RP26 - Integrating Information Extraction, Knowledge Representation, and User Interaction - Personalization of Medical Decision-making at the Point of Care using Large Language Models and Chatbots

The management of malignant melanoma requires a nuanced and personalized approach, with decisions based on a thorough understanding of each patient’s unique situation. With the increasing availability of diverse data sources and the innovative tools and applications developed by the first cohort of WisPerMed, healthcare professionals are empowered to make more accurate decisions. However, with the increasing complexity and volume of relevant medical data from clinical sources and literature, the challenge is to harness this information in a way that enables personalised, context-aware decision-making at the point of care.

The advent of LLMs such as GPT-4 has opened the door to innovative techniques for understanding and generating human-like text. These models transform text into high-dimensional vectors or embeddings that capture the semantic meaning of the text. Unlike traditional NLP models, which rely on static word embeddings, transformational models create contextualised word embeddings that vary according to the context in which a word is used. These embeddings enable complex operations on text, such as finding similar words, phrases or documents, which are central to applications such as chatbots, recommendation engines and more, and are critical in the context of malignant melanoma, where personalization is key.

Vector databases are at the intersection of these advances. They provide efficient storage solutions for handling the indexing and querying of high-dimensional embeddings, enabling efficient similarity searches. This efficiency is particularly relevant to the medical field, where vast amounts of unstructured data (e.g. PDF, MS Word documents) related to the treatment of malignant melanoma need to be made accessible and actionable. By encoding this data into word embeddings and storing them in a vector database, a semantic search can retrieve the precise information needed for a specific query. The use of vector databases greatly enhances the ability of LLMs to contextualize information. This is particularly valuable in medical scenarios where models may ‘hallucinate’ or produce inaccurate output without sufficient context. The process involves encoding large amounts of melanoma-related data into vectors, storing them in a vector database, and then querying this database to retrieve just enough contextually relevant data. This allows for the creation of personalized, contextual recommendations or explanations that adapt to individual patient situations and the specific needs of healthcare providers.

By leveraging the capabilities of LLMs and vector databases, the treatment of malignant melanoma can be transformed into a highly personalied, efficient, interactive and context-aware process. This integration of technology facilitates the extraction of the right information at the right time, helping healthcare professionals to make more informed decisions at the point of care.

This project is strategically positioned to respond to this challenge by harnessing the power of LLMs to add conversational capabilities to existing WisPerMed tools. Through this integration, the initiative aims to cultivate a more intuitive, interactive and efficient suite of tools. These enhancements will not only increase the effectiveness of current systems, but will also underscore the broader goals of the WisPerMed framework, further aligning it with the overarching goals of personalized and adaptive medical care in the field of malignant melanoma. The interdisciplinary collaboration within the RTG will ensure that this technology is tailored to the specific needs of melanoma care, fostering innovation and setting a precedent for personalized medicine.

[1] Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023 Jul 17. doi: 10.1038/s41591-023-02448-8.

[2] Guo R, Sun P, Lindgren E, Geng Q, Simcha D, Chern F, and Kumar S. 2020. Accelerating large-scale inference with anisotropic vector quantization. In Proceedings of the 37th International Conference on Machine Learning (ICML'20), Vol. 119. JMLR.org, Article 364, 3887–3896.