RP30 - Combining LLMs with Logical Reasoning for Aggregating the Findings of Medical Studies

Aug 27, 2026

LLMs are useful tools for extracting information from text and summarizing longer texts. In ongoing work, we develop methods for the multi-document summarization of medical studies, beginning with information extraction of the four elements of the PICO (Population, Intervention, Comparison, Outcome) model [1]. For aggregating this information, however, there are additional problems:

Heterogeneity: Studies may vary in how they define the population, intervention, and comparison components of the PICO model, as well as the outcome.
Incompleteness: When comparing two or more studies, some details are often missing, making it difficult to determine whether the studies use the same or different settings and whether this may affect outcomes.
Uncertainty: Medical studies not only report different “success rates” of the intervention investigated, but they also vary in population size and level of evidence [2].
Contradictions: Studies might even contradict in their outcomes, raising the question of possible explanations for this difference

Since LLMs are not well suited to these issues, this thesis project will investigate combining them with logical reasoning models that can handle the issues mentioned above, e.g., probabilistic four-valued logic [3].

The envisaged solution should not only provide an explanation component, but also traceability by allowing users to trace any output back to the underlying original publications. Moreover, the resulting system should support a high degree of interactivity.

[1] Bohao Chu, Meijie Li, Sameh Frihat, Chengyu Gu, Georg Lodde, Elisabeth Livingstone, Norbert Fuhr: TracSum: A New Benchmark for Aspect-Based Summarization with Sentence-Level Traceability in Medical Domain. EMNLP 2025: 844-864

[2] S. Frihat and N. Fuhr. Supporting evidence-based medicine by finding both relevant and significant works. Inf. Retr. Res. J., 1(1):93–108, 2025. doi: 10.54195/IRRJ.19784. URL: https://doi.org/10.54195/irrj.19784.

[3] N. Fuhr and T. Rölleke. HySpirit – a probabilistic inference engine for hypermedia retrieval in large databases. In Proceedings of the 6th International Conference on Extending Database Technology (EDBT), pages 24–38, Heidelberg et al., 1998. Springer.