MAGDA: Multi-agent guideline-driven diagnostic assistance

In emergency departments, rural hospitals, or clinics in less developed regions, clinicians often lack fast image analysis by trained radiologists, which can have a detrimental effect on patients' healthcare.

Large Language Models (LLMs) have the potential to alleviate some pressure from these clinicians by providing insights that can help them in their decision-making.

While these LLMs achieve high test results on medical exams showcasing their great theoretical medical knowledge, they tend not to follow medical guidelines.

In this work, we introduce a new approach for zero-shot guideline-driven decision support.

We model a system of multiple LLM agents augmented with a contrastive vision-language model that collaborate to reach a patient diagnosis.

After providing the agents with simple diagnostic guidelines, they will synthesize prompts and screen the image for findings following these guidelines.

Finally, they provide understandable chain-of-thought reasoning for their diagnosis, which is then self-refined to consider inter-dependencies between diseases.

As our method is zero-shot, it is adaptable to settings with rare diseases, where training data is limited, but expert-crafted disease descriptions are available.

We evaluate our method on two chest X-ray datasets, CheXpert and ChestX-ray 14 Longtail, showcasing performance improvement over existing zero-shot methods and generalizability to rare diseases.