Researchers have introduced AfriMed-QA, a benchmark question–answer dataset designed to test large language models (LLMs) on medical and health questions across Africa. The dataset combines consumer-style questions and medical school–type exams from 60 medical schools in 16 countries in Africa.
The project aims to address uncertainty about whether LLMs can generalize beyond existing medical benchmarks, especially when disease types, symptoms, language, linguistics, and local medical knowledge differ from traditional Western settings. The source says that without diverse benchmark datasets reflecting real-world contexts, it is difficult to train or evaluate models for these environments.
AfriMed-QA was developed in collaboration with Intron health, Sisonkebiotik, University of Cape Coast, the Federation of African Medical Students Association, and BioRAMP, which together form the AfriMed-QA consortium. The work also had support from PATH/The Gates Foundation.
The researchers evaluated LLM responses on the dataset by comparing them with answers provided by human experts and rating the responses according to human preference. The source says the approach can be scaled to other locales where digitized benchmarks are not yet available.
The dataset is intended to help assess whether LLMs can serve as decision-support tools in low-resource settings, including for clinical diagnostic accuracy, accessibility, multilingual clinical decision support, and health training.
Source: research.google.
Companies can share verified announcements through Newz9’s international press release submission page.
