Artificial intelligence (AI) is changing how we review scientific literature, especially in health research. At the Cochrane Collaboration, where I work as editor-in-chief, we focus on systematic reviews. These reviews are essential for guiding clinical practices and public health policies. We are exploring how AI can help streamline our processes, but so far, it’s clear that AI isn’t ready to replace human effort completely.
The stakes are high. Flawed reviews can mislead patients or result in health systems wasting money on ineffective treatments. Currently, AI tools mimic the way humans conduct reviews. They identify relevant studies, pull data, and even draft reports. But systematic reviews are not just about processing data; they require human insight. Experts define important questions, evaluate study relevance, and interpret findings in context. AI struggles with the context and nuances that are crucial for decision-making.
At Cochrane, we’ve tested various AI tools for screening studies and extracting data. However, these tools often come from private companies, raising concerns about bias, especially in medical device and drug evaluations. Many of these AI models are “black boxes,” meaning we can’t see how they operate or if they favor certain outcomes.
The practicalities also pose challenges. Training both the AI and the humans who operate it can take a long time. Surprisingly, we found that the AI-assisted process often takes longer than doing the work manually.
Experts in AI and health research suggest a shift in strategy. Instead of using AI to replicate human processes, the focus should be on collaborative systems where humans and AI complement each other. This would ensure more reliable reviews and better health outcomes.
A recent study from the National Institutes of Health (NIH) shows that while AI can handle large data sets, its effectiveness in nuanced evaluations is still limited. According to this research, only 30% of current AI models can accurately interpret the clinical relevance of findings without human oversight.
As we move forward, it’s vital to refine AI applications in systematic reviews. Balancing efficiency with accuracy will ultimately lead to better health policies and outcomes. For more on this, you can check out the NIH’s perspective on AI in healthcare here.
Source link
Machine learning,Medical research,Publishing,Research data,Science,Humanities and Social Sciences,multidisciplinary
