Artificial intelligence (AI) is making waves in health care. It helps analyze medical images, predict health risks, and even monitor patients remotely. But there’s a catch: AI can make mistakes, especially when the data it learns from isn’t diverse or balanced.
A recent study from UC Davis, led by Professor Courtney Lyles, highlights the need for human oversight in AI decision-making. Published in Social Science and Medicine, this study shows how human insight can help reduce bias and enhance patient safety.
Professor Lyles is also the co-founder of UC S.O.L.V.E Health Tech, a project that brings together researchers and digital health companies to ensure fairer AI development. In a Q&A, she shared valuable insights about AI in health care and the importance of preventing bias.
What’s the Study About?
The study partnered with Google and scientists from various universities, using a human-centered approach to assess AI models for bias. They gathered a panel of experts to dig deep into the factors that might lead to AI bias in health care.
Why Does Bias Matter?
AI systems need to consider social and structural influences on health data. If they don’t, the output can be misleading or even harmful. That’s why combining AI with human expertise is vital—it creates safer, more accurate health care solutions.
Understanding Explainable AI
Explainable AI (XAI) helps us understand how AI models come to their conclusions. It allows us to peel back the layers and see the decision-making process, making it easier to spot potential bias.
How Are Experts Involved?
In the study, experts from diverse fields—including medicine and engineering—analysed AI outputs to offer real-world context. They asked critical questions to identify the root causes of any biases. This interdisciplinary approach not only enhances accuracy but also fosters trust in AI systems.
Real-World Application
The study included a case where experts reviewed AI results in medical imaging, offering actionable recommendations. This method ties together human judgment with AI capabilities, ensuring that decisions are grounded in reality.
The Role of Public-Private Partnerships
Stronger collaboration between public institutions and private companies can improve AI technology in health care. UC S.O.L.V.E Health Tech is a prime example of this. It unites scholars and industry to focus on equity in health technologies.
UC Davis Health Innovations
UC Davis Health has pioneered the use of AI in various clinical areas. Their governance committee, led by Professor Jason Adams, evaluates new AI models to ensure they meet equity standards. For instance, Professor Reshma Gupta’s team has developed methods to reduce bias in predictive models, improving patient care.
One exciting advancement is the AI Scribe program, launched in 2024. This tool generates clinical notes from recorded patient interactions. A recent study found that the AI-generated notes were mostly accurate, with an impressive 94.7% free from significant errors. However, it emphasizes the need for physicians to review these notes to catch smaller errors, underscoring the necessity of human involvement.
Conclusion
As AI becomes integral to health care, we must balance technology with human oversight. By bringing together diverse expertise and experiences, we can create AI tools that are not only efficient but also equitable and safe for all patients.
For more on AI in health care, you can read the study published in Social Science and Medicine here and explore initiatives like UC S.O.L.V.E Health Tech for further insights.
