Celebrating the launch of a new augmented intelligence (AI) tool in healthcare is great, but it’s just the beginning. The real work comes next! As technology advances, clinical guidelines and work processes will change. It’s essential to regularly assess how well these AI tools are performing.
Margaret Lozovatsky, MD, the AMA’s chief medical information officer, emphasizes the need for ongoing evaluation: “Technology is changing rapidly, and organizations must measure the success of these tools over time.” In fact, a significant shift is happening—nearly two-thirds of doctors reported using AI tools in 2024, almost doubling from the previous year. With this growth, organizations must ensure that AI integration isn’t just a trend but a valuable asset.
- Ensuring accountability
- Boosting transparency and trust
- Reducing bias
- Identifying risks or data security issues
- Improving overall performance
- Remaining compliant with regulations
- Measuring the tools’ business value
The AMA STEPS Forward® “Governance for Augmented Intelligence” toolkit is a great resource for healthcare organizations. It provides a straightforward eight-step guide for managing and scaling AI solutions. Key elements of responsible AI governance include:
- Defining executive accountability
- Forming focused working groups
- Assessing existing policies
- Creating new AI-related policies
- Defining project processes
- Updating planning methods
- Establishing oversight
- Preparing the organization for AI adoption
The AMA prefers the term “augmented intelligence” to highlight that AI supports healthcare professionals instead of replacing them. This is about collaboration, not competition.
To keep AI tools effective, healthcare systems need regular monitoring. For instance, if an AI tool changes how it assesses patient risk, it can greatly impact clinical decisions. Dr. Lozovatsky stresses the importance of having experts who can oversee these tools and ensure they’re functioning correctly.
The AMA toolkit also outlines five crucial steps for developing an effective monitoring process:
- Create a monitoring plan: Track metrics like data quality, algorithm performance, and user satisfaction.
- Build a multidisciplinary team: Include a clinical expert, a data scientist, and an administrative leader to oversee monitoring.
- Stay updated on regulations: Regularly check for changes in AI best practices and privacy laws.
- Implement the monitoring plan: Set clear roles and audit methods for the AI tools.
- Communicate findings: Share concerns with developers and inform users about the tool’s performance.
As AI evolves, recent data from the AMA shows that healthcare providers are adapting quickly. For instance, the Permanente Medical Group highlights how they’re using AI effectively, as discussed in an episode of the “AMA STEPS Forward Podcast.”
In addition to monitoring, the AMA is also advocating for ethical practices in AI usage. They focus on:
- Oversight of AI in healthcare
- Ensuring transparency in AI’s development and use
- Policies around AI and generative technologies
- Managing physician liabilities regarding AI
- Protecting data privacy
- Understanding payer uses of AI
As AI technology continues to grow, staying informed and proactive is crucial. For more on AI’s evolving role in healthcare, check out the AMA’s “AMA ChangeMedEd® Artificial Intelligence in Health Care Series.”
