What’s Happening:
- A new partnership is working to set standards for using AI in healthcare. This effort includes a registry for hospitals and developers to find and share information on AI tools.
- The registry, called the model card registry, was launched by the Coalition for Health AI (CHAI) alongside major health organizations like Providence and Kaiser Permanente. Its aim is to simplify the process for healthcare providers to evaluate and purchase validated AI products.
- AI companies can add their tools to this registry, even if they aren’t CHAI members. Uploading model cards and accessing the registry is currently free, though future paid features may be introduced.
Why It Matters:
AI is making its way into healthcare, helping with tasks like clinical note-taking and patient communication. According to a recent survey, 86% of healthcare organizations are now using AI, primarily for administrative tasks. However, there’s a growing interest in more advanced applications like personalized treatment plans and diagnostic assistance.
Despite this interest, many healthcare organizations are still new to AI. Some have only started using the technology in the past six months. A major hurdle is keeping pace with the rapid changes in AI offerings. Hospitals often struggle to choose between the many options available, which makes procurement challenging. They need to sift through technical details and marketing claims from different providers.
Furthermore, as AI adoption accelerates, questions arise about ethical implementation and patient safety. Providers want to ensure that AI tools are used responsibly and effectively. This is where CHAI comes in, aiming to set clear standards for responsible AI use in healthcare.
The model card registry is designed to standardize how information about AI tools is presented. It will include model cards, which serve as documentation of an AI tool’s development, capabilities, and limitations. These cards function like nutritional labels, providing essential information to help healthcare providers make informed decisions.
The model card registry was developed in collaboration with Avanade, a partnership between Microsoft and Accenture. Hospitals can find all the necessary AI information in one place, and developers can showcase their products to potential buyers. Elisabeth Garwood, an informatics officer, has highlighted the importance of model cards in streamlining information for various stakeholders, including radiologists and legal teams.
Last year, CHAI shared its model card template for public feedback. Responses varied, with stakeholders suggesting changes based on their industry needs. While CHAI hasn’t finalized these changes, they’re working to ensure the model cards contain relevant fields for different types of AI, including generative models, which are generally harder to evaluate.
CHAI is also focused on creating a user-friendly registry that provides timely information based on users’ needs. This process will require extensive testing to integrate feedback from different organizations and ensure the registry fits seamlessly into their workflows.
Since its founding in 2021, CHAI has expanded to include 1,300 member organizations, including tech giants and healthcare providers. The consortium aims to establish a network of quality assurance labs to evaluate AI models and set best practices for their usage.
Overall, CHAI represents a broader effort among technology firms, developers, and healthcare institutions to develop standards in an area where government regulation has been absent. This initiative is crucial as AI continues to evolve and become more integral to healthcare delivery.
Various other organizations, like the Trustworthy & Responsible AI Network (TRAIN) and Nvidia in partnership with the clinical AI company Aidoc, are also working toward establishing their own standards for AI in healthcare.
