A study of an AI system for mammogram screening evaluated both standalone performance and how it could fit into real NHS workflows. The first study was split into two phases: a large-scale retrospective test and a prospective, non-interventional deployment study.
In the first phase, researchers analysed mammograms from 125,000 women, or 115,973 after applying inclusion/exclusion criteria, screened at five NHS screening services in the UK. The services covered three different clinical workflows, varying by whether the second reader was blinded to the first and how cases were selected for arbitration.
AI operating points, described as the threshold that determines the conservativeness with which the AI flags cases, were determined separately at each screening service to adjust for local differences in screening populations and workflows.
The primary endpoints assessed the sensitivity and specificity of the AI system in detecting cancer compared to the historical (original) first reader for the case. The study used a 39-month follow-up window to measure the AI system’s incremental benefit in detecting interval and next-round cancers before they became clinically symptomatic.
The analysis also compared AI performance with second and consensus readers, and examined lesion-level localization and fairness. The lesion-level analysis was intended to test whether the AI system correctly identified the relevant abnormality in the breast rather than relying on spurious correlations.
The retrospective phase was designed to validate AI performance at scale and did not involve collecting additional interpretations from human readers or prospective deployment.
Source: research.google.
Companies can share verified announcements through Newz9’s international press release submission page.
