Artificial Intelligence in Mammography Screening (AIMS)
Research type
Research Study
Full title
Clinical validation of an artificial intelligence system to improve the quality, efficiency and experience of breast cancer screening
IRAS ID
303782
Contact name
Ara Darzi
Contact email
Sponsor organisation
Imperial College London
Duration of Study in the UK
1 years, 11 months, 31 days
Research summary
Research Summary
This project aims to evaluate the potential for artificial intelligence (AI)-enabled NHS breast screening to increase accuracy, safety, cost-effectiveness, and clinician/patient experience, while demonstrating clinical feasibility.In the UK breast screening programme, two expert readers (radiologists) assess each mammogram (x-ray of the breast), with any disagreements in opinion reviewed by an arbitration panel of two further readers. However, a radiologist workforce crisis threatens the screening programme’s long-term sustainability. Google’s AI system identified cancer in mammograms with greater accuracy than specialists (Nature, 2020), suggesting potential to: reduce radiologist workload; increase service capacity; improve accuracy and outcomes, and reduce variability; and reduce time to results, improving patient experience. This project brings this initial research towards real world impact.
Firstly in Part A, we will compare the Google AI system to radiologists in large historical populations at two NHS sites with diverse multicultural patient populations. This will allow detailed assessments of accuracy of the AI; ensure fair, equitable performance; and support modelling of workforce and economic impacts.
Secondly in Part B, we will perform a large diagnostic study to re-read historical mammograms from approximately 10,000 women across two hospitals to explore how expert readers in arbitration panels interact with the AI when used as the second reader. This will explore complex human factors involved, measure overall accuracy of AI-enabled screening, and enable comprehensive assessments of NHS health economic impacts. Readers will only access anonymised data.Throughout the study, we will run PPIE workshops with diverse groups of up to 14 members of the public, with previous experience of breast cancer and/or screening mammography, co-facilitated by a lay partner. We will discuss patients’ ideas, concerns and expectations for the project, which will feed into project design. We have successfully run 4 patient and public workshops in 2021.
Summary of Results
This study explored how artificial intelligence (AI) could support breast cancer screening in the NHS, focusing on accuracy, fairness, how specialists interact with AI, and how it works in real hospital settings. It also involved patients and the public to ensure their views were considered.
________________________________________
Part A: Testing AI Accuracy and Fairness We tested an AI system using breast screening data from five NHS hospitals. The goal was to see how well the AI could detect cancer compared to human specialists and whether it worked fairly across different groups of women.
• What we did: We used 25,000 anonymised cases from each hospital and made sure the data was suitable for testing.
• What we found: The AI was better at spotting cancers (higher sensitivity) and just as good or better at avoiding false alarms (specificity) compared to human readers.
________________________________________
Part B: How Specialists Work with AI
We looked at how breast screening specialists interact with AI when it replaces one of the two human readers in the usual screening process.
• What we did: We ran a study where AI acted as the second reader and observed how specialists made decisions when they disagreed with the AI.
• What we found:
o The AI system detected slightly more cancers and had fewer false positives than the human-only system.
o Some cancers found by the AI weren’t followed up, showing that human decisions still play a big role.
o We’re analysing feedback from specialists to improve how AI fits into their workflow.
________________________________________
Part D: Involving Patients and the Public We held workshops with women who use breast screening services to understand their thoughts on using AI.
• What we learned:
o Their input helped shape the study and made it more relevant and respectful.
o They suggested ways to make screening more inclusive, like using different formats (audio, multilingual materials).
o Educating the public about AI’s role in screening is key to building trust and acceptance.REC name
East Midlands - Nottingham 1 Research Ethics Committee
REC reference
22/EM/0038
Date of REC Opinion
14 Feb 2022
REC opinion
Favourable Opinion