Chest X-ray AI study exposes lung cancer detection gaps
A head-to-head study of 7 commercial AI devices found sensitivity ranged from 20.8% to 77.8% for lung cancer detection on chest X-rays.
Commercial AI devices varied widely in lung cancer detection on chest X-rays, according to a head-to-head study published May 19 in Radiology.
Ahmed Maiter, MB BChir, of Sheffield Teaching Hospitals NHS Foundation Trust, and colleagues tested 7 AI devices on 5,235 posteroanterior chest radiographs from 5,235 adult patients at a single U.K. center. The radiographs were acquired between July 2020 and February 2021.
Confirmed lung cancer with a visible tumor was present in 1.4% of patients. Median patient age was 60 years, 53.4% were female, and 79.4% were White.
Participating manufacturers included Annalise.ai, now Harrison.ai, Gleamer, InferVision, Milvue, Oxipit, Qure.ai, and Rayscape. Device outputs were later anonymized in the analysis.
Performance varied across key metrics. Area under the receiver operating characteristic curve ranged from 0.80 to 0.94, sensitivity from 20.8% to 77.8%, specificity from 58.9% to 98.4%, and positive predictive value from 1.5% to 28.4%.
Pairwise comparisons showed significant differences in 39 of 44 classification-result comparisons. Device classification agreement was minimal, with a Fleiss κ of 0.24.
Compared with radiologist reports, 3 devices detected more tumors and 4 detected fewer tumors. Additional false-positive results ranged from 10 to 2,039, depending on the device.
False-positive failure analysis found that erroneous detection of other pathologies was the most common cause of false-positive results, with a median of 71%.
Study authors concluded that commercial AI tools for this use case do not perform equally. They said future work should examine the impact of different devices on radiologist accuracy, reporting behavior, patient outcomes, and healthcare service delivery.
About the author
RadiologySignal.com writersEditorial Team
Radiology Signal Staff covers developments across medical imaging, radiology AI, imaging informatics, clinical research, and radiology business. The team monitors primary sources, peer-reviewed studies, company announcements, society updates, and healthcare industry news to deliver concise reporting for imaging professionals.
More from this section
Pediatric ED imaging gaps persist by race and insurance status
A cohort study of 857,034 pediatric ED visits found that stronger hospital pediatric capability was not associated with narrower imaging-use differences by race, ethnicity, or insurance status.
MRI radiomics score flags aggressive colorectal cancer subtype
The multicenter study found an MRI-based machine-learning radiomics score predicted CMS4 colorectal cancer and stratified recurrent metastasis risk.
Contrast tied to higher AKI risk in neonatal ICU study
The retrospective study found higher acute kidney injury rates in neonates exposed to iodinated contrast media, although most cases were stage 1.
Quantum BioPharma reports early PET tracer signal in MS study
The ongoing MGH study is evaluating [18F]3F4AP PET imaging for demyelinated neurons with intact axons in patients with multiple sclerosis.
PSA screening review finds fewer deaths, more diagnoses
The updated review found PSA screening likely reduces prostate cancer deaths, but evidence remains unclear for newer screening strategies combining PSA, kallikrein testing, and MRI.
Stage 4 breast cancer diagnoses rise over decade
A cohort study of 761,471 people found de novo stage 4 breast cancer incidence increased by 1.2% per year from 2010 to 2021.
SIR backs genicular artery embolization for knee osteoarthritis
The Society of Interventional Radiology issued a position statement supporting genicular artery embolization for symptomatic knee osteoarthritis.
Fluoride, lead co-exposure linked to lower youth BMD
Researchers analyzed NHANES data from 2,378 U.S. children and adolescents who underwent DEXA scans from 2013 to 2016.
Stanford project aims to make MR lymphangiography faster
Stanford Medicine received up to $3.9M from ARPA-H to advance MR lymphangiography. Andreas Loening, MD, PhD, will lead work on faster scans, lymphatic-specific contrast agents, and contrast-free flow assessment.
Bayer PET tracer meets cardiac amyloidosis endpoints
Bayer reported positive topline phase 3 REVEAL results for I-124 evuzamitide, an investigational PET/CT radiotracer for suspected cardiac amyloidosis. The study met its sensitivity and specificity endpoints.
14-minute MRx scan maps 20+ brain biomarkers without contrast
University of Illinois researchers developed multiplexed MRI to map more than 20 brain biomarkers in a single scan using standard clinical MRI systems.
PSMA PET appears to shift prostate cancer treatment intensity
A national claims analysis found that PSMA PET use was associated with higher initiation of androgen receptor pathway inhibitors and systemic therapy in newly diagnosed advanced prostate cancer.