Radiologist-AI pairing shows high sensitivity for PE detection

A radiologist-AI workflow showed 99.2% sensitivity for pulmonary embolism detection in a large real-world CT pulmonary angiography study, according to the Harvey L. Neiman Health Policy Institute.

Researchers from Northwell Health analyzed 32,501 CTPA exams performed across an integrated healthcare system over an 18-month period. Overall agreement between the Aidoc AI tool and radiologist interpretations was 97.8%.

Concordance was higher for negative exams than positive exams, at 98.18% versus 93.75%, respectively. The Neiman Institute said the finding points to the algorithm’s strength in helping rule out pulmonary embolism.

Discordant cases were independently reviewed by expert thoracic radiologists. In those disagreements, radiologists were correct in 88.7% of cases, while AI was correct in 11.3%.

Among confirmed pulmonary embolism cases, 15% were identified by radiologists but missed by the AI tool alone. Matthew Barish, MD, said the findings showed that “radiologist oversight remained necessary.”

The AI system flagged suspected positive CTPA cases to assist radiologists with triage inside the Northwell Health workflow. Agreement was highest for acute and central emboli, the cases associated with greater clinical urgency and mortality risk, according to the release.

Pulmonary embolism is responsible for 5% to 10% of in-hospital deaths and more than 300,000 deaths annually in the U.S., the Neiman Institute noted. The study authors said few large-scale evaluations have assessed FDA-cleared PE detection tools using a human-in-the-loop adjudication model in routine clinical practice.

The results were published in Radiology: Artificial Intelligence.