BIOMAT Database Logo BIOMAT Database Logo

COVID-19 on Chest Radiographs: A Multireader Evaluation of an Artificial Intelligence System. 2020

Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
From the Diagnostic Image Analysis Group, Radboud University Medical Center, Geert Groteplein 10, Nijmegen 6500 HB, the Netherlands (K.M., E.T.S., S.S., C.M.S., B.v.G.); Department of Radiology, Bernhoven Hospital, Uden, the Netherlands (H.S.); Department of Radiology, Jeroen Bosch Hospital, 's-Hertogenbosch, the Netherlands (A.J.G.K., M.B.J.M.K., T.S., M.R.); Department of Radiology, Meander Medisch Centrum, Amersfoort, the Netherlands (C.M.S.); and Thirona, Nijmegen, the Netherlands (R.H.H.M.P., A.M., J.M.).

Background Chest radiography may play an important role in triage for coronavirus disease 2019 (COVID-19), particularly in low-resource settings. Purpose To evaluate the performance of an artificial intelligence (AI) system for detection of COVID-19 pneumonia on chest radiographs. Materials and Methods An AI system (CAD4COVID-XRay) was trained on 24 678 chest radiographs, including 1540 used only for validation while training. The test set consisted of a set of continuously acquired chest radiographs (n = 454) obtained in patients suspected of having COVID-19 pneumonia between March 4 and April 6, 2020, at one center (223 patients with positive reverse transcription polymerase chain reaction [RT-PCR] results, 231 with negative RT-PCR results). Radiographs were independently analyzed by six readers and by the AI system. Diagnostic performance was analyzed with the receiver operating characteristic curve. Results For the test set, the mean age of patients was 67 years ± 14.4 (standard deviation) (56% male). With RT-PCR test results as the reference standard, the AI system correctly classified chest radiographs as COVID-19 pneumonia with an area under the receiver operating characteristic curve of 0.81. The system significantly outperformed each reader (P < .001 using the McNemar test) at their highest possible sensitivities. At their lowest sensitivities, only one reader significantly outperformed the AI system (P = .04). Conclusion The performance of an artificial intelligence system in the detection of coronavirus disease 2019 on chest radiographs was comparable with that of six independent readers. © RSNA, 2020.

UI MeSH Term Description Entries
D008297 Male Males
D008875 Middle Aged An adult aged 45 - 64 years. Middle Age
D011024 Pneumonia, Viral Inflammation of the lung parenchyma that is caused by a viral infection. Pneumonias, Viral,Viral Pneumonia,Viral Pneumonias
D011857 Radiographic Image Interpretation, Computer-Assisted Computer systems or networks designed to provide radiographic interpretive information. Computer Assisted Radiographic Image Interpretation,Computer-Assisted Radiographic Image Interpretation,Radiographic Image Interpretation, Computer Assisted
D005260 Female Females
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000073640 Betacoronavirus A genus of the family CORONAVIRIDAE which causes respiratory or gastrointestinal disease in a variety of mostly mammals. Human betacoronaviruses include HUMAN ENTERIC CORONAVIRUS; HUMAN CORONAVIRUS OC43; MERS VIRUS; and SEVERE ACUTE RESPIRATORY SYNDROME-RELATED CORONAVIRUS. Members have either core transcription regulatory sequences of 5’-CUAAAC-3’ or 5’-CUAAAC-3’ and mostly have no ORF downstream to the N protein gene. HCoV-HKU1,Human coronavirus HKU1,Pipistrellus bat coronavirus HKU5,Rousettus bat coronavirus HKU9,Tylonycteris bat coronavirus HKU4,Betacoronaviruses
D000086382 COVID-19 A viral disorder generally characterized by high FEVER; COUGH; DYSPNEA; CHILLS; PERSISTENT TREMOR; MUSCLE PAIN; HEADACHE; SORE THROAT; a new loss of taste and/or smell (see AGEUSIA and ANOSMIA) and other symptoms of a VIRAL PNEUMONIA. In severe cases, a myriad of coagulopathy associated symptoms often correlating with COVID-19 severity is seen (e.g., BLOOD COAGULATION; THROMBOSIS; ACUTE RESPIRATORY DISTRESS SYNDROME; SEIZURES; HEART ATTACK; STROKE; multiple CEREBRAL INFARCTIONS; KIDNEY FAILURE; catastrophic ANTIPHOSPHOLIPID ANTIBODY SYNDROME and/or DISSEMINATED INTRAVASCULAR COAGULATION). In younger patients, rare inflammatory syndromes are sometimes associated with COVID-19 (e.g., atypical KAWASAKI SYNDROME; TOXIC SHOCK SYNDROME; pediatric multisystem inflammatory disease; and CYTOKINE STORM SYNDROME). A coronavirus, SARS-CoV-2, in the genus BETACORONAVIRUS is the causative agent. 2019 Novel Coronavirus Disease,2019 Novel Coronavirus Infection,2019-nCoV Disease,2019-nCoV Infection,COVID-19 Pandemic,COVID-19 Pandemics,COVID-19 Virus Disease,COVID-19 Virus Infection,Coronavirus Disease 2019,Coronavirus Disease-19,SARS Coronavirus 2 Infection,SARS-CoV-2 Infection,Severe Acute Respiratory Syndrome Coronavirus 2 Infection,COVID19,2019 nCoV Disease,2019 nCoV Infection,2019-nCoV Diseases,2019-nCoV Infections,COVID 19,COVID 19 Pandemic,COVID 19 Virus Disease,COVID 19 Virus Infection,COVID-19 Virus Diseases,COVID-19 Virus Infections,Coronavirus Disease 19,Disease 2019, Coronavirus,Disease, 2019-nCoV,Disease, COVID-19 Virus,Infection, 2019-nCoV,Infection, COVID-19 Virus,Infection, SARS-CoV-2,Pandemic, COVID-19,SARS CoV 2 Infection,SARS-CoV-2 Infections,Virus Disease, COVID-19,Virus Infection, COVID-19
D000086402 SARS-CoV-2 A species of BETACORONAVIRUS causing atypical respiratory disease (COVID-19) in humans. The organism was first identified in 2019 in Wuhan, China. The natural host is the Chinese intermediate horseshoe bat, RHINOLOPHUS affinis. 2019 Novel Coronavirus,COVID-19 Virus,COVID19 Virus,Coronavirus Disease 2019 Virus,SARS Coronavirus 2,SARS-CoV-2 Virus,Severe Acute Respiratory Syndrome Coronavirus 2,Wuhan Coronavirus,Wuhan Seafood Market Pneumonia Virus,2019-nCoV,2019 Novel Coronaviruses,COVID 19 Virus,COVID-19 Viruses,COVID19 Viruses,Coronavirus 2, SARS,Coronavirus, 2019 Novel,Coronavirus, Wuhan,Novel Coronavirus, 2019,SARS CoV 2 Virus,SARS-CoV-2 Viruses,Virus, COVID-19,Virus, COVID19,Virus, SARS-CoV-2,Viruses, COVID19
D000368 Aged A person 65 years of age or older. For a person older than 79 years, AGED, 80 AND OVER is available. Elderly

Related Publications

Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
COVID-19 pneumonia accurately detected on chest radiographs with artificial intelligence.
December 2020, Intelligence-based medicine,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs.
February 2022, Clinical imaging,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
Multi-center validation of an artificial intelligence system for detection of COVID-19 on chest radiographs in symptomatic patients.
January 2023, European radiology,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
An automated COVID-19 triage pipeline using artificial intelligence based on chest radiographs and clinical data.
January 2022, NPJ digital medicine,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
A Deep-Learning Diagnostic Support System for the Detection of COVID-19 Using Chest Radiographs: A Multireader Validation Study.
June 2021, Investigative radiology,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
Deep transfer learning artificial intelligence accurately stages COVID-19 lung disease severity on portable chest radiographs.
January 2020, PloS one,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
COVID-19 Diagnosis on Chest Radiograph Using Artificial Intelligence.
November 2022, Cureus,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
The Potential of Artificial Intelligence to Analyze Chest Radiographs for Signs of COVID-19 Pneumonia.
April 2021, Radiology,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
The role of artificial intelligence in plain chest radiographs interpretation during the Covid-19 pandemic.
January 2022, BJR open,
Keelin Murphy, and Henk Smits, and Arnoud J G Knoops, and Michael B J M Korst, and Tijs Samson, and Ernst T Scholten, and Steven Schalekamp, and Cornelia M Schaefer-Prokop, and Rick H H M Philipsen, and Annet Meijers, and Jaime Melendez, and Bram van Ginneken, and Matthieu Rutten
Development and evaluation of an artificial intelligence system for COVID-19 diagnosis.
October 2020, Nature communications,
Copied contents to your clipboard!