Validation of Automated Data Extraction From the Electronic Medical Record to Provide a Pediatric Risk Assessment Score. 2023

Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
From the Departments of Cardiology.

Although the rate of pediatric postoperative mortality is low, the development and validation of perioperative risk assessment models have allowed for the stratification of those at highest risk, including the Pediatric Risk Assessment (PRAm) score. The clinical application of such tools requires manual data entry, which may be inaccurate or incomplete, compromise efficiency, and increase physicians' clerical obligations. We aimed to create an electronically derived, automated PRAm score and to evaluate its agreement with the original American College of Surgery National Surgical Quality Improvement Program (ACS NSQIP)-derived and validated score. We performed a retrospective observational study of children <18 years who underwent noncardiac surgery from 2017 through 2021 at Boston Children's Hospital (BCH). An automated PRAm score was developed via electronic derivation of International Classification of Disease (ICD) -9 and -10 codes. The primary outcome was agreement and correlation among PRAm scores obtained via automation, NSQIP data, and manual physician entry from the same BCH cohort. The secondary outcome was discriminatory ability of the 3 PRAm versions. Fleiss Kappa, Spearman correlation (rho), and intraclass correlation coefficient (ICC) and receiver operating characteristic (ROC) curve analyses with area under the curve (AUC) were applied accordingly. Of the 6014 patients with NSQIP and automated PRAm scores (manual scores: n = 5267), the rate of 30-day mortality was 0.18% (n = 11). Agreement and correlation were greater between the NSQIP and automated scores (rho = 0.78; 95% confidence interval [CI], 0.76-0.79; P <.001; ICC = 0.80; 95% CI, 0.79-0.81; Fleiss kappa = 0.66; 95% CI, 0.65-0.67) versus the NSQIP and manual scores (rho = 0.73; 95% CI, 0.71-0.74; P < .001; ICC = 0.78; 95% CI, 0.77-0.79; Fleiss kappa = 0.56; 95% CI, 0.54-0.57). ROC analysis with AUC showed the manual score to have the greatest discrimination (AUC = 0.976; 95% CI, 0.959,0.993) compared to the NSQIP (AUC = 0.904; 95% CI, 0.792-0.999) and automated (AUC = 0.880; 95% CI, 0.769-0.999) scores. Development of an electronically derived, automated PRAm score that maintains good discrimination for 30-day mortality in neonates, infants, and children after noncardiac surgery is feasible. The automated PRAm score may reduce the preoperative clerical workload and provide an efficient and accurate means by which to risk stratify neonatal and pediatric surgical patients with the goal of improving clinical outcomes and resource utilization.

UI MeSH Term Description Entries
D007223 Infant A child between 1 and 23 months of age. Infants
D007231 Infant, Newborn An infant during the first 28 days after birth. Neonate,Newborns,Infants, Newborn,Neonates,Newborn,Newborn Infant,Newborn Infants
D011183 Postoperative Complications Pathologic processes that affect patients after a surgical procedure. They may or may not be related to the disease for which the surgery was done, and they may or may not be direct results of the surgery. Complication, Postoperative,Complications, Postoperative,Postoperative Complication
D002648 Child A person 6 to 12 years of age. An individual 2 to 5 years old is CHILD, PRESCHOOL. Children
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D012189 Retrospective Studies Studies used to test etiologic hypotheses in which inferences about an exposure to putative causal factors are derived from data relating to characteristics of persons under study or to events or experiences in their past. The essential feature is that some of the persons under study have the disease or outcome of interest and their characteristics are compared with those of unaffected persons. Retrospective Study,Studies, Retrospective,Study, Retrospective
D012307 Risk Factors An aspect of personal behavior or lifestyle, environmental exposure, inborn or inherited characteristic, which, based on epidemiological evidence, is known to be associated with a health-related condition considered important to prevent. Health Correlates,Risk Factor Scores,Risk Scores,Social Risk Factors,Population at Risk,Populations at Risk,Correlates, Health,Factor, Risk,Factor, Social Risk,Factors, Social Risk,Risk Factor,Risk Factor Score,Risk Factor, Social,Risk Factors, Social,Risk Score,Score, Risk,Score, Risk Factor,Social Risk Factor
D057286 Electronic Health Records Media that facilitate transportability of pertinent information concerning patient's illness across varied providers and geographic locations. Some versions include direct linkages to online CONSUMER HEALTH INFORMATION that is relevant to the health conditions and treatments related to a specific patient. Electronic Health Record Data,Electronic Medical Record,Electronic Medical Records,Computerized Medical Record,Computerized Medical Records,Electronic Health Record,Medical Record, Computerized,Medical Records, Computerized,Health Record, Electronic,Health Records, Electronic,Medical Record, Electronic,Medical Records, Electronic
D018570 Risk Assessment The qualitative or quantitative estimation of the likelihood of adverse effects that may result from exposure to specified health hazards or from the absence of beneficial influences. (Last, Dictionary of Epidemiology, 1988) Assessment, Risk,Benefit-Risk Assessment,Risk Analysis,Risk-Benefit Assessment,Health Risk Assessment,Risks and Benefits,Analysis, Risk,Assessment, Benefit-Risk,Assessment, Health Risk,Assessment, Risk-Benefit,Benefit Risk Assessment,Benefit-Risk Assessments,Benefits and Risks,Health Risk Assessments,Risk Analyses,Risk Assessment, Health,Risk Assessments,Risk Benefit Assessment,Risk-Benefit Assessments

Related Publications

Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
September 2021, Journal of general internal medicine,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
December 2018, BMC neurology,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
September 2018, Journal of medical systems,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
March 2016, Journal of the American Medical Directors Association,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
December 2015, Journal of nursing measurement,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
January 2021, AMIA ... Annual Symposium proceedings. AMIA Symposium,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
June 2014, Pharmacoepidemiology and drug safety,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
January 2020, International journal of medical informatics,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
July 1996, International journal of bio-medical computing,
Eleonore Valencia, and Steven J Staffa, and Yousuf Aslam, and David Faraoni, and James A DiNardo, and Shawn J Rangel, and Viviane G Nasr
January 2017, Journal for healthcare quality : official publication of the National Association for Healthcare Quality,
Copied contents to your clipboard!