Evaluating the reliability of an injury prevention screening tool: Test-retest study. 2016

Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
From the Division of Emergency Medicine (M.A.G., M.F.), Cincinnati Children's Hospital, Cincinnati, Ohio; Brown School at Washington University in St. Louis (M.K.), St. Louis, Missouri; Division of Emergency Medicine (S.D.), Nationwide Children's Hospital, Columbus, Ohio; American Academy of Pediatrics, Ohio Chapter (M.W.), Worthington, Ohio; and James M. Anderson Center for Health Systems Excellence (A.C.C., C.A.M.), Cincinnati Children's Hospital, Cincinnati, Ohio.

A standardized injury prevention (IP) screening tool can identify family risks and allow pediatricians to address behaviors. To assess behavior changes on later screens, the tool must be reliable for an individual and ideally between household members. Little research has examined the reliability of safety screening tool questions. This study utilized test-retest reliability of parent responses on an existing IP questionnaire and also compared responses between household parents. Investigators recruited parents of children 0 to 1 year of age during admission to a tertiary care children's hospital. When both parents were present, one was chosen as the "primary" respondent. Primary respondents completed the 30-question IP screening tool after consent, and they were re-screened approximately 4 hours later to test individual reliability. The "second" parent, when present, only completed the tool once. All participants received a 10-dollar gift card. Cohen's Kappa was used to estimate test-retest reliability and inter-rater agreement. Standard test-retest criteria consider Kappa values: 0.0 to 0.40 poor to fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, and 0.81 to 1.00 as almost perfect reliability. One hundred five families participated, with five lost to follow-up. Thirty-two (30.5%) parent dyads completed the tool. Primary respondents were generally mothers (88%) and Caucasian (72%). Test-retest of the primary respondents showed their responses to be almost perfect; average 0.82 (SD = 0.13, range 0.49-1.00). Seventeen questions had almost perfect test-retest reliability and 11 had substantial reliability. However, inter-rater agreement between household members for 12 objective questions showed little agreement between responses; inter-rater agreement averaged 0.35 (SD = 0.34, range -0.19-1.00). One question had almost perfect inter-rater agreement and two had substantial inter-rater agreement. The IP screening tool used by a single individual had excellent test-retest reliability for nearly all questions. However, when a reporter changes from pre- to postintervention, differences may reflect poor reliability or different subjective experiences rather than true change.

UI MeSH Term Description Entries
D007223 Infant A child between 1 and 23 months of age. Infants
D007231 Infant, Newborn An infant during the first 28 days after birth. Neonate,Newborns,Infants, Newborn,Neonates,Newborn,Newborn Infant,Newborn Infants
D008297 Male Males
D008403 Mass Screening Organized periodic procedures performed on large groups of people for the purpose of detecting disease. Screening,Mass Screenings,Screening, Mass,Screenings,Screenings, Mass
D011795 Surveys and Questionnaires Collections of data obtained from voluntary subjects. The information usually takes the form of answers to questions, or suggestions. Community Survey,Nonrespondent,Questionnaire,Questionnaires,Respondent,Survey,Survey Method,Survey Methods,Surveys,Baseline Survey,Community Surveys,Methodology, Survey,Nonrespondents,Questionnaire Design,Randomized Response Technique,Repeated Rounds of Survey,Respondents,Survey Methodology,Baseline Surveys,Design, Questionnaire,Designs, Questionnaire,Methods, Survey,Questionnaire Designs,Questionnaires and Surveys,Randomized Response Techniques,Response Technique, Randomized,Response Techniques, Randomized,Survey, Baseline,Survey, Community,Surveys, Baseline,Surveys, Community,Techniques, Randomized Response
D005260 Female Females
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D000061 Accidents, Home ACCIDENTS which occur within the home. Home Accidents,Accident, Home,Home Accident
D014947 Wounds and Injuries Damage inflicted on the body as the direct or indirect result of an external force, with or without disruption of structural continuity. Injuries,Physical Trauma,Trauma,Injuries and Wounds,Injuries, Wounds,Research-Related Injuries,Wounds,Wounds and Injury,Wounds, Injury,Injury,Injury and Wounds,Injury, Research-Related,Physical Traumas,Research Related Injuries,Research-Related Injury,Trauma, Physical,Traumas,Wound
D015203 Reproducibility of Results The statistical reproducibility of measurements (often in a clinical context), including the testing of instrumentation or techniques to obtain reproducible results. The concept includes reproducibility of physiological measurements, which may be used to develop rules to assess probability or prognosis, or response to a stimulus; reproducibility of occurrence of a condition; and reproducibility of experimental results. Reliability and Validity,Reliability of Result,Reproducibility Of Result,Reproducibility of Finding,Validity of Result,Validity of Results,Face Validity,Reliability (Epidemiology),Reliability of Results,Reproducibility of Findings,Test-Retest Reliability,Validity (Epidemiology),Finding Reproducibilities,Finding Reproducibility,Of Result, Reproducibility,Of Results, Reproducibility,Reliabilities, Test-Retest,Reliability, Test-Retest,Result Reliabilities,Result Reliability,Result Validities,Result Validity,Result, Reproducibility Of,Results, Reproducibility Of,Test Retest Reliability,Validity and Reliability,Validity, Face

Related Publications

Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
February 2017, The journal of trauma and acute care surgery,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
August 2022, BMJ open,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
December 2010, Military medicine,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
July 1981, Journal of clinical psychology,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
January 2022, PloS one,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
October 2021, Journal of patient safety,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
June 2012, Australian occupational therapy journal,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
September 2021, Journal of stroke and cerebrovascular diseases : the official journal of National Stroke Association,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
January 1995, The American journal of sports medicine,
Michael A Gittelman, and Madeline Kincaid, and Sarah Denny, and Melissa Wervey Arnold, and Michael FitzGerald, and Adam C Carle, and Constance A Mara
April 1998, The American journal of occupational therapy : official publication of the American Occupational Therapy Association,
Copied contents to your clipboard!