Supplemental Material for: Development of a protocol to retrospectively identify acute traumatic brain injury using electronic medical records from an academic health system data warehouse

Introduction: As recommended in the National Institute of Neurological Disorders and Stroke TBI Classification and Nomenclature Initiative, retrospective identification of traumatic brain injury (TBI) from electronic medical record (EMR) data offers major potential for research and surveillance. Enterprise data warehouses integrating structured and unstructured EMR data could support this work, yet few studies have evaluated their use for TBI case ascertainment. This study describes a manual chart review protocol to identify acute TBI among patients receiving head CT scan from a large health system’s enterprise data warehouse. Methods: We identified encounters potentially related to TBI using clinical documentation files (e.g., Emergency Department Triage, History & Physical, Discharge Summary) within our institution’s enterprise data warehouse. Two independent raters applied the protocol to 126 patients who presented in 2019 and underwent head computed tomography (CT). Sampling was limited to patients receiving a head CT scan to enrich the cohort with both TBI and non‑TBI cases. We developed a decision tree workflow to guide documentation of a plausible mechanism of head injury accompanied by clinical symptoms and/or clinically indicated neuroimaging. Criteria were adapted from the American Congress of Rehabilitation Medicine definition of mild TBI and modified for retrospective EMR data. Raters classified each case as (1) no evidence of acute TBI, (2) sufficient evidence of acute TBI, or (3) inconclusive evidence due to missing data or confounding conditions (e.g., syncope). Results: Inter‑rater reliability demonstrated strong agreement across raters (88.1% agreement; Cohen’s κ = 0.80 (95% CI: 0.71-0.89), p < 0.0001); inter-rater reliability was similar across subgroups of age (± 65 years), sex, and payor (government/non-government insurance). Among the 126 cases reviewed, there was agreement on sufficient evidence of acute TBI for 49 cases (86.0% agreement between rater 1 and 2), no evidence of acute TBI for 54 cases (96.4% agreement), and inconclusive evidence for 8 cases (61.5% agreement). Sensitivity analyses reweighting the sample to parent cohort marginal distributions produced similar results (94.2% agreement, κ = 0.74 (95% CI: 0.58-0.90). Conclusion: This manual chart review protocol showed strong inter‑rater reliability for identifying acute TBI using EMR data. Future research should examine external reliability in other health systems and compare its performance with International Classification of Diseases‑based definitions and prospective diagnosis. As enterprise data warehouses become more ubiquitous, reliable case ascertainment protocols such as this could facilitate standardized, scalable retrospective identification of TBI, supporting retrospective TBI cohort studies.