This paper is in the following e-collection/theme issue:

Nursing in a Hospital Setting (96) Registered Report (528) Clinical Informatics (919) Artificial Intelligence (1216) Machine Learning (1401) Nursing (87) Nursing Education and Training (113)

Published on in Vol 6 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/41331, first published .
The Use and Structure of Emergency Nurses’ Triage Narrative Data: Scoping Review

The Use and Structure of Emergency Nurses’ Triage Narrative Data: Scoping Review

The Use and Structure of Emergency Nurses’ Triage Narrative Data: Scoping Review

Authors of this article:

Christopher Picard1 Author Orcid Image ;   Manal Kleib1 Author Orcid Image ;   Colleen Norris1 Author Orcid Image ;   Hannah M O'Rourke1 Author Orcid Image ;   Carmel Montgomery1 Author Orcid Image ;   Matthew Douma2 Author Orcid Image
Article Authors Cited by (3) Tweetations (5) Metrics

    Review

    1Faculty of Nursing, University of Alberta, Edmonton, AB, Canada

    2School of Nursing, Midwifery and Health Systems, University College Dublin, Dublin, Ireland

    Corresponding Author:

    Christopher Picard, MN

    Faculty of Nursing

    University of Alberta

    Graduate Office

    4-171 Edmonton Clinic Health Academy

    Edmonton, AB, T6G 1C9

    Canada

    Phone: 1 (780) 492 4567

    Email: picard.ct@gmail.com


    Background: Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner.

    Objective: The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted.

    Methods: We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research.

    Results: We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses’ narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%).

    Conclusions: The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines.

    International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-055132

    JMIR Nursing 2023;6:e41331

    doi:10.2196/41331

    Keywords

    nursing; artificial intelligence; machine learning; triage; review; narrative 


    Overview

    There are an estimated 46.6 emergency department (ED) visits per 100 people in the United States or 142 million annual visits to Canadian and American EDs combined [1,2]. EDs sort and prioritize patients using triage to ensure that patients with the highest level of acuity are provided care quickly and safely. Modern electronic health records allow for the large-scale collection of triage data, such as time stamps, vital signs, screening assessments, and free-text descriptions [3,4]. These data can be used to track ED volumes and guide local and national policy decisions [5]. Machine learning (ML) and artificial intelligence have allowed the data to be examined for a range of purposes [5,6]. Despite the ubiquity of triage and triage-related data collection, the potential research impact of using triage narrative data remains largely unrealized [7,8].

    Background

    Triage is the process of sorting patients. It originated during the Napoleonic wars [9] and was introduced into civilian practice in the 1960s [10]. Triage was formalized using validated tools in the 1980s [11] and was first implemented in Australia as a national system in 1994 [12]. Most countries use a formal triage system [13] associated with improved patient safety and service efficiency outcomes [14]. Triage is typically performed by experienced ED nurses [15] who are specially trained to use formally validated triage assessment tools to prioritize patient care [13]. Triage assessment typically consists of a brief history and physical assessment of the patient, followed by the assignment of a visit category and triage priority level by the nurse [15].

    Several countries have standardized the mandatory collection of ED data. Canadian [16,17] and Australian [18] EDs report a triage minimum data set of structured complaint code fields. In addition to these nationally coordinated triage data collection efforts, there are regional databases for the local monitoring of injuries or syndromic surveillance (eg, toxic drug supplies and infectious disease outbreaks) [19]. The triage data collected between systems will vary, but the data types can be grouped into either structured or unstructured data, with each data type having its own strengths and weaknesses.

    Structured data force the triage nurse to select from one of several preformatted options and restrict the types of data that can be entered into any given data field. Examples of structured triage data include arrival time, vital signs, demographic information (ie, age and sex), insurance status, categorical chief complaints, and numerical triage acuity score. Structured data are the most frequently reported data generated during triage [4,5]. Structured data are readily available (owing to their routine collection) and simple to analyze and report compared with unstructured data; however, this convenience comes at a loss of contextual detail that is available from unstructured narratives [5].

    Unstructured clinical data include free-text written notes or “narrative” [20]. Narratives generated at triage vary in length and structure depending on the electronic health record and triage system used. The narrative typically includes the triage nurse’s assessment and the patient’s reported reason for visiting the ED. These data are unstructured and allow nurses to record the chief complaint in the patient’s own words, descriptions of events associated with the ED presentation, and their physical examination findings [21].

    Two systematic reviews that focused on injuries examined whether unstructured clinical narratives, including those generated at triage, could be used for large-scale injury surveillance [22,23]. These reviews summarized how narrative data were used to gather injury information and highlighted how data fields were interrogated [22,23]. Cumulatively, the reviews examined 2831 studies published over 18 years and included 56 studies, 13 of which used ED triage data [22,23]. They reported that narrative data use has increased over time and that analyzing the data required automatic or manual extraction of keywords or ML techniques. The review authors were critical of data heterogeneity and called for improved data collection methods [22,23]. The heterogeneity noted in these studies may be partially explained by the wide range of administrative data set types interrogated. A more homogeneous data set (ie, triage narratives alone) may have offered alternative insights.

    Two additional review studies published in 2013 focused their analyses on studies using triage narratives for syndromic surveillance systems (ie, programs that monitor for disease outbreaks) [19,24]. Syndromic classifiers use chief complaint narratives to group patient visits into categories to monitor for changes (eg, outbreaks) in disease presentations. The first systematic review screened 89 studies identified through a structured search limited to PubMed to examine syndromic classifiers for detecting influenza in ED triage data sets [24]. The authors included 38 studies that met their inclusion criteria: (1) examined clinical data, which was (2) generated in the ED, and (3) examined influenza. The most commonly used method to identify cases was chief complaint classification. The authors noted that ED triage narratives allowed for large-scale research and program evaluation, but no details on the structure of or methods for extracting chief complaint classification data were offered [24].

    The second 2013 nonsystematic review also focused on syndromic surveillance. This review offered descriptive details on the structure of syndromic surveillance systems and their data [19]. The review included 17 studies drawn from an undisclosed initial sample and identified 15 chief complaint classifier systems of interest. The authors described the geographic location where each system was in use and the process used by each system to group visits into syndromes and detailed the relative strengths and weaknesses of each system. The review noted that all but 1 system (from Canada) was American and that the classifiers used differing degrees of computer text parsing to assign patients into groups (eg, ranged from 4 to 9 syndromes) and classified the approach of each system by keyword, statistical, or linguistic methods. The authors highlighted that statistical methods relied on large data volumes to be robust to the “noisy” inputs found in narrative text. By contrast, keyword and linguistic methods used keyword-based strategies and were described as disadvantageous because time-intensive adjustments were needed to accommodate variations in triage vocabulary. The drawbacks of keyword-based methods were balanced by the transparency offered when compared with ML studies. The authors argued that triage narratives are of great utility for disease surveillance and were less critical of variations in the initial data quality, concluding that there is a need for common syndromic definitions to improve the utility of these data.

    Despite the use of triage data for multiple purposes, there is a criticism of the methods used to classify triage narratives and a call for improved consistency and quality in their collection. There are documented efforts to create common data definitions for triage narratives [25] and to create national ED nursing data sets [26]; however, unstructured data are not as widely collected as structured data [7], and there is a paucity of literature examining what structures are common to triage narratives. This scoping review addresses these concerns and examines peer-reviewed literature to identify what ED triage narrative data have been used for, studies that may be sufficiently similar to compare, and the need for additional research. This scoping review systematically examines the evidence to determine what, if any, structures underlie these narrative data and describes what the data have been used for.

    Objectives

    The objectives of this review were as follows:

    1. Describe the current literature on the use of ED nurses’ triage narratives
    2. Describe the objectives and findings of the included studies
    3. Determine whether there are sufficient data to systematically review the structure or descriptions of triage narratives
    4. Determine whether there is adequate consistency in the included studies to support further review of the outcomes.

    Overview

    In this review, we used the scoping framework proposed by Arksey and O’Malley [27,28]. The protocol was published previously [29]. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) framework was used to guide reporting [30]. To identify studies that examined unstructured narratives in the ED, we conducted a search using controlled terminology for the main topics of health record narratives, emergency, and triage. A medical librarian refined the search terms, and prespecified filters were used for ED [31-34]. To maximize the breadth of the retrieved studies, a comprehensive search was conducted in CINAHL, Ovid MEDLINE, Ovid Embase, Cochrane Library (via Wiley), and ProQuest Central. The search was limited to peer-reviewed literature published after 1990, four years before the first nationally implemented triage system [12]. The reference lists of select excluded studies, namely those that examined the free-text narratives of emergency physicians and review studies that included triage narratives, were hand searched for inclusion. There were no deviations from the published protocol [29].

    Data were downloaded into Covidence (Veritas Health Innovation) for screening. The studies were screened independently by 2 authors (CP and MJD) in 2 stages (title plus abstract and then full text) using prepiloted screening forms. Any peer-reviewed studies that examined unstructured narratives [22,35] that were generated within an ED [36] by a nurse [37] were included. Studies that examined disaster triage systems, studies that did not have full text (ie, abstracts only), and non-English studies were excluded. Cohen κ was used to gauge agreement during screening, and all conflicts were settled by consensus. There were no deviations from the study protocol, which outlined the screening forms and operational definitions [29].

    Data Extraction

    The data were extracted into Microsoft Excel (version 2019, Microsoft Corp; by CP) using prepiloted forms. The results were independently confirmed by a second reviewer (MJD). Counts and proportions were used to describe categorical and numeric values. The extracted categorical values included study variables such as study design, country of origin, triage system used, and the use of ML. The extracted numerical data included the publication year, number of EDs from which the data were drawn, number of visits or patients included in the initial and final samples, and the number of nurses included in each study. For studies that reported data as minimum values (ie, “there were over three million of visits”) [38-45], values were recorded as the minimum stated value (ie, 3 million). When studies reported using quality or reporting frameworks, we reported the tool by name. The main conceptual categories of each study (ie, the objectives, design, population, and results) were described [46]. We summarized the descriptions of the triage narratives and keywords when the narratives were reported in the study. When 5 or fewer keywords were used, they were recorded verbatim.

    Data Analysis

    Owing to the wide distribution of data, estimates of central measures were calculated using both median (with IQRs) and minimum and maximum counts. Statistical analyses were performed using SPSS (version 25, IBM Corp). Citation management was performed using Zotero (Corporation for Digital Scholarship). The study objectives were categorized dichotomously (ie, yes or no) based on whether ML was used in the study (defined as any form of artificial intelligence), and the y were grouped into exclusive categories according to the primary use of the triage narratives: case identification, predictor variable, or quality improvement.


    Overview of Studies

    A total of 25,091 studies were identified in the initial search, and after deduplication, 18,074 (72.03%) studies underwent title and abstract screening. The proportionate agreement between reviewers (CP and MJD) during screening was 97.4% for the excluded studies and 98.1% for the included studies (Cohen κ=0.250). A full-text review was completed for 214 studies, and 131 (61.2%) studies were excluded at this stage, primarily for not specifying whether the narratives were generated by a nurse at triage (67/131, 51.1%). All review studies identified in the initial search that discussed narrative (although excluded) underwent citation screening in the primary search that discussed triage or ED narratives underwent citation screening. An additional 13 studies were included at this stage (Figure 1).

    ‎
    Figure 1. PRISMA diagram.
    View this figure

    Study Designs

    Retrospective design was the most common approach (80/96, 83%; Multimedia Appendix 1). Data were typically drawn (in part or entirely) from electronic databases, except in earlier studies, in which data were manually abstracted from paper charts [47-49]. The studies used data from hospitals (63/96, 66%) or regional databases (33/96, 34%). All studies reported on the unstructured narratives generated at triage; however, there was significant variation in the types and details of additional data reported. The most commonly collected non–triage-narrative data were patient demographic data, namely age (63/96, 66%), sex (60/96, 62%), and vital signs (29/96, 30%); visit details, namely chief complaint codes (57/96, 59%), discharge status (53/96, 55%), arrival date (35/96, 36%), and time (32/96, 33%); and ED data, namely triage system used (41/96, 43%; Multimedia Appendix 2 [22,36]). There was a weak relationship between the number of items collected and time, with 12% (R2=0.122) of the variation being attributable to publication date (r94=0.35; P<.001). The number of EDs included was reported in 92% (88/96) of studies. The initial data set size was reported in 81% (78/96) of studies. Of the 96 included cases, 76 (79%) reported the number of visits, and 28 (29%) reported the number of patients. The number of nurses who generated the narratives used in the study was reported in 9% (9/96) of studies [38,48-55].

    The median study size included 12,103 (IQR 803-150,089) visits or 391 (IQR 391-76,069) patients from an initial sample of 60,231 (IQR 2943-461,435) visits from (IQR 1-12) 2 EDs (Table 1). There was a large variation in the numbers of visits and departments examined, with the included sample sizes ranging from fewer than 100 to >2 million visits. These visits were drawn from databases ranging from 100 to >14 million visits and reflected as few as 1 ED and as many as 496 EDs (Table 1). There was an increase in the number of studies performed and median sample size of studies in each 6-year period between 1998 and 2021, with 61% (59/96) of the studies published in the last 6 years, that is, after 2015. The median sample sizes increased after 2009 from 7951 (IQR 518-55,952) to 160,944 (IQR 19,418-501,758). There was a concurrent increase in the frequency of ML use as a primary tool, with 77% (30/39) of studies after 2017 using ML use as a primary tool (Table 1). We noted that ML was used more frequently in predictive studies (21/25, 84%) than in studies using narratives for case identification (17/58, 29%) or quality (1/13, 8%; Figure 2).

    Geographically, the United States (43/96, 45%), Australia (31/96, 32%), and Canada (5/96, 5%) represented 82% (79/96) of the published papers; 1 study was reported each from South America and Africa (Table 2). The studies coming from countries with official languages other than English [51,54,56-59] were from countries that adopted or adapted the existing triage systems. Other countries with large English-speaking populations are either underrepresented (England and New Zealand) or not represented at all (South Africa, Wales, Ireland, and Scotland; Multimedia Appendix 1).

    Table 1. Study characteristics by publication year.

    		Studies (n=96)Included EDsaInitial sampleIncluded visitsIncluded patientsIncluded nursesStudies using MLb methods (n=39)



    		Totalc (n=2438)Studiesd (n=88)Totalc (n=63,413,919)Studiesd (n=78)Totalc (n=12,220,866)Studiesd (n=76)Totalc (n=1,804,813)Studiesd (n=28)Totalc (n=3844)Studiesd (n=9)
    Year, n (%)

    		19981 (1.04)1 (0.04)1 (1.14)104 (0.0001)1 (1.28)104 (0.0008)1 (1.32)104 (0.01)1 (3.57)0 (0)0 (0)0 (0)

    		19992 (2.08)2 (0.08)2 (2.27)100 (0.001)2 (2.56)100 (0.0008)2 (2.63)100 (0.01)2 (7.14)24 (0.62)2 (2.22)0 (0)

    		20000 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)0 (0)

    		20012 (2.08)497 (20.39)2 (2.27)98,672 (0.16)2 (2.56)84,000 (0.69)1 (1.32)0 (0)0 (0)0 (0)0 (0)0 (0)

    		20021 (1.04)1 (0.04)1 (1.14)11,861 (0.02)1 (1.28)0 (0)0 (0)305 (0.02)1 (3.57)0 (0)0 (0)0 (0)

    		20032 (2.08)5 (0.21)2 (2.27)43,078 (0.07)1 (1.28)17,413 (0.14)2 (2.63)0 (0)0 (0)0 (0)0 (0)1 (2.56)

    		20043 (3.12)23 (0.94)3 (3.41)1,021,949 (1.61)3 (3.85)21,949 (0.18)2 (2.63)73,115 (4.05)2 (7.14)0 (0)0 (0)2 (5.13)

    		20054 (4.17)14 (0.57)3 (3.41)579,032 (0.91)3 (3.85)1510 (0.01)2 (2.63)86,079 (4.77)2 (7.14)0 (0)0 (0)3 (7.69)

    		20061 (1.04)1 (0.04)1 (1.14)46,602 (0.07)1 (1.28)45,329 (0.37)1 (1.32)0 (0)0 (0)50 (1.3)1 (11.11)0 (0)

    		20071 (1.04)1 (0.04)1 (1.14)521 (0.0008)1 (1.28)419 (0.003)1 (1.32)0 (0)0 (0)0 (0)0 (0)0 (0)

    		20082 (2.08)95 (3.9)2 (2.27)119,479 (0.19)2 (2.56)5956 (0.05)2 (2.63)0 (0)0 (0)0 (0)0 (0)1 (2.56)

    		20092 (2.08)14 (0.57)2 (2.27)3,556,352 (5.61)2 (2.56)1,089,984 (8.92)1 (1.32)389 (0.02)1 (3.57)0 (0)0 (0)0 (0)

    		20101 (1.04)2 (0.08)1 (1.14)263,937 (0.42)1 (1.28)19,252 (0.16)1 (1.32)0 (0)0 (0)0 (0)0 (0)0 (0)

    		20111 (1.04)6 (0.25)1 (1.14)794 (0.001)1 (1.28)794 (0.001)1 (1.32)0 (0)0 (0)2 (0.05)1 (11.11)1 (2.56)

    		20125 (5.21)182 (7.47)5 (5.68)12,810,122 (20.2)3 (3.85)71,427 (0.58)4 (5.26)519 (0.03)1 (3.57)27 (0.7)2 (22.22)1 (2.56)

    		20133 (3.12)4 (0.16)2 (2.27)348,895 (0.55)1 (1.28)41,624 (0.34)1 (1.32)798 (0.04)1 (3.57)0 (0)0 (0)0 (0)

    		20143 (3.12)282 (11.57)3 (3.41)16,074,953 (25.35)3 (3.85)43,114 (0.35)2 (2.63)38,479 (2.13)1 (3.57)3538 (92.04)1 (11.11)1 (2.56)

    		20153 (3.12)74 (3.04)3 (3.41)13,051,141 (20.58)2 (2.56)310,353 (2.54)3 (3.95)0 (0)0 (0)0 (0)0 (0)1 (2.56)

    		20164 (4.17)109 (4.47)3 (3.41)13,194 (0.02)3 (3.85)2972 (0.02)2 (2.63)369 (0.02)1 (3.57)0 (0)0 (0)0 (0)

    		20177 (7.29)345 (14.15)5 (5.68)2,450,310 (3.86)5 (6.41)2,287,592 (18.72)7 (9.21)0 (0)0 (0)0 (0)0 (0)2 (5.13)

    		20189 (9.38)18 (0.74)8 (9.09)195,014 (0.31)8 (10.26)59,801 (0.49)8 (10.53)183 (0.01)2 (7.14)10 (0.26)1 (11.11)3 (7.69)

    		201912 (12)641 (26.29)10 (11.36)5,453,665 (8.6)10 (12.82)3,426,182 (28.04)10 (13.16)153,145 (8.49)3 (10.71)193 (5.02)1 (11.11)7 (17.95)

    		202014 (14.58)29 (1.19)14 (15.91)4,183,453 (6.6)12 (15.38)3,372,239 (27.6)10 (13.16)1,029,147 (57.02)7 (25)0 (0)0 (0)10 (25.64)

    		202113 (13.54)92 (3.77)13 (14.77)3,090,691 (4.87)10 (12.82)1,318,752 (10.79)12 (15.79)422,081 (23.39)3 (10.71)0 (0)0 (0)6 (15.38)
    Value, median (IQR)e2.5 (1-4.25)2 (1-12)N/Af60,231 (2943-461,435)N/A12,103 (803-150, 089)N/A391 (240-76,069)N/A15 (10-50)N/A1 (0-2)
    Value, rangee0-141-496N/A50-14,000,000N/A29-2,100,000N/A29-412,858N/A2-3538N/A0-10

    aED: emergency department.

    bML: machine learning.

    cThe totals represent pooled data from all studies generated in that particular year.

    dThe number of studies represents how many studies the total was distributed across.

    eMedian (IQR) and range values were calculated based on individual study sample sizes; results reported by year are pooled.

    fN/A: not applicable.

    ‎
    Figure 2. Triage narrative uses.
    View this figure
    Table 2. Study characteristics by country.
    CountryStudies (n=96), n (%)Included EDsaInitial sampleIncluded patientsIncluded visitsIncluded nursesStudies using MLb methods (n=39), n (%)


    		Total (n=2438), nc (%)Studies (n=88), nd (%)Total (n=63,413,919), nc (%)Studies (n=79), nd (%)Total (n=1,804,813), nc (%)Studies (n=28), nd (%)Total (n=12,220,866), nc (%)Studies (76), nd (%)Total (n=3844), nc (%)Studies (n=9), nd (%)
    United States43 (44.79)2008 (82.36)39 (44.32)36,528,693 (57.6)35 (44.30)916,955 (50.81)12 (42.86)4,986,560 (40.80)34 (44.74)3781 (98.36)3 (33.33)22 (56.41)
    Australia31 (32.29)404 (16.57)27 (30.68)23,110,878 (36.44)24 (30.38)1996 (0.11)5 (17.86)4,784,753 (39.15)24 (31.58)2 (0.05)1 (11.11)10 (25.64)
    Canada5 (5.21)7 (0.29)5 (5.68)6450 (0.01)4 (5.06)573 (0.03)3 (10.71)19,727 (0.16)4 (5.26)20 (0.52)1 (11.11)1 (2.56)
    Israel3 (3.12)3 (0.12)3 (3.41)1,586,760 (2.50)3 (3.8)648,294 (35.92)3 (10.71)1,361,455 (11.14)2 (2.63)0 (0)0 (0)2 (5.13)
    Great Britain2 (2.08)2 (0.08)2 (2.27)11,911 (0.02)2 (2.53)355 (0.02)2 (7.14)50 (0.0004)1 (1.32)4 (0.1)1 (11.11)0 (0)
    Brazil1 (1.04)1 (0.04)1 (1.14)499,853 (0.79)1 (1.27)0 (0)0 (0)499,853 (4.09)1 (1.32)0 (0)0 (0)1 (2.56)
    Switzerland1 (1.04)1 (0.04)1 (1.14)0 (0)0 (0)519 (0.03)1 (3.57)519 (0.004)1 (1.32)15 (0.39)1 (11.11)0 (0)
    China1 (1.04)1 (0.04)1 (1.14)44,237 (0.07)1 (1.27)295 (0.02)1 (3.57)295 (0.002)1 (1.32)0 (0)0 (0)0 (0)
    Spain1 (1.04)1 (0.04)1 (1.14)2080 (0.003)1 (1.27)0 (0)0 (0)1572 (0.01)1 (1.32)0 (0)0 (0)0 (0)
    Finland1 (1.04)2 (0.08)1 (1.14)52,032 (0.08)1 (1.27)0 (0)0 (0)42,247 (0.35)1 (1.32)12 (0.31)1 (11.11)0 (0)
    France1 (1.04)1 (0.04)1 (1.14)80,320 (0.13)1 (1.27)0 (0)0 (0)806 (0.01)1 (1.32)0 (0)0 (0)0 (0)
    Iran1 (1.04)1 (0.04)1 (1.14)537 (0.0008)1 (1.27)0 (0)0 (0)537 (0.004)1 (1.32)10 (0.26)1 (11.11)1 (2.56)
    South Korea1 (1.04)1 (0.04)1 (1.14)142,972 (0.23)1 (1.27)0 (0)0 (0)138,022 (1.13)1 (1.32)0 (0)0 (0)1 (2.56)
    New Zealand1 (1.04)1 (0.04)1 (1.14)1000 (0.001)1 (1.27)0 (0)0 (0)1000 (0.01)1 (1.32)0 (0)0 (0)0 (0)
    Portugal1 (1.04)1 (0.04)1 (1.14)599,276 (0.95)1 (1.27)235,826 (13.07)1 (3.57)0 (0)0 (0)0 (0)0 (0)1 (2.56)
    Portugal and United States1 (1.04)2 (0.08)1 (1.14)719,925 (1.14)1 (1.27)0 (0)0 (0)356,475 (2.92)1 (1.32)0 (0)0 (0)0 (0)
    Uganda1 (1.04)1 (0.04)1 (1.14)26,995 (0.04)1 (1.27)0 (0)0 (0)26,995 (0.22)1 (1.32)0 (0)0 (0)0 (0)

    aED: emergency department.

    bML: machine learning.

    cThe totals represent pooled data from all studies generated in that particular country.

    dThe number of studies represents how many studies the total was distributed across.

    Study Objectives

    The most common objectives for studies using triage narratives were to perform case identification (59/96, 61%), to use narratives as a predictor variable in ML models (21/96, 22%), and to use narratives for quality improvement (16/96, 17%; Table 3). Studies categorized with case identification as their primary objective sought to describe incidence or prevalence estimates or populations of interest. Studies that used narratives as a predictor variable predicted patient acuity scores, resource use, or specific diagnoses.

    Quality improvement studies used triage narratives to increase clinician or system safety and were subdivided as pertaining to reliability, accuracy, and validity or safety and efficiency. Reliability and validity studies examined interrater reliability and were used to assess whether the triage classification matched specific populations with specific categorical assignments or triage acuity scores. Safety and efficiency studies examined narratives to improve data quality or reduce errors and effort (Table 3).

    ML consisted of several models, and we used an inclusive approach by combining all ML, natural language processing, and other artificial intelligence models. We noted the frequency of ML use to be increasing and that ML was more frequently used in predictive studies (21/25, 84%) than in studies using narratives for case identification (17/58, 29%) or quality (1/13, 8%; Figure 2).

    Table 3. Summary of study objectives.
    Study category and types of papers in the categoryExplanation
    Quality improvement

    		Accuracy, validity, and reliabilityStudies used triage narratives from previous EDa visits as a research instrument. These studies would have nurses or physicians rescore visits and compare the scores to calculate the reliability, validity, accuracy, or interrater agreement of providers for specific triage systems [48,49,51,54,60-62].

    		Safety and efficiencyThese studies examined quality as the completeness of triage data [47], as how time-sensitive presentations were handled at triage [63,64], and to identify or improve errors in acuity or category assignment [51,54,62,65,66]. Other studies focused on improving triage and measured the amount of duplicate or redundant information within triage narratives [67] or the efficiency [42,54,55], accuracy [55], and completeness [55,58,68,69] of narratives.
    Case identification

    		Syndromic classificationThese studies had a primary objective of developing, describing, or comparing syndromic surveillance systems. These systems attempt to group all patients from a single large cohort into one of several broadly defined groups to assign a reason for visit category [38,42-44,50,52,53,70-76].

    		Estimate incidence or describe a populationTriage narratives have been used as an alternate means of identifying general or specific presentations. General grouping included cases related to drugs or alcohol [39,40,77-86], sports [69,87-90], motor vehicle collisions [41,90-93], mental health–related presentations, [94-100], environmental injuries [45,101-103], infections [104,105], assaults [106,107], and animal bites [45,108,109]. Narratives seem to be particularly good at identifying rare cases [107,110-113]. Narratives have also been used to provide granular data about patients, such as temporal information [114], to complete missed vitals [115] and to provide contextual details such as events leading to an injury [39,78,87,89-91,116].
    Prediction

    		Acuity or resource usePredictions using triage narratives attempted to forecast the resource uses by patients in general [117] or addressed specific aspects of care, including the need for admission [118-122], triage acuity [57,59,123-126], length of stay [119], critical illness [124], and mortality, [57,127,128].

    		Specific diagnosesTriage narratives were used as a covariate for machine learning models that predicted specific resource or admission needs. Admission destinations and resources of interest included advanced diagnostic imaging use [56,129,130], mental health admission [131], ICUb admission [132], or neuro-intensive care unit admission [133].

    aED: emergency department.

    bICU: intensive care unit.

    Descriptions of Triage Narratives

    The quality and structure of the triage narratives used in each study were not clearly stated. Of the 96 studies included, only 30 (31%) described the narrative. The most common approach to describing narratives was a description of the triage narrative or of the keywords used to search within the narrative (Table 4).

    Table 4. Descriptions of the structure of triage narratives.
    Study, yearDescription of the triage narrativeaKeywords, nbKeyword topics
    Travers and Haas [75], 2003There was a description of the characteristic components of the narrative chief complaints that were not matched by machine learning: punctuation, truncations, modifiers, and qualifiers were discussedN/AcN/A
    Chapman et al [104], 2004N/A5The following fever-related keywords were used: “fever(s),” “Febrile,” “chill*,” and “low grade temp*”
    Day et al [43], 2004The mean length of the triage narratives was 14.6 (SD 7.9) words in each database6Shortness of breath and difficulty in breathing were examined
    Thompson et al [38], 2006The maximum allowable space for triage narratives was 40 characters>100Keywords for chest pain, syncope, earache, and others
    Indig et al [39], 2010The average triage note was 35 words (including abbreviations) per presentation; there was a secondary text field that was not discussed>160Drug and alcohol keywords
    Bregman and Slavinski [109], 2012N/A2Mammal bite–related terms and their associated animals were examined using the search terms “bite” and “animal”
    McKenzie et al [108],2010N/A50Work, worker, and work-related keywords and truncations
    Vallmuur et al [79], 2013N/A18Alcohol-related keywords
    Mitchell and Bambach [91], 2015N/A32Alcohol- and vehicular collision–related keywords
    Luther et al, [101], 2016N/A1Presentations with the keyword “heat”
    Rahme et al [95], 2016N/A16Suicide-related keywords were identified in both English and French
    Whitlam et al [81], 2016N/A12Alcohol-related keywords
    DeYoung et al [40], 2017N/A>150Cannabis-related keywords
    Kondis et al [107], 2017N/A2“Crying” and “fussy” were the search keywords reported; however, variations in these terms were also included (although not specified by the authors)
    Harduar Morano et al [102], 2017N/A11Heat injury–related keywords
    Zhang et al [118], 2017N/A25A list of keywords predictive of patient admission
    Chu et al [110], 2018N/A1“Headache”
    Gligorijevic et al [117], 2018N/A24Mixed keywords for a variety of presentations
    Goldman-Mellor et al [131], 2018N/A8Mental health and substance use–related keywords
    Hargrove and Waller [92], 2018N/A23Vehicle collision–related keywords
    Nagabhushan and Webley [111], 2018N/A2Specific chest pain feature keywords “ripping” and “tearing”
    Chen et al [89], 2019N/A2“Tramp” and “bounce” were specified, but other terms may have been used
    Eley et al [90], 2019N/A14Bicycle-related keywords
    Marx et al [83], 2019N/A8Marijuana-related keywords
    Trivedi et al [93], 2019N/A3Electric scooter–related brand names “bird” and “lime” as well as “scooter” were the keywords
    Sterling et al [126], 2020The mean length of triage narrative was 143.17 (SD 77.8) characters (excluding spaces) or 64.3 (SD 35.2) wordsN/AN/A
    Vernon et al [41], 2020N/A3Electric scooter–related keywords and their variations were searched. “Scooter,” “e-scooter,” and “electric-scooter” were offered as specific terms
    Ivanov et al [125], 2021The average number of clinical features per text entry was 12.79. There was no discussion about character or word countsN/AN/A
    Rahilly-Tierney et al [86], 2021N/A3“Heroin” and “overdose” were specified as inclusion terms and “detoxification” as an exclusion term; although there may have been additional terms included, they were not specified
    Rozova et al [99], 2021The average triage note was 127 characters long (notes with <30 characters were excluded)40Suicide-related keywords

    aStudies reporting only the process of cleaning and normalizing unstructured narratives were not included.

    bVariations in spelling, abbreviations, bigram duplications, and negation terms were counted if specified.

    cN/A: not applicable.

    There were 7 studies that described triage narratives [38,39,43,75,99,125,126]. The descriptions included the counts of characters and words used in the typical triage narrative. The length of the triage narrative entries in these studies ranged from 40 [38] to 127 characters [99] and 14.6 [43] to 35 words (including abbreviations) [39] (Table 4). One study described the narratives in terms of “clinical features” [125]. “Clinical features” in this study were Unified Medical Language System clinical terms that the authors derived using a natural language processing algorithm (C-NLP), but it is unclear how much these differ from their input data or whether they can be compared with those in other studies.

    There were 27 studies that reported on the specific keywords that were present within the narratives [38-41,43,79, 81,83,86,89-93,95,99,101,102,104,107-111,117,118,125,126,131]. The number of keywords ranged from 1 [101] to >160 [39], with a median number of 11 (IQR 3-24.5) keywords reported (Table 4). However, 11% (3/27) of studies did not report the exact number of keywords used [38-40]. The authors reported the use of express keywords with correct spellings [86,93,101,107,109-111] as well as intentional variations such as misspellings, abbreviations, or truncations [39,40,81,92,108]. One of the studies searched for terms using keywords in 2 languages (English and French) [95].

    In total, 9 studies reported the number of nurses who generated the narratives [38,48-55]. The total number of nurses whose documentation was assessed in these studies was 3844. The median sample size of nurses was 15 (IQR 10-50), and the sample size ranged from 2 [50] to 3538 [53]. These 9 studies represent only 3% of the total sample size (n=367,946). One of the studies reported on both the structure of triage narratives and the number of nurses included in the sample [38].

    The most in-depth descriptions were provided by Travers and Haas [75], who explored triage narratives in depth by describing the structure of the narratives and regional variations. This 3-center retrospective cohort study used verbatim triage chief complaint narratives drawn from EDs in the United States. In a corpus of 13,494 unique chief complaint narratives drawn from 39,038 visits, they used manual and automated techniques to identify chief complaint concepts using the Unified Medical Language System data definitions. Concepts that were not readily classified using ML models were described in both form and function, and the authors detailed the function of the punctuation, acronyms and abbreviations, truncations, modifiers, and qualifier words used in triage narratives [75].

    Although quality appraisal can be incorporated into scoping reviews [30], we did not opt to include one because our primary aim was to describe the literature rather than assess each study’s findings [27,28]. Consequently, we are limited to reporting that 8% (8/96) of the included studies used an Enhancing the Quality and Transparency of Health Research Network quality reporting guideline (Table 5). In total, 4% (4/96) of studies used reporting guidelines specifically for predictive models [62,99,124,129], and 1% (1/96) of studies reported using a quality framework to guide data cleaning and the protection of patient information [124].

    Table 5. Studies that used reporting guidelines and the types of guidelines used.
    Study, yearReporting guidelineGuideline body
    Chu et al [110], 2018The RECORDa statementEQUATORb Network
    Jones et al [82], 2019The STROBEc statement: guidelines for reporting observational studiesEQUATOR Network
    Trivedi et al [93], 2019The STROBE statement: guidelines for reporting observational studiesEQUATOR Network
    Zhang et al [129], 2019GRRASdEQUATOR Network
    Joseph et al [124], 2020(1) HIPAAe Safe Harbor method and (2) The TRIPODf statement(1) US Department of Health and Human Services and (2) EQUATOR Network
    Cheung and Leung [62], 2021GRRASEQUATOR Network
    Lam et al [85], 2021The RECORD statementEQUATOR Network
    Rozova et al [99], 2021The TRIPOD statementEQUATOR Network

    aRECORD: Reporting of Studies Conducted Using Observational Routinely Collected Health Data.

    bEQUATOR: Enhancing the Quality and Transparency of Health Research.

    cSTROBE: Strengthening the Reporting of Observational Studies in Epidemiology.

    dGRRAS: Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research.

    eHIPAA: Health Insurance Portability and Accountability Act.

    fTRIPOD: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.


    Principal Findings

    We performed a scoping review to examine studies reporting on the structure and use of triage nurse narratives. Our search was systematic, used a prepublished protocol, and screened a significant number of studies published over a 32-year period. Our study protocol was registered and published and used standardized screening templates and data extraction forms [29]. Our search intentionally sacrificed specificity for sensitivity, including a substantial number of studies in keeping with the scoping review design. The volume of studies retrieved demonstrates that identifying triage narrative in academic literature is difficult and that straightforward ways of identifying pertinent studies are needed. Studies would be more readily identifiable if their keywords, titles, and abstracts were clear and consistent.

    In addition to the triage narrative, we found that the most frequently reported data were patient age, sex, chief complaint category, discharge status, and triage acuity, similar to a 2020 systematic review of ML for clinical decision support in the ED [5]. Similar to other review studies, we found an increase in the number of studies conducted over time [3]. We found a sharp increase in the sample size of studies after 2008. Our findings also support that the studies using ML lag behind studies of health record data. However, we noted that this trend continued only until 2017, when ML became the most common approach reported in the literature. Wang et al [3] tabulated the top sources of electronic health record narratives and determined that the most common sources were discharge summaries (n=26, 45% of studies), progress notes (n=15, 26%), admission notes (n=9, 16%), operative notes (n=5, 9%), and primary care notes (n=3, 5%). We identified 5 studies [71,72,108,121,134] that used ML. ML studies were challenging to identify through structured searches. Similar to our review, Wang et al [3] determined that most studies were conducted in the United States. They identified fewer (3/263, 1%) studies from Australia. In comparison, our study identified that 56% (10/18) of the studies originated from Australia during the same period [50,108]. Our results differ in part because we did not restrict our search in the same manner as Wang et al [3], who explicitly examined ML, and rather focused on unstructured narratives as a primary search concept.

    The previously discussed reviews and several other studies included in this review established that triage narratives can improve case identification when used in isolation or when added to diagnosis codes [22]. The use of narratives for these purposes was reported as a straightforward process in several studies that showed that their inclusion or exclusion can substantially impact the number of cases identified [72,78,79]. Refinement of these techniques may improve the sensitivity of searches and have significant impacts on disease prevalence estimates for diagnoses (eg, rare illnesses) that may not be well captured with administrative discharge codes, a common method for tracking population illnesses [113,135]. The methods used in keyword-based case identification studies are well positioned for implementation, given their clearly defined and reproducible methods and long history of being used for these purposes. Studies of disease prevalence were among the first to use narratives collected on a large scale [42,75]. The potential improvements to the sensitivity and specificity of case identification may justify the systematic review of the studies included in this review. In addition, future research could focus on clearly defining the improvements that narrative data analysis can offer to case identification studies.

    There is a pressing need to collect nursing data [7], and triage has been identified as one of the most important areas for quality improvement [136]. Several studies have reviewed quality improvement efforts at triage [8] and called to include narratives in these efforts [137], but significant work is still needed. A renewal of early efforts to establish a minimum ED nurse data set [26] and efforts to create common definitions for narrative elements are needed [25], as is additional research to describe the structures of triage narratives in general. This work is required to determine whether there is a common structure in the data. Our results showed that even though 31% (30/96) of studies offered a description of narratives, only 1% (1/96) provided significant depth. A fulsome description is needed to ensure that triage nursing contextual data are not lost through text normalization (a typical early step in data cleaning for models), given that nurses use unique punctuation and abbreviations while recording triage narratives [75]. Finally, given the wide regional variations in the breadth and depth of information collected at triage, research is needed to identify the specific details that triage narratives should contain.

    The Strengthening the Reporting of Observational Studies in Epidemiology and Reporting of Studies Conducted Using Observational Routinely Collected Data guidelines were published in 2007 [138] and 2015 [139], respectively. However, only 8% (8/96) of the studies reported using a reporting guideline, even though 86% (83/96) of these studies were reported after 2007. Recently published reporting guidelines such as the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis [140] may contribute to more consistent reporting guideline use, and 2021 saw the highest (3/13, 23%) proportion of studies using a reporting guideline. The use of reporting guidelines will help reduce the heterogeneity noted in reporting.

    Limitations

    In total, 3 concepts (emergency, triage, and narrative) were searched using an inclusive search approach, resulting in a substantial number of studies. The level of agreement during screening was fair, but it was likely reduced owing to the large number of studies reviewed and the need for full-text reading to determine whether the narrative was nurse generated. Future refinements to the search strategy may enable a less wide-reaching search, and more clearly defined methods to identify nurse-generated narratives may decrease the number of studies for screening. In addition, clear methods for identifying when narratives are generated by nurses may prevent researchers from pooling nurses’ triage narratives with narratives generated by other clinicians such as physicians, which may result in more studies being positively identified as originating from triage nurses.

    Conclusions

    This review identified 96 studies that used triage narratives to achieve quality improvement, perform case identification, or make predictions about clinical outcomes. We have described how narrative use is changing to incorporate larger samples and more ML methods for interrogating the data. We have provided a strong argument that there is a considerable lack of research on the structure of triage narratives. Future research should not only focus on the outcomes of their study but also describe in detail the data sources used. Future researchers should strive to follow reporting guidelines to improve the quality of data reporting and increase the ability to pool and compare study findings. Emergency nursing scholars can encourage the national collection of triage data to allow comparison between regions if the common structures of data are better articulated.

    Authors' Contributions

    All the authors contributed to the design of this study. CP, MK, and MD initiated the project. The protocol was drafted by CP and refined by MK, HMO, CN, CM and MD. Screening was performed by CP and MD. Data extraction was performed by CP and confirmed by MD. CP performed the statistical analyses and was responsible for drafting the manuscript. MK supervised this study. All the authors have contributed to the manuscript read, refined, and approved the final manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Summary of the included studies.

    DOCX File , 74 KB
    ‎
    Multimedia Appendix 2

    Data items collected.

    DOCX File , 73 KB
    1. National hospital ambulatory medical care survey: 2019 emergency department summary tables. Open Minds. 2022 Dec 5.   URL: https://openminds.com/market-intelligence/resources/082222nhamcs2019edtables/ [accessed 2022-06-07]
    2. NACRS Emergency Department Visits and Length of Stay, 2018–2019 (XLXS). Canadian Institute for Health Information. 2019.   URL: https://www.cihi.ca/en/nacrs-emergency-department-visits-and-length-of-stay-2018-2019-xlxs [accessed 2022-06-07]
    3. Wang Y, Wang L, Rastegar-Mojarad M, Moon S, Shen F, Afzal N, et al. Clinical information extraction applications: a literature review. J Biomed Inform 2018 Jan;77:34-49 [FREE Full text] [CrossRef] [Medline]
    4. Ben-Israel D, Jacobs WB, Casha S, Lang S, Ryu WH, de Lotbiniere-Bassett M, et al. The impact of machine learning on patient care: a systematic review. Artif Intell Med 2020 Mar;103:101785. [CrossRef] [Medline]
    5. Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JM. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med 2020 Jan;102:101762. [CrossRef] [Medline]
    6. Sánchez-Salmerón R, Gómez-Urquiza JL, Albendín-García L, Correa-Rodríguez M, Martos-Cabrera MB, Velando-Soriano A, et al. Machine learning methods applied to triage in emergency services: a systematic review. Int Emerg Nurs 2022 Jan;60:101109. [CrossRef] [Medline]
    7. Picard C, Kleib M. Advancing emergency nurses’ leadership and practice through informatics. Can J Emerg Nurs 2020 Sep 17;43(3). [CrossRef]
    8. Austin EE, Blakely B, Tufanaru C, Selwood A, Braithwaite J, Clay-Williams R. Strategies to measure and improve emergency department performance: a scoping review. Scand J Trauma Resusc Emerg Med 2020 Jun 15;28(1):55 [FREE Full text] [CrossRef] [Medline]
    9. Nakao H, Ukai I, Kotani J. A review of the history of the origin of triage from a disaster medicine perspective. Acute Med Surg 2017 Oct 14;4(4):379-384 [FREE Full text] [CrossRef] [Medline]
    10. Mayer TA. Triage: history and horizons. Adv Emerg Nurs J 1997;19:1-11.
    11. Thompson J. Comprehensive Triage. Virginia: Reston Pub Co; 1982.
    12. Australasian College for Emergency Medicine. The Australasian Triage Scale. Emerg Med (Fremantle) 2002 Sep;14(3):335-336. [CrossRef] [Medline]
    13. Hinson JS, Martinez DA, Cabral S, George K, Whalen M, Hansoti B, et al. Triage performance in emergency medicine: a systematic review. Ann Emerg Med 2019 Jul;74(1):140-152. [CrossRef] [Medline]
    14. Harding KE, Taylor NF, Leggat SG. Do triage systems in healthcare improve patient flow? A systematic review of the literature. Aust Health Rev 2011;35(3):371. [CrossRef]
    15. FitzGerald G, Jelinek GA, Scott D, Gerdtz MF. Emergency department triage revisited. Emerg Med J 2010 Feb;27(2):86-92. [CrossRef] [Medline]
    16. Innes G, Murray M, Grafstein E. A consensus-based process to define standard national data elements for a Canadian emergency department information system. CJEM 2001 Oct 21;3(4):277-284. [CrossRef] [Medline]
    17. Grafstein E, Unger B, Bullard M, Innes G. Canadian Emergency Department Information System (CEDIS) presenting complaint list (Version 1.0). CJEM 2003 Jan 21;5(1):27-34. [CrossRef] [Medline]
    18. Non-admitted patient emergency department care NMDS 2019–20. Australian Institute of Health and Welfare. 2019.   URL: https://meteor.aihw.gov.au/content/index.phtml/itemId/699738 [accessed 2022-04-24]
    19. Conway M, Dowling JN, Chapman WW. Using chief complaints for syndromic surveillance: a review of chief complaint based classifiers in North America. J Biomed Inform 2013 Aug;46(4):734-743 [FREE Full text] [CrossRef] [Medline]
    20. Gandomi A, Haider M. Beyond the hype: big data concepts, methods, and analytics. Int J Inform Manag 2015 Apr;35(2):137-144. [CrossRef]
    21. Gilboy N, Travers D, Wuerz R. Re-evaluating triage in the new millennium: a comprehensive look at the need for standardization and quality. J Emergency Nursing 1999 Dec;25(6):468-473. [CrossRef]
    22. McKenzie K, Scott DA, Campbell MA, McClure RJ. The use of narrative text for injury surveillance research: a systematic review. Accid Anal Prev 2010 Mar;42(2):354-363. [CrossRef] [Medline]
    23. Vallmuur K. Machine learning approaches to analysing textual injury surveillance data: a systematic review. Accid Anal Prev 2015 Jun;79:41-49. [CrossRef] [Medline]
    24. Hiller KM, Stoneking L, Min A, Rhodes SM. Syndromic surveillance for influenza in the emergency department-A systematic review. PLoS One 2013 Sep 13;8(9):e73832 [FREE Full text] [CrossRef] [Medline]
    25. Barthell EN, Aronsky D, Cochrane DG, Cable G, Stair T, Frontlines Work Group. The Frontlines of Medicine Project progress report: standardized communication of emergency department triage data for syndromic surveillance. Ann Emerg Med 2004 Sep;44(3):247-252. [CrossRef] [Medline]
    26. Bradley V. Toward a common language: Emergency Nursing Uniform Data Set (ENUDS). J Emergency Nursing 1995 Jun;21(3):248-250. [CrossRef]
    27. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Social Res Methodol 2005 Feb;8(1):19-32. [CrossRef]
    28. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010 Sep 20;5:69 [FREE Full text] [CrossRef] [Medline]
    29. Picard CT, Kleib M, O'Rourke HM, Norris CM, Douma MJ. Emergency nurses' triage narrative data, their uses and structure: a scoping review protocol. BMJ Open 2022 Apr 13;12(4):e055132 [FREE Full text] [CrossRef] [Medline]
    30. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [FREE Full text] [CrossRef] [Medline]
    31. Kung J. A filter to retrieve studies related to emergency departments from the EMBASE database. John W. Scott Health Sciences Library, University of Alberta. 2021 Sep 20.   URL: https://docs.google.com/document/d/1sILOgAVTzMI192Mb5kLvVEoC7RO1UoiPm7dDfr-eYgU/edit [accessed 2022-06-07]
    32. Campbell S. A filter to retrieve studies related to emergency departments from the OVID MEDLINE database. John W. Scott Health Sciences Library University of Alberta. 2021 Apr 20.   URL: https://docs.google.com/document/d/1VH1Un8TzC3EXEKfytIF_W7I8-5LwRhU7JcIBDiPpaio/edit# [accessed 2022-06-07]
    33. Campbell S. A filter to retrieve studies related to emergency departments from the EBSCO CINAHL database. John W. Scott Health Sciences Library University of Alberta. 2021 Jun 19.   URL: https://docs.google.com/document/d/10OC5XozEekvZT4P-ymzHrvnXkBheYd5HzUI3-siuofc/edit# [accessed 2022-06-07]
    34. Campbell S. A filter to retrieve studies related to Emergency Departments from the Cochrane Library Databases. John W. Scott Health Sciences Library University of Alberta. 2021.   URL: https://docs.google.com/document/d/1rVKs9q5H1SVu2k5LcCXUdHywPp2MtQ46iyI7UotEbsU/edit [accessed 2022-12-08]
    35. Scheurwegs E, Luyckx K, Luyten L, Daelemans W, Van den Bulcke T. Data integration of structured and unstructured sources for assigning clinical codes to patient stays. J Am Med Inform Assoc 2016 Apr;23(e1):e11-e19 [FREE Full text] [CrossRef] [Medline]
    36. Khangura J, Flodgren G, Perera R, Rowe B, Shepperd S. Primary care professionals providing non-urgent care in hospital emergency departments. Cochrane Database of Systematic Reviews. 2012 Nov 14.   URL: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD002097.pub3/full [accessed 2022-12-20]
    37. Nursing definitions. International Council of Nurses.   URL: https://www.icn.ch/nursing-policy/nursing-definitions [accessed 2022-04-24]
    38. Thompson DA, Eitel D, Fernandes CM, Pines JM, Amsterdam J, Davidson SJ. Coded chief complaints—automated analysis of free-text complaints. Acad Emergency Med 2006 Jul;13(7):774-782. [CrossRef]
    39. Indig D, Copeland J, Conigrave K, Arcuri A. Characteristics and comorbidity of drug and alcohol-related emergency department presentations detected by nursing triage text. Addiction 2010 May;105(5):897-906. [CrossRef] [Medline]
    40. DeYoung K, Chen Y, Beum R, Askenazi M, Zimmerman C, Davidson AJ. Validation of a syndromic case definition for detecting emergency department visits potentially related to marijuana. Public Health Rep 2017 Jun 06;132(4):471-479 [FREE Full text] [CrossRef] [Medline]
    41. Vernon N, Maddu K, Hanna TN, Chahine A, Leonard CE, Johnson J. Emergency department visits resulting from electric scooter use in a major southeast metropolitan area. Emerg Radiol 2020 Oct 05;27(5):469-475. [CrossRef] [Medline]
    42. Aronsky D, Kendall D, Merkley K, James BC, Haug PJ. A comprehensive set of coded chief complaints for the emergency department. Acad Emerg Med 2001 Oct;8(10):980-989 [FREE Full text] [CrossRef] [Medline]
    43. Day FC, Schriger DL, La M. Automated linking of free-text complaints to reason-for-visit categories and International Classification of Diseases diagnoses in emergency department patient record databases. Annals Emergency Med 2004 Mar;43(3):401-409. [CrossRef]
    44. Lee SH, Levin D, Finley PD, Heilig CM. Chief complaint classification with recurrent neural networks. J Biomed Inform 2019 May;93:103158 [FREE Full text] [CrossRef] [Medline]
    45. Rhea SK, Weber DJ, Poole C, Waller AE, Ising AI, Williams C. Use of statewide emergency department surveillance data to assess incidence of animal bite injuries among humans in North Carolina. J Am Vet Med Assoc 2014 Mar 01;244(5):597-603. [CrossRef] [Medline]
    46. Peters M, Godfrey C, McInerney P, Munn Z, Tricco A, Khalil H. Chapter 11: Scoping Reviews (2020 version). In: Joanna Briggs Institute Reviewer's Manual. Adelaide, Australia: The Joanna Briggs Institute; 2017.
    47. Kabir A, Hanson R, Mellis CM, van Asperen PP. Is asthma documentation improved by computer-facilitated data entry? J Qual Clin Pract 1998 Sep;18(3):187-193. [Medline]
    48. Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S. Reliability of the Canadian emergency department triage and acuity scale: interrater agreement. Annals Emergency Med 1999 Aug;34(2):155-159. [CrossRef]
    49. Goodacre SW, Gillett M, Harris RD, Houlihan KP. Consistency of retrospective triage decisions as a standardised instrument for audit. J Accid Emerg Med 1999 Sep 01;16(5):322-324 [FREE Full text] [CrossRef] [Medline]
    50. Wagholikar AS, Lawley MJ, Hansen DP, Chu K. Identifying symptom groups from Emergency Department presenting complaint free text using SNOMED CT. AMIA Annu Symp Proc 2011;2011:1446-1453 [FREE Full text] [Medline]
    51. Grossmann FF, Zumbrunn T, Frauchiger A, Delport K, Bingisser R, Nickel CH. At risk of undertriage? Testing the performance and accuracy of the emergency severity index in older emergency department patients. Ann Emerg Med 2012 Sep;60(3):317-25.e3. [CrossRef] [Medline]
    52. Malmström T, Huuskonen O, Torkki P, Malmström R. Structured classification for ED presenting complaints - from free text field-based approach to ICPC-2 ED application. Scand J Trauma Resusc Emerg Med 2012 Nov 24;20(1):76 [FREE Full text] [CrossRef] [Medline]
    53. Haas SW, Travers D, Waller A, Mahalingam D, Crouch J, Schwartz TA, et al. Emergency Medical Text Classifier: new system improves processing and classification of triage notes. Online J Public Health Inform 2014 Oct 16;6(2):e178 [FREE Full text] [CrossRef] [Medline]
    54. Dehghani Soufi M, Samad-Soltani T, Shams Vahdati S, Rezaei-Hachesu P. Decision support system for triage management: a hybrid approach using rule-based reasoning and fuzzy logic. Int J Med Inform 2018 Jun;114:35-44. [CrossRef] [Medline]
    55. Greenbaum NR, Jernite Y, Halpern Y, Calder S, Nathanson LA, Sontag DA, et al. Improving documentation of presenting problems in the emergency department using a domain-specific ontology and machine learning-driven user interfaces. Int J Med Inform 2019 Dec;132:103981. [CrossRef] [Medline]
    56. Klang E, Barash Y, Soffer S, Bechler S, Resheff YS, Granot T, et al. Promoting head CT exams in the emergency department triage using a machine learning model. Neuroradiology 2020 Feb 10;62(2):153-160. [CrossRef] [Medline]
    57. Klug M, Barash Y, Bechler S, Resheff YS, Tron T, Ironi A, et al. A gradient boosting machine learning model for predicting early mortality in the emergency department triage: devising a nine-point triage score. J Gen Intern Med 2020 Jan;35(1):220-227 [FREE Full text] [CrossRef] [Medline]
    58. Mor S, Lev-Rn Z, Tal S. Is family history of coronary artery disease important in the emergency department triage? Int Emerg Nurs 2020 May;50:100855. [CrossRef] [Medline]
    59. Choi SW, Ko T, Hong KJ, Kim KH. Machine learning-based prediction of Korean triage and acuity scale level in emergency department patients. Healthc Inform Res 2019 Oct;25(4):305-312 [FREE Full text] [CrossRef] [Medline]
    60. Baumann MR, Strout TD. Evaluation of the Emergency Severity Index (version 3) triage algorithm in pediatric patients. Acad Emerg Med 2005 Mar 01;12(3):219-224 [FREE Full text] [CrossRef] [Medline]
    61. Jones PG, Dance A, Shah N. Accuracy of SNOMED-CT Chief Presenting Complaint Codes: an audit of 1,000 cases. N Z Med J 2020 Dec 04;133(1526):67-75. [Medline]
    62. Cheung KY, Leung LP. Validity and reliability of the triage scale in older people in a regional emergency department in Hong Kong. Hong Kong J Emergency Med 2020 Nov 18;28(2):65-71. [CrossRef]
    63. Mosley I, Morphet J, Innes K, Braitberg G. Triage assessments and the activation of rapid care protocols for acute stroke patients. Australas Emerg Nurs J 2013 Feb;16(1):4-9. [CrossRef] [Medline]
    64. Petruniak L, El-Masri M, Fox-Wasylyshyn S. Exploring the predictors of emergency department triage acuity assignment in patients with sepsis. Can J Nurs Res 2018 Jun 13;50(2):81-88. [CrossRef] [Medline]
    65. Hendin A, Eagles D, Myers V, Stiell IG. Characteristics and outcomes of older emergency department patients assigned a low acuity triage score. CJEM 2018 Sep;20(5):762-769. [CrossRef] [Medline]
    66. Metzger P, Allum L, Sullivan E, Onchiri F, Jones M. Racial and language disparities in pediatric emergency department triage. Pediatr Emer Care 2021 Apr 26;38(2):e556-e562. [CrossRef]
    67. Berendsen Russell S, Dinh MM, Bell N. Triage, damned triage… and statistics: sorting out redundancy and duplication within an Emergency Department Presenting Problem Code Set to enhance research capacity. Australas Emerg Nurs J 2017 Feb;20(1):48-52. [CrossRef] [Medline]
    68. Rodríguez Vico A, Sánchez Hernández F. Nursing triage in acute stroke. Enf Global 2021 Oct 08;20(4):108-130. [CrossRef]
    69. Gray SE, Finch CF. Assessing the completeness of coded and narrative data from the Victorian Emergency Minimum Dataset using injuries sustained during fitness activities as a case study. BMC Emerg Med 2016 Jul 12;16(1):24 [FREE Full text] [CrossRef] [Medline]
    70. Begier EM, Sockwell D, Branch LM, Davies-Cole JO, Jones LH, Edwards L, et al. The National Capitol Region's Emergency Department syndromic surveillance system: do chief complaint and discharge diagnosis yield different results? Emerg Infect Dis 2003 Mar;9(3):393-396. [CrossRef] [Medline]
    71. Chapman WW, Christensen LM, Wagner MM, Haug PJ, Ivanov O, Dowling JN, et al. Classifying free-text triage chief complaints into syndromic categories with natural language processing. Artif Intell Med 2005 Jan;33(1):31-40. [CrossRef] [Medline]
    72. Chapman WW, Dowling JN, Wagner MM. Classification of emergency department chief complaints into 7 syndromes: a retrospective analysis of 527,228 patients. Ann Emerg Med 2005 Nov;46(5):445-455. [CrossRef] [Medline]
    73. Muscatello DJ, Churches T, Kaldor J, Zheng W, Chiu C, Correll P, et al. An automated, broad-based, near real-time public health surveillance system using presentations to hospital Emergency Departments in New South Wales, Australia. BMC Public Health 2005 Dec 22;5(1):141 [FREE Full text] [CrossRef] [Medline]
    74. Bouchouar E, Hetman BM, Hanley B. Development and validation of an automated emergency department-based syndromic surveillance system to enhance public health surveillance in Yukon: a lower-resourced and remote setting. BMC Public Health 2021 Jun 29;21(1):1247 [FREE Full text] [CrossRef] [Medline]
    75. Travers DA, Haas SW. Using nurses' natural language entries to build a concept-oriented terminology for patients' chief complaints in the emergency department. J Biomed Inform 2003 Aug;36(4-5):260-270 [FREE Full text] [CrossRef] [Medline]
    76. Rice BT, Bisanzo M, Maling S, Joseph R, Mowafi H, Global Emergency Care Investigators Group (Study Group). Derivation and validation of a chief complaint shortlist for unscheduled acute and emergency care in Uganda. BMJ Open 2018 Jun 27;8(6):e020188 [FREE Full text] [CrossRef] [Medline]
    77. Indig D, Copeland J, Conigrave K, Rotenko I. Why are alcohol-related emergency department presentations under-detected? An exploratory study using nursing triage text. Drug Alcohol Rev 2008 Nov;27(6):584-590. [CrossRef] [Medline]
    78. Indig D, Copeland J, Conigrave KM. Comparing methods of detecting alcohol-related emergency department presentations. Emerg Med J 2009 Aug 22;26(8):596-600. [CrossRef] [Medline]
    79. Vallmuur K, Limbong J, Barker R, Hides L. A comparison of methods to identify alcohol involvement in youth injury-related emergency department presentation data. Drug Alcohol Rev 2013 Sep 06;32(5):519-526. [CrossRef] [Medline]
    80. Hides L, Limbong J, Vallmuur K, Barker R, Daglish M, Young RM. Alcohol-related emergency department injury presentations in Queensland adolescents and young adults over a 13-year period. Drug Alcohol Rev 2015 Mar 10;34(2):177-184. [CrossRef] [Medline]
    81. Whitlam G, Dinh M, Rodgers C, Muscatello DJ, McGuire R, Ryan T, et al. Diagnosis-based emergency department alcohol harm surveillance: what can it tell us about acute alcohol harms at the population level? Drug Alcohol Rev 2016 Nov 27;35(6):693-701 [FREE Full text] [CrossRef] [Medline]
    82. Jones R, Woods C, Barker R, Usher K. Patterns and features of methamphetamine-related presentations to emergency departments in QLD from 2005 to 2017. Int J Ment Health Nurs 2019 Aug 09;28(4):833-844. [CrossRef] [Medline]
    83. Marx GE, Chen Y, Askenazi M, Albanese BA. Syndromic surveillance of emergency department visits for acute adverse effects of marijuana, tri-county health department, Colorado, 2016-2017. Public Health Rep 2019;134(2):132-140 [FREE Full text] [CrossRef] [Medline]
    84. Delany C, Crilly J, Ranse J. Drug- and alcohol-related emergency department patient presentations during the 2018 Commonwealth Games: a multi-site retrospective analysis. Emerg Med Australas 2021 Oct;33(5):826-833. [CrossRef] [Medline]
    85. Lam T, Hayman J, Berecki-Gisolf J, Sanfilippo P, Lubman DI, Nielsen S. Pharmaceutical opioid poisonings in Victoria, Australia: rates and characteristics of a decade of emergency department presentations among nine pharmaceutical opioids. Addiction 2022 Mar 18;117(3):623-636 [FREE Full text] [CrossRef] [Medline]
    86. Rahilly-Tierney C, Altincatal A, Agan A, Albert S, Ergas R, Larochelle L, et al. Linking ambulance trip and emergency department surveillance data on opioid-related overdose, Massachusetts, 2017. Public Health Rep 2021 Nov 02;136(1_suppl):47S-53S [FREE Full text] [CrossRef] [Medline]
    87. Burt CW, Overpeck MD. Emergency visits for sports-related injuries. Ann Emerg Med 2001 Mar;37(3):301-308. [CrossRef] [Medline]
    88. Mitchell R, Finch C, Boufous S, Browne G. Examination of triage nurse text narratives to identify sports injury cases in emergency department presentations. Int J Inj Contr Saf Promot 2009 Sep;16(3):153-157. [CrossRef] [Medline]
    89. Chen M, Cundy P, Antoniou G, Williams N. Children bouncing to the emergency department: changes in trampoline injury patterns. J Paediatr Child Health 2019 Feb 09;55(2):175-180. [CrossRef] [Medline]
    90. Eley R, Vallmuur K, Catchpoole J. Value of emergency department triage data to describe and understand patterns and mechanisms of cycling injuries. Emerg Med Australas 2019 Apr 15;31(2):234-240. [CrossRef] [Medline]
    91. Mitchell RJ, Bambach MR. Examination of narratives from emergency department presentations to identify road trauma, crash and injury risk factors for different age groups. Health Inf Manag 2015 Mar 01;44(1):21-29. [CrossRef] [Medline]
    92. Hargrove J, Waller A. Motor vehicle crash case definitions and how they impact injury surveillance. N C Med J 2018 Nov 05;79(6):351-357 [FREE Full text] [CrossRef] [Medline]
    93. Trivedi HM, Panahiazar M, Liang A, Lituiev D, Chang P, Sohn JH, et al. Large scale semi-automated labeling of routine free-text clinical records for deep learning. J Digit Imaging 2019 Feb 20;32(1):30-37 [FREE Full text] [CrossRef] [Medline]
    94. Liljeqvist HT, Muscatello D, Sara G, Dinh M, Lawrence GL. Accuracy of automatic syndromic classification of coded emergency department diagnoses in identifying mental health-related presentations for public health surveillance. BMC Med Inform Decis Mak 2014 Sep 23;14(1):84 [FREE Full text] [CrossRef] [Medline]
    95. Rahme E, Low NC, Lamarre S, Daneau D, Habel Y, Turecki G, et al. Correlates of attempted suicide from the emergency room of 2 general hospitals in Montreal, Canada. Can J Psychiatry 2016 Apr 05;61(7):382-393. [CrossRef]
    96. Kuramoto-Crawford SJ, Spies EL, Davies-Cole J. Detecting suicide-related emergency department visits among adults using the district of Columbia syndromic surveillance system. Public Health Rep 2017 Jul 10;132(1_suppl):88S-94S [FREE Full text] [CrossRef] [Medline]
    97. Stapelberg NJ, Randall M, Sveticic J, Fugelli P, Dave H, Turner K. Data mining of hospital suicidal and self-harm presentation records using a tailored evolutionary algorithm. Mach Learn Applications 2021 Mar;3:100012. [CrossRef]
    98. Robinson J, Witt K, Lamblin M, Spittal MJ, Carter G, Verspoor K, et al. Development of a self-harm monitoring system for Victoria. Int J Environ Res Public Health 2020 Dec 15;17(24):9385 [FREE Full text] [CrossRef] [Medline]
    99. Rozova V, Witt K, Robinson J, Li Y, Verspoor K. Detection of self-harm and suicidal ideation in emergency department triage notes. J Am Med Inform Assoc 2022 Jan 29;29(3):472-480. [CrossRef] [Medline]
    100. Sveticic J, Stapelberg NC, Turner K. Suicidal and self-harm presentations to Emergency Departments: the challenges of identification through diagnostic codes and presenting complaints. Health Inf Manag 2020 Jan 04;49(1):38-46. [CrossRef] [Medline]
    101. Luther M, Gardiner F, Hansen C, Caldicott D. Hot of not: physiological versus meteorological heatwaves-support for a mean temperature threshold. Int J Environ Res Public Health 2016 Jul 26;13(8):753 [FREE Full text] [CrossRef] [Medline]
    102. Harduar Morano L, Waller AE. Evaluation of the components of the North Carolina syndromic surveillance system heat syndrome case definition. Public Health Rep 2017 Jul 10;132(1_suppl):40S-47S [FREE Full text] [CrossRef] [Medline]
    103. Rhea S, Ising A, Fleischauer AT, Deyneka L, Vaughan-Batten H, Waller A. Using near real-time morbidity data to identify heat-related illness prevention strategies in North Carolina. J Community Health 2012 Apr 1;37(2):495-500. [CrossRef] [Medline]
    104. Chapman WW, Dowling JN, Wagner MM. Fever detection from free-text clinical records for biosurveillance. J Biomed Inform 2004 Apr;37(2):120-127 [FREE Full text] [CrossRef] [Medline]
    105. Horng S, Sontag D, Halpern Y, Jernite Y, Shapiro N, Nathanson LA. Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning. PLoS One 2017;12(4):e0174708 [FREE Full text] [CrossRef] [Medline]
    106. Howe A, Crilly M. Identification and characteristics of victims of violence identified by emergency physicians, triage nurses, and the police. Inj Prev 2002 Dec;8(4):321-323 [FREE Full text] [CrossRef] [Medline]
    107. Kondis JS, Muenzer J, Luhmann JD. Missed fractures in infants presenting to the emergency department with fussiness. Pediatric Emergency Care 2017;33(8):538-543. [CrossRef]
    108. McKenzie K, Campbell MA, Scott DA, Discoll TR, Harrison JE, McClure RJ. Identifying work related injuries: comparison of methods for interrogating text fields. BMC Med Inform Decis Mak 2010 Apr 07;10(1):19 [FREE Full text] [CrossRef] [Medline]
    109. Bregman B, Slavinski S. Using emergency department data to conduct dog and animal bite surveillance in New York City, 2003-2006. Public Health Rep 2012 Mar 01;127(2):195-201 [FREE Full text] [CrossRef] [Medline]
    110. Chu KH, Mahmoud I, Hou X, Winter CD, Jeffree RL, Brown NJ, et al. Incidence and outcome of subarachnoid haemorrhage in the general and emergency department populations in Queensland from 2010 to 2014. Emerg Med Australas 2018 Aug 05;30(4):503-510. [CrossRef] [Medline]
    111. Nagabhushan M, Webley J. Ripping the myth: patients' symptomatic descriptions of acute thoracic aortic dissection. Spartan Med Res J 2018 Apr 27;3(1):6783 [FREE Full text] [CrossRef] [Medline]
    112. Personnic J, Titomanlio L, Auvin S, Dozières-Puyravel B. Neurological disorders encountered in a pediatric emergency department. Eur J Paediatr Neurol 2021 May;32:86-92. [CrossRef] [Medline]
    113. Nanda G, Vallmuur K, Lehto M. Semi-automated text mining strategies for identifying rare causes of injuries from emergency room triage data. IISE Transact Healthcare Syst Eng 2019 Mar 25;9(2):157-171. [CrossRef]
    114. Irvine AK, Haas SW, Sullivan T. TN-TIES: a system for extracting temporal information from emergency department triage notes. AMIA Annu Symp Proc 2008 Nov 06;2008:328-332 [FREE Full text] [Medline]
    115. Genes N, Chandra D, Ellis S, Baumlin K. Validating emergency department vital signs using a data quality engine for data warehouse. Open Med Inform J 2013 Dec 13;7(1):34-39 [FREE Full text] [CrossRef] [Medline]
    116. Gillam C, Meuleners L, Versluis L, Hendrie D, Sprivulis P. Electronic injury surveillance in Perth emergency departments: validity of the data. Emerg Med Australas 2007 Aug;19(4):309-314. [CrossRef] [Medline]
    117. Gligorijevic D, Stojanovic J, Satz W, Stojkovic I, Schreyer K, Del Portal D, et al. Deep attention model for triage of emergency department patients. arXiv 2018 [FREE Full text] [CrossRef]
    118. Zhang X, Kim J, Patzer RE, Pitts SR, Patzer A, Schrager JD. Prediction of emergency department hospital admission based on natural language processing and neural networks. Methods Inf Med 2017 Oct 26;56(5):377-389. [CrossRef] [Medline]
    119. Bacchi S, Gluck S, Tan Y, Chim I, Cheng J, Gilbert T, et al. Prediction of general medical admission length of stay with natural language processing and deep learning: a pilot study. Intern Emerg Med 2020 Sep 02;15(6):989-995. [CrossRef] [Medline]
    120. Roquette BP, Nagano H, Marujo EC, Maiorano AC. Prediction of admission in pediatric emergency department with deep neural networks and triage textual data. Neural Netw 2020 Jun;126:170-177. [CrossRef] [Medline]
    121. Handly N, Thompson DA, Li J, Chuirazzi DM, Venkat A. Evaluation of a hospital admission prediction model adding coded chief complaint data using neural network methodology. Eur J Emergency Med 2015;22(2):87-91. [CrossRef]
    122. Tahayori B, Chini-Foroush N, Akhlaghi H. Advanced natural language processing technique to predict patient disposition based on emergency triage notes. Emerg Med Australas 2020 Oct 11 (forthcoming)(forthcoming). [CrossRef] [Medline]
    123. Sterling NW, Patzer RE, Di M, Schrager JD. Prediction of emergency department patient disposition based on natural language processing of triage notes. Int J Med Inform 2019 Sep;129:184-188. [CrossRef] [Medline]
    124. Joseph JW, Leventhal EL, Grossestreuer AV, Wong ML, Joseph LJ, Nathanson LA, et al. Deep-learning approaches to identify critically Ill patients at emergency department triage using limited information. J Am Coll Emerg Physicians Open 2020 Oct;1(5):773-781 [FREE Full text] [CrossRef] [Medline]
    125. Ivanov O, Wolf L, Brecher D, Lewis E, Masek K, Montgomery K, et al. Improving ED emergency severity index acuity assignment using machine learning and clinical natural language processing. J Emerg Nurs 2021 Mar;47(2):265-78.e7 [FREE Full text] [CrossRef] [Medline]
    126. Sterling NW, Brann F, Patzer RE, Di M, Koebbe M, Burke M, et al. Prediction of emergency department resource requirements during triage: an application of current natural language processing techniques. J Am Coll Emerg Physicians Open 2020 Dec 14;1(6):1676-1683 [FREE Full text] [CrossRef] [Medline]
    127. Fernandes M, Mendes R, Vieira SM, Leite F, Palos C, Johnson A, et al. Risk of mortality and cardiopulmonary arrest in critical patients presenting to the emergency department using machine learning and natural language processing. PLoS One 2020 Apr 2;15(4):e0230876 [FREE Full text] [CrossRef] [Medline]
    128. Klang E, Levin MA, Soffer S, Zebrowski A, Glicksberg BS, Carr BG, et al. A simple free-text-like method for extracting semi-structured data from electronic health records: exemplified in prediction of in-hospital mortality. Big Data Cogn Comput 2021 Aug 29;5(3):40. [CrossRef]
    129. Zhang X, Bellolio MF, Medrano-Gracia P, Werys K, Yang S, Mahajan P. Use of natural language processing to improve predictive models for imaging utilization in children presenting to the emergency department. BMC Med Inform Decis Mak 2019 Dec 30;19(1):287 [FREE Full text] [CrossRef] [Medline]
    130. Zhang X, Kim J, Patzer RE, Pitts SR, Chokshi FH, Schrager JD. Advanced diagnostic imaging utilization during emergency department visits in the United States: a predictive modeling study for emergency department triage. PLoS One 2019 Apr 9;14(4):e0214905 [FREE Full text] [CrossRef] [Medline]
    131. Goldman-Mellor S, Jia Y, Kwan K, Rutledge J. Syndromic surveillance of mental and substance use disorders: a validation study using emergency department chief complaints. Psychiatr Serv 2018 Jan 01;69(1):55-60 [FREE Full text] [CrossRef] [Medline]
    132. Fernandes M, Mendes R, Vieira SM, Leite F, Palos C, Johnson A, et al. Predicting Intensive Care Unit admission among patients presenting to the emergency department using machine learning and natural language processing. PLoS One 2020 Mar 3;15(3):e0229331 [FREE Full text] [CrossRef] [Medline]
    133. Klang E, Kummer BR, Dangayach NS, Zhong A, Kia MA, Timsina P, et al. Predicting adult neuroscience intensive care unit admission from emergency department triage using a retrospective, tabular-free text machine learning approach. Sci Rep 2021 Jan 14;11(1):1381 [FREE Full text] [CrossRef] [Medline]
    134. Mikosz C, Silva J, Black S, Gibbs G, Cardenas I. Comparison of two major emergency department-based free-text chief-complaint coding systems. MMWR Suppl 2004 Sep 24;53:101-105 [FREE Full text] [Medline]
    135. Simonsen L, Gog JR, Olson D, Viboud C. Infectious disease surveillance in the big data era: towards faster and locally relevant systems. J Infect Dis 2016 Dec 01;214(suppl_4):S380-S385 [FREE Full text] [CrossRef] [Medline]
    136. Wolf L, Delao A, Perhats C, Baker K, Olson C. Development of nurse-sensitive, emergency department-specific quality indicators using a modified Delphi technique. J Nurs Care Qual 2022;37(4):E59-E66. [CrossRef] [Medline]
    137. Cotton R, Drew R, Douma M, O'Dochartaigh D, Keddie C, Muncaster K, et al. An analysis of individual and department triage variances to identify, quantify, and improve markers of triage nurse accuracy. CJEN 2021 Jul 20;44(2):19-20. [CrossRef]
    138. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007 Oct 16;147(8):573-577 [FREE Full text] [CrossRef] [Medline]
    139. Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, RECORD Working Committee. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med 2015 Oct 6;12(10):e1001885 [FREE Full text] [CrossRef] [Medline]
    140. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015 Jan 07;350:g7594 [FREE Full text] [CrossRef] [Medline]

    ‎
    ED: emergency department
    ML: machine learning
    PRISMA-ScR: Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews

    Edited by G Eysenbach; submitted 21.07.22; peer-reviewed by M Fernandes; comments to author 22.10.22; revised version received 24.11.22; accepted 28.11.22; published 13.01.23

    Copyright

    ©Christopher Picard, Manal Kleib, Colleen Norris, Hannah M O'Rourke, Carmel Montgomery, Matthew Douma. Originally published in JMIR Nursing (https://nursing.jmir.org), 13.01.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Nursing, is properly cited. The complete bibliographic information, a link to the original publication on https://nursing.jmir.org/, as well as this copyright and license information must be included.