We used data extracted from a large urban home health care organization in Northeastern United States. The home health care agency provides skilled home health care services, including nursing, physical and occupational therapy, and social work.

The study was approved by the institutional review board of the participating organization, VNS Health (IRB I22-001).

This study examined data collected during routine home health care services between January 1 and December 31, 2019. All data were extracted from the home health care agency’s electronic health record system. The data included the patient’s sociodemographic information, specifically the patient’s gender and race or ethnicity, collected using a federally mandated assessment data set called the “Outcome and Assessment Information Set” (OASIS) that captures race or ethnicity with the following categories: Asian, White, Black, Hispanic, and other (eg, Native Hawaiian or Pacific Islander). Since only a small number of patients identified as “other” race or ethnicity, we removed this group from this analysis. Although the OASIS allows for multiselect in race and ethnicity, to establish mutually exclusive groups, if Hispanic was one of the selections, then the individual was categorized in the Hispanic group. In addition, we extracted narrative clinical notes, clinician information (ie, clinician author ID for each narrative note), and the length of time spent by a clinician in the patient’s home (in minutes). Home health care clinicians in this study included nurses, physical or occupational therapists, and social workers.

Narrative notes were visit notes that home health care clinicians used to document the patient’s symptoms and health care inventions that occurred during home visits. Overall, we extracted 264,146 visit notes documented for 45,384 patients, with an average of 6 visit notes per patient. Visit note average length was 298 characters, corresponding to about 4-6 sentences.

Based on previous literature [27,28], we identified a specific type of language that potentially suggests judgment of patients—“judgment words.” Judgment words allow the speaker to distance themselves from the source of knowledge and directly question the speaker’s credibility. The initial list of judgment words was extracted from previous literature and included the following terms: “adamant,” “apparently,” “claims,” “insists,” and “states” [27].

In previous work [27], the use of quotes was also found to indicate judgment of patient’s claims; for example, “the patient reports she had a ‘reaction’ to the medication.” However, in home health care clinical notes, we found that clinicians use quotes very infrequently. Hence, we decided to omit this category from the analysis.

We used our previously developed and validated open-source natural language processing system, NimbleMiner [33,34], to expand the initial vocabularies of evidential and judgment words. Specifically, NimbleMiner was implemented in 3 steps that are briefly described below and captured in Figure 1 (a complete software architecture description is available elsewhere [33,34]).

First, we used chi-square tests to conduct bivariate analyses to examine differences in the use of judgment language by patients’ race or ethnic groups. Further, we conducted an adjusted analysis to explore associations between the patients’ race or ethnic group and the use of judgment language in their clinical notes. To examine whether the analysis should be adjusted for individual clinicians’ writing style, we implemented and compared the results of two logistic regression models: (1) a general logistic regression model adjusted for patients’ gender and (2) a mixed-effects logistic regression model adjusted for patients’ gender (fixed effects) and clinician ID (random effects). The most appropriate model was selected by comparing differences in the log-likelihood between the models [36]. Finally, we examined the association between visit time and the use of judgment words. Specifically, we implemented linear regression to analyze this association adjusted for the patient’s gender and race or ethnic group. All analyses were implemented in Stata statistical software (version 17; StataCorp) [37].

This study was the first to investigate the use of stigmatizing language in home health care. Specifically, we developed and applied a natural language processing system that identified the language of judgment in clinical notes. We found that such language appeared relatively frequently in clinical notes, with nearly 40% of patients having at least one instance of such language in their notes. Overall, approximately 1 in 10 clinical notes included judgment language, which is similar to the previous literature in hospital settings [27,29,38]. These numbers highlight the need for further studies with larger data sets of clinical data that will enable estimating the general prevalence and use of judgment language in health care.

Further, our findings helped identify racial and ethnic differences in the use of judgment language. Previous studies primarily focused on language differences between Black and White patients [27,29,38], while our sample also included a significant number of Hispanic and Asian patients. We found that judgment language was more frequently documented in clinical notes of Black and Hispanic patients. Specifically, in an adjusted analysis, Black and Hispanic patients had up to 9% higher odds of having judgment language in their clinical notes than White patients. These results are lower but in the same direction as previous findings from the hospital settings, indicating that Black patients have up to 25% higher odds of having judgment language in their clinical notes than White patients [27]. We further expand these results and show that similar to Black patients, Hispanic patients have high levels of judgment language in clinical notes.

Several potential explanations can help describe these differences. First, clinicians’ personal biases might manifest in written documents [27-29]; hence, we find the language of judgment to be more prevalent in clinical notes of minority patient populations. Our analysis also shows that some clinicians are more likely to use the language of judgment than others. This further supports the need for more research to test the hypothesis that personal differences among clinicians play a significant role in shaping their language. In addition, institutional biases toward certain racial and ethnic groups might shape clinicians’ views, resulting in documentation differences. For example, numerous previous studies found that racist institutional policies lead to worse quality of care and outcomes among Black patients [10].

In addition to the previously mentioned factors that may contribute to the use of judgment language in clinical notes, it is important to consider the role of patient-provider interactions in shaping the use of this type of language. The use of judgment language may not be solely the result of clinician biases but may also be influenced by the specific circumstances of the patient-provider interaction. For example, when a patient is not following instructions or refusing self-management, a clinician may be more likely to use judgment language in their documentation. Similarly, in complex clinical scenarios, a clinician may use more judgment language as they navigate a difficult case in which diagnosing a patient’s condition is complicated. Further, in some clinical situations, clinicians might use harsher, more critical, or more negative language. Further research is needed to understand specific contextual factors that may contribute to the use of judgment language in clinical notes.

Finally, our results hint at the potential association between the language of judgment and quality of care. Specifically, we found that clinicians spend less time with patients for whom they document the language of judgment. This is concerning since shorter home health care visits are associated with a higher risk for poor outcomes (eg, higher risk of hospitalizations) [39,40]. Another testable hypothesis might be that home health care clinicians spend less time with patients they perceive negatively, which is reflected by judgment language.

Our findings have several implications at the health care policy and management levels. First, health care organizations might need to develop guidelines to help shape more inclusive and neutral documentation patterns. For example, certain words and expressions of judgment might need to be limited or require thorough justification when used. Further, targeted clinician training in improving documentation styles might be needed for some clinicians who frequently use such language in their documentation. In addition, counseling or educational interventions to reduce implicit clinician biases might help decrease stigmatizing language in clinical practice. Finally, clinicians might need more training about engaging with patients who are not following instructions or refuse self-management to increase time spent with those patients in productive motivational conversations and similar interventions [41].

This work has several significant limitations. First, the judgment language in the notes might appear in descriptions of “nonjudgmental” clinical situations. For example, words like “state(s)” are often used to describe patient’s symptoms and other reports with little evidence of judgment. Additionally, the study’s approach to identifying judgment language is based on the frequency of certain words rather than considering the context in which they are used. This means that the study may not accurately capture the nuances of how judgment language is used in clinical notes, and therefore, may not fully capture the extent to which clinical notes are racially charged. Other natural language processing methods computationally tied to the clinical note context (eg, sequence of words, topic modeling, or Bidirectional Encoder Representations from Transformers [BERT]) [42] might help identify the judgment language more accurately.

Second, this analysis did not adjust for clinical factors that might interfere with judgment language prevalence (eg, the patient’s cognitive status or mental health conditions). Further work is needed to generate comparisons adjusted for such additional health conditions.

Home health care visit length might not necessarily reflect the quality of care provided. Further, clinical encounter time as well as documentation time and quality might be affected by multiple factors, such as administrative concerns or needing to visit another patient’s home as soon as possible. Associations between clinicians’ documentation and encounter times should be explored in future studies.

Further work is needed to understand whether differences in judgment language prevalence exist between different disciplines (ie, nursing, occupational or physical therapy, and social work). In addition, further exploration of the effect of culture on the language used during and length of clinical encounters is needed. For example, Asian patients might respond differently during clinician interaction [43], which might affect clinician documentation. Further, future studies should emphasize the frequency of the judgment words used rather than focusing solely on the specific vocabulary used. This will test whether a higher frequency of judgment words in the clinical notes may indicate a more intentional use of judgmental language, as opposed to a lower frequency, which may suggest a less intentional use. Further qualitative research with the clinicians who wrote the clinical notes is needed to gain a deeper understanding of the use of judgment language and the context in which it was used. Finally, the generalizability of this work is limited to one, albeit large, home health care agency.

This study’s findings indicate that language of judgment appears more frequently in clinical notes of Black and Hispanic patients as compared to White and Asian patients. We also found that clinicians spend less time in patients’ homes when judgment language is used. Since the language clinicians use in documentation is associated with care quality, policy and clinical practice steps are needed to address biases associated with racial and ethnic differences in the prevalence of judgment language. Further research is needed to fully understand the prevalence and root causes of stigmatizing language and to test interventions to eliminate their use.