Digital Leadership Scale for Clinical Nurses: Development and Validation of an Instrument

Introduction

The digital transformation era has accelerated the adoption of digital technologies in the health care sector, particularly after COVID-19, which catalyzed the integration of such technologies into clinical practice [1-3]. Clinical nurses are increasingly using digital tools to ensure safe care environments and to provide evidence-based nursing [4,5]. Common technologies used by nurses include wearable devices, electronic health records, and Barcode Medication Administration systems [6,7], all of which enhance patient safety and efficiency. The adoption of these digital tools in clinical settings has been shown to improve nursing efficiency and patient outcomes [8].

As health care environments become increasingly digitalized, nurses are expected not only to use digital technologies in practice but also to adapt to and support digital change in clinical settings. Nurses serve as coordinators, ensuring effective communication among patients, families, health care professionals, and nursing peers, which is directly related to patient safety [4,9]. In this context, digital leadership encompasses not only the technical ability to understand and apply digital tools [10] but also the capacity to introduce new communication systems and foster collaboration among team members [11]. Therefore, to successfully implement digitalization and health information technologies within hospital organizations, leadership from nurses who continuously enhance and manage their clinical competencies is essential [12,13]. The level of leadership demonstrated by nurses is closely linked to the quality of nursing services [14].

Digital leadership combines traditional nursing competencies with digital capabilities, thereby improving the quality and efficiency of care delivery [8]. In rapidly evolving health care environments, digital leadership is no longer a personal attribute of individual nurses but a comprehensive competency influencing teams and organizations [15]. Nurses must demonstrate adaptability to digital changes [2], solve problems arising during digital transitions [11,16], and support peer education and motivation [2,17]. Effective digital leadership also includes ethical considerations, such as the protection of patient information in the context of big data usage [18]. Existing tools for measuring digital leadership have primarily been developed for managers or employees in general corporate settings, focusing on behaviors and skills required in digital work environments, including vision sharing and resource support. These tools aim to assess general leadership attributes applicable in digital contexts [19,20]. However, they are limited in that they do not reflect the specific characteristics of nursing practice, which involves unique therapeutic relationships between nurses and patients [21], nor do they account for the evolving nature of contemporary clinical settings due to the introduction of new digital technologies [3]. While digital leadership among nurses—encompassing the ability to adapt to digital transformation and effectively integrate digital technologies with health data—is increasingly essential [2,22,23], there remains a lack of measurement tools that are specifically developed to reflect the unique roles and characteristics of clinical nurses.

The purpose of this study is to develop and validate the Digital Leadership Scale for Clinical Nurses (DLS-CN). This scale is intended to accurately assess the level of digital leadership among clinical nurses and provide a foundation for evaluating and enhancing their leadership capabilities in digital health care environments.

Methods

Study Design

This study is a quantitative methodological research conducted to develop and evaluate a measurement tool for assessing digital leadership among clinical nurses. The development of the DLS-CN was carried out in 2 phases, based on the tool development process proposed by DeVellis and Thorpe [24]. In the first phase, preliminary items were developed, followed by content validity testing, cognitive interviews, and a pilot survey. In the second phase, the tool was administered to clinical nurses to examine its reliability and validity.

Scale Development Process

Phase 1: Preliminary Item Development

Clarification of the Conceptual Definition

This study was based on the hybrid model of Schwartz-Barcott and Kim [25], applying theoretical, fieldwork, and final analytic phases. Based on a previous study analyzing the concept of digital leadership [15], a literature review was conducted, and concept attributes were integrated to establish a conceptual framework and identify core attributes. Through this process, 6 components of digital leadership were identified: digital problem-solving ability, digital ethical sensitivity, digital mindset, self-growth, digital collaboration, and organizational influence.

Item Generation

A total of 104 initial items were generated across 6 components of digital leadership for clinical nurses. Specifically, 21 items measured digital problem-solving ability, 19 items measured digital ethical sensitivity, 16 items measured digital mindset, 9 items measured self-growth, 17 items measured digital collaboration, and 22 items measured organizational influence.

Selection of Measurement Format

Since the items were designed to measure the level of agreement with statements regarding digital leadership among clinical nurses, the widely used Likert scale was deemed appropriate [24]. Therefore, a 5-point Likert scale was selected.

Expert Content Validity Assessment

To evaluate the content validity of the preliminary items, an expert panel of 9 members was formed: 2 clinical nurses with more than 5 years of experience; 2 nurse unit managers; 2 nursing professors; and 1 expert each in digital health care, leadership, and hospital management. After the researchers explained the concept of digital leadership for clinical nurses, the experts rated the appropriateness of the item content and components using a 4-point Likert scale. For item-level evaluation, the item-level content validity index was considered acceptable if it was 0.78 or higher. To assess overall content validity, the scale-level content validity index was calculated by averaging, with 0.90 or higher judged as acceptable [26]. Two rounds of expert validation were conducted. As a result of the first round, 71 items were selected. After the second round, the number was reduced to 46 items. During this process, sentence order and expressions were revised for better clarity and specificity, especially when abstract or ambiguous wording was identified.

Cognitive Interviews

Cognitive interviews were conducted with 5 clinical nurses who met the inclusion criteria of the main study. Each interview lasted 30 to 40 minutes and aimed to understand how participants interpreted and responded to the preliminary items. During the interviews, participants were asked to explain their thought processes while answering each item, including how they understood the wording, whether any terms were unclear, and whether the items were easy or difficult to answer. The interview data were analyzed by reviewing participants’ explanations and feedback on item comprehension, wording, and response processes. Based on these findings, items were revised to improve clarity and reduce potential response errors. When similar or overlapping content was identified, the researchers rechecked whether the same concern was expressed by other interviewees before making revisions. The revised items were then reviewed again with the interviewees to confirm their appropriateness. Some participants questioned the appropriateness of the term “hospital information system,” but after reviewing related tools, literature, and expert texts, the term was retained to encompass systems such as electronic medical records and order communication systems. Regarding terms describing digital collaboration tools or systems, alternatives like “intranet” and “groupware” were suggested; however, “digital collaboration system” was found to be the most understandable and suitable. In response to feedback, the term “real-time” was added to better reflect the advantages of system usage. While some general staff nurses noted difficulty adopting new technologies and resources, replacing terms like “utilize” or “apply” was found to weaken alignment with intended attributes, so the original expressions were maintained. After revising the items, the modified 46 items were confirmed for appropriateness through follow-up review with the interviewees.

Pilot Study

To identify issues with the preliminary items, a pilot study was conducted with 30 clinical nurses who met the same criteria as those in the main study. This was conducted via an online survey identical in format to the main study. Respondents rated item comprehensibility on a 5-point scale (1=“very difficult” to 5=“very easy”), yielding an overall average of 4.14 (SD 0.24), indicating that the items were well understood. The time required to complete the survey ranged from 5 to 25 minutes, with an average of 11 (SD 1.84) minutes. Open-ended questions confirmed that item length, font size, and type were appropriate. Based on the pilot results, items with duplicate meanings or unclear wording were revised. Additionally, a new item was added to the organizational influence component, “I can help organizational members experiencing difficulties in digitalized work environments,” to complement the existing item, “I can ask organizational members for help when I face difficulties.” All items were revised through linguistic review to ensure correct spacing, terminology, and order. Ultimately, the preliminary item pool was finalized with 47 items across 6 components.

Phase 2: Instrument Evaluation—Reliability and Validity Testing

Participants

Participants were 446 clinical nurses working at tertiary or general hospitals in South Korea. Eligible participants were nurses who were directly involved in patient care and had at least 1 year of clinical experience. Nurses with less than 1 year of experience were excluded because they were considered to be in the novice stage of acquiring the knowledge and skills necessary for nursing practice [27,28]. Nurses in administrative, educational, or nondirect care roles were also excluded. A minimum sample size of 200 is generally recommended for factor analysis, and for confirmatory factor analysis (CFA), a commonly used rule of thumb is that the sample should be at least 10 times the number of estimated parameters [29,30]. Therefore, the total sample size of 446 was considered adequate. To examine the factor structure and cross-validate the model, the total sample was randomly divided into 2 equal groups, with 223 participants assigned to exploratory factor analysis (EFA) and 223 to CFA. This equal split was used to ensure independent and sufficiently large samples for both analyses.

Instruments

To evaluate convergent validity, 2 instruments were selected: the Nursing Informatics Competency Scale [31] and the Self-Leadership Scale [32]. The Nursing Informatics Competency Scale includes 20 items across 5 components: “Basic information and communication technology use” (3 items), “Utilization of nursing information” (5 items), “Professional responsibility and ethics” (5 items), “Use of information and communication technology in nursing” (4 items), and “Attitudes toward nursing informatics” (3 items). Each item is rated on a 4-point Likert scale. The reliability of the original scale was a Cronbach α of 0.91 [31], and in this study, internal consistency reliability was assessed using Cronbach α based on the responses of the study participants, with a Cronbach α coefficient of 0.91, indicating excellent internal consistency. The Self-Leadership Scale consists of 16 items across 4 components: “Collaborative self-dialog strategies” (4 items), “Physical vitality enhancement strategies” (4 items), “Goal-oriented self-training strategies” (4 items), and “Self-respect pursuit strategies” (4 items). This instrument uses a 5-point Likert scale. The original reliability was a Cronbach α of 0.86 [32], and in this study, the Cronbach α was also calculated using the current dataset, yielding a coefficient of 0.91, which indicated excellent internal consistency.

Data Collection

Data were collected from March 10, 2024, to March 30, 2024, through a self-administered online survey using a nonprobability convenience sampling method. Participants were able to complete the survey at their preferred time and location without direct contact with the researchers. To recruit nurses from diverse regions, recruitment notices containing the contact information of the researchers and the survey link (URL) or QR code were posted on online nursing community boards and workplace forums, allowing eligible participants to voluntarily access and complete the survey. All items were set as mandatory, preventing participants from proceeding without responding; consequently, no missing data were observed in the submitted questionnaires. Data collection continued until the predetermined sample size required for item analysis was reached, after which the survey was closed to prevent additional responses. Because the survey link was distributed through open online communities and workplace forums, the total number of individuals who viewed the recruitment notice could not be determined, and thus the response rate could not be calculated.

Data Analysis

Collected data were analyzed using IBM SPSS Statistics version 25.0 and AMOS version 23.0. First, participants’ general characteristics were analyzed using frequencies, percentages, means, and SDs. Second, for the item analysis, means, SDs, skewness, kurtosis, inter-item correlations, item-total correlations, and reliability (if an item was deleted) were calculated. Third, to test the suitability of EFA, the Kaiser-Meyer-Olkin (KMO) measure and the Bartlett test of sphericity were used. Fourth, principal component analysis with varimax rotation was performed to extract factors. Fifth, CFA was conducted to evaluate model fit and test convergent and discriminant validity. Sixth, convergent validity was also examined using the Pearson correlation coefficient. Finally, internal consistency reliability was tested using Cronbach α.

Ethical Considerations

This study was approved by the Institutional Review Board of Daegu Catholic Medical Center (CR-23-148). The online survey introduction clearly explained the study’s purpose, informed consent, procedures, voluntary participation, confidentiality, potential benefits and risks, and stated that the data would be used only for research purposes. Participants received a small token of appreciation for their involvement. The study was conducted in accordance with the ethical principles of the Declaration of Helsinki.

Results

General Characteristics of Participants

A total of 446 clinical nurses participated in the survey (Table 1). Among them, 223 participants were included in the EFA, with an average age of 30.71 (SD 5.45) years. Most participants were female (208, 93.3%). Regarding the type of hospital where they worked, 137 participants (61.4%) were employed at tertiary hospitals and 86 (38.6%) at general hospitals. The most common region was Seoul, the capital, with 106 participants (47.5%). The average length of clinical experience was 70.41 (SD 46.68) months. In terms of work departments, 137 participants (61.4%) worked in general wards, 52 (23.3%) in special units (eg, intensive care units, operating rooms, emergency rooms), 17 (7.6%) in outpatient clinics, and 17 (7.6%) in other departments. For the CFA, 223 participants were included, with an average age of 31.07 (SD 4.81) years. The majority were female participants (200, 89.7%). A total of 131 participants (58.7%) worked in tertiary hospitals and 92 (41.3%) in general hospitals. The highest number of participants was also from Seoul (99, 44.4%). The average length of clinical experience was 74.31 (SD 57) months. Regarding work departments, 147 participants (65.9%) worked in general wards, 38 (17.0%) in special units, 25 (11.2%) in outpatient clinics, and 13 (5.8%) in other departments.

Item Analysis

To examine the distribution of items in the 223 collected responses for factor analysis, the mean, SD, skewness, and kurtosis were calculated for each item. Since none of the items had an absolute skewness value greater than 3.0 or an absolute kurtosis value greater than 8.0, all items met the assumption of normality [33]. Item-total correlation coefficients were reviewed, and items with coefficients below 0.30 or above 0.80 were removed, as these values suggest low contribution to the scale or item redundancy [34]. Seventeen items were removed due to low internal consistency reliability and weak correlation between item pairs. Based on corrected item-total correlation and changes in reliability after item deletion [33], the corrected item-total correlations ranged from 0.43 to 0.72, and the reliability did not significantly change upon item deletion (Table 2). As a result, 30 items were retained for further analysis.

Exploratory Factor Analysis

EFA using principal component analysis with varimax rotation was conducted on the 30 items of the DLS-CN. Prior to the factor analysis, data adequacy was assessed using the KMO measure and the Bartlett test of sphericity. The KMO value was 0.96, indicating excellent sampling adequacy, and the Bartlett test of sphericity was statistically significant [35], confirming the data’s suitability for factor analysis. To determine the number of factors, eigenvalues greater than 1.0 and a cumulative variance explanation rate exceeding 60% were used based on Kaiser rule. Scree plot analysis and the interpretability of the factor structure were also considered. Items were retained based on communalities greater than 0.40, factor loadings above 0.40, and at least 3 items per factor [36]. One item with a factor loading below 0.40 was removed. The final analysis included 29 items, with a KMO of 0.95 and a statistically significant Bartlett test (χ²₈₂₀=4799.3; P<.001), indicating the appropriateness of factor analysis. The EFA extracted 4 factors with eigenvalues greater than 1.0, explaining a cumulative variance of 56.9%. Communality values ranged from 0.52 to 0.64, and all factor loadings exceeded 0.40, indicating good model fit. The purpose of factor analysis is to understand the pattern of data and clarify the interpretability of factors [37]. The factor loadings and item contents were examined to ensure conceptual coherence. Cross-loading items that were theoretically essential were assigned to conceptually valid factors. The inter-item correlation coefficients within subfactors were as follows: factor 1: 0.43 to 0.64; factor 2: 0.35 to 0.57; factor 3: 0.35 to 0.51; and factor 4: 0.39 to 0.052, indicating that the model estimation was appropriate (Table 3).

Confirmatory Factor Analysis

CFA was conducted to verify whether the 4 factors and 29 items extracted during the EFA appropriately reflected the constructs of digital leadership among clinical nurses. The model’s goodness-of-fit was assessed to confirm the adequacy of the structure. To evaluate the model fit, the chi-square statistic was examined. A model was considered acceptable if the normed chi-square χ²/df was less than 3, the comparative fit index was above 0.90, the root mean square error of approximation was below 0.06, and the standardized root mean squared residual was below 0.80 [38]. The results of the CFA indicated that the chi-square value was statistically significant, and the normed chi-square χ²/df was 1.8, satisfying the criterion of being below 3. The comparative fit index was 0.91, exceeding the threshold of 0.90. The root mean square error of approximation was 0.06, which meets the criterion for a good model fit. The standardized root mean squared residual was 0.05, which is below the acceptable cutoff of 0.08, indicating that the model had a generally good fit (Table 2).

To verify the convergent validity of each factor, the standardized factor loading was required to be 0.5 or higher, the significance of the unstandardized regression coefficient (critical ratio) was required to exceed 1.97 (P<.05), the construct reliability had to be 0.7 or higher, and the average variance extracted (AVE) needed to be 0.5 or higher [39]. The results of the convergent validity test showed that the critical ratio values of the unstandardized regression coefficients ranged from 7.83 to 10.63, significantly exceeding the threshold [30], confirming that the measured items met the conditions necessary for validity testing. The standardized factor loadings for all factors met the criterion of 0.5 or higher. The AVE values for each factor ranged from 0.52 to 0.57, satisfying the minimum requirement of 0.50. The construct reliability values ranged from 0.85 to 0.90, all exceeding the minimum requirement of 0.70. Therefore, the DLS-CN demonstrated adequate convergent validity in terms of item construction. Through this, it was confirmed that 4 factors consistently measured the construct of “Digital Leadership Scale for Clinical Nurses,” and that the 29 items consistently represented the corresponding factors. To assess discriminant validity, interconstruct correlation matrices were examined (Table 4). Discriminant validity was evaluated by checking whether the AVE of each factor was greater than the squared correlation coefficients between the factors [39]. In some cases, such as the ability to use digital technology–digital collaboration mindset, ability to use digital technology–organizational influence, digital safety management–digital collaboration mindset, and digital collaboration mindset–organizational influence, the squared correlations between factors exceeded the AVE of the individual constructs, indicating that discriminant validity was only partially satisfied.

Convergent Validity Testing

To verify convergent validity, the correlations with the Nursing Informatics Competency Measurement Tool [31] and the Self-Leadership Scale [32] were examined. The correlation with the Nursing Informatics Competency Scale was 0.82 (P<.001), and the correlation with the Self-Leadership Scale was 0.83 (P<.001). Since the correlation coefficients were all above 0.70, convergent validity was confirmed.

Reliability Testing

Reliability was tested using Cronbach α, with a criterion of 0.70 or higher [29,40]. Cronbach α for the 29 items measuring DLS-CN was 0.95.

By subfactor, Cronbach α was as follows: factor 1 (ability to use digital technology): 0.88; factor 2 (digital safety management): 0.78; factor 3 (digital collaboration mindset): 0.86; and factor 4 (organizational influence): 0.87.

Since Cronbach α for all subfactors and the overall tool exceeded 0.70, reliability was confirmed.

Final Items

In this study, the final version of the DLS-CN consisted of 29 items across 4 factors (Table 5).

By subfactor, the items were as follows: 8 items for the ability to use digital technology, 5 items for digital safety management, 8 items for a digital collaboration mindset, and 8 items for organizational influence.

Discussion

Principal Findings

This study was conducted to develop a tool to measure digital leadership among clinical nurses, based on the instrument development steps proposed by DeVellis and Thorpe [24]. The research was carried out in 2 phases: the development of preliminary items and the evaluation of the instrument’s validity and reliability.

During the item development phase, a conceptual framework was established by integrating the components of digital leadership identified in previous concept analyses and literature reviews on digital leadership in clinical nursing. Preliminary items were selected from the initial items through a content validity assessment by experts, followed by cognitive interviews and a pilot survey, which were used to derive the final set of preliminary items. Throughout this process, expert feedback and identified issues were systematically reviewed and refined.

In the instrument evaluation phase, data were collected from 446 clinical nurses through an online survey. A random sample of 223 responses was used for EFA, and another 223 responses were used for CFA. Construct validity was tested through these analyses, and convergent validity was assessed by examining the correlation with instruments measuring nursing informatics competency and self-leadership. Internal consistency reliability was confirmed with a Cronbach α of 0.95.

The final tool developed through this study consists of 29 items across 4 factors: 8 items for the ability to use digital technology, 5 items for digital safety management, 8 items for a digital collaboration mindset, and 8 items for organizational influence. The tool uses a 5-point Likert scale for self-report surveys, with a total score range from 29 to 145 points. Higher scores indicate a higher level of digital leadership among clinical nurses. Overall, the developed scale demonstrated strong reliability and validity for assessing digital leadership in clinical nursing practice. However, the tool did not fully satisfy the criteria for discriminant validity. Even when variables are used as conceptually independent constructs, correlations between related factors are common in practice [29]. This is consistent with findings from a study on the development of an integrated nursing leadership scale for Korean nurses [41], which also reported overlaps among constructs such as data management and communication technologies. In particular, relatively high correlations were observed between the ability to use digital technology and organizational influence, as well as between digital collaboration mindset and organizational influence. These results suggest that while the factors are conceptually distinct, they may be closely related in actual clinical settings where digital competencies are integrated with leadership behaviors. Although some aspects of discriminant validity were supported, these findings indicate that the boundaries between certain factors were not clearly distinguished. Despite these limitations, this tool has the advantage of including items that measure competencies needed to adapt to rapidly changing digital health care environments, such as work performance, learning, and adaptability [42]. Accordingly, future research is needed to more clearly distinguish the unique conceptual domains measured by each factor. Furthermore, the “digital safety management” factor included fewer items compared to other factors. Given the importance of patient safety and information security in digital health care environments, future research should consider expanding and refining items in this domain to enhance the comprehensiveness of the scale.

Given that leadership competencies required in digitally transforming health care environments are increasingly complex and diverse [13], this DLS-CN reflects the unique characteristics of Korean clinical settings and is the first scale of its kind to undergo statistical validation. Nevertheless, because data collection was conducted via an online survey using convenience sampling, caution is needed when interpreting the results. Additionally, the scale was developed for general clinical nurses and excluded nurses with less than 1 year of experience, nurse managers not involved in direct care, and those in administrative or educational roles, thereby limiting its generalizability. Moreover, the study did not include test-retest reliability, which represents another limitation.

Based on these findings, several implications can be drawn. In particular, the developed digital leadership scale should be applied to nurses from diverse countries and cultural backgrounds, as well as to nurses in various roles and settings, such as hospitals, community health centers, and educational institutions. Repeated studies are recommended to verify the reliability and validity of the instrument in diverse environments.

Future research should use this tool to assess digital leadership levels and develop tailored educational or intervention programs that enhance digital leadership according to workplace and contextual characteristics.

In addition, as this scale may be applicable not only to health care professionals but also to professionals in management, public administration, and related fields, further research should explore its broader applicability across disciplines.

Conclusions

The DLS-CN, developed through this study, is a systematic tool designed to measure and evaluate digital leadership among nurses in clinical practice settings. The DLS-CN consists of 4 core domains: ability to use digital technology, digital safety management, digital collaboration mindset, and organizational influence. It proposes a new standard for nursing leadership in health care environments driven by digital technologies.

In particular, the tool is useful for developing various educational programs aimed at identifying emerging nurse leaders and enhancing the digital leadership of clinical nurses in hospital environments where frequent changes in digital health care systems, technologies, and organizational structures occur. Furthermore, the development of this tool confirms that digital leadership extends beyond the mere acquisition of digital skills, encompassing the competence of clinical nurses to actively engage in and lead change. This is a meaningful contribution to the study.

Since this tool was developed based on data collected from clinical nurses in South Korea, and the items were created accordingly, further validation studies are needed in diverse health care settings and across cultural contexts. In the rapidly evolving digital health care environment, this tool can serve both as a foundational resource and a structured instrument for strengthening clinical nurses’ digital leadership.