Socioemotional competencies (SEC) play a large role in the day-to-day work of a nurse, both to provide quality care as well as to maintain their own well-being [1-5]. As our understanding of the importance of these nursing competencies grows, so too does the interest in how to effectively teach [3,6-8] and assess [1,9] SEC in nursing programs.

A recent systematic review underlines the importance of developing SEC during clinical practice [10], showing nursing students are unprepared for the social dynamics of the workplace otherwise. Traditionally, however, the focus of clinical placement has been on improving technical skills [11-14]. Concurrently, nontechnical skills such as SEC are expected to be learned through a social and more implicit learning process [15-17]. The review’s findings suggest supporting SEC development within the clinical learning environment requires specific didactical attention. Yet, there is little research on what clinical placement needs or how to implement it there.

Research findings indicate that SEC require nursing programs to adopt active and innovative learning methodologies in their curricula [8,18,19]. Studies show that when reflection is used as an active learning methodology, SEC development is supported [20-23]. Reflection helps students to process their clinical experiences and learn from them [23]. Specific to SEC, reflection can support the development of interprofessional communication and collaboration skills [20,21], as well as a better understanding of oneself and the workplace experience [22-24]. Thus, using reflection specifically to make the implicit learning process of SEC more explicit warrants further exploration.

Over the past decades, studies on reflection within clinical placement have explored the potential of smartphone technology for its ease of use and ability to support students’ development when workplace coaching is limited [11,14,25,26]. When it comes to clinical skills, research shows this to be an effective approach to improve learning during clinical placement [26-28]. Moreover, many studies show nursing students, educators, and preceptors favor digital apps over paper-based methods of reflection [13,27-31]. Reasons for this preference include greater flexibility, a more convenient method of managing one’s reflections, and better visualization of one’s progress [29]. Although smartphone-based reflection can support clinical skill development, there is little research on how such apps can support SEC development.

This study aims to address this gap and explore the potential of smartphone-based reflection for SEC development during clinical placement. The study is part of a larger project in which we developed a reflection app called Nurse-IT in co-creation with nursing students and educators. The app aims to help students structurally reflect on their clinical experiences for SEC development purposes.

In the development process of the app, design guidelines from both theory and practice have been translated to concrete design features (eg, daily and weekly reflections). Given the lack of applied research on this topic, our study’s main focus is to explore the in situ impact of a SEC-focused reflection app on students’ learning processes. By focusing on students’ experiences with the app from a learning process perspective, we better understand not just the app’s impact, but also how it interacts with the learning environment in which it is introduced. Doing so provides a more nuanced and practical understanding of the developed app and the underpinning design guidelines. This leads us to the following design research question: How do the app’s features impact nursing students’ socioemotional competency development during clinical placement?

To answer this question, we conduct an embedded mixed methods field trial of the Nurse-IT app. By gathering a diverse set of data during the field trial, we aim to get a detailed view on how nursing students incorporate the app’s structural reflection in their learning process. We also measure their perception and usage of the app throughout the clinical placement to better understand how usage develops over time.

The study used an embedded mixed methods research approach within a pragmatist paradigm to capture the potential richness of a naturalistic field trial. The goal of mixed methods research is to integrate knowledge of several methods, both quantitative and qualitative, allowing meta-inferences to be drawn [40]. These meta-inferences aim to explain phenomena in a way that exceeds the sum of a study’s parts (ie, each individual method). Thus, describing the selection of methods as well as the purpose and timing of integrating results in mixed methods studies is important.

For this study, reflection contents, open-ended surveys, and interviews give a qualitative perspective on how students used the tool, how that usage developed over time, and a rich description of their context of use. Embedded within this qualitative study are log data. These give a quantitative perspective on how participants engaged with the tool. By combining perspectives, our understanding is informed both from the actual usage of the tool, as well as the participants’ context-rich experiences with the tool. Together, both strands help illuminate the app’s role within the student’s learning process: which parts of the app are used, what needs do they meet, and why?

It is recommended to visualize the mixed methods design in a diagram to condense its complexity in an explainable manner [41]. In Figure 5, we provide a chronological, vertical diagram of the above-mentioned methods. Arrows between methods highlight how insights from one method informed the data collection of another (ie, building [40]). Data interpretation (ie, analysis and integration) is simplified in this diagram and will be discussed in the Data Interpretation section.

The goal of mixed methods research is to find meta-inferences through the integration of qualitative and quantitative findings. One strategy to do so is through warranted assertion analysis [42], which consists of four steps:

This inductive approach toward mixed methods analysis was chosen as each of our measures shed light on a different aspect of the research question. Although some revealed more than others, they needed to be treated equally valuable for meta-inferences to be drawn. Put in Smith’s [42] words:

We will first report our initial analysis on a measure-to-measure basis. We then describe how we integrated qualitative and quantitative data to reach warranted assertions. The findings section will describe the findings of each measure, after which we discuss our warranted assertions in the Discussion section.

Log data were analyzed quantitatively through descriptive measures (medians, means, and SDs when applicable) to see students’ actual usage of the app. Students’ activity was quantified through (1) the number of days between first and last use; (2) the number of sessions within that time period; (3) the number of learning goals, daily reflections, and weekly reflections created; and (4) the number of tags and emoticons used (both total and average per session).

Learning goal contents were analyzed to see what topics students used and whether they were SEC or clinical skill-related. This was done by (1) creating an overview of all learning goals, categorized per participant; (2) comparing learning goal phrasing; and (3) cross-referencing it with the corresponding daily reflections to see how the learning goal was used in practice. An additional category (introductory period) was later added to refine the categorization.

During this round, reflection contents were only analyzed to get a first impression of how students used both the daily and weekly reflections. This was done by (1) creating an overview of an individual participant’s daily and weekly reflections, (2) reading the reflections, (3) summarizing the participant’s approach to reflecting (what fields were used and overall focus of the reflection).

The qualitative survey data were analyzed for perceived ease of use and utility of the daily and weekly reflections. Per question, responses were analyzed using content analysis. These results were then compared between the 2 questionnaires to analyze potential differences.

The interview schema was used as a lens to structure the semistructured interview analysis. Each transcript was analyzed separately to describe each student’s individual learning process and how they incorporated the app.

The data integration process consisted of repeatedly engaging with the results of the data analyses and the raw data. Performing the data analysis and comparing the results between measures was the first step in identifying potential warranted assertions. Then, using the semistructured interview findings as a starting point, these potential warranted assertions were refined, and new ones were found by importing [40] the other methods’ findings one by one. For example, importing the reflection and learning goal contents helped us understand patterns of use in relation to students’ approaches to learning and their learning environment.

Throughout this process, the integration shifted between a student-specific focus and an overarching one. Doing so allowed us to scrutinize the warranted assertions for disconfirming evidence (ie, if a student’s learning process did not fit an assertion), resulting in a more nuanced analysis.

A risk to study participation was the added workload of maintaining learning goals and reflections in the app, with no guarantee that this work would aid in passing the clinical placement itself. As the clinical placement is partially graded based on personal growth with regard to several learning outcomes (including 2 personal learning goals), we arranged with the clinical placement coordinator that the activity within the app could be used as evidence for such growth. In doing so, we aimed to lower the risk of participation and ensured that students gained something by participating. We prepared alternative material in case technical errors would occur (same format and questions but in separate Word files).

Ethical approval was obtained from the Hanze Ethical Advisory Committee before the commencement of this study (heac.T2024.032).

Informed consent was obtained at the end of the start-up session. Data were stored on the cloud storage of the researchers’ university. As multiple data gathering tools were used, participant data were pseudonymized to allow for data comparison and importation. The translation key was stored in a separate cloud storage which only the lead researcher had access to. The data will be deleted 5 years after the project’s completion. Due to low response rates to the first questionnaire, a small financial compensation was added to the study. The reasoning behind this choice is discussed further in the Limitations section.

In total, 13 participants responded to our call for a posttrial interview, 2 of whom had not used the tool but had enrolled for the study at the start. We specifically included nonusers to better understand the perspective of students who had the opportunity to use the app but chose not to engage with it.

Nearly all app-using participants remarked that usage faded over time. A commonality among the reasons given was the transition students experienced from an initially challenging environment to one in which they experienced more confidence in their capabilities and were less in need of ways to structure their experiences. This decreasing urgency is mirrored by the decrease of in-app activity as seen in the log data.

Combining students’ accounts of this transition with the reflection contents and learning goal categories, we see one distinct usage pattern which affects this (refer to Table 3 for a joint display). The pattern consists of students who used the app predominantly for clinical skill learning goals by logging performed procedures. They describe the transition as moving from a “learner” to a “worker” attitude or “just doing the work.”

For some of these students, this shift in attitude coincided with a sense of having learned all there is to learn for this clinical placement.

Most students who used the app for SEC learning goals or as a more personal journal of their learning experience quit due to the severe technical issue. For them, the app provided a consistent place for experiencing a sense of growth and peace of mind, and kept them from forgetting their experiences.

For all students, no longer using the app meant their conscious learning shifted to unconscious and, for some, more passive learning. The coaches also remarked on this:

Coaches acknowledged the transition being a common occurrence during clinical placement. Coaches mentioned that a low level of detail in a student’s plan of approach, written at the start of the clinical placement, often leads to this passive attitude. Other reasons came down to:

In line with the coaches’ observation, most participants remarked that no longer using the app also resulted in less awareness of their personal and professional growth.

The 2 students who had not used the app seemed to already have a structured learning approach: meaningful situations were noted in a physical journal and reflected upon. Additionally, both could identify and act upon relevant “learning moments” during their shift. This indicates that students who can self-regulate their learning might have less need for this app.

Other students shifted their reflective needs to colleagues or peers, depending on the clinical placement’s own learning environment. Like the app, colleagues in a learning-inducive environment could provide students with insight into their learning and progress. However, those discussions rarely focused on SEC.

Due to the severe technical issue, some students chose to use our nonapp alternative, albeit only briefly. The main reasons for students to prefer the app came down to (1) a clearer connection between learning goals and daily experiences, (2) a stronger sense of achievement upon seeing their progress in the app, and (3) the ease of use of a predefined format for reflection.

This study explored how and why nursing students used the Nurse-IT reflection app’s features to structure their learning process for developing SEC during clinical placement. Using an embedded mixed methods approach and a naturalistic setting, we were able to capture a detailed and realistic picture of how the app was used and how this usage changed during the clinical placement. We will first discuss how students used the app to structure their learning. We then highlight how that pattern of use differed from our intended use and what we can learn from it. Finally, we discuss students’ apparent changing needs during clinical placement and what that means for the design of such reflective applications.

The overarching design goal of the app was to stimulate learner agency through scaffolding a self-regulated learning process (Figure 1). We modeled this learning process by defining learning goals and tasks (forethought phase), daily reflections (monitoring phase), and weekly reflections (self-reflection phase and forethought phase). This section discusses how and why parts of this process supported students in structuring their learning process.

Having the app facilitate a self-regulated learning process alleviated students of the burden of setting up such a process themselves. This is seen in how students valued the app for lowering their mental workload. For example, writing experiences down meant no longer having to remember them. Finally, students remarked how the preformatted questions made it easy for them to engage in goal-setting and reflection. These findings underline the importance of applying persuasive systems design principles when designing such tools [34]. Thus, taking away practical barriers makes engaging in self-regulated learning manageable for students.

Structural engagement with the app made students more aware of their learning process (eg, remembering learning goals during a shift). In turn, this awareness helped them identify learning opportunities and connect those to their learning goals. Other studies focusing on supporting reflection in nursing students show similar effects [12,28]. Survey and interview findings show students valued these effects the most. Overall, this highlights that structurally making one’s learning explicit enables students to act on it.

Moreover, and in line with other studies, students who structurally used the app also remarked how its various overviews helped them see and appreciate their growth [12,35]. This boosted confidence and motivated app usage. This effect was best seen in students who used the app as a reflective journal or to capture and evaluate experiences in relation to SEC-related learning goals. In contrast, students who only recorded clinical procedures appreciated their development, but noted it became repetitive and questioned its long-term value. Thus, making one’s learning explicit also motivates further action, especially when it is focused on SEC-related learning.

Arguably, this could be achieved through more affordable, physical notebooks. However, students who switched to our nonapp alternative said they missed how the app structured their learning process. For example, the link between learning goals, tasks, and daily reflections, and how the app summarized their growth. As a result, they stopped using the alternative, further underlining the benefits of smartphone-based support systems [29-31]. Thus, empowering students to have an overview and awareness of their learning process scaffolds a self-regulated learning process.

A final key finding was the decline in app use, with nearly all students no longer using the app after the halfway point (10 wk). Interestingly, other studies with similar app interventions do not report this decline in use while having similar, mostly positive experiences.

One reason could be that those studies featured shorter clinical placement periods, such as 3 weeks [27,28], 6 weeks [20,46,47], or 8 weeks [12]. Yet, some studies that spanned a similar period of time do not report a decline in use [13,31]. From what we can tell, the difference there lies in whether using the tool was a required part of clinical placement learning. Shajani et al [13] digitized an existing, required reflection exercise, and Slade and Downer [31] measured students’ perceptions of electronic portfolios already being used in a curriculum. Thus, one reason for our dropout rate could be that it was optional to use.

Looking at the interview and focus group findings, we believe the decline is partially due to students experiencing a transitional phase during clinical placement. Students valued the structure provided by the app in the initial, tumultuous phase. Students’ reflections and goals reflect this as they focused mostly on learning protocols, how the team operates, and building rapport with patients. Similarly, this is when they learned clinical skills through hands-on experience. Thus, the urgency of the learning goals was high—“I need to function within the team!”—and one’s day-to-day growth was more palpable—“I am doing things I could not do last week!”

As this phase ended, how students used the app influenced further use. Students whose learning goals focused on clinical skills lacked urgency to further monitor their progress and quit the app. These students often did not manage their learning goals, whereas students with a large set of goals or who used a journaling approach stayed active longer. This indicates certain types of use are more conducive to self-regulated learning and should be advised.

Study coaches remarked that it is harder to get students to critically reflect on their learning process in the second half of the clinical placement. This is seen in how students lost awareness of their learning process and stopped reflecting after app use faded. Moreover, some students missed the oversight and, in hindsight, felt like they would have benefited from further use. This indicates that the work-oriented attitude and its unintentional learning process can hinder self-regulated learning.

The interviewees described clinical placement as being more conducive to learning the starting-phase skills. Most colleagues were willing and capable of helping students reach these goals, for example, through modeling procedures, giving clinical feedback, or explaining protocols. Contrastingly, and in line with other research [15,17], SEC—such as giving feedback or conflict management—were harder to discuss and thus found less uptake by students. Additionally, such skills bear less urgency and are more abstract than, for example, learning a new procedure. Thus, clinical placement as a learning environment and the nature of learning SEC within such an environment also influenced whether students were able to use it for that purpose.

Given the above finding, we believe other intervention components are needed to prepare students for practicing SEC during clinical placement. The app’s goal-setting and reflective structure then enables further learner agency and self-regulated learning of SEC. For example, only the few students in our field trial with clear goals regarding SEC managed to work on them. The study coaches acknowledged this, saying students who start with clearer goals are more likely to maintain a learning attitude. Simulation learning could help prepare students to practice with SEC before clinical placement [3,19,48,49]. Subsequently, SEC becomes more recognizable and their urgency clearer, enabling students to set concrete goals and tasks.

Overall, the app’s structure fulfills students’ early clinical placement learning needs. However, the app’s current design and how it was introduced to students do not support the more intentional approach needed for prolonged use. Looking at a recent similar study [12], we believe a stronger integration with the coaching process could help stimulate a more intentional approach (eg, revising or setting new learning goals). Thus, the design and implementation of a reflection app like ours needs to consider these changing needs to reach its intervention goals.

Two data collection issues occurred during the field trial. First, we observed low response rates to our first survey. Students struggled to keep up with our intended pace for surveys, as they were overwhelmed by the first weeks of clinical placement. Moreover, the surveys were the only part of the field trial that did not benefit their own learning process. We took two measures to address the low response rates:

The second issue was technical and required us to stop the trial in the 13th week. As a result, we could not send out the third qualitative survey nor gather further log data. At that time, there were only 2 active users left, meaning comparatively little log data were lost. Similarly, the third survey would have had a low response rate. Nevertheless, the technical issues did bar us from measuring how students used the reflections to prepare for their clinical placement assessment, a question which could be addressed in future research.

The open setup of the study made it harder for us to evaluate our intended reflective process. We did explain the idea behind the types of reflection and the self-regulated learning cycle as a potential way to use the app. However, we did not require this pattern of use due to the exploratory nature of the study; we were interested in how they themselves would use it. Thus, although our approach led to insights in the variety of ways students used the app in their learning process, our study lacked insight into the potential effect of our intended reflective process.

Similarly, the small sample size, lack of a control group, and design of our study limited our capability to measure the effect of the app on student learning and SEC development. Nevertheless, this means our findings should be interpreted with caution and for their descriptive value of students’ experiences with the app and its role in their learning process. Future experimental research should look into the effect of reflection apps on SEC development during clinical placement. Finally, our small sample size and focus on one specific clinical placement in one specific nursing program limit the generalizability of our findings. We addressed this inherent limitation by supporting transferability of our findings to different contexts. For example, transparency in the design of the app and its underlying guidelines helps other researchers understand the intent behind design choices and gives context to our findings. Similarly, the mixed methods approach provides a rich description of the context, which further supports other researchers’ understanding of the relevance of our findings to their context. In doing so, we aim to support future research on this topic.

The study’s exploratory and naturalistic nature allowed us to follow students’ developments throughout the clinical placement and the role of the app within it. The findings show that supporting students in structuring and monitoring their clinical placement learning helps them during the initial phase of the clinical placement. Although limited, our study indicates that capturing experiences as opposed to logging clinical procedures is key in using reflection for SEC development purposes. Concurrently, we see that the social norms of the workplace impact students’ SEC development, indicating interventions should address this in their design.

Future research should investigate how educational interventions can adapt to students’ changing attitudes toward learning as the clinical placement progresses. Additionally, for SEC development specifically, research into multicomponent intervention designs is needed to help students practice these competencies during clinical placement. Our findings can inform designers and nursing educators of the complexities in designing for nursing students’ SEC development during clinical placement.