This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Nurse practitioners are a common resource for victims of intimate partner violence (IPV) presenting to health care settings. However, they often have inadequate knowledge about IPV and lack self-efficacy and confidence to be able to screen for IPV and communicate effectively with patients.
The aim of this study was to develop and test the usability of a blended learning system aimed at educating nurse practitioner students on topics related to IPV (ie, the mPOWERED system [Health Equity Institute]).
Development of the mPOWERED system involved usability testing with 7 nurse educators (NEs) and 18 nurse practitioner students. Users were asked to complete usability testing using a speak-aloud procedure and then complete a satisfaction and usability questionnaire.
Overall, the mPOWERED system was deemed to have high usability and was positively evaluated by both NEs and nurse practitioner students. Respondents provided critical feedback that will be used to improve the system.
By including target end users in the design and evaluation of the mPOWERED system, we have developed a blended IPV learning system that can easily be integrated into health care education. Larger-scale evaluation of the pedagogical impact of this system is underway.
Violence in interpersonal relationships is a substantial health and social problem in the United States, with approximately 1 in 4 women and 1 in 7 men in the United States reporting being a victim of intimate partner violence (IPV) at least once in their lifetime [
Although IPV affects everyone regardless of age, socioeconomic status, sexual orientation, gender, race, religion, or nationality, research has demonstrated that lesbian, gay, bisexual, transgender, or queer (LGBTQ) individuals experience IPV at rates that are similar to or higher than heterosexual individuals [
More than 75% of practicing nurse practitioners provide care in primary care settings [
An effective method by which nurses’ skills and knowledge about IPV screening and intervention can be improved is to educate them on the dynamics of IPV and the importance of intervention and appropriate care [
The blended learning approach that is becoming an increasingly widespread approach in higher education institutions might provide a useful way to augment face-to-face nursing education regarding IPV. In addition to improving learning satisfaction, communication self-efficacies [
In summary, there is good evidence to support the integration of Web-based modalities to augment face-to-face nursing education. However, successful implementation of a blended learning system requires careful thought about how the course content will be delivered [
Existing literature indicates that interventions and treatments focused on reducing IPV must address myths and misconceptions and provide IPV-related training that is based on up-to-date and geographically and culturally contextualized empirical evidence. Using the most current empirical evidence and input from NEs and IPV health care providers, the current version of the mPOWERED system comprises 4 modules (
Screenshot of the 4 mPOWERED system modules: (A) Prevalence of Intimate Partner Violence (IPV), (B) Types of IPV, (C) IPV and Health, and (D) At-Risk Populations.
The first module (
The graphical components of the mPOWERED system were designed using Adobe Creative Suite in accordance with Mayer’s principles of multimedia design [
To gain the necessary insight into the needs, requirements, and expectations of the mPOWERED system, semistructured interviews were held with NEs and NSs from the San Francisco Bay Area. To be eligible to participate, NEs needed to have experience of working in clinical settings and sufficient experience of teaching in nurse education programs. NSs were eligible to participate if they were currently enrolled in a nurse practitioner certificate program at a US university. Research from the field of usability testing indicates that 80% of usability issues that can be identified in wider implementation will be identified with samples as small as 4 to 5 participants [
Method triangulation was used to increase the validity of findings and gain a more comprehensive understanding of pedagogical usability issues [
At the start of testing, participants were familiarized with the mPOWERED system for approximately 5 min. The participants then performed 4 tasks of varying levels of complexity that covered the full range of functions offered by the mPOWERED system: (1) log in to the system and read through the introduction to the system, (2) work through the first module (
As participants were navigating their way through these tasks, they were encouraged to vocalize anything that crosses their mind (eg, thoughts about any aspect of the system that they liked or disliked, found easy or difficult to understand, or found confusing or contradictory and when they encountered technical difficulties). In addition, participants could receive help from the moderator if they encountered problems or could not manage to go further in the system.
Immediately following usability testing, participants were asked a series of semistructured questions on the perceived usefulness, ease of use, and clarity of information of each system module as well as their experiences and perceptions of the mPOWERED system. The goal of this process was to obtain the participants’ immediate interpretation of a given task scenario and system design and to facilitate the elaboration of usability issues and increasing insight and design suggestions [
After testing, participants completed a Web-based survey implemented using Qualtrics Web-based survey software that contained questions about the perceived usability of the system, participant demographics (age, gender, position title, and prior experience with IPV), and technology use. The perceived usability of the system was evaluated by the 39-item e-learning usability questionnaire [
To evaluate respondent’s prior experience with IPV, we first provided the Centers for Disease Control and Prevention’s definition of IPV and then asked “With this definition in mind, please mark all of the following categories in which (to your knowledge) intimate partner violence has occurred, or is currently occurring,” with the following categories: (1) in your personal experience, (2) immediate family members, (3) close friends, (4) extended family and friends, (5) coworkers or clientele in a work environment, and (6) no instances of IPV. If the participants responded in the affirmative to categories (1) to (6), they answered a follow-up Likert scale question regarding the cumulative number of instances of IPV that occurred in each selected category. Response options for this question were (1) never, (2) once, (3) more than once, (4) a few times, and (5) many times.
With prior approval of the participants, the interviews and focus groups were video recorded using Morae usability and analytics software. All participant and moderator comments along with feedback were independently transcribed using qualitative content analysis [
The participants’ characteristics are reported in
Participants’ demographics, clinical and teaching experience, electronic learning, and intimate partner violence experience.
Characteristics | Nurse educators | Nursing students | Overall | |
Age (years), mean (SD) | 58.4 (11) | 37 (7) | 47.7 (15) | |
|
|
|
|
|
|
Male | 1 (14) | 3 (16) | 4 (15) |
|
Female | 5 (71) | 16 (84) | 21 (81) |
|
Genderqueer | 1 (14) | 0 (0) | 1 (4) |
|
||||
|
Caucasian | 2 (29) | 5 (28) | 7 (28) |
|
Black/African American | 1 (14) | —a | 1 (4) |
|
Asian/Asian American | 3 (43) | 9 (50) | 12 (48) |
|
Hispanic/Latino | 1 (14) | 2 (11) | 3 (12) |
|
Hawaiian or Pacific Islander | — | 1 (6) | 1 (4) |
|
Other | — | 1 (6) | 1 (4) |
|
||||
|
<1 | — | 4 (22) | 4 (16) |
|
1-5 | — | 4 (22) | 4 (16) |
|
5-10 | 1 (14) | 6 (33) | 7 (28) |
|
>10 | 7 (100) | 4 (22) | 11 (44) |
Teaching experience (years), mean (SD) | 12 (3) | N/Ab | — | |
|
||||
|
Daily | 4 (57) | 13 (72) | 17 (68) |
|
Twice weekly | 2 (29) | 3 (17) | 5 (20) |
|
Twice monthly | 1 (14) | 2 (11) | 3 (12) |
|
||||
|
No instances of IPV | — | 2 (11) | 2 (8) |
|
Current or past relationships | 2 (29) | 6 (33) | 8 (32) |
|
Immediate family members | 1 (14) | 8 (44) | 9 (36) |
|
Close friends | 2 (29) | 6 (33) | 8 (32) |
|
Coworkers or clientele in a work environment | 3 (43) | 6 (33) | 9 (36) |
aNo respondents.
bN/A: not applicable.
cIPV: intimate partner violence.
Overall, 3 themes emerged from the data to describe NEs’ and NSs’ views about the usability of the mPOWERED system. These were (1) ease of use, (2) usefulness, and (3) aesthetics. Selected ad verbatim quotations from the interviews are presented to illustrate these themes.
This theme included the subthemes content presentation and navigational fidelity. The general opinion among respondents was that IPV content was presented in small understandable chunks (NE mean 4.86; NS mean 4.71) and in a way that supports learning (NE mean 4.86; NS mean 4.65):
I like that you guys broke it down into modules, like small little chunks. And that you can do the quiz and see the answers right away. Yeah, full of rich information.
I would say this was put forth very well, because you don’t overwhelm in a module. I have modules for nursing where there’s 27 topics just in one, in one go- so it’s really hard to remember some of that stuff. But here, you’ve outline- to me the most important things that you’ve highlighted is the prevalence of IPV, who are the victims- so we know that it’s pretty much all ages, um all socioeconomic um demographics. And you’ve included statistics as well, and the types of IPV. So I think that it’s presented very straightforward, not too much.
An important facet of e-learning and blended learning is enabling the learner to control the amount of material they consume as well as when they learn the material. Usability testing indicated that most respondents found that the mPOWERED system enabled them to control the pace of learning (NE mean 4.43; NS mean 4.71), with 1 NS reporting:
I like that this whole system is very simple and straight forward. It makes it easier. I think any students who are using this will enjoy it, and it’s not really a looong, dry, or boring lecture. It’s actually quite interesting. It keeps it very simple to the point.
Participants felt that content was presented in a consistent manner (NE mean 4.71; NS mean 4.59) but mentioned that integrating audio narration into the system would emphasize the organization of key points in the material and reduce the overall cognitive demands placed on the user:
There is a lot of statistics and a lot of information, like minute information. So I would say having this presentation narrated it’s gonna be helpful. And forcing the reader to stop and look at the different types of x, y, and z.
Aspects of usefulness included consistency with the curriculum and system interactivity. Overall, both NEs and NSs expressed liking the mPOWERED system, finding that the content was congruent with the manner in which IPV content is taught in the classroom setting (NE mean 4.86; NS mean 4.65):
I thought this was an excellent way to present IPV to students. It was informational, provided interaction with the module, and user friendly.
Participants were of the view that the mPOWERED system could augment classroom learning of IPV (NE mean 4.86; NS mean 4.45), with all NEs stating that the system added flexibility in the teaching and learning environment, and would allow them to have deeper and richer teacher-student interactions during in-person classroom meetings:
I would use it to do, like before going to class for the lecture. Cos [Because] that will have them primed to start conversations and share stories about IPV, whether they know about it, or heard about it, or seen it, in a clinical setting.
Participants also stated that the mPOWERED system engaged the user (NE mean 4.71; NS mean 4.61) and motivated them to learn about IPV (NE mean 4.43; NS mean 4.71). This was especially true for the short multiple-choice quizzes at the end of each module. Both NSs and NEs unanimously felt that quizzes encouraged active learning and especially appreciated that the system provided adaptive feedback on students’ answers after each question:
I like the quiz because it’s interactive and you click on it and it gives you the answer right away. As opposed to answering the questions and then having the answers at the end. So I like this format. So keep that definitely.
One facet of the mPOWERED system that NSs particularly liked was the health care provider slides (see
Exemplar screenshots of a (A) health care provider slide and (B) interactive quiz question.
My favorite thing is the doctor/nurse speaking to you at various junctions of the program. This strengthens the messages.
Love the use of “I,” and the communicative approach. [...] She’s telling you this. That’s what I first thought at the beginning, I really like that you’ve kept it going throughout.
In contrast, many NEs did not like the health care provider slides, with respondents expressing a dislike for the first-person perspective:
It was distracting and it made me think “oh well ok, they are trying to plant the seed in my head that I’m supposed to think this way.” I see it as a distraction and the way I perceived it in my head is that they were trying to subconsciously make me think that way.
In general, participants were satisfied with the mPOWERED system, finding that it improved their understanding of the content material. However, both NSs and NEs stated that the system content should be expanded to include information regarding identifying and evaluating IPV and how to offer first-line support for IPV:
Well this just says how it impacts, it doesn’t tell me what I can do.
This tells me about it [IPV], but not what we do about it. I’d want to know what to do.
The theme aesthetics explored aspects including visual design and understandable and meaningful symbolic representations. First, there was a high level of satisfaction among the NSs and NEs regarding the balance between text and graphics on each slide (NE mean 4.78; NS mean 4.82), with participants reporting that the IPV-related icons were easy to interpret (NE mean 4.57; NS mean 4.18):
I think the size of the icons are good here, they are clear as to what they represent...These as well. So without the words I could likely put together a lot of>what’s going on here.
The icons are good. They’re pretty straightforward, in terms of like if you took away the titles you could probably deduce what they’re meaning.
In addition to comments about the symbolic representations, the use of icons to illustrate IPV-related scenarios was well received by participants and conveyed the seriousness of the issue in a compelling fashion (NE mean 4.71; NS mean 4.18):
I do really like the pictures. Yeah, those are good. I think it shows distress in a sense that, I guess it’s just a matter of visualizing the significance of violence, you know? And kind of seeing what someone is going through in the intimacy of their home and just kind of seeing it as a kind of significant thing. It has this emotional thing where you are more connected to the content. You think it could come off as cheesy, or minimizing it, like characters getting beaten up. But I think it helps you just understand the importance of “this is what someone is going through.” And especially in this particular way, and seeing the way the body is contorted, it doesn’t make it so graphic. But it still is kind of like “ohhh”.
I like the fact that its pictures, and not real people. Cos [Because] that doesn’t sit well with me when I watch actors screaming at each other.
That said, others suggested the use of real graphics to improve the graphical depiction of scenarios. For example, 1 NS recommended adding actual photos to the system to complement the existing icons (see
Screenshots of (A) initial and (B) modified intimate partner violence (IPV) and health module slides revised to medical images and radiologic findings of physical injuries associated with IPV.
You know, actually, for healthcare professionals or people that you want to show this information to I think real photos would probably really hit home. In terms of like drawing them into the emotional side, the magnitude of what’s going on.
In general, participants felt that the system avoided reinforcing negative stereotypes (NE mean 4.43; NS mean 4.59), with icons portraying diverse demographics and numerous situations in which IPV can manifest itself. However, 1 NE with expertise in LGBT issues felt that some of the icons were too stereotypical and outdated:
Here’s another thing to think about. Its binary, it’s always women and men. I think that’s something that we are having to look at when we are discussing things now. Is it important to say women and men, or people who identify as women, or people who identify with me. Because I would say this number shoots up when it is a transwoman. And actually a transman. Actually they are really vulnerable in relationships.
In general, this is men on women, which I think for nurses I think that they should understand that there are different types of violence. Like slashing his tire wheels, cutting up his clothes that are in his closet...I would want to see it if we are teaching all sorts of health professionals for it to be more broader.
The aim of this study was to conduct usability testing of the mPOWERED system of IPV education and to determine the acceptance of the system and its features among NEs and NSs. Overall, the mPOWERED system was well accepted by most respondents as an alternative teaching/learning modality. Participants’ comments suggest that the ease of navigating the system and amount of content were appreciated and that the e-learning system provides a user-friendly, visually appealing means that can alleviate the difficulty of adding IPV education in already content-heavy nursing program curriculums [
Positive user experience is of prime importance to educators and educational systems. As adult learners, students are more engaged when learning directly relates to their professional career [
The usability of e-learning software products is a key characteristic to achieve the acceptance of academic users [
Second, participants expressed that adding medical images and radiologic findings to the mPOWERED modules would provide users with realistic images that could facilitate the identification and evaluation of IPV. Cognizant of the fact that 25% of women and 10% of men in the United States have been a victim of IPV during their lifetime [
Third, participants commented that the mPOWERED system should include a module providing evidence-based information on how to (1) identify and evaluate IPV and (2) offer first-line support for individuals experiencing IPV. The expressed desire for this specific additional information is consistent with the literature indicating that the lack of education and training poses a significant obstacle to IPV screening and support [
This study makes a number of unique contributions to the development of blended learning systems for health care–related topics. Using a purposive sample methodology, we were able to get feedback from participants who would likely use this technology, both from the perspective of the educator and the student. After implementation of suggested changes, we plan to evaluate the pedagogical impact of the mPOWERED IPV education system to improve IPV knowledge, attitudes, and skills in students enrolled in our nurse practitioner program and then expand its use in our prelicensure nursing program. In addition, we will explore the ability of the system to educate other health care professionals (eg, physical therapists, occupational therapists, and physician assistants) on this serious public health topic.
NEs need to ensure that students are adequately prepared and competent to identify and evaluate IPV, are able to provide adequate assistance and support to victims, and can refer victims to specialized support services. A usability test conducted to explore users’ experiences with the mPOWERED system indicated that the system was useful, usable, and satisfying. Furthermore, participants proposed that the mPOWERED system would be a useful tool to augment traditional education and were keen to see more blended learning in their program curriculum. Both NEs and NSs identified design issues that will provide direction in the next stages of product development. The mPOWERED system has considerable potential to augment traditional classroom learning about health topics such as IPV.
electronic learning
intimate partner violence
lesbian, gay, bisexual, transgender, or queer
nurse educator
nursing student
None declared.