systematic literature review of virtual reality

• Reference Manager
• Simple TEXT file

People also looked at

Systematic review article, enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis.

• 1 College of Nursing, Hanyang University, Seoul, Republic of Korea
• 2 Department of Nursing Science, Chungbuk National University, Cheongju, Republic of Korea

Aim: Studies on the effectiveness of virtual reality (VR) in nursing education have explored its impact on learning outcomes, emotional immersion and engagement, learner self-confidence, and satisfaction, generally showing positive aspects. However, there is a need for a systematic review to examine the specific influence of VR-based education on nursing students’ practical competency.

Method: According to the PRISMA 2020 guidelines, 22 studies were selected based on inclusion criteria from 579 articles, published from January 1, 2018, to March 31, 2024, across nine major databases including PubMed and EMbase. The target population comprised nursing students, and the intervention focused on VR-based simulations aimed at enhancing competency, compared to control groups receiving either no intervention or conventional non-virtual simulation. The primary outcome, nursing competency, was analyzed using MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) to calculate pooled effect sizes.

Result: The pooled effect size for nursing competency was determined to be large, with Hedge’s g = 0.88 (95% CI, 0.47 to 1.29). Meta-regression analysis identified several factors associated with an increase in nursing competency. These included studies published after 2022, approval of an IRB, absence of funding, randomized controlled trials (RCTs), interventions reported as shorter than 4 weeks or not reported, sessions fewer than 4 or not reported, session duration under 1 h or not reported, and observational measurement methods. Additional factors enhancing nursing competency were the inclusion of a pre-briefing before simulations, the absence of a debriefing afterward, and the exclusion of other activities during the simulation.

Conclusion: By combining the results of the included studies, the systematic review and meta-analysis accounted for variations in sample size, study methodology, and independent intervention effects, providing an overall evaluation of the effectiveness of simulation-based education in improving nursing students’ competency.

Limitation: The selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results.

Systematic review registration: PROSPERO International Prospective Register of Systematic Reviews: http://www.crd.york.ac.uk/PROSPERO/ , identifier CRD42023446348.

1 Introduction

Nursing education is an applied discipline in which theory and practical education are combined; prospective nurses prepare to become competent by applying the knowledge learned in theoretical education to the practical education process. The need for nursing education to train professionals who provide nursing and medical services to humans utilizing digital-based, non-face-to-face media such as artificial intelligence (AI) and big data has recently become more urgent ( 1 ). In nursing education, there has been an increasing interest in virtual-reality simulation (VRS) education as an alternative and complementary method to traditional simulation education, providing students with new learning experiences in a reproduced clinical environment and enhancing clinical adaptability ( 2 ). Virtual reality (VR) is defined as “the use of partial immersion through a digital learning environment (computer, tablet, phone, screen, etc.) to foster a perceived lived experience for an intended outcome (e.g., learning and entertainment)” ( 3 ). This study defines VRS to include VR and its derivatives, augmented reality (AR), and mixed reality (MR), using the terminology consistently. In VRS, learners can collaborate with other healthcare professionals to provide interventions, such as solving patients’ problems or practicing simple skills ( 4 , 5 ). Improved clinical performance skills, knowledge, and metacognition, as well as enhanced learning satisfaction, communication, self-efficacy, confidence, and teamwork have been reported as effects of these VR programs ( 4 , 6 ). In addition, studies on the effectiveness of nursing education using VR have been conducted on learning effectiveness, emotional engagement and immersion, learner confidence, and satisfaction ( 7 , 8 ). Reportedly, VRS programs for nursing skills are effective in improving skills ( 9 ) and have the advantage of enabling safe and repetitive training without time and space constraints ( 10 ). Thus, learning through VRS has demonstrated improvement in various factors related to clinical nursing competency, albeit often assessed in a fragmented manner. As various forms of VRS are being applied in nursing education, and diverse elements contributing to nursing competency are considered, there is a need to comprehend the holistic outcomes of these studies. Consequently, this study aims to comprehensively review the results, considering nursing competency in a broader sense that encompasses collaboration, interpersonal relationships, communication, professional development, and the nursing process, skills, and education ( 11 ).

Moreover, a systematic review and analysis of nursing students’ outcomes are essential for determining specific factors that are deemed effective. Systematic reviews and meta-analyses can amalgamate the results of included studies, accounting for differences in sample size, variations in research approaches, and intervention effects among independent studies. We believe that the systematic review and meta-analysis in this study will enable an assessment of the overall effect of VRS-based education on nursing students’ nursing competency. Consequently, this study aims to provide foundational data on VRS by conducting a systematic literature review and meta-analysis, investigating the improvement effect of VRS on nursing students’ nursing competency as a primary outcome, and examining knowledge, self-efficacy, problem-solving skills, confidence, and satisfaction as secondary outcomes.

This study aims to acquire and analyze evidence regarding the enhancement of nursing students’ nursing competency through VRS. The primary outcome focuses on nursing students’ self-reported feelings and reactions, while the secondary outcome assesses nursing students’ nursing competency following exposure to VRS.

2 Materials and methods

2.1 search strategy and data sources.

The search was jointly conducted by two researchers, Cho, M.-K. and Kim, M.Y., across nine electronic databases or e-journals: PubMed, Cochrane, EMBASE-OVID, CINAHL, World of Science, SCOPUS, PQDT, APA PsycArticles, and Research Information Sharing Service. The primary search, conducted from July 18, 2023, to August 20, 2023, targeted articles published in English and Korean from January 1, 2003, to April 30, 2023. A secondary search was carried out from April 6, 2024, to April 9, 2024, focusing on articles published from May 1, 2023, to March 31, 2024, also in English and Korean. The search strategy and formula, following the PICO-SD framework (population, intervention, comparison, outcome, study design), are detailed in Table 1 . The keywords employed in search terms across the nine databases included combinations and variations of “nursing students,” “virtual reality,” “augmented reality,” “extended reality,” “metaverse,” “competency-based education,” “clinical competence,” “competency,” and “controlled clinical trial.” These keywords were chosen to comprehensively capture studies relevant to the impact of virtual reality simulation on nursing competency.

Table 1 . Search strategy according to PICO.

2.2 Inclusion and exclusion criteria

The reporting of the results adhered to the PRISMA 2020 checklist. Inclusion criteria comprised nursing students aged 19 years or older (Population), interventions involving VRS (Intervention), with conventional learning methods or no intervention as the control (comparison). The primary outcome was nursing competency, and secondary outcomes included knowledge, self-efficacy, problem-solving, confidence, satisfaction, and other variables, which were concurrently measured. If multiple measurements were conducted post-intervention, the first measurement was used to calculate the effect size. Only studies presenting subject numbers, means, and standard deviations in the results were selected for precise effect-size calculation. The study designs included randomized controlled trials (RCTs) and quasi-experimental studies. Exclusion criteria included studies encompassing students from majors other than nursing, interventions using conventional simulation-learning methods instead of VRS, the absence of nursing competency as an outcome variable, studies not reported in Korean or English, studies with inaccessible original texts, and single-group studies lacking a control group.

2.3 Data extraction

Two researchers, Cho, M.-K. and Kim, M.Y., independently conducted searches and selected studies for analysis based on the predefined inclusion and exclusion criteria. The selected studies were extracted, incorporating information such as author, year of publication, country, publication language, number of schools, institutional review board (IRB) approval, funding details, number of participants, study design, intervention characteristics (type, facilitator, duration, session, time/session, pre-briefing, debriefing, other activities, outcome measurement time, and measurement method), quality assessment score, and dependent variables. This information was meticulously recorded in a coding book created using the Microsoft Excel spreadsheet software. Any disparities in coding were addressed by revisiting the original text to ascertain and input the final coding values ( Table 2 ).

Table 2 . Descriptive summary of the included studies.

2.4 Quality assessment

The quality assessment of selected articles was independently performed by Cho, M.-K. and Kim, M.Y. using the Joanna Briggs Institute (JBI) Checklist for RCTs and the Checklist for Quasi-Experimental Studies. Five RCTs were assessed using the 13-question JBI Checklist; the average score was 8.40, and all five studies lacked clear reporting on “Q2. Was allocation to treatment groups concealed?” and “Q4. Were participants blind to treatment assignment?” Quasi-experimental studies comprised eight articles, and on evaluation using the 9-item JBI Checklist for Quasi-Experimental Studies ( 32 ), the average score was 8.50, with generally well-reported items ( Table 3 ).

Table 3 . Quality assessment of the included studies.

2.5 Statistical analyses

MIX 2.0 Pro (Ver. 2.0.1.6, BiostatXL, 2017) was used to calculate and merge effect sizes for both the primary outcome of nursing competency and secondary outcomes. The overall effect was determined using a random-effects model, considering between-subject variability and heterogeneity between studies. Hedge’s g was employed for effect-size calculation, and significance was assessed using 95% confidence intervals (CIs), Z tests, and p- values. The weight of each effect size was determined using the inverse of variance ( 33 ). Heterogeneity was evaluated using Higgin’s I 2 ( 34 ), with an I 2 of >50% indicating heterogeneity ( 35 ). Subgroup analysis, meta-regression, and exclusion-sensitivity analysis were conducted for nursing competency to identify factors contributing to heterogeneity. Publication bias was examined using funnel plots, trim-and-fill plots, Begg’s test, Egger’s regression, and the trim-and-fill method to correct for the overall effect ( 36 ).

3.1 Characteristics of the included studies

A total of 579 articles were initially identified from 9 databases following the search strategy. After excluding duplicates, 373 articles were extracted. Following the application of the inclusion and exclusion criteria, 21 research articles were ultimately selected. The research by Sahin Karaduman and Basak ( 28 ) was designed using two experimental groups and was analyzed as two separate studies, resulting in 22 studies being analyzed ( Figure 1 ). Of these, six studies were published before 2022; three were conducted in the United States (USA), twelve studies were published in English; nineteen were conducted at a single university; nineteen and nine studies had IRB approval and funding, respectively. The study designs included seven RCTs, fifteen quasi-experimental studies, and eight studies with fewer than 60 participants. Interventions comprised 18 VR/AR simulations and four metaverse. Eight studies had a professor as a facilitator, four had an intervention duration of more than 4 weeks, two had four or more intervention sessions, eight had an intervention time of more than 1 h per session, 19 had a pre-briefing, and nine had a debriefing. Dependent-variable measurements were taken immediately after the intervention in 20 studies, the measurement method was observational measurement in 12 studies, 19 studies had no additional activities, such as reflection, besides the simulation, and 14 studies had an above-average quality assessment score ( Table 2 ).

Figure 1 . PRISMA flow diagram. An article by Sahin Karaduman and Basak ( 28 ), designed using two experimental groups, was divided into two studies.

3.2 Effect of VRS-based intervention on nursing competency

The overall effect of nursing competency, as the primary outcome for the 22 VRSs, was found to be Hedge’s g  = 0.88 (95% CI: 0.47 to 1.29). This was interpreted as a large effect based on the criteria provided by Brydges ( 37 ) for interpreting effect sizes ( Figure 2 ). The high degree of heterogeneity among the studies, indicated by Higgins’s I 2 of 91.8% in the heterogeneity test, prompted subgroup and meta-regression analyses to explore factors contributing to this heterogeneity.

Figure 2 . The effect of virtual reality simulation-based intervention on nursing competency. ES, effect size; CI, confidence interval. An article by Sahin Karaduman and Basak ( 29 ), designed using two experimental groups, was divided into two studies (18 a and 18 b ). *Removal of the number of duplicate subjects in the 18th study.

In subgroup analyses, the characteristics of studies significantly associated with improvements in nursing competency IRB-approved studies (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); interventions with a duration not reported or those with a duration of less than 4 weeks (Hedge’s g = 1.05, 95% CI: 0.56, 1.53); interventions with sessions not reported or those with less than 4 sessions (Hedge’s g = 0.95, 95% CI: 0.50, 1.39); those with outcome measurement immediately after the intervention (Hedge’s g = 0.93, 95% CI: 0.50, 1.37); those with pre-briefing before the simulation (Hedge’s g = 0.71, 95% CI: 0.23, 1.20); those with debriefing after the simulation (Hedge’s g = 1.02, 95% CI: 0.57, 1.48); and those with no other activities besides the simulation, such as keeping a reflective journal (Hedge’s g = 1.03, 95% CI: 0.55, 1.50). Publication year, Country, publication language, number of schools, funding status, research design, number of participants, intervention type, facilitator, intervention time per session, measurement method, debriefing, and quality assessment score also showed statistically significant effect sizes ( Table 4 ).

Table 4 . Subgroup analysis of nursing competency according to study characteristics.

Univariate meta-regression identified factors influencing the overall effect. Publication year after 2022 ( Z = 2.68, p = 0.007); having an IRB ( Z  = 5.17, p  < 0.001); having an fund ( Z = −2.61, p = 0.009); RCT ( Z = 2.02, p = 0.044); intervention duration over than 4 weeks ( Z = −3.33, p < 0.001); intervention session over than 4 sessions ( Z = −3.01, p < 0.001); intervention time/session over than 1 h ( Z  = −5.20, p  < 0.001); observational measurement rather than self-reporting ( Z = 3.21, p = 0.001); having a pre-briefing before the simulation ( Z = 3.76, p < 0.001); having a debriefing after the simulation ( Z = −4.41, p < 0.001); and having other activities besides the simulation ( Z = −4.41, p < 0.001) had statistically significant effects on nursing competency ( Table 5 ).

Table 5 . Meta-regression analysis to evaluate competency.

The exclusion-sensitivity test ( 38 ), excluding one study at a time, showed Hedge’s g ranging from 0.67 to 0.94, indicating a moderate to large effect size. The 95% CI (0.36 ~ 0.53, 0.98 ~ 1.36) did not include 0, signifying statistical significance. The effect sizes from the exclusion-sensitivity test were not significantly different from Hedge’s g  = 0.88, which included all 22 studies ( Table 6 ).

Table 6 . Exclusion-sensitivity test of the virtual-reality simulation-based intervention.

3.3 Effect of intervention program on secondary outcomes

The secondary outcomes in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge, self-efficacy, confidence, and satisfaction exhibited statistically significant changes. After VRS, knowledge and self-efficacy showed significant increases, with moderate effect sizes of Hedge’s g  = 0.60 (95% CI: 0.07, 1.14) and Hedge’s g = 0.53 (95% CI: 0.09, 0.97), respectively. Confidence and satisfaction exhibited substantial increases, with large effect sizes of Hedge’s g = 1.02 (95% CI: 0.48, 1.57) and Hedge’s g = 1.35 (95% CI: 0.43, 2.28), respectively ( Table 7 ).

Table 7 . Effects of virtual reality simulation-based intervention on other variables.

3.4 Publication bias

To evaluate publication bias, funnel-plot and trim-and-fill plot analyses were conducted. Represented by the black circle, the individual effect sizes of the 22 studies included in the study were asymmetrical— slightly skewed to the right—indicating some potential publication bias ( Figure 3 ). The trim-and-fill plot suggested the addition of one study, represented by the white circle, skewing to the left ( Figure 4 ). Further analysis, The coefficient of the bias was 8.58, indicating the initial value (intercept) and the p -value was 0.001. Thus, the null hypothesis was rejected, and the existence of a publication bias could be confirmed. Unlike Egger’s regression test result, Begg’s test for rank correlation (Tau b = 0.27, ties = 0; Z = 1.75, p = 0.080) confirmed the absence of publication bias. Moreover, the trim-and-fill method suggested adding one article; the effect size of the 23 corrected articles was 0.60 (95% CI: 0.49, 0.72). Although the effect size of nursing competency was somewhat smaller after correction than before, it remained statistically significant. In conclusion, this study was deemed free of publication bias ( Table 8 ).

Figure 3 . Funnel plot of virtual reality simulation-based intervention on nursing competency. Precision = 1/standard error; 0.05; limit line = 95% confidence limit.

Figure 4 . Trim and fill plot of virtual reality simulation-based intervention on nursing competency. Precision = 1/standard error; 0.05; limit line = 95% confidence limit.

Table 8 . Publication bias test of virtual reality simulation-based intervention on competency.

4 Discussion

In this study, the impact of simulation-based programs on nursing competency demonstrated a significant effect size of 0.88. It’s notable that this simulation-based program yielded encouraging results by positively enhancing nursing competency. This is consistent with similar improvements observed in self-efficacy, a factor linked to nursing competency ( 19 ), enhanced knowledge, educational satisfaction, and academic achievement through VR in a hospital environment ( 39 ); and improved nursing-process performance ( 40 ), heightened critical thinking, clinical performance, and practice satisfaction through vSim for Nursing ( 41 ). Additionally, these results partially correlate with those in a study indicating that hands-on training utilizing scenario-based admission management in VR increased learning immersion, learner confidence, and learning satisfaction ( 7 ).

In the meta-regression analysis evaluating nursing competency, several factors emerged as influential. First, in cases where the publication year was 2022 or later, nursing competency was found to be significantly improved compared to studies that received IRB approval, compared to studies published before then. In the evolving landscape of clinical practice, recent emphasis on patient safety and rights has shifted the focus toward observing nursing behavior rather than direct patient care ( 42 ). This shift underscores the active implementation of simulation-based education, suggesting a more systematic adaptation of teaching methods and educational systems to enhance nursing competency compared to previous approaches. Moreover, studies with an intervention duration not reported or one of less than 4 weeks demonstrated a significant effect on nursing competency compared to those lasting more than 4 weeks. In cases of intervention with fewer than four sessions, competency was significantly improved compared to intervention sessions with four or more sessions. Similarly, interventions with time per session not reported or those lasting less than 1 h were associated with a significant improvement in nursing competency compared to those lasting more than 1 h. These findings suggest that shorter, more intensive interventions may be more effective in enhancing nursing competency through VRS. Establishing short-term intensive courses could thus be a meaningful approach. Even in the case of pre-briefings, which are recognized for their utility, the introduction and assignment of roles and expectations during pre-briefings may not be optimal. This is because simulation anxiety is linked to higher levels of extraneous cognitive load ( 43 ). Further investigation into the timing and temporal aspects of these activities is warranted to optimize their effectiveness. Therefore, further research specifically focusing on the temporal aspect is deemed necessary to comprehensively understand its implications.

Furthermore, pre-briefing before simulation emerged as a significant factor contributing to the improvement of nursing competency compared to that in the control group. This is consistent with the recognized importance of pre-briefing in face-to-face simulations, in which it influences simulation readiness ( 44 ). Given that most included studies conducted virtual pre-briefing activities individually, such as pre-briefing lessons and quizzes, the findings imply that virtual pre-briefing can be actively utilized with comparable effectiveness in face-to-face simulations. Various pre-briefing methods, including role rubrics, are currently under development ( 45 ). Further research will be necessary to ascertain the effectiveness of these diverse pre-briefing approaches.

Moreover, this study identified that post-simulation debriefing had a more significant effect of improving nursing competency compared to non-simulation debriefing. This could be attributed to the characteristic of VRS that enables repeated and reflective learning through debriefing with immediate feedback, thus providing learner-customized learning ( 46 ). The ability to facilitate individual improvement in nursing competency through immediate feedback is consistent with previous studies emphasizing the effectiveness and importance of debriefing in simulation ( 47 ). While debriefing in a virtual setting may differ from team interaction, reflection, and discussion in a face-to-face simulation, the results underscore the crucial role of debriefing in VRS situations.

Competency improved significantly when observation was measured rather than self-report. Role assignment in nursing simulation often elicits significant anxiety stemming from uncertainty, performing in front of faculty and peers, and social evaluation ( 45 ). Moreover, many individuals perceive themselves as lacking proficiency, particularly in terms of nursing competency. Consequently, self-reported improvements in nursing competency may underestimate actual progress observed through objective evaluation. Hence, effective communication and encouragement regarding the significance of simulation are vital when implementing simulation programs.

Nursing competency was statistically significantly improved when compared to those who did not engage in any other activities other than simulation. Other activities take as much time, which suggests that core simulation activities are important for improving nursing competency. Non-simulation activities, denoting the absence of activities other than simulation, exhibited a significant effect on nursing competency. While non-simulation activities may improve competencies such as team cooperation, communication, or empathy, they were not associated with improvements in nursing competency. This suggests that clear simulation content, along with pre-briefing and debriefing activities tailored to enhance nursing competency, directly influence this competency.

Meanwhile, several variables did not demonstrate a statistically significant effect of improving nursing competency. The country, number of centers, funding status, research design, and all the variables related to the operation of the intervention program (participants, intervention type, facilitator, intervention session, and outcome-measurement time), as well as the quality score, did not show significant differences in improving nursing competency. The inconsistency in trends observed across these variables can be attributed to the diverse definitions and measurements of nursing competency utilized in the included studies. This variability in research outcomes underscores the absence of a standardized measurement tool for nursing competency, potentially leading to increased heterogeneity in results.

Furthermore, the secondary outcomes measured alongside nursing competency in this study included knowledge, self-efficacy, problem-solving, confidence, and satisfaction. Among these, knowledge and confidence demonstrated statistically significant improvement. These variables, particularly knowledge and confidence, are closely related to nursing competency and can concurrently improve with it in VRS. Conversely, self-efficacy, problem-solving, and satisfaction did not show significant improvement. This is consistent with previous research indicating that VR nursing education improves knowledge ( 48 ) and increases learning satisfaction ( 49 ) but does not enhance technical skills ( 48 ) or significantly impact self-efficacy ( 49 ). This suggests that while VRS is effective in improving knowledge-related competencies, consistent improvements in self-efficacy, problem-solving, and satisfaction may depend on its design and utilization.

Given that learning immersion through simulation has been demonstrated to impact the development of clinical-nursing competence ( 50 ), and VR-based programs have been effective in improving cognitive performance, such as theoretical knowledge, through realism ( 51 ), VRS holds promise in nursing education. However, the results of this study underscore the need to carefully consider elements that are more challenging to implement in virtual situations than in face-to-face scenarios. Therefore, further research, such as systematic reviews and meta-analyses exploring other variables in VRS, is recommended for a more comprehensive understanding of its impact on nursing education.

VR-based nursing education represents an innovative field that has not been previously explored. These simulators offer a range of environments that transcend physical constraints, enabling participants to immerse themselves within the virtual space ( 52 ). It’s crucial for educators responsible for program development to grasp the distinctions between virtual reality and reality to facilitate effective education.

This study underscores the significance of pre-briefing and debriefing elements in VR-based simulation, highlighting the importance of their organization. Rather than focusing solely on operational time or the duration of the simulation itself, the key lies in how these elements are implemented for optimal educational outcomes. Additionally, when assessing effectiveness, we advocate for a combined approach utilizing both self-reported evaluations and objective evaluations through observation or assessment.

4.1 Limitations of the study

This study acknowledges several limitations. First, there is variability in reporting randomization methods among the included studies, with some providing comprehensive discussions on the topic while others lack detailed information on the methods employed. Second, the diverse interpretations and definitions of nursing competency across the included studies may introduce variability in the study outcomes. Third, the absence of a standardized measurement tool for nursing competency could contribute to increased heterogeneity. Fourth, the selection criteria for the studies analyzed, which included only those published in English or Korean and reported precise means, standard deviations, and sample sizes, could lead to selection bias and limit the generalization of our study results. Additionally, the studies encompass sample sizes from different countries, further contributing to overall heterogeneity. To enhance the robustness of future research and validate the effectiveness of interventions for nursing students, larger sample sizes and higher-quality studies are recommended.

5 Conclusion

The meta-analysis of nursing competency in VRS revealed the latter’s effectiveness in enhancing nursing competency. Notably, the incorporation of key elements from face-to-face simulation, such as pre-briefing and debriefing, significantly improved nursing competency compared to scenarios in which these elements were absent. This study suggests the importance of reflecting core simulation elements in virtual simulations and underscores the need to enhance the quality of pre-briefing and debriefing in virtual contexts. Moreover, the findings suggest that intensively operating VRS over a short period could be more effective in improving nursing competency. This implies the significance of considering the effectiveness of short-term intensive courses for nursing-competency improvement within virtual spaces. The study findings provide valuable insights for the design of VRS aimed at enhancing nursing competency.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Author contributions

M-KC: Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. MK: Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing.

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by the National Research Foundation of Korea grant funded by the South Korea Government (MSIT; no. 2022R1F1A1076248).

Acknowledgments

The authors sincerely thank those who participated in this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

1. Han, D-L . Nursing students’ perception of virtual reality (VR) and needs assessment for virtual reality simulation in mental health nursing. J Digit Contents Soc. (2020) 21:1481–7. doi: 10.9728/dcs.2020.21.8.1481

Crossref Full Text | Google Scholar

2. Foronda, CL, Fernandez-Burgos, M, Nadeau, C, Kelley, CN, and Henry, MN. Virtual simulation in nursing education: a systematic review spanning 1996–2018. Simul Healthc . (2020) 15:46–54. doi: 10.1097/SIH.0000000000000411

3. Foronda, CL . What is virtual simulation? Clin Simul Nurs . (2021) 52:8. doi: 10.1016/j.ecns.2020.12.004

PubMed Abstract | Crossref Full Text | Google Scholar

4. Shin, H, Rim, D, Kim, H, Park, S, and Shon, S. Educational characteristics of virtual simulation in nursing: an integrative review. Clin Simul Nurs . (2019) 37:18–28. doi: 10.1016/j.ecns.2019.08.002

5. Foronda, C, Gattamorta, K, Snowden, K, and Bauman, EB. Use of virtual clinical simulation to improve communication skills of baccalaureate nursing students: a pilot study. Nurse Educ Today . (2014) 34:e53–7. doi: 10.1016/j.nedt.2013.10.007

6. Irwin, P, and Coutts, R. A systematic review of the experience of using second life in the education of undergraduate nurses. J Nurs Educ . (2015) 54:572–7. doi: 10.3928/01484834-20150916-05

7. Kim, YJ . Development and application of scenario-based Admission Management VR contents for nursing students. J Korea Soc Comput Inf . (2021) 26:209–16. doi: 10.9708/jksci.2021.26.01.209

8. Butt, AL, Kardong-Edgren, SK, and Ellertson, A. Using game-based virtual reality with haptics for skill acquisition. Clin Simul Nurs . (2018) 16:25–32. doi: 10.1016/j.ecns.2017.09.010

9. Jung, A, Kwon, E, and Seo, J. Effects of nursing skills simulation program using virtual reality (VR) on learning flow, nursing skills confidence, nursing skills performance and usability verification. J Korea Acad-Ind Coop Soc . (2022) 23:127–35. doi: 10.5762/KAIS.2022.23.11.127

10. Kim, JW . Virtual reality (VR) based sustainable food education contents for elementary school students. Korean Assoc Pract Arts Edu . (2019) 32:45–63. doi: 10.24062/kpae.2019.32.4.45

11. Ahn, MK . The development and effects of metaverse-based core nursing skill contents of vital signs measurements and subcutaneous injections for nursing students. J Korean Acad Soc Nurs Educ . (2022) 28:378–88. doi: 10.5977/jkasne.2022.28.4.378

12. Lee, JS . Implementation and evaluation of a virtual reality simulation intravenous injection training system. Int J Environ Res Public Health . (2022) 19:5439. doi: 10.3390/ijerph19095439

13. Ahn, MK, and Lee, CM. Development and effects of head-mounted display-based home-visits virtual reality simulation program for nursing students. Korean Soc Nurs Sci . (2021) 51:465–77. doi: 10.4040/jkan.21051

14. Rossler, KL, Sankaranarayanan, G, and Duvall, A. Acquisition of fire safety knowledge and skills with virtual reality simulation. Nurse Educ . (2019) 44:88–92. doi: 10.1097/NNE.0000000000000551

15. Aebersold, M, Voepel-Lewis, T, Cherara, L, Weber, M, Khouri, C, Levine, MD, et al. Interactive anatomy, augmented virtual simulation training. Clin Simul Nurs . (2018) 15:34–41. doi: 10.1016/j.ecns.2017.09.008

16. An, J, Oh, J, and Park, K. Self-regulated learning strategies for nursing students: a pilot randomized controlled trial. Int J Environ Res Public Health . (2022) 19:9058. doi: 10.3390/ijerph19159058

17. Chang, CY, Panjaburee, P, and Chang, SC. Effects of integrating maternity VR based situated learning into professional training on students’ learning performances. Interact Learn Environ . (2022) 2022:1–15. doi: 10.1080/10494820.2022.2141263

18. Kim, MS, and Jeong, HC. The effects and adaptation of augmented reality–based intradermal injection practice education for nursing students. J Korean Soc Simul Nurs . (2022) 10:93–104. doi: 10.17333/JKSSN.2022.10.2.93

19. Ha, YO, Kwon, SJ, Kim, J, and Song, JH. Effects of nursing skills practice using VR (virtual reality) on competency and confidence in nursing skills, learning self-efficacy, and satisfaction of nursing students. J Ind Converg . (2022) 20:47–55. doi: 10.22678/JIC.2022.20.4.047

20. You, H, and Yang, B. The effects of virtual reality simulation scenario application on clinical competency, problem solving ability and nursing performance confidence. J Korea Acad Ind Coop Soc. (2021) 22:116–26. doi: 10.5762/KAIS.2021.22.9.116

21. Bae, YS, and Shin, KM. Effects of virtual reality simulation of core fundamental nursing skills for intravenous fluid infusion on nursing students. Korean J Care Manag . (2023) 46:95–119. doi: 10.22589/kaocm.2023.46.95

22. Song, YM . Online and blended learning application in psychiatric and mental health nursing practice program among nursing students. J Learn Cent Curric Instr . (2021) 21:289–303. doi: 10.22251/jlcci.2021.21.11.289

23. Raman, S, Labrague, LJ, Arulappan, J, Natarajan, J, Amirtharaj, A, and Jacob, D. Traditional clinical training combined with high fidelity simulation based activities improves clinical competency and knowledge among nursing students on a maternity nursing course. Nurs Forum . (2019) 54:434–40. doi: 10.1111/nuf.12351

24. Cho, IY, Yun, JY, and Moon, SH. Development and effectiveness of a metaverse reality-based family-centered handoff education program in nursing students. J Pediatr Nurs . (2024) 76:176–91. doi: 10.1016/j.pedn.2024.02.005

25. Lee, E, and Baek, G. Development and effects of a virtual reality simulation nursing education program combined with clinical practice based on an information processing model. Comput Inform Nurs . (2023) 41:1016–25. doi: 10.1097/CIN.0000000000001051

26. Kim, J, and Heo, N. Effect of augmented reality smart glasses-based nursing skills training for nursing students’ medication administration safety competency: a quasi-experimental study. J Korean Acad Fundam Nurs. (2023) 30:449–58. doi: 10.7739/jkafn.2023.30.4.449

27. Park, S, and Yoon, HG. Effect of virtual-reality simulation of indwelling catheterization on nursing students’ skills, confidence, and satisfaction. Clin Simul Nurs . (2023) 80:46–54. doi: 10.1016/j.ecns.2023.05.001

28. Karaduman, GS, and Basak, T. Is virtual patient simulation superior to human patient simulation: a randomized controlled study. CIN Comput Inform Nu . (2023) 41:467–76. doi: 10.1097/CIN.0000000000000957

29. Moon, SH . Metaverse based emergency nursing educational program using V-story. Crisis . (2023) 19:79–89.

Google Scholar

30. Lee, JJ . The effect of virtual reality simulation training on critical thinking disposition, clinical competency, and self-efficacy of nursing students. J Korea Acad Ind Coop Soc . (2023) 24:390–7. doi: 10.5762/KAIS.2023.24.12.390

31. Ahn, MK . Development and effects of metaverse-based CPR training. J Digit Contents Soc . (2023) 24:1347–52. doi: 10.9728/dcs.2023.24.6.1347

32. Tufanaru, C, Munn, Z, Aromataris, E, Campbell, J, and Hopp, L. Chapter 3. Systematic reviews of effectiveness In: E Aromataris and Z Munn, editors. JBI manual for evidence synthesis (2020). JBI; 2024.

33. Borenstein, M, Hedges, LV, Higgins, JPT, and Rothstein, HR. Introduction to Meta-analysis . West Sussex, UK: John Wiley & Sons (2009).

34. Higgins, JPT, and Thompson, SG. Quantifying heterogeneity in a meta-analysis. Statist Med . (2002) 21:1539–58. doi: 10.1002/sim.1186

35. Higgins, JPT , Green SE (2011). Available at: http://www.cochrane-handbook.org (Accessed September 13, 2023).

36. Mavridis, D, and Salanti, G. How to assess publication bias: funnel plot, trim-and-fill method and selection models. Evid Based Ment Health . (2014) 17:30. doi: 10.1136/eb-2013-101699

37. Brydges, CR . Effect size guidelines, sample size calculations, and statistical power in gerontology. Innov. Aging . (2019) 3:igz036. doi: 10.1093/geroni/igz036

38. Bown, MJ, and Sutton, AJ. Quality control in systematic reviews and meta-analyses. Eur J Vasc Endovasc Surg . (2010) 40:669–77. doi: 10.1016/j.ejvs.2010.07.011

39. Kim, MG, and Kim, HW. The effects of classes using virtual reality simulations of the hospital environment on knowledge of the hospital environment, academic self-efficacy, learning flow, educational satisfaction and academic achievement in nursing students. J Korean Acad Fundam Nurs . (2021) 28:520–9. doi: 10.7739/jkafn.2021.28.4.520

40. Lim, JH . The effect of virtual reality simulation education on nursing process competency. J Digit Converg . (2021) 19:401–9. doi: 10.14400/JDC.2021.19.9.401

41. Kim, S, and Kim, MJ. Effect of learner-centered virtual reality simulation education. J Digit Converg . (2022) 20:705–13. doi: 10.14400/JDC.2022.20.4.705

42. Yang, SJ, and Chae, MJ. Effect of nursing students’ practical training on nursing simulation for blood transfusion recipients using online virtual reality. J Digit Contents Soc . (2024) 25:143–51. doi: 10.9728/dcs.2024.25.1.143

43. Fredericks, S, ElSayed, M, Hammad, M, Abumiddian, O, Istwani, L, Rabeea, A, et al. Anxiety is associated with extraneous cognitive load during teaching using high-fidelity clinical simulation. Medical education. Online . (2021) 26:1994691. doi: 10.1080/10872981.2021.1994691

44. Brennan, BA . The impact of self-efficacy based prebriefing on nursing student clinical competency and self-efficacy in simulation: an experimental study. Nurse Educ Today . (2022) 109:105260. doi: 10.1016/j.nedt.2021.105260

45. Dodson, TM, and Reed, JM. Enhancing simulation preparation: Presimulation role rubrics and expert Modeling videos. Clin Simul Nurs . (2024) 87:101498. doi: 10.1016/j.ecns.2023.101498

46. Lim, S, and Yeom, YR. The effect of education integrating virtual reality simulation training and outside school clinical practice for nursing students. J Converg Inf Technol . (2020) 10:100–8.

47. Loomis, A, Dreifuerst, KT, and Bradley, CS. Acquire, apply, and retain knowledge through debriefing for meaningful learning. Clin Simul Nurs . (2022) 68:28–33. doi: 10.1016/j.ecns.2022.04.002

48. Chen, FQ, Leng, YF, Ge, JF, Wang, DW, Li, C, Chen, B, et al. Effectiveness of virtual reality in nursing education: a meta-analysis. J Med Internet Res . (2020) 22:e18290. doi: 10.2196/18290

49. Padilha, JM, Machado, PP, Ribeiro, A, Ramos, J, and Costa, P. Clinical virtual simulation in nursing education: randomized controlled trial. J Med Internet Res . (2019) 21:e11529. doi: 10.2196/11529

50. Kim, HW, and Suh, EY. Nursing students’ immersion experience in a comprehensive simulation scenario using high-fidelity human patient simulator among nursing students: a phenomenological study. J Mil Nurs Res . (2012) 30:89–99.

51. Shorey, S, and Ng, ED. Use of virtual reality simulation among nursing students and registered nurses: a systematic review. Nurse Educ Today . (2021) 98:104662. doi: 10.1016/j.nedt.2020.104662

52. Hwang, YJ, Jeong, JY, and Jeong, YM. A study on the feasibility of introducing XR in nursing education Core fundamental nursing skills. J Digit Contents Soc . (2023) 24:775–83. doi: 10.9728/dcs.2023.24.4.775

Keywords: virtual reality, simulation, nursing students, competency, meta-analysis

Citation: Cho M-K and Kim MY (2024) Enhancing nursing competency through virtual reality simulation among nursing students: a systematic review and meta-analysis. Front. Med . 11:1351300. doi: 10.3389/fmed.2024.1351300

Received: 20 January 2024; Accepted: 17 April 2024; Published: 07 May 2024.

Reviewed by:

Copyright © 2024 Kim and Cho. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Mi Young Kim, [email protected]

This article is part of the Research Topic

Future Prospects of Learning in the Clinical Environment: Exploring the Technological Revolution

Impact of immersive realities and metaverse in the university training for health professions and medicine: a systematic review of preliminary evidence

• Review Article
• Published: 29 April 2024

Cite this article

• S Gholamalishahi 1 ,
• V. I. Barletta 1 ,
• M. V. Manai 1 ,
• F. Battisti 2 ,
• L. Centonze 3 ,
• G. La Torre 1 &
• A. Mannocci 4 , 5  

81 Accesses

Explore all metrics

Extended reality (XR) provides technologies that can create engaging and interactive learning resources easily updatable in line with the learning needs. This systematic review aimed to investigate the academic performance (AP), level satisfaction (LS), and acceptability (Ac) of XRs in university education students of medicine and health sciences.

The Preferred Reporting Items for Systematic Reviews statement was applied.

Nineteen experimental studies were included, which examined AP, 15 LS and 4 AC. No study was found that applies the Metaverse, 15 considered virtual reality (VR), 3 Augmented Reality (AR), 1 Mixed Reality (MR). Ten studies showed a statistically significant increase in AP in students who adopted XRs compared to traditional teaching methodologies. Six 5 studies showed a significant increase in LS after VR use. Two studies reported a high percentage of significant acceptability of XR use.

The review provides that Metaverse results still need to be implemented; VR was a valid tool to support the improvement of AP and LS of students; VR turns out to be more widespread than other XRs. There are several discrepancies in the studies that confirm the need to promote the understanding of VR simulation’s effects on the acquisition of skills.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design

Gamification and Game Based Learning for Vocational Education and Training: A Systematic Literature Review

The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality

Data availability.

The data used to support the findings of this study are included within the article.

Aebersold M, Voepel-Lewis T, Cherara L et al (2018) Interactive anatomy-augmented virtual simulation training. Clin Simul Nurs 15:34–41. https://doi.org/10.1016/j.ecns.2017.09.008

Article   PubMed   PubMed Central   Google Scholar  

Alawadhi M, Alhumaid K, Almarzooqi S et al (2022) Factors affecting medical students’ acceptance of the metaverse system in medical training in the United Arab Emirates. South Eastern European J Public Health 19

Almarzouqi A, Aburayya A, Salloum SA (2022) Prediction of user’s intention to use metaverse system in medical education: a hybrid SEM-ML learning approach. IEEE Access 10:43421–43434

Article   Google Scholar  

Altman DG (1996) Better reporting of randomised controlled trials: the consort statement. BMJ 313:570–571

Article   CAS   PubMed   PubMed Central   Google Scholar  

Asad MM, Naz A, Churi P, Tahanzadeh MM (2021) Virtual reality as pedagogical tool to enhance experiential learning: a systematic literature review. Educ Res Int 2021. https://doi.org/10.1155/2021/7061623

Bandura A (1977) Social learning theory. Prentice Hall, Englewood Cliffs

Google Scholar  

Behmadi S, Asadi F, Okhovati M, Sarabi RE (2022) Virtual reality-based medical education versus lecture-based method in teaching start triage lessons in emergency medical students: virtual reality in medical education. J Adv Med Educ Prof 10:48–53. https://doi.org/10.30476/jamp.2021.89269.1370

Beulens AJW, Hashish YAF, Brinkman WM et al (2021) Training novice robot surgeons: proctoring provides same results as simulator-generated guidance. J Robot Surg 15:397–428. https://doi.org/10.1007/s11701-020-01118-y

Article   CAS   PubMed   Google Scholar  

Billinghurst M, Clark A, Lee G (2014) A survey of augmented reality. Found Trends Human-Comput Interact 8:73–272. https://doi.org/10.1561/1100000049

Bimber O, Raskar R (2005) A brief introduction to augmented reality. In: Spatial augmented reality. A K Peters/CRC Press

Birrenbach T, Zbinden J, Papagiannakis G et al (2021) Effectiveness and utility of virtual reality simulation as an educational tool for safe performance of COVID-19 diagnostics: prospective, randomized pilot trial. JMIR Serious Games 9: https://doi.org/10.2196/29586

Bordage G (2009) Conceptual frameworks to illuminate and magnify. Med Educ 43:312–319. https://doi.org/10.1111/j.1365-2923.2009.03295.x

Article   PubMed   Google Scholar  

Bridge P, Gunn T, Kastanis L et al (2014) The development and evaluation of a medical imaging training immersive environment. J Med Rad Sci 61:159–165. https://doi.org/10.1002/jmrs.60

Chang C-Y, Sung H-Y, Guo J-L et al (2022) Effects of spherical video-based virtual reality on nursing students’ learning performance in childbirth education training. Interact Learn Environ 30:400–416. https://doi.org/10.1080/10494820.2019.1661854

Article   CAS   Google Scholar  

Chang S-C, Tung F-C (2007) An empirical investigation of students’ behavioural intentions to use the online learning course websites. Br J Educ Technol 0:070625111823003-??? https://doi.org/10.1111/j.1467-8535.2007.00742.x

Chao Y-P, Kang C-J, Chuang H-H et al (n.d.) Comparison of the effect of 360 degrees versus two-dimensional virtual reality video on history taking and physical examination skills learning among undergraduate medical students: a randomized controlled trial. Virtual Reality. https://doi.org/10.1007/s10055-022-00664-0

Chen F-Q, Leng Y-F, Ge J-F et al (2020) Effectiveness of virtual reality in nursing education: meta-analysis. J Med Internet Res 22:e18290. https://doi.org/10.2196/18290

Copson B, Wijewickrema S, Sorace L et al (2021) Development of a virtual reality clinically oriented temporal bone anatomy module with randomised control study of three-dimensional display technology. BMJ Simul Technol Enhanced Learn 7:352–359. https://doi.org/10.1136/bmjstel-2020-000592

Dhar E, Bah AN, Chicchi Giglioli IA et al (2023) A scoping review and a taxonomy to assess the impact of mobile apps on cancer care management. Cancers (basel) 15:1775. https://doi.org/10.3390/cancers15061775

Ekstrand C, Jamal A, Nguyen R et al (2018) Immersive and interactive virtual reality to improve learning and retention of neuroanatomy in medical students: a randomized controlled study. CMAJ Open 6:E103–E109. https://doi.org/10.9778/cmajo.20170110

Engum S, Jeffries P, Fisher L (2003) Intravenous catheter training system: computer-based education versus traditional learning methods. Am J Surg 186:67–74. https://doi.org/10.1016/S0002-9610(03)00109-0

Falah J, Khan S, Alfalah T et al (2014) Virtual reality medical training system for anatomy education. 2014 Science and Information Conference. IEEE, London, pp 752–758

Chapter   Google Scholar  

Gnanasegaram JJ, Leung R, Beyea JA (2020) Evaluating the effectiveness of learning ear anatomy using holographic models. J Otolaryngol-Head Neck Surg 49. https://doi.org/10.1186/s40463-020-00458-x

Gray M, Downer T, Hartz D et al (2022) The impact of three-dimensional visualisation on midwifery student learning, compared with traditional education for teaching the third stage of labour: a pilot randomised controlled trial. Nurse Educ today 108. https://doi.org/10.1016/j.nedt.2021.105184

Halpern SM, Joanne H Douglas (2005) Appendix: jadad scale for reporting randomized controlled trials. In: Evidence-based obstetric anesthesia. John Wiley & Sons, Ltd, pp 237–238

Han SG, Kim YD, Kong TY, Cho J (2021) Virtual reality-based neurological examination teaching tool(VRNET) versus standardized patient in teaching neurological examinations for the medical students: a randomized, single-blind study. BMC Med Educ 21. https://doi.org/10.1186/s12909-021-02920-4

Jadad AR, Moore RA, Carroll D et al (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17:1–12. https://doi.org/10.1016/0197-2456(95)00134-4

Jallad ST, Isik B (2022) The effectiveness of virtual reality simulation as learning strategy in the acquisition of medical skills in nursing education: a systematic review. Ir J Med Sci 191:1407–1426. https://doi.org/10.1007/s11845-021-02695-z

Kim H-Y, Kim E-Y (2023) Effects of medical education program using virtual reality: a systematic review and meta-analysis. Int J Environ Res Public Health 20:3895. https://doi.org/10.3390/ijerph20053895

Kucuk S, Kapakin S, Goktas Y (2016) Learning anatomy via mobile augmented reality: effects on achievement and cognitive load. Anat Sci Educ 9:411–421. https://doi.org/10.1002/ase.1603

Lebdai S, Mauget M, Cousseau P et al (2021) Improving academic performance in medical students using immersive virtual patient simulation: a randomized controlled trial. J Surg Educ 78:478–484. https://doi.org/10.1016/j.jsurg.2020.08.031

Liberati A, Altman DG, Tetzlaff J et al (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6:e1000100. https://doi.org/10.1371/journal.pmed.1000100

Lorenzo-Alvarez R, Rudolphi-Solero T, Ruiz-Gomez MJ, Sendra-Portero F (2019) Medical student education for abdominal radiographs in a 3D virtual classroom versus traditional classroom: a randomized controlled trial. Am J Roentgenol 213:644–650. https://doi.org/10.2214/AJR.19.21131

Losco CD, Grant WD, Armson A et al (2017) Effective methods of teaching and learning in anatomy as a basic science: A BEME systematic review: BEME guide no. 44. Med Teach 39:234–243. https://doi.org/10.1080/0142159X.2016.1271944

Mahtab EAF, Egorova AD (2022) Current and future applications of virtual reality technology for cardiac interventions. Nat Rev Cardiol 19:779–780. https://doi.org/10.1038/s41569-022-00789-4

Maresky HS, Oikonomou A, Ali I et al (2019) Virtual reality and cardiac anatomy: exploring immersive three-dimensional cardiac imaging, a pilot study in undergraduate medical anatomy education. Clin Anat 32:238–243. https://doi.org/10.1002/ca.23292

Mystakidis S (2022) Metaverse Encyclop 2:486–497

Park BJ, Hunt SJ, Martin C et al (2020) Augmented and mixed reality: technologies for enhancing the future of IR. J Vasc Interv Radiol 31:1074–1082. https://doi.org/10.1016/j.jvir.2019.09.020

Pickering JD, Panagiotis A, Ntakakis G et al (2022) Assessing the difference in learning gain between a mixed reality application and drawing screencasts in neuroanatomy. Anat Sci Educ 15:628–635. https://doi.org/10.1002/ase.2113

Ryan GV, Callaghan S, Rafferty A et al (2022) Learning outcomes of immersive technologies in health care student education: systematic review of the literature. J Med Internet Res 24:e30082. https://doi.org/10.2196/30082

Smith HP (2021) The role of virtual reality in criminal justice pedagogy: an examination of mental illness occurring in corrections. J Crim Justice Educ 32:252–271. https://doi.org/10.1080/10511253.2021.1901948

Smith S, Farra SL, Hodgson E (2021) Evaluation of two simulation methods for teaching a disaster skill. BMJ Simul Technol Enhanced Learn 7:92–96. https://doi.org/10.1136/bmjstel-2019-000572

Tudor Car L, Soong A, Kyaw BM et al (2019) Health professions digital education on clinical practice guidelines: a systematic review by digital health education collaboration. BMC Med 17:139. https://doi.org/10.1186/s12916-019-1370-1

Ulrich F, Helms NH, Frandsen UP, Rafn AV (2021) Learning effectiveness of 360 degrees video: experiences from a controlled experiment in healthcare education. Interact Learn Environ 29:98–111. https://doi.org/10.1080/10494820.2019.1579234

Wang Y, Su Z, Zhang N et al (2022) A survey on metaverse: fundamentals, security, and privacy. IEEE Communications Surveys & Tutorials

Wu M-L, Chao L-F, Xiao X (2022) A pediatric seizure management virtual reality simulator for nursing students: a quasi-experimental design. Nurse Educ Today 119: https://doi.org/10.1016/j.nedt.2022.105550

Wu T-C, Ho C-TB (2022) A scoping review of metaverse in emergency medicine. Australasian Emerg Care. https://doi.org/10.1016/j.auec.2022.08.002

Yu Z, Xu W (2022) A meta-analysis and systematic review of the effect of virtual reality technology on users’ learning outcomes. Comput Appl Eng Educ 30:1470–1484. https://doi.org/10.1002/cae.22532

Download references

Author information

Authors and affiliations.

Unit of Respiratory Medicine, Department of Experimental Medicine and Surgery, “Tor Vergata” University, Rome, Italy

S Gholamalishahi, V. I. Barletta, M. V. Manai & G. La Torre

Department of Information Engineering, University of Padua, Padua, Italy

F. Battisti

Faculty of Society and Communication, Universitas Mercatorum, Rome, Italy

L. Centonze

Department for the Promotion of Human Sciences and Quality of Life, Telematic University San Raffaele, di Val Cannuta Street 247, 00166, Rome, Italy

A. Mannocci

Faculty of Economics and Law, Universitas Mercatorum, Rome, Italy

You can also search for this author in PubMed   Google Scholar

Contributions

Shima Gholamalishahi: performed literature search, article selection, analysis, and manuscript editing;Vanessa India Barletta: performed literature search, article selection, analysis; Maria Vittoria Manai: performed literature search, article selection, analysis ; Federica Battisti: provided data clarifications and revision; Laura Centonze: article selection, construct of the themes, manuscript review;Giuseppe La Torre: concepts, design, the definition of intellectual content and revised the manuscript; Alice Mannocci: conceived of the study, supervised the project, assisted with analysis, manuscript writing and revised the manuscript. All authors discussed the results and contributed to the final manuscript.

Corresponding author

Correspondence to A. Mannocci .

Ethics declarations

Ethics approval.

This work is a systematic review and thus no ethical approval is required.

Competing interests

The authors have no financial or non-financial interests to disclose.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 31 KB)

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Gholamalishahi, S., Barletta, V.I., Manai, M.V. et al. Impact of immersive realities and metaverse in the university training for health professions and medicine: a systematic review of preliminary evidence. J Public Health (Berl.) (2024). https://doi.org/10.1007/s10389-024-02268-1

Download citation

Received : 22 November 2023

Accepted : 05 April 2024

Published : 29 April 2024

DOI : https://doi.org/10.1007/s10389-024-02268-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Extended reality
• Virtual reality
• University teaching
• Health professions students
• Find a journal
• Publish with us
• Track your research

The Scope of Virtual Reality Simulators in Radiology Education: Systematic Literature Review

Affiliations.

• 1 Department of Oral and Craniofacial Health Sciences, College of Dental Medicine, University of Sharjah, Sharjah, United Arab Emirates.
• 2 Department of Oral Medicine and Radiology, AB Shetty Memorial Institute of Dental Sciences, Nitte (Deemed to be University), Mangalore, India.
• 3 Department of Preventive and Restorative Dentistry, College of Dental Medicine, University of Sharjah, Sharjah, United Arab Emirates.
• 4 Department of Oral Medicine and Radiology, Manipal College of Dental Sciences, Manipal Academy of Higher Education, Manipal, India.
• 5 Department of Clinical Sciences, College of Dentistry, Ajman University, Ajman, United Arab Emirates.
• PMID: 38722205
• DOI: 10.2196/52953

Background: In recent years, virtual reality (VR) has gained significant importance in medical education. Radiology education also has seen the induction of VR technology. However, there is no comprehensive review in this specific area. This review aims to fill this knowledge gap.

Objective: This systematic literature review aims to explore the scope of VR use in radiology education.

Methods: A literature search was carried out using PubMed, Scopus, ScienceDirect, and Google Scholar for articles relating to the use of VR in radiology education, published from database inception to September 1, 2023. The identified articles were then subjected to a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)-defined study selection process.

Results: The database search identified 2503 nonduplicate articles. After PRISMA screening, 17 were included in the review for analysis, of which 3 (18%) were randomized controlled trials, 7 (41%) were randomized experimental trials, and 7 (41%) were cross-sectional studies. Of the 10 randomized trials, 3 (30%) had a low risk of bias, 5 (50%) showed some concerns, and 2 (20%) had a high risk of bias. Among the 7 cross-sectional studies, 2 (29%) scored "good" in the overall quality and the remaining 5 (71%) scored "fair." VR was found to be significantly more effective than traditional methods of teaching in improving the radiographic and radiologic skills of students. The use of VR systems was found to improve the students' skills in overall proficiency, patient positioning, equipment knowledge, equipment handling, and radiographic techniques. Student feedback was also reported in the included studies. The students generally provided positive feedback about the utility, ease of use, and satisfaction of VR systems, as well as their perceived positive impact on skill and knowledge acquisition.

Conclusions: The evidence from this review shows that the use of VR had significant benefit for students in various aspects of radiology education. However, the variable nature of the studies included in the review reduces the scope for a comprehensive recommendation of VR use in radiology education.

Keywords: PRISMA; Preferred Reporting Items for Systematic Reviews and Meta-Analyses; VR; education; imaging; literature review; medical imaging; meta analysis; radiology; radiology education; simulators; student; students; systematic review; virtual reality.

© Shishir Shetty, Supriya Bhat, Saad Al Bayatti, Sausan Al Kawas, Wael Talaat, Mohamed El-Kishawi, Natheer Al Rawi, Sangeetha Narasimhan, Hiba Al-Daghestani, Medhini Madi, Raghavendra Shetty. Originally published in JMIR Medical Education (https://mededu.jmir.org).

Publication types

• Systematic Review
• Radiology* / education
• Simulation Training / methods
• Virtual Reality*

This paper is in the following e-collection/theme issue:

Published on 8.5.2024 in Vol 10 (2024)

The Scope of Virtual Reality Simulators in Radiology Education: Systematic Literature Review

Authors of this article:

• Shishir Shetty 1 , PhD ; 
• Supriya Bhat 2 , MDS ; 
• Saad Al Bayatti 1 , MSc ; 
• Sausan Al Kawas 1 , PhD ; 
• Wael Talaat 1 , PhD ; 
• Mohamed El-Kishawi 3 , PhD ; 
• Natheer Al Rawi 1 , PhD ; 
• Sangeetha Narasimhan 1 , PhD ; 
• Hiba Al-Daghestani 1 , MSc ; 
• Medhini Madi 4 , MDS ; 
• Raghavendra Shetty 5 , PhD

1 Department of Oral and Craniofacial Health Sciences, College of Dental Medicine, University of Sharjah, , Sharjah, , United Arab Emirates

2 Department of Oral Medicine and Radiology, AB Shetty Memorial Institute of Dental Sciences, Nitte (Deemed to be University), , Mangalore, , India

3 Department of Preventive and Restorative Dentistry, College of Dental Medicine, University of Sharjah, , Sharjah, , United Arab Emirates

4 Department of Oral Medicine and Radiology, Manipal College of Dental Sciences, Manipal Academy of Higher Education, , Manipal, , India

5 Department of Clinical Sciences, College of Dentistry, Ajman University, , Ajman, , United Arab Emirates

Corresponding Author:

Supriya Bhat, MDS

Background: In recent years, virtual reality (VR) has gained significant importance in medical education. Radiology education also has seen the induction of VR technology. However, there is no comprehensive review in this specific area. This review aims to fill this knowledge gap.

Objective: This systematic literature review aims to explore the scope of VR use in radiology education.

Methods: A literature search was carried out using PubMed, Scopus, ScienceDirect, and Google Scholar for articles relating to the use of VR in radiology education, published from database inception to September 1, 2023. The identified articles were then subjected to a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)–defined study selection process.

Results: The database search identified 2503 nonduplicate articles. After PRISMA screening, 17 were included in the review for analysis, of which 3 (18%) were randomized controlled trials, 7 (41%) were randomized experimental trials, and 7 (41%) were cross-sectional studies. Of the 10 randomized trials, 3 (30%) had a low risk of bias, 5 (50%) showed some concerns, and 2 (20%) had a high risk of bias. Among the 7 cross-sectional studies, 2 (29%) scored “good” in the overall quality and the remaining 5 (71%) scored “fair.” VR was found to be significantly more effective than traditional methods of teaching in improving the radiographic and radiologic skills of students. The use of VR systems was found to improve the students’ skills in overall proficiency, patient positioning, equipment knowledge, equipment handling, and radiographic techniques. Student feedback was also reported in the included studies. The students generally provided positive feedback about the utility, ease of use, and satisfaction of VR systems, as well as their perceived positive impact on skill and knowledge acquisition.

Conclusions: The evidence from this review shows that the use of VR had significant benefit for students in various aspects of radiology education. However, the variable nature of the studies included in the review reduces the scope for a comprehensive recommendation of VR use in radiology education.

Introduction

The use of technology in education helps students achieve improved acquisition of professional knowledge and practical skills [ 1 - 3 ]. Virtual reality (VR) is a modern technology that simulates experience by producing 3D interactive situations and presenting objects in a virtual world with spatial dimensions [ 4 , 5 ]. VR technology can be classified as nonimmersive or immersive [ 6 ]. In a nonimmersive VR, the simulated 3D environment is experienced through a computer monitor [ 6 ]. On the other hand, an immersive VR provides a sense of presence in a computer-generated environment, created by producing realistic sights, sounds, and other sensations that replicate a user’s physical presence in a virtual environment [ 6 , 7 ]. Using VR technology, a person can look about the artificial world, navigate around in it, and interact with simulated objects or items [ 5 , 8 ]. Due to the broad nature of VR technology, it has many applications, some of which are in the field of medicine [ 9 , 10 ].

The use of VR in medicine started in the 1990s when medical researchers were trying to create 3D models of patients’ internal organs [ 11 - 13 ]. Since then, VR use in the field of medicine and general health care has increased substantially to cover many areas including medical education. Radiology education has also come to see the use of VR technology in the recent past [ 14 ]. The use of VR in radiology education enables students to practice radiography in a virtual environment, which is radiation free [ 15 ]. Additionally, the use of VR enables effective and repeatable training. This allows trainees to recognize and correct errors as they occur [ 16 , 17 ]. The aim of this review is to explore the scope of VR in radiology education.

This systematic review has been performed using the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analysis) guidelines [ 18 ] [ Checklist 1 ]).

Information Sources and Study Selection

The bibliographic databases used were PubMed, Scopus, ScienceDirect, and Google Scholar. A systematic literature search was conducted for articles published from database inception to September 1, 2023. Topic keywords were used to generate search strings. The search strings that were used are provided in Table 1 . Only the first 10 pages of Google Scholar results were exported. The identified studies were then subjected to a study selection process. The search string for ScienceDirect was shorter because the database only allows a maximum of 8 Boolean operators, hence the sting had to be shortened. The search in PubMed was limited to the title and abstract. The searches in Scopus and ScienceDirect were limited to title, abstract, and keywords.

Inclusion and Exclusion Criteria

Original research articles written in the English language were included in the review. Studies conducted on medical, dental, and allied health sciences students (undergraduate and postgraduate) from any part of the world were included in the review. Studies exploring the use of VR learning in radiology education were included.

Narrative reviews, scoping reviews, systematic reviews, meta-analyses, editorials, and commentaries were excluded. Studies that did not align with the required study objective were excluded.

Method of Quality Assessment

Randomized controlled trials (RCTs) and randomized experimental studies were appraised using the RoB 2 tool from the Cochrane Collaboration [ 19 ]. A visualization of the risk-of-bias assessment was done using the web-based robvis tool [ 20 ]. Cross-sectional studies were appraised using the appraisal checklist for analytical cross-sectional studies from the Joanna Briggs Institute [ 21 ].

Data Extraction

Each article included in the review was summarized in a table, including basic study characteristics. The extracted attributes were study author(s), publication year, study design, type and number of participants, type of radiology education under study, and the outcome being assessed. The extracted data are provided in Table 2 .

a RCT: randomized controlled trial.

b CT: computed tomography.

Search Results

The database search identified a total of 2877 studies; 374 (13%) studies were from PubMed, 2169 (75.4%) were from Scopus, 234 (8.1%) were from ScienceDirect, and 100 (3.5%) were from Google Scholar. Before the screening procedure, 37 duplicates were removed. During title and abstract screening, 2808 articles were excluded since they did not align with the eligibility criteria. The remaining 32 articles were then subjected to a full-text review, and 15 were excluded for reasons provided in Figure 1 , which shows the study selection process [ 38 ]. At the end of the process, 17 studies were found eligible for inclusion in the review.

Characteristics of Included Studies

Among the 17 studies, 3 (18%) RCTs, 7 (41%) randomized experimental trials, and 7 (41%) cross-sectional studies were included. The studies encompassed various aspects of radiology education, including dental radiology [ 28 , 29 ], diagnostic radiology [ 22 , 24 ], and interventional radiology [ 25 , 31 ].

Results of Quality Assessment

Among the 7 cross-sectional studies, 2 (29%) scored “good” in overall quality and the remaining 5 (71%) scored “fair.” The results for the quality appraisal of cross-sectional studies are shown in Table 3 . Studies were appraised using the checklist for analytical cross-sectional studies from the Joanna Briggs Institute [ 21 ].

Among the 10 randomized trials, 3 (30%) had a low risk of bias, 5 (50%) showed some concerns, and 2 (20%) had a high risk of bias. These results are shown in Table 4 . RCTs were appraised using the RoB 2 tool from the Cochrane Collaboration [ 19 ]. A risk-of-bias graph ( Figure 2 ) and a risk-of-bias summary ( Figure 3 ) are also provided.

a Item 1: were the criteria for inclusion in the sample clearly defined?

b Item 2: were the study subjects and the setting described in detail?

c Item 3: was the exposure measured in a valid and reliable way?

d Item 4: were objective, standard criteria used for measurement of the condition?

e Item 5: were confounding factors identified?

f Item 6: were strategies to deal with confounding factors stated?

g Item 7: were the outcomes measured in a valid and reliable way?

h Item 8: was appropriate statistical analysis used?

i N/A: not assessable.

a D1: risk of bias arising from the randomization process.

b D2: risk of bias due to deviations from the intended interventions (effect of assignment to intervention).

c D3: risk of bias due to missing outcome data.

d D4: risk of bias in measurement of the outcome.

e D5: risk of bias in selection of the reported result.

Type of VR Hardware and Software Used in the Studies

The studies used a wide range of VR software and hardware. Some of the studies used 3D simulation software packages displayed on 2D desktop computers [ 22 , 24 , 25 , 36 ], whereas others used headsets for an immersive VR environment [ 15 , 23 , 26 , 35 , 37 ]. The most used VR teaching software were the CETSOL VR Clinic software [ 33 , 35 ], Virtual Medical Coaching VR software [ 15 , 30 , 32 ], Projection VR (Shaderware) software [ 36 ], SieVRt VR system (Luxsonic Technologies) [ 37 ], medical imaging training immersive environment software [ 23 ], VR CT Sim software [ 25 ], VitaSim ApS software [ 26 ], VR X-Ray (Skilitics and Virtual Medical Coaching) software [ 27 ], and radiation dosimetry VR software (Virtual Medical Coaching Ltd) [ 31 ].

Effect of VR Teaching on Skill Acquisition

Ahlqvist et al [ 22 ] looked at how virtual simulation can be used as an effective tool to teach quality assessment of radiographic images. They also compared how it faired in comparison to traditional teaching. The study reported a statistically significant improvement in proficiency from before training to after training. Additionally, the study reported that the proficiency score improvement for the VR-trained students was higher than that for the students trained using conventional method.

In the study conducted by Sapkaroski et al [ 34 ], students in the VR group demonstrated significantly better patient positioning skills compared to those in the conventional role-play group. The positioning parameters that were assessed were digit separation and palm flatness (the VR group scored 11% better), central ray positioning onto the third metacarpophalangeal joint (the VR group scored 23% better), and a control position projection of an oblique hand. The results for the control position projection indicated no significant difference in positioning between the 2 groups [ 34 ].

Bridge et al [ 23 ] also performed a performance comparison between students trained by VR and traditional methods. They assessed skills about patient positioning, equipment positioning, and time taken to complete a performative role-play. Students in the VR group performed better than those in the control group, with 91% of them receiving an overall score of above average (>3). The difference in mean group performance was statistically significant ( P =.0366). Similarly, Gunn et al [ 24 ] reported improved and higher role-play skill scores for students trained using VR software simulation compared to those trained on traditional laboratory simulation. The mean role-play score for the VR group was 30.67 and that for the control group was 28.8 [ 24 ].

Another study reported that students trained using VR performed significantly better (ranked as “very good” or “excellent”) than the control group (conventional learning) in skills such as patient positioning, selecting exposure factors, centering and collimating the x-ray beam, placing the anatomical marker, appraisal of image quality, equipment positioning, and procedure explanation to the patient [ 30 ]. Another recently conducted study found that the VR-taught group achieved better test duration and fewer errors in moving equipment and positioning a patient. There was no significant difference in the frequency of errors in the radiographic exposure setting such as source-to-image distance between the VR and the physical simulation groups [ 32 ].

Nilsson et al [ 28 ] developed a test to evaluate the student’s ability to interpret 3D information in radiographs using parallax. This test was applied to students before and after training. There was a significantly larger ( P <.01) pre-post intervention mean score for the VR group (3.11 to 4.18) compared to the control group (3.24 to 3.72). A subgroup analysis was also performed, and students with low visuospatial ability in the VR group had a significantly higher improvement in the proficiency test compared to those in the control group. The same authors conducted another follow-up study to test skill retention [ 29 ]. Net skill improvement was calculated as the difference in test scores after 8 months. The results from the proficiency test showed that the ability to interpret spatial relations in radiographs 8 months after the completion of VR training was significantly better than before VR training. The students who trained conventionally showed almost the same positive trend in improvement. The group difference was smaller and not statistically significant. This meant that, 8 months after training, the VR group and the traditionally trained group had the same skill level [ 29 ].

Among the included studies, only 1 reported that the VR group had lower performance in proficiency tests and radiographic skill tests, compared to a conventionally trained group. The study, conducted in 2022, showed that the proficiency of the VR group was significantly lower than that of the conventional technique group in performing lateral elbow and posterior-anterior chest radiography [ 27 ]. An itemized rubric evaluation used in the study revealed that the VR group also had lower performance in most of the radiographic skills, such as locating and centering of the x-ray beam, side marker placement, positioning the x-ray image detector, patient interaction, and process control and safety [ 27 ]. The study concluded that VR simulation can be less effective than real-world training in radiographic techniques, which requires palpation and patient interaction. These results may be different from those of other studies due to different outcome evaluation methods and since they used head-mounted display VR coaching, whereas the other studies, except O’Connor et al [ 15 ], used VR on a PC monitor.

All of the studies except Kato et al [ 27 ] agreed that VR use was more effective for students in developing radiographic and radiologic skills. Despite this general agreement, there were slight in-study variations in learning outcomes, which made some of the studies look at factors that may influence skill and knowledge acquisition during VR use. In studies such as Bridge et al [ 23 ], it was noted that the arrangement of equipment had the greatest influence on the overall score. After performing a multivariable analysis, Gunn et al [ 24 ] reported that there was no effect of age, gender, and gaming skills or activity on the outcome of VR learning. In the study by Shanahan [ 36 ], a few students (19/84, 23%) had previously used VR simulation software. This had no bearing on the learning outcomes. Another observation in the same study was that student age was found to significantly affected the student’s confidence about skill acquisition after VR training [ 36 ].

Students’ Perception of VR Uses for Learning

The findings from the study by Gunn et al [ 25 ] revealed that 68% of students agreed or strongly agreed that VR simulation was significantly helpful in learning about computed tomography (CT) scanning. In another study by Jensen et al [ 26 ], 90% of the students strongly agreed that VR simulators could contribute to learning radiography, with 90% reporting that the x-ray equipment in the VR simulation was realistic. In the study by Wu et al [ 37 ], most of the students (55.6%) agreed or somewhat agreed that VR use was useful in radiology education. Similarly, 83% of the students in Shanahan’s [ 36 ] study regarded VR learning with an ease of use. In the same study, students also reported that one of the major benefits of VR learning include using the simulation to repeat activities until being satisfied with the results (95% of respondents). Students also stated that VR enabled them to quickly see images and understand if changes needed to be made (94%) [ 36 ]. In the study by Gunn et al [ 25 ], 75% of medical imaging students agreed on the ease of use and software enjoyment in VR simulated learning. In the same study, 57% of the students reported a positive perceived usefulness of VR. Most respondents (80%) in the study by Rainford et al [ 31 ] favored the in-person VR experience over web-based VR. Similarly, 58% of the respondents in the study conducted by O’Connor et al [ 15 ] reported enjoying learning using VR simulation. In the study by Wu et al [ 37 ], 83.3% of students agreed or strongly agreed that they enjoyed using VR for learning. Similarly, the studies by Rainford et al [ 31 ] and O’Connor et al [ 15 ] reported student recommendation of 87% and 94%, respectively, for VR as a learning tool.

Students’ Perceived Skill and Knowledge Acquisition

In the study by Bridge et al [ 23 ], students who trained using VR reported an increase in perceived skill acquisition and high levels of satisfaction. The study authors attributed this feedback to the availability of “gold standards” that showed correct positioning techniques, as well as instant feedback provided by the VR simulators. Gunn et al [ 25 ] examined students’ confidence in performing a CT scan in a real clinical environment after using VR simulations as a learning tool. The study reported an increase (from before to after training) in the students’ perceived confidence in performing diagnostic CT scans. Similarly, the study by Jensen et al [ 26 ] reported that the use of VR had influenced students’ self-perceived readiness to perform wrist x-ray radiographs. The study, however, found no significant difference in pre- and posttraining (perceived preparedness) scores. The pre- and posttraining scores were 75 (95% CI 54-96) and 77 (95% CI 59-95), respectively. The study by O’Connor et al [ 15 ] looked at the effect of VR on perceived skill adoption. Most of the students in the study reported high levels of perceived knowledge acquisition in the areas of beam collimation, anatomical marker placement, centering of the x-ray tube, image evaluation, anatomical knowledge, patient positioning, and exposure parameter selection to their VR practice. However, most students felt that VR did not contribute to their knowledge of patient dose tracking and radiation safety [ 15 ]. In the study by Rainford et al [ 31 ], 73% of radiography and medical students felt that VR learning increased their confidence across all relevant learning outcomes. The biggest increase in confidence level was regarding their understanding of radiation safety matters [ 31 ]. Sapkaroski et al [ 33 ] performed a self-perception test to see how students viewed their clinical and technical skills after using VR for learning. In their study, students reported a perceived improvement in their hand and patient positioning skills. Their study also compared 2 software, CETSOL VR Clinic and Shaderware. The cohort who used CETSOL VR Clinic had higher scores on perceived improvement [ 33 ]. Sapkaroski et al [ 35 ] compared the student’s perception scores on the educational enhancement of their radiographic hand positioning skills, after VR or clinical role-play scenario training. Although the VR group scored higher, there was no significant difference between the scores for the 2 groups [ 35 ]. In the study by Shanahan [ 36 ], when the perception of skill development was evaluated, most of the students reported that the simulation positively developed their technical (78%), radiographic image evaluation (85%), problem-solving (85%), and self-evaluation (88%) abilities. However, in the study by Kato et al [ 27 ], there was no difference in the perceived acquisition of knowledge among students using traditional teaching and VR-based teaching.

Principal Findings

The results presented in this review reveal strong evidence for the effectiveness of VR teaching in radiology education, particularly in the context of skill acquisition and development [ 22 , 24 , 27 , 30 , 32 , 34 ].

In this review, quality appraisal of the cross-sectional studies revealed that the strategies for deal with confounding factors was one of the factors directly affecting the reliability of the results. Similarly, the appraisal of the randomized trials revealed that the bias arising due to missing outcome data was one of the factors directly affecting the reliability of the results.

All the studies found that VR-based teaching had a positive impact on various areas of radiographic and radiologic skill development. In comparison to the traditional way of teaching, only 1 study by Kato et al [ 27 ] reported VR teaching as inferior to traditional teaching. The studies consistently reported better improvements in proficiency, patient positioning outcomes, equipment handling, and radiographic techniques among students trained using VR. According to Nilsson et al [ 29 ], O’Connor et al [ 15 ], and Wu et al [ 37 ], the improvements were due to the immersive and interactive nature of VR simulations, which allowed learners to engage with radiological scenarios in a dynamic and hands-on manner. The studies also revealed that VR learning has the ability to easily and effectively introduce students to new skills. It was also found that existing skills could be improved, mainly through simulation feedback that happens in real time during training [ 22 , 24 , 28 , 30 , 36 ].

The improvement of skills after VR training have been noted in different domains, including patient positioning, equipment positioning, equipment knowledge, assessment of radiographic image quality, and patient interaction. Improvement was also observed in other skills such as as central ray positioning, source-to-image distance, image receptor placement, and side marker placement [ 22 , 24 , 30 , 32 , 34 ]. Two studies, Nilsson et al [ 28 ] and Nilsson et al [ 29 ], looked at how VR affected the students’ ability to interpret 3D information in radiographs using parallax. They both reported a positive effect. Nilsson et al [ 29 ] also gave insights into the long-term benefits of VR training in radiology. Eight months after training, the control (traditionally taught) group in Nilsson et al [ 29 ] showed a slight increase in skills, but the VR-trained group still maintained a significantly higher skill level. This finding shows the enduring impact of VR-based education on skill acquisition in radiology. Although most studies supported the effectiveness of VR in radiology education, 1 study reported contrasting results [ 27 ]. VR-trained students were found to perform worse than traditionally trained students in conducting lateral elbow and posterior-anterior chest radiography in Kato et al [ 27 ]. This difference in results was, according to the authors, attributed to the use of a different rubric evaluation method and the use of a head-mounted display–based immersive VR system, which was not used in other studies. These 2 reasons may be the reason for the variation in study findings.

A wide range of VR software with different functions were used in the studies. In addition to acquiring radiographic images, the CETSOL VR Clinic software facilitated students to interact with their learning environment [ 33 , 35 ]. Students using the Virtual Medical Coaching VR software performed imaging exercise on a virtual patient with VR headsets and hand controllers [ 15 , 30 , 32 ]. The SieVRt VR system displayed Digital Imaging and Communications in Medicine format images in a virtual environment, thus facilitating teaching [ 37 ]. The medical imaging training immersive environment simulation software provided automated feedback to the learners including a rerun of procedures, thus highlighting procedural errors [ 23 ]. The VR CT Sim software allowed the student virtually to perform the complete CT workflow [ 25 ]. Students could manipulate patient positioning and get feedback from the VitaSim ApS software [ 26 ]. The VR X-Ray software allowed students to manipulate radiographic equipment and patient’s position with a high level of immersive experience [ 27 ]. Radiation dosimetry VR software facilitated virtual movement of the staff and equipment to radiation-free areas, thus optimizing radiation protection [ 31 ].

The included studies also looked at factors that could influence skill acquisition when VR is used in radiology education. Bridge et al [ 23 ], Gunn et al [ 24 ], Kato et al [ 27 ], and Shanahan [ 36 ] investigated factors such as age, gender, prior gaming experience, and familiarity with VR technology. However, these factors were shown to have no significant effect on VR learning outcomes. This shows that VR education can equally accommodate a wide range of learners, regardless of experience or existing attributes.

Across several studies, positive feedback emerged regarding the utility, ease of use, enjoyment, and perceived impact on skill and knowledge acquisition. The included studies consistently reported positive perceptions of VR use among students [ 25 , 26 , 37 ]. Gunn et al [ 25 ] reported that a significant proportion of medical imaging and radiation therapy students found the use of VR simulation to be significantly helpful in learning about CT scanning. Similarly, Jensen et al [ 26 ] and Wu et al [ 37 ] reported that a majority of students agreed on the usefulness of VR in radiology education. Another aspect that received positive feedback was the ease of use. Students liked the ability to repeat tasks until they were satisfied with the results and the ability to quickly visualize radiographs to determine the need for revisions [ 36 ]. Rainford et al [ 31 ] and O’Connor and Rainford [ 30 ] found that most students would recommend VR as a learning tool to other students.

Several studies investigated student’s perceptions of skill and knowledge acquisition when using VR for radiology education. Bridge et al [ 15 ] and O’Connor et al [ 23 ] discovered an increase in students’ perceived acquisition of radiographic skills. Gunn et al [ 25 ] reported an increase in students’ perceived confidence to perform CT scans after learning using VR simulations. According to Rainford et al [ 31 ], a large percentage of radiography and medical students felt that VR learning boosted their confidence across all relevant learning outcomes, with the highest levels of confidence recorded in radiation safety. Sapkaroski et al [ 33 ] discovered that after using VR for learning, students experienced an improvement in their hand and patient placement skills. In summary, the positive feedback from the students shows that VR use in radiology education is a useful, engaging, and effective teaching tool. This perceived acquisition of skills is backed by the results from the proficiency tests.

The VR modalities used in some of the studies allowed remote assistance from an external agent (teacher), as the VR training is conducted in front of a screen while being part of a team, with the teacher making constant corrections and indications [ 22 , 24 , 27 ]. However, researchers are looking into VR systems with artificial intelligence–supported tutoring, which includes the assessment of learners, generation of learning content, and automated feedback [ 39 ].

Findings from the included studies show that VR-based teaching offers substantial benefits in various aspects of radiographic and radiologic skill development. The studies consistently reported that students educated using VR systems improved significantly in overall proficiency, patient positioning, equipment knowledge, equipment handling, and radiographic techniques. However, the variable nature of the studies included in the review reduces the scope for a comprehensive recommendation of VR use in radiology education. A key contributing factor to relatively better learning outcomes was the immersive and interactive nature of VR systems, which provided real-time feedback and dynamic learning experiences to students. Factors such as age, gender, gaming experience, and familiarity with VR systems did not significantly influence learning outcomes. This shows that VR can be used for diverse groups of students when teaching radiology. Students generally provided positive feedback about the utility, ease of use, and satisfaction of VR, as well as its perceived impact on skill and knowledge acquisition. These students’ reports show the value of VR as an important, interesting, and effective tool in radiology education.

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

• Ding X, Li Z. A review of the application of virtual reality technology in higher education based on Web of Science literature data as an example. Front Educ. Nov 17, 2022;7:1048816. [ CrossRef ]
• Oyelere SS, Bouali N, Kaliisa R, Obaido G, Yunusa AA, Jimoh ER. Exploring the trends of educational virtual reality games: a systematic review of empirical studies. Smart Learning Environments. Oct 19, 2020;7:31. [ CrossRef ]
• Zamyatina NV, Ushakova IA, Mandrikov VB. Medical education in digital transformation. In: Proceedings of the International Scientific Conference “Digitalization of Education: History, Trends and Prospects” (DETP 2020). Atlantis Press; 2020;324-327. [ CrossRef ]
• Kardong-Edgren S, Farra SL, Alinier G, Young HM. A call to unify definitions of virtual reality. Clin Simul Nurs. Jun 2019;31:28-34. [ CrossRef ]
• Sousa Santos B, Dias P, Pimentel A, et al. Head-mounted display versus desktop for 3D navigation in virtual reality: a user study. Multimed Tools Appl. Jan 2009;41(1):161-181. [ CrossRef ]
• Hamad A, Jia B. How virtual reality technology has changed our lives: an overview of the current and potential applications and limitations. Int J Environ Res Public Health. Sep 8, 2022;19(18):11278. [ CrossRef ] [ Medline ]
• Kaplan-Rakowski R, Meseberg K. Immersive media and their future. In: Branch RM, Lee H, Tseng S, editors. Educational Media and Technology Yearbook. Vol 42. Springer; 2019;143-153. [ CrossRef ]
• Kuliga SF, Thrash T, Dalton RC, Hölscher C. Virtual reality as an empirical research tool — exploring user experience in a real building and a corresponding virtual model. Comput Environ Urban Syst. Nov 2015;54:363-375. [ CrossRef ]
• Guan H, Xu Y, Zhao D. Application of virtual reality technology in clinical practice, teaching, and research in complementary and alternative medicine. Evid Based Complement Alternat Med. Aug 11, 2022;2022:1373170. [ CrossRef ] [ Medline ]
• Pottle J. Virtual reality and the transformation of medical education. Future Healthc J. Oct 2019;6(3):181-185. [ CrossRef ] [ Medline ]
• Satava RM, Jones SB. Current and future applications of virtual reality for medicine. Proc IEEE. Mar 1998;86(3):484-489. [ CrossRef ]
• Satava RM. Medical virtual reality. the current status of the future. Stud Health Technol Iform. 1996;29:100-106. [ Medline ]
• Rosenberg LB, Stredney D. A haptic interface for virtual simulation of endoscopic surgery. Stud Health Technol Inform. 1996;29:371-387. [ Medline ]
• Chytas D, Salmas M, Demesticha T, et al. A review of the use of virtual reality for teaching radiology in conjunction with anatomy. Cureus. Dec 5, 2021;13(12):e20174. [ CrossRef ] [ Medline ]
• O’Connor M, Stowe J, Potocnik J, Giannotti N, Murphy S, Rainford L. 3D virtual reality simulation in radiography education: the students' experience. Radiography (Lond). Feb 2021;27(1):208-214. [ CrossRef ] [ Medline ]
• Hart R, Karthigasu K. The benefits of virtual reality simulator training for laparoscopic surgery. Curr Opin Obstet Gynecol. Aug 2007;19(4):297-302. [ CrossRef ] [ Medline ]
• Thomas DJ, Singh D. Letter to the editor: virtual reality in surgical training. Int J Surg. May 2021;89:105935. [ CrossRef ] [ Medline ]
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372(71):n71. [ CrossRef ] [ Medline ]
• Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. Aug 28, 2019;366:l4898. [ CrossRef ] [ Medline ]
• McGuinness LA, Higgins JPT. Risk-of-bias visualization (robvis): an R package and shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. Jan 2021;12(1):55-61. [ CrossRef ] [ Medline ]
• Critical appraisal tools. JBI. URL: https://jbi.global/critical-appraisal-tools [Accessed 2024-04-25]
• Ahlqvist JB, Nilsson TA, Hedman LR, et al. A randomized controlled trial on 2 simulation-based training methods in radiology: effects on radiologic technology student skill in assessing image quality. Simul Healthc. Dec 2013;8(6):382-387. [ CrossRef ] [ Medline ]
• Bridge P, Gunn T, Kastanis L, et al. The development and evaluation of a medical imaging training immersive environment. J Med Radiat Sci. Sep 2014;61(3):159-165. [ CrossRef ] [ Medline ]
• Gunn T, Jones L, Bridge P, Rowntree P, Nissen L. The use of virtual reality simulation to improve technical skill in the undergraduate medical imaging student. Interactive Learning Environments. 2018;26(5):613-620. [ CrossRef ]
• Gunn T, Rowntree P, Starkey D, Nissen L. The use of virtual reality computed tomography simulation within a medical imaging and a radiation therapy undergraduate programme. J Med Radiat Sci. Mar 2021;68(1):28-36. [ CrossRef ] [ Medline ]
• Jensen J, Graumann O, Jensen RO, et al. Using virtual reality simulation for training practical skills in musculoskeletal wrist x-ray - a pilot study. J Clin Imaging Sci. Jul 11, 2023;13:20. [ CrossRef ] [ Medline ]
• Kato K, Kon D, Ito T, Ichikawa S, Ueda K, Kuroda Y. Radiography education with VR using head mounted display: proficiency evaluation by rubric method. BMC Med Educ. Jul 28, 2022;22(1):579. [ CrossRef ] [ Medline ]
• Nilsson TA, Hedman LR, Ahlqvist JB. A randomized trial of simulation-based versus conventional training of dental student skill at interpreting spatial information in radiographs. Simul Healthc. 2007;2(3):164-169. [ CrossRef ] [ Medline ]
• Nilsson TA, Hedman LR, Ahlqvist JB. Dental student skill retention eight months after simulator-supported training in oral radiology. J Dent Educ. May 2011;75(5):679-684. [ Medline ]
• O’Connor M, Rainford L. The impact of 3D virtual reality radiography practice on student performance in clinical practice. Radiography (Lond). Jan 2023;29(1):159-164. [ CrossRef ] [ Medline ]
• Rainford L, Tcacenco A, Potocnik J, et al. Student perceptions of the use of three-dimensional (3-D) virtual reality (VR) simulation in the delivery of radiation protection training for radiography and medical students. Radiography (Lond). Jul 2023;29(4):777-785. [ CrossRef ] [ Medline ]
• Rowe D, Garcia A, Rossi B. Comparison of virtual reality and physical simulation training in first-year radiography students in South America. J Med Radiat Sci. Jun 2023;70(2):120-126. [ CrossRef ] [ Medline ]
• Sapkaroski D, Baird M, McInerney J, Dimmock MR. The implementation of a haptic feedback virtual reality simulation clinic with dynamic patient interaction and communication for medical imaging students. J Med Radiat Sci. Sep 2018;65(3):218-225. [ CrossRef ] [ Medline ]
• Sapkaroski D, Baird M, Mundy M, Dimmock MR. Quantification of student radiographic patient positioning using an immersive virtual reality simulation. Simul Healthc. Aug 2019;14(4):258-263. [ CrossRef ] [ Medline ]
• Sapkaroski D, Mundy M, Dimmock MR. Virtual reality versus conventional clinical role-play for radiographic positioning training: a students' perception study. Radiography (Lond). Feb 2020;26(1):57-62. [ CrossRef ] [ Medline ]
• Shanahan M. Student perspective on using a virtual radiography simulation. Radiography (Lond). Aug 2016;22(3):217-222. [ CrossRef ]
• Wu Y, Mondal P, Stewart M, Ngo R, Burbridge B. Bringing radiology education to a new reality: a pilot study of using virtual reality as a remote educational tool. Can Assoc Radiol J. May 2023;74(2):251-263. [ CrossRef ] [ Medline ]
• Haddaway NR, Page MJ, Pritchard CC, McGuinness LA. PRISMA2020: an R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Syst Rev. Mar 27, 2022;18(2):e1230. [ CrossRef ] [ Medline ]
• King S, Boyer J, Bell T, Estapa A. An automated virtual reality training system for teacher-student interaction: a randomized controlled trial. JMIR Serious Games. Dec 8, 2022;10(4):e41097. [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Hasan Sapci, Taiane de Azevedo Cardoso; submitted 23.09.23; peer-reviewed by FernandezHerrero Jorge, Stacey Kassutto; final revised version received 01.02.24; accepted 31.03.24; published 08.05.24.

© Shishir Shetty, Supriya Bhat, Saad Al Bayatti, Sausan Al Kawas, Wael Talaat, Mohamed El-Kishawi, Natheer Al Rawi, Sangeetha Narasimhan, Hiba Al-Daghestani, Medhini Madi, Raghavendra Shetty. Originally published in JMIR Medical Education (https://mededu.jmir.org), 8.5.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Education, is properly cited. The complete bibliographic information, a link to the original publication on https://mededu.jmir.org/ , as well as this copyright and license information must be included.