virtual reality research paper

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 08 December 2022

Enhancing learning and retention with distinctive virtual reality environments and mental context reinstatement

• Joey Ka-Yee Essoe   ORCID: orcid.org/0000-0002-7802-4200 1 , 2 ,
• Nicco Reggente   ORCID: orcid.org/0000-0002-0511-9962 2 , 3 ,
• Ai Aileen Ohno   ORCID: orcid.org/0000-0002-5577-480X 2 , 4 ,
• Younji Hera Baek 2 , 5 ,
• John Dell’Italia 2 , 6 &
• Jesse Rissman   ORCID: orcid.org/0000-0001-8889-5539 2 , 7 , 8 , 9  

npj Science of Learning volume  7 , Article number:  31 ( 2022 ) Cite this article

6759 Accesses

9 Citations

174 Altmetric

Metrics details

• Learning and memory

Memory is inherently context-dependent: internal and environmental cues become bound to learnt information, and the later absence of these cues can impair recall. Here, we developed an approach to leverage context-dependence to optimise learning of challenging, interference-prone material. While navigating through desktop virtual reality (VR) contexts, participants learnt 80 foreign words in two phonetically similar languages. Those participants who learnt each language in its own unique context showed reduced interference and improved one-week retention (92%), relative to those who learnt the languages in the same context (76%)—however, this advantage was only apparent if participants subjectively experienced VR-based contexts as “real” environments. A follow-up fMRI experiment confirmed that reinstatement of brain activity patterns associated with the original encoding context during word retrieval was associated with improved recall performance. These findings establish that context-dependence can be harnessed with VR to optimise learning and showcase the important role of mental context reinstatement.

Similar content being viewed by others

The language network as a natural kind within the broader landscape of the human brain

Memorability shapes perceived time (and vice versa)

EEG is better left alone

Introduction.

Considerable research has documented that human memory is inherently context-dependent 1 , 2 . During learning, contextual cues—whether environmental (e.g., a specific room) or internal (e.g., an emotional state)—become bound to the information being encoded. Although some of these cues may be relevant to the to-be-learnt materials, many will be seemingly irrelevant. Despite their relevance, the later presence of these same contextual cues can facilitate memory recall, whereas their absence can hinder recall 3 . Perhaps the most iconic example of this effect is Godden & Baddeley’s 4 demonstration that scuba divers were better able to recall words that they had studied underwater when tested underwater, and better able to recall words studied on land when tested on land, but impaired when these study and test contexts were mismatched. Context effects can be observed with far less dramatic environmental changes (e.g., being tested in a different room 5 , or in a more quiet/noisy environment 6 ), are most robust when memory is probed with recall rather than recognition tests 1 , 7 .

One situation where context effects can be particularly impactful for learning is when multiple sets of information are studied in close temporal proximity. When the to-be-learnt content is similar across these sets, the build-up of interference can make it difficult to maintain clear mental representations of each set and cause confusion between the sets. For instance, reading two conceptually similar scientific papers within the same hour may lead one to mentally misattribute a finding of one paper to another. Likewise, while traveling to a place where two phonetically similar languages are spoken, it might be challenging to keep vocabulary items in these two languages appropriately compartmentalised in one’s memory if they are studied on the same plane flight. Some research has shown that learning each information set in its own distinctive context can improve recall by reducing this type of interference 8 , 9 . Specifically, a distinctive context provides unique cues that will become bound to items from a given information set. This supports learners’ abilities to maintain separate mental representations, reducing interference between the sets. This context-induced benefit increases in magnitude when the contexts are more distinctive, and when fewer items are affiliated with each context 1 , 8 , 9 , 10 .

Although distinctive learning contexts have the potential to reduce interference, they run the risk of creating context-dependent associations that could hinder later recall under circumstances where those contextual cues are no longer present. Whenever individuals have the luxury of studying information and repeatedly taking practice tests on that information in a single context, they may acquire the information quickly and perform quite well without realising the extent to which they are using the contextual cues as a “crutch” to facilitate learning and retrieval 9 , 11 . Only when later struggling to recall the information in a new context—such as a foreign traveller trying to use vocabulary that had only ever been practiced in a classroom setting—does their reliance on this contextual crutch become apparent. In most real-world settings, it is impossible or impractical for learners to physically return to the original encoding context as a means to gain access to helpful retrieval cues. Fortunately, mental reinstatement—the act of vividly imagining oneself in the original encoding environment—presents one solution to promote information transfer across contexts. Indeed, mental reinstatement can be nearly as effective as physically returning to the learning context 2 , 12 . Thus context change-induced forgetting may be mitigated by mentally “returning” to the learning context during recall.

Learning protocols that harness the beneficial aspects of context-dependence while ameliorating the deleterious effects are likely to yield the best outcomes. How best to achieve this balance thus remains an active and important area of research. Designing and controlling distinct contexts in practice is challenging, for experimenters and learners alike. Manipulating one’s physical context can influence learning and recall, but doing so can be costly, time-consuming, and difficult to control. Background images 13 and videos 14 have been used as contexts in an effort to increase experimental control. While these can serve as proximate contextual cues in experiments, they do not allow navigation or immersion like real-world contexts, and thus ecological validity suffers 15 .

Virtual reality (VR) offers a powerful means to create immersive learning environments that are highly distinctive and well-controlled, in order to examine and exploit context-based memory modulation 15 , 16 . Indeed, one recent study used two distinctive VR-based contexts—one underwater and one on the surface of Mars 17 —to conceptually replicate Godden & Baddeley’s classic finding of context-dependent recall. When using VR environments as contexts, it is valuable to measure participants’ sense of presence 18 , 19 , 20 , 21 , which refers to their sense of experiencing a VR-based environment as a place that one has actually inhabited, rather than something that one was merely watching passively (e.g., “I feel like I am in this space station, walking around,” vs “I am watching this space station on a screen while sitting in a lab.”). If an individual does not perceive VR-based contexts as actual environments, then these contexts may have little or no effect on memory outcomes because the “contexts” themselves would not be subjectively valid.

Here, we aimed to leverage the benefits of context-dependence to enhance learning and retention. We chose to focus on foreign vocabulary learning as it is a domain of practical value to many people, while also being a paradigmatic paired associate learning task. To rigorously test this approach, we selected learning material to maximise potential interference and used a challenging recall test. English-speaking participants learnt the meanings and pronunciations of 80 foreign words from two phonetically similar Bantu languages: Swahili and Chinyanja. During testing, participants were prompted to verbally pronounce foreign words when cued with their English translations (note that this is far more difficult than being cued with the foreign word and recalling the English translation 22 ).

Two custom, first-person desktop VR environments served as contexts, which enabled maximal experimental control over the learning contexts and subsequent guided mental reinstatement. First, we investigated whether contextual support could improve learning outcomes by reducing interference and promoting transfer. To this end, participants were randomly assigned to one of two groups: a single-context ( n  = 24) group that learnt both languages in a single VR context, and a dual-context ( n  = 24) group that learnt each language in its own unique VR context. We hypothesised that dual-context participants would be better able to keep track of which translations went with which language and thus would show fewer intrusions (i.e., producing the Chinyanja translation of a word when cued to recall the Swahili translation), and greater long-term retention (as measured on a surprise recall test conducted one week later). Moreover, we predicted that the magnitude of these context effects might be contingent on whether participants subjectively experienced the VR-based contexts as actual environments they had inhabited (i.e., did they have a strong sense of presence?). Thus, a 10-item presence scale (range 1-5) from a prior study was used to measure the degree to which participants felt “as one” with their first-person avatar and experienced the VR as real environments 19 . To assess the role of mental context reinstatement, our paradigm explicitly cued participants to imagine themselves in a specified place prior to each vocabulary recall trial. This allowed us to measure the impact of context reinstatement congruency (i.e., whether they reinstated the same or different context in which they had learnt a given language) on recall performance. Finally, to further explicate a potential mechanism for contextually supported recall, we examined a separate group of dual-context participants ( n  = 22) during recall, using functional magnetic resonance imaging (fMRI) to provide a neural index of context-specific reinstatement on each retrieval trial 23 , 24 . We hypothesised that elevated reinstatement of brain activity patterns linked to the original encoding context would enhance the likelihood that participants would be able to successfully recall the cued foreign vocabulary item. Given the universal desire to develop protocols for memory enhancement across disciplines, this investigation holds considerable promise for fields such as cognitive research, pedagogy, and psychotherapies that involve therapeutic skill learning.

Initial learning: contextual crutch and desirable difficulties

Across two consecutive days, participants encoded a total of 80 foreign vocabulary items in two languages, in one learn-only round (Round 1), followed by three test-learn cycles (Rounds 2–4, retrieval attempts during these tests were scored as recall data for Times 1-3; T1-T3 . See Fig. 1 , Fig. 2e , Methods, and Supplementary Video 1 ). They learnt 10 words in Swahili only, 10 in Chinyanja only, and 30 words in both languages. To induce contextual crutch effects, test-learn cycles occurred within the learning context(s) as participants navigated along a predetermined path (Fig. 2a–d ). To further bolster initial learning we integrate a “desirable difficulties” technique 25 called expanding retrieval practice, in which the time interval between successive learning and testing opportunities progressively increased 26 . Differences between the single-context and dual-context groups were not expected to emerge during the initial learning stage, as the magnitude of context effects has been shown to increase with the length of the retention interval 1 .

a Encoding tasks in VR-based contexts across Days 1 and 2. a1, In an underwater practice context, participants learnt VR navigation and received tasks instructions from “the teacher.” a2, Task Practice (under experimenter supervision). a3, Context A Encoding. In each of Context A’s nine named “rooms”, participants stood on a location marker and performed two clock-wise rotations (720°), while imagining themselves as tourists who forgot their camera, trying to remember what it felt like to be there. a4, Language 1 Encoding. Participants remained in Context A to encode Language 1 (Rounds 1–3, 40 words per round). a5, Context B Encoding. a6, Language 2 Encoding (Rounds 1–3). All participants experienced the same procedures except for the context in which Language 2 was encoded. Single-context participants returned to Context A to encode Language 2, while dual-context participants remained in Context B to encode Language 2. On Day 2 participants performed Rounds 4 of Language 1 and Language 2 Encoding. b Day 2: short-delay recall (T4). After a short delay, participants were tested outside of the VR contexts, in the laboratory or MRI scanner. In each of 80 trials, participants first mentally reinstated an auditorily cued room from one context before recalling the foreign translation of a cued word. In congruent reinstatement trials, the mentally reinstated room was the learning context of the cued word. In incongruent reinstatement trials, the mentally reinstated room was in the opposite context. c Day 8: one-week-delayed recall (T5). Participants were telephoned, ostensibly for an interview; experimenters then cued recall for all 80 foreign words. Image attribution: The VR environments and content depicted here were created by J.K.-Y.E or by Forde Davidson as commissioned by the research team, or were from the OpenSim community shared under the Creative Commons 0 License. The image of the telephone and computer monitor were modified from public domain images, and the image of the MRI scanner was provided by the UCLA Brain Mapping Center.

Two custom-built VR-based contexts were used in this study. a “Fairyland Garden” was a fantasy-fiction inspired context that was bright, verdant, visually open, with lakes and wooden rooms opened to the outdoors. b Fairyland Garden’s predetermined path used in language encoding. This path’s hints were bright green footsteps; its pedestals tree stumps. c “Moon Base” was a science-fiction inspired context that was dark, rocky, closed-in, with narrow hallways and artificially coloured metallic rooms, and participants were confined indoors at all times. d Moon Base’s predetermined path used in language encoding. This path’s hints were bright yellow arrows; its pedestals yellow stands as shown in 2e. e Language encoding task. In each round of language encoding, participants interacted with 40 concrete objects representing each of the foreign words (e.g., a rooster), organised along a predetermined path. The VR environments were experienced through a first-person perspective (a visible avatar is only present in this figure for illustrative purposes). e1, Participants followed visual hints (e.g., arrows) to an object; these hints were transient and disappeared after use. After arriving at the object, participants first verbally say its English name (e.g., “rooster”), printed in floating text above the object. During Round 1 of each language, participants then ‘clicked’ the object. e2, During Rounds 2–4, participants first attempted to verbally recall the foreign words (T1-T3) before clicking the object. e3, When the object was clicked, participants would hear the foreign translation (e.g., Swahili word “jogoo,” meaning rooster) three times. They were to repeat aloud after it each time. Then they clicked the object’s pedestal to reveal transient path hints to the next object. Image attribution: The VR environments and content depicted here were created by J.K.-Y.E or by Forde Davidson as commissioned by the research team, or were from the OpenSim community shared under the Creative Commons 0 License.

Across groups, participants recalled 42% (±17%) of the 80 foreign words after two exposures (T2); note that each “exposure” refers to encountering an object and hearing and repeating back its translation three times in rapid succession (Fig. 3 ). This learning rate was considerably higher than expectations (22–26%) set based on a previous study that used similar learning material (42 Swahili-English word pairs; no secondary foreign language was learnt in that study), but did not employ distinctive learning contexts (see Supplementary Discussion: D 2 for additional discussion) 27 . After the third exposure to the foreign words, our participants were not tested until the following day (T3), and yet their recall performance remained robust at 42% (±17%). As expected, no group differences emerged during the initial learning stage ( p  > 0.05).

a overall recall performance, split by context group and presence. b Main effect of mental reinstatement on T4 recall. c Main effect of context group condition on intrusions. d Interactions of context group and presence in one-week retention. * denotes statistical significance, error bars denote standard error of the mean.

Transfer and mental reinstatement

Transfer was measured by recall during a non-VR test (T4), which was the first test that occurred outside of the learning context. Across conditions, participants recalled 48% (±18%) in T4. A controlled mental reinstatement protocol was employed to maximise consistency across participants and across experiments (Fig. 4 ; see Methods). On each trial, participants were first cued to mentally reinstate a specific area within a given learning context (e.g., “Moon Base: Airlock”). Then, they were prompted by audio cues (e.g., “Swahili: dog”) to attempt to covertly retrieve the appropriate foreign translation, and finally a beep sound cued them to verbally pronounce the word. Two mental reinstatement conditions were employed: congruent reinstatement (when the original learning context of the to-be-recalled word was mentally reinstated) and incongruent reinstatement (when a different context was mentally reinstated). During T4, congruent mental reinstatement trials exhibited significantly greater recall (52% ± 18%) than incongruent reinstatement trials (47% ± 19%), RM-ANOVA, p  = 0.009, η p 2  = 0.31; Fig. 3b ; see Supplementary Note 1: A2, A3 ). This demonstrated that when recalling in a new context, transfer is enhanced when the learning context is mentally reinstated. This effect did not interact with context-group membership, suggesting that even those participants who learnt both languages in a single context still benefitted when prompted to mentally reinstate that context relative to when they reinstated a context in which neither language had been learnt.

An example trial of the short-delay non-VR test. Each trial consisted of the following periods: Mental reinstatement, language recall, imagery vividness rating, and two arithmetic questions (which served as an active baseline period between trials). The words “Get Ready” appeared to indicate the start of each trial. Mental Reinstatement: Participants heard via headphone the name a room they had visited (e.g., “Moon Base: Airlock”). Then the screen turns black, cuing participants to close their eyes and mentally “place” themselves back in that room. They pressed Button 1 to indicate that they had successfully “arrived” and oriented themselves. Then they mentally performed the same rotations they had done in the context encoding task (Figs. 1 a.3, 1a.5 ), while pushing Buttons 2 and 3 to indicate their mental reinstatement progress until they heard a beep. In the fMRI experiment, brain activity patterns related to mental imagery were extracted for the period between the Button 1 press and the beep. Language Recall: Participants heard the language recall cue (e.g., “Swahili: Dog”). Participants began to covertly retrieve the foreign word and made a button-press to indicate success or failure of retrieval; they then continued thinking about that word until they heard a beep. Upon the beep, they verbally pronounced the foreign word, or the portion of it they could recall. In the fMRI experiment, brain activity patterns related to language recall were extracted from the 6 s after the audio cue offset. Imagery Rating: Participants rated how vivid the previous mental reinstatement had been. These ratings were later used for trial exclusion for analyses involving mental reinstatement. Arithmetic Questions: At the end of each trial, participants answered two simple arithmetic questions. Each involved a display of two single-digit integers, and they were to press Button 1 if the product of these numbers was odd, and Button 2 if even.

Interference reduction

Interference was measured by intrusions from the opposite language (i.e., producing the Chinyanja translation of a word when cued to recall the Swahili translation, or vice versa), as these indicate a failure to maintain clear and distinctive representations between the two languages. While the intrusion count was generally low (less than 10 items out of 80), dual-context participants exhibited 38% fewer intrusions (4.09 ± 4.82) than the single-context (6.57 ± 4.69) participants (Fig. 3c ; RM-ANOVA, p  = 0.014, η p 2  = 0.13; see Supplementary Note 1: A3 ). This suggests that learning each language in its own distinctive context helped participants to maintain better separated mental representations and reduced interference.

One-week retention

A surprise memory test (T5; Fig. 1d ) was conducted via telephone one-week after T4. In a pre-scheduled “follow-up interview,” experimenters asked participants several interview questions and then began to conduct T5 (e.g., “How do you say ‘cherry’ in Chinyanja?”). Retention score was the percentage of information that survived the one-week delay interval, after it had been previously recalled in T4 (i.e., words that were not successfully recalled in T4 were excluded, see Methods). Furthermore, as the context manipulation was conducted via VR, presence (one’s sense of inhabiting a VR-based context as a real location) was entered into the analyses as a factor—if participants did not experience the VR environments as real contexts, then the context manipulation should have little to no effect.

Results showed that amongst participants who reported high presence (based on a mean split of presence scores, see Supplementary Table 2 ), the dual-context group exhibited a striking 92% (±7%) one-week retention rate, which was significantly higher than 76% (±12%) retention rate exhibited by the single-context group (Fig. 3d ; RM-ANOVA interaction, p  = 0.03, η p 2  = 0.11; simple main effect, p  = 0.002; see Supplementary Note 1: A4 ). Single- and dual-context participants who reported low presence did not perform differently on one-week retention (simple main effect for low-presence participants, p  = 0.47), nor did they differ from single-context participants reporting high presence (all contrasts p  > 0.05). Collectively, these results demonstrate that contextual support from unique contexts dramatically enhanced one-week retention, but only when participants subjectively perceived the contexts as actual environments they had inhabited.

Neural correlates of contextually supported recall

To further investigate the mechanisms by which distinctive learning contexts can later be brought back to mind to support the recall of foreign vocabulary items, we conducted a follow-up fMRI experiment. We recruited a separate group of participants ( n  = 23; analyses included n  = 22; see Methods) and assigned them all to the dual-context learning condition, since our goal was to measure context-specific reactivation on individual recall trials so as to characterise the behavioural advantage afforded by such reactivation. Given resource constraints, it was not possible for us to scan a separate group of single-context participants, nor would fMRI data from such participants be especially useful for our primary research question.

The use of verbal material separated the sensory modalities between contexts (visuospatial) and memoranda (verbal/auditory), allowing us to disentangle the neural correlates of contextual support from the memory retrieval itself. First, a whole-brain Searchlight Multi-Voxel Pattern Analysis (Supplementary Fig. 1 ; SL-MVPA) identified brain regions whose local fMRI activity patterns could most accurately discriminate between the two contexts during the mental reinstatement period. Each participant’s resulting searchlight map was thresholded to create an individualised binary mask, indicating which 2000 voxels would be used for the subsequent steps. Because the particular voxels selected for each participant will differ, we are unable to make claims about how individual brain regions contributed to our analyses. However, in an effort to provide a coarse portrait of which regions’ local activity patterns tended to be most able to facilitate context decoding, the group mean of the searchlight map is visualised in Supplementary Fig. 2 and shows that peak decoding was observed in bilateral visual association regions (superior lateral occipital cortex, ventral occipito-temporal cortex, fusiform gyrus), medial parietal regions (precuneus, posterior cingulate cortex), lateral parietal regions (intraparietal sulcus and superior parietal lobule), and the left inferior frontal sulcus. Second, a brain-response pattern was derived within this mask for each of the two learning contexts (Fig. 5a ; context template). Third, a Representational Similarity Analysis (Fig. 5a ; RSA) produced a similarity score between (1) the brain patterns during covert retrieval of each word and (2) the context template of the learning context of that word. This RSA score provided an objective, quantitative measure for mental contextual reinstatement during verbal recall for each individual trial, which we will refer to as its “representational fidelity.” Fourth, the verbal recall scores of words with high vs low representational fidelity (mean-split within-subject) were compared—which allowed us to examine whether trials with greater evidence for contextually supported retrieval enjoyed a behavioural performance advantage relative to those with less evidence for contextually supported retrieval.

After feature selection, fMRI activity patterns from each participant’s top 2000 voxels were used in a within-subject representational similarity analysis (RSA); RSA output was used to analyse verbal recall data. a RSA computed the correlations between activity patterns for each word during covert word recall (right) and the context template (left) of the word’s original learning context. The context template was an average of all the imagery patterns for a given context. The resulting correlation values were then used to divide recall trials into high fidelity vs low fidelity reinstatement trials, and verbal recall results were examined for each trial type. The effects of reinstatement prompt (congruent vs. incongruent) and/or reinstatement fidelity (high vs. low) on recall are plotted respectively for: ( b ), all non-VR tests (T4 and T5; collapsed across reinstatement prompt conditions), ( c ), short-delay non-VR test (T4), and ( d ), one-week-delayed non-VR test (T5). * denotes statistical significance for pairwise tests; see main text for description of interaction effects. Image attribution: The VR environments depicted here were created by J.K.-Y.E. or by Forde Davidson as commissioned by the research team, or were from the OpenSim community shared under the Creative Commons 0 License. The icons used were either created by J.K.-Y.E. or were modified from stock icons in MS PowerPoint or public domain.

A main effect of representational fidelity was observed (RM-ANOVA, F (1, 21) = 13.712, p  = 0.001, η p 2  = 0.395; see Supplementary Note 2 ), where high representational fidelity trials (0.50 ± 0.17) were associated with 5% higher recall than low representational fidelity trials (0.45 ± 0.18), collapsing across the short-delay test (T4) and one-week-delayed test (T5). When broken down by Times (Fig. 5b ), the effect of representational fidelity was significant at both T4 (RM-ANOVA, F (1, 21) = 8.60, p  = 0.008, η p 2  = 0.29; High = 0.56 ± 0.19; Low = 0.51 ± 0.20) and T5 (RM-ANOVA, F (1, 21) = 8.53, p  = 0.008, η p 2  = 0.29; High = 0.44 ± 0.19; Low = 0.39 ± 0.20) in follow-up analyses. Furthermore, a significant interaction between reinstatement prompt and representational fidelity was observed across T4 and T5 (RM-ANOVA, F (1, 21) = 6.59, p  = 0.02, η p 2  = 0.24; not shown). This examined how recall performance was impacted by the relationship between representational fidelity and the reinstatement prompt at the beginning of each trial (i.e., whether participants were cued to recall a room in a context congruent or incongruent with the language that was about to be probed). Follow-up analyses revealed that this interaction was driven by T5 one-week delayed recall ( simple interaction: p  = 0.006; Fig. 5d ), and not T4 short-delay recall ( p  > 0.05; Fig. 5c ). After incongruent mental reinstatement, if representational fidelity had been high during T4 recall, participants enjoyed a 10.1% advantage one week later (0.45 ± 0.19) as compared to if representational fidelity had been low (0.35 ± 0.20). This effect was absent in the trials preceded by congruent mental reinstatement, and recall was still high for both conditions (both 0.43 ± 0.20).

These findings indicated that we were able to quantify contextual support via mental reinstatement—by identifying neural representations of the two learning contexts and measuring their expression during each covert word retrieval attempt. Overall, we found a striking relationship between trial-specific evidence of context reinstatement fidelity and the likelihood of successfully recalling the cued word in the specified language on that trial. The behavioural advantage of high-fidelity reinstatement was not only present in the immediate term (T4 recall) but also persisted after a one-week delay (T5 recall). That this advantage was most apparent during incongruent reinstatement trials indicates that as long as participants were able to reinstate the original learning context during the word recall phase (despite having been prompted to imagine a different context several seconds earlier) they could minimise the potential disadvantage of this contextual incongruency.

By using distinctive virtual reality environments to provide rich contextual support, our behavioural protocol facilitated robust learning of highly challenging material—foreign vocabulary in two phonetically similar languages—while ameliorating the negative effects of context-dependence via “desirable difficulties” and mental reinstatement. These memorable contexts could later serve as retrieval cues when mentally reinstated during recall. After only four learning sessions, participants were able to recall nearly half of the 80 foreign words they had studied, and they showed relatively little forgetting after one week (up to 92% retention). Importantly, the knowledge acquired within the VR-based contexts transferred well to support recall in non-VR settings (i.e., a laboratory testing room, an MRI scanner, and a surprise telephone test), despite the fact that the learning contexts shared relatively few cues with real-world environments. In so doing, we leveraged the benefits of the “contextual crutch” phenomenon whereby rapid acquisition was facilitated by repeatedly learning and testing in the same context while mitigating the deficits of transfer and retention that typically accompany this occurrence (See Supplementary Discussion: D 3 ) 1 , 11 , 28 .

Our results provide evidence that contextual support optimises language learning in a manner that leads to high retention—but only when three critical conditions are met: First, participants must subjectively experience the VR-based contexts as actual environments that they feel like they are physically inhabiting during learning (i.e., they must report a high sense of presence). Second, a unique context must support the learning of each language. A high degree of presence, on its own, was insufficient to enhance retention for those participants in the single-context group who learnt the two languages in the same VR-based context. Only those participants assigned to the dual-context group—and who exhibited high presence during learning—showed superior retention of the material at the long-delayed test conducted one-week later. These high-presence dual-context participants were subjectively learning the two languages while actively navigating through two very different places, whereas low-presence participants presumably felt like they were learning both languages while sitting in a laboratory testing room. Third, benefits to memory recall must be evaluated after a long delay. Although dual-context participants did show fewer intrusions of the incorrect language translations (e.g., producing the Swahili translation when cued to recall the Chinyanja translation) at the immediate non-VR test (i.e., T4 on Day 2), they didn’t show an overall improvement in recall performance on this test. The dual-context participants’ advantage only emerged after the passage of one week’s time (i.e., T5 on Day 8). This finding illustrates that learning the two languages in two distinctive contexts can protect against forgetting, but only if participants felt highly present within the contexts. That the benefit was only observed after a long delay is consistent with previous reports that context-dependent effects tend to increase with longer retention intervals 1 , 29 . This may be due to the fact that that at shorter retention intervals a greater number of internal contextual cues (e.g., moods, levels of hunger or fatigue, private thoughts, etc.) may match those present during learning, thus outshining the effects of environmental context. Because we only assessed memory immediately after learning and at a one-week delay, we are unable to draw precise conclusions about the time course of the dual-context advantage. It is possible that the advantage could have emerged sooner (e.g., on Day 3 after one additional night of sleep), and it is also possible the magnitude of the effect could have grown even larger over time (e.g., if we waited two weeks before conducting the surprise memory test).

One critical attribute of our task design was the experimentally cued mental reinstatement of a specific environmental context prior to each vocabulary recall trial. This manipulation gave us precise experimental control over participants’ mental content immediately preceding each retrieval attempt. The cued context could either be congruent with the information the participant was about to be tested on (i.e., imagining themselves in the exact same ‘room’ where they had learnt that vocabulary item) or it could be incongruent (i.e., imagining themselves in a different ‘room’ from a completely different environment). Consistent with prior evidence for the benefits of mental reinstatement 2 , 12 , we found that imagery-based reinstatement of the congruent learning context enabled better recall in the short-delay non-VR test (i.e., T4).

In order to gain further insight into the impact of context reinstatement, we devised a follow-up experiment that used fMRI to measure neural correlates of context representations. This provided an objective index of the degree to which learning contexts were mentally reinstated during the language recall period of each trial. Unlike the behavioural experiment, the fMRI experiment enabled us to quantify mental reinstatement without relying on inferring mental reinstatement based on task instructions and participants’ subjective reports, nor to rely on the assumption that the reinstatement state would linger from the mental reinstatement period into the language recall period. Our fMRI experiment revealed evidence for contextually-supported retrieval of verbal materials. The results demonstrated that increased brain pattern similarity to the original learning context during covert verbal retrieval was associated with more successful recall performance. Trials with high reinstatement fidelity scores yielded short-delay recall performance (i.e., recall that took place seconds later) that was 5% higher than trials with low reinstatement fidelity scores. These high-fidelity reinstatement trials continued to enjoy the 5% recall advantage when memory was again tested one week later. This result expands upon a recent demonstration that context-specific fMRI activity patterns, induced through a closed-loop neurofeedback procedure, could facilitate verbal recall when the reinstated context was congruent with the learning context 30 .

When we examined the joint effects of mental reinstatement prompts and representational fidelity, we noted an interesting pattern. While high-fidelity mental reinstatement during recall improved short-delay recall regardless of pre-recall reinstatement prompts, after a one-week delay (T5) this advantage only appeared for words that had been paired with an incongruent pre-recall reinstatement prompt during T4. Thus, instructions to imagine oneself in a context that, just moments later, turns out to be incongruent with the learning context of the prompted language will serve to diminish the one-week retention of that word unless the participant manages to counteract this initial miscue and engage in high-fidelity reinstatement of the original learning context during word recall. In this sense, the act of overcoming incongruently cued context reinstatement by rapidly bringing the correct context back to mind may be considered a “desirable difficulty,” 25 given its ability to promote one-week retention.

Although our study did not systematically compare the influence of spatial contexts with other aspects of event representation, our findings are consistent with the notion that spatial context is crucial in event representations. There is growing evidence that spatial context is possibly a dominant attribute over and above other episodic details (e.g., objects and persons) 31 , 32 . Intracranial electroencephalographic recordings from human hippocampus show that spatial context information is often reactivated earliest in the retrieval process and guides recall of items learnt in that context 33 . When recalling short stories, spatial cues lead to quicker and more detailed memories about events 34 . In a VR learning paradigm based on the Method of Loci mnemonic techniques, we previously demonstrated that memory for the spatial layout of VR environments is correlated with participants’ ability to recall words learnt in those environments 35 . Even though the contexts used in the present study’s foreign vocabulary learning task bore no direct relevance to the verbal content being learnt, these richly detailed virtual environments provided a consequential scaffolding that helped mitigate potential interference 36 and provided memorable spatial cues that learners could later think back to when attempting word recall. While we did not directly test for this, the ability of our participants to actively navigate through the contexts during learning was likely an important determinant of the contextual effects we observed. One prior study investigating context-dependency used VR environments as passively presented backgrounds during word learning and found no impact of context reinstatement on behaviour 37 , 38 . Although there were other critical differences between our respective paradigms, this suggests that investigation of context effects will benefit when contexts are experienced in a more ecologically valid manner—such as the navigable, interactive desktop VR used here. When such contexts are experienced in VR, our results expand upon prior work emphasizing the importance of high presence in mediating the mnemonic benefits 37 . More broadly, our results showcase the critical importance of context in learning and bolster recent calls for cognitive neuroscientists to move beyond the study of isolated decontextualised stimuli 39 .

Presence, in addition to enabling virtual environments to serve as contexts for context-dependent memory effects, may be contributing to enhance learning in its own right. The recent Cognitive Affective Theory of Immersive Learning (CAMIL) 40 would predict that VR experiences that induce a sense of presence can increase learner interest and intrinsic motivation, which in turn generates greater learner efforts and willingness to attend the task, thereby facilitating learning and recall. Indeed, engaging learning environments using head-mounted display (HMD)-based VR, and generative learning activities therein, have been found to lead to better transfer 41 , 42 . Although we did not quantitatively examine our participants’ interest, intrinsic motivation, or engagement, these advantageous internal contexts during our desktop VR-based learning tasks may have contributed to our participants recalling 42% of the 80 foreign words after only two exposures, considerably higher than a previous non-VR study (22–26%) that used arguably easier to-be-learnt material without distinctive learning contexts 27 . Furthermore, CAMIL would posit that if the VR contexts were more meaningful and relevant to the to-be-learnt items, the learning enhancement effects would be greater still due to an increased sense of presence and agency.

Our study has several limitations that should be addressed in future work. In an effort to gain greater experimental control, we elected to cue mental reinstatement of a specific context immediately prior to each foreign word recall prompt. While this manipulation allowed us to examine the effects of reinstatement congruency and facilitated our effort to create context-specific brain activity templates, it prevented us from knowing how our participants would have performed—and to what degree neural reinstatement would have predicted their performance—had we not invoked any explicit reinstatement instructions. Also, our use of fMRI was focused on using neural measures to index putative mental states, which we could then relate to behaviour. Although our whole-brain multivariate pattern analysis approach afforded us enhanced power in our ability to measure context reactivation effects (which could incorporate perceptual, semantic, and emotional attributes of the respective contexts, represented across a wide array of brain regions), it limited our ability to draw conclusions about the role of specific brain structures in supporting context reinstatement and vocabulary recall. Furthermore, as the context-dependent learning enhancement effect was contingent on participants’ subjective sense of presence, future research using newer, more immersive HMD-based VR systems—especially those using omnidirectional treadmills for navigation—may find even stronger context-dependent effects due to the likely increased sense of presence. Additional studies with larger sample sizes will be necessary to characterise more fully how individual differences in presence levels impact the degree of context-dependence in VR learning tasks. Finally, along with CAMIL 40 , recent work has shown that the relevance of an environmental context to the information being learnt in that context is consequential for that information’s memorability 17 and transfer 41 , 42 . In our task, the relationship of the contexts to the languages and vocabulary being learnt was completely arbitrary. Future studies may confer additional memory advantages if language learning occurs in VR-based replicas of familiar real-world environments where that language would actually be useful (e.g., learning fruit vocabulary while navigating through the produce section of a grocery store or outdoor farmer’s market). Moreover, investigators should systematically quantify potentially relevant factors such as engagement, intrinsic motivation, interest, and agency in addition to measuring presence.

In summary, this study successfully harnesses context-dependence to enhance the learning of highly challenging and interference-prone material, while remedying the negative effects of context-dependence. After leveraging “contextual crutch” and “desirable difficulties” to enable a rapid learning rate, contextual support and mental reinstatement enabled transfer and overcame context change-induced forgetting, facilitating the real-world retrieval of information learnt in VR. This approach led to strikingly high one-week retention (92%) in participants who received unique contextual support for each language they had learnt, as long as they subjectively perceived the VR-based contexts as actual environments they had inhabited. Moreover, using neuroimaging to quantify mental context reinstatement during vocabulary recall, we found that trials with higher fidelity reinstatement of the learning context were associated a better ability to recall the foreign words they had learnt in that context. As learning and memory are involved in nearly every aspect of life—and they must always occur in some form of contexts—harnessing context-dependence to enhance memory bears far ranging practical implications for education, skill training, health care, as well as a potential to enhance therapeutic learning in evidence-based psychotherapy.

Participants

Data from forty-eight adult participants (26 females, age range 18–27 years; Supplementary Table 1 ) were included in the analyses for the behavioural experiment; participants were randomly assigned to one of two context conditions (single- and dual-context, each n  = 24). Data from twenty-two different adult participants (12 females, age range 19–25 years) were included in the analyses for the fMRI experiment; all were assigned to the dual-context condition.

Participants were recruited through flyers posted around the campus of the University of California, Los Angeles (UCLA) and social media advertisements targeting the same geographical area. Participants were tested individually, and they received course credit or were compensated monetarily ($20 per hour for fMRI procedures, $10 per hour for non-fMRI procedures). All participants provided written informed consent, and all study procedures were approved by the Institutional Review Board at the UCLA.

Eligibility screening was conducted using the Research Electronic Data Capture (REDCap) online survey system 43 . Inclusion criteria were as follows: (1) being monolingual English speakers (with no more than high school language courses for any other language) for the behavioural experiment, and being bilingual English speakers (having more than high school language courses for exactly one other language) for the fMRI experiment—this criterion was established for the fMRI experiment to increase baseline recall levels based on pilot results showing that bilingual participants learnt novel foreign vocabulary more quickly; (2) having limited (<5 h) prior exposure to the VR platform used in the experiment; (3) having normal or corrected-to-normal vision and audition; (4) having no diagnosis of learning disabilities; (5) reporting no substance dependence; and (6) not taking any psychotropic medications. Behavioural experiment data from an additional 13 people were acquired but excluded from analyses: five did not complete the procedure due to technical difficulties, three withdrew due to motion sickness during their desktop-VR experience, three did not return for Day 2 procedures, and two were excluded for not following instructions. fMRI experiment data from one additional person was acquired but excluded from analyses, for this individual reported falling asleep during procedure.

In the behavioural experiment, participants were randomly assigned to one of the two conditions (single- or dual-context); all participants in the fMRI experiment were assigned to the dual-context condition. All participants underwent the same procedural sequence (Fig. 1 ): Context A encoding, Language 1 encoding in Context A, Context B encoding, Language 2 encoding in Context A (single-context condition) or Context B (dual-context condition), non-VR test (in laboratory or in MRI scanner), and surprise telephone test.

This experiment measured recall at five time-points (Times 1–5, hence T1–T5). Each language was encoded four times in the VR-based learning contexts: one initial study session followed by three test-study cycles (T1–T3) across two lab visits on consecutive days. At the end of the Day 2 visit, participants were tested outside of the VR learning contexts (T4), either in the lab or in the MRI scanner, and tested again over the telephone one week later (T5).

Virtual reality

Two distinctive VR-based contexts were used for the learning task (Fig. 2a–d ). Participants navigated the world using a computer mouse and keyboard, where the mouse aimed the avatar and the arrow-key press translated to movement in the direction of the given key. They were instructed that the up-arrow (forward motion) was the least likely to lead to simulator sickness. Participants interacted with 3D objects via mouse clicks, and used headphones with a built-in microphone to hear the stimuli and communicate with experimenters. All graphics were displayed on a 27” LED monitor.

“Fairyland Garden” was a fantasy-fiction type context that was bright, verdant, visually open, and expansive. This context’s landscape was rich with water and trees, the buildings were wooden, every room was opened to the outdoors, with birdsongs, crickets, and nature-based ambient sounds (Fig. 2a ). “Moon Base,” on the other hand, was a science-fiction type context in which participants were confined indoors within the base, whose structure featured metallic walls, narrow hallways, electronic control panels, artificial colours, mechanical ambient sounds, and participants were always confined indoors (Fig. 2c ). Each context contained nine named areas (hence, “rooms”); the names of each room were displayed in English on signs at the boundaries.

The VR-based contexts displayed different experimental objects during the context encoding phase and language encoding phase. During context encoding, location markers were placed in each room to demarcate the location for participants to “stand” as they encoded the context. During language encoding, interactive 3-D objects representative of the to-be-learnt words were placed on “pedestals” in each room, organised along a hinted floor path that displayed transient markers between pedestals (Fig. 2b , d ).

An additional VR environment (Fig. 1a.1 , 1a.2 ) was used for participants to learn to control their avatars, receive task instructions, and practice the Context Encoding Task and the Language Encoding Task. This training environment was underwater in honour of one of the pioneering demonstrations of context-dependent memory 4 . It was designed to be visually attractive and highly fantastical (e.g., swimming fishes, shifting lights), so as to allow participants time to adjust to the other-worldly nature of VR experience. This aimed to allow participants to focus on the learning tasks without being distracted by the novelty of the VR experience itself.

These desktop-VR-based contexts were created for this study using the open source OpenSimulator platform (v0.8.2.1, Diva Distribution). Firestorm Viewer v4.4.2-v5.0.7 (2014–2017) rendered content, presented on a computer running Windows 7 Professional. A high-resolution (2560 x 1440) flatscreen display, which participants viewed in close proximity in a darkened room, was used instead of a head-mounted display (HMD). Our initial piloting with an HMD (Oculus RIFT DK1) found that many participants experienced eventual motion sickness that interfered with their ability to concentrate on the task. Switching to an LED monitor (often referred to as “desktop VR”) largely ameliorated this issue, although this may have led to some of our participants reporting a limited sense of “presence” in the VR worlds.

During the VR tasks, an experimenter was present to monitor the behaviour of the participant and to communicate with the participant over headphones. While experimenter and participant were in same room, they were separated by cubicle wall such that they were out of sight from one another.

Word list, cues, and testing

The to-be-learnt word lists were designed to be as similar, and thus as confusable, as possible. A total of 60 English words, and their translations in two phonetically similar Bantu languages—Swahili and Chinyanja—were used in the experiment. Each participant learnt to pronounce altogether 80 foreign words: 10 learnt in Swahili only, 10 in Chinyanja only, 30 in both languages. The Swahili word list was drawn from Carpenter & Olson (2012) 27 , and the Chinyanja versions of these words were translated using Google Translate™ and modified (see Appendix I. for the word lists and details regarding the modifications).

Audio stimuli for language learning and testing

During language encoding, audio recordings of the foreign words accompanied their written form. These recordings were pronounced by a single speaker who had no formal training with Bantu languages (J.K.-Y.E.). This was an intentional decision to ensure the foreign words were readily pronounceable by English speakers, as this experiment prioritised the memory aspect of the task over the degree of linguistic authenticity.

As Smith, Glenberg, and Bjork (1978) 5 found that experimenters constituted part of the learning contexts, we took precautions to prevent uncontrolled context reinstatement by virtue of subject-experimenter interactions. First, a single speaker recorded audio for both languages during the learning task—to ensure that speaker identity or voice would not serve as context cues between the languages. Every attempt was made by this speaker to not speak to participants during experimental procedures—only providing supervision for the study team in a separate office during the behavioural experiment procedure, and in the fMRI experiment, greeting participants by gestures, then managing equipment in the MRI control room (when asked, participant-facing researchers explained that this person was not to speak to them for scientific reasons, and that the team can answer questions on this matter at the end of their participation). Second, tests that were conducted outside of the learning contexts were cued by other speakers. The English audio cues used in T4 were recorded by A.O., and T5 was conducted by a team of research assistants.

Testing software

The short-delay non-VR test (T4; Fig. 4 ) was presented using PsychoPy2 44 , 45 . The long-delay surprise memory test was administered over telephone calls using Google’s Hangouts™ communication platform (audio-only), digitally recorded with participant permission, with foreign vocabulary recall cued conversationally by experimenters.

fMRI protocol and in-scanner verbal response recording

Fmri protocol.

fMRI data were collected with a Siemens 3.0 Tesla Magnetom Prisma scanner at the UCLA Ahmanson-Lovelace Brain Mapping Center, using a 64-channel head coil. Functional data were acquired using T 2 *-weighted simultaneous multislice echoplanar imaging (EPI) sequences (TR = 1.0 s; TE = 30 ms; flip angle = 52°; FoV = 20.8 cm; multiband acceleration factor = 5; 65 oblique axial slices; voxel resolution 2 × 2 × 2 mm). Each of the 10 runs consisted of 330 volumes and included eight trials of the task (we did not discard initial volumes as the version of Syngo software did not begin recording until T1 stabilised). Additionally, a T1-weighted structural MRI [axial magnetisation-prepared rapid gradient-echo (MPRAGE), 0.8 mm 3 ] was obtained for spatial registration of the functional data.

Auditory stimuli were presented via OptoActive™ noise cancelling headphones, which were equipped with the FOMRI III™ + microphone (Fig. 1c ) to record participants’ verbal responses during fMRI scans. This system provided online noise cancellation, which enabled high-quality recordings of participants’ vocalisations and allowed participants to clearly hear the audio stimuli despite the scanner noise. No post-experimental denoising of the verbal response was required. Button responses were recorded via CurrentDesign Fibre Optic Response Pads, an MR-compatible button box device. MR-compatible goggles were used to for visual presentations.

Procedure: day 1 and day 2, context and language encoding (T1–T3)

Familiarisation, instructions, and practice.

After informed consent and general instructions, participants “entered” the introductory VR environment. Therein, participants first familiarised themselves with the navigational controls. They then received instructions for the context- and language encoding tasks by watching a video on a screen within the world (Fig. 1a.1 , Supplementary Video 2 ), and practiced the two tasks (Fig. 1a.2 ) under the supervision of an experimenter, who provided corrective feedback to ensure that participants had proper understanding of the tasks. Participants practiced the context encoding task (see below) by performing it in the practice context. Then they practiced the language encoding task by learning the translations of a set of practice items in the pseudo-language ‘Pig Latin’.

Context A encoding (Fig. 1a.3 )

Participants were then “teleported” to Context A (Moon Base or Fairyland Garden, counterbalanced across participants), where they performed a guided encoding task of the VR-based context itself. Each context contained 9 “rooms,” each equipped with a location marker. In each room, participants were instructed to walk to the marker and do two full clock-wise rotations (720°) within 30 s while looking around the room. Participants were instructed to pretend that they were a tourist who had forgotten their camera and that they should try to remember what it felt like to be in that particular place. As participants entered and exited each room, the experimenter informed participants the names of the rooms (e.g., “You are now leaving Sickbay and entering Airlock.”).

Language 1 encoding (T1–T2; Fig. 1a.4 , Supplementary Video 1 )

There were four rounds of language encoding for each language (three rounds on Day 1, and one on Day 2). Before each round, participants were told which language they would be learning. After Context A encoding and a mandatory 2-min break, participants re-entered Context A for Round 1 of Language 1 encoding (Swahili or Chinyanja, counterbalanced across participants).

In each round, participants navigated along the hinted walking path (Fig. 2b , d ) and encountered a series of 40 pedestals (with 3–5 pedestals in each room). Upon each pedestal hovered a slowly rotating, 3-D object representation of the to-be-learnt word (e.g., a rooster), with its English name floating above to ensure that participants could have certainty about what that object was (i.e., so they knew it was not a hen or turkey). As Fig. 2e denotes, participants were instructed to walk up to each object, read its English name aloud, and then to click on it. The click changed the floating English text to reveal the foreign transliteration, and participants would hear the foreign pronunciation three times via headphones, evenly spaced across 10 s. Participants were instructed to repeat after the audio each time by pronouncing the foreign word aloud. Upon completion, they would then click the pedestal to reveal a visible path marking the way to the next pedestal with the next object. The path hints were transient and disappeared after use. Object sequences were controlled so that they were consistent within each language. That is, for a given participant, the same object always appeared in the same location for one language, but always in a different location for the other language. The pedestal locations and navigational route remained consistent across all rounds. A 5-min break was inserted between Rounds 2 and 3.

Retrieval practice (Fig. 2e.2 )

Retrieval practice was incorporated into Rounds 2–4. During Rounds 2–4, after participants walked up to each object and spoke aloud its English name, they were to first attempt to verbally recall its foreign translation before clicking the object. If the participant did not recall the translation and did not wish to attempt a guess, they had the option to say “pass.” They then clicked the object, which triggered the transliteration of the foreign word to the appear and the audio of its pronunciation to be played. Thus, regardless of whether they were correct, incorrect, or passed, the participant received feedback as to the correct answer. Then, as with Round 1, participants heard and repeated after the audio three times within a 10 s period. Participants’ verbal responses were digitally recorded and used to index their memory recall ability during each round, with performance summarised as: T1 (recall during Round 2 before the 2nd encoding), T2 (recall during Round 3 before the 3rd encoding), and T3 (recall after an overnight delay, before the 4th and the final encoding). In the rare cases when participants neglected to attempt recall or say “pass” before clicking an object, the associated vocabulary words were dropped from analysis after that time point. For example, consider a participant who clicked the 3-D boat object during Round 3 before attempting to recall the Swahili word for “boat.” Even though the participant would continue to encounter the boat in Round 4 to maintain consistency across participants, that word would be excluded in analyses of that participant’s T3, T4, and T5 data.

Context B encoding (Fig. 1a.5 )

After Round 3 of Language 1 encoding, participants encoded Context B. The procedure was identical to Context A encoding, except it occurred in the other VR-based context. This was followed by a 5-min break.

Language 2 encoding (T1–T2; Fig. 1a.6 )

After the break, participants began Language 2 encoding. This is the only portion of the procedures in which the experiences of the two context groups diverged. Dual-context participants remained in Context B to encode Language 2, while single-context participants were teleported back to Context A to encode Language 2 (note that single-context participants never learnt any language in Context B). The encoding procedure was identical to Language 1 encoding.

Post-VR questionnaires

Thereafter, participants completed on REDCap 43 a presence scale used in a prior study 19 , an immersion survey (this survey was not used in the analysis) 18 , 46 , the Simulator Sickness Questionnaire 47 , and the Pittsburgh Sleep Quality Index 48 . They were then reminded of their appointment the next day, and sent home.

Participants returned the next day around the same time of day to perform Language 1 Encoding Round 4 (T3). Then, following a 2-min break, participants performed Language 2 Encoding Round 4 (T3). Round 4 was participants’ last exposure to the foreign words and VR contexts.

Procedure: day 2, short-delay, non-VR testing (T4)

Language encoding was followed by a 10-min break (behavioural experiment) or 30-min break (fMRI experiment), after which participants were tested for the first time outside of the VR-based learning contexts (T4), either in the lab (behavioural experiment) or in the MRI scanner (fMRI experiment). During the break, participants in the behavioural experiment were unoccupied for 10 min under supervision, seated in a waiting room without using internet-capable devices. A 30-min interval was scheduled for participants in the fMRI experiment. During this time, each participant was escorted by their experimenter to the Ahmanson-Lovelace Brain Mapping Center (an 8-min walk from the laboratory), underwent final MRI safety screening, and was set up in the MRI scanner.

T4 consisted of 80 trials (one for each foreign word learnt) evenly divided into 10 runs. Each trial (Fig. 4 ) consisted of the following periods: “Ready” screen, mental reinstatement, language recall, imagery vividness rating, and two trials of an arithmetic task that served as active baseline for fMRI data analysis. T4 procedures were identical in the behavioural and fMRI experiments.

Ready (1 s)

A grey screen with the words “Get Ready” printed was presented to mark the beginning of each trial.

Mental reinstatement (10 s)

The mental reinstatement period began with an audio cue for each trial, which stated the name of a VR-based context, followed by that of a room therein (e.g., “Moon Base: Airlock”). Following the audio cue, the screen turned black, and based on instructions provided to the participants before the scan, they knew that this meant that they should close their eyes, imagine themselves back in that specific room, and mentally perform the full rotations (as they had practiced the prior day in the VR-based context encoding task) until they heard a beep. Participant used a series of button presses to indicate the progress of their imagined rotation: mentally “placed” themselves on the marker, rotated 180°, 360°, 540° and so on. If participants completed a full rotation before the allotted time, they were instructed to continue mentally rotating and button-pushing until the beep. Upon hearing the beep, which sounded 10 s after audio cue offset, participants were to cease performing the mental rotation task and open their eyes to prepare for the next phase of the trial.

In the congruent reinstatement condition, participants were cued to reinstate the specific room in which they had learnt the word to be recalled later in this trial. In the incongruent condition, they were cued to reinstate a room from the other context (for dual-context participants, this was the context where they had learnt the other language; for single-context participants, this was the context where they had not encoded any language). These conditions were pseudo-randomly intermixed.

Language recall (8 s)

The language recall period began 2 s after the onset of the previous beep. Participants first heard an audio cue, which stated a language, then an English word whose translation they had learnt in the stated language (e.g., “Chinyanja: rooster”). After hearing the cue, participants were to covertly retrieve the English word’s translation in the cued language (i.e., to mentally recall the foreign word without saying it aloud). If they felt they were successful, they were to push Button 1 and to continue thinking about the word until they heard a beep. If they failed to retrieve the foreign word, they were to push Button 2 and continue to attempt retrieval until the beep—should they succeed at any point after indicating failure, they were to push Button 1 at the moment of successful retrieval. The beep sounded 8 s after the cue offset, at which point participants were to verbally pronounce the foreign word, or as much of it as they could remember. These responses were recorded and scored as T4 data. The length of the verbal response recording period varied between 6.5–7.0 s depending on the length of the cue (3.0–3.5 s), so that the combined duration of the two always summed to 10 s.

Imagery vividness rating (2 s)

After verbal recall, participants were then asked to rate how vivid the previous mental reinstatement had been (1 for very vivid, 2 for vivid, 3 for not vivid, and 4 for unsuccessful). These ratings were later used for trial exclusion during the analyses involving mental reinstatement.

Arithmetic task (5 s)

At the end of each trial, participants performed an arithmetic task. Participants saw a display (2.5 s) with two single-digit integers, and they were to push Button 1 if the product of these numbers was odd, and Button 2 if even. Then a new pair of digits appeared (2.5 s) and participants performed the same task.

Procedure: day 2, post-experimental survey

After T4, participants completed a short survey to ask them about what strategies (if any) they had implemented to learn and recall the words, and if there was anything else they would like to communicate to the experimenters.

Procedure: day 8, one-week delay, surprise testing (T5)

On Day 8, participants were telephoned for a scheduled “follow-up interview” with the understanding that an experimenter would “ask them about things they had experienced in the VR.” The only instruction they received about the phone call was that they were to be at home, seated in a quiet place. Participants were not informed that they would be tested again.

During the call, the experimenter requested permission to record the participant’s responses. After permission was granted, experimenter asked the following questions: (1) Had they looked up or studied any of the Swahili or Chinyanja words during the preceding week? (2) Had they expected to be tested again? (3) What percentage of the words did they expect to recall? (see Supplementary Note 3 ).

The experimenter then conducted a cued recall test to test participants’ memory for all 80 of the foreign words they had learnt. On each trial, the experimenter cued the participant with an English word and a language that it was to be translated into (e.g., “How do you say ‘cherry’ in Swahili?”). The order in which the words are tested was fully randomised, such that testing hopped back and forth between the two foreign languages. Participants’ vocal responses were recorded and scored as T5 data.

Language test scoring

Digital recordings of the verbal responses from T1–T5 were scored offline by two scorers. The score for each word was the number of correct phonemes divided by the number of total phonemes. Scorers were trained to use a detailed decision tree, and when the two scorers disagreed, the average between the two scores was used as the final recall score for that word. The partial word score was used to provide more fine-grained results than binary (correct vs incorrect) word recall. In this scoring scheme, phonemes in shorter words were weighed more heavily than phonemes in longer words. This weighting mirrors the consequences of phonemic errors in real-world communication. When one mistakenly places, for instance, a “P” instead of an “V” in the word “van” it tends to be more consequential than in a longer word like “supervisor,” and a lot more difficult for the listeners to guess the intended meaning.

Retention measures

Retention was measured inversely via a forgetting score between two tests. Overnight retention (reported in Supplementary Note 1: A4 ) was computed based on the difference between T3 and T2. One-week retention was computed based on the difference between T5 and T4.

Forgetting score

The forgetting score was computed as follows: First, an item-wise forgetting index was computed for each word with a non-zero score in the earlier test (i.e., if no phonemes were recalled in T4, the word was excluded from this computation for one-week forgetting). These forgetting indices measured loss between the two tests: a negative forgetting index would mean the word was recalled worse after one-week, and a forgetting index of zero would mean no forgetting, thus perfect one-week retention. For example, consider a word that had a recall score of 1 (full, correct recall) on T4, but only 0.5 (half of the phonemes were missing or incorrect) in T5. It would receive a “−0.5” on the forgetting index, indicating half of the word had been forgotten. On the other hand, if a word had a score of 1 on both T4 and T5, it would receive a “0” on the forgetting index, indicating perfect retention. These forgetting indices were then averaged within each participant (across all eligible words) to produce a forgetting score. The forgetting score was a metric of forgetting, or the inverse of retention—the more negative the score, the more forgetting and thus the poorer retention.

Retention score

For the ease of interpretation, a positive retention score was computed by 1 minus averaged forgetting score. In which 1 indicates perfect retention across all eligible words, 0.5 indicates half of the information was retained, while 0 means no information were retained.

Intrusion measure

When scoring T4 and T5, scorers were instructed to compare the transliteration of each word to its counterpart in the other language, and to determine from experience whether the word in question was similar to any other words in either language (see Appendix II for intrusion coding). The scorers were experimenters who became highly familiar with the words in both languages. In addition to formal training, scorers spent 2–6 h each week monitoring participants during language encoding, testing participants during T5, or scoring verbal response offline. Despite this, “similarity” between words remains arbitrary and experience-based. Therefore, two cautions were introduced: a newer scorer was always paired with a very experienced one in the scoring assignments, and the maximum code was used when the scorers disagreed—as the higher ratings denote more severe intrusions, and preliminary examination revealed that novice scorers tend to underrate intrusion rather than overrate them.

Behavioural data analysis

Multiple statistical tests were conducted using SPSS 26.0 49 . The between-subject factors were Context Group (single- vs . dual-context) and Presence (high- vs . low-presence, a mean-split grouping using the presence scale 19 ). The within-subject factors were Times (T1–T5), Language Order (Language 1 vs 2; not reported, see Supplementary Note 1: A1 ), and Reinstatement (congruent vs . incongruent reinstatement). The dependent variables were intrusions (number of items coded to be intrusions from the opposite language, out of a total of 80 items), recall (mean of item-wise percentage phonemes correct for a given test), and retention (see Retention Score above).

fMRI data analysis

Fmri pre-processing.

Functional data were pre-processed without spatial smoothing, pre-whitening, nor B0 unwarping using the FMRI Software Library 5.0.4 and Advanced Normalisation Tools (ANTS 2.0) 50 . FSL Brain Extraction Tool (BET2) 51 was used to perform brain extraction. FSL 52 FEAT 53 was used to apply a high-pass temporal filter (128 Hz). Timeseries alignment, motion correction, and registration to standard Montreal Neurological Institute (MNI) template was performed using FMRIB’s Linear Image Registration Tool (FLIRT) 54 , 55 , 56 , Motion Correction FLIRT (MCFLIRT) 54 , and ANTS.

fMRI task timing and trial categorisation

The mental reinstatement (Fig. 4 “Imagery”) and language retrieval (Fig. 4 “Language”) periods from each trial were extracted from the dataset. The BOLD timeseries for these periods were extracted using the adjusted onset and offset times (5 s, i.e., 5 TRs, were added to onsets and offsets to account for the lagging hemodynamic response, or HDR). The resulting truncated timeseries was then temporally averaged at each voxel, yielding one averaged imagery pattern and one averaged language pattern for each trial.

Each “Imagery” period began when participants indicated that they had mentally “placed” themselves in the to-be-reinstated context via a button push (Fig. 4 “Orient”), and end at the beep onset (the beep which informed participants to open their eyes and end mental reinstatement). The onset for each trial was based on participants’ responses, thus the imagery period duration varied in length. Imagery period data were labelled as Moon base or Fairyland Garden, based on the world that participants were cued to reinstate. Trials were excluded if participants reported they were “unsuccessful” during the imagery rating portion, or did not push buttons to report mental reinstatement rotation progress.

Each “Language” period began with the onset of the audio cue, and ended 6 s afterwards. The duration of this period was task-based, and fixed in length. Language period data were labelled by the foreign word to be recalled (e.g., Chinyanja: Dress).

Searchlight multi-voxel pattern analysis (SL-MVPA)

A SL-MVPA was conducted using the Imagery patterns to identify regions in the brain that expressed multivariate patterns of activity capable of discriminating between a participant’s mental reinstatement of Moon Base vs. Fairyland Garden (Supplementary Fig. 1 ). To this end, we employed a support vector machine (SVM) classifier with a linear kernel using libSVM (c-SVC, c = 1) 57 and a whole-brain searchlight mapping approach (radius = 4 voxels). Classification was cross-validated using a leave-one-run-out method—the classifier was trained on valid trials from 9 runs (9 × 8 trials), and tested on the valid trials from the left-out run (8 trials). Trial labels were balanced prior to classification by randomly sampling from the overrepresented trials to match the underrepresented trial types. The entire cross-validation procedure was repeated over 10 iterations (one for each run) and the classification results were averaged. This produced a brain map whose voxel values reflected the classifier’s cross-validation accuracy when the searchlight sphere was centred on that voxel (Supplementary Fig. 1.4 ). The top 2000 voxels with the highest classification accuracies were identified for each participant, and used to create a distributed region of interest for the subsequent representational similarity analysis as a within-subject feature selection (Supplementary Fig. 1.5 ).

Representational similarity analysis (RSA)

For each word that each participant had learnt, the RSA produced a value of similarity between (1) the brain response pattern when the participant was recalling this word, and (2) the averaged brain response pattern when the participant was mentally reinstating that word’s learning context (Fig. 5a ).

This within-subject RSA was conducted using custom MATLAB code. First, trial-specific imagery and language patterns (produced by the aforementioned temporal average of HDR-adjusted timeseries within trial period) for each participant were masked using the participant’s top 2000 voxels identified in the SL-MVPA. Second, the imagery patterns for each learning context were averaged within-subject to produce a participant-specific mental reinstatement template for Moon Base and Fairyland Garden. Third, the language pattern for each word was then correlated (Pearson’s r ) with the reinstatement template of its learning context. For instance, consider a participant who had learnt “banana” in Chinyanja in Fairyland Garden. The language period during the covert retrieval of the word “banana” in Chinyanja would be correlated with the Fairyland Garden template—an average of all imagery patterns during the mental reinstatement of Fairyland Garden. Fourth, the resultant r -values were Fisher transformed to normally distributed z -values to allow for comparison across trial-types. Lastly, a mean split was performed on the z -values to categorise each trial as either a high-fidelity reinstatement trial or a low-fidelity reinstatement trial to analyse the verbal response data.

Repeated measure analysis of variance (RM-ANOVA)

A 2 × 2 × 2 × 2 RM-MANOVA was performed on with the factors Times (T4, T5) × Reinstatement instructions (congruent vs incongruent) × RSA (high- vs low-RSA) × Presence (high- vs low-presence) on recall using SPSS 26.0 49 . The dependent variables were proportion syllables recalled during T4 (short-delay recall in the MRI scanner) and T5 (one-week-delayed recall over the telephone).

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

De-identified data available from the corresponding author, upon request.

Code availability

The MATLAB scripts used for fMRI data preprocessing and statistical analysis are available from the corresponding author, upon request.

Smith, S. M. & Vela, E. Environmental context-dependent memory: a review and meta-analysis. Psychon. Bull. Rev. 8 , 203–220 (2001).

Article   CAS   Google Scholar  

Smith, S. M. Remembering in and out of context. J. Exp. Psychol.: Hum. Learn. Mem. 5 , 460 (1979).

Google Scholar  

Tulving, E. & Thomson, D. M. Encoding specificity and retrieval processes in episodic memory. Psychological Rev. 80 , 352–373 (1973).

Article   Google Scholar  

Godden, D. R. & Baddeley, A. D. Context‐dependent memory in two natural environments: on land and underwater. Br. J. Psychol. 66 , 325–331 (1975).

Smith, S. M., Glenberg, A. & Bjork, R. A. Environmental context and human memory. Behav. Res. Methods, Mem. Cognition, Mem. Cognition. 6 , 342–353 (1978).

Grant, H. M. et al. Context-dependent memory for meaningful material: information for students. Appl. Cogn. Psychol. 12 , 617–623 (1998).

Godden, D. & Baddeley, A. When does context influence recognition memory? Br. J. Psychol. 71 , 99–104 (1980).

Smith, S. M. Effects of environmental context on human memory. The SAGE Handbook of Applied Memory 162 (2013).

Smith, S. M. & Handy, J. D. Effects of varied and constant environmental contexts on acquisition and retention. J. Exp. Psychol.: Learn., Mem., Cognition 40 , 1582–1593 (2014).

Bjork, R. A. & Richardson-Klavehn, A. On the puzzling relationship between environmental context and human memory in Current issues in cognitive processes : The Tulane Flowerree Symposium on Cognition. (ed. Izawa, C.) 313–344 (Erlbaum, 1989).

Smith, S. M. & Handy, J. D. The crutch of context-dependency: effects of contextual support and constancy on acquisition and retention. Memory 1–8 https://doi.org/10.1080/09658211.2015.1071852 (2015).

Bramão, I., Karlsson, A. & Johansson, M. Mental reinstatement of encoding context improves episodic remembering. Cortex 94 , 15–26 (2017).

Wang, W.-C., Yonelinas, A. P. & Ranganath, C. Dissociable neural correlates of item and context retrieval in the medial temporal lobes. Behavioural Brain Res. 254 , 102–107 (2013).

Smith, S. M., Handy, J. D., Angello, G. & Manzano, I. Effects of similarity on environmental context cueing. Memory 22 , 493–508 (2014).

Reggente, N. et al. Enhancing the ecological validity of fMRI memory research using virtual reality. Front. Neurosci . 12 , 408 (2018).

Smith, S. A. Virtual reality in episodic memory research: a review. Psychon. Bull. Rev. 26 , 1213–1237 (2019).

Shin, Y. S., Masís-Obando, R., Keshavarzian, N., Dáve, R. & Norman, K. A. Context-dependent memory effects in two immersive virtual reality environments: on Mars and underwater. Psychon. Bull. Rev. 28 , 574–582 (2021).

Slater, M., Usoh, M. & Steed, A. Depth of presence in virtual environments. Presence.: Teleoperators Virtual Environ. 3 , 130–144 (1994).

Fox, J., Bailenson, J. & Binney, J. Virtual experiences, physical behaviors: The effect of presence on imitation of an eating avatar. Presence.: Teleoperators Virtual Environ. 18 , 294–303 (2009).

Bowman, D. A. & McMahan, R. P. Virtual reality: how much immersion is enough? Computer 40 , 36–43 (2007).

Sanchez-Vives, M. V. & Slater, M. From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6 , 332–339 (2005).

Kroll, J. F. & Stewart, E. Category interference in translation and picture naming: evidence for asymmetric connections between bilingual memory representations. J. Mem. Lang. 33 , 149–174 (1994).

Rissman, J. & Wagner, A. D. Distributed representations in memory: insights from functional brain imaging. Annu Rev. Psychol. 63 , 101–128 (2012).

Levy, B. J. & Wagner, A. D. Measuring memory reactivation with functional MRI: implications for psychological theory. Perspect. Psychol. Sci. 8 , 72–78 (2013).

Bjork, R. A. & Bjork, E. L. Desirable difficulties in theory and practice. J. Appl. Res. Mem. Cognition 9 , 475–479 (2020).

Storm, B. C., Bjork, R. A. & Storm, J. C. Optimizing retrieval as a learning event: When and why expanding retrieval practice enhances long-term retention. Mem. Cognition 38 , 244–253 (2010).

Carpenter, S. K. & Olson, K. M. Are pictures good for learning new vocabulary in a foreign language? Only if you think they are not. J. Exp. Psychol.: Learn., Mem., Cognition 38 , 92–101 (2012).

Lamers, M. H. & Lanen, M. Changing between virtual reality and real-world adversely affects memory recall accuracy. Front. Virtual Real. 2 , 602087 (2021).

Niki, K. et al. Immersive virtual reality reminiscence reduces anxiety in the oldest-old without causing serious side effects: a single-center, pilot, and randomized crossover study. Front Hum. Neurosci. 14 , 598161 (2021).

deBettencourt, M. T., Turk-Browne, N. B. & Norman, K. A. Neurofeedback helps to reveal a relationship between context reinstatement and memory retrieval. NeuroImage 200 , 292–301 (2019).

Robin, J. Spatial scaffold effects in event memory and imagination. WIREs Cogn. Sci. 9 , e1462 (2018).

Robin, J., Buchsbaum, B. R. & Moscovitch, M. The primacy of spatial context in the neural representation of events. J. Neurosci. 38 , 2755–2765 (2018).

Herweg, N. A. et al. Reactivated spatial context guides episodic recall. J. Neurosci. 40 , 2119–2128 (2020).

Robin, J., Wynn, J. & Moscovitch, M. The spatial scaffold: The effects of spatial context on memory for events. J. Exp. Psychol.: Learn., Mem., Cognition 42 , 308–315 (2016).

Reggente, N., Essoe, J. K. Y., Baek, H. Y. & Rissman, J. The method of loci in virtual reality: explicit binding of objects to spatial contexts enhances subsequent memory recall. J. Cogn. Enhanc. 4 , 12–30 (2020).

Kyle, C. T., Stokes, J. D., Lieberman, J. S., Hassan, A. S. & Ekstrom, A. D. Successful retrieval of competing spatial environments in humans involves hippocampal pattern separation mechanisms. Elife 4 , e10499 (2015).

Schomaker, J., van Bronkhorst, M. L. V. & Meeter, M. Exploring a novel environment improves motivation and promotes recall of words. Front. Psychol. 5 , 918 (2014).

Wälti, M. J., Woolley, D. G. & Wenderoth, N. Reinstating verbal memories with virtual contexts: myth or reality? PLOS ONE 14 , e0214540 (2019).

Willems, R. M. & Peelen, M. V. How context changes the neural basis of perception and language. iScience 24 , 102392 (2021).

Makransky, G. & Petersen, G. B. The cognitive affective model of immersive learning (CAMIL): a theoretical research-based model of learning in immersive virtual reality. Educ. Psychol. Rev. 33 , 937–958 (2021).

Makransky, G. et al. Investigating the feasibility of using assessment and explanatory feedback in desktop virtual reality simulations. Educ. Technol. Res. Dev. 68 , 293–317 (2020).

Parong, J. & Mayer, R. E. Learning science in immersive virtual reality. J. Educ. Psychol. 110 , 785–797 (2018).

Harris, P. A. et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 42 , 377–381 (2009).

Peirce, J. W. PsychoPy—psychophysics software in Python. J. Neurosci. Methods 162 , 8–13 (2007).

Peirce, J. W. Generating stimuli for neuroscience using PsychoPy. Front. Neuroinformatics 2 , 10 (2009).

Slater, M., Usoh, M. & Chrysanthou, Y. The influence of dynamic shadows on presence in immersive virtual environments. in Virtual environments’ 95 , 8–21 (Springer, 1995).

Kennedy, R. S., Lane, N. E., Berbaum, K. S. & Lilienthal, M. G. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3 , 203–220 (1993).

Buysse, D. J. et al. Quantification of subjective sleep quality in healthy elderly men and women using the Pittsburgh Sleep Quality Index (PSQI). Sleep 14 , 331–338 (1991).

CAS   Google Scholar  

SPSS, I. IBM SPSS Statistics for Windows, Version 20.0. (IBM Corp Armonk, NY, 2011).

Avants, B. B. et al. A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage 54 , 2033–2044 (2011).

Smith, S. M. Fast robust automated brain extraction. Hum. Brain Mapp. 17 , 143–155 (2002).

Jenkinson, M., Beckmann, C. F., Behrens, T. E., Woolrich, M. W. & Smith, S. M. Fsl. Neuroimage 62 , 782–790 (2012).

Woolrich, M. W., Ripley, B. D., Brady, M. & Smith, S. M. Temporal autocorrelation in univariate linear modeling of FMRI data. NeuroImage 14 , 1370–1386 (2001).

Jenkinson, M., Bannister, P., Brady, M. & Smith, S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17 , 825–841 (2002).

Jenkinson, M. & Smith, S. A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5 , 143–156 (2001).

Greve, D. N. & Fischl, B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage 48 , 63–72 (2009).

Chang, C.-C. & Lin, C.-J. LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2 , 1–27 (2011).

Download references

Acknowledgements

The authors gratefully acknowledge the funding agencies and the following individuals for their contribution to this manuscript: research assistant team (Priyanka Mehta, Alvin T. Vuong, Jacob Yu Villa, Gabriel Hughes, Alana Sanchez-Prak, Ruwanthi Ekanayake, and Hugo Shiboski, Daniel Lin); J.K-Y.E.’s dissertation committee (Drs. Elizabeth L. Bjork, Robert A. Bjork, and Kimberley Gomez) for valuable theoretical input; Forde “JubJub” Davidson for help with VR content development and custom functionality; the OpenSim community for VR content published under CC licensing; Andrew E. Silva, Ph.D. for data analysis advice; Joseph F. McGuire, Ph.D. and Joshua M. Essoe for manuscript editing. This work was supported by a Defense Advanced Research Project Agency (DARPA) Research Grant awarded to J.R. (D13AP00057) and National Science Foundation (NSF) Graduate Research Fellowships awarded to J.K-Y.E. (DGE-1144087), N.R. (DGE-1650604), and J.D. (DGE-1144087).

Author information

Authors and affiliations.

Center for OCD, Anxiety, and Related Disorders for Children, Division of Child and Adolescent Psychiatry, Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University School of Medicine, Baltimore, MD, 21205, USA

Joey Ka-Yee Essoe

Department of Psychology, University of California, Los Angeles, CA, 90095, USA

Joey Ka-Yee Essoe, Nicco Reggente, Ai Aileen Ohno, Younji Hera Baek, John Dell’Italia & Jesse Rissman

Institute for Advanced Consciousness Studies, Santa Monica, CA, 90403, USA

Nicco Reggente

School of Medicine, California University of Science and Medicine, Colton, CA, 92324, USA

Ai Aileen Ohno

Division of Psychology, Communication, and Human Neuroscience, School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, M13 9PL, UK

Younji Hera Baek

Birmingham Veterans Affairs, Birmingham, AL, 35233, USA

John Dell’Italia

Department of Psychiatry & Biobehavioral Sciences, University of California, Los Angeles, CA, 90095, USA

Jesse Rissman

Brain Research Institute, University of California, Los Angeles, CA, 90095, USA

Integrative Center for Learning and Memory, University of California, Los Angeles, CA, 90095, USA

You can also search for this author in PubMed   Google Scholar

Contributions

J.K.-Y.E. and J.R. conceived the study idea. J.K.-Y.E., J.R., N.R. designed the study. J.K.-Y.E. created and programmed the VR-based contexts, scripted and managed data collection. A.A.O. coordinated the experiment and contributed to RA team management. A.A.O., Y.H.B., and RAs collected and scored the behavioural data. J.K.-Y.E., A.A.O., Y.H.B., and N.R. collected the fMRI data. J.K.-Y.E. analysed the behavioural data. J.K.-Y.E., N.R., and J.D. pre-processed the fMRI data. N.R. and J.K.-Y.E. analysed the fMRI data. J.R. and J.D. advised on fMRI data analyses. J.K.-Y.E., J.R., and N.R. wrote the manuscript. All authors read and revised the manuscript and provided critical intellectual contributions.

Corresponding author

Correspondence to Jesse Rissman .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Demonstration of language encoding task, day 1 instructional video for participant, revised supplemental material, reporting summary checklist, supplementary videos legends, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Essoe, J.KY., Reggente, N., Ohno, A.A. et al. Enhancing learning and retention with distinctive virtual reality environments and mental context reinstatement. npj Sci. Learn. 7 , 31 (2022). https://doi.org/10.1038/s41539-022-00147-6

Download citation

Received : 14 May 2022

Accepted : 21 October 2022

Published : 08 December 2022

DOI : https://doi.org/10.1038/s41539-022-00147-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Stand in surgeon’s shoes: virtual reality cross-training to enhance teamwork in surgery.

• Benjamin D. Killeen
• Mathias Unberath

International Journal of Computer Assisted Radiology and Surgery (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Virtual, mixed, and augmented reality: a systematic review for immersive systems research

• Original Article
• Published: 03 January 2021
• Volume 25 , pages 773–799, ( 2021 )

Cite this article

• Matthew J. Liberatore   ORCID: orcid.org/0000-0002-5741-6723 1 &
• William P. Wagner 2  

7469 Accesses

55 Citations

1 Altmetric

Explore all metrics

Immersive systems can be used to capture new data, create new experiences, and provide new insights by generating virtual elements of physical and imagined worlds. Immersive systems are seeing increased application across a broad array of fields. However, in many situations it is unknown if an immersive application performs as well or better than the existing application in accomplishing a specific task. The purpose of this study is to conduct a systematic review of the literature that addresses the performance of immersive systems. This review assesses those applications where experiments, tests, or clinical trials have been performed to evaluate the proposed application. This research addresses a broad range of application areas and considers studies that compared one or more immersive systems with a control group or evaluated performance data for the immersive system pre- and post-test. The results identify those applications that have been successfully tested and also delineate areas of future research where more data may be needed to assess the effectiveness of proposed applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Augmented Reality: A Comprehensive Review

Virtual Reality Therapy in Social Anxiety Disorder

Eye Tracking in Virtual Reality: a Broad Review of Applications and Challenges

Availability of data and material.

Results of systematic literature search are available.

Code availability

Code used for literature search included as “ Appendix .”

Abdullah M, Shaikh ZA (2018) An effective virtual reality based remedy for acrophobia. Int J Adv Comput Sci Appl 9(6):162–167

Google Scholar  

Anstadt S, Bradley S, Burnett A (2013) Virtual worlds: relationship between real life and experience in second life. Int Rev Res Open Distance Learn 14(4):160–190

Article   Google Scholar  

Azuma R (1997) A survey of augmented reality. Presence: Teleoper Virtual Environ 6(4):355–385

Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, Macintyre B (2001) Recent advances in augmented reality. IEEE Comput Graphics Appl 21:34–47

Blum J, Rockstroh C, Göritz AS (2019) Heart rate variability biofeedback based on slow-paced breathing with immersive virtual reality nature scenery. Front Psychol 10:2172

Boas Y (2013) Overview of virtual reality technologies. School of Electronics and Computer Science, University of Southampton, Southampton, UK. http://static1.squarespace.com/static/537bd8c9e4b0c89881877356/t/5383bc16e4b0bc0d91a758a6/1401142294892/yavb1g12_25879847_finalpaper.pdf . Accessed 1 Aug 2020

Boell S, Cecez-Kecmanovic D (2015) On being ‘systematic’ in literature reviews in IS. J Inf Technol 30(2):161–173

Bogicevica V, Seob S, Kandampullyc J, Liuc S, Rudd N (2019) Virtual reality presence as a preamble of tourism experience: the role of mental imagery. Tour Manag 74:55–64

Bordnick PS, Traylor AC, Carter BL, Graap KM (2012) A feasibility study of virtual reality-based coping skills training for nicotine dependence. Res Soc Work Pract 22(3):293–300

Borsci S, Lawson G, Jha B, Burges M, Salanitri D (2016) Effectiveness of a multidevice 3D virtual environment application to train car service maintenance procedures. Virtual Real 20(1):41–55

Bouzit M, Popescu G, Burdea G, Boian R (2002) The Rutgers master II-ND force feedback glove. In: Proceedings of IEEE VR 2002 haptics symposium, Orlando FL, March, pp 256–263

Bowman D, McMahan R (2007) Virtual reality: how much immersion is enough? Computer 40:36–43

Bramley I, Goode A, Anderson L, Mary E (2018) Researching in-store, at home: using virtual reality within quantitative surveys. Int J Mark Res 60(4):344–351

Cardoş RA, David OA, David DO (2017) Virtual reality exposure therapy in flight anxiety: a quantitative meta-analysis. Comput Hum Behav 72:371–380

Carlson P, Peters A, Gilbert SB, Vance JM, Luse A (2015) Virtual training: learning transfer of assembly tasks. IEEE Trans Visual Comput Gr 21(6):770–782

Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M (2011) Augmented reality technologies, systems and applications. Multimedia Tools Appl 51(1):341–377

Carù A, Cova B (2006) Expériences de marque: comment favoriser l’immersion du consommateur? Décis Mark 41:43–52

Choi S, Cheung H (2008) A versatile virtual prototyping system for rapid product development. Comput Ind 59(5):477–488

Corbetta D, Imeri F, Gatti R (2015) Rehabilitation that incorporates virtual reality is more effective than standard rehabilitation for improving walking speed, balance and mobility after stroke: a systematic review. J Physiother 61(3):117–124

Cruz-Neira C, Sandlikn D, DeFanti T, Kenyon R, Hart J (1992) The CAVE: audio visual experience automatic virtual environment. Commun ACM 35(6):65–72

Cummings J, Bailenson J (2016) How immersive is Enough? A meta-analysis of the effects of immersive technology on user preference. Media Psychol 19:272–309

CyberGlove Systems (2020) CyberGrasp. http://www.cyberglovesystems.com/cybergrasp . Accessed 11 Nov 2020

Czub M, Piskorz J (2018) Body movement reduces pain intensity in virtual reality–based analgesia. Int J Hum-Comput Interact 34(11):1045–1051

de Rooij IJ, van de Port IG, Meijer JWG (2016) Effect of virtual reality training on balance and gait ability in patients with stroke: systematic review and meta-analysis. Phys Ther 96(12):1905–1918

Dobrowolski P, Pochwatko G, Skorko M, Bielecki M (2014) The effects of virtual experience on attitudes toward real brands. Cyberpsychol Behav Soc Netw 17(2):125–128

Dunn J (2017) A visual history of Nintendo’s video game consoles. http://www.businessinsider.com/nintendo-consoles-in-history-photos-switch-2017-1/#before-there-was-the-nes-there-was-the-color-tv-game-nintendo-first-dipped-its-toes-into-console-gaming-by-launching-five-of-these-rectangles-between-1977-and-1980-all-in-its-native-japan-1. Accessed 1 Aug 2020

Ferrer-Garcia M, Pla-Sanjuanelo J, Dakanalis A, Vilalta-Abella F, Riva G, Fernandez-Aranda F, Ribas-Sabaté J (2019) A randomized trial of virtual reality-based cue exposure second-level therapy and cognitive behavior second-level therapy for bulimia nervosa and binge-eating disorder: outcome at six-month follow up. Cyberpsychol Behav Soc Netw 22(1):60–68

Fink A (2005) Conducting research literature reviews: From the Internet to paper. Sage Publications, Thousand Oaks

Flavián C, Ibáñez-Sánchez S, Orús C (2019) Integrating virtual reality devices into the body: effects of technological embodiment on customer engagement and behavioral intentions toward the destination. J Travel Tour Mark 36(7):12. https://doi.org/10.1080/10548408.2019.1618781

Fodor L, Coteț C, Cuijpers P, Szamoskozi Ș, David D, Cristea I (2018) The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: a meta-analysis. Sci Rep 8(1):10323. https://doi.org/10.1038/s41598-018-28113-6

Gerçeker G, Binay Ş, Bilsin E, Kahraman A, Yılmaz H (2018) Effects of virtual reality and external cold and vibration on pain in 7-to 12-year-old children during phlebotomy: a randomized controlled trial. J PeriAnesthesia Nurs 33(6):981–989

Gleasure R, Feller J (2015) A rift in the ground: theorizing the evolution of anchor values in crowdfunding communities through the oculus rift case study. J Assoc Comput Syst 17(1):708–736

Glennon C, McElroy S, Connelly L, Lawson L, Bretches A, Gard A, Newcomer L (2018) Use of virtual reality to distract from pain and anxiety. Oncol Nurs Forum 45(4):545–552. https://doi.org/10.1188/18.ONF.545-552

Glueck A, Han D (2020) Improvement potentials in balance and visuo-motor reaction time after mixed reality action game play: a pilot study. Virt Real 24(2):223–229. https://doi.org/10.1007/s10055-019-00392-y

Gonzalez-Franco M, Pizarro R, Cermeron J, Li K, Thorn J, Hutabarat W, Bermell-Garcia P (2017) Immersive mixed reality for manufacturing training. Frontiers in Robotics and AI, 4. http://journal.frontiersin.org/article/10.3389/frobt.2017.00003/full . Accessed 1 Aug 2020

Gordon NS, Merchant J, Zanbaka C, Hodges LF, Goolkasian P (2011) Interactive gaming reduces experimental pain with or without a head mounted display. Comput Hum Behav 27(6):2123–2128

Gumaa M, Rehan Youssef A (2019) Is virtual reality effective in orthopedic rehabilitation? A systematic review and meta-analysis. Phys Ther 99(10):1304–1325

Guo C, Deng H, Yang J (2015) Effect of virtual reality distraction on pain among patients with hand injury undergoing dressing change. J Clin Nurs 24(1–2):115–120

Huang TL (2019) Psychological mechanisms of brand love and information technology identity in virtual retail environments. J Retail Consum Serv 47:251–264

Igna R, Stefan S, Onac I, Ungur RA, Tatar AS (2014) Mindfulness-based cognitive-behavior therapy (MCBT) versus virtual reality (VR) enhanced CBT, versus treatment as usual for chronic back pain. A clinical trial. J Evid-Based Psychother 14(2):229

igroup.org (2016) igroup presence questionnaire (IPQ) overview. http://www.igroup.org/pq/ipq/index.php Accessed 1 Aug 2020

Israel K, Zerres C, Tscheulin DK (2019) Presenting hotels in virtual reality: does it influence the booking intention? J Hosp Tour Technol 10(3):473–493

Javornik A (2016) Augmented reality: research agenda for studying the impact of its media characteristics on consumer behavior. J Retail Consumer Serv 30:252–261

Jennett C, Cox A, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A (2008) Measuring and defining the experience of immersion in games. Int J Hum Comput Stud 66(9):641–661

Jo D, Kim GJ (2019) IoT + AR: pervasive and augmented environments for “digi-log” shopping experience. Hum-Centric Comput Inf Sci. https://doi.org/10.1186/s13673-018-0162-5

Juan MC, Calatrava J (2011) An augmented reality system for the treatment of phobia to small animals viewed via an optical see-through HMD: comparison with a similar system viewed via a video see-through HMD. Int J Hum–Comput Interact 27(5):436–449

Kalawsky RS (1996) AGOCG Report. Exploiting virtual reality techniques in education and training: technological issues. http://www.agocg.ac.uk/reports/virtual/vrtech/toc.htm Accessed 1 Aug 2020

Karafotias G, Korres G, Teranishi A, Park W, Eid M (2017) Mid-air tactile stimulation for pain distraction. IEEE Trans Haptics 11(2):185–191

Kawulich B, D’Alba A (2019) Teaching qualitative research methods with second life: a 3-dimensional online virtual environment. Virtual Real 23(4):375–384

Kim IC, Lee BH (2012) Effects of augmented reality with functional electric stimulation on muscle strength, balance and gait of stroke patients. J Phys Ther Sci 24(8):755–762

Krumins A (2017) Haptic bodysuits and the strange new landscape of immersive VR. Jan 4, blog entry at https://www.extremetech.com/extreme/241917-haptic-bodysuits-strange-new-landscape-immersive-virtual-reality . Accessed 1 Aug 2020

Ku J, Kim YJ, Cho S, Lim T, Lee HS, Kang YJ (2019) Three-dimensional augmented reality system for balance and mobility rehabilitation in the elderly: a randomized controlled trial. Cyberpsychol Behav Soc Netw 22(2):132–141

Kumar R, Oskiper T, Naroditsky O, Samarasekera S, Zhu Z, Kim J (2017) System and method for generating a mixed reality environment. US Patent No. 9,600,067 B2

Laha B, Sensharma K, Schiffbauer JD, Bowman DA (2012) Effects of immersion on visual analysis of volume data. IEEE Trans Visual Comput Gr 19(4):597–606

Latif U, Shin S (2019) OP-MR: the implementation of order picking based on mixed reality in a smart warehouse. Vis Comput. https://doi.org/10.1007/s00371-019-01745-z

Lau K (2015) Organizational learning goes virtual? A study of employees’ learning achievement in stereoscopic 3D virtual reality. Learn Organ 22(5):289–303

Lee C, Kim Y, Lee B (2014) Augmented reality-based postural control training improves gait function in patients with stroke: randomized controlled trial. Hong Kong Physiother J 32(2):51–57

Lee J, Yoo H, Lee B (2017) Effects of augmented reality-based Otago exercise on balance, gait, and physical factors in elderly women to prevent falls: a randomized controlled trial. J Phys Ther Sci 29(9):1586–1589

Lessiter J, Freeman J, Keogh E, Davidoff J (2001) A cross-media presence questionnaire: the ITC sense of presence inventory. Presence: Teleoper Virtual Environ 10(3):282–297

Li C, Liang W, Quigley C, Zhao Y, Yu L (2017) Earthquake safety training through virtual drills. IEEE Trans Vis Comput Gr 23(4):1275–1284

Liberati N (2013) Improving the embodiment relations by means of phenomenological analysis on the “reality” of ARs. In: 2013 IEEE international symposium on mixed and augmented reality-arts, media, and humanities (ISMAR-AMH) 0, 13–17, 2013. http://doi.ieeecomputersociety.org/10.1109/ISMAR-AMH.2012.6483983

Liberati N (2016) Augmented reality and ubiquitous computing: the hidden potentialities of augmented reality. AI Soc 31(1):17–28

Lima J, McCabe-Bennett H, Antony M (2018) Treatment of storm fears using virtual reality and progressive muscle relaxation. Behav Cognit Psychother 46(2):251–256

Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. J Comput-Med Commun 3(2):1083–6101

Lombard M, Ditton T, Weinstein L (2013) Measuring presence: the temple presence inventory (TPI). Updated September, 15 . http://matthewlombard.com/research/p2_ab.html . Accessed 1 Aug 2020

Loreto-Quijada D, Gutiérrez-Maldonado J, Nieto R, Gutiérrez-Martínez O, Ferrer-García M, Saldana C, Liutsko L (2014) Differential effects of two virtual reality interventions: distraction versus pain control. Cyberpsychol Behav Soc Netw 17(6):353–358

Manzoni GM, Cesa GL, Bacchetta M, Castelnuovo G, Conti S, Gaggioli A, Riva G (2016) Virtual reality–enhanced cognitive–behavioral therapy for morbid obesity: a randomized controlled study with 1 year follow-up. Cyberpsychol Behav Soc Netw 19(2):134–140

Martínez-Navarro J, Bigné E, Guixeres J, Alcañiz M, Torrecilla C (2019) The influence of virtual reality in e-commerce. J Bus Res 100:475–482

Maskey M, Rodgers J, Grahame V, Glod M, Honey E, Kinnear J, Parr J (2019) A randomised controlled feasibility trial of immersive virtual reality treatment with cognitive behaviour therapy for specific phobias in young people with autism spectrum disorder. J Autism Dev Disord 49(5):1912–1927

McLay R, Wood D, Webb-Murphy J, Spira J, Wiederhold M, Pyne J, Wiederhold B (2011) A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychol Behav Soc Netw 14(4):223–229

McLay R, Baird A, Webb-Murphy J, Deal W, Tran L, Anson H, Klam W, Johnston S (2017) A randomized, head-to-head study of virtual reality exposure therapy for posttraumatic stress disorder. Cyberpsychol Behav Soc Netw 20(4):218–224

McMahan A (2003) Immersion, engagement and presence: a method for analyzing 3-D video games. In: Wolf M, Perron B (eds) The video game theory reader, chap 3. Routledge, New York, pp 67–86

Meng F, Zhang W, Yang R (2014) The development of a panorama manifestation virtual reality system for navigation and a usability comparison with a desktop system. Behav Inf Technol 33(2):133–143

Merel T (2017) The reality of VR/AR growth. Tech Crunch. https://techcrunch.com/2017/01/11/the-reality-of-vrar-growth/ . Accessed 1 Aug 2020

Michaliszyn D, Marchand A, Bouchard S, Martel M, Poirier-Bisson J (2010) A randomized, controlled clinical trial of in virtuo and in vivo exposure for spider phobia. Cyberpsychol Behav Soc Netw 13(6):689–695

Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77-D(12):1321–1329

Montero-López E, Santos-Ruiz A, García-Ríos M, Rodríguez-Blázquez R, Pérez-García M, Peralta-Ramírez M (2016) A virtual reality approach to the Trier Social Stress Test: contrasting two distinct protocols. Behav Res Methods 48(1):223–232

Motraghi T, Seim R, Meyer E, Morissette S (2014) Virtual reality exposure therapy for the treatment of posttraumatic stress disorder: a methodological review using CONSORT guidelines. J Clin Psychol 70(3):197–208

Muhanna M (2015) Virtual reality and the CAVE: taxonomy, interaction challenges and research directions. J King Saud Univ—Comput Inf Sci 27(3):344–361

Murcia-Lopez M, Steed A (2018) A comparison of virtual and physical training transfer of bimanual assembly tasks. IEEE Trans Vis Comput Gr 24(4):1574–1583

Narayan M, Waugh L, Zhang X, Bafna P, Bowman D (2005) Quantifying the benefits of immersion for collaboration in virtual environments. In: Proceedings of the ACM symposium on virtual reality software and technology, Monterey, California, USA, 7–9 November

Neguţ A, Matu S, Sava F, David D (2016) Task difficulty of virtual reality-based assessment tools compared to classical paper-and-pencil or computerized measures: a meta-analytic approach. Comput Hum Behav 54:414–424

Ng Y-L, Ma F, Ho F, Ip P, Fu K-W (2019) Effectiveness of virtual and augmented reality-enhanced exercise on physical activity, psychological outcomes, and physical performance: a systematic review and meta-analysis of randomized controlled trials. Comput Hum Behav 99:278–291

Nilsson S, Johansson B, Jonsson A (2010) Cross-organizational collaboration supported by augmented reality. IEEE Trans Visual Comput Graphics 17(10):1380–1392

Nilsson N, Nordahl R, Serafin S (2016) Immersion revisited: a review of existing definitions of immersion and their relation to different theories of presence. Hum Technol 12(2):108–134

Okoli C, Schabram K (2010) A guide to conducting a systematic literature review of information systems research. Sprouts: working papers on information systems, vol 10, no. 26. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1954824 . Accessed 1 Aug 2020

Oleksy T, Wnuk A (2016) Augmented places: an impact of embodied historical experience on attitudes towards places. Comput Hum Behav 57:11–16

Paré G, Jaana M, Sicotte C (2007) Systematic review of home telemonitoring for chronic diseases: the evidence base. J Am Med Inform Assoc 14(3):269–277. https://doi.org/10.1197/jamia.M2270

Paré G, Trudel M-C, Jaana M, Kitsiou S (2015) Synthesizing information systems knowledge: a typology of literature reviews. Inf Manag 52(2):183–199

Paré G, Tate M, Johnstone D, Kitsiou S (2016) Contextualizing the twin concepts of systematicity and transparency in information systems literature reviews. Eur J Inf Syst 25(6):493–508

Parsons T, Gaggioli A, Riva G (2017) Virtual reality research in social neuroscience. Brain Sci 7(4):42

Parveau M, Adda M (2018) 3iVClass: a new classification method for virtual, augmented and mixed realities. Procedia Comput Sci 141:263–270. https://doi.org/10.1016/j.procs.2018.10.180

Perret J, Vander Poorten E (2018) Touching virtual reality: a review of haptic gloves. In: ACTUATOR 2018: 16th international conference on new actuators, June 25–27, pp 270–274

Pickering C, Byrne J (2013) The benefits of publishing systematic quantitative literature reviews for PhD candidates and other early-career researchers. High Educ Res Dev 33(3):534–548. https://doi.org/10.1080/07294360.2013.841651

Pickering C, Grignon J, Steven R, Guitart D, Byrne J (2015) Publishing not perishing: how research students transition from novice to knowledgeable using systematic quantitative literature reviews. Stud High Educ 40(10):1756–1769. https://doi.org/10.1080/03075079.2014.914907

Piskorz J, Czub M (2014) Distraction of attention with the use of virtual reality. Influence of the level of game complexity on the level of experienced pain. Pol Psychol Bull 45(4):480–487

Regenbrecht H, Schubert T (2002) Real and illusory interactions enhance presence in virtual environments. Presence: Teleoper Virtual Environ 11(4):425–434

Reger G, Koenen-Woods P, Zetocha K, Smolenski D, Holloway K, Rothbaum B, Gahm G (2016) Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consul Clin Psychol 84(11):946–959

Rehman U, Cao S (2019) Comparative evaluation of augmented reality-based assistance for procedural tasks: a simulated control room study. Behav Inf Technol. https://doi.org/10.1080/0144929X.2019.1660805

Repetto C, Gaggioli A, Pallavicini F, Cipresso P, Raspelli S, Riva G (2013) Virtual reality and mobile phones in the treatment of generalized anxiety disorders: a phase-2 clinical trial. Pers Ubiquit Comput 17(2):253–260

Riva G, Waterworth JA (2003) Presence and the self: a cognitive neuroscience approach. Presence-Connect, 3(3)

Rodríguez C, Areces D, Garcia T, Cueli M, González Castro P (2018) Comparison between two continuous performance tests for identifying ADHD: traditional vs. virtual reality. Int J Clin Health Psychol 18:254–263

Ronchi E, Mayorga D, Lovreglio R, Wahlqvist J, Nilsson D (2019) Mobile-powered head-mounted displays versus cave automatic virtual environment experiments for evacuation research. Comput Anim Virtual Worlds 30(6):e1873. https://doi.org/10.1002/cav.1873

Rowe F (2014) What literature review is not: diversity, boundaries, and recommendations. Eur J Inf Syst 23(3):241–255

Sacks R, Perlman A, Barak R (2013) Construction safety training using immersive virtual reality. Constr Manag Econ 31(9):1005–1017. https://doi.org/10.1080/01446193.2013.828844

Sadowsky W, Stanney K (2002) Measuring and managing presence in virtual environments. In: Stanney K (ed) Handbook of virtual environments technology. Lawrence Erlbaum Associates, Mahway, pp 791–806

Schoonheim M, Heyden R, Wiecha JM, Henden T (2014) Use of a virtual world computer environment for international distance education: lessons from a pilot project using second life. BMC Med Educ. https://doi.org/10.1186/1472-6920-14-36

Schroeder R (1996) Possible worlds: the social dynamic of virtual reality technologies. Westview Press, Boulder

Schryen G (2015) Writing qualitative IS literature reviews—guidelines for synthesis, interpretation and guidance of research. Commun Assoc Inf Syst 37:286–325

MathSciNet   Google Scholar  

Schryen G, Benlian A, Rowe F, Shirley G, Larsen K, Petter S, Wagner G, Haag S, Yasasin E (2017) Literature reviews in IS research: what can be learnt from the past and other fields? Commun Assoc Inf Syst. https://doi.org/10.17705/1CAIS.04130

Schubert T, Friedmann F, Regenbrecht H (2001) The experience of presence: factor analytic insights. Teleoper Virtual Environ 10(3):266–281

Shu Y, Huang YZ, Chang SH, Chen MY (2019) Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments. Virtual Real 23(4):437–446

Slater M (2003) A note on presence terminology. Presence Connect 3(3):1–5

Slater M, Wilbur S (1997) A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence: Teleoper Virtual Environ 6(6):603–616

Slater M, Usoh M, Steed A (1994) Depth of presence in virtual environments. Presence: Teleoper Virtual Environ 3(2):130–144

Smink A, Frowijn S, van Reijmersdal E, van Noort G, Neijens P (2019) Try online before you buy: how does shopping with augmented reality affect brand responses and personal data disclosure. Electron Commer Res Appl 35:100854

Solomon B (2014) Facebook buys oculus, virtual reality gaming startup, for $2 billion. https://www.forbes.com/sites/briansolomon/2014/03/25/facebook-buys-oculus-virtual-reality-gaming-startup-for-2-billion/#d8d8b7024984 . Accessed 1 Aug 2020

Suh A, Prophet J (2018) The state of immersive technology research: a literature analysis. Comput Hum Behav 86:77–90

Suso-Ribera C, Fernández-Álvarez J, García-Palacios A, Hoffman HG, Bretón-López J, Banos RM, Botella C (2019) Virtual reality, augmented reality, and in vivo exposure therapy: a preliminary comparison of treatment efficacy in small animal phobia. Cyberpsychol Behav Soc Netw 22(1):31–38

Tang Y, Au K, Lau H, Ho G, Wu G (2020) Evaluating the effectiveness of learning design with mixed reality (MR) in higher education. Virtual Real. https://doi.org/10.1007/s10055-020-00427-9

Teel E, Gay M, Johnson B, Slobounov S (2016) Determining sensitivity/specificity of virtual reality-based neuropsychological tool for detecting residual abnormalities following sport-related concussion. Neuropsychology 30(4):474–483

Templier M, Paré G (2018) Transparency in literature reviews: an assessment of reporting practices across review types and genres in top IS journals. Eur J Inf Syst 27(5):503–550. https://doi.org/10.1080/0960085X.2017.1398880

Thompson C (2017) Stereographs were the original virtual reality. Smithsonian Magazine . https://www.smithsonianmag.com/innovation/sterographs-original-virtual-reality-180964771/ . Accessed 1 Aug 2020

Thompson T, Steffert T, Steed A, Gruzelier J (2011) A randomized controlled trial of the effects of hypnosis with 3-d virtual reality animation on tiredness, mood, and salivary cortisol. Int J Clin Exp Hypn 59(1):122–142

Turk V (2016) Face electrodes let you taste and chew in virtual reality. https://www.newscientist.com/article/2111371-face-electrodes-let-you-taste-and-chew-in-virtual-reality/ . Accessed 1 Aug 2020

UQO Cyberpsychology Lab. Presence Questionnaire. (2002). http://w3.uqo.ca/cyberpsy/wp-content/uploads/2019/04/QEP_vf.pdf . Accessed August 1, 2020

Valtchanov D, Barton KR, Ellard C (2010) Restorative effects of virtual nature settings. Cyberpsychol Behav Soc Netw 13(5):503–512

Van Baren J, IJsselsteijn W (2004) Measuring presence: a guide to current measurement approaches. http://www8.informatik.umu.se/~jwworth/PresenceMeasurement.pdf . Accessed 1 Aug 2020

Van Kerrebroeck H, Brengman M, Willems K (2017) When brands come to life: experimental research on the vividness effect of virtual reality in transformational marketing communications. Virtual Real 21(4):177–191

vom Brocke J, Simons A, Riemer K, Niehaves B, Plattfaut R, Cleven A (2015) Standing on the shoulders of giants: challenges and recommendations of literature search in information systems research. Commun Assoc Inf Syst 37:205–224

Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26(2):xiii–xxiii

Wechsler TF, Mühlberger A, Kümpers F (2019) Inferiority or even superiority of virtual reality exposure therapy in phobias?—A systematic review and quantitative meta-analysis on randomized controlled trials specifically comparing the efficacy of virtual reality exposure to gold standard in vivo exposure in agoraphobia, specific phobia and social phobia. Front Psychol 10:1758. https://doi.org/10.3389/fpsyg.2019.01758

Westerfield G, Mitrovic A, Billinghurst M (2015) Intelligent augmented reality training for motherboard assembly. Int J Artif Intell Educ 25(1):157–172

Wiederhold M, Crisci M, Patel V, Nonaka M, Wiederhold B (2019) Physiological monitoring during augmented reality exercise confirms advantages to health and well-being. Cyberpsychol Behav Soc Netw 22(2):122–126

Wilkerson W, Avstreih D, Gruppen L, Beier K-P, Woolliscroft J (2008) Using immersive simulation for training first responders for mass casualty incidents. Acad Emerg Med 15(11):1152–1159. https://doi.org/10.1111/j.1553-2712.2008.00223.x

Wissmath B, Weibel D, Mast F (2010) Measuring presence with verbal versus pictorial scales: a comparison between online- and ex post- ratings. Virtual Real 14(1):43–53

Witmer B, Singer M (1998) Measuring presence in virtual environments: a presence questionnaire. Presence: Teleoper Virtual Environ 7(3):225–240

Witmer B, Jerome C, Singer M (2005) The factor structure of the presence questionnaire. Presence 14(3):298–312

Yang S, Xiong G (2019) Try it on! Contingency effects of virtual fitting rooms. J Manag Inf Syst 36(3):789–822

Yang Z, Shi J, Jiang W, Sui Y, Wu Y, Ma S, Li H (2019) Influences of augmented reality assistance on performance and cognitive loads in different stages of assembly task. Front Psychol 10:1703. https://doi.org/10.3389/fpsyg.2019.01703

Yoo SC, Drumwright M (2018) Nonprofit fundraising with virtual reality. Nonprofit Manag Leadersh 29(1):11–27

Download references

Not applicable.

Author information

Authors and affiliations.

Department of Management and Operations, Villanova School of Business, Villanova University, Villanova, PA, 19085, USA

Matthew J. Liberatore

Department of Accounting and Information Systems, Villanova School of Business, Villanova University, Villanova, PA, 19085, USA

William P. Wagner

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed equally to this research.

Corresponding author

Correspondence to Matthew J. Liberatore .

Ethics declarations

Conflicts of interest.

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: Keywords used in Scopus literature search

((TITLE (“immersive system” OR “virtual reality” OR “mixed reality” OR “augmented reality”) OR ABS (“immersive system” OR “virtual reality” OR “mixed reality” OR “augmented reality”))) AND ((TITLE (“experiment” OR “trial” OR “test”) OR ABS (“experiment” OR “trial” OR “test”))) AND (LIMIT-TO (SRCTYPE, “j”)) AND (EXCLUDE (SUBJAREA, “MEDI”) OR EXCLUDE (SUBJAREA, “MATH”) OR EXCLUDE (SUBJAREA, “PHYS”) OR EXCLUDE (SUBJAREA, “NEUR”) OR EXCLUDE (SUBJAREA, “MATE”) OR EXCLUDE (SUBJAREA, “BIOC”) OR EXCLUDE (SUBJAREA, “AGRI”) OR EXCLUDE (SUBJAREA, “EART”) OR EXCLUDE (SUBJAREA, “ENVI”) OR EXCLUDE (SUBJAREA, “CENG”) OR EXCLUDE (SUBJAREA, “IMMU”) OR EXCLUDE (SUBJAREA, “DENT”) OR EXCLUDE (SUBJAREA, “PHAR”) OR EXCLUDE (SUBJAREA, “VETE”) OR EXCLUDE (SUBJAREA, “Undefined”) OR EXCLUDE (SUBJAREA, “ENER”) OR EXCLUDE (SUBJAREA, “CHEM”) OR EXCLUDE (SUBJAREA, “ENGI”)) AND (EXCLUDE (PUBYEAR, 2009) OR EXCLUDE (PUBYEAR, 2008) OR EXCLUDE (PUBYEAR, 2007) OR EXCLUDE (PUBYEAR, 2006) OR EXCLUDE (PUBYEAR, 2005) OR EXCLUDE (PUBYEAR, 2004) OR EXCLUDE (PUBYEAR, 2003) OR EXCLUDE (PUBYEAR, 2002) OR EXCLUDE (PUBYEAR, 2001) OR EXCLUDE (PUBYEAR, 2000) OR EXCLUDE (PUBYEAR, 1999) OR EXCLUDE (PUBYEAR, 1998) OR EXCLUDE (PUBYEAR, 1997) OR EXCLUDE (PUBYEAR, 1996) OR EXCLUDE (PUBYEAR, 1995) OR EXCLUDE (PUBYEAR, 1994) OR EXCLUDE (PUBYEAR, 1993) OR EXCLUDE (PUBYEAR, 1992) OR EXCLUDE (PUBYEAR, 1991) OR EXCLUDE (PUBYEAR, 1984)) AND (EXCLUDE (EXACTSRCTITLE, “Computers And Education”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Advanced Oxidation Technologies”) OR EXCLUDE (EXACTSRCTITLE, “Computer Applications In Engineering Education”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Online Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Computing In Civil Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Turkish Online Journal Of Educational Technology”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of recent Technology And Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Advanced Engineering Informatics”) OR EXCLUDE (EXACTSRCTITLE, “Computers In Education Journal”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Computing And Information Science In Engineering”) OR EXCLUDE (EXACTSRCTITLE, “AES Journal Of The Audio Engineering Society”) OR EXCLUDE (EXACTSRCTITLE, “Advances In Mechanical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Transactions On Biomedical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Multimedia And Ubiquitous Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Telecommunication Electronic And Computer Engineering”) OR EXCLUDE (EXACTSRCTITLE, “British Journal Of Educational Technology”) OR EXCLUDE (EXACTSRCTITLE, “Educational Technology And Society”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Advanced Research In Dynamical And Control Systems”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Medical And Biological Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Professional Issues In Engineering Education And Practice”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Science Education And Technology”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Journal On Emerging And Selected Topics In Circuits And Systems”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Innovative Technology And Exploring Engineering”) OR EXCLUDE (EXACTSRCTITLE, “Journal Of Educational Computing Research”) OR EXCLUDE (EXACTSRCTITLE, “Annals Of Biomedical Engineering”) OR EXCLUDE (EXACTSRCTITLE, “IEEE Transactions On Circuits And Systems For Video Technology”) OR EXCLUDE (EXACTSRCTITLE, “Interactive Technology And Smart Education”) OR EXCLUDE (EXACTSRCTITLE, “International Journal Of Applied Engineering Research”) OR EXCLUDE (EXACTSRCTITLE, “Asia Pacific Education Researcher”)) AND (LIMIT-TO (LANGUAGE, “English”)) AND (LIMIT-TO (DOCTYPE, “ar”) OR LIMIT-TO (DOCTYPE, “re”)).

Rights and permissions

Reprints and permissions

About this article

Liberatore, M.J., Wagner, W.P. Virtual, mixed, and augmented reality: a systematic review for immersive systems research. Virtual Reality 25 , 773–799 (2021). https://doi.org/10.1007/s10055-020-00492-0

Download citation

Received : 02 August 2020

Accepted : 27 November 2020

Published : 03 January 2021

Issue Date : September 2021

DOI : https://doi.org/10.1007/s10055-020-00492-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Immersive systems
• Virtual reality
• Augmented reality
• Mixed reality
• Empirical research
• Systematic review
• Find a journal
• Publish with us
• Track your research

Study and Analysis of Virtual Reality and its Impact on the Current Era

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Accessibility Links

• Skip to content
• Skip to search IOPscience
• Skip to Journals list
• Accessibility help
• Accessibility Help

Click here to close this panel.

Purpose-led Publishing is a coalition of three not-for-profit publishers in the field of physical sciences: AIP Publishing, the American Physical Society and IOP Publishing.

Together, as publishers that will always put purpose above profit, we have defined a set of industry standards that underpin high-quality, ethical scholarly communications.

We are proudly declaring that science is our only shareholder.

Virtual reality and augmented reality displays: advances and future perspectives

Kun Yin 1 , Ziqian He 1 , Jianghao Xiong 1 , Junyu Zou 1 , Kun Li 2 and Shin-Tson Wu 3,1

Published 8 April 2021 • © 2021 The Author(s). Published by IOP Publishing Ltd Journal of Physics: Photonics , Volume 3 , Number 2 Citation Kun Yin et al 2021 J. Phys. Photonics 3 022010 DOI 10.1088/2515-7647/abf02e

Article metrics

14822 Total downloads

Share this article

Author e-mails.

[email protected]

Author affiliations

1 College of Optics and Photonics, University of Central Florida, Orlando, FL 32816, United States of America

2 Goertek Electronics, 5451 Great America Parkway, Suite 301, Santa Clara, CA 95054, United States of America

Author notes

3 Author to whom any correspondence should be addressed.

Shin-Tson Wu https://orcid.org/0000-0002-0943-0440

• Received 10 December 2020
• Accepted 18 March 2021
• Published 8 April 2021

Peer review information

Method : Single-anonymous Revisions: 1 Screened for originality? Yes

Buy this article in print

Virtual reality (VR) and augmented reality (AR) are revolutionizing the ways we perceive and interact with various types of digital information. These near-eye displays have attracted significant attention and efforts due to their ability to reconstruct the interactions between computer-generated images and the real world. With rapid advances in optical elements, display technologies, and digital processing, some VR and AR products are emerging. In this review paper, we start with a brief development history and then define the system requirements based on visual and wearable comfort. Afterward, various VR and AR display architectures are analyzed and evaluated case by case, including some of the latest research progress and future perspectives.

Export citation and abstract BibTeX RIS

Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 license . Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

1. Introduction

As a promising next-generation display, virtual reality (VR) and augmented reality (AR) provide an attractive new way for people to perceive the world. Unlike conventional display technologies, such as TVs, computers, and smartphones that place a panel in front of the viewer, VR and AR displays are designed to revolutionize the interactions between the viewer, display, and surrounding environment. As a kind of information acquisition medium, VR and AR displays bridge the gap between computer-generated (CG) images and the real world. On the one hand, VR displays generate a fully immersive virtual environment based on CG images, which has a sufficient field of view (FOV) to provide refreshing virtual experience without relying on the viewer's real environment. On the other hand, AR display offers see-through capability with an enriched surrounding environment. By overlapping virtual images with the real world, viewers can immerse in an imaginative world that combines fiction and reality.

Although some commercial VR and AR displays have emerged in recent years, the origin of this technology can be traced back to the last century [ 1 ]. With the introduction of head-mounted display (HMD) and the virtual environment in the 1960s [ 2 , 3 ], such a novel display concept was once considered as state-of-the-art. However, due to the lack of flat panel displays, image rendering capabilities, related sensors, wireless data transfer, and well-designed optical components, this display technology, which was ahead of its time then, came to an end. Fortunately, with the rapid development of optics [ 4 – 6 ], high resolution displays [ 7 ], and information technologies [ 8 ] in recent years, VR and AR are blooming again. Because of the impressive visual experience and high degrees of interaction between viewers and CG images, VR and AR are promising for widespread applications, including, but not limited to, healthcare, education, engineering design, manufacturing, and entertainment.

The goals of VR and AR displays are to provide reality-like clear images that can simulate, merge into, or rebuild the surrounding environment without wearer discomfort [ 9 , 10 ]. Specifically, visual comfort has to meet the requirements of the human visual system based on the eye-to-brain imaging process, otherwise the viewer will feel unreal, unclear, or even dizzy and nauseous. Usually, the human eye has a large FOV: about 160° in the horizontal and 130° in the vertical directions for each eye (monocular vision). The overlapped binocular vision still has 120° FOV in the horizontal direction [ 11 ]. In parallel, the dioptre and rotation of the human eye lens can collaborate to focus on different positions of a real object with the correct depth of field and blur the other portions [ 12 ]. Therefore, to achieve visual comfort, the optical system should provide an adequate FOV, generate 3D images with matched depth and high resolution, and offer sufficient contrast and brightness, to name just a few examples. Regarding wearer comfort, a compact and lightweight structure is desired for long-time use. At present, due to the pros and cons between different optical components and system designs, it is still challenging for VR and AR to meet these goals. Therefore, in this paper, we focus on advanced VR and AR architectures aiming at visual and wearer comfort, and a more comprehensive understanding of the status quo.

2. Advanced architectures for VR displays

Figure 1 (a) depicts a schematic diagram of a VR optical system. For visual comfort, a broad FOV covering the human vision range can be achieved by designing a compact eyepiece with a low f -number ( f /#) [ 13 ]. However, due to the immersive experience with a completely virtual environment, the main issue is with the CG-3D image generation. When evaluating the capability of generating 3D images in VR, an important aspect of the human visual system is stereo perception. The real observation of a 3D object induces an accommodation cue (the focus of the eyes) and a vergence cue (relative rotation of the eyes), that match with each other (figure 1 (b): left) [ 14 , 15 ]. However, in most of the current VR systems, there is only one fixed display plane with different rendered contents. To capture the image information, the viewer's eyes will focus on the display plane, but the position of the CG-3D object is usually not in the display plane. As a result, the visual system in the viewer's brain will force the eyeball to focus on the virtual 3D object, while the eye lens focuses on the display plane, which leads to mismatched accommodation distance and vergence distance (figure 1 (b): right). This phenomenon is called vergence–accommodation conflict (VAC) [ 16 ], which causes dizziness and nausea. Besides visual comfort, the overall weight and volume of the system will also limit the usage time and applications. To achieve wearer comfort, the system should be as light as possible while keeping a broad FOV in the virtual space. In this section, we will focus on advanced VR architectures that address 3D image generation to mitigate VAC and reduce the headset volume.

Figure 1.  (a) The layout of a VR optical system. (b) The root cause of the VAC issue. The accommodation cue coincides with the vergence cue when viewing a real object (left). Mismatch occurs when viewing a virtual object displayed in a fixed plane (right).

Download figure:

2.1. VAC mitigation

2.1.1. multi-focal system.

The multi-focal display was proposed to solve the VAC problem of HMDs in the late 1990s [ 17 ]. The basic principle of a multi-focal system is to generate multiple image planes or shift the position of image planes to match the vergence distance and accommodation distance, thereby overcoming the VAC issue. Based on different architectures and principles, multi-focal VR systems can be categorized into space multiplexing, time multiplexing, and polarization multiplexing systems.

Space multiplexing can simultaneously generate multiple image planes with different depths. To achieve this goal, Rolland et al [ 18 ] proposed a very straightforward method to physically place multiple screens based on transparent panels, as illustrated in figure 2 (a). However, the transparent panels will not only increase the cost but also exhibit obvious moiré patterns after stacking multiple panels together [ 19 ]. To avoid this problem, beam splitters (BSs) can be utilized to help establish the space multiplexing system, as figure 2 (b) shows [ 20 ]. In this design, the display panel is placed on one side, while the BSs reflect different parts of the display. Since the distance between each BS and the human eye is different, the image is displayed at different depths. Space multiplexing provides a direct solution to address VAC in VR displays and maintains image quality and frame rate. However, this architecture requires multiple display panels or BSs, which leads to dramatically increased weight and volume. Recently, a focal plane display with a phase-only spatial light modulator (SLM) has been demonstrated [ 21 ]. This architecture can achieve multi-focal planes with reduced system size and weight, but it requires an expensive SLM, and the image quality is not ready for commercial products yet.

Figure 2.  A schematic diagram of space multiplexing using multiple (a) transparent panels and (b) BSs. Time multiplexing by (c) shifting the display and (d) applying a tunable lens. Polarization multiplexing by (e) applying PPML.

The time multiplexing method relies on dynamic components and can timely change the panel distance (figure 2 (c)) or the effective focal length (figure 2 (d)) [ 22 , 23 ]. The panel distance is usually changed by a mechanical motor, which leads to a lack of stability and modulation rate. For time multiplexing, the modulation rate of the dynamic components should be at least N times ( N is the number of image planes) the display frame rate to avoid motion blurs. Therefore, compared with mechanically tuning the panel position, tuning the effective focal length through an electrically driven eyepiece is more favourable. Although it is still challenging to fabricate an adaptive lens with a wide tuning range and fast response time, this method can reduce the number of physical elements, so the system volume is much more compact than that of spatial multiplexing.

Polarization multiplexing generates multiple image planes based on different polarization states. To distinguish different polarization states, the most critical optical component is a polarization-dependent lens with different focal lengths for two orthogonal polarization states. Two such examples are: (a) the Pancharatnam–Berry phase lens, based on left-handed circularly polarized light (LCP)/right-handed circularly polarized light (RCP), and (b) the birefringent lens based on horizontal/vertical linearly polarized light [ 24 , 25 ]. Figure 2 (e) depicts the basic polarization multiplexing system. The light emitted from the display panel transmits through a pixelated polarization modulation layer (PPML), which can modulate the ratio of two orthogonal polarization states, so the light intensity of each pixel in the corresponding focal plane can be adjusted independently. PPML can be a polarization rotator for a linear polarization system [ 26 ] or an integrated polarization rotator and quarter-wave plate for a circularly polarized system [ 27 ]. The advantage of polarization multiplexing is that it can generate multiple image planes without sacrificing the frame rate or an enlarged system volume. However, the major limitation of polarization multiplexing is that only two orthogonal polarization states can be utilized. It should be mentioned that these multiplexing approaches can be combined. For example, time multiplexing or space multiplexing can be combined with polarization multiplexing to increase the number of focal planes [ 27 , 28 ].

2.1.2. Micro-lens array system

Unlike using a large single lens as an eyepiece, another advanced architecture involves adding a micro-lens array (MLA) in front of the display panel to globally or individually change the position of virtual images in a VR system [ 10 ]. When the MLA is precisely aligned with the display panel, a small movement of the MLA can lead to a large focus change for the virtual image. As a result, instead of moving a thicker display panel or bulkier lens over a longer range, pushing or pulling the MLA plate a small distance can significantly mitigate the VAC. It is worth mentioning that the focus of an MLA based on liquid crystal materials can be switched dynamically for several microns, which means the movement of virtual images can be obtained without any mechanical motion, as shown in figure 3 (a). Furthermore, as figure 3 (b) shows, if each MLA element can be precisely controlled independently, then we can produce a specific focus for each lenslet and generate pixelated depth. These techniques are suitable for VR displays as well as for free-space-based couplers in AR displays. It is worth mentioning that in the MLA system, the resolution is usually an important issue, which needs to be further improved.

Figure 3.  A schematic diagram of focus tuning systems based on (a) an electronically addressable MLA and (b) an individually tunable MLA. A schematic diagram of (c) a real object and (d) light field with an MLA.

2.1.3. Light field system

To mitigate VAC, both temporally and spatially changed displays have been proposed. However, due to a limited or discrete tuning range, these methods can only partially recreate the 3D object with the correct depth. Rather than changing the image focus, light field displays ideally recreate a physical wavefront similar to that created by a real object. The light field capture (e.g. integral imaging) [ 29 – 31 ] can be achieved by a lens array to convert the light from display pixels to rays with arbitrary spatial angles. As depicted in figure 3 (d), the spatial points correspond to the pixels on the display panel. To display a virtual 3D object, we trace the points on the object and light the corresponding pixels on the display panel. Then, the light field on those points can be approximated with discrete emitting rays. Although this method can provide correct depth information and retinal blur, the resolution is sacrificed. If the amount of information is taken into consideration, it is not surprising that these approaches that aim to show true 3D information cannot offer sufficient resolution due to the limited bandwidth of the current devices. Generally, the resolution is limited by the display and the individual lens. Although a high-resolution display has been proposed, the pixel pitch is still determined by the diffraction limit of the employed lens [ 29 ]. These approaches should gradually mature in the long run and eventually reach a satisfactory level for viewers. But at the current stage, the main drawbacks of this architecture are resolution loss, refresh rate increase, and/or redundancy of display panels.

2.2. Pancake VR

As discussed above, aside from visual comfort, wearable comfort is another important consideration. To reduce the volume and weight of a VR system, thereby improving its wearable comfort, a compact optical design, while taking the headset's central gravity into consideration, is urgently needed. Recently, polarization-based folded optics (or pancake optics) with further reduced form factors have attracted increasing attention. The system was originally proposed for use in flight simulators [ 32 ] and it has gained renewed interest due to rapid development of VR [ 33 , 34 ]. The basic concept is to create a cavity to fold the optical path into a smaller space. The working mechanism is illustrated in figure 4 (a). The cavity lies between a BS and a reflective polarizer. The BS (including a metallic or dielectric half mirror) has 50% transmittance and it flips the handedness of incident polarized light upon reflection. The reflective polarizer selectively transmits light with one polarization state and reflects the orthogonal one, which can be achieved by a wire-grid polarizer, birefringent multi-layer film, or cholesteric liquid crystal (CLC). The former two respond to a linear polarization, while the latter respond to a circular polarization. To explain the working principle, we use a circularly polarized light as an example. As shown in figure 4 (a), the incoming RCP light in region A firstly passes through the BS (50%) and gets reflected by the reflective polarizer. Then, it is reflected by the BS again (25%), while flipped to the LCP state. Finally, the LCP light passes the reflective polarizer and enters region C. Because of the BS, only 25% of the total energy is delivered to the viewer's side. Therefore, system efficiency is an important issue in the pancake VR system. Practical systems often involve one or more refractive elements, which can be placed in any of the specified ABC regions. The surfaces of the reflective polarizer and BS can also be curved according to design requirements. An example with a refractive lens placed in region B is plotted in figure 4 (b). The BS (half reflector) in this case is coated on the curved surface of the lens.

Figure 4.  Polarization-based folded optics. (a) An illustration of the working principle. (b) An example of folded optics with a refractive lens. (c) A CLC-based reflective polarizer with optical power. (d) A reflective hologram with angular selectivity.

All the above discussions only consider traditional geometric optics, where the optical power is provided by reflection or refraction of the curved surfaces. Recent advances in holographic optics, however, offer an even wider range of choices for optical elements. Both the reflective polarizer and BS can be flat holographic films [ 35 ]. As figure 4 (c) shows, the reflective polarizer can have a focusing power by patterning the CLC molecules. The polarization selectivity of CLC leads to an optical power for one circular polarization and total transparency for the other. The BS can also be replaced by a phase hologram. Such a phase hologram is often fabricated by holographic exposure of a photopolymer [ 36 ]. Its index modulation is usually small, resulting in narrow angular and spectral responses. This angular selectivity can be utilized to boost the overall system efficiency. As depicted in figure 4 (d), for a certain reflective hologram, light within the angular response is reflected with flipped handedness. Other incident lights that do not meet the Bragg conditions will traverse the hologram. With this feature, the BS efficiency can potentially reach 100% because both the transmission and reflection efficiencies can reach 100%. This means the overall system efficiency can be improved from 25% to nearly 100%. However, the narrow angular and spectral selectivity also indicates the requirement for a directional backlight with narrow spectral linewidth, which could be challenging for practical implementation.

3. Advanced architectures for AR displays

In contrast to the immersive experience provided by VR displays, AR displays aim for see-through systems that overlap CG images with physical environments. To obtain this unique visual experience with wearable comfort, the near-eye systems need to possess high transmittance with sufficient FOV and a compact form factor. Therefore, freeform optics with broad FOV and high transmittance are essential for AR displays. However, due to the prism-shape, this architecture presents a relatively large volume and heavy weight. To reduce the system size while keeping a sufficient FOV, a lightguide-based structure and free-space coupler are commonly used to create a delicate balance between visual comfort and wearable comfort.

3.1. Freeform prisms and BS architectures

Freeform prisms have been extensively investigated due to the development of diamond-turning machines. Typically, the freeform prisms used in an AR system need a partially reflective surface and a total internal reflection (TIR) surface to overlap the CG images and transmit the surrounding environments. As shown in figure 5 (a), this configuration sophisticatedly incorporates two refraction surfaces, a TIR surface, and a partial reflection surface into one element, and therefore allows extra design freedom [ 37 , 38 ]. This design provides high-quality images with a wide FOV, but due to its volume limitation the entire system will be bulky and heavy. Another common example of a freeform-based AR device uses a designed BS cube as the coupler. In figure 5 (b), the magnifying optics is a reflective concave mirror disposed directly on the BS cube, which has more freedom to be further optimized. This device architecture provides the simplest solution to AR display with a broad FOV but a larger form factor. Moreover, there is another trade-off between the FOV and eyebox (or exit pupil) due to the conservation of étendue, which is the product of the FOV and eyebox. Therefore, the larger the FOV, the smaller the eyebox [ 39 ].

Figure 5.  A schematic AR diagram with a (a) a freeform prism and (b) a specially designed BS cube.

3.2. Lightguide-based architectures

Compared to the freeform design, the lightguide-based structure has a more balanced performance between visual comfort and wearable comfort, especially in the compact and thin form factor [ 40 , 41 ]. Over the past decade, lightguide-based near-eye display (LNED) has become one of the most widely used architectures for AR displays and is applied in many commercial products, such as HoloLens 2, Magic Leap 1, and Lumus. For an LNED, input and output couplers are pivotal optical elements affecting the system's performance. Typically, the input coupler has a high efficiency enabling it to fully utilize the light emitted from the optical engine. In contrast, the output coupler has low and gradient efficiency across the exit pupil to ensure an expanded and uniform eyebox. According to different coupler designs, LNEDs can be categorized into grating-based lightguides (figure 6 (a)) and geometrical lightguides (figure 6 (b)).

Figure 6.  Schematic diagrams of (a) grating-based and (b) geometrical lightguide-based AR architectures.

3.2.1. Grating-based lightguide

As shown in figure 6 (a), the display light is coupled into the lightguide by an input grating and then propagates inside the lightguide through TIR. When it encounters the output grating, the light is replicated and diffracted into the viewer's eyes. To provide a comprehensive understanding, we will theoretically analyze the FOV limit and discuss the commonly used grating couplers. For a diffractive grating, the first-order grating equation can be stated as:

where θ in and θ out represent the incident angle and diffracted angle, respectively, n in and n out are the refractive index of the incident medium and output medium, λ is the wavelength in vacuum, and Λ is the grating period. With this simple grating equation, the maximum system FOV can be calculated. If we assume the FOV in air is centrosymmetric, then the viewing angle in air ( θ air ) is related to the minimum/maximum guiding angles ( θ min / θ max ) in the lightguide as:

where n g is the refractive index of the lightguide, n air is the refractive index of air, θ min can be set to the TIR angle in the lightguide, and θ max should be less than 90°. Thus, the maximum horizontal FOV is [ 42 ]:

Figure 7 (a) shows the FOV as a function of n g and θ max . In an ideal case where θ max = 90° and n g = 2, the maximum system FOV is only 60°. In practical designs, such a high index lightguide substrate is still challenging to achieve, and θ max cannot approach 90° due to image quality considerations. This FOV limit is generally true for most grating-based lightguide AR. However, some methods can be employed to circumvent this limit. For instance, using a different system configuration [ 42 ], FOV can be expanded to 100°, or by leveraging polarization-dependent optical elements the FOV can be nearly doubled [ 43 ]. In equation ( 3 ), it seems that the FOV is independent of the wavelength, but the wavelength dependency is implicitly embedded in equation ( 2 ). For the extreme case with θ max = 90° and n g = 2, if the waveguide is designed at 535 nm, then the grating period is calculated to be 357 nm and the horizontal FOV is [−30°, 30°]. Utilizing such a grating period for blue (e.g. 450 nm) and red (e.g. 630 nm) with the assumption that the angle ranges in the lightguide are the same will lead to an FOV of [−15°, 48°] and [−50°, 14°], respectively. Thus, more than one grating is needed to obtain the same FOV for RGB colours. Although implementation of three gratings with narrow spectral bandwidths for R, G, and B in one lightguide is possible, it is still hard to eliminate colour crosstalk among different gratings. A more common choice is to have two (e.g. one for R, and one for G and B) or three (e.g. R, G, and B) lightguides [ 44 ], where the system's compactness is slightly sacrificed. Another important aspect is that the spectral response of most gratings depends on the incident angle. This can be well illustrated using a volume Bragg grating (VBG) as an example. For a VBG, the central wavelength is defined by the Bragg condition as:

Figure 7.  (a) FOV as a function of lightguide refractive indexes and maximum guiding angles. (b) Angle dependency of a VBG ( n eff = 1.5) designed for 535 nm and a diffraction angle of 50° at normal incidence. The inset shows the definition of θ , which is the angle of incident light relative to the normal direction of Bragg planes. For reflective VBGs: diffraction efficiency as a function of (c) wavelength and (d) incident angle; for transmissive VBGs: diffraction efficiency as a function of (e) wavelength and (f) incident angle. Simulations are based on rigorous coupled wave analysis.

where θ represents the incident light angle with respect to the normal direction of Bragg planes (see the inset in figure 7 (b)), and n eff is the effective refractive index of the VBG. If a VBG (e.g. n eff = 1.5) is designed for a normally incident green light ( λ = 535 nm) with 50° diffraction angle in a lightguide, then the angle-dependent central wavelength can be calculated, as figure 7 (b) depicts. For such a VBG, the central wavelength would shift from green to blue as the incident angle increases. Therefore, when designing a VBG-based lightguide AR for full-colour operation, such a colour crosstalk should be carefully analyzed.

In terms of selecting grating couplers, two types of gratings are commonly used in lightguide AR: holographic VBGs and surface relief gratings (SRGs). In holographic VBGs, sinusoidal refractive index modulation in the volume is introduced by interference exposure of holographic photopolymers. The refractive index modulation can be described by [ 45 ]:

Unlike holographic VBGs that have refractive index modulation in the bulk, SRGs have specially designed microstructures on the surface, which can be massively produced by nanoimprinting [ 48 ]. The surface structures have a large design degree of freedom. The shapes of grating structures can be blazed, slanted, binary, and even analogue, according to different needs [ 10 ]. The spectral and angular responses of SRGs strongly depend on the shape of surface structures. Due to high refractive index contrast between the substrate and air, the structure height can be submicron to achieve high diffraction efficiency.

Besides holographic VBG and SRG, CLC-based polarization volume grating (PVG) is also a strong contender [ 49 , 50 ]. Due to their volume grating nature, PVGs can be treated as a branch of holographic VBGs and their spectral and angular responses are very similar. However, PVGs exhibit some unique properties. First, PVGs are strongly circular-polarization dependent originating from CLCs [ 51 ], while VBGs and SRGs have weak polarization dependency on linear polarizations. For example, for a left-handed reflective PVG, it only diffracts the LCP light within the bandwidth into the first order, while transmitting the RCP light. This feature is useful for designing polarization-dependent optical elements. Second, if we use equation ( 5 ) to approximately describe the behaviour of PVGs (in fact, to describe PVGs the refractive indices in equation ( 5 ) should be replaced by dielectric constants), Δ n can be very large. For instance, if the host liquid crystal has a birefringence of 0.2, the effective Δ n can be as large as 0.5 ∼ 0.6 for a VBG. As a result, its spectral and angular bandwidths can be much larger than those of holographic VBGs. Moreover, recent studies show that multi-layer PVGs or gradient-pitch PVGs can be easily achieved to further enlarge the angular bandwidth [ 52 , 53 ].

3.2.2. Geometrical lightguide

Compared to grating-based lightguides, geometrical lightguides need more complex designs (e.g. spatial variant coatings) to achieve gradient efficiency, and it is relatively hard to add a lens power to the output. However, the working principle is very simple, and all the designs are based on surface reflection. Generally, geometrical lightguides use embedded reflective surfaces as the exit pupil expander to reflect and replicate the light [ 54 , 55 ].

As figure 6 (b) shows, a series of cascaded, embedded, and partially reflective surfaces can be used as output couplers in the geometrical lightguide architecture. As the embedded surface is reflective, it yields good colour uniformity over the entire FOV. However, this cascaded design produces the Louver effect [ 10 ], which is unfavourable for see-through devices. Recently, this effect has been reduced due to better cutting, polishing, coating, and design, but it is still a limitation. In addition, these complicated fabricating processes put more burdens on manufacturers. As an extension, the embedded partially reflective surfaces can be designed as flat surfaces (figure 6 (b)), pin-shaped mirror arrays (figure 8 (a)), microprism arrays (figure 8 (b)), or a curved lightguide with curved surfaces (figure 8 (c)) [ 56 ].

Figure 8.  A schematic diagram of geometrical lightguide AR: (a) microprism array, (b) pin-shaped mirror array, and (c) curved coupler.

3.3. Free-space coupler-based architectures

Unlike freeform optical devices or LNEDs, free-space couplers have greater freedom in the architecture, and there are no special restrictions on volume or TIR. Undoubtedly, due to large degrees of freedom, numerous architectures based on free-space couplers have been proposed, but each design has its pros and cons. These systems can be classified into three categories based on the working principles: reflective coupler, diffusive coupler, and diffractive coupler.

3.3.1. Reflective coupler

A reflective free-space coupler is based on the surface reflection of a flat or curved surface. Due to the high transmittance requirement, these surfaces should be partially reflective with sufficient reflection and transmittance. Figure 9 (a) depicts the most straightforward architecture with a flat coupler, which is a tilted partial reflective surface. The CG images emitted from the display are collimated by the lens and then reflected into the viewer's eye through the flat coupler. To further simplify the system, such a flat coupler can be replaced by a partially reflective curved or freeform surface with a specially designed profile, as shown in figure 9 (b). This design is aimed at smartphone displays rather than complex off-axis imaging and micro-display. This architecture has been successfully applied to Meta 2 by Meta Vison, DreamGlass by Dream World, and NorthStar by LeapMotion. Due to a large display panel and curved reflective surface, such a reflective coupler exhibits a relatively broad FOV but also a large system volume.

Figure 9.  A schematic diagram of reflective free-space coupler-based AR: (a) a flat coupler, and (b) a curved coupler. llustrations of diffusive free-space coupler-based AR: (c) a single diffuser, and (d) multiple diffusers.

3.3.2. Diffusive coupler

A diffusive free-space coupler is based on the light scattering of optical elements [ 57 ]. In such a system, the displayed images are directly projected onto the coupler, which is usually a diffuser with a flat or curved surface. As illustrated in figure 9 (c), the light is scattered by the coupler and then the image is displayed on the diffuser surface. Usually, the image source is a liquid-crystal-on-silicon (LCoS) or digital micro-mirror device, and the image resolution is controlled by the display and projection lens. To keep see-through capability, the diffuser should have angular selectivity to scatter the off-axis incident image and transmit the environment light in front of the eye. Therefore, the system can accommodate more than one diffuser, and thereby has the space to construct a 3D image with multiple planes [ 58 ], similar to the multiplane design in a VR system. As depicted in figure 9 (d), each diffuser scatters the incoming light with the corresponding incident angle and do not interfere with each other.

3.3.3. Diffractive coupler

A diffractive free-space coupler is based on flat diffraction optical elements with designed phase profiles, such as lens or freeform optics [ 59 , 60 ]. More specifically, the architectures based on diffractive couplers can be divided into free-space systems, Maxwellian systems, and integral imaging systems. The free-space-based diffractive couplers, as illustrated in figure 10 (a), are utilized in a pupil-forming system, which means it uses relay optics to first image the object and then deliver the relayed image to the viewer's eye with the diffractive coupler [ 61 , 62 ]. The image source includes but is not limited to a conventional 2D display and laser light source. However, due to the nature of diffractive flat optics and off-axis system configuration, aberrations like coma and astigmatism are large and need to be tackled with sophisticated optical design or image pre-processing. The Maxwellian system adopts the principle of a Maxwellian view [ 63 ], which directly forms a focus-free image on the retina. The diffractive couple can be a reflective off-axis lens with a designed focal length [ 64 , 65 ]. It is worth mentioning that because the light needs to be focused on the pupil, the eyebox in the Maxwellian system is relatively small. To expand the eyebox, an exit pupil shifting can be applied to increase the area covered by the focal point [ 66 ]. Generally, the image light is focused by the coupler and the focal spot is located at the eye lens. As a result, the image on the retina stays in-focus no matter how much the optical power of the eye lens changes. Depending on the image source, the system can be achieved by an LCoS (figure 10 (a)) or a laser beam scanner (LBS) (figure 10 (c)) for a simpler design. The light field system with an MLA can also be applied to the AR system, such as the light field in a VR display [ 67 , 68 ]. As depicted in figure 10 (d), a typical configuration is the projection system which is used to relay the original image from the image source to near the focus of the diffractive coupler, similar to the free-space combination system. The relayed image then works in the same way as depicted in figure 3 (d) and produces the light field to display 3D virtual objects. Similar to the multiplexing method in VR displays, these different AR architectures with fictional optics are not independent. On the contrary, they can be combined with each other to balance their respective advantages and trade-offs, and even enable new features [ 69 ].

Figure 10.  A schematic diagram of diffractive free-space coupler-based AR: (a) a free-space diffractive coupler, a Maxwellian system with (b) SLM and (c) LBS, and (d) an integral imaging system.

To quantitatively summarize the performance of AR architectures based on visual comfort and wearable comfort, table 1 compares the form factor and FOV among different coupling methods. It should be mentioned that for each architecture, the performance can be further improved based on the current value but at the cost of other parameters. Therefore, the contents listed in table 1 are the general conditions rather than strict restrictions.

Table 1.  Performance comparison of various AR architectures.

a These not only depend on the FOV and eyebox design but also include an optical engine part. b These typical values come from products and prototypes.

4. Conclusions

In this review, we summarize the advanced architectures with different optical components in the rapidly evolving VR and AR systems, including the most recent optical research and products, and analyze the systems based on the visual and wearable comforts case by case. Because of the various advanced architectures with unique features, such as reducing VAC through adjustable lenses, solving compact size issues using polarizing films, and providing a large FOV through freeform optics, VR and AR displays present both scientific significance and broad application prospects. Although, at the current stage, it is still challenging for these architectures to meet all the requirements for visual and wearable comfort, learning about and reviewing advanced systems will certainly help us focus on unresolved issues and inspire more elegant solutions.

Acknowledgments

The authors are indebted to GoerTek Electronics for financial support.

Data availability statement

All data that support the findings of this study are included within the article (and any supplementary files).

This paper is in the following e-collection/theme issue:

Published on 26.4.2024 in Vol 12 (2024)

This is a member publication of Glasgow Caledonian University (Jisc)

Views of Specialist Clinicians and People With Multiple Sclerosis on Upper Limb Impairment and the Potential Role of Virtual Reality in the Rehabilitation of the Upper Limb in Multiple Sclerosis: Focus Group Study

Authors of this article:

Original Paper

• Amy Webster 1 , MSc   ; 
• Matthieu Poyade 2 , PhD   ; 
• Elaine Coulter 1 , PhD   ; 
• Lisa Forrest 1 , MSc   ; 
• Lorna Paul 1 , PhD  

1 School of Health and Life Sciences, Glasgow Caledonian University, Glasgow, United Kingdom

2 School of Simulation and Visualisation, Glasgow School of Art, Glasgow, United Kingdom

Corresponding Author:

Amy Webster, MSc

School of Health and Life Sciences

Glasgow Caledonian University

Cowcaddens Road

Glasgow, G4 0BA

United Kingdom

Phone: 44 141 273 1345

Email: [email protected]

Background: Finding enjoyable and effective long-term approaches to rehabilitation for improving the upper limb (UL) function of people with multiple sclerosis (MS) is challenging. Using virtual reality (VR) could be a solution to this challenge; however, there is a lack of reporting on the views of people with MS and clinicians on VR-based approaches and recommendations for games for rehabilitation.

Objective: This study aims to identify common UL problems and their related current therapeutic approaches for people with MS, and to explore the opinions of people with MS and specialist clinicians on VR and obtain suggestions for the development and design of VR games.

Methods: Separate focus groups were conducted with people with MS, recruited through the MS Society UK’s research network, and clinicians, recruited through the MS Trust Therapists in MS network. A total of 10 people with MS (2 focus groups) and 8 clinicians (5 physiotherapists, 2 occupational therapists, and 1 MS nurse in 2 focus groups) were involved. The focus groups were recorded and transcriptions were analyzed using theme-based content analysis.

Results: People with MS commonly reported that their UL problems interfered with activities of daily living and resulted in the loss of meaningful hobbies such as writing. Many people with MS neglected UL exercise and found strategies for adapting to the UL impairments. Similarly, clinicians stated UL rehabilitation was neglected within their service and that it was challenging to find interesting treatment strategies. VR was suggested by both participant groups as a solution, as it was convenient for people with MS to access and it could provide a more engaging and disguised approach to exercise. There were shared concerns with cybersickness and disengagement with using VR approaches. Both groups agreed games should be meaningful and adaptable for users but suggested different VR activities, with clinicians suggesting games directly reflecting activities of daily living and people with MS suggesting more abstract activities.

Conclusions: VR was well received by both people with MS and clinicians for UL rehabilitation. Recommendations were made for the development of VR rehabilitation games which are personalized and customizable for the varying abilities of people with MS.

Introduction

Multiple sclerosis (MS) is an inflammatory demyelination disorder of the central nervous system that is estimated to affect 2.8 million people worldwide [ 1 ]. Over a third of the people with MS have upper limb (UL) dysfunction, including weakness, tremors, and spasms in one or both ULs [ 2 ]. This can result in difficulties with activities of daily living (ADL), negatively impacting quality of life and the likelihood of remaining in employment [ 3 , 4 ]. Problems specifically with dexterity are related to higher health care costs [ 5 ] and a higher association with depression-like psychological measures compared to problems with lower limb function [ 6 ]. Rehabilitation and physical exercise improve motor function for people with MS [ 7 , 8 ]. The evidence regarding UL rehabilitation is lacking in comparison with the lower limb, despite the high frequency of UL impairments and their impact on ADL [ 9 ]. In addition, there are particular challenges in finding effective yet motivating rehabilitation strategies in MS due to the long-term, progressive nature of the disease and diversity of symptoms [ 10 ].

Virtual reality (VR) is increasing in popularity in rehabilitation research and is proposed as a possible approach to encourage long-term rehabilitation [ 11 ]. VR includes digital environments that often simulate real-world experiences with reported benefits of high motivation and engagement, with real-time feedback [ 12 ]. VR has shown promising results within MS populations, but this evidence is limited in comparison with stroke, especially regarding UL function [ 13 ]. Our systematic review, investigating the effect of VR in improving UL function in MS, found early, but limited, evidence suggesting VR has the potential to improve function in people with MS [ 14 ]. There was also a low number of dropouts in most studies within the review, supporting that VR could improve adherence compared with conventional rehabilitation; therefore, VR could be useful in conditions such as MS, where prolonged rehabilitation is required.

VR is often investigated alongside video games played within a VR setting, which can be commercially available or specifically tailored games designed with a target population in mind. Commercially available exercise games, targeted at a healthy population, can be unsuitable for disabled individuals and lead to discouragement and anxiety [ 15 ]. It is beneficial to involve a sample of target users in the creation and development of effective VR-based gamified approaches [ 16 ]. This process is known as coproduction [ 17 ]. To date, no study has systematically coproduced VR games specifically for UL rehabilitation in people with MS.

The aims of this study were to determine the views of people with MS and specialist clinicians on UL dysfunction or function in MS, challenges faced by clinicians when delivering UL therapy, barriers and motivators for exercise in MS, opinions on VR, and suggestions for development and design of VR games. These findings will guide the future development of VR applications and interventions for UL rehabilitation for people with MS.

Ethical Considerations

Ethics approval for this study was provided by the School of Health and Life Sciences Ethics Committee at Glasgow Caledonian University (HLS/PSWAHS/20/002). Informed consent was obtained from the participants and clinicians.

Recruitment

The study aimed to recruit up to 12 people with MS and 12 specialist MS clinicians to take part in online focus groups. The sample size was determined in line with the design of other similar studies and general recommendations for qualitative analysis [ 18 , 19 ]. To be included in the study, people with MS were required to be aged ≥18 years and have a diagnosis of MS (self-reported) with self-reported UL impairment. Clinicians were required to have experience (any duration) in delivering MS rehabilitation within the National Health Service (NHS) or the third sector. In addition, all participants were required to have access to and the ability to operate videoconference software. There were no specified exclusion criteria. Participants with MS were identified through the MS Society UK’s research network, which advertised the study to its members. Those who were interested in participating contacted the research team directly and were emailed a participant information sheet. In terms of recruitment of clinicians, the MS Trust Therapists in MS network advertised the study to its members. Interested clinicians contacted the research team and were emailed a participant information sheet.

Coproduction Focus Groups

The focus groups for people with MS and clinicians were conducted separately with a maximum of 5 people per focus group. To comply with COVID-19 pandemic regulations at the time, focus groups were held online using Zoom (Zoom Video Communications) or Teams (Microsoft Corp) videoconference software; this also provided an opportunity for recruitment of participants from across the United Kingdom and Ireland. The focus groups were conducted in a semistructured style using a focus group schedule split broadly into three sections important for the development of VR interventions for UL problems in people with MS: (1) UL dysfunction and exercise or therapy; (2) opinions on VR; and (3) suggestions for development and design of any developed VR games ( Multimedia Appendix 1 ). In addition, clinicians were asked what information and feedback they would want from a patient’s VR therapy session. The questions included prompts that allowed more targeted responses from participants regarding their experiences and views [ 20 ]. Within the focus groups, participants were shown three videos demonstrating different commercially available head-mounted devices (HMDs) and hand-tracking devices: (1) a nonimmersive VR set up using a Leap Motion controller and computer monitor, which is a hand motion capture device that allows users to visualize their hand movements and interact with virtual environments; (2) immersive VR using the Oculus Rift HMD with a mounted Leap Motion device for hand tracking; and (3) immersive VR using the Oculus Quest, with in-built hand tracking ( Figure 1 [ 21 , 22 ]; Multimedia Appendix 2 ). Videos were shown as participants were unable to try these devices since the focus groups were online due to the COVID-19 pandemic. These videos attempted to contextualize and demonstrate the different VR and motion capture devices in terms of users interacting with environments, possible hand movements, and previous games developed from prior research. After watching the videos, participants were encouraged to share their initial thoughts on each of the technologies. The focus groups involving people with MS and clinicians lasted approximately 90 minutes and 60 minutes, respectively. The focus groups were facilitated by a female researcher (AW) who had been involved in the recruitment of participants and an additional senior, female researcher (LP) attended.

Data Analysis

All focus groups were audio recorded and transcribed verbatim. Qualitative analysis of the data was performed based on theme-based content analysis (TBCA) as described in the study by Neale and Nichols [ 23 ]. This qualitative method groups responses into content-related themes to enable researchers to view the user preferences more easily and has been used to influence the development or evaluation of a VR environment [ 23 - 25 ]. TBCA is a flexible qualitative data analysis method that involves five key steps: (1) data collection, (2) data collation, (3) raw theme definition and classification, (4) higher order theme selection, and (5) presentation of classification matrix [ 23 ]. Owing to the large number of higher order themes, we added an additional step by grouping the higher order themes into main themes. The raw themes were assigned independently by 2 researchers in the transcripts of people with MS (AW and LF) and clinicians (AW and LP). After agreement on the raw themes, the responses were then independently grouped by 2 researchers (AW and LP) into higher order themes. Any discrepancies in assigning the themes were resolved through consultation with a third reviewer, if necessary. Once the higher order themes were determined, the main themes were determined by 2 researchers (AW and LP). The main themes with their associated raw and higher order themes are presented in tables. The raw and higher order themes were quantified manually within the matrix based on the number of responses necessary to display popularity or consensus [ 23 ], and example quotes for each higher order theme were included. Focus groups of people with MS and clinicians were analyzed separately to allow comparison of the findings between the 2 groups.

Participant Demographics

A total of 10 people with MS were recruited to the study and took part in 1 of 2 focus groups, each of which had 5 participants. Most participants with MS were female (7/10, 70%), with a mean age of 56.4 (SD 16.5) years and a mean time since diagnosis of 14.4 (SD 12.3) years. Participants had varying MS types ( Table 1 ).

A total of 8 clinicians were recruited (5 physiotherapists, 2 occupational therapists, and 1 MS specialist nurse). Among them, 6 participants worked in the NHS and 2 worked in other settings. There were 2 focus groups for clinicians with 4 participants in each group. All clinicians were female, with a mean age of 46.2 (SD 9.6) years, and the mean length of experience was 17.9 (SD 10. 2) years.

a SPMS: secondary progressive multiple sclerosis.

b RRMS: relapsing-remitting multiple sclerosis.

c PPMS: primary progressive multiple sclerosis.

People With MS: TBCA

Following TBCA of the focus groups of people with MS, 20 higher order themes were determined based on the grouping of the assigned raw themes. These 20 higher order themes were grouped into four main themes: (1) Impact of MS on the UL; (2) Exercising with MS; (3) Views of people with MS on VR; and (4) Recommendations for development and user requirements ( Table 2 ). A full version of this table, including more example quotes from participants, is available in Multimedia Appendix 3 .

a MS: multiple sclerosis.

b UL: upper limb.

c ADL: activities of daily living.

d VR: virtual reality.

e HMD: head-mounted device.

Impact of MS on the UL

The most common higher order theme was “Interference with functional activities” with 35 responses ( Table 2 ). Participants reported a wide range of activities they found difficult to perform due to their MS, the most frequent being ADL, including personal care, eating, and carrying heavy items. “Symptoms and signs that impact activities” had the second highest number of responses (n=25), where participants particularly noted the impact of fatigue on activity (n=10); however, sensory problems such as numbness and pins and needles were also highlighted. Other MS symptoms impacting UL function were, for example, weakness, tremors, and coordination problems. In “Strategies people with MS adopt to assist with ADL” (n=24), because of losing function, participants discussed the use of assistive equipment, for example, button fasteners, specialized cups, and voice control. Other strategies included using their less affected hand or pacing to manage fatigue. The remaining 4 higher order themes had fewer responses. In brief, dexterity, range of joint movement, and grip were the main “UL actions people with MS find difficult” (n=13). These were often compounded by the unpredictability and progressive nature of MS (“Difficulty with progression and unpredictable nature of MS,” n=10). Participants reported the emotional impact of losing the ability to carry out personal and meaningful activities specifically because of loss of UL function (“Struggle with loss of meaningful activities and skills,” n=14), with one participant stating the following:

I used to be a writer and it was very, very hard because I couldn’t write anymore...I was really motivated [to relearn writing], felt really cut off from the world. [P8; age 70 years; male participant with primary progressive multiple sclerosis]

The final higher order theme was “Sharing and sympathy” (n=13 responses), where participants empathized and shared experiences and suggestions of assistive equipment.

Exercising With MS

Most responses under this main theme related to “Views and attitudes on exercise” (n=49; Table 2 ). Participants were motivated to exercise with a " use it or lose it ” attitude and a desire to, if not improve then at least maintain, their function and prevent further deterioration. Participants also described negative perceptions of exercise, such as finding it “ very boring ” and guilt from not participating in exercise. In “Previous experience with UL rehabilitation or exercise” (n=40), many participants (5/10, 50%) discussed not undertaking any UL exercise or rehabilitation, currently or previously. Many UL programs previously undertaken by some participants aimed to build strength, reduce pain, and improve hand function with varying outcomes. There were similar numbers of responses in terms of “Barriers to exercise” (n=28) and “Facilitators to exercise” (n=28). Personal barriers to exercise included comorbidities, MS symptoms (fatigue, pain, and bladder and bowel dysfunction), difficulty using exercise equipment, and expense. The COVID-19 pandemic had negatively impacted the participants’ exercise due to services closing down. Environmental barriers to exercise included lack of local facilities and not having space to exercise at home. Verbal encouragement was described as both a barrier (could be off putting) and a facilitator (motivating) to exercise. Other facilitators were seeing improvements, feeling motivated, and the attitudes of health care professionals, personal trainers, and carers. Conversely, health care professionals with a lack of experience in MS overwork people with MS, leading to exhaustion (“Adverse effects of exercise,” n=11). Participants undertook many different forms of exercise (“Approaches to exercise used by people with MS,” n=26), including exercise bikes, Pilates and yoga, dog walking, and gym exercises. There were varying “Views on group versus individual exercise” (n=26). Some found competition within a group to be motivating while others did not, with one participant suggesting social support and camaraderie was more important than competition:

I’m not too fussed about being in competition with others, but if it was a more social thing that would maybe encourage me to perhaps join in a group that’s doing something together. [P4; age 58 years; female participant with secondary progressive multiple sclerosis (SPMS)]

Negative aspects of group exercise included the fear of letting others down.

Views on VR

The initial reaction to VR was positive (“Positive views on VR,” n=55; Table 2 ). Participants stated it looked fun or enjoyable with the potential to improve or maintain muscle strength, dexterity, and spatial awareness, especially with repeating the actions and concurrently perhaps learning a new skill (for example, playing the piano):

I think [VR’s] still very good because... it’s...maintaining those motor skills that is so easily slip away when you’re not using them. [P9; age 60 years; male participant with SPMS]

There were positive comments in relation to the convenience and accessibility of VR facilitating exercise at home at a suitable time and eliminating travel to physiotherapy services and gyms. Participants highlighted that the wireless HMD was more convenient as it was portable and did not need a computer. The advantage of linking up with others online was raised. However, “Negative views on VR” (n=40) were related to concerns regarding cybersickness, linked to dizziness and balance problems:

With MS a lot of people suffer from nausea or motion sickness. That can be a concern for the headsets. [P6; age 28 years; female participant with relapse and remitting multiple sclerosis]

Other negative responses related to the HMD discomfort regarded weight, usability concerns, wearing it with glasses, and being disconnected from the real world. Two participants indicated that interest in VR may reduce over time. Participants were also concerned about fatigue and the usefulness of VR for UL sensory dysfunction. Most participants (6/10, 60%) expressed they were open to trying VR (“Views on trying or participating in VR rehabilitation,” n=25), but would like to understand the benefits, long-term outcomes, and any safety issues.

Recommendations for Development and User Requirements

With regard to “Considerations for development of VR games” (n=84), a variety of UL movements was desirable with clarity in terms of the aim and outcome in relation to the UL being important ( Table 2 ). Competition within the VR games, interacting with others or challenging themselves, were frequently discussed as being motivating. Tracking improvements during VR gameplay was vital to some participants, including monitoring improvements in score, exercise time (rather than countdown which could be stressful), and progressive challenges. The games should offer the ability to challenge users, with one participant saying the following:

That challenge to try and be better the next time, whereas if you’ve got no idea...you’ve got nothing to fight against or to work against. [P10; age 84 years; female participant with SPMS]

Conversely, other participants emphasized the potential demotivating effect of feedback given the progressive nature of MS, by warning that score feedback should not be “ disheartening ,” and should therefore be made optional to the user. There was a strong feeling that the VR games should be “ fun ” with abstract gameplay potentially being more fun. Participants felt that demonstrations and supervision to assess progress were important. They also stated that the VR games had to account for the differences in the ability of people with MS and that older people may need more basic VR games. The idea of the VR games having an educational outcome or in learning a new skill was suggested to help with engagement. Participants suggested that reaching, punching, and other aerobic activities could be incorporated (“Suggestions for VR activities,” n=36). Having haptic approaches was frequently proposed with gripping, squishing games, such as kneading bread. Participants proposed activities with a cognitive element, such as a puzzle or maze, and whole limb movements, such as Whack-a-Mole (Mattel), writing or drawing. Participants liked the VR piano which had been demonstrated. There was a variety of opinions in terms of abstract or real-life activity with most preferring abstract games but some ADL-type activity was also suggested. “Importance of choice” (n=23) related to having variety in games, UL movements, and levels of difficulty with abstract games or real-life gamified tasks, with 1 participant declaring the following:

I’d like to make sure I’m not doing a whole lot of exercises that are all doing the same things...Got to be mixing them up: one for coordination, one for dexterity. [P1; age 60 years; female participant with SPMS]

Clinicians: TBCA

From the clinician focus groups, there were 15 higher order themes grouped into four main themes: (1) Current methods and challenges for delivering UL rehabilitation; (2) Clinicians’ views on VR; (3) Recommendations for development and user requirements; and (4) Implementation of VR into practice ( Table 3 ). A full version of this table, including more example quotes from participants, is available in Multimedia Appendix 4 .

a UL: upper limb.

b MS: multiple sclerosis.

c VR: virtual reality.

d HMD: head-mounted device.

Current Methods and Challenges for Delivering UL Rehabilitation

“Recommended UL exercises for people with MS” (n=29) included strength training and active movements related to functional activity, such as hand-to-mouth movements ( Table 3 ). Treatment for the UL often involved equipment such as Therabands and Theraputty but also technology such as the Gloreha robotic system and functional electrical stimulators with different models of care for UL exercises described as part of community-based classes, within third sector organizations and online programs (“Current methods of UL exercise delivery for people with MS,” n=15). Within “Factors clinicians consider when prescribing exercise for the UL” (n=22), most responses were regarding meaningful and goal-focused exercises. Clinicians also considered the patient’s symptoms, for example, spasticity, pain, and the ability of patients. The importance of repetition of movement was reinforced. Most responses (n=52) were in relation to “Challenges clinicians face when delivering exercise for people with MS.” Clinicians expressed that UL-focused exercise was neglected compared to the lower limb and the challenge of making UL exercise interesting:

A bit more difficult for upper limb things...it’s much easier to maybe...go for a walk with somebody or you know, or cycle or whatever. Upper limb is maybe a wee bit more difficult. [C6; physiotherapist]

Clinicians also mentioned the use of Theraputty described as “ juvenile ” and lists of exercises “ boring .”

Service-related challenges included limited time and capacity to see patients and large geographic areas to cover. Other challenges were keeping patients engaged with exercise in the long term, especially at home, and finding an activity that would be attractive to patients. Under “Experience with long-term, progressive condition” (n=24), clinicians raised being realistic about improvements with a progressive condition while also keeping patients motivated, minimizing deterioration or maintenance, rather than improving:

Trying to motivate people with progressive MS, you’re trying to get them to continue to maintain where they are rather than improve. [C5; occupational therapist]

Clinicians expressed the positive benefits of “Socializing in exercise” (n=14) for support and motivation.

Clinicians’ Opinions on VR

Clinicians were very positive about VR (n=50), describing it as being interactive, fun, meaningful, and a novel potential approach to rehabilitation, which could help engagement ( Table 3 ). They were positive about the escapism aspect and the potential to improve mental health:

What appeals about VR stuff is that it is focused and takes you into a different place...You’re doing tai chi on a beautiful, Japanese garden rather than actually in your grumpy living room...I think even that in terms of the escapism aspect, maybe from a mental wellbeing. [C1; physiotherapist]

Clinicians liked the visual feedback to help with, for example, coordination, but which could also reinforce movements and introduce a cognitive component. Clinicians commented that VR provided the opportunity to undertake activities not possible within the clinic and to exercise without the activity seeming like an exercise. Most of the “Negative views of VR” (n=38) were regarding patient safety using VR headsets, especially cybersickness, including dizziness and disorientation, specifically in patients with vestibular issues. Other general concerns with HMDs were usability with glasses, the weight of the HMD, and feeling claustrophobic. Clinicians suggested that VR activities should not be too simplistic to avoid patronizing patients and at an appropriate skill level. The longevity of engagement of patients after the initial novelty was questioned. Clinicians also questioned the use of VR for activities that can be done in the real world and similarly how VR activities might translate to real function. The importance of feedback on the quality of movement as well as the quantity was highlighted. Finally, accessibility and digital poverty were also raised. The final higher order theme was “Questioning benefits and the unknowns of VR” (n=14), where some clinicians felt there was insufficient evidence on the purpose and benefits of VR and its effect on neural mechanisms:

I think it’s important to think about how is [VR] different to just doing [activities] in real life as well...What can you augment in your rehab through this virtual reality that you can’t just do in real life anyway? [C7; physiotherapist]

Under “Considerations for developing VR games for people with MS” (n=41), clinicians discussed the importance of the VR games having purposeful activity, translation of tasks into real life, and having an end point ( Table 3 ). The games should consider movements of individual joints of the UL with extension movements at the wrist and fingers being important as where people with MS lose the most function. Games should incorporate strength, coordination, proprioception, and range of motion exercise as well as exercises for the core. Feedback was important, with clinicians able to monitor the program. Clinicians were not interested in scores for the games but wished feedback on the quality of the movements and patient engagement. Clinicians stated that undertaking VR activities with others or in group settings with elements of competition was desirable. Clinicians provided “Suggestions for VR activities” (n=18), including ADL such as putting on makeup; writing or chopping vegetables; and hobbies including pottery, sewing, or piano playing. Clinicians raised the “Importance of choice” (n=15) in the VR setup, choice of games, and choice within games, for example, levels of difficulty, to appeal to as many people as possible:

I think, it is about having a variety of things that push as many buttons with patients that you can manage and cover as many options as you can. [C2; physiotherapist]

Implementation of VR Into Practice

Under “Suggestions for incorporation of VR into practice” (n=18), clinicians felt long-term, regular use of VR was needed for positive outcomes ( Table 3 ). Home use was felt to encourage frequent use, with clinicians monitoring progress remotely, thus saving in person contact time. There were a number of “Challenges with implementation of VR into practice” (n=24) with cost and funding (service and individual) being the most commonly reported, which included potential increased demand on services:

I know if I brought it to my bosses they would want a breakdown of cost of monthly rate, how are we going to utilise it, how often are we going to utilise it. What figures could we get from this particular item and what outcomes could we achieve. [C4; MS specialist nurse]

Equipment-related challenges were ownership, availability, supply of equipment, and infection control. A full risk assessment would be required before implementation and guidance would be needed on intervention duration and frequency. Clinicians discussed for whom VR would be appropriate, in terms of age or other factors, and identified this as an area for future research (“Finding the target audience for VR,” n=8).

Principal Findings

This study aimed to explore the views of people with MS and clinicians on UL impairment associated with MS and the potential role of VR as a rehabilitation approach to address this impairment. The discussion focuses on the combined findings from the 2 groups of participants: people with MS and clinicians ( Figure 2 ). Figure 2 is a visual representation of the principal findings based on the higher number of responses assigned, which should inform the development of VR applications and interventions aiming to improve the UL function for people with MS and how VR could tackle challenges of existing UL exercise raised by clinicians and people with MS in this study. The findings agree with those of previous studies that people with MS commonly have UL impairments that impact function, including problems with dexterity and ADL, which leads to loss of meaningful activities [ 26 - 28 ]. Despite UL difficulties, UL exercise was neglected due to MS symptoms, such as fatigue, lack of motivation, and dislike of exercise, as well as the challenges clinicians faced regarding time constraints and finding appropriate therapies that were not childlike or boring. Lack of focus on UL rehabilitation has been reported previously in MS [ 9 ] and in other long-term neurological conditions such as stroke [ 29 ]. The progressive and unpredictable nature of MS was raised by both groups, and consequently, clinicians raised the importance of setting realistic expectations with therapy, sometimes focusing on maintenance of function rather than improvement.

Both groups (people with MS and clinicians) were optimistic about the use of VR and believed VR could be a solution to their exercise challenges. Positive comments, including avoiding traveling, being accessible, and being engaging or fun addressed the identified barriers for UL rehabilitation. This concurs with previous VR studies [ 30 , 31 ] and specifically in UL rehabilitation in MS, with a recent home-based, feasibility study using the Oculus Quest 2 VR headset in which participants described VR as fun, interesting, and innovative [ 32 ]. Participants in the study by Kamm et al [ 32 ] suggested adding difficulty levels and scoring to their exercises, competitive elements previously described to be motivating by people with MS using nonimmersive exercise games delivered through the Nintendo Wii [ 33 ]. In this study, both groups were especially positive regarding the immersive approach of the Oculus Quest. Participants thought the escapism properties and visualization of movements could potentially “disguise exercise,” which may occur with the “fun” element of VR reducing the perception of exertion during exercise [ 34 ], therefore encouraging more UL therapy.

Negative views about VR were also expressed, mainly the potential for cybersickness. Cybersickness is thought to be caused by conflict of stimuli, leading to nausea, disorientation, and pain in the eyes and head [ 35 ]. Women are more susceptible to cybersickness [ 36 ], which is relevant in MS, with a higher number of women affected. Although cybersickness with VR has been reported previously in people with MS [ 37 ], there are development strategies for reducing cybersickness, such as designing VR activity with less overall movement within the virtual environment. Cybersickness is, however, thought to reduce over time with exposure to VR [ 38 ]. There were unnecessary concerns raised for those wearing glasses as the HMD can accommodate glasses, but there were valid concerns about the weight of the HMD for some users. Disengagement was another concern both groups expressed, with limited data on long-term adherence to VR in MS rehabilitation. Exercise is a behavioral intervention, and long-term adherence to exercise can be supported by evidence-based behavior change techniques [ 39 ]. These behavior change techniques, such as goal setting, rewards, and feedback, can be incorporated into VR games or activity to support long-term engagement in UL exercise. While VR can be more engaging than other methods of exercise [ 40 ], frequent performance, feedback on progress, and adjusting levels of difficulty can maximize VR engagement for those with long-term neurological conditions [ 41 ]. Finally, clinicians had specific concerns regarding digital poverty, the technical ability of people with MS, and insufficient technical services to support VR.

Considerations for VR game development align with user-centered design principles for VR in motor rehabilitation in survivors of stroke, such as being fun, tracking progress, having an element of competition, challenging oneself, and providing feedback [ 42 ], and are not specific to any clinical population. Participants raised that VR development should be mindful of the different end users (people with MS) who may differ in ability and preferences. Clinicians suggested VR would appeal to younger individuals with MS, whereas people with MS felt older people with MS might need more basic gameplay. While there is some, albeit limited, evidence for lower usability scores for older VR users compared with younger users, there can be higher user enjoyment [ 43 ], and there is moderate evidence for good usability of VR in older populations [ 44 ]; therefore, this concern may be overly cautious.

Consideration of the end user links to the importance of choice when designing VR interventions, with a variety of games to appeal to as many as possible. Participants felt the games should include different movements, levels of immersion, level of difficulty, or feedback on performance. Accommodating individual preferences is a key element for the design of VR games for rehabilitation, as it increases user engagement [ 45 ]. However, our previous systematic review found that a choice of games was rarely included in VR interventions in MS [ 14 ].

There were differing views in terms of the type of feedback people wished from VR. Some people with MS wanted to track scores and visualize results, which is supported by reward theories for users during both entertainment and serious games [ 46 ]. Conversely, concerns were raised about feedback potentially being discouraging or demotivating, especially given the variable nature of MS. As an example, countdown timers provide slight pressure to motivate players to increase engagement [ 47 ]; however, in this study, people with MS felt they could be stressful. Feedback on the duration of exercise completed was appealing to people with MS, as reported previously [ 19 ]. As well as the quantity of VR exercise, clinicians also wished feedback on the quality of movement when performing the games. Rehabilitation often involves highly repetitive movements to stimulate neuroplasticity; however, stroke specialist therapists have also previously reported concerns that quality of movement in VR rehabilitation for UL maybe sacrificed for a good gaming outcome [ 18 ], although this has not been explored in people with MS. Both groups were interested in the reported outcomes of using VR approaches which, if positive, would increase engagement.

Clinicians and people with MS felt VR activity had to be related to the patient’s personalized and meaningful goals, which is known to increase motivation in physiotherapy settings [ 48 ]; however, this is often neglected in VR regimes [ 14 ]. Goals need to be adjusted over time in a progressive condition, such as MS, and to avoid disengagement as raised earlier. Participants with MS frequently stated that their goals were related to not only improvement but also the maintenance of ability and the prevention of further deterioration. In terms of suggestions for VR activities, the groups differed with clinicians suggesting ADL or hobby simulations and people with MS being more ambivalent, stressing activity to be fun with a variety of real-life and abstract VR games. Previous studies of VR have often involved ADL activities such as cooking or other kitchen activity [ 49 , 50 ]. Although VR can provide a safe environment to practice ADL for people with mobility issues [ 51 ] people with MS in this study were less interested in ADL, especially kitchen simulations. Both groups suggested an “end result,” such as creating a drawing, or learning a new skill would be positive and facilitate a feeling of accomplishment. There were also suggestions from people with MS to incorporate haptic activities, such as grabbing and gripping. However, the user is not able to receive tactile feedback when interacting with a virtual environment, and handheld controllers may need to be considered for some VR activities [ 52 ]. Another solution could be to incorporate pseudo haptics, the use of different stimuli such as visual or auditory stimuli, to mimic a variety of haptic properties in a virtual environment [ 53 ]. This is an emerging field that could be explored in VR for people with MS. Similarly, as many of the participants suggested finger-related exercises, it is important that VR systems use good hand-tracking motion capture devices to allow visualization of the movement of fingers and wrists within a VR setting.

Many people with MS were supportive of VR for home use, as being more convenient and accessible. However, there was recognition that users needed demonstration of the technology and a level of clinician supervision. Assessing quality of movements and monitoring of patient progress are reported challenges for VR home use [ 54 ]. A recent study with a small number of participants found VR to be feasible for home-based UL rehabilitation in people with MS, after 3 supervised sessions [ 32 ], but larger studies of home-based VR for UL rehabilitation are required. There was agreement in both groups that an element of social interaction could be considered in the development of VR games. Generally, there is a lack of evidence on the effect of socialization within UL therapy, but it may improve adherence and motivation [ 55 ] and provide better outcomes [ 56 ]. Specifically in relation to VR, there is some evidence that social aspects increase motivation through competition [ 57 ], but participants in our study were more interested in self-competition rather than competing against others. This is similar to a study of a walking app for MS where users were less interested in sharing their goals or achievements with others [ 58 ].

Strengths and Limitations

Recruiting participants through online sources may result in a biased sample, as those comfortable with technology and access to online services are more likely to take part. Being online allowed the involvement of people with MS with varying abilities and clinicians who worked in the NHS and the third sector across the United Kingdom. However, the online nature meant it was not possible for participants to physically test the VR equipment and explore their reactions. While it can also be challenging to engage all participants in online focus groups, this was resolved by asking questions using participants’ names or by getting participants to use the raise hand function within the videoconference software and encourage discussion between participants.

The TBCA methodology groups responses into themes to quantify them but does not allow consideration of the interaction between participants. Participants had a number of specific questions, such as the long-term outcomes of using immersive VR, the optimal target users for VR (level of disability), and the extent of translation of VR activity into “real-life” function. However, there is currently a lack of literature to provide responses to these questions, which highlights areas for future research.

Conclusions

This is the first study exploring the views of people with MS and clinicians in terms of VR for UL rehabilitation for people with MS and has highlighted the current challenges in UL rehabilitation even though UL impairment is common and impacts meaningful activity. Overall, people with MS often found dexterity-related activities difficult, which impacted multiple ADL and challenges faced in therapy related to motivation, lack of resources, and difficulty finding interesting UL exercises. There was positive support for VR for UL exercise. Overall, to improve engagement and satisfaction for the user, this study suggests any VR games developed for people with MS should (1) be fun and engaging; (2) have clear aims related to the individual user’s goals; (3) offer personalization, such as a variety of games (abstract and ADL based), different movements, levels of difficulty, and methods of feedback; (4) monitor quality as well as quantity of movement during gameplay; (5) incorporate design features to reduce the potential for cybersickness; (6) consider if the games can incorporate education or skill development; (7) incorporate aspects of social interaction; and (8) consider including haptic properties. The findings support the need for the creation of bespoke serious games rather than using commercially available exercise games, which can discourage users with motor dysfunction [ 15 , 59 ]. Overall, future development of VR games for UL rehabilitation should focus on a personalized and customizable approach to encourage long-term engagement to improve meaningful outcomes for people with MS.

Acknowledgments

The authors would like to thank all the participants who took part in this study, as well as the MS Society, UK, for funding this study (grant number 115).

Authors' Contributions

AW, MP, EC, and LP designed and conceptualized this study. AW, LF, and LP performed the analysis. AW and LP wrote and prepared the manuscript, with support from MP, EC, and LF.

Conflicts of Interest

None declared.

Focus group interview questions for both participant groups.

Compilation of 3 videos shown to participants during focus groups showing 3 different virtual reality systems.

Main, higher order, and raw themes from theme-based content analysis of people with multiple sclerosis focus groups, with example quotes.

Main, higher order, and raw themes from theme-based content analysis of clinician focus groups, with example quotes.

• Walton C, King R, Rechtman L, Kaye W, Leray E, Marrie RA, et al. Rising prevalence of multiple sclerosis worldwide: insights from the Atlas of MS, third edition. Mult Scler. Dec 11, 2020;26(14):1816-1821. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marrie RA, Cutter GR, Tyry T, Cofield SS, Fox R, Salter A. Upper limb impairment is associated with use of assistive devices and unemployment in multiple sclerosis. Mult Scler Relat Disord. May 2017;13:87-92. [ CrossRef ] [ Medline ]
• Simmons RD, Tribe KL, McDonald EA. Living with multiple sclerosis: longitudinal changes in employment and the importance of symptom management. J Neurol. Jun 2010;257(6):926-936. [ CrossRef ] [ Medline ]
• Goverover Y, Genova HM, DeLuca J, Chiaravalloti ND. Impact of multiple sclerosis on daily life. In: Chiaravalloti ND, Goverover Y, editors. Changes in the Brain: Impact on Daily Life. New York, NY. Springer; 2017;145-165.
• Koch MW, Murray TJ, Fisk J, Greenfield J, Bhan V, Jacobs P, et al. Hand dexterity and direct disease related cost in multiple sclerosis. J Neurol Sci. Jul 15, 2014;341(1-2):51-54. [ CrossRef ] [ Medline ]
• Alonso RN, Eizaguirre MB, Cohen L, Quarracino C, Silva B, Pita MC, et al. Upper limb dexterity in patients with multiple sclerosis: an important and underrated morbidity. Int J MS Care. 2021;23(2):79-84. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dalgas U, Ingemann-Hansen T, Stenager E. Physical exercise and MS recommendations. Int MS J. May 2009;16(1):5-11. [ Medline ]
• Etoom M, Khraiwesh Y, Lena F, Hawamdeh M, Hawamdeh Z, Centonze D, et al. Effectiveness of physiotherapy interventions on spasticity in people with multiple sclerosis: a systematic review and meta-analysis. Am J Phys Med Rehabil. Dec 2018;97(11):793-807. [ CrossRef ] [ Medline ]
• Lamers I, Maris A, Severijns D, Dielkens W, Geurts S, Van Wijmeersch B, et al. Upper limb rehabilitation in people with multiple sclerosis: a systematic review. Neurorehabil Neural Repair. Oct 10, 2016;30(8):773-793. [ CrossRef ] [ Medline ]
• Burks JS, Bigley GK, Hill HH. Rehabilitation challenges in multiple sclerosis. Ann Indian Acad Neurol. Oct 2009;12(4):296-306. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Howard MC. A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Comput Human Behav. May 2017;70:317-327. [ CrossRef ]
• Cano Porras D, Siemonsma P, Inzelberg R, Zeilig G, Plotnik M. Advantages of virtual reality in the rehabilitation of balance and gait: systematic review. Neurology. May 29, 2018;90(22):1017-1025. [ CrossRef ] [ Medline ]
• Maggio MG, Russo M, Cuzzola MF, Destro M, La Rosa G, Molonia F, et al. Virtual reality in multiple sclerosis rehabilitation: a review on cognitive and motor outcomes. J Clin Neurosci. Jul 2019;65:106-111. [ CrossRef ] [ Medline ]
• Webster A, Poyade M, Rooney S, Paul L. Upper limb rehabilitation interventions using virtual reality for people with multiple sclerosis: a systematic review. Mult Scler Relat Disord. Jan 2021;47:102610. [ CrossRef ] [ Medline ]
• Plow M, Finlayson M. A qualitative study exploring the usability of Nintendo Wii Fit among persons with multiple sclerosis. Occup Ther Int. Mar 2014;21(1):21-32. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kristensson P, Matthing J, Johansson N. Key strategies for the successful involvement of customers in the co-creation of new technology-based services. Int J Serv Ind Manag. 2008;19(4):474-491. [ CrossRef ]
• Windasari NA, Visita L. User engagement mechanisms of online co-design service: does user innovativeness matter? Asian Acad Manag J. Jun 27, 2019;24(1):59-82. [ CrossRef ]
• Schmid L, Glässel A, Schuster-Amft C. Therapists' perspective on virtual reality training in patients after stroke: a qualitative study reporting focus group results from three hospitals. Stroke Res Treat. 2016;2016:6210508. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. Dec 21, 2018;18(1):148. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiménez TR, Orozco M. Prompts, not questions: four techniques for crafting better interview protocols. Qual Sociol. Jun 05, 2021;44(4):507-528. [ CrossRef ]
• Webster A, Poyade M, Rea P, Paul L. The co-design of hand rehabilitation exercises for multiple sclerosis using hand tracking system. In: Rea PM, editor. Biomedical Visualisation. Cham, Switzerland. Springer; 2019;83-96.
• Hollywood RA, Poyade M, Paul L, Webster A. Proof of concept for the use of immersive virtual reality in upper limb rehabilitation of multiple sclerosis patients. In: Rea PM, editor. Biomedical Visualisation. Volume 11. Cham, Switzerland. Springer; 2022;73-93.
• Neale H, Nichols S. Theme-based content analysis: a flexible method for virtual environment evaluation. Int J Hum Comput Stud. Aug 2001;55(2):167-189. [ CrossRef ]
• Junus IS, Santoso HB, Isal RY, Utomo AY. Usability evaluation of the student centered e-learning environment. Int Rev Res Open Dis Learn. Nov 02, 2015;16(4):62-82. [ CrossRef ]
• Latham B, Poyade M, Finlay C, Edmond A, McVey M. New tools in education: development and learning effectiveness of a computer application for use in a university biology curriculum. In: Rea PM, editor. Biomedical Visualisation. Volume 2. Cham, Switzerland. Springer; 2019;29-46.
• Bertoni R, Lamers I, Chen CC, Feys P, Cattaneo D. Unilateral and bilateral upper limb dysfunction at body functions, activity and participation levels in people with multiple sclerosis. Mult Scler. Oct 06, 2015;21(12):1566-1574. [ CrossRef ] [ Medline ]
• Cattaneo D, Lamers I, Bertoni R, Feys P, Jonsdottir J. Participation restriction in people with multiple sclerosis: prevalence and correlations with cognitive, walking, balance, and upper limb impairments. Arch Phys Med Rehabil. Jul 2017;98(7):1308-1315. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Newsome SD, von Geldern G, Shou H, Baynes M, Marasigan RE, Calabresi PA, et al. Longitudinal assessment of hand function in individuals with multiple sclerosis. Mult Scler Relat Disord. Jul 2019;32:107-113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Meadmore KL, Hallewell E, Freeman C, Hughes A. Factors affecting rehabilitation and use of upper limb after stroke: views from healthcare professionals and stroke survivors. Top Stroke Rehabil. Mar 13, 2019;26(2):94-100. [ CrossRef ] [ Medline ]
• Dias P, Silva R, Amorim P, Lains J, Roque E, Pereira IS, et al. Using virtual reality to increase motivation in poststroke rehabilitation. IEEE Comput Grap Appl. Jan 1, 2019;39(1):64-70. [ CrossRef ]
• Pagliari C, Di Tella S, Jonsdottir J, Mendozzi L, Rovaris M, De Icco R, et al. Effects of home-based virtual reality telerehabilitation system in people with multiple sclerosis: a randomized controlled trial. J Telemed Telecare. Dec 01, 2021;30(2):344-355. [ CrossRef ]
• Kamm CP, Blättler R, Kueng R, Vanbellingen T. Feasibility and usability of a new home-based immersive virtual reality headset-based dexterity training in multiple sclerosis. Mult Scler Relat Disord. Mar 2023;71:104525. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Forsberg A, Nilsagård Y, Boström K. Perceptions of using videogames in rehabilitation: a dual perspective of people with multiple sclerosis and physiotherapists. Disabil Rehabil. 2015;37(4):338-344. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Palacios-Ceña D, Ortiz-Gutierrez R, Buesa-Estellez A, Galán-Del-Río F, Cachon Perez JM, Martínez-Piedrola R, et al. Multiple sclerosis patients' experiences in relation to the impact of the kinect virtual home-exercise programme: a qualitative study. Eur J Phys Rehabil Med. Jul 2016;52(3):347-355. [ FREE Full text ] [ Medline ]
• LaViola Jr JJ. A discussion of cybersickness in virtual environments. SIGCHI Bull. Jan 01, 2000;32(1):47-56. [ CrossRef ]
• Stanney K, Fidopiastis C, Foster L. Virtual reality is sexist: but it does not have to be. Front Robot AI. Jan 31, 2020;7:4. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kalron A, Frid L, Fonkatz I, Menascu S, Dolev M, Magalashvili D, et al. The design, development, and testing of a virtual reality device for upper limb training in people with multiple sclerosis: single-center feasibility study. JMIR Serious Games. Oct 12, 2022;10(3):e36288. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kemeny A, Chardonnet JR, Colombet F. Getting Rid of Cybersickness: In Virtual Reality, Augmented Reality, and Simulators. Cham, Switzerland. Springer; 2020.
• Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. May 23, 2011;6:42. [ CrossRef ] [ Medline ]
• Mouatt B, Smith AE, Mellow ML, Parfitt G, Smith RT, Stanton TR. The use of virtual reality to influence motivation, affect, enjoyment, and engagement during exercise: a scoping review. Front Virtual Real. Dec 23, 2020;1 [ CrossRef ]
• Zimmerli L, Jacky M, Lünenburger L, Riener R, Bolliger M. Increasing patient engagement during virtual reality-based motor rehabilitation. Arch Phys Med Rehabil. Oct 2013;94(9):1737-1746. [ CrossRef ] [ Medline ]
• Charles D, Holmes D, Charles T. Virtual reality design for stroke rehabilitation. In: Rea PM, editor. Biomedical Visualisation. Volume 6. Cham, Switzerland. Springer; 2020;53-87.
• Lorenz M, Brade J, Klimant P, Heyde C, Hammer N. Age and gender effects on presence, user experience and usability in virtual environments-first insights. PLoS One. Mar 27, 2023;18(3):e0283565. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tuena C, Pedroli E, Trimarchi PD, Gallucci A, Chiappini M, Goulene K, et al. Usability issues of clinical and research applications of virtual reality in older people: a systematic review. Front Hum Neurosci. Apr 8, 2020;14:93. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lohse K, Shirzad N, Verster A, Hodges N, Van der Loos HF. Video games and rehabilitation: using design principles to enhance engagement in physical therapy. J Neurol Phys Ther. Dec 2013;37(4):166-175. [ CrossRef ] [ Medline ]
• Richter G, Raban DR, Rafaeli S. Studying gamification: the effect of rewards and incentives on motivation. In: Reiners T, Wood LC, editors. Gamification in Education and Business. Cham, Switzerland. Springer; 2015;21-46.
• Yildirim IG. Time pressure as video game design element and basic need satisfaction. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2016. Presented at: CHI EA '16; May 7-12, 2016;7-12; San Jose, CA. URL: https://dl.acm.org/doi/10.1145/2851581.2892298 [ CrossRef ]
• Dekker J, de Groot V, Ter Steeg AM, Vloothuis J, Holla J, Collette E, et al. Setting meaningful goals in rehabilitation: rationale and practical tool. Clin Rehabil. Jan 18, 2020;34(1):3-12. [ CrossRef ] [ Medline ]
• Høeg E, Becermen B, Bruun-Pedersen JR, Serafin S. Co-creating virtual reality applications for motor rehabilitation with physiotherapists. In: Proceedings of the 8th EAI International Conference, ArtsIT 2019, and 4th EAI International Conference, DLI 2019 on Interactivity, Game Creation, Design, Learning, and Innovation. 2019. Presented at: ArtsIT DLI '19; November 6-8, 2019;379-389; Aalborg, Denmark. URL: https://link.springer.com/chapter/10.1007/978-3-030-53294-9_26 [ CrossRef ]
• Pau M, Cocco E, Arippa F, Casu G, Porta M, Menascu S, et al. An immersive virtual kitchen training system for people with multiple sclerosis: a development and validation study. J Clin Med. May 30, 2023;12(9):3222. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lee JH, Ku J, Cho W, Hahn WY, Kim IY, Lee S, et al. A virtual reality system for the assessment and rehabilitation of the activities of daily living. Cyberpsychol Behav. Aug 2003;6(4):383-388. [ CrossRef ] [ Medline ]
• Knierim P, Kosch T, Schwind V, Funk M, Kiss F, Schneegass S, et al. Tactile drones - providing immersive tactile feedback in virtual reality through quadcopters. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2017. Presented at: CHI EA '17; May 6-11, 2017;6-11; Denver, CO. URL: https://dl.acm.org/doi/10.1145/3027063.3050426 [ CrossRef ]
• Collins K, Kapralos B. Pseudo-haptics: leveraging cross-modal perception in virtual environments. Senses Soc. Oct 14, 2019;14(3):313-329. [ CrossRef ]
• Threapleton K, Drummond A, Standen P. Virtual rehabilitation: what are the practical barriers for home-based research? Digit Health. Apr 29, 2016;2:2055207616641302. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Clarke R, Coote S. Perceptions of participants in a group, community, exercise programme for people with multiple sclerosis. Rehabil Res Pract. 2015;2015:123494-123497. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arntzen EC, Straume BK, Odeh F, Feys P, Zanaboni P, Normann B. Group-based individualized comprehensive core stability intervention improves balance in persons with multiple sclerosis: a randomized controlled trial. Phys Ther. Aug 01, 2019;99(8):1027-1038. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Anderson-Hanley C, Snyder AL, Nimon JP, Arciero PJ. Social facilitation in virtual reality-enhanced exercise: competitiveness moderates exercise effort of older adults. Clin Interv Aging. 2011;6:275-280. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Geurts E, Van Geel F, Feys P, Geurts E. WalkWithMe: personalized goal setting and coaching for walking in people with multiple sclerosis. In: Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. 2019. Presented at: UMAP '19; June 9-12, 2019;51-60; Larnaca, Cyprus. URL: https://dl.acm.org/doi/10.1145/3320435.3320459 [ CrossRef ]
• Cuesta-Gómez A, Sánchez-Herrera-Baeza P, Oña-Simbaña ED, Martínez-Medina A, Ortiz-Comino C, Balaguer-Bernaldo-de-Quirós C, et al. Effects of virtual reality associated with serious games for upper limb rehabilitation inpatients with multiple sclerosis: randomized controlled trial. J Neuroeng Rehabil. Jul 13, 2020;17(1):90. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by C Prahm; submitted 02.08.23; peer-reviewed by J Wu, C Srikesavan, L Bulle; comments to author 21.12.23; revised version received 14.02.24; accepted 14.03.24; published 26.04.24.

©Amy Webster, Matthieu Poyade, Elaine Coulter, Lisa Forrest, Lorna Paul. Originally published in JMIR Serious Games (https://games.jmir.org), 26.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on https://games.jmir.org, as well as this copyright and license information must be included.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Environ Res Public Health

The Effectiveness of Virtual Reality Exercise on Individual’s Physiological, Psychological and Rehabilitative Outcomes: A Systematic Review

1 School of Kinesiology, the University of Minnesota, 1900 University Ave. SE, Minneapolis, MN 55455, USA; ude.nmu@3310naiq (J.Q.); ude.nmu@5870odcm (D.J.M.)

2 Department of Physical Education, Shanghai Jiao Tong University, Minhang District, Shanghai 200240, China

Daniel J. McDonough

Objective purpose: This review synthesized the literature examining the effects of virtual reality (VR)-based exercise on physiological, psychological, and rehabilitative outcomes in various populations. Design: A systematic review. Data sources: 246 articles were retrieved using key words, such as “VR”, “exercise intervention”, “physiological”, “psychology”, and “rehabilitation” through nine databases including Academic Search Premier and PubMed. Eligibility criteria for selecting studies: 15 articles which met the following criteria were included in the review: (1) peer-reviewed; (2) published in English; (3) randomized controlled trials (RCTs), controlled trials or causal-comparative design; (4) interventions using VR devices; and (5) examined effects on physiological, psychological, and/or rehabilitative outcomes. Descriptive and thematic analyses were used. Results: Of the 12 articles examining physiological outcomes, eight showed a positive effect on physical fitness, muscle strength, balance, and extremity function. Only four articles examined the effects on psychological outcomes, three showed positive effects such that VR exercise could ease fatigue, tension, and depression and induce calmness and enhance quality of life. Nine articles investigated the effects of VR-based exercise on rehabilitative outcomes with physiological and/or psychological outcomes, and six observed significant positive changes. In detail, patients who suffered from chronic stroke, hemodialysis, spinal-cord injury, cerebral palsy in early ages, and cognitive decline usually saw better improvements using VR-based exercise. Conclusion: The findings suggest that VR exercise has the potential to exert a positive impact on individual’s physiological, psychological, and rehabilitative outcomes compared with traditional exercise. However, the quality, quantity, and sample size of existing studies are far from ideal. Therefore, more rigorous studies are needed to confirm the observed positive effects.

1. Introduction

Over the past decades, the effects of physical activity (PA) on individual’s health have been well documented [ 1 , 2 , 3 ]. However, despite the well-known benefits of PA participation, according to the World Health Organization (WHO) approximately 25% of adults and 80% of adolescents around the world are physically inactive partly due to societal and lifestyle changes [ 4 ]. Exercise (i.e., planned, structured and repetitive PA) is often perceived as boring and hard, thereby causing adults and students shy away from PA-related behaviors after long days of work and/or school. Instead, individuals are more interested in leisure activities, such as video games, where entertainment can be obtained while relaxing (i.e., sedentary behavior). Thus, the combination of video games and engaging in PA (e.g., virtual reality (VR)-integrated exercise) may trigger their interest and improve their PA behavior.

In recent years, VR exercise has been recognized as a new approach to promote PA and health behaviors [ 5 ] and is becoming increasingly used in health promotion. Researchers have observed VR exercise to enhance the psychological benefits of exercise and increase the likelihood of long-term adherence to exercise [ 6 , 7 ]. VR is operationally defined as digital technology wherein sensory experiences, (e.g., visual, auditory, touch, and scent stimuli) are artificially created, prompting users to manipulate the objects within virtual environment [ 8 ]. In general, there are three types of VR: immersive, non-immersive, and interactive. Immersive VR utilizes head-mounted displays, body movement sensors, real-time graphics, and advanced interface devices (e.g., dedicated headsets) to simulate a completely virtual environment for users, whereas non-immersive VR utilizes an interface, such as a flat screen TV/computer screen, and requires the use of a corresponding keyboard, controller and/or joystick [ 9 , 10 ]. Interactive VR is centered on the user’s ability to interact with virtual objects through devices (e.g., gloves, digital glasses) which produce the sensation of manipulating real items, such as picking up an apple [ 11 ].

The development of VR technology and its utility during PA via its integration with traditional exercise equipment and rehabilitation practices has attracted attention in the fields of kinesiology and public health. As a therapeutic tool, VR offers the opportunity to intensify repetitive tasks and increase visual and auditory feedback, making VR therapy more interesting than traditional physical therapy and without posing any serious threat or physical limitations to participants [ 11 ]. Previous reviews have examined the effectiveness of VR exercise on physiological, psychological or rehabilitative outcomes. For example, researchers suggested that VR could promote the lower limb function of patients who suffered from stroke [ 12 ]. VR exercise has also shown to have a significant effect on the balance ability of patients who suffered from stroke, Parkinson’s disease (PD) or children with cerebral palsy (CP) [ 13 ]. Additionally, the effectiveness of the application of VR in psychological treatment in psychotherapy has been widely supported [ 14 ]. For instance, VR exercise has been observed to relieve anxiety and depression [ 15 ]. As for rehabilitative effectiveness, VR technology in disease rehabilitation has been widely applied, namely to help disabled patients acquire lost motor skills caused by injury or illness and ensure these individuals are able to carry out activities of daily living. As such, the effectiveness of VR exercise on physiological and rehabilitative outcomes have been mostly related and combined.

Thus far, a meta-analysis demonstrated the positive effects of VR exercise on balance function in stroke patients [ 16 ]. Similarly, another review suggested that VR was useful for enhancing motor control, functional, and cognitive abilities and balance in Parkinson’s patients [ 13 ]. Distinctly, the target populations of the studies within the preceding literature review were narrow and limited to specific diseases, such as strokes and Parkinson’s disease. However, there are other vulnerable populations who may benefit from VR exercise, such as the elderly who could benefit from improved balance and other physical abilities to facilitate better health-related quality of life (HRQoL). However, reviews examining the utilization of VR exercise to intervene in such populations are sparse. Moreover, many relevant reviews are outdated, and thus there is a need to synthesize more updated research. Furthermore, some reviews included single-case experimental designs (i.e., studies with no control group), and thus these reviews were not based on high-quality research and the findings need to be further explored. It has been suggested researchers should include more rigorous study designs like randomized controlled trials (RCTs).

Therefore, this review aimed to fill the existing research gaps. Specifically, most of the articles included in this review were RCTs. Even if some studies could not be randomly grouped on a large scale due to the sample limitations, studies were only included if there were control group(s) and employed a comparative analysis between experimental groups and control groups, as well as between baseline- and post-tests for the examined health outcomes. Further, the target population of this review was relatively extensive, including clinical and healthy populations, allowing the overall effectiveness of VR exercise to be established. Taken together, the purpose of this review is to systematically synthesize the literature examining the effects of VR exercise on the physiological, psychological, and rehabilitative outcomes in various populations.

The framework and reporting of this review were based on the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement [ 17 ].

2.1. Information Sources and Search Strategies

The electronic databases were used to conduct the literature search were as follows: Academic Search Complete, Communication and Mass Media Complete, Education Resources Information Center (ERIC), PubMed, Scopus, Web of Science, PsycINFO, SPORTDiscus, and Medline. Within these databases, the following terms and phrases were searched: (“virtual reality” OR “VR exercise” OR “head-mounted display”) AND (“exercise” OR “physical activity” OR “sports” OR “bike” OR “treadmill”) AND (“physical” OR “movement” OR “physiological outcomes”) AND (“psychology” OR “cognition” OR “mental health” OR “psychological outcomes”) AND (“rehabilitation” OR “recovery” OR “rehabilitative process”). The literature search was conducted independently by all authors and all relevant studies were placed in a shared Google folder.

2.2. Eligibility Criteria

Five main eligibility criteria were used to evaluate each study. In detail, articles were included in this review if they: (1) were peer-reviewed and published in English between January 2000 and April 2020; (2) used VR-integrated exercise equipment; (3) intervened on human subjects; (4) used quantitative methods to evaluate the results related to physiological, psychological, and/or rehabilitative outcomes; and (5) employed an established study design (e.g., RCT, controlled trial, and causal-comparative design).

2.3. Data Extraction

Three reviewers (authors J.Q., D.M., and Z.G.) screened the titles of potentially relevant articles. The abstracts of these articles were then further reviewed to ensure relevance to the research topic. Data extraction was completed by one reviewer (J.Q.) and checked for accuracy by another reviewer (D.M.). All potential articles were downloaded in full and stored in a shared Google folder and three authors (J.Q., D.M., and Z.G.) reviewed each article independently to ensure that only relevant entries were included. We extracted the following information: (1) publication year and country; (2) study design (i.e., sample characteristics, study duration, VR exposure, results related to physiological, rehabilitation, and/or psychological outcomes, and instruments used); and (3) key findings regarding the effectiveness of VR in outcomes related to the preceding outcomes. Finally, we cross-referenced the bibliographies of the selected articles to further identify the relevant research. It is important to note that no reviewers were influenced by the authors of selected publications or members of journals, nor did we attempt to contact the investigators or journals of the original study for any missing information in the included article.

2.4. Risk of Bias in Individual Studies

The process of assessing risk of bias for each study was independently performed by two reviewers (J.Q. and D.M.), using eight quality assessment tools from the previous literature ( Table 1 ) [ 9 , 18 , 19 , 20 ]. When the project was clearly described and presented, the study was recorded as “+” (positive) and if the project description was inadequate or missing, the study was recorded as “−” (negative). Further, two reviewers (J.Q. and D.M.) rated each article independently to ensure reliability of the quality assessment. When these two reviewers had different opinions, the third reviewer (Z.G.) re-evaluated the objection. Among all eight indicators, randomization, pre-test/post-test study designs, study retention, and the use a power analysis were considered as the most important factors as they had the most profound impact on research results. The final score for each study was calculated from the sum of all “+” evaluations. The studies which were evaluated as “high quality and low risk of bias” were signified by a score greater than the median score of 5, whereas “low quality and high risk for bias” studies were those which scored lower than the median score 5.

Design quality analysis.

Note: + refers to positive (explicitly described and present in details); − refers to negative (inadequately described and absent); YES effectiveness indicates significant positive effect; NA indicates no significant effect; Median score = 5. Retention: retaining more than 70% of the participants; Follow-up: following more than 6 months after experiment.

3.1. Study Selection

This study initially included 246 related articles after the initial research. After further examination of the titles and abstracts of these articles, duplicate papers were excluded. Further, by meeting all predetermined eligibility criteria, 15 articles met the inclusion requirements and were included in this review ( Figure 1 ). We chose to disregard some studies upon further investigation for several reasons: (1) articles were not peer-reviewed; (2) articles were not published in English; and (3) articles which did not use VR exercise (e.g., some focused on VR only or VR exposure therapy (VRET) or Virtual Reality in Psychological Treatment (VRT)).

Flow diagram of studies through the review process. Note. * reasons for exclusions included ineligible age, ineligible exposure, ineligible analysis; ** reasons for exclusions included ineligible outcomes and lack of means/standard deviations.

The characteristics of the included studies are shown in the Table 2 , which includes 11 RCTs [ 7 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 30 , 32 , 34 ], three controlled trials [ 28 , 31 , 33 ] and one causal-comparative study [ 29 ]. In detail, all 11 RCTs used baseline and post-test results from the intervention and control groups as the basis of their conclusions [ 7 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 30 , 32 , 34 ]. Moreover, the three controlled trials compared the inferential statistics at baseline and post-test of the predetermined control and experimental groups [ 28 , 31 , 33 ]. Lastly, the causal-comparative study analyzed differences between the control group and the experimental group [ 29 ].

Descriptive characteristics of included studies.

Note: RCT = Randomized Controlled Trial; VR = Virtual Reality; FES = Functional Electrical Stimulation; IDD = Intellectual and Developmental Disability; GMFCS = Gross Motor Function Classification System; CPc = Children with Cerebral Palsy; TA = Tibialis Anterior; MG = Medial Gastrocnemius.

Experiments were conducted between 2009 and 2019 and were conducted in different countries: six in South Korea [ 21 , 24 , 25 , 27 , 32 , 33 ], two in Canada [ 28 , 30 ], and one in Brazil [ 22 ], Taiwan [ 23 ], Israel [ 26 ], Australia [ 29 ], the US [ 7 ], Belgium [ 31 ], and Spain [ 34 ]. The sample size varied from 11 to 121 and the age of participants ranged from children (five years old) to the elderly (≥70 years old), among which the majority of articles targeted adults and the older populations. The included studies also included specific groups, such as pregnant women [ 22 ] and patients with certain diseases (e.g., stroke, cognitive decline) [ 21 , 23 , 24 , 26 , 28 , 30 , 31 , 32 , 34 ]. Intervention periods varied by study as different studies examined different cause–effect relationships. Studies examining psychological outcomes primarily employed acute interventions (i.e., a single bout of intervention). Further, interventions that examined physiological outcomes employed interventions ranging from two weeks to 12 weeks and were primarily concerned with the effect of VR exercise on the function of the upper and lower limbs, balance and body fitness. The third research query focused on rehabilitation from a disease, such as stroke hemodialysis, spinal-cord, intellectual and development disabilities (IDD), cerebral palsy (CP), and cognitive decline (CD), with the physiological and psychological outcomes mentioned previously. These studies typically had flexible intervention periods, depending on the status of the patient and time of discharge. In these studies, the VR interventions most commonly used were Nintendo products and the specific games played varied based on the different research purposes across studies. For example, Nintendo Balance was used during interventions in which balance was the primary outcome. In terms of study setting, four studies were conducted in a rehabilitation center [ 21 , 28 , 30 , 31 ], three in a laboratory setting [ 7 , 22 , 23 , 29 ], and the rest did not report specifically where the study was conducted [ 24 , 25 , 26 , 27 , 32 , 33 , 34 ].

3.2. Quality and Risk of Bias Assessment

The risk assessment table for bias among the included studies is shown in Table 1 . In detail, study quality ranged from three points to seven points with a median of five points. Eight of the included studies scored equal to or greater than median score of five and were therefore considered high quality, while six of the included studies scored lower than median score of five and were consequently considered low quality. The majority of the articles retained at least 70% of the participants and the measurement tools in all articles were valid. However, only six articles accounted for the analysis of missing values, five articles employed a power analysis prior to experiment, and no articles reported six-month follow-up post-intervention. The low scores were attributed to missing data, the absence of a power analysis and a lack of follow-up.

3.3. Data Items

Among all included studies, the outcomes of interest were divided into three categories: physiology, psychology, and rehabilitation. Noteworthy is the fact that rehabilitation outcomes could not be assessed independently from physiological and/or psychological outcomes which were indicators of the rehabilitation effect. In these articles, the physiological indicators included upper and lower limb function, balance, fitness, body composition, muscle function, and muscular strength. The psychological indicators included tension, depression, affection, attention, and fatigue. Additionally, the diseases of rehabilitation in studies were stoke disease, hemodialysis, spinal cord, intellectual and development disabilities, cerebral palsy on young people and cognitive decline.

3.4. Measurement Protocol

The measurement methods in the included studies were valid measurement methods or scales and the process of data collection was carried out by experienced staff. For example, for balance, the Berg Balance Scale (BBS) [ 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 ] was the most common measurement tool, followed by the Good Balance System and the one-legged stance test [ 25 ], the Trunk Control Measurement Scale (TCMS) [ 31 ], and the balance testing paradigm used in their previous research [ 28 ]. One article used Qualisys Track Manager (QTM) to measure the kinematic variables on sit-to-stand balance in pregnant women [ 22 ]. As for limb function and strength, the Wolf Motor Function Test (WMFT) [ 21 , 30 ] was primarily used to assess the function of the upper limbs, followed by the Fugl-Meyer Assessment-Upper Extremity (FMA-UE) [ 21 ]. The sit-to-stand test was used to assess function the lower limbs [ 25 ], and electromyography was used in the measurement of limb and lower limb muscle activity [ 33 ]. As for fitness, two articles on hemodialysis patients measured muscular strength, flexibility, balance, body component, and fatigue by some physical function tests, physical activity questionnaire and HRQoL [ 24 , 34 ], while the other measured heart rate (HR) [ 26 ].

As for the psychological outcomes, the Activation-Deactivation Adjective Check List (AD-ACL) was used to assess mood state [ 7 , 23 ], the Physical Activity Affect Scale (PAAS) for affect [ 29 ], the Measure of Attentional Focus (MAF) for attention [ 29 ], the Geriatric Depression Scale-Korean (GDS-K) and the Korean version of quality of life Alzheimer’s disease (KQOL-AD) scale for HRQoL [ 32 ].

3.5. The Effectiveness of VR on Physiological Outcomes

A total of 12 studies included the examination of physiological indexes (including eight studies which also targeted rehabilitation), among which eight studies observed VR to have positive effects on certain physiological indexes, including upper and lower limb function, fitness, body composition, balance, muscle function and muscular strength.

All the articles which examined the effectiveness of VR on physiological indicators observed balance ability to be most strongly positively affected, especially among the elderly, specifically the healthy elderly [ 25 , 27 ] and elderly with CD [ 32 ]. Further, a positive effect was also found on sitting balance in children with CP [ 31 ], while no impact on postural control balance for CP was observed [ 28 ]. One more article which examined the balance of sit-to-stand (STS) in the second and third stages of pregnancy also failed to show improvement [ 22 ].

In terms of limb function, the included studies observed VR to have a significant impact on lower limb function in elderly people [ 25 ] and trunk and lower limb function in young people [ 33 ]. Interestingly, however, two studies which examined the function of upper limbs in stroke patients found differential effects [ 21 , 30 ].

Three studies examined individuals’ fitness-related outcomes, but the fitness outcomes were different. Two articles examining balance, flexibility and muscular strength among hemodialysis patients demonstrated a significant impact [ 24 , 34 ]. As for HR on IDD, even if there was a significant improvement statistically, the researchers claimed that it still could not be proven to improve physical fitness [ 26 ].

In summary, four studies observed VR to have null effect on physiological outcomes, most notably among pregnant women, those with IDD, patients with CP and patients with chronic stroke.

3.6. The Effectiveness of VR on Psychological Outcomes

Among the articles included in this review, there were relatively fewer studies which examined the effect of VR exercise on psychological outcomes. Three of four articles (including one which also targeted rehabilitation) demonstrated positive effects. All of these studies, however, demonstrated VR exercise to effectively relieve fatigue, reduce depression tendency and increase HRQoL, whether the subjects were healthy or suffering from chronic illness [ 7 , 23 , 32 ]. However, the psychological indexes regarding affection and attention demonstrated null effects [ 29 ].

3.7. The Effectiveness of VR on Rehabilitative Outcomes

The main difference between the rehabilitation-centered articles and the other included articles was that the participants who were in rehabilitation suffered from physical and/or mental illnesses. Among nine articles, six indicated VR exercise to have positive effects on various rehabilitative outcomes.

In detail, the results showed that VR exercise elicited positive effects on physiological factors, balance [ 31 , 32 ], extremity function [ 21 ] and fitness [ 24 ], psychological outcomes, and a weakened nervous system from disease [ 23 ]. Two studies had a sample of patients with CP, one of which observed null effect on body control [ 28 ], while the other demonstrated a positive effect on sitting balance [ 31 ]. The results regarding physical fitness showed hemodialysis patients to have positive effects [ 24 , 34 ], while IDD patients observed null effects [ 26 ]. In terms of easing tension and depression, both spinal cord and CD patients using VR exercise saw significant reductions [ 23 , 32 ]. Moreover, VR exercise also had a positive effect on the balance of CD patients [ 32 ]. As mentioned above, however, the two studies which examined the effect of VR exercise on middle-aged stroke patients’ upper limb function showed different results [ 21 , 30 ].

4. Discussion

Exercise has been considered as a prescription for healthy individuals and clinical populations with various diseases [ 35 ]. The main purpose of this review was to synthesize and review the latest available literature examining the effects of VR exercise on individuals’ physiological, psychological, and rehabilitation outcomes among various populations ( Table 3 ). Fifteen studies were included in this review, with 12 examining physiological outcomes, four examining psychological outcomes, and nine examining rehabilitation in which the judgment of rehabilitation was determined by physical and/or psychological indicators. Among these studies, the positive effect of VR on health outcomes was greater than 60%. Some studies demonstrated null effects, but no study observed negative effects. Interestingly, some studies observed different findings despite using similar or identical instruments and measurement procedures due to the different study protocols and samples.

The main pathologies, function and effect of VR among various populations.

Abbreviations: CPc = Children with Cerebral Palsy.

With regard to physiological outcomes, the two studies that examined CP in minors (i.e., those >18 years old) [ 28 , 31 ] demonstrated opposing conclusions. Although balance was the main outcome of each study, conflicting tasks and study protocols prompted different results. Indeed, the studies which focused on position control observed null effects [ 28 ] while others focusing on sitting balance displayed positive effects [ 31 ]. Sitting balance was considered to be relatively easy compared to posture control, and thus short-term effects were more readily observed. Moreover, STS exercises required torque on each joint to complete the task [ 36 ] and therefore when going from a seated position to standing specific strategies should be employed to promote proper implementation. For example, when the elderly perform such tasks, support from upper extremities may serve as a beneficial tool by which to increase the stability of the position [ 22 ]. Similarly, another study discerning the body control of pregnant women [ 22 ] failed to demonstrate a positive effect. Unlike healthy/normal populations during exercise, pregnant women emphasize the safety of their babies rather than trying to enhance their physical fitness or attenuate their chronic diseases, and therefore tend to reduce the range of motion of the exercises [ 22 ]. Moreover, it was found that STS movement did not observe any significant differences between pregnant women within different gestation periods, which suggests that pregnant women tend to be aware of their postural instability and in turn this may make them more cautious about falling during functional activities [ 22 ].

These findings were also observed concerning the upper limb function of stroke patients, such that conflicting results were observed despite using the same measurement tools and protocols [ 21 , 30 ]. Different instruments and different intervention periods likely contributed to the conflicting results. Functional Electrical Stimulation (FES) prompted beneficial effects in the upper limb functional rehabilitation after a stroke had been reported in a number of clinical trials including the improvement of motor function, movement range, activities of daily living (ADL) and flexibility [ 37 , 38 , 39 , 40 ] and was chosen to be a basic treatment of the upper limbs in Lee’s experiment [ 21 ]. VR (which used wearable gloves) was used as an aid in the treatment of FES in order to induce persistent movement [ 21 ]. However, based on conventional rehabilitation, VR (Nintendo Wii) was applied as an add-on in therapies which aimed to compare against general entertainment, such as bingo [ 30 ]. Moreover, the general entertainment was similar to the VR exercise environment so that participants were unable to distinguish between these two studies. Furthermore, these studies employed entirely different intervention periods. Specifically, the study which observed positive effects lasted for four weeks, while the other study only lasted for two weeks. Indeed, such short intervention periods made it difficult to distinguish the true effects of similar add-on therapies. A previous systematic review also reached a similar conclusion that virtual reality technology is more often used as a dose-increasing purpose to enhance the effectiveness of treatment [ 41 ]. However, a simple comparison between a comparison system and a commercial gaming system did not reveal significant differences in their effectiveness [ 41 ].

In light of psychological outcomes, VR exercise showed positive impacts on relieving mental tension [ 23 , 32 ] and improving HRQoL [ 7 , 32 ]. Traditional physical therapy may be perceived as a repetitive and lifeless task and participants often feel intimidated when confronted with cold, robotic rehabilitation equipment. However, when participants cycled in an interactive virtual environment, they enjoyed cycling, which was deemed as a fun activity rather than a daunting task like traditional therapy. Such improvements in mood and affect during exercise may facilitate greater endurance during exercise and may therefore promote greater effects [ 23 ]. VR-based exercise programs, for example, were always seen by patients as games rather than therapeutic situations. Responding to the audio-visual stimuli received through the screen made participants more interested in the program, which facilitated their motor persistence and concentration. Furthermore, patients who were immersed in play-based sports were encouraged by a competitive spirit and motivation to score more points during the game. In these circumstances, the reward circuit of the brain may be activated and dopamine production may increase as a result of the placebo, thereby easing depression [ 42 ]. However, little effect was observed for emotion and attention during VR exercises [ 29 ]. Young adults, who were accustomed to VR devices found it difficult to concentrate on the exercise and gained satisfaction from it, especially those who were familiar with computer games [ 7 ]. On the contrary, the older populations might have more potential to fulfill their curiosity when using novel equipment and may therefore gain pleasure from using equipment like VR during exercise. Gender was also a factor which may have influenced similar conclusions. Generally speaking, male participants tended to play more video games than females, and thus they were more familiar with the VR-enhanced computer experience [ 42 , 43 ] and observed less novelty during experiment. Hence, the experience level of participants with VR apparatuses may have contributed to the observed different results.

In regards to rehabilitative outcomes, the HR results of IDD patients [ 26 ] were controversial. Due to the disability status of IDD patients [ 26 ], PA is always facilitated by guardians or caretakers, and therefore, this population is heavily sedentary and often has poor health statuses [ 26 ]. Additionally, those with IDD lack the ability to participate in sports activities [ 44 ], especially those who live in residential environments [ 45 ]. But the main reason behind the contradictory results was the choice of fitness-related outcomes. Balance, speed, flexibility, and other outcomes are all indexes of physical fitness. However, these studies used HR as an outcome, which was not a representative fitness indicator.

Overall, VR has potential for enhancing physiological, psychological, and rehabilitative outcomes among healthy and clinical populations, although a few articles observed null effects. To discern the reasons underlying the contradictory results, we must examine the experimental protocols used in the included studies—task, sample, instrument, intervention length, intervention fidelity, experience level with VR, and specific items selected from general measurements were the key to intervention success. We speculate that VR will promote positive effects in the long-term. However, some null effects of immersive devices were found during testing. It was reported that some participants experienced disorientation, nausea and vision problems during or after the use of the VR devices [ 46 ].

Since the majority of the included studies were RCTs, the conclusions of this review are relatively valid. However, some limitations of this review must be noted: (1) only articles published in English were included, which may exclude relevant research published in other languages; (2) due to our rigorous inclusion criteria, only a limited number of studies were included in this review; (3) a few included studies had based their protocols in highly-controlled laboratory settings, which may limit the external validity of the findings to some extent, and some studies neglected to report their experimental environment and thus the clinical results need to be further confirmed. Future research should employ high quality designs (e.g., RCTs) among various potential populations (e.g., a depression population) [ 47 ]. Additionally, the intervention length of future studies should be extended, and follow-ups should be executed post-intervention to ensure the long-term intervention effectiveness. Moreover, potential confounding factors, such as gender, age, and socio-economic status should also be taken into account. Lastly, different perceptions of VR may produce different effects, and therefore should be discussed carefully in different situations and contexts. Nevertheless, based upon our review of the available literature, we conclude that VR is safe and has potential for improving physiological, psychological, and rehabilitative outcomes if implemented well by trained professionals [ 48 ].

The findings of this review have practical implications for researchers and health practitioners. Indeed, utilizing VR-based exercise rehabilitation training in hospitals might be conducive to the physical and mental health of patients due to the fact that VR has positive effects on some aspects of physiology and stress relief during the rehabilitation period. Similar advice may be applied to the public as well, such that using VR properly can be a more life-like and effective way to develop and maintain healthy lifestyles. Given the potential negative effects of immersive VR devices (e.g., motion sickness), rehabilitation centers and healthcare providers can choose other devices, such as non-immersive or interactive devises, based upon reality. Further, the price of VR equipment is inconsistent and must be taken into account. For ordinary consumers, they may choose a light and simple device or use apps on their smartphone which are similar to the VR devices, while the professional rehabilitation centers may choose higher-quality, multi-function VR devices to meet the needs of target populations.

5. Conclusions

Our findings suggest that VR exercise has the potential to exert a positive impact on individual’s physiological, psychological, and rehabilitative outcomes compared with traditional exercise. However, the quality, quantity, and sample size of existing studies are far from ideal. Therefore, more rigorous studies are needed to confirm these positive effects.

Author Contributions

The author would like to thank the co-authors for their help in completing this study. During the construction of this study, J.Q. played a role in data collection, sorting, analysis, and writing the article; D.J.M. played a role in data sorting and helping write the article; Z.G. played a role in developing the research ideas, overseeing data collection and analysis, and revising the article. All authors have read and agreed to the published version of the manuscript.

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.