The ”Human-Centered Intelligent Realities” (HINTS) laboratory is a strategic infrastructure project aiming to support research that advances the development of immersive, user-aware, and intellige...The ”Human-Centered Intelligent Realities” (HINTS) laboratory is a strategic infrastructure project aiming to support research that advances the development of immersive, user-aware, and intelligent digital environments by integrating augmented reality (AR), virtual reality (VR), extended reality (XR), artificial intelligence (AI), and machine learning (ML). By combining virtual reality and communication-computing continuums, the HINTS environment seeks to create innovative concepts, methods, and tools that empower users to engage with digital systems in novel, efficient, and effective ways. Research in the HINTS laboratory focuses on experience assessment, new digital environments and interaction techniques, visual analytics, adaptive AI, and networking. This paper presents the HINTS laboratory, ongoing activities, and opportunities and challenges for the future.
Recent advances relating to hands-free interaction methods for the physical control of telepresence robots provide alternatives for conditions where hand-use is not an option. Gaze interaction comes w...Recent advances relating to hands-free interaction methods for the physical control of telepresence robots provide alternatives for conditions where hand-use is not an option. Gaze interaction comes with unique advantages compared to other input methods. However, the use of gaze control for navigating telepresence robots with a head-mounted display (HMD) is a subject that ought to be further explored. Additionally, it is necessary to explore more effective techniques for assessing key metrics in telepresence robot navigation, particularly when evaluating novel interaction methods. In particular, situational awareness (SA) is a key psychological metric, but challenges remain regarding the reliability and validity of its measurement. We conducted an experiment to: 1) compare gaze control with manual control (joystick) of telepresence robots using an HMD; 2) compare two types of SA measurement techniques: a post-trial subjective measure, the Situation Awareness Rating Technique (SART) and our proposed adaptation of an existing real-time objective measure, the Situation Present Assessment Method (SPAM). Results showed that the gaze control method differs significantly from the manual control with a joystick, leading to poorer navigation performance, higher workload, lower SA (SPAM-based measures), and less accurate post-trial estimation and recollection. Regarding the SA techniques, comparison results indicated the SPAM-based method provided more reliable data than the SART questionnaire.
Large-scale screening programs for vision impairments can incur substantial costs. Computer-based screening methods, which combine different measurements within a single system, can facilitate and red...Large-scale screening programs for vision impairments can incur substantial costs. Computer-based screening methods, which combine different measurements within a single system, can facilitate and reduce the costs of such programs. Here, we present a virtual reality (VR) software, which includes tests for the assessment of visual acuity, stereoacuity, eye misalignments, and interocular suppression as well as games targeting different visual functions that may serve as treatment methods. The software can be easily extended to incorporate new tests and games. We present a proof of concept demonstrating the functionality of the software and its applicability in individuals with impaired binocularity. We evaluate a stereoacuity test in VR based on disparity detection using contoured objects by comparing its results to those obtained by standard clinical tests, i.e., TNO, Randot, and Titmus, for 7 amblyopes and 6 healthy controls. We evaluate the applicability of a new VR-based suppression test in 10 amblyopes and 6 healthy individuals. For the latter, we exploit the effects of short-term monocular deprivation, which induce a change of ocular dominance. Finally, we outline technical limitations and discuss potential applications.
This study explores dynamic and adaptable human cooperative behavior in team sports such as soccer or handball, emphasizing the sharing of intentions and actions among players. A key factor in this co...This study explores dynamic and adaptable human cooperative behavior in team sports such as soccer or handball, emphasizing the sharing of intentions and actions among players. A key factor in this context is the gaze direction of players, which is crucial for assessing situations and inferring teammates’ and opponents’ intentions, ultimately guiding practical actions. Recent advances in virtual reality (VR) technology have enabled detailed analysis of such behaviors and decision-making processes, facilitating experimental learning scenarios in cooperative settings. In this research, we investigate human gaze behavior in soccer from a firstperson perspective, using head-mounted displays (HMDs) and virtual environments to develop supportive learning systems. Through experiments in which subjects experience offensive scenarios from a real player’s viewpoint within a VR environment, we analyze how their gaze behavior changes during different phases of passing and attacking in the game.
Objective
This study aimed to analyze gaze behavior when viewing real-world driving videos in a virtual reality (VR) environment.
Methods
A total of 22 driving instructors and 36 older drivers, a...Objective
This study aimed to analyze gaze behavior when viewing real-world driving videos in a virtual reality (VR) environment.
Methods
A total of 22 driving instructors and 36 older drivers, all with regular driver’s licenses, participated in this study. Participants watched 360° real-world driving videos using a head-mounted display equipped with eye-tracking functionality (FOVE 0). Areas of interest (AOIs), such as traffic signals and side mirrors, were designated within the videos. Gaze behavior was evaluated using indicators such as time to first fixation, dwell time (DT), and revisit count (RC).
Results
In many scenarios, older drivers reached the AOIs significantly later, had shorter DT, and lower RC than those of driving instructors. These differences were particularly pronounced during right and left turns and lane changes, suggesting that older drivers may have insufficient recognition of surrounding risks. In addition, older drivers exhibited delays in visual attention, indicating a lack of attention to the surrounding environment.
Conclusions
Gaze analysis in a VR environment is a valuable method for safely evaluating the gaze behavior of older drivers. This study revealed important differences in gaze behavior between driving instructors and older drivers. These findings have practical implications for improving the safety of older drivers. By understanding their unique gaze behavior, targeted interventions can be developed to improve the safety of older drivers.
Pupillometry is an emerging, noninvasive technique that measures pupil reactivity and provides deeper insights into ophthalmic and neurologic function. Extended reality (XR) technology has also emerge...Pupillometry is an emerging, noninvasive technique that measures pupil reactivity and provides deeper insights into ophthalmic and neurologic function. Extended reality (XR) technology has also emerged with powerful clinical capabilities in various medical specialties, particularly in neuro-ophthalmology functional testing. This article explores the use of XR technology in pupillometry. XR encompasses various immersive technologies, including virtual reality, augmented reality, and mixed reality. By integrating eye-tracking technology into these systems, precise measurements of ocular movements, including pupil dilation and constriction, can be obtained. We propose the term “XR-Pupillometry” to describe this novel approach. Our manuscript discusses the potential benefits and limitations of XR-Pupillometry and highlights its applications in various fields, including psychology, neuroscience, space research, and health care. We also provide an overview of existing devices and software available for XR-Pupillometry. As this technology continues to evolve, it has the potential to revolutionize the way we understand human behavior and emotions.
In order to explore presenters' social presence, this study analyzed what elements of English presentation videos affected viewers' attention. Brain and eye movements of two university students were m...In order to explore presenters' social presence, this study analyzed what elements of English presentation videos affected viewers' attention. Brain and eye movements of two university students were measured while watching two presentation videos. An interview was conducted to help construe those data. The data was analyzed, and results showed an increase in cerebral blood flow in situations where the presenter expressed opinions, along with the importance of elaboration on structuring of a presentation and prosody. Furthermore, eye tracking and interview revealed that gestures positively affected the presenters' social presence. However, when opinions were being stated, the gaze was directed to the text rather than the presenters' face, suggesting that looking at the presenter's face did not necessarily lead to a meaningful improvement in social presence.
Aim
This study aimed to verify the usability of our newly developed virtual reality-based cognitive function examination (VR-E) to differentiate mild cognitive impairment (MCI) from normal cognition ...Aim
This study aimed to verify the usability of our newly developed virtual reality-based cognitive function examination (VR-E) to differentiate mild cognitive impairment (MCI) from normal cognition and mild dementia.
Method
The subjects of analysis were 71 people (26 males and 45 females, aged from 59 to 94), including 31 with normal cognitive function, 26 with MCI, and 14 with mild dementia, according to the Clinical Dementia Rating (CDR). The total score and each cognitive domain (memory, judgement, spatial cognition, calculation, language function) score of VR-E were compared among CDR0, CDR0.5 and CDR1. In addition, for CDR0 vs. CDR0.5 and CDR 0.5 vs. 1, the areas under the curve (ROC) were examined using the total score and each cognitive domain of VR-E.
Results
There were significant differences between the three CDR groups in the VR-E scores as well as each VR-E cognitive domain score. The ROC analysis with an AUC value for the VR-E scores between CDR 0 and 0.5 was 0.71, and between CDR 0.5 and 1 was 0.92. For each VR-E cognitive domain, the ROC analysis with an AUC value between CDR 0 and 0.5 was 0.63 for memory, 0.79 for judgment, 0.70 for spatial cognition, 0.62 for calculation, and 0.57 for language. Between CDR 0.5 and 1, the AUC values were 0.81 for memory, 0.75 for judgment, 0.82 for spatial cognition, 0.88 for calculation, and 0.86 for language.
Conclusion
The results suggest that VR-E is useful for differentiating mild cognitive impairment from normal cognition and early dementia.
Objective: To document (1) oculomotor (OM) and vestibulo-ocular (VO) function in children with concussion who were symptomatic at the time of assessment and to compare it with that in children with co...Objective: To document (1) oculomotor (OM) and vestibulo-ocular (VO) function in children with concussion who were symptomatic at the time of assessment and to compare it with that in children with concussion who were clinically recovered (asymptomatic) and in children with no concussive injury, and (2) the extent to which OM and VO function relates to postconcussion symptom severity in injured children.
Setting: Participants were recruited from a concussion clinic or the community.
Participants: A total of 108 youth with concussion (72 symptomatic; 36 recovered) and 79 healthy youth (aged 9-18 years). Youth with concussion were included if aged 9 to 18 years, had no previous concussion within the last 12 months, less than 90 days since injury, and no known existing visual disorders or learning disabilities.
Study design: A prospective cross-sectional study.
Main measures: All participants were tested for OM and VO function with a commercial virtual reality (VR) eye-tracking system (Neuroflex ® , Montreal,Québec, Canada). Participants in the concussion group who completed the postconcussion symptoms were scored with the Post-Concussion Symptom Inventory.
Results: There was a significant group effect for vergence during smooth pursuit ( F2,176 = 10.90; P .05), mean latency during saccades ( F2,171 = 5.99; P = .003), and mean response delay during antisaccades ( F2,177 = 9.07; P .05), where children with symptomatic concussion showed poorer performance than clinically recovered and healthy children. Similar results were found in VO for average vestibular ocular reflex gain in the horizontal leftward ( F2,168 = 7; P = .001) and rightward directions ( F2,163 = 13.08; P .05) and vertical upward ( F2,147 = 7.60; P = .001) and downward directions ( F2,144 = 13.70; P .05). Mean saccade error was positively correlated to total Post-Concussion Symptom Inventory scores in younger clinically recovered children.
Conclusion: VR eye tracking may be an effective tool for identifying OM and VO deficits in the subacute phase (90 days) postconcussion.
With improved portability and affordability, eye tracking devices have facilitated an expanding range of cycling experiments aimed at understanding cycling behavior and potential risks. Given the comp...With improved portability and affordability, eye tracking devices have facilitated an expanding range of cycling experiments aimed at understanding cycling behavior and potential risks. Given the complexity of cyclists’ visual behavior and gaze measurements, we provide a comprehensive review with three key focuses: 1) the adoption and interpretation of various gaze metrics derived from cycling experiments, 2) a summary of the findings of those experiments, and 3) identifying areas for future research. A systematic review of three databases yielded thirty-five articles that met our inclusion criteria. Our review results show that cycling experiments with eye tracking allow analysis of the viewpoint of the cyclist and reactions to the built environment, road conditions, navigation behavior, and mental workload and/or stress levels. Our review suggests substantial variation in research objectives and the consequent selection of eye-tracking devices, experimental design, and which gaze metrics are used and interpreted. A variety of general gaze metrics and gaze measurements related to Areas of Interest (AOI) are applied to infer cyclists’ mental workload/stress levels and attention allocation respectively. The diversity of gaze metrics reported in the literature makes cross-study comparisons difficult. Areas for future research, especially potential integration with computer vision are also discussed.
