Using the method presented in study [I], various HMDs can be evaluated with regard to their ability to display visual stimuli at a specific position in the FOV independently of the current viewing direction. To obtain a good assessment of the overall eye tracking quality, other metrics have to be considered as well. In addition to sensor latency and delay, these include precision and accuracy of the estimated eye direction (Lohr, Friedman, and Komogortsev 2019; Sipatchin, Wahl, and Rifai 2021; Aziz et al. 2024). To measure precision, the root-mean-square of the inter-sample angular distances between successive gaze directions during a fixation is usually calculated (Blignaut and Beelders 2012). Eye tracking accuracy is often defined as the average error between the position of a fixation target and the measured fixation, while precision describes the dispersion of gaze data while the gaze is fixed on a static position in space (Nyström et al. 2025; Clemotte et al. 2014). Under optimal conditions, the HTC Vive Pro Eye shows eye tracking data delays of about 50 ms, an accuracy below 1° in central, but up to 10° in strong peripheral positions (27°), while eye tracking precision varied between 1.4 and 3.6° (Stein et al. 2021; Sipatchin, Wahl, and Rifai 2021).

Low eye tracking precision can sometimes be corrected at the expense of additional latency, by applying a time-wise filter on the eye tracking signal (Holmqvist et al. 2011). However, to detect rapid eye movements such as saccades as early as possible, it is best to collect eye tracking data at a high temporal sampling rate and with the lowest possible latency.

Accuracy depends on the calibration of the eye tracker. During this process, the eye tracking system creates a mapping of the distance between the tracked pupil and the visible infrared reflections of the cornea, which were extracted from the recorded image of the eyes and the angle of the distance vector to positions on the screen. (Harezlak, Kasprowski, and Stasch 2014; Adhanom, MacNeilage, and Folmer 2023). In most cases, the user is shown a series of (usually 5-16) small targets that must be looked at in sequence (Adhanom, MacNeilage, and Folmer 2023). To ensure consistent accuracy of the calibrated assignment, it is also important to check whether the worn eye tracker changes position during the experiment (Niehorster et al. 2020). If there is too much movement, recalibration must be performed. Although they are not (yet) widely used, there are also some VR algorithms for calibrating eye tracking that can be run in the background without the user having to perform a separate calibration process. For example, gaze movements can be compared with the user’s interactions with the user interface in order to continuously update the mapping (Hou et al. 2025). Alternatively, it is also possible to use an algorithm to find connections between eye movements and visible motion on the screen (Tripathi and Guenter 2017).

Determining Gaze Position in VR

After calibration, current VR eye trackers usually provide two three-dimensional eye direction vectors. These can easily be converted into vertical and horizontal angles, by comparing the x and y-coordinates with the head direction. The gaze direction, which is calculated by combining the eye and head orientation vectors, can then be assigned to virtual objects on individual images of the VE.

Although the vectors of eye direction are often represented as three-dimensional vectors, they usually do not contain any information about the gaze distance. Instead, the third coordinate is typically the result of normalizing the vectors to a length of 1.

To estimate the viewing distance by exclusively using eye tracking data, the vergence angle between the left and right gaze direction can be calculated and calibrated to different gaze depths. However, we tend to make large vergence angle errors, especially when looking at near targets (Cornell et al. 2003). The same errors can be observed in VR and can not be reduced even when using optical lenses specifically implemented to reduce the VAC (McAnally, Grove, and Wallis 2024). An alternative method of obtaining a rough estimate of the user’s viewing distance is to measure the interpupillary distance resulting from the estimated eye positions (Arefin et al. 2022).

Another way to determine the three-dimensional viewing position is to combine the eye movement data with the structure of the VE. Once the head and eye signals have been combined, a virtual ray can be created from the eye position in the direction of gaze. If it crosses a surface of one of the virtual objects in the VE, the collision point can be used as the current three-dimensional gaze position, with the virtual object serving as a label. Since these objects update their position frame by frame, the rendering frequency of the VE influences the recorded three-dimensional gaze positions. Without further adjustments, the rendering frequency is limited by the maximum refresh rate of the display and how well the available hardware copes with the complexity of the scene. Thus, the quality of the resulting gaze position depends on the quality of the eye tracking and the quality of the head and position tracking. By default, the gaze ray method returns the first object the ray collides with. Since the accuracy and precision are not perfect in current VR eye trackers, it is possible that a gaze position from an object behind the current object of interest is selected, especially if the gaze target is small. This limitation also applies to desktop eye tracking configurations. In VR however, a small measurement error in vertical and horizontal eye direction angle can lead to comparably large errors in three-dimensional gaze points. To some extent, this problem can be detected by examining the distance between the user and the fixation position over time. Unusual spikes while no saccades are present in the eye movement signal may indicate that a gaze signal is bouncing between two objects in the line of sight.

Eye Movement Classification

In the next step, each of the continuous data points from the stream of gaze data is classified as a type of eye movement. In experiments, in which visual stimuli are presented on a fixed screen on a desktop, fixations and saccades are usually classified depending on threshold algorithms that are based on the eye movement velocity (Holmqvist et al. 2011) and therefore can in theory also be applied when using mobile eye tracking in natural environments. However, most of the commonly used algorithms are based on velocity thresholds and are optimised to work in static environments in scenarios without head movements. In experiments with head movements, differentiating clearly between smooth pursuit, saccades and eye movements that stabilise an object on the retina during an ongoing head movement can be challenging since these movements interact and their conceptual separation becomes less clear (Drewes, Feder, and Einhäuser 2021; Lappi 2015; Steinman and Collewijn 1980). In VR, data from VE, such as fixation marks based on the positions of virtual objects can also be taken into account. For example, one criterion for classifying a series of eye movements as fixation or smooth pursuit could be that they are continuously directed at the same virtual object. In addition, the positions and rotations of the user’s head, eyes and the virtual object being looked at could be used to distinguish smooth pursuit to track a moving object from eye movements to stabilise a fixed object during a head movement. This means that contextual information is used to classify the gaze data. Thus, additional efforts should be made to show that the gaze patterns identified using this method are transferable to different types of VEs and are not specific to eye movements in VR.

Manual classification of fixations has proven to be an unreliable standard since untrained coders often disagree with each other (Hooge et al. 2018). Nevertheless, the manual inspection of eye tracking data is a useful step and can be especially helpful in finding and improving incorrect classifications when using algorithms.

After classification, we can observe, replay and analyse how VR users are looking around in the presented virtual worlds. This setup is particularly interesting for research purposes since it offers a simple way to measure eye, head and thus also gaze movements while strictly controlling the (visual) inputs that participants receive. In comparison to real-world eye tracking experiments, gaze fixation points can be automatically linked to the inspected virtual objects. The resulting data therefore contains a detailed stream of when certain virtual objects were in the user’s FOV and which of these objects the user viewed while solving various VR tasks. VR experiments thus make it possible to collect a large amount of labelled data that describes how participants explore their environment using the same movements as in the real world. As a result, VR research can help us better understand complex human behaviours and could be a helpful addition to field experiments where both the control of visual stimuli and the labelling of eye movement data involve significantly more effort.

References

• ← Previous
• Next →