Eye tracking - Revision history

Xinreality at 21:21, 7 May 2025

2025-05-07T21:21:43Z

← Older revision		Revision as of 21:21, 7 May 2025
Line 40:		Line 40:
	Eye tracking unlocks numerous capabilities enhancing VR and AR experiences:		Eye tracking unlocks numerous capabilities enhancing VR and AR experiences:

	'''[[Foveated rendering]]''': Arguably the most impactful application for performance. By knowing precisely where the user is looking, the system renders the scene at maximum [[resolution]] only in the small, central area of the user's gaze (corresponding to the eye's high-acuity fovea), while rendering the peripheral areas at progressively lower resolutions. This mimics human vision and can drastically reduce the computational load on the [[graphics processing unit]] (GPU) – potentially by 30% to over 70% – allowing for higher fidelity graphics, increased frame rates, reduced [[latency]], or lower power consumption, all without a noticeable loss in perceived visual quality.<ref name="Patney2016">Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35(6), 179.</ref><ref name="TobiiFoveated">[https://www.tobii.com/blog/eye-tracking-in-vr-a-vital-component Tobii Blog: Eye Tracking in VR — A Vital Component]. Retrieved Nov 17, 2023.</ref><ref name="NvidiaFoveated">NVIDIA Corporation. (n.d.). Maximize VR Performance with Foveated Rendering*. NVIDIA Developer. Retrieved November 16, 2023, from https://developer.nvidia.com/vrworks/graphics/foveatedrendering</ref>		'''[[Foveated rendering]]''': Arguably the most impactful application for performance. By knowing precisely where the user is looking, the system renders the scene at maximum [[resolution]] only in the small, central area of the user's gaze (corresponding to the eye's high-acuity fovea), while rendering the peripheral areas at progressively lower resolutions. This mimics human vision and can drastically reduce the computational load on the [[graphics processing unit]] (GPU) – potentially by 30% to over 70% – allowing for higher fidelity graphics, increased frame rates, reduced [[latency]], or lower power consumption, all without a noticeable loss in perceived visual quality.<ref name="Patney2016">Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35(6), 179.</ref><ref name="TobiiFoveated">[https://www.tobii.com/blog/eye-tracking-in-vr-a-vital-component Tobii Blog: Eye Tracking in VR - A Vital Component]. Retrieved Nov 17, 2023.</ref><ref name="NvidiaFoveated">NVIDIA Corporation. (n.d.). Maximize VR Performance with Foveated Rendering*. NVIDIA Developer. Retrieved November 16, 2023, from https://developer.nvidia.com/vrworks/graphics/foveatedrendering</ref>

	*'''Natural Interaction / Gaze-Based Interaction''': Eye tracking enables more intuitive control schemes:		*'''Natural Interaction / Gaze-Based Interaction''': Eye tracking enables more intuitive control schemes:

Xinreality: Undo revision 34816 by Xinreality (talk)

2025-05-01T16:21:38Z

Undo revision 34816 by Xinreality (talk)

Show changes

Xinreality at 16:21, 1 May 2025

2025-05-01T16:21:15Z

Show changes

Xinreality: Text replacement - "e.g.," to "for example"

2025-04-29T04:23:11Z

Text replacement - "e.g.," to "for example"

Show changes

Xinreality: /* Standards and Regulations */

2025-04-24T16:47:37Z

Standards and Regulations

← Older revision		Revision as of 16:47, 24 April 2025
Line 167:		Line 167:
	==Standards and Regulations==		==Standards and Regulations==
	Efforts are underway to standardize aspects of eye tracking technology and address its implications:		Efforts are underway to standardize aspects of eye tracking technology and address its implications:
	*'''[[Khronos Group]] / [[OpenXR]]''': Defines standard APIs for accessing eye tracking data (e.g., `XR_EXT_eye_gaze_interaction`).		*'''[[Khronos Group]] / [[OpenXR]]''': Defines standard APIs for accessing eye tracking data (e.g., [[XR_EXT_eye_gaze_interaction]]).
	*'''[[IEEE Standards Association\|IEEE]]''': Working groups like IEEE P2048.5 focus on standards for immersive learning, potentially including eye tracking metrics.		*'''[[IEEE Standards Association\|IEEE]]''': Working groups like IEEE P2048.5 focus on standards for immersive learning, potentially including eye tracking metrics.
	*'''[[Virtual Reality Industry Forum]] (VRIF)''': Develops guidelines for VR implementation, potentially covering eye tracking best practices.		*'''[[Virtual Reality Industry Forum]] (VRIF)''': Develops guidelines for VR implementation, potentially covering eye tracking best practices.

Xinreality at 16:46, 24 April 2025

2025-04-24T16:46:28Z

Show changes

Xinreality: /* Applications in VR and AR */

2025-04-24T16:43:21Z

Applications in VR and AR

Xinreality: /* Technical Principles */

2025-04-24T16:42:04Z

Technical Principles

Xinreality at 16:40, 24 April 2025

2025-04-24T16:40:48Z

Xinreality: Undo revision 34459 by Xinreality (talk)

2025-04-24T16:21:55Z

Undo revision 34459 by Xinreality (talk)

Show changes

← Older revision		Revision as of 16:43, 24 April 2025
Line 37:		Line 37:
	*'''Blinks''': Detection of eyelid closures.		*'''Blinks''': Detection of eyelid closures.

	== Applications in VR and AR ==		==Applications in VR and AR==
	Eye tracking unlocks numerous capabilities enhancing VR and AR experiences:		Eye tracking unlocks numerous capabilities enhancing VR and AR experiences:

	* '''[[Foveated rendering]]''': Arguably the most impactful application for performance. By knowing precisely where the user is looking, the system renders the scene at maximum [[resolution]] only in the small, central area of the user's gaze (corresponding to the eye's high-acuity fovea), while rendering the peripheral areas at progressively lower resolutions. This mimics human vision and can drastically reduce the computational load on the [[graphics processing unit]] (GPU) – potentially by 30% to over 70% – allowing for higher fidelity graphics, increased frame rates, reduced [[latency]], or lower power consumption, all without a noticeable loss in perceived visual quality.<ref name="Patney2016">Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35(6), 179.</ref><ref name="TobiiFoveated">[https://www.tobii.com/blog/eye-tracking-in-vr-a-vital-component Tobii Blog: Eye Tracking in VR — A Vital Component]. Retrieved Nov 17, 2023.</ref><ref name="NvidiaFoveated">NVIDIA Corporation. (n.d.). Maximize VR Performance with Foveated Rendering. NVIDIA Developer. Retrieved November 16, 2023, from https://developer.nvidia.com/vrworks/graphics/foveatedrendering</ref>		'''[[Foveated rendering]]''': Arguably the most impactful application for performance. By knowing precisely where the user is looking, the system renders the scene at maximum [[resolution]] only in the small, central area of the user's gaze (corresponding to the eye's high-acuity fovea), while rendering the peripheral areas at progressively lower resolutions. This mimics human vision and can drastically reduce the computational load on the [[graphics processing unit]] (GPU) – potentially by 30% to over 70% – allowing for higher fidelity graphics, increased frame rates, reduced [[latency]], or lower power consumption, all without a noticeable loss in perceived visual quality.<ref name="Patney2016">Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35(6), 179.</ref><ref name="TobiiFoveated">[https://www.tobii.com/blog/eye-tracking-in-vr-a-vital-component Tobii Blog: Eye Tracking in VR — A Vital Component]. Retrieved Nov 17, 2023.</ref><ref name="NvidiaFoveated">NVIDIA Corporation. (n.d.). Maximize VR Performance with Foveated Rendering*. NVIDIA Developer. Retrieved November 16, 2023, from https://developer.nvidia.com/vrworks/graphics/foveatedrendering</ref>

	* '''Natural Interaction / Gaze-Based Interaction''': Eye tracking enables more intuitive control schemes:		*'''Natural Interaction / Gaze-Based Interaction''': Eye tracking enables more intuitive control schemes:
	* '''Gaze Selection/Pointing''': Allows users to select objects, menu items, or targets simply by looking at them. This is often combined with a confirmation action like a button press on a [[controller (computing)\|controller]], a [[hand tracking]] gesture (e.g., pinch), or a short dwell time.<ref name="Piumsomboon2017">Piumsomboon, T., Lee, G., Lindeman, R. W., & Billinghurst, M. (2017). Exploring natural eye-gaze-based interaction for immersive virtual reality. IEEE Symposium on 3D User Interfaces, 36-39.</ref>		*'''Gaze Selection/Pointing''': Allows users to select objects, menu items, or targets simply by looking at them. This is often combined with a confirmation action like a button press on a [[controller (computing)\|controller]], a [[hand tracking]] gesture (e.g., pinch), or a short dwell time.<ref name="Piumsomboon2017">Piumsomboon, T., Lee, G., Lindeman, R. W., & Billinghurst, M. (2017). Exploring natural eye-gaze-based interaction for immersive virtual reality. IEEE Symposium on 3D User Interfaces*, 36-39.</ref>
	* '''Intent Prediction''': Systems can anticipate user actions or needs based on gaze patterns (e.g., highlighting an object the user looks at intently).		**'''Intent Prediction''': Systems can anticipate user actions or needs based on gaze patterns (e.g., highlighting an object the user looks at intently).
	* '''Gaze-directed Locomotion''': Steering movement within the virtual world based on gaze direction.		**'''Gaze-directed Locomotion''': Steering movement within the virtual world based on gaze direction.

	* '''Enhanced Social Presence / [[Avatar]] Realism''': Realistic eye movements, including subtle saccades, blinks, and responsive gaze shifts, can be mirrored onto a user's avatar in social VR applications. This significantly enhances [[non-verbal communication]] and the feeling of [[social presence]] and connection when interacting with others.<ref name="MetaAvatarsEyeTracking">Meta Platforms, Inc. (2022, October 11). Meta Quest Pro: A New Way to Work, Create and Collaborate. Meta Quest Blog. https://www.meta.com/blog/quest/meta-quest-pro-vr-headset-features-price-release-date/</ref>		'''Enhanced Social Presence / [[Avatar]] Realism''': Realistic eye movements, including subtle saccades, blinks, and responsive gaze shifts, can be mirrored onto a user's avatar in social VR applications. This significantly enhances [[non-verbal communication]] and the feeling of [[social presence]] and connection when interacting with others.<ref name="MetaAvatarsEyeTracking">Meta Platforms, Inc. (2022, October 11). Meta Quest Pro: A New Way to Work, Create and Collaborate*. Meta Quest Blog. https://www.meta.com/blog/quest/meta-quest-pro-vr-headset-features-price-release-date/</ref>

	* '''User Analytics and Research''': Eye tracking provides invaluable objective data for:		*'''User Analytics and Research''': Eye tracking provides invaluable objective data for:
	* '''[[Usability testing]] & User Experience (UX) Research''': Understanding how users visually explore and interact with interfaces or environments.		**'''[[Usability testing]] & User Experience (UX) Research''': Understanding how users visually explore and interact with interfaces or environments.
	* '''[[Attention mapping]]''': Creating heatmaps and gaze plots to visualize areas of interest and attention duration.		**'''[[Attention mapping]]''': Creating heatmaps and gaze plots to visualize areas of interest and attention duration.
	* '''Cognitive Load Assessment''': Measuring mental workload through metrics like pupil dilation, blink rate, and fixation patterns.<ref name="Clay2019">Clay, V., König, P., & König, S. (2019). Eye tracking in virtual reality. Journal of Eye Movement Research, 12(1).</ref>		*'''Cognitive Load Assessment''': Measuring mental workload through metrics like pupil dilation, blink rate, and fixation patterns.<ref name="Clay2019">Clay, V., König, P., & König, S. (2019). Eye tracking in virtual reality. Journal of Eye Movement Research, 12*(1).</ref>
	* '''Training and Simulation Analysis''': Assessing trainee attention, situational awareness, and decision-making processes in professional simulations (e.g., medical, aviation).		**'''Training and Simulation Analysis''': Assessing trainee attention, situational awareness, and decision-making processes in professional simulations (e.g., medical, aviation).

	* '''Automatic [[Interpupillary distance\|IPD]] Adjustment''': Some headsets utilize the eye tracking cameras to automatically measure the user's interpupillary distance (the distance between the centers of the pupils) and mechanically adjust the lens spacing for optimal visual clarity, stereo depth perception, and user comfort.		*'''Automatic [[Interpupillary distance\|IPD]] Adjustment''': Some headsets utilize the eye tracking cameras to automatically measure the user's interpupillary distance (the distance between the centers of the pupils) and mechanically adjust the lens spacing for optimal visual clarity, stereo depth perception, and user comfort.

	* '''[[Accessibility]]''': Eye tracking offers a powerful hands-free input modality for users with limited physical mobility, enabling them to navigate interfaces, communicate (e.g., gaze typing), and control applications within VR/AR.		*'''[[Accessibility]]''': Eye tracking offers a powerful hands-free input modality for users with limited physical mobility, enabling them to navigate interfaces, communicate (e.g., gaze typing), and control applications within VR/AR.

	* '''Adaptive Optics / [[Varifocal display]]s''': Eye tracking is essential for dynamic varifocal displays, which adjust their focal plane based on where the user is looking in virtual depth. This helps address the [[vergence-accommodation conflict]], potentially reducing eye strain and improving visual realism.<ref name="Akeley2004">Akeley, K., Watt, S.J., Girshick, A.R., & Banks, M.S. (2004). A stereo display prototype with multiple focal distances. ACM Transactions on Graphics, 23(3), 804–813.</ref>		'''Adaptive Optics / [[Varifocal display]]s''': Eye tracking is essential for dynamic varifocal displays, which adjust their focal plane based on where the user is looking in virtual depth. This helps address the [[vergence-accommodation conflict]], potentially reducing eye strain and improving visual realism.<ref name="Akeley2004">Akeley, K., Watt, S.J., Girshick, A.R., & Banks, M.S. (2004). A stereo display prototype with multiple focal distances. ACM Transactions on Graphics, 23*(3), 804–813.</ref>

	* '''[[Dynamic Distortion Compensation]]''': Real-time adjustments to lens distortion correction based on precise eye position relative to the lens center can improve perceived sharpness across the field of view.<ref name="TobiiFoveated"/>		*'''[[Dynamic Distortion Compensation]]''': Real-time adjustments to lens distortion correction based on precise eye position relative to the lens center can improve perceived sharpness across the field of view.<ref name="TobiiFoveated"/>

	== Current Implementations ==		== Current Implementations ==

← Older revision		Revision as of 16:42, 24 April 2025
Line 4:		Line 4:
	While its integration into consumer VR/AR is relatively recent, the systematic study of eye movements dates back to the 19th century. [[Louis Émile Javal]] noted in 1879 that reading involved discrete stops (fixations) and rapid movements (saccades).<ref name="Rayner1998">Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422.</ref> Early tracking devices included [[Edmund Huey]]'s contact lens-based tracker (~1908)<ref name="Huey1908">Huey, E.B. (1908). The Psychology and Pedagogy of Reading. Macmillan.</ref> and [[Guy Thomas Buswell]]'s film-based systems in the 1930s. [[Alfred L. Yarbus]]'s work in the 1960s highlighted how viewing patterns are task-dependent.<ref name="Yarbus1967">Yarbus, A.L. (1967). Eye Movements and Vision. Plenum Press.</ref> These foundational efforts paved the way for modern video-based and integrated tracking systems.		While its integration into consumer VR/AR is relatively recent, the systematic study of eye movements dates back to the 19th century. [[Louis Émile Javal]] noted in 1879 that reading involved discrete stops (fixations) and rapid movements (saccades).<ref name="Rayner1998">Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422.</ref> Early tracking devices included [[Edmund Huey]]'s contact lens-based tracker (~1908)<ref name="Huey1908">Huey, E.B. (1908). The Psychology and Pedagogy of Reading. Macmillan.</ref> and [[Guy Thomas Buswell]]'s film-based systems in the 1930s. [[Alfred L. Yarbus]]'s work in the 1960s highlighted how viewing patterns are task-dependent.<ref name="Yarbus1967">Yarbus, A.L. (1967). Eye Movements and Vision. Plenum Press.</ref> These foundational efforts paved the way for modern video-based and integrated tracking systems.

	== Technical Principles ==		==Technical Principles==

	=== Tracking ~~Methods~~ ===		===Tracking Method===
	Most eye tracking systems in contemporary VR and AR headsets employ one of several core technologies:		Most eye tracking systems in contemporary VR and AR headsets employ one of several core technologies:

	* '''[[Pupil Center Corneal Reflection]] (PCCR)''': This is the predominant method. It involves:<ref name="Holmqvist2011">Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.</ref><ref name="RoadToVREyeTracking">Lang, B. (2021, July 21). Casual Explainer: What is Eye-tracking & How Does it Work? Road to VR. https://www.roadtovr.com/casual-explainer-what-is-eye-tracking-how-does-it-work/</ref>		'''[[Pupil Center Corneal Reflection]] (PCCR)''': This is the predominant method. It involves:<ref name="Holmqvist2011">Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.</ref><ref name="RoadToVREyeTracking">Lang, B. (2021, July 21). Casual Explainer: What is Eye-tracking & How Does it Work?* Road to VR. https://www.roadtovr.com/casual-explainer-what-is-eye-tracking-how-does-it-work/</ref>
	* '''Illumination:''' [[Infrared]] (IR) light-emitting diodes ([[LED]]s) safely illuminate the eye. IR light is used because it is invisible to the human eye, preventing distraction, and provides high contrast for [[camera]]s.		*'''Illumination:''' [[Infrared]] (IR) light-emitting diodes ([[LED]]s) safely illuminate the eye. IR light is used because it is invisible to the human eye, preventing distraction, and provides high contrast for [[camera]]s.
	* '''Imaging:''' Small, high-[[frame rate]] infrared cameras capture images of the eye, specifically tracking the center of the [[pupil]] and the reflection(s) of the IR light off the surface of the [[cornea]] (known as glints).		*'''Imaging:''' Small, high-[[frame rate]] infrared cameras capture images of the eye, specifically tracking the center of the [[pupil]] and the reflection(s) of the IR light off the surface of the [[cornea]] (known as glints).
	* '''[[Algorithm\|Algorithmic Processing]]:''' Sophisticated [[computer vision]] and [[image processing]] algorithms analyze the captured images. By calculating the vector between the pupil center and the corneal reflection(s), the system determines the eye's orientation and calculates the user's gaze point with high accuracy.		*'''[[Algorithm\|Algorithmic Processing]]:''' Sophisticated [[computer vision]] and [[image processing]] algorithms analyze the captured images. By calculating the vector between the pupil center and the corneal reflection(s), the system determines the eye's orientation and calculates the user's gaze point with high accuracy.
	* '''[[Calibration]]:''' A per-user calibration process is usually required upon first use, and sometimes periodically, to account for individual differences in eye physiology (e.g., corneal shape, pupil size range) and the precise fit of the headset. This typically involves the user looking at specific points displayed within the headset.		*'''[[Calibration]]:''' A per-user calibration process is usually required upon first use, and sometimes periodically, to account for individual differences in eye physiology (e.g., corneal shape, pupil size range) and the precise fit of the headset. This typically involves the user looking at specific points displayed within the headset.

	* '''Video-based eye tracking (Shape/Feature Tracking)''': Uses cameras aimed at the eyes to capture images, which are then analyzed using computer vision algorithms to identify eye features (pupil outline, iris texture, blood vessels) to determine gaze direction without necessarily relying on corneal reflections.<ref name="Hansen2010">Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 478-500.</ref> PCCR is often considered a subset of this broader category.		'''Video-based eye tracking (Shape/Feature Tracking)''': Uses cameras aimed at the eyes to capture images, which are then analyzed using computer vision algorithms to identify eye features (pupil outline, iris texture, blood vessels) to determine gaze direction without necessarily relying on corneal reflections.<ref name="Hansen2010">Hansen, D. W., & Ji, Q. (2010). In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32*(3), 478-500.</ref> PCCR is often considered a subset of this broader category.

	* '''[[Electrooculography]] (EOG)''': Measures the electrical potential difference between electrodes placed on the skin around the eyes. This potential changes predictably as the eye rotates. While less common for high-accuracy gaze pointing in consumer VR/AR due to lower precision and susceptibility to muscle noise, it can be used for detecting larger eye movements or in specialized applications.<ref name="Bulling2011">Bulling, A., Ward, J. A., Gellersen, H., & Tröster, G. (2011). Eye movement analysis for activity recognition using electrooculography. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 741-753.</ref>		'''[[Electrooculography]] (EOG)''': Measures the electrical potential difference between electrodes placed on the skin around the eyes. This potential changes predictably as the eye rotates. While less common for high-accuracy gaze pointing in consumer VR/AR due to lower precision and susceptibility to muscle noise, it can be used for detecting larger eye movements or in specialized applications.<ref name="Bulling2011">Bulling, A., Ward, J. A., Gellersen, H., & Tröster, G. (2011). Eye movement analysis for activity recognition using electrooculography. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33*(4), 741-753.</ref>

	* '''[[Scleral search coil]]''': Involves wearing a special contact lens containing a wire coil. The user sits within a magnetic field, and eye movements induce currents in the coil, providing extremely precise measurements. This is highly invasive and primarily used in laboratory research, not consumer VR/AR.<ref name="Robinson1963">Robinson, D.A. (1963). A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Transactions on Biomedical Engineering, BME-10(4), 137–145.</ref>		'''[[Scleral search coil]]''': Involves wearing a special contact lens containing a wire coil. The user sits within a magnetic field, and eye movements induce currents in the coil, providing extremely precise measurements. This is highly invasive and primarily used in laboratory research, not consumer VR/AR.<ref name="Robinson1963">Robinson, D.A. (1963). A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Transactions on Biomedical Engineering, BME-10*(4), 137–145.</ref>

	=== Key Components (PCCR-based) ===		===Key Components (PCCR-based)===
	A typical PCCR eye tracking system integrated into a VR/AR headset consists of:		A typical PCCR eye tracking system integrated into a VR/AR headset consists of:
	* '''Illuminators''': Infrared LEDs providing consistent, non-visible lighting.		*'''Illuminators''': Infrared LEDs providing consistent, non-visible lighting.
	* '''Cameras''': Specialized high-speed infrared cameras positioned to capture clear images of both eyes.		*'''Cameras''': Specialized high-speed infrared cameras positioned to capture clear images of both eyes.
	* '''Processing Unit''': Either onboard [[System on a Chip\|SoC]] resources or dedicated hardware to run the detection and gaze calculation algorithms in real-time.		*'''Processing Unit''': Either onboard [[System on a Chip\|SoC]] resources or dedicated hardware to run the detection and gaze calculation algorithms in real-time.
	* '''Calibration Software''': [[Software]] routines guiding the user through calibration and storing individual profiles.<ref name="Kar2017">Kar, A., & Corcoran, P. (2017). A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms. IEEE Access, 5, 16495-16519.</ref>		'''Calibration Software''': [[Software]] routines guiding the user through calibration and storing individual profiles.<ref name="Kar2017">Kar, A., & Corcoran, P. (2017). A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms. IEEE Access, 5*, 16495-16519.</ref>

	=== Eye Movement Types Measured ===		===Eye Movement Types Measured===
	Eye tracking systems in VR/AR can detect and analyze various types of eye movements and states:		Eye tracking systems in VR/AR can detect and analyze various types of eye movements and states:
	* '''[[Fixations]]''': Periods when the gaze remains relatively stable on a specific area (typically > 100-200 ms), indicating visual attention.		*'''[[Fixations]]''': Periods when the gaze remains relatively stable on a specific area (typically > 100-200 ms), indicating visual attention.
	* '''[[Saccades]]''': Rapid, ballistic movements shifting the gaze between fixation points (typically 30-120 ms).		*'''[[Saccades]]''': Rapid, ballistic movements shifting the gaze between fixation points (typically 30-120 ms).
	* '''[[Smooth pursuit]]''': Movements allowing the eyes to smoothly follow a moving object.		*'''[[Smooth pursuit]]''': Movements allowing the eyes to smoothly follow a moving object.
	* '''[[Vergence]]''': The simultaneous movement of both eyes in opposite directions to obtain or maintain single binocular vision, critical for focusing on objects at different depths.		*'''[[Vergence]]''': The simultaneous movement of both eyes in opposite directions to obtain or maintain single binocular vision, critical for focusing on objects at different depths.
	* '''[[Pupil dilation\|Pupil Size / Dilation]]''': Changes in pupil diameter (pupillometry), which can correlate with changes in light levels, cognitive load, emotional arousal, or interest.<ref name="Leigh2015">Leigh, R. J., & Zee, D. S. (2015). The neurology of eye movements. Oxford University Press.</ref>		'''[[Pupil dilation\|Pupil Size / Dilation]]''': Changes in pupil diameter (pupillometry), which can correlate with changes in light levels, cognitive load, emotional arousal, or interest.<ref name="Leigh2015">Leigh, R. J., & Zee, D. S. (2015). The neurology of eye movements*. Oxford University Press.</ref>
	* '''Blinks''': Detection of eyelid closures.		*'''Blinks''': Detection of eyelid closures.

	== Applications in VR and AR ==		== Applications in VR and AR ==

← Older revision		Revision as of 16:40, 24 April 2025
Line 57:		Line 57:
	* '''Automatic [[Interpupillary distance\|IPD]] Adjustment''': Some headsets utilize the eye tracking cameras to automatically measure the user's interpupillary distance (the distance between the centers of the pupils) and mechanically adjust the lens spacing for optimal visual clarity, stereo depth perception, and user comfort.		* '''Automatic [[Interpupillary distance\|IPD]] Adjustment''': Some headsets utilize the eye tracking cameras to automatically measure the user's interpupillary distance (the distance between the centers of the pupils) and mechanically adjust the lens spacing for optimal visual clarity, stereo depth perception, and user comfort.

	* '''[[Accessibility]]''': Eye tracking offers a powerful hands-free input modality for users with limited physical mobility, enabling them to navigate interfaces, communicate (e.g., gaze typing), and control applications within VR/AR.<ref name="Yuan2020">Yuan, Z., Bi, T., Muntean, G. M., & Ghinea, G. (2020). Perceived synchronization of mulsemedia services. IEEE Transactions on Multimedia, 17(7), 957-966. Note: While this ref discusses synchronization, accessibility is a widely cited benefit of eye tracking.</ref>		* '''[[Accessibility]]''': Eye tracking offers a powerful hands-free input modality for users with limited physical mobility, enabling them to navigate interfaces, communicate (e.g., gaze typing), and control applications within VR/AR.

	* '''Adaptive Optics / [[Varifocal display]]s''': Eye tracking is essential for dynamic varifocal displays, which adjust their focal plane based on where the user is looking in virtual depth. This helps address the [[vergence-accommodation conflict]], potentially reducing eye strain and improving visual realism.<ref name="Akeley2004">Akeley, K., Watt, S.J., Girshick, A.R., & Banks, M.S. (2004). A stereo display prototype with multiple focal distances. ACM Transactions on Graphics, 23(3), 804–813.</ref>		* '''Adaptive Optics / [[Varifocal display]]s''': Eye tracking is essential for dynamic varifocal displays, which adjust their focal plane based on where the user is looking in virtual depth. This helps address the [[vergence-accommodation conflict]], potentially reducing eye strain and improving visual realism.<ref name="Akeley2004">Akeley, K., Watt, S.J., Girshick, A.R., & Banks, M.S. (2004). A stereo display prototype with multiple focal distances. ACM Transactions on Graphics, 23(3), 804–813.</ref>
Line 142:		Line 142:
	<ref name="Duchowski2018">Duchowski, A. T., Krejtz, K., Krejtz, I., Biele, C., Niedzielska, A., Kiefer, P., Raubal, M., & Giannopoulos, I. (2018). The index of pupillary activity: Measuring cognitive load vis-à-vis task difficulty with pupil oscillation. CHI Conference on Human Factors in Computing Systems, 1-13.</ref>		<ref name="Duchowski2018">Duchowski, A. T., Krejtz, K., Krejtz, I., Biele, C., Niedzielska, A., Kiefer, P., Raubal, M., & Giannopoulos, I. (2018). The index of pupillary activity: Measuring cognitive load vis-à-vis task difficulty with pupil oscillation. CHI Conference on Human Factors in Computing Systems, 1-13.</ref>

	== Software Development and APIs ==		==Software Development and APIs==

	Integrating eye tracking into applications requires specific software support:		Integrating eye tracking into applications requires specific software support:

	~~###~~ Development Frameworks and APIs		===Development Frameworks and APIs===
	* '''[[Unity (game engine)\|Unity]]''': Provides APIs through its XR Interaction Toolkit and potential vendor-specific SDKs (e.g., Tobii XR SDK).		*'''[[Unity (game engine)\|Unity]]''': Provides APIs through its XR Interaction Toolkit and potential vendor-specific SDKs (e.g., Tobii XR SDK).
	* '''[[Unreal Engine]]''': Offers native interfaces and plugins for accessing eye tracking data.		*'''[[Unreal Engine]]''': Offers native interfaces and plugins for accessing eye tracking data.
	* '''[[OpenXR]]''': The cross-platform standard includes an `XR_EXT_eye_gaze_interaction` extension, allowing developers to write more portable code.		*'''[[OpenXR]]''': The cross-platform standard includes an `XR_EXT_eye_gaze_interaction` extension, allowing developers to write more portable code.
	* '''Vendor SDKs''': Companies like [[Tobii]] provide dedicated Software Development Kits offering fine-grained control and optimized features for their hardware.<ref name="TobiiSDK">Tobii Technology. (2023). Tobii XR SDK Documentation. Tobii.com.</ref>		'''Vendor SDKs''': Companies like [[Tobii]] provide dedicated Software Development Kits offering fine-grained control and optimized features for their hardware.<ref name="TobiiSDK">Tobii Technology. (2023). Tobii XR SDK Documentation*. Tobii.com.</ref>

	~~###~~ Data Access Levels		===Data Access Levels===
	Developers typically access eye tracking data at different levels of abstraction:		Developers typically access eye tracking data at different levels of abstraction:
	* '''Raw Data''': Direct coordinates of pupil centers, glints, eye openness, etc. Requires significant processing by the application.		*'''Raw Data''': Direct coordinates of pupil centers, glints, eye openness, etc. Requires significant processing by the application.
	* '''Gaze Data''': Processed output providing calibrated gaze origin and direction vectors, or intersection points on virtual surfaces.		*'''Gaze Data''': Processed output providing calibrated gaze origin and direction vectors, or intersection points on virtual surfaces.
	* '''Eye Movement Events''': Classified data identifying fixations, saccades, and blinks.		*'''Eye Movement Events''': Classified data identifying fixations, saccades, and blinks.
	* '''Semantic Data''': Higher-level interpretations, such as the specific object being looked at (gaze target) or estimated attention levels.		*'''Semantic Data''': Higher-level interpretations, such as the specific object being looked at (gaze target) or estimated attention levels.
	<ref name="Kumar2016">Kumar, D., Dutta, A., Das, A., & Lahiri, U. (2016). SmartEye: Developing a novel eye tracking system for quantitative assessment of oculomotor abnormalities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(10), 1051-1059.</ref>		<ref name="Kumar2016">Kumar, D., Dutta, A., Das, A., & Lahiri, U. (2016). SmartEye: Developing a novel eye tracking system for quantitative assessment of oculomotor abnormalities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 24(10), 1051-1059.</ref>

	== Ethical and Privacy Considerations ==		==Ethical and Privacy Considerations==

	The collection and use of eye tracking data necessitate careful ethical consideration:		The collection and use of eye tracking data necessitate careful ethical consideration:

	* '''Biometric Data Privacy''': Gaze patterns can be unique identifiers. This data requires strong [[data security]] measures and compliance with regulations like [[GDPR]] (which may classify it as sensitive biometric data).		*'''Biometric Data Privacy''': Gaze patterns can be unique identifiers. This data requires strong [[data security]] measures and compliance with regulations like [[GDPR]] (which may classify it as sensitive biometric data).
	* '''Inference and Profiling''': The potential to infer sensitive information (health, emotions, interests) without explicit consent raises ethical questions.		*'''Inference and Profiling''': The potential to infer sensitive information (health, emotions, interests) without explicit consent raises ethical questions.
	* '''Attention Monitoring''': Concerns about workplace surveillance or manipulative advertising based on attention analytics.		*'''Attention Monitoring''': Concerns about workplace surveillance or manipulative advertising based on attention analytics.
	* '''Informed Consent''': Users must be clearly informed about what eye data is collected, how it is processed, stored, and shared, and for what purposes. Opt-in mechanisms are crucial.		*'''Informed Consent''': Users must be clearly informed about what eye data is collected, how it is processed, stored, and shared, and for what purposes. Opt-in mechanisms are crucial.
	* '''Data Minimization and Anonymization''': Collect only necessary data, anonymize or aggregate whenever possible, and define clear data retention policies.		*'''Data Minimization and Anonymization''': Collect only necessary data, anonymize or aggregate whenever possible, and define clear data retention policies.
	<ref name="Kroger2020"/>		<ref name="Kroger2020"/>

	== Standards and Regulations ==		==Standards and Regulations==

	Efforts are underway to standardize aspects of eye tracking technology and address its implications:		Efforts are underway to standardize aspects of eye tracking technology and address its implications:
	* '''[[Khronos Group]] / [[OpenXR]]''': Defines standard APIs for accessing eye tracking data (e.g., `XR_EXT_eye_gaze_interaction`).		*'''[[Khronos Group]] / [[OpenXR]]''': Defines standard APIs for accessing eye tracking data (e.g., `XR_EXT_eye_gaze_interaction`).
	* '''[[IEEE Standards Association\|IEEE]]''': Working groups like IEEE P2048.5 focus on standards for immersive learning, potentially including eye tracking metrics.		*'''[[IEEE Standards Association\|IEEE]]''': Working groups like IEEE P2048.5 focus on standards for immersive learning, potentially including eye tracking metrics.
	* '''[[Virtual Reality Industry Forum]] (VRIF)''': Develops guidelines for VR implementation, potentially covering eye tracking best practices.		*'''[[Virtual Reality Industry Forum]] (VRIF)''': Develops guidelines for VR implementation, potentially covering eye tracking best practices.
	* '''Data Protection Regulations''': [[GDPR]] (Europe), [[CCPA]]/[[CPRA]] (California), and similar laws globally impose requirements on handling personal and biometric data, including eye tracking data.		*'''Data Protection Regulations''': [[GDPR]] (Europe), [[CCPA]]/[[CPRA]] (California), and similar laws globally impose requirements on handling personal and biometric data, including eye tracking data.
	* '''[[ISO]]/[[IEC]] JTC 1/SC 24''': Committee working on international standards for computer graphics, image processing, and environmental data representation, relevant to VR/AR interfaces.<ref name="ISO_SC24">International Organization for Standardization. (2023). ISO/IEC JTC 1/SC 24 - Computer graphics, image processing and environmental data representation. ISO.org.</ref>		'''[[ISO]]/[[IEC]] JTC 1/SC 24''': Committee working on international standards for computer graphics, image processing, and environmental data representation, relevant to VR/AR interfaces.<ref name="ISO_SC24">International Organization for Standardization. (2023). ISO/IEC JTC 1/SC 24 - Computer graphics, image processing and environmental data representation*. ISO.org.</ref>

	== See Also ==		==See Also==
	* [[Augmented reality]]		* [[Augmented reality]]
	* [[AR headset]]		* [[AR headset]]
Line 205:		Line 202:
	* [[VR headset]]		* [[VR headset]]

	== References ==		==References==
	<references />		<references />