Depth cue: Difference between revisions
Xinreality (talk | contribs) |
Xinreality (talk | contribs) m Text replacement - "e.g.," to "for example" Tags: Mobile edit Mobile web edit |
||
| Line 40: | Line 40: | ||
====[[Relative Height]] (Elevation in the Visual Field)==== | ====[[Relative Height]] (Elevation in the Visual Field)==== | ||
For objects resting on the same ground plane, those that are higher in the visual field (closer to the horizon line) are typically perceived as being farther away. For objects above the horizon line ( | For objects resting on the same ground plane, those that are higher in the visual field (closer to the horizon line) are typically perceived as being farther away. For objects above the horizon line (for example clouds), those lower in the visual field are perceived as farther. <ref name="CuttingVishton1995"/> <ref name="OoiHeight2001">Ooi, Teng Leng, Bing Wu, and Zijiang J. He. (2001). Distance determined by the angular declination below the horizon. *Nature, 414*(6860), 197-200.</ref> | ||
====[[Linear Perspective]]==== | ====[[Linear Perspective]]==== | ||
| Line 82: | Line 82: | ||
====The [[Vergence-Accommodation Conflict]] (VAC)==== | ====The [[Vergence-Accommodation Conflict]] (VAC)==== | ||
A major limitation in most current VR/AR displays is the mismatch between vergence and accommodation cues. Most headsets use [[fixed-focus display]]s, meaning the optics present the virtual image at a fixed focal distance (often 1.5-2 meters or optical infinity), regardless of the simulated distance of the virtual object. <ref name="ARInsiderVAC">(2024-01-29) Understanding Vergence-Accommodation Conflict in AR/VR Headsets - AR Insider. Retrieved April 25, 2025, from https://arinsider.co/2024/01/29/understanding-vergence-accommodation-conflict-in-ar-vr-headsets/</ref> <ref name="WikiVAC">Vergence-accommodation conflict - Wikipedia. Retrieved April 25, 2025, from https://en.wikipedia.org/wiki/Vergence-accommodation_conflict</ref> <ref name="DeliverContactsFocus">(2024-07-18) Exploring the Focal Distance in VR Headsets - Deliver Contacts. Retrieved April 25, 2025, from https://delivercontacts.com/blog/exploring-the-focal-distance-in-vr-headsets</ref> While the user's eyes converge appropriately for the virtual object's simulated distance ( | A major limitation in most current VR/AR displays is the mismatch between vergence and accommodation cues. Most headsets use [[fixed-focus display]]s, meaning the optics present the virtual image at a fixed focal distance (often 1.5-2 meters or optical infinity), regardless of the simulated distance of the virtual object. <ref name="ARInsiderVAC">(2024-01-29) Understanding Vergence-Accommodation Conflict in AR/VR Headsets - AR Insider. Retrieved April 25, 2025, from https://arinsider.co/2024/01/29/understanding-vergence-accommodation-conflict-in-ar-vr-headsets/</ref> <ref name="WikiVAC">Vergence-accommodation conflict - Wikipedia. Retrieved April 25, 2025, from https://en.wikipedia.org/wiki/Vergence-accommodation_conflict</ref> <ref name="DeliverContactsFocus">(2024-07-18) Exploring the Focal Distance in VR Headsets - Deliver Contacts. Retrieved April 25, 2025, from https://delivercontacts.com/blog/exploring-the-focal-distance-in-vr-headsets</ref> While the user's eyes converge appropriately for the virtual object's simulated distance (for example 0.5 meters), their eyes must maintain focus (accommodate) at the fixed optical distance of the display itself to keep the image sharp. This mismatch between the distance signaled by vergence and the distance signaled by accommodation is known as the '''[[vergence-accommodation conflict]]''' (VAC). <ref name="HoffmanVAC2008">Hoffman, D. M., Girshick, A. R., Akeley, K., & Banks, M. S. (2008). Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. *Journal of Vision, 8*(3), 33. doi:10.1167/8.3.33</ref> <ref name="FacebookVAC2019">Facebook Research. (2019, March 28). *Vergence-Accommodation Conflict: Facebook Research Explains Why Varifocal Matters For Future VR*. YouTube. [https://www.youtube.com/watch?v=YWA4gVibKJE]</ref> <ref name="KramidaVAC2016">Kramida, Gregory. (2016). Resolving the vergence-accommodation conflict in head-mounted displays. *IEEE transactions on visualization and computer graphics, 22*(7), 1912-1931.</ref> | ||
The VAC forces the brain to deal with conflicting depth information, potentially leading to several issues: | The VAC forces the brain to deal with conflicting depth information, potentially leading to several issues: | ||
| Line 91: | Line 91: | ||
*Reduced realism and immersion | *Reduced realism and immersion | ||
The VAC is particularly problematic for interactions requiring sustained focus or high visual fidelity at close distances ( | The VAC is particularly problematic for interactions requiring sustained focus or high visual fidelity at close distances (for example virtual surgery simulation, detailed object inspection, reading text on near virtual objects). <ref name="HowardRogers2012"/> | ||
====Other Limitations==== | ====Other Limitations==== | ||
| Line 101: | Line 101: | ||
To mitigate or eliminate the VAC and provide more accurate depth cues, researchers and companies are actively developing advanced display technologies: | To mitigate or eliminate the VAC and provide more accurate depth cues, researchers and companies are actively developing advanced display technologies: | ||
*'''[[Varifocal Displays]]''': These displays dynamically adjust the focal distance of the display optics ( | *'''[[Varifocal Displays]]''': These displays dynamically adjust the focal distance of the display optics (for example using physically moving lenses/screens, [[liquid lens]] technology, or [[deformable mirror]] devices) to match the simulated distance of the object the user is currently looking at. <ref name="KonradVAC2016">Konrad, R., Cooper, E. A., & Banks, M. S. (2016). Towards the next generation of virtual and augmented reality displays. *Optics Express, 24*(15), 16800-16809. doi:10.1364/OE.24.016800</ref> <ref name="DunnVarifocal2017">Dunn, David, et al. (2017). Wide field of view varifocal near-eye display using see-through deformable membrane mirrors. *IEEE transactions on visualization and computer graphics, 23*(4), 1322-1331.</ref> This typically requires fast and accurate [[eye tracking]] to determine the user's point of gaze and intended focus depth. Varifocal systems often simulate [[Depth of Field]] effects computationally, blurring parts of the scene not at the current focal distance. <ref name="ARInsiderVAC"/> Prototypes like Meta Reality Labs' "Half Dome" series have demonstrated this approach. <ref name="ARInsiderVAC"/> | ||
*'''[[Multifocal Displays]] (Multi-Plane Displays)''': Instead of a single, continuously adjusting focus, these displays present content on multiple discrete focal planes simultaneously or in rapid succession. <ref name="AkeleyMultifocal2004">Akeley, Kurt, Watt, S. J., Girshick, A. R., & Banks, M. S. (2004). A stereo display prototype with multiple focal distances. *ACM transactions on graphics (TOG), 23*(3), 804-813.</ref> The visual system can then accommodate to the plane closest to the target object's depth. Examples include stacked display panels or systems using switchable lenses. Magic Leap 1 used a two-plane system. <ref name="ARInsiderVAC"/> While reducing VAC, they can still exhibit quantization effects if an object lies between planes, and complexity increases with the number of planes. | *'''[[Multifocal Displays]] (Multi-Plane Displays)''': Instead of a single, continuously adjusting focus, these displays present content on multiple discrete focal planes simultaneously or in rapid succession. <ref name="AkeleyMultifocal2004">Akeley, Kurt, Watt, S. J., Girshick, A. R., & Banks, M. S. (2004). A stereo display prototype with multiple focal distances. *ACM transactions on graphics (TOG), 23*(3), 804-813.</ref> The visual system can then accommodate to the plane closest to the target object's depth. Examples include stacked display panels or systems using switchable lenses. Magic Leap 1 used a two-plane system. <ref name="ARInsiderVAC"/> While reducing VAC, they can still exhibit quantization effects if an object lies between planes, and complexity increases with the number of planes. | ||
| Line 107: | Line 107: | ||
*'''[[Light Field Displays]]''': These displays aim to reconstruct the [[light field]] of a scene – the distribution of light rays in space – more completely. By emitting rays with the correct origin and direction, they allow the viewer's eye to naturally focus at different depths within the virtual scene, as if viewing a real 3D environment. <ref name="WetzsteinLightField2011">Wetzstein, Gordon, et al. (2011). Computational plenoptic imaging. *Computer Graphics Forum, 30*(8), 2397-2426.</ref> <ref name="Lanman2013">Lanman, D., & Luebke, D. (2013). Near-eye light field displays. *ACM Transactions on Graphics (TOG), 32*(6), 1-10. doi:10.1145/2508363.2508366</ref> This can potentially solve the VAC without requiring eye tracking. However, generating the necessary dense light fields poses significant computational and hardware challenges, often involving trade-offs between resolution, field of view, and form factor. <ref name="ARInsiderVAC"/> Companies like CREAL are developing light field modules for AR/VR. <ref name="WikiVAC"/> | *'''[[Light Field Displays]]''': These displays aim to reconstruct the [[light field]] of a scene – the distribution of light rays in space – more completely. By emitting rays with the correct origin and direction, they allow the viewer's eye to naturally focus at different depths within the virtual scene, as if viewing a real 3D environment. <ref name="WetzsteinLightField2011">Wetzstein, Gordon, et al. (2011). Computational plenoptic imaging. *Computer Graphics Forum, 30*(8), 2397-2426.</ref> <ref name="Lanman2013">Lanman, D., & Luebke, D. (2013). Near-eye light field displays. *ACM Transactions on Graphics (TOG), 32*(6), 1-10. doi:10.1145/2508363.2508366</ref> This can potentially solve the VAC without requiring eye tracking. However, generating the necessary dense light fields poses significant computational and hardware challenges, often involving trade-offs between resolution, field of view, and form factor. <ref name="ARInsiderVAC"/> Companies like CREAL are developing light field modules for AR/VR. <ref name="WikiVAC"/> | ||
* '''[[Holographic Displays]]''': True [[holography|holographic]] displays aim to reconstruct the wavefront of light from the virtual scene using diffraction, which would inherently provide all depth cues, including accommodation, correctly and continuously. <ref name="MaimoneHolo2017">Maimone, A., Georgiou, A., & Kollin, J. S. (2017). Holographic near-eye displays for virtual and augmented reality. *ACM Transactions on Graphics (TOG), 36*(4), 1-16. doi:10.1145/3072959.3073610</ref> This is often considered an ultimate goal for visual displays. However, current implementations suitable for near-eye displays face major challenges in computational load, achievable [[field of view]], image quality ( | * '''[[Holographic Displays]]''': True [[holography|holographic]] displays aim to reconstruct the wavefront of light from the virtual scene using diffraction, which would inherently provide all depth cues, including accommodation, correctly and continuously. <ref name="MaimoneHolo2017">Maimone, A., Georgiou, A., & Kollin, J. S. (2017). Holographic near-eye displays for virtual and augmented reality. *ACM Transactions on Graphics (TOG), 36*(4), 1-16. doi:10.1145/3072959.3073610</ref> This is often considered an ultimate goal for visual displays. However, current implementations suitable for near-eye displays face major challenges in computational load, achievable [[field of view]], image quality (for example [[speckle noise]]), and component size. <ref name="MaimoneHolo2017"/> <ref name="ARInsiderVAC"/> | ||
*'''[[Retinal Projection]] (Retinal Scan Displays)''': These systems bypass intermediate screens and project images directly onto the viewer's retina, often using low-power lasers or micro-LED arrays. <ref name="ARInsiderVAC"/> Because the image is formed on the retina, it can appear in focus regardless of the eye's accommodation state, potentially eliminating VAC. This approach could enable very compact form factors. Challenges include achieving a sufficiently large [[eye-box]] (the area where the eye can see the image), potential sensitivity to eye floaters or optical path debris, and safety considerations. <ref name="ARInsiderVAC"/> Examples include the discontinued North Focals smart glasses. | *'''[[Retinal Projection]] (Retinal Scan Displays)''': These systems bypass intermediate screens and project images directly onto the viewer's retina, often using low-power lasers or micro-LED arrays. <ref name="ARInsiderVAC"/> Because the image is formed on the retina, it can appear in focus regardless of the eye's accommodation state, potentially eliminating VAC. This approach could enable very compact form factors. Challenges include achieving a sufficiently large [[eye-box]] (the area where the eye can see the image), potential sensitivity to eye floaters or optical path debris, and safety considerations. <ref name="ARInsiderVAC"/> Examples include the discontinued North Focals smart glasses. | ||
| Line 127: | Line 127: | ||
*'''Visual Fatigue and Discomfort:''' The [[vergence-accommodation conflict]] is a primary contributor to eye strain, headaches, blurred vision, and general visual discomfort, especially during prolonged use. <ref name="HoffmanVAC2008"/> <ref name="ARInsiderVAC"/> | *'''Visual Fatigue and Discomfort:''' The [[vergence-accommodation conflict]] is a primary contributor to eye strain, headaches, blurred vision, and general visual discomfort, especially during prolonged use. <ref name="HoffmanVAC2008"/> <ref name="ARInsiderVAC"/> | ||
*'''Spatial Perception Errors:''' Inaccurate or conflicting depth cues can lead to misjudgments of distance, size, and the spatial relationships between objects, potentially affecting user performance in tasks requiring precise spatial awareness or interaction. <ref name="JonesVAC2008"/> <ref name="WillemsenHMD2009">Willemsen, Peter, Colton, M. B., Creem-Regehr, S. H., & Thompson, W. B. (2009). The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments. ''ACM Transactions on Applied Perception (TAP), 6''(2), 1-14.</ref> | *'''Spatial Perception Errors:''' Inaccurate or conflicting depth cues can lead to misjudgments of distance, size, and the spatial relationships between objects, potentially affecting user performance in tasks requiring precise spatial awareness or interaction. <ref name="JonesVAC2008"/> <ref name="WillemsenHMD2009">Willemsen, Peter, Colton, M. B., Creem-Regehr, S. H., & Thompson, W. B. (2009). The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments. ''ACM Transactions on Applied Perception (TAP), 6''(2), 1-14.</ref> | ||
*'''[[Simulator Sickness]]:''' Inconsistencies between visual depth cues and other sensory information ( | *'''[[Simulator Sickness]]:''' Inconsistencies between visual depth cues and other sensory information (for example vestibular signals from the inner ear) can contribute to symptoms like nausea, disorientation, and dizziness. <ref name="VosVAC2005"/> <ref name="WannAdaptation1995">Wann, John P., Simon Rushton, and Mark Mon-Williams. (1995). Natural problems for stereoscopic depth perception in virtual environments. *Vision research, 35*(19), 2731-2736.</ref> | ||
==Design Considerations for VR/AR Developers== | ==Design Considerations for VR/AR Developers== | ||
| Line 144: | Line 144: | ||
*'''Perceptual Adaptation:''' Studying how users adapt to inconsistent or unnatural depth cues over time, potentially leading to training paradigms or design strategies that improve comfort on current hardware. <ref name="WannAdaptation1995"/> | *'''Perceptual Adaptation:''' Studying how users adapt to inconsistent or unnatural depth cues over time, potentially leading to training paradigms or design strategies that improve comfort on current hardware. <ref name="WannAdaptation1995"/> | ||
*'''Personalized Depth Rendering:''' Calibrating depth cue presentation based on individual user characteristics ( | *'''Personalized Depth Rendering:''' Calibrating depth cue presentation based on individual user characteristics (for example IPD, visual acuity, refractive error, sensitivity to VAC) for optimized comfort and performance. <ref name="WillemsenHMD2009"/> | ||
*'''[[Cross-modal interaction|Cross-Modal Integration]]:** Investigating how integrating depth information from other senses ( | *'''[[Cross-modal interaction|Cross-Modal Integration]]:** Investigating how integrating depth information from other senses (for example [[spatial audio]], [[haptic feedback]]) can enhance or reinforce visual depth perception. <ref name="ErnstCrossModal2002">Ernst, Marc O., and Martin S. Banks. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. ''Nature, 415''(6870), 429-433.</ref> | ||
*'''[[Neural rendering|Neural Rendering]] and AI:''' Utilizing machine learning techniques ( | *'''[[Neural rendering|Neural Rendering]] and AI:''' Utilizing machine learning techniques (for example [[Neural Radiance Fields]] (NeRF)) to potentially render complex scenes with perceptually accurate depth cues more efficiently by learning implicit scene representations. <ref name="MildenhallNeRF2020">Mildenhall, Ben, et al. (2020). Nerf: Representing scenes as neural radiance fields for view synthesis. *European conference on computer vision*. Springer, Cham.</ref> | ||
==References== | ==References== | ||