|
|
| (4 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| {{see also|Terms|Technical Terms}}
| | #REDIRECT [[Near-eye light field display]] |
| A '''Near-eye lightfield display''' (NELFD) is a type of [[Near-eye display]] (NED), often implemented in a [[Head-mounted display]] (HMD), designed to reproduce a [[lightfield]], the complete set of light rays filling a region of space, rather than just a single flat [[image]] for the viewer. By emitting light rays with potentially correct spatial *and* angular distribution, a NELFD allows the viewer’s [[eye]]s to engage natural [[Vergence|vergence]] and [[Accommodation (visual)|accommodation]] (focusing) responses simultaneously. This capability aims to resolve the [[vergence-accommodation conflict]] (VAC), a common source of visual discomfort and fatigue in conventional [[stereoscopic]] displays used in [[virtual reality]] (VR) and [[augmented reality]] (AR)<ref name="Hoffman2008">Hoffman, D. M., Girshick, A. R., Akeley, K., & Banks, M. S. (2008). Vergence–accommodation conflicts hinder visual performance and cause visual fatigue. ''Journal of Vision'', 8(3), 33. doi:10.1167/8.3.33</ref>, leading to potentially sharper, more comfortable, and more realistic three-dimensional vision.
| |
| | |
| ==Principle of Operation==
| |
| Unlike traditional displays that emit light from each [[pixel]] as if originating from a single fixed-focus plane, lightfield displays attempt to reconstruct the four-dimensional function describing light rays passing through space by their position and direction. In the context of a near-eye display, this means controlling the [[intensity]], [[color]], and crucially, the '''direction''' of light rays that enter the viewer's [[pupil]] within a specific viewing volume known as the [[Eye Box|eyebox]]. A sufficiently large eyebox allows for some natural eye movement without losing the effect, related to the concept of the [[Exit pupil]] in optical systems.
| |
| | |
| Common technical approaches to generating the lightfield include:
| |
| | |
| * '''[[Microlens Array]] (MLA) based:''' An array of tiny lenses is placed over a high-resolution [[display panel]] (like an [[OLED]] or [[LCD]]). Each microlens samples a portion of the underlying pixels and projects them in specific directions, creating different views for different parts of the eye's pupil. This technique is related to [[integral imaging]] or [[plenoptic]] camera principles<ref name="Lanman2013">Lanman, D., & Luebke, D. (2013). Near-eye light field displays. ''ACM Transactions on Graphics (TOG)'', 32(4), Article 138. Presented at SIGGRAPH 2013. [https://research.nvidia.com/sites/default/files/pubs/2013-11_Near-Eye-Light-Field/NVIDIA-NELD.pdf PDF Link]</ref>, but inherently trades [[spatial resolution]] for [[angular resolution]] (i.e., number of views or depth cues).
| |
| * '''Multi-layer Displays:''' Using multiple stacked, typically transparent, display layers (e.g., LCDs) that multiplicatively modulate light passing through them. By controlling the patterns on each layer, often using [[computational display]] techniques, the directional light distribution can be approximated, potentially offering more continuous focus cues<ref name="Huang2015">Huang, F. C., Wetzstein, G., Barsky, B. A., & Heide, F. (2015). The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues. ''ACM Transactions on Graphics (TOG)'', 34(4), Article 60. Presented at SIGGRAPH 2015.</ref>.
| |
| * '''Varifocal / Multifocal Displays:''' Using optics whose focal length can be changed rapidly, such as [[Tunable lens|tunable lenses]] or mechanically actuated lenses/displays. These systems present images at different focal distances sequentially (time-multiplexed) or simultaneously. The visual system integrates these into a perception of depth, approximating a lightfield effect, particularly addressing accommodation<ref name="Akşit2019">Akşit, K., Lopes, W., Kim, J., Shirley, P., & Luebke, D. (2019). Manufacturing application-driven near-eye displays by combining 3D printing and thermoforming. ''ACM Transactions on Graphics (TOG)'', 38(6), Article 183. Presented at SIGGRAPH Asia 2019. (Discusses varifocal elements)</ref>.
| |
| * '''Scanning / Projection:''' Using light sources like [[laser]]s combined with scanning [[mirror]]s (such as [[MEMS]]) or projection [[optics]] to directly synthesize the lightfield point-by-point or line-by-line towards the eye's pupil<ref name="Schowengerdt2015">Schowengerdt, B. T., & Seibel, E. J. (2015). True 3D scanned voxel displays using single or multiple light sources. US Patent 9,025,213 B2.</ref>.
| |
| * '''[[Holographic display|Holographic]] Approaches:''' Using a [[Spatial light modulator]] (SLM), typically phase-only, to diffract light (usually from a laser) and reconstruct the [[wavefront]] of the desired 3D scene directly. This can potentially provide highly accurate focus cues but faces challenges like [[Speckle pattern|speckle]] and high computational requirements<ref name="Maimone2017">Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., & Fuchs, H. (2017). Holographic near-eye displays for virtual and augmented reality. ''ACM Transactions on Graphics (TOG)'', 36(4), Article 85. Presented at SIGGRAPH 2017.</ref>. [[Holographic optical element]]s (HOEs) or [[Metasurface]]s may also be used to manipulate light directionally, often in combination with a microdisplay.
| |
| * '''Computational Approaches:''' Combining specialized optics with sophisticated [[rendering]] algorithms to generate the lightfield effect, sometimes using techniques like compressive light field displays or optimized light patterns projected onto diffusers or specialized optical elements<ref name="Wetzstein2012">Wetzstein, G., Luebke, D., Heidrich, W. (2012). Hand-held Computational Light Field Photography and Display. ''IEEE Computer Graphics and Applications'', 32(1), 8-13.</ref>.
| |
| | |
| ==Advantages==
| |
| * '''Correct Focus Cues:''' The primary intended advantage. By reproducing directional light rays, NELFDs allow the eye's natural accommodation mechanism to work more naturally, mitigating or resolving the vergence-accommodation conflict (VAC).
| |
| * '''Improved [[Depth Perception]]:''' Providing multiple [[Depth cue|depth cues]] (binocular disparity, vergence, accommodation, [[defocus blur]]) leads to more realistic and accurate perception of 3D space.
| |
| * '''Reduced [[Visual Fatigue]]:''' By reducing the VAC, NELFDs can potentially decrease eye strain, headaches, and [[simulator sickness]] associated with prolonged use of conventional stereoscopic displays<ref name="Hoffman2008" />.
| |
| * '''Smoother [[Parallax]]:''' Can provide more continuous motion parallax as the viewer moves their eye slightly within the eyebox.
| |
| * '''Thinner/Lighter Form Factor (Potentially):''' MLA-based designs, for example, can replace bulky magnifying optics with compact microlens arrays, potentially leading to thinner and lighter HMDs<ref name="Lanman2013" />.
| |
| * '''Wider [[Eye Box]] (Potentially):''' Some lightfield display designs can offer a larger eyebox compared to conventional NED designs with small exit pupils, although this often involves trade-offs.
| |
| * '''Potential for [[Prescription]] Correction:''' Some lightfield approaches might computationally correct for the viewer's refractive errors (like myopia or hyperopia), a potential benefit demonstrated in early prototypes and an active area of research<ref name="Pamplona2012">Pamplona, V. F., Oliveira, M. M., Aliaga, D. G., & Raskar, R. (2012). Tailored displays to compensate for visual aberrations. ''ACM Transactions on Graphics (TOG)'', 31(4), Article 99. Presented at SIGGRAPH 2012.</ref><ref name="Lanman2013" />.
| |
| | |
| ==Challenges==
| |
| * '''Resolution Trade-off:''' Often a fundamental trade-off between spatial resolution (perceived sharpness) and angular resolution (number of distinct directions/depths). For MLA-based systems, the perceived resolution is reduced relative to the native microdisplay resolution, often proportional to the ratio of the lens focal length to the eye relief<ref name="Lanman2013" />.
| |
| * '''Computational Complexity:''' Generating the complex image data required for a lightfield display (lightfield rendering or hologram computation) is computationally intensive, requiring significant [[GPU]] power, high [[bandwidth]], and sophisticated algorithms. Real-time rendering for interactive applications is a major hurdle<ref name="Maimone2017" />. Techniques like GPU-accelerated ray tracing or specialized rasterization are often employed<ref name="Lanman2013" />.
| |
| * '''[[Form Factor]] and [[Weight]]:''' While some approaches promise thinner designs, implementing the necessary optics (MLAs, multiple layers, SLMs, scanning systems, varifocal mechanisms) within strict wearable constraints remains difficult.
| |
| * '''[[Field of View (FoV)]]:''' Achieving a wide field of view simultaneously with high spatial resolution, high angular resolution, a large eyebox, and compact form factor is extremely challenging.
| |
| * '''[[Brightness]] and [[Contrast ratio|Contrast Ratio]]:''' Some approaches, particularly those involving multiple layers, masks, MLAs, or diffractive elements, can suffer from reduced light efficiency (lower brightness) and potentially lower contrast compared to direct-view displays.
| |
| * '''[[Artifacts]]:''' Specific implementations can suffer from unique visual artifacts, such as [[Speckle pattern|speckle]] in holographic systems<ref name="Maimone2017" />, latency or visible plane-switching in varifocal systems, diffraction effects from small features, MLA boundary effects, or image discontinuities at the edge of the eyebox.
| |
| * '''Calibration:''' Precise manufacturing, alignment (rotational and lateral), and calibration of the optical components and display panels are critical and often complex, potentially requiring software correction<ref name="Lanman2013" />.
| |
| | |
| Recent reviews discuss ongoing research to overcome these challenges through advancements in display technology and computational techniques<ref name="Nature2024">[Naked-eye light field display technology based on mini/micro light emitting diode panels: a systematic review and meta-analysis | Scientific Reports](https://www.nature.com/articles/s41598-024-75172-z)</ref><ref name="Frontiers2022">[Frontiers | Challenges and Advancements for AR Optical See-Through Near-Eye Displays: A Review](https://www.frontiersin.org/journals/virtual-reality/articles/10.3389/frvir.2022.838237/full)</ref>.
| |
| | |
| ==Historical Development and Notable Examples==
| |
| While the underlying concept of capturing and displaying light fields dates back to Gabriel Lippmann's [[Integral imaging|integral photography]] (1908)<ref name="Lippmann1908">Lippmann, G. (1908). Épreuves réversibles donnant la sensation du relief [Reversible proofs giving the sensation of relief]. ''Journal de Physique Théorique et Appliquée'', 7(1), 821–825.</ref>, focused development on near-eye versions intensified with the rise of modern VR/AR and the identification of the VAC problem (around 2008)<ref name="Hoffman2008" />.
| |
| | |
| Key milestones and prototypes include:
| |
| * '''NVIDIA Near-Eye Light Field Display (2013):''' Presented by Lanman and Luebke (NVIDIA Research) at SIGGRAPH 2013, this influential prototype used [[Microlens Array|microlens arrays]] (Fresnel #630) over high-resolution [[OLED]] microdisplays (Sony ECX332A, 1280x720 per eye, 12µm pixel pitch) to demonstrate accurate focus cues resolving the VAC in a thin form factor (1cm eyepiece thickness). It also showed the potential for software-based [[Prescription|prescription correction]] and analyzed the spatio-angular trade-offs (achieving ~146x78 pixel resolution at ~29°x16° FoV in the demo)<ref name="Lanman2013" />.
| |
| * '''Stanford / NVIDIA Light Field Stereoscope (2015):''' An HMD demonstration using two stacked LCD layers to provide accommodation cues over a continuous range (0.2m to infinity) within a ~30° FoV<ref name="Huang2015" />.
| |
| * '''NVIDIA / UNC Holographic HMD (2017):''' Showcased a prototype using a 2k x 2k phase SLM and GPU computation to generate real-time holograms at 90 Hz with an 80° FoV<ref name="Maimone2017" />.
| |
| * '''Avegant Light Field Technology (2017 onwards):''' Demonstrated mixed reality prototypes using multiple simultaneous focal planes (~2-3 planes, ~40° FoV)<ref name="AvegantBlog2017">Avegant (2017, January 4). Avegant Demonstrates Light Field Technology For Mixed Reality Experiences. PR Newswire. https://www.prnewswire.com/news-releases/avegant-introduces-light-field-technology-for-mixed-reality-300423855.html</ref>.
| |
| * '''[[Magic Leap]] One (2018):''' The first widely available commercial HMD marketed with lightfield concepts ("photonic lightfield chip"), implemented using waveguides providing two fixed focal planes (~0.5m and infinity) over a ~50° diagonal FoV<ref name="MagicLeapSpecs">Based on technical specifications and reviews published circa 2018-2019. Original spec links may be defunct. Example review: UploadVR (2018). Magic Leap One Creator Edition In-Depth Review. [Link to reliable review or archived spec sheet if available]</ref>.
| |
| * '''[[Meta Reality Labs Research]] (formerly Facebook Reality Labs) Half-Dome Series (2018-2020):''' A series of research prototypes exploring varifocal displays. Half-Dome 1 used mechanical actuation; later versions like Half-Dome 3 used a stack of liquid crystal lenses to achieve 64 discrete focal planes electronically, combined with [[eye tracking]] and a wide FoV (~140°)<ref name="AbrashBlog2019">Abrash, M. (2019, September 25). Oculus Connect 6 Keynote [Video]. YouTube. Retrieved from https://www.youtube.com/watch?v=7YIGT13bdXw (Relevant discussion on Half-Dome prototypes)</ref>.
| |
| * '''CREAL (2020 onwards):''' A company developing compact lightfield display engines for AR, often using time-multiplexed micro-LED projection or scanning combined with holographic combiners. Prototypes aim for continuous focus (e.g., 0.15m to infinity) within a ~50-60° FoV in a glasses-like form factor<ref name="CrealWebsite">CREAL (n.d.). Technology. Retrieved from https://creal.com/technology/</ref>.
| |
| | |
| ==Applications==
| |
| The primary goal of NELFDs is to enhance visual realism and comfort in VR and AR:
| |
| * '''VR Comfort & Presence:''' Eliminating the VAC can significantly reduce eyestrain during long sessions and improve the sense of presence and depth judgment, aiding tasks requiring precise spatial awareness or interaction.
| |
| * '''AR Depth Coherence:''' Allows virtual objects to appear at specific, correct optical depths that match the real world, enabling seamless integration for applications like surgical overlays, industrial assembly guidance, and design visualization.
| |
| * '''Training & Simulation:''' More accurate rendering of depth and focus cues benefits tasks requiring precise hand-eye coordination, such as flight, driving, or medical simulators.
| |
| * '''Productivity & Close Work:''' Enables clear viewing of virtual text, user interfaces, or detailed objects at close distances, which is often problematic in fixed-focus HMDs.
| |
| | |
| ==Current Status and Future Outlook==
| |
| Near-eye lightfield displays remain predominantly in the research and development phase, although some aspects (like multi-plane or varifocal systems) are appearing in niche or high-end devices. The significant challenges, particularly the trade-offs between resolution, computation, FoV, and form factor, have prevented widespread adoption in consumer HMDs.
| |
| | |
| Ongoing research focuses on:
| |
| * Developing novel display panels (e.g., high-pixel-density [[MicroLED|microLEDs]] or OLEDs) and optics (HOEs, metasurfaces, advanced lens designs, potentially curved MLAs<ref name="Lanman2013"/>) to improve the spatio-angular resolution trade-off.
| |
| * Creating more efficient lightfield rendering algorithms, potentially using [[Artificial intelligence|AI]] / [[Machine learning|machine learning]] for reconstruction or up-sampling, and dedicated [[hardware acceleration]].
| |
| * Integrating high-speed, high-accuracy [[eye tracking]] to enable [[foveated rendering]] adapted for lightfields or dynamic optimization of the display based on gaze, potentially relaxing eyebox constraints or improving resolution/computational load.
| |
| * Exploiting redundancy in lightfields for error correction (e.g., compensating for dead pixels)<ref name="Lanman2013"/>.
| |
| * Hybrid approaches combining several techniques (e.g., a few focal planes with some angular diversity per plane) to achieve a "good enough" lightfield effect with current technology.
| |
| | |
| Longer-term advances in display panels, ultrafast SLMs, efficient computational methods, and compact diffractive or [[Metasurface|metasurface]] optics hold the potential for true continuous lightfield displays in lightweight, eyeglass-sized hardware, potentially making digital imagery optically much closer to viewing the real world.
| |
| | |
| ==See Also==
| |
| * [[Lightfield]]
| |
| * [[Near-eye display]]
| |
| * [[Head-mounted display]]
| |
| * [[Vergence-accommodation conflict]]
| |
| * [[Accommodation (visual)]]
| |
| * [[Vergence]]
| |
| * [[Depth perception]]
| |
| * [[Depth cue]]
| |
| * [[Microlens array]]
| |
| * [[Integral imaging]]
| |
| * [[Plenoptic camera]]
| |
| * [[Computational display]]
| |
| * [[Holographic display]]
| |
| * [[Spatial light modulator]]
| |
| * [[Volumetric display]]
| |
| * [[Virtual reality]]
| |
| * [[Augmented reality]]
| |
| * [[Mixed reality]]
| |
| * [[Eye tracking]]
| |
| * [[Foveated rendering]]
| |
| * [[Optics]]
| |
| * [[Eye Box]]
| |
| * [[Exit pupil]]
| |
| * [[Speckle pattern|Speckle]]
| |
| * [[Metasurface]]
| |
| * [[Emerging Technologies]]
| |
| | |
| ==References==
| |
| <references />
| |
| | |
| [[Category:Display Technology]]
| |
| [[Category:Virtual Reality]]
| |
| [[Category:Augmented Reality]]
| |
| [[Category:Optics]]
| |
| [[Category:Emerging Technologies]]
| |
