Light field stereoscope
More actions
The light field stereoscope is a prototype device made at the computational imaging lab at Stanford University. It is an evolution of the stereoscope that incorporates light field rendering for a multifocal display.[1] The design was published by Fu-Chung Huang, Kevin Chen and Gordon Wetzstein in ACM Transactions on Graphics as a SIGGRAPH 2015 technical paper and demonstrated at SIGGRAPH 2015 in Los Angeles in August 2015.[2][3]
The headset places two stacked, transparent LCD panels behind a pair of magnifying lenses, so that each eye sees a four-dimensional light field instead of a single flat image. Because the image changes slightly across the viewer's pupil, the eye can focus at different depths in the virtual scene, which conventional stereoscopic head-mounted displays do not allow. The authors built it to address the vergence-accommodation conflict, and described it as the first wearable VR display to support both high image resolution and focus cues.[2]
Background
Conventional VR headsets reproduce binocular disparity, vergence and motion parallax, but the eyes must stay focused at the fixed distance of the screen's virtual image while they converge on objects at other simulated distances. Huang, Chen and Wetzstein cite studies linking this vergence-accommodation conflict to visual discomfort, fatigue, eyestrain, double vision, headaches, nausea and compromised image quality.[2] Stanford's announcement of the project put the problem in plain terms: in current "flat" stereoscopic headsets each eye sees only one image, and the eye is forced to focus on a single plane.[3]
The paper reviews earlier focus-supporting approaches and their trade-offs for wearable use. The extreme requirements of dynamic holography on feature size, computation and light-source coherence made it, in the authors' view, ill-suited to headsets in the near future. Volumetric displays with spinning or switchable diffusers were too large. Multi-focal-plane displays approximate a volume with a few additive image planes; citing MacKenzie et al., the authors note that about five planes spaced one diopter apart would cover accommodation from 25 cm to optical infinity, but that this would require display refresh rates of at least 300 Hz so that each plane is updated at 60 Hz, and that such time-multiplexed displays can flicker. Microlens-based near-eye light field displays, such as the 2013 design by Douglas Lanman and David Luebke, trade spatial resolution directly for the supported depth range.[2]
The light field stereoscope builds instead on earlier factored or "compressive" multilayer light field displays, in which stacked attenuating layers are driven by computed patterns. The authors describe their work as the first to apply this approach to near-eye VR displays, and as a hybrid between light field displays and multi-focal-plane displays.[2] The paper traces the conventional stereoscopic half of the design to Charles Wheatstone's 1838 stereoscope,[2] while the SIGGRAPH Emerging Technologies description of the demonstration names Brewster's original stereoscope as the starting point.[4]
How it works
Each eye looks through a magnifying lens at two stacked liquid crystal panels lit by a uniform backlight. The two panels modulate the backlight multiplicatively, and the lens forms a magnified virtual image of each panel at a different distance: one close to the viewer and one close to optical infinity.[2] A ray of light that reaches the pupil passes through one pixel on each panel, so different points on the pupil receive different combinations of pixels. The pair of panels can therefore reproduce a light field with parallax over the eye box, rather than a single two-dimensional image.[2][3]
The patterns shown on the two panels are computed by nonnegative light field factorization. For every frame the system renders a target light field (5 x 5 views per eye in the prototype) and solves for the front and rear panel images whose product best approximates it, using iterative update rules similar to those of earlier factored displays.[2] The central observation of the paper is that a near-eye display only has to cover a very small viewing region: the eye box is just slightly larger than the pupil. Over such a short baseline the views of the light field are very similar, so a rank-1 factorization, a single pair of panel images shown at once, is enough to produce correct or nearly correct focus cues. Earlier factored light field displays for television-type viewing needed time-multiplexing on high-speed panels, whereas the light field stereoscope works with the refresh rate of conventional panels. Because the light field is computed for the whole eye box, the design needs no eye tracking.[2][1]
The light field representation also allows lens aberrations to be corrected in four dimensions. The authors model lens distortion with a radially symmetric polynomial and pre-distort the target light field with the inverse distortion before factorizing it. Without this correction the prototype showed strong pincushion distortion, and images on the front and rear panels did not line up.[2] The coefficients can be chosen per wavelength to account for chromatic aberration as well.[2]
In simulations comparing retinal blur, the authors found that multiplicative layers reproduce blur and monocular occlusions better than additive multi-focal-plane displays. An additive display can only emit light, so a dark object in front of a bright background appears semi-transparent; stacked attenuating LCDs can block light and avoid this. Microlens-based near-eye light field displays could produce good retinal blur in the same comparison, but the authors judged their loss of spatial resolution unacceptable for most applications.[2]
Prototype
The team built several prototypes with different liquid crystal panels, and settled on Chimei Innolux N070ICG-LD1 panels as one of the lowest-cost and most widely available options.[2] Stanford described the prototype as made from off-the-shelf parts, with the two transparent LCDs separated by a spacer.[3] The published parameters of the prototype are:
| Parameter | Value |
|---|---|
| LCD panels | Two stacked Chimei Innolux N070ICG-LD1 panels, 1280 x 800 pixels[2] |
| Lenses | Aspheres with a focal length of 5 cm and a diameter of 2 inches, separated by 6.4 cm[2] |
| Panel positions | Physical panels 3.8 cm and 4.8 cm from the lenses; virtual images at 19 cm and 123 cm (5.26 D and 0.81 D)[2] |
| Field of view | 87 x 91 degrees[2] |
| Accommodation range | Approximately 0.2 m to 1.2 m[2] |
| End-to-end latency | About 65 ms[2] |
| Housing | Modified from Adafruit's 3D-printed wearable video goggles, fabricated on a Makerbot Replicator 2[2] |
| Host computer | Dell XPS 8700 workstation, Intel i7 CPU, 8 GB RAM, NVIDIA GTX 970 GPU[2] |
| Software | C++, OpenGL for rendering, CUDA for the factorization[2] |
| Frame rate | About 20 fps on a GTX 970 and about 35 fps on a Quadro K6000, for simple scenes at 640 x 800 pixels with 5 x 5 views per eye[2] |
The lab published the 3D-printable housing files (.stl) and additional results as supplementary material.[1] The factorization runs one iteration per displayed frame and uses the previous result as the starting point for the next, so static objects are refined over several frames while fast-moving objects show slight motion blur. The measured runtimes for four test scenes on the GTX 970, with 5 iterations per frame, were:[2]
| Model | Vertices / faces | Draw, mono (ms) | Solve, mono (ms) | Draw, stereo (ms) | Solve, stereo (ms) | Stereo frame rate (fps) |
|---|---|---|---|---|---|---|
| Lion | 5K / 10K | 4.5 | 57.5 | 9 | 54.5 | 15.7 |
| Hands | 10K / 20K | 7.5 | 55.0 | 14 | 49.5 | 15.7 |
| Bench | 30K / 50K | 15.5 | 49.5 | 31 | 37.5 | 14.6 |
| Armadillo | 25K / 50K | 15.5 | 50.5 | 29.5 | 41.0 | 14.2 |
The researchers measured the gamma of the displays as 2.4 and calibrated their black levels. They corrected lateral misalignment between the panels mechanically and then in software.[2]
Captured content
To show that photographic content could be prepared for the display, the team mounted a Lytro Illum light field camera on a manual translation stage and captured two light fields about 6.4 cm apart, matching a typical interpupillary distance. At its shortest focal length the camera's aperture corresponded to a pupil or eye box diameter of 4.75 mm, and the resulting stereo light fields were factored and shown on the prototype.[2]
Limitations
The authors name diffraction as the main limitation. The virtual image of the rear panel is always seen through the front panel, whose pixel structure diffracts light and blurs the rear image, so raising the resolution of the front panel lowers the resolution observed on distant objects. They recommend a lower-resolution front panel (possibly grayscale) with a high-resolution rear panel, or reflective microdisplays such as liquid crystal on silicon if pixel fill factors near 100 percent became available.[2] The rank-1 constraint can also limit sharpness for objects between the two panel planes.[2]
Stacking two LCDs lowers light throughput. The authors argue that this matters little in immersive VR, where the visual system adapts to the display brightness, but that it is a serious problem for augmented reality displays that must compete with bright outdoor backgrounds, and they consider the design most suitable for VR.[2] The extra computation adds latency, and light field rendering was the performance bottleneck: each frame required 2 x 5 x 5 rendered images, and rendering time grows quadratically with the number of views.[2] In its coverage, Road to VR raised a separate concern that the design relies on LCD panels at a time when headset makers were favoring OLED displays with low persistence to reduce motion blur.[5]
The 2015 paper did not include a user study of accommodation. The authors wrote that the device's small form factor did not allow them to evaluate it with the Grand Seiko autorefractor available to the team, because the optical paths could not be matched, and they stated that the ability of people to accommodate to the display still had to be verified experimentally. They proposed eye tracking and foveated rendering of the light field as future work.[2]
Presentation and coverage
The paper appeared in volume 34, issue 4 of ACM Transactions on Graphics, the SIGGRAPH 2015 proceedings.[2][6] A working prototype was also shown in the Emerging Technologies program of SIGGRAPH 2015; that demonstration is credited to Fu-Chung Huang, David Luebke and Gordon Wetzstein.[4] NVIDIA Research lists the demonstration with a date of 20 July 2015, giving Huang's affiliation as NVIDIA and Wetzstein's as Stanford University.[7]
Stanford announced the project in early August 2015, saying the findings would be presented and demonstrated with help from NVIDIA at SIGGRAPH 2015 in Los Angeles on 9-13 August.[3][8] Wetzstein, then an assistant professor of electrical engineering, said: "You have a virtual window which ideally looks the same as the real world, whereas today you basically have a 2D screen in front of your eye." The announcement named robotic surgery, phobia treatment, education and entertainment as applications that could benefit from more comfortable long-term viewing.[3] Road to VR reported at the time that the unit was being developed at NVIDIA Research.[5]
Later research
In a 2016 overview of focus-supporting near-eye displays, Wetzstein grouped the approaches into adaptive focus, volumetric, light field and holographic displays, and listed the light field stereoscope alongside the designs of Hua and Javidi (2014) and Lanman and Luebke (2013) as current-generation light field displays that still offer limited resolution.[6] His group's later work on focus-tunable near-eye displays used programmable liquid lenses instead.[6] A 2020 Stanford study of ocular parallax rendering by Robert Konrad, Anastasios Angelopoulos and Wetzstein listed the light field stereoscope among the light field displays that inherently provide ocular parallax cues.[9]
See also
References
- ↑ 1.0 1.1 1.2 "The Light Field Stereoscope - SIGGRAPH 2015". Stanford Computational Imaging Lab. Stanford University. https://www.computationalimaging.org/publications/the-light-field-stereoscope/. Retrieved 2026-09-27.
- ↑ 2.00 2.01 2.02 2.03 2.04 2.05 2.06 2.07 2.08 2.09 2.10 2.11 2.12 2.13 2.14 2.15 2.16 2.17 2.18 2.19 2.20 2.21 2.22 2.23 2.24 2.25 2.26 2.27 2.28 2.29 2.30 2.31 2.32 Fu-Chung Huang, Kevin Chen, Gordon Wetzstein (2015). "The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues". ACM Transactions on Graphics, vol. 34, no. 4 (Proc. SIGGRAPH 2015). doi:10.1145/2766922. https://www.computationalimaging.org/wp-content/uploads/2015/06/TheLightFieldStereoscope-SIGGRAPH2015.pdf. Retrieved 2026-09-27.
- ↑ 3.0 3.1 3.2 3.3 3.4 3.5 Vignesh Ramachandran (2015-08-04). "Stanford researchers unveil virtual reality headset that reduces eye fatigue, nausea". Stanford Engineering. Stanford University. https://engineering.stanford.edu/news/stanford-researchers-unveil-virtual-reality-headset-reduces-eye-fatigue-nausea. Retrieved 2026-09-27.
- ↑ 4.0 4.1 "The Light Field Stereoscope by Huang, Luebke and Wetzstein". ACM SIGGRAPH History Archive. ACM SIGGRAPH. https://history.siggraph.org/experience/the-light-field-stereoscope-by-huang-luebke-and-wetzstein/. Retrieved 2026-09-27.
- ↑ 5.0 5.1 Paul James (2015-08-05). "Stanford Unveils 'Light Field Stereoscope' a VR Headset to Reduce Fatigue and Nausea". Road to VR. https://roadtovr.com/stanford-unveils-light-field-stereoscope-a-vr-headset-to-reduce-fatigue-and-nausea/. Retrieved 2026-09-27.
- ↑ 6.0 6.1 6.2 Gordon Wetzstein (2017). "Computational Near-Eye Displays: Engineering the Interface to the Digital World". Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2016 Symposium. National Academies Press. doi:10.17226/23659. https://www.nationalacademies.org/read/23659/chapter/4. Retrieved 2026-09-27.
- ↑ "The Light Field Stereoscope". NVIDIA Research. NVIDIA. 2015-07-20. https://research.nvidia.com/publication/2015-07_light-field-stereoscope. Retrieved 2026-09-27.
- ↑ Vignesh Ramachandran (2015-08-03). "Stanford researchers unveil virtual reality headset that reduces eye fatigue, nausea". Stanford Report. Stanford University. https://news.stanford.edu/stories/2015/08/virtual-reality-headset-080315. Retrieved 2026-09-27.
- ↑ Robert Konrad, Anastasios Angelopoulos, Gordon Wetzstein (2020). "Gaze-Contingent Ocular Parallax Rendering for Virtual Reality". ACM Transactions on Graphics, vol. 39, no. 2 (arXiv preprint). https://arxiv.org/abs/1906.09740. Retrieved 2026-09-27.