Focal surface display: Difference between revisions
Xinreality (talk | contribs) No edit summary |
Xinreality (talk | contribs) No edit summary |
||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 7: | Line 7: | ||
Focal surface display is a technology developed by [[Oculus]] Research that improves focus on images generated by a [[virtual reality]] (VR) [[head-mounted display]] (HMD) by simulating the way the eyes naturally focus at real object of varying depths (Figure 1). <ref name=”1”>Oculus VR (2017). Oculus Research to present focal surface display discovery at SIGGRAPH. Retrieved from https://www.oculus.com/blog/oculus-research-to-present-focal-surface-display-discovery-at-siggraph/</ref> | Focal surface display is a technology developed by [[Oculus]] Research that improves focus on images generated by a [[virtual reality]] (VR) [[head-mounted display]] (HMD) by simulating the way the eyes naturally focus at real object of varying depths (Figure 1). <ref name=”1”>Oculus VR (2017). Oculus Research to present focal surface display discovery at SIGGRAPH. Retrieved from https://www.oculus.com/blog/oculus-research-to-present-focal-surface-display-discovery-at-siggraph/</ref> | ||
While modern VR experiences are superior to what they were just a few years ago, the Oculus focal surface display addresses a perceptual limitation of current HMDs: not being able to display scene content at correct focal depths. These HMDs have a fixed-focus accommodation determined by the headset’s eyepiece focal length. Although they give the illusion of depth from the stereo images, the images are essentially flat, at a fixed perceived distance from the face and with a focus selected by the software instead of the eyes. Scene content with a virtual distance from the viewer different than the fixed focal distance of the headset’s screen will lead to a vergence-accommodation conflict - arising from binocular disparity cues (vergence) in conflict with focus cues (accommodation). The vergence-accommodation conflict prevents the VR content scenes from appearing sharply in focus and may contribute to user’s fatigue and discomfort. <ref name=”2”>Comp Photo Lab. Focal surface displays. Retrieved from http://compphotolab.northwestern.edu/project/focal-surface-displays/</ref> <ref name=”3”>Miller, P. (2017). Oculus Research's focal surface display could make VR much more comfortable for our eyeballs. Retrieved from https://www.theverge.com/circuitbreaker/2017/5/19/15667172/oculus-research-focal-surface-display-vr-comfort-eye-tracking</ref> <ref name=”4”>Coppock, M. (2017). Oculus developing ‘focal surface display’ for better VR image clarity. Retrieved from https://www.digitaltrends.com/computing/oculus-working-on-focal-surface-display-technology-for-improved-visual-clarity</ref> | While modern VR experiences are superior to what they were just a few years ago, the Oculus focal surface display addresses a perceptual limitation of current HMDs: not being able to display scene content at correct focal depths. These HMDs have a fixed-focus accommodation determined by the headset’s eyepiece focal length. Although they give the illusion of depth from the stereo images, the images are essentially flat, at a fixed perceived distance from the face and with a focus selected by the software instead of the eyes. Scene content with a virtual distance from the viewer different than the fixed focal distance of the headset’s screen will lead to a [[vergence-accommodation conflict]] - arising from binocular disparity cues (vergence) in conflict with focus cues (accommodation). The vergence-accommodation conflict prevents the VR content scenes from appearing sharply in focus and may contribute to user’s fatigue and discomfort. <ref name=”2”>Comp Photo Lab. Focal surface displays. Retrieved from http://compphotolab.northwestern.edu/project/focal-surface-displays/</ref> <ref name=”3”>Miller, P. (2017). Oculus Research's focal surface display could make VR much more comfortable for our eyeballs. Retrieved from https://www.theverge.com/circuitbreaker/2017/5/19/15667172/oculus-research-focal-surface-display-vr-comfort-eye-tracking</ref> <ref name=”4”>Coppock, M. (2017). Oculus developing ‘focal surface display’ for better VR image clarity. Retrieved from https://www.digitaltrends.com/computing/oculus-working-on-focal-surface-display-technology-for-improved-visual-clarity</ref> | ||
According to Oculus Research, the focal surface display has a new approach to avoid the vergence-accommodation conflict by changing the way light enters the display using spatial light modulators (Figure 2) to bend the HMD’s focus around 3D objects. This results in an increased depth and maximizes the amount of space represented. <ref name=”1”></ref> | According to Oculus Research, the focal surface display has a new approach to avoid the vergence-accommodation conflict by changing the way light enters the display using spatial light modulators (Figure 2) to bend the HMD’s focus around 3D objects. This results in an increased depth and maximizes the amount of space represented. <ref name=”1”></ref> | ||
| Line 15: | Line 15: | ||
The development of the Oculus focal surface display was an interdisciplinary task, “combining leading hardware engineering, scientific and medical imaging, computer vision research, and state-of-the-art algorithms to focus on next-generation VR.” This technology could even allow people who wear corrective lenses use a VR HMD without glasses. <ref name=”1”></ref> | The development of the Oculus focal surface display was an interdisciplinary task, “combining leading hardware engineering, scientific and medical imaging, computer vision research, and state-of-the-art algorithms to focus on next-generation VR.” This technology could even allow people who wear corrective lenses use a VR HMD without glasses. <ref name=”1”></ref> | ||
There had been previous attempts to solve the vergence-accommodation conflict such as using integral imaging techniques to synthesize light | There had been previous attempts to solve the vergence-accommodation conflict such as using integral imaging techniques to synthesize [[light field]]s from scene content or displaying multiple focal planes, but these suffered from such problems as low fidelity accommodation cues, low resolution, and low field of view. The focal surface display is expected to generate high fidelity accommodation cues using off-the-shelf optical components. The spatial light modulator - placed between the display screen and eyepiece - produces variable focus along the display field of view. <ref name=”2”></ref> | ||
As of 2017, there was no known planned commercial release for focal surface display technology. <ref name=”6”></ref> | |||
==Development and announcement of the focal surface display== | ==Development and announcement of the focal surface display== | ||
The Oculus focal surface display project was a long time in development. According to a research scientist at Oculus Research, “manipulating focus isn’t quite the same as modulating intensity or other more usual tasks in computational displays, and it took us a while to get to the correct mathematical formulation that finally brought everything together. Our overall motivation was to do things the | The Oculus focal surface display project was a long time in development. According to a research scientist at Oculus Research, “manipulating focus isn’t quite the same as modulating intensity or other more usual tasks in computational displays, and it took us a while to get to the correct mathematical formulation that finally brought everything together. Our overall motivation was to do things the "right" way: solid engineering combined with the math and algorithms to back it up. We weren’t going to be happy with something that only worked on paper or a hacked together prototype that didn’t have any rigorous explanation of why it worked.” <ref name=”1”></ref> | ||
On May, 2017, the VR and AR R&D division of Oculus - Oculus Research - announced the new display technology. During the same period, they published a research paper about their focal surface display, authored by Oculus scientists Nathan Matsuda, Alexander Fix, and Douglas Lanman. The research was also presented at the SIGGRAPH conference in July, 2017. <ref name=”7”>Lang, B. (2017). Oculus Research reveals “groundbreaking” focal surface display. Retrieved from https://www.roadtovr.com/oculus-research-demonstrate-groundbreaking-focal-surface-display/</ref> | On May, 2017, the VR and AR R&D division of Oculus - Oculus Research - announced the new display technology. During the same period, they published a research paper about their focal surface display, authored by Oculus scientists Nathan Matsuda, Alexander Fix, and [[Douglas Lanman]]. The research was also presented at the SIGGRAPH conference in July, 2017. <ref name=”7”>Lang, B. (2017). Oculus Research reveals “groundbreaking” focal surface display. Retrieved from https://www.roadtovr.com/oculus-research-demonstrate-groundbreaking-focal-surface-display/</ref> | ||
==Focal Surface display technology== | ==Focal Surface display technology== | ||
| Line 33: | Line 33: | ||
Different HMD architectures have been proposed to solve this problem and depict correct or near-correct retinal blur (Figure 3). The focal surface displays augment regular HMDs with a spatial light modulator that “acts as a dynamic freeform lens, shaping synthesized focal surfaces to conform to the virtual scene geometry.” Furthermore, Oculus Research has introduced “a framework to decompose target focal stacks and depth maps into one or more pairs of piecewise smooth focal surfaces and underlying display images,” building on “recent developments in "optimized blending" to implement a multifocal display that allows the accurate depiction of occluding, semi-transparent, and reflective objects.” <ref name=”5”></ref> | Different HMD architectures have been proposed to solve this problem and depict correct or near-correct retinal blur (Figure 3). The focal surface displays augment regular HMDs with a spatial light modulator that “acts as a dynamic freeform lens, shaping synthesized focal surfaces to conform to the virtual scene geometry.” Furthermore, Oculus Research has introduced “a framework to decompose target focal stacks and depth maps into one or more pairs of piecewise smooth focal surfaces and underlying display images,” building on “recent developments in "optimized blending" to implement a multifocal display that allows the accurate depiction of occluding, semi-transparent, and reflective objects.” <ref name=”5”></ref> | ||
Contrary to multifocal displays with fixed focal surfaces, the phase modulator shapes focal surfaces to conform to the scene geometry. A set of color images are produced and mapped onto a corresponding focal surface (Figure 4), with visual appearance being rendered by “tracing rays from the eye through the optics, and accumulating the color values for each focal surface.” Furthermore, Matsuda ''et al''. (2017) explain that their “algorithm sequentially solves for first the focal surfaces, given the target depth map, and then the color | Contrary to multifocal displays with fixed focal surfaces, the phase modulator shapes focal surfaces to conform to the scene geometry. A set of color images are produced and mapped onto a corresponding focal surface (Figure 4), with visual appearance being rendered by “tracing rays from the eye through the optics, and accumulating the color values for each focal surface.” Furthermore, Matsuda ''et al''. (2017) explain that their “algorithm sequentially solves for first the focal surfaces, given the target depth map, and then the color images, full joint optimization is left for future work. Focal surfaces are adapted by nonlinear least squares optimization, minimizing the distance between the nearest depicted surface and the scene geometry. The color images, paired with each surface, are determined by linear least squares methods.” <ref name=”5”></ref> | ||
The focal surface display research team demonstrated that the technology depicts more accurate retinal blur, with lesser multiplexed images, with high resolution being maintained throughout the user’s accommodative range. <ref name=”5”></ref> | The focal surface display research team demonstrated that the technology depicts more accurate retinal blur, with lesser multiplexed images, with high resolution being maintained throughout the user’s accommodative range. <ref name=”5”></ref> | ||