View source for Near-eye light field display

[[File:Lightfields.jpg|thumb|Figure 1. Light fields (Image: fudzilla.com)]]
__NOTOC__
'''Near-eye light field display'''s introduce a light-field-based approach to near-eye displays. This allows for HMDs capable of depicting accurate accommodation, convergence, and binocular-disparity [[depth cue]]s that are also thinner and lighter. The two human eyes perceive the world slightly differently. In the same way, light rays that enter the pupil at different location will encode a slightly different picture of the world being observed <ref name=”3”></ref> <ref name=”4”> Fattal, D. (2016). The ultimate guide to 3D technologies. Retrieved from https://thenextweb.com/insider/2016/04/23/guide-to-3d-tech/#</ref>. A light field is composed of all the light rays at every point in space travelling in every direction. It is a 4D data, since every point in three-dimensional space is attributed a direction. This concept came about in the 1990s as a solution to problems in computer graphics and vision <ref name=”5”> LightField Forum. Refocus your Eyes: Nvidia presents Near-Eye Light Field Display Prototype. Retrieved from http://lightfield-forum.com/2013/07/refocus-your-eyes-nvidia-presents-near-eye-light-field-display-prototype/</ref>. Near-eye light field displays must independently render light rays that are coming from every direction through every point in space in order to trigger accommodation. Sharp images from out-of-focus display elements are depicted by synthesizing these light fields that correspond to virtual scenes located within the viewer’s natural accommodation range (Figure 1). Lanman and Luebke (2013) mention that “conventional displays are intended to emit light isotropically. In contrast, a light ﬁeld display supports the control of tightly-clustered bundles of light rays, modulating radiance as a function of position and direction across its surface.” <ref name=”2”></ref> <ref name=”3”></ref> <ref name=”4”></ref>

Near-eye displays (NEDs) project images into a viewer’s eyes, creating a virtual image in their field of view. The image appears at a distance, and larger than the small display panel and optics used to create it. However, according to [[Doug Lanman]] and Luebke (2013), these kind of displays have a fundamental problem: the unaided human eye cannot accommodate (focus) on objects placed in close proximity <ref name=”1”> Bhakta, V.R., Richuso, J. and Jain, A. (2014). DLP ® Technology for Near Eye Display. Retrieved from http://www.ti.com/lit/wp/dlpa051/dlpa051.pdf</ref> <ref name=”2”> Lanman, D. and Luebke, D. (2013). Near-Eye Light Field Displays. ACM Transactions on Graphics, 32(6)</ref> <ref name=”3”> Stanford University. Near-Eye Light Field Displays. Retrieved from https://talks.stanford.edu/douglas-lanman-near-eye-light-field-displays/</ref>.

NEDs are also known as [[Head-mounted Display|head-mounted displays]] (HMDs) and encompass electronic viewfinders. Bhakta et al. (2014) referred that “near-eye displays are the headphones of the display world, creating small, portable, personal viewing experiences.” They have several advantages over traditional displays, such has a compact size, being lightweight, demanding low power, and can be see-through, being able to produce a virtual image that looks like a big screen TV from a small form factor. Furthermore, NEDs can be placed in two general categories: immersive and see-through. Immersive NEDs block the user’s view of the real world, creating a large field of view image (e.g., VR headset). See-through NEDs allow for the user to see the real world, generating a transparent image or a very small opaque image that blocks a small portion of the user’s peripheral vision. Examples of see-through NEDs are [[augmented reality]] headsets or smart glasses like the [[Google Glass]] <ref name=”1”></ref> <ref name=”2”></ref> <ref name=”3”></ref>.

Traditional HMDs only provide a single display plane; without a proper focus cue, the display decouples accommodations from the vergence of the eyes. Since there is a mismatch, the observer has to rely only on the binocular vision to perceive a 3D space. This can lead to visual discomfort, fatique, eye strain, and headaches <ref name=”6”> Stanford Computational Imaging Lab (2015). The Light Field Stereoscope - SIGGGRAPH 2015 [Video]. Retrieved from https://www.youtube.com/watch?v=YJdMPUF8cDM</ref>.

There have been demonstrations that light field displays allow for small form factors of NEDs. This was made by placing a microlens array on a small screen close to the eye. In near-eye light field displays the image created appears to be floating outside the physical device enclose, and the observer can accommodate with a narrow range. However, the lens used in the studies have a tradeoff between achieved spatial resolution and the supported depth range <ref name=”2”></ref> <ref name=”7”> Huang, Fu-Chung, Chen, K. and Wetzstein, G. (2015). The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues. ACM Transactions on Graphics, 34(4)</ref>. Another technique used to implement light field displays is to stack liquid crystal displays (LCDs). In this case, the image formation is multiplicative, allowing for correct or nearly-correct focus cues to be supported over larger depth ranges. Alternatively, it reduces the number of required display planes <ref name=”7”></ref>.

With the [[Oculus Rift]], the commercial interest in HMDs increased. Indeed, over the last few years interest in VR has been increasing, both by researchers as well as consumers. NED technology has a vast range of applications besides gaming and entertainment. It can be applied in education, teleconferencing, scientific visualization, training and simulation, phobia treatment, and surgical training, for example. Immersive VR has also been demonstrated to be effective in the treatment of post-traumatic stress disorder. For the continuing development of VR. It is essential to have a visually comfortable experience, such as diminishing the vergence-accommodation conflict that occurs in most HMDs. The improvement of light field displays is a path to the creation of better and more visually comfortable headsets <ref name=”2”></ref> <ref name=”7”></ref> <ref name=”8”> Huang, Fu-Chung, Chen, K. and Wetzstein, G. (2015). The Light Field Stereoscope | SIGGRAPH 2015. Retrieved from http://www.computationalimaging.org/publications/the-light-field-stereoscope/</ref>. Huang et al. (2015) wrote that “correct or nearly correct focus cues signiﬁcantly improve stereoscopic correspondence matching, 3D shape perception becomes more veridical, and people can discriminate different depths better. Vergence and accommodation cues are neurally coupled in the human brain; it seems intuitive that displays supporting all depth cues improve visual comfort and performance in long-term experiences.” <ref name=”7”></ref>

==Near-eye light field display prototypes==

[[File:NE-LF prototype.png|thumb|Figure 2. NVIDIA’s near-eye light field display prototype (image: Lanman and Luebke, 2013)]]

[[File:Lightfield stereoscope.jpg|thumb|Figure 3. The light field stereoscope prototype (Image: Huang et al., 2015)]]

[[File:LFS images.jpg|thumb|Figure 4. Images with front and rear focus produced by the light field stereoscope (Image: Huang et al., 2015)]]

[[File:Nvidia-light-field-stereoscope-schematic.jpg|thumb|Figure 5. Light Field Stereoscope schematic (Image: fudzilla.com)]]

During the 2013 SIGGRAPH (Special Interest Group on Computer Graphics and Interactive Techniques) conference, NVIDIA showed the product of its research sector: a near-eye light field display prototype (Figure 2). It consisted of a pair of Sony ECX332A OLED micro-displays with a pixel density of about 2100 ppi. The display panels measured 15.36 x 8.64 mm, with a resolution of 1280 x 720 through 24-bit color pixels, and were installed on a glasses-like frame with a small box of electronics on top <ref name=”5”></ref><ref name=”9”> Steele, B. (2013). NVIDIA Research's near-eye light field display prototype eyes-on (video). Retrieved from https://www.engadget.com/2013/07/24/nvidia-research-near-eye-light-field-display-prototype/</ref>.

The microlens arrays mounted in front of the displays are used to convert pixels to individual light rays, generating a light field in front of the eye. It allows the viewer to focus at multiple depths and create a field of view of approximately 70 degrees. Users who experimented the prototype during the conference confirmed both aspects. Furthermore, they reported that despite being situated close to the eye, the prototype still provided some sharp images. Nevertheless, the proximity caused some pixel loss due to a decreased spacial resolution. Another interesting aspect of this prototype is that adjustments can be made, at the level of software, to take into account the user’s glasses or contacts prescription. The software is powered by NVIDIA GPUs and OpenGL <ref name=”5”></ref><ref name=”9”></ref>.

NVIDIA, in collaboration with Stanford Computational Imaging, presented a new near-eye display technology that supports focus cues (accommodation and retinal blur) and high image resolution during SIGGRPAH’s 2015 conference (Figure 3). The prototype was based on Wheatstone’s original stereoscope, augmented with modern factored light field synthesis via stacked liquid crystal panels <ref name=”8”></ref>. Huang et al. (2015) explain that “the light field stereoscope is a near-eye display that facilitates immersive computer graphics via stereoscopic image synthesis with correct or nearly correct focus cues. As opposed to presenting conventional 2D images, the display shows a 4D light ﬁeld to each eye, allowing the observer to focus within the scene. The display comprises two stacked liquid crystal displays (LCDs) driven by nonnegative light ﬁeld factorization.” (Figure 4) <ref name=”7”></ref> 

A light field is presented to each eye, providing a more natural viewing experience than conventional NEDs. The required field of view is very small (the size of the pupil), and it produces correct or nearly-correct focus cues. These cues are important for diminishing visual discomfort and contributing to comfortable, long-term immersive experiences. The developers of the light field stereoscope have the main goal of providing a practical, inexpensive display technology that supports focus cues in a wearable form factor <ref name=”7”></ref><ref name=”8”></ref>. 

NVIDIA showed a revision of its light field prototype during the Virtual Reality LA Expo of 2016, in Southern California. The unit was attached to a desktop PC equipped with a Maxwell-based GPU. To compute the light fields images in real-time, NVIDIA used an algorithm based on its CUDA parallel programming language for GPUs. Still in collaboration with Stanford University, the new prototype eliminates the headphone requirement with a VR headset design that does not produce motion sickness side effects. The light field stereoscope uses two layered displays with back-to-back LCD panels. These are in-between a backlight and two circular viewing lenses (Figure 5). The device creates a natural depth-of-field in each eye, combining it with a stereoscopic technique (showing images at slightly different angles in each eye) <ref name=”10”> Worrel, J. (2016). Nvidia shows off its 'Light Field' VR headset at VRLA 2016. Retrieved from http://www.fudzilla.com/news/graphics/39762-nvidia-shows-off-its-light-field-vr-headset-at-vrla-2016</ref>.

==Main characteristics of near-eye light field displays==

The main benefits of the near-eye light field displays are reduced thickness and weight. In the 2013 prototype this was achieved by substituting a compact microlens array for compound magnifying optics. Adding to this, the device addresses the accommodation-convergence conflict with binocular configurations by approximating retinal defocus blur. It also accounts for the user’s glasses or contact prescription through software <ref name=”2”></ref> <ref name=”5”></ref>.

On the other hand, reduced spatial resolution was the primary limitation of this device. Lanman and Luebke (2013) indicate that the resolution is proportional to the ratio of the microlens focal length to the distance of the display from the eye, creating tension between compact form factors and higher resolutions. Furthermore, they wrote that “practical applications will necessitate manufacturing larger microdisplays with smaller pixel pitches, enabling wide ﬁelds of view and high resolutions, respectively.” <ref name=”2”></ref>

The near-eye light field stereoscope prototype of 2015 also addresses the accommodation-convergence conflict that leads to motion sickness, resulting in a more natural and flexible experience. In this case, the device uses two layers of LCD separated by approximately 5 millimeters. These send an entire “4D” light field of images so the eye can set focus between objects more naturally <ref name=”11”> Pirzada, Usman (2015). Nvidia Working On Next Generation Virtual Reality Device; The Near Eye Light Field Stereoscope, Aims To Bring It To The Market by 2018. Retrieved from http://wccftech.com/nvidia-virtual-reality-device-light-field-2018/</ref>.

The resolution was also a limitation in this prototype. While the screen resolution was 1280x800, the HMD splits the screen into two halves, making the effective resolution decrease to 640x800 per eye. This is too low for a current setting, but it could be improved in the future since this is still in the prototype stage. Indeed, NVIDIA intends to bring this device into the market by 2018. While NVIDIA could be developing the new standard for HMDs, it could also mean that by the time the device is released to the market conventional VR makers may have solved the problems related to motion sickness in their own devices <ref name=”11”></ref>.

==References==
<references/>

[[Category:Terms]]
[[Category:Technical Terms]]