Foveated rendering: Difference between revisions

| name = Foveated Rendering

| image = [[File:Foveated rendering illustration.svg|250px]]

| caption = Illustration of foveated rendering, showing high resolution in the foveal region and reduced resolution in the periphery

| type = [[Computer graphics]] technique

| industry = [[Virtual reality]] (VR), [[Augmented reality]] (AR)

| application = Performance optimization, human vision simulation

| hardware = [[Graphics processing unit|GPU]]s, [[Head-mounted display]]s with [[eye tracking]]

Implementations of foveated rendering are broadly categorized into two types: '''[[Fixed foveated rendering]]''' (FFR), which assumes the user is always looking at the center of the screen, and '''dynamic (or eye-tracked) foveated rendering''' (ETFR or DFR), which uses integrated [[eye tracking]] hardware to update the high-quality region in real-time to match the user's gaze.<ref name="MetaETFRvsFFR">{{cite web |url=https://developers.meta.com/horizon/blog/save-gpu-with-eye-tracked-foveated-rendering/ |title=Save GPU with Eye Tracked Foveated Rendering}}</ref>

As one moves away from the fovea into the '''[[peripheral vision]]''', the density of cone cells decreases rapidly, while the density of [[rod cell]]s, which are more sensitive to light and motion but not to color or fine detail, increases.<ref name="TOYF_Paper">{{cite web |url=https://research.manchester.ac.uk/files/296585058/toyf.pdf |title=Type of Movement and Attentional Task Affect the Efficacy of a Foveated Rendering Method in Virtual Reality}}</ref> This anatomical arrangement means that our ability to perceive detail, color, and stereoscopic depth diminishes significantly with increasing [[eccentricity]] (the angular distance from the point of gaze).<ref name="EyeTrackingVRReview">{{cite web |url=https://pmc.ncbi.nlm.nih.gov/articles/PMC10449001/ |title=Eye tracking in virtual reality: a comprehensive overview of the human visual system, eye movement types, and technical considerations}}</ref> However, our peripheral vision is highly attuned to detecting motion and flicker.<ref name="HVS_VR_Context_2">{{cite web |url=https://pmc.ncbi.nlm.nih.gov/articles/PMC10449001/ |title=Eye tracking in virtual reality: a comprehensive overview}}</ref>

The first is '''[[saccadic masking]]''' (also known as saccadic suppression), a mechanism where the brain selectively blocks visual processing during a saccade.<ref name="FoveatedRenderingExplainedReddit">{{cite web |url=https://www.reddit.com/r/oculus/comments/afj50w/eye_tracking_foveated_rendering_explained_what_it/ |title=Eye Tracking & Foveated Rendering Explained}}</ref> This prevents the perception of motion blur as the eyes sweep across the visual field, effectively creating a brief window of functional blindness. This period of suppressed sensitivity begins about 50 ms before a saccade and lasts until about 100 ms after it begins.<ref name="LatencyRequirements">{{cite web |url=https://research.nvidia.com/sites/default/files/pubs/2017-09_Latency-Requirements-for/a25-albert.pdf |title=Latency Requirements for Eye-Tracked Foveated Rendering}}</ref> This window is crucial for dynamic foveated rendering systems. If the system's total [[latency]]—from detecting the eye movement to displaying the updated foveated image—is shorter than the saccadic masking window, the transition of the high-resolution region can occur without the user ever perceiving the intermediate low-resolution state at their new point of gaze. This direct link between a biological phenomenon and a hardware specification means that low-latency eye tracking is not just a desirable feature but a hard requirement for effective ETFR.

The second phenomenon is '''[[visual attention]]'''. Research has shown that the HVS's capabilities are not static but are modulated by cognitive factors. When a user is concentrating on a visually demanding task at their point of gaze, their [[contrast sensitivity]] in the periphery drops significantly.<ref name="AttentionAware">{{cite web |url=https://www.computationalimaging.org/publications/attention-aware/ |title=Towards Attention-Aware Foveated Rendering}}</ref><ref name="AttentionAwareSIGGRAPH">{{cite web |url=https://history.siggraph.org/learning/towards-attention-aware-foveated-rendering-by-krajancich-kellnhofer-and-wetzstein/ |title=“Towards Attention–Aware Foveated Rendering” by Krajancich, Kellnhofer and Wetzstein}}</ref> This implies that current foveation models, which are based solely on the fixed acuity falloff of the retina, may be too conservative. Attention-aware foveation systems could potentially apply even more aggressive degradation in the periphery during periods of high foveal concentration, unlocking further performance gains without any perceptual loss.

# '''Display Update:''' The final, composited multi-resolution image is presented to the user. This entire loop must be completed within the frame budget (e.g., under 11.1 ms for a 90 Hz display) to ensure a smooth experience.

* '''Shading Rate Reduction:''' This method focuses on reducing the workload of the [[pixel shader]] (also known as a fragment shader). Instead of executing a complex shading program for every single pixel in the periphery, a single shader result can be applied to a block of multiple pixels. This is the core mechanism behind [[Variable rate shading|Variable Rate Shading (VRS)]].<ref name="TOYF_Paper" /><ref name="AutoVRSE">{{cite web |url=https://www.autovrse.com/foveated-rendering |title=What is Foveated Rendering? - autovrse}}</ref>

* '''Geometric Simplification:''' The geometric complexity of the scene can be reduced in the periphery. This involves using lower-polygon [[Level of detail|Level of Detail (LOD)]] models for objects that are outside the user's direct gaze.

* '''Other Methods:''' More advanced or experimental techniques include chromatic degradation (reducing color precision, since the periphery is less sensitive to color), simplifying lighting and shadow calculations, and spatio-temporal deterioration, which involves reducing quality across both space and time.

{| class="wikitable"

| Multi-spatial Resolution Sampling || High-res fovea, low-res periphery || 2–4× || Guenter et al. (2012)<ref name="Guenter2012">{{Cite conference |last=Guenter |first=Brian |last2=Grimes |first2=Mark |last3=Nehab |first3=Diego |last4=Sander |first4=Pedro V. |last5=Summa |first5=João |title=Efficient rerendering in viewport space |journal=ACM Transactions on Graphics |volume=31 |issue=6 |pages=1–13 |year=2012 |doi=10.1145/2366145.2366195 |description=Key 2012 foveated graphics paper.}}</ref>

| Neural Reconstruction || AI-based upsampling || 5–8× || NVIDIA DeepFovea<ref name="DeepFovea">{{Cite conference |last=Kaplanyan |first=Anton |et al. |title=DeepFovea: Neural reconstruction for foveated rendering |journal=ACM Transactions on Graphics |volume=38 |issue=6 |pages=1–15 |year=2019 |doi=10.1145/3355089.3356559 |description=Neural upsampling for foveation.}}</ref>

[[Variable rate shading|Variable Rate Shading]] (VRS) is a hardware feature available on modern GPUs (e.g., Nvidia Turing architecture and newer, AMD RDNA 2 and newer) that provides fine-grained control over the pixel shading rate.<ref name="TOYF_Paper" /><ref name="OpenXRToolkit">{{cite web |url=https://mbucchia.github.io/OpenXR-Toolkit/fr.html |title=Foveated Rendering - OpenXR Toolkit}}</ref> It allows a single pixel shader operation to compute the color for a block of pixels, such as a 2x2 or 4x4 block, instead of just a single pixel.<ref name="VarjoAPI">{{cite web |url=https://developer.varjo.com/docs/native/foveated-rendering-api |title=Foveated Rendering - Varjo for Developers}}</ref><ref name="PicoUnrealOpenXR">{{cite web |url=https://developer.picoxr.com/document/unreal-openxr/fixed-foveated-rendering/ |title=Foveated rendering - PICO Unreal OpenXR Plugin}}</ref> For foveated rendering, the GPU can be instructed to use a 1x1 shading rate (full quality) for the foveal region and progressively coarser rates (e.g., 2x2, 4x4, or even culling) for the periphery. This is an extremely efficient, GPU-centric way to reduce shading load with minimal CPU overhead.

An alternative approach, notably used by [[Varjo]] and available in [[Unreal Engine]], is to render multiple distinct views for each eye.<ref name="VarjoAPI" /><ref name="DCS_Forum_QuadViews">{{cite web |url=https://www.reddit.com/r/hoggit/comments/15ep59q/dcs_dynamic_foveated_rendering_available_for_more/ |title=DCS Dynamic Foveated Rendering available for more headsets}}</ref> For example, a "Quad Views" implementation renders four views in total for a stereo image: a high-resolution central "focus" view for each eye, and a lower-resolution peripheral "context" view for each eye. These are then composited into the final image.<ref name="PimaxQuadViews">{{cite web |url=https://pimax.com/blogs/blogs/quad-views-foveated-rendering-for-pimax-crystal |title=Quad Views Foveated Rendering for Pimax Crystal}}</ref> This technique can dramatically reduce the total number of pixels that need to be shaded, often yielding greater performance gains than VRS, especially in fill-rate limited scenarios. However, it comes at the cost of increased CPU overhead, as the scene's geometry must be submitted and processed multiple times (once for each view).<ref name="DCS_Forum_QuadViews" /> The choice between VRS and a multi-view approach thus represents a fundamental trade-off. A developer must consider whether their application is primarily bottlenecked by the GPU's pixel shading capabilities (favoring VRS) or by its overall pixel throughput, and whether they have sufficient CPU headroom to manage the multiple rendering passes required by a multi-view system.

A related concept, known as foveated transport or foveated streaming, applies the same principles to bandwidth optimization rather than real-time rendering. This is primarily used in cloud-streamed VR or 360° video playback.<ref name="FOVAS_Pixvana">{{cite web |title=Pixvana’s FOVAS Technology Delivers 8K VR Video on Today’s Headsets |url=https://www.businesswire.com/news/home/20160907005500/en/Pixvana’s-FOVAS-Technology-Delivers-8K-VR-Video-on-Today’s-Headsets |publisher=Business Wire |date=2016-09-07 |access-date=2025-10-26}}</ref> In this model, the server sends a high-resolution, high-bitrate video stream only for the small portion of the scene the user is currently looking at, while sending a much lower-quality stream for the periphery. This drastically reduces the bandwidth required to achieve a high-quality experience. Technologies like [[FOVAS]] (Field of View Adaptive Streaming) pioneered this technique.<ref name="FOVAS_Pixvana" />

* '''Visible Artifacts:''' If a user moves their eyes to look at the periphery without turning their head, they can easily notice the drop in resolution, which can be distracting and break immersion.<ref name="MetaFFR_OS" /><ref name="UnityOpenXR_FR">{{cite web |url=https://docs.unity3d.com/Packages/[email protected]/manual/features/foveatedrendering.html |title=Foveated rendering in OpenXR - Unity Manual}}</ref>

Dynamic Foveated Rendering represents the full realization of the concept. It requires a [[head-mounted display|HMD]] with integrated eye-tracking cameras to determine the user's precise point of gaze in real-time.<ref name="VarjoWhatIs" /><ref name="WikipediaFR" /> The high-resolution foveal region is then dynamically moved to match this gaze point on a frame-by-frame basis, ensuring that the user is always looking at a fully rendered part of the scene.<ref name="TobiiDFR">{{cite web |url=https://www.tobii.com/resource-center/reports-and-papers/eye-tracking-and-dynamic-foveated-rendering |title=Eye tracking and dynamic foveated rendering - Tobii}}</ref>

* '''Maximum Performance:''' ETFR allows for much more aggressive foveation—a smaller foveal region and a more significant quality reduction in the periphery—because the low-quality areas are always guaranteed to be outside the user's direct line of sight. This results in substantially greater performance and power savings.<ref name="VRX_FR_Types" /><ref name="PicoUnrealOpenXR" />

* '''Perceptually Seamless:''' When implemented with low latency, the effect is imperceptible to the user. The entire virtual world appears to be rendered in high resolution, as the degradation is always hidden in the natural blur of peripheral vision.<ref name="VarjoWhatIs" />

* '''Sensitivity to Latency:''' The technique is highly sensitive to system latency. If the delay between an eye movement and the corresponding display update is too long, the user will perceive artifacts, which negates the primary benefit.<ref name="LatencyRequirements" />

! Feature

! Fixed Foveated Rendering (FFR)

! Dynamic (Eye-Tracked) Foveated Rendering (ETFR/DFR)

 

 

| Assumes user's gaze is fixed at the center of the display.

| Tracks the user's real-time gaze to place the high-quality region precisely where they are looking.

 

| None (beyond a capable [[GPU]]).

| Integrated [[eye tracking]] cameras and processing hardware.

 

| Static, located in the center of the screen. Must be conservatively large.

| Dynamic, moves with the user's fovea. Can be made smaller and more aggressive.

 

| Generally effective, but artifacts can be noticed if the user looks to the periphery without moving their head.<ref name="MetaFFR_OS" /><ref name="UnityOpenXR_FR" />

| When latency is low, the effect is imperceptible to the user, providing a consistently high-quality experience.<ref name="VarjoWhatIs" />

 

| Moderate. Reduces GPU load by rendering fewer pixels in the periphery (e.g., 26-43% savings reported for Meta Quest).<ref name="QuestProPerformance">{{cite web |url=https://www.uploadvr.com/quest-pro-foveated-rendering-performance/ |title=Quest Pro Foveated Rendering GPU Savings Detailed}}</ref>

| Significant. Allows for more aggressive degradation, leading to greater GPU savings (e.g., 33-52% savings reported for Meta Quest Pro).<ref name="VRX_FR_Types" /><ref name="QuestProPerformance" />

 

| Cost-sensitive standalone headsets, applications without eye-tracking support, platforms where simplicity is prioritized.<ref name="JigSpace" />

| High-end PC VR and standalone headsets, demanding simulations, applications seeking maximum visual fidelity and performance.<ref name="JigSpace" />

 

| Sub-optimal performance gains and potentially visible artifacts.

| Increased hardware cost, complexity, power consumption, and high sensitivity to system latency.

* '''Predictive Foveation:''' Some systems attempt to predict the landing point of a saccade based on its initial trajectory and velocity. This allows the rendering system to begin shifting the foveal region to the target destination ''before'' the eye movement is complete, effectively hiding some of the system's latency and making artifacts less likely.<ref name="FoveatedRenderingExplainedReddit" /><ref name="VRX_FR_Types" />

* '''Attention-Aware Foveation:''' This is a cutting-edge research area that aims to model the user's cognitive state of attention. As noted earlier, peripheral visual sensitivity decreases when foveal attention is high. An attention-aware system could leverage this by applying even more aggressive foveation during tasks that require intense focus, further optimizing performance in a way that is tailored to the user's cognitive load, not just their retinal physiology.<ref name="AttentionAware" /><ref name="AttentionAwareSIGGRAPH" />

* '''GPU Frame Time:''' The most direct measurement of performance. This is the time, in milliseconds (ms), that the GPU takes to render a single frame. Foveated rendering directly reduces this value, and a lower frame time is always better.<ref name="PSVR2_GDC_Unity">{{cite web |url=https://www.playstationlifestyle.net/2022/03/28/psvr-2-specs-eye-tracking-foveated-rendering/ |title=PSVR 2 Specs Run 3.6x Faster Using Eye-Tracking Technology}}</ref>

* '''Frames Per Second (FPS):''' Lower frame times enable higher and more stable frame rates. Maintaining a high FPS (typically 90 FPS or more) is critical for a comfortable and immersive VR experience, as dropped frames or low FPS can induce [[virtual reality sickness]].<ref name="PimaxDFR">{{cite web |url=httpshttps://pimax.com/blogs/blogs/the-crystal-supers-secret-weapon-dynamic-foveated-rendering |title=The Crystal Super's Secret Weapon: Dynamic Foveated Rendering}}</ref>

* '''Increased Visual Fidelity:''' Instead of simply increasing FPS, developers can reinvest the saved GPU performance. This "performance headroom" can be used to render the scene at a higher base resolution ([[supersampling]]), enable more complex lighting and shader effects, or use higher-quality assets, resulting in a visually richer experience at the same target frame rate.<ref name="AutoVRSE" /><ref name="MetaETFR_Blog">{{cite web |url=https://developers.meta.com/horizon/blog/save-gpu-with-eye-tracked-foveated-rendering/ |title=Save GPU with Eye Tracked Foveated Rendering}}</ref>

|+ FFR Performance on Meta Quest Devices<ref name="QuestProPerformance" />

| [[Meta Quest 2]] || Level 1 || 4× (2×2) || 26%

| Meta Quest 2 || Level 3 || 16× (4×4) || 36%

| [[Meta Quest 3]] || Level 1 || 4× (2×2) || 26%

| Meta Quest 3 || Level 3 || 16× (4×4) || 43%

|+ ETFR Performance Comparison (GPU Savings)<ref name="QuestProPerformance" /><ref name="UploadVR_PSVR2">{{cite web |url=https://www.uploadvr.com/psvr-2-eye-tracking-foveated-rendering-gdc/ |title=PSVR 2 Foveated Rendering Provides 3.6x Faster Performance - Unity}}</ref><ref name="VarjoDevs">{{cite web |title=Foveated Rendering |url=https://developer.varjo.com/docs/native/foveated-rendering-api |publisher=Varjo for developers |year=2023}}</ref>

| [[Meta Quest Pro]] || 26-43% || 33-52% || 7-9%

| [[PlayStation VR2]] || ~60% (2.5×) || ~72% (3.6×) || ~12%

| [[Varjo Aero]] || 30-40% || 50-60% || 10-20%

* '''[[Meta Quest Pro]] (ETFR):''' In performance tests conducted by Meta, ETFR demonstrated significant savings over both non-foveated rendering and FFR. At 1.5 times the default resolution, FFR provided a performance saving of 34-43%, while ETFR delivered a saving of 36-52%.<ref name="QuestProPerformance" /><ref name="80lvQuestPro">{{cite web |url=https://80.lv/articles/quest-pro-s-foveated-rendering-saves-up-to-52-performance |title=Quest Pro's Foveated Rendering Saves up to 52% Performance}}</ref> The developers of the game ''Red Matter 2'' utilized ETFR to increase the rendered pixel density by 33%, which equates to 77% more total pixels in the optical center, leading to a much sharper image.<ref name="MetaETFR_Blog" />

=== Factors Influencing Efficacy ===

The performance gains from foveated rendering are not linear. The technique primarily alleviates bottlenecks related to pixel shading and fill rate. Once this bottleneck is removed, the application's performance will become limited by another part of the system, such as CPU performance (e.g., draw calls, physics simulations) or memory bandwidth. At that point, applying more aggressive foveation will not improve frame rates further and will only serve to degrade visual quality. This reality has led to the development of "dynamic foveation" features in some SDKs, which automatically adjust the foveation level based on the current GPU load to maintain a target frame rate, thus finding the optimal balance between performance and quality on-the-fly.<ref name="MetaQuestProUnityETFR">{{cite web |url=https://developers.meta.com/horizon/documentation/unity/unity-eye-tracked-foveated-rendering/ |title=Eye Tracked Foveated Rendering - Unity}}</ref><ref name="MetaUnrealETFR">{{cite web |url=https://developers.meta.com/horizon/documentation/unreal/unreal-eye-tracked-foveated-rendering/ |title=Eye Tracked Foveated Rendering - Unreal}}</ref>

== Applications and Use Cases ==

While foveated rendering is most prominent in VR/AR gaming and simulation to achieve higher frame rates, its principles are applied in other domains:

* '''360° Video Streaming:''' As noted in the Foveated Transport section, foveated encoding can reduce bandwidth requirements for streaming high-resolution (e.g., 8K) 360° video by up to 80%.<ref name="Foveated360Video">{{Cite conference |last=Li |first=Jian |et al. |title=Foveated video coding for cloud-based VR streaming |journal=2021 IEEE International Conference on Multimedia and Expo |pages=1–6 |year=2021 |doi=10.1109/ICME51207.2021.9428350 |description=Foveated 360° video.}}</ref>

* '''Medical Visualization:''' In high-fidelity medical imaging, foveated rendering allows for real-time interaction with massive volume datasets (e.g., from CT or MRI scans) for surgical planning and simulation.<ref name="MedicalVolumeRender">{{Cite conference |last=Gallo |first=Luigi |et al. |title=Foveation for 3D volume rendering |journal=2013 IEEE 15th International Conference on e-Health Networking, Applications and Services |pages=1–6 |year=2013 |doi=10.1109/HealthCom.2013.6720650 |description=Medical volume foveation.}}</ref>

* '''Holographic and Light Field Displays:''' These next-generation displays require enormous computational power to calculate light propagation. Foveated rendering is a critical area of research for making these displays computationally feasible.<ref name="HolographicFoveation">{{Cite conference |last=Chakravarthula |first=Praneeth |et al. |title=To the holograms born |journal=ACM Transactions on Graphics |volume=40 |issue=6 |pages=1–16 |year=2021 |doi=10.1145/3486108 |description=Holographic foveation.}}</ref>

== Ecosystem Integration ==

For foveated rendering to be widely adopted, it must be accessible to developers through common game engines and standardized APIs. The ecosystem has matured from requiring bespoke, hardware-specific implementations to being a more integrated feature of modern development platforms.

==== [[Unity (game engine)|Unity]] ====

Unity provides native support for foveated rendering on supported XR platforms through its [[Universal Render Pipeline|Scriptable Render Pipeline]]s (URP and HDRP).<ref name="UnityFR_Roadmap">{{cite web |url=https://unity.com/roadmap/1356-xr-foveated-rendering |title=XR Foveated Rendering - Unity Roadmap}}</ref><ref name="UnityDocsFR">{{cite web |url=https://docs.unity3d.com/6000.2/Documentation/Manual/xr-foveated-rendering.html |title=Foveated rendering - Unity Manual}}</ref> Developers can enable the feature within the '''XR Plug-in Management''' project settings. At runtime, the strength of the foveation effect is controlled by setting the <code>XRDisplaySubsystem.foveatedRenderingLevel</code> property to a value between 0 (off) and 1 (maximum). To enable gaze-based foveation on supported hardware, the <code>foveatedRenderingFlags</code> property must be set to allow gaze input.<ref name="UnityOpenXR_FR" /><ref name="UnityDocsFR" />

In addition to this core support, platform-specific SDKs often provide their own wrappers and APIs. For example, the Meta XR SDK and the PICO Unity Integration SDK expose dedicated components and functions for enabling and configuring both FFR and ETFR, offering different levels of quality and performance.<ref name="PicoUnityFFR" /><ref name="MetaUnityFFR">{{cite web |url=https://developers.meta.com/horizon/documentation/unity/unity-fixed-foveated-rendering/ |title=Using Fixed Foveated Rendering - Unity}}</ref><ref name="PicoUnityETFR">{{cite web |url=https://developer.picoxr.com/document/unity/eye-tracked-foveated-rendering/ |title=Eye tracked foveated rendering - PICO Unity Integration SDK}}</ref>

Support for foveated rendering in Unreal Engine is also robust, though often managed through platform-specific plugins like the Meta XR Plugin or the PICO Unreal OpenXR Plugin.<ref name="ViveUnrealFFR">{{cite web |url=https://developer.vive.com/resources/openxr/unreal/unreal-tutorials/rendering/foveated-rendering/ |title=Foveated Rendering - VIVE OpenXR Unreal}}</ref><ref name="PicoUnrealOpenXR" /> Configuration is typically handled through a combination of '''Project Settings''' (under the Plugins section for the specific XR platform) and console variables.<ref name="MetaUnrealETFR" /><ref name="PicoUnrealOpenXR" /> For example, developers can set the foveation level using a console command like <code>xr.OpenXRFBFoveationLevel=2</code> for medium foveation. These settings can also be controlled dynamically at runtime via [[Blueprints (visual scripting)|Blueprints]] or C++.<ref name="PicoUnrealOpenXR" /> On mobile platforms, ETFR support is often a [[Vulkan (API)|Vulkan]]-only feature and may require using a specific version or fork of the engine to ensure compatibility.<ref name="MetaUnrealETFR" /><ref name="PicoUnrealOpenXR" />

The Vulkan API, in particular, offers powerful and flexible extensions for foveation. The <code>VK_EXT_fragment_density_map</code> extension allows the VR runtime to provide the GPU with a custom texture that dictates the rendering resolution across the framebuffer, enabling highly tailored foveation patterns.<ref name="VulkanMobileVR" /><ref name="VulkanFDMExtension" /> This approach is a cornerstone of foveation implementation on modern Android-based standalone headsets.

[[OpenXR]] is a royalty-free, open standard from the [[Khronos Group]] that provides high-performance access to AR and VR platforms and devices. It plays a crucial role in standardizing foveated rendering for developers. By defining a common set of extensions, OpenXR allows game engines and applications to implement foveation in a vendor-agnostic way. Key extensions include:

These extensions allow an application to query for foveation support, configure its parameters (like quality levels), and enable or disable it at runtime.<ref name="PicoUnrealOpenXR" /><ref name="ViveUnrealFFR" /> This level of abstraction is a sign of a maturing technology; it moves developers away from needing to write hardware-specific code and toward a "write once, run anywhere" paradigm. This trend, where low-level hardware features are abstracted first by graphics APIs, then by standards like OpenXR, and finally by game engines, is critical for lowering the barrier to entry and encouraging widespread adoption of foveated rendering.

 

Furthermore, tools like the '''OpenXR Toolkit''' demonstrate the power of this layered approach by acting as an API layer that can inject foveated rendering capabilities into OpenXR applications that may not natively support it, provided the user's hardware is capable.<ref name="OpenXRToolkit" /><ref name="DCS_Forum_QuadViews" />

== Hardware Support and Implementation ==

 

=== Consumer Headsets ===

 

* '''[[Apple Vision Pro]]:''' Apple's "spatial computer," released in 2024, also features a sophisticated eye-tracking system as a primary input method. It leverages this for foveated rendering to help drive its very high-resolution displays with the onboard Apple M2 chip.<ref name="PCMagBestVR">{{cite web |url=https.www.pcmag.com/picks/the-best-vr-headsets |title=The Best VR Headsets for 2025}}</ref>

! Headset

! Release Year

! Display Resolution (per eye)

! Eye Tracking

! Eye Tracker Specs

! Foveated Rendering Support

 

 

| 2020

| 1832 x 1920

| No

| N/A

| Fixed Foveated Rendering (FFR) only<ref name="VRX_FR_Types" />

 

| 2022

| 1800 x 1920

| Yes

| Internal cameras, gaze prediction<ref name="MetaETFR_Blog" />

| Eye-Tracked Foveated Rendering (ETFR) & FFR<ref name="WikipediaFR" /><ref name="MetaQuestProUnityETFR" />

 

| 2023

| 2000 x 2040

| Yes

| 1x IR camera per eye<ref name="PSVR2_TechSpecs" />

| Eye-Tracked Foveated Rendering (ETFR)<ref name="WikipediaFR" /><ref name="PSVR2_GDC_Unity" />

! [[Varjo Aero]]

 

| 2021

| 2880 x 2720

| Yes

| 200Hz, sub-degree accuracy<ref name="VarjoAeroSpecs" /><ref name="VarjoEyeTrackingSpecs" />

| Dynamic Foveated Rendering (Dynamic Projection & VRS)<ref name="VarjoAPI" /><ref name="VarjoFoveationPage">{{cite web |url=https://support.varjo.com/hc/en-us/foveated-rendering |title=Foveated rendering - Varjo Support}}</ref>

! [[Pimax Crystal]]

 

| 2023

| 2880 x 2880

| 120Hz, Tobii-powered<ref name="PimaxDFR" /><ref name="PimaxDFR_About">{{cite web |url=httpshttps://pimax.com/blogs/blogs/about-dynamic-foveated-rendering-dfr-in-virtual-reality-vr |title=About Dynamic Foveated Rendering (DFR) in Virtual Reality (VR)}}</ref>

| Dynamic Foveated Rendering (VRS & Quad Views)<ref name="PimaxDFR" /><ref name="PimaxQuadViews" />

| 2022

| Eye-Tracked Foveated Rendering (ETFR) & FFR<ref name="PicoUnityETFR" /><ref name="PicoUnrealLegacy">{{cite web |url=https://developer.picoxr.com/document/unreal/fixed-foveated-rendering/ |title=Foveated rendering - PICO Unreal Integration SDK}}</ref>

| No (Add-on available)

| ~3660 x 3200 (est.)

| High-speed cameras and IR illuminators

| Eye-Tracked Foveated Rendering<ref name="PCMagBestVR" />

* '''Latency:''' As previously established, high end-to-end latency is the primary antagonist of foveated rendering. If the system cannot update the foveal region before the user's saccade completes and saccadic masking wears off, the user will perceive a blurry or low-resolution image at their new point of focus. This is a jarring artifact known as "pop-in."<ref name="UnityFR_Test_Reddit" /> Research indicates that while latencies of 80-150 ms cause significant issues, a total system latency of 50-70 ms can be tolerated without a major impact on the acceptable level of foveation.<ref name="LatencyRequirements" /> Meta reports the end-to-end latency on the Quest Pro to be in the 46-57 ms range, which is within this acceptable threshold.<ref name="MetaETFR_Blog" />

* '''Accuracy and Jitter:''' The tracking system must be accurate enough to place the foveal region correctly. Inaccuracies can lead to the user's actual fovea landing on a medium- or low-resolution part of the image. Furthermore, the tracking data must be stable. "Jitter," or small, rapid fluctuations in the reported gaze position, can cause the high-resolution area to shimmer or vibrate, which is highly distracting.

To combat these issues, developers employ several mitigation strategies. Creating a smooth "blend" region between the high- and medium-quality zones, rather than a sharp cutoff, can make the transition less obvious.<ref name="IntegrativeView" /> Some advanced renderers apply a contrast enhancement pass to the periphery to counteract the contrast loss caused by blurring, which helps restore the apparent detail and reduce the sense of tunnel vision.<ref name="SIGA16_GazeTracked" /> Finally, using sophisticated [[anti-aliasing]] algorithms that are aware of the multi-resolution nature of the image can help to stabilize the periphery and reduce flicker.<ref name="SIGA16_GazeTracked" />

== History and Future Directions ==

Foveated rendering has evolved from a theoretical concept in academic papers to a cornerstone technology for modern VR, with ongoing research continuing to push its boundaries.

Research into gaze-contingent rendering can be traced back to at least 1990, when scientists began proposing that rendering computations could be reduced by taking the human visual acuity falloff into account.<ref name="Levoy1990">{{Cite journal |last=Levoy |first=Marc |last2=Whitaker |first2=Robert |title=Gaze-directed volume rendering |journal=Proceedings of the 1990 Symposium on Interactive 3D Graphics |pages=361–369 |year=1990 |doi=10.1145/91394.91431 |description=Foundational gaze-directed rendering paper.}}</ref> However, early experiments were limited by the available hardware. The path to commercialization began in earnest with the modern resurgence of VR.

* '''2014:''' The startup '''[[FOVE]]''' unveiled a prototype HMD at TechCrunch Disrupt, bringing the concept of an eye-tracked VR headset with foveated rendering into the public eye.<ref name="FOVE_TechCrunch">{{Cite web |title=FOVE Uses Eye Tracking To Make Virtual Reality More Immersive |url=https://techcrunch.com/2014/09/09/fove/ |publisher=TechCrunch |date=2014-09-10 |access-date=2025-10-26}}</ref> The company launched a successful Kickstarter in 2015, and the FOVE 0 headset shipped to developers in late 2016, becoming the first commercially available HMD with integrated eye tracking.<ref name="FOVE_Kickstarter">{{Cite web |title=FOVE: The World's First Eye Tracking Virtual Reality Headset |url=https://www.kickstarter.com/projects/fove/fove-the-worlds-first-eye-tracking-virtual-reality |publisher=Kickstarter |date=2015-09-01 |access-date=2025-10-26}}</ref>

* '''2016:''' At CES, '''SensoMotoric Instruments (SMI)''' demonstrated a 250 Hz eye-tracking system retrofitted into an Oculus Rift, showcasing a low-latency foveated rendering solution that observers found imperceptible.<ref name="SMI_UploadVR">{{Cite web |title=SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real |url=https://uploadvr.com/smi-hands-on-250hz-eye-tracking/ |publisher=UploadVR |date=2016-01-15 |access-date=2025-10-26}}</ref> Later that year at [[SIGGRAPH]], '''[[NVIDIA]]''' demonstrated a new, perceptually-based method claimed to be invisible to users, reporting a 50-66% reduction in pixel shading load.<ref name="Nvidia_Blog_SMI">{{Cite web |title=NVIDIA Partners with SMI on Innovative Rendering Technique That Improves VR |url=https://blogs.nvidia.com/blog/2016/07/21/rendering-foveated-vr/ |publisher=NVIDIA |date=2016-07-21 |access-date=2025-10-26}}</ref>

* '''2017:''' '''[[Qualcomm]]''' announced its Snapdragon 835 VRDK, which included support for a foveation technique called "Adreno Foveation," signaling the technology's arrival on mobile processors.<ref name="Qualcomm835">{{Cite web |title=Qualcomm Introduces Snapdragon 835 Virtual Reality Development Kit |url=https://www.qualcomm.com/news/releases/2017/02/23/qualcomm-introduces-snapdragon-835-virtual-reality-development-kit |publisher=Qualcomm |date=2017-02-23 |access-date=2025-10-26}}</ref>

* '''2019:''' Eye-tracking began to enter mainstream devices. The '''[[HTC Vive Pro Eye]]''' was announced at CES 2019 as a commercial headset with native eye tracking.<ref name="ViveProEye_Verge">{{Cite web |title=HTC announces new Vive Pro Eye virtual reality headset with native eye tracking |url=https://www.theverge.com/2019/1/7/18172700/htc-vive-pro-eye-tracking-virtual-reality-headset-features-pricing-release-date-ces-2019 |publisher=The Verge |date=2019-01-07 |access-date=2025-10-26}}</ref> Later that year, Meta (then Facebook) provided developers with access to Fixed Foveated Rendering in the Oculus Quest SDK, marking its first large-scale commercial deployment.<ref name="QuestFFR_VentureBeat">{{Cite web |title=Oculus Quest gets dynamic fixed foveated rendering |url=https://venturebeat.com/2019/12/22/oculus-quest-gets-dynamic-fixed-foveated-rendering/ |publisher=VentureBeat |date=2019-12-22 |access-date=2025-10-26}}</ref>

* '''2022-2023:''' The launch of the '''[[Meta Quest Pro]]''' (2022) and '''[[PlayStation VR2]]''' (2023) established Eye-Tracked Foveated Rendering as a flagship feature in the consumer VR market, cementing its role as a critical enabling technology for next-generation immersive experiences.<ref name="PSVR2_Blog">{{Cite web |title=PlayStation VR2 and PlayStation VR2 Sense controller: The next generation of VR gaming on PS5 |url=https://blog.playstation.com/2022/01/04/playstation-vr2-and-playstation-vr2-sense-controller-the-next-generation-of-vr-gaming-on-ps5/ |publisher=PlayStation Blog |date=2022-01-04 |access-date=2025-10-26}}</ref>

* '''2024:''' The release of the '''[[Apple Vision Pro]]''' further solidified the technology's importance, incorporating high-speed eye tracking as a core component for both rendering optimization and user interaction.<ref name="VisionPro_Dev">{{Cite web |title=Discover visionOS |url=https://developer.apple.com/visionos/ |publisher=Apple Developer |date=2023-06-07 |access-date=2025-10-26}}</ref>

A cutting-edge frontier in foveation involves [[neural rendering]], which uses [[artificial intelligence]] to reconstruct a high-quality image from sparsely rendered data.

* '''[[DeepFovea]]:''' Developed by Facebook Reality Labs and presented at SIGGRAPH Asia 2019, DeepFovea pioneered neural reconstruction for foveated rendering. The system renders only 10% of peripheral pixels and then uses a [[convolutional neural network]] to "fill in" the missing information, enabling a potential 10-14× reduction in pixel shading.<ref name="DeepFovea" /><ref name="DeepFovea_Paper">{{cite web |title=DeepFovea: Neural Reconstruction for Foveated Rendering |url=https://dl.acm.org/doi/10.1145/3306307.3328186 |publisher=ACM SIGGRAPH |year=2019}}</ref>

* '''Recent Advances:''' Research has continued to build on this concept. FoVolNet (2022) achieved a 25× speedup over DeepFovea for volume rendering.<ref name="fovolnet2022">{{cite web |title=FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks |url=https://arxiv.org/abs/2209.09965 |publisher=arXiv |year=2022}}</ref> VR-Splatting (2024) combines [[3D Gaussian Splatting]] with foveated rendering to achieve photorealistic VR at 90Hz.<ref name="vrsplatting2024">{{cite web |title=VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points |url=https://dl.acm.org/doi/10.1145/3728302 |publisher=ACM |year=2024}}</ref>

=== Future Developments ===

The progression of foveated rendering represents a paradigm shift in computer graphics, moving away from a brute-force approach toward an intelligent, perceptually-driven allocation of resources.

* '''Near-term (2025-2026):''' Increased adoption of neural reconstruction techniques in consumer devices, potentially accelerated by on-chip [[NPU]]s. Standardization of foveation APIs via OpenXR will eliminate platform fragmentation.

* '''Mid-term (2026-2028):''' Foveated rendering will become mandatory for "retinal resolution" displays (60-70 pixels per degree) to be computationally feasible. Research will focus on adaptive foveation that personalizes the quality falloff curve for individual users.

* '''Long-term (2028+):''' The technology will likely be fully integrated with [[Neural Radiance Fields]] (NeRF) and cloud streaming (foveated transport) to deliver photorealistic, real-time graphics. This could also extend to other senses, such as multi-modal foveation for spatial audio and haptics.

* [[3D Gaussian Splatting]]

* [[Fixed foveated rendering]]

* [[Human visual system]]

* [[Level of detail]]

<ref name="StateOfArtSurvey">{{cite web |url=https://www.researchgate.net/publication/366842988_Foveated_rendering_A_state_of_the_art_survey |title=Foveated rendering: A state-of-the-art survey}}</ref>

<ref name="AttentionAwareSIGGRAPH">{{cite web |url=https://history.siggraph.org/learning/towards-attention-aware-foveated-rendering-by-krajancich-kellnhofer-and-wetzstein/ |title=“Towards Attention–Aware Foveated Rendering” by Krajancich, Kellnhofer and Wetzstein}}</ref>

<ref name="FOVAS_Pixvana">{{cite web |title=Pixvana’s FOVAS Technology Delivers 8K VR Video on Today’s Headsets |url=https://www.businesswire.com/news/home/20160907005500/en/Pixvana’s-FOVAS-Technology-Delivers-8K-VR-Video-on-Today’s-Headsets |publisher=Business Wire |date=2016-09-07 |access-date=2025-10-26}}</ref>

<ref name="PimaxDFR">{{cite web |url=httpshttps://pimax.com/blogs/blogs/the-crystal-supers-secret-weapon-dynamic-foveated-rendering |title=The Crystal Super's Secret Weapon: Dynamic Foveated Rendering}}</ref>

<ref name="UploadVR_PSVR2">{{cite web |url=https://www.uploadvr.com/psvr-2-eye-tracking-foveated-rendering-gdc/ |title=PSVR 2 Foveated Rendering Provides 3.6x Faster Performance - Unity}}</ref>

<ref name="VarjoDevs">{{cite web |title=Foveated Rendering |url=https://developer.varjo.com/docs/native/foveated-rendering-api |publisher=Varjo for developers |year=2023}}</ref>

<ref name="80lvQuestPro">{{cite web |url=https://80.lv/articles/quest-pro-s-foveated-rendering-saves-up-to-52-performance |title=Quest Pro's Foveated Rendering Saves up to 52% Performance}}</ref>

<ref name="UnityFR_Test_Reddit">{{cite web |url=https://www.reddit.com/r/oculus/comments/3bls3q/unity_foveated_rendering_test_4x_fps_increase/ |title=unity foveated rendering test 4x fps increase with a pretty simple rendering strategy}}</ref>

<ref name="PicoUnityFFR">{{cite web |url=https://developer.picoxr.com/document/unity/fixed-foveated-rendering/ |title=Fixed foveated rendering - PICO Unity Integration SDK}}</ref>

<ref name="MetaQuestProUnityETFR">{{cite web |url=https://developers.meta.com/horizon/documentation/unity/unity-eye-tracked-foveated-rendering/ |title=Eye Tracked Foveated Rendering - Unity}}</ref>

<ref name="MetaUnrealETFR">{{cite web |url=https://developers.meta.com/horizon/documentation/unreal/unreal-eye-tracked-foveated-rendering/ |title=Eye Tracked Foveated Rendering - Unreal}}</ref>

<ref name="Foveated360Video">{{Cite conference |last=Li |first=Jian |et al. |title=Foveated video coding for cloud-based VR streaming |journal=2021 IEEE International Conference on Multimedia and Expo |pages=1–6 |year=2021 |doi=10.1109/ICME51207.2021.9428350 |description=Foveated 360° video.}}</ref>

<ref name="MedicalVolumeRender">{{Cite conference |last=Gallo |first=Luigi |et al. |title=Foveation for 3D volume rendering |journal=2013 IEEE 15th International Conference on e-Health Networking, Applications and Services |pages=1–6 |year=2013 |doi=10.1109/HealthCom.2013.6720650 |description=Medical volume foveation.}}</ref>

<ref name="HolographicFoveation">{{Cite conference |last=Chakravarthula |first=Praneeth |et al. |title=To the holograms born |journal=ACM Transactions on Graphics |volume=40 |issue=6 |pages=1–16 |year=2021 |doi=10.1145/3486108 |description=Holographic foveation.}}</ref>

<ref name="UnrealVRS_FFR">{{cite web |url=https.dev.epicgames.com/documentation/en-us/unreal-engine/vr-performance-features?application_version=4.27 |title=VR Performance Features - Unreal Engine 4.27 Documentation}}</ref>

<ref name="PSVR2_TechSpecs">{{cite web |url=https.www.playstation.com/en-ca/ps-vr2/ps-vr2-tech-specs/ |title=PlayStation VR2 tech specs | PlayStation}}</ref>

<ref name="PCMagBestVR">{{cite web |url=https.www.pcmag.com/picks/the-best-vr-headsets |title=The Best VR Headsets for 2025}}</ref>

<ref name="PimaxDFR_About">{{cite web |url=httpshttps://pimax.com/blogs/blogs/about-dynamic-foveated-rendering-dfr-in-virtual-reality-vr |title=About Dynamic Foveated Rendering (DFR) in Virtual Reality (VR)}}</ref>

<ref name="PicoUnrealLegacy">{{cite web |url=https://developer.picoxr.com/document/unreal/fixed-foveated-rendering/ |title=Foveated rendering - PICO Unreal Integration SDK}}</ref>

<ref name="PerceptuallyBasedFR_Nvidia">{{cite web |url=https::research.nvidia.com/publication/2016-07_perceptually-based-foveated-virtual-reality |title=Perceptually-Based Foveated Virtual Reality | Research}}</ref>

<ref name="QualcommSpacesUnityFR">{{cite web |url=httpss://docs.spaces.qualcomm.com/unity/setup/foveated-rendering |title=Foveated Rendering - Snapdragon Spaces}}</ref>

<ref name="Levoy1990">{{Cite journal |last=Levoy |first=Marc |last2=Whitaker |first2=Robert |title=Gaze-directed volume rendering |journal=Proceedings of the 1990 Symposium on Interactive 3D Graphics |pages=361–369 |year=1990 |doi=10.1145/91394.91431 |description=Foundational gaze-directed rendering paper.}}</ref>

<ref name="FOVE_TechCrunch">{{Cite web |title=FOVE Uses Eye Tracking To Make Virtual Reality More Immersive |url=https://techcrunch.com/2014/09/09/fove/ |publisher=TechCrunch |date=2014-09-10 |access-date=2025-10-26}}</ref>

<ref name="FOVE_Kickstarter">{{Cite web |title=FOVE: The World's First Eye Tracking Virtual Reality Headset |url=https://www.kickstarter.com/projects/fove/fove-the-worlds-first-eye-tracking-virtual-reality |publisher=Kickstarter |date=2015-09-01 |access-date=2025-10-26}}</ref>

<ref name="SMI_UploadVR">{{Cite web |title=SMI's 250Hz Eye Tracking and Foveated Rendering Are For Real |url=https://uploadvr.com/smi-hands-on-250hz-eye-tracking/ |publisher=UploadVR |date=2016-01-15 |access-date=2025-10-26}}</ref>

<ref name="Nvidia_Blog_SMI">{{Cite web |title=NVIDIA Partners with SMI on Innovative Rendering Technique That Improves VR |url=https://blogs.nvidia.com/blog/2016/07/21/rendering-foveated-vr/ |publisher=NVIDIA |date=2016-07-21 |access-date=2025-10-26}}</ref>

<ref name="Qualcomm835">{{Cite web |title=Qualcomm Introduces Snapdragon 835 Virtual Reality Development Kit |url=https://www.qualcomm.com/news/releases/2017/02/23/qualcomm-introduces-snapdragon-835-virtual-reality-development-kit |publisher=Qualcomm |date=2017-02-23 |access-date=2025-10-26}}</ref>

<ref name="ViveProEye_Verge">{{Cite web |title=HTC announces new Vive Pro Eye virtual reality headset with native eye tracking |url=https://www.theverge.com/2019/1/7/18172700/htc-vive-pro-eye-tracking-virtual-reality-headset-features-pricing-release-date-ces-2019 |publisher=The Verge |date=2019-01-07 |access-date=2025-10-26}}</ref>

<ref name="QuestFFR_VentureBeat">{{Cite web |title=Oculus Quest gets dynamic fixed foveated rendering |url=https://venturebeat.com/2019/12/22/oculus-quest-gets-dynamic-fixed-foveated-rendering/ |publisher=VentureBeat |date=2019-12-22 |access-date=2025-10-26}}</ref>

<ref name="PSVR2_Blog">{{Cite web |title=PlayStation VR2 and PlayStation VR2 Sense controller: The next generation of VR gaming on PS5 |url=https://blog.playstation.com/2022/01/04/playstation-vr2-and-playstation-vr2-sense-controller-the-next-generation-of-vr-gaming-on-ps5/ |publisher=PlayStation Blog |date=2Details. |access-date=2025-10-26}}</ref>

<ref name="VisionPro_Dev">{{Cite web |title=Discover visionOS |url=https://developer.apple.com/visionos/ |publisher=Apple Developer |date=2023-06-07 |access-date=2025-10-26}}</ref>

<ref name="DeepFovea_Paper">{{cite web |title=DeepFovea: Neural Reconstruction for Foveated Rendering |url=https://dl.acm.org/doi/10.1145/3306307.3328186 |publisher=ACM SIGGRAPH |year=2019}}</ref>

<ref name="fovolnet2022">{{cite web |title=FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks |url=https://arxiv.org/abs/2209.09965 |publisher=arXiv |year=2022}}</ref>

<ref name="vrsplatting2024">{{cite web |title=VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points |url=https://dl.acm.org/doi/10.1145/3728302 |publisher=ACM |year=2024}}</ref>