Foveated rendering: Difference between revisions
Xinreality (talk | contribs) |
Xinreality (talk | contribs) m Text replacement - "e.g.," to "for example" |
||
| Line 57: | Line 57: | ||
# '''Display Update:''' The final, composited multi-resolution image is presented to the user. | # '''Display Update:''' The final, composited multi-resolution image is presented to the user. | ||
This entire loop must be completed within the frame budget ( | This entire loop must be completed within the frame budget (for example under 11.1 ms for a 90 Hz display) to ensure a smooth experience. | ||
=== Methods of Quality Reduction === | === Methods of Quality Reduction === | ||
| Line 63: | Line 63: | ||
The term "reducing quality" encompasses several distinct techniques that can be applied to the peripheral regions to save computational power. These methods can be used individually or in combination:<ref name="IntegrativeView" /> | The term "reducing quality" encompasses several distinct techniques that can be applied to the peripheral regions to save computational power. These methods can be used individually or in combination:<ref name="IntegrativeView" /> | ||
* '''Resolution Scaling / Subsampling:''' This is the most common and intuitive method. The peripheral regions are rendered into a smaller off-screen buffer ( | * '''Resolution Scaling / Subsampling:''' This is the most common and intuitive method. The peripheral regions are rendered into a smaller off-screen buffer (for example at half or quarter resolution) and then upscaled to fit the final display. This directly reduces the number of pixels that need to be processed and shaded.<ref name="GazeContingent2D">{{cite web |url=http://stanford.edu/class/ee367/Winter2017/mehra_sankar_ee367_win17_report.pdf |title=Gaze Contingent Foveated Rendering for 2D Displays}}</ref> | ||
* '''Shading Rate Reduction:''' This method focuses on reducing the workload of the [[pixel shader]] (also known as a fragment shader). Instead of executing a complex shading program for every single pixel in the periphery, a single shader result can be applied to a block of multiple pixels. This is the core mechanism behind [[Variable Rate Shading]] (VRS).<ref name="TOYF_Paper" /><ref name="AutoVRSE">{{cite web |url=https://www.autovrse.com/foveated-rendering |title=What is Foveated Rendering? - autovrse}}</ref> | * '''Shading Rate Reduction:''' This method focuses on reducing the workload of the [[pixel shader]] (also known as a fragment shader). Instead of executing a complex shading program for every single pixel in the periphery, a single shader result can be applied to a block of multiple pixels. This is the core mechanism behind [[Variable Rate Shading]] (VRS).<ref name="TOYF_Paper" /><ref name="AutoVRSE">{{cite web |url=https://www.autovrse.com/foveated-rendering |title=What is Foveated Rendering? - autovrse}}</ref> | ||
* '''Geometric Simplification:''' The geometric complexity of the scene can be reduced in the periphery. This involves using lower-polygon [[level of detail]] models for objects that are outside the user's direct gaze. | * '''Geometric Simplification:''' The geometric complexity of the scene can be reduced in the periphery. This involves using lower-polygon [[level of detail]] models for objects that are outside the user's direct gaze. | ||
| Line 74: | Line 74: | ||
==== Variable Rate Shading (VRS) ==== | ==== Variable Rate Shading (VRS) ==== | ||
[[Variable Rate Shading]] (VRS) is a hardware feature available on modern GPUs ( | [[Variable Rate Shading]] (VRS) is a hardware feature available on modern GPUs (for example [[NVIDIA]] Turing architecture and newer, [[AMD]] RDNA 2 and newer, [[Intel]] Gen11+) that provides fine-grained control over the pixel shading rate.<ref name="TOYF_Paper" /><ref name="OpenXRToolkit">{{cite web |url=https://mbucchia.github.io/OpenXR-Toolkit/fr.html |title=Foveated Rendering - OpenXR Toolkit}}</ref><ref name="microsoft2019">{{cite web |url=https://devblogs.microsoft.com/directx/variable-rate-shading-a-scalpel-in-a-world-of-sledgehammers/ |title=Variable Rate Shading: a scalpel in a world of sledgehammers |publisher=Microsoft DirectX Blog |year=2019}}</ref> It allows a single pixel shader operation to compute the color for a block of pixels, such as a 2x2 or 4x4 block, instead of just a single pixel.<ref name="VarjoAPI">{{cite web |url=https://developer.varjo.com/docs/native/foveated-rendering-api |title=Foveated Rendering - Varjo for Developers}}</ref><ref name="PicoUnrealOpenXR">{{cite web |url=https://developer.picoxr.com/document/unreal-openxr/fixed-foveated-rendering/ |title=Foveated rendering - PICO Unreal OpenXR Plugin}}</ref> The technique supports shading rates from 1×1 (full quality) to 4×4 (coarse, one shade per 16 pixels). | ||
==== Multi-View Rendering & Quad Views ==== | ==== Multi-View Rendering & Quad Views ==== | ||
| Line 135: | Line 135: | ||
|- | |- | ||
! Performance Savings | ! Performance Savings | ||
| Moderate. Reduces GPU load by rendering fewer pixels in the periphery ( | | Moderate. Reduces GPU load by rendering fewer pixels in the periphery (for example 26-43% savings reported for Meta Quest).<ref name="QuestProPerformance">{{cite web |url=https://www.uploadvr.com/quest-pro-foveated-rendering-performance/ |title=Quest Pro Foveated Rendering GPU Savings Detailed}}</ref> | Significant. Allows for more aggressive degradation, leading to greater GPU savings (for example 33-52% savings reported for Meta Quest Pro).<ref name="VRX_FR_Types" /><ref name="QuestProPerformance" /> | ||
|- | |- | ||
! Ideal Use Cases | ! Ideal Use Cases | ||