Occlusion culling: Difference between revisions

For [[VR]]/[[AR]] specifically, occlusion culling addresses the unique challenge of [[stereoscopic rendering]], which requires rendering each scene twice—once for each eye—effectively doubling the rendering workload. [[John Carmack]] at [[Oculus]] established that VR requires below 20 milliseconds of motion-to-photons [[latency]] to feel imperceptible to humans, making every optimization critical.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 1: Developing a Custom Solution". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/</ref> Modern VR-specific innovations like [[Umbra Software]]'s "Stereo Camera" feature perform a single occlusion query for a spherical volume encompassing both eyes, effectively halving the required processing time compared to traditional per-eye approaches.<ref>Road to VR. "Umbra Positioning Occlusion Culling Tech for 120 FPS VR Gaming". https://www.roadtovr.com/umbra-software-occlusion-culling-120-fpt-virtual-reality-gaming/</ref>

The modern era of occlusion culling began with [[Ned Greene]], Michael Kass, and Gavin Miller's seminal 1993 [[SIGGRAPH]] paper "Hierarchical Z-buffer Visibility."<ref>NVIDIA. "Chapter 29. Efficient Occlusion Culling". GPU Gems. https://developer.nvidia.com/gpugems/gpugems/part-v-performance-and-practicalities/chapter-29-efficient-occlusion-culling</ref> This groundbreaking work introduced the hierarchical visibility algorithm using [[octrees]] for scene organization and the Z-pyramid concept—a hierarchical image pyramid of depth values enabling efficient culling of occluded regions. Their technique generated roughly one hundred times fewer depth comparisons than conventional Z-buffering, establishing hierarchical approaches as the path forward.

Modern engines now employ two-phase [[hierarchical depth buffer]] (HiZ) culling: rendering objects visible in the previous frame, building a depth pyramid, then testing newly visible objects—eliminating CPU-GPU synchronization while maintaining efficiency.<ref>Vulkan Guide. "Compute based Culling". https://vkguide.dev/docs/gpudriven/compute_culling/</ref> By the 2020s, mobile VR/AR devices (e.g., Meta Quest series) necessitated custom, lightweight implementations, blending traditional methods with AI-accelerated depth estimation for dynamic scenes.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 1: Developing a Custom Solution". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/</ref>

Occlusion culling operates on the principle that not all scene geometry contributes to the final image, as closer opaque objects can fully obscure distant ones. It is distinct from but complementary to other culling methods. Techniques are broadly classified as image-space (pixel-level, e.g., Z-buffering) or object-space (geometry-level, e.g., PVS), with hybrids common in practice.<ref>Wikipedia. "Occlusion culling". https://en.wikipedia.org/wiki/Occlusion_culling</ref>

The fundamental challenge is pipeline latency—queries travel from CPU to GPU queue to rasterization and back, typically requiring 1-3 frames for results. Naïve implementations that wait for query results cause CPU stalls that starve the GPU, negating any performance benefit. The solution is the coherent hierarchical culling algorithm developed by Bittner and Wimmer, which exploits [[temporal coherence]] by assuming objects visible in the previous frame remain visible, rendering them immediately while asynchronously querying previously occluded objects organized in a spatial hierarchy like an [[octree]] or [[k-d tree]].<ref>NVIDIA. "Chapter 6. Hardware Occlusion Queries Made Useful". GPU Gems 2. https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-6-hardware-occlusion-queries-made-useful</ref>

These occlusion queries (sometimes called "Hierarchical Z-buffer" or "Z-culling" when done at a coarse level) allow dynamic, on-the-fly culling of arbitrary objects without precomputed data. To mitigate latency, engines often use techniques like '''temporal reprojection''' or '''multi-frame queries''' (e.g., issuing queries for many objects and using last frame's results to decide what to draw in the current frame, sometimes known as "round-robin" occlusion culling). This reduces stalls by giving the GPU more time to produce query results in parallel. Unreal Engine, for example, can use asynchronous occlusion queries and even has a "Round Robin Occlusion" mode optimized for VR to distribute query workload across frames.<ref>Epic Games. "Visibility and Occlusion Culling in Unreal Engine". https://dev.epicgames.com/documentation/en-us/unreal-engine/visibility-and-occlusion-culling-in-unreal-engine</ref>

The primary advantage is zero-frame latency—visibility determination happens entirely before rendering begins, preventing the pop-in artifacts common with asynchronous GPU queries. This makes it particularly valuable for VR where even single-frame delays create noticeable temporal artifacts.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 1: Developing a Custom Solution". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/</ref> The technique also frees GPU resources for rendering rather than visibility testing, crucial on bandwidth-constrained mobile VR platforms.

[[Hierarchical Occlusion Maps]] (HOM), introduced by Hansong Zhang in 1997, separate visibility testing into a 2D overlap test and 1D depth test. The system renders occluders as white-on-black images without shading, builds an opacity pyramid through averaging, and tests objects by checking opacity values at appropriate hierarchy levels.<ref>NVIDIA. "Chapter 29. Efficient Occlusion Culling". GPU Gems. https://developer.nvidia.com/gpugems/gpugems/part-v-performance-and-practicalities/chapter-29-efficient-occlusion-culling</ref> An opacity threshold provides controllable approximate culling—accepting slight inaccuracies for performance.

The depth component uses a coarse grid (64×64 typical) storing the furthest Z-value per region estimated from occluder bounding box vertices. This two-stage approach reduces memory compared to full-resolution depth buffers while enabling automatic occluder fusion through the opacity pyramid. The technique's advantage over pure HZB is flexibility in accuracy—developers tune the threshold to balance culling aggressiveness against potential false positives. Techniques like Hierarchical Occlusion Maps use multi-resolution buffers to quickly cull objects by testing against progressively lower-detail occluder representations.<ref>NVIDIA. "Chapter 29. Efficient Occlusion Culling". GPU Gems. https://developer.nvidia.com/gpugems/gpugems/part-v-performance-and-practicalities/chapter-29-efficient-occlusion-culling</ref>

The cutting edge of occlusion culling as of 2024-2025 centers on fully GPU-driven rendering pipelines that eliminate CPU involvement entirely. [[Mesh shaders]], available since NVIDIA Turing (2018) and AMD RDNA2, enable programmable geometry processing with per-meshlet culling granularity.<ref>NVIDIA. "Introduction to Turing Mesh Shaders". https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/</ref> Meshlets—small clusters of 32-256 triangles—undergo individual frustum, backface cone, and occlusion tests on the GPU before rasterization.

The stringent latency requirements compound these challenges. John Carmack established that VR needs below 20 milliseconds motion-to-photons latency to avoid perceptible lag.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 1: Developing a Custom Solution". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/</ref> With rendering limited to approximately 3 milliseconds on the CPU thread before vertical sync on console VR, visibility culling becomes a critical bottleneck. Hardware occlusion queries with their 1-3 frame delay cause unacceptable pop-in artifacts—objects suddenly appearing or disappearing, which proves dramatically more noticeable in VR than flat-screen gaming and particularly jarring if occurring in only one eye.

For parts of the game with limited camera movement (e.g., along a fixed rail), the system was extended. Instead of baking a full PVS at many points along the path, which would be memory-intensive, they baked a PVS only at key points.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 2: Moving Cameras and Other Insights". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-part-2-moving-cameras-and-other-insights/</ref> For the space between two key points, they stored only the ''difference''—a small list of objects to enable or disable when transitioning from one PVS to the next. This "difference list" approach dramatically reduced the memory footprint and the computational cost of updating visibility as the camera moved.

[[Google]]'s [[ARCore]] Depth API, released from beta in June 2020, democratized this capability by using monocular depth estimation—capturing multiple images as the device moves and calculating depth through motion parallax triangulation.<ref>MDPI. "Occlusion Handling for Mobile AR Applications in Indoor and Outdoor Scenarios". https://www.mdpi.com/1424-8220/23/9/4245</ref> ARCore's Depth API produces a depth map of the environment using sensors and camera data (via techniques like [[structured light]], [[time-of-flight]], or stereo vision), which the AR application can use to allow real objects to occlude virtual objects.<ref>Google Developers. "Depth adds realism – ARCore Depth API documentation". https://developers.google.com/ar/develop/depth</ref> This works on standard mobile cameras across 200+ million devices without requiring specialized [[Time-of-Flight]] sensors. In practice, AR developers can enable "environment occlusion" so that, when the device knows the distance to real surfaces (walls, furniture, people, etc.), it will not draw virtual content that is behind those surfaces relative to the camera. Google emphasizes that "occlusion – accurately rendering a virtual object behind real-world objects – is paramount to an immersive AR experience".<ref>Google Developers. "Depth adds realism – ARCore Depth API documentation". https://developers.google.com/ar/develop/depth</ref>

AR developers are encouraged to use the same optimizations as VR: level-of-detail, batching, and occlusion culling for virtual content that might be off-screen or behind other virtual objects. In fact, Unity's AR Foundation toolkit integrates occlusion culling just as in VR — for example, Unity will not render virtual objects that are outside the camera view or completely behind other virtual objects in the scene. This helps save processing power on phones or AR glasses.<ref>Unity Technologies. "Optimizing your VR/AR Experiences – Unity Learn Tutorial". https://learn.unity.com/tutorial/optimizing-your-vr-ar-experiences</ref>

Techniques to gather environmental depth include [[structured light]], [[time-of-flight]] cameras, and stereo camera vision, each with limitations in range, lighting, and resolution. When high-quality depth data is available (e.g., LiDAR on high-end devices), AR apps can pre-scan the environment and even use a generated mesh as an occluder for virtual content, achieving very convincing occlusion effects. Developers often combine depth-based occlusion with other tricks (like shader-based depth masks or manual placement of invisible occlusion geometry in known locations) to handle occlusion in specific scenarios.<ref>Medium. "Occlusion Culling in Augmented Reality". https://medium.com/@ishtian_rev/occlusion-culling-in-augmented-reality-c1ee433598</ref>

GPU savings manifest through multiple mechanisms. [[Draw call]] reduction proves critical as each eliminated draw call saves CPU-side validation, state changes, and GPU command processing—Republique VR's reduction from 1,400 to 60 draw calls freed enormous overhead.<ref>Meta for Developers. "Occlusion Culling for Mobile VR - Part 1: Developing a Custom Solution". https://developers.meta.com/horizon/blog/occlusion-culling-for-mobile-vr-developing-a-custom-solution/</ref> [[Overdraw]] reduction directly saves fragment shading work; in indoor levels with high depth complexity, 50-90% of GPU fragment processing can be eliminated. [[Early-Z]] rejection operates at the hardware level but only helps when objects render front-to-back, whereas occlusion culling prevents occluded objects from entering the pipeline entirely.

[[Frustum culling]] removes objects outside the camera's field of view through simple geometric tests at very low cost—it should always run first and always be enabled. Occlusion culling then operates on the frustum-culled set, eliminating hidden objects within the view. The techniques multiply: approximately 3x from frustum times 3x from occlusion yields 8-9x combined as Intel demonstrated.<ref>Intel. "Software Occlusion Culling". https://www.intel.com/content/www/us/en/developer/articles/technical/software-occlusion-culling.html</ref>

[[Backface culling]] operates differently still—a hardware-accelerated fixed-function GPU unit that eliminates triangles facing away from the camera during rasterization using simple dot product tests. This provides approximately 50% fragment shading reduction for closed opaque meshes at zero CPU cost. It runs automatically after all visibility culling and should remain enabled for solid geometry.

This enables longer view distances compared to max draw distance settings and works for movable and non-movable Actors supporting opaque and masked blend modes. The inherent one-frame latency can cause "popping" with rapid camera movement—objects suddenly appearing as visibility predictions lag actual view changes.

[[Godot Engine]] implements occlusion culling in Godot 4.x using the [[Embree]] library for CPU-based rasterization—a software raytracing library from Intel. The system bakes simplified representations of static geometry using OccluderInstance3D nodes and tests [[AABBs]] against occluder shapes at runtime. Setup involves enabling Occlusion Culling in Project Settings Rendering section, creating occluders through automatic baking or manual authoring, and baking the occlusion data. The technique proves most effective for indoor scenes with many small rooms and particularly benefits the Mobile renderer which lacks depth prepass.

Profile first to confirm GPU-bound scenarios before implementing occlusion culling. Always combine with frustum culling for multiplicative gains. Use hierarchical testing by grouping objects and testing large nodes first to reduce query counts. Exploit temporal coherence by reusing previous frame results—objects visible last frame likely remain visible.<ref>NVIDIA. "Chapter 6. Hardware Occlusion Queries Made Useful". GPU Gems 2. https://developer.nvidia.com/gpugems/gpugems2/part-i-geometric-complexity/chapter-6-hardware-occlusion-queries-made-useful</ref>

Pipeline processing by working on frame N while rendering frame N+1 eliminates stalls. Conservative testing prevents missing visible objects—false negatives cause obvious artifacts while false positives merely reduce efficiency.

Mesh shader integration enables unprecedented culling granularity. Rather than culling entire meshes or objects, per-meshlet culling at 32-256 triangle granularity provides fine-grained efficiency.<ref>NVIDIA. "Introduction to Turing Mesh Shaders". https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/</ref> The performance benefits prove substantial—40-48% improvements documented across multiple shipping titles—but hardware requirements remain a constraint. As of 2025, mesh shaders require NVIDIA Turing/RTX 2000+ (2018), AMD RDNA2+ (2020), or Intel Arc, limiting deployment to relatively recent hardware.

For AR specifically, depth sensing democratization represents the major development. Google's ARCore Depth API release from beta in June 2020 brought real-world occlusion to 200+ million devices using monocular depth estimation—capturing multiple images as devices move and calculating depth through motion parallax without requiring Time-of-Flight sensors.<ref>MDPI. "Occlusion Handling for Mobile AR Applications in Indoor and Outdoor Scenarios". https://www.mdpi.com/1424-8220/23/9/4245</ref> This software-based approach works on standard mobile cameras, though it fuses ToF data when available on premium devices.

The future trajectory involves [[neural networks]] and [[machine learning]] for improved depth estimation and visibility prediction. Research demonstrates deep learning models predicting depth from single images with increasing accuracy, potentially replacing or augmenting physical depth sensors. However, real-time performance for complex dynamic scenes remains a research challenge as of 2025—practical deployment requires models running in milliseconds on mobile GPUs, a constraint current approaches struggle to meet.