Disocclusion

Disocclusion is the appearance, in a newly rendered or synthesized view, of parts of a scene that were hidden behind a nearer object in the image or images the new view is built from. Because the source image holds no color or depth for those surfaces, any method that creates a frame by warping, reprojecting or re-rendering existing pixels has to invent their content, and errors there show up as holes, stretched texture, trails or ghosting at the edges of foreground objects. A 2016 Oculus engineering post on Asynchronous Spacewarp described disocclusion as "a graphics term for an object moving out of the way of another".

The problem matters in virtual reality and augmented reality because several standard techniques generate frames from other frames: reprojection and frame extrapolation that cover for a missed frame or reduce latency, camera passthrough that has to move the image from the headset's cameras to the user's eyes, and temporal upscaling and anti-aliasing that reuse samples from earlier frames. Each of these must detect disoccluded pixels and fill them with plausible content.

Reviewed 15 October 2026. Every quotation, date, author, venue and technical claim checked against the cited ASW and ASW 2.0 posts, Microsoft WMR guide, AMD FSR 2 documentation and the full texts of the Mark, LDI, Xu, NeuralPassthrough, Passthrough+, TAA survey, GFFE, Shih and SynSin papers; bibliographic details checked via Crossref and arXiv. About review dates.

Definition

In image-based rendering, a new view is produced by moving pixels from one or more reference images to the positions they would occupy from a different camera position, usually using per-pixel depth. When a foreground object shifts relative to the background in the new view, it uncovers background that the reference camera could not see. Xu, Po and colleagues, writing about depth-image-based rendering (DIBR) for 3D video, describe these "newly exposed areas" as "disocclusions" or "holes" and note that, because the reference view holds no corresponding information, their texture and depth are uncertain. Xiao and colleagues at Meta's Reality Labs define the disocclusion region for passthrough as the scene points "that should be seen by the user's eyes but are occluded from the camera's perspective".

The same effect goes by other names in the older literature. William R. Mark's 1999 dissertation on post-rendering warping calls the gaps "visibility holes" and the resulting errors "occlusion artifacts". Mark also notes that not every visibility error appears as a hole: a missing part of a middle-depth object can be wrongly covered by background that falls on the same screen position, a case Leonard McMillan called an "exposure error".

Disocclusion is the counterpart of occlusion: occlusion handling decides which of several known surfaces is in front, while disocclusion handling has to supply a surface that the reference images never captured. As Mark put it, because the reference frames provide no information about the part of the scene covered by a hole, a warping system "must make some kind of assumption about the scene in order to fill the hole". Temporal methods use the word in the same sense: in temporal anti-aliasing, a pixel is disoccluded when the surface it shows in the current frame was hidden in the previous frame, so the history stored for that pixel belongs to a different surface.

Causes and geometry

Disocclusions form at depth discontinuities. Mark found that the holes left by a 3D warp "are always located at the boundary between a foreground object and the object(s) behind it", although not every such boundary produces a hole. He also pointed out that the effect follows from the reason depth-based warping is used at all: in a 3D warp, points move differently depending on their depth, and this same property is what causes visibility artifacts. If the holes were insignificant, a 3D warp would not be needed, because a perspective warp would serve just as well. A perspective warp can compensate for head rotation but generally not for head translation, unless every object in the scene lies on a single plane.

For VR passthrough, Xiao and colleagues modeled a near occluder in front of a far background plane and derived the width of the disoccluded region from the headset geometry. In their model, the disocclusion grows when the foreground object is closer, and it shrinks when the distance between the cameras and the eyes along the viewing axis (effectively the headset's thickness) is reduced. They also found that placing the two passthrough cameras farther apart than the user's interpupillary distance can remove horizontal disocclusion within a chosen angular region, which they described as a design choice "that diverges from current consumer products".

Errors in the predicted viewpoint add to the problem. In Mark's post-rendering warping system, which rendered reference frames at predicted future head positions, prediction error was "the biggest cause of visibility holes", and with perfect prediction very few holes remained.

History

Holes from depth-based image warping were discussed in graphics research before VR headsets were widely sold. According to Mark's review, Shenchang Eric Chen and Lance Williams' 1993 view interpolation system always warped two reference images to reduce occlusion artifacts, used a one-pixel reconstruction kernel that left frequent holes, and filled remaining hole pixels by interpolating colors from neighboring non-hole pixels. McMillan and Bishop's plenoptic modeling and stereo display systems warped one reference image at a time, so occlusion artifacts became severe as the viewer moved away from the reference viewpoint.

William R. Mark, Leonard McMillan and Gary Bishop presented post-rendering 3D warping at the 1997 Symposium on Interactive 3D Graphics. Mark's 1999 dissertation at the University of North Carolina at Chapel Hill, supervised by Bishop, describes the full system: it rendered every Nth frame conventionally and generated the frames in between with an image warper, and it chose pairs of reference images so that most needed parts of the scene would be visible in at least one of them. The dissertation states that the technique can also compensate for rendering-system and network latency, and one of its test paths was captured by walking through a kitchen model while wearing an HMD.

The 1998 Layered Depth Images (LDI) paper by Jonathan Shade, Steven Gortler, Li-wei He and Richard Szeliski took a different approach. An LDI stores several depth pixels along each line of sight from one camera, and the authors explain that "the farther depth pixels, which are occluded from the LDI center, will act to fill in the disocclusions that occur as the viewpoint moves away from the center".

Disocclusion filling later became a central problem in depth-image-based rendering for 3D video, which Xu and colleagues place in the context of autostereoscopic displays and free-viewpoint television. They summarize the main families of DIBR hole filling as linear interpolation, extrapolation and inpainting, and note that interpolation and extrapolation work for small holes but degrade large ones.

Handling methods

Because the missing content was never captured, all methods either gather the information from somewhere else or guess it. The main approaches are:

Approach	How it works	Examples
More reference views	Warp from two or more images taken from different positions, so a surface hidden in one is visible in another	Chen and Williams' view interpolation and Mark's post-rendering warping both warped two reference images
Layered representations	Store hidden surfaces behind the visible ones so they can be revealed when the viewpoint moves	Layered Depth Images; the 3D photo method of Shih et al., which builds an LDI with inpainted hidden layers
Background extension	Assume the background continues behind the foreground object and fill the hole from background pixels	Mark's hole-filling algorithm; the stretching of background in Oculus ASW
Inpainting	Synthesize texture for the hole from surrounding image content, by diffusion, exemplar patches or a learned model	Depth-aided exemplar-based inpainting for DIBR; learned color-and-depth inpainting for 3D photos
Filtering for real-time use	Fill holes with smoothed colors of nearby distant surfaces, trading detail for speed and temporal stability	Full disocclusion filtering in NeuralPassthrough
History rejection	In temporal methods, detect pixels whose stored history belongs to another surface and reduce or discard that history	Depth-based history validation in temporal anti-aliasing; the disocclusion mask in AMD FSR 2

Background-based filling relies on a property that several groups state directly. Mark's system "assumes that the background object extends into the hole", which he found usually holds. Xu and colleagues note that disoccluded regions are usually background, and that filling them with foreground information creates artifacts. Xiao and colleagues observed that disoccluded regions "are more often missing information from background objects rather than from foreground occluders".

Inpainting can produce convincing still images, but it is costly and can flicker in video. The NeuralPassthrough authors found that context-aware inpainting of the kind used by Shih et al. gave visually acceptable results for static images but introduced temporal flickering in dynamic passthrough video and was too computationally expensive for their real-time use.

Applications in VR and AR

Reprojection and frame extrapolation

When a VR application cannot render at the display's refresh rate, the runtime can synthesize the missing frames from earlier ones. A rotation-only warp such as orientation timewarp can compensate for head rotation but generally not for head translation, because, unlike a depth-based 3D warp, it cannot correct for objects at different depths; techniques that correct for translation or object motion move points differently depending on their depth, and so they produce disocclusions.

Asynchronous Spacewarp (ASW), which Oculus made available for the Oculus Rift with its 1.10 runtime in November 2016, generates extrapolated frames from previous frames rendered by the application. The engineers who introduced it listed "object disocclusion trails" among its known artifacts: ASW does not know what lies behind a moving object, "so the world behind will be stretched to fill the void". They added that because objects usually move only a short distance on the display between 45 fps frames, the trails are generally minimal and hard to spot.

ASW 2.0 combined ASW with Positional Timewarp (PTW), which uses the application's depth buffer to correct for both rotation and translation of the headset and, unlike orientation-only timewarp, corrects for parallax based on distance. In Meta's developer guide for ASW 2.0, Volga Aksoy and Dean Beeler showed in a Robo Recall comparison that after PTW the remaining slight corrections on a chair and on geometry edges were "mostly due to disocclusion and view-dependent shading", and were picked up by ASW's motion vectors after PTW.

Microsoft's guide to SteamVR on Windows Mixed Reality describes a similar motion reprojection mode, in which games render at half frame rate (45 fps instead of 90 fps) and GPU-generated motion vectors are used to extrapolate the next frame. Its list of expected visual artifacts says that sharp contrast edges or text, "especially on in-game HUDs or menus, may look temporarily warped or distorted because of disocclusion". Windows Mixed Reality has since been deprecated; Microsoft states that it is removed in Windows 11 version 24H2 and that existing devices continue to work with Steam through November 2026 on version 23H2.

Research on frame extrapolation for games faces the same issue. The authors of GFFE (2024) note that extrapolation, unlike interpolation methods such as DLSS 3 and FSR 3, adds no latency but is harder "because of the lack of information in the disocclusion regions"; their method analyzes different types of disocclusion and fills them before a shading-correction network runs.

Passthrough and mixed reality

A passthrough headset cannot place its cameras where the user's eyes are, so it has to resynthesize the camera images from the eye positions, and surfaces the cameras cannot see become disocclusions. The Passthrough+ system that Facebook shipped on the Oculus Quest builds a coarse 3D mesh from the headset's cameras and textures it with the left camera image for the left eye and the right camera image for the right eye. Chaurasia and colleagues wrote that warping each camera image over this shorter baseline gives "a plausible visual reproduction of (dis-)occlusion artifacts" without explicit edge constraints in their low-resolution depth map; they named as its most important limitation that the mesh can be inaccurate for nearby objects, because depth estimation around 30 cm becomes inaccurate.

The NeuralPassthrough paper presented at SIGGRAPH 2022 by Lei Xiao, Douglas Lanman and colleagues at Meta stated that existing passthrough methods show distracting artifacts "largely due to the lack of accurate depth information (especially for near-field and disoccluded objects)". Their prototype used a stereo pair of RGB cameras with a 10 cm baseline. The method forward-splats each camera image to the target eye view, fills "partial" disocclusions (holes in only one of the two warped images) by blending the two, and fills "full" disocclusions (holes in both) with a depth-assisted, anisotropic low-pass filter that uses only the colors of relatively far surfaces nearby. The authors compared their results with the passthrough of the Oculus Quest 2, which they described as closely related to Passthrough+.

Temporal anti-aliasing and upscaling

Temporal anti-aliasing (TAA) and temporal upscalers accumulate samples over several frames and reproject the stored history to the current frame with motion vectors. In their 2020 survey, Lei Yang, Shiqiu Liu and Marco Salvi of NVIDIA describe how comparing the reprojected depth of a pixel with the previous frame's depth identifies disoccluded pixels so that history is not reused there. They describe ghosting, which "often appears in newly disoccluded regions caused by moving objects", as a common artifact of screen-space temporal reuse, and note that newly disoccluded regions are prone to undersampling artifacts.

AMD's FidelityFX Super Resolution 2 documentation includes a "Depth clip" stage whose goal is "to produce a mask which indicates disoccluded areas of the current frame". It computes each pixel's depth from the previous camera position, compares it with the current depth against a separation threshold, and later reduces "the influence of the historical samples for areas which are disoccluded".

Novel view synthesis from photographs

Systems that let a viewer move around inside a captured photograph face disocclusion whenever the viewpoint leaves the capture position. Shih, Su, Kopf and Huang's CVPR 2020 method converts a single RGB-D image into a "3D photo", a multi-layer representation that contains hallucinated color and depth "in regions occluded in the original view", using a Layered Depth Image and a learned inpainting model, so that it can be rendered with motion parallax. Learned single-image methods such as SynSin (Wiles et al., 2020) likewise project scene features into the target view and use a refinement network to "inpaint missing regions".

See also

• Occlusion
• Asynchronous Spacewarp
• Positional Timewarp
• Timewarp
• Motion Smoothing
• Depth buffer
• Depth map
• Passthrough
• Neural Radiance Fields
• Optical flow