Passthrough: Difference between revisions
Xinreality (talk | contribs) No edit summary |
Xinreality (talk | contribs) No edit summary |
||
| Line 39: | Line 39: | ||
===Color Passthrough=== | ===Color Passthrough=== | ||
Uses [[RGB]] color cameras for a full-color view of the real world, greatly enhancing realism and enabling use cases like reading phone screens or interacting with colored objects. First widely available consumer example was Meta Quest Pro.<ref name="MixedNews_Cambria"/> Quality varies significantly based on camera resolution, processing, and calibration (for example Quest 3 offers ~10x the passthrough pixels of Quest 2).<ref name="UploadVR_specs"> | Uses [[RGB]] color cameras for a full-color view of the real world, greatly enhancing realism and enabling use cases like reading phone screens or interacting with colored objects. First widely available consumer example was Meta Quest Pro.<ref name="MixedNews_Cambria"/> Quality varies significantly based on camera resolution, processing, and calibration (for example Quest 3 offers ~10x the passthrough pixels of Quest 2).<ref name="UploadVR_specs"></ref> High-quality color passthrough (for example Varjo XR series, Vision Pro) aims for near-photorealism.<ref name="Skarredghost_Varjo"/><ref name="VisionProPassthrough"/> Requires more powerful hardware and sophisticated software. | ||
===Monoscopic vs. Stereoscopic=== | ===Monoscopic vs. Stereoscopic=== | ||
| Line 66: | Line 66: | ||
Creating high-quality, comfortable passthrough involves overcoming significant hurdles: | Creating high-quality, comfortable passthrough involves overcoming significant hurdles: | ||
*'''[[Latency (engineering)|Latency]]:''' The delay between real-world motion and the passthrough display update ([[photon-to-photon latency]]). High latency (>~20ms<ref name="LatencyThreshold"/>) causes disorientation, [[motion sickness]] ("world swimming"), and breaks immersion. Fast processing pipelines are essential.<ref name="PassthroughChallengesUploadVR"/> Residual latency can cause ghosting or trailing artifacts on moving objects.<ref name="UploadVR_ghosting"> | *'''[[Latency (engineering)|Latency]]:''' The delay between real-world motion and the passthrough display update ([[photon-to-photon latency]]). High latency (>~20ms<ref name="LatencyThreshold"/>) causes disorientation, [[motion sickness]] ("world swimming"), and breaks immersion. Fast processing pipelines are essential.<ref name="PassthroughChallengesUploadVR"/> Residual latency can cause ghosting or trailing artifacts on moving objects.<ref name="UploadVR_ghosting"></ref> | ||
*'''Resolution and Image Quality:''' Camera feeds are often lower resolution than human vision, leading to pixelation or blurriness, making fine details (like text) hard to see.<ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> Limited [[dynamic range]] struggles with bright highlights and dark shadows compared to the human eye. Poor [[low-light performance]] results in noisy, grainy images.<ref name="CameraLimitations"/> Achieving high resolution and good image quality requires better sensors and significant processing power. | *'''Resolution and Image Quality:''' Camera feeds are often lower resolution than human vision, leading to pixelation or blurriness, making fine details (like text) hard to see.<ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> Limited [[dynamic range]] struggles with bright highlights and dark shadows compared to the human eye. Poor [[low-light performance]] results in noisy, grainy images.<ref name="CameraLimitations"/> Achieving high resolution and good image quality requires better sensors and significant processing power. | ||
*'''Camera Placement and Perspective Mismatch:''' Cameras are offset from the user's eyes, causing [[parallax]] errors if not corrected. Naive display leads to distorted views, incorrect scale, and depth perception issues, especially for close objects.<ref name="PassthroughChallengesUploadVR"/> Sophisticated [[reprojection]] algorithms are needed to warp the camera view to match the eye's perspective, but perfect correction is difficult.<ref name="KGuttag_Align"/> This geometric misalignment can cause eye strain or discomfort.<ref name="KGuttag_Align"/> Close objects (<~0.5m) often appear warped even in good systems due to sensor/lens limitations and reprojection challenges.<ref name="UploadVR_Q3Review_MR"/> | *'''Camera Placement and Perspective Mismatch:''' Cameras are offset from the user's eyes, causing [[parallax]] errors if not corrected. Naive display leads to distorted views, incorrect scale, and depth perception issues, especially for close objects.<ref name="PassthroughChallengesUploadVR"/> Sophisticated [[reprojection]] algorithms are needed to warp the camera view to match the eye's perspective, but perfect correction is difficult.<ref name="KGuttag_Align"/> This geometric misalignment can cause eye strain or discomfort.<ref name="KGuttag_Align"/> Close objects (<~0.5m) often appear warped even in good systems due to sensor/lens limitations and reprojection challenges.<ref name="UploadVR_Q3Review_MR"/> | ||
| Line 193: | Line 193: | ||
<ref name="Auganix_Pico4Ultra">[https://www.auganix.org/vr-news-pico-launches-pico-4-ultra/ Auganix – Pico Unveils ‘Pico 4 Ultra’] (Aug 21 2024)</ref> | <ref name="Auganix_Pico4Ultra">[https://www.auganix.org/vr-news-pico-launches-pico-4-ultra/ Auganix – Pico Unveils ‘Pico 4 Ultra’] (Aug 21 2024)</ref> | ||
<ref name="UploadVR_Q3Review">[https://www.uploadvr.com/quest-3-review/ UploadVR – Quest 3 Review: Excellent VR With Limited MR] (Oct 9 2023)</ref> | <ref name="UploadVR_Q3Review">[https://www.uploadvr.com/quest-3-review/ UploadVR – Quest 3 Review: Excellent VR With Limited MR] (Oct 9 2023)</ref> | ||
<ref name="UploadVR_specs"> | <ref name="UploadVR_specs">[https://www.uploadvr.com/quest-3-specs/ UploadVR – Quest 3 Specs Compared to Quest 2 & Apple Vision Pro] (Sept 27 2023)</ref> | ||
<ref name="UploadVR_ghosting"> | <ref name="UploadVR_ghosting">[https://www.uploadvr.com/quest-3-review/#passthrough Ghosting issue noted in UploadVR Quest 3 Review] (Oct 9 2023)</ref> | ||
<ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> | <ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> | ||
<ref name="Varjo_blog">Varjo Blog - Video Pass-Through XR - Merge Real and Virtual (Urho Konttori, 2020)</ref> | <ref name="Varjo_blog">Varjo Blog - Video Pass-Through XR - Merge Real and Virtual (Urho Konttori, 2020)</ref> | ||