Passthrough: Difference between revisions
Xinreality (talk | contribs) No edit summary |
Xinreality (talk | contribs) No edit summary |
||
| Line 28: | Line 28: | ||
**The [[Apple Vision Pro]] (2023 announcement, 2024 release) emphasized passthrough-based MR ("[[spatial computing]]"), using dual high-resolution color cameras, advanced processing ([[Apple R1]] chip), and a [[LiDAR]] scanner for precise depth mapping.<ref name="VisionProPassthrough">[https://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/ Apple Vision Pro Announcement]</ref><ref name="Verge_VisionPro">The Verge - Apple Vision Pro review: magic, until it’s not (Nilay Patel, June 2023)</ref> | **The [[Apple Vision Pro]] (2023 announcement, 2024 release) emphasized passthrough-based MR ("[[spatial computing]]"), using dual high-resolution color cameras, advanced processing ([[Apple R1]] chip), and a [[LiDAR]] scanner for precise depth mapping.<ref name="VisionProPassthrough">[https://www.apple.com/newsroom/2023/06/introducing-apple-vision-pro/ Apple Vision Pro Announcement]</ref><ref name="Verge_VisionPro">The Verge - Apple Vision Pro review: magic, until it’s not (Nilay Patel, June 2023)</ref> | ||
**Other high-end devices like the [[Pimax Crystal]] (2023) and [[Varjo XR-4]] (late 2023) continued to push resolution and fidelity.<ref name="PimaxCrystal">[https://pimax.com/crystal/ Pimax Crystal Product Page]</ref><ref name="VarjoXR4">[https://varjo.com/products/xr-4/ Varjo XR-4 Product Page]</ref> | **Other high-end devices like the [[Pimax Crystal]] (2023) and [[Varjo XR-4]] (late 2023) continued to push resolution and fidelity.<ref name="PimaxCrystal">[https://pimax.com/crystal/ Pimax Crystal Product Page]</ref><ref name="VarjoXR4">[https://varjo.com/products/xr-4/ Varjo XR-4 Product Page]</ref> | ||
**Even mid-range devices began incorporating improved color passthrough and depth sensing, like the anticipated Pico 4 Ultra (2024).<ref name="Auganix_Pico4Ultra"> | **Even mid-range devices began incorporating improved color passthrough and depth sensing, like the anticipated Pico 4 Ultra (2024).<ref name="Auganix_Pico4Ultra"></ref> | ||
Passthrough has evolved from a basic safety utility to a core feature enabling sophisticated mixed reality experiences, blurring the lines between traditional VR and AR. | Passthrough has evolved from a basic safety utility to a core feature enabling sophisticated mixed reality experiences, blurring the lines between traditional VR and AR. | ||
| Line 57: | Line 57: | ||
===Mixed Reality Blending=== | ===Mixed Reality Blending=== | ||
Refers to how seamlessly the passthrough system integrates virtual content with the real-world camera feed. Advanced implementations aim to unify lighting, shadows, reflections, and occlusion across both realities. Examples include: | Refers to how seamlessly the passthrough system integrates virtual content with the real-world camera feed. Advanced implementations aim to unify lighting, shadows, reflections, and occlusion across both realities. Examples include: | ||
*Virtual objects casting realistic shadows on real surfaces.<ref name="Varjo_blog"> | *Virtual objects casting realistic shadows on real surfaces.<ref name="Varjo_blog"></ref> | ||
*Virtual elements being correctly hidden by real furniture or people. | *Virtual elements being correctly hidden by real furniture or people. | ||
*Virtual lighting affecting the appearance of the real world within the passthrough view (and vice-versa). | *Virtual lighting affecting the appearance of the real world within the passthrough view (and vice-versa). | ||
| Line 97: | Line 97: | ||
===Enterprise and Professional Uses=== | ===Enterprise and Professional Uses=== | ||
*'''Collaboration:''' Design reviews where virtual prototypes are viewed in a real meeting room alongside physical mockups or colleagues.<ref name="XRToday_enterprise"> | *'''Collaboration:''' Design reviews where virtual prototypes are viewed in a real meeting room alongside physical mockups or colleagues.<ref name="XRToday_enterprise"></ref> Remote collaboration where experts guide on-site technicians using virtual annotations overlaid on the real equipment view. | ||
*'''Training and Simulation:''' Combining virtual scenarios with physical controls or environments (for example flight simulation using a real cockpit visible via passthrough, medical training on physical manikins with virtual overlays).<ref name="VIVE_Blog_Sauce"/> | *'''Training and Simulation:''' Combining virtual scenarios with physical controls or environments (for example flight simulation using a real cockpit visible via passthrough, medical training on physical manikins with virtual overlays).<ref name="VIVE_Blog_Sauce"/> | ||
*'''Visualization:''' Architects visualizing 3D models on a real site, designers overlaying virtual concepts onto physical products. | *'''Visualization:''' Architects visualizing 3D models on a real site, designers overlaying virtual concepts onto physical products. | ||
*'''Productivity:''' Creating expansive virtual workspaces integrated with the physical office environment, improving multitasking while maintaining awareness.<ref name="XRToday_benefits"> | *'''Productivity:''' Creating expansive virtual workspaces integrated with the physical office environment, improving multitasking while maintaining awareness.<ref name="XRToday_benefits"></ref> | ||
===Industrial and Field Uses=== | ===Industrial and Field Uses=== | ||
| Line 131: | Line 131: | ||
*'''[[Lynx R1]]:''' Standalone headset project focusing specifically on delivering quality color passthrough at a competitive price point.<ref name="LynxR1">[https://www.lynx-r.com/ Lynx R1 Official Website]</ref> | *'''[[Lynx R1]]:''' Standalone headset project focusing specifically on delivering quality color passthrough at a competitive price point.<ref name="LynxR1">[https://www.lynx-r.com/ Lynx R1 Official Website]</ref> | ||
*'''[[PlayStation VR2]]:''' Features stereo black-and-white passthrough primarily for setup and quick environment checks.<ref name="RoadToVR_PSVR2"/> | *'''[[PlayStation VR2]]:''' Features stereo black-and-white passthrough primarily for setup and quick environment checks.<ref name="RoadToVR_PSVR2"/> | ||
*'''[[Valve Index]]:''' Basic stereoscopic monochrome passthrough via front cameras.<ref name="Index_Docs"> | *'''[[Valve Index]]:''' Basic stereoscopic monochrome passthrough via front cameras.<ref name="Index_Docs"></ref> | ||
==Future Developments== | ==Future Developments== | ||
| Line 141: | Line 141: | ||
*Implementing selective passthrough (showing only specific real-world elements like hands or keyboards) and potentially "augmented reality" filters applied to the real-world view. | *Implementing selective passthrough (showing only specific real-world elements like hands or keyboards) and potentially "augmented reality" filters applied to the real-world view. | ||
*Utilizing [[eye tracking]] for [[foveated rendering]] of the passthrough feed or dynamic depth-of-field adjustments. | *Utilizing [[eye tracking]] for [[foveated rendering]] of the passthrough feed or dynamic depth-of-field adjustments. | ||
*Exploring novel camera technologies like light field cameras (for example Meta's "Flamera" concept<ref name="KGuttag_Flamera"> | *Exploring novel camera technologies like light field cameras (for example Meta's "Flamera" concept<ref name="KGuttag_Flamera"></ref>) to better solve perspective issues. | ||
As technology matures, VST passthrough aims to provide a near-seamless blend between the virtual and physical worlds, potentially unifying VR and AR capabilities into single, versatile devices. | As technology matures, VST passthrough aims to provide a near-seamless blend between the virtual and physical worlds, potentially unifying VR and AR capabilities into single, versatile devices. | ||
| Line 191: | Line 191: | ||
<ref name="PimaxCrystal">[https://pimax.com/crystal/ Pimax Crystal Product Page]</ref> | <ref name="PimaxCrystal">[https://pimax.com/crystal/ Pimax Crystal Product Page]</ref> | ||
<ref name="VarjoXR4">[https://varjo.com/products/xr-4/ Varjo XR-4 Product Page]</ref> | <ref name="VarjoXR4">[https://varjo.com/products/xr-4/ Varjo XR-4 Product Page]</ref> | ||
<ref name="Auganix_Pico4Ultra"> | <ref name="Auganix_Pico4Ultra">[https://www.auganix.org/vr-news-pico-launches-pico-4-ultra/ Auganix – Pico Unveils ‘Pico 4 Ultra’] (Aug 21 2024)</ref> | ||
<ref name="UploadVR_Q3Review">[https://www.uploadvr.com/quest-3-review/ UploadVR – Quest 3 Review: Excellent VR With Limited MR] (Oct 9 2023)</ref> | <ref name="UploadVR_Q3Review">[https://www.uploadvr.com/quest-3-review/ UploadVR – Quest 3 Review: Excellent VR With Limited MR] (Oct 9 2023)</ref> | ||
<ref name="UploadVR_specs">UploadVR - Quest 3 Specs Compared To Quest 2 & Apple Vision Pro (David Heaney, Sep 27, 2023)</ref> | <ref name="UploadVR_specs">UploadVR - Quest 3 Specs Compared To Quest 2 & Apple Vision Pro (David Heaney, Sep 27, 2023)</ref> | ||
<ref name="UploadVR_ghosting">UploadVR - Quest 3 Review (on ghosting issue)</ref> | <ref name="UploadVR_ghosting">UploadVR - Quest 3 Review (on ghosting issue)</ref> | ||
<ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> | <ref name="CameraLimitations">[https://arstechnica.com/gadgets/2023/10/quest-3-review-finally-real-mixed-reality-for-under-500/ Ars Technica Quest 3 Review discussing passthrough quality]</ref> | ||
<ref name="Varjo_blog"> | <ref name="Varjo_blog">Varjo Blog - Video Pass-Through XR - Merge Real and Virtual (Urho Konttori, 2020)</ref> | ||
<ref name="TechTarget_ARdef">[https://www.techtarget.com/whatis/definition/augmented-reality-AR TechTarget: What is augmented reality (AR)?]</ref> | <ref name="TechTarget_ARdef">[https://www.techtarget.com/whatis/definition/augmented-reality-AR TechTarget: What is augmented reality (AR)?]</ref> | ||
<ref name="XRToday_enterprise"> | <ref name="XRToday_enterprise">[https://www.xrtoday.com/mixed-reality/meta-quest-for-enterprise-the-ultimate-guide-to-meta-quest-for-enterprise-applications/ XR Today – VR Passthrough in Enterprise] (2024)</ref> | ||
<ref name="XRToday_benefits"> | <ref name="XRToday_benefits">[https://www.xrtoday.com/virtual-reality/what-is-vr-passthrough-and-how-is-it-shaping-the-future-of-xr/ XR Today – What is VR Passthrough… (benefits)] (Dec 2024)</ref> | ||
<ref name="Index_Docs"> | <ref name="Index_Docs">[https://steamcdn-a.akamaihd.net/store/valve_index/ValveIndexManual032021.pdf Valve Index Hardware Manual (PDF)] (Mar 2021)</ref> | ||
<ref name="LynxR1">[https://www.lynx-r.com/ Lynx R1 Official Website]</ref> | <ref name="LynxR1">[https://www.lynx-r.com/ Lynx R1 Official Website]</ref> | ||
<ref name="KGuttag_Flamera"> | <ref name="KGuttag_Flamera">[https://kguttag.com/2023/09/26/apple-vision-pro-part-6-passthrough-mixed-reality-ptmr-problems/#flamera KGOnTech – Meta Flamera Light‑Field Passthrough] (Sept 26 2023)</ref> | ||
</references> | </references> | ||