Head-mounted display: Difference between revisions - VR & AR Wiki

Line 129:

*'''Display Panels & Optics''': Generate the visual output and direct it to the eyes (see Key Technical Specifications above).

* '''[[Sensor]]s''': Detect movement, user input, and sometimes the environment.

*'''[[Sensor]]s''': Detect movement, user input, and sometimes the environment.

* [[Inertial Measurement Unit]] (IMU): Core component for rotational (3DoF) tracking. Essential for low-latency orientation updates.

**[[Inertial Measurement Unit]] (IMU): Core component for rotational (3DoF) tracking. Essential for low-latency orientation updates.

* [[Camera]]s: Crucial for modern HMDs. Can include:

**[[Camera]]s: Crucial for modern HMDs. Can include:

* Visible light cameras: Used for inside-out positional (6DoF) tracking, [[video passthrough]] (especially color passthrough), environment mapping, [[Hand tracking|hand tracking]].

***Visible light cameras: Used for inside-out positional (6DoF) tracking, [[video passthrough]] (especially color passthrough), environment mapping, [[Hand tracking|hand tracking]].

* [[Infrared]] (IR) cameras: Often used for inside-out tracking (less susceptible to ambient light changes), controller tracking (detecting IR LEDs on controllers), and [[eye tracking]].

***[[Infrared]] (IR) cameras: Often used for inside-out tracking (less susceptible to ambient light changes), controller tracking (detecting IR LEDs on controllers), and [[eye tracking]].

* [[Depth-sensing Cameras|Depth Sensors]]: (e.g., [[Time-of-flight camera|Time-of-Flight]], Structured Light, Active Stereo IR). Used in some AR/MR HMDs (like HoloLens, Vision Pro) for accurate [[Spatial analysis|spatial mapping]], environment understanding, hand tracking, and occlusion.

**[[Depth-sensing Cameras|Depth Sensors]]: (e.g., [[Time-of-flight camera|Time-of-Flight]], Structured Light, Active Stereo IR). Used in some AR/MR HMDs (like HoloLens, Vision Pro) for accurate [[Spatial analysis|spatial mapping]], environment understanding, hand tracking, and occlusion.

* [[Eye tracking|Eye Tracking Cameras]]: Small internal IR cameras pointed at the user's eyes to monitor gaze direction and pupil characteristics.

**[[Eye tracking|Eye Tracking Cameras]]: Small internal IR cameras pointed at the user's eyes to monitor gaze direction and pupil characteristics.

* '''Processors''':

*'''Processors''':

* CPU/GPU: Handle rendering, tracking calculations, application logic (essential in standalone HMDs, often a mobile [[System on a chip|SoC]] like [[Qualcomm Snapdragon Spaces|Snapdragon XR series]]).

**CPU/GPU: Handle rendering, tracking calculations, application logic (essential in standalone HMDs, often a mobile [[System on a chip|SoC]] like [[Qualcomm Snapdragon Spaces|Snapdragon XR series]]).

* Specialized Processors: May include [[Vision Processing Unit]]s (VPUs) or [[Neural Processing Unit]]s (NPUs) to efficiently handle computer vision tasks (SLAM, hand/eye tracking) or [[Artificial intelligence|AI]] workloads, offloading the main CPU/GPU. Microsoft's [[Holographic Processing Unit]] (HPU) in HoloLens is an example.<ref name="HoloLensSensors">Microsoft Learn. "HoloLens 2 hardware". Retrieved 2023-10-27. [https://learn.microsoft.com/en-us/hololens/hololens2-hardware Link]</ref>

**Specialized Processors: May include [[Vision Processing Unit]]s (VPUs) or [[Neural Processing Unit]]s (NPUs) to efficiently handle computer vision tasks (SLAM, hand/eye tracking) or [[Artificial intelligence|AI]] workloads, offloading the main CPU/GPU. Microsoft's [[Holographic Processing Unit]] (HPU) in HoloLens is an example.<ref name="HoloLensSensors">Microsoft Learn. "HoloLens 2 hardware". Retrieved 2023-10-27. [https://learn.microsoft.com/en-us/hololens/hololens2-hardware Link]</ref>

* '''Memory & Storage''': [[RAM]] for active processing and onboard [[Flash memory|storage]] (in standalone HMDs) for the operating system, applications, and media.

*'''Memory & Storage''': [[RAM]] for active processing and onboard [[Flash memory|storage]] (in standalone HMDs) for the operating system, applications, and media.

* '''Audio System''': Integrated speakers, microphones, headphone jacks (see Key Technical Specifications above).

*'''Audio System''': Integrated speakers, microphones, headphone jacks (see Key Technical Specifications above).

* '''Connectivity Hardware''': Wi-Fi, [[Bluetooth]] radios (especially in standalone HMDs), [[USB]] ports, video input ports (in tethered HMDs).

*'''Connectivity Hardware''': Wi-Fi, [[Bluetooth]] radios (especially in standalone HMDs), [[USB]] ports, video input ports (in tethered HMDs).

* '''Power System''': [[Battery]] (in standalone or wireless HMDs), power regulation circuitry, charging ports (often USB-C).

*'''Power System''': [[Battery]] (in standalone or wireless HMDs), power regulation circuitry, charging ports (often USB-C).

* '''Mechanical Structure & Ergonomics''': Housing, straps, facial interface, IPD adjustment mechanisms, thermal management (fans, heat sinks).

*'''Mechanical Structure & Ergonomics''': Housing, straps, facial interface, IPD adjustment mechanisms, thermal management (fans, heat sinks).

* '''Input Mechanisms''': How the user interacts with the system.

*'''Input Mechanisms''': How the user interacts with the system.

* [[Motion Controllers]]: Handheld devices tracked in 3D space (usually 6DoF), typically including buttons, triggers, joysticks/touchpads, and [[Haptic technology|haptic feedback]]. Primary input for most VR systems (e.g., Meta Touch controllers, Valve Index controllers, PS VR2 Sense controllers).

**[[Motion Controllers]]: Handheld devices tracked in 3D space (usually 6DoF), typically including buttons, triggers, joysticks/touchpads, and [[Haptic technology|haptic feedback]]. Primary input for most VR systems (e.g., Meta Touch controllers, Valve Index controllers, PS VR2 Sense controllers).

* [[Hand tracking]]: Camera-based systems that track the user's bare hands and finger movements without requiring controllers. Offers natural interaction but lacks physical buttons and haptic feedback. Increasingly standard on standalone VR/MR headsets (Quest series, Vision Pro).

**[[Hand tracking]]: Camera-based systems that track the user's bare hands and finger movements without requiring controllers. Offers natural interaction but lacks physical buttons and haptic feedback. Increasingly standard on standalone VR/MR headsets (Quest series, Vision Pro).

* [[Eye tracking]]: Used for gaze-based selection, foveated rendering, and social presence.

**[[Eye tracking]]: Used for gaze-based selection, foveated rendering, and social presence.

* [[Voice Commands]]: Using built-in microphones and [[Speech recognition|speech recognition software]] for hands-free control.

**[[Voice Commands]]: Using built-in microphones and [[Speech recognition|speech recognition software]] for hands-free control.

* [[Brain-Computer Interface|Brain-Computer Interfaces]] (BCI): Experimental interfaces reading neural signals, potentially via electrodes integrated into the HMD, for direct thought control. Still largely in research phases for consumer HMDs (e.g., [[NextMind]], [[CTRL-labs]] research).

**[[Brain-Computer Interface|Brain-Computer Interfaces]] (BCI): Experimental interfaces reading neural signals, potentially via electrodes integrated into the HMD, for direct thought control. Still largely in research phases for consumer HMDs (e.g., [[NextMind]], [[CTRL-labs]] research).

==Applications==