Toggle search
Toggle menu
4.9K
5.5K
229
46.8K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

Mixed Reality Toolkit

From VR & AR Wiki
More actions
Mixed Reality Toolkit
Information
Type Software development kit
Industry Mixed reality, augmented reality, virtual reality
Developer Mixed Reality Toolkit Organization (MRTK3); Microsoft (HoloToolkit and MRTK v2)
Written In C#
License BSD 3-Clause (MRTK3); MIT License (MRTK v2)
Supported Devices HoloLens 2, Meta Quest, Magic Leap 2, Android XR devices, Lenovo ThinkReality A3; experimental: Windows Mixed Reality, SteamVR, Varjo XR-3
Release Date March 30, 2016 (HoloToolkit); August 21, 2019 (MRTK v2.0.0); September 6, 2023 (MRTK3 general availability)
Website https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity


The Mixed Reality Toolkit (MRTK) is an open-source software development kit for building mixed reality, augmented reality and virtual reality applications in the Unity game engine. It provides a cross-platform input system, building blocks for spatial interactions and user interface, and in-editor simulation for rapid prototyping.[1] Microsoft started the project as HoloToolkit for HoloLens development in 2016, rebranded the repository as MixedRealityToolkit-Unity in 2017, and released a rewritten second generation (MRTK v2) in 2019.[2][3][4]

The third generation, MRTK3, is built on Unity's XR Interaction Toolkit and OpenXR.[5] In January 2023 Microsoft cut the team that developed MRTK.[6] In August 2023 the project moved to an independent GitHub organization governed by a steering committee that included Microsoft, Magic Leap and Qualcomm, and MRTK3 reached general availability on September 6, 2023.[7][8] Its supported devices include HoloLens 2, Meta Quest, Magic Leap 2 and Android XR headsets.[5]

Reviewed 4 October 2026. Release dates, release notes, licensing, governance, supported devices and press claims checked against GitHub, Microsoft Learn, cited news pages and the ARETT paper. About review dates.

History

HoloToolkit (2016-2019)

The toolkit began as HoloToolkit-Unity. Microsoft's release notes for the HoloToolkit-Unity 1.5.4.0 release, published in November 2016, describe it as containing "all the contributions since HoloToolkit first launched on 30th March 2016".[2] The 1.2017.1.0 release for Unity 2017.1, published on August 12, 2017, announced that the repository branding had been updated to MixedRealityToolkit-Unity. Around the same time the toolkit gained support for Windows Mixed Reality immersive headsets: a developer release in July 2017 added Xbox controller input, motion controller support, teleportation and stage bounds for immersive applications.[3][9] Releases under the HoloToolkit name continued into early 2019, with HoloToolkit 2017.4.3.0 issued in November 2018 and a refresh in February 2019.[10]

MRTK v2 (2018-2022)

Microsoft's rewrite of the toolkit, first called "vNext", appeared as an alpha release on August 9, 2018. The alpha required Unity 2018.1 or later and introduced a new architecture with a core device abstraction layer and an SDK layer, an input and interaction system, teleportation, pointing rays and a boundary system. It supported Windows Mixed Reality and OpenVR.[11] After beta and release-candidate builds between October 2018 and June 2019, Microsoft published MRTK v2.0.0 on August 21, 2019, calling it "the first official release of Microsoft Mixed Reality Toolkit version 2". The release included prebuilt MRTK Examples Hub app packages for HoloLens, HoloLens 2 and immersive headsets.[4][10]

MRTK v2 includes support for articulated hand tracking and eye tracking on HoloLens 2, spatial awareness of the physical environment, speech and dictation input, and UX controls such as buttons, bounds controls, hand menus, near menus, slates and tooltips.[12] Microsoft continued the 2.x line through version 2.8. MRTK 2.8.0, released on May 31, 2022, improved support for the OpenXR pipeline and recommended Unity 2019.4 at minimum. The same release announced "MRTK3" as the name of the next version.[13] The last MRTK2 release on the repository's releases page, 2.8.3, was published on December 8, 2022.[14]

At Augmented World Expo 2021 in November 2021, Microsoft's Mixed Reality Toolkit for Unity won the Auggie Award for Best Developer Tool.[15]

MRTK3 preview and the 2023 layoffs

Microsoft opened the MRTK3 public preview on June 8, 2022, with preview updates planned roughly every two to four weeks until general availability. Microsoft introduced MRTK3 in sessions at its Mixed Reality Dev Days 2022 event.[1] In January 2023, after Microsoft cut as many as 10,000 jobs, The Register reported that the company "cut the entire team behind MRTK" and noted that a new release had been scheduled for February.[6]

Independent organization (2023 onward)

On August 21, 2023, Microsoft announced that MRTK3 had moved to an independent organization on GitHub. Robin Seiler, Microsoft's corporate vice president and chief operating officer of the Windows + Devices organization, said: "We believe that to be a true multi-platform toolkit that enables greater third-party collaboration, MRTK needs to be recognized as an independent organization that expands beyond Microsoft's sole jurisdiction." Microsoft stated that the move did not change its investment in MRTK3.[16][7] Magic Leap and Qualcomm joined Microsoft on the new organization's steering committee.[7][16]

The same day, preview 18 was the first release under the new organization. It removed the "Microsoft" prefix from namespaces (which now begin with MixedReality.Toolkit), renamed packages from com.microsoft.* to org.mixedrealitytoolkit.*, and relicensed the code under the BSD 3-Clause license, replacing the MIT License used before. Development moved to a new repository, while the old repository stayed under Microsoft's management for MRTK2; Microsoft said it would continue to address critical MRTK2 issues until MRTK2 is deprecated.[17][18]

The organization released MRTK3 for general availability on September 6, 2023. The release notes said it would be the final "multi-package" release, with later releases made per package.[8] In an October 2023 post, Qualcomm announced that it had joined the steering committee and that some of the original architects and team members of the MRTK project had joined its Snapdragon Spaces team. Qualcomm described MRTK as "largely a UI and interaction toolkit", while Snapdragon Spaces provides device enablement.[19]

Governance

The organization's charter makes the steering committee responsible for technical oversight, project approval, policy oversight and trademark management. It aims to decide by consensus and falls back to a vote when consensus fails. Members are added or removed by at least a three-quarters affirmative vote.[20] The committee list, last updated in September 2026, names members affiliated with Microsoft, Magic Leap, Qualcomm and Ultraleap, with one Magic Leap seat vacant. Ultraleap's representative was added in September 2024.[21]

Features

MRTK3 is built on Unity's XR Interaction Toolkit and the Unity Input System and is focused on OpenXR.[1] According to its documentation, it rewrote most features and systems, targets zero per-frame memory allocation, and is tuned for resource-constrained mobile platforms.[5] Interaction models include gaze-pinch indirect manipulation, and the UI system combines Unity Canvas with 3D UX and dynamic auto-layout. Gamepad, mouse and accessibility input share one 2D and 3D input path, and data binding handles theming and dynamic data.[1]

Its UX building blocks include a volumetric button that accepts poking, gaze-pinch, ray, mouse and gamepad input; bounds control and an object manipulator for one- and two-handed manipulation; hand menus and near menus; sliders; dialogs; slates; and solvers for object placement behaviors such as tag-along, body-lock, constant view size and surface magnetism.[5] A prerelease Figma toolkit gives 2D versions of the components for UI layout work.[1]

Starting with MRTK3 GA, each package is versioned on its own under Semantic Versioning. HoloToolkit and MRTK v2 instead shipped all packages under one version number. Microsoft's documentation notes that the "3" in MRTK3 marks the generation of the architecture, not a version number.[1] The GA release notes recommend obtaining MRTK3 through Microsoft's Mixed Reality Feature Tool.[8]

Package Identifier
Audio Effects org.mixedrealitytoolkit.audio
Core Definitions org.mixedrealitytoolkit.core
Diagnostics org.mixedrealitytoolkit.diagnostics
Extended Assets org.mixedrealitytoolkit.extendedassets
Input org.mixedrealitytoolkit.input
Spatial Manipulation org.mixedrealitytoolkit.spatialmanipulation
Standard Assets org.mixedrealitytoolkit.standardassets
Tools org.mixedrealitytoolkit.tools
UX Components org.mixedrealitytoolkit.uxcomponents
UX Components (Non-Canvas) org.mixedrealitytoolkit.uxcomponents.noncanvas
Windows Speech org.mixedrealitytoolkit.windowsspeech
Accessibility (early preview) org.mixedrealitytoolkit.accessibility

Package list from the project README.[5]

The README lists minimum requirements of OpenXR, Unity 2022.3 LTS and XR Interaction Toolkit 3.0.[5] The code is mainly written in C#.[5]

Supported devices

Platform Devices (MRTK3) Status
OpenXR Android XR Supported
OpenXR HoloLens 2 Supported
OpenXR Meta Quest Supported
OpenXR Magic Leap 2 Supported
OpenXR Lenovo ThinkReality A3 (with Snapdragon Spaces) Supported
OpenXR Windows Mixed Reality Experimental
OpenXR SteamVR Experimental
OpenXR Varjo XR-3 Experimental
Windows Flat-screen desktop Experimental

Source: project README.[5]

The project's Android XR deployment guide covers the Android XR hand mesh data feature for rendering the system hand mesh.[22] Input 3.3.0, released in November 2025, added a hand tracking permission for Android XR.[23] Magic Leap documents an early-access MRTK3 package for Magic Leap 2, listing controller, hand tracking, eye tracking, keyword recognition and meshing as pre-release features.[24]

For HoloLens 2, Microsoft's MRTK3 documentation recommends Unity 2022 with MRTK 2.8. It names core v3.2.2 as the last known good MRTK3 release tested on HoloLens by Microsoft.[1]

MRTK v2 reached devices through several Unity XR plug-ins. The Unity OpenXR plug-in covered HoloLens 2, Windows Mixed Reality headsets, Meta Quest and SteamVR devices; the Windows XR plug-in covered HoloLens and HoloLens 2; the Oculus XR plug-in covered Quest. ARCore and ARKit supported Android and iOS through AR Foundation, and the Ultraleap Leap Motion controller was supported for hand tracking.[12]

Release history

Release Date Notes
HoloToolkit launch March 30, 2016 Date given in the 1.5.4.0 release notes[2]
1.2017.1.0 August 12, 2017 Repository rebranded MixedRealityToolkit-Unity[3]
vNext Alpha 2018.7.0.0 August 9, 2018 First preview of the rewritten architecture[11]
MRTK v2.0.0 August 21, 2019 First official MRTK v2 release[4]
MRTK 2.8.0 May 31, 2022 Improved OpenXR support; MRTK3 name announced[13]
MRTK3 public preview June 8, 2022 Public preview opened[1]
MRTK 2.8.3 December 8, 2022 Final MRTK2 release[14]
MRTK3 preview 18 August 21, 2023 First release from the independent organization; BSD 3-Clause license[17]
MRTK3 v3.0.0 September 6, 2023 General availability[8]
Core v3.2.2 September 11, 2024 Last MRTK3 build tested on HoloLens by Microsoft[1]
Core v3.3.0 November 21, 2025 Latest stable core package as of October 2026[23]
Core v4.0.0-pre.3, Theming v1.0.0-pre.1 May 22, 2026 Pre-release; new theming package, minimum editor 6000.0.66f2[25]

Current development

The organization started publishing pre-release builds of a 4.0.0 generation of the core, input, spatial manipulation and UX packages in November 2025.[26] The 4.0.0 pre-release packages move to XR Interaction Toolkit 3.0.3 and add "controllerless" versions of the MRTK XR rig and controller prefabs.[27] The May 2026 pre-release added a UX theming package (org.mixedrealitytoolkit.theming) for switching themes at runtime across MRTK canvas UI, and raised the minimum Unity editor version to 6000.0.66f2.[25]

Related projects

Microsoft also published UX Tools for Unreal Engine, an Unreal Engine plugin with code, blueprints and example assets for mixed reality UX. Its documentation refers to building apps "with MRTK-Unreal and UX Tools". Its README says all OpenXR devices should be supported, although only HoloLens 2 and Windows Mixed Reality VR devices had been tested. Releases ran from version 0.8.0 in May 2020 to a 0.12.1 build for Unreal Engine 5.0 in October 2022, and the repository is now archived.[28]

Use in research

MRTK appears in academic work on HoloLens 2. In ARETT, an open-source eye tracking toolkit for head-mounted displays published in Sensors in 2021, Kapp and colleagues note that HoloLens 2 gaze data is available in Unity through MRTK, which "primarily focuses on enabling gaze-based interaction via an easy-to-use API". They also note that MRTK does not offer recordings for research purposes and does not guarantee a fixed sampling rate, which is tied to the Unity update rate. ARETT instead reads gaze data through the Universal Windows Platform API, which the authors say provides unsmoothed data and more stable data rates, but uses MRTK's spatial mapping for gaze ray hits on the real environment when that option is enabled.[29]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 "Mixed Reality Toolkit 3 Developer Documentation - MRTK3". Microsoft Learn. Microsoft. 2023-08-30. https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk3-overview/. Retrieved 2026-10-04.
  2. ↑ 2.0 2.1 2.2 "HoloToolkit-Unity Thanksgiving release compatible with Unity 5.4". GitHub. Microsoft. 2016-11-24. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v1.5.4.0. Retrieved 2026-10-04.
  3. ↑ 3.0 3.1 3.2 "MixedRealityToolkit-Unity for Unity 2017.1.0f3". GitHub. Microsoft. 2017-08-12. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v1.2017.1.0. Retrieved 2026-10-04.
  4. ↑ 4.0 4.1 4.2 "Microsoft Mixed Reality Toolkit v2.0.0". GitHub. Microsoft. 2019-08-21. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v2.0.0. Retrieved 2026-10-04.
  5. ↑ 5.0 5.1 5.2 5.3 5.4 5.5 5.6 5.7 "MixedRealityToolkit-Unity: Mixed Reality Toolkit for Unity". GitHub. Mixed Reality Toolkit Organization. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity. Retrieved 2026-10-04.
  6. ↑ 6.0 6.1 Jeff Burt (2023-01-24). "Microsoft's mixed reality idea meets the reality of job cuts". The Register. https://www.theregister.com/2023/01/24/microsoft_mixed_reality_cuts/. Retrieved 2026-10-04.
  7. ↑ 7.0 7.1 7.2 Laurent Giret (2023-08-21). "Microsoft Mixed Reality Toolkit Becomes an Independent Project". Thurrott.com. https://www.thurrott.com/microsoft/287452/microsoft-mixed-reality-toolkit-independent-qualcomm-magic-leap. Retrieved 2026-10-04.
  8. ↑ 8.0 8.1 8.2 8.3 "MRTK v3.0.0 GA". GitHub. Mixed Reality Toolkit Organization. 2023-09-06. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases/tag/v3.0.0. Retrieved 2026-10-04.
  9. ↑ "HoloToolkit-Unity for Unity MRTP9 with Windows Mixed Reality support". GitHub. Microsoft. 2017-07-26. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v1.Dev.MRTP9. Retrieved 2026-10-04.
  10. ↑ 10.0 10.1 "Releases - microsoft/MixedRealityToolkit-Unity". GitHub. Microsoft. https://github.com/microsoft/MixedRealityToolkit-Unity/releases. Retrieved 2026-10-04.
  11. ↑ 11.0 11.1 "Mixed Reality Toolkit vNext Alpha v2018.7.0.0". GitHub. Microsoft. 2018-08-09. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/2018.7.0.0-vNextAlpha. Retrieved 2026-10-04.
  12. ↑ 12.0 12.1 "MRTK2-Unity Developer Documentation - MRTK 2". Microsoft Learn. Microsoft. 2021-11-10. https://learn.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/mrtk2/. Retrieved 2026-10-04.
  13. ↑ 13.0 13.1 "Microsoft Mixed Reality Toolkit v2.8.0". GitHub. Microsoft. 2022-05-31. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v2.8.0. Retrieved 2026-10-04.
  14. ↑ 14.0 14.1 "Microsoft Mixed Reality Toolkit v2.8.3". GitHub. Microsoft. 2022-12-08. https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v2.8.3. Retrieved 2026-10-04.
  15. ↑ Sam Sprigg (2021-11-11). "2021 Auggie Award winners announced at Augmented World Expo". Auganix. https://www.auganix.org/2021-auggie-award-winners-announced-at-augmented-world-expo/. Retrieved 2026-10-04.
  16. ↑ 16.0 16.1 Sam Sprigg (2023-08-24). "Microsoft MRTK3 now an independent organization within GitHub". Auganix. https://www.auganix.org/xr-news-microsoft-announces-mixed-reality-toolkit-3-is-now-an-independent-organization-within-github-with/. Retrieved 2026-10-04.
  17. ↑ 17.0 17.1 "MRTK v3.0.0 Preview 18 Release & New Mixed Reality Toolkit Organization". GitHub. Mixed Reality Toolkit Organization. 2023-08-21. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases/tag/v3.0.0-pre.18. Retrieved 2026-10-04.
  18. ↑ "microsoft/MixedRealityToolkit-Unity". GitHub. Microsoft. https://github.com/microsoft/MixedRealityToolkit-Unity. Retrieved 2026-10-04.
  19. ↑ Maria Stenina (2023-10-04). "Qualcomm Joins MRTK Steering Committee". Snapdragon Spaces. Qualcomm. https://spaces.qualcomm.com/all/qualcomm-joins-mrtk-steering-committee/. Retrieved 2026-10-04.
  20. ↑ "Charter for the MixedRealityToolkit Organization". GitHub. Mixed Reality Toolkit Organization. https://github.com/MixedRealityToolkit/MixedRealityToolkit-MVG/blob/main/org-docs/CHARTER.md. Retrieved 2026-10-04.
  21. ↑ "Steering Committee". GitHub. Mixed Reality Toolkit Organization. https://github.com/MixedRealityToolkit/MixedRealityToolkit-MVG/blob/main/org-docs/STEERING-COMMITTEE.md. Retrieved 2026-10-04.
  22. ↑ "Deploy to an Android XR device". GitHub. Mixed Reality Toolkit Organization. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/blob/main/docs/mrtk3-overview/test-and-deploy/android-xr-deployment.md. Retrieved 2026-10-04.
  23. ↑ 23.0 23.1 "Audio v3.0.5, Core v3.3.0, Diagnostics v3.0.3, ExtendedAssets v3.0.4, Input v3.3.0, SpatialManipulation v3.4.0, StandardAssets v3.2.1, Tools v3.0.5, UXComponents v3.4.0, UXComponents.NonCanvas v3.1.5, UXCore v3.3.0, WindowsSpeech v3.0.4". GitHub. Mixed Reality Toolkit Organization. 2025-11-21. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases/tag/core-v3.3.0. Retrieved 2026-10-04.
  24. ↑ "Mixed Reality ToolKit 3 Overview". Magic Leap Developer Documentation. Magic Leap. https://developer-docs.magicleap.cloud/docs/guides/third-party/mrtk3/mrtk3-overview/. Retrieved 2026-10-04.
  25. ↑ 25.0 25.1 "Core v4.0.0-pre.3, StandardAssets v3.3.0-pre.1, Theming v1.0.0-pre.1, UXComponents v4.0.0-pre.3, UXCore v4.0.0-pre.3". GitHub. Mixed Reality Toolkit Organization. 2026-05-22. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases/tag/theming-v1.0.0-pre.1. Retrieved 2026-10-04.
  26. ↑ "Releases - MixedRealityToolkit/MixedRealityToolkit-Unity". GitHub. Mixed Reality Toolkit Organization. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases. Retrieved 2026-10-04.
  27. ↑ "Core v4.0.0-pre.1, Input v4.0.0-pre.1, SpatialManipulation v4.0.0-pre.1, UXComponents v4.0.0-pre.1, UXComponents.noncanvas v4.0.0-pre.1, UXCore v4.0.0-pre.1". GitHub. Mixed Reality Toolkit Organization. 2025-11-18. https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity/releases/tag/core-v4.0.0-pre.1. Retrieved 2026-10-04.
  28. ↑ "MixedReality-UXTools-Unreal: UX tools and components for developing Mixed Reality applications in UE4". GitHub. Microsoft. https://github.com/microsoft/MixedReality-UXTools-Unreal. Retrieved 2026-10-04.
  29. ↑ Sebastian Kapp, Michael Barz, Sergey Mukhametov, Daniel Sonntag, Jochen Kuhn (2021-03-23). "ARETT: Augmented Reality Eye Tracking Toolkit for Head Mounted Displays". Sensors, vol. 21, no. 6, article 2234. https://doi.org/10.3390/s21062234. Retrieved 2026-10-04.
Retrieved from "https://vrarwiki.com/index.php?title=Mixed_Reality_Toolkit&oldid=46443"