ARKit

ARKit
Information
Type	Software
Industry	Virtual reality, Augmented reality
Developer	Apple
Operating System	iOS, iPadOS, visionOS; macOS for remote Vision Pro rendering
Supported Devices	Compatible iPhone and iPad models; Apple Vision Pro; compatible Mac and Vision Pro combinations for remote rendering. Requirements vary by feature.
Release Date	Developer preview: June 5, 2017; public iOS 11 release: September 19, 2017
Website	https://developer.apple.com/documentation/arkit

Reviewed 12 October 2026. Framework history; mobile and visionOS APIs, requirements and permissions; remote rendering; historical demonstrations; three dated tracking studies. About review dates.

ARKit is Apple's framework for augmented reality and spatial sensing. It combines motion tracking, scene analysis and anchors that relate virtual content to a physical environment. Its iPhone and iPad interfaces support camera-based AR experiences; its visionOS interfaces expose selected tracking and scene data to immersive applications on Apple Vision Pro. The two platforms use different session APIs and permissions.

Introduction and development

Apple introduced ARKit in the iOS 11 developer preview on June 5, 2017, during Worldwide Developers Conference. The announcement described AR experiences built from an iPhone or iPad's camera, motion sensors and processors, including games, shopping and industrial design. Public availability followed with iOS 11 on September 19, 2017. The June developer preview and September operating-system release were separate events.

ARKit's capabilities expanded beyond its initial horizontal-surface placement. ARKit 1.5, released with iOS 11.3 on March 29, 2018, added vertical-surface detection and recognition of images such as posters and artwork. Later mobile interfaces added body tracking, people occlusion, collaborative sessions and LiDAR scene data, each with its own operating-system and hardware requirements.

ARKit on iPhone and iPad

An ARSession coordinates camera capture, motion-sensor readings and image analysis. An application runs the session with an ARConfiguration subclass suited to the experience, then receives frames, anchors and session events. ARKit supplies tracking data; an application uses a renderer to draw the associated virtual content.

Visual-inertial odometry and world tracking

Mobile world tracking uses visual-inertial odometry to combine camera-image features with motion-sensor data. ARWorldTrackingConfiguration tracks both translation and rotation in six degrees of freedom. This allows a virtual object to appear stationary in the environment as a person moves the device around it.

Tracking depends on the visible scene. Poor lighting, featureless surfaces, motion blur and excessive device movement can reduce quality. Initial plane detection estimates can change as ARKit gathers more observations, including a detected plane's position and extent. Applications can inspect tracking states and show guidance while tracking initializes or becomes limited.

Scene understanding and lighting estimation

World-tracking sessions can detect horizontal and vertical planes, recognize reference images and objects, and perform ray casts to find surfaces corresponding to a screen position. Plane anchors provide locations for content such as a virtual object on a table. A reference image must be supplied by the application; image-only tracking and world tracking with image detection have different behavior when that image leaves the camera view.

When light estimation is enabled, an ARFrame can include an ARLightEstimate with ambient intensity and color-temperature estimates. A renderer can use these estimates to make virtual lighting more consistent with the camera image. World-tracking configurations also expose environment texturing and environment-probe anchors for reflections.

Face and body tracking

ARFaceTrackingConfiguration uses the front camera and returns face anchors describing position, orientation, surface geometry and expressions. Apple documents TrueDepth camera requirements for iOS and iPadOS 13 and earlier, and support on devices with an Apple Neural Engine from iOS and iPadOS 14. Applications must check isSupported on the device. Face tracking exposes personal facial information, and Apple requires a privacy policy explaining the use of face data.

ARBodyTrackingConfiguration, available from iOS and iPadOS 13, uses the rear camera to track a detected person's body and supplies ARBodyAnchor data. Its frame semantics also provide detected 2D body-joint positions. Body tracking can be combined with plane or image detection to place an animated virtual character in the scene.

People occlusion, LiDAR and scene reconstruction

People occlusion prevents virtual content from drawing over regions occupied by a detected person. personSegmentation masks people regardless of their distance, while personSegmentationWithDepth allows a person to cover virtual content when the person is nearer to the camera. Apple identifies A12 and later devices as supporting people occlusion. Applications can check supportsFrameSemantics before enabling it.

On supported LiDAR devices, scene reconstruction estimates the physical environment as a polygonal mesh, divided among mesh anchors. Apple's reconstruction sample, available from iOS and iPadOS 13.4, demonstrates ray casting on nonplanar surfaces, mesh classification, occlusion and collisions between virtual content and real-world geometry. Plane detection can flatten parts of the reconstructed mesh where ARKit detects a plane.

The separate scene-depth interface provides per-frame depth maps, confidence values and a temporally smoothed depth option. Apple's sample requires a LiDAR Scanner and checks device support before requesting these frame semantics. Depth maps and reconstructed meshes are different representations; the sample's enlargement of a depth map interpolates its low-resolution data rather than adding new sensor observations.

Persistence, collaboration and location anchors

From iOS 12, an ARWorldMap can store world-tracking state and anchors for later relocalization. Restoring a map requires ARKit to recognize the environment again, and changes in lighting or scene features can prevent recovery. Saving a map therefore does not guarantee that content will return to its prior physical location.

Collaborative sessions exchange world-mapping information and participant anchors between nearby devices. The application enables collaboration on a world-tracking configuration and implements transmission of the resulting ARSession.CollaborationData. ARKit supplies the data, while the application chooses the networking framework and handles transport. The devices need overlapping observations so ARKit can merge their maps.

ARGeoTrackingConfiguration, available from iOS and iPadOS 14, relates location anchors to latitude, longitude and optional altitude. It combines GPS and other location information with Apple's previously captured localization imagery. Apple limits geotracking to supported outdoor locations, requires an internet connection, and documents A12 or later hardware with cellular/GPS capability. Applications must check both configuration support and availability at the requested location.

Device support and permissions

Apple's mobile setup documentation specifies iOS 11 or later and an A9 or later processor for the base framework, while warning that individual features need later systems or specific hardware. Applications should check the relevant configuration's isSupported property and feature-specific support methods. The original 2017 device list is consequently a historical list, rather than an exhaustive description of later ARKit support.

Mobile AR sessions require the person's camera permission and an NSCameraUsageDescription explanation in the application's information property list. The arkit required-device-capability key can restrict installation when AR is essential; an application with optional AR can instead test support at runtime and offer another interface.

ARKit for visionOS

On visionOS, ARKitSession runs a set of data providers that deliver anchors for individual features. These include WorldTrackingProvider, PlaneDetectionProvider, SceneReconstructionProvider, ImageTrackingProvider and HandTrackingProvider. Hand tracking supplies left- and right-hand anchors and skeletal-joint data that an application can use for custom gestures or content attached to a hand.

Access requires the application to present a Full Space, which can use a mixed immersion style; this does not require replacing the entire physical surroundings with a virtual scene. Hand tracking requires a hands-tracking usage description and authorization. Image tracking, plane detection and scene reconstruction require a world-sensing usage description and authorization. World tracking is distinct from world sensing and does not require that authorization. Applications must handle denied or revoked access and check provider support.

Dated spatial-computing additions

Introduction	Documented ARKit changes	Scope and qualifications
visionOS 2, presented at WWDC24	Room tracking, slanted-plane detection, object tracking, display-aligned hand updates and hand-anchor prediction.	Room tracking describes the currently occupied room and needs world-sensing authorization. Object tracking uses reference objects trained in Create ML from USDZ models. Apple's 2024 presentation describes statically placed objects. Predicted hand anchors trade some accuracy for lower delay.
visionOS 26, presented at WWDC25	RealityKit exposes ARKit anchor data through its spatial-tracking and anchor-event APIs.	A SpatialTrackingSession, AnchorStateEvents and ARKitAnchorComponent provide anchor transforms and other data to RealityKit applications.
visionOS 27 and iOS 27, presented at WWDC26	Higher-frequency tracking of moving reference objects, an extended Create ML training mode, and a choice between rendered and metric object-pose coordinate spaces on visionOS.	The iOS example uses ARWorldTrackingConfiguration, with detectionObjects for mostly stationary objects and trackingObjects for moving ones. It delivers ARObjectAnchor updates through an ARSessionDelegate, rather than adopting visionOS's data-provider API. These are version-specific additions described in the 2026 presentation.

Rendering and development tools

Apple's documentation includes RealityKit views for rendering AR content, as well as the older SceneKit and SpriteKit view interfaces. A custom renderer can use Metal. ARKit configuration choices also affect computation and energy use, so Apple's guidance recommends enabling only the features required by an experience.

Unity provides an Apple ARKit XR Plug-in for handheld AR through AR Foundation. The version 6.1 documentation identifies the implemented features and its Unity and Xcode requirements. Unreal Engine also provides an Apple ARKit plugin and documents supported handheld-AR workflows. An engine's support and toolchain requirements must be checked separately from the base framework's availability.

Apple extended ARKitSession to macOS 26 for remote immersive rendering to Vision Pro. In that workflow, a Mac application connects its session to a remote device identifier and uses world tracking to query Vision Pro's pose. Apple's WWDC25 demonstration combines ARKit with Compositor Services and Metal; it does not describe a Mac using its own camera as a substitute for the headset's tracking sensors.

Uses and early demonstrations

The 2017 developer-preview period produced experimental applications in painting, measurement, games, product visualization and education. Contemporary coverage showed Normal's demonstration connecting a HTC Vive painting experience to an ARKit-enabled iPad, Wingnut AR's noninteractive battlefield presentation using Unreal Engine 4, and a solar-system demonstration by the developer krutosh. The same coverage described Jelmer Verhoog's vehicle-preview prototype depicting a Tesla Model 3. These demonstrations documented applications during the preview period; they are not a list of current released products.

Smart Picture 3D's Measurement Cloud demonstrated room measurement, while Nexus Interactive Arts experimented with an iPhone 7's ARKit tracking and passthrough imagery in a Google Cardboard headset. These are historical examples of ARKit's use in measurement and VR experiments; they do not establish measurement accuracy or compatibility with later devices.

Tracking evaluation and limits

Independent research has evaluated ARKit with different hardware, trajectories and reference measurements. Results need to retain those conditions: a measured trajectory error is not an accuracy specification for every application or later version.

Study	ARKit setup and method	Interpretation
ADVIO, ECCV 2018	iPhone 6s running iOS 11.0.3; 23 indoor and outdoor recordings. Reference trajectories combined calibrated inertial data with manually identified position fixes.	The authors reported mostly successful tracking along with failure cases and severe drift in some outdoor sequences. Their reference was an estimated trajectory, with uncertainty in the manual fixes. The experiment concerns the tested iPhone 6s and iOS 11.0.3 setup.
Marino and colleagues, Sensors, 2022	Eight mobile devices, including an iPad Pro and iPhone 11, tested with ARKit or ARCore in a laboratory in 2021. An external OptiTrack system supplied comparison poses for three paths, each repeated ten times.	The translational absolute trajectory error and relative pose error measures differed with device and path. The authors limit their conclusions to the tested systems and laboratory trajectories. Their suggestion that LiDAR contributed to the iPad's results is not an isolated measurement of LiDAR's contribution.
Kim and colleagues, 2022 preprint	iPhone 12 Pro Max running iOS 14.7.1, compared with ARCore, RealSense T265 and ZED 2 in six environments. Tests measured final drift after returning to the start point and compared trajectory shapes with maps.	Endpoint drift is different from error measured against a continuously observed ground-truth trajectory. These results describe the particular devices and tests; the paper's vehicle experiments were outside the officially supported uses of the tested systems.