ARCore
More actions
| ARCore | |
|---|---|
| Information | |
| Type | Software Development Kit |
| Industry | Augmented reality |
| Developer | |
| Operating System | Android; selected APIs for iOS |
| License | Component-specific terms; Apache 2.0 for most SDK source files and samples |
| Supported Devices | Certified Android devices; feature-specific requirements apply |
| Release Date | 29 August 2017 (preview); 23 February 2018 (1.0) |
| Website | https://developers.google.com/ar/ |
ARCore is Google's platform for building augmented reality experiences, with software development kits and APIs that connect virtual content to a phone's camera view. Its basic capabilities are motion tracking, environmental understanding and light estimation. Google provides development paths for native Android apps, Unity, Unreal Engine and the web, with selected APIs also available on iOS.[1]
ARCore's initial phone AR implementation built on work from Tango, using existing phone hardware rather than requiring additional hardware. Google released its developer preview in August 2017 and version 1.0 in February 2018. Android version alone does not establish compatibility: Google certifies individual device models, and features such as Depth and Geospatial have further requirements.[2][3][4]
Devices that support ARCore
Certification checks a device's camera, motion sensors, hardware design and processing capability. The supported-device list gives model-specific minimum Android versions and capabilities, including Depth API support. It also separates devices using Google Play, supported devices distributed in China and iOS devices for selected features.[4]
| Context | Requirements and limits |
|---|---|
| Android with Google Play | A listed device must originally have shipped with the Google Play Store and run the minimum Android version specified for its model. Where the list gives no other minimum, Android 7.0 or later is required.[4] |
| AR Required app | The app needs ARCore for basic functionality. Google Play restricts its availability to supported devices and installs Google Play Services for AR alongside it. The documented minimum SDK level is 24.[5] |
| AR Optional app | The app retains functionality without ARCore. The documented minimum SDK level can be 19, but its AR functionality still needs a supported device running at least Android 7.0. The app must arrange installation of Google Play Services for AR when needed.[5] |
| Depth API | Support is a subset of ARCore-compatible Android devices. Apps must check depth support and enable it in the session; depth is disabled by default.[6] |
| Geospatial API | ARCore compatibility does not imply Geospatial compatibility. Android apps must check the feature separately, configure the service and obtain precise location permission.[7] |
On Android, the SDK communicates with Google Play Services for AR, which Google Play updates separately from the app. Both AR Required and AR Optional apps still perform runtime checks for device support and installation of a compatible service version before creating an AR session. They also require camera permission.[5]
How does ARCore work?
ARCore combines camera feature points with measurements from the phone's inertial measurement unit to estimate camera position and orientation. Google describes this as simultaneous localization and mapping. An app uses that pose for its virtual camera so that rendered content follows the perspective of the physical camera.[8]
| Capability | Information available to an app |
|---|---|
| Motion tracking | The estimated camera pose, comprising position and orientation relative to the world.[8] |
| Environmental understanding | Detected feature points and geometric planes, including plane boundaries. Flat surfaces with little texture can be difficult to detect.[8] |
| Light estimation | Lighting information for rendering virtual objects. Ambient Intensity supplies average pixel intensity and color correction; Environmental HDR supplies a main directional light, ambient spherical harmonics and an HDR cubemap.[9] |
A hit-test projects a ray from a screen coordinate into the camera's view and returns intersections with detected geometry. Apps can use the intersection pose to place an object on a surface.[8] Lighting estimates provide inputs to the app's rendering system; the app uses them to render effects such as shadows and reflections.[9]
Anchors and tracking limits
An anchor is a tracked position and orientation to which an app attaches virtual content. ARCore can revise its estimate of world coordinates as it gathers observations; an anchor's pose can therefore change. Objects that share an anchor retain their position and orientation relative to each other through those updates. Anchors can attach to a trackable such as a plane, or be created in the ARCore session's world space.[10]
Google recommends reusing anchors for nearby objects, detaching unused anchors and avoiding objects more than 8 meters from their anchor because world-coordinate updates can produce unexpected rotational movement. Local anchors are valid for the particular app instance; Cloud Anchors and Geospatial anchors provide different ways to share or geographically position content.[10]
Textureless surfaces, dim or very bright lighting, transparent or reflective materials and moving surfaces can hinder environmental understanding. Applications need to communicate tracking problems and guide users toward suitable conditions.[11]
Depth and occlusion
The Depth API produces depth maps using depth from motion: it compares camera views captured as the device moves. It can combine this information with a supported hardware depth sensor, such as a time-of-flight sensor, but a dedicated depth sensor is not required. Google describes the most accurate results as occurring about 0.5-5 meters from the scene and notes that low-texture surfaces can yield imprecise depth.[12]
Apps can compare real-world depth with the depth of rendered objects to implement occlusion, hiding virtual content behind nearer physical objects. Depth also supports surface collisions, distance-based effects and placement on non-planar surfaces.[12] The Android depth image stores depth in millimeters; its value describes distance projected onto the camera's principal axis, rather than distance along the viewing ray. Depth data can be unavailable before sufficient motion or when tracking is lost.[6]
Raw Depth supplies a depth image and a corresponding confidence image. Some pixels have no valid estimate. The full Depth API instead supplies a smoothed image, using interpolation to fill depth estimates. Google describes raw estimates as more accurate for the pixels where data is available; the two outputs serve different rendering and geometry-processing needs.[13]
Image recognition
Augmented Images recognizes supplied reference images, such as posters or packaging, and estimates their position and orientation. It uses the supplied physical size, or estimates size if the app does not provide it. The reference database stores extracted grayscale features. Google documents up to 20 simultaneously tracked images and up to 1,000 references per database, with one database active at a time. Detection and tracking run on the device without an internet connection. Initial detection requires a flat image occupying at least 25% of the camera frame; repeated patterns and images with few distinctive features can perform poorly.[14]
Cloud Anchors and Geospatial
Cloud Anchors share a local spatial reference between app instances, including compatible Android and iOS devices. Hosting uploads anchor data to Google's ARCore API endpoint and returns an identifier that the app distributes. A resolving device observes the same physical environment, and the service compares its visual features with the stored 3D feature map. Hosting and resolving require an internet connection. Google describes Cloud Anchors as suited to room-scale shared or persistent AR experiences.[15]
The Geospatial API combines device sensors and GPS with Google's Visual Positioning System (VPS), whose localization model is derived from Google Street View images. It relates local tracking coordinates to geographic coordinates and lets developers attach content to locations remotely. VPS localization compares recognizable visual features from the device with that model.[16]
VPS coverage and conditions matter. Google's Android guide recommends checking availability at the intended location; where GPS is weak, the API depends on VPS coverage for accurate poses. The guide also requires location permissions and internet access, and states that approximate location alone cannot be used for the Geospatial API.[7]
| Geospatial anchor type | Altitude reference |
|---|---|
| WGS84 | An altitude in meters above the WGS84 ellipsoid, supplied by the app. Ground level is not necessarily zero.[17] |
| Terrain | A height relative to ground or floor known to VPS at the chosen latitude and longitude. Terrain resolution uses the service.[17] |
| Rooftop | A height relative to a building's rooftop, resolved using the service. Locations away from buildings fall back to terrain altitude.[17] |
History of ARCore
On 29 August 2017, Google announced the ARCore developer preview for existing Android phones, initially naming the Pixel and Samsung Galaxy S8. The announcement identified Java/OpenGL, Unity and Unreal development paths and described ARCore as building on technology developed for Tango.[2]
Version 1.0 left preview on 23 February 2018. Google said it supported 13 phone models at launch and enabled developers to publish AR apps on Google Play. Those launch devices and capabilities are historical, rather than a complete current compatibility list.[3] On 25 June 2020, Google announced the Depth API in ARCore 1.18 for Android and Unity, including AR Foundation.[18]
Developers and development environments
Google documents Kotlin/Java and native C development on Android, Unity integration, Unreal Engine integration and web AR through WebXR. Availability of a particular capability depends on the platform and integration.[1]
In Unity, AR Foundation supplies components such as plane detection, local anchors, raycasting, point clouds and occlusion. ARCore Extensions exposes additional ARCore functionality, including Cloud Anchors. Android builds use the ARCore XR plugin provider, while iOS builds use the ARKit XR plugin.[19]
Google's iOS integration includes Cloud Anchors, Geospatial and Augmented Faces. For Cloud Anchors and Geospatial, the documented workflow runs an ARKit ARSession and passes its ARFrame objects into Google's GARSession.[20] Augmented Faces uses a separate GARAugmentedFaceSession; the documented sample captures front-camera images with AVCaptureSession and passes image buffers, timestamps and recognition rotation into that face session.[21]
Licensing and services
The Android SDK's license distinguishes components. Most SDK source files and samples are Apache 2.0 licensed, except files covered by the other sections. The C API header, listed arcoreimg executables, release APKs and Maven com.google.ar:core AAR files are governed by the ARCore Additional Terms of Service. Dependencies have their own license notices.[22]
The Additional Terms, last modified 11 May 2022, apply alongside the Google APIs Terms of Service. They include user disclosure requirements and prohibit Cloud Anchors and Geospatial use in applications deemed directed to children under the Children's Online Privacy Protection Act.[23]
Permissions and privacy
Camera permission is required before creating an Android AR session.[5] Geospatial additionally requires precise location access. On Android 12 and later, the app declares both fine and coarse location permissions to accommodate Android's permission choices, but Geospatial cannot operate with only approximate location.[7]
Google requires apps to prominently disclose their use of Google Play Services for AR and its data processing. Cloud Anchors and Geospatial have an additional disclosure at API activation about Google's processing of sensor data, including camera and location data.[24] Google's user documentation describes collection of data to power applicable features and for analytics, and directs users to apps' Google Play Data safety sections for applicable disclosures.[25]
Uses of ARCore
Early Daydream Labs prototypes
In September 2017, Google published six Daydream Labs examples while ARCore was in preview. These were experiments and prototypes, rather than a list of shipping ARCore features or proof of current app availability.[26]
| Prototype | Demonstrated interaction |
|---|---|
| Street View window | Viewing British Museum and Great Russell Street panoramas through an AR window.[26] |
| Architectural visualization | Overlaying a model on a construction site to show a proposed finished home.[26] |
| Espresso Trainer | Instructions for making espresso, adapted from a VR prototype.[26] |
| Street View movement | Walking in physical space to change virtual position in a panorama.[26] |
| Content emphasis | Changing depth of field and camera-feed desaturation with user movement.[26] |
| VPS position sharing | An experiment using Google's then-beta Visual Positioning Service to guide a friend to a shared position.[26] |
Depth research and interaction
Valentin and colleagues' 2018 Depth from Motion for Smartphone AR presented a pipeline for dense depth estimation from a single moving color camera. Its research included keyframe selection, stereo matching, depth post-processing and a late rendering step for occlusion. The authors identified limitations from camera-pose errors, rolling shutter, motion blur and ambiguity in low-texture regions.[27]
The 2020 DepthLab paper described a Unity library that processed ARCore depth maps into localized, surface and dense-depth representations. Its examples included occlusion, shadows, collisions, scene relighting and depth-of-field effects without requiring a persistent reconstructed surface model. Performance tests used a Pixel 3 and Unity 2019.2; those measurements describe the tested research implementation, rather than a universal performance specification for ARCore.[28]
References
- ↑ 1.0 1.1 "Overview of ARCore and supported development environments". Google for Developers. https://developers.google.com/ar/develop. Retrieved 2026-10-12.
- ↑ 2.0 2.1 Dave Burke (2017-08-29). "ARCore: Augmented reality at Android scale". The Keyword. https://blog.google/products-and-platforms/products/google-ar-vr/arcore-augmented-reality-android-scale/. Retrieved 2026-10-12.
- ↑ 3.0 3.1 Anuj Gosalia (2018-02-23). "Announcing ARCore 1.0 and new updates to Google Lens". The Keyword. https://blog.google/products-and-platforms/products/google-ar-vr/announcing-arcore-10-and-new-updates-google-lens/. Retrieved 2026-10-12.
- ↑ 4.0 4.1 4.2 "ARCore supported devices". Google for Developers. https://developers.google.com/ar/devices. Retrieved 2026-10-12.
- ↑ 5.0 5.1 5.2 5.3 "Enable AR in your Android app". Google for Developers. https://developers.google.com/ar/develop/java/enable-arcore. Retrieved 2026-10-12.
- ↑ 6.0 6.1 "Use Depth in your Android app". Google for Developers. https://developers.google.com/ar/develop/java/depth/developer-guide. Retrieved 2026-10-12.
- ↑ 7.0 7.1 7.2 "Enable the Geospatial API for your Android app (Kotlin/Java)". Google for Developers. https://developers.google.com/ar/develop/java/geospatial/enable. Retrieved 2026-10-12.
- ↑ 8.0 8.1 8.2 8.3 "Fundamental concepts". Google for Developers. https://developers.google.com/ar/develop/fundamentals. Retrieved 2026-10-12.
- ↑ 9.0 9.1 "Get the lighting right". Google for Developers. https://developers.google.com/ar/develop/lighting-estimation. Retrieved 2026-10-12.
- ↑ 10.0 10.1 "Working with Anchors". Google for Developers. https://developers.google.com/ar/develop/anchors. Retrieved 2026-10-12.
- ↑ "Environment". Google for Developers. https://developers.google.com/ar/design/environment/definition. Retrieved 2026-10-12.
- ↑ 12.0 12.1 "Depth adds realism". Google for Developers. https://developers.google.com/ar/develop/depth. Retrieved 2026-10-12.
- ↑ "Use Raw Depth in your Android app". Google for Developers. https://developers.google.com/ar/develop/java/depth/raw-depth. Retrieved 2026-10-12.
- ↑ "Add dimension to images". Google for Developers. https://developers.google.com/ar/develop/augmented-images. Retrieved 2026-10-12.
- ↑ "Cloud Anchors allow different users to share AR experiences". Google for Developers. https://developers.google.com/ar/develop/cloud-anchors. Retrieved 2026-10-12.
- ↑ "Build global-scale, immersive, location-based AR experiences with the ARCore Geospatial API". Google for Developers. https://developers.google.com/ar/develop/geospatial. Retrieved 2026-10-12.
- ↑ 17.0 17.1 17.2 "Use Geospatial anchors to position real-world content on Android SDK (Kotlin/Java)". Google for Developers. https://developers.google.com/ar/develop/java/geospatial/anchors. Retrieved 2026-10-12.
- ↑ Rajat Paharia (2020-06-25). "A new wave of AR Realism with the ARCore Depth API". Google Developers Blog. https://developers.googleblog.com/a-new-wave-of-ar-realism-with-the-arcore-depth-api/. Retrieved 2026-10-12.
- ↑ "AR Foundation and ARCore Extensions for Unity capabilities and features". Google for Developers. https://developers.google.com/ar/develop/unity-arf/features. Retrieved 2026-10-12.
- ↑ "Configure an ARCore session in iOS". Google for Developers. https://developers.google.com/ar/develop/ios/session-config. Retrieved 2026-10-12.
- ↑ "Augmented Faces developer guide for iOS". Google for Developers. https://developers.google.com/ar/develop/ios/augmented-faces/developer-guide. Retrieved 2026-10-12.
- ↑ "ARCore SDK for Android: LICENSE". GitHub. https://github.com/google-ar/arcore-android-sdk/blob/main/LICENSE. Retrieved 2026-10-12.
- ↑ "ARCore Additional Terms of Service". Google for Developers. 2022-05-11. https://developers.google.com/ar/develop/terms. Retrieved 2026-10-12.
- ↑ "User privacy requirements". Google for Developers. https://developers.google.com/ar/develop/privacy-requirements. Retrieved 2026-10-12.
- ↑ "How Google Play Services for AR handles your data". Google AR Help. https://support.google.com/ar/answer/12148145. Retrieved 2026-10-12.
- ↑ 26.0 26.1 26.2 26.3 26.4 26.5 26.6 Chris Kelley (2017-09-11). "Daydream Labs: Experiments with ARCore". The Keyword. https://blog.google/products-and-platforms/products/google-ar-vr/daydream-labs-experiments-arcore/. Retrieved 2026-10-12.
- ↑ Julien Valentin and colleagues (2018). "Depth from Motion for Smartphone AR". ACM Transactions on Graphics 37 (6): Article 193, 1-19. doi:10.1145/3272127.3275041. Retrieved 2026-10-12.
- ↑ Ruofei Du and colleagues (2020). "DepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality". Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. pp. 829-843. doi:10.1145/3379337.3415881. https://storage.googleapis.com/gweb-research2023-media/pubtools/5679.pdf. Retrieved 2026-10-12.