Fiducial markers

Early Lighthouse prototype, an inside-out tracking system with QR codes as fiducial markers.

Fiducial markers are visible objects placed in the field of view of an imaging system, usually a camera, to act as fixed points of reference. In computer vision and augmented reality, a fiducial marker is a printed or physical pattern with high contrast and a known geometry that detection software can find reliably in a camera image. Because the real size and shape of the marker are known in advance, the software can measure where the marker appears in the picture and work backward to recover the camera's position and orientation, known together as its pose, relative to the marker. The same idea lets a system overlay virtual content on the real world or track a moving object in three dimensions.

In VR and AR, fiducial markers are used for positional tracking of HMDs and Input Devices. They can be used in both inside-out tracking and outside-in tracking.

In inside-out tracking, markers such as QR codes can be placed in the environment to serve as reference points that an outward-facing camera on the headset or device detects to locate itself in the room.

In outside-in tracking, markers such as light-emitting diodes can be placed onto the object so it can be tracked by one or more external cameras. The camera or cameras follow the movement and position of the marker to figure out its position and orientation in 3D space. Different markers can be placed on different objects so they can be tracked independently.

How fiducial markers work

A fiducial marker is built so that it stands out clearly from a normal scene and is hard to confuse with anything else. Most marker systems used in AR and robotics are flat, square, black and white patterns. The high contrast between black and white makes the marker easy to find under a wide range of lighting, and the straight edges of the square give the software well defined corners to work with.

The general detection pipeline is similar across systems. The software first converts the camera frame to a form where the marker stands out, often by thresholding the image into black and white or by looking for sharp edges. It then locates candidate four-sided shapes, identifies the marker's corners, and reads the pattern inside the border to confirm the marker and find its identity. Because the four corners of a known square map to known points in the real world, the system can solve for the geometric transformation between the marker and the camera. This recovers the full six degrees of freedom pose, which is three values for position and three for orientation, from a single image.

Many marker systems carry a binary code inside the square border rather than a free-form picture. The code acts as an identifier, so a system can tell dozens or thousands of distinct markers apart, and it can include checksums and error correction so that a misread or partly hidden marker is rejected instead of being confused for another one.

Major marker systems

ARToolKit

ARToolKit is one of the earliest and most widely used marker systems for augmented reality. It was developed by Hirokazu Kato of the Nara Institute of Science and Technology in 1999 and released publicly through the University of Washington's HIT Lab. An ARToolKit marker is a square with a thick black border surrounding an image pattern. The software looks for the square border in the video stream, then compares the pattern inside against a database of stored templates by correlation to identify the marker. Once a marker is found, the library calculates the camera's position and orientation relative to it in real time, so a virtual object can be drawn as if it sits on the marker. ARToolKit was released as open source and its robust performance led to many applications across AR and computer vision research.

ARTag

ARTag is a marker system introduced by Mark Fiala in 2005 as an improvement on ARToolKit. Instead of matching free-form picture templates, ARTag markers are bi-tonal square patterns that encode a unique ID number using digital coding methods, including checksums and forward error correction. This design gives ARTag very low and measurable false positive and inter-marker confusion rates, removes the need to store and correlate image templates, and does not require a fixed grayscale threshold. ARTag could encode a large library of unique IDs and was the first marker system to detect markers from their edges and to use forward error correction.

AprilTag

AprilTag is a visual fiducial system introduced by Edwin Olson in a 2011 paper at the IEEE International Conference on Robotics and Automation. An AprilTag uses a 2D barcode style square pattern that allows full six degrees of freedom localization from a single image. Compared with earlier systems it adds a fast and robust line detection stage, a stronger digital coding scheme, and greater robustness to occlusion, image warping, and lens distortion. The coding scheme uses a modified lexicode method and rejects codes that are too simple, which would otherwise look like ordinary geometric patterns. AprilTag is fully open and is widely used in robotics for tasks such as navigation, ground truthing, and camera calibration.

ArUco

ArUco is a square marker system originally developed in 2014 by S. Garrido-Jurado and colleagues at the University of Cordoba in Spain, and the name stands for Augmented Reality University of Cordoba. An ArUco marker is a synthetic square made of a wide black border around an inner binary matrix that encodes the marker's identifier. The black border allows fast detection in the image, and the binary code allows identification together with error detection and correction. Markers are organized into dictionaries, where a dictionary is a set of markers of the same size and a marker's ID is simply its index within that dictionary. ArUco is included in the OpenCV computer vision library through its ArUco module, which makes it a common choice for marker based pose estimation.

QR codes and data markers

QR codes and similar 2D data matrix codes are another family of patterns that can serve as fiducial markers. They were designed mainly to store data rather than to recover pose, so a QR code can hold a web address or other payload that an AR application reads when the code is seen. Because a QR code is a high contrast square pattern with recognizable corner finder patterns, it can also be used as a reference point for positioning, as in the early Lighthouse prototype pictured above, which used QR codes placed in the environment. In practice, dedicated systems such as ARTag, AprilTag, and ArUco are usually preferred for precise tracking, because their codes are tuned for reliable detection, low confusion, and accurate pose estimation rather than for carrying large amounts of data.

Marker based versus markerless tracking

Tracking systems that use fiducial markers are described as marker based, while systems that work without any added markers are called markerless. The two approaches trade off in predictable ways.

Marker based tracking is robust and cheap. A marker is just a printed pattern, so it costs almost nothing to produce, and it gives precise and stable positioning as long as the marker is clearly visible to the camera at a reasonable size, distance, and angle. The main limitation is that someone has to place the marker in the scene in advance, and the camera has to actually see it. If the marker is missing, obscured, or poorly lit, tracking can fail.

Markerless tracking removes the need to prepare the environment by detecting natural features such as edges, textures, and surfaces already present in the scene. This gives more freedom in where content can be placed, but it tends to be less accurate than marker based tracking in difficult conditions, depends more on the capability of the device's camera and processor, and struggles with plain, untextured surfaces that offer no features to lock onto.

Common uses

Fiducial markers were central to early augmented reality, where systems like ARToolKit let a camera anchor virtual graphics to a printed marker on a desk or page. Beyond AR, markers are widely used in robotics, where tags such as AprilTag and ArUco give a robot or drone a reliable reference for navigation and localization. They are also used for camera calibration, where a known marker or board of markers provides the precise reference geometry needed to estimate a camera's parameters, and more broadly for Tracking the pose of cameras and objects in computer vision research.