Toggle search
Toggle menu
4.9K
5.5K
229
46.8K
VR & AR Wiki
Toggle preferences menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

ARToolKit

From VR & AR Wiki
More actions
ARToolKit
Information
Type Software development kit
Industry Augmented reality
Developer Hirokazu Kato and Mark Billinghurst (original); ARToolworks; DAQRI; artoolkitX project
Written In C, C++
Operating System Windows, macOS, Linux, iOS, Android (earlier versions also SGI IRIX)
License GPL (versions 1.0 to 2.x); LGPL v3 (version 5.2 and later, and artoolkitX)
Release Date 1999 (first demonstration); 2001 (first open-source release)
Website https://www.artoolkitx.org

ARToolKit is an open-source software library for building augmented reality applications. It finds square black-bordered markers in live camera video, calculates the position and orientation of the camera relative to each marker in real time, and lets an application draw 3D graphics from that viewpoint so they appear fixed to the marker.[1] The toolkit was developed in 1999 by Hirokazu Kato and Mark Billinghurst at the Human Interface Technology Laboratory (HIT Lab) of the University of Washington, and its tracking method was described in their 1999 paper on marker tracking and head-mounted display calibration.[2][3]

Versions 1.0 through 2.x were released under the GNU General Public License, while ARToolworks, a Seattle company, sold a proprietary version for commercial use.[4] In May 2015 DAQRI acquired ARToolworks and released the formerly commercial version as open source under the GNU Lesser General Public License version 3 (LGPL v3).[5][6][7] Development continues in artoolkitX, a successor project started by former ARToolworks executives, and a JavaScript port of the library provides the marker tracking in the WebAR library AR.js.[8][9]

Reviewed 4 October 2026. Checked the HIT Lab ARToolKit documentation and licensing pages, the IWAR 99 and MagicBook papers, ISMAR award list, ARToolKit5 changelog and README, artoolkitX site, README and release notes, DAQRI acquisition coverage (TechCrunch, Business Wire), and the port and ARTag sources against the article text. About review dates.

History

Origins at the HIT Lab (1999)

According to the toolkit's documentation, ARToolKit was developed in 1999 when Kato arrived at the HIT Lab. Its first demonstration was at SIGGRAPH 1999 as part of the lab's Shared Space project. The documentation credits support for Windows DirectShow video with helping the toolkit spread to many exhibitions, and states that Kato and Billinghurst remained the original creators even as later versions were packaged by other people.[2]

The method behind the toolkit appeared in "Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System", presented by Kato (then at Hiroshima City University) and Billinghurst (HIT Lab) at the 2nd International Workshop on Augmented Reality (IWAR 99) in San Francisco in October 1999. The paper described an AR conferencing system in which remote collaborators appear on "Virtual Monitors" that a user can place around the room, along with a shared virtual whiteboard. The prototype used Virtual i-O i-glasses fitted with a small color camera and connected to an SGI O2 workstation, and ran at 7 to 10 frames per second.[3] At ISMAR 2012 in Atlanta the paper received the symposium's 10 Years Lasting Impact Award.[10]

GPL releases and ARToolworks (2001-2015)

ARToolKit used a dual-license model. The HIT Lab licensing page states that, since its first public release as version 1.0, the toolkit has been freely available for non-commercial use under the GPL, and that versions 1.0 through 2.x are GPL-licensed. A proprietary version was developed in parallel by ARToolworks, Inc. of Seattle, which handled commercial licenses, support and custom development.[4] ARToolworks was founded in 2001 to offer commercial services for ARToolKit and held several AR patents.[5] The artoolkitX project describes ARToolKit as first released as an open-source project in 2001.[8]

The GPL 2.x line ran on Windows, Linux, Mac OS X and SGI systems, and could be downloaded with plain OpenGL graphics or with the OpenVRML renderer for loading VRML 97/2.0 models.[11][12] Its API was written in C, with bindings for Java and Matlab, and it shipped with a camera calibration routine and a GLUT-based graphics library.[12] Later GPL releases were distributed through SourceForge; version 2.71.3, announced by Philip Lamb in June 2006, added Universal binaries for Intel-based Macs.[13]

The commercial branch was sold as ARToolKit Professional, in versions 4.0 through 5.1.7.[7] Its change log begins with version 4.1.1 in April 2007. Version 5.0, released on 6 September 2013, added OpenGL ES 2.0 support, a new format for natural feature tracking (NFT) datasets, and native handling of the camera video formats on Android and iOS.[14]

DAQRI and ARToolKit 5 (2015-2017)

On 13 May 2015 DAQRI, then preparing the market launch of its Smart Helmet, announced that it had acquired ARToolworks and would make the features of the commercial Pro version, including the natural feature tracking libraries and the iOS, Android and Unity versions, free and open source under the LGPL v3.[6] TechCrunch quoted DAQRI chief executive Brian Mullins as saying the company saw value in the team and the intellectual property.[5] ARToolKit 5.2 was released the same day with full source code supplied, and Java package names changed from "com.artoolworks" to "org.artoolkit".[14] The release notes describe 5.2 as the first major release under an open-source license in several years, representing several years of commercial development by ARToolworks; for the first time, full source was provided for the NFT libraries libAR2 and libKPM.[7] The abstract for a July 2015 seminar given by DAQRI's Ben Vaughan at the Open University's Knowledge Media Institute described ARToolKit as "the first commercially available SDK for Augmented Reality".[15]

Version 5.3 (31 July 2015) replaced libKPM, the key-point matching library used to recognize NFT images, with a version based on the FREAK feature detector contributed by DAQRI. The earlier commercial libKPM used SURF features; the release notes say FREAK is not encumbered by patents, which allowed libKPM to join the other core libraries under the LGPL v3.[7][14] Version 5.4, assembled by Philip Lamb and dated 20 September 2017, overhauled the video input modules: AVFoundation replaced QuickTime and QTKit on macOS, the DirectShow modules were dropped on Windows, and Video4Linux2 became the Linux default.[7]

artoolkitX (2018-present)

The artoolkitX website states that ARToolKit continued as an open-source project under ARToolworks until the company was sold in 2015, and that Ben Vaughan and Phil Lamb, the former chief executive and chief technology officer of ARToolworks, created artoolkitX to keep the software developed and maintained.[8] artoolkitX 1.0 was released on 26 March 2018. It added a new 2D texture tracker built on OpenCV primitives, kept the square-marker and NFT trackers from the LGPL ARToolKit 5, and came with a Java library (ARXJ) and a C# library for Unity.[16] In January 2018 the Chinese AR company Realmax said its Realmax Studio authoring tool had been "built in partnership with the team behind ARToolKit".[17]

artoolkitX is licensed under the LGPL v3, which allows linking into both closed-source and open-source software. It targets iOS, Android, macOS, Windows and Linux, with experimental web support through Emscripten. As of October 2026 the project's README names IQXR, LLC as its sponsor, while the project website names Ethar Inc. of Huntsville, Alabama.[18][8] Version 1.1.0 (23 February 2023) added a native Camera2 video module on Android and full support for Apple Silicon Macs.[16] The newest GitHub tagged release is 1.1.22 (13 December 2024); the release notes also list a version 1.1.23 dated 26 May 2026, with preliminary support for Linux on ARM.[16][18]

How it works

In ARToolKit's basic loop, the camera sends video of the real world to the computer; the software searches each frame for square shapes; when it finds one, it calculates the camera's position relative to the black square; it then draws the computer graphics model from that same position over the live video, so the model appears attached to the marker.[1] The library reports the marker's pose in the camera's coordinate system and uses the OpenGL matrix convention for placing virtual objects.[19]

Kato and Billinghurst's paper gives the details. The input image is thresholded, and regions whose outlines can be fitted by four line segments are extracted, with the four corner points taken from the intersections of those lines. Each candidate region is normalized and the image inside it is compared, by template matching, with patterns registered in advance, so a marker can carry a user's name or photo as its identifying pattern. Markers of known size then give the transformation from marker coordinates to camera coordinates.[3] The same paper described a calibration method for an optical see-through head-mounted display built on the marker tracker: for each eye, the user lines up the center of a hand-held marker with a crosshair shown in the display while moving it from near to far, so the user does not need to keep the head still. In tests with an 80 mm marker, tracking accuracy fell as the marker moved farther from the camera.[3]

ARToolKit 5 and its JavaScript port support square pictorial markers, square barcode markers, multi-marker sets, and NFT markers, which track ordinary images instead of black squares.[20]

Limitations

The HIT Lab documentation lists the main limits of a purely vision-based tracker. Virtual objects appear only while their marker is in view, and if a hand or other object covers part of the pattern the virtual object disappears. Range depends on marker size and pattern complexity: in the documentation's tests, replacing the 4.25-inch pattern with a much more complex pattern of the same size cut the usable range from 34 to 15 inches, and patterns with large black and white regions worked best. Recognition also becomes less reliable as markers tilt away from the camera, and overhead lighting can cause glare on paper markers; the documentation suggests non-reflective materials such as black velvet glued to a white base.[1]

Usable tracking range by marker size (markers placed perpendicular to the camera)[1]
Pattern size (inches) Usable range (inches)
2.75 16
3.50 25
4.25 34
7.37 50

Ports and derivatives

Because the source was available, researchers and developers carried ARToolKit to platforms beyond desktop computers.

Port or derivative Year Platform Notes
ARToolKit on a PDA 2003 Personal digital assistant Daniel Wagner and Dieter Schmalstieg described what they called the first stand-alone AR system with self-tracking running on an unmodified PDA with a commercial camera, with ARToolKit running directly on the device[21]
ARToolKit for Symbian 2005 Mobile phones (Symbian) Anders Henrysson, Mark Billinghurst and Mark Ollila built a custom Symbian port and used it for a face-to-face collaborative AR game, reported at ISMAR 2005[22]
FLARToolKit 2008 Adobe Flash (ActionScript 3) Announced in May 2008 by the developer Saqoosha; rewritten in ActionScript 3 from NyARToolKit, a Java port of ARToolKit, and licensed under the GPL because NyARToolKit is[23]
jsartoolkit5 Not stated in sources Web browsers (JavaScript, WebAssembly) Emscripten port of ARToolKit 5 to JavaScript, now maintained under the artoolkitX organization[20]
AR.js 2017 (repository created) Web browsers Web AR library created by Jerome Etienne that uses jsartoolkit5 for tracking and renders with Three.js or A-Frame; now maintained by the AR.js organization under the MIT License[9][24][25]
artoolkitX 2018 iOS, Android, macOS, Windows, Linux Successor SDK built on the LGPL ARToolKit 5 trackers[16]

Applications and influence

The HIT Lab used ARToolKit in its own projects. In the MagicBook, Billinghurst, Kato and Ivan Poupyrev used the library for head tracking: a reader looking through a handheld display at the pictures in a real book saw virtual scenes on the pages, and could flick a switch on the handle to fly into the scene as an immersive virtual environment.[26] The toolkit's download page also distributed the BlackMagic Book, which used handheld displays and printed pages to tell the story of the America's Cup.[11]

ARToolKit's square markers also influenced later fiducial marker systems. In a 2004 National Research Council of Canada report introducing ARTag, Mark Fiala called ARToolKit "a very successful and robust marker system used for Augmented Reality" whose performance "has spawned many applications in AR and computer vision"; ARTag was presented as an improved marker system based on it, and the HIT Lab download page listed ARTag as a modified marker detection module.[11][27] The PDA and Symbian ports described above ran ARToolKit's tracking on handheld consumer devices.[21][22]

See also

References

  1. ↑ 1.0 1.1 1.2 1.3 "ARToolKit Documentation (How does ARToolKit work?)". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm. Retrieved 2026-10-04.
  2. ↑ 2.0 2.1 "ARToolKit Documentation (History)". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/documentation/history.htm. Retrieved 2026-10-04.
  3. ↑ 3.0 3.1 3.2 3.3 Hirokazu Kato, Mark Billinghurst (1999). "Marker Tracking and HMD Calibration for a Video-based Augmented Reality Conferencing System". Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99), San Francisco. doi:10.1109/IWAR.1999.803809. https://www.hitl.washington.edu/artoolkit/Papers/IWAR99.kato.pdf. Retrieved 2026-10-04.
  4. ↑ 4.0 4.1 "ARToolKit licensing". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/license.html. Retrieved 2026-10-04.
  5. ↑ 5.0 5.1 5.2 Frederic Lardinois (2015-05-13). "DAQRI Acquires AR Pioneer ARToolworks". TechCrunch. https://techcrunch.com/2015/05/13/daqri-acquires-ar-pioneer-artoolworks/. Retrieved 2026-10-04.
  6. ↑ 6.0 6.1 "DAQRI Acquires Augmented Reality Software Pioneer ARToolworks, Makers of Open Source Augmented Reality Platform, ARToolKit". Business Wire (archived). 2015-05-13. https://web.archive.org/web/2015/https://www.businesswire.com/news/home/20150513006304/en/DAQRI-Acquires-Augmented-Reality-Software-Pioneer-ARToolworks-Makers-of-Open-Source-Augmented-Reality-Platform-ARToolKit. Retrieved 2026-10-04.
  7. ↑ 7.0 7.1 7.2 7.3 7.4 "Read me for ARToolKit (version 5.4)". ARToolKit5 repository, GitHub. 2017-09-20. https://github.com/artoolkit/ARToolKit5/blob/master/README.md. Retrieved 2026-10-04.
  8. ↑ 8.0 8.1 8.2 8.3 "artoolkitX: Open-source, multi platform augmented reality". artoolkitX. https://www.artoolkitx.org/. Retrieved 2026-10-04.
  9. ↑ 9.0 9.1 "AR.js Documentation". AR.js. https://ar-js-org.github.io/AR.js-Docs/. Retrieved 2026-10-04.
  10. ↑ "ISMAR - Past Symposiums". International Symposium on Mixed and Augmented Reality. https://www.ismar.net/past_symposiums/. Retrieved 2026-10-04.
  11. ↑ 11.0 11.1 11.2 "ARToolKit Download". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/download/. Retrieved 2026-10-04.
  12. ↑ 12.0 12.1 "ARToolKit Documentation (Feature List)". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/documentation/features.htm. Retrieved 2026-10-04.
  13. ↑ Philip Lamb (2006-06-19). "ARToolKit 2.71.3 released". ARToolKit Mailing List Archive. https://www.hitl.washington.edu/artoolkit/mail-archive/message-thread-00977-ARToolKit-2-71-3-release.html. Retrieved 2026-10-04.
  14. ↑ 14.0 14.1 14.2 "ARToolKit ChangeLog". ARToolKit5 repository, GitHub. https://github.com/artoolkit/ARToolKit5/blob/master/ChangeLog.txt. Retrieved 2026-10-04.
  15. ↑ "ARToolKit Open Source Software (seminar by Ben Vaughan, DAQRI)". Knowledge Media Institute, The Open University. 2015-07-07. https://kmi.open.ac.uk/seminars/2548. Retrieved 2026-10-04.
  16. ↑ 16.0 16.1 16.2 16.3 "artoolkitX Release Notes". artoolkitX repository, GitHub. https://github.com/artoolkitx/artoolkitx/blob/master/Release%20Notes.md. Retrieved 2026-10-04.
  17. ↑ "Realmax previews 100 degree mobile Augmented Reality Glasses, and Multi-Platform AR Creation Tool". PR Newswire. Realmax. 2018-01-08. https://www.prnewswire.com/news-releases/realmax-previews-100-degree-mobile-augmented-reality-glasses-and-multi-platform-ar-creation-tool-300577931.html. Retrieved 2026-10-04.
  18. ↑ 18.0 18.1 "artoolkitX". GitHub. https://github.com/artoolkitx/artoolkitx. Retrieved 2026-10-04.
  19. ↑ "ARToolKit Documentation (Tutorial 2: Camera and Marker Relationships)". ARToolKit, Human Interface Technology Laboratory, University of Washington. http://www.hitl.washington.edu/artoolkit/documentation/tutorialcamera.htm. Retrieved 2026-10-04.
  20. ↑ 20.0 20.1 "jsartoolkit5: Javascript ARToolKit v5.x". GitHub. https://github.com/artoolkitx/jsartoolkit5. Retrieved 2026-10-04.
  21. ↑ 21.0 21.1 Daniel Wagner, Dieter Schmalstieg (2003). "First Steps Towards Handheld Augmented Reality". Seventh IEEE International Symposium on Wearable Computers (ISWC 2003), pp. 127-135. doi:10.1109/ISWC.2003.1241402. https://www.ims.tuwien.ac.at/publications/tuw-138074. Retrieved 2026-10-04.
  22. ↑ 22.0 22.1 Anders Henrysson, Mark Billinghurst, Mark Ollila (2005). "Face to Face Collaborative AR on Mobile Phones". Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 05). doi:10.1109/ISMAR.2005.32. https://digitalnz.org/records/514867. Retrieved 2026-10-04.
  23. ↑ Saqoosha (2008-05-16). "ARToolKit を Flash に移植したよ。 (ARToolKit ported to Flash)". Saqoosha. https://saqoo.sh/a/665. Retrieved 2026-10-04.
  24. ↑ "AR.js - Augmented Reality on the Web". GitHub. https://github.com/AR-js-org/AR.js. Retrieved 2026-10-04.
  25. ↑ "AR.js: Efficient Augmented Reality for the Web". GitHub. https://github.com/jeromeetienne/AR.js. Retrieved 2026-10-04.
  26. ↑ Mark Billinghurst, Hirokazu Kato, Ivan Poupyrev (2001). "The MagicBook: Moving Seamlessly between Reality and Virtuality". IEEE Computer Graphics and Applications, Projects in VR column. https://www.hitl.washington.edu/artoolkit/Papers/cgaMay2001.pdf. Retrieved 2026-10-04.
  27. ↑ Mark Fiala (2004). "ARTag, an improved marker system based on ARToolkit". NRC Publications Archive, National Research Council Canada, NRCC 47166. https://nrc-publications.canada.ca/eng/view/object/?id=b080c6d3-c00c-4527-863b-ee575af5045a. Retrieved 2026-10-04.
Retrieved from "https://vrarwiki.com/index.php?title=ARToolKit&oldid=46427"