Facial Action Coding System

The Facial Action Coding System (FACS) is an anatomically based system for describing visible facial movement. It breaks an expression down into numbered action units (AUs), each tied to the action of one or more facial muscles. The psychologists Paul Ekman and Wallace V. Friesen published the original FACS manual in 1978, building on earlier work by the anatomist Carl-Herman Hjortsjö, and Ekman, Friesen and Joseph C. Hager published a significant revision in 2002.

A 1999 paper on automatic facial action recognition described FACS as widely used in behavioral investigations of emotion, cognitive processes and social interaction. Its named facial actions are also used in computer animation, where FACS has guided the construction of the target shapes in blendshape face rigs. Some face tracking systems for virtual reality (VR) use FACS-based blendshapes to animate avatars. Meta's Face Tracking API for the Meta Quest Pro and other Quest headsets reports expressions that Meta describes as based on FACS, as weights that drive an avatar's blendshapes.

Reviewed 21 October 2026. Checked the FACS definition, history, 2002 code counts, intensity scale, training and coding figures, action-unit muscle table, automated-coding papers, blendshape and VR uses (Li 2015, Meta Face Tracking API, OpenXR XR_FB_face_tracking/2 expression lists, EmojiHeroVR papers) against the cited Paul Ekman Group page, papers via Crossref/arXiv/full text, Meta docs and the Khronos registry. About review dates.

Definition

The Paul Ekman Group describes FACS as "a comprehensive, anatomically based system for describing all visually discernible facial movement". The manual gives the criteria for observing and coding each action unit and describes how action units appear in combination. A 2020 systematic review in Frontiers in Psychology by Elizabeth A. Clark and colleagues summarizes the design: each action unit has a number, a shorthand name and an anatomical basis, and the numbers themselves are arbitrary and carry no meaning.

FACS is descriptive. It contains no emotion labels, although researchers often use it to interpret facial expressions related to emotion and other states. Companion resources such as EMFACS, the FACS Investigators' Guide and the FACS interpretive database are used when researchers want to make emotion-based inferences from single action units or combinations of them. The review also lists two limits of the system: it deals with clearly visible changes in facial movement and does not account for subtle changes such as muscle tonus, and it excludes other facial phenomena such as changes in skin coloration, sweating and tears.

Action units

Using the 2002 version, a coder breaks an expression down into the action units that produce it. Citing the 2002 manual, Clark and colleagues list nine action units in the upper face and 18 in the lower face. FACS also has codes for 14 head positions and movements, nine eye positions and movements, five miscellaneous action units, nine action descriptors (movements whose anatomical basis is unspecified), nine gross behaviors and five visibility codes.

Each action unit can be scored for intensity on a five-point scale from A to E: A is trace, B slight, C marked or pronounced, D severe or extreme, and E maximum. Action units often occur together. In a 2001 Carnegie Mellon technical report, Ying-li Tian, Takeo Kanade and Jeffrey F. Cohn explained that a combination may be additive, where the constituents keep their individual appearance (AU 1+5), or non-additive, where one action changes the appearance of another (AU 1+4). They noted that non-additive combinations make automatic recognition more difficult.

The table below lists a selection of action units with the muscles given for them in the review's table of single action units from the 2002 manual. The last column shows the expression with the same name in the face-tracking expression set of Meta's OpenXR extension XR_FB_face_tracking2, described in the VR section below.

AU	FACS name	Muscle(s)	Same-named expression in XR_FB_face_tracking2
1	Inner brow raiser	Frontalis (pars medialis)	INNER_BROW_RAISER (left and right)
2	Outer brow raiser	Frontalis (pars lateralis)	OUTER_BROW_RAISER (left and right)
4	Brow lowerer	Depressor glabellae, depressor supercilii, corrugator supercilii	BROW_LOWERER (left and right)
5	Upper lid raiser	Levator palpebrae superioris, superior tarsal muscle	UPPER_LID_RAISER (left and right)
6	Cheek raiser	Orbicularis oculi (pars orbitalis)	CHEEK_RAISER (left and right)
7	Lid tightener	Orbicularis oculi (pars palpebralis)	LID_TIGHTENER (left and right)
9	Nose wrinkler	Levator labii superioris alaeque nasi	NOSE_WRINKLER (left and right)
12	Lip corner puller	Zygomaticus major	LIP_CORNER_PULLER (left and right)
15	Lip corner depressor	Depressor anguli oris (triangularis)	LIP_CORNER_DEPRESSOR (left and right)
17	Chin raiser	Mentalis	CHIN_RAISER (top and bottom)
20	Lip stretcher	Risorius with platysma	LIP_STRETCHER (left and right)
24	Lip pressor	Orbicularis oris	LIP_PRESSOR (left and right)
26	Jaw drop	Masseter; relaxed temporalis and internal pterygoid	JAW_DROP

History

Ekman and Friesen built on the work of Hjortsjö, who, in the Paul Ekman Group's words, "did important groundwork in identifying the units of action based on facial muscle groups". Hjortsjö's book Människans ansikte och mimiska språket appeared in Swedish in 1969 and was published in English in 1970 as Man's Face and Mimic Language.

Ekman and Friesen described their method in the paper "Measuring facial movement", published in the first issue of Environmental Psychology and Nonverbal Behavior in 1976. According to Clark and colleagues, FACS was developed as a comprehensive system to distinguish all possible visible, anatomically based facial movements. The FACS manual was first published in 1978 and last revised in 2002. The Paul Ekman Group states that the 2002 manual is the current version and the only one that should be used for scoring. Variants have been adapted from the adult system; Baby FACS, for example, is an anatomically based coding system for infants adapted from the 2002 adult FACS.

Coding and training

The FACS manual is self-instructional: the user reads it and practices coding pictures and videos. The Paul Ekman Group says self-study usually takes 50 to 100 hours and recommends training in a group. Its FACS Final Test, a 34-question video-based exam, is described by the group as "the only standard for proficiency in FACS coding". EMFACS (Emotion FACS) is a selective application of FACS scoring in which coders score only behavior likely to have emotional significance, and its instructions are available only to people who have passed the Final Test.

Manual coding is slow. Clark and colleagues cite an estimate by Cohn and colleagues (2007) that it takes about three months to become proficient enough to pass the Final Test, and state that comprehensively coding one minute of video then takes on average more than two hours. The same review cites findings of good to excellent reliability for the occurrence, intensity and timing of individual action units (Sayette and colleagues, 2001), and notes that human coders find it very difficult to code the dynamics of an expression, as opposed to which muscles are active. Of the 55 consumer-research studies in the review, 40 (73 percent) coded facial responses only by hand, while 11 (20 percent) used only automatic coding.

Linking action units to emotions is a separate step from coding them. Clark and colleagues note that the FACS Investigators' Guide makes it possible to map action units onto basic emotion categories with a finite number of rules, and warn that when researchers deviate from these rules and define action units and emotions by their own criteria, their emotional interpretation is more likely to be faulty unless a large body of empirical research supports it. In a 2019 review in Psychological Science in the Public Interest, Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M. Martinez and Seth D. Pollak concluded that people do smile when happy or scowl when angry more often than chance, but that the way people express anger, disgust, fear, happiness, sadness and surprise varies substantially across cultures, situations and individuals, and that the same configuration of facial movements can express more than one emotion category or something other than an emotion.

Automated coding

According to Clark and colleagues, the subjectivity and time demands of manual coding led to the adoption of FACS in automated systems, which use computer vision to analyze faces in images and video. In a 1999 paper in IEEE Transactions on Pattern Analysis and Machine Intelligence, G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman and T. J. Sejnowski compared image-analysis techniques for recognizing facial actions in image sequences. Their best methods, Gabor wavelet and independent component representations, reached 96 percent accuracy in classifying 12 facial actions of the upper and lower face. The paper notes that FACS coding was then performed by highly trained human experts. Tian, Kanade and Cohn's 2001 system recognized 8 upper-face action units and 13 lower-face action units plus a neutral expression, whether they appeared alone or in combinations, with average recognition rates of 92.7 percent and 87.4 percent after refinement.

A 2019 survey in IEEE Transactions on Affective Computing by Brais Martinez, Michel F. Valstar, Bihan Jiang and Maja Pantic called FACS "one of the most comprehensive and objective ways to describe facial expressions" and reviewed the components of automatic systems, from pre-processing and feature extraction to machine coding, along with the available FACS-coded databases. Open-source tools such as OpenFace, whose 2.0 version was presented by Tadas Baltrušaitis and colleagues at the 2018 IEEE Conference on Automatic Face and Gesture Recognition, combine facial landmark detection, head pose estimation, facial action unit recognition and eye-gaze estimation. Automated facial expression analysis software, including several commercial systems, classifies expressions into basic emotion categories, and some software also assesses action unit activation, intensity and duration.

Use in computer animation

The Paul Ekman Group notes that FACS is used by animators and by computer scientists interested in facial recognition. In their 2014 Eurographics state-of-the-art report on blendshape facial models, J. P. Lewis and co-authors describe a blendshape model as producing a facial pose from a weighted linear combination of target expressions. They write that FACS "has been used to guide the construction of the target shapes", and that many targets in such rigs approximate the linearized effect of individual facial muscles.

Shared FACS-style targets also make it possible to move a performance from one face to another. When a source model and a target model have targets with the same semantic function, "typically FACS inspired", the blendshape weights can be copied directly from one to the other. The report credits the film projects Monster House and King Kong with introducing expression cloning based on FACS expressions to the movie industry. The authors add two caveats. A source model can be built by having an actor perform a set of basis expressions, for example by mimicking FACS expressions, but producing expressions made of individual facial muscles is unnatural and difficult for many people. And although one of the authors had heard claims in the entertainment industry that targets modeled after the FACS poses were the only approach that produced all and only the valid face shapes, the authors show that this approach and the "delta" blendshape form are equivalent in expressive power. See Facial motion capture for the capture methods that drive such rigs.

Applications in VR and AR

Facial capture inside a headset

A head-mounted display covers much of the upper face, which defeats conventional camera-based facial capture. In a 2015 ACM Transactions on Graphics paper, Hao Li and colleagues from the University of Southern California and Oculus and Facebook built a prototype on an Oculus Rift DK2 with eight ultra-thin strain gauges in the foam liner to measure upper-face expressions and an Intel RealSense RGB-D camera mounted on the headset for the mouth region. Like many real-time facial animation systems, it used linear blendshape models to produce animations "based on FACS expressions", and the authors noted that the meaning of each blendshape allows the performance to be retargeted to arbitrary digital characters.

Meta Movement SDK and OpenXR

On the Meta Quest Pro, Meta's Face Tracking API detects facial movement with the headset's inward-facing cameras and converts it into activations of FACS-based expressions. Meta's developer documentation states: "The Face Tracking API supports expressions based on the Facial Action Coding System (FACS)". The API returns a weight for the strength of each expression, and an application uses those weights to drive blendshapes that an artist has created for an avatar. Meta's documentation refers to 70 FACS expressions, including seven tongue blendshapes, and to a separate set of 15 visemes for mouth shapes during speech. It also says an avatar does not need a blendshape for every expression; a simple avatar could have only a neutral and a smile shape. The camera-based feature, which Meta calls Natural Facial Expressions, is limited to the Quest Pro. On Meta Quest 2, Meta Quest 3 and Meta Quest 3S, an "Audio To Expressions" machine learning model estimates the same expressions or visemes from the microphone audio, through the same API.

The data reaches applications through two Meta extensions to OpenXR. XR_FB_face_tracking (extension number 202) defines 63 expressions, and its successor XR_FB_face_tracking2 (extension number 288) adds seven blend shapes that estimate tongue movement, for 70 in total, and lets the application and runtime indicate whether visual or audio data is used. Many expression names in these extensions, such as BROW_LOWERER, CHEEK_RAISER, LIP_CORNER_PULLER and JAW_DROP, are the names of FACS action units, although the extension text itself defines the expressions by name and does not mention FACS. The specification describes face tracking data as sensitive personal information and says applications storing or transferring it should ask the user for active and specific acceptance. Meta's help page for Natural Facial Expressions states that images the Quest Pro captures of the user's face never leave the headset and are deleted after processing, and that the system produces a set of numbers estimating gaze direction and facial movement.

Emotion recognition research

The occlusion caused by headsets also complicates emotion recognition in VR. Thorben Ortmann, Qi Wang and Larissa Putzar recorded 37 participants playing EmojiHeroVR, an affective VR game, with external cameras and a Meta Quest Pro. Their EmoHeVRDB database, presented at the 2024 International Conference on Affective Computing and Intelligent Interaction (ACII), contains 3,556 labeled facial images of 1,778 reenacted emotions plus the headset's activations for 63 facial expressions per frame, which the authors describe as based on FACS. Before playing, participants were introduced to FACS and trained for about 10 minutes on reenacting the game's seven emotions. A baseline image classifier reached 69.84 percent accuracy across seven categories (six basic emotions and neutral). In a follow-up paper at IEEE AIxVR 2025, the same authors classified emotions from the headset's 63 expression activations alone with up to 73.02 percent accuracy, and reached 80.42 percent by fusing the activations with image data.

See also

• Face tracking
• Facial motion capture
• Avatar
• Codec Avatars
• Digital human
• Eye tracking
• Computer vision