Chapter 1 - How to Read the AU Sections

Each AU is identified by a number and name (for example, the first AU explained is "AU 4 - Brow Lowerer"). Names like "Brow Lowerer" are provided as a more meaningful handle than the more arbitrary numbers, and might make it easier for you to relate to the AUs as you begin learning this Facial Action Coding System. You need not try to memorize the names because as you learn, you will find that you stop relying on the names and increasingly refer to the AUs by numbers. Experienced FACS coders name facial behaviors with a string of numbers for each AU present, e.g., one-two-five-twenty. It is important that you learn the number that designates each AU, but not the name. Unfortunately, we have not been able to make this task easier by any logical relationships among the numbers. The assignment of an AU to a number is arbitrary and has no mnemonic, for example, consecutive AU numbers do not correspond to AUs with similar appearance changes. Likewise, the first AU in the chapter on Upper Face AUs is Action Unit 4, rather than AU 1.

The description of each AU begins with a short paragraph highlighting the muscle that underlies the AU, its location and general action. This description refers to the illustrations at the beginning of each chapter showing muscular anatomy and muscular action. Begin by taking a look at AU 4 on page 17, reading the opening description, and looking at the illustrations in Figure 2-1 relevant to AU 4. Knowing where in the face the muscle acts and its general mode and direction of action is the first step in understanding how to score the action. Look at how the muscle is shaped and from where and to where it runs. Notice that AU 4 runs from the lateral parts of the nasal root obliquely to insert above the center of the eyebrow in the forehead and that two other strands of AU 4 run from near the nasal root upwards into the forehead. From the "Muscular Action" diagram of Figure 2-1, you know that this muscle will pull the skin at the center of the eyebrow towards the root of the nose and pull the skin in the center of the forehead down. Now you are able to understand how this muscle produces the appearance changes described in Section A. Study the opening description and the corresponding illustrations at the beginning of each chapter carefully every time you read about a new AU. Think about how AU 4 is located in respect to the other muscles in the illustrations.

Section A - Appearance Changes

Section A lists the most important appearance changes produced by the muscle that allow this action to be distinguished from other AUs. This section includes both a description of the movements that occur with the action of the muscle and the static appearance of the face when that action is held for a period of time (or frozen in a still photograph). For example, read each of these appearance changes for AU 4 on page 18 and try to visualize each appearance change that is described. Be sure you know the meaning of any specialized terms describing the face and the muscular action. You may need to refer to the definitions of terms in the tables in this chapter beginning on page 2.

At the end of Section A is a short summary of the relevant images and videos suggested for study. The caption or title of a reference example image or video merely indicates what AUs the example represents, not its complete FACS score. Appendix II contains the complete FACS score for each example. Thumbnails of the images in Section A provide a visual overview and index of the images relevant to the current AU, but for studying the appearances, you should examine larger images. If you are reading online, click on the blue link of the thumbnail caption's text to see the Manual page with the image, or click on the thumbnail itself to see the image in your external viewer. Click on the "page xx for score" blue text link to see the full FACS score for the item. When you browse the Reference Example section, you can also show the neutral for the person in the image by clicking on the image's caption, though this link is not highlighted by color. If you are reading a paper copy, page to the page number indicated to see the example image. If you are reading an online version and want to see the video, click on the thumbnail caption's blue text link to view the video with your internal viewer or the thumbnail itself to view it in an external viewer. If you are reading a printed version, look up the video filename in Appendix I and view the file on the CD ROM using your computer and the appropriate video viewer.

Look for each appearance change listed in Section A in each of the AU depictions. Not all appearance changes may be apparent in any given example, but you should decide whether you can see each or not. As you read through the appearance changes for an AU, study the accompanying image(s) and video(s) carefully. Identify:

If you find that the appearance changes resulting from the action are difficult to detect, compare the image showing the AU with a image where no action has occurred (i.e., the face is neutral). The video can be more helpful than the image when studying subtle actions due to the motion that can be seen. Read the scoring commentaries in Appendix II to clarify what appearance changes are relevant to scoring each example image or video.

Section B - How to do the AU

One of the most important activities in the practice of scoring faces is making AUs on your own face while looking in a mirror to compare with what you see on another's face. Thus, it is very important to try to make the action on your own face accurately. Section B describes how to make the AU, suggests techniques for recruiting the muscle, and offers tips on the most likely mistakes to avoid. Your goal is to try to produce the one muscular action described without any other AU. Do not settle for less than perfection. Keep working on making the action until you can perform it well. It may take daily practice for months before you can do some of the difficult AUs. No one we have observed is able to make every AU with little or no practice, but everyone does better with more practice. Even if you cannot perform an AU perfectly, you must know what it is about your performance that is deficient.

Imitate the image and video and compare your face with those in the illustrations. Be certain that you have made the correct AU and no additional AUs. If you fail to mimic the appearance in a image or video, place your hand along the Action Unit's muscle pathway. Try to feel the muscle contract under your fingers. Flatten your hand against your face so you can distinguish other AUs in the surrounding area. Persist until you can move only the AU you are studying. When you are satisfied that you can make the AU, do it repeatedly and study the effects it has on the surface of your face. If you have difficulty, try making the AU on only one side of your face. If possible, add the action on the other side. If you cannot do that, study the effect of the unilateral action.

Show your performance of each AU to other people who are learning FACS with you. Let them study the effects on your face. Let them correct your attempts if they believe you are making other AUs.

Look at the faces of other people in your training group to see how their face appears. Carefully note variations you observe on others' faces. Make suggestions about what is accurate or inaccurate about each other's attempts to do the action. The more faces you see doing the action, the more you will understand the range of appearances that the AU produces, and also how other actions contribute to the appearance of the target action. If you are doubtful that others are making the correct AU or are making additional AUs, touch their faces. Feel whether there is tension along the pathway of the muscles that underlie the AU you are studying, and not tension elsewhere.

You may find some AUs are difficult, if not impossible, to perform singly. Learn the appearance changes and the movement of such an Action Unit from the images and video and from what you can see on the faces of the other members of your group. Don't become discouraged. Continue trying to make each AU as you proceed through the Manual. As you learn to perform other neighboring AUs, return to those that gave you difficulty. Because you become more aware of the way your face feels when making each of the AUs, you may find it possible to do AUs you earlier found to be difficult.

AU 4 is a good action to start performing as it is easy to do. Look now at the B section for AU 4 on page 18. The instruction for this easy movement is short, and no special motivation for its performance appears, but several common problems that can occur are noted, and last resort measures for producing this action are suggested.

Section C - Intensity Scoring

FACS uses conventions or rules to set thresholds for scoring the intensities of an Action Unit. Section C for each Action Unit or combination describes the changes in appearance that distinguish one AU from another and provide criteria for scoring intensities of an AU. Whether you score intensity or not, and for which AUs intensity is scored, will depend upon the purposes of the investigation (see the Investigator's Guide to FACS).

When evidence of any specific AU is absent, the face is in a neutral (or baseline) scoring condition. When evidence of an AU is present, intensity of an Action Unit can be scored on a five-point ordinal scale. The letters A, B, C, D, and E refer to the intensity of an action. These letters are written immediately after the number of the AU to indicate how much of the total appearance change that can potentially be caused by the AU is actually present, e.g., 4B or 4E. There is a correspondence between the five point FACS intensity notation and the scale of evidence of an AU's presence that defines the thresholds or criteria for scoring intensity levels listed in Section C.

The general relationship between the scale of evidence and the A-B-C-D-E intensity scoring is diagrammed in Figure 1-2.

Generally, the A level refers to a trace of the action; B, slight evidence; C, marked or pronounced; D, severe or extreme; and E, maximum evidence. For some AUs, this relationship is a bit different, so for each AU, the criteria for each intensity level in terms of the scale of evidence is listed in Section C.

There are two important points about the A-B-C-D-E scoring scale to keep in mind. It is not an equal interval scale; the C and D levels cover a larger range of appearance changes than the other levels, and most of the range of AU movement falls in these levels. The A, B, and E levels are defined as very narrow ranges. Second, each letter refers to a range of behaviors; even the trace of A and the maximum of E refer to a limited range of appearance changes rather than a single point, as might be implied otherwise by the term "maximum." In the paragraphs below, the criteria for scoring intensity are considered in more detail.

If there is little action, it may be very difficult to distinguish an AU from no activity. Sometimes it may not be possible to determine whether any action at all has occurred, or you may think something moved or happened, but it is not possible to determine exactly which AU. This is especially so when you are scoring a still photograph where you cannot see the movement itself, only the consequence of the action. In such situations, a change in lighting, head position, or transient shadows can give the impression of a different expression even when no action occurs. Even with a motion record, however, it may not be possible to distinguish an action from a baseline, or one AU from another if the action is below a certain threshold. On these occasions, you will not score a change in facial behavior, i.e., when you see something so barely noticeable that it is too small to allow you to choose an AU to describe what you see. Such unscorable changes could be merely artifacts in the visual record or a result of a mechanical force other than muscular action (e.g., a gust of wind); or they could be changes that are so subtle, that even though it is certain something happened, it is not possible to determine which AU or combination of AUs produced it.

On the other hand, the activity in the face may be barely noticeable, but trace evidence of an Action Unit is sufficient for you to decide that a particular AU is present. This AU has an intensity of A, a trace of evidence, but enough to convince you of its presence. After several hundred hours of facial scoring with FACS, you will become expert at discriminating these traces of evidence about an AU as you practice careful examination of minute changes in the face. You will learn to infer that a particular AU has occurred as you become increasingly adept at discriminating appearance changes in the face and as you discuss with other FACS scorers what it takes to be sure about trace evidence. Accurately detecting trace evidence of an AU is an ability that will elevate your skills to the expert class. This learning process is another important reason why training with others and discussing your scoring of faces that show subtle changes are important activities.

A crucial anchor for scoring the range of intensities of each AU is the set of criteria for scoring a B level of intensity. All the AUs have explicit criteria in Section C for scoring the B intensity. The set of criteria for scoring the B intensity of an AU may be only one appearance change, or it may include more than one, or it may allow a choice among appearance changes. More than one appearance change in the set of criteria is indicated by an "and" between appearance changes. Choice among appearance changes is indicated by "or" between appearance changes. If the evidence for an AU is present, but does not meet the B level criteria, then the intensity must be A. If the criteria are met, then the intensity must be B or greater. You must obtain enough facial scoring skill to be able accurately to detect and identify AUs that have a B intensity in order to function as an effective facial scorer. This level of skill is the minimum objective of the training provided by this manual, and is required to pass the FACS Final Test. Thorough study of this manual and sufficient practice will enable you to reliably score B level AUs; subsequent experience will enable you see all the A level actions.

The C level of intensity is typically distinguished from the B level by a set of criteria that establishes how much more evidence of the AU is required beyond the B level to score it as C. As muscular activity increases, more appearance changes become visible, and appearance changes already present become more evident. Your task in scoring the intensity of an AU is to assess the amount of appearance change present and represent it as an intensity score. It is not possible for us to enumerate all the combinations of appearance changes that provide the evidence for each intensity level beyond the B range. Instead, you must rely on the meaning of the scale of evidence to score these higher intensities. We provide examples of what appearance changes constitute a set of criteria for each level. For example, the B level of an AU might require the slight evidence of an appearance change, while the C level requires the marked evidence of the same change. Alternatively, the B level might require slight evidence on either one or another change, while the C level requires both changes to be present and at least one marked. These are merely examples. Other combinations of appearance changes listed in Section A might also comprise evidence of the C level, but we have not provided an explicit guideline. Your experience in scoring the whole range of intensities for each AU will allow you to assess the overall degree of appearance change and cast it into an intensity score. Again, it is by practice and experience, together with discussions among your facial scoring colleagues, that will enable you to reliably assess intensities of AUs.

Similar to the distinction between B and C, the threshold for the D level of intensity from the C level is a different, higher point on the scale of evidence for the appearance changes that identify the AU. Again, the criteria for the C, D, and E levels are example criteria; you must factor in the range of appearance changes possible to assign an intensity score. The D level is more than the C level of evidence, but less than the maximum evidence usually required for E.

The E level of intensity is generally scored when the maximum evidence, or nearly maximum evidence, of change is present. The intense muscular contractions of the E level combine with the person's individual physical characteristics to produce changes in appearance that vary somewhat across different people. Your experience in scoring different individuals will enable you to grasp what constitutes the maximum level of appearance change for each person and AU. Again, considerable practice and comparison of your scores and assessment of faces with other coders is an important aspect of the learning process.

There are some additional important points about scoring intensity. One concern is scoring intensity when you do not actually observe the motion in the face that is caused by the muscular action. Sometimes, you may not be able to see a movement occur because there is a break in the motion record, noise, the person moves out of view, etc. Also, you may be using FACS to score still photographs. In Section C, you sometimes find explicit criteria that must be met if you cannot see the actual movement. Sometimes, no such instructions are given. In either case, you score the AU if you can infer that the consequence of the movement has occurred as described by the intensity criteria. In instances where there are separate criteria for when no motion is observed, you use the changed intensity criteria. Generally, for the B level, the criteria to be met when the motion is not observed are stricter or harder to meet than those listed for when motion is observed. In a motion record, score the AU at the B level if it meets either set of criteria, for when motion is observed or when it is not.

Sometimes, the involvement of other AUs with an AU you are scoring for intensity affects the criteria for scoring the B level of intensity. For example, you may be considering scoring two AUs, one of which produces some of the same appearance changes that are produced by the other. In such a case, additional evidence for one AU may be required beyond the evidence that might be produced by the other AU. Guidelines for scoring in these situations are provided in the Reference section that may appear for an AU.

The Manual describes the appearance changes due to combinations of AUs, as well as those of individual AUs. The list of examples of appearance changes for scoring intensity in a combination of AUs is often limited to combinations of AUs with the same intensity or to intensity of only a target AU with the other AU intensities unspecified. Of course, combinations of AUs with different intensities are more common than combinations of all equal intensities, but the number of possible combinations of intensities increases vastly with each additional AU in a combination. It is not practical to list all these combinations, and this list is not needed. You should be able to use the guidelines that are provided to extrapolate intensity scoring to combinations of intensities where no example is provided. You should assess the magnitude of the appearance changes in total, the relative contribution of each AU to the total appearance, and the magnitude of appearance change due to each separate AU to produce a set of intensities for the AUs that best represents these assessments.

Most intensity criteria refer to the degree of an appearance change or to the number of appearance changes. The intensity scores for some AUs involve a criterion in terms of a time duration or some other benchmark.

In summary, it is important to remember that movements of the skin and other features of the face are the basis for determining both the particular AU that has occurred and the intensity of the AU. These movements have a direction, and they produce bunching, bagging, pouching, and wrinkling between the origin of the muscle and its attachment, as well as flattening, stretching, and pulling in other areas. Section A lists the most important appearance changes characteristic of each AU. Section C lists how some or all of these appearance changes are used to score the intensity of the AU, as well as further information about how these changes distinguish each AU. FACS coders are like detectives searching for evidence in the changing facial appearance that shows which AU (or AUs) acted. The coder's task is use the guidelines described in Sections A and C to deduce from the signs observed what AUs (or AU) moved the skin and other features, and how much of each AU occurred. In making these decisions, the strategy is twofold: score the AU or AU combination that best explains the appearances detected, and include only the AUs necessary to explain the appearances. Use the criteria provided in the Manual as guidelines, together with your own experience and understanding of the range of appearance changes each individual can manifest, to determine the scores that you assign. Practice in scoring many individuals in different situations is an essential ingredient in obtaining the experience needed to detect the signs and assign the score.

Look now at Section C for AU 4 on page 18. Study the criteria for each level, and notice how closely the levels for AU 4 correspond to the general description of intensity levels above. Look at the images and video of AU 4 to see how these criteria were applied to each example. At this point, the material in this section and the next may not be completely clear to you, but after you learn Chapter 2, read through the sections here again with more concrete examples in mind, and any confusion will evaporate.