Chapter 1 - Introduction to the Reference Sections

The descriptions of AUs and AU combinations in the A, B, C sections, together with the example images and video, are the primary learning vehicles in this Manual, but you will also use this material frequently as reference when you score the face. Beyond these didactic materials, some sections of the Manual become more important after you have learned the AUs and combinations, when you begin to score faces productively. This material is primarily in the form of tables. Some of the material is specific to a particular AU, and is found after the AU's A, B, C sections. Other material is relevant to contrasts among multiple AUs or combinations, and is generally found at the end of chapters or between major parts of chapters. Both types of reference materials are discussed further below, but first, we must look in more detail at the general effects of multiple simultaneous AUs, or co-occurrence.

Co-occurring actions can produce appearance changes that are relatively independent, changes in which one action masks another, or a new and distinctive set of appearances. When the combination of Action Units responsible for a set of appearance changes is additive, the decisions you need to make to score an AU are still relatively straightforward. In such additive cases, the appearance changes of each separate AU are relatively independent and produce in the combination the sum of all these distinct changes caused by the separate AUs. Evidence of each AU remains clearly recognizable because they have combined without distorting or changing the appearances due to each separate AU. In some cases, additive changes are totally independent, with no AU affecting the appearance of any other, such as when the AUs are in different areas of the face. These combinations are the easiest to score by extrapolating from the appearances described for each separate AU, and are not represented in separate A, B, C sections of the Manual. In other cases, the co-occurring AUs are in the same area of the face and combine to produce an appearance that is a straightforward sum of their separate appearances. Many of these additive combinations have their own AU Combination A, B, C sections that describe the combination, which you read while learning just as the single AU sections. A good example of such an additive combination is 1+2, which is described beginning on page 54. Look at the Section A appearance changes for combination 1+2 and compare them to the appearance changes for AU 1 on page 20 and for AU 2 on page 22. You can see that the appearance changes for 1+2, although they describe a different appearance from either AU 1 or AU 2 alone, are a sum of the appearance changes due to the separate AUs, with neither AU distorting or masking the appearances of the other. These additive changes make scoring the constituent AUs relatively easy.

Scoring the AUs responsible for an appearance change becomes more difficult, however, if the combination of AUs is not additive, but produces an appearance that is in some way different from the separate appearances caused by the individual AUs. A combination which is different is one where the appearance changes are not simply the sum of the appearances of the constituent AUs, but create distinctively new appearance changes. All such combinations that create different, distinctive appearances in the upper face have been explained and their appearances detailed in separate AU Combination A, B, C sections. For example, look now at one of these distinctive combinations, 1+2+4 beginning on page 56. Read the appearance changes in Section A and compare them to the appearance changes for AU combination 1+2 on page 54 and AU 4 on page 17. You can see that the opposed lifting of the eyebrows due to 1+2 and the lowering due to 4 tend to counteract each other, masking and changing the signs of each. The pulling together of skin in the forehead by AU 4 also creates a new pattern of lines in the forehead that differs from 4 alone and from the relatively straight, horizontal wrinkles of 1+2. This is a pattern that is distinctive of 1+2+4.

When the appearance changes associated with a particular AU are altered by the action of other AUs or combinations of AUs, the signs relevant to the criteria for scoring an AU's intensity level can become easier or harder to detect. The most common example of this interaction of AUs is that two or more of the AUs in a combination produce some of the same appearance changes used as intensity criteria for at least one of the AUs. This overlapping of appearance changes might require setting criteria for the B intensity higher, in the form of an increased appearance change or an additional appearance change, to adjust for their effects in the combined appearance. Occasionally, the effect on facial appearance of the interacting combination is noticeably different from what might be expected from summing the effects of each individual AU in the combination. In these instances it is sometimes necessary to specify different criteria for the scoring the B level of intensity of one or more of the AUs involved.

When different criteria have been specified for an AU in situations like those described above, these criteria are described after the criteria for the single AU, or there is a cross-reference to a Section C or Reference section where the criteria for the combination were described. You do not need to read these qualified criteria in your first pass through the material, and you do not need to understand them in your initial study of the single actions. They are explained later in the sections for AU combinations. These qualified criteria are listed in this manner for later convenience when you use Section C and the Reference for single AUs as a reference in scoring. You do need to know what these tables contain and how they work, so take a brief look now at the Reference section for AU 4 on page 19. The first column indicates the problematic combination of AUs; the second indicates the target AU, or the AU whose intensity criteria are being modified. This table shows, for example, that when AU 4 co-occurs with AUs 1 or 2 or both, the criteria for scoring 4B change. The third column of the table shows what the changed criteria are, or where to find the criteria. For example, when 4 co-occurs with both 1 and 2, lookup the changed criteria in Section C of the description for AU combination 1+2+4 (or in our abbreviated notation, 1+2+4secC). When 4 co-occurs with 1, use the criteria in the row for combination 1+4. In cases like these, where the appearance changes produced by the AUs (e.g., 1+2 v. 4) are opposite, you generally need less evidence of the target AU or to emphasize another criterion. Now look at the line in the table for 4+6. In this case, because 6 can produce some of the same appearance changes that 4 produces, we require somewhat more evidence of 4 in order to score 4B.

Subtle Differences sections contain reference material that deals with multiple actions and combinations, and they usually appear near the end of chapters. The purpose of Subtle Differences tables is to highlight signs that are useful in distinguishing among similar AUs and combinations. They may contrast the appearance differences between single AUs, combinations, or single AUs and combinations. Subtle Differences sections do not describe all differences, but only those aspects of movement and appearance changes that are most relevant in distinguishing two Action Units or Action Unit combinations or in detecting the AU's action.

The Subtle Differences tables also contain notes about the interaction of appearance changes when multiple actions co-occur. When actions co-occur, one AU can interfere with detecting another AU by hiding or masking the signs of its action. The AU that obscures is call the dominant AU because its appearances are apparent, while tending to eclipse the signs of the other, or subordinate, AU. Look in the Subtle Differences table beginning on page 66 and find the comparison 6 versus 6+7. AU 6 is the dominant AU because it masks the presence of AU 7. FACS scorers say that 6 dominates 7, and thus special care needs to be taken when looking for signs of 7 when 6 is present. The discussion of 6 versus 6+7 points out the difficulties and suggests techniques for clarifying the presence of AU 7 when AU 6 acts.

We urge you read the Subtle Differences tables carefully as you work through each chapter the first time. By reading them, you will become aware of the appearance changes that are important for distinguishing among AUs and sharpen your understanding of the appearance changes that cause problems when trying to distinguish AUs. They also provide warnings about discerning actions when AUs co-occur. Subtle Differences tables become really important later when you are scoring faces in a productive environment. These tables are one of the first places to turn when you are trying to decide between scoring one AU or another. A master table of Subtle Differences is in Appendix III.

Alternative scores describe yet another relationship among AUs. Alternatives are mutually exclusive scores. The many reasons why two AUs are alternatives include:

Table 2-3 on page 71 shows the alternative rules for upper face actions. In the first column of the table, the AUs that are alternatives are listed. The alternative nature of the pair is designated using the @ sign. The second column of the table provides the rationale for why the AUs are alternative. You can score one or the other of the AUs, but not both. A master table of alternatives is in Appendix IV.

Now that our introduction to the features of the AU chapters is complete, you are ready to read Chapter 2 from beginning to end. This chapter contains much material and introduces you to new ways of seeing faces. Take your time and work through all the exercises and share experiences with your training cohort. After studying Chapter 2 carefully, you will be ready to practice what you have learned by scoring upper face behaviors. Chapter 3 explains how to score facial behavior using the Facial Action Coding System.