| |
|
|
Measuring Facial Movement1
Paul Ekman and Wallace V. Friesen
This article reports a new method of describing facial movement based on an anatomical analysis of facial action. Most research on facial behavior has not measured the face itself, but instead measured the information that observers were able to infer from the face. Examples of the questions asked are: can observers make accurate inferences about emotion? Can observers detect clinical change or diagnosis? Do observers from different cultures interpret facial expression differently? Are observers influenced by contextual knowledge in their judgments of the face? Do observers attend more to the face than to the voice, etc.?
Few studies have measured the face itself. Examples of the type of questions which could be asked are: which movements signal emotion? Do facial actions change with clinical improvement or differentiate among types of psychopathology? Do the same facial movements occur in the same social contexts in different cultures? Are certain facial actions inhibited in certain social settings? Which facial movements punctuate conversation, etc.? The differences between these two approaches to the study of facial behavior (i.e., observer's inference vs. facial measurement) were discussed and the literature reviewed by Ekman, Friesen and Ellsworth (1972).
Research focused on the face has been impeded by the problems of devising an adequate technique for measuring the face. Over the years various procedures for facial measurement have been invented. Early work has rarely been cited by current investigators, e.g., Frois-Wittmann (1930), Fulcher (1942), Landis (1924) or Thompson (1941). More current approaches to facial measurement have varied in methodology, ranging from analogic notations of specific changes within a part of the face (Birdwhistell, 1970), to photographic depictions of movements within each of three facial areas (Ekman, Friesen & Tomkins, 1971), to verbal descriptions of facial gestalts (Young & Decarie, 1977).
No consensus has emerged about how to measure facial behavior. No tool has been developed which has become the standard, used by all investigators. Each investigator has almost been in the position of inventing his own tool from scratch. The only exception has been that the category lists of facial behavior described by some human ethologists (Blurton Jones, 1971; Grant, 1969; McGrew, 1972) have influenced other human ethologists studying children.
Although different in almost all other respects, most facial measurement techniques have shared a focus upon what is visible, what a rater can differentiate when he sees a facial movement. An exception, Schwartz, et al. (1976) used electromyographic (EMG) measurement to study changes in muscle tone which are not visible. EMG could also be used to measure visible changes in muscle tone which do not involve a noticeable movement, but such work has not been done. EMG also could be employed to study visible movement, but we think it is unlikely that surface electrodes could distinguish the variety of visible movements which most other methods delineate.
Vascular changes in the face are another aspect of facial behavior which can occur without visible movement and which, like muscle tonus, could be measured directly with sensors. No such work has been published on coloration or skin temperature although Schwartz2 in unpublished studies has found thermal measures useful in measuring affective responses. Some of the measurement procedures which utilize observers to rate visible movement, have included a reference to a "reddened" face.
Elsewhere (Hager, 1986, Ekman, 1981)3 we have compared other methods for measuring facial movement with our own method, contrasting the assumptions which underlie each method, explaining how units of measurement were derived, and providing point by point comparisons of the measurement units. Here, we will only selectively contrast other methods with our own to explain our technique.
Background of the Facial Action Coding System
Our primary goal in developing the Facial Action Coding System (FACS) was to develop a comprehensive system which could distinguish all possible visually distinguishable facial movements. Most other investigators developed their method just to describe the particular sample of behavior they were studying. Our earlier approach, the Facial Affect Scoring Technique (FAST) (Ekman, Friesen & Tomkins, 1971), also had a more narrow objective. It was designed primarily to measure facial movement relevant to emotion. While we remain interested in describing the emotion signals, to do so we needed a measurement scheme that could distinguish among all visible facial behavior. We were also interested in a tool which would allow study of facial movement in research unrelated to emotion: e.g., facial punctuators in conversation, facial deficits indicative of brain lesions, etc. With comprehensiveness as our goal it was necessary to build the system free of any theoretical bias about the possible meaning of facial behaviors.
The interest in comprehensiveness also led us to reject an inductive approach to developing FACS. Most other investigators devised their descriptive system on the basis of careful inspection of some sample of the behavior they intended to measure. While their system might contain gaps, as long as its purpose was simply to measure a prescribed sample of events it was perfectly practicable. With comprehensiveness as a goal an inductive method would require inspecting a very large and diversified sample of behavior.
We chose to derive FACS from an analysis of the anatomical basis of facial movement. Since every facial movement is the result of muscular action, a comprehensive system could be obtained by discovering how each muscle of the face acts to change visible appearance. With that knowledge it would be possible to analyze any facial movement into anatomically based minimal action units.
No other investigator has so exclusively focused on the anatomy of facial movement as the basis for his descriptive measurement system. Blurton Jones (1971) considered anatomy in developing his descriptive categories, but it was not the main basis of his measurement system. He did not attempt to provide a description of the full range of minimal actions.
Our interest in comprehensiveness was motivated not only by the diverse applications we had in mind, but by an awareness of the growing need for a common nomenclature for this field of research. Comparisons of the measurement units employed by other investigators would be facilitated if the particular units used in each study could be keyed to a single comprehensible list of facial actions. Also, a complete list of facial actions would reveal to the potential investigator the array of possibilities, so he could better select among them. And, of course, there might be some investigators who, like us, would want to measure not just some facial behavior, but all possible movement they could observe.
A constraint in the development of FACS was that it deals with what is clearly visible in the face, ignoring invisible changes (e.g., certain changes in muscle tonus), and discarding visible changes too subtle for reliable distinction. In part, this constraint of measuring the visible was willingly adopted, based on our interest in what could have social consequences. In part, the constraint of dealing only with the visible was based on our interest in a method which could be applied to any record of behavior - photographic, film or video - taken by anyone. If our descriptive system included the nonvisible, we would be limited only to situations where we could ourselves attach the apparatus (e.g., the leads for EMG). The visibility constraint was also dictated by our belief that if subjects know their face is being scrutinized their behavior may differ radically. The odd results obtained by Landis (l924) may have been due to this in part (cf., Ekman, Friesen & Ellsworth 1972, pp. 79-84, for a discussion of the Landis studies). A method based on visible behavior would use video or motion picture film records, which could be gathered without the subject's knowledge.
Another limitation was that FACS would deal with movement, not with other visible facial phenomena. These other facial signs would be important to a full understanding of the psychology of facial behavior, but their study requires a different methodology. (Elsewhere [Ekman, 1977] we have distinguished a variety of static and slow facial signs contrasting the types of information they may contain with rapid facial movement.) FACS excludes visible changes in muscle tonus which do not entail movement. These changes can be measured through EMG or by having observers make global inferences about brightness, alertness, soberness, etc. Changes in skin coloration are not usually visible on black and white records. Facial sweating, tears, rashes, pimples, and permanent facial characteristics were all excluded from FACS. As the name states, the Facial Action Coding System was developed to measure only movement of the face.
Ideally, the Facial Action Coding System would differentiate every change in muscular action. Instead, it is limited to what humans can reliably distinguish, since it is used by human operators viewing facial behavior, not a machine-based classification. FACS includes most but not all of the subtle differences in appearance which result from different muscle action. The fineness of the scoring categories in FACS depends upon what can be reliably distinguished when a facial movement is inspected repeatedly, and in stopped and slowed motion.
A system for measuring visible facial movements can follow one of two approaches. The minimal units of behavior can be specified, which can in combination account for any total behavior. Or, a list of possible facial gestalts can be listed. The sheer variety of possible actions which the facial musculature allows argues for the minimal units solution rather than gestalts if comprehensiveness is the goal. There are too many different possible total facial actions to list all of the gestalts. If the method specifies facial gestalts (e.g., Young & Decaries', 1977 list of 42 facial gestalts) it cannot score facial actions which show only part of the gestalt, or actions which combine some of the elements of three or four of the gestalts.
While most investigators have listed minimal units, they were not explicit as to how they derived their list. How did they determine how many separate facial actions are possible? How did they determine whether an action was minimal or, instead, a composite of two actions which might separately appear? Usually the decision was based on a hunch, speculation about signal value, or simply what was observed in a limited sample of facial behavior. It seemed to us that an answer would come from knowledge of the mechanics of facial action. We would have to determine the number of muscles which can fire independently, and whether each independent muscular action results in a distinguishable facial appearance. Such an anatomically-based list of facial appearances should allow description and differentiation of the total repertoire of visibly different facial actions.
Some might argue that there is no need to make such fine distinctions among facial actions. Indeed, there might not be a need; many differently appearing facial actions may serve the same function, or convey the same message. There may be facial synonyms, but that should be established empirically, not on a priori grounds. Only a measurement scheme which separately scores visibly different facial actions will permit the research that can determine which facial actions should be considered equivalent in a particular situation.
Another consideration which guided our development of the Facial Action Coding System was the need to separate inference from description. We are interested in determining which facial behavior is playful, or puzzled, or sad, but such inferences about underlying state, antecedent, or consequent actions should rest upon evidence. The measurement must be made in non-inferential terms that describe the facial behavior, so that the inferences can be tested by evidence. Almost all of the previous descriptive systems have combined inference-free descriptions with descriptions confounded with inference; e.g., "aggressive frown" (Grant, 1969); "lower lip pout" (Blurton Jones, 1971); "smile tight-loose 0" (Birdwhistell, 1970). Each of these actions could be described without inferential terms. Since humans do the measurement, the possibility of inferences cannot be eliminated, but they need not be encouraged or required. If a face is scored, for example, in terms of the lip corners moving up in an oblique direction which raises the infraorbital triangle, the person scoring the face still may make the inference that what he is describing is a smile. Our experience has been that when people use a measurement system which is solely descriptive, as time passes they increasingly focus on the behavioral discriminations and are rarely aware of the "meaning" of the behavior.
Another problem which has plagued previous attempts to measure facial movement has been how to describe most precisely each measurement unit. Blurton Jones (1971) noted that facial activity could be described in three ways: the location of shadows and lines; the muscles responsible; or the main positions of landmarks, such as mouth corners or brow location. He opted for the last basis, although he said he used the other two as well. He decided not to base his descriptions on muscular activity because it would be "more convenient if description could be given which did not require that anyone who uses them should learn the facial musculature first, although knowledge of the musculature obviously improves the acuity of one's observations" (p. 369).
We have taken almost the opposite position. The user of FACS must learn the mechanics - the muscular basis - of facial movement, not just the consequence of movement or a description of a static landmark. FACS emphasizes patterns of movement, the changing nature of facial appearance. Distinctive actions are described - the movements of the skin, the temporary changes in shape and location of the features, and the gathering, pouching, bulging and wrinkling of the skin.
FACS' emphasis on movement and the muscular basis of appearance change helps overcome the problems due to physiognomic differences. Individuals differ in the size, shape and location of their features, and in the wrinkles, bulges or pouches which become permanent in mid-life. The particular shape of a landmark may vary from one person to another; e.g., when the lip corner goes up the angle, shape or wrinkle pattern may not be the same for all people. If only the end result of movement is described, scoring may be confused by physiognomic variations. Knowledge of the muscular basis of action and emphasis on recognizing movements helps to deal with variations due to physiognomic differences.
Development of the Facial Action Coding System
Our first step in developing FACS was to study various anatomical texts to discover the minimal units. We expected to find a listing of the muscles which can fire separately, and how each muscle changes facial appearance. We were disappointed to find that most anatomists were seldom concerned with facial appearance. The anatomy texts for the most part described the location of the muscles. Capacity for separate action or visible changes in appearance was not the basis for the anatomists' designation of facial muscles. Instead, they distinguished muscles because of different locations, or if there was a similar location they separately named what appeared as separate bundles of muscle fibers4.
Duchenne (1862) was one of the first anatomists concerned with the question of how muscles change the appearance of the face. He electrically stimulated the facial muscles of a man without pain sensation, and photographed the appearance changes. By this means he was able to learn the function of some of the muscles. His method was problematic for exploring the action of all of the facial muscles. Many of the muscles of the face lie one over the other, and surface stimulation will fire a number of muscles. Inserting a needle or fine wire through the skin to reach a particular muscle may fire others as well.
Hjorstjo (1970) provided the most help. An anatomist interested in describing the visible appearance changes for each muscle, Hjorstjo learned to fire his own facial muscles voluntarily. He photographed his own face and described in drawings and words the appearance changes for each muscle. His aim was not to provide a measurement system, and so he did not consider many of the combinations of facial muscles, nor did he provide a set. of rules necessary for distinguishing between appearance changes which are in any way similar.
Following Hjorstjo's lead, we spent the better part of a year with a mirror, anatomy texts, and cameras. We learned to fire separately the muscles in our own faces. When we were confident we were firing intended muscles we photographed our faces. Usually there was little doubt as to whether we were firing the intended muscle. The problem instead was how to learn to do it at all. By feeling the surface of our faces we could usually determine whether the intended muscle was contracting. By checking Hjorstjo's account we could see whether the appearance on our faces was what he described and showed in his drawings. There were a few areas of ambiguity, and here we returned to a variation on Duchenne's method. A neuroanatomist placed a needle in one of our faces, inserting the needle into the muscle we were uncertain about. With the needle in place, the muscle was voluntarily fired, and electrical activity from that needle placement verified that indeed it was the intended muscle. As this method was uncomfortable, we used it rarely, and only when we were in doubt.
One limitation of this method of deriving facial units must be noted. If there are muscles which cannot be fired voluntarily, we cannot study them. This seems to be the case only with the Tarsalis muscle, and, as best we can determine, its action and effect on appearance are not different from those of one of the voluntarily controlled muscles, levator palpebrae.
Our next step was to examine the photographs taken of each of our faces, scrambling the pictures so we would not know what muscle had been fired. Our purpose was to determine if all the separate muscular actions could be distinguished accurately from appearance alone. Often, it was easy to determine, although it usually required comparing the appearance change with the resting or baseline facial countenance.
There were instances in which we found it difficult to distinguish among a set of muscles in accounting for a photograph of a facial appearance. Sometimes we could tell one muscular action from another, but the differentiation seemed so difficult that we prejudged it as not likely to be reliable. Sometimes the appearance changes resulting from two muscles seemed to differ mostly in intensity of the action, not in type of appearance. In either instance we designated and described one Action Unit which could be produced by 2 or 3 different muscles.
Note that we call the measurements action not muscle units. As just explained, this is because a few times we have combined more than one muscle in our unitization of appearance changes. The other reason for using the term Action Unit is because we also have separated more than one action from what most anatomists described as one muscle. For example, following Hjorstjo's lead, the frontalis muscle which raises the brow was separated into two Action Units, depending upon whether the inner or outer portion of this muscle lifts the inner or outer portions of the eyebrow.
Table 1-1 lists the names, numbers and anatomical basis of each Action Unit. Most of the Action Units involve a single muscle. The numbers are arbitrary and do not have any significance except that 1 through 7 refer to brows, forehead or eyelids.5
The table indicates where we have collapsed more than one muscle into a single Action Unit, or where we have distinguished more than one Action Unit from a single muscle. The FACS names given in the table are a shorthand, not meant to describe the appearance changes, but a convenience to call them to mind.
Table 1-2 lists an example of how each Action Unit (AU) is described in the FACS Manual. The description includes four types of information:
(1) The muscular basis of each AU is given in words and diagrams.
(2) Detailed description of the appearance changes are keyed to illustrative still photograph and film examples.
(3) Instructions are given as to how to make the movement on one's own face. This aids in learning the appearance changes particularly if FACS is learned by a group of people who can observe the variations in appearance on each other's faces. Learning how to do each AU also provides the user with a technique for later analyzing movements to be scored into their component parts. The user imitates the movement to be scored, noting which muscles he had to move in his own face to produce the movement to be scored. By this means the scoring of any novel, complex facial action can be determined.
(4) A rule is given specifying the minimal changes which must be observed in order to score a slight version of each AU.
The muscle underlying AU 15 emerges from the side of the chin and runs upwards attaching to a point near the corner of the lip. In AU 15 the corners of the lips are pulled down. Study the anatomical drawings which show the location of the muscle underlying this AU.
(1) Pulls the corners of the lips down.
(2) Changes the shape of the lips so they are angled down at the corner, and usually somewhat stretched horizontally.
(3) Produces some pouching, bagging, or wrinkling of skin below the lips corners, which may not be apparent unless the action is strong.
(4) May flatten or cause bulges to appear on the chin boss, may produce depression medially under the lower lip.
(5) If the nasolabial furrow1 is permanently etched, it will deepen and may appear pulled down or lengthened.
The photographs in FACS show both slight and strong versions of this Action Unit. Note that appearance change (3) is most apparent in the stronger versions. The photograph of 6+15 shows how the appearance changes due to 6 can add to those of 15. Study the film of AU 15.
Pull your lip corners downwards. Be careful not to raise your lower lip at the same time - do not use AU 17. If you are unable to do this, place your fingers above the lip corners and push downwards, noting the changes in appearance. Now, try to hold this appearance when you take your fingers away.
Minimum Requirements to score 152
Elongating the mouth is irrelevant, as it may be due to AU 20, AU 15, or AU 15+20.
(1) If the lip line is straight or slightly up in neutral face, then the lip corners must be pulled down at least slightly to score 15.
or (2) If lip line is slightly or barely down in neutral face, then the lip corners must be pulled down slightly more than neutral and not the result of AU 17 or AU 20.
Table 1-2: An Example of the Information Given in FACS for Each Action Unit
The determination of the single AUs (Table 1-1) and their description (as shown in Table 1-2) was the first step in developing FACS. The procedure of moving muscles, photographing the movement, and inspecting the pictures was reiterated with all the possible combinations of 2 AUs. There was no need to describe AU combinations which could not interact. For example, pulling the lip corners down is done by a muscle which cannot affect the muscles which control the position of the eyebrows. Two-way combinations were performed separately for the AUs controlling the brows, forehead, and upper and lower eyelids, and for those AUs controlling the lower eyelids, cheeks, and lower regions of the face.
There were a few hundred combinations to perform and examine, for only in a very few instances did we discover that two AUs could not occur simultaneously.
Study of the photographs of the AU combinations showed that most of the appearance changes were additive. The characteristic appearance of each of the two AUs was clearly recognizable and virtually unchanged. There were a few AU combinations which were not additive. The appearance change may have incorporated some of the evidence of the single AUs, but also new appearance changes from their joint action were evident. All of these distinctive combinations were added to FACS, each described in the same detail as were the single AUs.
Inspection of the photographs of the AU combinations revealed that the appearance changes may be neither additive nor distinctive, but there may be a relationship of dominance, substitution or alternatives for another AU. In dominance, the strong AU overshadows the weak one. It may completely conceal the appearance due to the subordinant AU, or it may make the evidence of the subordinant AU very difficult to detect. In order to enhance agreement in scoring, rules were established which prohibit the scoring of subordinant AUs when there is clear evidence of a dominant AU.6
In substitution the appearance of two different AU combinations is so similar, that in order to avoid disagreements, we designated only one of the combinations as the score to be used for either of the combinations.7
In alternatives, two AUs cannot both be scored because both cannot be performed simultaneously, or it is hard to distinguish one from the other, or the logic of other FACS rules does not allow scoring both. The coder determines which of the two alternatives best describes a particular action.8
After analyzing the pictures of all the combinations of 2 AUs, the process of performing, photographing, and then inspecting was reiterated, but this time with combinations of 3 AUs. Instead of hundreds there were thousands to so examine. Those which produced a distinctive rather than an additive combination of AUs were allotted their own entry in FACS with full descriptions as per Table 1-2. When we were ready to explore the combinations of 4 AUs, the number to consider was so great that we decided to make only selective study. On the basis of what we had learned from the 2-AU combinations and 3-AU combinations we extrapolated which further combinations were likely to result in distinctive facial movements. In total, between four and five thousand facial combinations were performed and examined. This included all the possible combinations of AUs in the upper regions of the face, and all 2-way and 3-way combinations in the lower face, plus some of the 4-, 5-, 6-, 7-, and 8-AU combinations in the lower region of the face.
The Manual for the Facial Action Coding System was written in a self-instructional format, to serve as an initial tutor and subsequently as a reference in scoring facial behavior. The Manual contains the following information:
1. Textual material describing each single AU listed in Table 1-1. Each AU is described in terms of its muscular basis, appearance changes, instructions for making the movement, and requirements which must be met for scoring slight versions (cf. example in Table 1-2).
2. The same information for each of more than 44 combinations of AUs.
3. A simple, less precise account of 11 additional single AUs listed in Table 1-3. Many of these AUs do not involve the facial muscles. We have not described them as finely as was done in Table 1-2.
4. Descriptors which can be used to measure head and eye position.
5. Tables comparing and contrasting AUs (or AU combinations) which differ only subtly. More than 400 such subtle differences are tabled.
6. Scoring rules based on the Dominance, Alternatives and Substitution relationships among AUs.9
7. A scoring sheet and a step-by-step procedure to follow in measuring a facial movement. The procedure contains a number of internal checks designed to increase inter-rater reliability.
Table 1-3: More Grossly Defined AUs in the Facial Action Coding System
There are also still photographic and motion picture film examples of all the single AUs in Tables 1-1 and 1-3, of the 44 AU combinations, and the head and eye position descriptors. Additional still photographs and motion picture film examples of facial behavior are provided to practice scoring facial movement. Correct scores are given, with commentary about the source of possible errors in scoring.10
In addition to the Manual, a separate publication, Analyzing Facial Action is in preparation11. This describes in detail the development of FACS, reliability, validity, results from experiments using FACS, and possible areas of application. Theoretically based predictions about the particular AUs and AU combinations which signal emotions and emotion blends are also included.
An Example of Scoring Faces
It is not feasible in a short article and without film or video to illustrate the actual use of FACS to score a facial movement. The logic which underlies FACS can be illustrated, however, with still photographs. See the seven facial behaviors shown in Figure 1-1.
They all involve some common elements in appearance, in particular the down curve to the line of the mouth. They also differ. Analysis of these faces in terms of the single AUs involved will allow precise differentiation among them.
These seven faces include three single AUs and the four combinations among these AUs. Figure 1-A is the appearance change due to AU 15 described earlier in Table 1-2. Figure 1-B shows AU 17, described in Table 1-4; Figure 1-C shows AU 10, described in Table 1-4.
If you read these verbal descriptions matching them to the photographs, you should then be able to "dissect" the other four faces in Figure 1 into their component AUs. Figure l-D combines AU 10 and 15; Figure l-E combines AU 10 and 17; Figure 1-F combines AU 15 and 17; and Figure l-G combines AU 10, 15, and 17.
Any complex facial behavior can be so analyzed into its component elements, if the single AUs have been learned, and if rules regarding combinations have been studied. The scoring procedure leads the user to break down any action into a set of single AU scores. When he is in doubt, he is encouraged to consult the verbal descriptions, photographic and film examples, and tables of contrasting subtle differences. The person is also encouraged to imitate the action he sees, observing his own face and noting what AUs he must use in order to reproduce the action he observes.
It is important not to be mislead by this example into thinking FACS is designed for scoring still photographs. FACS emphasizes movement and its chief purpose is to score facial actions seen on motion records, although it can be used with stills if there is also a picture of a "neutral" face.
It has been shown how FACS scoring differentiates the seven facial behaviors shown in Figure 1-1. They are not visibly the same. Are these seven the same functionally, psychologically, communicatively? Is one a sadness expression, another a pout, another a disbelief gesture, etc.? It is only if the facial measurement distinguishes among these behaviors that we can determine empirically how many of the distinctions are useful. Once we can measure their separate occurrence, we can examine the contexts in which the behaviors occur, or we can study preceding or consequent actions of other persons, isolate concomitant behavior in the person showing the behavior, study observers' inferences from viewing each behavior, etc.
The muscle underlying AU 10 emerges from the center of the infraorbital triangle1 and attaches in the area of the nasolabial furrow2. In AU 10 the skin above the upper lip is pulled upwards and towards the cheek, pulling the upper lip up.
(1) Raises the upper lip. Center of upper lip is drawn straight up, the outer portions of upper lip are drawn up but not as high as the center.
(2) Causes an angular bend in the shape of the upper lip.
(3) Raises the infraorbital triangle; and may cause the infraorbital furrow to wrinkle or deepen if it is already evident in neutral.
(4) Deepens the nasolabial furrows and raises the upper part of this furrow producing a shape as
.
The muscle underlying AU 17 emerges from an area below the lower lip and attaches far down the chin. In AU 17 the skin of the chin is pushed upwards, pushing up the lower lip.
(3) May cause wrinkles to appear on chin boss as skin is stretched, and may produce a depression medially under the lower lip.
Table 1-4: Appearance Changes Due to AU 10 and to AU 17
Concluding Comments
Six people have learned FACS. It required about 100 hours for them to learn and practice scoring. Reliability in scoring was satisfactory. The formula used was to divide the number of AU scores on which two persons agreed, by the sum of the number of AUs scored by each person. If there was perfect agreement on a facial movement the score would be 1.00. The average coefficient of agreement among all possible pairings of the 6 persons across the faces they measured was .8312.
FACS far exceeded our initial anticipation of what would be required to provide a comprehensive descriptive system for measuring facial action. Certainly, FACS is a very elaborate system, more comprehensive than any previous system by quite a margin. There is no facial action described by other systems which cannot be described by FACS, and there are many behaviors described by FACS not previously distinguished by others. FACS allows for measuring facial asymmetries, when different AUs appear on each side of the face. FACS does not include a measure of the intensity of action for every AU, although it does so for four of the AUs listed in Table 1-1. It would be possible for others to follow the procedure used for these AUs to elaborate intensity of action scoring for the other AUs.13
We are reasonably confident that FACS is complete for scoring the visible, reliably distinguishable actions of the brows, forehead, and eyelids. FACS probably does not include all of the visible, reliably distinguishable actions in the lower part of the face. The hinged jaw and rubbery lips allow a nearly infinite number of actions. We have included everything we could see, everything anyone else has included, and what are probably the most common elements and combinations of actions in the lower part of the face among children and adults. As we and others use FACS, we expect that some other AUs may need to be added; hopefully, not many. Others may well be interested in more finely discriminating separate AUs from the list of gross AUs in Table 1-3.
Some will ask the question whether FACS is too elaborate, too comprehensive and detailed. We believe it has been useful to attempt an approximation of the total repertoire of facial action, to isolate minimal Action Units which can combine to account for any facial movement. At the least, FACS provides a means to cross-reference with a common nomenclature the different scoring categories used by others. It may also serve to alert the investigator as to his choices, so he may, if he so chooses, be more explicit in his decisions about what to ignore when he does his measurement. No one knows at the outset how many of the variations in facial behavior can be ignored in any research study without losing important information. In preliminary observations, or pilot studies, investigators may wish to use FACS to comprehensively measure, and then, based on these results, more selectively score only certain AUs or AU combinations in their main study.
Apart from these more selective uses of FACS, there will be some who need a comprehensive measurement system. If we wish to learn all the facial actions which signal emotion (and those that do not), or whether facial emphasis markers are the same regardless of the content of speech so emphasized (to mention just two of our current interests), then a method such as FACS is needed.
Birdwhistell, R. I. Kinesics and context. Philadelphia: University of Pennsylvania Press, 1970.
Blurton-Jones, N. G. Criteria for use in describing facial expressions in children, Human Biology. 1971, 41, 365-413.
Duchenne, B. Mecanisme de la physionomie humaine ou, analyse electrophysioloqique de l'expression des passions. Paris: Bailliere, 1862.
Ekman, P. Facial signs. In T. Sebeok (Ed.), Sight, Sound, and Sense. Bloomington, Ind., Indiana University Press, 1977.
Ekman, P. Methods for measuring facial actions. In K. Scherer & P. Ekman (Eds.), Handbook of Methods of Nonverbal Communication Research. New York: Cambridge University Press, 1981.
Ekman, P., Friesen, W. V., & Tomkins, S. S. Facial affect scoring technique (FAST): A first validity study. Semiotica, 1971, 3 (1), 37-58.
Ekman, P., Friesen, W. V., & Ellsworth, P. Emotion in the human face: Guidelines for research and a review of findings. New York: Pergamon Press, 1972.
Ekman, P., & Friesen, W. V. Manual for the Facial Action Coding System. Palo Alto: Consulting Psychologists Press,1977
Frois-Wittmann, J. The judgment of facial expression. Journal of Experimental Psychology, 1930, 13, 113-151.
Fulcher, J. S. "Voluntary" facial expression in blind and seeing children. Archives of Psychology, 1942, 38, No. 272.
Grant, E. C. Human facial expression. Man, 4, 1969, 525-536.
Hager, J. C. A comparison of units for visually measuring facial actions. Behavior Research Methods, Instruments, & Computers, 1985, 17(4), 450-468.
Hjortsjo, C. B. Man's face and mimic language. Lund: Student-litteratur, 1970.
Landis, C. Studies of emotional reactions: II. General behavior and facial expression. Journal of Comparative Psychology, 1924, 4, 447-509.
McGrew, W. C. An ethological study of children's behavior. New York: Academic Press, 1972.
Schwartz, G.E., Fair, P.L., Salt, P., Mandel, M.R., & Klerman, G.L. Facial muscle patterning to affective imagery in depressed and non depressed subjects. Science, 1976, 192, 489-491.
Thompson, J. Development of facial expression of emotion in blind and seeing children. Archives of Psychology, 1941, 37, No. 264.
Young, G. & Decarie, T. G. An ethology-based catalogue of facial/vocal behaviors in infancy. Animal Behavior, 1977, 25, 95-107.
1We reprint here, with only minor changes, an early article about FACS, which appeared in the first issue of the Journal of Environmental Psychology and Nonverbal Behavior, 1976, 1(1), 56-75. The Journal of Environmental Psychology and Nonverbal Behavior, Randolf Lee, editor, is published by Human Sciences Press, 72 Fifth Avenue, New York, New York. Reprinted with permission.
4We are grateful to Sherwood Washburn (University of California) for explaining why the standard anatomy texts were of so little help and for encouraging our attempt to explicate the muscular basis of facial action.
5This table applies to the first edition. AUs 41, 42, and 43 are now combined; AU 44 is 7E; 25, 26 and 27 have modified definitions.
6Dominance rules were dropped from FACS in 1992 and are not included in the current version; the description of AU interactions and the issues these interactions pose remain valid.
11This work was not published as described, but other works contain these materials, including the currently in print volume What the Face Reveals by P. Ekman and E. Rosenberg (Eds.), Oxford University Press, 1997.
| |
|
|
.