A Human Face Asymmetries in Facial Actions
Title Page
Contents
Acknowledgements
Abstract
Introduction
Method
Results
Discussion
Summary and Conclusions
Tables
Appendix R
Appendix S
Appendix Y
References

Method

Subjects

Although gender might be an important variable affecting symmetry of facial actions (e.g., Alford & Alford, 1981; Borod & Caron, 1980), only women were studied in order to keep the sample size managable. Previous studies (Ekman, et al., 1981) indicated that the size of laterality effects observed required about 30 subjects for significant trends to emerge. Because 30 subjects was a substantial number to score, the subject sample was restricted to one gender. Women were chosen because previous studies of facial laterality had focused on women (e.g., Ekman et al., 1981; Schwartz et al., 1979).

Participants were recruited for this study with bulletin board advertisements placed around campus. Of the 37 women who completed the final experimental procedure, only the 33 righthanded Caucasians were retained for scoring and analysis. This selection was intended to make the sample as homogeneous as possible because right- and left-handers differ on many measures of hemispheric activity. The sample ranged in age from 18 to 53 with a mean of 27.5 years. They were paid $15 for their hour of participation.

Equipment

All of the facial actions subjects made were recorded on videotape for later analysis. The subject always faced directly into the camera lens so that straight ahead shots that were ideal for asymmetry scoring were obtained.

Previous researchers (e.g., Landis & Hunt, 1939; Ekman, Friesen, & Simons, in press) have frequently used acoustic stimuli to elicit startles, in part because they are perhaps the most effective and reliable for eliciting the startle response, including facial expressions, and are relatively easy to administer. Although firing a starter's pistol has commonly been used, this stimulus was inappropriate for this study for two important reasons. First, the sound pressure level created by such shots in the laboratory varied widely and might have been a hazard to subjects' hearing (Simons, personal communication). Second, a pilot study suggested that asymmetry of facial actions might depend upon the direction of the noise from the subject. It would not have been easy to control the directional properties of a gunshot or other point source stimulus in a small experimental room where the subject could vary the position of her head.

To solve these problems, startle sounds were 80 msec bursts of white noise produced electronically, amplified, and transduced by headphones worn by the subject and an array of speakers stacked directly behind her (see Appendix S for details). Sound pressure level from the speaker and headphone combination was 125 dB and was balanced to within 1 dB on either side of the subjects head so that the sound appeared to come from the center of the head. This arrangement allowed the control of stimulus intensity and minimized the effects of slight differences in the position of the subject's head by creating a wide sound field and by using headphones that moved with the subject. The imposing speaker arrays were hidden by a curtain in order to minimize any expectation about the magnitude of the stimulus.

All of the equipment for controlling the experiment was placed behind a partition so that the experimenter was hidden from view except when giving general instructions and questionaires. Thus, the subject could not see any subtle cues that the experimenter might have shown about how to do movements, when the unanticipated startle noise was to occur, etc.

Experimental Procedure

Figure 2 presents an outline of the experimental procedure that each subject completed individually. After a short, standard introduction, she gave consent and filled out the paper and pencil questionaires. These measures included a handedness questionaire (Johnstone, Galin, & Herron, 1979), an adjective self-rating scale (Veldman & Parker, 1970), the Social Desireability Scale (Crowne & Marlowe, 1964), and the Manifest Anxiety Scale (Taylor, 1953). The handedness questionaire was used to screen for right-handed subjects. The other measures were included to explore purported relations between asymmetry and personality and to check assertions that repression is the mediating variable in asymmetry (Schwartz, personal communication).

To summarize the startle part of the experiment, which was modeled after the procedure of Ekman et al. (in press), the subject simulated a startled expression twice and heard a loud noise in three different conditions. In the "unanticipated" condition the noise occurred unexpectedly. In the "anticipated" condition, the subject knew exactly when the noise would occur and reacted naturally. In the "inhibit" condition she knew exactly when the noise would occur and tried to act as though nothing had happened. Except for the "unanticipated" startle noise, each simulation or noise came after a countdown from ten to one. Between each noise was a five minute interview concerning her experience of the noise.

The first task was to make a startled expression at the tap of a pencil after the countdown. Ten seconds after making the simulated startle, she heard the unanticipated startle noise. During this interval, the experimenter attempted to shift her attention away from thinking about hearing a startle noise by asking her to help focus the camera by looking directly into the lens. This ploy not only minimized expectation of a noise, but also put her head in a good position for hearing the noise and for videotaping her response. No subject anticipated exactly when this noise would occur, even though all subjects knew that they would be startled sometime. The noise was completely unexpected for 24 subjects. The others thought the noise would occur sometime within the next few minutes of its actual occurrence. There were no differences between these two groups in the startle actions or their symmetry.

After the five minute interview, the subject heard the second noise, either in the anticipated or inhibit condition. These two conditions were counterbalanced across subjects, but were not counterbalanced with the unanticipated startle because it became more difficult to elicit an unanticipated startle as the experiment proceded. After the interview following the third noise, the subject again simulated a startle expression at the tap of a pencil. By obtaining a simulated startle both before and after the startle noises, any changes consequent to a recent startle experience could be examined.

The next part of the experiment was a modified version of Ekman and Friesen's Requested Facial Action Test (REFACT) (Note 1). First, the subject simulated six emotions: happy, sad, fear, anger, surprise, and disgust. The experimenter then asked "Now that this is done, aren't you glad its over?" in order to elicit a more spontaneous smile. This comment played on the difference between pretending to show an emotion and actually feeling it, and it appeared when tension from the demands of the task was relieved, a condition conducive to eliciting happiness (Tomkins, 1963). Demand for a deliberate or social smile was minimized because the experimenter was behind a curtain and the subject was not in a face-to-face interaction. Afterwards, the subject filled out rating scales that assessed how the subject went about making these emotion expressions and how difficult the task was.

Next, eight facial movements were verbally requested: smile, raise eyebrows up, lower eyebrows and pull them together, blink, pull upper lip up, wrinkle nose, stretch lip corners straight to sides, and squint. After completing these verbally requested actions, the subject rated how difficult each was to do. Then, the subject imitated 17 facial movements shown on a television monitor. These movements had been recorded earlier and were performed by an expert in facial movement so that they precisely matched the Action Units (AUs) described in the Facial Action Coding System (Ekman & Friesen, 1978). These actions included: raise eyebrows (AU 1+2), wrinkle nose (AU 9), pull lower lip down (AU 16), raise lip corners up (AU 12), pull eyebrows down and together (AU 4), raise eyelids (AU 5), tighten eyelids (AU 7), push chin up (AU 17), press lips together (AU 24), raise upper lip (AU 10), lower lip corners (AU 15), tighten lip corners (AU 14), stretch lip corners to side (AU 20), raise inner corners of brow (AU 1), raise outer corners of brow (AU 2). No coaching or verbal descriptions of the actions were given in this visually requested action condition.

The final task was to make muscle actions on only one side of the face without moving the other side. The ten actions requested were the same eight as those verbally requested previously, plus raising the inner and outer corners of the eyebrow. No mention was made of which side should show the action, but if a subject asked, she was told it did not matter. In the final task, the subject performed the same unilateral actions again, except that she was instructed to do an action first on one side, then on the other. Which side was requested first was counterbalanced across subjects.

Scoring Procedure

Scoring proceded in a series of steps, each completed before the next began. The first step was to score the five events in the startle part of the study. Only actions that were part of the startle response were scored. According to Ekman et al. (in press), startle actions that are either common or universal are AUs 6, 7, 20, 21, and 45, head movements, and shoulders and/or trunk up (see Figure 1). Following the scoring procedure of Ekman et al. (in press), any of these actions that began in the first 1/5th second after the noise were scored. Ekman et al. showed that this interval included the beginning of actions that comprise a characteristic startle response. In addition, idiosyncratic actions that were part of the startle response were also scored. Such actions were considered to be part of the startle if they onset before the end of the apex of AUs 6, 7, 20, or 21. In the simulate condition, AUs that began within 1/5th second of the onset of the first AU in the simulation were scored. Although actions in the simulation onset later than actions in the startle, the scoring interval included a distinct group of actions that onset at about the same time but excluded other actions that were apparently afterthoughts or signals to the experimenter. Each AU present in the simulation was identified and scored for asymmetry.

The second step in scoring was to score the verbally and visually requested facial actions in the same order as requested. All eight verbal requests were scored, but only the visual requests for AUs 1, 2, 1+2, 4, 7, 9, 10, 12, 15, 16, and 20 were scored. AUs 5, 14, 17, 22, and 24 were requested but not scored because they were not scored in previous studies and omitting them saved scoring time. For both verbal and visual requests, each individual AU was scored separately. Sometimes, a subject performed a requested action many times, but only a maximum of four actions were scored. If the subject performed the requested action four or fewer times, all were scored. If the action was performed more than four times, the first, second, and the last two times were scored. Whether or not an unrequested action co-occurred with the requested action did not affect this selection procedure for scoring, but the co-occurance of unrequested AUs was noted. By scoring the first two and the last two times each requested AU appeared, a more reliable asymmetry score was obtained and any change with practice could be examined without scoring the many movements that sometimes were produced. Requested blinks were scored both for strength and timing as in the startle conditions.

The third step in the scoring procedure was to score the emotion simulations: Happy, Sad, Fear, Anger, Disgust, Surprise, and the response to the question: "Aren't you glad its done?" For each simulation, the subject said "Okay, I'm done" when she was finished. The expression scored for asymmetry was the one immediately prior to this statement because this expression was likely to be the one that the subject thought was best and fulfilled the experimental instructions. To define the last expression, the scorer determined the AUs present just before the subject began to announce she was finished. Then, the onsets of these AUs were determined and the actions were scored for asymmetry as usual. Also scored for asymmetry were any actions that onset after the onset of the actions described above, but that offset before the end of the simulation.

The final step was to score the requests for unilateral actions. All requests were scored in the same order as requested in the experiment. Only the first action performed was included for analysis because this action was most likely to reveal the preferred side and the asymmetry present in actions that were not affected by recent practice. Only AUs that were correct actions for the AU requested were scored.

Measurement of Asymmetry

Measurement of asymmetry was similar to the procedure used by Ekman et al. (1981). Each individual muscular action was identified using Ekman and Friesen's Facial Action Coding System (FACS) (1978). The scorer located the beginning and apex of the action and assessed how much appearances were changed by the action in this interval. These changes were the basis for assigning FACS intensity scores to each side of the face separately. Then, the scorer used a video disk to look repeatedly in slow motion and real time for any differences in intensity between the two sides at the apex of the action (a single video frame showing both sides at greatest intensity). A score was assigned to indicate whether the action was symmetrical or asymmetrical. If asymmetrical, the score indicated how great the difference in intensity was, guided by the FACS intensity scores previously assigned.

The asymmetry scores provided two levels of measurement that were used in the analysis of this study. One level was simply whether the action or subject was right, left, or symmetrical. The other was a continuous measure from extremely left to symmetrical to extremely right, similar to that of Lynn and Lynn (1943). Each of these measures were useful for describing different aspects of the results, and the continuous measure permitted more complex analyses than the categorical measure.

Reliability

Reliability of asymmetry scoring was given careful attention in this study even though the main coder was very experienced in scoring asymmetry of facial actions and had demonstrated reliability in a previous study (Ekman et al., 1981). The purpose of the reliability studies was to re-confirm reliability and to investigate the reliability of scores for different Action Units. A more detailed account of these studies is contained in Appendix R.

One important issue arising from the fact that the experimenter was also the main coder was whether knowledge of the hypotheses could have biased scoring. Several precautions were taken to avoid this problem. In many instances, the potential for bias was minimized by the scoring procedure. All the actions in one condition were scored separately from actions in other conditions rather than scoring all the actions of each subject at once. Conditions were scored at least a month apart, making it virtually impossible for the coder to remember how subjects had been scored in different conditions. Conditions with the most obvious and easily scored asymmetries, unilateral actions, were scored last so that obvious patterns of asymmetry would not influence other, more difficult scoring. Even when the opportunity for bias to influence results was present, many of these results were opposite to important hypotheses and other hypotheses were not confirmed. This fact, along with the reliability coefficients discussed below, indicates that bias was not an important factor in the results.

The other important issue in reliability was whether differences in the asymmetry of actions were accurately scored, i.e., could be discriminated by more than one coder. To summarize, reliability was established between the main coder and three other coders. The Pearson correlation for all the actions scored for reliability was 0.72 (p<.0001). This coefficient was for scoring by the main coder versus scoring by all three reliability coders. The actions were sampled from all conditions in the study and included a variety of representative AUs. Scoring reliability varied with the AU, ranging from modest (about .5) to high (about 0.8). Coders agreed that some AUs (e.g., AU 6) were harder to score than others (e.g., AU 12). Harder AUs generally had lower reliability. The difficulty in discriminating the asymmetry of some actions suggests that the measurement procedure might not have been sensitive enough to detect real differences in asymmetry for these actions, so results for these actions should be interpreted with appropriate caution.

Kappa was used to assess the reliability of category scores. Because hypotheses about categories concerned only the relative number of right and left asymmetries, a Kappa was calculated for only these two categories. Across all AUs and coders, this Kappa was .88 (p<.0001) with 94 percent agreement. This coefficient indicates that when coders agreed that an action was asymmetrical, they almost always agreed about whether it was left or right. Reliability coders disagreed about whether an action was right or left on only two percent of all scores; almost all disagreements were about whether the action was symmetrical or not. The Kappa including the symmetrical category was .44 (p<.0001) with 64 percent agreement.

In addition to reliability coefficients, the scores of different coders provided the opportunity to compare results of the experiment as seen by different people. There was virtually no difference in results based on scores of different coders. This analysis of reliability scoring duplicated many of the major analyses reported below and showed that reliability coders produced the same results as the main coder. This agreement about results indicates that disagreements among coders about scores did not affect the findings about the phenomena studied.

Measures in the Study

Once asymmetry had been measured for each individual action, scores used in the analysis were calculated. There were three different behavioral scores for deliberate, requested actions and a score based on subjects' ratings of unilateral movements. The "bilateral asymmetry" score reflects the asymmetry of bilateral requested actions and is simply the asymmetry score as described above or averages of these scores if there was more than one action in a given condition. This bilateral asymmetry score was also used to analyze movements in the simulated startle and emotion conditions and the spontaneous action conditions. The other scores were only applicable to deliberate movements.

"Better unilateral control" scores were derived by calculating the difference between asymmetry scores for unilateral actions requested first on one side, then on the other (see Appendix Y for details). Thus, better unilateral control scores tell how much more asymmetrical the unilateral action was on one side than on the other. "Preferred side" scores are simply the asymmetry scores for requests to move one side without moving the other. In these requests, the side moved was chosen by the subject, i.e., it was the side she preferred for making unilateral movements, and the asymmetry score indicated the side she chose. Sometimes, subjects could not perform the requested unilateral action asymmetrically (i.e., the asymmetry score indicated symmetry). In such cases, preference could not be determined, and the subject was assigned to the "symmetrical" category. "Rated easier" scores were derived by calculating the difference between the subject's ratings of how easy it was to do the action unilaterally on each side of the face, providing a measure of how much easier one side was rated. Figure 3 describes these four measures for easy reference.

 

Procedure:

Type of Movement Obtained (Asymmetry Measure):

Paper and Pencil Questionaires

---

--Simulate Startle

Startle simulations

--Unanticipated startle

Spontaneous Startle Actions

--Anticipated startle - counterbalanced with the following -

"

--Inhibit startle

"

--Simulate startle

Startle simulations

--Simulate Six Emotions

Emotion simulations

--Humorous Comment by Experimenter

Spontaneous positive emotion

--Questionaire about Performance of Simulations

---

--Verbal Request for Eight Facial Actions

Deliberate actions (Bilateral Asymmetry)

--Questionaire about Difficulty of Above Actions

---

--Imitate Videotape of 17 Facial Actions

Deliberate actions (Bilateral Asymmetry)

--Request for Ten Unilateral Actions

Deliberate Unilaterals(Preferred Side)

--Same Requests except on Each Side Separately with Ratings of Difficulty

Deliberate Unilaterals (Better Unilateral Control) (Rated Easier)

NOTE: Simulations of emotion and startle produce a mixture of types of movement.

Measure

Derivation

Interpretation

Bilateral Asymmetry

Average of asymmetry scores for each AU elicited in specified condition.

Shows which side of bilateral actions had greater intensity of contraction and how much stronger it was.

Better Unilateral Control

Difference between the asymmetry scores for the right and left unilateral requested actions.

Defines the better unilateral action as the action that showed greater asymmetry and shows how much greater it  was. (see Appendix Y)

Preferred Side

Asymmetry score for the requested unilateral action when subjects were free to choose which side to move.

Shows side which was preferred in making a unilateral action and how asymmetrical this action was.

Rated Easier

Difference between the ratings of how easy the right and left unilateral requested actions were to perform.

Shows how much easier it was to do a unilateral action on one side versus the other.