1 Emotion Daniel Messinger, Ph.D.

2 Infant emotions Core elements of infant behaviorQuickly motivate behavior Hunger-Distress-Cry Interest-Attentive face Engaging playful other – joy - smile Organize action, physiology, cognition, & perception To meet environmental and internal demands Patterns constitute core aspects of temperament/personality functioning

3 Infant emotional developmentDistress is present at birth Interest and joy emerge in the first 2 mos. joy developing through at least 6 mos. Anger, sadness, fear differentiate after 4 m. Pride and shame develop between 1 & 2 years

4 Structuralist vs. functional emotion theoriesStructuralist (aka discrete, natural kinds) Emotions comprise unique patterns of subjective feeling, cognitive appraisal, physiological arousal, facial expressions Basic emotions promote survival and reproductive success

5 Discrete infant emotions

6 Is there emotional feeling without knowledge of feeling?Infantile memory Strong emotional associations Without explicit knowledge of associations Makes associations inaccessible to reflection and difficult to change Memories of smells, movements, even abuse

7 The Structuralist View“Many models assume that each emotion kind is characterized by a distinctive syndrome of hormonal, muscular, and autonomic responses that are coordinated in time and correlated in intensity “ p. 30 Barrett, 2006

8 But where are specific emotions?

9 Key brain regions implicated in emotion-related processing.

10 Facial affect programs?Current evidence: Relevant linked brain systems But not distinct affect programs Fear may be exception Panskepp and current animal work

11 What emotions do you see here?Cohn

12 Negative emotional expressions are not situationally specificThrough 2 months, Justine shows distress to bathing, being moved, & pacifier removal (inoculation and hunger) After 2 months, anger and, to a much lesser degree, sadness are most common reaction to all negative elicitors infants cry, not a specific reaction Camras, 1992

13 Infant negative expressions rated as distress(Oster et al., 1992)

14 Situational appropriateness: Production studiesPremise: In response to an appropriate elicitor (situation), hypothesized emotional expression should occur significantly more than other expressions

15 Examples Sad  distresssmile: Distress: Saddisress: https://www.youtube.com/watch?v=l7oD9WX-1CU SmiloeFear/orientdistress: Feardistress: (alligator bite) Sad: dad singing (lower lip in response to rasberries) Examples (Slides 3-10 are pictures) :

16 Maze game—Scary—children Slow-motion: Compendium: Long: 0:55-1:07, 1:45-2:30

17 Surprise! Its not in the faceCovert toy switch

18 Surprise examples Expression on demand: Coordinative structure? Posed adult: Girl and Dad 1:05—1:40.

19 Emotion (cont) Structuralist vs. functional perspectives on emotion (cont) Functionalist Emotions serve to establish, maintain, or change relation between person and environment on matters of significance to person

20 Developmental patternsSocialization Emotion displays become more restricted Full-face to partial face - miniaturization Cognitive input shame, guilt, contempt emerge involve rudimentary appraisal of self vis-à-vis other dynamic systems

21 Developmental Changes in EmotionPsychobiological foundations Subcortical mediation of basic emotions Developing subcortical-frontal connections permit more effective emotion regulation Emotion Perception Discrimination/categorization of expression by 5 months of age Rely on others’ reactions to interpret unfamiliar situations = social referencing (12+ months) Understanding of subjective state of emotion (24+ months), allows for prosocial displays of comforting etc.

22 Developmental Changes in Emotion (cont)Emotion and Self-Development Increases in self-awareness (2/3 yrs) leads to expression of new, more complex emotions Self-Conscious Emotions Pride Guilt Shame Embarrassment

23 Developmental Changes in Emotion (cont)Understanding effects of emotions on others: The use of display rules Increased ability to understand and apply social rules for display of emotion in social situations Emotion masking Primitive forms in preschool; more flexible, reasoned use in middle childhood

24 Developmental Changes in Emotion (cont)Emotion Regulation Adaptive management of emotional experiences Developmental transition from other-regulation to self-regulation Internalization of socialization experiences

25 Dynamic systems Development, interaction, and (emotional) behavior are complex involving multiple interfacing/interacting constituents which produce patterns we see as pre-designed regularities A bottom-up approach Discrete emotions as preferred states formed from the interface of multiple constituents

26 Facial expression communicationA systems view + Eye constriction - Eye constriction Positive Emotion Positive Rating 3 4 5 6 7 8 Negative Rating Messinger, 2002

27 But what about dynamic expressions?Messinger, Mattson, Mahoor, & Cohn, Emotion, 2012 Bolzani-Dinehart, et al., 2005

28 Automated MeasurementComputer Vision Pattern Recognition

29 Intensifiers of positive and negativeMessinger , Mattson, Mahoor, & Cohn, 2012

30 Intensifiers of positive and negativeMessinger , Mattson, Mahoor, & Cohn, 2012

31

32 Signals a focus on internal state to self and otherEye constriction has a common meaning—intensification—across positive and negative expressions Isre icon Signals a focus on internal state to self and other

33 Darwin’s Duchenne: Eye Constriction during Infant Joy and Distress(Mattson et al., 2013)

34 Darwin’s Duchenne (Mattson et al., 2013)What We Know… Research has supported Darwin’s proposal that smiles with eye constriction (Duchenne smiles) are indices of strong positive emotion In both infants & adults, Duchenne smiles are… A more frequent response to positive emotion elicitors Perceived as more joyful than other smiles Following early research by 19th century French neurologist Duchenne de Boulogne (who identified two types of smiles), Darwin highlighted the role of eye constriction (orbicularis oculi, pars orbitalis) in facial expressions of positive emotion Duchenne Smile: Engages multiple facial muscles and elicits wrinkling around the eyes

35 Darwin’s Duchenne (Mattson et al., 2013)What We Don’t Know… Darwin also proposed that eye constriction plays a role in weeping/crying (especially during infancy), but little parallel research has examined cry-faces No study has simultaneously examined the role of infant eye constriction in both smiling & cry-face expressions in reaction to experimental elicitation of positive and negative emotion Cry Face = Open-mouth expression with lateral lip stretching

36 Darwin’s Duchenne (Mattson et al., 2013)Q: Does eye constriction index the affective intensity of both positive and negative emotions ? Hypotheses: Duchenne marker (eye constriction) expected to be associated with both stronger smiles and cry faces; smiles during play would be more emotionally positive than still face; cry-faces would be more emotionally negative in still-face than play. Fixation Duration: Time between saccadic eye movements when eyes are relatively stable Infant attention correlates with attention in toddlerhood/preadolescence Infant attention correlates with IQ at age 11 Clennan

37 Darwin’s Duchenne (Mattson et al., 2013)Q: Does eye constriction index the affective intensity of both positive and negative emotions ? Fixation Duration: Time between saccadic eye movements when eyes are relatively stable Infant attention correlates with attention in toddlerhood/preadolescence Infant attention correlates with IQ at age 11

38 Darwin’s Duchenne (Mattson et al., 2013)Study 1: Face-to-Face/Still Face (FFSF) Play elicits more positive emotion (i.e. higher proportions of infant smiling) than the Still-Face; the Still- Face elicits more negative emotion (i.e. a higher proportion of cry- face expressions) than Play. Based on these findings, we reasoned that smiles during Play would be more emotionally positive than smiles in the Still-Face, and that cry-faces in the Still-Face would be more emotionally negative than cry-faces during Play. Accordingly, we hypothesized that a greater proportion of smiles would involve eye constriction during Play than during the Still- Face; and that a greater proportion of cry-faces would involve eye constriction during the Still-Face than during Play. In the current dataset and others [26], however, the strength of eye constriction covaries with the strength of smiles and cry-faces [9]. Consequently, we expected the Duchenne marker to be associated with both stronger smiles and cry-faces. PLAY 3 minutes Elicits more positive emotion Smiles during play more emotionally positive – Duchenne? STILL FACE 2 minutes Elicits more negative emotion Cry-faces during still face more emotionally negative – Duchenne?

39 Darwin’s Duchenne (Mattson et al., 2013)PARTICIPANTS 12 6-month old infants + parents (11 moms, 1 dad) 4 EA, 4 Hispanic American, 2 AA, 2 Asian American 66.7% male METHOD Facial Coding Repeated-measures ANOVAs Participants. Twelve six-month-olds and their parents (11 mothers, 1 father) were video-recorded in the FFSF [9] to elicit a range of negative and positive infant emotional expressions. Play lasted three minutes and the Still-Face lasted two minutes. The six- month-olds (M = 6.20, SD = 0.43) were 66.7% male and ethnically diverse (16.7% African American; 16.7% Asian American, 33.3% Hispanic American, and 33.3% European American). Clennan

40 Darwin’s Duchenne (Mattson et al., 2013)Study 1: Face-to-Face/Still Face (FFSF) PARTICIPANTS 12 6-month old infants + parents (11 moms, 1 dad) 66.7% male 2 4 2 Participants. Twelve six-month-olds and their parents (11 mothers, 1 father) were video-recorded in the FFSF [9] to elicit a range of negative and positive infant emotional expressions. Play lasted three minutes and the Still-Face lasted two minutes. The six- month-olds (M = 6.20, SD = 0.43) were 66.7% male and ethnically diverse (16.7% African American; 16.7% Asian American, 33.3% Hispanic American, and 33.3% European American). 4 METHOD Facial Coding Repeated-measures ANOVAs

41 Darwin’s Duchenne (Mattson et al., 2013)Study 1: Face-to-Face/Still Face (FFSF) RESULTS Figure 1. Time in smiling and cry-faces as a proportion of time in each episode of the Face-to-Face/Still-Face (FFSF) Smiles & cry-faces distributed differentially in Play & Still-Face episodes Overall still-face effects Figure 1. Time in smiling and cry-faces as a proportion of time in each episode of the Face-to-Face/Still-Face (FFSF). Error bars indicate standard errors of the mean. The images are of smiles and of cry-faces without eye constriction. The images are from a six-month-old in the FFSF in the current study. Written informed consent, as outlined in the PLOS consent form, was obtained for publication of these images. We posited that eye constriction would be differentially distributed with smiles and cry-faces during the FFSF. To lay the groundwork for testing this hypothesis, we used repeated- measures ANOVAs to ascertain whether there were still-face effects in the overall levels of smiles and cry-faces in the FFSF. Smiles and cry-faces were distributed differentially in the Play and Still-Face episodes of the FFSF, F (2, 22) = 7.24, p,.01, gp2 = .40 (see Figure 1). The mean proportion of time involving smiling declined from Play (M = .13, SD = .08) to the Still-Face (M = .02, SD = .03), F (1, 11) = 17.10, p,.01, gp2 = .61. The mean proportion of time involving cry-faces increased from Play (M = .11, SD = .25) to the Still-Face (M = .32, SD = .29), F (1, 11) = 4.97, p,.05, gp2 = .31. These overall still-face effects are the background against which we test whether the proportion of smiles involving eye constriction and the proportion of cry-faces involving eye constriction vary systematically over the course of the FFSF.

42 Darwin’s Duchenne (Mattson et al., 2013)Study 1: Face-to-Face/Still Face (FFSF) RESULTS Figure 2. Eye constriction is differentially associated with smiles & cry-faces in FFSF Greater proportion of smiles in Play with eye constriction Greater proportion of cry-faces in Still-Face with eye constriction Figure 2. Eye constriction (the Duchenne marker) is differentially associated with smiles and cry-faces in the Face-to-Face/ Still-Face (FFSF). Mean proportions of smiles and of cry-faces occurring with eye constriction. Error bars indicate standard errors of the mean. The images are of smiles and cry-faces with eye constriction. The images are from a six-month-old in the FFSF in the current study. Written informed consent, as outlined in the PLOS consent form, was obtained for publication of these images. A repeated-measures ANOVA indicated that proportions of smiling and cry-faces that involved eye constriction were differentially distributed over episodes of the FFSF, F (2, 22) = 11.28, p,.001, gp2 = .51 (see Figure 2). A higher proportion of smiles involved eye constriction during Play (M = .64, SD = .30) than did smiles during the Still-Face (M = .34, SD = .38), F (1, 11) = 6.70, p = .03, gp2 = .38. A higher proportion of cry-faces involved eye constriction during the Still-Face (M = .70, SD = .38) than during Play (M = .36, SD = .39), F (1, 11) = 11.43, p,.01, gp2 = .51. These results provide evidence in support of the hypothesis. In a positive-emotion eliciting context, smiling was more likely to be accompanied by eye constriction. In a negative- emotion eliciting context, cry-faces were more likely to be accompanied by eye constriction. Hypothesis

43 Experimental evidence for role of eye constrictionMattson, et al.,PLOS One, in press Mattson, et al., 2013, under review

44 Darwin’s Duchenne (Mattson et al., 2013)DISCUSSION During play, infant smiles are more emotionally positive than when trying to elicit a response from non-responsive parent When stymied by parent, infant cry-faces are more emotionally negative than cry-faces during play Duchenne marker (eye constriction) is associated with BOTH stronger smiles & stronger cry-faces The results indicate that when infants are engaged in play with a parent, their smiles are more emotionally positive than when they are trying to elicit a response from a non- responsive parent. Likewise, when infants are stymied by their non-responsive parent, their distress expressions are more emotionally negative than distress expressions that occur during play. Finally, the Duchenne marker, eye constriction, accompa- nied both stronger smiles and stronger cry-faces. Study 1 results add to a growing body of research suggesting that smiling with eye constriction is stronger and more likely to occur in situations that elicit positive emotion than smiles without eye constriction [5,6,34]. Early distress expressions that involved eye constriction also tended to be stronger and were more likely to occur in periods intended to elicit negative emotion than distress expressions without eye constriction. To ascertain the generality with which infants respond to strong elicitors of negative emotion with cry-faces involving eye constriction, we next examined responses to naturally occurring vaccination injections.

45 Darwin’s Duchenne (Mattson et al., 2013)Study 1: Face-to-Face/Still Face (FFSF) DISCUSSION During play, infant smiles are more emotionally positive than when trying to elicit a response from non-responsive parent When stymied by parent, infant cry-faces are more emotionally negative than cry-faces during play Duchenne marker (eye constriction) is associated with BOTH stronger smiles & stronger cry-faces The results indicate that when infants are engaged in play with a parent, their smiles are more emotionally positive than when they are trying to elicit a response from a non- responsive parent. Likewise, when infants are stymied by their non-responsive parent, their distress expressions are more emotionally negative than distress expressions that occur during play. Finally, the Duchenne marker, eye constriction, accompa- nied both stronger smiles and stronger cry-faces. Study 1 results add to a growing body of research suggesting that smiling with eye constriction is stronger and more likely to occur in situations that elicit positive emotion than smiles without eye constriction [5,6,34]. Early distress expressions that involved eye constriction also tended to be stronger and were more likely to occur in periods intended to elicit negative emotion than distress expressions without eye constriction. To ascertain the generality with which infants respond to strong elicitors of negative emotion with cry-faces involving eye constriction, we next examined responses to naturally occurring vaccination injections.

46 Darwin’s Duchenne (Mattson et al., 2013)Q: Do the cry-face results of the FFSF generalize to a naturalistic elicitor of intense negative emotion ? Fixation Duration: Time between saccadic eye movements when eyes are relatively stable Infant attention correlates with attention in toddlerhood/preadolescence Infant attention correlates with IQ at age 11

47 Darwin’s Duchenne (Mattson et al., 2013)Study 2: Vaccinations PARTICIPANTS Identified through YouTube videos of infant vaccinations 12 6-month old & 12-month old infants METHOD Facial Coding of 10 seconds following first injection Repeated-measures ANOVAs Participants. Twelve six-month-olds and their parents (11 mothers, 1 father) were video-recorded in the FFSF [9] to elicit a range of negative and positive infant emotional expressions. Play lasted three minutes and the Still-Face lasted two minutes. The six- month-olds (M = 6.20, SD = 0.43) were 66.7% male and ethnically diverse (16.7% African American; 16.7% Asian American, 33.3% Hispanic American, and 33.3% European American).

48 Darwin’s Duchenne (Mattson et al., 2013)Study 2: Vaccinations RESULTS Most cry-faces involved Duchenne marker 6-month olds = 12-month olds in… Proportion of time involving crying faces Proportion of cry-faces with eye constriction Results: We determined the overall proportion of time involving cry-face expressions after injections and tested whether this proportion varied with age. The mean proportion of time involving cry-faces was.50. There was not a significant difference in the proportion of time involving cry-faces at 6 months (M = .43, SD = .25) compared to 12 months (M=.56, SD=.24), F (1, 22)=1.71, p=.20. We next examined the proportion of time in which cry-faces were accompanied by eye constriction. The mean proportion of time in cry-faces which involved eye constriction was.87. There were no differences in the proportion of cry-faces involving eye constriction at 6 (M=.82, SD=.28) and 12 months (M=.92, SD=.07), F (1, 21) = 1.43, p = .24. These results indicate that in an intense negative-emotion eliciting context, cry-faces were likely to be accompanied by eye constriction at both 6 and 12 months of age. That is, the overwhelming majority of cry-faces occurring in response to a painful elicitor of negative emotion involved the Duchenne marker. Discussion: Study 2 vaccination findings extend the Study 1 FFSF cry-face results to a naturalistic elicitor of intense negative emotion. Immediately following vaccination injections, the overwhelming majority of cry-faces involved eye constriction at both 6 and 12 months. These results provide an anatomically specific documen- tation of infant eye constriction responses to painful injections In response to this elicitor of intense negative emotion, infants combined a cry-face expression with eye constriction, the same Duchenne marker they combine with smiling during the elicitation of intense positive emotion during play. These cry-faces involving eye constriction, termed Duchenne distress expressions (see Fridlund) [35], were the predominant responses both to parents’ abruptly ceasing playful interaction and to a noxious stimulus. DISCUSSION Extends results to naturalistic elicitor of negative emotion “Duchenne distress expression” is predominant response to both parents ceasing play & noxious stimulus

49 Darwin’s Duchenne (Mattson et al., 2013)Conclusions Results support hypotheses Eye constriction indexes the affective intensity of both positive & negative facial configurations “Duchenne distress expression” parallels “Duchenne smile” Facial action has consistent function in various facial expressions Parsimonious way to communicate emotional intensity Eye constriction Possible functions: (1) Regulates exposure to intense emotional stimuli, (2) Increase attention to internal emotional state Signals:  Intense positive engagement &  Intense need for comfort In adults: Pain, Orgasm Ultimately these findings support the contention of Darwin and others that a given facial action may have a consistent function in a variety of facial expressions [36,37,38]. Eye constriction (obicularis occuli pars orbitalis), the Duchenne marker, reduces but does not completely occlude the visual field. This suggests that the Duchenne marker may regulate exposure to intense emotional stimuli [34], and may also increase the expresser’s attention to his or her own internal emotional state [8]. Duchenne smiling and Duchenne distress appear to serve, respectively, to communicate intense positive engagement and an intense need for comfort. Smiling and the cry-face expression are the infant’s most commonly used facial configurations, suggesting the importance of eye constriction to early emotion expression and communication [12,13]. In adults, the role of eye constriction is well documented in Duchenne smiling [3,4,7]. Eye constriction is also a key element of the adult pain configuration [21,22], and may index the intensity of this expression which, when it is maximally displayed, is an adult analog of the infant cry-face. The same expressive configuration is present during other intense experiences such as orgasm [39], suggesting that eye constriction is associated with the intensity of facial expressions of extreme positive and negative valence in adults.

50 Darwin’s Duchenne (Mattson et al., 2013)Future Directions What about facial expression of other emotions? What about other modalities of emotional expression? Which other modalities are important for the expression of emotion? How do we make sense of inconsistent signals from different modalities of emotional expression? From an evolutionary perspective, eye constriction provides a parsimonious means for indexing the intensity of both positive and negative emotions. However, an evolutionary focus on the function of facial movements for the organism in its environment suggests that eye constriction will not have an identical role in all (adult) expressions of negative emotion [22]. Facial expressions of fear, for example, may serve to enhance sensory input, widening the visual field to facilitate quick defensive reactions [40]. Anger configurations in adults also involve eye opening (AU5) [15], which is likely to facilitate and communicate potential aggression to the target [35,41]. This suggests that the intensity of fear and anger might be indexed by eye opening rather than eye constriction. Disgust, by contrast, is characterized by rejecting sensory input through eye and nostril constriction [40]. Eye constriction might index the intensity of disgust, and of sadness, which may also involve a narrowing of the visual field in adults, but this remains a topic for future research. Ultimately, the current focus on the general functions of facial actions across a range of expressions (see Susskind et al.) [40] is likely to produce new insights into both expression-specific and pan-expression features of expressive action. The face is one modality of emotional expression. Aviezer, Trope, and Todorov found that other modalities of expression such as body posture carry more weight than the face when adults rate the valence of still images of other adults during positive and negative events [42]. Nevertheless the facial expressions in response to both positive and negative events (e.g., winning or losing a match point during a tennis match) exhibited in Aviezer et al.’s figures (Figures 1–4) [42] all involved eye constriction and/ or eye shutting, potentially underlining the role of these actions in communicating intense positive and negative emotional valence.

51 Darwin’s Duchenne (Mattson et al., 2013)Discussion Qs What are some other directions for future research? How could this work be extended? What are some potential implications for development? How might Duchenne expressions impact our perceptions & behavior? More time-points/extend beyond 6 and 12 months to examine developmental trajectory; compare to adults If studying children or adults, use self-report likert scale to match self-reported intensity with coded intensity 2. No change across development? Or only within the 1st 12 months? Do we change in the proportion of intense expressions as we age? 3. Other possible social functions? Communication beyond internal state – for a smile: trustworthiness, appeasement, friendliness? What about Duchenne distress? Are we more likely to help or offer comfort? Professional Photos Botox

52 Darwin’s Duchenne (Mattson et al., 2013)DISCUSSION QUESTIONS: How can these findings be extended (i.e. clinical implications)? Do you think these findings can be replicated in an adult population? If the findings are similar in an older sample, what does this imply? How might these Duchenne expressions (eye constriction) impact our behavior? From and evolutionary perspective, how are Duchenne expressions beneficial? 2. Communication beyond internal state – for a smile: trustworthiness, appeasement, friendliness? What about Duchenne distress? Are we more likely to help or offer comfort? 3. From an evolutionary perspective, eye constriction provides a parsimonious means for indexing the intensity of both positive and negative emotions. However, an evolutionary focus on the function of facial movements for the organism in its environment suggests that eye constriction will not have an identical role in all (adult) expressions of negative emotion [22]. Facial expressions of fear, for example, may serve to enhance sensory input, widening the visual field to facilitate quick defensive reactions [40]. Anger configurations in adults also involve eye opening (AU5) [15], which is likely to facilitate and communicate potential aggression to the target [35,41]. This suggests that the intensity of fear and anger might be indexed by eye opening rather than eye constriction. Disgust, by contrast, is characterized by rejecting sensory input through eye and nostril constriction [40]. Eye constriction might index the intensity of disgust, and of sadness, which may also involve a narrowing of the visual field in adults, but this remains a topic for future research. Clennan

53 Yearbook pictures …and life Messinger

54 Smile intensity & self-reported personalityMessinger

55 Smile intensity & other-reported personalityMessinger

56 Smile intensity and Observer Expected Interactions (n=114)Observer Expectations Positive emotional expression Expected positive emotions .70 Expected negative emotions -.57 Approach-acceptance .52 Messinger

57 Smile intensity and Life OutcomesPositive expression Controlling for Attract./Social Desirability Married by age 27 .19 .18/.16 Single into adulthood -.20 -.18/.20 Ever divorced .15 ~.15/~.15 Personal Well-being Age 21 (n=112) .20 .20/.11 Age 27 (n=86) .25 .26./.23 Age 43 (n=105) .18 .19/12 Age 52 (n=101) .27 .28/.24 Messinger

58 Expressions by behavior ratingWhy? 80th%ile. Higher % of their own expressions Keltner et al., 1999 Messinger

59 Functionalist theory Emotion is the person’s attempt or readiness to establish, maintain, or change the relation between the person and the environment on matters of significance to that person (Saarni et al., 1998). Emotion is associated with goal-attainment, social relationships, situational appraisals, action tendencies, self-understanding, self regulation, etc. Messinger

60 Halloween Candy Messinger

61 Emotions Organize action, physiology, cognition, and perception to meet ever-changing environmental and internal demands In patterns constituting core aspects of temperament/personality functioning Motivate action and thought, creating value in life—and impacting wellness and sickness