Asymmetric Effect of Expression Intensity on Evaluations of Facial Attractiveness

Many studies have shown that facial expression influences evaluations of attractiveness, but the effect of expression intensity remains unclear. In the present study, participants rated the expression intensity and attractiveness of faces with happy, neutral, or sad expressions. Sad faces, as anticipated, were judged as less attractive than neutral and happy faces. Among happy expressions, faces with more intense expressions were considered more attractive; for sad expressions, there was no significant relationship between rating and intensity. Multiple regression analyses further demonstrated that the attractiveness of a face with a sad expression could be predicted only by its baseline attractiveness (i.e., ratings of neutral expressions). We conclude that the intensity of positive and negative expressions asymmetrically influences evaluations of the attractiveness of a face. We discuss the results in terms of emotional contagion or sympathy.


Background
Our faces provide much information about our emotions and intentions. Facial characteristics and impressions can serve as crucial cues in social communication. In daily life, we often change our behavior depending on the information derived from others' faces. For example, various studies have indicated the presence of a "beauty-is-good" stereotype (Dion, Berscheid, & Walster, 1972), according to which attractive people are viewed as socially desirable (for a review, see Langlois et al., 2000) or competent (Hamermesh & Parker, 2005) or are more likely to be hired (Marlowe, Schneider, & Nelson, 1996;Watkins & Johnston, 2000).
Whereas these aspects of physical attractiveness are largely static and unchangeable, we can dramatically and instantly change our facial expressions. By doing so, we can send various messages to others; conversely, we tend to change our own attitude toward others depending on their facial expressions. Smiles and other happy expressions make a positive impression and increase a person's attractiveness (Reis et al., 1990). A study using economic games indicated that a happy face increases the perceived trustworthiness toward and elicits greater tolerance of a person (Scharlemann, Eckel, Kacelnik, & Wilson, 2001). Neuroimaging studies have demonstrated that this increased attractiveness of happy expressions is based on the reward system (O'Doherty et al., 2003; for a review, Phan, Wager, Taylor, & Liberzon, 2002). In contrast, negative expressions including sadness, anger, or disgust decrease attractiveness (Mueser, Grau, Sussman, & Rosen, 1984).
Haxby's well-known model (Haxby, Hoffman, & Gobbini, 2000), according to which distributed neural systems are associated with facial cognition processes, has been supported by numerous studies that examined the neural connections involved in facial perception (e.g., Winston, Henson, Fine-Goulden, & Dolan, 2004). It has also been assumed that two interacting criteria: rewarding beauty and aesthetic beauty, contributing to the evaluation of facial attractiveness are represented in the brain (Senior, 2003). A later experimental study has suggested that facial attractiveness represents a combination of sexual processes and aesthetic processes (Franklin & Adams, 2010). Other studies have confirmed that dynamically changing one's facial expression can influence evaluations of facial attractiveness.
Such evaluations may be affected not only by the type of facial expression but also by its intensity. Several studies have shown that, even in comparison with other happy expressions, intensely happy faces are rated as more attractive (Golle, Mast, & Lobmaier, 2014) or trustworthy (Oosterhof & Todorov, 2009;Schmidt, Levenstein, & Ambabar, 2012). In comparison with such studies of positive expressions, fewer reports examine negative expressions. Presumably, if one strongly expresses a negative emotion, observers should easily notice it, as can be predicted from research indicating that seeing a sad face elicits sadder feelings from the observer and encourages helping behaviors when compared with happy or neutral faces (Small & Verrochi, 2009). Social psychology studies have reported that attractive individuals receive more help, especially when the severity of an emergency is high (Benson, Karabenick, & Lerner, 1976;Harrell, 1978;West & Brown, 1975). Thus, a sad expression can indicate a need for help, in which attractiveness can play an important role. However, whether the intensity of a sad expression is also associated with the evaluation of attractiveness remains unknown. A more intense sad expression can indicate a more serious need for help; thus, we can predict a positive correlation between the intensity of a sad expression and an evaluation of the attractiveness of the faces.
In this study, we investigated the relationship between intensity and attractiveness of facial expression in both positive and negative expressions. The results demonstrated, unsurprisingly, that compared with neutral faces, happy faces were judged as more attractive and sad faces as less attractive. In a more novel finding, intensity of expression asymmetrically influenced the attractiveness of happy or sad faces.

Participants
Thirty volunteers (15 males; M age = 21.6 years) with no history of psychiatric disease participated in the study; all of them were compensated for their participation. After receiving a detailed description of the study, all the participants provided written informed consent in accordance with the Declaration of Helsinki and guidelines of the Japanese Psychological Association.

Stimuli
Forty-four Japanese facial images (16 males; age range: 18-25 years) were used as stimulus material. Each image was shown in four facial expressions (happy, angry, neutral, and sad), resulting in a total of 176 facial images. All of them were used in the first task (the expression intensity rating task), but only females' happy, neutral, and sad facial images were used in the subsequent task (the attractiveness rating task).
Facial images were first captured in movie form as the models displayed each expression by imitating the sample expression images from Ekman and Friesen (1976). Then, still images were cut off from the movies, and the surroundings of the face (e.g., hair or ornaments) were hidden so that they did not influence the evaluations (see Figure 1). The face images subtended roughly 17° horizontally and 23° vertically of visual angle at a standard viewing distance of 45 cm. This was accomplished using Adobe Premiere Pro and Photoshop. In the experimental tasks, each stimulus was presented in color on a 24.1-inch liquid crystal display (1920 × 1200 pixels), and the participant observed it, using a chin rest.

Experimental Paradigm
The experimental procedure consisted of two tasks for each participant: the expression intensity rating task and the attractiveness rating task. All participants completed the two tasks in this order.
Expression intensity rating task. In each trial, facial images were randomly presented on the screen, and each participant rated how intensely the person appeared to be happy, angry, disgusted, sad, surprised, fearful, and contemptuous on a 7-point scale (1 = not at all, 7 = very intense). The evaluation items were presented in Japanese, in this order. Facial images consisted of males and females' happy, angry, neutral, and sad faces. Participants rated the seven evaluation items for each stimulus with no time limit, and the face stimulus remained present until the participant had completed all ratings (see Figure 1). Between trials, a blank screen was presented for 100 ms.
Attractiveness rating task. For three reasons, we did not use male or female faces with angry, disgusted, surprised, fearful, and contemptuous expressions in the subsequent attractiveness rating task. First, our initial aim was to compare the effect of positive and negative expressions, so we used just one negative expression (i.e., sad faces) as a stimulus for Note. Each participant rated how intensely the person appeared to be happy, angry, disgusted, sad, surprised, fearful, and contemptuous on a 7-point scale (1 = not at all, 7 = very intense), in that order. There was no time limit for making these judgments, and the facial stimulus remained present until all ratings of that stimulus had been completed.
simplicity. Second, we included more rating categories than facial expressions for both genders in the intensity ratings because we wanted to prevent participants from recognizing the aim of our experiment. Finally, we wanted to avoid possible confounding factors caused by the gender of the stimuli, so we used only one gender in the attractiveness rating task.
In each trial of this task, facial images were presented on the screen in a random order, and the participant rated how attractive each person appeared on a 9-point scale (1 = very unattractive, 9 = very attractive). We used only facial images of females' happy, neutral, and sad faces as stimuli. Between trials, a blank screen was presented for 200 ms. Figure 2A shows the mean ratings in the expression intensity rating task. Only the rating data for females' happy and sad stimuli are shown here, because, as explained above, these were the only ratings relevant to the experiment's purpose. We conducted a two-way ANOVA of the mean ratings, with the participants' gender (male or female) as a between-subjects factor and the stimuli's facial expression (happy or sad) as a within-subjects factor. The ANOVA revealed that the stimuli's facial expression, F(1, 54) = 54.85, p < .001, η p 2 = 0.50, η G 2 = 0.29, presented significant main effects, whereas the main effect for the participant's gender, F(1, 54) = 0.85, p = .36, η p 2 = 0.02, η G 2 = 0.01, and the interaction effect, F(1, 54) = 1.36, p = .25, η p 2 = 0.02, η G 2 = 0.01, were not significant. Thus, participants rated happy expressions as more intense than sad expressions. Figure 2B shows the mean ratings in the attractiveness rating task. We also conducted a two-way ANOVA of the mean ratings in this second task, with the participant's gender (male or female) as a between-subjects factor and the stimuli's facial expression (happy, neutral, or sad) as a within-subjects factor. It yielded significant main effects of the participant's gender, F(1, 54) = 6.74, p = .01, η p 2 = 0.11, η G 2 = 0.08, and of the stimuli's facial expression, F(2, 108) = 99.12, p < .001, η p 2 = 0.65, η G 2 = 0.34, whereas the interaction effect was not significant, F(2, 108) = 1.28, p = .28, η p 2 = 0.02, η G 2 = 0.01.

Mean Ratings
Then, we conducted a post hoc analysis that involved multiple comparisons of the stimuli's facial expressions (happy, neutral, or sad) using Shaffer's modified sequentially rejective Bonferroni procedure. This analysis determined that happy expressions were judged as more attractive than neutral and sad expressions, and that sad expressions were judged as less attractive than neutral expressions (all p < .001). The rating of faces with happy expressions as most attractive and sad expressions as most unattractive was consistent with the results of previous studies (e.g., Mueser et al., 1984;Reis et al., 1990).

Correlation Analyses
Next, we directly examined whether the intensity of facial expressions was correlated with the evaluation of attractiveness. For this purpose, we computed Pearson's correlation between the ratings for expression intensity and attractiveness in happy and sad expressions across stimuli. As our measure of attractiveness ratings, we used the differences in attractiveness rating scores between happy or sad faces and neutral faces. First, we calculated the mean ratings of each facial stimulus by all participants, and we confirmed that there was no facial stimulus with mean rating scores (for both intensity of expression and attractiveness) more than 3 standard deviations above or below the overall mean. We therefore used the mean rating scores for all stimuli in the correlation analyses. For happy expressions, there was a significant positive correlation between expression intensity ratings and attractiveness scores (r = .50, p = .01; Figure 3A). In contrast, there was no significant correlation between expression intensity ratings and attractiveness scores for sad expressions (r = −.21, p = .27; Figure 3B). In summary, for happy expressions, a face with a more intense expression was seen as more attractive, consistent with a previous study (Golle et al., 2014), but there was no relationship  between intensity of expression and attractiveness for sad expressions. Note that, however, these analyses ignored the effect of the baseline attractiveness of each face (i.e., the attractiveness ratings for neutral expression). We therefore went on to conduct multiple regression analyses.

Multiple Regression Analyses
From the correlation analyses, we found an asymmetrical effect of expression intensity on the attractiveness of happy or sad faces, in that evaluations of the attractiveness of sad faces were unrelated to the intensity of expression. To investigate in greater detail the features that explain the attractiveness of happy or sad faces, we conducted multiple regression analyses to confirm the relative impact of the intensity of expression for attractiveness of stimuli. We conducted separate analyses for happy and sad expressions.
The participants' attractiveness ratings for each stimulus with a happy expression were analyzed with a linear mixedeffects model (Baayen, Davidson, & Bates, 2008) using the packages lme4 (Bates, Maechler, Bolker, & Walker, 2014) and lmerTest (Kuznetsova, Brockhoff, & Christensen, 2015), available for R statistical software (R Core Team, 2014). We included each stimulus's baseline attractiveness (i.e., the attractive ratings for each stimulus with a neutral expression), the intensity of happy expressions (i.e., the expression intensity ratings for each stimulus with a happy expression), and their interaction effect as fixed effects. All variables were mean centered prior to the analysis. We also included random intercepts for stimuli and participants (Barr, Levy, Scheepers, & Tily, 2013). The model's two kinds of R 2 (one is Marginal R 2 , representing the variance explained by fixed factors; the other is Conditional R 2 , representing the variance explained by both fixed and random factors) were also estimated using the R package MuMIn (Barton, 2015). The regression analysis (Marginal R 2 = .14, Conditional R 2 = .60) demonstrated significant main effects of the baseline attractiveness (B = 0.42, t = 13.37, p < .001) and the intensity of happy expressions (B = 0.12, t = 2.68, p = .01), but there was no significant interaction effect (B = 0.02, t = 1.30, p = .19), confirming that both the baseline attractiveness and the intensity of expressions influence the attractiveness evaluation of faces with a happy expression.
Next, the participants' attractiveness ratings for each stimulus with a sad expression were analyzed in a similar way. We included each stimulus's baseline attractiveness (i.e., the attractive ratings for each stimulus with a neutral expression), the intensity of sad expressions (i.e., the expression intensity ratings for each stimulus with a sad expression), and their interaction effect as fixed effects. We also included random intercepts for stimuli and participants. The regression analysis (Marginal R 2 = .21, Conditional R 2 = .42) demonstrated a significant main effect of the baseline attractiveness (B = 0.45, t = 14.82, p < .001), but there was no main effect of the intensity of sad expressions (B = −0.002, t = −0.06, p = .95) or interaction effect (B = −0.02, t = −1.59, p = .11). Therefore, unlike in the case of a happy expression, the attractiveness of faces with a sad expression is independent of its intensity.
To confirm the asymmetrical effects of the intensity of happy and sad expressions on attractiveness evaluation, we conducted an additional multiple regression analysis (Marginal R 2 = .27, Conditional R 2 = .42) using the dummycoded expression condition (1 = happy, 2 = sad), baseline attractiveness, the intensity of the expression, and their interaction effects as independent variables. We included random intercepts for the stimuli and the participants. Consistent with the results of the separate models described above, we found a significant three-way interaction effect (B = −0.06, t = −2.29, p = .02).
In summary, the attractiveness of a face with a happy expression could be predicted by its baseline attractiveness and intensity of expression, whereas the attractiveness of a face with a sad expression could be predicted only by its baseline attractiveness.

Discussion
We investigated the relationship between the intensity of facial expressions and evaluations of the attractiveness of both positive and negative expressions. The analysis of mean attractiveness ratings showed that, as expected, a face with a happy expression was judged as more attractive than one with a neutral expression, while a face with a sad expression was considered less attractive than one with a neutral expression. The correlation and multiple regression analyses revealed that, consistent with previous studies (e.g., Golle et al., 2014), faces with intense happy expressions were judged as more attractive. However, contrary to the prediction, there was no relationship between intensity and attractiveness for sad expressions; only the face's baseline attractiveness could predict the attractiveness of sad expressions. Therefore, we conclude that the intensity of positive and negative expressions asymmetrically influences evaluations of the attractiveness of each face.
Previous studies have shown that positive emotional faces make a positive impression, and we tend to evaluate more positive faces as more attractive (Golle et al., 2014;Oosterhof & Todorov, 2009;Reis et al., 1990;Schmidt et al., 2012). Thus, as also observed in the present study, the evaluation of positive faces depends on the intensity of happy expressions. Similarly, stronger negative expressions should communicate stronger negative messages or impressions to observers, so it might be expected that sadder faces would be evaluated as less attractive. However, we found that this was not true. Rather, only the baseline attractiveness of each face predicted the evaluation of the attractiveness of faces with sad expressions. Therefore, our findings indicate that positive and negative expressions affect evaluation of facial attractiveness in different ways.
This asymmetrical relationship might be associated with social interaction, especially emotional contagion or sympathy. A sad face indicates a troubled feeling. One research study on the relationship between facial expressions and sympathy found that sad faces elicit sadder feelings from observers when compared with happy or neutral faces (Small & Verrochi, 2009). As noted in the introduction, attractiveness is an important factor for receiving help from others (Benson et al., 1976;Harrell, 1978;West & Brown, 1975). The processing of sympathy in response to negative expressions might be qualitatively different from the evaluation of attractiveness, as people may perceive that an individual with a sad expression might be in immediate need of help. In contrast, there is a weaker relationship between positive facial expressions and the need for support, as seeing someone with a positive facial expression is encouraging and increases one's positive judgment of that person. As mentioned above, previous studies have shown that more positive faces are judged as more attractive (Golle et al., 2014) and more trustworthy (Oosterhof & Todorov, 2009;Schmidt et al., 2012). Therefore, our findings that expression intensity asymmetrically influences attractiveness in happy and sad faces might reflect a difference in processing related to feelings of sympathy. This explanation remains speculative, but we hope that future research can reveal how emotional expressions affect an observer's behavior and whether expression intensity is truly unrelated to the effect in the case of sad expressions. Experimental studies using economic games may reveal how facial attractiveness and expression intensity can influence the helping motivation of observers more directly.
Our study has limitations in three ways. First, we only used sadness as the negative expression, but it will be important to investigate other negative emotions such as anger, disgust, or contempt. As each negative expression sends different messages or impressions to observers or causes varying degrees of arousal, faces displaying these emotions may evoke different mental processes. Second, we did not investigate the relationship between expression and the sex of faces. Some studies have suggested that female faces are more likely to be stereotypically associated with happy and sad expressions than male faces (for a review, see Adams, Hess, & Kleck, 2015), which may play a role in the evaluation of a person's attractiveness or impression. For instance, males who look strongly happy or sad may violate the expectations of observers, which could lead to their decreased attractiveness. Given this stereotypical evaluation process, it might also be important to investigate cultural differences. It has been suggested that there could be some differences in facial expression cognition depending on the social context between American and Japanese people (Nagashima & Schellenberg, 1997); thus, cultural-dependent stereotypes may modify the relationship between expression intensity and attractiveness evaluation. Finally, an important future research direction relates to the possibility of individual differences in evaluation. For example, individuals with greater emotional sensitivity could be more influenced by the emotional intensity of other people's faces when evaluating their attractiveness. These questions lie beyond the scope of the present study, but we have demonstrated the important and novel finding that, although the emotional valence of a face has significant impact on its perceived attractiveness, the intensity of positive and negative expressions asymmetrically influences evaluations of attractiveness.

Conclusion
In this study, we investigated the relationship between intensity and attractiveness of facial expression in both positive and negative expressions. The results demonstrated, unsurprisingly, that compared with neutral faces, happy faces were judged as more attractive and sad faces as less attractive. In a more novel finding, intensity of expression asymmetrically influenced the attractiveness of happy or sad faces. Our findings might be associated with psychological process of sympathy.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This study was supported by JSPS Grant-in-Aid for Scientific Research 15H01690 for Hiroshi Ashida.