Do Sexist Comments Hinder Participation in Online Political Discussions?
A Preregistered Experiment
Abstract
Abstract: Women who visibly engage in politics online experience a lot of backlash. The presented study investigates sexist incivility against women in online political spaces as a possible explanation for the gender gap in online political discussion and expression. Online sexism solidifies the masculine norm in online political spaces. Drawing on social cognitive theory and the theory of normative social behavior, we understand online incivility as communication mechanisms that enforce gender norms in online political discussions. We use a preregistered online survey experiment with German Internet users to investigate how sexist comments in online political discussions affect women’s participation in the discussion, perceived social norms about participating, and their internal political efficacy. We found no effects of sexist comments on the propensity to engage in the discussion or the political efficacy to contribute. However, the presence of sexist comments increased the fear of sanctions in both men and women. The null findings of the preregistered experiment occurred despite sufficient statistical power and a successful treatment check. We discuss several possible explanations for the null effects and ways forward.
Political participation is central to democracy. Over the past decades, the Internet and in particular social media have enabled a variety of means for online political and civic participation (Theocharis, 2015). Political expression (PE) in online political discussions is essential to online participation (Papacharissi, 2004). Sadly, the online environments keep replicating offline inequalities: Women participate less visibly, especially in expressive forms of political engagement (Bode, 2017). In this article, we consider this gender gap from a gender role perspective. Politics is socialized as a masculine domain where agentic traits and behaviors are expected (Schneider & Bos, 2019). Women in counter-stereotypical domains are penalized for not behaving stereotypically feminine, which is rather communal as opposed to agentic (Spence, 1984). Negative evaluations and objectification are well-established ways of sanctioning women for counter-stereotypical roles and behaviors (Rudman & Glick, 1999), such as participating in political discussions.
Extensive research suggests that there is a systematic and, thus, sexist bias against women who visibly engage in politics. Female political activists describe negative reactions to sharing political opinions online (Sobieraj, 2018; Vochocová, 2018). Female politicians (Rheault et al., 2019; Southern & Harmer, 2021) and journalists (Gardiner, 2018) become the target of online incivility more frequently than their male colleagues. Women indicate having experienced mansplaining (i.e., explanation by a man, typically to a woman in a patronizing manner) on Twitter more often when discussing politics (Koc-Michalska et al., 2019). In an international survey, 47% of interviewed young women aged 15–24 years report that they have been attacked for voicing their opinion (PLAN International, 2020). Our survey corroborated that various forms of online harassment and incivility are common among German women: Almost half of the women under the age of 30 experienced sexual harassment (47%), followed by being ridiculed (39%), mansplaining (38%), objectification (37%), and sustained harassment (37%; Reich & Bachl, 2022b).
Consequently, we construe incivility and harassment against women in digital spaces as a form of sexism that solidifies the masculine norms in politics, comprising many possible actions (Reich & Bachl, 2022b). While not all sexist attacks are explicitly referencing sex or gender, for reasons of operationalization and definition we concentrate on sexist attacks that do reference sex. We draw on social cognitive theory (SCT; Bandura, 1986) as well as the theory of normative social behavior (TNSB; Rimal & Real, 2005) to understand online incivility as communication mechanisms that enforce gender norms in online PE by sanctioning women for their visibility. Using a preregistered online survey experiment, we investigate how this public enforcement of gender roles through sexism affects women’s political expression.
Social Norms of Online Political Expression
Our rationale builds on two premises. First, according to SCT, we learn by observing others (Bandura, 1986). Attentional and representational processes determine what we selectively observe in the social environment and what rules prescribe. Secondly, political behavior is gendered (Schneider & Bos, 2019). In the media, male political experts and politicians still outnumber female political experts and politicians (Kitzinger, 2008; Prommer & Stüwe, 2020) and online discussions are dominated by male participants (Duyn et al., 2019; Ziegele et al., 2013). As a result, representation in the political domain is skewed, enforcing masculine dominance in this domain. Through mass media and social media, PE shapes users’ descriptive norms through communication with and observation of referent others (Geber & Hefner, 2019). The rationale of TNSB is that the more a target behavior is considered the norm, the more likely this behavior gets adopted by the individual (Rimal & Real, 2005).
To counteract the skewed representation of women in the political domain, feminist scholars across fields have called for equal representation of female media experts and politicians (Prommer & Linke, 2019). Stronger representation of female voices, for example, in politics should affect normative beliefs about who contributes to politics at large. However, in line with the TNSB, we consider this only half of the truth. The visibility of women in counter-stereotypical domains is often tainted by the prevailing sexism visible women encounter in online settings as much as in offline settings (Krook, 2017).
The backlash against women who defy prescriptive stereotypes about how they are supposed to behave can take many forms: hate speech (Döring & Mohseni, 2020), microaggressions (Harmer & Southern, 2021), incivility (Rheault et al., 2019), or harassment (Chen et al., 2020). We adopt the definition of incivility by Papacharissi (2004), arguing that incivility includes a “set of behaviors that threaten democracy, deny people their freedoms, and stereotype social groups” (Papacharissi, 2004, p. 267) and that aim to obstruct a healthy discussion. Sexist comments are a form of incivility and likely act as social normative influences. TNSB conceptualizes the social approval of behavior as injunctive norms (Rimal & Lapinski, 2015). Online comments constantly display approval or disapproval and thereby contribute to injunctive norms (Geber & Hefner, 2019). Therefore, sexist incivility in online political discussions communicates who or, more broadly, which group is expected and accepted to contribute – and, more importantly, who is not welcome. If injunctive norms of PE are, in fact, gendered, they add to the gender gap in PE. Our first prediction is that women who observe sexist comments against a politically visible woman are less likely to participate in political discussions online.
Reading sexist comments (vs. benign comments) in a discussion decreases women’s likelihood to participate (H1a) and to share their own opinion (H1b), and the decrease is larger for women than for men.
One way by which incivility excludes individuals or groups from online political discussions is through public sanctions. TSNB (Rimal & Real, 2005) and Bandura’s SCT (1986) postulate that the expectation of benefits is an important predictor of human behavior. Negative outcomes like social sanctions reduce or even cancel out potential benefits. In the spiral of silence theory (Noelle-Neumann, 1974), the anticipation of social isolation is highlighted as the central inhibitor of opinion sharing. In the TSNB framework, expected social sanctions can be understood as a manifestation of injunctive norms. To investigate how sexist incivility affects the fear of social sanctions, we conceptualize expected social sanctions as domain-specific sanctions that are relevant to online political discussions (Neubaum & Krämer, 2018). In our second hypothesis, we thus expect that:
Reading sexist comments (vs. benign comments) increases the expected sanctions for women, and the increase is larger for women compared to men.
Sexist attacks online are supposed to intimidate. Therefore, effects on the self are also relevant. In order to actively engage in political discussions online, women need to feel capable. Thus, following the SCT, efficacy experiences are necessary predictors of behaviors (Bandura, 1977). In particular, internal political efficacy (Balch, 1974) is understood as the perception of one’s own capability to participate in politics. Beliefs about efficacy develop from direct and vicarious experiences of mastery, verbal encouragement, and physiological feedback. We argue that sexist online comments attacking politically active women provide negative experiences on at least two dimensions: Instead of encouragement, sexist comments show explicit discouragement and sexist comments provide a negative vicarious experience of visible political participation of women. As such, we expect sexism to negatively impact women’s, but not men’s, political efficacy.
Reading sexist comments (vs. benign comments) decreases the perceived competence for women but not for men.
Additional secondary hypotheses and research questions were part of the preregistration reported here and their analysis is reported on OSF for reference. For this short report, we concentrate on the central hypotheses.
Method
Preregistration, Open Data, Open Materials
The hypotheses, study procedures, stimulus materials, power calculations, and the complete data analysis plan for hypothesis testing were preregistered before the data collection started. The frozen preregistration and materials can be found in the study’s OSF repository (https://osf.io/xjac7/; Reich & Bachl, 2022a).
Sample
A sample of 750 participants fulfilling representative quotas of the German population in terms of gender (50% female), age (M = 44.56, SD = 14.35), and education (lower = 30%, medium = 34%, higher = 36%) was ordered from the online access panel by Respondi (http://www.respondi.com) in June 2020. The total sample size and design were based on a pragmatic decision given the available funding for the data collection (Lakens, 2022). A sample with 180 participants per cell corresponds to 76% power to detect small (d = 0.2) effects and to 99.99% power to detect medium (d = 0.5) effects as defined by the coefficients in an effect-coded linear model with two-tailed tests and α = .05. Data were screened for the requested quotas, plausibility and attention checks, and processing time per questionnaire page by the provider before they were delivered to the researchers. Incentives were issued by Respondi.
Design
Participants landed on a briefing page about the conditions of participating. We informed participants about the study procedure, voluntary participation, termination options, anonymity, and data use. After consent, participants provided demographics. The following pages asked for the participants’ typical participation in online political discussions and some additional items not relevant to the hypotheses tested here. After that, each participant was presented with one of two versions of the stimulus: a screenshot of the discussion board of a public broadcasting station in Germany (meta.tagesschau.de). A short teaser and photo of an online news article on parental leave were shown, featuring a direct and an indirect quote from a female scientist. Several neutral user comments were displayed below the article in the control condition. In the treatment condition, an additional comment with two replies (pile-on) was posted. The three treatment comments questioned the scientist’s ability based on her gender (cp. Figures E2 and E3, Electronic Supplementary Material, ESM 1; translations of the stimuli are available in the ESM 1, E4 and E5). The stimulus was followed by items about the likelihood of participating in the discussion and voicing an opinion. The next pages asked for the expected sanctions and participants’ perceived competence, followed by two treatment checks and some additional measures.
Measurements
All measurements used answer options on 7-point scales and are fully cited in ESM 1, E6. Four items assessed the online discussion frequency on different platforms (M = 2.23, SD = 1.61). PE was operationalized on two subdimensions using single items: “How likely is it that you would post a comment in this discussion?” (M = 2.23, SD = 1.61) and “How likely is it that you would post your personal opinion about this topic in this discussion?” (M = 2.31, SD = 1.70, higher values indicated a greater likelihood of participation). We used a 9-item scale on the expected sanctions when commenting in the displayed discussion, for example, “I would fear to be verbally attacked” or “I would fear getting mobbed.” Items were partially derived and extended from Neubaum and Krämer (2018), M = 2.79, SD = 1.49, ω = 0.89 (higher values indicated a greater likelihood of sanctions). Five items operationalized the perceived internal efficacy to contribute to the discussion (M = 3.52, SD = 1.55, ω = 0.78, higher values indicated higher efficacy). Items read, for example, “I do not believe I have anything to add to the discussion” or “I feel competent enough to gather information on the issue.” Two treatment checks were used to test whether the treatment was perceived as sexist against women (four items, e.g., “In the comments women were portrayed as incompetent”; M = 3.56, SD = 1.81, Cronbach’s α = 0.94, higher values indicated a stronger perception of the issues; the preregistered factor model underlying the ω coefficient did not converge, and thus we report Cronbach’s α) and whether it created a hostile climate for women in the comment section (eight items, e.g., “Women’s opinion is not taken seriously in this discussion”; M = 3.33, SD = 1.40, ω = 0.85, higher values indicated more hostility).
Data Analysis
We use an α level of 5% for all tests and one-tailed tests for all directional hypotheses. To assess the hypotheses, we conducted linear models with effect-coded factors.
Results
Treatment Check
We assessed whether participants recognized an attack against the female scientist cited in the article teaser and found a substantial effect of the sexist attack: d = 1.28, 95% CI [1.17, 1.39] (cp. ESM 1, Figure E1, left panel). We assessed whether the discussion climate was perceived as hostile towards women and found a substantial main effect of the treatment: d = 0.97, 95% CI [0.84, 1.09] (cp. ESM 1, Figure E1, right panel).
Hypothesis 1
Contrary to our expectations, women reading sexist comments (vs. benign comments) did not report a lower likelihood to participate, b = 0.11, t(746) = 0.68, p = .247 (H1a), and to share their own opinion, b = 0.04, t(746) = 0.23, p = .41 (H1b). The treatment effect was not larger for women than for men: b = −0.18, t(746) = −0.76, p = .225 (H1a); b = −0.24, t(746) = −0.95, p = .171 (H1b). Figure 1 shows that we found no significant model for either dependent variable.
Hypothesis 2
In line with our expectation, women reading sexist comments about a female scientist reported higher fear of sanctions: b = 0.38, t(746) = 2.51, p = .006. However, the treatment effect for men was not significantly different from that for women: b = −0.1, t(746) = −0.45, p = .325. Both men and women had a stronger fear of sanctions if reading sexist comments (cp. Figure 2, left panel).
Hypothesis 3
Contrary to our expectation, there was no statistically significant treatment effect on women’s internal efficacy, b = 0.18, t(746) = 1.15, p = .126 (cp. Figure 2, right panel). The absence of a small effect for men, defined as a difference of d < 0.2, could also not be established (standardized b = 0.11, 90% CI [−0.06, 0.28]).
Discussion
The present study was designed to determine the gender-related effects of sexist comments against visible women in online political discussions. Specifically, we tested whether such comments affected women in ways that reduce their willingness to express themselves relative to men, offering a possible explanation for the gender gap in online political expression. We found no effects of reading sexist comments on the propensity to engage in the discussion and the self-efficacy to contribute. The presence of sexist comments against the female scientist in the news article led to more fear of being sanctioned if participating in the discussion. Yet this effect was, contrary to our prediction, similar for both genders. This finding corroborates previous work that incivility in comment sections promotes the perception of visible PE as risky behavior. This runs counter to the deliberative norms of a democratic society (Papacharissi, 2004). No one should fear retribution if they voice their opinion. Concluding, our study lends no support to a gender-related effect of sexist uncivil comments on a gender participation gap in online political discussions.
The null findings of the preregistered experiment occurred despite sufficient statistical power and a successful treatment check. There are several possible explanations for the null effects and limitations to consider when evaluating these findings. For one, we did not employ blatant sexism in the form of name-calling, threatening comments, or objectification. This was reflected in the results of the treatment check. Although the attack against the female scientist in the treatment condition was perceived as substantially more hostile compared to the control, the absolute hostility ratings just reached the midpoint of the scales. Ethical considerations make this a rather tricky issue to investigate experimentally and existing data suggest that women in politics are sometimes met with recognizably more hostile sexism (e.g., Gardiner, 2018). Instead, we presented comments questioning women’s competence, a form of sexism often defended as opinion but happens to women systematically more often in online political discussions (Koc-Michalska et al., 2019, Reich & Bachl, 2022b). As online sexism is expressed in many different forms, we cannot assume that all forms of sexism are without consequences. Our sample recognized the non-blatant attacks very well but was not affected in their behavior. The natural progression of this work is to find an ethically sustainable design to analyze if and at what qualitative level attacks shift norms of PE. This leads to a second question that our research opens: Has the public, particularly female Internet users, become accustomed to a baseline level of sexism and incivility? This would show a normative shift in online PE that requires further assessment. Finally, the participants reported low baseline levels of participation in online political discussions and were, regardless of condition, unlikely to participate in the presented discussion. There was little room for the sexist attacks to further decrease participation. Adjusting for pretreatment differences in the frequency of participation in online political discussions did not change the results of the reported hypothesis tests (see analysis script in the OSF). Future studies might consider the effects of sexist attacks in populations and scenarios in which participation is higher, for example, among younger citizens, regular users of social media, or more popular platforms or topics. Gender role norms are powerful prescriptive of social life. Investigating social norms – particularly gender role norms – and their enforcement in political communication is important in order to shed light on the PE gap. Although we obtained null results with the current study, we outlined a gendered perspective on incivility that should be advanced in future research. Furthermore, the literature on incivility predominantly reports statistically significant effects. Therefore, the preregistered null result is relevant for the meta-study of incivility, highlighting that not every incivility manipulation has immediate effects.
Electronic Supplementary Material
The electronic supplementary material is available with the online version of the article at https://doi.org/10.1027/1864-1105/a000373
Sabine Reich (PhD, University of Mannheim, Germany) is a postdoctoral researcher at the Department of Media and Communication, Ludwig-Maximilians-University of Munich, Germany. Her research focuses on media psychology, online communication, and media effects, analyzing aspects of sexism, identity, and social needs.
Marko Bachl (PhD, University of Hohenheim, Germany) is a postdoctoral researcher at the Department of Communication Science at University of Hohenheim, Germany. His research interests are political communication, health communication, and quantitative methods.
References
(1974). Multiple indicators in survey research: The concept “sense of political efficacy”. Political Methodology, 1(2), 1–43.
(1977). Self-efficacy–toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215.
(1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
(2017). Closing the gap: Gender parity in political engagement on social media. Information, Communication & Society, 20(4), 587–603. https://doi.org/10.1080/1369118X.2016.1202302
(2020). “You really have to have a thick skin”: A cross-cultural perspective on how online harassment influences female journalists. Journalism, 21(7), 877–895. https://doi.org/10.1177/1464884918768500
(2020). Gendered hate speech in YouTube and YouNow comments: Results of two content analyses. Studies in Communication and Media, 9(1), 62–88. https://doi.org/10.5771/2192-4007-2020-1-62
(2019). The gender gap in online news comment sections. Social Science Computer Review, 089443931986487. https://doi.org/10.1177/0894439319864876
(2018). “It’s a terrible way to go to work:” What 70 million readers’ comments on The Guardian revealed about hostility to women and minorities online. Feminist Media Studies, 18(4), 592–608. https://doi.org/10.1080/14680777.2018.1447334
(2019). Social norms as communicative phenomena: A communication perspective on the theory of normative social behavior. Studies in Communication Media, 8(1), 6–28. https://doi.org/10.5771/2192-4007-2019-1-6
(2021). Digital microaggressions and everyday othering: An analysis of tweets sent to women members of Parliament in the UK. Information, Communication & Society, 24(14), 1998–2015. https://doi.org/10.1080/1369118X.2021.1962941
(2008). Gender, stereotypes and expertise in the press: How newspapers represent female and male scientists. UK Resource Centre for Women in Science, Engineering and Technology.
(2019). From online political posting to mansplaining: The gender gap and social media in political discussion. Social Science Computer Review, 39(2), 197–210. https://doi.org/10.1177/0894439319870259
(2017). Violence against women in politics. Journal of Democracy, 28(1), 74–88. https://doi.org/10.1353/jod.2017.0007
(2022). Improving your statistical inferences. https://doi.org/10.5281/ZENODO.6409077
(2018). What do we fear? Expected sanctions for expressing minority opinions in offline and online communication. Communication Research, 45(2), 139–164. https://doi.org/10.1177/0093650215623837
(1974). The spiral of silence: A theory of public opinion. Journal of Communication, 24(2), 43–51. https://doi.org/10.1111/j.1460-2466.1974.tb00367.x
(2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259–283. https://doi.org/10.1177/1461444804041444
. (2020). The state of the world’s girls 2020. https://www.plan.de/fileadmin/website/05._Ueber_uns/Maedchenberichte/Maedchenbericht_2020/Free_to_be_online_report_englisch_FINAL.pdf
(2019). Ausgeblendet: Frauen im deutschen Film und Fernsehen
[Hidden women: Women in German movies and television] . Herbert von Halem Verlag.(2020). Wer wird gefragt? Geschlechterverteilung in der Corona-Berichterstattung
[Who gets asked in times of crisis?] . Institut für Medienforschung, Philosophische Fakultät, Universität Rostock. https://malisastiftung.org/wp-content/uploads/Studie_MaLisa_Geschlechterverteilung_in_der_Corona_Berichterstattung.pdf(2022a). Preregistration, materials, and data for "Sexism in Online Discussions". https://osf.io/xjac7
(2022b). Prevalence of online harassment in German men and women [Preprint]. Open Science Framework. https://doi.org/10.31219/osf.io/6auq2
(2019). Politicians in the line of fire: Incivility and the treatment of women on social media. Research & Politics, 6(1), 205316801881622. https://doi.org/10.1177/2053168018816228
(2015). A re-explication of social norms, ten years later: Social norms. Communication Theory, 25(4), 393–409. https://doi.org/10.1111/comt.12080
(2005). How behaviors are influenced by perceived norms: A test of the theory of normative social behavior. Communication Research, 32(3), 389–414. https://doi.org/10.1177/0093650205275385
(1999). Feminized management and backlash toward agentic women: The hidden costs to women of a kinder, gentler image of middle managers. Journal of Personality and Social Psychology, 77(5), 1004–1010. https://doi.org/10.1037/0022-3514.77.5.1004
(2019). The application of social role theory to the study of gender in politics. Political Psychology, 40(S1), 173–213. https://doi.org/10.1111/pops.12573
(2018). Bitch, slut, skank, cunt: Patterned resistance to women’s visibility in digital publics. Information, Communication & Society, 21(11), 1700–1714. https://doi.org/10.1080/1369118X.2017.1348535
(2021). Twitter, incivility and “everyday” gendered othering: An analysis of tweets sent to UK members of parliament. Social Science Computer Review, 39(2), 259–275. https://doi.org/10.1177/0894439319865519
(1984). Masculinity, femininity, and gender-related traits: A conceptual analysis and critique of current research. Progress in Experimental Personality Research, 13, 1–97. https://doi.org/10.1016/B978-0-12-541413-5.50006-1
(2015). The conceptualization of digitally networked participation. Social Media + Society, 1(2), 1–14. https://doi.org/10.1177/2056305115610140
(2018). Witty divas, nice mothers and tough girls in a sexist world: Experiences and strategies of female influencers in online political debates. Media, Culture & Society, 40(4), 535–550. https://doi.org/10.1177/0163443717729211
(2013). Männlich, rüstig, kommentiert? Einflussfaktoren auf die Aktivität kommentierender Nutzer von Online-Nachrichtenseiten
[Male, spry, commenting? Predictors of activity of commenting users of online news web sites] . Studies in Communication Media, 2(1), 67–114. https://doi.org/10.5771/2192-4007-2013-1-67
