Abstract
Abstract: The inclusion of skin-tone modifiers into the standard emoji set in 2015 marked a shift toward increased skin-tone representation in emoji characters. We investigated whether implicit skin-tone bias, as typically found for human faces, also exists for emojis – a topic that is becoming increasingly relevant in computer-mediated communication (CMC) as more of our communication, education, and social exchange takes place in digital spaces. We systematically adapted Harvard University’s most recent skin-tone IAT (Implicit Associations Test) to assess implicit skin-tone bias for emojis. The reliability of our novel skin-tone IAT was good (internal consistency = .87). Data from a racially and ethnically diverse sample of 248 participants revealed that, on average, participants held more positive implicit associations for light than for dark skin-tone emojis: MDscore = 0.39; SDDscore = 0.42; t(247) = 14.76, p < .001; Cohen’s d = 0.93. Participants’ own skin tone predicted the strength of skin-tone bias, R2 = .02, F(1, 247) = 5.41, p = .02. The darker the skin tone of the participants, the less likely they were to exhibit bias in favor of light skin-tone emojis (β = −.14). These results align with the patterns of skin-tone bias that are typically observed for human faces, and extend those insights to emojis that are frequently used in CMC. The results also provide the first evidence that our novel emoji skin-tone IAT can be a useful tool for assessing emoji skin-tone bias.
References
(1988). Children and Prejudice. Blackwell Publishing.
(2016). Skin tone bias among African Americans: Antecedents and consequences across the life span. Developmental Review, 40, 93–116. https://doi.org/10.1016/j.dr.2016.03.002
(2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological Assessment, 28(6), 648–691. https://doi.org/10.1037/pas0000217
(1997). Shades of meaning: Skin tone, racial attitudes, and constructive memory in African American children. Journal of Experimental Child Psychology, 67(3), 363–388. https://doi.org/10.1006/jecp.1997.2413
(2018). How gender and skin tone modifiers affect emoji semantics in Twitter. Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (SEM), June 5–6, 101–106. https://orca.cardiff.ac.uk/id/eprint/114047/1/starSEM2018_emoji-modifiers.pdf
(2008). Practical advice for conducting ethical online experiments and questionnaires for United States psychologists. Behavior Research Methods, 40(4), 1111–1128.
(2018). Online networks of racial hate: A systematic review of 10 years of research on cyber-racism. Computers in Human Behavior, 87, 75–86.
(2010).
Ethical issues in psychological research on the Internet . In S. GoslingJ. JohnsonEds., Advanced methods for conducting online behavioral research (pp. 255–271). American Psychological Association.(2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13(2), 149–154. https://doi.org/10.1177/1745691617706516
(2019). Survey-software implicit association tests: A methodological and empirical analysis. Behavior Research Methods, 51(5), 2194–2208. https://doi.org/10.3758/s13428-019-01293-3
(2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavioral Research Methods, 46, 112–130. https://doi.org/10.3758/s13428-013-0365-7
(2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53–81. https://doi.org/10.1146/annurev-clinpsy-021815-093623
(2009). Access and accountability: The ecology of information sharing in the digital age. Anthropology News, 50(4), 4–5.
(1977). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.
(2013). Race and racism in Internet studies: A review and critique. New Media & Society, 15(5), 695–719. https://doi.org/10.1177/1461444812462849
(1991). Who is black? one nation’s definition. Pennsylvania State University Press.
(2019). Implicit bias is behavior: A functional-cognitive perspective on implicit bias. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 14(5), 835–840. https://doi.org/10.1177/1745691619855638
(2007). The Implicit Association Test outperforms the extrinsic affective Simon task as an implicit measure of inter-individual differences in attitudes. British Journal of Social Psychology, 46, 401–421. https://doi.org/10.1348/014466606X130346
(2017). Skin color and colorism: Global research, concepts, and measurement. Annual Review of Sociology, 43(1), 405–424. https://doi.org/10.1146/annurev-soc-060116-053315
(2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.
(2013).
The unbearable whiteness of emoji . In Mercedes KrausEd., Emoji: A special print edition of Womazine (pp. 26–27). Womazine + Emoji Art and Design Show. https://issuu.com/lindseyweber5/docs/emoji_by_womanzine(2022). Implicit bias ≠ bias on implicit measures. Psychological Inquiry, 33(3), 139–155. https://doi.org/10.1080/1047840X.2022.2106750
(1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102(1), 4–27. https://doi.org/10.1037/0033-295x.102.1.4
(2022). Best research practices for using the Implicit Association Test. Behavioral Research Methods, 54(3), 1161–1180. https://doi.org/10.3758/s13428-021-01624-3
(2020). Implicit social cognition. Annual Review of Psychology, 71, 419–445. https://doi.org/10.1146/annurev-psych-010419-050837
(1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480. https://doi.org/10.1037/0022-3514.74.6.1464
(2001). Health of the Implicit Association Test at age 3. Zeitschrift für Experimentelle Psychologie, 48(2), 85–93. https://doi.org/10.1026/0949-3946.48.2.85
(2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197–216. https://doi.org/10.1037/0022-3514.85.2.197
. (n.d.). Project implicit. https://implicit.harvard.edu/implicit/education.html
(2005). A meta-analysis on the correlation between the Implicit Association Test and explicit self-report measures. Personality and Social Psychology Bulletin, 31(10), 1369–1385. https://doi.org/10.1177/0146167205275613
(2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14, 399–425. https://doi.org/10.1007/s10683-011-9273-9
(2016). Colorism in the classroom: How skin tone stratifies African American and Latina/O Students. Theory into Practice, 55(1), 54–61. https://doi.org/10.1080/00405841.2016.1119019
. (n.d.). IATs in Qualtrics. https://iatgen.wordpress.com/
(2007).
Understanding and using the Implicit Association Test: IV: What we know (so far) about the method . In B. WittenbrinkN. Schwarz (Eds.), Implicit measures of attitudes (pp. 59–102). The Guilford Press.(2018). Who is studying online (and where). https://www.insidehighered.com/digital-leaming/article/2018/01/05/new-us-data-showcontinued-growth-college-students-studying
(2002). What are we really priming? Cue-based versus category-based processing of facial stimuli. Journal of Personality and Social Psychology, 82, 5–18. https://doi.org/10.1037/0022-3514.82.1.5
(2021). One part politics, one part technology, one part history: Racial representation in the Unicode 7.0 emoji set. New Media & Society, 23(3), 515–534. https://doi.org/10.1177/1461444819899623
. (n.d.). Amazon Mechanical Turk. https://www.mturk.com/
(2002). Cybertypes: Race, ethnicity, and identity on the Internet. Routledge.
(2002). Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dynamics: Theory, Research, and Practice, 6(1), 101–115. https://doi.org/10.1037/1089-2699.6.1.101
(2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psychology, 18(1), 36–88. https://doi.org/10.1080/10463280701489053
(2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872. https://doi.org/10.1016/j.jesp.2009.03.009
(2018). Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 21(4), 233–241. https://doi.org/10.1080/1097198x.2018.1542262
(2016). Psychological research in the internet age: The quality of web-based data. Computers in Human Behavior, 58, 354–360. https://doi.org/10.1016/j.chb.2015.12.049
(2008, September 1–3). Virtual humans elicit skin-tone bias consistent with real-world skin-tone biases. Paper presented at the 8th international conference on Intelligent Virtual Agents in Tokyo, Japan.
(2007). Negotiating interracial interactions. Current Directions in Psychological Science, 16(6), 316–320. https://doi.org/10.1111/j.1467-8721.2007.00528.x
(2015). Should we stop looking for a better scoring algorithm for handling Implicit Association Test data? Test of the role of errors, extreme latencies treatment, scoring formula, and practice trials on reliability and validity. PloS one, 10(6). https://doi.org/10.1371/journal.pone.0129601
(2011). Implicit measures for social and personality psychology. SAGE.
(2021). The implicit association test: A method in search of a construct. Perspectives on Psychological Science, 16(2), 396–414. https://doi.org/10.1177/1745691619863798
(2008). Assessment of individual differences in implicit cognition. European Journal of Psychological Assessment, 24(4), 210–217. https://doi.org/10.1027/1015-5759.24.4.210
(2019). Technically white: Emoji skin-tone modifiers as American technoculture. First Monday, 24(7). https://doi.org/10.5210/fm.v24i7.10060
(2014). Apple pledges to make emojis more ethnically diverse. Dazed Digital. https://www.dazeddigital.com/artsandculture/article/19367/1/apple-pledges-better-ethnic-diversity-for-emoji
(1997). Life on the screen: Identity in the age of Internet. MIT Press.
(2022). Bots amplify and redirect hate speech in online discourse about racism during the COVID-19 pandemic. Social Media + Society, 8(3), 1–14. https://doi.org/10.1177/20563051221104749
(2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493–504. https://doi.org/10.1037/pspa0000056
