Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article

Face, Construct and Criterion Validity, and Test-Retest Reliability, of the Adult Rejection Sensitivity Questionnaire

Published Online:June 29, 2023https://doi.org/10.1027/1015-5759/a000782

Abstract

Abstract: This research sought to test the face, construct and criterion validity, and test-retest reliability of the Adult Rejection Sensitivity Questionnaire (ARSQ). In Study 1, participants (n = 45) completed the ARSQ and questions assessing scale item relevancy, clarity, difficulty, and sensitivity. In Study 2, participants (n = 513) completed the ARSQ and demographic questions. In Study 3, participants (n = 244) completed the ARSQ and returned 2 weeks later to complete the ARSQ and measures of depression, anxiety, and self-silencing behavior. Study 1 provided strong support for face validity with all items deemed relevant, clear, easy to answer, and neither distressing nor judgmental. Study 2 provided adequate support for the factor structure of the ARSQ (single-factor model and two-factor model) but suggested modifications could be made to improve scale validity. Study 3 provided further support for an adequate (but not good) factor structure, and evidence for criterion validity established through medium-large effect size correlations with depression, anxiety, and self-silencing behavior. However, the 2-week scale stability appeared poor (r = .45) in a subsample of participants. Overall, the ARSQ showed sufficient validity to recommend its continued use, but we recommend further tests of scale reliability and potential modifications to increase construct validity.

References

  • Allen, M. S. (2023, April 25). Face, construct and criterion validity of the ARSQ. https://osf.io/kbec7 First citation in articleGoogle Scholar

  • Beck, A. T., Epstein, N., Brown, G., & Steer, R. (1993). Beck Anxiety Inventory. Journal of Consulting and Clinical Psychology, 56(6), 893–897. https://doi.org/10.1037/0022-006x.56.6.893 First citation in articleCrossref, Google Scholar

  • Beck, A. T., Steer, R. A., & Carbin, M. G. (1988). Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clinical Psychology Review, 8(1), 77–100. https://doi.org/10.1016/0272-7358(88)90050-5 First citation in articleCrossref, Google Scholar

  • Berenson, K. R., Gyurak, A., Ayduk, Ö., Downey, G., Garner, M. J., Mogg, K., Bradley, B. P., & Pine, D. S. (2009). Rejection sensitivity and disruption of attention by social threat cues. Journal of Research in Personality, 43(6), 1064–1072. https://doi.org/10.1016%2Fj.jrp.2009.07.007 First citation in articleCrossref, Google Scholar

  • Brenchley, K. J. M., & Quinn, D. M. (2016). Weight-based rejection sensitivity: Scale development and implications for well-being. Body Image, 16, 79–92. https://doi.org/10.1016/j.bodyim.2015.11.005 First citation in articleCrossref, Google Scholar

  • Brookings, J. B., Zembar, M. J., & Hochstetler, G. M. (2003). An interpersonal circumplex/five-factor analysis of the Rejection Sensitivity Questionnaire. Personality and Individual Differences, 34(3), 449–461. https://doi.org/10.1016/S0191-8869(02)00065-X First citation in articleCrossref, Google Scholar

  • Cavicchioli, M., & Maffei, C. (2020). Rejection sensitivity in borderline personality disorder and the cognitive-affective personality system: A meta-analytic review. Personality Disorders, 11(1), 1–12. https://doi.org/10.1037/per0000359 First citation in articleCrossref, Google Scholar

  • Chan, W., & Mendoza-Denton, R. (2008). Status-based rejection sensitivity among Asian Americans: Implications for psychological distress. Journal of Personality, 76(5), 1317–1346. https://doi.org/10.1111/j.1467-6494.2008.00522.x First citation in articleCrossref, Google Scholar

  • Connell, J., Carlton, J., Grundy, A., Buck, E. T., Keetharuth, A. D., Ricketts, T., Barkham, M., Robotham, D., Rose, D., & Brazier, J. (2018). The importance of content and face validity in instrument development: Lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL). Quality of Life Research, 27, 1893–1902. First citation in articleCrossref, Google Scholar

  • DeNeve, K. M., & Cooper, H. (1998). The happy personality: A meta-analysis of 137 personality traits and subjective well-being. Psychological Bulletin, 124(2), 197–229. https://doi.org/10.1037/0033-2909.124.2.197 First citation in articleCrossref, Google Scholar

  • Downey, G., & Feldman, S. I. (1996). Implications of rejection sensitivity for intimate relationships. Journal of Personality and Social Psychology, 70, 1327–1343. First citation in articleCrossref, Google Scholar

  • Downey, G., Lebolt, A., Rincón, C., & Freitas, A. L. (1998). Rejection sensitivity and children’s interpersonal difficulties. Child Development, 69(4), 1074–1091. https://doi.org/10.2307/1132363 First citation in articleCrossref, Google Scholar

  • Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and personality research: Current practice and recommendations. Social Psychological and Personality Science, 8(4), 370–378. https://doi.org/10.1177/1948550617693063 First citation in articleCrossref, Google Scholar

  • Foxhall, M., Hamilton-Giachritsis, C., & Button, K. (2019). The link between rejection sensitivity and borderline personality disorder: A systematic review and meta-analysis. British Journal of Clinical Psychology, 58(3), 289–326. https://doi.org/10.1111/bjc.12216 First citation in articleCrossref, Google Scholar

  • Funder, D. C., & Ozer, D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/2515245919847202 First citation in articleCrossref, Google Scholar

  • Gao, S., Assink, M., Cipriani, A., & Lin, K. (2017). Associations between rejection sensitivity and mental health outcomes: A meta-analytic review. Clinical Psychology Review, 57, 59–74. https://doi.org/10.1016/j.cpr.2017.08.007 First citation in articleCrossref, Google Scholar

  • Gao, S., Assink, M., Liu, T., Chan, K. L., & Ip, P. (2021). Associations between rejection sensitivity, aggression, and victimization: A meta-analytic review. Trauma, Violence, & Abuse, 22(1), 125–135. https://doi.org/10.1177/1524838019833005 First citation in articleCrossref, Google Scholar

  • Heene, M., Hilbert, S., Draxler, C., Ziegler, M., & Bühner, M. (2011). Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices. Psychological Methods, 16(3), 319–336. https://doi.org/10.1037/a0024917 First citation in articleCrossref, Google Scholar

  • Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466(7302), 29. https://doi.org/10.1038/466029a First citation in articleCrossref, Google Scholar

  • Howard, M. C., Cogswell, J. E., & Smith, M. B. (2020). The antecedents and outcomes of workplace ostracism: A meta-analysis. Journal of Applied Psychology, 105(6), 577–596. https://doi.org/10.1037/apl0000453 First citation in articleCrossref, Google Scholar

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Innamorati, M., Balsamo, M., Fairfield, B., Fabbricatore, M., Tamburello, A., & Saggino, A. (2014). Construct validity and reliability of the Adult Rejection Sensitivity Questionnaire: A comparison of three factor models. Depression Research and Treatment, 2014, Article 972424. First citation in articleCrossref, Google Scholar

  • Jack, D. C., & Dill, D. (1992). The Silencing the Self Scale: Schemas of intimacy associated with depression in women. Psychology of Women Quarterly, 16, 97–106. First citation in articleCrossref, Google Scholar

  • Kang, S. K., & Chasteen, A. L. (2009). The development and validation of the Age-Based Rejection Sensitivity Questionnaire. The Gerontologist, 49, 303–316. First citation in articleCrossref, Google Scholar

  • London, B., Downey, G., Romero-Canyas, R., Rattan, A., & Tyson, D. (2012). Gender-based rejection sensitivity and academic self-silencing in women. Journal of Personality and Social Psychology, 102(5), 961–979. https://doi.org/10.1037/a0026615 First citation in articleCrossref, Google Scholar

  • Maiolatesi, A. J., Clark, K. A., & Pachankis, J. E. (2022). Rejection sensitivity across sex, sexual orientation, and age: Measurement invariance and latent mean differences. Psychological Assessment, 34(5), 431–442. https://doi.org/10.1037/pas0001109 First citation in articleCrossref, Google Scholar

  • Mendoza-Denton, R., Downey, G., Purdie, V. J., Davis, A., & Pietrzak, J. (2002). Sensitivity to status-based rejection: Implications for African American students’ college experience. Journal of Personality and Social Psychology, 83(4), 896–918. https://doi.org/10.1037/0022-3514.83.4.896 First citation in articleCrossref, Google Scholar

  • Mishra, M., & Allen, M. S. (2023). Rejection sensitivity and romantic relationships: A systematic review and meta-analysis. Personality and Individual Differences, 208, Article 112186. First citation in articleCrossref, Google Scholar

  • Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill. First citation in articleGoogle Scholar

  • Pachankis, J. E., Goldfried, M. R., & Ramrattan, M. E. (2008). Extension of the rejection sensitivity construct to the interpersonal functioning of gay men. Journal of Consulting and Clinical Psychology, 76, 306–317. First citation in articleCrossref, Google Scholar

  • Park, L. E. (2007). Appearance-based rejection sensitivity: Implications for mental and physical health, affect, and motivation. Personality and Social Psychology Bulletin, 33, 490–504. First citation in articleCrossref, Google Scholar

  • Polit, D. F. (2014). Getting serious about test-retest reliability: A critique of retest research and some recommendations. Quality of Life Research, 23, 1713–1720. First citation in articleCrossref, Google Scholar

  • Romero-Canyas, R., Downey, G., Berenson, K., Ayduk, O., & Kang, N. J. (2010). Rejection sensitivity and the rejection-hostility link in romantic relationships. Journal of Personality, 78(1), 119–148. https://doi.org/10.1111/j.1467-6494.2009.00611.x First citation in articleCrossref, Google Scholar

  • Rosenbach, C., Renneberg, B., & Scheithauer, H. (2021). Psychometric properties of the German version of the Children’s Rejection Sensitivity Questionnaire (CRSQ). International Journal of Developmental Science, 15(1–2), 29–37. https://doi.org/10.3233/dev-210312 First citation in articleCrossref, Google Scholar

  • Shujja, S., Malik, F., & Khan, N. (2017). Translation and cross-cultural validation of Children Rejection Sensitivity Questionnaire (CRSC) for Pakistani adolescents. Pakistan Journal of Psychological Research, 1, 297–315. First citation in articleGoogle Scholar

  • Yu, Y. R., Youk, G. Y., & Lee, S. H. (2017). Adult attachment, parent child intimacy, rejection sensitivity of university students. Information, 20, 2793–2800. First citation in articleGoogle Scholar

  • Zaid, S. M. R., Jamaluddin, S., Baharuldin, Z., & Taresh, S. M. (2020). Psychometric properties of an adapted Yemeni version of Rejection Sensitivity Questionnaire. Pertanika Journal of Social Sciences and Humanities, 28, 2149–2165. First citation in articleGoogle Scholar