Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Free AccessOriginal Article

Workplace Stress in Real Time

Three Parsimonious Scales for the Experience Sampling Measurement of Stressors and Strain at Work

Published Online:August 03, 2022https://doi.org/10.1027/1015-5759/a000725

Abstract

Abstract: Experience sampling methods are increasingly used in workplace stress assessment, yet rarely developed and validated following the available best practices. Here, we developed and evaluated parsimonious measures of momentary stressors (Task Demand and Task Control) and the Italian adaptation of the Multidimensional Mood Questionnaire as an indicator of momentary strain (Negative Valence, Tense Arousal, and Fatigue). Data from 139 full-time office workers that received seven experience sampling questionnaires per day over 3 workdays suggested satisfactory validity (including weak invariance cross-level isomorphism), level-specific reliability, and sensitivity to change. The scales also showed substantial correlations with retrospective measures of the corresponding or similar constructs and a degree of sensitivity to work sampling categories (type and mean of job task, people involved). Opportunities and recommendations for the investigation and the routine assessment of workplace stress are discussed.

References

  • Avanzi, L., Balducci, C., & Fraccaroli, F. (2013). Contributo alla validazione italiana del Copenhagen Burnout Inventory (CBI) [A contribution to the Italian validation of the Copenhagen Burnout Inventory]. Psicologia Della Salute, 2, 120–135. https://doi.org/10.3280/PDS2013-002008 First citation in articleCrossref, Google Scholar

  • Balducci, C., Fraccaroli, F., & Schaufeli, W. B. (2010). Psychometric properties of the Italian version of the Utrecht Work Engagement Scale (UWES-9). European Journal of Psychological Assessment, 26(2), 143–149. https://doi.org/10.1027/1015-5759/a000020 First citation in articleLink, Google Scholar

  • Barbaranelli, C., Fida, R., & Gualandri, M. (2013). Assessing counterproductive work behavior: A study on the dimensionality of CWB-Checklist. TPM – Testing, Psychometrics, Methodology in Applied Psychology, 20(3), 235–248. https://doi.org/10.4473/TPM20.3.3 First citation in articleCrossref, Google Scholar

  • Beal, D. J. (2015). ESM 2.0: State of the art and future potential of experience sampling methods in organizational research. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 383–407. https://doi.org/10.1146/annurev-orgpsych-032414-111335 First citation in articleCrossref, Google Scholar

  • Bowling, N. A., Alarcon, G. M., Bragg, C. B., & Hartman, M. J. (2015). A meta-analytic examination of the potential correlates and consequences of workload. Work and Stress, 29(2), 95–113. https://doi.org/10.1080/02678373.2015.1033037 First citation in articleCrossref, Google Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum. First citation in articleGoogle Scholar

  • Fisher, C. D., & To, M. L. (2012). Using experience sampling methodology in organizational behavior. Journal of Organizational Behavior, 33(7), 865–877. https://doi.org/10.1002/job.1803 First citation in articleCrossref, Google Scholar

  • Flake, J. K., & Fried, E. I. (2020). Measurement schmeasurement: Questionable measurement practices and how to avoid them. Advances in Methods and Practices in Psychological Science, 3(4), 456–465. https://doi.org/10.1177/2515245920952393 First citation in articleCrossref, Google Scholar

  • Gabriel, A. S., Podsakoff, N. P., Beal, D. J., Scott, B. A., Sonnentag, S., Trougakos, J. P., & Butts, M. M. (2019). Experience sampling methods: A discussion of critical trends and considerations for scholarly advancement. Organizational Research Methods, 22(4), 969–1006. https://doi.org/10.1177/1094428118802626 First citation in articleCrossref, Google Scholar

  • Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138 First citation in articleCrossref, Google Scholar

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Hurrell, J. J., Nelson, D. L., & Simmons, B. L. (1998). Measuring job stressors and strains: Where we have been, where we are, and where we need to go. Journal of Occupational Health Psychology, 3(4), 368–389. https://doi.org/10.1037/1076-8998.3.4.368 First citation in articleCrossref, Google Scholar

  • Jak, S., & Jorgensen, T. D. (2017). Relating measurement invariance, cross-level invariance, and multilevel reliability. Frontiers in Psychology, 8, 1–9. https://doi.org/10.3389/fpsyg.2017.01640 First citation in articleCrossref, Google Scholar

  • Kamarck, T., Janicki, D., Shiggman, S., Polk, D., Muldon, M., Libenauer, L., & Schwartz, J. (2002). Psychosocial demands and ambulatory blood pressure: A field assessment approach. Physiology & Behavior, 77(4–5), 699–704. https://doi.org/10.1016/S0031-9384(02)00921-6 First citation in articleCrossref, Google Scholar

  • Karasek, R., Brisson, C., Kawakami, N., Houtman, I., Bongers, P., & Amick, B. (1998). The Job Content Questionnaire (JCQ): An instrument for internationally comparative assessments of psychosocial job characteristics. Journal of Occupational Health Psychology, 3(4), 322–355. https://doi.org/10.1037/1076-8998.3.4.322 First citation in articleCrossref, Google Scholar

  • Kim, E. S., Dedrick, R. F., Cao, C., & Ferron, J. M. (2016). Multilevel factor analysis: Reporting guidelines and a review of reporting practices. Multivariate Behavioral Research, 51(6), 881–898. https://doi.org/10.1080/00273171.2016.1228042 First citation in articleCrossref, Google Scholar

  • Kristensen, T. S., Borritz, M., Villadsen, E., & Christensen, K. B. (2005). The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work & Stress, 19(3), 192–207. https://doi.org/10.1080/02678370500297720 First citation in articleCrossref, Google Scholar

  • Menghini, L., Pastore, M., & Balducci, C. (2022). Open data and supplementary materials of the article “Workplace stress in real time: Three parsimonious scales for the experience sampling measurement of stressors and strain at work.” https://doi.org/10.17605/OSF.IO/87A9P First citation in articleCrossref, Google Scholar

  • Ohly, S., Sonnentag, S., Niessen, C., & Zapf, D. (2010). Diary studies in organizational research. Journal of Personnel Psychology, 9(2), 79–93. https://doi.org/10.1027/1866-5888/a000009 First citation in articleLink, Google Scholar

  • Pindek, S., Arvan, M. L., & Spector, P. E. (2019). The stressor–strain relationship in diary studies: A meta-analysis of the within and between levels. Work and Stress, 33(1), 1–21. https://doi.org/10.1080/02678373.2018.1445672 First citation in articleCrossref, Google Scholar

  • R Development Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing. http://www.r-project.org/ First citation in articleGoogle Scholar

  • Robinson, M. A. (2009). Work sampling: Methodological advances and new applications. Human Factors and Ergonomics in Manufacturing, 20(1), 42–60. https://doi.org/10.1002/hfm.20186 First citation in articleCrossref, Google Scholar

  • Semmer, N. K., Zapf, D., & Dunckel, H. (1995). Assessing stress at work: A framework and an instrument. In O. SvaneC. JohansenEds., Work and health – Scientific basis of progress in the working environment (pp. 105–113). Office for Official Publications of the European Communities. First citation in articleGoogle Scholar

  • Shrout, P. E., & Lane, S. P. (2012). Psychometrics. In M. S. MehlT. S. ConnerEds., Handbook of research methods for sudying daily life (pp. 302–320). The Guilford Press. First citation in articleGoogle Scholar

  • Spector, P. E., & Jex, S. M. (1998). Development of four self-report measures of job stressors and strain: Interpersonal Conflict at Work Scale, Organizational Constraints Scale, Quantitative Workload Inventory, and Physical Symptoms Inventory. Journal of Occupational Health Psychology, 3(4), 356–367. https://doi.org/10.1037/1076-8998.3.4.356 First citation in articleCrossref, Google Scholar

  • Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520. https://doi.org/10.3102/1076998616646200 First citation in articleCrossref, Google Scholar

  • Tabanelli, M. C., Depolo, M., Cooke, R. M. T., Sarchielli, G., Bonfiglioli, R., Mattioli, S., & Violante, F. S. (2008). Available instruments for measurement of psychosocial factors in the work environment. International Archives of Occupational and Environmental Health, 82(1), 1–12. https://doi.org/10.1007/s00420-008-0312-6 First citation in articleCrossref, Google Scholar

  • Thorsen, S. V., & Bjorner, J. B. (2010). Reliability of the Copenhagen Psychosocial Questionnaire. Scandinavian Journal of Public Health, 38(3 suppl), 25–32. https://doi.org/10.1177/1403494809349859 First citation in articleCrossref, Google Scholar

  • Toderi, S., Balducci, C., Edwards, J. A., Sarchielli, G., Broccoli, M., & Mancini, G. (2013). Psychometric properties of the UK and Italian versions of the HSE Stress Indicator Tool. European Journal of Psychological Assessment, 29(1), 72–79. https://doi.org/10.1027/1015-5759/a000122 First citation in articleLink, Google Scholar

  • Van Katwyk, P. T., Fox, S., Spector, P. E., & Kelloway, E. K. (2000). Using the Job-Related Affective Well-Being Scale (JAWS) to investigate affective responses to work stressors. Journal of Occupational Health Psychology, 5(2), 219–230. https://doi.org/10.1037/1076-8998.5.2.219 First citation in articleCrossref, Google Scholar

  • Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192–196. https://doi.org/10.3758/BF03206482 First citation in articleCrossref, Google Scholar

  • Warr, P. B. (1994). A conceptual framework for the study of work and mental health. Work & Stress, 8(2), 84–97. https://doi.org/10.1080/02678379408259982 First citation in articleCrossref, Google Scholar

  • Wetherell, M. A., & Carter, K. (2014). The multitasking framework: The effects of increasing workload on acute psychobiological stress reactivity. Stress and Health, 30(2), 103–109. https://doi.org/10.1002/smi.2496 First citation in articleCrossref, Google Scholar

  • Wilhelm, P., & Schoebi, D. (2007). Assessing mood in daily life: Structural validity, sensitivity to change, and reliability of a short-scale to measure three basic dimensions of mood. European Journal of Psychological Assessment, 23(4), 258–267. https://doi.org/10.1027/1015-5759.23.4.258 First citation in articleLink, Google Scholar

  • Xiong, H., Huang, Y., Barnes, L. E., & Gerber, M. S. (2016). Sensus: A cross-platform, general-purpose system for mobile crowdsensing in human-subject studies. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 415–426. https://doi.org/10.1145/2971648.2971711 First citation in articleCrossref, Google Scholar