Skip to main content

Validating Four Standard Scales in Spiritually Practicing and Nonpracticing Samples Using Propensity Score Matching

Published Online: Doi: https://doi.org/10.1027/1015-5759.24.3.165

Validation studies of standard scales in the particular sample that one is studying are essential for accurate conclusions. We investigated the differences in answering patterns of the Brief-Symptom-Inventory (BSI), Transpersonal Trust Scale (TPV), Sense of Coherence Questionnaire (SOC), and a Social Support Scale (F-SoZu) for a matched sample of spiritually practicing (SP) and nonpracticing (NSP) individuals at two measurement points (t1, t2). Applying a sample matching procedure based on propensity scores, we selected two sociodemographically balanced subsamples of N = 120 out of a total sample of N = 431. Employing repeated measures ANOVAs, we found an intersample difference in means only for TPV and an intrasample difference for F-SoZu. Additionally, a group × time interaction effect was found for TPV. While Cronbach’s α was acceptable and comparable for both samples, a significantly lower test-rest-reliability for the BSI was found in the SP sample (rSP = .62; rNSP = .78). Thus, when researching the effects of spiritual practice, one should not only look at differences in means but also consider time stability. We recommend propensity score matching as an alternative for randomization in variables that defy experimental manipulation such as spirituality.

References

  • Albani, C. , Bailer, H. , Blaser, G. , Geyer, M. , Brähler, E. , Grulke, N. (2003). Psychometrische Überprüfung der Skala “Transpersonales Vertrauen” (TPV) in einer repräsentativen Bevölkerungsstichprobe [A psychometric investigation of the transpersonal trust scale in a sample representative for the German population]. Transpersonale Psychologie und Psychotherapie, 9(1), 86–98. First citation in articleGoogle Scholar

  • Antonovsky, A. (1979). Health, stress, and coping: New perspectives on mental and physical well-being. San Francisco: Jossey Bass. First citation in articleGoogle Scholar

  • Antonovsky, A. (1993). The structure and properties of the Sense of Coherence Scale. Social Sciences and Medicine, 36, 725–733. First citation in articleCrossrefGoogle Scholar

  • Arrindell, W.A. (2001). Changes in waiting-list patients over time: Data on some commonly-used measures. Beware! Behavior Research and Therapy, 39, 1227–1247. First citation in articleCrossrefGoogle Scholar

  • Barker, C. , Pistrang, N. (2002). Psychotherapy and social support: Integrating research on psychological helping. Clinical Psychology Review, 22, 361–379. First citation in articleCrossrefGoogle Scholar

  • Belschner, W. (2001). Tun und Lassen: Ein komplementäres Konzept der Lebenskunst [Acting and resting: A complementary concept for balancing life]. Transpersonale Psychologie und Psychotherapie, 7(2), 85–102. First citation in articleGoogle Scholar

  • Belschner, W. (2003). Ergebnisse empirischer Forschung zur Transpersonalen Psychotherapie [Transpersonal psychotherapy – Some empirical results]. In J. Galuska, (Ed.), Den Horizont erweitern – Die transpersonale Dimension in der Psychotherapie [Broadening one’s horizons – The transpersonal dimension within psychotherapy] (pp. 93–135). Berlin, Germany: Ulrich Leutner. First citation in articleGoogle Scholar

  • Blalock, H.M. (1972). Social statistics. New York: McGraw-Hill. First citation in articleGoogle Scholar

  • Brookhart, M.A. , Schneeweiss, S. , Rothman, K.J. , Glynn, R.J. , Avorn, J. , Sturmer, T. (2006). Variable selection for propensity score models. American Journal of Epidemiology, 163, 1149–1156. First citation in articleCrossrefGoogle Scholar

  • Derogatis, L. , Lazarus, L. (1994). SCL-90-R, Brief Symptom Inventory, and matching clinical rating scales. The use of psychological testing for treatment planning and outcome assessment, 217–248. First citation in articleGoogle Scholar

  • Devon, B. (2005). Methodological pitfalls in the study of religiosity and spirituality. Western Journal of Nursing Research, 27, 628–647. First citation in articleCrossrefGoogle Scholar

  • Fydrich, T. , Sommer, G. , Brähler, E. (2002). F-SozU – Fragebogen zur sozialen Unterstützung (F-SozU) [F-SoZu: a scale for measuring social support]. In E. Brähler, J. Schumacher, B. Strauß, (Eds.), Diagnostische Verfahren in der Psychotherapie [Diagnostic instruments for psychotherapy] (pp. 150–153). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • George, L.K. , Larson, D.B. , Koenig, H.G. , McCullough, M.E. (2000). Spirituality and health: What we know, what we need to know. Journal of Social and Clinical Psychology, 19, 102–116. First citation in articleCrossrefGoogle Scholar

  • Geyer, S. (1997). Some conceptual considerations on the sense of coherence. Social Science and Medicine, 44, 1771–1779. First citation in articleCrossrefGoogle Scholar

  • Hill, P.C. , Pargament, K.I. (2003). Advances in the conceptualization and measurement of religion and spirituality: Implications for physical and mental health research. American Psychologist, 58, 64–74. First citation in articleCrossrefGoogle Scholar

  • Hogan, B.E. , Linden, W. , Najarian, B. (2002). Social support interventions: Do they work? Clinical Psychology Review, 22, 381–440. First citation in articleCrossrefGoogle Scholar

  • Klaiberg, A. (2002). Brief Symptom Inventory von L.R. Derogatis. In E. Brähler, H. Holling, D. Leutner, & F. Peterman, (Eds.), Brickenkamp Handbuch psychologischer und pädagogischer Tests [Brickenkamp manual of psychological and pedagogical tests] (pp. 822–825). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Koenig, H. , McCullough, M. , Larson, D. (Eds.). (2001). Handbook of religion and health. New York: Oxford University Press. First citation in articleCrossrefGoogle Scholar

  • Kohls, N. (2004). Außergewöhnliche Erfahrungen – Blinder Fleck der Psychologie? [Exceptional experiences: The blind spot of psychology?]. Münster, Germany: Lit. First citation in articleGoogle Scholar

  • Kohls, N. , Walach, H. (2006). Exceptional experiences and spiritual practice – A new measurement approach. Spirituality and Health International, 7(3), 125–150. First citation in articleCrossrefGoogle Scholar

  • Kohls, N. , Walach, H. (2007). Psychological distress, experiences of ego loss and spirituality. Exploring the Effects of Spiritual Practice Social Behavior and Personality, 35, 1301–1316. First citation in articleCrossrefGoogle Scholar

  • Kohls, N. , Hack, A. , Walach, H. (in press). Measure the unmeasurable by ticking boxes and actually opening pandoras box? The Archive for the Psychology of Religion. First citation in articleGoogle Scholar

  • Larson, D.B. , Larson, S.S. (2003). Spirituality’s potential relevance to physical and emotional health: A brief review of quantitative research. Journal of Psychology and Theology, 31, 37–51. First citation in articleGoogle Scholar

  • Luellen, J.K. , Shadish, W.R. , Clark, M.H. (2005). Propensity scores: An introduction and experimental test. Evaluation Review, 29, 530–558. First citation in articleCrossrefGoogle Scholar

  • Painter, J. (2004). Propensity matching via SPSS. Syntax for the SPSS software package. Retrieved January 15, 2007, from www.unc.edu/~painter/SPSSsyntax/propen.txt First citation in articleGoogle Scholar

  • Rosenbaum, P.R. (2002). Overt bias in observational studies. In P.R. Rosenbaum, , Observational studies (pp. 71–104). New York: Springer. First citation in articleCrossrefGoogle Scholar

  • Rosenbaum, P.R. , Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. First citation in articleCrossrefGoogle Scholar

  • Rosenbaum, P.R. , Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79(387), 516–524. First citation in articleCrossrefGoogle Scholar

  • Schmitz, N. , Hartkamp, N. , Kruse, J. , Franke, G.H. , Reister, G. , Tress, W. (2000). The Symptom Check-List-90-R (SCL-90-R): A German validation study. Quality of Life Research, 9, 185–193. First citation in articleCrossrefGoogle Scholar

  • Schumacher, J. , Gunzelmann, T. , Brähler, E. (2000). Deutsche Normierung der Sense of Coherence Scale von Antonovsky [Antonovsky‘s Sense of Coherence Scale – A German validation study]. Diagnostica, 46, 208–213. First citation in articleLinkGoogle Scholar

  • Schumacher, J. , Wilz, G. , Gunzelmann, T. , Brähler, E. (2000). Die Sense of Coherence Scale von Antonovsky – Teststatistische Überprüfung in einer repräsentativen Bevölkerungsstichprobe und Konstruktion einer Kurzskala [Antonovsky‘s Sense of Coherence Scale: Psychometric investigation in a sample representative of the German population and construction of a short form]. Psychotherapie, Psychosomatik, med. Psychologie, 50, 472–482. First citation in articleCrossrefGoogle Scholar

  • Schwartz, C.E. , Sprangers, M.A.G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48, 1531–1548. First citation in articleCrossrefGoogle Scholar

  • Sloan, R.P. , Bagiella, E. , Powell, T. (1999). Religion, spirituality and medicine. Lancet, 353, 664–667. First citation in articleCrossrefGoogle Scholar

  • Thoresen, C. , Harris, A. (2002). Spirituality and health: What’s the evidence and what’s needed? Annals of Behavioral Medicine, 24(1), 3–13. First citation in articleCrossrefGoogle Scholar

  • Walsh, R. , Shapiro, S.L. (2006). The meeting of meditative disciplines and Western psychology: A mutually enriching dialog. American Psychologist, 61, 227–339. First citation in articleCrossrefGoogle Scholar

Access content

To read the fulltext, please use one of the options below to sign in or purchase access.