The Lack of Cross-National Equivalence of a Therapist Adherence Measure (TAM-R) in Multisystemic Therapy (MST)
Abstract
Abstract. This two-study report investigates the equivalence of the Dutch Therapist Adherence Measure Revised (TAM-R) to the US original. The TAM-R is a questionnaire measuring therapist adherence to the treatment model of Multisystemic Therapy (MST). Monitoring of therapist adherence is essential for evidence-based interventions to ensure the quality of the delivered treatment. International implementation of the TAM-R assumes cross-national equivalence, even though this assumption has never been investigated. In study 1 Rasch analysis was applied to 1,875 Dutch TAM-R reports and the response category frequency distributions of the items of 1,875 US TAM-R reports. Response frequencies were more heavily skewed in the US compared to the Netherlands and several items showed Differential Item Functioning (DIF). Study 2 investigated whether adaptations to the translation of the items and response categories could improve equivalence. For this purpose, 237 families were randomly allocated to 1 of 3 versions (original TAM-R, adapted items only, adapted items and response categories) and the analyses from study 1 were replicated. Results indicated that equivalence was not improved by the adapted translations. The article concludes with a discussion of several potential other sources of bias, such as differences in sample characteristics, implementation of MST, and response styles.
References
(2001). Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermong, Research Center for Children, Youth & Families.
(2012). The Rasch rating model and the disordered threshold controversy. Educational and Psychological Measurement, 72, 547–573.
(2006). An investigation into the factors influencing extreme response style: Improving meaning of translated and culturally adapted rating scales. Educational and Psychological Measurement, 66, 374–392.
(2007). Applying the Rasch model: Fundamental measurement in human sciences. Mahwah, NJ: Erlbaum.
(2012). Structural Equation Modeling with Mplus. Basic concepts, applications, and programming. NY: Routledge Taylor & Francis Group.
(2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10, 107–132.
(1979). Reliability and validity assessment. Beverly Hills, CA: Sage.
(2006). The impact of response styles on the stability of cross-national comparisons. Journal of Business Research, 59, 925–935.
(2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.
(1991). Fundamentals of Item Response Theory. Newbury Park, CA: Sage.
(2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross Cultural Management, 6, 243–266.
(2011). Efficacy studies to large-scale transport: The development and validation of Multisystemic Therapy Programs. Annual Review of Clinical Psychology, 7, 351–381.
(2006). Multisystemic Therapy Adherence Scale-Revised (TAM-R) Unpublished instrument. Charleston, SC: Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina.
(1999). The role of quality assurance in achieving outcomes in MST programs. Journal of Juvenile Justice and Detention Services, 14, 1–17.
(2010). Multisysteem therapie voor kinderen en jongeren met antisociaal gedrag
[Multisystemic therapy for antisocial behavior in children and adolescents] . Amsterdam, The Netherlands: Uitgeverij SWP.(2013). Reflections on treatment integrity in a dissemination and implementation framework. Clinical Psychology: Science & Practice, 20, 114–119.
(2011a). Winsteps® Rasch measurement computer program, Beaverton, OR: Winsteps.com.
(2011b). Winsteps® Rasch measurement computer program User’s Guide. Beaverton, OR: Winsteps.com.
(2014). The Swedish implementation of Mutisystemic Therapy for adolescents: Does treatment experience predict treatment adherence? Journal of Clinical Child & Adolescent Psychology, 43, 643–655.
(2011). Rasch analysis of repeated measures. Rasch Measurement Transactions, 25, 1317.
(2011). Dealing with extreme response style in cross-cultural research: A restricted latent class factor analysis approach. Sociological Methodology, 41, 13–47.
. (n.d.). Adherence data: Tips and strategies. Retrieved from http://www.mstinstitute.org/qa_program/pdfs/TAMCollectionTips.pdf
. (n.d.). MST Therapist Adherence-Revised (TAM-R): Guidelines for administration. Retrieved from www.mstinstitute.org/qa_program/pdfs/TAMAdministration.pdf
. (2010). 2010 MST Data Report. Retrieved from http://www.mstinstitute.org/MST2010DataReportFinal.pdf
. (2012). MST Therapist Adherence Measure – Revised (TAM-R): Guidelines for Interpretation. Retrieved from http://www.mstinstitute.org/qa_program/tam.shtml
(2011). On the topic of treatment integrity. Clinical Psychology, Science and Practice, 18, 148–153.
(2013). Predictors of therapist adherence and emotional bond in Multisystemic Therapy: Testing ethnicity as a moderator. Journal of Child and Family Studies, 22, 122–136.
(2008). Towards evidence-based transport of evidence-based treatments: MST as an example. Journal of Child and Adolescent Substance Abuse, 17, 69–91.
(2011). It’s a bird, it’s a plane, it’s fidelity measurement in the real world. Clinical Psychology, Science and Practice, 18, 142–147.
(2008). Therapist adherence and organizational effects on change in youth behavior problems one year after Multisystemic Therapy. Administration and Policy in Mental Health and Mental Health Services Research, 35, 379–394.
(2009). Long-term youth criminal outcomes in MST transport: The impact of therapist adherence and organizational climate and structure. Journal of Clinical Child & Adolescent Psychology, 38, 91–105.
(2003). Client-level predictors of adherence to MST in community service settings. Family Process, 42, 345–359.
(2008). The international implementation of Multisystemic Therapy. Evaluation & the Health Professions, 31, 211–225.
(2008). Rasch fit statistics and sample size considerations for polytomous data. Medical Research Methodology, 8, 33. doi: 10.1186/1471-2288-8-33
(2007). The Rasch measurement model in rheumatology: What is it and why is it? When should it be applied and what should one look for in a Rasch paper? Arthritis & Rheumatism (Arthritis Care & Research), 57, 1358–1362.
(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 1–7.
(2004). Bias and equivalence in cross-cultural assessment: An overview. Revue Européenne de Psychologie Appliqué, 54, 119–135.
(2005). Translation and cross-cultural adaptation of assessment instruments used in psychological research with children and families. Clinical Child and Family Psychology Review, 8, 135–147.
