Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

German Language Adaptation of the Kalamazoo Communication Skills Assessment Form (KCSAF): A Multi-Method Study of Two Cohorts of Medical Students

Published Online:August 26, 2019https://doi.org/10.1024/1010-0652/a000241

Abstract

Abstract. This work investigates the German version of the Kalamazoo Communication Skills Assessment Form (KCSAFd) for three assessment methods: students' self-assessment (KCSAFd-self), assessment by standardised patients (KCSAFd-sPat) and video-assessment by trained raters (KCSAFd-video). Videotaped simulated patient consultations of N = 163 medical students from the first (n = 97) and the final clinical years (n = 66) were rated using the KCSAFd. Investigating the psychometric properties of the instrument, we found a two factor-construct with interpersonal and conversational competence. All methods showed good internal consistency and acceptable model fit values. Additionally, we found plausible relationships between the three methods and meaningful differences between the two groups of students.


Deutschsprachige Adaption des Kalamazoo Communication Skills Assessment Form (KCSAF): Eine multimethodale Studie zweier Kohorten Medizinstudierender

Zusammenfassung. Diese Arbeit untersucht die deutsche Version des Kalamazoo Communication Skills Assessment Form (KCSAFd) für die drei Methoden der Selbstbeurteilung der Studierenden (KCSAFd-self), der Beurteilung durch Simulationspatient_innen (KCSAFd-sPat) und der Beurteilung durch trainierte Rater_innen mittels Videoanalyse (KCSAFd-video). Videographierte simulierte Anamnesegespräche von N = 163 Medizinstudierenden aus dem ersten (n = 97) und aus dem letzten klinischen Jahr (n = 66) wurden mit Hilfe des KCSAFd ausgewertet. In der Analyse der psychometrischen Eigenschaften des KCSAFd zeigte sich ein zweidimensionales, die Faktoren zwischenmenschliche Fähigkeiten und Gesprächsführung umfassendes Konstrukt. Alle drei Methoden zeigten gute interne Konsistenz und akzeptable Model-Fit-Werte. Weiterhin konnten mithilfe des Instruments sinnvolle Zusammenhänge zwischen den Methoden und plausible Gruppenunterschiede aufgezeigt werden.

References

  • Amaral, A. B. C. N., Rider, E. A., Lajolo, P. P., Tone, L. G., Pinto, R. M. C., Lajolo, M. P., & Calhoun, A. W. (2016). Development of a Brazilian Portuguese adapted version of the Gap-Kalamazoo communication skills assessment form. International Journal of Medical Education, 7, 400 – 405. https://doi.org/10.5116/ijme.583a.df42 First citation in articleCrossref, Google Scholar

  • Blömeke, S., Gustafsson, J.-E., & Shavelson, R. J. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift Für Psychologie / Journal of Psychology, 223(1), 3 – 13. https://doi.org/10.1027/2151-2604/a000194 First citation in articleLink, Google Scholar

  • Calhoun, A. W., Rider, E. A., Meyer, E. C., Lamiani, G., & Truog, R. D. (2009). Assessment of communication skills and self-appraisal in the simulated environment: Feasibility of multirater feedback with gap analysis. Simulation in Healthcare: Journal of the Society for Simulation in Healthcare, 4(1), 22 – 29. https://doi.org/10.1097/SIH.0b013e318184377a First citation in articleCrossref, Google Scholar

  • Calhoun, A. W., Rider, E. A., Peterson, E., & Meyer, E. C. (2010). Multi-rater feedback with gap analysis: An innovative means to assess communication skill and self-insight. Patient Education and Counseling, 80(3), 321 – 326. https://doi.org/10.1016/j.pec.2010.06.027 First citation in articleCrossref, Google Scholar

  • Davis, D. A., Mazmanian, P. E., Fordis, M., van Harrison, R., Thorpe, K. E., & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296(9), 1094 – 1102. First citation in articleCrossref, Google Scholar

  • Desoete, A. (2008). Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3(3), 189 – 206. https://doi.org/10.1007/s11409-008-9026-0 First citation in articleCrossref, Google Scholar

  • Donnon, T., Al Ansari, A., Al Alawi, S., & Violato, C. (2014). The reliability, validity, and feasibility of multisource feedback physician assessment: A systematic review. Academic Medicine: Journal of the Association of American Medical Colleges, 89(3), 511 – 516. https://doi.org/10.1097/ACM.0000000000000147 First citation in articleCrossref, Google Scholar

  • Eid, M., & Diener, E. (Eds.). (2006). Handbook of multimethod measurement in psychology (1st ed.). Washington, DC: American Psychological Association. First citation in articleCrossref, Google Scholar

  • Eva, K. W., Cunnington, J. P. W., Reiter, H. I., Keane, D. R., & Norman, G. R. (2004). How can I know what I don't know? Poor self assessment in a well-defined domain. Advances in Health Sciences Education: Theory and Practice, 9(3), 211 – 224. https://doi.org/10.1023/B:AHSE.0000038209.65714.d4 First citation in articleCrossref, Google Scholar

  • Field, A. (2013). Discovering statistics using IBM SPSS Statistics: And sex and Drugs and rock ‘n’ roll (4th edition). London: Sage. First citation in articleGoogle Scholar

  • Harendza, S., Berberat, P. O., & Kadmon, M. (2017). Assessing competences in medical students with a newly designed 360-degree examination of a simulated first day of residency: A feasibility study. Journal of Community Medicine & Health Education, 07(04). https://doi.org/10.4172/2161-0711.1000550 First citation in articleCrossref, Google Scholar

  • Hartig, J. (2008). Psychometric models for the assessment of competencies. In J. HartigE. KliemeD. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 70 – 90). Göttingen, Germany: Hogrefe & Huber. First citation in articleGoogle Scholar

  • IBM Corp. (2017). IBM SPSS statistics for windows. Armonk, NY: IBM Corp. First citation in articleGoogle Scholar

  • Kiessling, C., Bauer, J., Gartmeier, M., Iblher, P., Karsten, G., Kiesewetter, J., Fischer, M. R. (2016). Development and validation of a computer-based situational judgement test to assess medical students' communication skills in the field of shared decision making. Patient Education and Counseling. Advance online publication. https://doi.org/10.1016/j.pec.2016.06.006 First citation in articleGoogle Scholar

  • Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press. First citation in articleGoogle Scholar

  • Kurtz, S., Silverman, J., & Draper, J. (2016). Teaching and learning communication skills in medicine (2nd ed.). Boca Raton: CRC Press. Retrieved from http://gbv.eblib.com/patron/FullRecord.aspx?p=4711378 First citation in articleGoogle Scholar

  • Langer, I., & Schulz von Thun, F. (2007). Messung komplexer Merkmale in Psychologie und Pädagogik [Measuring complex traits in psychology and education]. Münster: Waxmann. First citation in articleGoogle Scholar

  • Larsen, R. J., & Prizmic-Larsen, Z. (2006). Measuring emotions: Implications of a multimethod perspective. In M. EidE. Diener (Eds.), Handbook of multimethod measurement in psychology (1st ed., pp. 337 – 351). Washington, DC: American Psychological Association. https://doi.org/10.1037/11383-023 First citation in articleGoogle Scholar

  • Lie, D. A., Forest, C. P., & Richter-Lagha, R. (2018). Evaluating medical Spanish proficiency: A comparison of physician assistant student self-assessment to standardized patient and expert faculty member ratings. The Journal of Physician Assistant Education: The Official Journal of the Physician Assistant Education Association, 29(3), 162 – 166. https://doi.org/10.1097/JPA.0000000000000211 First citation in articleGoogle Scholar

  • Mercer, S. W., Maxwell, M., Heaney, D., & Watt, G. C. (2004). The consultation and relational empathy (CARE) measure: Development and preliminary validation and reliability of an empathy-based consultation process measure. Family Practice, 21(6), 699 – 705. https://doi.org/10.1093/fampra/cmh621 First citation in articleCrossref, Google Scholar

  • Muthén & Muthén. (2017). MPlus 8. Los Angeles: Muthén & Muthén. First citation in articleGoogle Scholar

  • Neumann, M., Edelhauser, F., Tauschel, D., Fischer, M. R., Wirtz, M., Woopen, C., Scheffer, C. (2011). Empathy decline and its reasons: A systematic review of studies with medical students and residents. Academic Medicine: Journal of the Association of American Medical Colleges, 86(8), 996 – 1009. https://doi.org/10.1097/ACM.0b013e318221e615 First citation in articleCrossref, Google Scholar

  • Neumann, M., Wirtz, M., Bollschweiler, E., Warm, M., Wolf, J., & Pfaff, H. (2008). Psychometrische Evaluation der deutschen Version des Messinstruments “Consultation and Relational Empathy” (CARE) am Beispiel von Krebspatienten [Psychometric evaluation of the German version of the “Consultation and Relational Empathy” (CARE) measure at the example of cancer patients]. Psychotherapie, Psychosomatik, Medizinische Psychologie, 58(1), 5 – 15. https://doi.org/10.1055/s-2007-970791 First citation in articleGoogle Scholar

  • Peterson, E. B., Calhoun, A. W., & Rider, E. A. (2014). The reliability of a modified Kalamazoo Consensus Statement Checklist for assessing the communication skills of multidisciplinary clinicians in the simulated environment. Patient Education and Counseling, 96(3), 411 – 418. https://doi.org/10.1016/j.pec.2014.07.013 First citation in articleCrossref, Google Scholar

  • R Core Team. (2014). R: A language and environment for statistical computing. Vienna, Austria: R foundation for statistical computing. Available from: http://www.R-project.org/ First citation in articleGoogle Scholar

  • Rider, E. A. (2010). Interpersonal and communication skills. In E. A. RiderR. H. Nawotniak (Eds.), A practical guide to teaching and assessing the ACGME core competencies (2nd ed., pp. 1 – 137). Marblehead, MA: HCPro, Inc. First citation in articleGoogle Scholar

  • Scheffer, S., Muehlinghaus, I., Froehmel, A., & Ortwein, H. (2008). Assessing students' communication skills: Validation of a global rating. Advances in Health Sciences Education: Theory and Practice, 13(5), 583 – 592. https://doi.org/10.1007/s10459-007-9074-2 First citation in articleCrossref, Google Scholar

  • Seidel, T. (Ed.). (2005). How to run a video study: Technical report of the IPN video study. Münster: Waxmann. Available from: http://www.content-select.com/index.php?id=bib_view&ean=9783830965695 First citation in articleGoogle Scholar

  • Simmenroth-Nayda, A., Heinemann, S., Nolte, C., Fischer, T., & Himmel, W. (2014). Psychometric properties of the Calgary Cambridge guides to assess communication skills of undergraduate medical students. International Journal of Medical Education, 5, 212 – 218. https://doi.org/10.5116/ijme.5454.c665 First citation in articleCrossref, Google Scholar

  • Weinert, F. E. (1999). Konzept of Kompete nz [Concept of competence]. Paris: OECD: First citation in articleGoogle Scholar

  • Wiesbeck, A. B. (2015). An evaluation of simulated conversations as an assessment of pre-service teachers' communication competence in parent-teacher conversations (Doctoral Dissertation). Technical University of Munich, München. https://mediatum.ub.tum.de/?id=1271404 First citation in articleGoogle Scholar

  • Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., & Erikson, P. (2005). Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value in Health: The Journal of the International Society for Pharmacoeconomics and Outcomes Research, 8(2), 94 – 104. https://doi.org/10.1111/j.1524-4733.2005.04054.x First citation in articleCrossref, Google Scholar

  • Wirtz, M. A., & Caspar, F. (2002). Beurteilerübereinstimmung und Beurteilerreliabilität: Methoden zur Bestimmung und Verbesserung der Zuverlässigkeit von Einschätzungen mittels Kategoriensystemen und Ratingskalen [Interrater agreement and Interrater reliability]. Göttingen: Hogrefe Verl. für Psychologie. First citation in articleGoogle Scholar

  • World Health Organization. (2016). Process of translation and adaptation of instruments. Available from: http://www.who.int/substance_abuse/research_tools/translation/en/ First citation in articleGoogle Scholar