How far are N-of-1 studies suitable evaluation designs in forensic psychiatric sports therapy?
A methodological discussion
Abstract
Abstract:Introduction: Some evidence showed positive effects of sports therapy in forensic settings. Most of these evaluation studies analyzed between-subject relations and used RCTs (randomized controlled trials) to investigate intervention effects. RCTs normally randomize participants to an intervention and a control group with the intention to compare averaged group-findings. However, such averaged results, which are based on group effects may not apply to every single forensic patient as they do not adequately address that these patients are unique due to their complex psychopathology. Thus, RCTs do not adequately address the following circumstances especially relevant in forensic settings: heterogeneity and low sample size of therapy groups and low compliance rate of forensic patients. To address these challenges, it could be worthwhile to focus on within-subject relations by using N-of-1 studies. Methods: This viewpoint summarizes the possibilities and limitations of N-of-1 study designs regarding the challenges heterogeneity, low sample size and a low compliance rate when evaluating forensic sports therapy. Results: N-of-1 studies offer a useful addition in the evaluation of sports therapy in forensic psychiatry. They are able to evaluate individual effects and they increase the power of the study by increasing the number of measurements. Nevertheless, they are associated with limitations, for example increased effort due to frequent measurements and long study duration or the difficulty to generalize the results. Conclusion: Future studies should implement N-of-1 study designs in forensic psychiatric sports therapy to gain evidence and should find solutions to deal with the limitations (e.g., digital technology).
To investigate health-enhancing effects of sports therapy in forensic psychiatry, evaluation studies normally use RCTs (randomized controlled trials) [1, 2]. However, such a design does not adequately address the challenges of patients with severe mental illnesses like forensic psychiatric patients [3]: First, each patient’s symptoms are highly individual with a different severity and complexity [4]. Thus, homogeneous patient groups are not possible due to the heterogeneous characteristics of forensic patients [3]. In sports therapy, all patients are treated together in groups, since no subdivision is made into diagnoses or paragraphs or severity of the disease, resulting in very heterogeneous groups of sports therapy [3]. Second, psychiatric facilities are characterized by a rather small patient group in sports therapy [5, 2] and lower fluctuations in the patient group, because forensic patients normally stay multiple months or years in stationary therapy facilities [6]. Third, patients with serious mental illnesses often show low treatment compliance [7, 8].
Previous evaluation studies are normally RCTs, which mainly investigated between-subject effects between patients of the intervention and control group and are suitable to investigate the average effect of an intervention for a certain group [9]. Such RCTs are suitable for single, well-defined disorders or diagnoses [10], but challenging when investigating patients with severe mental diseases [11]. One advantage of RCTs with sufficient power is to average effects of specific interventions in forensic sports therapy [9]. However, due to high heterogeneity within forensic patients, such averaged results may not apply to every single patient [12, 13] as they do not adequately address that forensic patients are unique due to their complex psychopathology [3]. Patients in forensic psychiatry vary strongly regarding their diagnosis and risk factors [14] as there are multiple factors that contribute to the development of mental illness and delinquent behavior in forensic psychiatric patients [15]. Therefore, the results of RCTs in forensic psychiatry cannot be transferred to each forensic patient [10] and indicate that this classic intervention design might not be appropriate for forensic psychiatry.
To conduct adequate evaluation studies in forensic sports therapy the specific circumstances of forensic settings (heterogeneity, low sample size, and low compliance rate) should be addressed. N-of-1 studies are able to face these challenges [12, 16, 17, 18] and are of particular interest in the healthcare sector [17]. They provide more detailed information about the individual effects in health and well-being outcomes than RCTs [16–18], cohort studies or case-control studies [12]. This brief report addresses the specific circumstances of forensic settings and presents possibilities and limitations of N-of-1 studies to evaluate effects of forensic sports therapy.
Definition of N-of-1 studies
N-of-1 studies (observational or experimental [12]) are flexible [13] and focus on a single patient [19]. Instead of the term N-of-1 study, the following terms case study [16, 17, 20], single-subject study [12], or single-patient study [21] are also used in the literature. In contrast, a case report presents a descriptive observation of a patient [21]. Compared to N-of-1 studies, case reports are often used in forensic psychiatry [22].
Possibilities and limitations of N-of-1 studies
Heterogeneity
Heterogeneity in forensic psychiatry means that each patient shows a different expression of complex symptoms [4] and thus no homogeneous picture of forensic patients can be depicted [3]. That argues for evaluating patients in N-of-1 studies to gain more details about the individual changes [16, 17, 18].
For example, a volleyball intervention with forensic psychiatric patients (N=19) improved social competent behavior [1]. A further evaluation would be valuable about the extent to which the different dimensions of social competence (communication skills, social responsibility, perspective taking) changed for whom of the forensic patients. Similar applies to another study [2], where forensic patients have been evaluated in an exercise program and reported positive effects on psychiatric symptoms (e.g., improving mood, coping with stress), level of fitness, confidence, and self-esteem for the intervention group. To address heterogeneity in forensic patients, literature shows evidence by combining an RCT (n=50) and an N-of-1 study (n=4) to investigate a neurofeedback protocol [10]. Besides the RCT design, four patients were additionally examined in an N-of-1 study design [10]. Results of the N-of-1 study showed that three patients completed all study phases, none of the patients followed the training like planned and the behavioral measures varied between the patients [23].
Nevertheless, averaged conclusions about the population based on N-of-1 studies are difficult [12]. A possible solution could be to conduct replications of various N-of-1 studies with the same research topic [12, 17]. Due to economic reasons and to get statements for more than one patient, aggregated or cumulative N-of-1 studies might be able to infer individual effects on specific patient groups or patient characteristics if the N-of-1 studies show similar effects with regard to sports therapy [16, 17]. Moreover, it could indicate the possibility to generalize the effects to other forensic patients [12].
Low sample size
A low sample size in psychiatric sports therapy groups [5] leads to low statistical power in evaluation studies [24]. To increase the power of a study [11, 12, 16], N-of-1 studies use data assessments over a long time (time-series data) [12, 21], because in N-of-1 studies the patient is in both, an intervention situation and a control situation [12]. However, assessing data over a long period puts high requirements [12] on forensic patients due to their low capacity of motivation [3].
Low compliance rate
Personalized feedback motivates people to increase their level of physical activity [25]. N-of-1 studies might face the challenge of low compliance rate due to personalized feedback that motivates the patient to endure until the end of the study or due to ABA-designs or observational study designs [12]. The long accommodation period of forensic patients offers the opportunity to conduct intensive longitudinal studies and to assess patients over a long period of time [6].
Conclusion
This brief report discusses the possibilities and limitations of N-of-1 studies to evaluate sports therapy in forensic settings. In health psychology and behavioral medicine, N-of-1 studies have still received too little recognition and application [26] and are rarely used in psychiatry due to the effort for the patients [12].
To evaluate sports therapy in forensic psychiatry settings, N-of-1 studies might be worthwhile to supplement existing evaluation designs, because they address the challenges of evaluation studies in forensic sports therapy more adequately, for example in the evaluation of individual effects or the increase of the power of the study by increasing the number of measurements (sample size). Nevertheless, they are associated with limitations, for example the high requirements due to the frequent measurements and the long study duration or the poor generalization of the results.
Future studies should implement N-of-1 study designs in forensic psychiatric sports therapy to gain evidence. If this type of study design contributes to more detailed knowledge, then solutions to deal with the limitations (e.g., digital technology) need to be considered [17, 20]. It is possible that for patients with the same diagnoses, the evidence of N-of-1 studies result in a recommended treatment.
This brief report was written within the context of a third party funded project. We would like to thank the Centre for Psychiatry Reichenau for financing the third-party project funding and the library of the University of Konstanz for funding the open access publication of this article.
References
1 . Promotion of social competence with the sports-therapeutic volleyball program GDIVP in forensic psychiatry: a pilot study. Dtsch Z Sportmed. 2022;73(2):70–6. https://doi.org/10.5960/dzsm.2021.518
2 . Evaluation of service users’ experiences of participating in an exercise programme at the Western Australian State Forensic Mental Health Services. Int J Ment Health Nurs. 2012;21:229–35. https://doi.org/10.1111/j.1447-0349.2011.00787.x
3 . Sporttherapie in der forensischen Psychiatrie und Psychotherapie – eine Standortbestimmung
[Sports therapy in the forensic psychiatry and psychotherapy - an assessment of the situation] . Forensische Psychiatrie und Psychotherapie. 2016;23(1):37–65.4 . Trauma exposure and posttraumatic stress disorder in adults with severe mental illness: a critical review. Clin Psychol Rev. 2011;31(6):883–99. https://doi.org/10.1016/j.cpr.2011.04.003
5 . Effects of exercise on mental and physical health parameters of persons with schizophrenia. Issues Menta Health Nurs. 2005;26(6):661–76. https://doi.org/10.1080/01612840590959551
6 . Rechtliche und psychosoziale Einflussfaktoren auf die Behandlungszeit von Patienten in forensisch-psychiatrischen Einrichtungen nach § 63 StGB. Forensische Psychiatrie und Psychotherapie. 2013;20:62–81.
7 . Mental health stigma and primary health care decisions. Psychiatry Res. 2014;218:35–8. https://doi.org/10.1016/j.psychres.2014.04.028
8 . What’s in a name? Compliance, adherence and concordance in chronic psychiatric disorders. World J Psychiatry. 2014;4(2):30–6. https://doi.org/10.5498/wjp.v4.i2.30
9 . Promoting both internal and external validity: designing the trial to match its intention. NIH Collaboratory. 2021. Available from: https://dcricollab.dcri.duke.edu/sites/NIHKR/KR/Promoting%20Internal%20and%20External%20Validity.pdf [26.04.2023].
10 . Neurofeedback as a treatment for impulsivity in a forensic psychiatric population with substance use disorder: study protocol of a randomized controlled trial combined with an N-of-1 clinical trial. JMIR Res Protoc. 2017;6(1):e13. https://doi.org/10.2196/resprot.6907
11 . Practice-based research in complementary medicine: could N-of-1 trials become the new gold standard? Healthcare (Basel). 2020;8(1):15. https://doi.org/10.3390/healthcare8010015
12 . Single-subject research in psychiatry: facts and fictions. Front Psychiatry. 2020;11:539777. https://doi.org/10.3389/fpsyt.2020.539777
13 . The family of single-case experimental designs. Harv Data Sci Rev. 2022;4(SI3):10.1162/99608f92.ff9300a8. https://doi.org/10.1162/99608f92.ff9300a8
14 . Patient profiles in Dutch forensic psychiatry based on risk factors, pathology, and offense. Int J Offender Ther Comp Criminol. 2017;61(12):1369–91. https://doi.org/10.1177/0306624X15619636
15 . Exploring heterogeneity among mentally disordered offenders – The key targeting Interventions. Aust N Z J Psychiatry. 2015;49(11):1063–4. https://doi.org/10.1177/0004867415592959
16 . Challenges and solutions for N-of-1 design studies in health psychology. Health Psychol Rev. 2019;13(2):163–78. https://doi.org/10.1080/17437199.2018.1564627
17 . N-of-1 trials in healthcare. Healthcare (Basel). 2021;9(3):330. https://doi.org/10.3390/healthcare9030330
18 . N-of-1 trials to facilitate evidence-based deprescribing: rationale and case study. Br J Clin Pharmacol. 2022;88(10):4460–73. https://doi.org/10.1111/bcp.15442
19 . Application of n-of-1 treatment trials in schizophrenia: a systematic review. Br J Psychiatry. 2018;213(1):398–403. https://doi.org/10.1192/bjp.2018.71
20 . The n-of-1 clinical trial: the ultimate strategy for individualizing medicine? Per Med. 2011;8(2):161–73. https://doi.org/10.2217/pme.11.7
21 . Dynamic modelling of n-of-1 data: powerful and flexible data analytics applied to individualised studies. Health Psychol Rev. 2017;11(3):222–34. https://doi.org/10.1080/17437199.2017.1343680
22 . Case reports: publication standards in forensic psychiatry. J Am Acad Psychiatry Law. 2014;42:297–304.
23 . Theta/SMR neurofeedback training works well for some forensic psychiatric patients, but not for others: a sham-controlled clinical case series. Int J Offender Ther Comp Criminol. 2019;63(14):2422–39. https://doi.org/10.1177/0306624X19849562
24 Power failure: why small sample size undermines the reliability of neuroscience. Nature Rev Neurosci. 2013;14(5):365–76. https://doi.org/10.1038/nrn3475
25 . User models for personalized physical activity interventions: scoping review. JMIR Mhealth Uhealth. 2019;7(1):e11098. https://doi.org/10.2196/11098
26 . The state of the art and future opportunities for using longitudinal n-of-1 methods in health behaviour research: a systematic literature overview. Health Psychol Rev. 2017;11(4):307–23. https://doi.org/10.1080/17437199.2017.1316672
