Dealing With Dependent Effect Sizes in MASEM
A Comparison of Different Approaches Using Empirical Data
Abstract
Abstract. The objective of the present study was to examine whether different methods for dealing with dependency in meta-analytic structural equation modeling (MASEM) lead to different results. Four different methods for dealing with dependent effect sizes in MASEM were applied to empirical data, including: (1) ignoring dependency; (2) aggregation; (3) elimination; and (4) a multilevel approach. Random-effects two-stage structural equation modeling was conducted for each method separately, and potential moderators were examined using subgroup analysis. Results demonstrated that the different methods of dealing with dependency in MASEM lead to different results. Thus, the decision on which approach should be used in MASEM-analysis should be carefully considered. Given that the multilevel approach is the only approach that includes all available information while explicitly modeling dependency, it is currently the theoretically preferred approach for dealing with dependency in MASEM. Future research should evaluate the multilevel approach with simulated data.
Meta-analytic structural equation modeling (MASEM) is an increasingly popular technique for summarizing findings from multivariate correlational research (Becker, 1992; Cheung & Chan, 2005; Viswesvaran & Ones, 1995). The goal of MASEM is to fit and interpret structural equation models in order to explain the (synthesized) correlations between variables. For most MASEM methods, the first step involves the estimation of a synthesized correlation matrix based on the studies’ observed correlation matrices.
An important assumption related to synthesizing effect sizes is that each effect size is independent of the other (e.g., Cheung, 2019). In MASEM, this implicates that each study may only provide one correlation coefficient for each cell (each relationship between variables) in the correlation matrix. This assumption often does not hold as dependence among effect sizes can occur for a variety of reasons (e.g., Ahn et al., 2012). For instance, multiple informants (e.g., mother- and father-report on parenting practices) or multiple measurement occasions (e.g., pre-and post-test measures) will lead to multiple correlation coefficients for the same relationship in a study. Failure to properly deal with dependency can lead to over- or underestimation of the available information, which has important implications for the statistical inferences (Cheung, 2019; Moeyaert et al., 2017; Wilson et al., 2016).
Dependency of effect sizes is a common issue in meta-analytic research (Cheung, 2019; Moeyaert et al., 2017). There have been several (methodological) reviews on the occurrence of dependent effect sizes. A recent review of 28 meta-analyses from educational research found that 57% of the studies reported dependent effect sizes (Rios et al., 2020). This is similar to the findings of Ahn and colleagues (2012), who found that of the 56 meta-analyses on educational research they reviewed, 62% reported multiple (dependent) effect sizes, and a review of 44 meta-analyses on randomized controlled trials reported that 70% of the studies included dependent effect sizes (Page et al., 2015).
Over time, several (ad hoc) solutions have arisen to overcome the issue of dependency in MASEM, which have not always been justified or well-examined for their statistical properties (Wilson et al., 2016). The objective of the present study was to examine whether applying different methods for dealing with dependent effect sizes to empirical data leads to different results when conducting MASEM analysis. Four methods were compared, including: (1) ignoring dependency; (2) aggregation; (3) elimination; and (4) a recently developed multilevel approach by Wilson and colleagues (2016), further referred to as the WPL-approach.
The next section describes the concept of MASEM in more detail. The section thereafter further elaborates on the issue of dependency and provides descriptions of the four methods for dealing with dependency, including a discussion of their (dis)advantages. The final section describes the application of the different methods for dealing with dependency to empirical data, including a comparison of the results.
Meta-Analytic Structural Equation Modeling
MASEM combines meta-analysis (MA) and structural equation modeling (SEM) and thereby overcomes some of the disadvantages of the separate techniques. SEM allows for testing more complex research questions, and MA provides sufficiently large samples to test these complex theories in SEM with sufficient statistical accuracy. There are many different ways to combine MA and SEM, but it mostly consists of two stages: (1) effect sizes from primary studies are synthesized to obtain a pooled correlation matrix; and (2) a structural equation model is fitted to the pooled correlation matrix from Stage 1 (e.g., Cheung & Chan, 2005; Jak, 2015; Viswesvaran & Ones, 1995). MASEM can be conducted using two-stage structural equation modeling (TSSEM). TSSEM was first developed for fixed-effects models (Cheung & Chan, 2005) and later extended to fit random-effects models by including study-specific random-effects (Cheung, 2014a), which is very similar to the GLS-approach by Becker (1992, 1995). Nowadays, random-effects models are preferred because fixed-effects models assume homogeneity of effect sizes which is often unrealistic (e.g., Cheung, 2014a; Yuan, 2016).
In Stage 1 of TSSEM, the correlation coefficients are weighed by their sampling variance (vi) and study-level variance (τ2). The random-effects model for the correlation vectors ri = vechs(Ri) in the ith correlation matrix Ri is
In Stage 2, the structural equation model is fitted to the pooled correlation matrix R (consisting of the estimates of ρRandom) of Stage 1 using weighted least squares (WLS) estimation. The weight matrix used in WLS-estimation is the inverse asymptotic covariance matrix of the Stage 1 estimates (Cheung & Chan, 2005). These weights ensure that correlation coefficients based on more information (on more studies and/or studies with larger sample sizes) get more weight in the estimation of the Stage 2 parameters. Since the between-studies variance is filtered out at Stage 1, it does not play a direct role at Stage 2 (Cheung, 2014a).
Different Methods for Dealing With Dependent Effect Sizes in MASEM
There are different ways of dealing with the dependency of effect sizes in MASEM. When not properly dealt with, dependent effect sizes may lead to under-or overestimation of standard errors (SEs) of the average effect sizes, which could result in inflation of Type I errors or reduced statistical power (Cheung, 2019; López-López et al., 2017). In the following section, the (potential) advantages and disadvantages of four different approaches for dealing with dependent effect sizes are described, including: (1) ignoring dependency; (2) aggregation; (3) elimination; and (4) the WPL-approach.
Ignoring Dependency of Effect Sizes
Ignoring dependency is a known-to-be incorrect strategy that is likely to bias results, to the extent that it threatens the validity of the inferences (Moeyaert et al., 2017; López-López et al., 2018). For one, studies with just one effect size will have a smaller influence on the resulting average effect size than studies with multiple effect sizes, which may result in biased estimates (Cheung, 2014b; Van den Noortgate et al., 2013). Second, simulation studies showed that the estimated SEs of the average effect sizes are underestimated, resulting in an increased likelihood of significant results (i.e., inflation of Type I errors; López-López et al., 2017; Moeyaert et al., 2017). One might incorrectly assume that the estimates are very precise and statistical inferences are more likely to be wrong (Cheung, 2019). The approach of ignoring dependency is a non-acceptable practice in meta-analysis and is merely presented in the current study to emphasize its inappropriateness and underline its (negative) implications.
Aggregation of Effect Sizes
Aggregation is a commonly used approach that involves averaging dependent effect sizes within a study before pooling effect sizes across studies (Cheung & Chan, 2004; Cheung, 2014b; Marín-Martínez & Sánchez-Meca, 1999). There are different ways to aggregate effect sizes. One option is simple aggregation, which involves calculating the arithmetic mean. Simple aggregation may be appropriate when sample sizes are (close to being) equal and when it is likely that population effect sizes are the same (Marín-Martínez & Sánchez-Meca, 1999; Moeyaert et al., 2017). However, in practice, this is often unrealistic.
Another option involves weighted aggregation. Here, effect sizes are averaged using some weighting scheme (e.g., by the inverse of the sampling variance; Marín-Martínez & Sánchez-Meca, 1999; Moeyaert et al., 2017). Weighted aggregation essentially involves cell-by-cell sub-meta-analyses. For each study that contributes multiple (dependent) effect sizes per cell, a pooled correlation matrix is estimated with a single pooled estimate within each cell (Wilson et al., 2016). An advantage of weighted aggregation – over simple aggregation – is that more weight is assigned to more precise estimates and less weight to less precise estimates.
An advantage of aggregation – both simple and weighted – is that it is a relatively intuitive and simple procedure. Disadvantages of aggregation are that it ignores within-study variability (López-López et al., 2018), and the loss of information limits the possibility to examine characteristics that can be used to evaluate effect size variability (Wilson et al., 2016). Also, a recent simulation study showed that the aggregation approach is too conservative, especially when the level of dependency is relatively low (Moeyaert et al., 2017). Their results showed that SEs are overestimated, which could lead to an inflation of Type II errors.
Thus, even though the aggregation approach is appealing and intuitive, given its disadvantages, it is not considered a state-of-the-art approach for dealing with dependent effect sizes (López-López et al., 2018; Moeyaert et al., 2017; Wilson et al., 2016).
Elimination of Effect Sizes
With elimination, one effect size per study is randomly picked or chosen based on some a priori decision rule, resulting in independent effect sizes (Cheung, 2014b; Cheung, 2019; Wilson et al., 2016). Randomly picking one effect size could be appropriate when effect sizes are assumed to be truly equivalent. However, this is a very strong assumption that rarely holds in practice. To test the assumption, one could conduct sensitivity analyses to compare results from the initial randomly picked effect sizes to another set of effect sizes (López-López et al., 2018).
Elimination based on an a priori decision rule may be appropriate when there are substantive (or validity) considerations for preferring one effect size over the other. For example, if a study includes multiple measurements of child delinquency, if reliability is higher for self-reported delinquency than for parent-reported child delinquency, the effect size pertaining to the self-report measure may be preferred. The disadvantages of elimination are similar to those of aggregation in that it affects statistical power and excludes the possibility to examine study characteristics that can be used to evaluate effect size variability.
Additionally, if the effect size is chosen based on some a priori decision rule, the fixed-effect estimates will likely show some bias towards the characteristics of the decision rule (Cheung, 2019). For example, choosing only the first measurement from longitudinal studies may bias the results to samples of younger ages. This may – depending on the specific association of interest – lead to systematically larger or smaller effects for the specific associations. Still, both with randomly picking or choosing an effect size, the resulting effect sizes will be less efficient because the information is lost. Thus, elimination may be appropriate when relevant to the research question, but it is an inappropriate method for solving dependency issues (Cheung, 2014b).
The WPL-Approach
Wilson and colleagues (2016) developed an approach to deal with dependency in MASEM, which combines three-level meta-analysis and TSSEM. A three-level random-effects meta-analysis is used to account for dependency in which participants (Level 1) are nested within effect sizes (Level 2) and effect sizes within clusters (Level 3; Van den Noortgate et al., 2013). Information from all available (dependent) effect sizes per study is incorporated in the pooled correlation matrix, and dependency is explicitly modeled.
The most important advantage of the WPL-approach is that all available information is incorporated, thus it does not reduce statistical power. Additionally, both within- and between cluster variance are taken into account, allowing for examination of heterogeneity at different levels (Cheung, 2014b; Cheung, 2019). One potential disadvantage is that the approach is somewhat more complex and not yet widely used, thus may pose more of a challenge for researchers. However, examples of studies that incorporated the WPL-approach are available (e.g., Graf-Drasch et al., 2019; Loignon & Woehr, 2018).
Empirical Application
The empirical application examined whether using the four different methods for dealing with dependency in MASEM would lead to different results. In case no (or minor) differences are found, one could conclude that the differences are mainly theoretical with no important practical implications. Then, deciding on how to deal with dependency may be based on personal preferences. However, if (large) differences are found that affect statistical inferences, the decision on which method to use for dealing with dependency in MASEM is an important one and should be carefully considered.
The next section describes the empirical application in further detail. To start, some background information is provided on the empirical data consisting of a meta-analysis on the intergenerational continuity of criminal behavior.
Background
The intergenerational continuity of criminal behavior has been well established. For instance, a meta-analysis found that children of criminal parents are at two times higher risk for criminal behavior themselves than children of non-criminal parents (Besemer et al., 2017). Explanatory mechanisms are not yet well studied, but from the literature, potential explanations can be derived. A potential mechanism through which criminal parents affect their children may be that criminal parents use less efficient (or even problematic) parenting practices. Evidence for this comes from a longitudinal study that found that mothers with a history of antisocial behavior show increased odds for problematic parenting behaviors, when compared to mothers without a history of antisocial behavior (Johnson et al., 2004). Finally, these problematic parenting practices are associated with child delinquency, with moderate associations between both behavioral control and parental support and child delinquency (Hoeve et al., 2009).
The empirical application examined the underlying mechanisms through which parental crime is associated with child delinquency. It was hypothesized that the effect of parental crime on child delinquency was fully mediated by parental support and behavioral control. The hypothesized full mediation model was compared to a partial mediation model in which a direct effect of parental crime on child delinquency was added. Given that the hypothesized model involved a path model, MASEM was necessary for the analyses.
Procedure
Sample of Studies and Selection Criteria
The selection of studies was derived from a meta-analysis on the relation between parenting practices and child delinquency (Hoeve et al., 2009), and an additional selection of studies (Silva Pinho, 2018; Van den Berg, 2018), which are part of a larger project ‘The potential mediating role of parenting on the intergenerational continuity of criminal behaviour’. The coding of studies and the manual search are still in progress; therefore, a subset of studies was included in this study.
Studies were selected using the following criteria: studies had to (1) focus on child delinquency, parental crime, and parenting behavior; (2) involve Western samples; and (3) report on bivariate associations. Child delinquency and parental crime were operationalized as all behavior prohibited by law. Broadly, parenting behaviors were defined such that all behaviors had to be directed at the child. Parental support includes all behaviou of the parent towards the child that makes the child feel comfortable and accepted. Behavioral control includes supervision, regulation, and active monitoring (excluding child disclosure and parental knowledge). Note that studies including negative support (e.g., rejection), and negative behavioral control (e.g., low supervision) were also included.
The articles were screened and coded for effect sizes on (1) parental crime and parenting behaviors, (2) parental crime and child delinquency, and (3) parenting behaviors. A more elaborate description of the search strategy, selection criteria, and the coding procedure can be found in the original meta-analysis of Hoeve and colleagues (2009) and the PRISMA flow diagram included in the Electronic Supplementary Material (ESM 1, Figure E3).
Classification and Computation of Effect Sizes
The Pearson product-moment correlation coefficient (r), further referred to as the correlation (coefficient), was used as the input effect size for the analyses because this is the only effect size suitable for conducting MASEM. Primary studies often report on a variety of effect sizes, be it due to different reporting standards across disciplines or differing nature of the variable included in the study (e.g., continuous versus categorical). The raw (non-correlation) effect sizes were converted to correlation coefficients using methods and formulae provided by Lipsey and Wilson (2001) and Borenstein and colleagues (2009). A total of 18 effect sizes were converted.
The directions of effect sizes were coded such that a positive effect indicated higher levels (e.g., more occurrences, increased severity) of child delinquency or parental crime. In case primary studies reported effect sizes that were not in line with the hypothesis of the current study, the effect sizes were reversed. For example, when support and behavioral control were negatively formulated, the effect sizes were reversed to indicate a negative association between parenting behavior and child delinquency.
Statistical Analyses
Evaluation of Publication Bias
Publication bias was evaluated using three-level funnel plots (Fernández-Castilla et al., 2020). The three-level funnel plot provides two graphs from which to evaluate publication bias: (1) a graph in which all effect sizes are plotted; and (2) a graph which plots the study-specific effects (i.e., amount of effect sizes reported per study, including their variability) against their meta-analytic standard errors.
Dealing With Dependent Effect Sizes
The procedures of the four approaches for dealing with dependency are described in the following section.
Ignoring Dependency
With ignoring dependency, no additional adjustments of the data or calculations were required. All effect sizes were included and treated as independent.
Aggregation
With simple aggregation, the arithmetic mean was calculated (i.e., the average of all effect sizes within a study). With weighted aggregation, the dependent effect sizes within a study were weighed using the inverse of the sampling variance (Cheung, 2014b). The sampling variance (vi) was estimated using
Elimination
With the elimination approach, one effect size per study was chosen based on a set of a priori decision rules. In case of multiple measurement occasions, only the effect size from the first measurement of child delinquency was included. In the case of both a boy and a girl sample, the girl sample was chosen because boys were overrepresented in the current sample of studies. In case of multiple samples or multiple informants, the sample or informant with the highest reliability was chosen. If no distinction could be made based on the described criteria, the first effect size that was reported was chosen.
WPL-Approach
With the WPL-approach, the synthesized correlation matrix was estimated using a three-level hierarchical model, thereby accounting for the statistical dependencies (Van den Noortgate et al., 2013; Wilson et al., 2016). Each unique effect size is coded with a unique effect size ID, and the effect sizes are nested within studies. Wilson and colleagues (2016) provide a nice illustration of how a dataset with such structure may be organized.
A random-effects no-intercept model was estimated using maximum likelihood (ML) estimation to synthesize correlations in each of the cells. Input required for the random-effects no-intercept model was the unique effect sizes and the variances of the effect sizes, which were calculated using simple sample size weighing (Schmidt & Hunter, 2014). Using a no-intercept model allows interpreting the regression coefficients as synthesized correlation coefficients, which are necessary for Stage 2 of the analysis. Also, the asymptotic covariance matrix of the pooled correlation matrix is available, which provides information on the precision of the pooled correlations (Wilson et al., 2016).
Random-Effect TSSEM Analysis
The hypothesized model was tested using random-effects TSSEM (Cheung, 2014a) and was overidentified with 1 df. For the WPL-approach, Stage 1 involved estimating a random-effects no-intercept model using ML estimation in which the effect sizes were nested within studies (Wilson et al., 2016). For the remaining approaches, a pooled correlation matrix was estimated in Stage 1 using ML estimation (Cheung, 2014a). The hypothesized model includes four variables, resulting in a pooled correlation matrix with six cells. Each cell contains a pooled estimate representing one of the associations of interest. In case the model did not reach convergence, the between-studies variance (τ2) was fixed at zero for the associations that seemed to lack heterogeneity.
The degree of heterogeneity was qualified using I2, which typically estimates how much of the total variance of effect sizes is due to between-study heterogeneity. Due to its three-level nature, the WPL-approach has the additional benefit of evaluating heterogeneity on both the within- and between-study level. The following rules of thumbs are used, with an I2 of .25, .50, and .75 indicating low, medium, and high levels of heterogeneity, respectively (Higgins et al., 2003).
At Stage 2, the hypothesized model was fitted on the pooled correlation matrix obtained at Stage 1 using weighted least squares (WLS) estimation (Cheung, 2014a). Model fit was evaluated using the chi-squared difference (Δχ2) test, using an α = .05 criterion for indicating a significant discrepancy between the (saturated) partial mediation model and the (more parsimonious) full mediation model. Note that with the evaluation of model fit in SEM, it is common to report alternative fit indices (e.g., RMSEA, CFI) because the Δχ2-tests are known to be very sensitive to small discrepancies when working with large sample sizes (e.g., Barret, 2007). Therefore, the RMSEAs (including their 95% CIs) are reported, using the following guidelines for adequate- to a good fit, respectively: RMSEA ≤ .08 and ≤ .05 (Hu & Bentler, 1998).
Finally, the parameter estimates of the retained model were interpreted. Criteria used to evaluate the size of the effects were based on the guidelines provided by Funder and Ozer (2019, p. 166), with an r of .05 indicating a very small effect, r of .10 a small effect, r of .20 a medium effect, r of .30 a large effect, and r of .40 a very large effect. These guidelines were originally developed for interpreting the size of correlations coefficients but are deemed appropriate for the interpretation of standardized parameter estimates.
Moderator Analysis
The (hypothesized) moderator involved the type of sample on which the effect size was based, being either a sample from the general community or a high-risk sample (e.g., a sample coming from high-crime neighborhoods, an offender sample). The moderator analyses were conducted using subgroup analysis (Jak & Cheung, 2018), which tests whether the parameter estimates are equal across groups.
The retained model was fitted to the pooled correlation matrices of each group separately. To test for subgroup differences, a model in which the parameter estimates were constrained to equality across groups was compared to a model without equality constraints. In the case of a significant Δχ2-test, the constrained model fits significantly worse than the model without the equality constraints which indicates that there are significant subgroup differences.
Software
Analyses were performed using R (version 3.5.1.; R Core Team, 2020) with the metafor package (version 2.4.0.; Viechtbauer, 2010) for Stage 1 of the WPL-approach, and the metaSEM package (version 1.2.4.; Cheung, 2015) for the MASEM and the subgroup analyses.
Results
Study Descriptives
The current sample of studies consisted of 140 manuscripts, with 114 unique samples and a total sample size of N = 163,709. Of the studies, 72.1% (k = 101) reported multiple (dependent) effect sizes. The studies contained a total of 764 effect sizes (see Table 1 for the number of effect sizes and the total sample sizes per association). There was an almost equal number of longitudinal (k = 68) and cross-sectional (k = 72) studies. Most studies were conducted with samples from North America (75%), with fewer studies conducted with European (22.9%) and Australian/New Zealand (2.1%) samples. With regard to sample type, 70.7% were general community samples, 22.9% were high-risk or delinquent samples, and 6.4% were other types of samples (e.g., combined samples of delinquents and non-delinquent). The studies included in the meta-analysis are listed in Table E1 (ESM 1), including some of their characteristics.
Comparison of Results From the Different Approaches for Dealing With Dependency
TSSEM Analysis
Stage 1 analyses were conducted to allow for the evaluation of the heterogeneity of effect sizes and to obtain the pooled correlation matrices needed for Stage 2 (see Figure 1). With the simple- and weighted aggregation approaches, running the Stage 1 model led to some convergence issues, which were likely due to the lack of heterogeneity in the associations between parental crime and support and parental crime and behavioral control. Thus, with the aggregation approaches, it seemed that the loss of information contributed to a lack of heterogeneity, leading to convergence issues, which was not the case with the other approaches.
Evaluation of I2 indicated large levels of heterogeneity (I2 = .94 to I2 = .97) for all approaches, with only small differences of .01 to .03. A benefit of the WPL-approach is the possibility to divide the overall heterogeneity into within- and between-cluster (i.e., studies) heterogeneity. Under the WPL-approach, 15% of the total variance was estimated to be due to between-study heterogeneity, and 81% due to within-study heterogeneity (with the remaining 4% due to random sampling variance). Note that under the other approaches, one may incorrectly infer that variability of effect sizes is mainly due to differences between studies, whereas the WPL-approach shows that most variability of effect sizes is due to differences within studies.
Next, the pooled correlation matrices for all approaches were compared, which are presented in Table E2 (ESM 1). Some differences were found in the size of the estimated pooled correlations. For example, with the simple aggregation approach, there is a large to the very large association between support and behavioral control (r = .37), which is small to moderate with the WPL-approach (r = .15). Also, there were differences regarding the significance of the associations. For example, the association between parental crime and behavioral control was non-significant with the WPL-approach but significant for the other approaches. Figure 1 presents the pooled correlation estimates, including their 95% confidence intervals (CIs) per approach. The width of the CIs of the ignoring dependency approach seems to be consistently smaller than the width of the CIs of the other approaches. In line with expectations, the ignoring dependency approach seems to overestimate the precision of the estimates, whereas the aggregation- and elimination approaches seem to underestimate their precision. Note that with the WPL-approach, the CIs of the associations coming from a larger number of effect sizes are also quite narrow and wider in association coming from less effect sizes. This is to be expected since a larger number of effect sizes should contribute to the precision of the estimates.
In Stage 2, both the hypothesized full mediation model and the partial mediation model were fitted to the pooled correlation matrices obtained at Stage 1. Inferences regarding model comparison were similar for all approaches. Model comparison showed significant differences between the full mediation model and the partial mediation model, indicating that the (more parsimonious) full mediation model fit significantly worse than the (saturated) partial mediation model, with Δχ2 = 47.33, Δdf = 1, p < .001, for the ignoring dependency approach, Δχ2 = 25.17, Δdf = 1, p < .001, for the simple aggregation approach, Δχ2 = 28.23, Δdf = 1, p < .001, for the weighted aggregation approach, Δχ2 = 19.56, Δdf = 1, p < .001, for the elimination approach, and, lastly, Δχ2 = 156.01, Δdf = 1, p < .001, for the WPL-approach. Each approach of dealing with dependency showed good fit of the full mediation model with RMSEAs ranging from .01 to .03, with RMSEA = .01, 95% CI [.01, .01] for the ignoring dependency approach, RMSEA = .01, 95% CI [.01, .02], for the simple aggregation approach, RMSEA = .01, 95% CI [.01, .01] for the weighted aggregation approach, RMSEA = .01, 95% CI [.01, .02], for the elimination approach, and RMSEA = .03, 95% CI [.03, .03] for the WPL-approach. Note that even though conclusions regarding model comparison are the same across approaches, the values of Δχ2-tests show seemingly large differences across approaches. Given the statistical power of the Δχ2-test, it may be that studies with smaller sample sizes would lead to different conclusions across the different approaches.
Next, the parameter estimates of the partial mediation model were compared, which are presented in Table E3 (ESM 1). Overall, the parameter estimates were quite similar in size across the approaches. Small differences in the point estimates were found, ranging from 0.003 to 0.071. For example, the effect of parental crime on child delinquency was small to moderate with the ignoring dependency approach (β = 0.16) and moderate with the weighted aggregation approach (β = 0.21). Also, differences were found regarding the statistical significance of the effects. For example, the effect of parental crime on support was non-significant with the weighted aggregation- and the WPL-approach but significant with the other approaches. This may be explained by this effect coming from the least amount of information (i.e., coming from the smallest number of effect sizes) and because of the relatively large amount of within-study heterogeneity, which is only accounted for by the WPL-approach. Thereby, the precision of the estimates may be smaller than portrayed by the other approaches.
Figure 2 presents plots of the parameter estimates, including their 95% CIs. It seems that with ignoring dependency, the CIs of the parameter estimates are consistently smaller, which is in line with expectations. The CIs of the simple- and weighted aggregation-, and elimination approaches seem consistently larger than, except for the CIs of the effects of the parenting behaviors on child delinquency (which come from the largest amount of effect sizes and largest sample sizes). Similar to the comparison of the pooled correlations, these results are somewhat in line with expectations. Again, it seems that the differences between the approaches are larger for effects coming from a smaller amount of information than for effects coming from a larger amount of information. This suggests that using the aggregation and/or elimination approach does not affect results as much if there is a sufficiently large dataset because then there will still be enough power.
The differences found in the parameter estimates across methods are also reflected in the residual variances. The residual (co)variances of the partial mediation model are presented in Table E4 (ESM 1). The variance in child delinquency explained by the partial mediation model was 6.7%, 10.1%, 9.3%, 7.7%, and 8% across the ignoring dependency-, simple aggregation-, weighted aggregation-, elimination-, and the WPL-approach, respectively. Figure 3 presents the final model estimated under the WPL-approach.
Comparison of Results From the Moderator Analyses
Moderator analyses were conducted using subgroup analysis with sample type (i.e., general community vs. high-risk) as the moderator. With the simple aggregation approach, it was impossible to conduct moderator analyses due to the lack of information on the association between parental crime and behavioral control for the general community subgroup. There were convergence issues when using the ignoring dependency approach for the high-risk subgroup. Additionally, with the weighted aggregation approach, there were convergence issues for both subgroups. In both cases, the between-studies variances (τ2) for the associations between parental crime and support and parental crime and behavioral control were fixed to zero.
Subgroup analyses showed similar results across approaches, except for the ignoring dependency approach, Δχ2 = 15.37, Δdf = 5, p = .009. With the weighted aggregation-, Δχ2 = 4.12, Δdf = 5, p = .532, elimination-, Δχ2 = 9.20, Δdf = 5, p = .101, and the WPL-approach Δχ2 = 2.10, Δdf = 5, p = .836, results showed no significant differences between the regression coefficients from the general community versus the high-risk subgroup.
Evaluation of Publication Bias With the WPL-Approach
Evaluation of publication bias was conducted using three-level funnel plots, which are presented in Appendix C (ESM 1). Figure E1 (ESM 1) shows the graph in which all effect sizes are plotted. Visual inspection of the effect size plot shows one effect size in the lower-right part of the graph, whereas there is no result with similar precision at the lower-left part of the graph, which may be a sign of publication bias. Figure E2 (ESM 1) shows the plot in which the study-specific effects are plotted against their meta-analytic standard errors. The study-funnel plot shows some signs of asymmetry, especially at the bottom of the graph. Concluding from both graphs, there may be some signs of publication bias, which should be taken into account when interpreting the meta-analytic results.
Discussion
The aim of this study was to examine whether applying different methods for dealing with dependency to empirical data leads to different results when conducting MASEM analysis. The empirical application demonstrated that the different approaches for dealing with dependency in MASEM are not only theoretically different but also lead to different results with important practical implications. An overview of the (dis)advantages of the four approaches is presented in Table 2.
The most important differences lie in the SEs of the parameter estimates. The SEs of the parameter estimates with the ignoring dependency approach seemed consistently smaller, and the SEs of the aggregation- and elimination approaches seemed consistently larger. The SEs of the WPL-approach did not seem consistently higher or lower across the different associations but, as one would expect, seemed to depend on both the amount of information available and the level of within- and between-study variability. Under- or overestimation of the SEs has important implications for statistical inferences. For instance, in the present study, with the WPL-approach, the effect of parental crime on parent support is not significant and therefore may be removed from the model, whereas this effect was significant with the other approaches.
Results from the subgroup analysis were also affected by the use of the different approaches for dealing with dependency. For one, with the simple aggregation approach, it was not possible to conduct subgroup analyses due to the lack of information available on the variable of interest. Second, with the ignoring dependency approach, significant differences were found between the subgroups. Given that this was the only approach showing significant differences, this may have been the result of overestimated precision of the estimates. Thus, using different methods for dealing with dependency in MASEM also has important practical implications with regard to the evaluation of (potential) moderators.
These findings are in line with previous research. By ignoring dependency, the available information was overestimated, thereby increasing the likelihood of Type I errors (Cheung, 2019; López-López et al., 2017; Moeyaert et al., 2017). Hereby, one may incorrectly infer that the estimates are very precise and important subgroup differences. Therefore, ignoring dependency is deemed non-acceptable in meta-analytic research.
With aggregation- and elimination of effect sizes, a lot of information was lost by reducing the available information to one effect size per study. Even though the parameter estimates seemed to show no specific bias, the standard errors were consistently larger in comparison to the other approaches. Overestimation of the standard errors is problematic because it affects statistical power, thereby increasing the likelihood of Type II errors (Cheung, 2014b; Moeyaert et al., 2017). Given that most parameter estimates were significant in the current study, the loss of information did not seem to affect statistical inferences. However, this study had a relatively large dataset to work with. It may be the case that in meta-analyses with a smaller number of studies, the lack of statistical power does affect results and fails to identify a potential effect. Thus, using aggregation- and elimination of effect sizes may not be problematic if there is sufficient number of studies and the level of dependency is relatively low (Moeyaert et al., 2017). Still, aggregation- and elimination of effect sizes, even though simple and intuitive, is deemed suboptimal for dealing with dependency in MASEM because these are less efficient approaches.
The WPL-approach showed no consistently higher or lower SEs across associations. One explanation for this may be the amount of within-study variability of effect sizes, which is not accounted for by the other approaches. The ability to account for within-study variability is another important benefit of the WPL-approach, as it gives a more accurate representation of the data. Accounting for both within- and between-study variability of effect sizes can lead to different inferences than when one can only examine between-study variability. In this study, with the ignoring dependency-, aggregation-, and elimination approaches, one would infer that there is large significant variability in effect sizes due to between-studies differences. However, the WPL-approach paints a very different picture and shows that most of the heterogeneity is due to within-study differences, with a moderate amount due to between-study differences.
The WPL-approach is the only approach where all available information is included while also explicitly modeling dependency by nesting the effect sizes within studies (Van den Noortgate et al., 2013; Wilson et al., 2016). Because all the available information is used, statistical power is not affected. By nesting effect sizes within studies, the dependency is properly accounted for, and therefore the precision of the estimates is not overestimated. Limitations of the WPL-approach are that is has not been evaluated in a simulation study and that it is not yet frequently used in practice. However, the paper by Wilson and colleagues (2016) describes the procedure extensively and provides the syntax in the supplementary materials. Also, some examples are available (e.g., Graf-Drasch et al., 2019; Loignon & Woehr, 2018). Based on the findings of the current study, the WPL-approach is the theoretically preferred approach for dealing with dependency in MASEM analysis.
Strengths, Limitations, and Future Directions
A strength of the current study is that – to the author’s knowledge – this study is the first to compare frequently used (ad hoc) methods for dealing with dependency in MASEM to the relatively new WPL-approach using empirical data. Additionally, this study aspires to facilitate the reproducibility of the analyses. Given that the WPL-approach may be viewed as somewhat more complex, the authors have provided the data (incl. the code book; Stolwijk et al., 2021a) and the R-script (Stolwijk et al., 2021b) for the WPL-approach in PsychArchives. Combined with the extensive description of the procedure by Wilson and colleagues (2016), this should aid interested researchers in conducting MASEM-analysis using the WPL-approach to handle dependency. Lastly, this study gives a comprehensive overview of commonly used approaches for dealing with dependency and shows its pitfalls. Providing an overview of the (dis)advantages hopefully aids researchers to decide on an appropriate method.
The current study is limited in that it offers a comparison based solely on empirical data, and inferences can stretch not much further than to the specifics of the current dataset. However, from this practical application, there is a basis from which to conduct a simulation study in order to examine the robustness of the WPL-approach under ideal and non-ideal conditions (e.g., Hallgren, 2013). For instance, it would be interesting to examine the effects of differences in the amount of overall heterogeneity that can be attributed to within-versus between-studies differences. Additionally, the level of dependency may be altered to evaluate the impact on the performance of the WPL-approach, relative to other approaches. Also, the minimum number of studies necessary to conduct the WPL-approach should be examined.
Conclusion
In summary, dependency is a non-avoidable issue in meta-analytic research. This study demonstrated that using different approaches for dealing with dependency in MASEM leads to different results, which can have important practical implications. Thus, the decision on which approach should be used in MASEM-analysis should be one that is carefully considered. Given that the WPL-approach is the only approach that includes all available information while explicitly modeling dependency, it is currently the theoretically preferred approach for dealing with dependency in MASEM. Future research should evaluate the multilevel approach with simulated data.
Electronic Supplementary Material
The electronic supplementary material is available with the online version of the article at https://doi.org/10.1027/2151-2604/a000485
This research was presented at the Research Synthesis & Big Data (virtual) Conference in May 2021. We thank Tessa van den Berg and Ana Silva Pinho for their efforts in the data collection.
References *References marked with an asterisk are included in the meta-analysis.
* (2010). Parental incarceration and multiple risk experiences: Effects on family dynamics and children’s delinquency. Journal of Youth and Adolescence, 39, 1471–1484. https://doi.org/10.1007/s10964-009-9458-0
* (2001). Self-control as a mediator for the effects of parental bonding, prosocial behavioural training and psychological autonomy on adolescent delinquency. Dissertation Abstracts International, 62, 4(A).
(2012). A review of meta-analyses in education: Methodological strengths and weaknesses. Review of Educational Research, 82(4), 436–476. https://doi.org/10.3102/0034654312458162
* (2005). The two faces of adolescents’ success with peers: Adolescent popularity, social adaptation, and deviant behaviour. Journal of Child Development, 76(3), 747–760. https://doi.org/10.1111/j.1467-8624.2005.00875.x
* (2000). Bullies and delinquents: Personal characteristics and parental styles. Journal of Community and Applied Social Psychology, 10(1), 17–31. https://doi.org/b39pnk
* (2006). Interpersonal violence in adolescence: Ecological correlates of self-reported perpetration. Journal of Interpersonal Violence, 21(10), 1314–1332. https://doi.org/10.1177/0886260506291657
* (2004). Self-reported juvenile delinquency in England and Wales, the Netherlands and Spain. European Institute for Crime Prevention and Control.
* (2005). Correlates of aggressive and delinquent conduct problems in adolescence. Aggressive Behaviour, 31, 24–39. https://doi.org/10.1002/ab.20033
(2007). Structural equation modelling: Adjudging model fit. Personality and Individual Differences, 42, 815–824. https://doi.org/10.1016/j.paid.2006.09.018
* (2006). Parental support, behavioural control, and psychological control among African American youth: The relationships to academic grades, delinquency, and depression. Journal of Family Issues, 27(10), 1335–1355. https://doi.org/10.1177/0192513X06289649
(1992). Using results from replicated studies to estimate linear models. Journal of Educational Statistics, 17, 341–362. https://doi.org/10.3102/10769986017004341
(1995). Corrections to “Using results from replicated studies to estimate linear models”. Journal of Educational Statistics, 20, 100–102. https://doi.org/10.3102/10769986020001100
(2017). A systematic review and meta-analysis of the intergenerational transmission of criminal behaviour. Aggression and Violent Behaviour, 37, 161–178. https://doi.org/10.1016/j.avb.2017.10.004
(2009). Introduction to meta-analysis. Wiley. https://doi.org/10.1002/9780470743386
* (2007). Parenting behaviours, association with deviant peers, and delinquency in African American adolescents: A mediated-moderation model. Journal of Youth and Adolescence, 36(4), 517–527. https://doi.org/10.1007/s10964-006-9117-7
* (2011). Autonomy-supportive parenting and adolescent delinquency [Unpublished doctoral dissertation]. North Carolina State University.
* (2001). Reactive and proactive aggression: Predictions to physical violence in different contexts and moderating effects of parental monitoring and caregiving behaviour. Journal of Abnormal Child Psychology, 29(4), 293–304. https://doi.org/10.1023/A:1010305828208
* (1995). The impact of parental controls on delinquency. Journal of Criminal Justice, 23, 111–126. https://doi.org/10.1016/0047-2352(95)00009-F
* (2011). The roles of mothers’ neighborhood perceptions and specific monitoring strategies in youths’ problem behaviour. Journal of Youth and Adolescence, 40, 347–360. https://doi.org/10.1007/s10964-010-9538-1
* (2006). Brief report: An examination of the relationships between parental monitoring, self-esteem and delinquency among Mexican American male adolescents. Journal of Adolescence, 29(3), 459–464. https://doi.org/10.1016/j.adolescence.2005.07.005
* (1987). Self-reported delinquency and home life: Evidence from a sample of British girls. Journal of Youth and Adolescence, 16(2), 167–177. https://doi.org/10.1007/BF02138918
* (2003). Continuity of parenting practices across generations in an at-risk sample: A prospective comparison of direct and mediated associations. Journal of Abnormal Child Psychology, 31(2), 127–142. https://doi.org/10.1023/A:1022518123387
* (1987). Family relationships and delinquency. Criminology, 25(2), 295–321.
* (2003). Examining intergenerational violence: Violent role modeling or weak parental controls? Violence and Victims, 18(2), 143–162. https://doi.org/10.1891/vivi.2003.18.2.143
(2014a). Fixed- and random-effect meta-analytic structural equation modelling: Examples and analyses in R. Behaviour Research methods, 46, 29–40. https://doi.org/10.3758/s13428-013-0361-y
(2014b). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modelling approach. Psychological Methods, 19(2), 211–229. https://doi.org/10.1037/a0032968
(2015). metaSEM: An R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5, Article 1521. https://doi.org/10.3389/fpsyg.2014.01521
(2019). A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychology Review, 29, 387–396. https://doi.org/10.1007/s11065-019-09415-6
(2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10(1), 40–64. https://doi.org/10.1037/1082-989X.10.1.40
(2004). Dependent effect sizes in meta-analysis: Incorporating the degree of interdependence. Journal of Applied Psychology, 89(5), 780–791. https://doi.org/10.1037/0021-9010.89.5.780
* (2015). DRD4 and DRD2 genes, parenting, and adolescent delinquency: Longitudinal evidence for a gene by environment interaction. Journal of Abnormal Psychology, 124, 791–802. https://doi.org/10.1037/abn0000091
* (2006). Relations between neighborhood factors, parenting behaviours, peer deviance, and delinquency among serious juvenile offenders. Developmental Psychology, 42(2), 319–331. https://doi.org/10.1037/0012-1649.42.2.319
* (2015). The relationship between parental warmth, the effects of childhood witnessed violence and pre-adolescent delinquency [Unpublished doctoral dissertation]. Cleveland State University.
* (2009). Parental competencies of juvenile probationers and adherence to court sanctions and recidivism rates. Virginia Commonwealth University.
* (1996). Family experience in preadolescence and the development of male delinquency. Journal of Marriage and the Family, 58(2), 491–501. https://doi.org/10.2307/353512
* (2010). Peer and neighborhood risk contexts, and adolescents’ delinquent behaviours: The protective potential of family and neighborhood connectedness [Unpublished doctoral dissertation]. Oklahoma State University.
* (2002). How parents find out about their teenagers’ activities: Validating an observational measure of monitoring as dyadic process [Unpublished doctoral dissertation]. Auburn University.
* (2006). Mediating mechanisms: Understanding the link between parenting and adolescent deviance [Unpublished doctoral dissertation]. Auburn University.
* (1976, April). Parent-child relationships, parental role models and reference others–their joint impact on juvenile delinquency. Paper presented at the Annual Meeting of the Midwest Sociological Society, St. Louis, MO.
* (2004). Opvoeding, delinquente vrienden en delinquent gedrag van jongeren
[Parenting, delinquent friends and delinquent behaviour of adolescents] . Pedagogiek, 24(3), 262–278.* (2016). Adolescent-parent attachment and externalizing behaviour: The mediating role of individual and social factors. Journal of Abnormal Child Psychology, 44, 283–294. https://doi.org/10.1007/s10802-015-9999-5
* (2012). Parent and peer pathways to adolescent delinquency: Variations by ethnicity and neighborhood context. Journal of Youth and Adolescence, 41, 1078–1094. https://doi.org/10.1007/s10964-012-9754-y
* (2004). Like father, like son: Toward a developmental model for the transmission of male deviance across generations. European Journal of Developmental Psychology, 1, 105–126. https://doi.org/10.1080/17405620444000094
* (1991). Family, school, and behavioural antecedents to early adolescent involvement with antisocial peers. Developmental Psychology, 27(1), 172–180. https://doi.org/10.1037/0012-1649.27.1.172
* (2008). Testing an idealized dynamic cascade model of the development of serious violence in adolescence. Child Development, 79, 1907–1927. https://doi.org/10.1111/j.1467-8624.2008.01233.x
* (2009). Parental monitoring, personality, and delinquency: Further support for a reconceptualization of monitoring. Journal of Research in Personality, 43, 49–59. https://doi.org/10.1016/j.jrp.2008.10.006
* (2008). Psychopathic features moderate the relationship between harsh and inconsistent parental discipline and adolescent antisocial behaviour. Journal of Clinical Child & Adolescent Psychology, 37, 472–476. https://doi.org/10.1080/15374410801955938
* (2005). The influence of violent behaviour and victimization at school on psychological distress: The role of parents and teachers. Adolescence, 40(157), 183–196.
* (2012). The effect of corporal punishment and verbal abuse on delinquency: Mediating mechanisms. Journal of Youth and Adolescence, 41, 1095–1110. https://doi.org/10.1007/s10964-012-9755-x
* (2001). The concentration of offenders in families, and family criminality in the prediction of boys’ delinquency. Journal of Adolescence, 24, 579–596. https://doi.org/10.1006/jado.2001.0424
* (2002). Are within-individual causes of delinquency the same as between-individual causes? Criminal Behaviour and Mental Health, 12(1), 53–68. https://doi.org/10.1002/cbm.486
* (2015). Intergenerational similarities in risk factors for offending. Journal of Developmental and Life-Course Criminology, 1, 48–62. https://doi.org/10.1007/s40865-015-0005-2
(2020). Visual representation of meta-analyses of multiple outcomes: Extensions to forest plots, funnel plots, and caterpillar plots. Methodology, 16(4), 299–315. https://doi.org/10.5964/meth.4013
* (2005). Parenting behaviour and adolescent behavioural and emotional problems: The role of self-control. International Journal of Behavioural Development, 29(1), 58–69. https://doi.org/10.1080/01650250444000333
* (1999). Who are they with and what are they doing? Delinquent behaviour, substance use, and early adolescents’ after-school time. American Journal of Orthopsychiatry, 69(2), 247–253. https://doi.org/10.1037/h0080426
* (2004). Parental influences on adolescent problem behaviour: Revisiting Stattin and Kerr. Child Development, 75(3), 781–796. https://doi.org/10.1111/j.1467-8624.2004.00706.x
(2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168. https://doi.org/10.1177/2515245919847202
* (1997). Deviance among the children of heroin addicts in treatment: Impact of parents and peers. Deviant Behaviour, 18(2), 143–159. https://doi.org/10.1080/01639625.1997.9968050
* (2012). It’s a two-way street: The bidirectional relationship between parenting and delinquency. Journal of Youth and Adolescence, 41, 121–145. https://doi.org/10.1007/s10964-011-9656-4
* (1996). An empirical assessment of the core elements of Gottfredson and Hirschi’s general theory of crime. Dissertation Abstracts International, 56, 4155-A.
* (2011). The relation between abuse and violent delinquency: The conversion of shame to blame in juvenile offenders. Child Abuse & Neglect, 35, 459–467. https://doi.org/10.1016/j.chiabu.2011.02.007
(2019, June). Clarifying the structure of collective intelligence in teams: A meta-analysis. Paper presented at the Proceedings of the Collective Intelligence Conference, Pittsburgh, PA.
* (1990). Juvenile delinquency in the Black community. Youth and Society, 22(1), 67–84. https://doi.org/10.1177/0044118X90022001005
* (2000). Parenting practices as predictors of substance use, delinquency, and aggression among urban minority youth: Moderating effects of family structure and gender. Psychology of Addictive Behaviours, 14(2), 174–184. https://doi.org/10.1037/0893-164X.14.2.174
* (2016). Associations between mother–child relationship quality and adolescent adjustment: Using a genetically controlled design to determine the direction and magnitude of effects. International Journal of Behavioural Development, 40, 196–204. https://doi.org/10.1177/0165025415620059
* (2000). Young offenders’ experiences of child protection services. Journal of Youth and Adolescence, 29, 355–371. https://doi.org/10.1023/A:1005151809736
* (2008). The continued importance of quality parent–adolescent relationships during late adolescence. Journal of Research on Adolescence, 18, 187–200. https://doi.org/10.1111/j.1532-7795.2008.00556.x
* (2013). Delinquent-oriented attitudes mediate the relation between parental inconsistent discipline and early adolescent behaviour. Journal of Family Psychology, 27, 293–302. https://doi.org/10.1037/a0031962
(2013). Conducting simulation studies in the R programming environment. Tutorials in Quantitative Methods for Psychology, 9(2), 43–60. https://doi.org/10.20982/tqmp.09.2.p043
* (2017). Bidirectional relationships between parenting processes and deviance in a sample of inner-city African American youth. Journal of Research on Adolescence, 27, 201–213. https://doi.org/10.1111/jora.12267
* (2001). Parenting, self-control, and delinquency: A test of self-control theory. Criminology, 39, 707–736. https://doi.org/10.1111/j.1745-9125.2001.tb00938.x
* (2003). Family strain, gender, and delinquency. Sociological Perspectives, 46(1), 107–135. https://doi.org/10.1525/sop.2003.46.1.107
* (2003). Contexts of risk? Explaining the link between girls’ pubertal development and their delinquency involvement. Social Forces, 82(1), 355–397. https://doi.org/10.1353/sof.2003.0093
* (1994). Family of origin, personality, and self-reported delinquency. Journal of Adolescence, 17, 445–459. https://doi.org/10.1006/jado.1994.1038
* (2004). The impact of adolescent and parental characteristics on adolescent levels of delinquency and depression. Personality and Individual Differences, 36(1), 173–185. https://doi.org/10.1016/S0191-8869(03)00077-1
* (2013). The longitudinal relationship between peer violence and popularity and delinquency in adolescent boys: Examining effects by family functioning. Journal of Youth and Adolescence, 42, 1651–1660. https://doi.org/10.1007/s10964-012-9859-3
* (2014). Delinquency in adolescent girls: Using a confluence approach to understand the influences of parents and peers. Criminal Justice and Behaviour, 41, 1327–1337. https://doi.org/10.1177/0093854814538624
* (1997). The influence of family regulation, connection, and psychological autonomy on six measures of adolescent functioning. Journal of Adolescent Research, 12(1), 34–67.
* (2000). Developmental risk factors for youth violence. Journal of Adolescent Health, 26(3), 176–186. https://doi.org/10.1177/0743554897121004
* (2015). Unique and combined contributions of callous-unemotional traits and parental incarceration on juvenile delinquency in an at-risk sample [Unpublished doctoral dissertation]. University of Southern Mississippi.
(2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327, 557–560. https://doi.org/10.1136/bmj.327.7414.557
* (1988). Gender, familial control, and delinquency. Criminology: An Interdisciplinary Journal, 26(1), 127–147. https://doi.org/10.1111/j.1745-9125.1988.tb00835.x
(2009). The relationship between parenting and delinquency: A meta-analysis. Journal of Abnormal Child Psychology, 37, 749–775. https://doi.org/10.1007/s10802-009-9310-8
* (2011). Maternal and paternal parenting styles: Unique and combined links to adolescent and early adult delinquency. Journal of Adolescence, 34, 813–827. https://doi.org/10.1016/j.adolescence.2011.02.004
* (2007). Long term effects of parenting and family characteristics on delinquency of male young adults. European Journal of Criminology, 4(2), 116–194. https://doi.org/10.1177/1477370807074854
(1998). Fit indices in covariance structure modelling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. https://doi.org/10.1037/1082-989X.3.4.424
* (2012). The roles of social bonds, personality, and perceived costs: An empirical investigation into Hirschi’s “New” Control Theory. International Journal of Offender Therapy and Comparative Criminology, 56, 1182–1200. https://doi.org/10.1177/0306624X11422998
(2015). Meta-analytic structural equation modeling. SpringerBriefs in research synthesis and meta-analysis. Springer International Publishing.
(2018). Testing moderator hypotheses in meta-analytic structural equation modeling using subgroup-analysis. Behavioural Research Methods, 50, 1359–1373. https://doi.org/10.3758/s13428-018-1046-3
* (2016). How is parenting related to adolescent delinquency? A between- and within-person analysis of the mediating role of self-control, delinquent attitudes, peer delinquency, and time spent in criminogenic settings. European Journal of Criminology, 13, 169–194. https://doi.org/10.1177/1477370815608881
(2004). Anti-social parental behaviour, problematic parenting and aggressive offspring behaviour during adulthood: A 25-year longitudinal investigation. British Journal of Criminology, 44, 915–930. https://doi.org/10.1093/bjc/azh041
* (1987). Mother’s versus father’s role in causing delinquency. Adolescence, 22(86), 305–315.
* (2000). Maternal and paternal parenting during adolescence: Forecasting early adult psychosocial adjustment. Adolescence, 35(139), 513–530.
* (2015). Personality traits, social intelligence, social support and juvenile delinquency [Conference session]. International Conference on Advances in Social Science, Economics and Management Study – SEM, Birmingham, United Kingdom
* (2004). Integrating models of risk and protection for the prevention of adolescent delinquency [Unpublished doctoral dissertation]. Michigan State University.
(2012). Forbidden friends as forbidden fruit: Parental supervision of friendships, contact with deviant peers, and adolescent delinquency. Child Development, 83, 651–666. https://doi.org/10.1111/j.1467-8624.2011.01701.x
* (2010). Reciprocal effects between parental solicitation, parental control, adolescent disclosure, and adolescent delinquency. Journal of Research on Adolescence, 20, 88–113. https://doi.org/10.1111/j.1532-7795.2009.00631.x
* (1999). To know you is to trust you: Parents’ trust is rooted in child disclosure of information. Journal of Adolescence, 22(6), 737–752. https://doi.org/10.1006/jado.1999.0266
* (2011). Parental incarceration during childhood, family context, and youth problem behaviour across adolescence. Journal of Offender Rehabilitation, 50, 18–36. https://doi.org/10.1080/10509674.2011.536720
* (1980). Social control and delinquent behaviour: An examination of the elements of the social bonds. Sociological Quarterly, 21, 529–543. https://doi.org/10.1111/j.1533-8525.1980.tb00634.x
* (1992). The measurement of family process variables: An examination of adolescent and parent perception of family life and delinquent behaviour. Journal of Quantitative Criminology, 8, 287–315. https://doi.org/10.1007/BF01064550
* (2002). Adolescent behavioural autonomy as a moderator between parenting practices and adolescent delinquency. Dissertation Abstracts International, 63, 5(B).
* (2014). Linking family economic pressure and supportive parenting to adolescent health behaviours: Two developmental pathways leading to health promoting and health risk behaviours. Journal of Youth and Adolescence, 43, 1176–1190. https://doi.org/10.1007/s10964-013-0060-0
* (2013). When love hurts: Assessing the intersectionality of ethnicity, socio-economic status, parental connectedness, child abuse, and gender attitudes in juvenile violent delinquency. Child Abuse and Neglect, 37, 1034–1049. https://doi.org/10.1016/j.chiabu.2013.07.001
* (2014). Ethnic differences in the effect of perceived parenting on juvenile violent delinquency of Dutch and Moroccan-Dutch boys. Journal of Child and Family Studies, 23, 333–346. https://doi.org/10.1007/s10826-013-9725-2
* (2016). Maternal sensitivity and effortful control in early childhood as predictors of adolescents’ adjustment: The mediating roles of peer group affiliation and social behaviours. Developmental Psychology, 52, 922–932. https://doi.org/10.1037/dev0000118
* (1990). Parental management: Mediator of the effect of socioeconomic status on early delinquency. Criminology, 28, 301–323.
(2001). Practical Meta-analysis. Sage.
(2018). Social class in the organizational sciences: A conceptual integration and meta-analytic review. Journal of Management, 44(1), 61–88. https://doi.org/10.1177/0149206317728106
(2017). Assessing meta-regression methods for examining moderator relationships with dependent effect sizes: A Monte Carlo simulation. Research Synthesis Methods, 8, 435–450. https://doi.org/10.1002/jrsm.1245
(2018). Dealing with dependent effect size multiplicity in systematic reviews and meta-analyses. Research Synthesis Methods, 9, 336–351. https://doi.org/10.1002/jrsm.1310
* (2007). Examining resource and protective factors in the adjustment of Latino youth in low income families: What role does maternal acculturation play? Journal of Youth and Adolescence, 36(4), 489–501. https://doi.org/10.1007/s10964-006-9124-8
* (2000). Parenting and friend affiliation in adolescent development: A cross-cultural comparison [Unpublished doctoral dissertation]. Wayne State University.
* (1994). Parental neglect and overprotection as risk factors in delinquency. Australian Journal of Psychology, 46(2), 107–111. https://doi.org/10.1080/00049539408259481
* (2006). Adolescent personality, problem behaviour and the quality of the parent-adolescent relationship. European Journal of Personality, 20, 237–254. https://doi.org/10.1002/per.574
* (2015). Person × environment interactions on adolescent delinquency: Sensation seeking, peer deviance and parental monitoring. Personality and Individual Differences, 76, 129–134. https://doi.org/10.1016/j.paid.2014.11.055
(1999). Averaging dependent effect sizes in meta-analysis: A cautionary note about procedures. The Spanish Journal of Psychology, 2(1), 32–38. https://doi.org/10.1017/S1138741600005436
* (1996). Neither too sweet nor too sour: Problem peers, maternal control, and problem behaviour in African American adolescents. Child Development, 67(5), 2115–2130. https://doi.org/10.2307/1131613
* (2013). Children’s delinquency after paternal incarceration [Unpublished doctoral dissertation]. Texas A&M University.
* (1991). Family relationships, juvenile delinquency and adult criminality. Criminology, 29, 397–417.
(2017). Methods for dealing with multiple outcomes in meta-analysis: A comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology, 20(6), 559–572. https://doi.org/10.1080/13645579.2016.1252189
* (2010). Trajectories of desistance and continuity in antisocial behaviour following court adjudication among serious adolescent offenders. Developmental Psychopathology, 22, 453–475. https://doi.org/10.1017/S0954579410000179
* (2007). Crime in adult offspring of prisoners: A cross-national comparison of two longitudinal samples. Criminal Justice and Behaviour, 34, 133–149. https://doi.org/10.1177/0093854806289549
* (2009). Frequency and seriousness of parental offending and their impact on juvenile offending. Journal of Adolescence, 32, 893–908. https://doi.org/10.1016/j.socscimed.2008.10.005
* (2001). Conditional associations between parental behaviour and adolescent problems: A search for personality-environment interactions. Journal of Research in Personality, 35(1), 1–26. https://doi.org/10.1006/jrpe.2000.2295
(1976).
Asymptotic distribution of functions of a correlation matrix . In S. IkedaEd., Essays in probability and statistics (pp. 235–251). Shinko Tsusho Co., Ltd.(2015). Methods to select results to include in meta-analyses deserve more consideration in systematic reviews. Journal of Clinical Epidemiology, 68, 1282–1291. https://doi.org/10.1016/j.jclinepi.2015.02.009
* (2010). Gender differences in predictors of assaultive behaviour in late adolescence. Youth Violence and Juvenile Justice, 8, 314–331. https://doi.org/10.1177/1541204009361173
* (2003). Effects of parenting, father absence, and affiliation with delinquent peers on delinquent behaviour among African-American male adolescents. Adolescence, 38(149), 15–34.
* (2016). A behavioural approach to adolescent cannabis use: Accounting for nondeliberative, developmental, and temperamental factors. Journal of Substance Use, 21, 506–514. https://doi.org/10.3109/14659891.2015.1076076
* (1985). Contributions of families and peers to delinquency. Criminology, 23(1), 63–79. https://doi.org/10.1111/j.1745-9125.1985.tb00326.x
* (2002). “Don’t forget the women”: A multi-level analysis of individual and contextual effects on girls’ and boys’ delinquency [Unpublished doctoral dissertation]. University of Nebraska.
* (2001). Antecedents and behaviour-problem outcomes of parental monitoring and psychological control in early adolescence. Child Development, 72(2), 583–598. https://doi.org/10.1111/1467-8624.00298
* (2004). Parent and child personality characteristics as predictors of negative discipline and externalizing problem behaviour in children. European Journal of Personality, 18, 73–102. https://doi.org/10.1002/per.501
. (2020). R: A language and environment for statistical computing. https://www.R-project.org/
* (1994). Parental attachments and delinquency. Criminology, 32(4), 495–515. https://doi.org/10.1111/j.1745-9125.1994.tb01163.x
* (2017). Callous–unemotional traits predict self-reported offending in adolescent boys: The mediating role of delinquent peers and the moderating role of parenting practices. Developmental Psychology, 53, 319–328. https://doi.org/10.1037/dev0000210
(2020). A topical and methodological systematic review of meta-analyses published in the educational measurement literature. Educational Measurement: Issues and Practice, 39(1), 71–81. https://doi.org/10.1111/emip.12282
* (2002). Family rituals and deviant behaviour [Unpublished doctoral dissertation]. University of North Texas.
* (1999). Multiple mediators of the effects of acculturation status on delinquency for Mexican American adolescents. American Journal of Community Psychology, 27(2), 189–210. https://doi.org/10.1023/A:1022883601126
* (1994). Urban poverty and the family context of delinquency: A new look at structure and process in a classic study. Child Development, 65(2), 523–540. https://doi.org/10.2307/1131400
(2014). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Sage.
* (1999). Factors predicting continued violence into young adulthood. Journal of Adolescence, 22(1), 3–20. https://doi.org/10.1006/jado.1998.0197
(2018). The effects of child personality traits on parenting and child delinquency: A meta-analytic approach [Unpublished master’s thesis]. University of Amsterdam.
* (2007). Identifying the psychological factors that mediate the association between parenting practices and delinquency. Criminology: An Interdisciplinary Journal, 45, 481–517. https://doi.org/10.1111/j.1745-9125.2007.00086.x
* (1989). The nature of the association between parental rejection and delinquent behaviour. Journal of Youth and Adolescence, 18(3), 297–310. https://doi.org/10.1007/BF02139043
* (2000). Associations between parenting, acculturation, and adolescent functioning among Chinese families in North America. International, 61(2), 784(A).
* (2009). Does parenting mediate the effects of exposure to violence on violent behaviour? An ecological–transactional model of community violence. Journal of Adolescence, 32, 1321–1341. https://doi.org/10.1016/j.adolescence.2008.12.003
* (2002). Beyond the interactional relationship between delinquency and parenting practices: The contribution of legal sanctions. Journal of Research in Crime and Delinquency, 39(1), 36–59. https://doi.org/10.1177/002242780203900102
(2021). Dealing with dependent effect sizes in MASEM: A comparison of different approaches using empirical data [Data set and code book]. PsychArchives. http://dx.doi.org/10.23668/psycharchives.5130
(2021). Dealing with dependent effect sizes in MASEM: A comparison of different approaches using empirical data [Code]. PsychArchives. http://dx.doi.org/10.23668/psycharchives.5129
* (2013). Moral emotions and offending: Do feelings of anticipated shame and guilt mediate the effect of socialization on offending? European Journal of Criminology, 10, 22–39. https://doi.org/10.1177/1477370812454393
* (2014). Adolescents’ information management: Comparing ideas about why adolescents disclose to or keep secrets from their parents. Journal of Youth and Adolescence, 43, 803–813. https://doi.org/10.1007/s10964-013-0008-4
* (2012). Personality, parenting and deviance among Spanish adolescents. Anales de Psicologia, 28, 654–664. https://doi.org/10.6018/analesps.28.3.155951
* (2006). Why is “bad” parenting criminogenic?: Implications from rival theories. Youth Violence and Juvenile Justice, 4(1), 3–33. https://doi.org/10.1177/1541204005282310
(2018). A comparison of meta-analytic structural equation modeling and univariate meta-analysis: An application in forensic child and youth care sciences [Unpublished master’s thesis]. University of Amsterdam.
(2013). Three-level meta-analysis of dependent effect sizes. Behavioural Research, 45, 576–594. https://doi.org/10.3758/s13428-012-0261-6
* (2012). The moderating role of empathy in the association between parental support and adolescent aggressive and delinquent behaviour. Aggressive Behaviour, 38, 368–377. https://doi.org/10.1002/ab.21435
* (1988). The impact of family structure and quality on delinquency: A comparative assessment of structural and functional factors. Criminology, 26, 235–261. https://doi.org/10.1111/j.1745-9125.1988.tb00840.x
* (2016). Why is young maternal age at first childbirth a risk factor for persistent delinquency in their male offspring? Examining the role of family and parenting factors. Criminal Behaviour and Mental Health, 26, 322–335. https://doi.org/10.1002/cbm.1959
* (1997). Early adolescent delinquent behaviours: Associations with family and school domains. Journal of Early Adolescence, 17(3), 271–293. https://doi.org/10.1177/0272431697017003002
* (2016). An empirical test of self-control theory in Roma adolescents. Journal of Criminal Justice, 44, 66–76. https://doi.org/10.1016/j.jcrimjus.2015.12.004
* (2008). Rural and non-rural African American youth: Does context matter in the etiology of problem behaviours? Journal of Youth and Adolescence, 37, 798–811. https://doi.org/10.1007/s10964-007-9239-6
* (1993). Acculturation and delinquent behaviour among Cuban American adolescents: Toward an empirical model. American Journal of Community Psychology, 21(1), 113–125. https://doi.org/10.1007/BF00938210
(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03
(1995). Theory testing: Combining psychometric meta-analysis and structural equations modelling. Personnel Psychology, 48, 865–885. https://doi.org/10.1111/j.1744-6570.1995.tb01784.x
* (2000). Influence of deviant friends on delinquency: Searching for moderator variables. Journal of Abnormal Child Psychology, 28(4), 313–325. https://doi.org/10.1023/A:1005188108461
* (2001). Ethnic differences in the effect of parenting on gang involvement and gang delinquency: A longitudinal, hierarchical linear modeling perspective. Child Development, 72(6), 1814–1831. https://doi.org/10.1111/1467-8624.00380
* (2014). Pathways to early delinquency: Exploring the individual and collective contributions of difficult temperament, low maternal involvement, and externalizing behaviour. Journal of Criminal Justice, 42, 321–326. https://doi.org/10.1016/j.jcrimjus.2014.04.003
* (1991). Monitoring and delinquency. Criminal Behaviour and Mental Health, 1(3), 268–281.
* (1988). Direct parental controls and delinquency. Criminology, 26, 263–285. https://doi.org/10.1111/j.1745-9125.1988.tb00841.x
* (2003). Family relationship quality and contact with deviant peers as predictors of adolescent problem behaviours: The moderating role of gender. Journal of Adolescent Research, 8(5), 454–480. https://doi.org/10.1177/0743558403255063
* (2004). Risk factors for selected health-related behaviours among American Indian Adolescents. Dissertation Abstracts International, 65, 3(B).
* (2011). Reciprocal relations between parenting and adjustment in a sample of juvenile offenders. Child Development, 82, 633–645. https://doi.org/10.1111/j.1467-8624.2010.01523.x
(2016). Fitting meta-analytic structural equation models with complex datasets. Research Synthesis Methods, 7(2), 121–139. https://doi.org/10.1002/jrsm.1199
* (2006). Parenting behaviour, quality of the parent-adolescent relationship, and adolescent functioning in four ethnic groups. Journal of Early Adolescence, 26(2), 133–159. https://doi.org/10.1177/0272431605285718
* (1999). Actor, situation, and context: A framework for delinquency theory integration. American Journal of Criminal Justice, 24, 117–138. https://doi.org/10.1007/BF02887621
* (2011). The role of deliberative decision making, parenting, and friends in adolescent risk behaviours. Journal of Youth and Adolescence, 40, 1607–1622. https://doi.org/10.1007/s10964-011-9644-8
* (2011). Gender differences in parent–child bonding: Implications for understanding the gender gap in delinquency. Journal of Criminal Justice, 34, 3–23. https://doi.org/10.1080/0735648X.2011.554744
* (2001). Parental efficacy and delinquent behaviour: Do control and support matter? Criminology, 39(3), 677–705. https://doi.org/10.1111/j.1745-9125.2001.tb00937.x
(2016). Meta analytical structural equation modeling: Comments on issues with current methods and viable alternatives. Research Synthesis Methods, 7, 215–231. https://doi.org/10.1002/jrsm.1213
