Skip to main content
Open AccessOriginal Article

A Comparison of Simple Structure Rotation Criteria in Temporal Exploratory Factor Analysis for Event-Related Potential Data

Published Online:https://doi.org/10.1027/1614-2241/a000175

Abstract

Abstract. It is challenging to apply exploratory factor analysis (EFA) to event-related potential (ERP) data because such data are characterized by substantial temporal overlap (i.e., large cross-loadings) between the factors, and, because researchers are typically interested in the results of subsequent analyses (e.g., experimental condition effects on the level of the factor scores). In this context, relatively small deviations in the estimated factor solution from the unknown ground truth may result in substantially biased estimates of condition effects (rotation bias). Thus, in order to apply EFA to ERP data researchers need rotation methods that are able to both recover perfect simple structure where it exists and to tolerate substantial cross-loadings between the factors where appropriate. We had two aims in the present paper. First, to extend previous research, we wanted to better understand the behavior of the rotation bias for typical ERP data. To this end, we compared the performance of a variety of factor rotation methods under conditions of varying amounts of temporal overlap between the factors. Second, we wanted to investigate whether the recently proposed component loss rotation is better able to decrease the bias than traditional simple structure rotation. The results showed that no single rotation method was generally superior across all conditions. Component loss rotation showed the best all-round performance across the investigated conditions. We conclude that Component loss rotation is a suitable alternative to simple structure rotation. We discuss this result in the light of recently proposed sparse factor analysis approaches.

Event-related potential (ERP) data are a common electrophysiological measure of brain activity that is time-locked to an event (e.g., a stimulus presented to the participant; Luck, 2014). They are computed from a continuous electro-encephalogram (EEG) typically recorded with a high sampling rate (e.g., 512 Hz) at multiple electrode sites (e.g., 64 or 128) from the participant’s scalp. For each participant, the continuous EEG signal is cut into epochs around the events of interest (e.g., stimuli in several experimental conditions) and averaged across repetitions of the same event to improve the signal-to-noise ratio. Assuming that the events of interest represent stimuli under different experimental conditions, this procedure results in an average time series for the electric potential for each experimental condition and electrode per participant. Throughout this paper, we will refer to such a dataset as an ERP dataset.

Researchers are typically interested in amplitude (or latency) differences between the ERPs from different experimental conditions. Such analyses are challenging for at least two reasons: First, due to the large number of sampling points and electrodes, comparisons between conditions typically face massive multiple testing problems.

Second, the observed voltage at the scalp is a 2D mixture of temporally and spatially overlapping source signals generated in a 3D space in the brain – complicating functional interpretations of the observed differences. Exploratory factor analysis (EFA) has been used to reduce the multiple testing problem (but see also Groppe, Urbach, & Kutas, 2011a, 2011b; Maris, 2004), and to characterize the mixture of signals in a data-driven way (i.e., without any anatomical knowledge; Chapman & McCrary, 1995; Dien, 2012; Donchin, 1966, 1978; Kayser & Tenke, 2005). In principle, EFA can be conducted in the temporal or in the spatial domain (or in both; Dien, 2010a; Dien & Frishkoff, 2005). Here, we will focus on EFA in the temporal domain.

For the sake of completeness, it should be noted that such analyses are often conducted using Principal Component Analysis (PCA) rather than EFA. The main difference between EFA and PCA is that the former includes explicit error terms for each variable (i.e., sampling point), whereas the latter does not (see e.g., Widaman, 2007, 2018). From this perspective, PCA estimates a restricted EFA model in which all error variances are fixed to zero (McDonald, 1996). Consequently, differences between EFA and PCA are negligible when the error variances approach zero and/or the number of observed variables is high. It has been argued that this is typically the case for ERP data (Dien, Beal, & Berg, 2005), but there is no guarantee that this precondition holds for every application EFA or PCA to ERP data. Therefore, we refer to this data analytic approach (i.e., either EFA or PCA) by the more general term EFA throughout this work.

When applying temporal EFA to ERP data, an important precondition for drawing valid substantive conclusions is the correct allocation of condition effects to the latent factors. For instance, when there are two factors in the population of which only one is affected by the experimental condition, the sample estimates should (on average) resemble this pattern. Situations in which this is not the case have been referred to as variance misallocation in the literature (Dien, 1998; Dien et al., 2005; Kayser & Tenke, 2003; Wood & McCarthy, 1984). Research has identified biased factor loading estimates as a major source of variance misallocation and emphasized the importance of the factor rotation step for variance misallocation (Dien, 2010a; Möcks & Verleger, 1986; Scharf & Nestler, 2018). With the present research, we aimed, first, to better characterize the role of factor rotation for the occurrence of variance misallocation, and, second, to compare the performance of a wide range of common rotation criteria, including the recently proposed Component loss rotations (Jennrich, 2004, 2006), for this specific application. This comparison is also relevant from a more general methodological perspective because some of the investigated rotation criteria have rarely been considered in previous simulation research (e.g., Schmitt & Sass, 2011).

The present article is organized as follows: First, we briefly explain the temporal EFA model in the context of ERP data and describe a prototypical data analytic procedure followed by a technical definition of variance misallocation. Then, we explain the factor rotation step in more detail and elaborate on how it is related to the variance misallocation problem. Afterward, we report the results of a simulation study in which we compared the performance of various rotation techniques under a variety of conditions in which we manipulated the amount of temporal overlap, the size of the factor correlations and the amount of topographic overlap. Finally, we derive recommendations regarding the choice of the rotation method and discuss future research questions.

Temporal EFA for ERP Data

Temporal EFA operates on a p × n ERP data matrix T in which the p sampling points are treated as variables (T-technique; Cattell, 1952), and the data from all electrodes, conditions, and participants are treated as observations of the data matrix (i.e., n = nelectrodes · nconditions · nparticipants). That is, the voltage at each sampling point is decomposed into a weighted sum of m underlying factors. This factor model can be expressed in matrix notation as (e.g., Mulaik, 2010):

(1) where Λ is a p × m matrix of factor loadings, η is an m × n matrix of factor scores, and ϵ is the p × n matrix of error terms. Put simply, the factor loadings reflect the time courses of the latent factors and factors that load higher on a specific sampling point contribute more to the voltage at that sampling point. The amplitudes of the factors are represented by the factor scores. Importantly, in this approach, the factor time courses are assumed to be (approximately) equal across all electrodes, conditions, and participants and only the amplitudes are allowed to vary. That is, scalar invariance is assumed (see Putnick & Bornstein, 2017, for an introduction).

In order to analyze amplitude differences between conditions, the factor scores are obtained (e.g., by utilizing the regression method; Thomson, 1935; Thurstone, 1935), and subjected to a general linear model, typically a (robust) analysis of variance (Dien, 2012, 2017) with the aim of identifying which factors are affected by an experimental manipulation. This information is a crucial ingredient for functional interpretations of the factors, that is, to determine which cognitive processes are related to the factors (Luck, 2014). It has been shown that EFA-based quantifications of ERP signals are superior to more naïve quantifications such as peak-picking or averaging the voltage in a time region of interest (Beauducel & Debener, 2003; Beauducel, Debener, Brocke, & Kayser, 2000).

Research on variance misallocation has shown that this generally useful analytic approach has certain limitations that follow from the specific characteristics of ERP datasets (Dien, 1998, 2010a; Dien et al., 2005; Kayser & Tenke, 2003; Scharf & Nestler, 2018; Wood & McCarthy, 1984). Most generally, variance misallocation can be defined as a bias in the effect size estimates (of the condition effects) with respect to the population model (Scharf & Nestler, 2018).1 Biases in the factor loading estimates are arguably the main source of variance misallocation (Möcks & Verleger, 1986). These biases can occur either due to an inappropriate choice of orthogonal rotation methods (orthogonality bias; Scharf & Nestler, 2018), or because of the rotation procedure in general which biases the results toward its simple structure criterion (rotation bias; Scharf & Nestler, 2018; Schmitt & Sass, 2011).

With respect to the orthogonality bias, the use of orthogonal rotation methods generally carries a high risk of variance misallocation because ERP datasets contain the data from all electrodes and conditions within each participant in the rows. This has the consequence that the factor (co-)variances are the sum of (co-)variance contributions of participants, electrodes, and conditions (Scharf & Nestler, 2018). Especially the contribution of the electrodes (i.e., the factor topography), makes it highly unlikely that the overall factor covariances will be zero (Dien, 2010a). This notion is in line with research showing the superiority of oblique rotation methods in EFA for ERP data (Dien, 1998; Dien et al., 2005).

The rotation bias is a consequence of the temporal overlap between the factors (Dien, 1998; Dien et al., 2005). Temporal overlap refers to sampling points that have non-zero loadings on more than one factor (i.e., sampling points with cross-loadings). It is well-known that the performance of factor rotation methods depends on the size of cross-loadings and that factor rotation methods differ in their tolerance for cross-loadings (Browne, 2001). More specifically, most rotation methods tend to underestimate the cross-loadings at the cost of inflated factor correlations, and they differ in the extent to which they are prone to these distortions (Schmitt & Sass, 2011). This implies that the choice of a suitable rotation technique for a specific application has a profound impact on the correctness of the factor solution.

Factor Rotation

In the following, we will briefly describe the mathematical foundations of factor rotation and introduce some common oblique factor rotation criteria. On the basis of the findings on the orthogonality bias, orthogonal rotation methods will not be further considered here. In general, an infinite set of parameters (i.e., factor loadings and factor correlations) fits the covariance matrix of the data equally well in EFA. This property of the EFA model is typically referred to as rotational indeterminacy, and the mathematical operation that transforms one set of parameters into another equally well-fitting set of parameters is called factor rotation (e.g., Mulaik, 2010). Mathematically, factor rotation can be expressed with the following equation (e.g., Mulaik, 2010, p. 276):

(2)

Here, H denotes the m × m rotation matrix. Notably, the error term is not affected by factor rotation (ϵrot = ϵ), reflecting the fact that the total amount of variance in the data that is accounted for by the factors remains unchanged by the rotation. Since H−1 · H = Im (where Im denotes an identity matrix of order m), any invertible matrix H could be used in this transformation, and additional criteria are required to achieve a unique solution. Typically, the rotation matrix is determined in a way that optimizes the interpretability of the factor loading matrix by striving for a simple structure (Thurstone, 1947). Put simply, the variables (i.e., sampling points) are assigned to the factors as distinctly as possible.

To this end, a variety of rotation techniques have been proposed that differ mainly in the mathematical criterion f(Λ) that is utilized as operationalization of the simple structure ideal. Table 1 provides an overview of common oblique rotation criteria. Many rotation criteria can be subsumed under the general Crawson-Ferguson (CF) rotation family that optimizes the criterion (e.g., Crawford & Ferguson, 1970; Sass & Schmitt, 2010):

(3)

Table 1 Overview of common oblique simple structure rotation techniques

The first and second terms reflect the variable (i.e., row) and factor (i.e., column) complexity of the factor loading matrix, respectively. The weight k = [0, 1] determines the extent to which each of these contributions is considered during factor rotation with higher values indicating more emphasis on factor complexity. Not all rotation techniques in Table 1 are members of the CF-family but the distinction between variable and factor complexity provides a good framework to describe the rationale of rotation techniques.

For instance, Geomin rotation minimizes the variable-wise geometric mean of the squared factor loadings. Specifically, the Geomin criterion can be described by the following equation (Browne, 2001; Yates, 1987):

(4)

The product term of a specific variable i in Equation (4) is small if at least one of the factor loadings for that variable is close to zero. In that sense, Geomin focuses on variable complexity. The rotation parameter ϵ is a small constant value, typically between 0.0001 and 0.01. This is necessary because, when ϵ is set to zero and one factor loading of a specific variable is zero, the product of that row would be zero irrespective of all remaining factor loadings in that specific row. Recently, a modified Geomin rotation with ϵ set to 0.5 has been recommended when high cross-loadings are expected (Marsh, Liem, Martin, Morin, & Nagengast, 2011; Marsh et al., 2010, 2009). With respect to Equation (4), a higher value of ϵ reduces the relative impact of a single small loading on the product term – consequently, the rotation criterion is less focused on variable complexity.

Component loss rotation is a notable exception from the reasoning of distinguishing row and column complexity (Jennrich, 2004, 2006; Mulaik, 2010, Chapter 12.7). Unlike most simple structure rotation methods, Component loss rotation does not impose any assumptions about the pattern of the non-zero loadings. Instead, a general loss function is minimized, for instance, the sum of the absolute values of the factor loadings:

(5)

Consequently, Component loss rotation aims for as many (close to) zero factor loadings as possible, irrespective of the distribution of the non-zero loadings across the factor loading matrix. Component loss rotation is both mathematically and conceptually very close to recently popularized regularized estimation methods (e.g., Hastie, Tibshirani, & Friedman, 2009) in which, for instance, the sum of the absolute values of the to-be-estimated parameters is added as a penalty term during parameter estimation (least absolute shrinkage operator, lasso; Tibshirani, 1996). The most important difference between a Component loss rotated EFA and a regularized factor analysis is that Component loss rotation operates on a “traditional” maximum likelihood or least-squares estimated initial model, whereas regularized estimation methods penalize the estimated parameters directly during the estimation.

Previous research has revealed that rotation methods differ in their ability to tolerate cross-loadings with a general tendency to underestimate cross-loadings while inflating the factor correlations in the presence of substantial cross-loadings (Asparouhov & Muthén, 2009; Browne, 2001; Sass & Schmitt, 2010; Schmitt & Sass, 2011). In addition, there is a trade-off in the sense that rotation techniques that perform well in the presence of cross-loadings often fail to recover simple structure when it exists. Overall, in the context of psychometric questionnaires, Geomin rotation has been found to yield reasonable results both for patterns with low cross-loadings and for patterns with high cross-loadings (Asparouhov & Muthén, 2009; Sass & Schmitt, 2010; Schmitt & Sass, 2011), and the modified Geomin with ϵ = 0.5 has been recommended when high cross-loadings are expected (Marsh et al., 2011, 2010, 2009). In the context of EFA for ERP data, however, a Promax rotation with Kaiser normalization has been established as the gold standard rotation in the temporal domain (Dien, 2010a; Dien et al., 2005; Dien, Khoe, & Mangun, 2007). An important limitation of previous research is that the range of rotation techniques that were considered has been rather narrow. For instance, Geomin rotation with modified rotation parameters has not yet been considered in the context of EFA for ERP data.

To our knowledge, Component loss rotation has not yet been included in any extensive comparison of rotation techniques – irrespective of the application context. We think that Component loss rotation is a very interesting approach because of its slightly different concept of simplicity. In Component loss rotation no assumptions are made about the pattern of the zero and non-zero loadings. Rather, many of the elements of the factor loading matrix are assumed to be zero, that is, that the population factor loading matrix is sparse. This less restrictive sparsity assumption suits common beliefs about ERP factors very well. Dien (2010a) emphasized that simple structure rotation is appropriate for temporal EFAs for ERP data due to the transient nature of ERP factors. That is, ERP factors tend to contribute to the observed electric potential only in a relatively narrow time range, and, hence, it seems reasonable to expect many zero loadings for each factor. However, as mentioned above, the temporal overlap between the factors poses a challenge for simple structure rotation. Component loss rotation may be especially appropriate for ERP factors because the underlying assumptions explicitly allow for high cross-loadings (i.e., temporal overlap).

The Present Study

In a simulation study, we investigated the rotation bias of oblique rotation methods as a source of variance misallocation, aiming to better characterize its behavior under various conditions. The present research extends previous efforts in at least two ways: First, previous simulation research included factors with varying temporal overlap (e.g., Dien, 1998; Dien et al., 2007) but was either limited to a small set of conditions or did not manipulate the temporal overlap in isolation, making it difficult to disentangle biases that were due to temporal overlap from biases that were due to other properties of the factors (e.g., their topographies). Here, we independently varied the amount of temporal overlap, the size of the factor correlation, and the topographic overlap of the factors, so that the relative contributions of all manipulations to variance misallocation could be investigated. Second, we considered many common rotation techniques (see Table 1), including two recently proposed criteria. Specifically, we included a modified Geomin rotation with a rotation parameter ϵ of 0.5 and Component loss rotation to see if they offer any advantages over traditional simple structure rotation techniques for ERP applications.

Method

Our simulation approach was based on recent simulations (Scharf & Nestler, 2018) in which simulated raw data were sampled on the basis of the common factor model (Equation 1). A similar approach has been taken, for instance, by Dien (2010a). We are aware that others have investigated the performance of factor rotation by directly rotating prototypical population patterns (Beauducel, 2018; Möcks & Verleger, 1986). However, whereas this approach is much simpler to implement, neither the consequences of the special structure of ERP datasets (i.e., multiple electrodes and conditions per participant) nor the potential differences in the standard errors of the estimates can be investigated with it.

Simulation Model

We investigated a set of simulation conditions in which we varied important determinants of the performance of factor rotation methods. Specifically, we varied the temporal overlap (5), the topographic overlap (2), and the between-participant correlation (2). The sample ERP datasets T were arranged as follows: The columns of T represented the 200 sampling points (spread over an epoch of 450 ms), and the rows contained the data from all 2 · 10 · 20 Condition × Electrodes × Participant combinations, respectively. To illustrate the electrode setup, one could think of 10 electrodes placed down the central line on the scalp where Electrode 1 is at the most anterior electrode site and Electrode 10 is at the most posterior electrode site.

The sample data were drawn from a matrix-variate normal distribution (e.g., Gupta, 2000). That is, T ∼ N (M, V, Σ), where M, V, and Σ were matrices containing the expected values, the row (co-)variances, and the column (co-)variances, respectively. The row covariance matrix was an identity matrix – resulting in independent samples between conditions and electrodes. The column covariance matrix was derived from the common factor model (e.g., Mulaik, 2010, p. 136, Equation 6.13). We specified a population factor loading matrix Λ with two factors. Figure 1 illustrates the population factor loading patterns. The time courses of both factors were created from Gaussian density functions with a standard deviation of 40 ms. Following the conventions of ERP research, we labeled the factors according to their temporal order rather than by the proportion of variance they explained. The first factor had its peak (i.e., mean) at 120 ms. The peak of the second factor was varied in order to manipulate the temporal overlap of the factors. We used values of 120 ms (L0), 150 ms (L1), 175 ms (L2), 200 ms (L3), 250 ms (L4), 300 ms (L5), that is, our simulation conditions covered the extreme cases of complete temporal overlap and complete temporal separation as well as a range of conditions with partial temporal overlap. The maximum loadings were 0.8 and 1.0 for the first and second factors, respectively. The factor variance was 1, and the factor correlation (φ12) was 0 or +0.3. For instance, in the simulation condition with a mildly positive correlation, participants with a more positive amplitude in the first factor were likely to show a more positive amplitude for the second factor. The error covariance matrix was a diagonal matrix with mutually uncorrelated errors (white noise) and a constant noise variance of 0.4 for all sampling points.

Figure 1 Population factor loading patterns with decreasing temporal overlap from L0 (left-most) to L5 (right-most). The solid and dashed lines represent the loadings of the first and second factors, respectively. Figure available from the OSF (https://osf.io/zmtcg/) under a CC-BY 4.0 license.

The matrix of expected values M was a 2 · 10 · 20 × 200 matrix that contained the expected time courses for each combination of participant, electrode, and condition in the rows. Assuming that all deflections of the voltage from zero were due to the factors, the expected time courses could be calculated as E(t= Λ · E(η), where E(η) contains the expected factor scores for the respective observation (Scharf & Nestler, 2018). For the simulation, we varied the expected factor scores E(η) as a function of electrode site and condition, but both the topographic and condition effects were held constant across participants (see also Beauducel & Debener, 2003). Both simulated factors were affected by the experimental condition: The first factor had expected factor scores of −1.5 (Condition 1) or −2.5 (Condition 2), and the second factor had expected factor scores of 2.5 (Condition 1) or 3.5 (Condition 2), respectively.

Following the principles of topographic component models (Achim & Bouchard, 1997; Möcks, 1988), we introduced topographic variance by defining a topographic weight for each electrode that was 1 at the topographic maximum and otherwise smaller than 1. These topographic weights were multiplied by the expected factor scores in each condition. For instance, if the topographic weight was 0.5, the corresponding expected values for Factor 1 was 0.5 · −1.5 = −0.75 (Condition 1) or 0.5 · 2.5 = −1.25 (Condition 2). This procedure resulted in realistic factors in the sense that the condition effects were maximal at the topographic maximum and otherwise followed the factor topography – including sign reversals.

We varied whether topography contributed to both the factor variances and the factor covariances or only to the factor variances (Figure 2; Scharf & Nestler, 2018). The topographic maximum of the first factor was always at Electrode 1 (“anterior” distribution), and the topographic weights decreased linearly toward a value of −0.5 at Electrode 10. In order to create simulation conditions with varying topographic overlap, we varied the topographic maximum of the second factors. In simulation conditions with overlapping topographies, the topographic maximum was at Electrode 10 (“posterior” distribution), resulting in a setup where both factors tended to have rather positive factor scores at posterior electrode sites. In simulation conditions with non-overlapping topographies, the topographic maximum was at Electrodes 5 and 6 (“central” distribution). In both cases, the topographic weights decreased linearly toward a value of 0.1 at the most distant electrode sites. These conditions represented two possible extreme cases: maximal and minimal topographic overlap. The interested reader is referred to Table S1 in the Electronic Supplementary Material, ESM 1, for the factor means derived from this population model.

Figure 2 Illustration of the simulated factor means of each factor in the experimental conditions – separately for overlapping (A) and non-overlapping (B) topographies. Figure available from the OSF (https://osf.io/zmtcg/) under a CC-BY 4.0 license.

Simulation Procedure

The simulations were conducted in R (Version 3.5.1, R Core Team, 2018). All simulation and analysis scripts are available at the OSF (https://osf.io/zmtcg/). In each simulation condition, 1,000 random samples were drawn using the package LaplacesDemon (Statisticat & LLC, 2016). For each sample, we ran a parallel analysis to determine the number of factors that would be used in an actual research setting (Horn, 1965). However, we always extracted a two-factor solution because we wanted to focus on a comparison of the rotated solutions between rotation techniques. The initial unrotated solution was estimated using a Maximum Likelihood approach (Lawley & Maxwell, 1971) as implemented in the package psych (Revelle, 2016).

The unrotated solution was rotated by applying each of the rotation techniques listed in Table 1. All rotation methods were used in their oblique versions to avoid biases due to inappropriate orthogonality constraints (Dien, 1998, 2010a; Scharf & Nestler, 2018). Promax rotation was applied using the psych package, that is, a Kaiser-normalized Varimax rotation was applied in a first step. Then, the Varimax-rotated loadings were raised to the fourth power and used as target matrix in the subsequent target rotation (see also Mulaik, 2010). Component-loss rotation was implemented using a derivative-free gradient projection algorithm (Jennrich, 2004).

Geomin (ϵ = 0.0001 or 0.5), Quartimin, Covarimin, Infomax, Parsimax, and Equamax rotated solutions were estimated using a gradient projection algorithm as implemented in the package GPArotation (Bernaards & Jennrich, 2005). We supplied the rotation matrix from an oracle target rotation (i.e., the population loadings were entered as target) as starting values to the gradient projection algorithm to avoid different local optima between samples (Hattori, Zhang, & Preacher, 2017; Weide & Beauducel, 2019).2 Except for Promax rotation, all rotations were applied without prior normalization of the factors.

For each rotated solution, order and sign indeterminacies inherent to the EFA model were resolved (e.g., Asparouhov & Muthén, 2009; Mulaik, 2010). Specifically, the factors were reordered according to the correlations of the estimated loadings with the population factor loadings, and the factors were multiplied by −1 if the sum of their loadings was smaller than one. Finally, the estimated experimental condition effects were quantified as the standardized differences (Cohen’s d; Cohen, 1977) in the factor scores at the topographic maximum of the respective factor. The factor scores were estimated using the regression method (Thomson, 1935; Thurstone, 1935). As in the ERP PCA Toolkit, the factor scores were not centered (Dien, 2010b).

Dependent Measures

We calculated measures of global model fit as well as measures of parameter recovery, and we quantified the amount of variance misallocation that could be expected in tests for differences between experimental conditions. In addition, we computed how often parallel analysis suggested the extraction of 1, 2, or more factors across all samples for each simulation condition. As a measure of global model fit, we report the standardized root mean residual (SRMR; Brown, 2014). Notably, this measure cannot be used to evaluate the performance of rotation techniques because all rotated solutions for a given dataset have the same model fit as the unrotated solution due to rotational indeterminacy (e.g., Mulaik, 2010). Rather, this measure served as an indicator of how successfully the observed data could be described by the estimated factor models.

The recovery of the factor loadings was quantified by the Tucker congruences (; Tucker, 1951) and the Pearson correlations () between the average estimated and the population factor loading matrix. The former is often used in simulation studies in a psychometric context (e.g., De Winter, Dodou, & Wieringa, 2009), whereas the latter is more common in research on EFA for ERP data (e.g., Dien, 1998). The most important difference is that the correlation is a measure of similarity (and not of congruence) and is thus not affected by changes in the average loadings. The recovery of the factor correlation was quantified as the absolute bias between the estimated and the population factor correlation (Biasφ). Finally, we calculated the absolute bias of the estimated effect size measure (Biasδ) in order to quantify the consequences of biases in the EFA parameters for the statistical tests of condition effects independent of statistical power (see Beauducel & Debener, 2003, for a discussion).

Results

The main results are summarized in Tables 2 and 3 for conditions with overlapping and non-overlapping topographies, respectively. For applied readers, we note that the bias in the estimated effect sizes (Biasδ) is the most relevant measure with respect to the consequences for consecutive statistical inference. For all conditions, the EFA model fit the data very well as indicated by SRMR values between 0.03 and 0.04 (Hu & Bentler, 1999). The model fit varied only as a function of temporal overlap – slightly decreasing as the temporal overlap increased (L0: SRMR = 0.037, L5: SRMR = 0.031). In conditions with partial temporal overlap (L1–L5), parallel analysis suggested the extraction of two factors in all samples with one notable exception: In the condition with high temporal overlap (L1), topographic overlap and a positive factor correlation (i.e., φ12 = 0.3), a one-factor solution was recommended in 16.5% of the samples. In conditions with complete temporal overlap (L0), the extraction of a single factor was indicated in all samples without exception.

Table 2 Simulation results for dependent measures as a function of the population factor correlation, temporal overlap, and rotation method for conditions with overlapping topographies
Table 3 Simulation results for dependent measures as a function of the population factor correlation, temporal overlap, and rotation method for conditions with non-overlapping topographies

Irrespective of rotation technique and simulation condition, the estimated factor loadings (Figures 3 and 4) were too high relative to the population loadings indicating a rescaling due to the topographic and condition variance (Scharf & Nestler, 2018). Apart from this, the recovery of the factor loadings (i.e., time courses) was sufficient for the majority of the investigated conditions and rotation techniques as indicated by high factor congruencies () and correlations (). When there was no temporal overlap (L5), the recovery of the factor loadings was nearly optimal. When the temporal overlap was substantial (especially L1–L3), all rotation methods tended to underestimate the cross-loadings (i.e., loadings within the region of temporal overlap). This pattern was slightly more pronounced in conditions with non-overlapping topographies and without between participant correlations (φ12 = 0). In conditions with perfect temporal overlap, factor rotation tended to collapse the factors yielding one major and one minor factor.

Figure 3 Average factor loadings (F1 solid lines; F2 dashed lines) as a function of rotation technique (columns) and temporal overlap (rows) for conditions with overlapping topographies and uncorrelated factors across participants. The population loadings (gray) are depicted as a reference line. For the sake of comprehensibility, only selected relevant temporal overlap conditions are displayed. Figure available from the OSF (https://osf.io/zmtcg/) under a CC-BY 4.0 license. The depicted conditions were representative of the general results pattern. Nevertheless, supplementary figures for the remaining conditions are available from the OSF. A version containing the complete information is available in ESM 2.
Figure 4 Average factor loadings (F1 solid lines; F2 dashed lines) as a function of rotation technique (columns) and temporal overlap (rows) for conditions with non-overlapping topographies and uncorrelated factors across participants. The population loadings (gray) are depicted as a reference line. For the sake of comprehensibility, only selected relevant temporal overlap conditions are displayed. Figure available from the OSF (https://osf.io/zmtcg/) under a CC-BY 4.0 license. The depicted conditions were representative of the general results pattern. Nevertheless, supplementary figures for the remaining conditions are available from the OSF. A version containing the complete information is available in ESM 3.

A similar results pattern was found for biases in the factor correlations (Biasφ) that tended to be more positive with increasing temporal overlap. In addition, the bias in the factor correlation tended to be strongly positive in the presence of topographic overlap. In the absence of topographic overlap, the bias in the estimated factor correlation was negative for low temporal overlap (L4 or L5) but positive for conditions with substantial topographic overlap (L1–L3). For perfect temporal overlap conditions (L0), factor rotation resulted either in nearly orthogonal factor (e.g., for Promax) or in extremely correlated factors (e.g., Geomin with ϵ = 0.5). The bias in the estimated effect sizes (Biasδ) generally followed the biases in the EFA parameters; that is, they were more biased the more the factor loadings and correlations were biased – with a general tendency to underestimate the population effect sizes.

The rotation techniques differed primarily in the extent to which they were prone to these distortions – except for Covarimin rotation, which yielded strongly inferior results for almost all conditions, and Infomax, which yielded extremely unstable estimates when the temporal overlap was perfect (L0). In the majority of the conditions, all rotation techniques performed quite similarly with small performance advantages for the Promax, Geomin and Component loss rotations. However, in conditions with very high temporal overlap (L1), substantial factor correlation and/or overlapping topographies, we observed a conflation of the factors for the Promax, Geomin (0.0001), and Quartimin rotations (e.g., upper-most subfigure in the left-most column in Figure 3). That is, they yielded a solution with a large factor that spanned the time range of both factors and an additional smaller factor that explained some of the remaining variance. This artifact was also visible in the biases of the factor correlations that became much less positive compared with rotation techniques that did not conflate the factors. When looking at the raw factor correlations, the factors in conflated solutions were (almost) uncorrelated. Conflated solutions occurred across all samples irrespective of the number of factors that was indicated by parallel analysis in the respective sample and resulted in profound biases in the effect size estimates (up to −0.50).

Of the remaining rotation techniques, Component loss performed best in conditions with temporal overlap (L1) – followed closely by the Geomin (0.5) rotation. Both rotation techniques were immune to conflated solutions and showed substantially smaller biases on all measures for these conditions. However, Geomin (0.5) yielded small spurious cross-loadings in conditions without temporal or topographic overlap. Therefore, considering the results across all conditions, Component loss rotation showed the best performance by far. It was not prone to conflated solutions when the temporal overlap was high, it was able to recover the factors in the absence of temporal overlap (L5), and it yielded generally almost unbiased effect size estimates. In fact, there were only two conditions (both L1 conditions without topographic overlap) in which Component loss rotation was slightly inferior to the Promax rotation.

Discussion

In the present study, we compared the performance of a variety of oblique simple structure rotation techniques and Component loss rotation for a wide range of conditions typical of EFA applications to ERP data. We observed characteristic biases in the factor loading and factor correlation estimates that were a function of temporal overlap and the factor topography. In line with its gold standard status in ERP applications of temporal EFA (Dien, 1998; Dien et al., 2005), Promax rotation performed very well compared with all other simple structure rotations – except for conditions with high temporal and topographic overlap in which it yielded conflated factors. However, Component loss rotation was clearly the most flexible rotation technique, performing best or second best in all of the conditions we investigated.

All observed biases are easy to explain when considering the consequences of the structure of ERP datasets (Scharf & Nestler, 2018). The factor variances and covariances are a mixture of (co-)variances due to participants, scalp topography, and condition effects. As a consequence, the factors are rescaled so that the total variance (instead of the participant-related variance only) is 1, explaining the rescaling of the factor loadings. In addition, the factor covariance is a function of the topographic overlap, the between-participant correlation, and the conditional overlap (i.e., the extent to which both factors are affected by the experimental condition). In our simulations, the topographic overlap was the largest contribution, resulting in large positive biases in the factor correlations. In conditions without topographic overlap, the factor correlation was underestimated due to the increased relative impact of the negative condition effect overlap (δF1 < 0, δF2 > 0). Considering this, the results are well in line with the literature that has shown that factor rotations tend to underestimate cross-loadings at the cost of inflated factor correlations when the cross-loadings (here: temporal overlap) are high in the population (Sass & Schmitt, 2010; Schmitt & Sass, 2011).

This interpretation is also in line with the occurrence of conflated solutions in our simulations. Taking a closer look at the specific simulation conditions that suffered from this artifact, one can see that conflated solutions were more likely to occur when the total factor correlation was high. For instance, in conditions with topographic overlap and high temporal overlap, where Promax yielded conflated solutions, the estimated factor correlation was between 0.70 and 0.95 for rotation techniques that did not conflate the factors. It is a known property of some rotation methods that they are prone to such conflated solutions when both the factor correlations and the cross-loadings are high (Schmitt & Sass, 2011).

The differential susceptibility to conflated factors and rotation biases between factor rotation techniques can be explained when considering the foundations of each technique.3 Promax is a two-step rotation where Varimax rotation is applied in a first step, the Varimax-rotated loadings are raised to the power κ (bringing near-zero loadings closer to zero) and subjected to a second (oblique) target rotation. As an orthogonal rotation the initial Varimax rotation is less prone to biases due to temporal overlap (e.g., Scharf & Nestler, 2019a) as long as the factors are not completely overlapping (Beauducel, 2018; Wood & McCarthy, 1984). This is presumably the main reason for the superior performance of Promax in previous simulations (Dien, 2010a). However, this advantage disappears once factors are highly correlated and temporally overlapping to an extent where the Varimax criterion prefers a conflated solution. Indeed, supplementary simulations suggested that conflated solutions for Varimax rotation occur in the same conditions as for Promax rotation.

In general, rotation criteria are more susceptible to conflated solutions, the more they focus on minimizing variable complexity (see Kaiser, 1958, for a similar argument). For instance, Quartimax which minimizes only variable complexity (i.e., k = 0) yielded conflated solutions whereas Parsimax that considers factor complexity as well (i.e., k > 0) did not. However, a too strong focus on the minimization of factor complexity yields rotation criteria that are unable to recover perfect simple structure (e.g., Scharf & Nestler, 2019b; Schmitt & Sass, 2011). This reasoning is applicable outside the CF-family as well: The Geomin criterion minimizes the variable-wise geometric mean of the squared factor loadings (Equation 4). We found that an increased rotation parameter ϵ of 0.5 leads to better tolerance of cross-loadings. This is in line with the notion that a larger value of ϵ decreases the relative impact of each small loading on the overall criterion. Hence, the emphasis on variable complexity is reduced and with it the risk of conflated factors. Finally, the fact that Component loss rotation (which aims for a factor loading matrix with as many zero elements irrespective of their distribution across variables or factor) was able to recover a wider range of factor loadings patterns further supports the notion that the distinction between variable and factor complexity is not beneficial when the complexity of the factor loading pattern cannot be anticipated – as in the case of ERP data.

For the choice of the most appropriate rotation technique, it is important to consider whether conflated factor solutions are desirable in a specific research context or not. Here, we generated data from a population with two factors that are clearly differentiable from a substantive point of view because they are characterized opposing condition effects (and different topographies in half of the conditions). A conflation of the factors results in a single factor with a condition effect that is a mixture of the condition effects from both factors. In the most extreme case, the conflated factor may yield a zero condition effect because both effects cancel out. Therefore, conflated solutions are clearly undesirable for our simulated data. Nevertheless, there may be situations in which a conflated solution is actually more appropriate. For instance, an experimental manipulation may result both in amplitude and latency changes and, when the latency shift between conditions is strong enough, EFA may yield two separate factors (one for each condition). This implies that these factors should be highly correlated both due to between-participant and topographic similarity as they actually reflect the same underlying factor. In such cases, a conflated solution may actually be desirable because it resembles the underlying reality more closely.

The present results indicate that parallel analysis may be able to differentiate between these two cases. In the condition which comes closest to the case of latency shift (i.e., extreme topographic overlap and positive factor correlation), a substantial proportion of samples would have yielded a one factor solution. The fact that the data generating two-factor model was still preferred is arguably due to the negative contribution of the condition effects to the overall factor covariance (see also Scharf & Nestler, 2018). We tested this interpretation by replicating the simulation condition but without experimental condition effects, that is, the condition effects did not reduce the overall factor correlation. Indeed, for an increased proportion of 40% of all samples parallel analysis indicated a one-factor solution in this condition. Although more systematic and direct investigations of the factor extraction step are necessary to settle this issue, we tend to conclude that conflated factors are undesirable solutions of the factor rotation step in most cases.

Taken together, the present results are well in line with previous investigations of temporal EFA for ERP data (Beauducel, 2018; Dien, 1998, 2010a; Dien et al., 2005; Möcks & Verleger, 1986; Scharf & Nestler, 2018; Wood & McCarthy, 1984). We confirmed that the best performing simple structure rotation techniques work very well across many plausible ERP factor patterns but also that they yield substantial biases when the temporal overlap is extreme. However, there are application scenarios in which even more extreme temporal overlap should be expected, for instance, slow-wave potentials that may overlap with all other factors of interest (see Beauducel, 2018; Möcks & Verleger, 1986, for some examples). Therefore, further research is needed to develop rotation techniques that are able to recover both highly overlapping and non-overlapping ERP factors – possibly challenging the predominance of the simple structure concept for ERP data. In the following, we outline some efforts that could be made in that direction.

Recently, the suggestion has been made that factor rotation criteria for ERP data can be improved by utilizing additional information that is unique for ERP data. Specifically, knowledge about plausible time courses (Beauducel, 2018) or the allocation of known experimental condition effects (Beauducel & Leue, 2015) can be useful for developing ERP-specific rotation criteria. We think that a comparison of these techniques with simple structure rotation would have gone beyond the scope of the present paper. However, future studies should include these approaches to evaluate whether they perform well under sufficiently general conditions to be useful for real-data applications.

An important limitation of ERP-specific rotation criteria is that they require sufficient and, above all, valid knowledge about plausible time courses (or condition effects). In the absence of such knowledge, the investigated Component loss rotation seems to be a promising alternative. We think that its good all-round performance is due to the inherent sparsity assumption. Like simple structure rotation, it is assumed that many loadings are zero, but no assumptions are made about the specific pattern of zero loadings (e.g., that a least one factor has zero loadings on each sampling point). In this sense, simple structure may be regarded as a special case of sparsity (Yamamoto, Hirose, & Nagata, 2017), and sparse estimation of factor models is a suitable alternative to simple structure rotation (Scharf & Nestler, 2019c; Trendafilov, 2014). The assumption that many loadings are zero is reasonable for temporal ERP factors (see also Dien, 2010a), but pattern assumptions are not necessarily justified (due to temporal overlap). Despite the promising results, further research should investigate the performance of sparsity-based approaches for real ERP datasets and for more challenging factor patterns such as overlapping slow-wave potentials.

Apart from the question which rotation technique should be applied in temporal EFA for ERP data, the present results are also relevant when applying exploratory structural equation modeling (ESEM) to ERP data (Scharf & Nestler, 2019a) in which factor rotation is an essential step as well. Rotation biases due to temporal overlap will also occur for ESEM factors because the very same rotation techniques are used. However, a crucial difference between EFA and ESEM is that ESEM properly acknowledges the topographic variance in its structural model. As a consequence, ESEM reduces the problem of very high factor correlations that resulted in conflated solutions. Therefore, ESEM might make the application to ERP data slightly less challenging for common rotation techniques. In this sense, the present results may also be interpreted as an additional argument in favor of approaches such as ESEM that are able to separate topographic, condition-related, and between-participant (co-)variance.

Finally, we want to acknowledge some limitations of the present study. First, we simulated an unrealistically simplistic factor pattern with only two factors. Realistic applications are characterized by solutions with many more factors (e.g., 12). Whereas this simplification enabled us to focus on the core principles that underlie the rotation biases in temporal EFA for ERP data, it might have concealed differential behavior of rotation techniques as a function of the number of factors. Second, the conditions with topographic overlap were characterized by the most extreme topographic overlap possible. That is, with respect to topographic overlap, we studied the worst-case scenario. This might have exaggerated some of the biases that we observed. However, it is impossible to have more than two topographically non-overlapping factors in temporal EFA for ERP data (Dien, 2010a). Given this and the typically high number of factors, it is reasonable to assume substantial topographic overlap – rendering our simulation conditions more realistic than they might seem at first sight.

Third, we did not systematically investigate the influence of normalization on the rotated solutions. Previous findings indicate that it may be beneficial to normalize the unrotated loadings during rotation (Dien et al., 2005). In a supplementary simulation, we did not observe such benefits but our simulated factors did not differ very much in their overall variance. Therefore, our results are not conclusive with respect to the question whether normalization is generally beneficial for the rotated solutions when EFA is applied to ERP data. Lastly, we always extracted the correct number of factors. For real data applications, this might not be the case. For most of the investigated conditions, the correct number of factors would have been extracted. However, especially in conditions with perfect temporal overlap, fewer factors would be extracted than we assumed. The determinants of correct factor extraction in temporal EFA for ERP data remain an open question and the present results underline the need for further investigations (see also Dien, 2006; Dien et al., 2005; Kayser & Tenke, 2003).

Conclusion

In the present study, we compared the performance of a variety of oblique simple structure rotation techniques and Component loss rotation for a wide range of conditions typical of EFA applications to ERP data. The results confirmed that the best-performing simple structure rotation techniques work very well across many plausible ERP factor patterns but also that they yield substantial biases when the temporal overlap is extreme. Geomin rotation with a rotation parameter of 0.5 and Promax were the best performing simple structure rotation techniques but Component loss rotation showed the best all-round properties of all investigated rotation techniques. We conclude that sparse (or regularized) factor models should be considered as an alternative to simple structure rotation for ERP data.

We thank Jana Pförtner for assisting in the preparation of the simulation study. We also thank two Reviewers whose thoughtful comments led to significant improvements of the manuscript.

Florian Scharf is a postdoctoral research at the University of Münster. His interests include: Structural Equation Modeling of neurophysiological data, performance of dimension reduction techniques for EEG data, validity and reliability of Electrophysiological Measures, and the prediction in auditory perception.

Steffen Nestler is a professor for psychological methods at the University of Münster. He is interested in statistical approaches to model social relation and social network data and intraindividual variability data.

1Due to the rotational indeterminacy of the EFA model (e.g., Mulaik, 2010), an infinite set of factor correlations and factor loadings have an equivalent fit to the observed covariance matrix. Hence, the term bias is sometimes considered inappropriate in the context of EFA. However, in this specific application context, a ground truth (i.e., a generator pattern in the brain) exists – justifying the question to which extent EFA can blindly recover certain population parameters.

2More generally, local optima should be avoided by repeating the rotation procedure with multiple random starts. We also ran the simulation conditions with high temporal overlap (L0, L1, L2) for which local optima are more frequent (Hattori et al., 2017) with 100 random starts for each rotation. The results remained the same.

3We also tested the alternative interpretation that the performance of Promax differs because it is not estimated by gradient projection which is prone to local optima. However, the same results occur when using gradient projection to estimate the initial Varimax.

References

  • Achim, A., & Bouchard, S. (1997). Toward a dynamic topographic components model. Electroencephalography and Clinical Neurophysiology, 103, 381–385. https://doi.org/10.1016/S0013-4694(97)96055-0 First citation in articleCrossrefGoogle Scholar

  • Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397–438. https://doi.org/10.1080/10705510903008204 First citation in articleCrossrefGoogle Scholar

  • Beauducel, A. (2018). Recovering Wood and McCarthy’s ERP-prototypes by means of ERP-specific procrustes-rotation. Journal of Neuroscience Methods, 295, 20–36. https://doi.org/10.1016/j.jneumeth.2017.11.011 First citation in articleCrossrefGoogle Scholar

  • Beauducel, A., & Debener, S. (2003). Misallocation of variance in event-related potentials: Simulation studies on the effects of test power, topography, and baseline-to-peak versus principal component quantifications. Journal of Neuroscience Methods, 124, 103–112. https://doi.org/10.1016/S0165-0270(02)00381-3 First citation in articleCrossrefGoogle Scholar

  • Beauducel, A., Debener, S., Brocke, B., & Kayser, J. (2000). On the reliability of augmenting/reducing: Peak amplitudes and principal component analysis of auditory evoked potentials. Journal of Psychophysiology, 14, 226–240. https://doi.org/10.1027/1614-2241.14.4.226 First citation in articleLinkGoogle Scholar

  • Beauducel, A., & Leue, A. (2015). Controlling for experimental effects in event-related potentials by means of principal component rotation. Journal of Neuroscience Methods, 239, 139–147. https://doi.org/10.1016/j.jneumeth.2014.10.008 First citation in articleCrossrefGoogle Scholar

  • Bernaards, C. A., & Jennrich, R. I. (2005). Gradient projection algorithms and software for arbitrary rotation criteria in factor analysis. Educational and Psychological Measurement, 65, 676–696. https://doi.org/10.1177/0013164404272507 First citation in articleCrossrefGoogle Scholar

  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Press: New York, NY. First citation in articleGoogle Scholar

  • Browne, M. W. (2001). An overview of analytic rotation in Exploratory Factor Analysis. Multivariate Behavioral Research, 36, 111–150. https://doi.org/10.1207/S15327906MBR360105 First citation in articleCrossrefGoogle Scholar

  • Carroll, J. B. (1957). Biquartimin criterion for rotation to oblique simple structure in factor analysis. Science, 126, 1114–1115. https://doi.org/10.1126/science.126.3283.1114 First citation in articleCrossrefGoogle Scholar

  • Cattell, R. B. (1952). The three basic factor-analytic research designs – their interrelations and derivatives. Psychological Bulletin, 49, 499–520. https://doi.org/10.1037/h0054245 First citation in articleCrossrefGoogle Scholar

  • Chapman, R. M., & McCrary, J. W. (1995). EP component identification and measurement by principal components analysis. Brain and Cognition, 27, 288–310. https://doi.org/10.1006/brcg.1995.1024 First citation in articleCrossrefGoogle Scholar

  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press. First citation in articleGoogle Scholar

  • Crawford, C. B., & Ferguson, G. A. (1970). A general rotation criterion and its use in orthogonal rotation. Psychometrika, 35, 321–332. https://doi.org/10.1007/BF02310792 First citation in articleCrossrefGoogle Scholar

  • De Winter, J. C. F., Dodou, D., & Wieringa, P. A. (2009). Exploratory Factor Analysis with small sample sizes. Multivariate Behavioral Research, 44, 147–181. https://doi.org/10.1080/00273170902794206 First citation in articleCrossrefGoogle Scholar

  • Dien, J. (1998). Addressing misallocation of variance in principal components analysis of event-related potentials. Brain Topography, 11, 43–55. https://doi.org/10.1023/A:1022218503558 First citation in articleCrossrefGoogle Scholar

  • Dien, J. (2006). Progressing towards a consensus on PCA of ERPs. Clinical Neurophysiology, 117, 699–702. https://doi.org/10.1016/j.clinph.2005.09.029 First citation in articleCrossrefGoogle Scholar

  • Dien, J. (2010a). Evaluating two-step PCA of ERP data with Geomin, Infomax, Oblimin, Promax, and Varimax rotations. Psychophysiology, 47, 170–183. https://doi.org/10.1111/j.1469-8986.2009.00885.x First citation in articleCrossrefGoogle Scholar

  • Dien, J. (2010b). The ERP PCA Toolkit: An open source program for advanced statistical analysis of event-related potential data. Journal of Neuroscience Methods, 187, 138–145. https://doi.org/10.1016/j.jneumeth.2009.12.009 First citation in articleCrossrefGoogle Scholar

  • Dien, J. (2012). Applying principal components analysis to event-related potentials: A tutorial. Developmental Neuropsychology, 37, 497–517. https://doi.org/10.1080/87565641.2012.697503 First citation in articleCrossrefGoogle Scholar

  • Dien, J. (2017). Best practices for repeated measures ANOVAs of ERP data: Reference, regional channels, and robust ANOVAs. International Journal of Psychophysiology, 111, 42–56. https://doi.org/10.1016/j.ijpsycho.2016.09.006 First citation in articleCrossrefGoogle Scholar

  • Dien, J., Beal, D. J., & Berg, P. (2005). Optimizing principal components analysis of event-related potentials: Matrix type, factor loading weighting, extraction, and rotations. Clinical Neurophysiology, 116, 1808–1825. https://doi.org/10.1016/j.clinph.2004.11.025 First citation in articleCrossrefGoogle Scholar

  • Dien, J., & Frishkoff, G. A. (2005). Principal components analysis of event-related potential datasets. In T. HandyEd., Event-related potentials: A methods handbook (pp. 189–208). Cambridge, MA: MIT Press. First citation in articleGoogle Scholar

  • Dien, J., Khoe, W., & Mangun, G. R. (2007). Evaluation of PCA and ICA of simulated ERPs: Promax vs. infomax rotations. Human Brain Mapping, 28, 742–763. https://doi.org/10.1002/hbm.20304 First citation in articleCrossrefGoogle Scholar

  • Donchin, E. (1966). A multivariate approach to the analysis of average evoked potentials. IEEE Transactions on Biomedical Engineering, 13, 131–139. https://doi.org/10.1109/TBME.1966.4502423 First citation in articleCrossrefGoogle Scholar

  • Donchin, E., & Heffley, E. F. (1978). Multivariate analysis of ERP data: A tutorial review. In D. OttoEd., Multidisciplinary Perspectives in Event-Related Brain Potential Research (pp. 555–572). Washington, DC: U.S. Government Printing Office. First citation in articleGoogle Scholar

  • Groppe, D. M., Urbach, T. P., & Kutas, M. (2011a). Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review. Psychophysiology, 48, 1711–1725. https://doi.org/10.1111/j.1469-8986.2011.01273.x First citation in articleCrossrefGoogle Scholar

  • Groppe, D. M., Urbach, T. P., & Kutas, M. (2011b). Mass univariate analysis of event-related brain potentials/fields II: Simulation studies. Psychophysiology, 48, 1726–1737. https://doi.org/10.1111/j.1469-8986.2011.01272.x First citation in articleCrossrefGoogle Scholar

  • Gupta, A. K. (2000). Matrix variate distributions. Boca Raton, FL: Chapman & Hall/CRC. https://doi.org/10.1201/9780203749289 First citation in articleGoogle Scholar

  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. Elements, 1, 337–387. https://doi.org/10.1007/b94608 First citation in articleGoogle Scholar

  • Hattori, M., Zhang, G., & Preacher, K. J. (2017). Multiple local solutions and geomin rotation. Multivariate Behavioral Research, 52, 720–731. https://doi.org/10.1080/00273171.2017.1361312 First citation in articleCrossrefGoogle Scholar

  • Hendrickson, A. E., & White, P. O. (1964). PROMAX: A quick method for rotation to oblique simple structure. British Journal of Statistical Psychology, 17, 65–70. https://doi.org/10.1111/j.2044-8317.1964.tb00244.x First citation in articleCrossrefGoogle Scholar

  • Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30, 179–185. https://doi.org/10.1007/BF02289447 First citation in articleCrossrefGoogle Scholar

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossrefGoogle Scholar

  • Jennrich, R. I. (2004). Derivative free gradient projection algorithms for rotation. Psychometrika, 69, 475–480. https://doi.org/10.1007/BF02295647 First citation in articleCrossrefGoogle Scholar

  • Jennrich, R. I. (2006). Rotation to simple loadings using component loss functions: The oblique case. Psychometrika, 71, 173–191. https://doi.org/10.1007/s11336-003-1136-B First citation in articleCrossrefGoogle Scholar

  • Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187–200. https://doi.org/10.1007/BF02289233 First citation in articleCrossrefGoogle Scholar

  • Kaiser, H. F. (1959). Computer program for varimax rotation in factor analysis. Educational and Psychological Measurement, 19, 413–420. https://doi.org/10.1177/001316445901900314 First citation in articleCrossrefGoogle Scholar

  • Kayser, J., & Tenke, C. E. (2003). Optimizing PCA methodology for ERP component identification and measurement: Theoretical rationale and empirical evaluation. Clinical Neurophysiology, 114, 2307–2325. https://doi.org/10.1016/S1388-2457(03)00241-4 First citation in articleCrossrefGoogle Scholar

  • Kayser, J., & Tenke, C. E. (2005). Trusting in or breaking with convention: Towards a renaissance of principal components analysis in electrophysiology. Clinical Neurophysiology, 116, 1747–1753. https://doi.org/10.1016/j.clinph.2005.03.020 First citation in articleCrossrefGoogle Scholar

  • Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method (2nd ed.). London, UK: Butterworths. First citation in articleGoogle Scholar

  • Luck, S. J. (2014). An introduction to the event-related potential technique (2nd ed.). Cambridge, MA: MIT Press. First citation in articleGoogle Scholar

  • Maris, E. (2004). Randomization tests for ERP topographies and whole spatiotemporal data matrices. Psychophysiology, 41, 142–151. https://doi.org/10.1111/j.1469-8986.2003.00139.x First citation in articleCrossrefGoogle Scholar

  • Marsh, H. W., Liem, G. A. D., Martin, A. J., Morin, A. J. S., & Nagengast, B. (2011). Methodological measurement fruitfulness of exploratory structural equation modeling (ESEM): New approaches to key substantive issues in motivation and engagement. Journal of Psychoeducational Assessment, 29, 322–346. https://doi.org/10.1177/0734282911406657 First citation in articleCrossrefGoogle Scholar

  • Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471–491. https://doi.org/10.1037/a0019227 First citation in articleCrossrefGoogle Scholar

  • Marsh, H. W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A. J. S., & Trautwein, U. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439–476. https://doi.org/10.1080/10705510903008220 First citation in articleCrossrefGoogle Scholar

  • McDonald, R. (1996). Path analysis with composite variables. Multivariate Behavioral Research, 31, 239–270. https://doi.org/10.1207/s15327906mbr3102 First citation in articleCrossrefGoogle Scholar

  • Möcks, J. (1988). Topographic components model for event-related potentials and some biophysical considerations. IEEE Transactions on Biomedical Engineering, 35, 482–484. https://doi.org/10.1109/10.2119 First citation in articleCrossrefGoogle Scholar

  • Möcks, J., & Verleger, R. (1986). Principal component analysis of event-related potentials: A note on misallocation of variance. Electroencephalography and Clinical Neurophysiology/Evoked Potentials, 65, 393–398. https://doi.org/10.1016/0168-5597(86)90018-3 First citation in articleCrossrefGoogle Scholar

  • Mulaik, S. A. (2010). Foundations of factor analysis (2nd ed.). Boca Raton, FL: CRC Press. First citation in articleGoogle Scholar

  • Putnick, D. L., & Bornstein, M. H. (2017). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004 First citation in articleCrossrefGoogle Scholar

  • R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. First citation in articleGoogle Scholar

  • Revelle, W. (2016). psych: Procedures for psychological, psychometric, and personality research. Evanston, IL: Northwestern University. First citation in articleGoogle Scholar

  • Sass, D. A., & Schmitt, T. A. (2010). A comparative investigation of rotation criteria within Exploratory Factor Analysis. Multivariate Behavioral Research, 45, 73–103. https://doi.org/10.1080/00273170903504810 First citation in articleCrossrefGoogle Scholar

  • Scharf, F., & Nestler, S. (2018). Principles behind variance misallocation in temporal exploratory factor analysis for ERP data: Insights from an inter-factor covariance decomposition. International Journal of Psychophysiology, 128, 119–136. https://doi.org/10.1016/j.ijpsycho.2018.03.019 First citation in articleCrossrefGoogle Scholar

  • Scharf, F., & Nestler, S. (2019a). Exploratory structural equation modeling for event-related potential data – An all-in-one approach? Psychophysiology, 56, e13303. https://doi.org/10.1111/psyp.13303 First citation in articleGoogle Scholar

  • Scharf, F., & Nestler, S. (2019b). Should regularization replace simple structure rotation in exploratory factor analysis? Structural Equation Modeling, 26, 576–590. https://doi.org/10.1080/10705511.2018.1558060 First citation in articleCrossrefGoogle Scholar

  • Scharf, F., & Nestler, S. (2019c). Should regularization replace simple structure rotation in Exploratory Factor Analysis? Structural Equation Modeling, 26, 576–590. 10.1080/10705511.2018.1558060 First citation in articleCrossrefGoogle Scholar

  • Schmitt, T. A., & Sass, D. A. (2011). Rotation criteria and hypothesis testing for Exploratory Factor Analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, 71, 95–113. https://doi.org/10.1177/0013164410387348 First citation in articleCrossrefGoogle Scholar

  • Statisticat LLC. (2016). LaplacesDemon: Complete environment for Bayesian inference. Retrieved from http://www.Bayesian-Inference.com First citation in articleCrossrefGoogle Scholar

  • Thomson, G. H. (1935). The definition and measurement of “g” (general intelligence). Journal of Educational Psychology, 26, 241–262. https://doi.org/10.1037/h0059873 First citation in articleCrossrefGoogle Scholar

  • Thurstone, L. L. (1935). The vectors of mind: Multiple-factor analysis for the isolation of primary traits. Chicago, IL: University of Chicago Press. https://doi.org/10.1037/10018-000 First citation in articleCrossrefGoogle Scholar

  • Thurstone, L. L. (1947). Multiple factor analysis. Chicago, IL: University of Chicago Press. First citation in articleGoogle Scholar

  • Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58, 267–288. First citation in articleCrossrefGoogle Scholar

  • Trendafilov, N. T. (2014). From simple structure to sparse components: A review. Computational Statistics, 29, 431–454. https://doi.org/10.1007/s00180-013-0434-5 First citation in articleCrossrefGoogle Scholar

  • Tucker, L. R. (1951). A method for synthesis of factor analysis studies. Princeton, NJ: Educational Testing Service. First citation in articleCrossrefGoogle Scholar

  • Weide, A. C., & Beauducel, A. (2019). Varimax rotation based on gradient projection is a feasible alternative to SPSS. Frontiers in Psychology, 10, 645. https://doi.org/10.3389/fpsyg.2019.00645 First citation in articleCrossrefGoogle Scholar

  • Widaman, K. F. (2007). Common factors versus components: Principals and principles, errors and misconceptions. In R. CudeckR. C. MacCallumEds., Factor analysis at 100: Historical developments and future directions (pp. 177–203). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Widaman, K. F. (2018). On common factor and principal component representations of data: Implications for theory and for confirmatory replications. Structural Equation Modeling, 25, 1–19. https://doi.org/10.1080/10705511.2018.1478730 First citation in articleCrossrefGoogle Scholar

  • Wood, C. C., & McCarthy, G. (1984). Principal component analysis of event-related potentials: Simulation studies demonstrate misallocation of variance across components. Electroencephalography and Clinical Neurophysiology, 59, 249–260. https://doi.org/10.1016/0168-5597(84)90064-9 First citation in articleCrossrefGoogle Scholar

  • Yamamoto, M., Hirose, K., & Nagata, H. (2017). Graphical tool of sparse factor analysis. Behaviormetrika, 44, 229–250. https://doi.org/10.1007/s41237-016-0007-3 First citation in articleCrossrefGoogle Scholar

  • Yates, A. (1987). Multivariate exploratory data analysis: A perspective on exploratory factor analysis. Albany, NY: State University of New York Press. First citation in articleGoogle Scholar

Florian Scharf, Department of Psychology, Statistics & Psychological Methods, University of Münster, Fliednerstraße 21, 48149 Münster, Germany, E-mail