Understanding Complexity as a Construct and as a Formally Scored Variable
Abstract
Abstract: This article serves three goals. First, I review complexity in Rorschach responding. As a construct, complexity illuminates ways people differentially register experiences, which produces distinct patterns of expressed behavior when completing the task. Rorschach first described this dimension, creating novel terminology for it, and it was central to Rapaport, Gill, and Schafer’s system and Schachtel’s classic text. As a scored variable, Viglione and Meyer defined it when they were brought together by Exner to work on advancing the Comprehensive System (CS) through his Rorschach Research Council; later it was adopted in the Rorschach Performance Assessment System (R-PAS). Second, I review early factor analytic research identifying complexity and provide new data to document how Complexity as a scored variable is an excellent index of the first unrotated principal component when factoring individually assigned Rorschach scores. Third, I document a number of assertions published about Complexity by Fontan and Andronikof that are incorrect and misleading. I correct those assertions by means of explanation and also statistical results from two data sets. I close by offering 10 basic conclusions about complexity.
At the outset of this article I describe a dimension of complexity evident in respondent behavior when completing the Rorschach task, both as a construct and as a scored variable. As a construct, I focus primarily on Hermann Rorschach’s views about this dimension of functioning, and secondarily on similar views embedded in several other systems. As a scored variable, I trace its history and document its use in clinical practice, research, and training. Second, I review some of the factor analytic research identifying its importance as the first principal component among scored Rorschach variables, which also helps to differentiate it from the number of responses (R) and proportion of Form only determinants (Lambda or F%) in a protocol. Third, I document some incorrect statements that have appeared in the published literature by Fontan and Andronikof (2021) concerning complexity as a variable and as a dimension or principal component. Subsequently, I provide new research and data analyses in order to address the last two issues with evidence. As part of this, I present new evidence showing how the Complexity variable is an excellent marker for the largest source of shared variance among scored variables. I also further illustrate and correct some of the false and misleading assertions that have been made concerning complexity.
The Complexity Construct
As a construct, complexity is a dimension of simplicity versus richness in a protocol, which can be and is observed by assessors using any scoring system or no formal scoring at all. It encompasses the extent to which respondents generate responses to the task, visually structure the elements of their responses, animate and enliven their perceptions, recognize and incorporate various inkblot features into their responses, envision a range of different types of objects in the cards, and richly communicate, among other dimensions.
Rorschach (1921/2021) considered this dimension to be so fundamental and important that he created the novel terms coarctation versus dilation to define its opposing poles (see pp. 103–105). He introduced the ideas associated with the dimension by referring to a number of clinical conditions on the simplistic, coarctated end of the spectrum (i.e., pedantic, depressed, demented, and schizophrenic) and contrasted them with conditions on the rich, dilated end of the spectrum (i.e., manic, catatonic1, and multitalented healthy individuals). Rorschach articulated how each of these types of people brought to the task their typical manner of being, repeating their usual life pattern in their inkblot responses. He characterized the coarctated type as being ruled by form, abhorring fantasy, inhibited, paralyzed, self-controlling, unable to think, unimaginative, having a paucity of ideas, affectively dull, without initiative, and barren. By contrast, Rorschach characterized the dilated type as having inner richness and a wealth of ideas. Excepting those with catatonia, he saw them as able to do anything, addicted to affective rapport, and having intensive and extensive rapport. He also said they were productive, creative, original, easy to understand, and adroit motorically. Finally, he noted the influence of these poles on the determinants that he coded at the time, which were perceiving Human Movement (M) and responding to the influence of Color (C), as opposed to giving responses defined by just their shape or form (F). Rorschach viewed the coarctated type as dominated by F with an absence of M or C, while the dilated type had an abundance of both M and C and a relative absence of F.
Rorschach (1921/2021) was thus contrasting people who give simplistic or barren protocols with few minimally communicated responses to those who give rich or elaborate protocols with many complexly articulated responses. However, he also clarified how the coarctated and dilated “types” are endpoints on a continuum of degrees, like the introverted type caps a dimension of tending toward increasing introversiveness. He also noted how M and C are indicators of the dimension, not the dimension itself. Thus, he said they just “represent (not are but represent)” this dimension, which also encompasses “other factors” (p. 106, italics in the original). This was evident from his descriptions, as well as from the ways he saw the M and C codes intersecting with the other variables he scored, which included location, content, form quality, and Originality (e.g., see his Table 8 or Table 15 and the text associated with both).
Importantly, Rorschach viewed this dimension as illustrating and dictating or determining how respondents experience life, which likely is the reason why he created novel terminology for it. With this dimension he said, “We do not know their experiences, but we do know the experience apparatus with which they receive experiences from the inside and from the outside, and to which the respondents initially subject their experiences to processing” (p. 106, italics in the original). Thus, the dimension, and its various forms of emphasis or differential weighting of functions, indicates the scope of and richness with which people register and process information. It is useful to consider this internal experiential register like other sensory abilities because it can help ground Rorschach’s unique views in more familiar terms. If this register or apparatus was vision, for instance, it would range from something akin to legal blindness (e.g., 20/200 vision) on the low end to eagle-eyed heightened acuity (e.g., 20/10 vision) on the high end.
Rorschach’s conceptualization of this simplicity–richness dimension, including the coarctated and dilated terminology, was incorporated as a crucial dimension for interpretation in some subsequent approaches to using the measure (e.g., Rapaport et al., 1968; Schachtel, 1966/2001; see also Zulliger, 1969). For instance, Rapaport et al. (1968) labeled this dimension “the qualitative wealth of the record” (p. 293), differentiating it from a dimension of quantitative productivity generating responses. For them, as for Rorschach, the poverty of a record is indicated by F, while its wealth is marked in part by M and C. However, like Rorschach, the dimension also encompasses “the number of original responses, of carefully articulated observations, and of combinations and constructions” because those variables show how the associative process “may follow a free and unstereotyped course, drawing material from many conceptual realms, elaborating upon the ideas that suggest themselves, going beyond what is merely seen, and attributing convincing and imaginative relationships to the sections of the inkblot” (p. 293). They note how healthy people with dilated records show “interesting, rich, freely variable, versatile thought and feeling life, and mobile activity,” while those with coarctated records – even if high in R – show “inhibited, rationalistic cognitive and feeling life in which there is relatively little variability or enjoyment and the realm of action is narrowed” (p. 390). In people with psychological disturbance, “the qualitatively rich protocol refers to a variability and luxurious development of symptoms; while a meager protocol refers to inhibitory, inert, and relatively colorless symptoms” (p. 390).
Schachtel (1966/2001), more so than Rapaport et al. (1968) or any other writer, amplified Rorschach’s conceptualization of this dimension. He quoted the same passage from Rorschach about the “experience apparatus” as I did above to support his conclusion that Rorschach “considered as the most important result of his test the fact that it enables us to see how a person experiences” (p. 75, italics in the original). In turn, Schachtel saw the coarctated to dilated dimension as the experiential foundation for respondents; that is, consistent with the title of his book, Experiential Foundation of Rorschach’s Test, he believed this simplicity-to-complexity dimension demonstrated how respondents experienced the inkblot task and, in parallel, life itself (e.g., see pp. 75–78).
Expressed in the most general way, the reactions of testees range all the way from a full encounter with the inkblots in which the whole personality with all its layers is engaged on a wide range of levels of functioning, resulting in a considerable variety and flexibility of experiences and responses, to an almost complete avoidance of the encounter either by rejection of the test task or, more frequently, by the mobilization of massive defenses against all but the most superficial, stereotyped, and rigidly controlled reactions. Most testees’ reactions lie somewhere between these two extremes … (p. 44)
Schachtel (1966/2001) went on to describe a “healthy” coarctated record as having “an empty, matter-of-fact, impersonal, rather stereotype quality” and noted the people giving these records “usually do not encounter the phantastic, the great, and the unknown in the world of the inkblots or in the real world, just as they do not encounter what may be buried in the depth of their own person” (p. 49). By contrast, as in their experiences of daily life, the records of dilated respondents are “able to give free play to all [their] capacities in the encounter with the inkblot before [them], thereby making available a wide range of possible perceptions and associations” where “free play is a condition for a rich, varied, and personally meaningful experience and interpretation” (p. 57).
Zulliger (1969) used only a three-card set of inkblots. Nonetheless, he described the dimension similarly, with coarctated people as “the pedants, the depressively ill-tempered individuals. They cannot be moved; they are arid, ‘wooden’ human beings” (p. 128). By contrast, dilated people “are cheerful, capable in many different areas, strong and gay by disposition. They are aware of their creative powers and enjoy them. Frequently, they are artists. They possess great vitality” (p. 129).
Despite the signature role this dimension played for Hermann Rorschach and others in understanding how people experience life, it did not retain a central role in all systems for using the Rorschach task. Rather, it seems that with the empirical and nomothetically focused systems relying on scores and norms, the focus shifted from a complex dimension of simplicity versus richness to a concern with and focus on the statistical consequences of variability in R (e.g., Cronbach, 1949; Exner, 1974; Fiske & Baughman, 1953). Rightfully, this was a salient concern, given that in early systems some respondents provided fewer than 10 responses in total (i.e., not even one to each card) and others provided more than 100 or 150 responses (i.e., 10–15 or more to each card). The concern with variability in R impairing the use of norms, as well as nomothetic research and interpretation, also likely emerged because assessors could do more to control nomothetic variability in R than they could the richness of individual responses (e.g., Exner, 1974, pp. 26–31).
The Complexity Variable
As a formally scored variable, Complexity was developed as a score for Exner’s (1974, 1986) Comprehensive System. Viglione initially approached this conceptually based on markers of guardedness versus richness (e.g., see McGuire et al., 1995; Morgan & Viglione, 1992), using variables like R, the proportion of form only (Lambda) or animal only (A%) responses, and synthetic or integrative responses (synthetic Developmental Quality, or DQ+). Meyer (1992b) initially approached this empirically from factor analytic research using many CS scores, identifying the first unrotated principal component (FUPC) as a simplicity-versus-complexity dimension of task engagement. This dimension was initially labeled “Response Articulation” and later, as it replicated in other samples using a larger array of variables, it was renamed “Response Engagement” (R-Engagement) or “Response Complexity,” with the poles described as coarctated or constricted versus dilated or engaged (Meyer, 1997; Meyer et al., 2000). Viglione and Meyer (1998) reconciled these two traditions by formulating procedures to calculate Complexity from CS scores. They did so while working together as part of Exner’s (1997) Rorschach Research Council (RRC), which was a group of seven members whom he brought together twice a year to advance the CS. (There were 11 members in total. Initial members were Thomas Boll, John Exner, Mark Hilsenroth, Gregory Meyer, William Perry, Donald Viglione, and Irving Weiner; final members were Exner, Meyer, and Viglione, as well as Phillip Erdberg, Christopher Fowler, Roger Greene, and Joni Mihura.) The algorithm they formalized in 1998 was prepared for the second meeting of the Council members. It draws on locations and DQ, determinants, contents, and R, as indicated in Table 1. Viglione and Meyer also documented that Complexity calculated this way was the best marker of the FUPC derived from CS data, with loadings on the component or simple correlations of the scale with the component of about .95 (e.g., RRC, 1999).
Exner subsequently included Complexity in the program he created to read files containing the sequence of scores from the second and third editions of the Rorschach Interpretation Assistance Program (RIAP). Later, he included it in the scoring program he developed for in-house use and sharing (but not for sale) when RIAP4 was published.2 In 1999, Exner was developing the Perceptual Thinking Index (PTI) to replace the Schizophrenia Index (SCZI; see Exner, 2000) with input from the RRC, including data Viglione provided to show how low Complexity could add meaningfully to the PTI by providing incremental validity over Form Quality and scores of disordered thought processes when assessing patients with schizophrenia (RRC, 1999). Although Exner did not add it to the PTI, Complexity was included in subsequent RRC-inspired CS research studies (e.g., Dean et al., 2007; RRC, 1999).
Separately, in an effort to make the first principal component model more accessible to and usable in clinical practice, Meyer (1999) drew on Exner’s (1993) descriptions of defensive or overly involved responding to formally define elevations and suppressions on the Response-Complexity dimension using R and the inverse of Lambda (or F%, the psychometrically superior alternative to Lambda; see Meyer et al., 2001). Meyer and Viglione (2006, 2007), based on input from other RRC members, subsequently generated CS norms stratified for Complexity, using three or four levels of R and three levels of F% as its markers. They did this using a heterogeneous clinical sample and Exner’s CS normative data (N = 450) but shifted to use the Composite International Reference Values once those were published (Meyer et al., 2007). Meyer and Viglione provided these norms, as well as visually plotted T-Score graphs, to CS users and illustrated case interpretations with them as part of training workshops provided in multiple venues in the United States, including the Annual CS Reunion Conference in Asheville, NC (Erdberg & Meyer, 2008; Meyer, 2007a), and internationally, including in Belgium (Viglione & Meyer, 2008), Brazil (Meyer, 2007d), Finland (Meyer, 2005), Italy (Meyer, 2006), The Netherlands (Meyer, 2007b, 2007c), and Sweden (Meyer, 2007e). These norms were the first Complexity Adjusted norms, and they were developed specifically to enhance CS practice.
Exner (1997) anticipated the RRC would take over developments with the CS after he retired. For instance, in his 1997 annual update to CS users, Exner (1997, p. 2) explained, “I am currently in the process of creating a Rorschach Research Council which will consist of seven accomplished researchers …” After describing its three-pronged mission to advance the CS through its elements, modified interpretation, or needed research, he concluded, “In effect, the Council will ultimately have the responsibility for the continued development of the Comprehensive System.” However, he died unexpectedly in February 2006 without leaving a mechanism in place to facilitate or guide this transition. At the time, Rorschach Workshops, which published and sold CS products, was being run by one of Exner’s sons and overseen by his wife. Several RRC members (Meyer, Viglione, Erdberg, and Mihura) wanted to continue the work they started in 1997 on Exner’s behalf to advance the CS. They discussed this possibility with both heirs until 2009 when his wife decided against further changes to the CS in order to ensure it would always reflect his lasting legacy. Faced with the options of doing nothing or pursuing an alternative system that would bring to fruition some of the improvements Exner had them developing, the Council members began working on what became the Rorschach Performance Assessment System (R-PAS; Meyer et al., 2011).
Not surprisingly, Complexity was one of the variables being investigated by the RRC that was carried over from the CS to R-PAS. As the FUPC of all assigned scores, the authors recognized its importance for influencing a range of other scores in a protocol. As such, after considering test-taking behaviors, standard R-PAS interpretation begins with Complexity and – following the tradition Meyer and Viglione initiated for the CS – users have the option of obtaining normed scores that adjust for it.
Early Factor Analytic Research
The initial factor analytic work documenting the FUPC ended up obtaining a two-dimensional solution for the subset of 27 variables considered by Meyer (1992b). These uncorrelated dimensions are illustrated in Table 2, which provides a version of the FUPC as Component 1 on the left. The numeric values are factor loadings or component coefficients and they indicate the degree of correlation between the variable and the underlying dimension. Thus, Color Blended with Shading (CBlend) has a correlation of .66 with C1, the complexity dimension.
By reviewing the pattern of coefficients for that dimension, one can see there are two conceptual elements to Complexity. In part, Complexity is a function of R (loading = .54); that is, of response fluency, ranging from few to many responses. However, it also is a function of what can be considered density or the complexity and richness of each response itself. That density can be approximated using the inverse of Lambda (loading = −.49) or F%, as Meyer and Viglione (2006, 2007) did, to identify attributions of movement (loadings from .42 to .54) or clear use of the chromatic or achromatic color, shading, or dimensionality of the inkblots (loadings from .47 to .66). At the same time, as Table 1 indicates, determinants are only a facet of density. Consistent with the simple-to-rich dimension described by Rorschach (1921/2021), Rapaport et al. (1968), and Schachtel (1966/2001), complexity as a dimension and as a formal score also intersects with and encompasses other variables, with the density component seen here also including the sophistication of visual structure in a response (S = .52, W = .46) and the range of ideas expressed within a response, as reflected in its distinct content categories (represented in a limited way in this data set through MOR = .37). Of course, other variables could also serve as markers of the density or richness of individual responses, including word counts of spontaneous speech or vocabulary level and quality.
Component 2 on the left in Table 2 is a narrowly focused dimension. It is a dimension of frequently responding (R = .76, Afr = .45) to the smaller and simpler locations of the inkblot (D = .83, Dd = .66) in a simplistic Form-dominated manner (Lambda = .58).
On the right in Table 2 are the same 27 variables but now described by two alternative perpendicular dimensions that have been rotated relative to the original ones. That rotation isolated the effects of R on Component 1a, which served to reproduce a dimension that was repeatedly obtained by earlier researchers (see Meyer, 1992a, 1992b). The rotation also accentuated the effects of Lambda and its inverse on Component 2a.
The two halves of Table 2 are alternative ways to figuratively map the same terrain. The terrain in this case is the relationship of each variable with every other variable when considered from a two-dimensional perspective. Figure 1 shows this terrain, with each variable depicted by a dot and their clustering dictated by the strength of their inter-correlations (i.e., closer variables are more positively correlated). The primary coordinates are the solid vertical and horizontal lines that meet in the center of the figure, with C1 being the vertical dimension and C2 the horizontal dimension. Those dimensions are defined by the placement of the variables in the plot and they are situated to optimally explain that placement using the simplest notation possible (when the goal is to have each variable fall as close as possible to just one dimension or the other while maximizing the distances of all variables from the center point of .00). Only three of the variables are labeled. However, the dimensions can be read as a set of coordinates to find each of them. For instance, Figure 1 shows that both R and Lambda fall between and thus help define both C1 and C2, with them falling at opposite ends of the vertical C1 dimension but at the same end of the horizontal C2 dimension. Table 2 indicates more specifically that R is positioned .54 units up on C1 and .76 units out on C2, while Lambda is .49 units down on C1 and .58 units out on C2.
However, the position of the variables in two dimensions can be explained by two perpendicular dimensions residing at any location, like spokes in a wheel. For instance, the light dotted axes map the same terrain but do so at a 45° angle to the original dimensions, forming C1a and C2a. This can be envisioned as turning the original axes by 45° or, given the reversal of sign for C2a, as if the original vertical line was first rotated 45° to the right and then 45° to the left. The value of this rotation is that it isolates the two primary markers of complexity; R and the inverse of Lambda (or F%), as each of those variables now has the largest defining coefficient on its own dimension. Thus, when complexity is parsed apart, R and the opposite of Lambda (or F%) are the primary variables that emerge. Lambda is now on the negative end of the dimension rather than the positive end as before. Dimensions often change sign with rotation, which mathematically is simply multiplying coefficients by −1.
Note also that Table 2 and Figure 1 depict a two-component solution. If instead just the FUPC had been extracted, there would not be a horizontal axis in Figure 1. All the data points would essentially move to the center and reside on the vertical line where C2 is .00. However, some of the coefficients also would become larger; they become somewhat smaller by the presence of the second component. Given the important role that the coarctation-to-dilation dimension has influencing scores derived from the inkblot task, the first aim of this article was to provide an overview of the history of this concept from its beginnings with Hermann Rorschach to its currently scored format. A second aim was to provide an overview of the factor analytic foundation for complexity. Subsequently, I return to this second aim with new data on its currently scored format, using the R-PAS norms to provide previously unpublished data on the analyses completed by the R-PAS authors in order to evaluate how well the conceptually specified Complexity variable replicates the empirical FUPC, and thus how well Complexity works as a marker of the coarctation–dilation dimension. The third aim of this article was to present and correct some inaccurate statements Fontan and Andronikof (2021) published about Complexity. I address that topic next in order to present the statements and offer some logical rebuttals, after which I turn to data to further illustrate the inaccuracy of their statements.
Inaccurate Information About Complexity
Sensitivity of Complexity to Genuine Individual Differences
Early in their article, Fontan and Andronikof (2021) suggest Complexity is insensitive to genuine differences in response complexity. They state, “the same weight is given for a simple Whole response on cards I or V (e.g., Bat) and for highly synthetic responses on Card X (e.g., a firework in Paris above the Eiffel Tower)” (p. 27). As Table 1 indicates, the first example would receive LSO = 1, Cont = 0, and Det = 0 (most likely), while the second example would receive LSO = 3, Cont = 2 (at least), and Det = 2 (likely at least), making Complexity scores of about 1 versus 7, respectively. Those weights are not nearly the same.
Tautologies and Complexity
Fontan and Andronikof (2021) assert that Complexity is “problematic on conceptual grounds” (p. 28) by quoting my 1992b article where I describe Component 1 in Table 2 by saying that the “response articulation factor [is] a factor that reflects the somewhat tautological3 position that frequent responding leads to increased scoring across all determinant categories” (Meyer, 1992b, p. 124). However, Fontan and Andronikof change what I said into an illogical statement, creating a straw man fallacy, which occurs when someone restates an argument in a way that makes it easy to prove illogical. They say my statement:
Can be generalized to “frequent responding leads to increased scoring across all Rorschach count variables.” From a logical point of view, this statement corresponds to the following implication: if (A) R increases then (B) the total number of codes in a protocol will increase. (p. 24, italics added)
The italicized phrases in the quote indicate when the authors transform what I said into something else. The result is a final statement that is true if R increases and there is just one more count code in a protocol and it is false only if R increases but the total number of count codes does not. Fontan and Andronikof state that the latter is impossible, which of course it is. R is a count variable. By definition, one more R will lead to an increase in the total number of count codes in a protocol. Thus, their statement is a genuine tautology.
However, they return to my original statement rather than their notably modified actual argument to conclude that it, as well as Complexity itself, is unfalsifiable and unscientific (Fontan & Andronikof, 2021):
Consequently, the statement “frequent responding leads to increased scoring across all determinants categories” cannot be proven false: it is unfalsifiable. According to Popper (1959), unfalsifiable statements are unscientific. It follows that Complexity one of the most important components of R-PAS interpretation might be unscientific. (pp. 24–25)
They thus end with a conclusion about Complexity and determinant use even though their argument was about R and an increase in any of the count codes in a protocol. This is illogical and it misleads readers, particularly when the illogicality is attributed to Meyer (1992b).
Component 1 in Table 2 had already falsified the argument that all codes or even all determinants increase with increases in R or complexity. Form is a determinant count code and Component 1 shows that the relative frequency of Form declines as one increases in complexity, or as one increases in R on the complexity dimension. This has been documented statistically since 1992 when those data were published. It has been documented conceptually since 1921 when Rorschach defined markers of the coarctated–dilated dimension as M and C as opposed to F.
Random Coding and Complexity
Data presented by Fontan and Andronikof (2021) purport to show how Complexity scores from a sample of nonpatients did not assess a genuine construct because scores from “random data” in “randomly scored protocols” (p. 29) showed a very strong correlation of r = .82 with real Complexity scores. Because this finding was “corroborated empirically” (p. 29), Fontan and Andronikof conclude Complexity is a tautological statistical artifact with a confirmed unscientific status that should not be interpreted as a general personality dimension. They say, “the ‘Rorschach 1st factor’ is tautological. It follows that this notion cannot be refuted and thus it is unscientific according to Karl Popper’s criteria (1959), as Complexity findings can be replicated even with random and meaningless data” (p. 32). They add, “strictly speaking, this means that the ‘Rorschach 1st factor’ simply does not exist” (p. 33). They conclude by saying:
Basically, our results demonstrate that the Complexity score has little to do with the properties of the responses given by people taking the test. Consequently, we highly recommend for psychologists using R-PAS not to draw any clinical inferences based on Complexity scores. (p. 33)
I will demonstrate that the FUPC does in fact exist and that real Complexity scores – the ones quantifying properties associated with real people – are statistically unrelated to Fontan and Andronikof’s randomized version of Complexity. Also, I suspect Hermann Rorschach would probably be surprised to hear that the coarctated-versus-dilated dimension of functioning he named, described, and considered central to the results of the task “simply does not exist.”
Complexity in R-PAS Versus the CS
Fontan and Andronikof (2021) say all the problems they identify apply to R-PAS but do not apply to the CS, “as the CS was not designed with a dimensional approach and does not include the Complexity model, the issue raised in this paper impacts R-PAS but not the CS” (p. 33). However, it is illogical to believe a dimension ranging from coarctated and simple to dilated and complex influences the responses of people assessed by R-PAS (as well as people assessed by Rorschach; Rapaport, Gill, and Schafer; and Schachtel) but not people assessed by the CS. Indeed, the CS is the very system for which Complexity was developed. To demonstrate this point, I will provide data to show that the FUPC is larger in the CS than it is in R-PAS. This is because R-PAS administration dramatically reduces variability in R relative to the CS. Because R is part of Complexity, Complexity also is less variable in R-PAS than the CS (Meyer et al., 2011; Pianowski et al., 2016, 2021). With R and Complexity more variable in the CS than R-PAS, the FUPC also is larger, such that any problems that may exist from Complexity or the FUPC actually affect the CS more than they affect R-PAS.
Factor Analytic Decisions and Complexity
Fontan and Andronikof (2021) argue that Complexity, as an index of the FUPC, reflects an extreme form of reductionism and a problematic underextraction of components for a correct model of R-PAS (or CS) data. For instance, they assert:
It is very concerning that (1) all Rorschach raw count variables are reduced to a single dimension in R-PAS, which is an extreme form of reductionism and (2) that this decision is not supported by any statistical argument in R-PAS manual… Given these two points, R-PAS Complexity model very likely represents an extreme form of underextraction, which is a severe problem, even if the tautological issue of Complexity was not addressed. (p. 34)
R-PAS does not reduce all count variables to a single dimension. To the contrary, two Profile Pages display 60 scores central to interpretation, including 27 raw count variables (e.g., R, Pr, SR, An, Sy, P, M). Thus, almost half (45%) of the key interpretive variables are raw counts interpreted on their own, not as part of any overarching dimension.
With respect to the statistical argument for extracting the FUPC, Fontan and Andronikof (2021, p. 28) actually quote the argument provided in the R-PAS manual. The full quote is (Meyer et al., 2011):
Complexity is the best marker of the “first factor” of the Rorschach, which emerges consistently in factor analytic research. As such, it is the variable that defines the biggest source of variability in the test; through its links to other codes, it is the most important thing that makes one person look different from another person. (p. 319)
Meyer et al. (2011) provide the same rationale later when describing the statistical reason for extracting the FUPC and correlating Complexity with it:
Complexity has a very high correlation with the first factor among Rorschach scores, so that it represents a relatively easily understandable marker for it … To illustrate this we completed a principal components analysis in the normative reference sample (N = 640) using sums of all of the individually assigned codes … and extracted the first component. The correlation between the first factor and Complexity was .95. (pp. 442–443)
If a researcher’s goal is to know and identify the FUPC, like the g factor of intelligence (e.g., Lubinski, 2004) or the p factor of psychopathology (e.g., Caspi et al., 2014), there is never a problem with extracting only that first dimension. Further, it is well known that broad FUPC dimensions can meaningfully give rise to lower-order dimensions, such as the Cattell–Horn–Carroll model of cognitive functioning with g at its apex (Schneider & McGrew, 2018), the Hierarchical Taxonomy of Psychopathology (HiTOP; Kotov et al., 2017) model of psychopathology with p at its apex, or the hierarchical model of personality that yields a 4- or even 7-level model with general adaptive fitness at its apex (e.g., Goldberg, 2006). Like our analyses with Complexity, all of these models have at their apex a meaningful-on-its-own FUPC. Indeed, Ree et al. (2015) have urged researchers to focus attention on recognizing the pervasiveness and importance of only the FUPC across a range of psychological constructs.
As Goldberg and Velicer (2006) note, the FUPC has useful properties when the variables being analyzed serve as markers of the same construct, such as markers of response complexity or richness. They say the FUPC “provides an index of whatever is most in common to the variables” and can “be used as a surrogate for the underlying ‘latent’ construct” with its component loadings “reflecting the correlations of each variable with that construct” (p. 226).
I have argued before that underextraction is problematic when one’s aim is to correctly model what in the population is a genuine multifactor structure (e.g., see Hoelzle & Meyer, 2013). However, when one’s aim is to correctly model the largest source of variance among a set of variables, obtaining the single FUPC is exactly the right number of dimensions to extract; there is no underextraction problem. That is, when the goal is to understand the FUPC, it is impossible to engage in underextraction when extracting and examining the single FUPC. Underextraction when modeling the FUPC can only occur if a researcher extracts 0 dimensions instead of 1 and thus effectively ignores the influence of complexity on Rorschach scores.
An Earlier Source of Similarly Inaccurate Information About Complexity
Seeing the Fontan and Andronikof (2021) article was surprising because the authors had published a version of it in a different journal 5 years earlier (Fontan & Andronikof, 2016). When I learned of the 2016 publication, I wrote the first author to express alarm at its false assertions (G. Meyer, personal email to P. Fontan, February 23, 2017). Fontan insisted that he had withdrawn the submission and that he had no knowledge it had been published, for which he provided copies of his journal correspondence (P. Fontan, personal email to G. Meyer, February 27, 2017). After seeing he had tried to withdraw the article, I explained to him the primary reasons why the analyses in that article were false and misleading, saying the following (G. Meyer, personal communication to P. Fontan, March 10, 2017):
Had you ever approached me to have a serious discussion of the issues raised in your paper, I would have been happy to discuss them. What your paper claims about Complexity being an R-PAS variable is false. Complexity was created for the CS, long before there was any notion of R-PAS. What your paper documents is the serious confounding problems associated with R. When you randomly shuffle variables across R but do not randomly vary the number of responses allocated to a protocol, the correlation between the real and the random data remains high because the correlation of R is 1.0. Had you included R in [your table of results] this would have been apparent and it would have demonstrated that there is a constant across the real and the random data that is behind the seeming similarity of the other variables.
Subsequently, Phil Erdberg and I had extended discussions with Fontan about these issues at the Society for Personality Assessment convention in March 2017, which included my offer to provide him with data sets to ensure there were no lingering ambiguities about the flaws in their analyses (e.g., G. Meyer, personal communication to P. Fontan and A. Andronikof, March 17, 2017). Subsequently, Fontan successfully retracted the article and he and I, along with Anne Andronikof, Phil Erdberg, and Joni Mihura, had additional lengthy discussions touching on the issues it raised at the International Society for the Rorschach congress in July 2017. Following this, I assumed this issue was settled. That is, I believed Fontan and Andronikof maintained the revoked publication status of their 2016 article because they realized that Complexity was developed for the CS, that the reason for their seeming nonsensical results were a consequence of keeping R fixed in their “random rescoring” of protocols, that they had confused problems related to R with problems related to Complexity, and that their critiques of the way Complexity was used in R-PAS were mistaken and unfounded.
However, their 2021 article differed only minimally from what they had published and retracted in 2016. The main change was that Fontan and Andronikof (2021) indirectly acknowledged the problems we had identified with their analyses and conclusions, saying, “concerns were raised, in professional congresses, about our randomization procedure” (p. 32). In response, they conducted an alternative randomizing procedure. However, that alternative did not address the issues we had noted, as described in my email to Fontan on March 10, 2017.
In the present paper, I replicate their main randomization findings using the same data set they used. After that, I demonstrate why those results occur and show how they have nothing to do with Complexity, but everything to do with holding R constant in their random analyses. I extend the analyses by documenting the same results emerge using two weighted composite variables unrelated to Complexity, the Weighted Sum of Color (WSumC) and the Weighted Sum of Cognitive Codes (WSumCog). Finally, I provide a simple equation that shows how Complexity and their random version of it are actually unrelated once the confounding role of R is considered. The aim of these analyses is primarily to correct the scientific record, which seems particularly important given heightened concerns in psychology (e.g., Funder et al., 2014; Shrout & Rodgers, 2018) and science at large (e.g., Nosek et al., 2015) about reproducibility, questionable research practices, and publication bias. However, these results also inform broader psychometric issues about creating composite scales and forcing variables to show a correlation when in fact they are unrelated.
Method
Participants
This article uses two data sets. First, because Fontan and Andronikof’s (2021) criticisms were aimed at R-PAS, I use the adult R-PAS normative sample (Meyer et al., 2011, pp. 442–443) to illustrate the FUPC and the association of Complexity with that dimension. I also use this data set to show that the partial randomization used by Fontan and Andronikof affects WSumCog and WSumC the same way it affects Complexity. These norms consist of 640 adult protocols modeled for R-Optimized administration. That mode of administration greatly improves the proportion of protocols in an optimal range for interpretation and produces protocols with scores that are more valid than their CS counterparts (Dean et al., 2007; Pianowski et al., 2021, 2023). Protocols were from Argentina (n = 25), Belgium (55), Brazil (36), Denmark (54), Finland (44), France (47), Greece (34), Israel (26), Italy (38), Portugal (43), Romania (47), Spain (54), and the United States (137 from three subsamples). The mean age of the respondents was 37.3 years (SD = 13.4) and with a mean 13.3 years (SD = 3.6) of education. About half (55.3%) identified as female; 52.3% were married, followed by 35.5% who were single. Given the countries sampled, most participants identified as White (66.8%), followed by other or mixed race (19.4%), Hispanic (8.7%), Black (2.6%), and Asian (2.6%). Assessors were typically substantially trained and typically had collected and coded a considerable number of protocols before data collection. However, the Belgium data were collected by assessors for whom the “Rorschach administrations were their first ones” (Mormont et al., 2007, p. S27).
Fontan and Andronikof (2021) used the CS Belgian data set (Mormont et al., 2007) for their random rescoring analyses. Christian Mormont kindly shared those protocols with me for use in further research, including in the R-PAS norms (C. Mormont, personal email to G. Meyer, June 17, 2010). As such, I used those data to replicate and extend the prior randomization analyses. I also use this data set to extract the FUPC in order to compare it with the dimension from the R-PAS norms. Fontan and Andronikof (2021, p. 29) said they used 98 of the 100 cases in the data set because, “one protocol included more than 50 responses and could not be scored in CHESSSS, and one other protocol was found to be a duplicate.” However, I could not find any duplicates in the data set used for Mormont and colleagues’ (2007) published results. The closest are two cases with 17 responses, each of which have W = 7, D = 8, Dd = 2, and S = 1. However, the similarity stops there. For instance, one has Lambda = 0.70 and Afr = .31 but the other has Lambda = 0.21 and Afr = .70. Given this, and given there was no need to omit the protocol with 56 responses, the analyses reported here use the full sample of 100 cases. The mean age of these respondents was 37.0 (SD = 13.3); 55% identified as female and 100% as White; 93% had at least 12 years of education. Information about marital status was not obtained. Assessors were receiving their initial training and were in their fourth or fifth year of university study.
Measures
In both the R-PAS and Belgian samples, the FUPC was extracted using all count variables included in R-PAS, which encompassed 71 variables for R-PAS (as shown in Table 3) and 62 for the CS. CS codes were converted to their R-PAS counterparts for Sy, Vg, An, and NC. For Belgium, the CS code S was used rather than SI and SR; CT, AGC, MAH, MAP, and ODL were omitted because they are not coded; and DV2, DR2, and CON were omitted because they had just a single code present. Other measures examined include real and randomized versions of the primary variables (FUPC and Complexity, and their basic markers, R and F%) and the secondary variables (SumC, WSumC, SumCog, and WSumCog).
Procedures
For the R-PAS sample, I ran a principal components analysis on the 71 variables and extracted only the first dimension. As a single dimension, it cannot be rotated. Component scores for each case were saved using the regression method. In order to be able to compare the size of FUPCs in R-PAS and the CS, I also ran the analyses a second time using just the 61 count variables found in the Belgian data set and did the same in the Belgian data set itself.
For randomizing the Belgian data, I followed two procedures, one is Partially Random that replicates Fontan and Andronikof’s (2021) procedure and the other is Fully Random. Partially Random kept R (and R8910) fixed on a per-protocol basis. However, for each response, the coding was drawn randomly with replacement from all possible responses in the data set using the following variable sets: Location, S, DQ, Determinant(s), Content(s), FQ, Popular, Pair, DV, DR, INC, FAB, PEC, ABS, AGM, COP, MOR, PER, and HR. For instance, consistent with the procedures followed by Fontan and Andronikof (2021), for the first response of the first protocol in the actual data set, the procedure randomly selected a response from the full data set and assigned its Location code to the target response. Next, it randomly selected a response again and assigned whatever coding was present for the S variable to the target response. Next, the procedure randomly selected a response again and assigned whatever DQ coding was present to the target response. This iterative process of randomly reselecting a response and assigning its assigned code to the target response proceeded through to the HR variable. It then started over with the second response to the first protocol as the target and repeated all of the same steps, terminating only when all the responses in the data set had been randomly assigned a new set of codes. Once these codes were assigned, a version of the Complexity score was computed and all protocol-level scores were created for analysis to generate the FUPC.
The Fully Random procedure followed the same steps. However, those steps were taken after also randomizing the number of responses in a protocol using sampling with replacement. Thus, for the first protocol in the data set, the procedure randomly selected a protocol from the full data set and assigned its R to the target protocol. It then repeated the random selection and assignment process for each protocol in turn. Scores then were randomly assigned to these responses using the Partially Random steps, as were IDs for matching with the original protocols. Protocol-level scores were generated in order to again compute the FUPC.
Subsequently, the Partially Random and Fully Random protocol-level scores for R, F%, Complexity, and the FUPC were joined with the genuine scores for each respondent. The genuine scores were correlated with their Partially Random and Fully Random counterparts, anticipating strong correlations between Real and Partially Random scores but no correlations between Real (or Partially Random) and Fully Random scores. To compare correlation magnitudes, I used the formula appropriate for dependent correlations (i.e., from the same participants) with overlapping variables (Meng et al., 1992).
Complexity is a weighted composite scale, with an anchor variable, consisting of each R in a protocol, weighted by an intensity or degree factor, consisting of the sum of points obtained for the three other Complexity subcomponents. To further show that the partial randomization issue is not specific to Complexity as a construct, I use the adult R-PAS norms to demonstrate the same phenomena using two other weighted composite scales, WSumC and WSumCog. These demonstrations show how each weighted composite, WSumC and WSumCog, is tied to its anchor variable, SumC and SumCog, in the same manner that Complexity is tied to R. For these illustrations, I randomly reassigned WSumC weights to each SumC response and randomly reassigned WSumCog weights to each SumCog response, using random sampling without replacement (i.e., each relevant response was randomly assigned one of the actual weights, with the randomized weights having distributions that exactly matched the actual weights). This permits analyses correlating the actual version of each scale with its Partially Random counterpart (i.e., SumC and SumCog remained fixed and unchanged for each person). Fully Random versions of SumC and SumCog were not created because the difference between Partial and Fully Random scores is being illustrated by the previously described analyses.
Finally, to demonstrate that the real Complexity score is statistically unrelated to the partially random Complexity score, I provide semipartial correlations. The semipartial correlations indicate the contribution of Complexity to the prediction of Partially Random Complexity after controlling for the influence of R. Because R has the same contribution to both versions of Complexity, both real Complexity and Partially Random Complexity are correlated due to the part of each score that is R. But that correlation is a mirage; it is unrelated to the real Complexity scores. This is what I had explained to Fontan on March 10, 2017: When response elements are randomly allocated but R remains fixed, “the correlation between the real and the random data remains high because the correlation of R is 1.0” and that constant causes “the seeming similarity of the other variables.”
More generally, I show that a real composite has nothing to do with its partially random counterpart after accounting for the fixed anchor variable. Thus, in addition to Complexity and R, I provide parallel semipartial correlations for WSumC and SumC, as well as WSumCog and SumCog. These results could have been generated from regression equations predicting the Partially Random composite from the anchor variable and real composite. However, for simplicity, I generated the semipartial coefficients just from the correlations that will be reported by using formula 3.3.8 from Cohen et al. (2003). The equation is 
, where PR = the Partially Random composite, Real = the real composite, Anchor = the anchor variable, and - = “with.”
Results
The FUPC in the R-PAS Norms
In the adult R-PAS norms, the FUPC accounted for 11.7% of the total variance among all 71 protocol-level scores and 13.2% of the variance using just the 62 variables in the Belgian data set. Table 3 provides the loadings of all 71 variables on the FUPC. As can be seen, it is defined about equally by F (inversely, Row 13) and R (Row 16), although also by Blends (Row 1), synthesized wholes (Rows 2 and 6), and embellished and interactive activity (Rows 3–5, 8, 9, 14, 17). The negative loading from F mirrors the negative loading from Lambda on this dimension from Meyer (1992b), as reported in Table 2. It also shows that all scores – or even more narrowly, all determinants – do not increase tautologically with increasing complexity. As complexity (or R) increases on this component, F codes decline. Further, the absence of salient loadings for many variables (e.g., Rows 60–71) also contradicts the assertion that increasing Complexity tautologically leads to an increase in all scores. In this data set, the FUPC and Complexity correlate at r = .94, while the FUPC correlates with R at .43 (the same as its loading in Table 3) and F% at −.70. (The R-PAS manual [Meyer et al., 2011, p. 443] reports the Complexity-FUPC correlation as .95. The actual value is .944775. By default, that displays in SPSS output as .945, which rounds to .95 when copying that value to Excel, as we did when preparing the manual text.)
The FUPC in the Belgian CS Sample and Compared to the R-PAS FUPC
In the Belgium sample, the FUPC accounted for 21.7% of the total variance among the 62 protocol-level scores. The FUPC is 1.64 times larger in this data set than in the R-PAS norms (i.e., 21.7% of total variance vs. 13.2%). This is due to differences in the standard deviation of R, which is 8.85 in the Belgian data (on M = 24.4) but only 4.69 in the R-PAS data (on M = 24.2). Thus, because R is 1.89 times more variable in CS than R-PAS data, the FUPC is 1.64 times larger in CS data. Complexity itself is not higher in the Belgian data (M = 71.32, SD = 26.79) than in R-PAS (M = 74.62, SD = 23.13), though its slightly larger standard deviation (1.16 times larger) is again due to the larger standard deviation of R in the Belgian data.
FUPC, Complexity, R, and F% Correlations in the Real and Randomized Belgian CS Data
The correlations across the Real, Partially Random, and Fully Random data sets tell a simple story (see Table 4). As expected, when R is held constant for real-case protocols and their partially random counterparts (i.e., r = 1.00), there are very strong associations between the FUPC, Complexity, and R across the Real and Partially Random data sets (r values from .82 to 1.00, M = .89). However, as expected, those associations completely disappear when the Real (and Partially Random) scores are correlated with their fully randomized counterparts (r values from −.11 to .10, M = −.03). Furthermore, as expected based on the historical conceptualization of Complexity within the CS as a variable defined both by R and by F%, both variables are correlated with the genuine FUPC and Complexity, but are themselves uncorrelated independent dimensions.
Finally, the Real and Partially Random results from the current analyses (Table 4, lower triangle) nicely replicate Fontan and Andronikof’s (2021) results (upper triangle), with the nine cross-sample convergent r values ranging from .82 to 1.0 in each set of correlations and M = .89 in our analysis and .92 in theirs. Note also how the correlation that the real R has with Complexity changes significantly (p < .00001) in magnitude going from each of the real data sets to each of the Partially Random data sets. In the current analyses, the change is from .83 to .97 and, in their analyses, the change was from .82 to .95. As important, in each analysis, the real R is more highly correlated (p < .00001) with Partially Random Complexity than the real Complexity is correlated with Partially Random Complexity. That is, two different variables, R and Complexity, have r values of .95 and .97, but two instances of the same variable, Complexity, have r values of just .82 and .82. This occurs because holding R constant in the Real and Partially Random data sets forces the Partially Random version of Complexity to be more similar to R and essentially identical to it, while also making it less similar to actual Complexity. This also is why Real F% is no longer correlated with the Partially Random FUPC and Complexity scores.
Two Other Examples of Real and Partially Random Score Correlations
In the adult R-PAS norms, SumC correlated with WSumC at r = .92, while both variables correlated with Partially Random WSumC at .95 and .87, respectively. Similarly, SumCog correlated with WSumCog at .93 and those variables correlated with Partially Random WSumCog at .94 and .88, respectively. Both sets of variables show the same pattern seen with Complexity; the randomly weighted score is notably and significantly (p < .00001) less correlated with the original weighted score than with the original unweighted anchor score. That is, Partially Random WSumC correlates less with WSumC (.87) than with SumC (.95) and Partially Random WSumCog correlates less with WSumCog (.88) than with SumCog (.94), reflecting how the anchor score remains perfectly correlated in the real and partially random data.
Real Composite Scales Show No Semipartial Correlations With Partially Randomized Scales
Table 5 provides the correlations used to generate the semipartial correlations (sr), as well as sr values and their t and p values. As the table shows, in no case are the real data for the composite (e.g., Complexity) associated with the partially randomized data for the composite (e.g., Partially Random Complexity) after accounting for the anchor scale. The seeming correlation in the fourth data column labeled “C-PRC” is merely an artifact of the methodology used to create the partially randomized data, which held constant the anchor variable (e.g., R) so that it had an r = 1.0 with its partially randomized counterpart. Thus, Table 5 indicates that the real composite scales have no correlation with their randomized versions (i.e., r = .00 in the population) once the anchor scale is accounted for.
Discussion
This article had three primary aims. One was to review how complexity in Rorschach responding was a fundamental dimension of personality and psychological experiencing first recognized by Rorschach. Much later it was identified as a key dimension evident in CS data that ultimately was operationalized as a formally scored CS variable and, much later still, adopted as a central variable in R-PAS. The second aim was to review the early principal components research that found a complexity dimension as the first unrotated principal component (FUPC) within 27 CS variables and to further update that information to show how the currently scored Complexity variable was an excellent marker of the FUPC of Rorschach data comprised of 71 R-PAS variables. The final aim was to clarify and correct some disparaging but false assertions that have been published about Complexity and the FUPC of CS or R-PAS data. The historical and statistical data presented in this article underscore 10 main points in relation to these three aims.
First, the FUPC is clearly present in the R-PAS norms (Table 3), defined by richness in determinants, perceptual structure, and concepts with salient loadings from both R and inversely F, much as it was in the component analyses that first identified a complexity dimension (see Table 2; Meyer, 1992b). As expected, Complexity was an excellent marker for this dimension, having a correlation of .94 with it.
Second, Fontan and Andronikof (2021) asserted that it is tautological that all scores will rise with increasing Complexity (or with increasing R). The negative and near-zero loadings in Tables 2 and 3 falsify this supposed tautology. However, Fontan and Andronikof (2021) defined the tautology in several different ways, while treating each definition as if it was the same. The data presented here negate the version of the tautology that says an increase in R on the complexity dimension leads to a rise in all coded determinants. The same data negate the version that says an increase in R or Complexity leads to an increase in all assigned count codes. However, the data do not negate the version of the tautology that says when R increases, the protocol gains at least one additional raw count code. That is an actual tautology; the outcome has to happen for CS or R-PAS protocols, given that R itself is a count code.
Third, not only is the FUPC real, it is larger in CS data than in R-PAS data. This means that protocol complexity is more of an issue for CS protocols than it is for R-PAS protocols, regardless of whether CS users calculate a Complexity score or overtly follow an FUPC model. The reason the FUPC is larger for the CS is because R is more variable in CS protocols than in R-PAS protocols. The problematic variability in R was a topic addressed in most of the RRC meetings (e.g., RRC, 1999, 2000a, 2000b, 2003, 2005), which led Exner (2003) to slightly modify administration instructions to encourage additional responses when R was limited or to curtail responding when R was ample. It is also what led the R-PAS authors to take even more notable steps to manage responding, which allowed them to attain the goal of reduced variability in R (Hosseininasab et al., 2019; Meyer et al., 2011). Reducing variability in R has had the expected benefit of enhancing the utility and validity of R-PAS scores over their CS counterparts (Pianowski et al., 2021, 2023). In addition, it has had the unanticipated benefit of providing two helpful new variables for interpretation related to under- and overproductive behavior (i.e., Prompts and Pulls).
Fourth, the “random protocols” or “randomly rescored protocols” (e.g., p. 27) that Fontan and Andronikof (2021) created to document the meaninglessness of Complexity were not random at all. To the contrary, there was a perfect correlation of r = 1.0 linking the real data set to its supposedly random counterpart. There is nothing random about a correlation of 1.0 across two data sets. However, as Table 4 clearly shows, when protocols really are randomly rescored, there is no association between real Complexity scores and random Complexity scores or between any real scores and any random scores.
Further, the authors should have known that what they presented as random was not really random. I told Fontan this in writing and also explained it multiple times in person; for example, “the correlation between the real and the random data remains high because the correlation of R is 1.0 … there is a constant across the real and the random data that is behind the seeming similarity of the other variables” (G. Meyer, personal email to P. Fontan, March 10, 2017). It is not clear why Fontan and Andronikof (2021) did not address this crucial issue, or fully report the between-sample correlation of 1.0 for R. Rather than address this flaw in their analyses, they published inaccurately labeled and described data. They did so while claiming proof that R-PAS was fundamentally flawed and founded on “a statistical artifact” because their analyses confirmed “the tautological nature of Complexity and its unscientific status” (p. 29), which then justified them to “highly recommend for psychologists using R-PAS not to draw any clinical inferences based on Complexity scores” (p. 33). Further, they claimed the CS was immune to all the problems they said R-PAS possessed, concluding, “the issue raised in this paper impacts R-PAS but not the CS” (p. 33). It is unclear what prompts such inaccurate and illogical statements. However, in combination, they have the applied effect of alleging impaired validity of a key R-PAS variable when that is not true and assailing the reputation of that system when that is not warranted, while fostering a confidence-inspiring illusion for CS users.
Fifth, the data presented here document how the strong observed correlation between Complexity and a partially randomized version of Complexity is not specific to Complexity as a construct or as a variable. The same outcome can be demonstrated for any weighted composite scale (e.g., Complexity) and its unweighted anchor variable (e.g., R), so long as a researcher does not randomize the anchor variable (i.e., r = 1.0 across data sets). This remains true even if the researcher does not provide readers with the 1.0 correlation between the two anchor variables or claims the data are random when they are not.
Sixth, the data presented in this article document how the seemingly strong observed correlation between Complexity and a partially randomized version of Complexity (r = .82) actually has nothing to do with Complexity statistically or psychometrically. Once the correlation of r = 1.0 between R in the actual data and the partially random data is accounted for, the mirage disappears and the seeming correlation of r = .82 between both versions of Complexity is shown to really be the equivalent of just r = .00. It is misleading to publish a correlation of r = .82 for Complexity while never providing the associated correlation of r = 1.0 for R, much less the correlation of r = .00 that is present for Complexity after accounting for R.
Seventh, as described in the Introduction, Rorschach (1921/2021) was very aware of the dimension of complexity that runs through the protocols obtained from people completing the task. He viewed this dimension as so important and crucial to understanding people and how they experience life that he devised novel terminology for this coarctated-versus-dilated dimension. It was not accidental that Meyer (1992b, 1997; Meyer et al., 2000) applied the same terms to the FUPC that emerged from the component analysis of CS data.
Eighth, and related to the last point, it was not only Rorschach (1921/2021), Viglione (e.g., McGuire et al., 1995; Morgan & Viglione, 1992), Meyer (1992b, 1997; Meyer et al., 2000), or Viglione and Meyer (1998, 2008) who recognized the importance of this dimension of responding. Rather, this dimension of simple and constricted versus rich and complex was a cornerstone for understanding how people experience themselves and the world around them for other systems (Rapaport et al., 1968; Schachtel, 1966/2001). Knowing the views of Rorschach, Rapaport et al., and Schachtel may provide readers who do not follow all of the statistical arguments made in this article with a degree of trust in the reality of this dimension of Rorschach task behavior and the even more important dimension of psychological experience and functioning behind it. As Rorschach first characterized it, this dimension provides indicators of the experience apparatus that illustrates the scope of and richness with which people register and process information. It thus has profound clinical implications for understanding clients and their experiences. Given the centrality of this dimension of functioning, everyone engaged in clinical practice with the Rorschach should also see ample evidence for this dimension with their own eyes.
Ninth, with his RRC, Exner brought six others together to join him in advancing the CS through ongoing research and biannual meetings that he hosted. By the second RRC meeting, Viglione and Meyer (1998) used this opportunity for collaboration to specify a Complexity index derived from CS scores. The formula in Table 1 has not changed in the 25 years since then and it continues to provide an excellent correlation (~.95) with the FUPC derived from summary scores of individually assigned CS or R-PAS codes. As part of their work with the RRC, Meyer and Viglione (2006, 2007) also showed how Complexity was approximated by just R and the inverse of F%, which led to CS normative data stratified by those two variables that were used in CS training workshops in the United States and multiple other countries (e.g., Viglione & Meyer, 2008). These norms were the first Complexity Adjusted norms, and they were developed specifically to enhance CS practice. This documented history concerning the development and use of Complexity contradicts any claim that Complexity is specific to R-PAS rather than the CS. It also contradicts the idea that Complexity is irrelevant for CS users or that Complexity does not influence CS scores simply because one does not calculate a complexity index.
As the 10th and final point, the history presented here should help readers understand how much R-PAS authors are indebted to John Exner. As members of the RRC, he brought them together to work with him on an ongoing basis over 9 years to identify CS limitations and strive to fix them. Were it not for him and his desire to improve Rorschach assessment, the R-PAS would not exist.
I am grateful for the helpful input on an earlier version of this article from Donald J. Viglione, Joni L. Mihura, Philip Erdberg, Luciano Giromini, and Ruam P. F. A. Pimentel. I also appreciate Alejandra Palacios Banchero, Emiliano Muzio, and Tomoko Muramatsu for their help translating the summary.
1Rorschach viewed the mental activity and richness associated with M as inversely related to physical action (Akavia, 2013; Meyer & Friston, 2022). He understood catatonia to result from profound absorption in an intensely active and rich mental life where the physical inactivity associated with high M ideation joined with the compelling vividness of high C reactivity to produce paralysis (e.g., see Rorschach, 1921/2021, p. 105).
2Exner liked the simplicity of RIAP2 or RIAP3 code sequence files and never shifted to the more complex Microsoft Access relational databases that were the foundation for RIAP4 and RIAP5. He developed the alternative program to continue using a simple file format.
3In logic, a tautology is something that is always true or true by definition; it has to be and cannot be otherwise. Examples could be the either–or statements “all birds fly or all birds do not fly” and “a = b or a ≠ b.” They can also be empty statements such as, “As you get heavier you end up weighing more.” I said the relation between R and all determinant categories via the FUPC is just somewhat tautological because it is probabilistic, not deterministic. As R increases there is a probability all determinant categories will also increase, but this is not required or certain. Some will probably increase more than others (e.g., Human Movement more than Texture, Form Dominated Color more than Formless Color), one category will almost certainly decrease (pure Form), and these rates will probably differ by person (e.g., Diffuse Shading may increase faster than Vista for person A but vice versa for person B).
References
(2013). Subjectivity in motion: Life, art, and movement in the work of Hermann Rorschach (2012–34295-000). Routledge/Taylor & Francis Group.
(2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2(2), 119–137. https://doi.org/10.1177/2167702613497473
(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed., xxviii, 703). Lawrence Erlbaum Associates Publishers.
(1949). Statistical methods applied to Rorschach scores: A review. Psychological Bulletin, 46(5), 393–429. https://doi.org/10.1037/h0059467
(2007). A Method to optimize the response range while maintaining Rorschach Comprehensive System validity. Journal of Personality Assessment, 89(2), 149–161. https://doi.org/10.1080/00223890701468543
(2008, May 22). Case 246: A 12-year old female threatening suicide, CS Reunion Rorschach Training Program, Asheville, NC.
(1974). The Rorschach: A comprehensive system (1st ed., Vol. 1). John Wiley & Sons.
(1986). The Rorschach: A comprehensive system (2nd ed., Vol. 1). John Wiley & Sons.
(1993). The Rorschach: A comprehensive system, (3rd ed., Vol. 1, pp. xxiii, 642). John Wiley & Sons.
(1997). Rorschach workshops and the future (Alumni Newsletter). Rorschach Workshops.
(2000). A primer for Rorschach interpretation. Rorschach Workshops.
(2003). The Rorschach: A comprehensive system, (4th ed., Vol. 1). John Wiley & Sons.
(1953). Relationships between Rorschach scoring categories and the total number of responses. The Journal of Abnormal and Social Psychology, 48(1), 25–32. https://doi.org/10.1037/h0059193
(2016). Complexity, pseudoscience and Rorschach Performance Assessment System. Clinical and Experimental Psychology, 2(4), 1–5. https://tinyurl.com/SemanticScholarLink
(2021). The complexity of Rorschach random protocols: A fundamental issue with the dimensional approach to the Rorschach. SIS Journal of Projective Psychology & Mental Health, 28(1), 27–37.
(2014). Improving the dependability of research in personality and social psychology: Recommendations for research and educational practice. Personality and Social Psychology Review, 18(1), 3–12. https://doi.org/10.1177/1088868313507536
(2006). Doing it all bass-ackwards: The development of hierarchical factor structures from the top down. Journal of Research in Personality, 40(4), 347–358. https://doi.org/10.1016/j.jrp.2006.01.001
(2006).
Principles of exploratory factor analysis . In S. StrackEd., Differentiating normal and abnormal personality (2nd ed., pp. 209–237). Springer Publishing Company.(2013).
Exploratory factor analysis: Basics and beyond . In J. A. SchinkaW. F. VelicerEds., Handbook of psychology: Research methods in psychology (2nd ed., Vol. 2, pp. 164–188). John Wiley & Sons.(2019). The effect of CS administration or an R-optimized alternative on R–PAS variables: A meta-analysis of findings from six studies. Journal of Personality Assessment, 101(2), 199–212. https://doi.org/10.1080/00223891.2017.1393430
(2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. https://doi.org/10.1037/abn0000258
(2004). Introduction to the special section on cognitive abilities: 100 years after Spearman’s (1904) “‘General intelligence’, objectively determined and measured.”. Journal of Personality and Social Psychology, 86(1), 96–111. https://doi.org/10.1037/0022-3514.86.1.96
(1995).
Some special issues in data analysis . In J. E. ExnerEd., Issues and methods in Rorschach research (pp. 227–250). Lawrence Erlbaum Associates.(1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111(1), 172–175. https://doi.org/10.1037/0033-2909.111.1.172
(1992a). Response frequency problems in the Rorschach: Clinical and research implications with suggestions for the future. Journal of Personality Assessment, 58(2), 231–244. https://doi.org/10.1207/s15327752jpa5802_2
(1992b). The Rorschach’s factor structure: A contemporary investigation and historical review. Journal of Personality Assessment, 59(1), 117–136. https://doi.org/10.1207/s15327752jpa5901_10
(1997). On the integration of personality assessment methods: The Rorschach and MMPI. Journal of Personality Assessment, 68(2), 297–330.
(1999). The convergent validity of MMPI and Rorschach scales: An extension using profile scores to define response and character styles on both methods and a reexamination of simple Rorschach response frequency. Journal of Personality Assessment, 72(1), 1–35. https://doi.org/10.1207/s15327752jpa7201_1
(2005, November 14). Applying current evidence and issues to Comprehensive System cases [Invited workshop]. The 10th Anniversary of the Finnish Rorschach Association for the Comprehensive System, Jyväskylä, Finland
(2006, August 24). The impact of R and Lambda on the interpretation of Comprehensive System cases [Invited workshop]. The VIII Congress of the European Rorschach Association for the Comprehensive System, Padova, Italy
(2007a, May 17). The impact of R and Lambda on the interpretation of a Comprehensive System case [Invited workshop]. CS Reunion Rorschach Training Program, Asheville, NC.
(2007b, June 27). The impact of R and Lambda on interpretation and current research issues with a look to the future [Invited workshop]. Dutch Rorschach Society, Utrecht, The Netherlands.
(2007c, June 29). An overview of current Rorschach research issues with a look to the future [Invited keynote address]. Dutch Rorschach Society, Utrecht, The Netherlands.
(2007d, July 25). Challenges in Rorschach-based assessment [Invited workshop]. III Congresso Brasileiro de Avaliação Psicológica e XII Conferência Internacional de Avaliação Psicológica (III Brazilian Congress of Psychological Assessment and XI International Conference of Psychological Assessment), João Pessoa, Brazil
(2007e, October 11). Advancing Rorschach assessment [Invited workshop]. Swedish Rorschach Associations, Stockholm, Sweden.
(2023). Understanding complexity – syntax and output [Data set]. https://osf.io/8prc4/
(2007). Toward international normative reference data for the Comprehensive System. Journal of Personality Assessment, 89(S1), S201–S216.
(2022). The active Bayesian brain and the Rorschach task. Rorschachiana, 43(2), 128–150. https://doi.org/10.1027/1192-5604/a000158
(2000). A replication of Rorschach and MMPI-2 convergent validity. Journal of Personality Assessment, 74(2), 175–215. https://doi.org/10.1207/S15327752JPA7402_3
(2006, March 23). The influence of R, Form%, R-Engagement, and Complexity on interpretive benchmarks for Comprehensive System variables [Oral presentation]. Annual meeting of the Society for Personality Assessment, San Diego, CA
(2007, March 7). The evidence base and recommended changes for the Comprehensive System [Workshop]. Annual meeting of the Society for Personality Assessment, Washington, DC
(2001). Superiority of Form% over Lambda for research on the Rorschach Comprehensive System. Journal of Personality Assessment, 76(1), 68–75.
(2011). Rorschach Performance Assessment System: Administration, coding, interpretation, and technical manual. Rorschach Performance Assessment System LLC.
(1992). Sexual disturbances, Rorschach sexual responses, and mediating factors. Psychological Assessment, 4(4), 530–536. https://doi.org/10.1037/1040-3590.4.4.530
(2007). Rorschach Comprehensive System data for a sample of 100 adult nonpatients from the Belgian French-speaking community. Journal of Personality Assessment, 89(Supplement 1), S26–S34. https://doi.org/10.1080/00223890701582871
(2015). Promoting an open research culture. Science, 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
(2023). Comparing the validity of the Rorschach Performance Assessment System (R-PAS) and Exner’s Comprehensive System (CS) to differentiate patients and nonpatients. Assessment. Advance online publication. https://doi.org/10.1177/10731911221146516
(2016). The impact of R-optimized administration modeling procedures on Brazilian normative reference values for Rorschach scores. Journal of Personality Assessment, 98(4), 408–418. https://doi.org/10.1080/00223891.2016.1148701
(2021). Does the Rorschach Performance Assessment System (R-PAS) differ from the Comprehensive System (CS) on variables relevant to interpretation? Journal of Personality Assessment, 103(1), 132–147. https://doi.org/10.1080/00223891.2019.1677678
(1968). Diagnostic psychological testing (R. R. Holt, Ed.; Revised). International Universities Press.
(2015). Pervasiveness of dominant general factors in organizational measurement. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(3), 409–427. https://doi.org/10.1017/iop.2015.16
(2021). Hermann Rorschach’s psychodiagnostics: Newly translated and annotated 100th anniversary edition (P. J. KeddyR. SignerP. ErdbergA. Schneider-StockingTrans. & Eds.). Hogrefe Publishing. (Original work published 1921)
. (1999). Notes – Rorschach Research Council meeting, October 9–10, 1999, https://r-pas.org/
. (2000a). Notes – Rorschach Research Council meeting, May 20–21, 2000. https://r-pas.org
. (2000b). Notes – Rorschach Research Council meeting, October 28–29, 2000. https://r-pas.org
. 2003 Rorschach Research Council Meeting agenda and notes, June 7–8, 2003. https://r-pas.org
. (2005). Meeting agenda, October 29–30, 2005 and file “The Impact of R and Form%, R-Engagement, and Complexity on CS Interpretive Benchmarks.” https://r-pas.org
(2001). Experiential foundations of Rorschach’s Test (1st ed.). The Analytic Press. https://www.routledge.com/Experiential-Foundations-of-Rorschachs-Test/Schachtel/p/book/9780881633542. (Original work published 1966)
(2018).
The Cattell–Horn–Carroll theory of cognitive abilities . In D. P. FlanaganE. M. McDonoughEds., Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). The Guilford Press.(2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69(1), 487–510. https://doi.org/10.1146/annurev-psych-122216-011845
(1998). Complexity Index scoring principles [Unpublished document]. California School of Professional Psychology, Alliant International University.
(2008, July 23). An international system as a solution to problems in Rorschach reliability, validity, and utility [Plenary session]. The XIXth Congress of the International Rorschach Society, Leuven, Belgium.
(1969). The Zulliger individual and group test (1st ed.). International Universities Press.
Summary
This article serves three primary goals. First, I review complexity evident in respondent behavior when completing Rorschach’s inkblot task. As a construct, complexity illuminates ways people differentially register experiences, which produces distinct patterns of expressed behavior when completing the task. Rorschach (1921/2021; see pp. 103–106) first described this dimension and identified its central importance as the experience apparatus that shows how respondents sense and process internal and external experiences. Reflecting its importance, he created novel terminology for its coarctated (i.e., underproductive, barren, simplistic, unimaginative) and dilated (i.e., highly productive, rich, complex, creative) poles. The same dimension and terms were central to Rapaport and colleagues’ (1968) system and this dimension formed the foundation for Schachtel’s (1966/2001) classic text. As a scored variable, Viglione and Meyer (1998) defined it when they were brought together by Exner (1997) to work on advancing the Comprehensive System (CS) through his Rorschach Research Council. I trace its history as a variable and document its use in CS clinical practice, research, and training, including how it eventually becoming a valuable variable in the Rorschach Performance Assessment System (R-PAS; Meyer et al., 2011).
Second, I review the early factor analytic research identifying complexity, which also helps to differentiate that dimension from two of its primary constituents, the number of responses (R) and proportion of Form only determinants (Lambda or F%) in a protocol. Subsequently, I provide new data to document how Complexity as a scored variable is an excellent index of the first unrotated principal component when factoring individually assigned Rorschach scores and thus also an excellent index of the coarctated to dilated dimension.
Third, I document some incorrect statements that Fontan and Andronikof (2021) published concerning complexity as a variable and as a dimension or principal component. These include that complexity is an illogical, unfalsifiable, unscientific tautology, that it can be reproduced with random data, and that it is a problem for R-PAS but not the CS. I show how each of these assertions (and others) are incorrect using logic and explanation, much as I had done in 2017 when the authors published and then retracted a very similar article. Now, I also use statistical results from two data sets to show how their arguments and analyses are flawed and unwarranted. I close by offering 10 basic conclusions about complexity.
Résumé
Cet article répond à trois objectifs principaux. Premièrement, j'examine la complexité évidente dans le comportement des répondants lorsqu'ils accomplissent la tâche des taches d'encre du Rorschach. En tant que concept, la complexité met en lumière les façons dont les gens enregistrent différemment les expériences, ce qui produit des modèles distincts de comportement exprimé lors de l'accomplissement de la tâche. Rorschach (1921/2021 ; voir pp. 103-106) a été le premier à décrire cette dimension et à identifier son importance centrale en tant qu'appareil d'expérience qui montre comment les répondants perçoivent et traitent les expériences internes et externes. Reflétant son importance, il a créé une nouvelle terminologie pour ses pôles coarté (c'est-à-dire sous-productif, stérile, simpliste, sans imagination) et dilaté (c'est-à-dire hautement productif, riche, complexe, créatif). La même dimension et les mêmes termes étaient au cœur du système de Rapaport, Gill et Schafer (1968) et cette dimension a constitué la base du texte classique de Schachtel (1966/2001). En tant que variable cotée, Viglione et Meyer (1998) l'ont définie lorsqu'ils ont été réunis par Exner (1997) pour travailler sur l'avancement du Système Intégré (SI) dans le cadre de son Rorschach Research Council. Je retrace son histoire en tant que variable et je documente son utilisation dans la pratique clinique, la recherche et la formation du SI, y compris la façon dont elle est éventuellement devenue une variable importante dans le Rorschach Performance Assessment System (R-PAS ; Meyer et al., 2011).
Deuxièmement, je passe en revue les premières recherches par analyse factorielle identifiant la complexité, ce qui permet également de différencier cette dimension de deux de ses principaux constituants, le nombre de réponses (R) et la proportion de déterminants de pure forme (Lambda ou F%) dans un protocole. Ensuite, je fournis de nouvelles données pour documenter comment la Complexité, en tant que variable cotée, est un excellent indice de la première composante principale non-rotative lors de la factorisation de codes du Rorschach individuellement attribués, et donc aussi un excellent indice de la dimension coartée à dilatée.
Troisièmement, je documente certaines affirmations incorrectes que Fontan et Andronikof (2021) ont publiées concernant la complexité en tant que variable, et en tant que dimension ou composante principale. Il s'agit notamment de l'affirmation que la complexité est une tautologie illogique, non-falsifiable et non scientifique, qu'elle peut être reproduite avec des données aléatoires, et qu'elle est un problème pour le R-PAS, mais pas pour le SI. Je montre comment chacune de ces affirmations (et d'autres) est incorrecte en utilisant logique et explication, un peu comme j'avais fait en 2017, lorsque les auteurs ont publié puis rétracté un article très similaire. Maintenant, j'utilise également les résultats statistiques de deux séries de données pour montrer comment leurs arguments et analyses sont erronés et injustifiés. Je termine en proposant 10 conclusions de base sur la complexité.
Resumen
Este artículo tiene tres objetivos principales. En primer lugar, he revisado la complejidad evidente en el comportamiento de los evaluados al completar la tarea de las manchas de tinta Rorschach. Como constructo, la complejidad ilustra formas diferenciadas de como las personas registran las experiencias, lo que produce distintos patrones de comportamiento expreso al completar la tarea. Rorschach (1921/2021; ver pp. 103-106) es el primero que describió esta dimensión e identificó su importancia fundamental como el aparato de experiencia que muestra cómo los evaluados perciben y procesan las experiencias internas y externas. Mostrando su importancia, creó una terminología novedosa para sus polos, el coartado (es decir, poco productivo, estéril, simplista, poco imaginativo) y el dilatado (es decir, altamente productivo, rico, complejo, creativo). La misma dimensión y términos fueron fundamentales para el sistema de Rapaport, Gill y Schafer (1968) y esta dimensión formó la base para el texto clásico Schachtel (1966/2001). Como variable de puntuación, Viglione y Meyer (1998) la definieron cuando fueron reunidos por Exner (1997) para trabajar en el avance del Sistema Comprehensivo (SC) en su Consejo de Investigación Rorschach. Tracé su historia en el SC como variable y documenté su uso en la práctica clínica, la investigación y la formación, incluyendo en cómo eventualmente se ha convertido en una variable valiosa en el Sistema Rorschach de Evaluación del Desempeño (R-PAS; Meyer et al, 2011).
En segundo lugar, revisé las primeras investigaciones de análisis factorial que identificaban la complejidad, lo que también ayudó a diferenciar esa dimensión de dos de sus componentes principales, el número de respuestas (R) y la proporción de Forma (Lambda o F%) como único determinante en un protocolo. Posteriormente, proporcioné nuevos datos para documentar cómo la Complejidad, como variable de puntuación, es un excelente índice del primer componente principal no rotado cuando se factorizan las puntuaciones Rorschach asignadas individualmente y, por tanto, también un excelente índice de la dimensión coartada a dilatada.
En tercer lugar, documenté algunas afirmaciones incorrectas que Fontan y Andronikof (2021) publicaron sobre la complejidad como variable y como dimensión o componente principal. Estas incluían que la complejidad es una tautología ilógica, infalsificable y no científica, que puede ser reproducida con datos aleatorios, y que es un problema para el R-PAS pero no para el SC. Mostré cómo cada una de estas afirmaciones (y otras) son incorrectas, utilizando la lógica y la explicación, de forma muy similar a como lo hice en el 2017, cuando los autores publicaron y luego se retractaron de un artículo muy similar. Ahora, también utilizo resultados estadísticos de dos conjuntos de datos para demostrar cómo sus argumentos y análisis son erróneos e injustificados. Cierro ofreciendo 10 conclusiones básicas sobre la complejidad.
要 約
この論文の主要な目的は3つある。まず、ロールシャッハのインクブロット課題に答える人の行動に見られる複雑性について検討する。複雑性とは人々が様々な経験を銘記する方法を示す構成概念であり、その結果、課題を完了する際に異なる表現行動パターンが生じる。ロールシャッハ(1921/2021; p. 103-106参照) は、この次元を最初に説明し、回答者が内外の経験をどのように感知し、処理するかを示す経験装置としての中心的な重要性を指摘した。その重要性を反映して彼は、圧縮（すなわち、生産性の低さ、不毛、単純、想像力がない）、および拡張（すなわち、生産性の高さ、豊かさ、複雑さ、創造的）という両極の新しい用語を作った。同じ次元と用語は、ラパポートらが作ったシステムの中心であり、シャハテル (1966/2001)の古典的テキストの基礎を形作っていった。スコア化された変数として、Viglione とメイヤー（1998）は、エクスナー（1997）が彼のRorschach Research Council を通じて、包括システム（CS）を進める作業をするために引き合わされた時にそれを定義しました。私は変数としてのその歴史をたどり、最終的にR-PAS（Meyer et al., 2011)の貴重な変数になった経緯を含む、CS臨床実践、研究、トレーニングにおけるその使用について記す。
2つ目に、複雑性を同定した初期の因子分析的研究をレビューする。このレビューはその次元について、複雑性を構成する主要な要素のうちの2つ、すなわち反応の数とプロトコル内の形態のみの決定要因（ラムダまたはF%）を区別するにも役に立つものであった。次にスコア化された変数としての複雑性が、個々に割り当てられたロールシャッハのスコアを因数分解した時に、回転していない第1主成分の優れた指標となり、そして圧縮から拡張への次元を示す新しいデータを提供する。
3つ目に私は、フォンタンとアンドロニコフ（2021）が発表した変数としての複雑性、次元や主成分としての複雑性に関するいくつかの誤った記述について述べる。これらには、非論理的で、反証不可能であり、非科学的であること、ランダムなデータで再現できること、R-PASの問題ではあるがCSには関係しないこと、などが含まれる。私は、2017年に著者が非常に類似した論文を発表し、その後撤回したのと同様に、論理と解釈を使って、これらの主張（及びその他の主張）のそれぞれがいかに間違っているのかを示す。今回は、2つのデータセットからの統計結果も使用し、彼らの主張と分析にいかに欠陥があり、根拠がないかを示している。そして最後に複雑性に関する10の基本的な結論を提示する。
