Abstract. An important problem in multilevel modeling is what constitutes a sufficient sample size for accurate estimation. In multilevel analysis, the major restriction is often the higher-level sample size. In this paper, a simulation study is used to determine the influence of different sample sizes at the group level on the accuracy of the estimates (regression coefficients and variances) and their standard errors. In addition, the influence of other factors, such as the lowest-level sample size and different variance distributions between the levels (different intraclass correlations), is examined. The results show that only a small sample size at level two (meaning a sample of 50 or less) leads to biased estimates of the second-level standard errors. In all of the other simulated conditions the estimates of the regression coefficients, the variance components, and the standard errors are unbiased and accurate.
Afshartous D.(1995, April). Determination of sample size for multilevel model design. . Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA Berkhof J. , Snijders T. A. B.(2001). Variance component testing in multilevel models. Educational and Behavioral Statistics, 26, 133– 152 Browne W. J.(1998). Applying MCMC methods to multilevel models . Unpublished doctoral dissertation, University of Bath, UK Browne W. J. , Draper D.(2000). Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models. Computational Statistics, 15, 391– 420 Bryk A. S. , Raudenbush S. W.(1989). Methodology for cross-level organizational research. Research in Organizational Behavior, 7, 233– 273 Busing F.(1993). Distribution characteristics of variance estimates in two-level models . Unpublished manuscript, Leiden University, the Netherlands Cohen J.(1988). Statistical power analysis for the behavioral sciences . Mahwah, NJ: Erlbaum Cohen J. , Cohen P.(1983). Applied multiple regression analysis for the behavioral sciences . Hillsdale, NJ: Erlbaum Elston D. A.(1998). Estimation of the denominator degrees of freedom of F-distributions for assessing Wald statistics for fixed-effect factors in unbalanced mixed designs. Biometrics, 41, 477– 486 Goldstein H.(1995). Multilevel statistical models . (2nd ed.). London: Edwards Arnold. New York: Halstead Goldstein H.(2003). Multilevel statistical models . (3rd ed.). London: Arnold Gulliford M. C. , Ukoumunne O. C. , Chinn S.(1999). Components of variance and intraclass correlations for the design of community-based surveys and intervention studies. American Journal of Epidemiology, 149, 876– 883 Heck R. H. , Thomas S. L.(2000). An introduction to multilevel modeling techniques . Mahwah, NJ: Erlbaum Hox J. J.(2002). Multilevel analysis: Techniques and applications . Mahwah, NJ: Erlbaum Kim K.-S.(1990). Multilevel data analysis: A comparison of analytical alternatives . Unpublished doctoral dissertation, University of California at Los Angeles Kish L.(1965). Survey sampling . New York: Wiley Klein K. , Kozlowski S. W. J.(Eds.) (2000). Multilevel theory, research, and methods in organizations . San Francisco: Jossey-Bass Kreft I. G. G.(1996). Are multilevel techniques necessary? An overview, including simulation studies . Unpublished manuscript, California State University at Los Angeles. Retrieved July 6, 2005 from www.calstatela.edu/faculty/ikreft/quarterly.html Kreft I. G. G. , De Leeuw J.(1998). Introducing multilevel modeling . Newbury Park, CA: Sage Manor O. , Zucker D. M.(2004). Small sample inference for the fixed effects in the mixed linear model. Computational Statistics and Data Analysis, 46, 801– 817 Muthén B. , Satorra A.(1995). Complex sample data in structural equation modeling. In P. V. Marsden (Ed.),Sociological methodology (pp. 267-316). Oxford, England: Blackwell Rasbash J. , Browne W. , Goldstein H. , Yang M. , Plewis I. , Healy M.(2000). A user’s guide to MLwiN . London: Multilevel Models Project, University of London Raudenbush S. W.(1989). The analysis of longitudinal, multilevel data. International Journal of Educational Research, 13, 721– 740 Raudenbush S. W. , Bryk A. S.(2002). Hierarchical linear models . Newbury Park, CA: Sage Snijders T. A. B.(1996). Analysis of longitudinal data using the hierarchical linear model. Quality and Quantity, 30, 405– 426 Snijders T. A. B. , Bosker R. J.(1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling . Thousand Oaks, CA: Sage Van der Leeden R. , Busing F.(1994). First iteration versus IGLS RIGLS estimates in two-level models: A Monte Carlo study with ML3 . Unpublished manuscript, Leiden University, the Netherlands Van der Leeden R. , Busing F. , Meijer E.(1997, April). Applications of bootstrap methods for two-level models . Paper presented at the Multilevel Conference, Amsterdam Welham S. J. , Thompson R.(1997). Likelihood ratio tests for fixed model terms using residual maximum likelihood. Journal of the Royal Statistical Society, Series B, 59, 701– 714 Yung Y.-F. , Chan W.(1999). Statistical analysis using bootstrapping: Concepts and implementation. In R. H. Hoyle (Ed.),Statistical strategies for small sample research (pp. 82-105). Thousand Oaks, CA: Sage