Classification Accuracy of Neural Networks vs. Discriminant Analysis, Logistic Regression, and Classification and Regression Trees
Abstract
Abstract. This paper compares the predictive accuracy of three commonly used parametric methods for group classification, linear discriminant analysis, quadratic discriminant analysis, and logistic regression, with two less common approaches, neural networks and classification and regression trees. The simulation study examined the impact of such factors as inequality of covariance matrices, distribution of predictors, and group size ratio (among others) on the performance of each method. Results indicate that quadratic discriminant analysis always performs as well as the other methods while neural networks behave very similarly to linear discriminant analysis and logistic regression.
References
Adebi, J. (1991). Predicting graduate academic success from undergraduate academic performance: A canonical correlation study. Educational and Psychological Measurement, 51, 151– 160Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken, NJ: WileyArminger, G. , Enache, D. , Bonne, T. (1997). Analyzing credit risk data: A comparison of logistic discrimination, classification tree analysis, and feedforward networks. Computational Statistics, 12, 293– 310Baird, L.L. (1975). Comparative prediction of first year graduate and professional school grades in six fields. Educational and Psychological Measurement, 35, 941– 946Balakrishnan, P.V. , Cooper, M.C. , Jacob, V.S. , Lewis, P.A. (1994). A study of the classification capabilities of Neural Networks using unsupervised learning: A comparison with K-means clustering. Psychometrika, 59, 509– 525Baron, A.E. (1991). Misclassification among methods used for multiple group discrimination: The effects of distributional properties. Statistics in Medicine, 10, 757– 766Breiman, L. , Friedman, J.H. , Olshen, R.A. , Stone, C.J. (1984). Classification and regression trees . Monterey, CA: Wadsworth & Brooks/ColeByers-Gonzalez, J.M. , DesJardins, S.L. (2002). Artificial Neural Networks: A new approach to predicting application behavior. Research in Higher Education, 43, 235– 258Dey, E.L. , Astin, A.W. (1993). Statistical alternatives for studying college student retention: A comparative analysis of logit, probit, and linear regression. Research in Higher Education, 34, 569– 581Fan, X. , Wang, L. (1999). Comparing linear discriminant function with logistic regression for the two-group classification problem. The Journal of Experimental Education, 67, 265– 286Ferrer, A.J.A. , Wang, L. (1999, April). Comparing the classification accuracy among nonparametric, parametric discriminant analysis and logistic regression methods . Paper presented at the annual meeting of the American Educational Research Association, Montreal, CanadaFidell, S. , Pearsons, K. , Tabachnick, B. , Howe, R. , Silvati, L. , Barber, D.S. (1995). Field study of noise-induced sleep disturbance. Journal of the Acoustical Society of America, 98, 1025– 1033Finch, W.H. , Schneider, M.K. (2005). Misclassification rates for four methods of group classification: Impact of predictor distribution, covariance inequality, effect size, sample size, and group size ratio. Educational and Psychological Measurement, 66, 240– 257Garson, G.D. (1998). Neural networks: An introductory guide for social scientists . London: SageHemphill, J.F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, 78– 80Hess, B. , Olejnik, S. , Huberty, C.J. (2001). The efficacy of two improvement-over-chance effect sizes for two-group univariate comparisons under variance heterogeneity and nonnormality. Educational and Psychological Measurement, 61, 909– 936Johnson, R.A. , Wichern, D.W. (2002). Applied multivariate statistical analysis (5th ed.). Upper Saddle River, NJ: Prentice HallMcClachlan, G.J. (1992). Discriminant analysis and statistical pattern recognition . New York: WileyMarshall, D.B. , English, D.J. (2000). Neural network modeling of risk assessment in child protective services. Psychological Methods, 5, 102– 124Meshbane, A. , Morris, J.D. (1996, April). Predictive discriminant analysis versus logistic regression in two-group classification problems . Paper presented at the annual meeting of the American Educational Research Association, New YorkRemus, W. , Wong, C. (1982). An evaluation of five models for the admission decision. College Student Journal, 16, 53– 59Richmond, R. (1992). Discriminating variables among psychotherapy dropouts from a psychological training clinical. Professional Psychology: Research and Practice, 23, 123– 130Schwarzer, G. , Vach, W. , Schumacher, M. (2000). On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Statistics in Medicine, 19, 541– 561Schumacher, M. , Robner, R. , Vach, W. (1996). Neural networks and logistic regression: Part I. Computational Statistics and Data Analysis, 21, 661– 682Sidak, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62, 626– 633Stevens, J. (1996). Applied multivariate statistics for the social sciences . Mahwah, NJ: ErlbaumTabachnick, B.G. , Fidell, L.S. (2000). Using multivariate statistics . New York: Pearson, Allyn, & BaconWilliams, C.J. , Lee, S.S. , Fisher, R.A. , Dickerman, L.H. (1999). A comparison of statistical methods for prenatal screening for Down's syndrome. Applied Stochastic Models in Business and Industry, 15, 89– 101Wilson, R.L. , Hardgrave, B.C. (1995). Predicting graduate student success in an MBA program: Regression versus classification. Educational and Psychological Measurement, 55, 186– 195
