Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Predictive Validity of the Air Force Officer Qualifying Test (AFOQT) for Pilot Performance

A Meta-Analytic Investigation at the Subtest Level

Published Online:July 14, 2020https://doi.org/10.1027/2192-0923/a000190

Abstract

Abstract. The Air Force Qualifying Test (AFOQT) has been the primary selection test battery for officer candidates in the US Air Force since 1953. Despite a wealth of literature on the validity of the AFOQT in predicting pilot performance, there is less evidence on its validity generalization. This study investigated the predictive validity of 16 AFOQT subtests and its Pilot composite via psychometric meta-analytic procedures. Based on 32 independent samples from 26 studies, results indicated that pilot performance is best predicted by subtests indicative of perceptual speed, aviation-related aptitude and knowledge, and quantitative ability constructs, and least predicted by subtests indicative of verbal ability construct. Evidence for validity generalization of AFOQT subtests is presented, and implications for practical use are discussed.

References References marked with an asterisk (*) indicate studies included in the meta-analysis.

  • Aguilar, I. D. (2017). Comparison of the Air Force Officer Qualifying Test Form T and Form S: Initial item-and subtest-level analyses (AFCAPS-TR-2017–0002). Randolph AFB, TX: Air Force Personnel Center. First citation in articleGoogle Scholar

  • ALMamari, K., & Traynor, A. (2019). Multiple test batteries as predictors for pilot performance: A meta‐analytic investigation. International Journal of Selection and Assessment, 27(4), 337–356. https://doi.org/10.1111/ijsa.12258 First citation in articleCrossref, Google Scholar

  • *Arth, T. O., Steuck, K. W., Sorrentino, C. T., & Burke, E. F. (1990). Air Force Officer Qualifying Test (AFOQT): Predictors of undergraduate pilot training and undergraduate navigator training success (AFHRL-TP-89–52). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleCrossref, Google Scholar

  • *Baker, L. E. (1989). The effect of higher education variables on cadet performance during 1987 light aircraft training (AU-ARI-88–9). Maxwell AFB, AL: Air Univ, Airpower Research Institute. First citation in articleGoogle Scholar

  • *Barron, L. G., Carretta, T. R., & Rose, M. R. (2016). Aptitude and trait predictors of manned and unmanned aircraft pilot job performance. Military Psychology, 28(2), 65–77. https://doi.org/10.1037/mil0000109 First citation in articleCrossref, Google Scholar

  • *Barron, L. G., & Rose, M. R. (2013). Relative validity of distinct spatial abilities: An example with implications for diversity. International Journal of Selection and Assessment, 21(4), 400–406. https://doi.org/10.1111/ijsa.12049 First citation in articleCrossref, Google Scholar

  • Bartram, D., & Baxter, P. (1996). Validation of the Cathay Pacific Airways pilot selection program. The International Journal of Aviation Psychology, 6(2), 149–169. https://doi.org/10.1207/s15327108ijap0602_3 First citation in articleCrossref, Google Scholar

  • *Bordelon, V. P., & Kantor, J. E. (1986). Utilization of psychomotor screening for USAF pilot candidates: Independent and integrated selection methodologies (AFHRL-TR-86–4). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleGoogle Scholar

  • Brannick, M. T., Yang, L. Q., & Cafri, G. (2011). Comparison of weights for meta-analysis of r and d under realistic conditions. Organizational Research Methods, 14(4), 587–607. https://doi.org/10.1177/1094428110368725 First citation in articleCrossref, Google Scholar

  • *Carretta, T. R. (1987a). Field dependence independence and its relationship to flight training performance(AFHRL-TP-87–36). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleCrossref, Google Scholar

  • *Carretta, T. R. (1987b). Spatial ability as a predictor of flight training performance (AFHRL-TP-86–70). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleGoogle Scholar

  • *Carretta, T. R. (1988). Relationship of encoding speed and memory tests to flight training performance (AFHRL-TP-87–49). Brooks AFB, San Antonio, TX: Air Force Human Resources Laboratory. https://doi.org/10.1037/e532992009-001 First citation in articleGoogle Scholar

  • *Carretta, T. R. (1992). Understanding the relations between selection factors and pilot training performance: Does the criterion make a difference? The International Journal of Aviation Psychology, 2(2), 95–105. https://doi.org/10.1207/s15327108ijap0202_2 First citation in articleCrossref, Google Scholar

  • *Carretta, T. R. (1997). Group differences on US Air Force pilot selection tests. International Journal of Selection and Assessment, 5(2), 115–127. https://doi.org/10.1111/1468-2389.00051 First citation in articleCrossref, Google Scholar

  • Carretta, T. R. (2000). US Air Force pilot selection and training methods (AFRL-HE-WP-TR-2000–0122). Wright-Patterson AFB, OH: Air Force Research Laboratory. First citation in articleCrossref, Google Scholar

  • *Carretta, T. R. (2005). Development and validation of the Test of Basic Aviation Skills (TBAS). Wright-Patterson AFB, OH: Air Force Research Laboratory. First citation in articleCrossref, Google Scholar

  • Carretta, T. R. (2010). Predictive validity of the Air Force Officer Qualifying Test for non-rated officer specialties. Military Psychology, 22(4), 450–464. https://doi.org/10.1080/08995605.2010.513261 First citation in articleCrossref, Google Scholar

  • *Carretta, T. R. (2011). Pilot candidate selection method: Still an effective predictor of US Air Force pilot training performance. Aviation Psychology and Applied Human Factors, 1(1), 3–8. https://doi.org/10.1027/2192-0923/a00002 First citation in articleLink, Google Scholar

  • Carretta, T. R., King, R. E., Ree, M. J., Teachout, M. S., & Barto, E. (2016). Compilation of cognitive and personality norms for military aviators. Aerospace Medicine and Human Performance, 87(9), 764–771. https://doi.org/10.3357/AMHP.4545.2016 First citation in articleCrossref, Google Scholar

  • *Carretta, T. R., & Ree, M. J. (1994). Pilot-candidate selection method: Sources of validity. The International Journal of Aviation Psychology, 4(2), 103–117. https://doi.org/10.1207/s15327108ijap0402_1 First citation in articleCrossref, Google Scholar

  • *Carretta, T. R., & Ree, M. J. (1995). Air Force Officer Qualifying Test validity for predicting pilot training performance. Journal of Business and Psychology, 9(4), 379–388. https://doi.org/10.1007/BF02230977 First citation in articleCrossref, Google Scholar

  • Carretta, T. R., & Ree, M. J. (1996). Factor structure of the Air Force Officer Qualifying Test: Analysis and comparison. Military Psychology, 8(1), 29–42. https://doi.org/10.1207/s15327876mp0801_3 First citation in articleCrossref, Google Scholar

  • Carretta, T. R., Rose, M. R., & Trent, J. D. (2016). Air Force Officer Qualifying Test Form T: Initial Item-, Test-, Factor-, and Composite-Level analyses (AFRL-RH-WP-TR-2016–0093). Wright-Patterson AFB, OH: Human Performance Wing. First citation in articleGoogle Scholar

  • *Cowan, D. K., Barrett, L. E., & Wegner, T. G. (1990). Air Force officer training school selection system validation (AFHRL-TR-89–65). Brooks AFB, TX: Air Force Human Resources Lab. First citation in articleCrossref, Google Scholar

  • Dahlke, J. A., & Wiernik, B. M. (2019). psychmeta: An R package for psychometric meta-analysis. Applied Psychological Measurement, 43(5), 415–416. https://doi.org/10.1177/0146621618795933 First citation in articleCrossref, Google Scholar

  • *Davis, R. A. (1989). Personality: Its use in selecting candidates for US Air Force undergraduate pilot training (No. AU-ARI-88–8). Maxwell AFB, AL: Airpower Research Institute, Air University. https://doi.org/10.21236/ADA271002 First citation in articleCrossref, Google Scholar

  • De Kock, F. S. (2004). The validation of the selection battery for pilots of the South African Air Force. (Master’s thesis). Retrieved from http://scholar.sun.ac.za/handle/10019.1/50198 First citation in articleGoogle Scholar

  • Drasgow, F., Nye, C. D., Carretta, T. R., & Ree, M. J. (2010). Factor structure of the Air Force Officer Qualifying Test Form S: Analysis and comparison with previous forms. Military Psychology, 22(1), 68–85. https://doi.org/10.1080/08995600903249255 First citation in articleCrossref, Google Scholar

  • *Duke, A. P., & Ree, M. J. (1996). Better candidates fly fewer training hours: Another time testing pays off. International Journal of Selection and Assessment, 4(3), 115–121. https://doi.org/10.1111/j.1468-2389.1996.tb00068.x First citation in articleCrossref, Google Scholar

  • Dye, D. A., Reck, M., & McDaniel, M. A. (1993). The validity of job knowledge measures. International Journal of Selection and Assessment, 1(3), 153–157. https://doi.org/10.1111/j.1468-2389.1993.tb00103.x First citation in articleCrossref, Google Scholar

  • Hardison, C. M., Sims, C. S., & Wong, E. C. (2010). The Air Force officer qualifying test: Validity, fairness, and bias. Santa Monica, CA: RAND Corporation, Project Air Force. https://doi.org/10.1037/e647322010–001 First citation in articleGoogle Scholar

  • Herniman, D. D. (2013). Investigating predictors of primary flight training in the Canadian Forces. (Doctoral dissertation). Retrieved from https://curve.carleton.ca First citation in articleCrossref, Google Scholar

  • Howse, W. R., & Damos, D. L. (2011). A bibliographic database for the history of pilot training selection (DAS-2011–02). Gurnee, IL: Damos Aviation Services. https://doi.org/10.21236/ADA553711 First citation in articleCrossref, Google Scholar

  • *Hunter, D. R., & Burke, E. F. (1994). Predicting aircraft pilot-training success: A meta-analysis of published research. The International Journal of Aviation Psychology, 4(4), 297–313. https://doi.org/10.1207/s15327108ijap0404_1 First citation in articleCrossref, Google Scholar

  • Hunter, D. R., & Thompson, N. A. (1978). Pilot Selection System Development (AFHRL-TR-78–33). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleCrossref, Google Scholar

  • Ingurgio, V., & Crawford, C. V. (2017). Revalidation of the selection instrument for flight training (ARI-RR-2002). Fort Belvoir, VA: Army Research Institute for the Behavioral and Social Sciences. First citation in articleGoogle Scholar

  • Johnson, J. F., Barron, L. G., Carretta, T. R., & Rose, M. R. (2017). Predictive validity of spatial ability and perceptual speed tests for aviator training. The International Journal of Aerospace Psychology, 27(3–4), 109–120. https://doi.org/10.1080/24721840.2018.1442222 First citation in articleCrossref, Google Scholar

  • Kay, G. G. (1995). CogScreen Aeromedical edition professional manual. Odessa, FL: Psychological Assessment Resources. First citation in articleGoogle Scholar

  • *Keener, R. A. (2003). Use of multivariate techniques to validate and improve the current USAF pilot candidate selection model (AFIT/GOR/ENS/03–13). Wright-Patterson AFB, OH: Air Force Institute of Tech, School of Engineering and Management. First citation in articleGoogle Scholar

  • King, R. E., Carretta, T. R., Retzlaff, P., Barto, E., Ree, M. J., & Teachout, M. S. (2013). Standard cognitive psychological tests predict military pilot training outcomes. Aviation Psychology and Applied Human Factors, 3(1), 28–38. https://doi.org/10.1027/2192-0923/a000040 First citation in articleLink, Google Scholar

  • Kock, F. D., & Schlechter, A. (2009). Fluid intelligence and spatial reasoning as predictors of pilot training performance in the South African Air Force (SAAF). SA Journal of Industrial Psychology, 35(1), 31–38. https://doi.org/10.4102/sajip.v35i1.753 First citation in articleGoogle Scholar

  • *LeMaster, W. D., & Gray, T. H. (1974). Ground training devices in job sample approach to UPT [Undergraduate Pilot Training] selection and screening. Williams AFB, AZ: Air Force Human Resources Laboratory. First citation in articleGoogle Scholar

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications. First citation in articleGoogle Scholar

  • Lynch, W. E. (1991). A Meta-analysis of pilot selection tests: Success and performance in pilot training (AFIT/GLM/LSM/91S–44). Wright-Patterson AFB, OH: Air Force Institute of Technology, School of Systems and Logistics. First citation in articleGoogle Scholar

  • Martinussen, M. (1996). Psychological measures as predictors of pilot performance: A meta-analysis. The International Journal of Aviation Psychology, 6(1), 1–20. https://doi.org/10.1207/s15327108ijap0601_1 First citation in articleCrossref, Google Scholar

  • Martinussen, M., & Torjussen, T. (1998). Pilot selection in the Norwegian Air Force: A validation and meta-analysis of the test battery. The International Journal of Aviation Psychology, 8(1), 33–45. https://doi.org/10.1207/s15327108ijap0801_2 First citation in articleCrossref, Google Scholar

  • Mount, M. K., Oh, I. S., & Burns, M. (2008). Incremental validity of perceptual speed and accuracy over general mental ability. Personnel Psychology, 61(1), 113–139. https://doi.org/10.1111/j.1744-6570.2008.00107.x First citation in articleCrossref, Google Scholar

  • Nye, C. D., Chernyshenko, O. S., Stark, S., Drasgow, F., Phillips, H. L., Phillips, J. B., & Campbell, J. S. (2018). More than g: Evidence for the incremental validity of performance‐based assessments for predicting training performance. Applied Psychology, 69(2), 302–324. https://doi.org/10.1111/apps.12171 First citation in articleCrossref, Google Scholar

  • *Olea, M. M., & Ree, M. J. (1994). Predicting pilot and navigator criteria: Not much more than g. Journal of Applied Psychology, 79(6), 845–851. https://doi.org/10.1037/0021-9010.79.6.845 First citation in articleCrossref, Google Scholar

  • Paullin, C., Katz, L. C., Bruskiewicz, K. T., Houston, J., & Damos, D. (2006). Review of aviator selection (No. TR-493). Minneapolis, MN: Personnel Decisions Research Institute. First citation in articleGoogle Scholar

  • *Ree, M. J., Carretta, T. R., & Teachout, M. S. (1995). Role of ability and prior knowledge in complex training performance. Journal of Applied Psychology, 80(6), 721–730. https://doi.org/10.1037/0021-9010.80.6.721 First citation in articleCrossref, Google Scholar

  • *Roomsburg, J. D. (1990). Utility as a function of selection ration and base rate: An empirical investigation of military aviation selection (AFIT/CI/CIA-90–004D). Wright-Patterson AFB, OH: Air Force Institute of Technology. First citation in articleGoogle Scholar

  • Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage. https://doi.org/10/b6mg First citation in articleCrossref, Google Scholar

  • Skinner, J., & Ree, M. J. (1987). Air Force Officer Qualifying Test (AFOQT): Item and Factor Analysis of Form O (AFHRL-TR-86–68). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleGoogle Scholar

  • *Siem, F. M., & Carretta, T. R. (1988). Personality, attitudes, and pilot training performance: Final analysis (No. AFHRL-TP-88–23). Brooks AFB, TX: Air Force Human Resources Laboratory. First citation in articleGoogle Scholar

  • *Stauffer, J., & Ree, M. J. (1996). Predicting with logistic or linear regression: Will it make a difference in who is selected for pilot training? The International Journal of Aviation Psychology, 6(3), 233–240. https://doi.org/10.1207/s15327108ijap0603_2 First citation in articleCrossref, Google Scholar

  • *Walters, L. C., Miller, M. R., & Ree, M. J. (1993). Structured interviews for pilot selection: No incremental validity. The International Journal of Aviation Psychology, 3(1), 25–38. https://doi.org/10.1207/s15327108ijap0301_2 First citation in articleCrossref, Google Scholar

  • Wee, S. (2018). Aligning predictor-criterion bandwidths: Specific abilities as predictors of specific performance. Journal of Intelligence, 6(3), 40. https://doi.org/10.3390/jintelligence6030040 First citation in articleCrossref, Google Scholar

  • Whitener, E. M. (1990). Confusion of confidence intervals and credibility intervals in meta-analysis. Journal of Applied Psychology, 75(3), 315–321. https://doi.org/10.1037/0021-9010.75.3.315 First citation in articleCrossref, Google Scholar

  • Zierke, O. (2014). Predictive validity of knowledge tests for pilot training outcome. Aviation Psychology and Applied Human Factors, 4(2), 98–105. https://doi.org/10.1027/2192-0923/a000061 First citation in articleLink, Google Scholar