Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Next article
Original Articles and Reviews

Integrating Technology in Neuropsychological Assessment

A Narrative Review of the Main Obstacles, New Developments, and Potential Issues in Assessment Through Tablets

Published Online:September 14, 2022https://doi.org/10.1027/1016-9040/a000484

Abstract

Abstract. Although neuropsychological assessments include some measures that are administered, scored, or interpreted using new technologies, most researchers in this area advocate that more technology should be integrated. The current situation in neuropsychological assessment may be conceptualized as triggering a crisis leading to a paradigm shift, as there is some resistance to adopting more technology. In this paper, the context of the present crisis in neuropsychological assessment, the main obstacles, and new developments will be discussed. An example of a new computerized assessment tool, the NIH Toolbox, is highlighted. Also addressed are potential issues: in the assessment with tablets illustrating it with the older adult population and how to ensure the compatibility of data collected through these devices within the framework of the European General Data Protection Regulation (GDPR). Recommendations for research, test development, and clinical practice are also provided.

References

  • Aarestrup, F. M., Albeyatti, A., Armitage, W. J., Auffray, C., Augello, L., Balling, R., Benhabiles, N., Bertolini, G., Bjaalie, J. G., Black, M., Blomberg, N., Bogaert, P., Bubak, M., Claerhout, B., Clarke, L., De Meulder, B., D’Errico, G., Di Meglio, A., Forgo, N., … Van Oyen, H. (2020). Towards a European health research and innovation cloud (HRIC). Genome Medicine, 12(1), 18. https://doi.org/10.1186/s13073-020-0713-z First citation in articleCrossref, Google Scholar

  • Almeida, L. S., Simões, M. R., Almiro, P. A., & Seabra-Santos, M. J. (2018, September 7). Práticas de avaliação psicológica em Portugal: Resultados gerais de inquérito realizado sobre a utilização de testes e outros instrumentos [Practice of psychological assessment in Portugal: General results of the survey on the use of tests and other instruments]. Abstract communication, Proceedings of the 10th Congress AIDAP/AIDEP: Diagnóstico e Avaliação psicológica [Psychological Diagnosis and Assessment], Coimbra, Portugal. First citation in articleGoogle Scholar

  • Arango-Lasprilla, J. C., Stevens, L., Paredes, A. M., Ardila, A., & Rivera, D. (2017). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24(4), 318–330. https://doi.org/10.1080/23279095.2016.1185423 First citation in articleCrossref, Google Scholar

  • Baethge, C., Goldbeck-Wood, S., & Mertens, S. (2019). SANRA – A scale for the quality assessment of narrative review articles. Research Integrity and Peer Review, 4, Article 5. https://doi.org/10.1186/s41073-019-0064-8 First citation in articleCrossref, Google Scholar

  • Battista, P., Salvatore, C., Berlingeri, M., Cerasa, A., & Castiglioni, I. (2020). Artificial intelligence and neuropsychological measures: The case of Alzheimer’s disease. Neuroscience & Biobehavioral Reviews, 114, 211–228. https://doi.org/10.1016/j.neubiorev.2020.04.026 First citation in articleCrossref, Google Scholar

  • Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., & Naugle, R. I. (2012). Computerized neuropsychological assessment devices: Joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Archives of Clinical Neuropsychology, 27(3), 362–373. https://doi.org/10.1093/arclin/acs027 First citation in articleCrossref, Google Scholar

  • Bilder, R. M. (2011). Neuropsychology 3.0: Evidence-based science and practice. Journal of the International Neuropsychological Society, 17(1), 7–13. https://doi.org/10.1017/S1355617710001396 First citation in articleCrossref, Google Scholar

  • Bilder, R. M., & Reise, S. P. (2019). Neuropsychological tests of the future: How do we get there from here? The Clinical Neuropsychologist, 33(2), 220–245. https://doi.org/10.1080/13854046.2018.1521993 First citation in articleCrossref, Google Scholar

  • Bleck, T. P., Nowinski, C. J., Gershon, R., & Koroshetz, W. J. (2013). What is the NIH Toolbox, and what will it mean to neurology? Neurology, 80(10), 874–875. https://doi.org/10.1212/WNL.0b013e3182872ea0 First citation in articleCrossref, Google Scholar

  • Canini, M., Battista, P., Della Rosa, P. A., Catricalà, E., Salvatore, C., Gilardi, M. C., & Castiglioni, I. (2014). Computerized neuropsychological assessment in aging: Testing efficacy and clinical ecology of different interfaces. Computational and Mathematical Methods in Medicine, 2014, 1–13. https://doi.org/10.1155/2014/804723 First citation in articleCrossref, Google Scholar

  • Chanda, K. (2016). Password security: An analysis of password strengths and vulnerabilities. International Journal of Computer Network and Information Security, 8(7), 23–30. https://doi.org/10.5815/ijcnis.2016.07.04 First citation in articleCrossref, Google Scholar

  • Choudhury, S., Fishman, J. R., McGowan, M. L., & Juengst, E. T. (2014). Big data, open science and the brain: Lessons learned from genomics. Frontiers in Human Neuroscience, 8, Article 239. https://doi.org/10.3389/fnhum.2014.00239 First citation in articleCrossref, Google Scholar

  • Cipresso, P., & Immekus, J. C. (2017). Back to the future of quantitative psychology and measurement: Psychometrics in the twenty-first century. Frontiers in Psychology, 8, Article 2099. https://doi.org/10.3389/fpsyg.2017.02099 First citation in articleCrossref, Google Scholar

  • Crutzen, R., Ygram Peters, G.-J., & Mondschein, C. (2019). Why and how we should care about the General Data Protection Regulation. Psychology & Health, 34(11), 1347–1357. https://doi.org/10.1080/08870446.2019.1606222 First citation in articleCrossref, Google Scholar

  • Dede, E., Zalonis, I., Gatzonis, S., & Sakas, D. (2015). Integration of computers in cognitive assessment and level of comprehensiveness of frequently used computerized batteries. Neurology, Psychiatry and Brain Research, 21(3), 128–135. https://doi.org/10.1016/j.npbr.2015.07.003 First citation in articleCrossref, Google Scholar

  • Duggan, E. C., Awakon, L. M., Loaiza, C. C., & Garcia-Barrera, M. A. (2019). Contributing towards a cultural neuropsychology assessment decision-making framework: Comparison of WAIS-IV norms from Colombia, Chile, Mexico, Spain, United States, and Canada. Archives of Clinical Neuropsychology, 34(5), 657–681. https://doi.org/10.1093/arclin/acy074 First citation in articleCrossref, Google Scholar

  • Engelhard, G., & Wang, J. (2014). Alternative measurement paradigms for measuring executive functions: SEM (Formative and Reflective Models) and IRT (Rasch Models). Measurement: Interdisciplinary Research and Perspectives, 12, 102–108. https://doi.org/10.1080/15366367.2014.943592 First citation in articleCrossref, Google Scholar

  • European Parliament & Council. (2016). General Data Protection Regulation [GDPR] Regulation (EU) 2016/679. https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=CELEX:32016R0679 First citation in articleCrossref, Google Scholar

  • European Parliament & Council. (2020). COMMUNICATION: Data protection as a pillar of citizens’ empowerment and the EU’s approach to the digital transition: Two years of application of the General Data Protection Regulation COM/2020/264 final. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52020DC0264&from=EN First citation in articleGoogle Scholar

  • Fox, R. S., Manly, J. J., Slotkin, J., Devin Peipert, J., & Gershon, R. C. (2021). Reliability and validity of the Spanish-language version of the NIH toolbox. Assessment, 28(2), 457–471. https://doi.org/10.1177/1073191120913943 First citation in articleCrossref, Google Scholar

  • García-Casal, J. A., Franco-Martín, M., Perea-Bartolomé, M. V., Toribio-Guzmán, J. M., García-Moja, C., Goñi-Imizcoz, M., & Csipke, E. (2017). Electronic devices for cognitive impairment screening: A systematic literature review. International Journal of Technology Assessment in Health Care, 33(6), 654–673. https://doi.org/10.1017/S0266462317000800 First citation in articleCrossref, Google Scholar

  • Gershon, R. C., Cella, D., Fox, N. A., Havlik, R. J., Hendrie, H. C., & Wagster, M. V. (2010). Assessment of neurological and behavioural function: The NIH Toolbox. The Lancet Neurology, 9(2), 138–139. https://doi.org/10.1016/S1474-4422(09)70335-7 First citation in articleCrossref, Google Scholar

  • Gershon, R. C., Wagster, M. V., Hendrie, H. C., Fox, N. A., Cook, K. F., & Nowinski, C. J. (2013). NIH Toolbox for assessment of neurological and behavioral function. Neurology, 80(11 Supplement 3), S2–S6. https://doi.org/10.1212/WNL.0b013e3182872e5f First citation in articleCrossref, Google Scholar

  • Hesse, B. W. (2018). Can psychology walk the walk of open science? American Psychologist, 73(2), 126–137. https://doi.org/10.1037/amp0000197 First citation in articleCrossref, Google Scholar

  • Hoofnagle, C. J., van der Sloot, B., & Borgesius, F. Z. (2019). The European Union general data protection regulation: What it is and what it means. Information & Communications Technology Law, 28(1), 65–98. https://doi.org/10.1080/13600834.2019.1573501 First citation in articleCrossref, Google Scholar

  • Human Brain Project. (2020). Human brain project. https://www.humanbrainproject.eu/en/ First citation in articleGoogle Scholar

  • Iliescu, D., & Greiff, S. (2019). The impact of technology on psychological testing in practice and policy: What will the future bring. European Journal of Psychological Assessment, 35(2), 151–155. https://doi.org/10.1027/1015-5759/a000532 First citation in articleLink, Google Scholar

  • Jenkins, A., Lindsay, S., Eslambolchilar, P., Thornton, I. M., & Tales, A. (2016). Administering cognitive tests through touch screen tablet devices: Potential issues. Journal of Alzheimer’s Disease, 54(3), 1169–1182. https://doi.org/10.3233/JAD-160545 First citation in articleCrossref, Google Scholar

  • Koch, C., & Jones, A. (2016). Big science, team science, and open science for neuroscience. Neuron, 92(3), 612–616. https://doi.org/10.1016/j.neuron.2016.10.019 First citation in articleCrossref, Google Scholar

  • Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press. First citation in articleGoogle Scholar

  • Kwak, S., Oh, D. J., Jeon, Y.-J., Oh, D. Y., Park, S. M., Kim, H., & Lee, J.-Y. (2022). Utility of machine learning approach with neuropsychological tests in predicting functional impairment of alzheimer’s disease. Journal of Alzheimer’s Disease, 85(3), 1357–1372. https://doi.org/10.3233/JAD-215244 First citation in articleCrossref, Google Scholar

  • Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Neuropsychological assessment (5th ed.). Oxford University Press. First citation in articleGoogle Scholar

  • Marques-Costa, C., Almiro, P. A., & Simões, M. R. (2018). Computerized cognitive tests (CCT) in elderly: A psychometric review. Revue Européenne de Psychologie Appliquée, 68(2), 61–68. https://doi.org/10.1016/j.erap.2018.04.002 First citation in articleCrossref, Google Scholar

  • Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey of North American professionals. The Clinical Neuropsychologist, 29(6), 741–776. https://doi.org/10.1080/13854046.2015.1087597 First citation in articleCrossref, Google Scholar

  • Mascalzoni, D., Bentzen, H. B., Budin-Ljøsne, I., Bygrave, L. A., Bell, J., Dove, E. S., Fuchsberger, C., Hveem, K., Mayrhofer, M. T., Meraviglia, V., O’Brien, D. R., Pattaro, C., Pramstaller, P. P., Rakic, V., Rossini, A., Shabani, M., Svantesson, D. J. B., Tomasi, M., Ursin, L., … Kaye, J. (2019). Are requirements to deposit data in research repositories compatible with the European Union’s General Data Protection Regulation? Annals of Internal Medicine, 170(5), 332–334. https://doi.org/10.7326/M18-2854 First citation in articleCrossref, Google Scholar

  • Meiland, F., Innes, A., Mountain, G., Robinson, L., van der Roest, H., García-Casal, J. A., Gove, D., Thyrian, J. R., Evans, S., Dröes, R.-M., Kelly, F., Kurz, A., Casey, D., Szcześniak, D., Dening, T., Craven, M. P., Span, M., Felzmann, H., Tsolaki, M., & Franco-Martin, M. (2017). Technologies to support community-dwelling persons with dementia: A position paper on issues regarding development, usability, effectiveness and cost-effectiveness, deployment, and ethics. JMIR Rehabilitation and Assistive Technologies, 4(1), Article e1. https://doi.org/10.2196/rehab.6376 First citation in articleCrossref, Google Scholar

  • Miller, J. B. (2019). Big data and biomedical informatics: Preparing for the modernization of clinical neuropsychology. The Clinical Neuropsychologist, 33(2), 287–304. https://doi.org/10.1080/13854046.2018.1523466 First citation in articleCrossref, Google Scholar

  • Miller, D. D., & Brown, E. W. (2018). Artificial intelligence in medical practice: The question to the answer? The American Journal of Medicine, 131(2), 129–133. https://doi.org/10.1016/j.amjmed.2017.10.035 First citation in articleCrossref, Google Scholar

  • Miller, J. B., & Barr, W. B. (2017). The technology crisis in neuropsychology. Archives of Clinical Neuropsychology, 32(5), 541–554. https://doi.org/10.1093/arclin/acx050 First citation in articleCrossref, Google Scholar

  • Mittelstadt, B. (2017). From individual to group privacy in big data analytics. Philosophy & Technology, 30, 475–494. https://doi.org/10.1007/s13347-017-0253-7 First citation in articleCrossref, Google Scholar

  • Mulder, T. (2019). Health apps, their privacy policies and the GDPR. European Journal of Law and Technology, 10(1). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3506805 First citation in articleGoogle Scholar

  • National Institutes of Health. (2020). The brain initiative. NIH. https://braininitiative.nih.gov/ First citation in articleGoogle Scholar

  • National Institutes of Health [NIH] & Northwestern University. (2016). NIH Toolbox® scoring and interpretation guide [Manual]. http://assistly-production.s3.amazonaws.com/228622/kb_article_attachments/98102/Toolbox_Scoring_and_Interpretation_Guide_for_iPad_v1.7_original.pdf?AWSAccessKeyId=AKIAJNSFWOZ6ZS23BMKQ&Expires=1531157325&Signature=F9GRT8bn%2Bsbijcg62zP%2Fcm1vuvQ%3D&response-content-disposition=filename%3D%22Toolbox_Scoring_and_Interpretation_Guide_for_iPad_v1.7.pdf%22&response-content-type=application%2Fpdf First citation in articleCrossref, Google Scholar

  • NIH Toolbox. (2020, November). Hardware requirements: iPad/iOS. https://nihtoolbox.force.com/s/article/hardware-requirements-ipadios First citation in articleGoogle Scholar

  • Noorbakhsh-Sabet, N., Zand, R., Zhang, Y., & Abedi, V. (2019). Artificial intelligence transforms the future of healthcare. The American Journal of Medicine, 132(7), 795–801. https://doi.org/10.1016/j.amjmed.2019.01.017 First citation in articleCrossref, Google Scholar

  • Olabarrieta-Landa, L., Caracuel, A., Pérez-García, M., Panyavin, I., Morlett-Paredes, A., & Arango-Lasprilla, J. C. (2016). The profession of neuropsychology in Spain: Results of a national survey. The Clinical Neuropsychologist, 30(8), 1335–1355. https://doi.org/10.1080/13854046.2016.1183049 First citation in articleCrossref, Google Scholar

  • Order of Portuguese Psychologists. (2014). Resultados do censo dos membros efectivos da OPP [Results of the full members census of the OPP]. OPP. https://www.ordemdospsicologos.pt/pt/noticia/1246 First citation in articleGoogle Scholar

  • Parsey, C. M., & Schmitter-Edgecombe, M. (2013). Applications of technology in neuropsychological assessment. The Clinical Neuropsychologist, 27(8), 1328–1361. https://doi.org/10.1080/13854046.2013.834971 First citation in articleCrossref, Google Scholar

  • Parsons, T. D., McMahan, T., & Kane, R. (2018). Practice parameters facilitating adoption of advanced technologies for enhancing neuropsychological assessment paradigms. The Clinical Neuropsychologist, 32(1), 16–41. https://doi.org/10.1080/13854046.2017.1337932 First citation in articleCrossref, Google Scholar

  • Perna, G., Grassi, M., Caldirola, D., & Nemeroff, C. B. (2018). The revolution of personalized psychiatry: Will technology make it happen sooner? Psychological Medicine, 48(5), 705–713. https://doi.org/10.1017/S0033291717002859 First citation in articleCrossref, Google Scholar

  • Pearson. (2022a). Q-interactive. https://www.pearsonassessments.com/professional-assessments/digital-solutions/q-interactive/about.html First citation in articleGoogle Scholar

  • Pearson. (2022b). Q-interactive France. https://www.pearsonclinical.fr/evaluation-numerique/q-interactive First citation in articleGoogle Scholar

  • Pearson. (2022c). Q-GLOBAL. https://www.pearsonclinical.es/digital/evaluacion-online First citation in articleGoogle Scholar

  • Piotrowski, C. (2017). Neuropsychological testing in professional psychology specialties: Summary findings of 36 studies (1990–2016) in applied settings. Journal of the Indian Academy of Applied Psychology, 43(1), 134–144. First citation in articleGoogle Scholar

  • Price, W. N., & Cohen, I. G. (2019). Privacy in the age of medical big data. Nature Medicine, 25(1), 37–43. https://doi.org/10.1038/s41591-018-0272-7 First citation in articleCrossref, Google Scholar

  • Prieto, G., Delgado, A., Perea, M., & Ladera, V. (2009). Scoring neuropsychological tests using the Rasch model: An illustrative example with the Rey-Osterrieth Complex Figure. The Clinical Neuropsychologist, 24(1), 45–56. https://doi.org/10.1080/13854040903074645 First citation in articleCrossref, Google Scholar

  • Pucchio, A. D., Vanacore, N., Marzolini, F., Lacorte, E., Fiandra, T. D., Group, I.-D., & Gasparini, M. (2018). Use of neuropsychological tests for the diagnosis of dementia: A survey of Italian memory clinics. BMJ Open, 8(3), Article e017847. https://doi.org/10.1136/bmjopen-2017-017847 First citation in articleCrossref, Google Scholar

  • Rabin, L. A., Paolillo, E., & Barr, W. B. (2016). Stability in test-usage practices of clinical neuropsychologists in the United States and Canada over a 10-year period: A follow-up survey of INS and NAN members. Archives of Clinical Neuropsychology, 31(3), 206–230. https://doi.org/10.1093/arclin/acw007 First citation in articleCrossref, Google Scholar

  • Rabin, L. A., Spadaccini, A. T., Brodale, D. L., Grant, K. S., Elbulok-Charcape, M. M., & Barr, W. B. (2014). Utilization rates of computerized tests and test batteries among clinical neuropsychologists in the United States and Canada. Professional Psychology: Research and Practice, 45(5), 368–377. https://doi.org/10.1037/a0037987 First citation in articleCrossref, Google Scholar

  • Ramprasad, C., Tamariz, L., Garcia-Barcena, J., Nemeth, Z., & Palacio, A. (2019). The use of tablet technology by older adults in health care settings – Is it effective and satisfying? A systematic review and meta analysis. Clinical Gerontologist, 42(1), 17–26. https://doi.org/10.1080/07317115.2017.1322162 First citation in articleCrossref, Google Scholar

  • Robillard, J. M., Lai, J. A., Wu, J. M., Feng, T. L., & Hayden, S. (2018). Patient perspectives of the experience of a computerized cognitive assessment in a clinical setting. Alzheimer’s & Dementia: Translational Research & Clinical Interventions, 4, 297–303. https://doi.org/10.1016/j.trci.2018.06.003 First citation in articleCrossref, Google Scholar

  • Schmand, B. (2019). Why are neuropsychologists so reluctant to embrace modern assessment techniques? The Clinical Neuropsychologist, 33(2), 209–219. https://doi.org/10.1080/13854046.2018.1523468 First citation in articleCrossref, Google Scholar

  • Seabra-Santos, M. J., Almiro, P. A., Simões, M. R., & Almeida, L. S. (2019). Testes psicológicos em Portugal: Atitudes, problemas e perfil dos utilizadores [Psychological tests in Portugal: Attitudes, problems and user profiles]. Revista Iberoamericana de Diagnóstico y Evaluación Psicológica, 4(53), 101–112. https://doi.org/10.21865/RIDEP53.4.08 First citation in articleGoogle Scholar

  • Solias, A., Chaldogeridis, A., Batzikosta, A., & Tsolaki, M. (2020). Tablet-administered screening tests for the detection of major and mild cognitive disorders: Preliminary findings of a comparative study. International Journal of Interactive Mobile Technologies, 14(11), 200–223. https://doi.org/10.3991/ijim.v14i11.14629 First citation in articleCrossref, Google Scholar

  • Thomas, M. L. (2010). The value of item response theory in clinical assessment: A review. Assessment, 18(3), 291–307. https://doi.org/10.1177/1073191110374797 First citation in articleCrossref, Google Scholar

  • Thornton, I. M., & Horowitz, T. S. (2004). The multi-item localization (MILO) task: Measuring the spatiotemporal context of vision for action. Perception & Psychophysics, 66(1), 38–50. https://doi.org/10.3758/BF03194859 First citation in articleCrossref, Google Scholar

  • Wild, K., Howieson, D., Webbe, F., Seelye, A., & Kaye, J. (2008). The status of computerized cognitive testing in aging: A systematic review. Alzheimer’s & Dementia: the Journal of the Alzheimer’s Association, 4(6), 428–437. https://doi.org/10.1016/j.jalz.2008.07.003 First citation in articleCrossref, Google Scholar

  • Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393 First citation in articleCrossref, Google Scholar

  • Yıldırım, M., & Mackie, I. (2019). Encouraging users to improve password security and memorability. International Journal of Information Security, 18(6), 741–759. https://doi.org/10.1007/s10207-019-00429-y First citation in articleCrossref, Google Scholar

  • Zanon, C., Hutz, C. S., Yoo, H. H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexão e Crítica, 29(1), 1–10. https://doi.org/10.1186/s41155-016-0040-x First citation in articleGoogle Scholar

  • Zygouris, S., & Tsolaki, M. (2015). Computerized cognitive testing for older adults: A review. American Journal of Alzheimer’s Disease & Other Dementias, 30(1), 13–28. https://doi.org/10.1177/1533317514522852 First citation in articleCrossref, Google Scholar