Development, Measurement, and Validation of a Managerial Competency Model in Spain
The PUMAC Questionnaire
Abstract
Abstract: Modernization of Public Administration (PA), to provide a high-quality service, requires improvements in managerial personnel assessment processes. For this purpose, we developed and validated a management competency model that would enable Spanish PA to enhance organizational achievements and prevent corrupt practices. Relations between managers’ competencies and their performance would be perceived as a fairer assessment of human resources processes and ultimately lead to the promotion of ethical behavior within the organization. We followed a mixed-method approach. Study 1 began with a content analysis of articles referring to managerial competencies, followed by Focus Groups and a Delphi panel procedure with subject matter experts, which resulted in a model of eight competencies. Based on this model, in Study 2, we developed a reliable and invariant-across-gender questionnaire (Public Manager Competencies Questionnaire – PUMACQ), which was validated with composite performance criteria (task, contextual, and unethical pro-organizational behavior), by means of a cross-sectional design survey on public managers from around Spain (N = 439), using structural equation analyses. Our main results show significant relationships between job performance and the competencies of leadership, communication, engagement with PA, innovation orientation, and ethics. Finally, we discuss the practical applications of the results in the public context.
References
2001). Managerial competencies and the managerial performance appraisal process. Journal of Management Development, 20(10), 842–852. https://doi.org/10.1108/0262171011041084
(1994). Relationship of job stressors to job performance: Linear or inverted-U? Psychological Reports, 75(1), 547–558. https://doi.org/10.2466/pr0.1994.75.1.547
(2019). A new civil service to strengthen confidence in public institutions. Revista Vasca de Gestión de Personas y Organizaciones Públicas, 16, 36–53.
(2017). Nuevos tiempos para la función pública: Propuestas para atraer y desarrollar el talento en la Administración General del Estado
([New times for the public function: Proposals to attract and develop talent in the General State Administration] . INAP.2003). The normalization of corruption in organizations. Research in Organizational Behavior, 25, 1–52. https://doi.org/10.1016/S0191-3085(03)25001-2
(2005). The Great Eight Competencies: A criterion-centric approach to validation. Journal of Applied Psychology, 90(6), 1185–1203. https://doi.org/10.1037/0021-9010.90.6.1185
(2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061
(2010). Achieving competencies in public service. The professional edge. M. E. Sharpe.
(1993).
(Alternative ways of assessing model fit . In K. A. BollenJ. S. LongEds., Testing structural equation models (pp. 136–162). Sage.2021). Assessing moral judgment maturity using the Sociomoral Reflection Measure – Short Form Objective. European Journal of Psychological Assessment, 39(2), 85–95. https://doi.org/10.1027/1015-5759/a000680
(2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. Routledge.
(2021). Desarrollo de un catálogo de competencias para el personal directivo público español
([Development of a competency catalogue for Spanish public managers] . INAP.2019). “Why can’t I become a manager?” – A systematic review of gender stereotypes and organizational discrimination. International Journal of Environmental Research and Public Health, 16(10), Article
(1813 . https://doi.org/10.3390/ijerph161018132018). Validity evidence of the Organizational Justice Scale in Spain. Psicothema, 30(3), 344–350. https://doi.org/10.7334/psicothema2017.415
(2019). Competency vocational training as a strategy for improving public management. Aula Abierta, 48(4), 393–406. https://www.unioviedo.es/reunido/index.php/AA/article/view/14506
(2023). Supplementary material: Development, measurement and validation of a managerial competency model in Spain: The PUMAC Questionnaire [Data set]. http://hdl.handle.net/10651/66875
(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
(2013). Managerial competencies and career advancement: A comparative study of managers in two countries. Journal in Business Research, 66(3), 345–353. https://doi.org/10.1016/j.jbusres.2011.08.015
(2013). A review of the empirical ethical decision-making literature: 2004–2011. Journal of Business Ethics, 117(2), 221–259. https://doi.org/10.1007/s10551-012-1518-9
(2015).
(La función pública ante un escenario de gobernanza multinivel: Crisis y reinvención [The civil service in a multilevel governance scenario: Crisis and reinvention] . In J. CrespoEd., Crisis y reinvención de la función pública en un escenario de gobernanza multinivel (pp. 39–62). INAP.2019). Development of a brief version of the Personality Inventory IPIP-Revised: Control of the acquiescence response bias. Acta Colombiana de Psicología, 22(1), 248–272. https://doi.org/10.14718/ACP.2019.22.1.12
(2013). In search for a public health leadership competency framework to support leadership curriculum – A consensus study. The European Journal of Public Health, 24(5), 850–856. https://doi.org/10.1093/eurpub/ckt158
(2011). Facing an incompetent leader: The effects of a nonexpert leader on subordinates’ perception and behavior. European Journal of Work and Organizational Psychology, 20(2), 239–265. https://doi.org/10.1080/13594320903429576
(2017). Explaining unethical business decisions: The role of personality, environment, and states. Personality and Individual Differences, 117, 188–197. https://doi.org/10.1016/j.paid.2017.06.007
(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
(2017). The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioral Science/Revue Canadienne des Sciences du Comportement, 49, 78–88. https://doi.org/10.1037/cbs0000069
(1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312
(2022). Can psychological assessment contribute to a better world? European Journal of Psychological Assessment, 38(5), 347–355. https://doi.org/10.1027/1015-5759/a000739
(2022). Work characteristics and occupational health: Validation and proposal of a shortened version of the Work Design Questionnaire. Anales de Psicología/Annals of Psychology, 38(1), 149–162. https://doi.org/10.6018/analesps.480481
(2020). Developing biodata for public manager selection purposes: A comparison between fuzzy logic and traditional methods. Journal of Work and Organizational Psychology, 36(3), 231–242. https://doi.org/10.5093/jwop2020a22
(2013). To act out, to withdraw, or to constructively resist? Employee reactions to supervisor abuse of customers and the moderating role of employee moral identity. Human Relations, 66(7), 925–950. https://doi.org/10.1177/0018726713482992
(2013). An analysis of the management competency in Spanish companies. From theory to practice. Contaduría y Administración, 58(1), 251–288. https://doi.org/10.1016/S0186-1042(13)71205-1
(2014).
(Contemporary trends and dilemmas of administrative ethics in the developing world . In M. S. de VriesP. KimEds., Value and virtue in public administration. A comparative perspective (pp. 169–185). Palgrave Macmillan.2012). Der Beitrag Sozialer Kompetenz zur Vorhersage Beruflicher Leistung: Inkrementelle Validität Sozialer Kompetenz Gegenüber der Leistung im Assessment Center und Im Interview
([The contribution of social competence to the prediction of job performance: Incremental validity of social competence beyond assessment center and interview performance] . Zeitschrift für Arbeits- und Organisationspsychologie A&O, 56(2), 87–97. https://doi.org/10.1026/0932-4089/a0000772007, December 13). In leadership, “hard” skills trump “soft” skills. Harvard Business Review. https://hbr.org/2007/12/in-leadership-hard-skills-trum
(2020). Why dark personalities can get ahead: Extending the toxic career model. Personality and Individual Differences, 156, Article
(109792 . https://doi.org/10.1016/j.paid.2019.1097922011). La gestión de recursos humanos y el desempeño laboral [PhD Thesis, Universitat de València]. TDX (Tesis Doctorals en Xarxa). http://www.tdx.cat/handle/10803/81889
(2017). Comparative analysis of work values across four nations. Journal of Employment Counseling, 54(3), 132–144. https://doi.org/10.1002/joec.12061
(2006). Measuring the relationship between managerial competencies and performance. Journal of Management, 32(3), 360–380. https://doi.org/10.1177/0149206305280789
(2016, December 6). Measurement invariance with categorical indicators [Conference session]. Applied Psychometric Strategies Lab – Applied Quantitative and Psychometric Series, University of Kentucky, Lexington, Kentucky. https://education.uky.edu/edp/apslab/events/#MeasurementInvariance
(2014). Exploratory item factor analysis: A practical guide revised and up-dated. Anales de Psicología, 30(3), 1151–1169. https://doi.org/10.6018/analesps.30.3.199361
(2012). Common method bias in marketing: Causes, mechanisms, and procedural remedies. Journal of Retailing, 88(4), 542–555. https://doi.org/10.1016/j.jretai.2012.08.001
(2019). Development and psychometric evaluation of the Questionnaire of Ethical Leadership (QueL). European Journal of Psychological Assessment, 36(4), 635–645. https://doi.org/10.1027/1015-5759/a000533
(2005). Selecting individuals in team settings: The importance of social skills, personality characteristics, and teamwork knowledge. Personnel Psychology, 58(3), 583–611. https://doi.org/10.1111/j.1744-6570.2005.655.x
(2007). Competency management in EU public administrations – EUPAN Human Resources Working Group. https://www.eupan.eu/wp-content/uploads/2019/05/2007_2_PT_Competency_Management_in_EU_Public_Administrations.pdf
(2020).
(E-Kompetenzen [E-competences] . In T. KlenkF. NullmeierG. WewerEds., Handbuch Digitalisierung in Staat und Verwaltung (pp. 611–622). Springer Fachmedien. https://doi.org/10.1007/978-3-658-23668-7_572017). Derecho de la función pública
([Civil service law] . Civitas, Thomson Reuters.2012). Técnicas de gestión de recursos humanos por competencias
([Human resources management techniques by competencies] . Centro de Estudios Ramón Areces.2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879
(2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63(1), 539–569. https://doi.org/10.1146/annurev-psych-120710-100452
(2018). La nueva gestión del empleo público. Recursos Humanos e Innovación de la Administración
([The new management of public employment. Human resources and administration innovation] . Tibidabo Ediciones SA.2015). Exploring the factorial structure of the Sport Anxiety Scale – 2: Invariance across language, gender, age and type of sport. Psicothema, 27(2), 174–181. https://doi.org/10.7334/psicothema2014.263
(1995). Adolescent suicidal behaviors as a function of depression, hopelessness, alcohol use, and social support: A longitudinal investigation. American Journal of Community Psychology, 23(3), 329–354. https://doi.org/10.1007/BF02506948
(1996). Metodología de la investigación cualitativa
([Methodology of qualitative research] . Universidad de Deusto.2011). Performance appraisal in the public administration of the Principality of Asturias: An analysis of psychometric properties. Journal of Work and Organizational Psychology, 27(2), 75–91. https://doi.org/10.5093/tr2011v27n2a1
(2022). A review of measurement equivalence in organizational research: What’s old, what’s new, what’s next? Organizational Research Methods, 25(4), 741–785. https://doi.org/10.1177/10944281211056524
(2020). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semTools Packages. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 111–130. https://doi.org/10.1080/10705511.2019.1602776
(2012). Ethical leadership: An overview and future perspectives. Journal of Personnel Psychology, 11(1), 1–6. https://doi.org/10.1027/1866-5888/a000059
(2010). Unethical behavior in the name of the company: The moderating effect of organizational identification and positive reciprocity beliefs on unethical pro-organizational behavior. Journal of Applied Psychology, 95(4), 769–780. https://doi.org/10.1037/a0019214
(2016). Ética púbica y buen gobierno
([Public ethics and good governance] . Tecnos.