Development and validation of an infectious disease control competency scale for public health professionals

Abstract

Background

Infectious diseases persistently pose global threats, and it is imperative to accelerate the professionalization of public health workforce. This study aimed to develop and validate the infectious disease control competency scale (IDCCS) for public health professionals to fill a theoretical gap and elevate practical capabilities by informing public health professionals’ development goals.

Methods

The initial item pool was generated through a literature review, and categorized into three dimensions (knowledge, practical skills, and leadership) based on the competency iceberg model and public health leadership framework. A two-round Delphi process was conducted to determine indicators within the scale. A pilot survey was utilized for item analysis and exploratory factor analysis (EFA). A formal survey was employed for confirmatory factor analysis (CFA). The weight value of each indicator was calculated using the analytic hierarchy process.

Results

An initial scale with three primary items, 14 secondary items, and 81 tertiary items was generated. Twenty experts participated in the two rounds of the Delphi process. Authority coefficients exceeded 0.9 in both rounds. Kendall's W was 0.29 and 0.19, respectively (both P < 0.001). Item analysis presented a Cronbach's Alpha of 0.98, with corrected item-total correlation coefficients ranging from 0.33 to 0.78. EFA demonstrated that cumulative variance explanations for the four primary dimensions (knowledge, practical skills, leadership, and personal quality) were 77.463%, 73.976%, 81.174%, and 68.654%, respectively. CFA indicated that all composite reliability values and average variance extracted surpassed 0.8 and 0.5, respectively. The standardized factor loadings of the items ranged from 0.630 to 0.977. Among the seven model fit indices, each of the four dimensions satisfied at least five criteria. A final three-level scale comprising four primary items, 14 secondary items, and 64 tertiary items was constructed. The weight values for the four primary items were 0.4064, 0.2878, 0.2082, and 0.0981, respectively.

Conclusions

The IDCCS was established to evaluate the competencies of knowledge, practical skills, leadership, and personal quality for public health professionals in infectious disease control. This scale demonstrates good reliability and validity, and can be used for performance evaluation, recruitment processes, curriculum development, and individual self-assessment.

Introduction

Infectious diseases persistently pose global threats, manifested in periodic outbreaks of epidemics such as influenza and coronavirus disease 2019, and compounded by rising antimicrobial resistance [1]. Despite biomedical advances reducing associated morbidity and mortality, newly emerging and re-emerging infectious diseases with pandemic potential remain a pressing public health concern in our interconnected world [2]. Enhancing outbreak prediction and response requires more advanced skills among public health professionals, including utilizing new technologies such as internet-based surveillance systems and computational modeling to analyze pathogen transmission and impacts [3].

Currently, numerous countries are experiencing significant shortages in their public health workforce, both in terms of quantity and quality, which is hindering efforts to control infectious diseases. Data reveal over a 15% decline in the public health workforce in the United States alone over the past decade [4]. The World Health Organization (WHO) estimated a global shortage of 18 million health workers by 2030 [5]. Moreover, inadequate professional training is widespread among practitioners. Nearly two-thirds of personnel who carry out fundamental public health functions in some countries lack formal specialized public health education [6]. With intensifying structural challenges such as population aging and emerging infectious diseases, it is imperative to enhance capabilities in these fields through improved health workforce professionalization.

After outbreak of severe acute respiratory syndrome in 2003, China has been investing in infectious disease control, exemplified by revising the Law on the Prevention and Treatment of Infectious Diseases, enhancing emergency response systems, and establishing national disease reporting and surveillance networks [7, 8]. However, deficiencies persist regarding capacities for early outbreak detection and response, technical expertise, and interagency coordination [9]. Additionally, public health agencies in China also suffer from workforce attrition and skill gaps. Even model public health systems, such as that of Beijing, struggle to recruit and retain high-caliber personnel [10]. Consequently, the State Council emphasized key initiatives, including establishing high-caliber public health talent cultivation projects and training core experts from Centress for Disease Control and Prevention (CDCs) in 2023 guidelines on "Promoting High-Quality Development of Disease Prevention and Control" [11].

Several international competency frameworks for public health personnel exist, including the WHO's guidelines and the United Kingdom's Public Health Skills and Knowledge Framework [12, 13]. However, these frameworks focus on generalist skills without addressing specialized technical capacities such as infectious disease control. There was minimal research explicitly examining outbreak response competencies for public health professionals in China, contrasting with substantial assessments among doctors and nurses. More than half of these existing assessments relied on non-standardized tools lacking rigorous validation [14,15,16,17]. As jurisdictions such as Shanghai start planning specialized assessments among disease control experts, public health emergency responders and other critical personnel [18], a pressing need arises for unified, rigorously-developed instruments tailored to infectious disease competencies, which also capture the leadership capabilities central to guiding multisectoral response efforts. The absence of definitive standards underscores the significance of developing a competency scale for infectious disease control. This scale would fill a theoretical gap and enhance practical capabilities by informing public health professionals’ development goals. Crucially, in the face of changing public health threats, a validated tool is essential for strengthening overall preparedness to infectious diseases.

In this study, we aimed to develop a competency scale focusing on infectious disease prevention and control among public health professionals based on the competency iceberg model [19] and the frame of public health leadership [20]. Public health professionals engaged in infectious disease control were enrolled for scale validation.

Methods

Study design

A research group was assembled to construct and revise the infectious disease control competency scale (IDCCS), comprising two experts specializing in infectious disease prevention and control, two PhD students, and three master's students. This study adhered to published recommendations for scale development and reporting [21, 22]. The study encompassed four phases (Fig. 1): construction of the initial scale, Delphi process [23], evaluation and validation of the scale, and the analytic hierarchy process (AHP) [24].

Fig. 1

The study flow chart

Construction of the initial scale

Relevant initial items were gathered from the expert experience, literature [20, 25,26,27,28] and documents [29] related to personnel competencies in infectious disease prevention and control or public health emergency management. The sources included PubMed, China National Knowledge Infrastructure, Google Scholar, and official websites of CDCs in the United States, European Union, and China. Based on the competency iceberg model [19] and the leadership attributes of public health professionals, an initial scale with a three-level structure was developed through research group discussion.

Delphi process

Two rounds of Delphi process were conducted via email between March and May 2023. Experts meeting the following criteria were invited: (1) possessing over 10 years of work or research experience in the prevention and control of infectious disease; (2) holding senior titles, such as associate professor, professor, or equivalent senior positions in research or healthcare institutions. Experts' regional distribution, professional background, and multidisciplinary expertise were considered for representativeness and authority. If an expert who completed the first round could not participate in the second round, an alternative expert with an equivalent or stronger professional background was identified. A 5-point Likert scale [30] measured the importance of each item, and the suggestions of experts were collected. Experts reported judgment criteria (Ca) and familiarity (Cs), with the authority coefficient (Cr) calculated as (Ca + Cs)/2 (Cr ≥ 0.7 considered acceptable). After each round, items were adjusted based on the critical value method and the suggestions of experts, retaining items with average importance score ≥ 4.05, coefficient of variation (CV) ≤ 0.17, and full mark rate ≥ 26.0% in the first round; and average score ≥ 4.34, CV ≤ 0.14, and full mark rate ≥ 41.9% in the second round. If any one of these three indicators was unsatisfied, the research group discussed adjustment methods. The degree of coordination of experts’ scores was determined by Kendall’s coefficients of concordance (Kendall’s W).

Evaluation and validation of the scale

In June 2023, a pilot survey was conducted in Guizhou and Inner Mongolia, China. The subjects in the pilot study were individuals aged 35–50 years without disabilities or severe illnesses. They were involved in public health emergency or infectious disease control work at provincial or municipal CDCs and held intermediate or higher professional titles. All subjects provided informed consent. An electronic questionnaire was developed on the Wen-Juan Xing platform (Changsha Ranxing Information Technology Co., Ltd., Hunan, China) and distributed through internal CDC communication channels (e.g., WeChat group). The questionnaire included sociodemographic characteristics and the revised scale version after the Delphi method.

For exploratory factor analysis (EFA), a sample size of 150–200 is generally considered sufficient when the data set exhibits high communalities (above 0.5) or when there is a ratio of approximately 10 subjects per item with factor loadings of |0.4| or higher [21]. Data were analyzed using SPSS 23.0 (IBM Corporation, New York, NY, United States) and STATA 17.0 (StataCorp, TX, United States). Item analysis and EFA were used to refine and improve the scale structure. Specifically, t-tests were used to compare the upper 27% and lower 27% score groups. Corrected item-total correlation coefficients between individual items and the overall scale score were calculated, with coefficients above 0.4 indicating satisfactory results. The internal consistency of items was assessed by Cronbach's Alpha coefficient, where values above 0.7 indicated strong internal consistency. Cronbach's Alpha was calculated after removing individual items, and higher coefficients might indicate items that could be removed. The Kaiser–Meyer–Olkin (KMO) test and Bartlett's test of sphericity (P < 0.05) were used to assess EFA adequacy, with KMO values > 0.7 indicating suitability for factor analysis. A fixed number of factors were extracted according to the secondary item count in each dimension. Principal component analysis (PCA) with varimax orthogonal rotation was employed for data extraction. Items with factor loadings < 0.45 required adjustment. Items failing item analysis and EFA underwent research team discussion to determine adjustment methods. EFA was rerun after adjustment until all items fitted the preset factor frame.

The formal survey in July 2023 across 11 provinces targeted the same population. A minimum of five cases per item was recommended for sample size [21]. The questionnaire included the updated scale version after item analysis and EFA.

Amos 28 software (IBM Corporation, New York, NY, United States) was used to conduct the confirmatory factor analysis (CFA). Convergent validity was assessed through standardized factor loadings, composite reliability (CR), and average variance extracted (AVE). Satisfactory discriminant validity was determined by a higher square root of AVE than the correlation coefficients between factors. Convergent validity criteria included CR values ≥ 0.70 and AVE values ≥ 0.50. CFA model fit appropriateness was determined by: Chi-square to degree of freedom ratio (χ²/df) < 3, goodness of fit index (GFI) > 0.9, comparative fit index (CFI) > 0.9, normed fit index (NFI) > 0.9, Tucker-Lewis index (TLI) > 0.9, standardized root mean square residual (SRMR) < 0.05, and root mean square error of approximation (RMSEA) < 0.08. Meeting most of these indicators signified acceptable model fit.

Analytic hierarchy process

The analytic hierarchy process (AHP) was used to define the weight of each indicator [31]. For this three-level structure, all indicators were compared pairwise within each level according to the mean of expert scoring in the second Delphi round. Saaty's fundamental 9-point scale was adopted to determine the relative importance of one indicator compared to another [32]. Δ represents the difference in mean importance scores between any two indicators. The scoring standards were as follows: a score of 3 was assigned when 0.25 < Δ ≤ 0.50; 5 when 0.75 < Δ ≤ 1.00; 7 when 1.25 < Δ ≤ 1.50; and 9 when Δ > 1.75. For Δ values falling between these ranges, the scores were interpolated as 2, 4, 6, or 8, accordingly. The established expert judgment matrixes were entered into yaahp 10.1 (yaahp software, Taiyuan, Shanxi, China) to calculate the consistency ratio of each judgment matrix and the combined weight of each indicator.

Results

Summaries of the initial scale

The research group categorized the initial index pool into three dimensions: knowledge, practical skills, and leadership. The preliminary index pool comprised three primary items, 14 secondary items, and 81 tertiary items (Supplementary Table 1).

Scale modification via the Delphi method

Delphi round one

Twenty experts participated in the first round from 24 experts we initially contacted, yielding a response rate of 83.3%. The majority were male (13/20, 65%), aged over 50 years (14/20, 70%), working in CDCs or research institutions (19/20, 95%), and having a background in public health and preventive medicine (18/20, 90%). 95%(19/20) of the experts held a master's degree or above, 90% (18/20) possessed senior professional titles, and 85% (17/20) had at least 20 years of work experience (Table 1). The average Cr values for the three primary dimensions were 0.935, 0.933, and 0.908, respectively (Supplementary Table 2), representing a high authority.

Table 1 Characteristics of Delphi experts

Across all indicators, experts’ mean importance scores ranged from 3.32 to 4.95, with CV values ranged from 0.04 to 0.24, and full mark proportions ranging from 0 to 95% (Table 2). According to the critical value method and experts’ suggestions, one secondary item and three tertiary items were deleted, three secondary items and 30 tertiary items were revised or merged, and one primary item, two secondary items, and 11 tertiary items were newly added (Supplementary Table 3). A new primary item named "Personal Quality" was introduced, encompassing two secondary items (professional qualifications and professional quality). Kendall’s W was 0.285 (P < 0.001), indicating an acceptable coordination among experts (Supplementary Table 4). The revised scale after the first Delphi round comprised four primary items, 15 secondary items, and 73 tertiary items (Supplementary Table 5).

Table 2 Experts’ scores in the first round of Delphi

Delphi round two

Twenty experts participated in the second round from 27 experts contacted, resulting in a response rate of 74.1%, with similar basic characteristics to those in the first round (Table 1). The average Cr values of four primary dimensions were 0.938, 0.925, 0.933, and 0.945, respectively (Supplementary Table 6), representing a high authority.

Across all indicators, experts’ mean importance scores ranged from 3.85 to 4.85, with CV values ranged from 0.07 to 0.18, and full mark proportions ranging from 10 to 85% (Table 3). Based on the critical value method and experts’ suggestions, three tertiary items were deleted, two tertiary items were revised, four tertiary items were merged into two, and one tertiary item was newly added (Supplementary Table 7). Kendall’s W was 0.192 (P < 0.001), indicating an acceptable coordination among experts (Supplementary Table 8). The revised scale after the second Delphi round comprised four primary items, 15 secondary items, and 69 tertiary items (Supplementary Table 9).

Table 3 Experts’ scores in the second round of Delphi

Evaluation and validation of the scale

Participants’ characteristics

A total of 155 subjects were enrolled in the pilot survey for item analysis and EFA, and 476 subjects were enrolled in the formal survey for CFA (Table 4). Age, gender, education level, years of work related to infectious diseases, position, and postgraduate supervisor qualification did not differ statistically between subjects in the two surveys. In contrast, major of bachelor/college, professional title, job type, monthly income, experience in prevention and control of infectious diseases, and number of participations in outbreak response varied between the groups.

Table 4 Basic characteristics of participants in the pilot survey and formal survey

Item analysis and EFA results

The group with the lower 27% of total scores had significantly lower mean scores than the group with the upper 27% for all indicators (Supplementary Table 10), indicating a high degree of discriminant validity. Cronbach’s Alpha values of the four primary dimensions exceeded 0.893 in the pilot survey (Supplementary Table 11). The corrected item-total correlation coefficients between individual items and the overall scale score ranged from 0.33 to 0.78, with all P < 0.001 (Supplementary Table 12). Cronbach’s Alpha of the total scale remained stable and exceeded 0.98 when any individual item was deleted.

The criteria for conducting EFA were met because the KMO values of four primary dimensions were 0.940, 0.929, 0.943, and 0.868, respectively (all P < 0.05). The cumulative variance explanations for the four dimensions were 77.463%, 73.976%, 81.174%, and 68.654%, respectively. Three rounds of EFA and item adjustment were conducted to ensure all items fitted the preset theoretical frame (Supplementary Table 12−20). In summary, four tertiary items were deleted based on item analysis or EFA, two secondary items and two tertiary items were merged based on item analysis or EFA, and one tertiary item was moved to another secondary item based on EFA (Supplementary Table 21). The revised scale after item analysis and EFA comprised four primary items, 14 secondary items, and 64 tertiary items (Supplementary Table 22).

CFA results

Cronbach’s Alpha values of the four primary dimensions exceeded 0.898 in the formal survey (Supplementary Table 11). All CR values and AVE values exceeded 0.8 and 0.5, respectively, representing a satisfactory convergent validity (Table 5). The standardized factor loading of the items ranged from 0.630 to 0.977. The path diagrams with standardized factor loadings for the four dimensions are shown in Supplementary Figure 1 to 4. Good discriminant validity was observed in each dimension, with the square root of AVE exceeding the correlation coefficients between factors (Supplementary Table 23−26). Among the seven model fit indices, all four dimensions satisfied at least five criteria, indicating a good fit effectiveness in CFA (Supplementary Table 27).

Table 5 Confirmatory factor analysis and convergent validity of the scale

Weight determination of each indicator through AHP

The expert judgment matrixes for the three hierarchies are shown in Supplementary Table 28−46. The consistency ratios were less than 0.1 for all the matrixes, indicating good consistency in each. The combined weight for the four primary items were 0.4064 for Knowledge, 0.2878 for Practical skills, 0.2082 for Leadership, and 0.0981 for Personal quality (Table 6). The secondary items with the top three combined weights were Knowledge of infectious diseases (0.1993), Knowledge of public health emergency management (0.1267), and Public health response to infectious diseases (0.1199). The tertiary items with the top five combined weights were Basic epidemiological knowledge of infectious diseases (0.0823), Criteria for judging aggregates epidemics and outbreaks of infectious diseases (0.0486), Mastering knowledge and skills of personal protection (0.0486), Command, coordination and control of health emergency (0.0413), and Analyzing the situation of public health incidents and proposing targeted prevention and control measures (0.0410). The final versions of the IDCCS in English and Chinese are presented in Supplementary Table 47.

Table 6 The weight values of final three-level scale

Discussion

Key findings of this study

A key strength of the IDCCS lies in its robust development process, which combined expert consensus through the Delphi method with rigorous statistical validation. The high authority coefficient and opinion coordination coefficient achieved across two Delphi rounds underscore the reliability and representativeness of the scale items. The validation process, including item analysis, EFA, and CFA, demonstrated the IDCCS's strong psychometric properties. Notably, the scale exhibited excellent internal consistency and satisfactory construct validity, with factor loadings, CR, and AVE values all meeting or exceeding standard thresholds. These findings support the reliability and validity of the IDCCS as a measurement tool for infectious disease control competencies.

This study marks a significant advancement in the field of public health competency assessment by developing and validating the IDCCS specifically tailored for public health professionals in China. The IDCCS represents a comprehensive and multidimensional approach to competency evaluation, encompassing four primary dimensions: knowledge, practical skills, leadership, and personal qualities.

Theoretical underpinnings and dimensionality of the IDCCS

The IDCCS was established based on well-founded theoretical frameworks. Knowledge, practical skills, and personal qualities were derived from the competency iceberg model [19], while leadership stemmed from the public health leadership framework [20]. The iceberg model suggests that hidden factors such as motives, traits, self-image, and social roles are key to understanding competency. Visible competencies, such as knowledge and skills can be readily developed through training and skill-building exercises. However, the invisible competencies are more challenging to assess and cultivate [19]. Thus, more items related to knowledge and practical skills were designed to improve the scale's practicality. The frame of public health leadership was developed through expert consensus to inform a leadership curriculum for public health professionals in Europe [20]. We incorporated most of its domains such as systems thinking, collaborative leadership, leadership and communication, leading change, and organizational learning and development. Political leadership related to European public health governance was excluded. Public health leadership is currently lacking in higher education [33, 34]. The inclusion of this dimension effectively complements the competency measurement for public health professionals, aligning with their work needs and personal development goals.

The IDCCS is the first of its kind, exclusively designed to assess competencies in infectious disease control among public health professionals. Existing public health competency scales often cover a broad range of skills not limited to infectious diseases or focus on medical staff rather than public health professionals. For instance, the Core Competencies for Public Health Professionals developed by the Council on Linkages Between Academia and Public Health Practice provide a foundational framework for generic public health competencies, but it is not specific to infectious disease control [35]. Similarly, the Regional Core Competency Framework for Public Health developed by the WHO exhibited six domains, where only one domain (surveillance and control of risks and threats) was directly related to infectious disease control [12]. Thus, these broader competency models, while valuable for setting macro-level development goals, often lack the specificity required for precise measurement of infectious disease control competencies. IDCCS addresses this gap by providing a specialized tool for assessing competencies in infectious disease control among public health professionals. In contrast to previous scales that primarily focused on the field rescue capabilities of medical staff in managing infectious diseases [16, 25], the IDCCS differs from them in several key aspects. First, the IDCCS is tailored to the Chinese public health system, reflecting the unique structure of China's CDC system. Second, the IDCCS integrates the competency iceberg model with the public health leadership framework, while most previous scales were established without clear theoretical basis. Notably, the incorporation of public health leadership as a competency dimension is a novel and distinctive feature of the IDCCS. While earlier scales concentrated on the clinical aspects of infectious disease management, our scale recognizes the pivotal role of public health leadership in effective disease control strategies.

The results of AHP revealed that knowledge and practical skills were weighted most heavily among the primary dimensions. This highlights the high demand for specialized and technical expertise in infectious disease control. Experts tend to view specialized knowledge and practical abilities as prerequisites, thus assigning them greater weights over leadership and personal qualities when judging competencies. Within the secondary dimensions, knowledge of infectious diseases, knowledge of public health emergency management, and public health response to infectious diseases received the highest weights. This premium on specialized knowledge and practical skills aligns with the competency weights at the primary level, revealing more nuanced demands. Infectious disease knowledge forms the vital theoretical foundation; emergency management represents the capability to respond amidst unpredictable crises; and public health response encompasses the comprehensive expertise to coordinate interventions. Collectively, these secondary competencies underscore the diverse abilities required for real-world infectious disease control. Among the tertiary items, the five with the highest weights all belonged to the knowledge and practical skills dimensions, further validating their significance. The premium placed on self-regulation and communication abilities within the leadership competency dimension highlights the high-pressure nature of infectious disease control work and the critical importance of coordination in outbreak response systems [36]. In the personal quality dimension, professional quality carried a higher weight than professional qualifications, indicating that competency evaluation should focus on applicants' professional sentiments, attitudes, and learning abilities, rather than solely relying on resumes while ignoring character and growth potential.

Potential applications and impact of the IDCCS

Tailored for the Chinese context, the IDCCS was developed and validated with participants recruited from multiple provinces in China. The Delphi technique employed in its construction ensured the authority and representativeness of the scale's items. Furthermore, the incorporation of the AHP method facilitated the scale's practical operationalization among public health professionals, augmenting its usability and applicability in real-world settings. By combining expert consensus and prioritization methods, the IDCCS effectively balances theoretical rigor with practical considerations. This makes it useful for assessing and nurturing competencies vital to infectious disease control within the Chinese public health system. The IDCCS holds significant potential for practical applications in performance evaluation, recruitment, curriculum development, and self-assessment within public health agencies. During infectious disease outbreaks, the scale can facilitate identifying and deploying highly competent professionals to frontline response efforts and critical incident management. Furthermore, it can serve as a self-assessment tool, promoting intrinsic motivation for learning and professional development among public health professionals. For institutes or agencies, the IDCCS guides the identification of workforce competency gaps, enabling targeted capacity-building initiatives tailored to specific needs and job requirements. By recognizing the varying weights assigned to different competency items, professionals can prioritize and focus on developing the core critical abilities essential for their roles. Moreover, the IDCCS informs the design and evaluation of training programs and simulation exercises, providing a theoretical framework for assessing their effectiveness.

While the IDCCS was developed within the Chinese context, its framework and methodology have potential global applications. For adaptation to other regions, we have several recommendations. First, the core dimensions (knowledge, practical skills, leadership, personal qualities) are suggested to be retained while adjusting specific indicators to reflect local public health systems and infectious disease control priorities. Second, new Delphi processes are suggested to be conducted through engaging local public health experts to refine indicators and recalibrate weights according to regional needs and perspectives. Third, new indicators related to local cultural contexts, policy environments, and specific infectious disease challenges should be incorporated. Additionally, cross-cultural validity testing is necessary to ensure the adapted scale's reliability and validity in the new context [37].

Limitations

This study has several limitations that should be considered. Methodologically, the Delphi process excluded health officials and administrators, which may have limited the scale's reflection of implementation and policy perspectives. The competency evaluation relied on self-reported assessments, which may introduce leniency bias. In terms of scope and applicability, the IDCCS was specifically developed for public health professionals in infectious disease control within the CDC system, potentially limiting its suitability for evaluating professionals beyond this context or in other public health domains. Geographically and culturally, our study sample was limited to multiple provinces in China, which may restrict the scale's applicability to regions with distinct cultural, socioeconomic, and public health system characteristics. Future research should address these limitations by including a broader range of stakeholders in the development process, incorporating objective assessment methods, and exploring the scale's cross-cultural validity and potential adaptations required for broader national or international applications.

Conclusions

This study has established, for the first time, a competency framework tailored specifically for public health professionals in infectious disease control. The IDCCS covers a comprehensive spectrum of competencies, addressing knowledge, practical skills, leadership, and personal qualities. It holds significant potential for application in performance evaluation, recruitment processes, curriculum development, and individual self-assessment within public health agencies. While providing crucial support in addressing key public health issues, the scale requires practical validation to continually enhance and optimize its utility.