Skip to main content
Download PDF

The construction of Chinese indicator system on public health field investigation and short-term study hub: experience and implications

Global Health Research and Policy volume 7, Article number: 40 (2022) Cite this article

Abstract

Background

The increasing of exchange activities among public health institutes and experts globally calls for a standardized operation to construct public health field investigation and short-term study hub (Field Study Hub). This can funcion as a platform to share experience in public health development in an accurate and comprehensive manner that would benefit global practices. This research aims to establish a supportive indicator system to guide the construction work.

Methods

Delphi method including two rounds of surveys were conducted among 82 senior public health experts. A structured questionnaire was designed to collect the opinions of the experts on the necessity of setting and feasibility of measurement for proposed 5 dimensions of 49 indicators and 7 additionally proposed ones. Percentage and score were used to describe the assessments, χ2 and t tests to compare differences, Kappa and Cronbach’s alpha values to assess intra-rater and inter-rater reliabilities. Significance level α was 0.05. Bonferroni adjustment was used in the comparison of experts’ judgment basis.

Results

The percentages of experts choosing “Very good” or “Good” for necessity and feasibility in rounds 1 and 2 were 73.1–97.6% (85.8% ± 7.5%), 64.6–93.9% (82.8% ± 6.7%), 73.8–100% (91.0% ± 6.2%) and 72.5–100% (89.2% ± 7.3%) respectively. The scores of necessity were higher than those of feasibility, and the differences in the dimensions of “Key experience”, “Capacity for logistic support” and the total were statistically significant (t11 = 2.920, t12 = 3.035, t31 = 4.448, t32 = 2.664, tt1 = 3.794, tt2 = 3.007, P < 0.05). The fourteen most necessary indicators were identified. The judgment bases of “Theory” and “Experience” were higher than “Knowledge” and “Intuition” statistically significantly (round 2: χTK2 = 39.020, χEK2 = 67.692, χTI2 = 45.823, χEI2 = 76.515, P < 0.0125). The Kappa values exceeded 40 with the maximum as 75 and the Cronbach’s alphas exceeded 0.8000 with the maximum as 0.9732.

Conclusions

A set of 5 dimensions of 56 indicators with good necessity and feasibility were developed to technically support and well evaluate the construction of field study hub in public health institutions. This was of high significance because it tended to provide a preliminary baseline for the standardized practice in global health. Also, the present research might serve as a methodological reference for the development of other indicator sets.

Introduction

The promotion of global health cooperation greatly increased the exchange activities among global public health institutes and health experts [1]. Commissioners of public health in each country made persistent and tremendous efforts to strengthen public health development in order to achieve public health goals, such as Sustainable Development Goals, which brought about remarkable achievements and equipped all the masses in each country with wealth of extensive health knowledge, collaborations and experience [2,3,4,5,6,7,8,9,10,11].

Some countries played crucial roles in the development of communication mechanism and working platform in public health with public health institutes, especially for centers for disease control and prevention, and healthcare departments in women and children’s healthcare hospitals/centers at national and international levels. Such crucial role entails designing and implementing a capacity building program on global public health development cooperation [12]. In the implementation of the programs of global public health, professional public health experts were invited to different countries to have investigation visits in the centers for disease control and prevention at national, state and grass-root level. Against this background, it was popular for these programs to build qualified field study hubs inside each county’s public health system to enhance the hosting capacity of multiple levels of public health institutes, particularly in the developing countries. Such approaches would provide conducive environment for global public health experts to accurately share each country’s public health experience with international colleagues and ensure the implementation of standardized global public health practices and measures. However, few researches were conducted to identify a set of indicators pertaining to hub development for field investigation and short-term study.

According to the proposal from O’Donnell in 2020 [13], an indicator could be considered as a measure that provided an insight into relative positions in a given area or sector (e.g. public health). Evaluation of these indicators was proven to be beneficial, because it pointed out a new direction of changes in an area over a period of time and future trends [14]. Selection of suitable set of indicators relevant to establishment and evaluation of study hub entailed a high-level of judgment and consensus building among the health parastatals and health users around world [15]. However, a large number of variables might influence the development of the hub for field investigation and short-term study, hence a Delphi based-approach was necessary because a consensus might be attained amongst the public health experts. Therefore, a technical framework of indicator system to guide the development of hubs needed to be established for following reasons: the hub construction meant a considerable input of human, financial and material resources; the indicator system was expected to be a veritable tool with the capacity to circumvent possible risks from significant inputs, support to obtain satisfactory input–output ratio, accurate achievement of set goals and fully share experience after the hub’s completion.

Accordingly, the “Key experience” was set as the first dimension of the indicator system, then following four dimensions were developed as: “Capacity on experience demonstration”, “Capacity on reception”, “Capacity to host short-term study”, and “Significance of the hub construction”. Referring to professional classification of public health, international development documents such as Sustainable Development Goals [16], Agenda 2063 etc. [17] and key points were expected to be shared. We summarized China’s experience in public health into seven key areas: (1) introduction of advanced techniques to public health laboratory, (2) prevention and control of major infectious disease, (3) maternal and child healthcare, (4) disease surveillance and response, (5) public health emergency, (6) public health infrastructure, (7) prevention and control of non-communicable disease. Other indicators in the 5 dimensions were all designed through the approach of brain storming among the research group, foreign visitors and reception personnel participating in past exchange activities including field investigation and short-term study in China’s public health institutes organized by the researchers in the global public health program [12].

This research described the indicator system framework, assessed the necessity of setting and feasibility measurement of proposed indicators by invited experts and determined their intra-rater and inter-rater agreements of the experts in the two rounds of surveys.

Method

Study design

Delphi method including two rounds of surveys were conducted among the invited public health experts in November 2019 and from August to September 2020, respectively. A total of 82 Chinese experts from different public health workplaces including China’s national, provincial centers for disease control and prevention, women and children’s healthcare hospitals/centers/institutes, China’s universities and general hospitals were included.

General setting

China has 23 provinces, 5 autonomous regions, and 4 municipalities (Beijing, Tianjin, Shanghai and Chongqing) and 2 special administrative regions. Each of them consists of prefectures, districts/counties, communities/townships, neighborhood committees/villages (urban/rural) respectively [18]. China has one national level center for disease control and prevention, China CDC. Each province (autonomous region, municipality), prefecture and district/county has a full government-sponsored center for disease control and prevention (CDC) which constitute China’s local triple-level CDC system consisting of 3384 CDCs in total. Besides, each province (autonomous region, municipality), prefecture and district/county has a full government-sponsored hospital/center/institute for women and children’s health (WCH) which constitute China’s local triple-level WCH system consisting of 3052 WCH institutes in total [19]. Inside China CDC, there is a National Center for Women and Children’s Health (NCWCH). China CDC plays a technical guidance role to local CDCs and WCH institutions.

The public health field investigation and short-term study hub is aimed to establish a professional exchanging platform through which the counterparts could conduct exchange visits and study activities in general or some specific areas in public health to share experience and learn from each other. In China, it is expected to be constructed in national and provincial CDCs and WCH hospitals basing on their present resources mainly with necessary enhancement. The construction of such hub also means to increase the administrative and professional functions of these institutes to receive foreign counterpart professionals and share China’s experience in public health development including concrete knowledge and skill transfer with foreign professionals through two forms of exchanging activities. One activity is field investigation through which the foreign visitors are invited to visit all levels of public health institutes for a week to understand the good practices of China in public health from various aspects. The other activity is short-term study through which the foreign participants work in these institutes to learn professional knowledge and skills for 12 weeks under the guidance of Chinese experts.

Development of indicator framework: dimensions and indicator set

Since the main aim of the construction and operation of the public health field investigation and short-term study hub is to share China’s experience in public health development with foreign counterpart professionals accurately and comprehensively. For this, the first step is to extract the experience (Dimension 1), then carry out field investigation and short-term study activities with certain capacities to demonstrate and share the experiences (Dimensions 2 and 4), and logistic support (Dimension 3) is needed for the reception of the participants in field investigation and short-term study. Thus, the Dimensions 1–4 implement the functions and roles of the public health investigation and short-term study hub. In total, the public health investigation and short-term study hub will have multiple significances (Dimension 5). So, the 5 dimensions were developed according to the construction framework of the public health field investigation and short-term study hub (Fig. 1). The indicators in each dimension were generated respectively afterwards.

Fig. 1
figure 1

Framework the hub construction to develop the dimensions of indicator system

Full size image

We adopted the Delphi method to attain expert consensus on the indicators. Consequently, a structured questionnaire (Additional file 1) was designed to assess the opinions and agreements of these experts on the degrees of necessity of setting and feasibility of measurement for proposed indicators. A five-point Likert scale including very good, good, middle, poor and very poor was used. The experts were also given an option in the questionnaire to show their judgment basis regarding theory, experience, international and domestic understanding, and intuitiveness. The judgment basis degrees were classified as high, middle and low. The Delphi process took two rounds of surveys through electronic system. Initial sets including five dimensions and their corresponding indicators were shown in Table 1. During the first round, the experts were encouraged to add indicators if they thought them necessary and feasible. These indicators were added to the pool, sorted and finalized in the second round.

Table 1 Dimensions one to five and their indicators
Full size table

Data collection

The self-administered questionnaire was distributed to the same experts in both first and second rounds of surveys. The distribution was conducted via two channels: directly through WeChat or email for the national level experts, and through office automatic (OA) system or fax from China CDC to provincial CDCs and WCH institutes who were asked to recommend experts with appropriate professional background as well as senior professional and technical title from their own institutes to participate in the evaluation. Filled forms were then returned to the researcher within 3 weeks by emails. Experts in the first round included 30 nation-level experts. The provincial CDCs and WCH hospitals recommended 52 local experts, hence a total of 82 experts responded to the questionnaire. In the second round, two local experts could not participate due to the retirement and emergency work respectively.

Statistical analysis

The data collected was double-entered with validation using Epidata Entry version 3.1 and exported into Statistical Package for Social Science (SPSS) version 22 for data analysis. The distributions of the responses were described by number of counts (percentage, %), minimum, maximum and mean ± standard deviation (sd). Chi-square (χ2) test and student’s t test were used to compare the differences between necessity of setting and feasibility of measurement in rounds 1 and 2. The intra-rater and inter-rater agreements of expert judgments were assessed by Kappa and Cronbach’s alpha values respectively.

For necessity of setting and feasibility of measurement of the indicators, the scales of “Very good”, “Good”, “Middle”, “Poor” and “Very poor” corresponding to 5, 4, 3, 2 and 1 respectively. The scores were then calculated by using the formula: Score = Number of Very good*5 + Number of Good*4 + Number of Middle*3 + Number of Poor*2 + Number of Very poor*1. For the expert judgment basis, the degree of “High” valued 5, “Middle” valued 3 and “Low” valued 1. The scores were then calculated by using the formula: Score = Number of High*5 + Number of Middle*3 + Number of Low*1. Their percentages were obtained by the formula: Score*100/[82*(80 for Round 2)*5(dimensions)*5]. The indicators with higher degrees of necessity and feasibility as well as with higher agreement were considered as qualified ones. Significance level α was set to be 0.05. Bonferroni adjustment was used in the comparison of experts’ judgment basis and the adjusted α was 0.0125 (0.05/4 times of comparisons).

Results

Expertise areas of the experts

In the two rounds of surveys, all questionnaires were returned valid (100%), although only 2 experts were unavailable in the second round of survey. The experts that responded to the questionnaire covered 9 major expertise areas in public health (Additional file 2). Some experts are professionals in more than one area of specialization. The top 3 areas were health administration, prevention and control of major infectious diseases, and women and children’s health.

Necessity of setting and feasibility of measurement by the experts

In both rounds, most experts agreed with the necessity of setting and feasibility of measurement of the indicators (Additional files 3). For both necessity and feasibility, the average percentages of experts who chose “Very good” or “Good” were larger than 80% with standard deviations being less than 8% totally, and the percentages in round 2 were higher than those in round 1 statistically significant (tnecessity = 3.443, tfeasibility = 4.143, P < 0.05) (Additional files 4). Over 85% of the indicators obtained higher ratios of “Very good” plus “Good” in the second round than in the first round, and some differences were statistically significant (P < 0.05) (Fig. 2A, B).

Fig. 2
figure 2

The percentages of experts who considered the necessities or feasibilities were “Very good” or “Good”. Note “Ind” represents “Indicator”

Full size image

In both rounds, the scores of necessities were higher than those of the feasibilities for almost all of the indicators, and the differences in Dimension 1, Dimension 3 and the total were statistically significant (t11 = 2.920, t12 = 3.035, t31 = 4.448, t32 = 2.664, tt1 = 3.794, tt2 = 3.007, P < 0.05). The top indicators in necessity in round 2 achieving higher scores were: Indicators 1.2.1, 1.4.1 and 1.1.1 in Dimension 1, Indicators 2.5.1, 2.5.2 and 2.1.1 in Dimension 2, Indicators 3.5.1 and 3.4.1 in Dimension 3, the three indicators of 4.4.1, 4.3.3, 4.3.2 in Dimension 4 and Indicator 5.1.1 in Dimension 5 in order respectively (Table 2).

Table 2 The scores of expert judgments for the indicators in necessity and feasibility
Full size table

The judgment bases of the experts

The percentages of the scores for “Theory” and “Experience” were higher than those of “Knowledge about international and domestic situation” and “Intuition” statistically significantly in both round 1 (χTK2 = 27.617, χEK2 = 49.377, χTI2 = 17.329, χEI2 = 35.261, P < 0.0125) and round 2 (χTK2 = 39.020, χEK2 = 67.692, χTI2 = 45.823, χEI2 = 76.515, P < 0.0125) (Additional file 5, Fig. 3).

Fig. 3
figure 3

The illustration of the comparison processes of four judgment bases in round 1 and round 2, separately. Note *P < 0.05.

Full size image

Coherence of expert’s judgment

The Actual agreement of Kappa values of necessity and feasibility between the two rounds were 37.50–77.50 and 35.00–61.25 respectively. Twenty-eight and twenty-four indicators’ Kappa values in necessity and feasibility respectively were statistically significant (P < 0.05) and larger than 40.00 (P < 0.05), in which indicator 3.5.1’s in necessity was up to 75.00 (Fig. 4). For judgment bases, except for “Theory” in Dimensions 2 and 6 and “Knowledge about international and domestic situation” in Dimensions 3 and 4, all Kappa values were statistically significant (P < 0.05) and larger than 40.00 (Fig. 4B). Cronbach’s alphas demonstrated that the coherence among experts in each round of survey were larger than 0.8000 except for Dimension 2 of round 2 (0.7637). Interestingly, all Cronbach’s alphas in round 1 were larger than 0.9000 (Fig. 5).

Fig. 4
figure 4

Actual agreements of Kappa values for necessity, feasibility and judgment bases between two rounds

Full size image
Fig. 5
figure 5

Cronbach’s alphas among experts

Full size image

Additional proposed indicators by experts

Seven indicators distributed in the 5 dimensions were proposed additionally by experts in round 1 (Additional file 6). In round 2, 77.5–96.3% (90.1% ± 7.1%) and 67.6–97.6% (87.0% ± 10.8%) experts considered that the added indicators were “Very good” or “Good” in necessity and feasibility respectively (t = 0.575, P > 0.05) (Additional file 7).

Discussion

The present research efficiently utilized resources and wealth of practical experience from the experts in multiple branches of public health areas from China’s both national and provincial public health institutes. Also, the selection of experts from different public health institutes and areas played a positive role to circumvent a certain degree of bias due to the same background. Through the Delphi-based approach, we obtained the necessity of setting and feasibility of measurement of 56 indicators of 5 dimensions which would form a guideline and baseline study for building an indicator system which was expected to enhance the construction of an efficient and effective public health field investigation and short-term study hub of significant scientific value in China.

In this research, we took advantage of China’s public health network in organization and mobilization during the data collection of the expert opinions, thus ensuring the participation of a wide range of experts and their opinions gained. We also emphasized on the accuracy in experience sharing with the public health professionals globally. Although there have been a few of researches on the indicator system in health area [20,21,22], there was a dearth of information from previous researches on the construction of hub to share public health development experience. The indicator system developed in this study is the first set of indicators to guide, evaluate and monitor the hub construction for global sharing of public health experience in China. Its whole process of the development was scientific and consistent. Attainment of a consensus tends to be an established standard for the completion of Delphi process; and capacity to reduce the variance in the outcomes is the priority for establishing the consensus [23]. Established on these virtues, this research provides a model on the methodology and procedure to the development of supportive guidelines before the actions in public health hub construction. Here, we will discuss the necessity and feasibility, coherence and future needs of the indicator system.

The judgments of the experts on both necessities of setting and feasibility of measurement of indicators were found to be different in rounds 1 and 2 of the consultations. The percentage of experts who considered necessities or feasibilities on a point scale of “Very good” or “Good” were higher in round 2 compared to round 1 consultations, and 7 and 11 indicators were statistically different (P < 0.05) for necessity and feasibility respectively (Fig. 2). The increased percentage observed in round 2 could be that the experts’ view on the critical value of the indicators gained more recognition. The increased Cronbach’s alphas among experts in the significance of the field investigation and short-term study hub construction for Dimension 5 validates the above rationale although the Cronbach’s alphas among experts for other dimensions were slightly lower in round 2 than in round 1 (Fig. 5).

Necessity of setting and feasibility of measurement were the two essential factors considered for the value of each indicator in this research. Interestingly, the necessity received better judgments than the feasibility for almost all of the indicators, and the differences of Dimension 1 (experience sharing), Dimension 3 (demonstration capacity) and the total were statistically significant (P < 0.05). While for the additional proposed indicators by experts, the necessity and feasibility values were similar. This attractive finding indicated that feasibility of indicators merited more attention when creating indicator system. The indicators should be designed specifically to get the measurement target. If necessary, the indicator should be adjusted to be more measurable and ensure a pre-test run before adoption for normal use. Meanwhile, the actual agreements of Kappa values of necessity between the two rounds were larger than those of feasibility for most indicators (Fig. 4). Perhaps the reason for this is that necessity is easier to be achieved than feasibility when designing an indicator. The statistically significant lower scores for feasibility than necessity for experience sharing and logistic support also illustrated that they are themselves relatively more difficult parts to operate besides the measurement aspect. Critical analysis of these two parts would provide a better reflection of the inherent characteristics of the public health field investigation and short-term study hub, hence more deliberations on them are essential.

Furthermore, during the round 2 survey, fourteen indicators with highest scores in necessity in their own dimensions were identified. These highlighted the following crucial points: prevention and control of major infectious diseases, health emergency and introduction of new technique, and construction of lab network were the most important areas of Chinese experience in public health; accurate content and language capacity were the most important points for experience demonstration; cross-cultural awareness and humanitarian act as well as security were important principles in reception; the importance of the sustainability of cooperation, research work and plan in short-term study were notable; and the reception experience was emphasized. For additionally proposed indicators, “Has perfect management system” (Dimension 3) and “Trainee’s evaluation on the Hub” (Dimension 5) got highest scores in necessity together with best feasibility. These fourteen indicators can be used as a group of core and short-list indicators to guide and evaluate the hub construction, especially when time is relatively tight, multiple periodic verifications on the public health field investigation and short-term study hub construction progress are needed and so on. Moreover, taken into consideration that expert opinion scores on necessity was statistically significantly higher than that of feasibility for Dimension 1 and 3 indicators as mentioned above, it is highly recommended that specific and in-depth research on the content and requirement of the six key indicators in Dimensions 1 and 3 be conducted.

Generally speaking, the Kappa value between 40 and 75 represents a middle degree of agreement, being equal or larger than 75 means a good agreement [24, 25]. The Cronbach’s alpha between 0.7 and 0.8 indicates acceptable consistency, and between 0.8 and 0.9 considerable consistency, being equal or larger than 0.9 means a very good consistency [26, 27]. The present research invited more than 80 experts from both national and provincial levels, but achieved good consistency among experts and between the two rounds of surveys, in both indicators and expert judgment basis, illustrating a good credibility of the indicators. Therefore, all initially proposed indicators were retained after the two rounds of expert consultation. The fact that the expert judgment basis was more from “Theory” and “Experience” than “Knowledge about international and domestic situation” and “Intuition” especially in the second round was consistent with the actual situation that the consulted experts might be the participants of the field investigation and short-term study hub construction and the users of the indicator system as well.

Some question items were designed as “open question”, especially for those the experience extraction in Dimension one. The responses to them are expected to be analyzed by using the method for qualitative survey materials analysis. The answers to the open questions are sorted out and analyzed to describe the responses results on the basis of the topic, the case and the case code classification. Semantic analysis of artificial intelligence could be used. For experience extraction, the resources input (human, financial and material reserves and mobilization), strategies (key points, risk avoidance) and actions (content, frequency, intensity), the significance and effect (importance, positive impact) are used as the primary (secondary) classification criteria. A method based on Bert (Bidirectional Encoder Representation from Transformers) model could be used for analysis [28, 29]. In practical operations, minor adjustment to Bert could be made according to specific downstream tasks to adapt to the text characteristics. Otherwise, we may also analyze the text about the public health experience from the perspective of topic extraction to use the author topic model for topic analysis [30]. The core content of the comment could be extracted and analyzed. In the situation where the collected responses about the experience can mostly be classified into pre-established groups, the Bert classification method can be used. When the collected responses were innovative comparatively to have more text that is difficult to be classified into the established categories, the topic analysis method will be more appropriate.

The initial purpose of present indicator system was to technically guide but not limited in evaluating the construction of the field investigation and short-term study hub in public health. However, the present research has highlighted the capacity of this indicator system to evaluate the construction of such hub objectively, which demonstrates the innovative and unique nature of present research. A step further to consolidate it may invite foreign counterparts who have been or may become participants of the field investigation and short-term study in the Hubs in China to contribute their ideas on the necessity, feasibility and supplementation of the indicators. In addition, in the course of monitoring these indicators, one could ascertain the reliability of them.

The present research would not be oblivious of certain limitation, in as much as its details were described for the experts prior to the survey, the experts’ understandings of the questionnaire might vary because the survey was carried out through email instead of face to face. Further research would be conducted with the public health experts to investigate the significance and value of each indicator. This pragmatic approach would ensure that the indicator system is a practical tool to observe the progress of the construction work about the public health field investigation and short-term study hub health supportively.

A healthy nation is a wealthy nation and the health labor force is the future driving force of the country’s overall sustainable development. With the increasing demand for standard global public health practices, the populace needs to have access to enhanced health facilities. Identification and development of these indicator sets will provide baseline for the implementation of better health strategies and healthy policies that will promote the overall capacity of public health sector including CDCs and WCH institutes in China. Furthermore, the development is helpful for the establishment, evaluation and monitoring of the hub system for field investigation and short-term study, hence meeting up with health demands now and in future using a scientific approach.

Conclusions

A set of 5 dimensions with 56 indicators were developed to technically support and well guide a standardized construction of investigation visit and short-term study hub in public health in China. Such indicator system was found to have good necessity of setting and feasibility of measurement with good levels of agreements between two rounds of expert consultations. This was of high significance in the public health sector as the present research tended to provide a preliminary baseline for field study hub construction and evaluation in public health. Also, the set of indicators might serve as a methodological reference for the development of other indicator sets.

Availability of data and materials

All data generated or analyzed during this research are included in this published article.

Abbreviations

CDC:

Center for Disease Control and Prevention

MCH:

Maternal and Child Health

China CDC:

Chinese Center for Disease Control and Prevention

NCWCH:

National Center for Women and Children’s Health

OA:

Office automatic

WCH:

Women and Children’s Health

References

  1. The State Council Information Office of the People’s Republic of China. China’s International Development Cooperation in the New Era. http://www.scio.gov.cn/zfbps/32832/Document/1696686/1696686.htm. 10th January, 2021.

  2. Weizhong YANG. China’s achievements in public health in the past 70 years. Mod Prev Med. 2019;46(16):7–10.

    Google Scholar 

  3. National Health Commission of the People’s Republic of China. China’s Women and Children’s Health Development Report 2019. http://www.nhc.gov.cn/fys/ptpxw/201905/bbd8e2134a7e47958 c5c9ef032e1dfa2.shtml. 27th May, 2019

  4. World Health Organization. From 30 million cases to zero: China is certified malaria-free by WHO. https://www.who.int/news/item/30-06-2021-from-30-million-cases-to-zero-china-is-certified-malaria-free-by-who. 30th June, 2021

  5. Lover AA, Baird JK, Gosling R, et al. Malaria elimination: time to target all species. Am J Trop Med Hyg. 2018;99(1):17–23. https://doi.org/10.4269/ajtmh.17-0869.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Liu J, Liang W, Jing W, et al. Countdown to 2030: eliminating hepatitis B disease, China. Bull World Health Organ. 2019;97(3):230–8.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Zou L-P, Yang G, Ding Y-X, et al. Two decades of battle against polio: opening a window to examine public health in China. Int J Infect Dis. 2010;14(Suppl 3):e9–13.

    Article  PubMed  Google Scholar 

  8. Bahl S, Bhatnagar P, Sutter RW, et al. Global polio eradication-way ahead. Indian J Pediatr. 2018;85(2):124–31. https://doi.org/10.1007/s12098-017-2586-8.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Vallée A, Gabet A, Grave C, et al. Patterns of hypertension management in France in 2015: the ESTEBAN survey. J Clin Hypertens (Greenwich). 2020;22(4):663–72. https://doi.org/10.1111/jch.13834.

    Article  Google Scholar 

  10. Kumosani TA, Al-Malki AL, Razvi SS, et al. Hemorrhagic fever in Saudi Arabia: challenge to public health, effective management and future considerations. Afr Health Sci. 2020;20(3):1153–63. https://doi.org/10.4314/ahs.v20i3.17.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jiapeng Lu, Yuan Lu, Wang X, et al. Prevalence, awareness, treatment, and control of hypertension in China: data from 17 million adults in a population-based screening study (China PEACE Million Persons Project). Lancet. 2017;390(10112):2549–58. https://doi.org/10.1016/S0140-6736(17)32478-9.

    Article  Google Scholar 

  12. Feng N, Edwards JK, Owiti PO, et al. Operational research capacity building through the Structured Operational Research Training Initiative (SORT-IT) in China: implementation, outcomes and challenges. Infect Dis Poverty. 2021;10(1):1–4. https://doi.org/10.1186/s40249-021-00865-w.

    Article  Google Scholar 

  13. O’Donnell S, Doyle G, O’Malley G, et al. Establishing consensus on key public health indicators for the monitoring and evaluating childhood obesity interventions: a Delphi panel study. BMC Public Health. 2020;20:1733. https://doi.org/10.1186/s12889-020-09814-y.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Gibney S, Sexton E, Shannon S. Measuring what matters: achieving consensus on a positive aging Indicator set for Ireland. J Aging Soc Policy. 2018;8(1–6):10. https://doi.org/10.1080/08959420.2018.1485393.

    Article  Google Scholar 

  15. Lippeveld T, Sauerborn R, Bodart C. Design and implementation of health information systems. Geneva: World Health Organization; 2000. https://apps.who.int/iris/handle/10665/42289. (Accessed 15/09/2020.)

  16. United Nations Industrial Development Organization. THE SDGS IN ACTION. https://www.undp.org/sustainable-development-goals.

  17. The African Capacity Building Foundation. African Union Agenda 2063: Capacity Requirements for the New African Vision: Agenda 2063-“The Africa We Want”. February 2016. The African Capacity Building Foundation. Zimbabwe.

  18. National Bureau of Statistics of China. 1–1 Divisions of Administrative Areas in China (End of 2020). China Statistical Yearbook 2021. http://www.stats.gov.cn/tjsj/ndsj/2021/indexch.htm.

  19. National Health Commission of the People’s Republic of China. China Health Statistics Yearbook 2021. Beijing. Peking Union Medical College Press. Beijing. 2021. pp 3–4.

  20. Suo-Wei Wu, Chen T, Pan Qi, et al. Establishment of a quantitative medical technology evaluation system and indicators within medical institutions. Chin Med J. 2018;131(11):1327–32. https://doi.org/10.4103/0366-6999.232804.

    Article  Google Scholar 

  21. Dai F, Wei K, Chen Y, et al. Construction of an index system for qualitative evaluation of undergraduate nursing students innovative ability: a Delphi study. J Clin Nurs. 2019;28(23–24):4379–88. https://doi.org/10.1111/jocn.15020.

    Article  PubMed  Google Scholar 

  22. Lin M, Zeng L, Huang L, et al. Application of Delphi method and analytic hierarchy process to establish indicator system for evaluation of rational drug use in children with primary nephrotic syndrome: observational study. Medicine (Baltimore). 2020;99(19): e19949. https://doi.org/10.1097/MD.0000000000019949.

    Article  Google Scholar 

  23. Jones J, Hunter D. Qualitative research: consensus methods for medical and health services research. BMJ. 1995;311(7001):376–80. https://doi.org/10.1136/bmj.311.7001.376.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Tang W, Hu J, Zhang H, et al. Kappa coefficient: a popular measure of rater agreement. Shanghai Arch Psychiatry. 2015;27(1):62–7. https://doi.org/10.11919/j.issn.1002-0829.215010.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Rigby AS. Statistical methods in epidemiology. V. Towards an understanding of the kappa coefficient. Disabil Rehabil. 2000;22(8):339–44. https://doi.org/10.1080/096382800296575.

    Article  CAS  PubMed  Google Scholar 

  26. Ventura-León J, Peña-Calero BN. The world should not revolve around Cronbach’s alpha more than 70. Adicciones. 2020. https://doi.org/10.20882/adicciones.1576.

    Article  Google Scholar 

  27. Ventura-León J. Is this the end for Cronbach’s alpha? Adicciones. 2019;31(1):80–1. https://doi.org/10.20882/adicciones.1037.

    Article  PubMed  Google Scholar 

  28. Ambalavanan AK, Devarakonda MV. Using the contextual language model BERT for multi-criteria classification of scientific articles. J Biomed Inform. 2020;112(12):1–8. https://doi.org/10.1016/j.jbi.2020.103578.

    Article  Google Scholar 

  29. Zhu R, Tu X, Huang JX. 5-Utilizing BERT for biomedical and clinical text mining. In: Data analytics in biomedical engineering and healthcare. Academic Press, 2021. 73–103. https://doi.org/10.1016/B978-0-12-819314-3.00005-7.

  30. Jiang S, Qian X, Shen J, et al. Author topic model-based collaborative filtering for personalized POI recommendations. IEEE Trans Multimed. 2015;17(6):907–18.

    Google Scholar 

Download references

Acknowledgements

The authors wish to thank all the experts who participated in the survey to contribute their opinions to the indicator system as well as their institutes. Also, the authors would like to thank Mr. Shuo Chen, Clerk, Human Resources and Social Security Bureau of South District in Qingdao, Shandong Province, China; Miss Yu-fei Wang, International Department, China Foundation for Poverty Alleviation and Miss Xiao-li He, Intermediate accountant, Xiamen City Center for Disease Control and Prevention to facilitate the collection of the questionnaires from the experts participating in the survey.

Funding

The source of funding for the research comes from China’s Public Health Development Assistance Capacity Building Program by China CDC. The funding body has financially supported the design of the study and collection, analysis, and interpretation of data and writing the manuscript that should be declared.

Author information

Authors and Affiliations

  1. Center for Global Public Health, Chinese Center for Disease Control and Prevention, Room 211, 155 Changbai Road, Changping District, Beijing, People’s Republic of China

    Ning Feng

  2. Institute of Child and Adolescent Health, School of Public Health, Peking University, Beijing, People’s Republic of China

    Yanhui Dong

  3. Department of Infectious Diseases, Zhejiang Provincial Center for Disease Control and Prevention, Hangzhou, Zhejiang, People’s Republic of China

    Shelan Liu

  4. National Institute for Viral Disease Control and Prevention, Chinese Center for Disease Control and Prevention, Beijing, People’s Republic of China

    Xiaoping Dong

Authors
  1. Ning Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanhui Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shelan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoping Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FN has engaged in the design of the research and collection, analysis, and interpretation of data and writing the manuscript. DYH has engaged in the analysis and interpretation of data. LSL, has engaged in the writing the manuscript. DXP has engaged the technical support to the designing the research and collection of data. All authors have read and approved the manuscript, and ensure that this is the case.

Corresponding author

Correspondence to Xiaoping Dong.

Ethics declarations

Ethics approval and consent to participate

This research was conducted according to the guidelines laid down in the Declaration of Helsinki. The aim and content of the research was explained to the experts prior to their participation of the survey. The experts participating of the research in the survey is regarded as informed consent which has been noted at the beginning of the questionnaire. There was a written expression on the title page of the questionnaire as: Participation in the questionnaire is voluntary, can be suspended at any time. The experts who didn't agree with the questionnaire would quit on their own. There is no biological sample collected from the experts who participated the questionnaire survey, neither individual information nor its correlation with the answers was publicized, data analysis focused on expert group in total but not the individual perspective. In China, there is no related provision to require that sociological survey research should go through ethics review. So, the ethics committee’s approval on the research was not mandatory.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Supplementary Information

Additional file 1

. Questionnaire on the construction of Chinese indicator system of public health field investigation and short-term study Hub.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, N., Dong, Y., Liu, S. et al. The construction of Chinese indicator system on public health field investigation and short-term study hub: experience and implications. glob health res policy 7, 40 (2022). https://doi.org/10.1186/s41256-022-00273-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41256-022-00273-z

Keywords