Table 1 Summary of evaluation design and methods
Step | Method/approach | Analysis and expected outcome |
|---|---|---|
Step 1: Context analysis: Analyze country context, state fragility, and health system contexts of the aid interventions under evaluation | Qualitative approach through realist synthesis methods | Identify drivers and barriers from state fragility and health system that can be shaped by the interventions and can influence intervention outcomes |
Step 2: Program design and mechanism analysis: Analyze program designs, elicit underlying program theories, and identify how aid interventions interact with the state fragility and health system context | Qualitative-interpretivist approach and constructivist-mixed methods approach through document review, semi-structured interviews of stakeholders; focus group discussions | Using both qualitative and quantitative analysis, elicit underlying program theories of each intervention and their implementation mechanisms. Construct stakeholders’ views on the extent each intervention adhered to different aspects of the Paris principles |
Step 3: Outcome analysis: Analyze program outcomes and their values in terms of achievement of targeted objectives and effect on health system | Flexible methods depending on type and purpose of an intervention. A suggested example is: quantitative analysis of cost-effectiveness of outcomes using epidemiological modeling exercise with actual, counterfactual and optimum scenario modeling | Analysis of degree of achievement of targeted outcomes, cost effectiveness and technical efficiency of each program |
Step 4: Comparative evaluation of inputs, process, and outcomes | Realist synthesis with comparative analysis of relevance and adherence to the Paris principles, efficiency, effectiveness, and likely sustainability of each intervention | Interpret significance of findings by comparing them with program theories, empirical evidences from qualitative and quantitative analysis, and plausible context-mechanism-outcome interactions. Investigate possible causal chain between adherence to the Paris principles and programmatic effectiveness |