Journal Design Clinical Emerald
African Food Systems Research (Interdisciplinary - incl Agri/Env) | 07 February 2024

A Methodological Evaluation and Difference-in-Differences Analysis of Public Health Surveillance System Adoption in Rwanda, 2000–2024

C, l, a, u, d, i, n, e, M, u, r, e, k, a, t, e, t, e, ,, J, e, a, n, d, e, D, i, e, u, U, w, i, m, a, n, a, ,, M, a, r, i, e, C, h, a, n, t, a, l, U, w, a, s, e, ,, J, e, a, n, P, a, u, l, M, u, g, a, b, o
Causal InferenceHealth SystemsSurveillance EvaluationRwanda
Critique reveals a predominant reliance on cross-sectional designs lacking counterfactual reasoning in surveillance evaluation.
Difference-in-differences analysis estimates a 23.4 percentage point increase in reporting completeness following system adoption.
Effect was statistically significant (p<0.001) and robust to multiple sensitivity checks.
Advocates for integrating quasi-experimental designs into routine surveillance system assessments.

Abstract

{ "background": "The methodological rigour of evaluating national public health surveillance systems in low-resource settings remains underdeveloped, limiting the ability to attribute health outcomes to specific system implementations.", "purpose and objectives": "This study aimed to conduct a methodological evaluation of surveillance system assessments and to quantify the causal impact of a major integrated disease surveillance and response (IDSR) adoption on reporting completeness.", "methodology": "We performed a systematic methodological critique of evaluation frameworks. A quasi-experimental difference-in-differences (DiD) design was employed, using panel data from administrative districts. The core model was $Y{it} = \\beta0 + \\beta1 (\\text{Treat}i \\times \\text{Post}t) + \\gammai + \\deltat + \\epsilon{it}$, where $\\gammai$ and $\\deltat$ are district and time fixed effects. Inference was based on cluster-robust standard errors at the district level.", "findings": "The methodological review identified a predominant reliance on cross-sectional designs lacking counterfactual reasoning. The DiD analysis estimated that IDSR adoption increased reporting completeness by 23.4 percentage points (95% CI: 18.1 to 28.7). This effect was statistically significant (p<0.001) and robust to multiple sensitivity checks.", "conclusion": "The adoption of the integrated system caused a substantial and significant improvement in surveillance reporting metrics. Methodological approaches incorporating causal inference designs are critical for generating credible evidence of public health programme impact.", "recommendations": "National health ministries should integrate quasi-experimental evaluation designs into routine surveillance system assessments. Future research should apply similar causal inference methods to evaluate the impact on direct health outcomes, such as outbreak detection timeliness.", "key words": "public health surveillance, impact evaluation, difference-in-differences, causal inference, health systems, reporting completeness", "contribution statement": "This paper provides a novel application of a robust quasi-experimental design