Journal Design Engineering Masthead
African Civil Engineering Journal | 26 June 2022

Replication and Methodological Evaluation of Quasi-Experimental Design for Manufacturing Systems Yield Optimisation in South Africa

T, h, a, n, d, i, w, e, N, k, o, s, i, ,, P, i, e, t, e, r, v, a, n, d, e, r, M, e, r, w, e, ,, K, a, g, i, s, o, M, o, k, o, e, n, a, ,, A, n, j, a, d, e, V, i, l, l, i, e, r, s
Quasi-experimental designReplication studyYield optimisationManufacturing systems
Direct replication shows attenuated 4.7% yield effect versus original study
Inter-group contamination identified as key threat to internal validity
Interrupted time-series design requires stricter isolation of control units
Robust standard errors essential for serial correlation in time-series data

Abstract

{ "background": "Quasi-experimental designs (QEDs) are increasingly employed in engineering to evaluate interventions in operational settings where randomisation is impractical. Their application in manufacturing systems for yield optimisation, particularly within an emerging economy context, requires rigorous methodological validation to ensure causal inferences are robust.", "purpose and objectives": "This study replicates and critically evaluates a published QED methodology for measuring yield improvement. The primary objective is to assess the internal validity and practical implementation of the design in South African manufacturing plants, focusing on its robustness to common threats such as selection bias and history.", "methodology": "We executed a direct replication across three automotive component manufacturing plants, applying the same interrupted time-series design with a non-equivalent control group. The core statistical model for the treatment group is $Yt = \\beta0 + \\beta1Tt + \\beta2Dt + \\beta3(Tt \\times Dt) + \\epsilont$, where $Yt$ is yield, $Tt$ is time, and $D_t$ is the intervention dummy. Inference was based on Newey-West robust standard errors to account for serial correlation.", "findings": "The replication confirmed a positive treatment effect but of a smaller magnitude than the original study, with a point estimate of a 4.7% yield increase (95% CI: 2.1% to 7.3%). Methodological evaluation revealed significant contamination between control and treatment groups in two plants, substantially inflating the original effect size estimate.", "conclusion": "While the QED framework is viable, its application in interconnected manufacturing environments is highly sensitive to inter-group contamination. The attenuated effect suggests earlier findings were partially confounded.", "recommendations": "Future applications must incorporate stricter physical or procedural isolation of control units and more frequent pre-intervention sampling to better model underlying trends. Sensitivity analyses for contamination should be mandatory.", "key words": "quasi-experimental design, replication, manufacturing yield, interrupted time series,