Methodological aspects in the construction of a composite indicator of service delivery in Uganda

Abstract

This study elaborates methodological aspects encountered in the building of composite indicators with application to service delivery in Uganda. This study amplified the three main stages, the selection of quality data, the building of the composite indicator itself and statistical approaches that may be utilized to model service delivery composite indicator (CI). This study formulated a data quality assessment framework (DQAF) to enhance the construction of a composite indicator. The DQAF was formulated with a dual orientation that prioritizes two user-oriented data quality components (DQCs) namely; relevance, and interpretability, and three producer-oriented DQCs of methodological soundness, accuracy, and statistical adequacy. The application of the DQAF to service delivery data resulted in the selection of 51 from a pool of 103 potential indicators, reflecting a 48.6% acceptability percentage. The composite indicator for statistical regions, which included five dimensions—education, health, water, agriculture, and roads—was developed utilizing official data from the 2021 National Service Delivery Survey conducted by the Uganda Bureau of Statistics, along with various sector performance reports from the Ministry of Health and the Ministry of Water and Environment. Additionally, the study developed an alternative composite indicator for district local governments, concentrating on the education, health, and water dimensions, which was modeled against potential covariates. The composite indicator for statistical regions indicated that Uganda achieved a score of 0.49 (0 ≤ composite indicator score ≤ 1) utilizing equal weighting, minimax transformation, and additive aggregation, whereas the score was 0.45 with equal weighting, distance-to-reference point transformation, and geometric aggregation. Min-max transformation yields higher scores compared to distance-to-reference point, attributable to the exogenously determined goalposts. Weights that are participatory determined were comparable with data-derived weights. Robustness tests demonstrated that the constructed composite indicator exhibited stability and can therefore be utilized. The absolute differences in ranks by region were observed, with Kampala and Lango exhibiting the lowest differences and Karamoja and Kigezi the highest, attributable to the presence of outliers and inequitable performance in the examined variables. The aggregation stage was the most sensitive accounting for nearly 60% of the total variance, primarily due to interaction with mainly the transformation stage; this underscores the necessity to cautiously select an aggregation method, as it greatly influences the robustness of the results. The absolute rank differences were highest in the education dimension at 2.00 and lowest in the roads and health dimension at 1.33, indicating the varying impact of excluding aspects from the composite index. In assessing the differentials of service delivery at local government level, the composite indicator scores ranged from 0.25 to 0.60, with a substantial portion of the density plot situated below 0.50, indicating inadequate service delivery levels. While beta regression adeptly models bounded data, random forest regression highlights the relative importance of predictors, and generalized additive model captures non-linear covariate effects. The comparable predictive accuracy of these methods, as evaluated using root mean square error, suggests their applicability to this investigation in accordance with the analytical objectives. It is recommended that data quality assessment frameworks should encompass producer and user data quality components that are developed collaboratively with potential users of the composite indicator, in addition to identifying and addressing redundancies among them. Given that the aggregation stage was the most sensitive, it is recommended to explore the use of penalization techniques to address the substitution of variables while maintaining official statistics.