Brief 58: Does Bottom-Up Accountability Work?

EGAP researchers: Doug Parkerson, Daniel Posner, Pia Raffler

Partners: Innovations for Poverty Action , GOAL Uganda, Coalition for Health Promotion and Social Development (HEPS Uganda), Kabarole Research and Resource Centre (KRC), Multi-Community Based Development Initiative (MUCOBADI)

Geographical region: Africa

Research question: Can bottom-up accountability generate improvements in health outcomes?

May 8, 2019

Background

Poor service delivery, particularly in primary health care, is a major problem in developing countries. This is also the case in Uganda. Although the country has recently seen a decline in child, infant, and neonatal mortality rates, the country still falls in the bottom quartile of the world distribution on most basic health measures.

In recent years, development funders and practitioners have embraced a potentially promising approach to the problem of poor service provision: the bottom-up monitoring of service providers by community members. Bottom-up monitoring provides citizens with information about service delivery shortfalls, along with information allowing them to compare local outcomes with national standards and with outcomes in other communities. Motivating the enthusiasm toward bottom-up accountability is the success of the widely praised 2009 Power to the People (P2P) program, which generated striking results on a smaller scale. The ACT intervention is a scaled-up program modeled after the P2P program.

The ACT Health intervention consisted of a Citizen Report Card (CRC) provided to health centers that contained citizen attitudes about their care and treatment, information services offered at the health center, and utilization rates, alongside district averages to offer benchmarks of relative performance. The CRC’s were then discussed during community meetings with local leaders and village health team members, which then lead to the development of an action plan with specific steps to improve service delivery. Parallel meetings with health center staff were also organized to identify steps that the staff could take to improve care and treatment. The ACT Health program then brought the health staff together with representatives of the community to discuss how they might work together to improve the quality of health care in the community. The output of the meetings was a social contract between the citizens and health care workers laying out specific steps that each could take to contribute to improvements in health outcomes. ACT Health implementation teams returned every six months to meet with community members and healthcare staff to check on the progress made toward the commitments in the social contract.

Research Design

This study took place in 16 districts across four regions of Uganda. The sample included 376 health centers, which encompasses nearly every functioning government-run health center in the study districts. In coordination with Innovations for Poverty Action (IPA), researchers conducted a randomized evaluation to measure the impact of the full ACT Health program and individual components of the program on utilization rates, treatment quality, patient satisfaction, and health outcomes, including child mortality. The unit of randomization is the health center and its associated catchment area. Before the program began, the research team collected data about health delivery in facilities in each of the communities and used this information to write CRCs. Then, researchers randomly assigned the 376 health centers and their catchment areas to one of four groups:

1. Full ACT Health program: information, mobilization and interface (92 health centers)
2. Partial ACT Health program: information and mobilization (92 health centers)
3. Partial ACT Health program: interface only (97 health centers)
4. Comparison: No program at the time of study (95 health centers)

The research team conducted three rounds of data collection using a household survey, a health clinic survey, and administrative data from the health centers (verified with physical checks of drug stocks and storage conditions). Surveys were conducted before the program was rolled out, and one and two years later.

Results

Overall, communities participating in the full ACT Health program reported receiving marginally better care and were more satisfied with their care than communities that did not receive any program. However, in striking contrast to what was found in P2P, no version of the program improved health outcomes or increased the number of people who sought care. In households that received the full ACT Health program, quality of care improved by 0.059 standard deviations, a very small effect. People in ACT Health program communities were more likely to report having had privacy during their most recent exam and having had their diagnosis clearly explained to them. Health centers in program communities were also less likely to have had stock outs of key drugs during the previous three months. Patient satisfaction overall improved by 0.079 standard deviations, also a very small effect. In addition, exposure to the program did not increase citizen monitoring, as the theory of change motivating the intervention would have expected.

Policy Implications

Results from the ACT Health evaluation provide no evidence that the small improvement in treatment quality was caused by an increase in monitoring or bottom-up pressure by community members. The stark contrast in results between the ACT Health and P2P programs suggest that subtle differences in implementation may provide an explanation for divergent findings. Another likely explanation is that health outcomes and service delivery have much improved in Uganda during the ten years that separated the two studies. It may have been easier for the P2P intervention to foster change when baseline levels were worse.

In sum, the findings suggest a combination of information provision and increased oversight can marginally change the behavior of frontline service providers, but cast doubt on the power of information to foster community monitoring or to generate improvements in health outcomes, including child mortality, at least in the short term.