Brief 69: Bottom-up accountability and public service provision in Brazil

EGAP researchers: Danilo Freire, Manoel Galdino, Umberto Mignozzetti

Geographical region: Latin America and the Caribbean

Research question: Does bottom-up, citizen-based monitoring improve public service delivery?

Preparer: Tanu Kumar

July 27, 2020

Background

In 2017, Transparência Brasil developed a cell phone application called Tá de Pé (TDP) to improve the accountability of government education spending to citizens (Figure 1). The TDP app collects crowdsourced data from citizens on school construction project completion rates in their neighborhoods to be assessed by a group of independent specialists. If the construction is behind schedule, the app provides a platform for citizens to make an anonymous complaint to the mayor. If the mayor’s office does not respond to the notice within 15 days, the notice is forwarded to the Brazilian Ministry of Education, which makes it more difficult for the municipality to access federal funds in the future. Transparência Brasil launched the app for Android in August 2017, and a version for the iOS platform came out six months later. A Facebook campaign to publicize the app successfully attracted 2028 new users within October 2017 alone.

Research Design

Between August 2017 and July 2019, the authors fielded two experiments to measure the effects of TDP on the following outcomes:

1. The percentage of school construction projects completed before the intervention (a placebo)
2. The percentage of school construction projects completed after the intervention
3. The difference between the percentage reported completed before and after the intervention
4. The number of completed projects
5. The number of cancelled projects
6. The number of projects for which construction companies updated completion dates

The first experiment ran from August 2017 to July 2018 on the Android platform only. Randomization occurred at the municipal level, with 2642 municipalities in the treatment group and 344 municipalities in the control group. For municipalities in the control condition, all information about school construction was removed from the TDP app. 

The second experiment ran from August 2018 to July 2018 on both the Android and iOS platforms. Randomization occurred at the school level, with 3717 schools in the treatment group and 659 schools in the control group. Randomization was stratified by state, school construction status, and municipal school spending level relative to the median.

Results

In the first experiment, the authors found that the treatment condition caused a 2.07% increase in cancellation rates, an effect that goes in the opposite direction from their hypotheses. This effect did not replicate in the second experiment. All other coefficients were not statistically significant at conventional levels, in either experiment.

The null results do not arise from low engagement with the app, as data from Google Analytics shows that each user launched, on average, 60 app sessions during the study period. 

Furthermore, coefficients are generally small and negative, suggesting that null results do not arise from low statistical power. Not only would the intervention have a small impact if the coefficients were statistically significant, but they would also go in the opposite direction of hypotheses generated by theories of bottom-up accountability.

Policy Implications

The findings do not provide evidence for a relationship between bottom-up accountability and local public service delivery. One possible reason for null results is general cuts in public investment in Brazil. These cuts, introduced by the federal government, reduced the ability of citizens to blame local politicians for the under-provision of public goods. The experiment was also fielded right after Brazil’s municipal elections, potentially leading to weak electoral incentives among mayors to respond to complaints. Overall, however, the experimental results suggest that accountability interventions that target elite groups, such as lobbyists or civil servants, may yield better school construction outcomes than those focused on citizens. Digital interventions, furthermore, may not be as effective as face-to-face communication. Whether the results generalize beyond school construction programs remains to be studied.