Mercury Project Spotlight: Jeremy Bowles, Kevin Croke, Horacio Larreguy, Shelley Liu, John Marshall, and Africa Check
In today’s Mercury Project Spotlight, we feature EGAP members, Horacio Larreguy (Instituto Tecnológico Autónomo de México) and John Marshall (Columbia University) along with their co-authors, Jeremy Bowles (Stanford University), Kevin Croke (Harvard University), and Shelley Liu (UC Berkeley). This team with Antonella Bandiera (ITAM), Romain Ferrali (Aix-Marseille School of Economics), and Daniela Pinto Veizaga (Tecnológico de Monterrey) was recently awarded a grant from the Social Science Research Council’s Mercury Project.
The Mercury Project teams consist of 12 grantees working on projects to increase take-up of vaccines and essential health services across 17 countries. EGAP is also a recipient of this grant as part of the Health Ambassadors in sub-Saharan Africa project, a research collaboration between EGAP, Partnership for African Social and Governance Research (PASGR) in Nairobi, and Centre de Recherche et d’Action pour la Paix (CERAP) in Abidjan.
Larreguy and Marshall’s team will examine the effects of digital literacy training and fact-checking dissemination by social media influencers on misinformation in three countries: Bolivia, Kenya, and South Africa. Their project, “Harnessing Influencers to Counter Misinformation” is an extension of previous research carried out by the team in collaboration with two fact-checking organizations, Africa Check and Chequea Bolivia. In this spotlight, we ask Larreguy and Marshall and their colleagues about the findings from their working paper, “Sustained exposure to fact-checks can inoculate citizens against misinformation in the Global South,” that will serve as a foundation for the Mercury Project work. We also spoke with Hlalani Gumpo, Impact Manager, from Africa Check, the first fact-checking organization in sub-Saharan Africa.
The study examines the effects of sustained exposure to fact-checks produced by Africa Check on citizen’s susceptibility to misinformation in South Africa. Treated study participants received fact-checks on various topics, including COVID-19, politics, economy and crime through WhatsApp messages every two weeks over a period of six months. Within the treatment, participants received fact-checks through four different modes of delivery: a short text message, short podcast, long podcast, and an empathetic variant of the podcast emphasizing the narrators’ understanding of fears and concerns that may lead individuals to believe false news.
The research team finds that the repeated exposure to fact-checks increased the ability of respondents to discern between true and fake news and increased their skepticism of conspiracy theories. In addition, after the sustained exposure to fact-checks, participants were more likely to doubt social media content and less willing to share it. The simplest mode of fact-check delivery, short text messages, along with the podcast with empathetic language, had the largest effects on reducing beliefs in and spread of misinformation. We asked the study authors and Africa Check about their findings and next steps for their project.
In your field experiment, treated participants received three fact-checks on a biweekly basis over a period of six months. Why did you choose to focus on the effects of sustained exposure to fact-checks?
Jeremy Bowles, Kevin Croke, Horacio Larreguy, Shelley Liu, and John Marshall (study authors): The lack of studies featuring sustained exposure seemed like a major gap in the literature to date. There are a number of papers which examine the effects of shorter run interventions, often delivered in artificial settings such as a survey experiment or a lab environment. While there are sometimes positive effects from such interventions, there is also evidence that these effects tend to fade over time. In contrast, a key mechanism – especially if inducing broader inoculation against misinformation – might be repeated exposure to fact-checks, which could help individuals “learn by doing” what to look for. Assessing whether sustained interventions over a longer term can produce more durable effects was thus a major impetus for this study.
Can you briefly describe misinformation in the South African context?
Hlalani Gumpo: Digital mis- and disinformation are becoming an increasingly common feature of Africa’s domestic political landscape. Mis- and disinformation are especially widespread during national election cycles. There are a number of elections intended to restart democratic processes and resume constitutional governance across the continent in 2022/2023 – South Africa among them – and online platforms are replete with examples of posts claiming a politician said something they didn’t. There is also a great deal of mistrust in political news. Previous research (Wasserman and Madrid-Morales 2019) in South Africa has found a high perceived exposure to disinformation – higher than in the USA – and low levels of trust in social and national media. This trust deficit may add to the allure of disinformation in the country, as readers turn to simple explanations to complex issues or information that confirms their political biases.
Study authors: Misinformation is an issue that the South African government has highlighted as a challenge, especially in the context of the COVID-19 pandemic. For example, the government has set up websites where citizens can report COVID misinformation, such as http://www.real411.org/, and made disseminating such information punishable by law. In addition, one feature that South Africa shares with other middle-income and low-income countries is that many internet users purchase more affordable mobile data packages which let you only access web content via social media. As a result, 70% of South Africa’s adult population uses WhatsApp, but some of these users have very limited access to the rest of the Internet. These users can be very vulnerable to misinformation that spreads via WhatsApp because they may not be receiving media from other sources.
This project was a result of a collaboration between the research team and Africa Check. How and why did you establish the partnership?
Study authors: For this project, we received funds from the Harvard Data Science Initiative (HDSI)’s Trust in Science Project. Initially, following up on a previous project with Kubatana in Zimbabwe, the main goal was to evaluate the effectiveness of a WhatsApp Chatbot that allowed Zimbabwean to fact check likely misinformation via WhatsApp. To decide which misinformation to flag on the bot, we thought to also partner with a fact checker. As the oldest fact-checking organization in the African continent, Africa Check is a very reputable organization, and was thus an obvious candidate for the partnership we sought. After initial conversations, Africa Check brought up that they would be interested in also evaluating their initiatives to counter misinformation.
Hlalani, what are some projects that Africa Check has previously implemented to combat misinformation?
HG: Africa Check implements a 360-degree approach to reduce the spread and impact of misinformation. There are certainly no simple solutions to disrupt the complex problem of misinformation, and we believe a multi-pronged approach, such as ours, is the most effective way to improve the quality of information. Simply fact-checking claims is not enough. Our approach thus focuses on the following activities: fact-checking claims made by public figures and the media; training media and other organizations to carry out non-partisan fact-checking themselves; providing fact-checking tools, tips, guides and accurate data on key topics to media, policymakers and the public; engaging with conduits of misinformation (social media platforms and search engines) to identify false claims and present facts; and promoting the practice of fact-checking and fostering awareness of our work.
Participants in the treatment group received the fact-checks via four distinct modes of delivery. Your findings reveal that the mode of conveying fact-checks shapes their effectiveness. Can you elaborate on this?
Study authors: This aspect of the project had both a practical dimension (it could help Africa Check optimize their delivery) but also spoke to some questions of theoretical interest, such as whether certain approaches which foreground empathy for the listener might be especially effective as well as whether information is more effectively delivered in short or longer forms. Depending on their treatment group, recipients could receive the information in text form, as a short podcast, as a standard (“long”) podcast, or the standard podcast augmented with some additional “empathetic” language. What we found is that the text and the empathetic podcast outperformed the standard podcast and the shorter version of the standard podcast. We think that there are two groups of compliers: (i) those that have limited time and thus only consume fact checks if they can easily access them whenever they have a break, and (ii) those who are attracted by more elaborated podcasts, but whose sustained engagement is only feasible if they include empathetic language.
Treated participants, specifically those who received the fact-checks via short, simple text messages, displayed a shift in COVID-19 beliefs including an increase in willingness to get vaccinated and decreased skepticism that COVID-19 is a hoax. What are the implications of these findings on the potential of fact-checks to transform attitudes and behaviors?
Study authors: Overall, we saw very positive effects on knowledge outcomes (respondents learned how to distinguish true from false new stories, and how to recognize conspiracy theories), and as you rightly note, there was also some evidence of COVID-related attitudinal and self-reported behavior change. One thing to emphasize about these results is that only one of the treatment arms produced this change – the text treatment. While there is still more to unpack on these outcomes, we were quite encouraged by the positive effects on willingness to get vaccinated. Indeed, much of the literature seeking to combat misinformation has focused on outcomes that are quite close to treatment – both in terms of time since treatment as well as logical distance between information and action/belief. Consequently, finding that fact-checks covering a wide variety of different topics could shift behaviors not directly linked to most of the fact-checks is quite notable – and suggests that sustained intervention produced an inoculating effect.
You plan to continue your work on the effects of sustained exposure to fact-checks in collaboration with Africa Check through your project funded by SSRC’s Mercury Project. For this project, “Harnessing Influencers to Counter Misinformation: Scalable Solution in the Global South,” you plan to provide digital literacy training resources and fact-checks to 600 social media influencers and study their effects on countering misinformation. How will the findings from this paper contribute to your upcoming project?
Study authors: Beyond helping to secure funding from the Social Science Research Council Mercury Fund, our findings will shape this future work in at least two ways. First, the new grant will allow us to conduct a follow-up survey to assess longer-term effects of the intervention we’ve been discussing. Second, the fund also includes a budget to assess whether social media influencers can be recruited to disseminate Africa Check and Chequea Bolivia’s fact-checking and digital-literacy-training content at a higher scale. Since this is a cheap and scalable way of combating misinformation, it is important to rigorously evaluate.