Priority Theme Spotlight: David Broockman and Joshua Kalla

April 29, 2022

Theme: Democracy, Conflict, & Polarization

In today’s Priority Theme Spotlight, we spoke with EGAP members, David Broockman (UC Berkeley) and Joshua Kalla (Yale University) about their working paper, “The manifold effects of partisan media on viewers’ beliefs and attitudes: A field experiment with Fox News viewers.”

Broockman and Kalla share their results from a field experiment conducted in 2020 measuring the impacts of partisan media on viewers’ beliefs and attitudes. By paying regular viewers of Fox News to watch CNN for 30 days, they found evidence of partisan coverage filtering, i.e. networks selectively reporting information based on whether or not that information is beneficial to their partisan alignments. Moreover, they found that changing the slant of partisan media viewers’ media diets has significant effects on the participants’ beliefs, attitudes, perceptions of issues’ importance, and overall political views.

What was the motivation behind this project, and how does it fit into your broader research agenda?

David Broockman and Joshua Kalla: Broadly speaking, in our research we seek to understand the persuasive effects of elite political communication in American politics. Our prior research has focused primarily on the effects of campaign outreach to voters, such as through TV ads and canvassing. Given the importance of the media in American politics, we wanted to expand our research agenda on political persuasion to include studies of media effects.

We know from several natural experiments that the partisan slant of cable news, notably Fox News, has had electoral impacts. Yet, we know little about what changes in viewers’ beliefs, attitudes, or priorities may underlie these shifts in voting behavior and whether there may be broader implications for American democracy.

Coming into this project, one big suspicion we had is that partisan media networks wouldn’t “tell the whole story” – especially when it came to mistakes their side might be making. Inspired by this hypothesis, our project investigates ways in which partisan outlets selectively report information favorable to their side, leading viewers to learn a biased set of information, something we refer to as partisan coverage filtering. We discuss this more below.

This study looks at the effects of partisan media on beliefs and attitudes. You conducted a field experiment, in which you incentivized Fox News viewers to watch CNN for 30 days. Tell us about your experimental design.

DB and JK: There are many potential experimental designs one could use to study the effects of partisan cable news. Given the realities of a budget constraint and the need for sufficient statistical power, we needed to pick just one experimental design, with one treatment group and one control group. Since we conducted this experiment in 2020 when Donald Trump was president, we wanted to focus the experiment on Fox News (a network generally aligned with Trump) and any effects it may have on incumbent accountability. To do that, we paid regular viewers of Fox News to watch CNN (which also reduced their Fox News consumption).

Specifically, we first used data from internet-connected (smart) televisions to identify households that, at baseline, were heavy viewers of Fox News and minimal viewers of CNN or MSNBC. We then sent these households a letter inviting them to go online to take a paid survey. In that survey, we gathered baseline demographic and political information, as well as confirming their television viewing habits.

We then invited participants to a second baseline survey, where we told them, “Some people may be selected to earn more than $10 per survey in September if they agree to watch a new channel for a few hours and answer questions about what they saw.” We let them select up to 7 prime time hours when they would be willing to watch CNN.

Among participants willing to be paid to watch CNN, we then block-randomized at the household level to either a treatment group that was paid $15 per hour to watch CNN for four weeks (median of 7 hours per week from Aug. 31 – Sept. 25, 2020) or to a control group that received no payment.

Throughout that month, we sent five surveys at random intervals to both treatment and control subjects. To enforce compliance, the surveys the treatment group received also contained several “pop quiz” items about what was on the CNN show they were supposed to be watching. We only paid the treatment group if they correctly answered these “pop quizzes.” On average, treatment group participants answered 12.4 out of 15 pop quiz questions correctly, suggesting very high compliance with watching CNN.

Finally, to assess outcomes, we conducted two post-treatment surveys: a midline survey launched three days after the incentives to watch CNN ended and an endline survey launched several months later. These surveys contained a variety of items about knowledge and perceptions of current events, political attitudes, and evaluations of Donald Trump.

This experimental design was very much inspired by Chen and Yang’s (2019) field experiment on the impact of media censorship in China in which they paid participants to read the Chinese edition of the New York Times.

What were the primary findings from this project?

DB and JK: First, descriptively, we found that Fox News and CNN covered different events and facts during this time. For example, CNN gave substantially more coverage to topics related to COVID and election integrity than Fox News, which instead focused on issues related to race and protests.

Second, we found manifold effects on participants’ factual beliefs, attitudes, perceptions of issues’ importance, and overall political views. A few examples:

The treatment group was far less likely to rate issues as important that Fox News heavily emphasized than the control group. For example, the treatment group rated COVID-19 as more important than the negative consequences of racial protests.
The treatment group learned more information flattering to the liberal side of a story on events that CNN predominantly covered and less information flattering to the conservative side of a story on events that Fox News predominantly covered. For example, the treatment group was less likely to believe that Donald Trump’s campaign was taking significant safety precautions at its rallies to reduce the risk that rally attendees spread the coronavirus to each other (something CNN heavily covered) and less likely to agree that Democrats were trying to steal the 2020 election with fraudulent mail-in ballots (something Fox News heavily covered).
The treatment caused individuals to become more negative in their evaluations towards Donald Trump and Republican politicians. These effects were small, but notable, such as a 3 point drop in a Trump feeling thermometer. However, we did not see any effect on vote choice in the 2020 presidential election.

Third, we also found that the treatment increased unfavorable attitudes towards Fox News. For example, we found a 0.20 standard deviation effect on disagreement with the statement “If Donald Trump did something bad, Fox News would discuss it.” On the flip side, we found no effect on attitudes towards CNN.

(For those interested, there are many more results in the paper. This is just a subset of the findings.)

Overall, these findings were somewhat surprising to us. People who watch cable news tend to be very politically engaged and have strong opinions about politics, limiting the impact of the media. Similarly, they also tend to be strong partisans who might not trust any source not associated with their party. The people in this experiment were overwhelmingly pro-Trump Republicans with negative views of CNN when we started this experiment. For the last few years, they had regularly been exposed to Trump’s attacks on CNN’s credibility. One expectation going into this project, based on the selective exposure literature, was that this audience would completely resist what CNN had to say, leading to no attitude change.

You attribute some of the shift in viewers’ beliefs and attitudes to a form of influence called partisan coverage filtering. Can you tell us a little bit about this mechanism for media influence? What are its implications for democratic accountability?

DB and JK: Partisan coverage filtering is the idea that networks selectively report information based on whether or not that information is beneficial to their partisan alignment. For example, we find that Fox News was far less likely than CNN to cover negative information about Donald Trump, such as his failures around COVID-19. Similarly, CNN was far less likely than Fox News to cover positive information about Donald Trump, such as his accomplishments at getting Israel, the UAE, and Bahrain to normalize relations.

We find that as a result of this partisan coverage filtering, viewers of CNN and Fox News learn different facts and have different understandings of the world. As a result, this has important implications for democratic accountability. When politicians do something bad, we hope that voters will punish them, irregardless of their party. Otherwise, politicians won’t have to work hard to make our lives better in order to keep their jobs. However, this type of behavior becomes less possible if the media engages in partisan coverage filtering. If CNN doesn’t cover bad things Democrats do or good things Republicans do, and if Fox News doesn’t cover bad things Republicans do or good things Democrats do, then voters become less likely to learn this information and less able to hold their elected officials accountable. This also means that elected officials will have less of an incentive to work hard for their constituents. This is troubling for the functioning of a healthy democracy.