close× Call Us
close×
Title The Effect of Redactions on Conspiracy Theory Belief
Post date 08/20/2015
C1 Background and Explanation of Rationale

The objective of this project is to investigate the causes of belief in conspiracy theories. Specifically, we hope to determine whether the presence of redacted text in government documents makes people more suspicious about coverups and hidden evidence and therefore more likely to believe in conspiracy theories.

(Note: This study is a replication of Nyhan et al. EGAP 20140502AA with slightly different introductory text used in the redacted document condition.)

C2 What are the hypotheses to be tested?

Hypotheses:

H1: The presence of redacted text in documents related to a conspiracy theory will increase belief in the conspiracy theory relative to textually identical documents without redactions.
H2: The effects of redactions will vary depending on respondents' predispositions to believe in conspiracy theories. Specifically, the presence of redacted text will increase belief in conspiracy theories relatively more among participants with high conspiracy predispositions than those with low conspiracy predispositions.

Research question:

RQ1: We do not have strong theoretical expectations about the effect of redactions on conspiracy belief relative to a control condition because the comparison confounds exposure to the contents of the documents with exposure to redactions. Absent strong theory about the effects of these documents, which are generally consistent with the official account of the event in question but have failed to quell a conspiracy theory about its causes, we consider the difference in means between the redaction and control condition - as well as the baseline effect of exposure to the contents of the unredacted documents relative to controls - to be a research question and do not propose a formal hypothesis.

(These are identical to EGAP 20140502AA.)

C3 How will these hypotheses be tested? *

Participants in this study are United States residents age 18 and over recruited on the Amazon Mechanical Turk online marketplace to take part in a survey experiments on Qualtrics. Some respondents read documents concerning a topic that is the subject of a conspiracy theory. Respondents are randomly assigned to read versions of the documents that either include putative redactions or not but actually feature identical text. These respondents will be compared to a control group that is randomly assigned to read unrelated documents instead. Random assignment to the three conditions is being conducted with equal probabilities.

We measure conspiracy predispositions using a scale combining responses to two questions from Oliver and Wood (2014): mean agreement on a six-point scale with ""Politics is ultimately a struggle between good and evil"" and ""Much of what happens in the world today is decided by a small and secretive group of individuals."" (If these questions do not scale together, we may interact the treatments with the two predisposition questions separately.)

There are six dependent variables (DV1-DV6) measuring belief in the conspiracy theory that allow responses on a six point scale from ""Very unlikely"" to ""Very likely"". We will recode them so their direction is consistent (specifically, that higher values indicate greater conspiracy belief) and test the effect of our treatments on each as well as a combined scale. (If the questions do not scale together, we may estimate treatment effects on a subset of the dependent variable questions that scale together.)

Specifically, we will run OLS regressions of the following forms in Stata 14 after appropriately recoding the dependent variables (redacted and unredacted are indicator variables for those experimental conditions; the control condition is the omitted group) and then estimate the difference in means between the redacted and unredacted conditions:

reg DV1 redacted unredacted, robust
lincom redacted-unredacted
reg DV2 redacted unredacted, robust
lincom redacted-unredacted
reg DV3 redacted unredacted, robust
lincom redacted-unredacted
reg DV4 redacted unredacted, robust
lincom redacted-unredacted
reg DV5 redacted unredacted, robust
lincom redacted-unredacted
reg DV6 redacted unredacted, robust
lincom redacted-unredacted
reg DVscale redacted unredacted, robust
lincom redacted-unredacted

We will also interact the experimental indicator variables with the conspiracist predisposition scale (redactedXconspdisp, unredactedXconspdisp) and estimate the following models:

reg DV1 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DV2 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DV3 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DV4 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DV5 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DV6 redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust
reg DVscale redacted unredacted consdisp redactedXconspdisp unredactedXconspdisp, robust

To illustrate the substantive effects of these variables, we will create marginal effects plots demonstrating how the estimated effect of the redaction varies relative to the unredacted condition and/or controls over the range of conspiracist predisposition values. If the conspiracist predisposition measure has outliers or a non-normal distribution, we will test that our results are robust to interacting the treatment measures with indicators for respondents with high conspiracy predispositions in a median or tercile split.

(Note: The above are identical to EGAP 20140502AA; the target sample size is identical [n=~2500]. We will replicate the analysis of that data presented in the April 2015 version of the Nyhan et al. manuscript based on this study, which follows the approach described above except for three specified deviations. First, we omit a dependent variable pertaining to “ignition of flammable fuel/air vapors in the fuel tank” from the analysis because that item did not scale with the other dependent variables. Second, we present a dichotomous conspiracy belief measure for ease of exposition as in the original manuscript, though we will again verify that results are identical with either measure. Finally, we will again compare response times for the stimulus documents and outcome variables and the frequency of open text responses between the redacted and unredacted conditions, a set of exploratory analyses prompted by comments on an initial version of the manuscript.)

C4 Country
C5 Scale (# of Units) not provided by authors
C6 Was a power analysis conducted prior to data collection? Yes
C7 Has this research received Insitutional Review Board (IRB) or ethics committee approval? Yes
C8 IRB Number not provided by authors
C9 Date of IRB Approval not provided by authors
C10 Will the intervention be implemented by the researcher or a third party? Researchers
C11 Did any of the research team receive remuneration from the implementing agency for taking part in this research? not provided by authors
C12 If relevant, is there an advance agreement with the implementation group that all results can be published? not provided by authors
C13 JEL Classification(s) not provided by authors