close× Call Us
Title Compliance in Survey Experimental Studies of Intent
Post date 03/04/2019
C1 Background and Explanation of Rationale

In a co-authored project with Sarah Kreps, we conducted a survey experiment in May 2017 that tested public concern for the intent behind civilian harm. The results indicate that over half of the respondents (53%) failed to identify the correct intent treatment in the manipulation check. Thus, the experiment was largely unsuccessful in the administration of the treatments—intentional, foreseeable but unintentional, and accidental civilian harm. The results indicate two potential causes for noncompliance: “satisficing,” i.e. insufficient effort in interpreting the content of the scenarios and post-treatment questions by the respondents, and/or “motivated moral reasoning,” i.e. the biasing effect of respondents' moral judgments on their perception of the details of the scenario. The results raise a puzzle about studying the effect of intent on public attitudes in the context of survey experiments—whether noncompliance is a result of lack of respondent motivation or an inherent feature of moral reasoning. I will investigate this puzzle using a survey experiment conducted through Amazon’s Mechanical Turk.

Scholars in the field of international relations will benefit from this study. The distinction between intentional, foreseeable but unintentional and accidental war-time civilian is codified in international humanitarian as the norm of noncombatant immunity—the legal principle that seeks to prevent harm to innocents during war. To date, scholars have not investigated the full-range of intent codified in this norm. The state of literature has, therefore, left scholars with a limited understanding of the impact of the norm of noncombatant immunity on public support for war. This study takes an important step towards improving scholarly knowledge by investigating how to improve data quality in the context of survey experiments that investigate intent and public attitudes towards conflict.

C2 What are the hypotheses to be tested?

Satisficing Hypothesis: (a) Respondents who engage in speeding are more likely to misperceive the intent behind civilian harm. (b) Survey policies that deter speeding will result in higher rates of compliance.

Motivated Moral Reasoning Hypothesis: (a) Liberals and individuals who judge harm as immoral are more likely to impute stronger intent, control and causation than conservatives and individuals who judge the harm as moral. (b) Survey policies that deter speeding will not result in higher rates of compliance.

C3 How will these hypotheses be tested? *

The survey will be fielded through Amazon’s Mechanical Turk. Participants will be given the option to complete an original survey that will take about 8 to 10 minutes. All participants will be asked to read a hypothetical scenario in which U.S. military leaders authorize airstrikes against a terrorist organization which has taken over towns in a country in West Africa. The airstrikes result in civilian casualties, which the leaders anticipated but did not directly intend prior to authorizing the airstrikes. The study will use a 2x2 mixed design. Participants will be randomly assigned to a treatment condition (T2 and T4) in which they are asked to commit to reading the scenario and questions carefully. Participants in the treatment condition will also be notified when they read questions faster than 215 milliseconds per word and asked whether or not they would like to reconsider the question. Participants who are randomly assigned to the control condition (T1 and T3) will NOT be asked to commit to careful reading nor notified when reading too quickly. Within the treatment and control conditions, participants will also be randomly assigned to receive one of two versions of the scenario. In the first scenario (T1 and T2), the intent behind the civilian casualties will be described as anticipated but not intentional. The second scenario (T3 and T4) will describe the intent in this way, while also stipulating that the airstrikes weakened the terrorist organization’s ability to carry out attacks.

Intent Intent + Effectiveness
Control T1 T3
Survey Policy T2 T4

After reading the scenario, the survey will collect participants’ answers to follow-up questions about whether or not they support the airstrikes and believe the leaders’ decision was moral. In addition, participants will be asked several questions testing their ability to recall details of the scenario, such as the intent behind the casualties. Responses to these questions will be used to determine whether or not the survey policy treatment improves compliance. Post-treatment questions will also be included to measure participants’ perception of the intent behind the airstrikes. These questions will determine whether supporters of the airstrikes and conservatives were more likely to misinterpret the civilian casualties as accidental (i.e. unanticipated), and whether opponents and liberals were more likely to misinterpret the casualties as deliberate. Such findings would be evidence of motivated moral reasoning.

C4 Country United States
C5 Scale (# of Units) 500
C6 Was a power analysis conducted prior to data collection? Yes
C7 Has this research received Insitutional Review Board (IRB) or ethics committee approval? Yes
C8 IRB Number 1902008586
C9 Date of IRB Approval February 19, 2019
C10 Will the intervention be implemented by the researcher or a third party? Amazon Mechanical Turk
C11 Did any of the research team receive remuneration from the implementing agency for taking part in this research? No
C12 If relevant, is there an advance agreement with the implementation group that all results can be published? not provided by authors
C13 JEL Classification(s) not provided by authors