By Anders Corr, Ph.D.

Responses of Don’t Know or No Answer for 28 New York Times/CBS polls. Blocks represent the density of observed DK/NA percentages for the 28 polls. The black line represents the probability of each DK/NA rate given a poisson distribution. The green dotted line depicts the mean of the observations. The black dotted lines depict two standard deviations above and below the mean. The most recent Syria poll, with a 14% DK/NA rate, is well above 10.5 (two standard deviations above the mean). The question is therefore highly unlikely to be valid and should be clarified for future polls.

CBS News and the New York Times released a poll yesterday on opinions regarding Syria. The poll posed a question to 965 adults nationwide worded as such: “Do you think the United States has a responsibility to do something about the fighting in Syria between government forces and anti-government groups, or doesn’t the United States have this responsibility?” Answers included “Yes” by 24% of respondents and “No” by 62% of respondents. CBS headlines interpreted the poll as “Americans Against Intervention in Syria.” The New York Times stated, “Americans are exhibiting an isolationist streak, with majorities across party lines decidedly opposed to American intervention in North Korea or Syria, according to the latest New York Times/CBS News poll.”

While the wording of the poll question on North Korea was acceptable, the question on Syria was ambiguous due to usage of the word “responsibility” and the phrase “do something”. It would be very possible to think that the US has no responsibility to act in Syria, but should do something nonetheless. It is also possible to think that the U.S. has a responsibility to do something, but should nevertheless refrain from acting. Finally, “doing something” is ambiguous, and does not necessarily mean “intervention” as asserted in the reporting of the poll by the New York Times. The biases are countervailing, and lead to ambiguity.

The ambiguity of the question is clear when examining the large number of Don’t Know (DK) and No Answer (NA) responses. Tellingly, an aggregated 14% of respondents answered either DK or NA. In contrast, the mean aggregated DK/NA response for all questions was 5.8% (2.4% standard deviation assuming a poisson probability distribution). This indicates that compared with other better-worded questions, an additional 8.2% of respondents on the Syria question were coded as DK/NA. The Syria DK/NA rate of 14% was well above the key value of 10.5 (two standard deviations above the mean). The question is highly unlikely to be valid, and should therefore be reworded for future polls as simply, “Should the US intervene militarily to stop the fighting between government forces and anti-government groups in Syria?”

5/3/2013: A Pew Research Center Poll on Syria released on May 1, 2013 posed a better-worded question that received 3% fewer DK/NA answers. Responses to the question, “Do you support or oppose your country sending arms and military supplies to antigovernment groups in Syria?” included 25% in favor, 64% opposed, and 11% “Don’t Know”. Regarding intervention, another Pew Research Center Poll conducted between April 25 and April 28, found that if Syrian government use of chemical weapons against opposition groups were confirmed, 45% of Americans would support military action against Syria, with 31% in opposition and 23% recorded as “Don’t Know”. These numbers are contrary to, and raise further questions as to the accuracy of, the New York Times/CBS News poll.