[Posted by Karl]
Jonathan Chait disagrees. He’s wrong, as he is about many things. But he’s wrong in ways worth discussing.
Chait’s target is the recent Third Way poll of independent voters, which he doesn’t like because “Third Way is an intra-party lobbying group that urges Democrats to adopt moderate, pro-business policies” and its poll tends to support its positions. He notes that if you frame poll questions differently you can get findings like those from Greenberg Quinlan Rosner (taken for the Center for American Progress), such as 81% agreeing that “[r]egular people work harder and harder for less and less, while Wall Street CEOs enjoy bigger bonuses than ever.”
To be sure, the wording of poll questions matters. However, Chait does not bother to quote the questions in the Third Way poll. It appears he does not like questions such as:
I’m going to name some topics that have angered some people in America. For each one, please tell me if it makes you very angry, somewhat angry, not too angry, or not angry at all.
The poll then asks about: Congressional gridlock; the national debt; Wall Street bailouts; America falling behind its global competitors; the wealthy not paying enough in taxes; the next generation’s ability to achieve the American Dream; corporate profits; and China’s economic rise. Another question that seems to bother Chait is: “What do you think would be the most effective way to strengthen our economy?”, giving reducing the deficit, reducing taxes and regulations, and reducing income inequality as options. Had Chait actually quoted the poll’s questions, it might have occurred to his readers that those questions sound much more neutral than the GQR questions he did quote. Indeed, the basic Third Way findings on economic opportunity vs. economic fairness are not much different from those of Gallup and Pew.
However, the issue of question neutrality goes to a larger problem with Chait’s general concept of “real” public opinion polling:
Pollsters understand that very slight differences in the wording of a question, or even in the ordering of questions, can produce dramatically different results. Polls that are actually designed to measure public opinion take great precautions to avoid tilting answers one way or another. They try to frame questions in as neutral fashion as possible, and when they do ask questions that gauge people’s ideological views, they measure it by looking at changes.
So, for instance, a poll might ask if you prefer a larger government with more services, or a smaller government with fewer services. That is a classic polling question. It’s not an accurate snapshot of public opinion, though, because even though it’s posed in a completely neutral way, in frames the question in abstract terms rather than specific terms. Its value as a measuring tool is simply that polls as the same question in the same way every year, and the changes in response to the same question can help tell you how public opinion is changing.
This is a wildly reductive view of public opinion polling, and especially reductive of political polling.
The information gathered from the sorts of polling Chait describes is valuable — even if the polls generated for the establishment media and by entities like Gallup and Pew often fall short of the ideal. However, it does not logically follow that “advocacy” polls are not “real” polls. The issue is the quality of a given poll for its purpose.
For example, another poll from Greenberg Quinlan Rosner — this time for Democracy Corps — looks at several of the messages Pres. Obama and Democrats have been putting out and tests them against hypothetical GOP messages (which are debatable, but beyond my scope today). It is not remotely neutral, but highly informative about public opinion for those actually conducting campaigns. It is advocacy polling like this (which the White House or the DNC surely conducts internally) that explains why Obama is mostly attacking the GOP instead of leading with claims that America is back or has made progress on job creation.
Election campaigns are not waged solely in the editorial bullpens of the New York Times and Washington Post, or the offices of Gallup. Rather, beyond the fundamentals of peace and prosperity, they are driven by candidates and their messages. The candidates, their supporters and their messages are not neutral.
In short, to suggest that advocacy polling is not “real” is in some senses exactly backwards. And to compare the recent Third Way poll to the Center for American Progress poll is laughable. Indeed, Third Way’s “advocacy” here rests primarily on the general, neutral approach of its poll.