Being someone who scours the Internet looking for clues as to how the election is going to turn out, I discovered the widely-referred to polling firm Rasmussen Reports. Their methodology is to sample 1,000 likely voters each day and report results on a 3 day rolling average basis. Which makes me wonder what value would this methodology have? I know rolling averages are useful in some situations But not here…
Here’s what I see as the problems with their methodology. Granted, I’m not a professional statistician, nor have I ever played one on TV. On the other hand, I have stayed at a Holliday Inn Express.
First, political surveys should reflect what people are thinking right now. Voters change their minds all the time. What’s the value in learning on Monday what some people were thinking on Friday? A campaign shouldn’t make decisions based on stale information – and how three day old information on how people intended to vote is stale information. Voters change their minds based on any number of factors – such as, for example, their reaction to watching a debate. Who wants to plan a campaign based on what people might no longer be thinking? How is each campaign supposed to respond to last night’s debate if they have to wait to get that data? Cheap analogy: who picks the winner of a baseball game based on how the teams were playing back in April?
Second, since Rasmussen doesn’t disclose the specifics of each day’s results, one can’t tell the trend over the three days of polling – it’s impossible for a campaign to figure out the impact a given day’s events have had on the voters. For example, when looking at results released on Friday (Tues-Wed-Thur polling), it’s impossible to tell what impact an event that took place on Wednesday would have had on the overall totals. One can only try to infer these numbers from playing with the numbers over a number of days – by that time, the ability to react is much diminished.
And third, if survey results are to include stale data, more weight should be given to the more recently obtained results. Otherwise, a sudden drop in support or a sudden boost in support wont’ be revealed for three days. For example, if I was looking to Rasmussen for hints as to how last night’s debate played out, I’d have to wait until sometime next week to get my answer – because until then, their results include totals from before the debate and would, again in my dreams, overstate the current level of support that Kerry has.