National Poll Shenanigans

Take a look at the Real Clear Politics' 2012 Presidential poll averages and you'll see, for the most part, Obama and Romney are within 2-4 pts of each other with Obama leading most of them. Then Pew released a poll showing Obama with a 10 point lead. How'd that happen? Well, several experienced and astute poll watchers noticed the discrepancy between the Democrats and Republicans sampled, which Pew itself laid out:

The current survey finds that 45% of independents back Romney and 43% Obama, which is virtually unchanged from earlier in July. Over the course of the year, independent support has wavered, with neither candidate holding a consistent advantage.

Both candidates have nearly universal backing within their party: Nine-in-ten Democrats support Obama and an identical share of Republicans support Romney. Obama’s overall edge at this point is based on the healthy advantage in overall party identification that Democrats have enjoyed in recent years.

Pew's aggregate shows that they significantly over-sampled Democrats compared to Republicans. That's how, even though 9 in 10 of each party stayed true and Romney has a slight lead over the President with Independents, Obama manages to have a 10 point lead. How can this be a legitimate way to conduct a poll when no one--and I mean no one--really things the Democrats will have a 10 point turnout advantage in November? Hugh Hewitt asked the Quinnipiac pollster--who also tends to over-sample Democrats--the same question.

HH: I want to start with the models, which are creating quite a lot of controversy. In Florida, the model that Quinnipiac used gave Democrats a nine point edge in turnout. In Ohio, the sample had an eight point Democratic advantage. What’s the reasoning behind those models?

PB: Well, what is important to understand is that the way Quinnipiac and most other major polls do their sampling is we do not wait for party ID. We ask voters, or the people we interview, do they consider themselves a Democrat, a Republican, an independent or a member of a minor party. And that’s different than asking them what their party registration is. What you’re comparing it to is party registration. In other words, when someone starts as a voter, they have the opportunity of, in most states, of being a Republican, a Democrat, or a member of a minor party or unaffiliated.

HH: Okay.

PB: So what’s important to understand is what we are doing is we’re asking voters what they consider themselves when we interview them, which was in the last week.

HH: Now what I don’t understand this, so educate me on it, if Democrats only had a three point advantage in Florida in the final turnout measurement in 2008, but in your poll they have a nine point turnout advantage, why is that not a source of skepticism for people?

PB: Well, I mean, clearly there will be some people who are skeptics. This is how we’ve always done our polls. Our record is very good in terms of accuracy. Again, remember, we’re asking people what they consider themselves at the time we call them.

HH: But I don’t know how that goes to the issue, Peter, so help me. I’m not being argumentative, I really want to know. Why would guys run a poll with nine percent more Democrats than Republicans when that percentage advantage, I mean, if you’re trying to tell people how the state is going to go, I don’t think this is particularly helpful, because you’ve oversampled Democrats, right?

PB: But we didn’t set out to oversample Democrats. We did our normal, random digit dial way of calling people. And there were, these are likely voters. They had to pass a screen. Because it’s a presidential year, it’s not a particularly heavy screen.

HH: And so if, in fact, you had gotten a hundred Democrats out of a hundred respondents that answered, would you think that poll was reliable?

PB: Probably not at 100 out of 100.

HH: Okay, so if it was 75 out of 100…

PB: Well, I mean…

For whatever reason, more Democrats seem to be answering the pollsters calls. There could be any number of reasons for this. The obvious error here is believing that people who pick up the phone for pollsters is an accurate reflection of those who will actually vote in November. Even though pollsters seem to know that the model isn't really incorrect, it sure is good for a headline, isn't it?