Insight into Narrative Building With Polls

I got this release (PDF) about post Presidential debate research from The Winston Group. Included in it is an analysis of a debate focus group put together by left-leaning Democracy Corps.

Democracy Corps conducted a post-debate discussion with 45 swing voters in Denver, which included 16 unmarried women. According to their analysis this group contained twice as many Republican leaning participants as Democratic leading participants. This qualitative study supports many of the findings of the quantitative studies of CBS and CNN. By a 2-1 margin they thought Governor Romney won (42% Romney, 20% Obama, 38% Neither). Additionally, this group showed a significant improvement for Governor Romney in favorables, much larger than the improvement for President Obama, to the point there was a potential for a significant difference between the two. It also showed movement in the ballot test as Governor Romney picked up more support than President Obama - although their analysis says that the movement to Romney came from Republican leaning voters only.

I stopped reading here because something seemed pretty clear to me: Democracy Corps had set up this focus group with 2-1 GOP leaning undecided for the express purpose of confirming the narrative they had prepared. That Obama had won the debate and convinced even GOP-leaning women to get on board. Woops.

Instead, as explained in the report, the Democracy Corps was left to spin:

Their conclusion was that “In the end, though, this debate did not emerge as the game-changer the Romney campaign needed.” However, most of the data in the focus group doesn’t support this conclusion and was more in line with the two quantitative studies by CBS and CNN which showed clear movement. Their rationale was that no supporter of President Obama in the focus group moved to Governor Romney. Looking at the numbers it looks like this represented about 14 people (out of 45), a small group to make such a definitive conclusion. This is a quantitative conclusion based on qualitative data from 45 people in Denver preselected to match certain demographic criteria. Focus groups can provide possible theories; they cannot provide quantitative conclusions.