Andrew Cuomo, Carl Paladino, and Those Nutty Polls
The day before Mike Bloomberg defeated Bill Thompson by 4.6 percent of the vote last November, Quinnipiac said the mayor would win by 12 points. Its poll the day before the 2005 election predicted Bloomberg would beat Fernando Ferrer by 38 points, twice his actual margin.
The good news for Andrew Cuomo in yesterday's Quinnipiac poll is that he's still winning, and that's what Q almost always gets right.
Its margins -- in this case a mere 6 point lead over Carl Paladino -- usually have more to do with capturing headlines than getting it right.
Their shocking findings landed a front page splash in this morning's Times, though the paper didn't take the bait and buried naming Quinnipiac until the jump, an omission that got old Timesman and current Quinnipiac mastermind Micky Carroll quite upset on the phone with me a few minutes ago. Quinnipiac knows that the more shocking the finding, the bigger the play. That's not to say they rig the findings; they just care more about making news than making sense.
Afterwards, no one, least of all the press that feeds off these persistently newsy but nutty numbers, remembers how far off they were.
Indeed, the Quinnipiac website boasts now about their 2009 findings as if they hit it on the head. And a New York Post editorial today, rushing to turn on the Democrat they've been temporarily championing, called Quinnipiac "generally authoritative."
The danger is that a bad poll can have a big impact, and reshape political reality. The Paladino camp told the Times he was hunkering down on the phone raising big bucks using Quinnipiac's numbers (imagine what he says to anyone who stiffs him). The Post was so jubilant they ascribed the poll results to Cuomo's alliance with the "egregiously corrupt Working Families Party," which it said was "brimming with sophistries," adding that the "voters have had it with sophistry." A week before the Post praised the WFP deal under the headline "Andrew's Coup," crowing "score one for Andrew Cuomo -- and maybe for New York," but poll-driven flip-flops on the editorial page are not sophistry.
The Times has no business serving up its top news spot to a pollster that missed the margin in the last gubernatorial race, between Eliot Spitzer and John Faso in 2006, by 11 points, and blew the last two mayoral contests. Obviously, every public poll gets attention, and others get them almost as wrong as Quinnipiac, but this placement decision was more about stirring a pot the Times delights in stirring than telling readers what's actually happening. At least the Times was self-conscious enough that it cluttered the story with caveats, and ran a sidebar from an expert tossing up more caveats, but still couldn't resist hyping the result.
As the online-but-not-in-the-paper sidebar by Nate Silver notes (how many readers get this far), to compare these numbers with the ones that preceded it from Quinnipiac or any other pollster is to miss the point. It's an entirely different universe. Every prior Q poll this year, like those of most other pollsters, surveyed registered voters. This one surveys "likely voters," a sudden narrowing of the field that no average reader would notice. Not only that, Q is not just asking the preferences of these so-called likely voters. If a voter refuses to state a preference, Q asks them if they are "leaning" toward any candidate. So this poll combines two highly speculative items: the leanings of likelies.
How does it determine who is a likely voter?
Let Mickey Carroll explain. He says the Ph.D who runs the poll, Doug Schwartz, uses a variety of screening questions to determine "how interested" the voter is and "winnows out" the registered voters who seem less engaged based on "his experience" at separating wheat and chaff, such as it is. Mickey calls their screening of likelies through evaluating multiple answers "the Cuisanart approach," but this result suggests it's more the spice and nut grinder than the elite food processor. "Their answers to multiple questions" produces a "cohort of people likely to vote." If that sounds like guesswork to you, it doesn't have to. Carroll freely concedes that other consultants like New York's eminently successful identifier of prime voters, Jerry Skurnik, is actually sitting on lists of people who reliably do vote, but Quinnipiac, despite missing the margin again and again, prefer not to buy his lists.
"Maybe we should," says Carroll. "Maybe we will. Would that be a better way? Prime voters might be a good thing."
Mickey has no idea why the pool of likelies in the last mayoral election was 1360 and this pool, for a statewide race, is a mere 751, but it does strike me that stretching fewer folks over a much larger space is likely, to choose a random word, to open up a few new holes and they had a pretty big problem with the pool in 2009.
Finally, there's the question of whether polling likely voters for the first time the day after the primary is the best time to identify who's really likely to vote. Or does Q do it because it knows that it will come in ahead of other major polls and grab the most gas out of the story? Sienna, for example, did its poll today and found Cuomo ahead of Paladino by 33 percent, but Siena is still surveying registered voters.
Here's why, from Steve Greenberg, its director.
Sienna did its first likely voter survey in September in 2006 and 2008 but decided it was just too fluid to do that early this year. "We talked about it a lot," said Greenberg. "But we thought to do it so soon after the primary, when so many voters didn't know Paladino, six weeks out from the election, would just be wrong. We don't think voters now know whether they'll be voting in this race. They haven't come to a decision on whether to vote in this and many of the other races. We will do likely voters in October." He says Sienna, which does not do city races and came closer than Quinnipiac to getting the Spitzer race right, uses a mix of Skurnik lists and screens. Their poll today will not get the front page of the Times.
"We don't think we're too soon," answered Carroll. "If we wait until just before the election, no one will be able to understand the swing." So why not October, when Greenberg says Sienna will do two with likelies?
Quinnipiac acknowledges that its likely voter pool skews the results toward Republicans, and Silver puts the range at between four and 10 points. Sienna kept Lazio in the poll, and he got 8 points. Q's decision to exclude him guaranteed that it would produce a closer race between Paladino and Cuomo. Q also disproportionately found more male leaners than female, the reverse of the usual final voter tallies.
Finally, Quinnipiac wound up with a 36 percent Democratic pool in a state with 49 percent Democratic registration and a 22 percent Republican pool with 25 percent Republican registration. That left 42 percent for independents, third party registrants, or people who wouldn't disclose their party registration, an inexplicable total that makes them the new dominant force in New York politics. (By the way, how does someone who refuses to disclose his party wound up counted as a likely voter?).
If you think none of this matters, because the polls will correct themselves, think again. After its final poll the day before the mayoral primary in 2005 said Ferrer would fall eight points short of winning the 40 percent necessary to avoid a runoff and proving wrong again, Quinnipiac's first poll of likelies right after the primary killed Ferrer's unity drive and showed Bloomberg magically widening his lead. I did a piece then about how juiced it was. Ferrer never got going after that poll. Fueled by Q's latest rush to relevance, Paladino is off dialing dollars. That may actually make the race closer than it would otherwise be.
Quinnipiac is a quiet suburban campus in Connecticut. The only time it's ever in the headlines is when it polls and shocks. When its numbers buried Ferrer, it had 6 Spanish-speaking callers out of the 155 students dialing into city homes. It has no claim to the Times front page.
Research Credit: Lily Altavena, Sam Cook, Ryan Gellis, Jared Green and Puneet Parhar.