Canada's NDP leader Jack Layton greets supporters as he arrives at a campaign rally in Winnipeg on April 27, 2011.FRED GREENSLADE/Reuters
When Frank Graves considered recent polling results generated by EKOS Research, he felt the sweat bead on his forehead as he tried to reconcile a report that was guaranteed to change the tone of the election.
The results were very different from what other pollsters had been reporting, and showed the NDP had moved past the Liberals into second place and could take as many as 100 seats nationally.
"I didn't sleep the night before we brought out that bombshell of a poll," he said. "I'd be in bed thinking about how my method works, worrying about whether the results are repeatable."
The poll, which was made public Monday, highlighted the growing divide among the handful of pollsters who have been trying to make sense of this election. It's not just that they're seeing different levels of support for different parties - that's normal. Finding that the NDP is unexpectedly rushing toward what could be its best showing in history is another matter, especially if you're the first out of the gate.
"If you get this wrong then people are going to say that you've just really maliciously and stupidly entered into an election debate and provided everyone with crappy advice," Mr. Graves said. "That's kind of a career destroyer."
Because the pollsters all use different methodologies, their results have appeared to conflict at times and have confused voters unsure how to interpret the results.
EKOS uses a method of polling in which an automated system calls random numbers, and relies on whomever answers the phone to punch in their answers on a keypad. Ipsos-Reid's most recent release was put together by asking questions of an online panel, and the company has also used the telephone. Nanos Research does a nightly poll of 400 people via phone.
While the pollsters generally agree on key trends - the NDP is eating into the Bloc Québécois base, the Liberals have been falling back - the biggest inconsistency is how the Conservatives have been faring. The pollsters are also on different timelines, so even when they agree it can take a few days for the data to catch up.
Nanos Research president Nik Nanos said he's reconciling the differences by focusing on trends rather than hard numbers. That's especially true, he said, when you get to regional breakdowns, which can be volatile. "We should be looking more at the direction and trend because it's more important than the actual number," he said.
Ipsos president Darrell Bricker said this is the first election he can think of that has seen pollsters use such a wide range of methods. "I've stopped looking at the other polls," he said. "All we can do is be open and transparent, and constantly questioning … and invalidating our own work. If we don't do that, the election results will humble us all."
While the NDP surge is undoubtedly real, it's impossible to say how that will translate into votes. And if the party's strong showing doesn't translate into an increased number of seats in the House of Commons, an industry that is already self-conscious about its track record will find it has some explaining to do.
Critics charge that small sample sizes, low response rates and the use of technology render the results all but useless, something the pollsters deny. They are able to target youth by randomly calling cellphone exchanges, for example.
"I hear people say that polling is completely invalid these days," said Mr. Bricker. "That's absolute crap - there are more ways to contact people than ever, so many avenues of evidence to pull together."
The Marketing Research and Intelligence Association, which represents market research companies in Canada, released a report called "There's No Margin of Error on the Truth" in an effort to debunk claims that the polling industry "is having a crisis of confidence."
The report shows that the final-week polling results came very close to actual results in both 2006 and 2008, and suggests that having so many polls through a campaign leads to a better system in which politicians find it harder to spin their messages.
For Mr. Graves, the only thing that matters now is how the seats break down on Monday. "Hopefully on election night I'll stay in with a glass of wine and a few friends," he said, "and chuckle about how right I got it once again."