Christy Clark joined Alberta Premier Alison Redford last night as the second premier in little over a year to defy the polls and pull off an upset victory against all expectations. If the polls had been reflective of the mood of the electorate, Adrian Dix would be celebrating his victory. Even if the polls had been only slightly inaccurate, the lead his B.C. New Democrats had been given was wide enough to ensure he only needed to prepare one speech.
It should not have happened again. And yet it did – the pollsters expressing disbelief and shock both on television and on Twitter. None of them predicted that the B.C. Liberals would beat the New Democrats by even a single point, let alone five. The NDP was given a cushion of between eight and nine points by the two experienced B.C. pollsters who were in the field as recently as the day before the election.
But though the surprise was similar to the one that occurred in Alberta's provincial election last year, the circumstances here were very different. The reasons that an Alberta Surprise was not to be expected in British Columbia still apply – the uncertainty that voters had with Wildrose should not have been as large of a factor with the NDP. The experience of the pollsters in the field was undoubted. And the pollsters were active in the final days of the campaign, some in the final hours.
Worse, the polls were not suggesting that the margin was closing. The final polls by Angus-Reid and Ipsos-Reid showed a margin that was holding steady or even widening. Some other surveys did show a closing gap, but that was compared to polls they had taken prior to the April 29 debate when the major shift in voting intentions occurred. Christy Clark's approval ratings were dismal. Those who did say they would vote Liberal expressed little enthusiasm in their support for the party.
Undoubtedly, the pollsters in British Columbia will try to explain what went wrong last night and work towards avoiding those errors in the future. Ipsos-Reid, for instance, was in the field on election day with an exit poll that should be revealing – though early word from the firm suggests that the results of the exit poll also misjudged the strength of the B.C. Liberals.
With every polling firm having done the same, there is not much of a prize to hand out. Forum Research did end up with the closest result, with a cumulative error of about eight points for the four major parties in their final poll. But it was taken six days before the vote, so it is debatable whether or not the firm was just lucky to be in the field at the right time. And the poll still gave the New Democrats a two-point advantage.
Ipsos-Reid and Angus-Reid, who were in the field as late as Monday, did the next best with a cumulative error of about 15 and 17 points, respectively. The other firms active within 30 days of the election (EKOS Research, Justason Market Intelligence, Hill & Knowlton, Oraclepoll Research, Insights West, and Abacus Data) did marginally worse.
That every polling firm in the field, using a mix of methodologies, was unable to get a good result (and they mostly showed consistency even at the regional levels) suggests that something systemically wrong was taking place in their sampling methods. Are pollsters not building a sample that is reflective of the broader population anymore? Are they not polling those who actually vote? Are people no longer responding to polls truthfully? Do the now ubiquitous online panels and automated telephone polls have intrinsic limitations that can be amplified under certain circumstances (both have had success, and failure, in the past)?
These questions will need to be answered. An almost literal last-minute swing in voting intentions worth about 13 points does not seem to be plausible. The effect of low turnout, and the inherent discrepancies it can cause in polling, may be a place to start.
But the polls did not get everything wrong. Most surveys did a decent job assessing the support levels of the Greens and Conservatives. Ipsos-Reid, Angus-Reid, and Forum were within about a point of the Green's vote share and within two points for the Conservatives. But as expected, every firm over-estimated these two parties' support. In most cases, the discrepancy was less than the error for the B.C. Liberals, who were underestimated by an average of nine points. By contrast, the final polls of the firms active in the campaign overestimated NDP support by an average of about three points – not overly catastrophic, but in the context of the Liberal underestimation it made all the difference.
For the people of British Columbia, who elected a Liberal government for the fourth consecutive time, these debates may be trivial. But the extent to which the polls influenced the tone of coverage of the campaign and voters' behaviour is unknowable. Reliable polls can help the electorate make a more informed choice, but polls do voters a disservice if they are not capable of accurately reflecting public opinion. Figuring out what went wrong, and ensuring it does not happen again, is absolutely vital if the polling industry is to regain any trust.
Éric Grenier writes about politics and polls at ThreeHundredEight.com .