Political polling in Canada is at a crossroads, as new technologies push old methods of polling aside and a series of missed election calls shine a spotlight on the industry. In the last of a series of three articles looking at how polling is done in Canada, we look at online polling.
Few industries have been able to resist the move to online, and the polling industry is no different. While the telephone remains an important way to reach Canadians for public opinion research, increasingly it seems that online polling will be the way of the future.
The move to online is fraught with danger to some in the polling industry. A random sample cannot be built online in the way that a properly conducted telephone poll can. But for others, the Internet opens up a world of possibilities that were unthinkable before the advent of online polling.
How online polls are done
The primary method of conducting polls online is to build a panel of respondents. There are several of these panels in Canada, with 100,000 people or more signed up. These panelists can be invited to take part in surveys, and are often rewarded with chances to win a draw or are paid small amounts.
One method of recruiting panelists is over the telephone. Theoretically, this injects a degree of randomness into the panel similar to a standard telephone poll, but other modes of recruitment are also used. River sampling is one, where Internet users are recruited by ads on websites, even if they are not already a member of a panel. Increasingly, firms are moving away from entirely opt-in panels to mixed samples that also invite respondents to take part in online surveys via river sampling or over the telephone.
Once recruited to a panel, however, panelists can be analyzed in great detail in order to fit their demographic profile into the sample.
"We profile on over 90 variables over time so we get a good grasp on who these panelists are," Christian Bourque, executive vice-president of Léger Marketing, in an e-mail. This information can help build a representative sample, ensuring that no demographic group is over-sampled when invitations are sent out.
The advantages
Once a panel is recruited and sampling techniques are developed, online polls are relatively inexpensive to run and can be done in a short period of time. That puts them at an advantage compared to live-caller telephone polls, which can be much more costly and take longer. But there are up-front costs.
"Building and maintaining an opt-in online panel requires far more labour than laypeople tend to appreciate," said Doug Anderson, senior vice-president of Harris-Decima. "It has often seemed to our clients...that it must be the simplest thing to just recruit people to a panel and get national survey results. It is not. It is, however, very easy to set something up and end up with garbage."
While coverage is not as high as it is with landline and cell phone polling, it still covers the vast majority of the population: "86 per cent of Canadians went online last week," said Mr. Bourque. "That's more than households that have a landline phone."
Online polls also have an advantage in greater control of the sample. Certain demographics that are harder to reach with other methods can be targeted. The method also gives firms the ability to ask longer questionnaires than other methods and more capabilities in terms of how the questionnaire is designed. They can be customized in ways that are impossible over the telephone without costs being prohibitive: during the last two B.C. elections, Angus-Reid Public Opinion was able to give respondents a "real ballot" showing the actual names of the candidates in their own riding.
The disadvantages
But while online polls can be cheaper and faster to run while giving clients more options in how to build a questionnaire (videos and pictures, for example, can be included), there are still some questions about their reliability.
Opt-in panels do not represent random samples of the population, as the general population does not have the same random chance of being invited to participate. They need to be Internet users and, in some cases, panel members. But those who do not have access to the Internet are "usually less affluent, more transient," said Darrell Bricker, CEO of Ipsos Public Affairs. "These groups...are also among the least likely members of society to vote." For some market and public policy research these groups do need to be reached by telephone but "for political research this isn't a major issue."
Over-reliance on the same panelists can also pose a problem, said David Coletto, CEO of Abacus Data: "Probability issues become more significant when respondents are over-surveyed, meaning they change their behaviour or attitudes because they are surveyed often." Steps need to be taken to ensure this does not happen.
Concerns still exist in the industry about the randomness of these samples. The AAPOR and MRIA (industry bodies in the U.S. and Canada, respectively) are against the inclusion of a margin of error with these polls, as it considers them non-applicable.
"The biggest disadvantage is that online research in politics is relatively new," said Mr. Bricker. "We're still learning every day about what potential issues might exist."
Track record and increasing presence
The novelty of online polls has not prevented them from being increasingly used in Canada. In the 2006 and 2008 federal elections, online polls were rare. In 2011, however, there were four national firms using online panels against five using live-caller telephone interviews, while in the last few months there have been as many firms releasing online poll results as those using live-caller and automated telephone polling methods – combined.
The performance of online polls has also been quite good. On average, they did the best of the three methods of contact in the last two federal elections as well as in Quebec's last provincial election. They also performed better than live-caller polls in Ontario's last vote, and were the strongest performers in last year's U.S. presidential election. But the methodology carries the miss in British Columbia's recent election particularly heavily.
The future of polling?
This series of articles has looked at the challenges telephone polling has faced as technologies change. But the greatest threat to its former dominance in the industry comes from online polling. Even Google is getting into the game. Is this the way all polls will be conducted in the future?
"We face a small-c conservative industry that is adverse to change and innovation," said Mr. Coletto. But in the future, he said, "the business of polling will be completely online, within the next 10 years nobody will answer their phone unless they know who is calling."
And as media and government polling budgets shrink, pollsters are increasingly looking at what their corporate clients want. As a result, they are setting the standards as they are buying the research.
"The research business is in major transition," said Mr. Bricker. "It's funny that we get caught up in conversations about data collection methods like online vs. offline, it's almost a bit quaint. The truth is that the marketplace has already decided on much of this – and online is winning in all markets where it's feasible."
Éric Grenier writes about politics and polls at ThreeHundredEight.com .