You might have missed the news that the polling industry (The British Polling Council) began its autopsy into its general election performance a few weeks ago. The investigation will be long with a final report not due until March 2016, but the initial meeting of the contributors threw up some interesting points.
Prominent amongst these was a question from Martin Boon of ICM who, according to the BBC report of the event (http://www.bbc.co.uk/news/uk-politics-33228669) asked, is it still possible to get a representative sample over the telephone?
With more people using mobiles and more likelihood of call screening or non-participation the number of calls that need to be made to achieve a representative sample is huge. Mr. Boon cited numbers of 30,000 random calls needed to gather a sample of 2,000. That means the chances of achieving a representative sample from such a low response rate are equally low.
Ok so their focus is on polling and the canvassing of political opinion which is never an easy subject to get respondents to engage with and still achieve representation (the chances are that the politically engaged are more likely to respond thereby skewing your sample) but the general principle is something for anyone undertaking research to be concerned about.
Take online sampling, the self-selected nature of the sample means it will always be inherently biased, of course as a research agency we would say that wouldn’t we? Well yes, but only because we care about the credibility of the process and the accuracy of results. If the data collection method skews the results then you have to think about your method.
Should you be worried and thinking about changing your methodology? Probably not. In the end you have to take a pragmatic approach. How you do that depends on your sector, the potential costs involved, the time available to complete your project, your objective and the decisions that will be made as result of your work.
In the end this topic highlights the importance of sampling and the real possibility that an ill-considered sample may deliver results that lead you in entirely the wrong direction.
Posted by Lisa
