Tom H. C. Anderson - Next Gen Market Research

Online Research Best, IVR Second Best, Telephone Third

Inside Research (IR) today helped confirm what I have always known to be true. Online research is better than traditional research methodologies for more reasons than speed and cost.

Ever since P&G’s Kim Dedeker made her comments about online panels two years ago a hot topic of discussion in Market Research has been that of online data quality. For me a far more interesting topic, especially in the short term, is quality of polling and predicting the winner of the US presidential election.

If you’ve read Freakonomics you will know that a lead in the polls doesn’t necessarily translate into winning the election, especially when race is a variable. In the NYC 1989 mayoral race Dinkins slender margin of victory came as a surprise when polls had shown him winning over Rudolph Giuliani by 15 points. Conversely, when white supremacist David Duke ran for US Senate in 1990 he garnered nearly 20 percent more of the vote than any polls had projected. Is race an issue and do people lie to pollsters? Absolutely!

IR’s front page this month featured article “Lying Voters Skew Results”. The article went on to discuss other stories such as WSJ “when Voters Lie” and shared findings from a matched sample survey done in parallel via telephone interviewing and online data collection. It concluded, “that people who take surveys in private [online] are less likely to lie than when an interviewer is listening”. This ranged over a variety of topics tested from alcohol and exercise to gambling. Examples from IR include: 56% told interviewer they went to church, synagogue or mosque most weeks; 25% told that to the computer. Exercise regularly 58% vs. 35%, alcohol 39% vs. 53%.

Later in the issue IVR polling was also discussed, and that while “watching the survey sausage get made isn’t pretty” (IVRs get low cooperation rates and don’t consider random selection of HH respondents) this methodology which can be fielded for one tenth of the cost of traditional CATI produces more accurate results. Strangely major news networks like NBC, CBS, ABC, Associated Press, NY Times, and Hotline still forbid their use.

These were issues I was familiar with. Strangely some of these media sources still actually also have prejudices against using online survey research! Anderson Analytics used IVR technology together with text analytics to accurately predict the Lieberman vs. Lamont race in 2006. Though IVR had low response rates (RDD calls to 10,000 Connecticut households yielded just 312 responses).

This too was an emotionally charged campaign with Lieberman having left the Democratic Party. Instead of asking who a respondent would vote for, Anderson Analytics relied more on asking who respondents thought would actually win the election.

Unfortunately political pollsters seem to get more glory in the media than they deserve. Comparing your average pollster to a full market researcher is like comparing a draftsman to an architect or a chiropractor to a back surgeon. By thinking more critically about the research problem it should be easy to construct an accurate measuring tool.

What I find most encouraging about the recent polling controversy though is that it gives additional confirmation that traditional research techniques can be replaced by cheaper, faster, more accurate next generation tools! Race is just one variable, it can’t be that hard to solve for.

Tom

Tweet This Post