After years of viewing the Web with suspicion, mainstream consumer researchers are starting to report that the rewards of online research far outweigh the risks.
"We're at the beginning of something very, very new - a whole new phase of the research industry," said Jim Spaeth, president of the Advertising Research Foundation, as he opened the group's daylong workshop Monday in Los Angeles, the first such event devoted to exploring the validity of online research techniques.
Consumer researchers presented the results of several parallel studies conducted using online and traditional mail or phone interviewing techniques. The unanimous result was that online and offline findings resulted in consistent business direction.
"Although the data may not be exactly the same, you would have made the same business decision," said Dennis Gonier, president of Digital Marketing Services, which runs the Opinion Place consumer research forum on AOL (dossier). Gonier presented findings from an Avon study that demonstrated the ability of online research to produce sales forecasts comparable to those produced using offline methods.
In the same way that it has affected many other industries, the Web has brought increased speed, lower costs and, in some cases, higher-quality techniques than offline research methods. But difficulty obtaining random samples and the upscale demographic skew of Internet surfers have kept many traditional research firms from encouraging clients to use the Web.
But offline survey techniques have their own set of drawbacks. While it's theoretically easy to draw a representative sample of respondents due to the ubiquity of telephones and the ready availability of census data, willing respondents are increasingly difficult to find. The percentage of respondents who refuse to participate in telephone surveys increased from 40 percent to 48 percent from 1988 to 1997. And that's not counting the large percentage of folks who are never contacted because they aren't home to answer the phone.
"The questions today about online research are the same questions that have been asked about other types of research in the not-too-distant past," said ARF's Spaeth, who pointed out that over time the use of the telephone to conduct research went from "disrepute to the gold standard."
In the past, Web users were a tiny and wholly unrepresentative group of Americans. But over the past few years, the demographic profile of Net surfers has begun to more closely resemble the profile reported by U.S. Census data. Research firm Market Facts (dossier) presented findings from their "Landmark Study" that show the differences between the demographic, psychographic and shopping profiles of online and offline adults to be largely "directional." This means researchers can survey within specific groups online and estimate behavior for the entire group - online and offline.
Rather than using random-sampling methods, researchers are starting to use more sophisticated techniques - such as propensity scores and Bayesian sampling to better estimate marketwide data - to poll online populations. Harris Black International reported the use of propensity scoring to accurately predict 21 out of 22 election races last November.
Another concern about online research is respondent "veracity." Some analysts have expressed fear that online respondents may respond untruthfully more often than offline respondents. Many speakers chalked up this perception to old-fashioned stereotypes about the freewheeling behavior of anonymous chat-room participants.
In fact, a Quaker Oats study presented by researcher MARC Online found that Internet respondents may be more honest on sensitive topics than survey subjects intercepted in shopping malls. For example, online respondents' answers about how many snack foods they eat per week were more in line with Quaker Oats' internal data than answers the firm received from people in shopping malls. This was attributed to the fact that respondents may be reluctant to fess up to a human interviewer about how many snacks they really eat every week.
The Quaker Oats (OAT) online study was also less expensive, larger in scale and completed quicker. The project was funded with half of the mall study's budget, and three times as many
