When people are careful and take their time answering surveys, the results will accurately reflect consumer opinion and can be used to properly innovate and improve the products we know and love, or know and hate as they case may be.
Some people, perhaps because they are in a hurry or are more interested in the incentive than sharing their full opinions, answer surveys as quickly as they can. This is where data quality suffers. These types of people may choose as few responses as they possibly can, provide low quality verbatims, not follow instructions, or straightline a lot.
But what about people at the other end of the spectrum? What about people who take a long time to complete their survey? Perhaps they are taking great care with every question, perhaps they are slow readers, perhaps they let the survey sit for a while in order to finish a load of laundry or eat their lunch, or perhaps they are having difficulty understanding and responding to the survey. Regardless of the reason, what does their data look like? Should we be worried that laggards aren’t providing high quality data?
Based on behaviors from nearly 2000 people, here’s what we know about the slowest 10% of people.
Laggards may be much less likely than speeders but more likely as average people to:
- check as few items as possible in response to a request to “Check all that apply”
- provide lower quality verbatims
- correctly recall the brand name from a commercial they just watched
- straightline on grid questions
- refuse an extra request
The effect is small but very clear so what can we conclude? We know that people who speed through surveys aren’t sharing the best possible data they can. But for some reason, something is holding back laggards. The effect may not be large enough to change trendlines or rankings, but it’s definitely worth considering when point estimates are an important part of a project.
Here’s hoping you spent a moderate amount of time reading this post!