How many survey questions are needed?
Yesterday’s Q&A with BRB prompted an interesting discussions on one of the research related LinkedIn groups I belong to. As a result today I have a semi hypothetical question I’d like to put out to the marketing research community.
Let’s assume that a customer satisfaction survey has these four quite common questions:
- What is your likelihood to recommend the brand/product?
(10 point scale) - What is your likelihood to try the brand/product again?
(10 point scale) - What is your overall satisfaction with the brand/product?
(10 point scale) - Please explain why you are satisfied/dissatisfied with the brand/product?
(text comment)
If you could predict the average of questions 1-3 (with say 80% accuracy), by analyzing just question Q4, would you bother asking Q1-Q3?
What if together with question 4 and any single one of the other likert scale questions you could predict the other two questions with >90% accuracy, would you still ask the other two questions?
![[Post to Twitter]](http://www.tomhcanderson.com/wp-content/plugins/tweet-this/icons/tt-twitter-big1.png){kind=icon}
3 responses so far ↓
1 Terry Grapentine // Jul 12, 2012 at 8:52 pm
There is a long answer to your question—I’ll give you the short version. The short answer is “it depends,” based on my experience. I have tested just your question—almost word for word, and I used computer based text analysis. IMO, it does not work well in categories where high satisfaction does not predict future brand purchase—such as cell phones (e.g., where new products are constantly coming out and consumers just want to try something different). It may work better where there is “brand purchase inertia.” It CAN work better the more attention (e.g., time and cost) is invested in coding the text data. It also helps if you can probe respondents on their open ended responses. Generally, the more probing and the more time spend coding, the closer the open ended will predict the close ended questions. But that coding can be quite difficult.
2 Ian Straus // Jul 13, 2012 at 5:39 pm
I am not too eager to ask the first two questions anyway, in my current job.
The fact is, in public transportation we’re not dealing with a big proportion of new users on a daily basis. So #2, a likelihood to repurchase question, is much less relevant than it would be for a manufacturer of big ticket items or a cruise line or…
and indeed in our context it sounds peculiar to most customers.
As for likelihood to recommend, I’ve asked it and realize first, that people don’t spend a lot of time recommending methods of commuting; second that the answer may say a lot more about the individual than about the service.
So the more honest and accurate the answer, the less we may find it useful. Whereas if it’s dishonest and really just a repetition of the satisfaction question it is redundant.
The situation is much different for movies or books or higher education, things that I perceive get more recommendation time.
3 andreibuspro // Jul 15, 2012 at 2:44 am
It depends really on what medium these questions are asked. But I find the first 2 questions noteworthy.
Leave a Comment