Bob Lederer, Editor of RFL Communications, probably one of the people most responsible for starting the current online panel quality debate, has taken up a new torch, ROI.
Just as he organized the first event tied to panel quality (together with support and serious accusations raised by then head of research at P&G, Kim Dedeker), Bob is now organizing the first “ROI on MR” conference (Chicago July 27-28th).
To be quite honest, I’m not sure where I stand on the ROI issue. While I do feel thinking about ROI is extremely critical in certain marketing research, such as ad campaign effectiveness testing, I’m less sure whether it is possible or even desirable to set out to measure ROI of marketing research in general.
Currently the Public Relations industry (its trades and several leading PR firms) are taking a stand against measuring their ROI, at least in the same way that advertising ROI is measured (Click through’s, views etc.). Their reasoning is that PR is different, more impactful and strategic, influencing key opinion leaders rather than the masses. Therefore, how can driving 1,000 magazine editors or bloggers and influencing them be equal to driving 1,000 consumers to certain content?
How can you value the market research that stops you from taking your business down an incorrect path and takes you in another? What is the value of strategic research such as a customer segmentation which shapes every part of your marketing strategy and tactics?
One thing I can say for certain is that marketing researchers are doing an extremely poor job in communicating their value. Whether ROI is the right way to help market researchers do this or not, I do think it’s an important topic to consider. I applaud Bob Lederer for continuing to raise often unpopular questions within our field.
![[Post to Twitter]](../../../../wp-content/plugins/tweet-this/icons/tt-twitter.png){kind=icon}
Tweet This Post
15 responses so far ↓
1 Dan Gersten // Mar 28, 2011 at 8:33 am
What’s the ROI of each class you’ve ever taken in school? What’s the ROI of obtaining information that makes you smarter in some degree? How do you measure the ROI of providing insightful information that isn’t acted upon for reasons that have nothing to do with the quality of information? What was the ROI of research on prospect of New York City tourism that was presented on 9/10/01?
2 Mike Gadd // Mar 28, 2011 at 8:33 am
Must say we’ll be going back to telephone for tracking studies and advocating more
triangulated studies (observation, motivational, customer journey …) for more in-depth understanding. Have seen some very questionable results based on online panel data.
3 Tom Thompson // Mar 28, 2011 at 2:02 pm
One of the bigger challenges of MR ROI measurement will likely be separating the return s generated on how well the client makes use of the findings from those returns that could be allocated to the “correctness” or “accuracy” of the findings/insights themselves
4 Leonard Murphy // Mar 28, 2011 at 2:46 pm
Great post Tom; are you going to attend Bob’s event? I went to his panel quality summit in Chicago a few years back and it was well done, although I’m not sure if it ever ultimately achieved the goals set out for it due to a sort of collective shrug by all involved in the value chain.
I hope this event can start a serious dialogue on the issue of how we calculate our own worth as an industry and what we communicate that to clients. In the age of “pay for performance”, this is is only going to become a bigger deal for us all.
5 Tom H C Anderson // Mar 28, 2011 at 2:59 pm
I definitely don’t think that was the case at all. There has been A LOT of research on research because of that initiative. I know, Anderson Analytics has conducted a fair amount of it as an ‘independent’ part on behalf of clients.
Even other trades got into it, such as R&R commissioned by the ARF and conducted by Surveys & Forecasts.
As part of the whole panel quality debate the Offshoring Transparency Initiative was also formed when I spoke at the event during the TMRE two years ago.
And I think the discussion is still continuing and is helping with faster acceptance of other techniques such as SMR.
Personally I’m somewhere in the middle on the issue. But it took guts by Bob to ask the questions and stir the pot, and that I certainly admire!
6 Mike Gadd // Mar 28, 2011 at 3:04 pm
Just to be clear I feel that the best ROi is to be had from research that helps clients make the best decisions with their brand positioning or messaging. Inaccurate or unreliable research based or poor representation of the target is worse than no research.
7 Leonard Murphy // Mar 28, 2011 at 3:08 pm
I agree that Bob should be commended for his efforts, and I am a supporter of both his drive to focus on quality and on ROI. Bob serves a vital role in the industry and he should be honored for his contributions to much needed dialogues and tracking of the industry.
I’m aware of all of the research on research momentum, but my point is that I never saw the big panel players give it too much attention, nor did most clients, which is the biggest issue of all. If I recall, at the time Kim Dedeker said that this was a supplier issue; she expected quality as a point of entry and really didn’t want to be involved in the debate. I think that was (and is) pretty indicative oif the client mindset.
I think it’s similar to the ARF Neurostandards initiative; a great idea and a big issue, but one that ultimately won’t produce much real change. Pragmatic business realities trump all, and both sample quality and neurostandards are examples of ideas that will be trumped.
The difference with ROI is that this is an issue that clients are acutely attuned to and that is impacting both suppliers and clients. Pragmatism is in our favor on this one: it’s in the best interests of all involved to tackle this issue head on.
8 Brian Singh // Mar 28, 2011 at 3:10 pm
Maybe the same questions should be asked of accountants, lawyers, public relations and management consultants.
9 Tom H C Anderson // Mar 28, 2011 at 3:15 pm
Good point
10 Brian Singh // Mar 28, 2011 at 3:17 pm
Funny thing, they never question most of these foks - it is about the cost of doing business. But the minute they want to grow their business, whammo! MR and Marketing get the ROI kick in the butt.
11 Brian F. Singh // Mar 28, 2011 at 3:19 pm
Maybe the same questions should be asked of accountants, lawyers, public relations and management consultants.
Funny thing, they never question most of these foks - it is about the cost of doing business. But the minute they want to grow their business, whammo! MR and Marketing get the ROI kick in the butt.
12 Tom H C Anderson // Mar 28, 2011 at 3:21 pm
MR fees are already typically much lower than consulting firms like McKinsey for basically the same work…
I wonder if ROI might not be an additional step in wrong direction.
Unless with ROI we would also get a % stake in client business, like Ad Agencies used to take back in the day. But doubtful clients would ever be on board for this again no matter what ROI you could prove.
Then it would seem to make perfect sense.
13 Kevin Gray // Mar 28, 2011 at 4:23 pm
A good source for this: http://www.gordon-mccallum.com/
14 Tom H C Anderson // Mar 28, 2011 at 5:00 pm
Thanks Kevin, they are also NGMR Top Bloggers by the way.
I was just thinking, what was the ROI on the analysis that told us Japan was attacking prior to Pearl Harbor? No one listened to it.
ROI of MR is something an entire company is responsible for, not the MR department. There’s already been a lot written on how companies who spend more on research and analytics outperform their competition whether the economy is good or bad.
15 Aaron Deacon // Mar 30, 2011 at 8:45 am
If by ROI you indeed mean a metric that is analogous to those used by media buyers, that’s definitely a step in the wrong direction. You can’t undersell the thought work that goes into the research design, analysis, and recommendations. Having easily defined metrics oversimplifies a complex process and commoditizes the work.
Of course, clients want simple and they’ve been accustomed to “ROI”, especially as the social media world tries so hard to crack that nut. The best demonstration of value is through case studies, where research results can be related to hard business metrics, even if the connection if more qualitative than quantitative.
But in order to get there, you need to follow through or follow up with clients, as there is often a tendency to leave the researcher out once findings are delivered.
Leave a Comment