Tom H. C. Anderson - Next Gen Market Research

The Best and Cheapest Panel Sample in the World, Really.

OR 

Our Survey Respondents are Unique - Just Like Snow Flakes

Some years ago I worked for a large full service research supplier. We had our own panel which we said was the best in the industry. One day while working up a proposal for my client I decided to check to see what would happen if I tried to source the sample elsewhere. Basically I wanted to know how our pricing differed. I called up one of the large panel & fielding only companies:

ME: “Hi, I’m calling from COMPANY NAME, and am interested in getting a quote on some sample I’m having difficulty sourcing internally… I’m looking for 750 completes with the following demographics and screeners….”

LARGE SAMPLE PROVIDER: “Great, I’ll work up a proposal for you and you should have it in a few hours”

10 MINUTES LATER… phone rings

ME: “Tom Anderson”

CALLER: “Hello, Tom Anderson?”

ME: “Yes?”

CALLER: “this is NAME, Panel Manager at COMPANY NAME [the company I worked for at the time], I understand from LARGE SAMPLE PROVIDER that you called them asking for a sample quote?!”

ME: “uuuh yeah I did” [thinking ohoh]

CALLER: “All external sample purchases go through our department, you should contact us for all sample needs. We will make purchases if necessary”

OK, so it’s obvious that my company didn’t want me to go outside for sample. However the scary part was how quickly they found out. I was a bit nervous about having broken company protocol (but of course as all successful managers know, “it’s easier to ask for forgiveness than permission”).

I was a bit surprised and upset about having been ratted out by the third party company I had contacted until I had time to think about what had just happened. My company was buying so much sample from this other company that they had a special account contact person managing all purchases. Obviously getting a little extra business from me was not worth compromising the large relationship with my company.

SKIP FORWARD SOME YEARS

After having given some pro bono marketing advice (nothing wrong with paying it forward) to a potential client at a very large financial company Anderson Analytics goes through an arduous vendor qualification procedure and gets put on the preferred vendor list.

LARGE FINANCIAL COMPANY CLIENT: “Tom, thanks for the help the other day. I don’t think this RFP is applicable for your company, but since you helped me out, if it’s something you are interested in and you want to take a shot at it feel free to submit a proposal.”

I look over the RFP and see it’s for a large volume of sample. It requires no analysis only a simple data merge with a third party geo-data service. I’m skeptical as to what value Anderson Analytics can add to this project as we do not have a panel for this demographic and would need to purchase it. But I figure, oh well, this is a new client, I’ll submit a proposal and at least I’ll know where we stand on these types of projects.

I follow Anderson Analytics procedure, get 3-5 competitive sample quotes from major companies with a panel and select the one that is reasonable and seems to be able to fulfill this amount quickly and accurately. I send out individual email requests.

I start by sending out to 1-2 companies that are top of mind. A few hours later when I have a break from the phone, I look up info for 3 others and send to them as well. Typically most call us back to go over the request on the phone briefly before sending on a quote. After the 5th email goes out I get a phone call from email #3 company:

PANEL CO. #3: “Hi Tom, thanks for your RFQ, can we go over your request briefly?”

ME: “Sure, I need…”

PANEL CO. #3: “[laughs] You know, I just saw this request come across. [reads specs I had provided WITH SAME WORDING VERBATIM]”

ME: “Yeah, that sounds like my RFQ allright??”

PANEL CO. #3: “OK, well I guess I’m just going to give you the same price I gave them”

ME: “Uhh, great, thanks”

Again, shocked though I shouldn’t be after what I had learned a year before. These are different companies I have contacted, not the two I had contacted a year ago. Yet once again I see how insestious the industry is. I think about what I should do. I know my client will have shopped this out to others as well and that Anderson Analytics was an afterthought. Still we’re not in the business of doing pro-bono work. I figure this will be a good test. I mark up the sample 30%, and add in our standard fee for merging the data, then send to my client. I’m thinking well, I just marked this up 30%, so I doubt we will get this, but at least my client will know where we stand price wise on something like this, as will I. Who wants to waste time on projects/relationships that aren’t mutually beneficial after all.

A couple days later after my client has received all his proposals I follow up with a call.

ME: “Hi, just wanted to follow up to see that you got our proposal and if you needed anything else?”

CLIENT: “Yes I did, I received four proposals for this project, and yours was the cheapest.!”

The client was actually a bit upset as well. He knew that I would source this elsewhere, probably from the very same companies he had requested a proposal from. However, he had assumed his position as client at a large PROMINENT FINANCIAL Co. doing a large amount of research on a regular basis would entitle him to some bargaining power/leverage.

What have we learned?

1. So, key learning here is, no one really knows where sample comes from unless you manage your own proprietary panel.

2. Smaller smarter firms can provide a higher quality product at a lower price (incredibly good news right; unless you’re a large market research co.)

3. Finally, if most data is generated from this global panel, and this is the data that we trend and compare to, does it matter? Does anyone care? Is “quality” just a cure word, and the real word is “commodity”, I think it’s more the latter, unfortunately or fortunately depending on your view point.

Yet, each time you call a few different vendors for exactly the same demo you usually get very different prices (reminds me about the Diamond Model we built). I’m aware that some smaller companies that specialize in a specific demo may really have high response, and totally confirmed respondents, yet many others are perhaps not much better then a direct mail company list.

Questions

Do you know where your survey respondents come from? Do they come from websites with pop-ups or banners “we want your opinion”, “win $1,000″, “get paid to take surveys”…
from games on Facebook “get your pirate booty!” Does it even matter?

Do you think there is any difference between the sample from at top-3 Honomichl 50 company and#49 or someone who’s not on the list at all? I’m beginning to think with a very few notable exceptions, there isn’t. If not should we buy pretty much solely on price? (After a minimal trust threshold is met). Perhaps this is our best bet…

Curious to hear your opinions?
- Tom H. C. Anderson
[NOTE: Anderson Analytics views sample quality as a serious issue and we consult with our clients to determine best and most efficient sample to meet their needs. If you think creatively about sample you will notice there are many different approaches. If Panel sample is desired we recommend you choose a company that you trust and understand how they acquire sample. We also recommend inserting red herring questions in the survey. We have selected panel partners which we feel are trustworthy, share information, have quality standards in place, and provide a reasonable price.]