Tom H. C. Anderson - Next Gen Market Research™

More Than Market Research - Gain The Information Advantage

Tom H. C. Anderson - Next Gen Market Research™ header image 6

New to Text Analytics?

September 3rd, 2014 · No Comments

6 lessons for CIOs and CMOs who are new to text analytics
(Free White Paper)

Unstructured data is the most prevalent form of information on the planet. It also underpins much of our communication. It exists in our e-mails, surveys, social media accounts, call center logs, etc. With a strong text analytics strategy in place, companies can get critical information from this data to drive better business decisions.

You may recall my blog interviews with several client side analytics managers ahead of this years’ Useful Business Analytics Summit.

Data Driven Business has compiled a free white paper which focuses on the business benefits (and challenges!) of text analytics from the perspectives of 4 of these experts from Highmark Health, Toyota, Mozilla, and Visa (All who are slated to speak at the 13th Annual Text Analytics Summit West on November 4-5 in San Francisco.

You can download the free whitepaper here: http://goo.gl/hxZg80

Some of the issues covered include:

· 6 lessons for CIOs and CMOs who are new to text analytics

· Identifying KPIs for your text analytics initiatives

· Barriers to adoption and how to overcome them

Big thanks to the following contributors:

Mark Pitts, Vice President, Enterprise Informatics, Data & Analytics at Highmark Health

Farouk Ferchichi, Executive Director at Toyota Financial Services

Matthew P.T. Ruttley, Manager of Data Science at Mozilla Corporation

Ramkumar Ravichandran, Director, Analytics at Visa

@TomHCAnderson

[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Post to Twitter] 

→ No CommentsTags: Text Analysis · Text Analytics · Text Analytics News · Text Analytics Software · Text Analytics Summit · text mining

Only Two Weeks Left to Nominate for 2014 NGMR Awards

August 25th, 2014 · No Comments

History of the NGMR Awards and 2014 Nomination Reminder

Today on The Market Research Event blog there is a post on how the annual Next Gen Market Research awards began.

Each year the winners are honored at the biggest US Market Research Event (TMRE) and then participate in a panel discussion about disruptive innovation. This year the post award discussion will be moderated by Kristin Luck, President & CMO at Decipher, and founder of WIRe (Women In Research).

Kristin Luck & Tom H. C. Anderson

You have less than two weeks left to submit your company or a deserving colleague at another company for consideration - so please submit your nominations ASAP! (Better to be early than late. We will ask for additional info if needed)

Nomination Instructions can be found here.

Check out interview with 2013 and 2012 winners here.

Good Luck!

@TomHCAnderson

[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Post to Twitter] 

→ No CommentsTags: Uncategorized

Text Analytics VS Other Next Gen Methods

August 18th, 2014 · No Comments

The Great Methodology Debate - IIEX2014

For those of you who weren’t able to attend the Insights Innovation Exchange in Atlanta this summer or missed the end of day 3 you can watch The Great (Next Gen Market Research) Methodology Debate below. I tried to explain in a limited time why I think Text Analytics is by far the most important methodology and the one with the biggest implications (extending far beyond traditional market research).

My fellow debaters made some interesting points as well. Curious to hear your thoughts.

@TomHCAnderson

[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Post to Twitter] 

→ No CommentsTags: Social Media Monitoring · Text Analysis · Text Analytics · social-media analytics · text mining

Ice Bucket Challenge

August 16th, 2014 · 1 Comment

I was challenged by Lenny Murphy Editor of Greenbook.

I’m Calling Out: Gregory Shapiro, Sid Banerjee, Grant McCracken and Joe Fernandez – YOU HAVE 24 HOURS!

@TomHCAnderson

PS. Bonus $$$ if IBM Pours Water on Watson

[More about ALS #IceBucketChallenge Here]

[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Post to Twitter] 

→ 1 CommentTags: Social Media · Uncategorized

Conference Tweets I Would Like to See

August 12th, 2014 · 5 Comments

By The Snarky Brothers (AKA Kevin Lonnie & Tom H. C. Anderson)

Ever get a little antsy after day 1 at a market research conference? If you attend a lot of conferences chances are you do, and that you’re not alone. Kevin Lonnie of KL Communications reached out to me and suggested a joint post entitled “Conference Tweets I Would Like to See“. I told Kevin, this isn’t too different than what I usually tweet at conferences, so why not.

Below are 24 of these tweets. We’d love to hear which if any you like, or better yet, please submit your own in the comment section below. We’ll buy whomever has the best suggested tweet a drink at the next research event.

Additionally, all the odd tweets are from one of us, and all the even tweets from the other, so if you’re looking for another challenge please feel free to guess which tweets belong to which Snarky Brother, DJ MROC or DOCTOR TEXT?!?

  1. Are they recording this presentation? I could play it back the next time I have trouble sleeping. #insomnia
  2. Oh, it’s Twitter data, why didn’t you say so, that’s why methodology doesn’t matter #GullibleMRXer
  3. Oh wow, chicken for lunch. What were the odds? #rubberchicken
  4. Did that focus group moderator actually just say “Big Data” #JustSlightlyOutOfBoundsExperienceWise
  5. Why don’t we cut to the chase and play paint ball in the exhibit hall. Any client who gets shot has to endure a sales pitch. Clients receive sales immunity from vendors they shoot. #hungergames
  6. Another F*ing Word Cloud, wow, how creative! #QualBigData
  7. Oh my, another “evolve or die” theme. I wonder what underpaid summer intern came up with this one? #deathofcreativity
  8. Just Wake me for Happy Hour #SmartAndExperiencedAttendee
  9. If the guy next to me leans any closer, he’ll be behind me. #marxbrothers
  10. Ohhh wow 10 Sample Providers in the Exhibit Hall each with biggest highest quality most unique panel #DejavuLOL
  11. I always learn the shortcut to the conference area on the last day. #gpschallenged
  12. “[Insert Quote]” - [Insert Big Brand Name] *REPEAT* #ShamelessToadyImpressedBybIGBrands
  13. Would it kill folks to actually time their presentations? I say we cut their mikes when their time expires. If they insist on still talking, then we start to zap them! #swisspercision
  14. We found xyz correlated with the price of cheese in Cambodia #I’mAstatisticianJustTrustMe
  15. Of course, the only two speeches I want to see are at the exact same time. #cloning
  16. Nother Client preso heavily edited by legal. Why don’t they just post their url and be done with it #Zzzzzz
  17. What the hell did that presentation have to do with innovation? It should have been titled: “Innovative ways to shamelessly plug my company.” #shamelesssalespitch
  18. Today we have replaced the coffee stations with beer kegs, enjoy! #wishfulthinking
  19. CONFERENCE RHETORIC: Nine tracks of learning! REALITY: Nine revenue streams! #cynic
  20. Innovative?!? My company was doing this 10 years ago #timetravel
  21. Lifetime achievement award winners. Also known as, “OMG, I can’t believe he’s still alive!” #goingstraighttohell
  22. I saw this same preso just last week at a conference in Chicago! #TooManyF*ngConferences
  23. I say instead of evolving, we regress back to a simpler time and place. Let’s forget about big data & data scientists and bring back the old dart board! #MarketingNeanderthal
  24. Hmmm, looks like I’m the only one on Twitter at this event *talking to myself* #AcademicConference

Thanks, hope you enjoyed and Tweet you soon!

@TomHCAnderson

@KLonnie

PS. If you enjoy research humor, check out last week’s ‘text analytics’ cartoon!

[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Post to Twitter] 

→ 5 CommentsTags: Conferences · Text Analytics · Twitter

Big Data Meets Text Analytics

August 5th, 2014 · 1 Comment

[Interview Re-posted from Text Analytics News via OdinText Blog]

Mr. Big Data & Mr. Text Analytics Weigh In

Structured VS. Unstructured Data

kirk_borne Text Analytics News

If you pay attention to Big Data news you’re sure to have heard of Kirk Borne who’s well respected views on the changing landscape are often shared on social media. Kirk is professor of Astrophysics and Computational Science at George Mason University. He has published over 200 articles and given over 200 invited talks at conferences and universities worldwide. He serves on several national and international advisory boards and journal editorial boards related to big data

tom_anderson Text Analytics News

Tom H. C. Anderson was an early champion of applied text analytics, and gives over 20 conference talks on the topic each year, as well as lectures at Columbia Business School and other universities. In 2007 he founded the Next Gen Market Research community online where over 20,000 researchers frequently share their experiences online. Tom is founder of Anderson Analytics, developers of text analytics software as a service OdinText. He serves on the American Marketing Association’s Insights Council and was the first proponent of natural language processing in the marketing research/consumer insights field.

Ahead of the Text Analytics Summit West 2014, Data Driven Business caught up with them to gain perspectives on just how important and interlinked Big Data is with Text Analytics.

Q1. What was the biggest hurdle that you had to overcome in order to reach your current level of achievement with Big Data Analytics?

KB: The biggest hurdle for me has consistently been cultural — i.e., convincing others in the organization that big data analytics is not “business as usual”, that the opportunities and potential for new discoveries, new insights, new products, and new ways of engaging our stakeholders (whether in business, or education, or government) through big data analytics are now enormous.

After I accepted the fact that the most likely way for people to change their viewpoint is for them to observe demonstrated proof of these big claims, I decided to focus less on trying to sell the idea and focus more on reaching my own goals and achievements with big data analytics. After making that decision, I never looked back — whatever successes that I have achieved, they are now influencing and changing people, and I am no longer waiting for the culture to change.

THCA: There are technical/tactical hurdles, and methodological ones. The technical scale/speed ones were relatively easy to deal with once we started building our own software OdinText. Computing power continues to increase, and the rest is really about optimizing code.

The methodological hurdles are far more challenging. It’s relatively easy to look at what others have done, or even to come up with new ideas. But you do have to be willing to experiment, and more than just willingness, you need to have the time and the data to do it! There is a lot of software coming out of academia now. They like to mention their institution in every other sentence “MIT this” or ‘UCLA that”. The problem they face is twofold. On the one hand they don’t have access to enough real data to see if their theories play out. Secondly, they don’t have the real world business experience and access to clients to know what things are actually useful and which are just novelty.

So, our biggest hurdle has been the time and effort invested through empirical testing. It hasn’t always been easy, but it’s put me and my company in an incredibly unique position.

Q2. Size of data, does it really matter? How much data is too little or too much?

THCA: Great question, with text analytics size really does matter. While it’s technically possible to get insights from very small data, for instance on our blog during the elections one of my colleagues did a little analysis of Romney VS. Obama debate transcripts, text analytics really is data mining, and when you’re looking for patterns in text, the more data you have the more interesting relationships you can find.

KB: Size of data doesn’t really matter if you are just getting started. You should get busy with analytics regardless of how little data you have. The important thing is to identify what you need (new skills, technologies, processes, and data-oriented business objectives) in order to take advantage of your digital resources and data streams. As you become increasingly comfortable with those, then you will grow in confidence to step up your game with bigger data sets. If you are already confident and ready-to-go, then go! The big data revolution is like a hyper-speed train — you cannot wait for it to stop in order to get on board — it isn’t stopping or slowing down! At the other extreme, we do have to wonder if there is such a thing as too much data. The answer to this question is “yes” if we dive into big data’s deep waters blindly without the appropriate “swimming instruction” (i.e., without the appropriate skills, technologies, processes, and ​data-oriented business objectives)​. However, with the right preparations, we can take advantage of the fact that bigger data collections enable a greater depth of discovery, insight, and data-driven decision support than ever before imagined.

Q3. What is the one thing that motivates and inspires you the most in your Big Data Analytics work?

KB: Discovery! As a scientist, I was born curious. I am motivated and inspired to ask questions, to seek answers, to contemplate what it all means, and then to ask more questions. The rewards from these labors are the discoveries that are made along the way. In data analytics, the discoveries may be represented by a surprising unexpected pattern, trend, association, correlation, event, or outlier in the data set. That discovery then becomes an intellectual challenge (that I love): What does it mean? What new understanding does this discovery reveal about the domain of study (whether it is astrophysics, or retail business, or national security, or healthcare, or climate, or social, or whatever)? The discovery and the corresponding understanding are the benefits of all the hard work of data wrangling.

THCA: Anyone working with analytics has to be curious by nature. Satisfying that curiosity is what drives us. More specifically in my case, if our clients get excited about using our software and the insights they’ve uncovered, then that really gets me and my whole team excited. This can be challenging, and not all data is created equal.

It can be hard to tell someone who is excited about trying Text Analytics that their data really isn’t suitable. The opposite is even more frustrating though, knowing that a client has some really interesting data but is apprehensive about trying something new because they have some old tools lying around that they haven’t used, or because they have a difficult time getting access to the data because it’s technically “owned” by some other department that doesn’t ‘Get’ analytics. But helping them build a case and then helping them look good by making data useful to the organization really feeds into that basic curiosity. We often discover problems to solve we had no idea existed. And that’s very inspiring and rewarding.

Q4. Which big data analytics myth would you like to squash right here and now?

KB: Big data is not about data volume! That is the biggest myth and red herring in the business of big data analytics. Some people say that “we have always had big data”, referring to the fact that each new generation has more data than the previous generation’s tools and technologies are able to handle. By this reasoning, even the ancient Romans had big data, following their first census of the known world. But that’s crazy. The truth of big data analytics is that we are now studying, measuring, tracking, and analyzing just about everything through digital signals (whether it is social media, or surveillance, or satellites, or drones, or scientific instruments, or web logs, or machine logs, or whatever). Big data really is “everything, quantified and tracked”. This reality is producing enormously huge data volumes, but the real power of big data analytics is in “whole population analysis”, signaling a new era in analytics: the “end of demographics”, the diminished use of small samples, the “segment of one”, and a new era of personalization. We have moved beyond mere descriptive analysis, to predictive, prescriptive, and cognitive analytics.

THCA: Tough one. There are quite a few. I’ll avoid picking on “social media listening” for a bit, and pick something else. One of the myths out there is that you have to be some sort of know it all ‘data scientist’ to leverage big data. This is no longer the truth. Along with this you have a lot dropping of buzz words like “natural language processing” or “machine learning” which really don’t mean anything at all.

If you understand smaller data analytics, then there really is no reason at all that you shouldn’t understand big data analytics. Don’t ever let someone use some buzz word that you’re not sure of to impress you. If they can’t explain to you in layman’s terms exactly how a certain software works or how exactly an analysis is done and what the real business benefit is, then you can be pretty sure they don’t actually have the experience you’re looking for and are trying to hide this fact.

Q5.What’s more important/valuable, structured or unstructured data?

KB: Someone said recently that there is no such thing as unstructured data. Even binary-encoded images or videos are structured. Even free text and sentences (like this one) are structured (through the rules of language and grammar). Even some meaning this sentence has. One could say that analytics is the process of extracting order, meaning, and understanding from data. That process is made easier when the data are organized into databases (tables with rows and columns), but the importance and value of the data are inherently no more or no less for structured or unstructured data​. Despite these comments, I should say that the world is increasingly generating and collecting more “unstructured data” (text, voice, video, audio) than “structured data” (data stored in database tables). So, in that sense, “unstructured data” is more important and valuable, simply because it provides a greater signal on the pulse of the world. But I now return to my initial point: to derive the most value from these data sources, they need to be analyzed and mined for the patterns, trends, associations, correlations, events, and outliers that they contain. In performing that analysis, we are converting the inherent knowledge encoded in the data from a “byte format” to a “structured information format”. At that point, all data really become structured.

THCA: A trick question. We all begin with a question and relatively unstructured data. The goal of text analytics is structuring that data which is often most unstructured.

That said, based on the data we often look at (voice of customer surveys, call center and email data, various other web based data), I’ve personally seen that the unstructured text data is usually far richer. I say that because we can usually take that unstructured data and accurately predict/calculate any of the available structured data metrics from it. On the other hand, the unstructured data usually contain a lot of additional information not previously available in the structured data. So unlocking this richer unstructured data allows us to understand systems and processes much better than before and allows us to build far more accurate models.

So yes, unstructured/text data is more valuable, sorry.

Q6. What do you think is the biggest difference between big data analysis being done in academia vs in business?

KB: Perhaps the biggest difference is that data analysis in academia is focused on design (research), while business is focused on development (applications). In academia, we are designing (and testing) the optimal algorithm, the most effective technique, the most efficient methodology, and the most novel idea. In business, you might be 100% satisfied to apply all of those academic results to your business objectives, to develop products and services, without trying to come up with a new theory or algorithm. Nevertheless, I am actually seeing more and more convergence (though that might be because I am personally engaged in both places through my academic and consulting activities). I see convergence in the sense that I see businesses who are willing to investigate, design, and test new ideas and approaches (those projects are often led by data scientists), and I see academics who are willing to apply their ideas in the marketplace (as evidenced by the large number of big data analytics startups with university professors in data science leadership positions). The data “scientist” job category should imply that some research, discovery, design, modeling, and hypothesis generation and testing are part of that person’s duties and responsibilities. Of course, in business, the data science project must also address a business objective that serves the business needs (revenue, sales, customer engagement, etc.), whereas in academia the objective is often a research paper, or a conference presentation, or an educational experience. Despite those distinctions, data scientists on both sides of the academia-business boundary are now performing similar big data analyses and investigations. Boundary crossing is the new normal, and that’s a very good thing.

THCA: I kind of answered that in the first question. I think academics have the freedom and time to pursue a research objective even if it doesn’t have an important real outcome. So they can pick something fun, that may or may not be very useful, such as are people happier on Tuesdays or Wednesday’s? They’ll often try to solve these stated objectives in some clever ways (hopefully), though there’s a lot of “Pop” research going on even in academia these days. They are also often limited in the data available to them, having to work with just a single data set that has somehow become available to them.

So, academia is different in that they raise some interesting fun questions, and sometimes the ideas borne out of their research can be applied to business.

Professional researchers have to prove an ROI in terms of time and money. Of course, technically we also have access to both more time and more money, and also a lot more data. So an academic team of researcher working on text analytics for 2-3 years is not going to be exposed to nearly as much data as a professional team.

That’s also why academic researchers often seem so in love with their models and accuracy. If you only have a single data set to work with, then you split it in half and use that for validation. In business on the other hand, if you are working across industries like we do, while we certainly may build and validate models for a specific client, we know that having a model that works across companies or industries is nearly impossible. But when we do find something that works, you can bet it’s going to be more likely to be useful.

[Post to Twitter] 

→ 1 CommentTags: Big Data · Odin Text · OdinText · Text Analysis · Text Analytics · Text Analytics News · Text Analytics Software · Text Analytics Summit · Text Mining Guru · text mining