Tom H. C. Anderson - Next Gen Market Research™phd literature review help
help me with my homework proxy
online help for writing research papers
writing windows services in vb
write my essay 4 me
top resume service online
help on writing scholarship essays
1984 george orwell help
order conspectus paper
cv writing service reviews uk
freelance writers online
custom paper napkins toronto
professional personal essay writers
help with writing a sonnet poem
custom essay papers 7
how to write a windows service in c 2008
essay good bad customer service
essay writing service legit
help with an essay
professional resume writers groupon
writers freelance contract
new zealand essay writing service
professional assignment writers australia
help with english homework ks3
what is the best college essay editing service
digest writing services london
abstract helpers
report writer cover letter
dissertation help new york
business finance homework help
custom made personal statement
buy a condensation
homework help textbook solutions
what should i write my essay about
help essays
cv writing services reviews uk
top 10 dissertation writers
ap english literature essay help
death penalty research paper help
my community service essay
do my assignment cheap
professional cv services
top recommendation letter writing services online
buy papers online
does homework help high school students
cover letter for internship help
physics homework helper
need synopsis help
i don39t want to do my homework help
customized paper bags uk
help me write an essay free
accounting homework services
admission essay help
customer service term paper topics
help with research paper thesis statement
abridgment help san diego
how to write my dissertation proposal
help with writing report cards
research paper on customer relationship management .pdf
top 10 resume writing services
jurisprudence essay help
buy cheap toilet paper
best online condensation writing services
custom essay writing services cheap
buy an essay in the uk
help writing a precis for a precis
homework help for 7th grade math
best professional abstract writing services
homework help computer science
sat essay help
places to buy resume paper
speech helpers
need help writing my paper
paper mario helpers
best resume writers in atlanta
best custom college papers
video games help improve critical thinking
resume and cover letter writing services
phd thesis help india
how does creative writing help kids
buy your own iron on paper
custom writer39s reference
i didn39t do my homework help
case study dissociative identity disorder
holt middle school math course 1 homework help
cheap papers for printer
certified federal resume writing service
conspectus help chicago
paperback writer chords beatles
annotated bibliography helper
cheap paperback books online
examples of customer service resume titles
buy paper bags online ireland
curriculum vitae professional service uk
management accounting homework help
create custom css tumblr
purchase a conspectus
custom made essays free
cv writing services in kenya
executive resume writing service minneapolis
christian book review of the help
business plan writer raleigh nc
essay about help poor people
resume writing services ottawa ontario
research paper order online
free help with writing papers
things to research when buying a house
purchase essay papers
custom writing paper template
order custom rolling papers
freelance online writer resume
free resume writer wizard
application letter for janitorial services
birth order essay outline
houston area resume writing service
homework helpers chemistry review
custom paper napkins
professional cv writing service reviews
reading helps critical thinking
argumentative essay about is homework helpful or harmful
professional resume writing service singapore
application letter for customer service agent
free help with cv writing
the help film review independent
custom research north america
buy college books online india
college essay service trip
buy authentic college football jerseys
essay writers free
help with a brief
help with assignments online
hiring a grant writer
how to get paid to write movie reviews
resume writing services columbia sc
help assignment locus
get paid to write reviews in india
discursive essay help
freelance writers paid
cv cover letter service
how much does a music ghostwriter make
college accounting homework help
personal statement community service example
best outline writing services in atlanta ga
count desk essay neatness writer
research papers customer retention strategies
medical cv writing service uk
research before buying a car
buy essay online for cheap
ghostwriting services australia
make business budget plan
buy your research paper customer reviews
need help making a compendium
homework help chemical equations
custom essay meister coupon
write my essay please
need help writing a persuasive essay
get homework done for free
buy a cv
buy a financial planning business
a case study on bipolar disorder
i need free math homework help
medical school personal statement service
how to write a essay to get into college
help with abridgments
cheap private universities
buy already written essays
resume services in miami fl
inexpensive resume writing services
executive digest writers nyc
graduate essay service
research proposal writing service
certified resume writers toronto
help with writing digest
best resume writers in uk
art homework help
criterion online essay evaluation services
sample cover letter for web content writer
buying dissertations
order recommendation letter paper
homework help hotline atlanta
cv writers in leicester
write my critical thinking
where to buy nice paper for resume
resume writing services oklahoma city ok
cv writing help london
customer service presentation powerpoint
how does critical thinking help a student
professional condensation services
essays on helping the community
online homework helper free
professional cv services limerick
best article writing services
write my high school admissions essay
famous essay writers
writing paper with borders printable
homework help riverside ca
help with brief
free online help with english homework
custom essay help

More Than Market Research - Gain The Information Advantage

Tom H. C. Anderson - Next Gen Market Research™ header image 6

Text Analytics Guru Interview

September 20th, 2010 · 11 Comments

Anderson Analytics’ Tom H. C. Anderson speaks with Seth Grimes about text mining

If you know of text analytics you know of Seth Grimes, he is a true Text Analytics Guru. While I may know everything there is to know about text analytics in market research, Seth’s knowledge is far broader encompassing all of Business intelligence (BI).

TomHCA: Welcome Seth, so happy to finally get to interview you for the blog. I very much enjoy your text analytics conferences and events!

Seth Grimes: Thanks Tom, and I’ll start by saying that I’ve learned a lot from your writing and presentations [Anderson Analytics], for instance about “triangulation“ methodologies for NGMR.

TomHCA: Thank You Seth, Likewise! As most of NGMR’s readers are market researchers, can you tell us a bit about how you define BI, and also of course your definition of Text Analytics?

Seth Grimes: Business Intelligence is a confluence of information, analysis software, and business processes that transform data into insights that support better business decision making.

Most of the BI world — especially software heavyweights including IBM/SPSS, SAS, SAP/Business Objects, Microsoft — has defined BI as analysis of sales, marketing, customer transactions, and other data from operational systems. But now they’re all seeing how limiting this view is, that customers need and want to bring social media, enterprise feedback, online news, and other “unstructured” sources into enterprise BI initiatives.

This “unstructured data challenge” is why I got into Text Analytics, and it’s the starting point for my latest conference, Smart Content, covering content analytics. It’s slated for October 19 in New York. We have some great speakers so do check it out.

In sum, getting back to the question: It wouldn’t be inapt to define Text Analytics as Business Intelligence focused on text.

TomHCA: Text Analytics has been around, well I guess you could say since
just after WWII with the first crypto/translation related efforts.
Given this, are you surprised we’re not farther along than we are now?

Seth Grimes: Yes, text analytics has been around for a long time. IBM researcher Hans Peter Luhn published seminal papers in the ’50s that actually defined BI as knowledge extraction from text,
but it’s obvious in retrospect why BI became something else, analysis of sales, financial, and marketing data and the like. That data is “low hanging fruit”: easy to analyze, containing A LOT of business value.

Contrast automated text analysis. In the words of expert systems pioneer Edward Feigenbaum, “Reading from text in general is a hard problem because it involves all of common sense knowledge.” Further, the link between information captured in text and business challenges is frequently not direct.

Net result: Business focused on the easier analysis need, on data in databases, but now we’ve begun to see the true potential of automated text analytics and we finally have the tools to do the job well.

TomHCA: When I started Anderson Analytics in 2005 with aim of bringing text analytics to market research, it seemed no one in my field had heard of it. Then later in 2007 Nielsen (BuzzMetrics) and TNS (Cymfony) got into it a little for the sake of social media. Now on the other hand, perhaps especially because of Twitter, it seems to be one of the hottest buzz words around! Is it just me or does there seem to be explosive growth in just the past 2-3 years? Surely it isn’t all do to social media monitoring. How have you seen the use of Text Analytics evolve more recently?

Seth Grimes: Yes, you were out ahead, and I think it’s in 2005 that we first met, at the first Text Analytics Summit. When I started looking at text analytics in 2002 or so — check out, for instance, The Word on Text Mining from 2003 — really only folks in life sciences and intelligence were using the technology.

Now there are solutions for every industry and business function that can benefit — and that means every organization that’s online or communicates electronically in any way. The growth in awareness and uptake does come from user-generated content — social platforms and also e-mail and messaging — and because publishing, marketing, advertising, and customer support has shifted its primary focus to online and other electronic channels.

Further, there are solutions that range from traditional installed software to online, as-a-service offerings, both free-standing text analytics for those who want it and, most importantly, built into line-of-business applications where the user doesn’t even know she’s doing text analytics.

TomHCA: OK, How about Natural Language Programming (NLP)? It seems to me, based on the many vendors we have worked with and investigated, that everyone claims their software is using some state of the art NLP algorithm. And of course it’s usually completely black box. It seems there is ‘A LOT’ of hype here. What are your thoughts about where we really are and where firms ‘claim’ to be. Is there a gap? What should customers look for?

Seth Grimes: Natural Language Processing: We could get into a deep discussion about statistical approaches versus the use lexicons and grammar rules of also machine learning. The science is published for anyone who wants to learn it, but most folks in business don’t want to, nor do they need to.

Business wants solutions that “just work,” and they can have them.
Fortunately, solutions are testable: How well do they “just work” for you own business problems, whether in market research or competitive intelligence or customer service? Sure, there’ll likely be a gap, wide if you choose the wrong solution, bridgeable if you choose well. There’s no one-size-fits-all set of selection criteria. I make this point over and over again to consulting clients, also that if create the right short list you’ll be most of the way there, to a solution that “just works” (for you).

TomHCA: Yes, makes sense… Taking sentiment as an example, a lot of fuss is made about how accurate this is, yet mostly it seems sentiment is off by +/- 20-30% what are your thoughts about where software vendors say they are and where we actually are in this regard? Also, does it really matter. I mean, as long as it’s consistently off differences can be measured right?

Seth Grimes: Untrained tools can be 50% accurate in sentiment classification, or untrained they can top 80% if they were designed for the business problem at hand. Train them, and you can beat 90%, which is as good as the agreement you’ll likely get out of two humans. But this is a red herring: The argument is a distraction.

The simple fact is that computers are faster (and yes, more consistent) than humans. Computers handle huge volumes of information, working 24/7, very often allowing you to tap information sources that would have been inaccessible ten years ago.

So the simple answer, for now, is to take a hybrid approach that combines human knowledge and judgment with machine power. You’ll get better results than with either humans or machines alone.

TomHCA: Yes, that’s what we have found, and I like that “it’s a distraction”, I may borrow that.

So what industries do you feel have used Text Analytics in creative ways, can you give some examples?

Seth Grimes: We all use text analytics. Here’s an example: Type “map massachusetts”
into Google or Bing. You’ll see, first up, a map of Massachusetts. That’s because the data scientists have studied searches and they understand that a searcher who asks a search engine “map <geographic area>” probably wants a map rather than a list of documents containing those words. And they did some “named entity recognition” that sees “massachusetts” as a geographic area. This is text analytics, and it’s creative, and most important, it delivers very broad business value.

Other examples? I love one from Gaylord Hotels, which used software from Clarabridge, a vendor that focuses on customer experience management (CEM). Here’s a case-study quotation
“Automated analysis of survey comments showed that customer experience was measurably enhanced when bell services staff accompanied lost guests to their destinations within a resort, as opposed to merely pointing them in the direction they needed to go.”

Creative is great, but there are much more compelling reasons to try automated text analytics. I remember a presentation by an EDS staffer back in 2005, that his company cut processing time for large-scale employee surveys from 5 staff-days to half a day. (That was using Megaputer’s PolyAnalyst software. That kind of ROI is pretty convincing.

TomHCA: Yes, Hospitality industry certainly is rich with VOC data, and we’ve done a lot of interesting work there as well with firms such as Starwood Hotels and Flyertalk for instance. But, how about the other side of the coin, are there any specific industries that you feel are behind the curve considering the potential ROI of text analytics for them?

Seth Grimes: There’s been across-the-board uptake, sometimes more enthusiastic, sometimes less. To me, the real behind-the-curve issue involves users who handle text in isolation. I’m thinking in particular of Social Media Analytics (SMA) (which relies on text analytics). I’m getting tired of people who think the business goal of social-media use is to gain follower, friends, and connections, that success is measured in and social-media mentions and “retweets.”

That attitude is silly. Social ROI is properly measured in the ability to drive business outcomes, and that means sales and cost reductions.
Social followers have no value unless they contribute to the corporate bottom line. The only correct way to measure social ROI is to link mentions to transactions: product and service sales, resolution of customer issues, etc. Linkage entails bridging social media with enterprise operational systems. Text analytics enables semantic integration. If you’re not working toward integration, toward data fusion, that’s when you’re behind the curve.

TomHCA: Interesting and challenging. You’re down in Washington DC, lots of Pentagon, NSA, CIA, FBI contract work. Some of the government stuff I’ve seen in the past has been pretty darn low tech. I’ve often had the feeling that what we’ve been using in market research has been more powerful. I realize this probably isn’t what most people would think given what we see on the tube and Hollywood screen with RAPTOR listening into every phone conversation and email. So what’s the truth here in your opinion. Is government further ahead as I’m sure they’d like everyone to believe, or is this false?

Seth Grimes: The government is ahead and behind. The government is early to recognize, cultivate, and adopt new technolgies — think of work at DARPA and funded by the CIA’s venture arm, In-Q-Tel — but the government remains plagued by insularity, mismanagement, territoriality, and political meddling when it comes to procurement, information sharing, technology scale-out, and executing on intelligence.

I’ll add that I’d absolutely love to work with government agencies on text analysis and semantic challenges, but as an independent, I can’t afford to work the procurement bureaucracy. It’s a shame.

TomHCA: Yes, working with procurement suck, especially academic and government. How about other industries? Pharma or Finance for instance. I know Finance industry were quiet early adopters. Can you speak at all to how effective predictive models using text analytics have been in predicting stock price fr instance?

Seth Grimes: Yup, pharma. My buddy Breck Baldwin of Alias-I thinks it’ll be just a few years before a Nobel Prize award for physiology or medicine will have involved the use of text analytics — mining scientific and clinical literature — for drug discovery or related goals.

The modeling problem in finance is trickier. People have been looking for “systems” for a long time. It’s not irrelevant that the early development of statistics was linked to gambling or that “Monte Carlo” methods, named for a casino locale, are a key simulation technique. Gambling and finance are kissing cousins.

Now we understand that news can move markets and the possibility, via text analytics, to automate the extraction of information from news that can be incorporated into models. The trick is extracting the right signals, quickly, and linking it to all the rest of the market data that’s out there in ways that can reliably inform trading strategies.

Does it work? Got me. But there are certainly folks out there who are trying. Check out, for instance, Thomson Reuters News Analytics.

TomHCA: For others who want to get their hands real dirty, which computer languages you have found are better/worse for handling text analytics? And how about free/academic resources for sentiment and/or NLP?

Seth Grimes: There are lots of ways to do text analytics, and not all of them require getting deep into the technology. You can find a business focused solutions that address business needs and problems, for instance, for survey or qualitative research or social CRM (Customer Relationship Management). But you’re right, users who want to a highly performing solution may have to build (or extend) it themselves or work with a services provider that can handle that technology.

Do-it-yourselfers can try traditional, installed software. There are many choices, including open source tools such as GATE, RapidMiner and modules for programming languages such as Python and Java services.

Or check out as-a-service semantic tagging, accessed via a Web application programming interface. Examples are Thomson Reuters’ Calais and Evri, which focus on entities and terms; AlchemyAPI, which adds in concepts and topics; topic-focused TextWise; Open Amplify for relationships and intent signals; and Lexascope from Lexalytics and the Clarabridge API for

There are other options: Text-analysis solutions from companies including IBM, SAS, SAP, Attensity, TEMIS, Open Text, SRA, and others; search-focused technology from Autonomy, Endeca, Exalead, and Open Text; and a myriad of “listening platforms” that focus on social media. If you don’t mind a plug: Advising users on solutions and strategy, and vendors on product and market positioning, is a large part of my consulting practice. Also, folks who want to learn more will have a great opportunity at the up-coming Smart Content content analytics conference, October 19 in New York.

TomHCA: Thanks Seth, certainly continues to be an interesting time for us

Seth Grimes: Tom, thanks for the opportunity to do a bit of market education.
Text analytics and semantics can and should be part of Next Generation Market Research initiatives, so I was glad to have a chance to explain how.

TomHCA: Always a pleasure Seth

Managing Partner
Anderson Analytics, LLC

[More on Seth - Seth Grimes is an analytics visionary: A consultant, writer, and industry analyst working in text analytics, business intelligence, data analysis and visualization, and information strategy as applied to information-age challenges. Seth founded consultancy Alta Plana in 1997 and is a long-time contributing editor at TechWeb's, a channel expert at TechTarget's BeyeNETWORK, and founding chair of the Smart Content: The Content Analytics Conference, the Text Analytics Summit, and Sentiment Analysis Symposium.]

[Post to Twitter] 

Tags: Anderson Analytics · Business Guru · Interview · Market Research · NGMR · Social Media · Text Analytics · Tom H. C. Anderson · Uncategorized · next gen market research · seth grimes · tomhcanderson

11 responses so far ↓

  • 1 Menno Mafait // Sep 21, 2010 at 11:20 am

    Tom, I have only a few questions in life. One of them is:

    • Why is Text Analysis considered to be unstructured? Isn’t grammar the structure in natural language? When people talk to each other or write texts, is that also considered to be unstructured (words)? Why not?

    Unless grammar is fully involved in the analysis process, software will never “understand” text or the meaning of the text writer.

  • 2 Tom H C Anderson // Sep 21, 2010 at 11:22 am

    I think linguistics (NLP) is way overhyped, I actually lean more toward the statistical side of Text Mining. Luckily, perfect understanding is not needed in order for there to be actionable business insights…

  • 3 Jon L ehto // Sep 21, 2010 at 1:55 pm

    Yes, considering most searches terms are noun phrases - extracting noun phrases in all their typoed, pre/suf-fixed synonymous glory is very useful. A first pass to get the high frequency terms, followed by a 2nd (logical) pass to build rules is very productive.

  • 4 Andrew Jeavons // Sep 21, 2010 at 6:17 pm

    I enjoyed the interview. One question. I know what NLP is however I would interested in what is termed the “statistical approach”. Exactly what sort of measurements and techniques do people see as part of this appraoch ? Oh and do Kohonen nets work out for text analytics ?

  • 5 Tom H C Anderson // Sep 21, 2010 at 6:57 pm

    I would say yes, any mathematical techniques including but not limited to clustering vs. trying to use linguistic approach. Of course many of us use or claim to use both. But somehow NLP seems to sound sexier to the layman, so I hear it touted more often than it probably should be (and even when it’s not in many cases), IMHO

  • 6 Menno Mafait // Sep 21, 2010 at 8:34 pm

    Tom, another burning question: Why are we still searching for words on web pages? Why don’t we get answers to our questions?

    • Why is “Name a measure for the ratio of reflected light” not simply answered by “albedo”?

    Because without perfect understanding the word “name” is considered to be a noun instead of an imperative, by which the software misses the clue that the user requests for a technical term.

  • 7 Andrew Jeavons // Sep 21, 2010 at 8:35 pm

    Thanks for the replies. The problem I have with NLP is that it is a procedural approach to process which for people is done via associative processing within the brain. You can have grammer but no meaning of course, “colorless green ideas sleeping furiously” to use Chomsky’s old example…

  • 8 Bill Porter // Sep 22, 2010 at 2:16 am

    A real advantage with linguistic approaches is that they are not black box - you do have the chance to adjust the software to suit the business. Each business situation is different after all.

  • 9 Clark Breyman // Sep 22, 2010 at 4:06 pm


    Thanks for sharing the interview. More insight into project genesis - how clients get these projects off the ground, select consulting partners, select vendors, etc. would be golden.

  • 10 dominiq // Apr 14, 2011 at 12:49 am

    No mention of WEKA?

  • 11 You can fake, but you can’t hide | // Jun 19, 2011 at 11:10 pm

    [...] analyzing the vast contents of online conversations is becoming more and more central to market research and business intelligence, and so knowing the demographics of the writers is becoming quite [...]

Leave a Comment