NGMR Text Analytics Predictions for 2012
Though I’ve been posting predictions all week I haven’t made any myself yet. Today I added my own prediction (at the bottom of this post). I wrote it before reading any of the others I received.
Looking over all the thoughtful prognostications from the ‘who’s who of text analytics’ below, while they may differ in terms of technical solutions as well as broadness and depth of use case, what I believe many have in common, and more and more of those of us working in text analytics seem to agree on, is that we will need more users with a basic understanding and skill in the use text analytics to continue to move forward.
I think we will see next year a move from a “service” model that either produce reports or allow users to install, train, etc. software for performing text analytics on their data, to “applications” that would attempt to interpret the results obtained by means of text analysis into suggested action.
Perhaps it is more a hope than a prediction.
Over the past couple of years text and sentiment analytics have been applied to traditional and social feedback (surveys, review sites, social media) to track issues, trends, correlations of experiences to survey or experience outcomes - using a mix of statistical and quantitative analysis techniques. 2012 will see a continuation of these techniques, but with a greater degree of multichannel source fusion and integration as companies look to merge all identities of a customer together, and all feedback channels into a holistic view. Additionally - I expect to see innovative organizations blurring the lines between market research and customer engagement through the application of analytics to deduce issues, problems, and sales/churn events as they are developing, and take proactive means to either: 1) engage with a customer operationally on a 1:1 basis through solutions that integrate survey and social text/sentiment analytics with customer relationship management (CRM) and business process management (BPM) solutions, or 2) resolve an issue at a more macroscopic level by using text/sentiment analytics to identify and tracking growth rates of significant but quantitatively infrequent issues and fixing them before they create a material impact on customer sentiment.
The imperative to transform text into insight and action, utilizing the ever-increasing ocean of information in social media, CRM notes, surveys, and beyond in practical applications, is driving the future of text analytics. The growth of social media especially as a source for analysis has resulted in a two-fold challenge: managing the costs and technical challenges of ingesting and processing all of that data, as well as developing new ways to make sense of it. And, of course, in the small world in which we live, you need to be able to handle multiple languages and idioms equally well. Compounding these challenges is the fact that applying text analytics to the new generation of routing and other “process-based” applications means you need far greater accuracy. I believe this means we will see more specialization within the field - vendors specializing in particular verticals, use cases, or languages - which will fuel the growth of new platform players and application providers alike.
Businesses who pay for sentiment analysis are going to start asking, “So what?”
It’s not that marketers don’t care how consumers feel about them, but that there’s not much they can do with a pie chart of percent positive, neutral or negative sentiment. The sharpest marketers will be looking for ways to make sentiment analysis pay.
2012 will be the year custom machine classifiers for unstructured text will become commonplace in corporate America. A new online, cloud-based marketplace for outstanding machine-learning classifiers will emerge.
Predictive analytics and text analytics start to merge into text predictive analytics. We will see use cases when both text and numerical analysis creates better predictions together than only looking at numerical predictions.
For instance when looking at predictions of sales volumes (or other economical data) the use of public text such as twitter could help. It’s not fiction since companies like Recorded Future do some stuff in this space and Saplo have projects where text predictive analytics have shown value.
This is one of the areas where we will see great value of text analytics on big data.
1.New applications of text analytics will center on business analysis
2. Marketers will outsource their social research and monitoring to third-parties skilled in listening, and use their own analysts for interpreting the data and drawing insights for brand strategy and tactics
3. The shortage of skilled text analysts remains
2012 will be the year of text analytics. It is now widely understood how much significant information is held in “unstructured data” and how important it is to be able to slice, dice, and mine it.
Key technologies such as entity recognition, sentiment analysis, and text classification are solid, and their benefits are now being recognized, while greater sophistication on the part of users is weeding out the snake-oil salesmen and raising the bar. The upcoming election season in the US will probably see wide deployment of text analytics for political analysis; this will raise awareness of the field even more.
We will see a rise in the variety and power of end-user applications, as vendors seek to reduce the heretofore large technical investments that have been needed to implement effective text analytics efforts.
An ongoing challenge will be to make text analytics more accessible while maintaining validity of analysis results. We will need a combination of better client education and more effective user interfaces to make this possible.
Text analytics technology has matured over the past several years. In 2012, text analytics will become a known commodity for businesses that are still unaware of this crucial technology, and will continue to be adopted at a rapid pace for many different uses like social media monitoring, improving the customer experience, big data management, real-time search, and sentiment analysis. Expect large increases in text analytics use across the healthcare, insurance, finance, legal, and hospitality industries.
This is the year that the text analytics market grows up. Smart people who always ask the margin of error for an election poll result somehow take text analytics claims at face value. I believe that buyers are becoming more sophisticated about both what text analytics can do and about how often it gets it wrong
The easiest prediction to make about 2012 text analytics is continued strong market growth — my estimate is 25% on a base that likely topped $1 billion globally in 2011 — as uptake expands throughout the enterprise and as the technology becomes a must-have value-booster for broad-market survey, social/media analytics, and CRM platforms.
With less certainty: We may look back on 2012 as the Year of Question Answering, of the deployment IBM Watson/Apple Siri-type technologies to respond to enterprise and consumer needs ranging from customer (self-)service to medical diagnosis, as a semanticized information-access replacement for tired old search.
And there are signs, from market leaders such as SAP and IBM and from innovative start-ups alike, that 2012 will be the year of *effective* data fusion across database and text (a.k.a. “unstructured”) sources.
Business can’t, won’t, wait for prescriptivist, rigid Semantic Web approaches but is instead applying analytics to the job, to discover the connections that make for truly rich data. You need analytics to operate in real time, to keep up with the data torrent. Many of those efforts will incorporate information mined from audio (speech), image, and video sources as a evolution from text analytics to content analytics picks up speed
Social media is the elephant in the room - no decision management system will escape to the impact of social media. My prediction is that social media data, still analyzed in a silo, will be fully part of the 360 degrees view on customers, improving how to attract, retain new customers, sell them more stuff or identify and reduce risk and fraud.
Social Media Monitoring, including sentiment analysis, will become more and more a commodity and focus will be on integration with decision-based systems
I think that some of the more established players are targets for acquisition by companies that serve the enterprise market. Large companies have vast stores of data, plus get a lot of activity in social media, and its harder to make sense of it for meaningful business decisions. Enterprises want to make sense of their private as well as public social media data, and it doesn’t make sense to use a variety of platforms to do that.
I also think there will be a continuing drive to tune text analytics to create richer results within industry or market sectors (finance, pharma, CPG, etc) and to get away for more generic analysis
The gap between software developer promises and user expectations will narrow as consumers choose text analytics software designed for their specific use cases. More importantly, 2012 will be the year of text analytics user education. Clients will finally begin to understand that text analytics software is not a black box where any data goes in and finished reports magically appear. Just as with structured data analytics software, a skilled user is necessary to get the best actionable insights desired. Even software firms which previously marketed their wares primarily via an NLP-smoke-and-mirrors (you don’t know what it’s doing, just trust us)-wow factor will realize that a more aware and educated customer is a happier customer.
Big thanks to all my fellow text analytics colleagues who responded with their predictions today.
Please check back tomorrow as I close the week out with predictions from some friends in the marketing research c-suite.