Tom H. C. Anderson - Next Gen Market Research™Time Management Essays Conclusion Meaning Of Alexander Pope#39s Essay On Criticism Mla Citing Sources Websites How To Write An Essay Plan Template Essay On Removing Corruption How To Write A Persuasive Research Paper Outline Accounting Research Paper Topics 2012 10 Cover Letter Mistakes Sample Cover Letter For Job Application Template Creative Writing Piece Examples Common Application Essay Writing Tips Case Study Paper In Apa Format Application Letter For Senior Administrative Assistant Essay Writing University Level University Of Utah Creative Writing Online How To Write A Good Compare And Contrast Essay Intro Sample Paper Term 2 Cbse Class 10 Annotated Bibliography American History Essay On Self Respect In Hindi Annotated Bibliography Apa 2012 Argumentative Essay About Is Homework Helpful Or Harmful General Essay Rubric High School Help Writing Your Personal Statement Ucas Application Letter Format Sample Job Argumentative Essay Verbs Apush Chapter 10 Essay Questions Glamour Essay Contest Winners Parts Of A Research Paper Middle School Mla Citation Pdf From Website Freelance Writing Jobs For Beginners Uk Opinion Essay Examples Pdf Argumentative Essay On Beauty Math Homework Help Chat Rooms Creative Writing Teacher Letter Of Employment Verification Template A Short Essay About Respect Normal Margins For Essay Descriptive Essay Writing Powerpoint Contracts 1 Essay Questions Personal Statement Colorado State University Essay About Life Is Like A Box Of Chocolates Steps In Writing Library Research Paper Easybib Student Case Study Entity Framework Persuasive Essay Topics For High School Students Writing Your Personal Statement For Medical School Cover Letter Template Office Nature 459 Essay Critical Thinking Rubric For Elementary Writing A Cause And Effect Essay Examples College Essay Topics About Yourself Coursework Extension Uea Argument Essay Body Image Annotated Bibliography On Business Ethics Letter Of Intent 501c3 Literature Review Teaching Writing Essay On Nightlife In Mumbai Conducting A Literature Review Dissertation Business Cover Letter Salutation Argumentative Essay Format Topics Dissertation Writing Case Study Examples Management Consulting Get Paid To Write Sports Articles Online Essay On Love Your Neighbour As Yourself Creative Writing High School Course Description Research Paper Third Person Writing How To Write A Paper In Apa Format In Microsoft Word Term Paper Outline Example Structure College Application Essay Set Up High School Research Paper Outline Example Introduction To Epidemiology Case Study Answers Essay Jackie Robinson Cover Letter Quality Assurance Position Job Reference Letter In Word Expository Essay Writing Prompts High School Creative Writing Exercises Middle School Purdue Owl Cover Letter Powerpoint Cover Letter Applying Online Job Literary Analysis Essay Introduction Examples Science Case Study Rubric Cover Letter On Email Template Spacing In Annotated Bibliography Mla Creative Writing Phd Waiver Sample Research Paper Methodology Section How To Write Joining Report After Leave Tips On Writing A Rhetorical Essay Creative Writing Digital Tools Annotated Bibliography For A Dictionary Law Coursework Answers Extended Essay Guidelines Word Count Essay About Meaningful Life Resumes Templates For High School Students Applying To College Creative Writing Instructor Wanted Creative Writing Career Advice Application Letter For Fresh Graduate Sample Write My Essay Melbourne Mla Citation No Author Same Title Kite Runner Critical Analysis Text Response Essay Example Ways To Help Kids Do Homework Argumentative Essay Samples Isb Video Essay Help Ut Online Homework Service Mla Research Paper Works Cited Page Example Personal Statement Chemistry Graduate School Words To Make An Essay Sound Better How To Write A History Essay Example Worksheets On Critical Thinking For Kids Essays Written By Zora Neale Hurston Essay Word Count 10 College Essay Writing Tips Examples Good Essay Writing Open University Cover Letter Apartment Address Help On Research Paper Outline Argumentative Essay Graphic Organizer Middle School A 6 Essay Sat Critical Thinking How To Assess Names Of Essay Titles Paperback Writer Bass Tablature Argumentative Essay Structure And Format Stat Homework Help Free Case Study Analysis Of Ikea Advantages Using Case Study 5 Paragraph Essay Rubric Elementary Literature Review Art History Example Essay On Urban And Rural Life In Urdu Creative Writing Research Topics Persuasive Essay Bubble Map Essay Writing Competition 2012 Case Study House 7 Research Paper Ideas 7th Grade Essay About Yourself 10 Years From Now Writing Papers In College Creative Writing For 3rd Grade Essay Topics For Scholarships Samples Cover Letter Marketing And Communications Coursework Mathematics T 2014 Stanford Undergraduate Admission Essay Expository Essay Samples For High School Homework Help Online English Case Study Marketing Planning Process Example Of Term Paper Cover Page Cover Letter Professor Application Writing Report Card Comments Grade 1 Words That Make An Essay Better Writing A College Essay Lesson Plans Need Essay Writers Of Research Papers 7th Edition Essay Defining Happiness Essay About Cyber Bullying Cheap Place To Buy Paper Products Personal Essay For Scholarship Sample California Critical Thinking Skills Test Sample Questions English Essay Competition 2012 Uk Essays On Leadership And Teamwork Cover Letter With No Name To Send To Sales Job Cover Letter No Experience Profile Writing Services Online Dating Mla Citation Self Quiz Essay On Peer Pressure Acts As A Morale Booster Creative Writing For Kids Brooklyn Sample Illustration Essay Write My Assignment For Me Uk Gre Analytical Writing Samples With Scores Essay Writing Custom Cover Letter Drilling Engineer Critical Thinking Questions About Art How Long Should A 9th Grade Essay Be Creative Writing Topics For Middle Schoolers Research Paper With Title Page Format Literature Review For Innovation How Does Critical Thinking Help In Learning Medical School Personal Statement What Not To Do Creative Writing Class Baltimore Technical Report Writing Pdf Free Download Write A Essay Introduction Esl Essay Rules Sample Phd Thesis Proposal Computer Science Question Stems To Provoke Critical Thinking Writing Academic English Answer Key Pdf Letter Writing Topics For High School Students Hart C. (1998) Doing A Literature Review Releasing The Social Science Research Viva Essays Research Paper Topics Literature Review Science Topics How To Write A Case Report For Publication Literature Review Exam Questions Critical Thinking Strategies Definition Creative Writing Programs Boston University Essay Writing Format Critical Thinking And Research Methods Summary Of Julius Caesar For Class 10 Sample Scholarship Essaypersonal Statement What Is Academic Writing Style Creative Writing Groups Victoria Bc Transfer Essays Common Application Supreme Court Case Study 3 Answers Cover Letter Samples Career Builder Persuasive Essay Grading Rubric Middle School Application Letter Format For Teacher Post How Do I Resume My Best Buy Application Homework Help Resources For Parents 4th Grade Hero Essay Top Ranked Creative Writing Graduate Programs Writing Five Paragraph Essay Outline App Mobile Case Study Book Thesis Proposal Writing Template Thesis Proposal Methods Section Help With Persuasive Speech Writing Research Paper Topics British History Standards Of Critical Thinking Nosich Free Essay On High Blood Pressure Annotated Bibliography Apa 6th Example Set Up Of Argumentative Essay Literature Review In Pdf Psychology Major Coursework Research Paper And Citations How To Do Literature Review In Research Paper College Personal Statement How Long Sample Application Letter Of Receptionist College Term Paper Guidelines Literature Review For Knowledge Management Essay Phrases To Conclude Case Study 1 Simple Bus-based Multiprocessor Essay On Corruption 250 Words Editing The Essay Parts One And Two Creative Writing Tips Esl Critical Thinking Moore Parker 10th Edition Answer Key How To Write A Good Thesis Statement For A Rhetorical Analysis Essay People To Write Papers For You Example Term Paper Bel 311 Creative Writing Colleges In Pennsylvania Research Paper Justification 5th Grade Math Essay Iowa University Admission Essay Can I Write A Dissertation In A Month Case Study Creative Thinkers Outline Persuasive Essay Topics For 11th Graders Creative Writing Unit Elementary School Sample Report Writing On Road Accident Sample Essay Editing Practice Review Online Will Writing Services Social Studies Homework Help Creative Writing Minor Bu Sample Cover Letter For Nursing Unit Clerk Argumentative Essay Violent Video Games Cover Letter With No Experience In Field How To Write A Group Paper Online Critical Analysis Kite Runner Literature Review On Diabetes Education 500 Word Paper On Respect How To Add Appendix To Research Paper Essay Writer App Volunteer Application Cover Letter Sample Essay Structure Mla Article Plagiarism Checker Free Writing On Paper Sound Effect Vanderbilt Admission Essay Prompt 5 Paragraph Essay Graphic Organizer Pdf What An Annotated Bibliography Should Look Like In Apa Format Five Paragraph Essay Examples For Kids Critical Thinking Moore 10th Edition Hamlet Important Quotes Act 4 Sample Essay Satire Example Of Application Letter For Fresh Nursing Graduates Objectives Of Thesis Proposal Cover Letter My Current Salary Case Study Houses Peter Gossel Admission Essay Service Application Letter For Mba Admission Is Critical Thinking And Logic The Same Examples Of University Application Essay Critical Thinking Reading And Writing Fifth Edition Case Study Of Education And Training Ib English Exam Essay Questions Writing A Critical Analysis Essay Sample Cover Letter Examples For High School Teachers English Creative Writing Quotes Informative Essay Topics For Elementary Students How To Do Annotated Bibliography Chicago Style Unsolicited Application Letter Wikipedia Case Study 21 Address Literature Review In Research Writing Biology Coursework Osmosis Free Urban Outfitters Case Study 2 Fcat Writing Prompts 4th Grade Expository Compare And Contrast Essay Online Shopping 5 Paragraph Essay On Jim Crow Laws John Dewey View On Critical Thinking Essay Research Examples A Good Personal Statement Opening Cover Letter Nursing Graduate Example Army Critical Thinking And Problem Solving Personal Statement Examples Computing Persuasive Essay Example 7th Grade Extended Essay Guide Music Kids Expository Essay Topics Creative Essay Titles For To Kill A Mockingbird Literature Review On Sources Of Water Term Paper On The Vietnam War How To Write Report Format Example

More Than Market Research - Gain The Information Advantage

Tom H. C. Anderson - Next Gen Market Research™ header image 6

Practical Sentiment Analysis and Lies

April 9th, 2012 · 4 Comments

Q&A with Prof. Bing Liu ahead of the Sentiment Analysis Symposium and Pre Symposium Tutorial

The Sentiment Analysis Symposium in NYC is just a month away (May 8th), so I thought I’d check out who was teaching the pre conference sentiment analysis tutorial this year. For those of us working with text analytics and in the New York area, Seth Grimes Sentiment Symposium has definitely made our annual must attend list. However, what most seem to miss is the half day workshop the day before the event each year. I started attending this component last year when researchers from Amazon.com were teaching it and decided it was definitely well worth half a day in the city to get a more tactical POV on Sentiment from someone who might have a slightly different use case or experience.

This year, data mining expert Bing Liu, a Professor at University of Illinois at Chicago’s Computer Science Department, will be teaching the workshop. Some of his work on text analytics and detecting fraud in online ratings was recently published in the NY Times and as I noticed we were connected on LinkedIn from a previous text analytics event, I called him up for a quick chat to learn a bit more about his work and what I might expect to learn at his pre Symposium workshop. We had an interesting talk and subsequently I sent him a few questions as I thought others would be interested as well.

I plan on being at both the Symposium and Pre Workshop again this year. Anyone else who is interested in attending feel free to use my discount code (OdinText). Do let me know if you’ll be attending so we can meet up, it’s a relatively small and informal group.

Now on to the Q&A…

Tom: Bing, how did you get into text analytics, and sentiment analysis?

Bing: My earlier research interests were in the areas of data mining and machine learning. In about year 2000, I started to get interested in Web mining and machine learning using text data. These two topics led me to the text on the Web. Reviews naturally come to mind because they are focused and well organized, which is great for data mining. I also quickly realized that sentiment analysis was a perfect research problem on its own (I called it opinion mining then due to my data mining background). It had so many applications as every individual and organization needs opinions for decision making. There was also a whole range of challenging research problems that had not been addressed by the natural language processing or the linguistics communities. We started to work on it in 2003 and published our first paper in KDD-2004 (ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). The paper basically defined the framework of feature or aspect-based sentiment analysis and opinion summarization, which is now widely used in the industry and in research.

Tom: False website reviews are an interesting application, and one that I’ve been keeping my eye on. I noticed the New York Times recently covered some of your work in this area. This type of text analytics research seems to be much more difficult than most people think. Can you tell us a bit about this problem from the text analytics perspective, and how it is different from simpler use cases like identifying spam email for instance?

Bing: Indeed, this is a very difficult problem. My group began to work on it in around 2006 or 2007 as we realized this was an important problem and would become more and more important. When we started to do it, we realized it was really hard. The main difficulty lies in the fact that it is very hard, if not impossible, to recognize fake reviews manually as it is fairly easy to craft a fake review and pose it as a genuine one. Email spam detection is a much easier problem because you will immediately recognize a spam mail when you see one. This means that spam and non-spam emails have clear differences, and that it is easy to produce training data for machine learning algorithms in order to produce predictive models and to evaluate the models.

However, for fake reviews, if one writes them very carefully, it is hard to recognize them just by reading the review text. In the extreme case, this is an impossible task logically. For example, one can write a genuine review for a good restaurant and post it as a fake review for a bad restaurant in order to promote the bad restaurant. There is no way to detect this fake review without considering information beyond the review text itself simply because one review cannot be both truthful and fake at the same time.

Tom: What do you see as some of the applications of this type of research?

Bing: Review hosting sites or any general social media sites all want their reviews and user comments to be trustworthy. They are thus interested in fake review detection algorithms. All text analytics systems that use reviews or any opinion data need to worry about this problem too. Social media is here to stay. Its content is also being used more and more in applications.

Something has to be done to ensure the integrity of this valuable source of information before it becomes full of fake opinions, lies and deceptive information. After all, there are strong motivations for businesses and individuals to post fake reviews for profit and fame. It is also easy and cheap to do so. Writing fake reviews has already become a very cheap way of marketing and product promotion.

Tom: Have you found there are certain approaches that work better than others?

Bing: It is still too early to tell. Researchers currently use both linguistic features and atypical behaviors of reviewers to detect fakes. I feel that algorithms that mine atypical behaviors of reviewers and reviews tend to produce more interpretable and trustworthy results. For example, if all 5-star reviews for a hotel were posted only by people from the surrounding area of the hotel, these reviews are clearly suspicious. This is a simple example. More sophisticated fake reviews need more involved modeling and algorithms to detect them.

Tom: It’s been my observation and experience that we as an industry are moving away from linguistic approach to text (sure, some of the basics are useful), but machine learning and statistical approaches seem more powerful. What are your thoughts on this?

Bing: For most tasks, machine learning and statistical approaches are indeed more effective than pure linguistic based approaches. Linguistic approaches are mostly based on heuristic rules and patterns (including grammar information). For those tasks that can be performed based on words, it is very hard for a linguistics based approach to beat a statistical machine learning algorithm simply because the signals used by a machine learning algorithm are far more numerous than the rules or patterns that a human person can design. Plus, machine learning algorithms optimize the performances. However, that being said, in many tasks, linguistics based signals and clues are used as features by machine learning algorithms.

Statistical approaches are not without their limits. Going forward, I believe that both linguistic knowledge and statistical modeling are important. We are working on integrating more linguistic knowledge into statistical modeling.

Tom: It seems to me a lot of folks get a little too caught up in differences between languages. My firm for instance has found it rather easy to add other European languages to our approach, and of course machine translation is always a possibility. What are your thoughts on this?

Yes, I agree. Although every language is different, different languages are still similar as they all consist of words and grammar. European languages have even more similarities due to their common roots. A learning algorithm can capture many types of grammar regularities from any language if there is a sufficient amount of training data. For those tasks that need only word or lexical information, the same algorithm can be used for any language with almost no modification because an algorithm treats words are symbols. In that sense, it does not matter what language it is.

Tom: What will you be covering during the tutorial at the sentiment symposium?

Bing: Sentiment analysis has been studied extensively for the past decade. A huge number of research papers have been published on it (probably more than 1000). It is impossible to cover them all. Therefore, I will try to cover the main threads of research that also contain aspects which can be of immediate use in practice.

In the tutorial, I will start with a short motivation and then go on to define the problem. This will provide an abstraction or statement of the problem, which will naturally introduce the key sub-problems. I will then discuss the current state-of-the-art approaches to solving these problems. Since this is a practical sentiment analysis tutorial, I will also describe how to build a practical sentiment analysis system based on my previous experience in building one. In the final part of the tutorial, I will introduce the problem of fake review detection.

A big thanks to Bing for our talk and the subsequent Q&A. Looking forward to meeting up at the Symposium.

@TomHCAnderson
@OdinText

[For those interested in more info about the sentiment tutorial a syllabus and outline is available here]

[Post to Twitter] 

Tags: Academia · Conferences · Datamining · Odin Text · OdinText · Sentiment Analysis · Sentiment Analysis Symposium · Text Analytics · Text Analytics Summit · Text Mining Guru · seth grimes · text mining · tomhcanderson

4 responses so far ↓

  • 1 Peter Szekeres // Apr 11, 2012 at 7:58 am

    Great interview. I really agree with Prof. Bing Liu. I think that the most effective sentiment analysis methods those which are using knowledge lexicons, grammar rules and statictical methods and assesments both. By combining these I built a quite good sentiment analysis system on Hungarian webpages (accuracy: 80%).
    I think one important thing wasn’t mentioned above: handling irony. Maybe it can be similar to recognizing fake reviews…

  • 2 chris west // Apr 19, 2012 at 6:47 am

    i’m someone that’s coming into Text Analytics from the Marketing World. Can anyone explain (simply) how ’statistical machine learning algorithm’ works: what do you give it as inputs? Does it look for wide variances from ‘typical’ or ‘mean’ to spot possible fakes?
    Any help appreciated
    Chris

  • 3 Tom H C Anderson // Apr 19, 2012 at 8:30 am

    Chris, good question. Yes most typically that approach has to do with the computer ‘learning’ how humans do it. So target variable would be how humans have coded it, be it sentiment (Pos, Neg etc.) or in this case I guess dishonest or honest.

  • 4 Fake Reviews a Growing and Tenacious Problem in Social Media : Beyond Search // Apr 20, 2012 at 12:07 am

    [...] Analysis Symposium in New York City early next month. He has titled his interview, “Practical Sentiment Analysis and Lies.” [...]

Leave a Comment