Text Analytics: From sending a short message to our loved ones to writing a formal email to a colleague, we use text in various forms. It is the most authentic form of information exchange that has enabled mankind to evolve faster and for the better.
Each text that we use has a meaning and a purpose of its own. Emails. Short text messages. Tweets. Facebook statuses. Blogs. Live chat excerpts. Customer support notes. Survey results. The list of textual data is endless.
All the forms of textual data that we listed above are unstructured data. Apart from their textual semblance, from a data perspective, they are all different from each other.
For example, the data structure of a Twitter tweet data would be entirely different compared to a Facebook status post.
What if all these texts that a specific cohort of users send and receive can be placed under a microscope?
That would enable us to pick up the nuances of communication, unspoken messages or even common behavioral traits. Text analytics is what enables all of this.
In simple terms, text analytics is all about converting unstructured data into meaningful data from which advanced insights can be picked up.
Text analytics is the process of picking up insights from clusters of textual data.
Demystifying Text Analytics
Imagine your business to have got 100 customer reviews. This collection of reviews is a cluster of textual data. They would have different words, dates and even varying tones of customers — from good to bad or even worse.
Text analytics can sweep through these 100 reviews and figure out whether a review is praising the product/service or bashing it for any reason. But isn’t that something that a person can do manually?
In the olden days that used to be the case. But, in the past decade, as the volume of unstructured data exploded, the manual approach to text analytics was proven to be ineffective and unproductive.
The amount of labor required to carry out text analytics made the manual approach inefficient.
Walmart, the American retailer handles about a million transactions every day (Source).
Imagine what a mammoth task it would be to analyze these transactions on a word-to-word basis.
Thanks to the arithmetic progression of computing power in agreement with Moore’s Law, text analytics is now largely automated.
Advanced data-crunching systems use a series of tools to facilitate text analytics.
Some of the processes include lexical analysis, categorization, clustering, pattern recognition, tagging, annotation, information extraction, link and association analysis, visualization and predictive analytics.
Apart from these ground-level processes, there are also several other approaches to text analytics.
The many approaches to text analytics
Each approach to text analytics brings out diverse results of text analytics. As you are reading this article, there are newer approaches that are being innovated and brought to the world.
Here are five top approaches that are widely used for text analytics.
Word spotting, also known as keyword spotting is a technique used to spot specific words that represent the entire meaning of the sentence. The assumption is that if a specific word is present, the entire sentence is about that particular word.
Let’s see how word spotting can be used to categorize the responses to an NPS survey.
As you can see from the image above, every review that spots the word ‘price’ the system assumes that the sentence is about pricing.
Similarly, the mention of the word ‘easy’ leads to the assumption that the sentence is about user-friendliness. When combined with the NPS score of 1 to 10, it gives a fair idea of whether the customer sentiment is positive or negative.
Manual rules are more like word spotting but work at an advanced level where more words in complex scenarios are categorized. Manual rules are useful for businesses that want to customize the rules on how various words would be treated by the text analytics program.
For example, the word ‘stock’ has an entirely different meaning in retail and financial industry parlance. Manual rules make it possible for businesses to set clear rules of how each word relating to their industry should be understood and analyzed by the system.
This is perhaps the approach to text analytics that is most prevalent today. It seeks the support of machine learning which picks up models from existing datasets, matches them to new datasets to create forecasts or suggestions.
In-text categorization, the system is fed a pre-built set of text examples and their relevant categories. The machine learning algorithm learns how each text is categorized and creates rules for itself.
When new text is presented, it applies these rules to categorize the new text into further categories. This supervised learning-based approach to text categorization saves time and effort to set and implement new rules.
The machine learning algorithm itself creates the classifiers for categorizing new data.
Topic modeling is an unsupervised approach to text analytics. In this approach, the algorithm is fed raw textual data as it is.
The algorithm itself picks up clusters of topics from the dataset to arrive at predictions.
The image below shows how topic modeling works to create models for topics from raw textual data.
To cite a real-world example, an input of customer responses to an NPS survey can result in a topic modeling that might look like:
X% easy-to-use, good service, friendly staff
X% bad, poor, expensive, not useful
While other text analytics approaches focus on clustering the text or topics, thematic analysis focuses on creating themes from the dataset provided to it.
Like topic modeling, it also takes the unsupervised training route. This makes it ideal for businesses that have large datasets of a specific nature.
For example, the customer ratings and reviews of a pizza delivery service. Thematic analysis can sift through customer data to bring up themes around which most customer feedback revolves.
Some examples include friendliness of staff, recipes of pizzas, service quality, etc.
Thematic analysis also can have several sub-themes under each of these main themes. However, to achieve that level of data extraction, the text analytics algorithm implemented should be advanced as well.
The importance of text analysis
Now all this dissection of text analytics begs the question of ‘why text analytics is important?’ What benefit can it accrue to businesses that spend the pain and resources to implement it?
Turns out there are three major gains that businesses of all nature can reap through reap analytics. They are:
- Understanding the tone of textual content
- Predict emerging topics of discussion
- Quick translation of multilingual customer feedback
Understanding the tone of textual content
Tweets with less than 140 characters can wreak more damage to a brand than a full-fledged press release. A one sentence-long NPS survey response can hint whether the customer is happy or not.
A page-long customer review on a third-party marketplace can tell stories about a brand’s customer service efficacy.
Text analytics can sift through all this structured and unstructured data to figure out whether the tone used in the text is positive or negative.
How does that magic happen? Sentiment analysis makes it happen.
Sentiment analysis is a systematic way of extracting subjective information using a variety of technologies including Natural Language Processing (NLP) and text analytics.
Using sentiment analysis, it is possible to determine whether the user’s perspective about a product or service is positive, negative or neutral.
Predict emerging topics of discussion
Text analytics empowers businesses with ‘social listening’ capabilities. It allows businesses to tune into structured and unstructured data across emails, text messages, emails, and customer reviews to narrow down on positive and negative topics.
For instance, text analytics can help determine the overall topic of discussion in response to a specific change in business offering like features, pricing, integrations, etc.
Such an overarching approach to identifying topics of interest would enable a business to identify positive and negative topics of high-impact that can be improved immediately.
Translation of multilingual customer feedback
For any globe-spanning business, there would be a diverse customer base that uses multiple languages for communication.
Translating every byte of communication of interpretation and analysis is a tough job with manual processes.
Text analytics helps cut through this linguistic barriers. It can translate unstructured text from all channels that customers are using.
Proper training models can also enable the algorithm to decipher urban slang-heavy or acronym-heavy internet speech.
In other words, text analysis can help you understand customer praise or ranting in any medium in any language in plain language that your business can understand.
Real-world use cases of text analytics
Given the various approaches and benefits of text analytics, what are the real-world use cases to which a business can apply text analytics? Here are some of the use cases.
There are three core traits that customer service demands — swift responses, empathy, and accurate solutions. Now if your customer support function is used to getting a never-ending queue of customer queries,
it could be a problem. It is not possible to read through all queries and provide swift responses. Also, a majority of the queries could be basic that can be easily resolved with an automated responder system.
Text analytics with its analytic capabilities can cluster customer queries of similar nature into categories. This would make it easier for customer support agents to attend them with a specific approach.
Even better, the business can figure out perennial problems that can be resolved with a permanent fix.
87% of shoppers now begin product searches online (Retail Dive). That is a huge heap of text that is being used there. Right from the preliminary text search advanced search regarding a specific brand or label, users use a variety of text during the research phase.
All this text can be mined to create effective product placement and online advertising campaigns. This is the same strategy that has made Google AdWords into an advertising behemoth.
Text analytics helps businesses go beyond search queries and pick up customer behavior patterns from social media and other digital mediums.
Every recruiter who has an ambitious target for hiring will testify how difficult it is to sift through resumes and online candidate profiles on career profiles to find the right candidate. It is the most time-consuming part of recruiting.
Text analytics can make a difference here. It can help recruiters to look for specific text phrases in the candidate’s profile that can help them filter potential individuals who could be personally interviewed.
How text analytics can add value to your NPS
Since the days of the Industrial Revolution, businesses have been trying hard to find out how customers feel about their products or surveys. The search reached some kind of conclusion in the last decade when the Net Promoter Score® (NPS®) was introduced.
NPS® aims to collect customer feedback in a concise form with a single question. “How likely are you to recommend this product/brand/service to your friends or colleagues?” depending on the response rate, the business can decide how well they are faring in serving customers.
The unique trait about NPS is that, although it is a measure of customer satisfaction, it is not an absolute indication of business performance.
Rather, it indicates areas where the business can improve its offerings to sustain loyal customer, turn delusional customers into loyal ones and to convert detractors into passive customers.
NPS® surveys receive their responses in textual form as well. In addition to the standard question, most surveys are also accompanied by open-ended questions that allow the customer to write at length their feedback about the product or service.
Analyze all survey responses without sampling
Imagine running an NPS survey and getting thousands of responses. It is almost impossible and also unproductive to go through each response manually. Sampling of responses is also not recommended since responses that portray serious flaws in the product might be missed out.
In such a scenario, it is text analytics that can come to your aid. It helps analyze all survey responses without the need for sampling. The thorough analysis of the responses help in arriving at a near accurate measure of promoters, passives, and detractors.
Categorization of survey responses
The purpose of Net Promoter Score® is to categorize users based on their loyalty into promoters, passives, and detractors. From a business perspective, the user responses must be also further segmented.
That calls for identifying a common pattern of responses — a common theme that every customer talks about is happy or unhappy about.
With text analytics, it is possible to quickly categorize the widely scattered text responses of users into common topics or themes.
For example, what do promoters feel about pricing and customer service? The same can be compared with the responses of passives to identify areas of improvement.
Data to craft a marketing story
Take a look at the world’s top brands. Apple. Airbnb. Amazon. Alibaba. Facebook. Apart from their business offerings, they are also known for their story of how the business was born out of an idea. And, countless stories revolve around their business data.
For example, Apple is a company which is hailed for its user data privacy. Airbnb is known for its wonderful experiences at unique host locations.
For your business, there could be a similar USP that could be indicated with data from NPS® survey responses. Text analytics help populate data of similar nature to craft a marketing story that can augment your brand image.
In a nutshell
Text analytics has been around for a long time. Ever since the days of cave drawings, mankind has been trying to decipher the hidden meaning in texts and symbols.
In today’s world, text analytics has leapfrogged to a new dimension. It is used to dig out the hidden meaning in text messages and content that users create in their daily life.
That search for meaning also enables businesses to understand their customers better and serve them better. To sum it up, text analytics is a must-have tool for businesses that want to read in between the lines of what their customers are writing about them.