Semantic tagging can elucidate unparalleled data insights for businesses both big and small. In turn, this information can catalyze solutions to longstanding problems in an array of industries. But the use cases and benefits of semantic tagging are not well understood. In this blog post, we’ll delve into some of the most relevant applications of semantic tagging today. We’ll also explore the common challenges organizations face when setting up a pipeline and choosing a semantic tagging classification model.
What's Semantic Tagging?
Before we define semantic tagging, we must first understand what text mining is. Text mining is a process that leverages artificial intelligence (AI) and natural language processing (NLP) technology to transform unstructured text into normalized, structured data that’s ready for analysis. After this text is organized, semantic tagging comes into the picture.
Semantic tagging is the process of connecting an element from an ontology with a database or document, such as a website or computer file. This form of text annotation allows us to efficiently describe key components of a data structure to facilitate better retrieval of them later on or understand how they relate with other resources in the same ontology. More simply put, semantic tagging takes a piece of text and a predefined tag as inputs and then predicts or outputs whether this text conveys the tag’s semantics.
Since the internet is filled with unstructured text, text mining and semantic tagging are invaluable in making sense of all this information. Semantic tagging is a core driver of several commercial applications from leading e-commerce companies such as Amazon and eBay. One of the best-known use cases of semantic tagging is sentiment analysis, which aggregates opinions from customer-generated texts (e.g., reviews) by indicating whether each one carries a positive or negative sentiment. These sentiment tags can then be exploited in downstream applications to help determine appropriate actions.
Examples of Semantic Tagging Applications
Sentiment analysis only scratches the surface of what semantic tagging is capable of accomplishing. Let’s examine some of the most pertinent use cases for semantic tagging today.
1. Suggestion Mining
Suggestion mining is one of the most popular applications of semantic tagging. A suggestion is a short yet practical piece of text that carries an actionable recommendation. Experience-sharing websites such as TripAdvisor and Yelp contain multitudes of suggestions in their user-generated text. You can see some examples in Figure 2 below:
While suggestions contain ample information that can help future customers improve their experiences, they are often buried in lengthy reviews that are laborious and time-consuming to read. Besides this, long reviews are usually not friendly to read on mobile devices due to their limited screen real estate. These issues often cause users to feel overwhelmed and miss key points.
To address this problem, Yelp [1] now encourages users to write tips other than full reviews to identify and suggest the most popular entrees of a restaurant near the top of its page on the platform’s mobile app interface. Similarly, Yahoo Search noticed that a substantial amount of searches contained how-to intent, such as “how to zest a lime without zester.” To streamline an adequate solution for users fielding these queries, the company mines, tags, and displays top-rated suggestions from its Yahoo Answers database at the top of search pages containing how-to intent.
By reducing the amount of information a user must process while still retaining the most prominent insights, suggestion mining allows users to quickly find actionable answers.
2. Product Description Generation
E-commerce websites such as Amazon and eBay collect innumerable customer reviews every day. Perhaps the most helpful part of these user-generated texts is the product description. Product descriptions from customer reviews often contain vital information that’s missing from their official counterparts.
A study of eBay [2] found that official descriptions for both new and old products were lacking in detail. The study authors used semantic tagging to identify customer-generated product descriptions and then evaluated if they provided more useful information to address this problem. Study participants found that these descriptions significantly simplified the purchase decisions for potential customers because they yielded informative, objective insights. Figure 3 below shows some of the potential insights that a tagged product description can provide:
By aggregating user review content to form additional product descriptions, e-commerce platforms can shine a light on complementary, supplemental, and even contradictory information about products. Because these product descriptions come directly from other customers, consumers find them attractive for many reasons:
- They are written by independent users with no personal connections to merchants. .
- They mention crucial factors concerning actual user experience.
- They introduce exceptions to avoid unnecessary orders when the product is not applicable.
For example, a tagged product description for a camera can discuss an actual user experience after they’ve used the camera for several weeks. It may also describe after-sale services, such as customer support or return policy. In turn, this helps potential buyers make more informed purchasing decisions.
3. Humor
Although informative, reading online user reviews can be tedious and unpleasant. But when you add humor into the mix, it can bring readers happiness and attract attention from more of them. Yelp [3] noticed this and started offering users the ability to label whether a review is funny or not. Figure 4 below shows two examples of such reviews:
Because of humor’s ability to alleviate the task of digesting user reviews, it has garnered the NLP research community’s attention. Semantic tagging of humorous and useful customer reviews can not only simplify and expedite the task of finding insights about products for potential customers, but it also makes the process more enjoyable by bringing some levity to the table.
4. Argument
Compared with sentimental comments like “the room is uncomfortable,” arguments provide factual evidence such as “the room has a smell of mold” to support or oppose an opinion. Since arguments are generally different opinions towards the same proposition, they often contain information from various perspectives that can be useful for explanations.
Figure 5 above displays argument and non-argument sentences for the proposition “nuclear energy is good.” Even with just a glance, it’s easy to see that the argument sentence provides more explanatory information about the proposition than the second statement, which is a rhetorical question on the effectiveness of green energy.
Argument mining, coupled with semantic tagging, has attracted considerable research attention because they can effectively depict the pros and cons of a targeted service, product, or even a controversial topic. In the domain of e-commerce, this can provide a potential customer with knowledge from opposing perspectives. In turn, this gives users a more realistic and objective expectation of the experience they will receive.
5. Spoilers
Nobody likes spoilers. But they commonly exist in the reviews of media works such as TV series or books. It isn’t the authors’ fault — sometimes, detailing specific plot points is the only way to support their content evaluation. Nevertheless, these types of spoilers can ruin the expectation and enjoyment of future audiences.
Spoiler alerts have become necessary to warn readers before they browse reviews for a work of media. For example, the online movie database IMDB [4] requires that reviewers add an alert message if they’re going to include spoilers such as those displayed in Figure 6 below.
Since manual spoiler alerts aren’t always put in place properly, automatic spoiler detection has become desirable. As of now, there are few open-source datasets for spoiler detection. But they’ve already received recognition in the news because they’ve elucidated more about the nature of spoilers in reviews. For example, the authors of one such dataset found that spoilers tend to occur later on in reviews.
Semantic tagging has great potential to accelerate automatic spoiler detection. We do not doubt that this process will play an integral role in enabling and improving this capability across media review sites such as IMDB and Rotten Tomatoes in the near future.
Common Challenges of Semantic Tagging
Despite the many benefits that semantic tagging offers, setting up an application pipeline for production data is not easy. Here are three common challenges:
Your pipeline will require extensive amounts of high-quality label data to be prepared for model training.
You’ll need to develop a series of computational operators to process raw data into numeric representations. Possible operators include data preprocessing, deduplication, and feature engineering. They can take an enormous amount of time and effort to create.
Pipeline performance strongly depends on selecting the most suitable machine learning model. Suboptimal models may tag undesired content such as fake or irrelevant comments or exclude legitimate sentiments.
To elucidate if deep learning models perform best for semantic tagging tasks, we conducted a systematic study that compares three prevalent ones (e.g., CNN, LSTM, BERT) against “simple models” (e.g., LR, SVM) over datasets with varying characteristics. Our paper, “Deep or Simple Models for Semantic Tagging? It Depends on Your Data”, details the results of this study and summarizes best practices for developing semantic tagging data pipelines. The leading computer science conference PVLDB recently accepted it. We have also made our collection of datasets and models publicly available. You can access them here.
While it’s largely believed that deep learning models are better for semantic tagging, there has been no extensive analysis of this assumption before our study. Consequently, practitioners often have to train different models to identify the one that works best for their specific semantic tagging task. By examining 21 real-world datasets, our study is not only the most comprehensive one available on this topic but also reveals that dataset characteristics are key to achieving better tagging quality with deep learning models over simple models.
Given the raw complexity of real-world datasets, it’s clear that choosing a suitable tagging model for a specific dataset rather than sticking with deep learning models is the optimal approach to performing tagging tasks. By considering scale, label ratio, and cleanliness, our study, especially the visualized performance heat map, can inform practitioners on how to choose a suitable tagging model for their dataset. In turn, this should help make semantic tagging less expensive and more efficient for researchers, engineers, and product managers worldwide. We hope this work advances the development and productization of semantic tagging technologies.
Interested in learning more about semantic tagging? Check out our research paper! Do you have any questions about how it works? Contact us today!
References
[1] Yelp. What are Tips? https://www.yelp-support.com/article/What-are-tips?l=en_US. 2020.
[2] S. Novgorodov, G. Elad, I. Guy, and K. Radinsky. Generating product descriptions from user reviews. In WWW, pages 1354–1364, 2019.
[3] Yelp Dataset Challenge. https://www.yelp.com/dataset/documentation/main. 2020.
[4] IMDB. User Review Guidelines. https://help.imdb.com/article/contribution/contribution-information/user-review-guidelines/GABTWSNLDNFLPRRH#. 2020.
Written by Jinfeng Li and Megagon Labs