Unless you’ve been living under a rock for the past 10 years, chances are you’ve used or interacted with a chatbot. This software conducts conversations and assists people via auditory or textual methods. But when it comes to supporting various types of businesses, such as hotels, chatbots can be limited in capabilities. To address this, we built ConciergeBot, an industrial-strength Q&A chatbot for the hospitality sector.
The Issue With Intent
“Hey Siri, read my new messages.”
“Alexa, how is the weather today?”
“Ok Google, play some dance music!”
Chatbots like the ones mentioned above can help you because of a process called intent classification. As its name alludes to, intent classification lets a chatbot analyze what you’ve said and understand your intent.
Most chatbots have predefined sets of classes (e.g., playing music, checking the weather, etc.). With recent developments in natural language processing (NLP) and machine learning such as pre-trained language models and reinforcement learning, chatbots are able to classify your input text into a class.
However, unique technical challenges arise when designing a chatbot to support different types of businesses. Take hotels, for instance; each one differs in the particular amenities and services they provide. Consequently, queries can range drastically in scope. Here are a few examples:
Is the wine complimentary?
Does the bowling alley open every night?
Is the stargazing event still happening this evening?
From these few questions alone, it’s clear that having a predefined set of intents to cover all possibilities is unviable. Theoretically, you could try to define new intents to answer new questions. But you’d need to start from scratch for every hotel that wishes to use a chatbot. And the process would never end. Simply put, this doesn’t scale feasibly.
Introducing ConciergeBot
We created ConciergeBot to assist hotel staff in answering the myriad of questions that guests ask. It reduces employee workload by automatically answering common questions like these:
What time is check-in?
Is the pool open tonight?
Is parking free?
Like other chatbots, ConciergeBot must classify the guests’ intents in order to provide relevant information. Originally, we were going to rely on the same arsenal of classification techniques that we described above to achieve this. But as we previously mentioned, predefined sets of intents only go so far.
More specifically, predefined sets of intents can cover roughly 80% of the queries that concierge staff encounter. The remaining 20% of questions usually revolve around the distinctive amenities and services each hotel provides.
How ConciergeBot Handles Unique Queries for Hotels
To solve this issue, we built ConciergeBot, which assumes there are no predefined sets of intents. Instead, any piece of information available for the hotel implicitly defines an intent (assuming the intent is a request for information).
These intents vary from one hotel to another. And they can change over time. ConciergeBot is not only robust to these changes but can deal with previously unseen intents as well. We accomplished this by designing ConciergeBot with two key observations in mind:
- By leveraging simple phrase matching techniques, a substantial number of questions become easy to answer. For example, if a guest inquires about “stargazing,” and there’s only one piece of information about this topic, then it must be the only possible answer.
- State-of-the-art NLP techniques have become remarkably proficient at identifying whether two questions are likely to be paraphrases or not. For instance, pre-trained language models can easily detect that the question “Can we go to the amateur astronomy event tonight?” is more similar to “Are you holding the stargazing event tonight?” than “Is the wine complimentary?”.
In a nutshell, ConciergeBot operates based on these principles.
The Components Behind ConciergeBot
For any input question, ConciergeBot’s system uses two semantic parsing techniques: LexParse, a rule-based program for analyzing machine-readable dictionaries; and semantic role labeling (SRL), a process that assigns labels to words or phrases to indicate their semantic role in a sentence. Together, LexParse and SRL let ConciergeBot align and match key parts of a question against the available hotel information and predict the most likely answer.
Concurrently, ConciergeBot relies on a similarity search component called QSim. This measures how much the input question has in common with prior questions for which the correct intent was identified. The assumption is that the question should have the same intent as similar queries.
Next, we must reconcile the signals from the parsing techniques (LexParse and SRL) and the similarity search (QSim). To do this, ConciergeBot uses a component which we call the MetaLearner. The MetaLearner decides which signal to trust more. If the semantic parsing results match perfectly with available information on the hotel, the MetaLearner chooses the signal from LexParse and SRL (Principle #1). Conversely, if the input question is extremely similar to previous questions, then it chooses the QSim component (Principle #2).
Evaluating ConciergeBot's Performance
The primary advantage of ConciergeBot’s setup is that the MetaLearner does not take the number of possible intents into account. Instead, it simply learns when to rely on each system to obtain the optimal results. This provides unprecedented versatility.
Even if ConciergeBot is trained with the available data from one hotel, it can be seamlessly deployed in other hotels without any need for specific tuning. As it adapts over time, ConciergeBot will improve its performance. ConciergeBot can also successfully bootstrap using very little data. With as few as 1,300 questions for training purposes, this system still managed to achieve a 78% precision rate and a 71% recall rate.
ConciergeBot proves that it’s possible to develop an adaptable, powerful chatbot with a limited amount of training data. This opens up various opportunities for potential applications beyond hotels as well as several directions for future work. The next steps in making chatbots more versatile include going beyond information-seeking questions, addressing follow-up queries, and handling multi-message threads (which requires context-based reasoning).
For now, ConciergeBot is already hard at work at many hotels. Perhaps one day, it will be able to assist you during your travels.
Are you interested in learning more about ConciergeBot works? Check out our research paper which was presented in the Conversational Access to Data (CAST) workshop at VLDB 2019. Do you have any questions about ConciergeBot? Feel free to contact us today!
Written by Behzad Golshan and Megagon Labs
References
Behzad Golshan, George Mihaila, Chen Chen, Jonathan Engel, Alon Halevy, Yoshihiko Suhara, Wang-Chiew Tan, Michael Matuschek (TrustYou), “Building a Hotel Concierge Bot: an industrial case study”, CAST 2019.