Unlocking Real Value from User Reviews With Subjective Data and Experiential Search — Part 1

User reviews now run the world’s business. Whether you’re selling a product, hiring employees, or managing a hotel, reviews can make or break your business. And it’s not just the quality of reviews that determines your fate; it’s how well the experience you offer aligns with expectations. Misalignment between these two factors can lead to disastrous results.

Sadly, the status quo doesn’t offer any easy fix. Today’s review platforms and e-commerce search engines only support querying by objective attributes. In the case of restaurants, that means you can find out things like price, location, or types of food. But confirming your dining experience will have a vibrant ambiance or romantic sunset view? Not so easy. You’ll likely have to read through dozens of user reviews — a monotonous, time-intensive task that’s not even guaranteed to pay off.

Finding services or products that satisfy unique, specific needs shouldn’t be so difficult for users. To solve this dilemma, we built OpineDB, a subjective data system that supports answering queries with experiential search.

Welcome to the first entry of our series on how subjective data and experiential search can unlock more value from user reviews. In this blog, we’ll examine the problematic results that we’ve all encountered by only searching with objective attributes. We’ll also take a look at the demand for experiential search and why it hasn’t been done before. Let’s get started.


The Need to Align Expectations With Experience

After a long day of travel, you’ve finally arrived at your hotel. It’s rated 4.5 on Yelp, so you can’t wait to see what’s inside. But as you walk into your hotel room, you quickly discover that it could use some more lighting. Also, you can hear constant noise coming through the walls. What gives? None of the top ten user reviews of this place mentioned these problems.

Confused, you log into Yelp to see if anyone else encountered the same issues. Lo and behold, a few reviews buried on page 12 mention dirty rooms and a noisy bar next door. This tragedy could have been avoided entirely if you had only spent a few more hours poring over these reviews!

Unfortunately, misalignments like these occur all too often. And, as we discussed, addressing them isn’t so easy. After all, who has time to read 12 pages of user reviews? Why should users have to do this for every product and service to ensure their experience is satisfactory? By only supporting queries involving objective attributes, current search engines and their database systems leave a lot of potential for improvement on the table.

The Case for Subjective Data and Experiential Search

The world is teeming with subjective attributes, and it turns out that users want to apply these to find the specific experiences they desire. We conducted a study on Amazon Mechanical Turk to determine the criteria that users consider while searching for certain types of entities.

We provided each participant in the study with questions such as “Other than cost, what are 7 separate criteria you’d likely value the most when deciding on a hotel?” After acquiring numerous responses for a variety of domains, we evaluated whether each criterion was subjective or objective. For example, Wi-Fi frequently shows up in hotel search criteria, but we interpret it as objective (Is there Wi-Fi?) rather than subjective (Is the Wi-Fi fast and reliable?).

The results of this experiment were astounding — almost 70% of participants used subjective queries. This ratio held true whether they were looking for careers, colleges, restaurants, or vacations.

But nearly all experiential technical and subjective data is relegated to text feedback, like what you’d find on Yelp or Glassdoor. In this case, users have no choice but to invest time in reading mountains of reviews to find what they want. It shouldn’t be this way. So why is it?


Why Hasn't Experiential Search Been Done Before?

Incorporating experiential search into online platforms could benefit every industry. But a few substantial hurdles stand in the way of making this type of search a reality.

First, the user reviews must be aggregated in a way that they can be queried efficiently and effectively. Second, an experiential search engine must be able to systematically answer queries, regardless of their complexity. Last but not least, the engine must be able to elegantly handle searches not using terms that fit neatly into the subjective database schema. Each of these issues represents a tremendous obstacle; solving just one of them would be significant progress towards experiential search.

OpineDB addresses all of them by modeling subjective data and processing queries in the user’s own words. By doing so, this subjective data system enables a new set of applications where users can search by their personal preferences — and find exactly what they’re looking for in an experience. With OpineDB, you can gain unprecedented insight into subtle yet vital factors, like if that car you’re thinking about buying has comfortable seats, or if the Wi-Fi at your next hotel is as slow as a snail.

In the second and final chapter of this series, we’ll illustrate how OpineDB seamlessly solves the three biggest challenges of experiential search. We’ll also explore how OpineDB compares to information retrieval-based search engines and attribute-based query engines with an experiment using real subjective data from Booking.com and Yelp. Stay tuned!

Are you interested in learning more about OpineDB? Contact us today!

Written by Yuliang Li and Megagon Labs



Yuliang Li, Aaron Feng, Jinfeng Li, Saran Mumick, Alon Halevy, Vivian Li, Wang-Chiew Tan, “Subjective databases,” VLDB, July 2019.


More Blog Posts: