Dan Zhang, Yoshihiko Suhara, Jinfeng Li, Madelon Hulsebos, Çağatay Demiralp, Wang-Chiew Tan
Detecting the semantic types of data columns in relational
tables is important for various data preparation and information retrieval tasks such as data cleaning, schema matching, data discovery, and semantic search. However, existing
detection approaches either perform poorly with dirty data,
support only a limited number of semantic types, fail to
incorporate the table context of columns or rely on large
sample sizes for training data. We introduce Sato, a hybrid machine learning model to automatically detect the
semantic types of columns in tables, exploiting the signals
from the table context as well as the column values. Sato
combines a deep learning model trained on a large-scale table corpus with topic modeling and structured prediction
to achieve support-weighted and macro average F1 scores
of 0.925 and 0.735, respectively, exceeding the state-of-theart performance by a significant margin. We extensively
analyze the overall and per-type performance of Sato, discussing how individual modeling components, as well as feature categories, contribute to its performance.