Human-Centered AI

AI, particularly deep learning, is making great advances in NLP and beyond. These tools are redefining business in many areas, including our focus on the Human Resources (HR) domain. That said, deploying AI models and applications in areas with direct human impact (like HR),  requires the participation of various stakeholders, from researchers and machine-learning engineers, to domain experts, product managers, and business owners. These groups are essential for exploring data, training models, monitoring predictions, and approving outcomes.

Our vision in the HITL space is to devise and build novel tools, workflows, and systems that incorporate input from a variety of stakeholders in the larger ML cycle. We strive for a holistic view with a common HITL backend that captures and incorporates various forms of human input, while at the same time creating specific solutions for stakeholders. Our recent work includes an exploratory programmable annotation framework geared toward NLP researchers, tools to support the exploration of text and knowledge graphs for subject-matter experts, and tools for tracking and planning model development for product managers.

Related Projects:


We present MEGAnno, an open-source data annotation tool that puts the data scientist first, enabling you to bootstrap annotation tasks and manage the continual evolution of annotations through the machine learning lifecycle.

Magneton: Transparent and Customizable Widget Framework for Jupyter Notebooks

We developed MAGNETON, a framework for authoring interactive widgets within computational notebooks that enables transparent, reusable, and customizable data science workflows. The framework enhances widgets to support fine-grained interaction history management, reusable states, and user-defined customizations.


For data-centric NLP, we present Weedle: Widget-Enabled Exploratory Data analysis for NLP Experts. Weedle offers global and local exploration of text data via built-in and customizable transformation operations.

Characterizing Human-Centered Information Extraction

In particular, we proposed feature- and system-specific guidelines for designing human-centered data systems. The feature-specific guidelines, inspired by cognitive engineering principles for enhancing human-computer performance, recommend automating the unwanted workload of humans.