Human Centered AI

AI, particularly deep learning, is making great advances in NLP and beyond. These tools are redefining business in many areas, including our focus on the Human Resources (HR) domain. That said, deploying AI models and applications in areas with direct human impact (like HR),  requires the participation of various stakeholders, from researchers and machine-learning engineers, to domain experts, product managers, and business owners. These groups are essential for exploring data, training models, monitoring predictions, and approving outcomes.

Our vision in the HITL space is to devise and build novel tools, workflows, and systems that incorporate input from a variety of stakeholders in the larger ML cycle. We strive for a holistic view with a common HITL backend that captures and incorporates various forms of human input, while at the same time creating specific solutions for stakeholders. Our recent work includes an exploratory programmable annotation framework geared toward NLP researchers, tools to support the exploration of text and knowledge graphs for subject-matter experts, and tools for tracking and planning model development for product managers.

Related Projects:

Characterizing Human-Centered Information Extraction

In particular, we proposed feature- and system-specific guidelines for designing human-centered data systems. The feature-specific guidelines, inspired by cognitive engineering principles for enhancing human-computer performance, recommend automating the unwanted workload of humans.


We present MegAnno, a novel exploratory annotation framework designed for NLP researchers and practitioners. Unlike existing labeling tools that focus on data labeling only, our framework aims to support a broader, iterative ML workflow, including data exploration and model development.