AI, particularly deep learning, is making great advances in NLP and beyond. These tools are redefining business in many areas, including our focus on the Human Resources (HR) domain. That said, deploying AI models and applications in areas with direct human impact (like HR), requires the participation of various stakeholders, from researchers and machine-learning engineers, to domain experts, product managers, and business owners. These groups are essential for exploring data, training models, monitoring predictions, and approving outcomes.
Our vision in the HITL space is to devise and build novel tools, workflows, and systems that incorporate input from a variety of stakeholders in the larger ML cycle. We strive for a holistic view with a common HITL backend that captures and incorporates various forms of human input, while at the same time creating specific solutions for stakeholders. Our recent work includes an exploratory programmable annotation framework geared toward NLP researchers, tools to support the exploration of text and knowledge graphs for subject-matter experts, and tools for tracking and planning model development for product managers.