The CHI 2022 conference took place last month in the vibrant city of New Orleans, Louisiana. It was a hybrid event — almost 50% of the participants joined in person, and the rest joined virtually. With my research interests spanning visualization, NLP, and human-AI interaction, I was looking forward to my first in-person visit to CHI. And it didn’t disappoint. I came back from the conference energized after seeing presentations of high-quality research work, coming across thought-provoking ideas, and syncing up with outstanding researchers.
In this blog post, I summarize my experience attending CHI, provide an overview of the keynotes and awards, and summarize several interesting papers on human-AI interaction, mixed-initiative system design, and visualization. Human-centered AI is a key research area of Megagon Labs where we explore challenges related to scalability, usability, and explainability in diverse projects such as data integration, natural language generation, and knowledge graphs. Therefore any research work at the intersection of NLP, data management, and HCI is of significant interest to us.
My Experience at CHI 2022 @ NOLA
I once read somewhere that science is a social process. That’s an observation I agree with entirely. Conferences provide the opportunity to connect with peers, meet new folks, and discuss exciting ideas and projects. Therefore, I was looking forward to going to New Orleans after two years of attending conferences virtually. In my view, the face-to-face discussions in the in-person setting felt more spontaneous and inspiring than the virtual Gather discussions.
I presented a paper on understanding the challenges data science workers face in performing information extraction tasks in the “Reasoning and Sensemaking” session on the final day of the conference. More details can be found in my previous blog post. Surprisingly, our session chair was remote while all the presenters were present in the conference room. The hybrid setup and minor technical issues aside, my overall experience at CHI was fantastic. The organizers did a phenomenal job in organizing and managing a hybrid event.
Overview of Papers, Keynotes, and Awards
First, I will highlight a few exciting papers on human-AI interaction, mixed-initiative systems, and data visualization. Then, I will briefly discuss the keynotes and awards, which often reflect what the community is thinking about and deems to be important topics and issues.
CHI welcomes diverse research topics with papers presented in several parallel sessions. So I had to prioritize which sessions to attend. Here, I will highlight papers/sessions that I found to be most interesting and relevant to our work at Megagon Labs. I have listed the papers with their ACM digital library link, which contains video presentations of the work.
Besides the aforementioned work, researchers at Google presented the idea of “chain of thought” prompting to elicit desired content from language models. In a chain, a problem can be broken down into a number of smaller sub-tasks. Each sub-task can be mapped to a distinct step with a corresponding natural language prompt. Then, results of one or more previous steps can be fed into the next step in aggregated form as an input prompt. The rationale behind chaining is that the process enables users to run the same model on multiple sub-tasks, thereby ensuring each sub-task has a higher likelihood of achieving the desired outcome. Here is a blog post that explains the idea in detail. In this work, the researchers explored whether humans can successfully operate within this chain-of-thought prompt setting and showed such a strategy might help calibrate model outcomes and debug unexpected model outputs. The paper can be found here:
Another work explored the text-to-image generation problem. Researchers studied how to effectively prompt generative models with style and subject keywords to generate artistic images and proposed several design guidelines:
The following are some additional pointers related to the work: project page, NLP demo, and vision demo. As I conclude the human-AI interaction discussion, I also wanted to highlight the “Human-centered Explainable AI” workshop at CHI which tackled important questions related to the study of explanation.
The paper I found most interesting at this year’s CHI was Symphony by researchers at Apple. The project is not open source, though. Symphony is a framework for composing interactive ML interfaces with task-specific, data-driven components that can be used across platforms such as computational notebooks and web dashboards. The extensible and adaptive component-driven design of the tool makes it very appealing. The extensibility is achieved through wrappers. These wrappers have two primary functions—first, passing data from a platform to Symphony in the correct format, and second, rendering Symphony components in the platform’s UI. Other exciting features include a persistent toolbar to structure users’ workflow and automate their unwanted workload. While Symphony seems to draw from existing formalization in visualization literature, it is not backed by any formal grammar. Our work on LEAM, presented more than a year ago at CIDR’21 (and surprisingly missing from the citations), explored similar ideas in the context of text analysis workflows. We proposed a formal grammar for visual interactive text analysis. We also presented ideas such as metadata and interaction state management which Symphony explored. Nevertheless, Symphony is an important work that opens the door to ongoing and exciting research. Here is the link to the paper:
Regarding mixed-initiative systems, one work explores the merits of the automated agents in the context of 3D touch interactive techniques and highlights challenges related to agency and expressiveness. The tension between human agency and automation is long-discussed in HCI research and poses vital challenges for designing and engineering mixed-initiative systems. How can we effectively integrate automated reasoning into interactive systems without impeding human agency? Then there is the question of trustworthiness. Another work explored how people trust and rely on an AI assistant in an object identification task. The work demonstrates that people can detect and rely on an AI assistant that performs both expertly and non-expertly in different parts of the same task. The papers can be found here:
Last but not least… visualizations!
There were several very interesting data visualization and analytics papers that I missed due to conflicts/overlaps with other sessions. However, I would love to explore the following papers in detail sometime soon and wanted to highlight them here:
- Varv: Reprogrammable Interactive Software as a Declarative Data Structure
- Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
- Structure-aware Visualization Retrieval
- ComputableViz: Mathematical Operators as a Formalism for Visualization Processing and Analysis
- Recommendations for Visualization Recommendations: Exploring Preferences and Priorities in Visualization Recommendations for Public Health
Now let’s briefly discuss the keynote talks and awards next.
The opening keynote at CHI 2022 was delivered by Kishonna Gray who talked about hybrid communities that online gamers form on various platforms such as Discord, Twitter, and Twine. More specifically, Kishonna talked about how gamers communicate, innovate, and create on these platforms and how the dynamics within such communities may help reimagine the future in an increasingly digital world.
I have personal experience engaging with such communities by playing online games like FIFA. However, those experiences revolved more around toxicity and how in-game settings almost felt like gambling. Here is a link to the talk.
The closing keynote resonated with me as I am originally from the global south (Bangladesh). Payal Arora is doing fantastic work on empowering women in the South Asian community. Her talk called attention to the fact that we need to recognize the varied dynamics of systemic inequality, shaped by race, caste, gender, and religion, as we develop technology solutions. For example, her talk pointed to US-centric industries and academia despite the US having only a small percentage of internet users. Often, papers on the global south are questioned on the ground of generalizability to, say, the US.
In CHI 2022, there were several (25) best paper award winners. A more detailed list can be found here. There were three dissertation award winners also. I would specifically like to highlight one of the winners, Fred Hohman (Georgia Tech.) Fred’s research focused on machine learning interpretability related to scaling for deep learning, communicating explanations, and investigating emerging practices. His dissertation spans both visualization and ML and proposes a number of open-source interactive interfaces, scalable algorithms, and novel communication paradigms. In my view, this recognition highlights that human-centered AI is of significant interest to the HCI community. As a researcher working on this topic at Megagon Labs, I am very excited about the prospect of seeing a lot more amazing work in the coming years! Please, check out Fred’s award talk here.
And with that, it’s a wrap. I hope the readers will find some of the highlighted papers interesting.