We are wrapping up another successful summer of working alongside talented interns from a variety of universities. As usual, we want to share with you the talented people who contributed to our lab projects and give them an opportunity to talk about their experiences.
Yoo Yeon Sung
University: University of Maryland
Program: PhD at the School of Information
Project: Verifying an LLM planner: Since LLM planners decompose the original tasks into subtasks, we verify each subtask to improve the overall result of the LLM task.
Project Mentors: Hannah Kim & Dan Zhang
How did your project tie in with your studies?
In my PhD studies, I focused on user studies and data collection utilizing the human-in-the-loop approach. I did use that insight for this project with Megagon, but it’s a different domain; I went from traditional NLP tasks to LLM chains. The tools tie in, but the domains and tasks were very different, so it was a good and new change.
What is some advice you would give future Megagon Labs interns?
Planning ahead might help. I feel I should have done more planning ahead before my internship even began, but now I can pass that advice on to incoming interns. It’s important to communicate with your mentor ahead of your internship and get the plans in advance. And while you’re working on formulating your research project with your mentor, it’s good to have a plan B or plan C, since things don’t always go as planned.
Simone Papicchio
University: Politecnico di Torino, Turin
Program: PhD in Computer and Control Engineering
Project: Creating a framework for automatic evaluation of RAG (retrieval augmented generation) systems. With different sources of information, we’d like to automatically generate question, answer, and composite questions, helping to generate benchmark datasets.
Project Mentor: Sajjadur Rahman
What's something you learned during your internship?
How to do research. It’s my first year in the PhD program, so this has helped me grow a lot as a researcher. The rigidity of a three-month internship has helped me be thorough. I learned to organize a research project, do a literature review, and write a research paper. This is usually something PhD candidates learn later, and often, we learn it alone. I was able to ask other researchers in the office how to write a literature review and a paper. I had guidance. I will also take the confidence I gained with the support I received from all the other researchers at Megagon Labs.
What is some advice you would give future Megagon Labs interns?
Enjoy the experience. It is a fun process. Don’t see it as just work; have fun with it. Fun is subjective but in the research world, this is fun. It’s a process where we do what we can to push the boundaries of technology, doing something that no one else is doing.
Don’t be afraid to communicate with your mentor and ask for help. It’s not only up to you to solve the problem; you’re working together. Don’t be intimidated by their seniority, and don’t be shy. They are very helpful. You can learn from everyone, and there is a great pool of researchers. Take advantage. It’s a unique experience, especially coming from Italy.
Yihao Hu
University: Duke University
Program: PhD in Computer Science
Project: Research on data lake repositories of abstract files: We wanted to find techniques to create interesting visualizations for the unstructured data—visualizations that gave us interesting metrics and allowed for better data exploration by the user.
Project Mentors: Jin Wang & Sajjadur Rahman
How has the Megagon team helped you broaden your research interest?
My PhD work primarily focuses on developing tools and frameworks for users to better understand how database queries work. I built a SQL query execution visualizer to show the user how SQL queries are executed in launch code. That was related in terms of visualization, but I was challenged to step out of my comfort zone by working with unstructured data. Working with some undefined tasks made a difference, and it has given me some good ideas to take back to the work I do for my PhD.
What is something you liked about working at Megagon during your internship?
I can honestly say it’s the best internship I’ve ever done. My internship has been great, especially in terms of opening my eyes and teaching me different aspects of research and different research mindsets. This experience represents the building blocks for my future research. The entire project was laid out in a very structured manner and gave me a very clear goal for my time with Megagon.
I can clearly see how my research fits into the entire Megagon project. I learned how things are tied together, which will help me position my future research in a larger context.
Jin and Sajjadur are great mentors, and everyone in the lab has been approachable. The experience was great.
What is something you liked about working at Megagon during your internship?
My mentor gave me literature to read. I advise other interns to prepare for their own topics by going over any literature and information given to them by their mentor. Mentally preparing for your internship is important—every second counts at work.
Communicate well with your mentor and update them on progress. Try and finish early to have a cushion in case something doesn’t work out. Turnaround time is very short, so try and finish everything as soon as possible.
It’s very difficult to finish everything assigned in the internship, but most things should be done. Be prepared for a very fast-paced and intense internship.
Catarina Belém
University: University of California, Irvine
Program: PhD in Computer Science
Project: Investigate how LLMs perform on tasks where it is required to process information across multiple documents, such as multi-document summarization, opinion summarization, or even resume and job description matching. The focus was on understanding the types of errors or hallucinations models make in these scenarios, especially concerning potential biases. The goal was to quantify these errors and develop simple techniques to mitigate them.
Project Mentor: Pouya Pezeshkpour
How did your project tie in with your studies?
My studies have largely focused on the fairness and trustworthiness of LLMs. Models that fabricate untruthful information or are unable to follow the specified instructions are considered untrustworthy. Although in a different setting and based on different tasks, by inspecting and characterizing the types of errors, their frequency, and their cause, we are, in some sense, contributing to the study of the trustworthiness of LLMs.
What is something you learned during your internship that you will take with you?
I learned to better utilize a project log to help structure my thinking. I used to do this before my internship, but it was not as structured. I liked having shorter cycles of interacting with my advisor to make sure the research direction was going well. Pouya helped me reevaluate and clarify techniques, and I also gained the courage to ask for help. I now know how to push back on research ideas. I can provide good insight to help build the project into something that is interesting and aligns with a broader field of research and inquiry. The entire internship has been a growth experience.
Haibo Sun
University: Brandeis University, Boston
Program: PhD in Computational Linguistics
Project: I sought to develop a way to make use of subjective information, such as reviews, for use in a conversational recommendation system.
Project Mentor: Naoki Otani with support from Hannah Kim, Nikita Bhutani, & Dan Zhang
How was your experience working with the team at Megagon Labs?
It was great to have this level of collaboration. It’s a small lab, which is slightly different from working in school (where we are all going in different directions and have ample time). Here at Megagon, we are all going in a similar direction and have limited time to finish projects. It has pushed me to be very focused, organized, and steadfast.
I liked the feeling of working on a real-world, applicable problem, not a benchmark or academic test, and then converting it into a research problem. I learned to connect academic research to a real-world problem to make the research more solid. I focused on a problem that was worth solving and found new ways to solve it.
I also had a great mentor, and my research interests aligned with his.
Coming from Boston, what did you enjoy most about the Bay Area?
The Bay Area is so much fun, and it’s always sunny. I go hiking, travel downtown to SF, and sometimes to the seaside to get some cooler weather. I love the views of the mountains from the office. The food is very good. There’s lots of authentic Asian food. Mountain View is very lively and has a good atmosphere.
We are grateful to have had a successful summer of projects, outings, and even a potluck. We leave you with a collection of images from our summer experiences.
Interns: We’d like to thank you for all of your outstanding work ethic, grace, and positive attitude during this intensive internship. It was a pleasure meeting and working with you all!
If you would like to learn more about our internship program, or to apply, check out our internship page.