From Knowledge Graphs to Self-Improving LLM-Based Agent Systems
Speaker: Estevam Hruschka (Megagon Labs)
In recent years, large language models (LLMs) have demonstrated remarkable generalization abilities, yet they remain inherently static (trained once and deployed) with limited capacity for continual adaptation. Conversely, Never-Ending Learning (NEL) and Lifelong Learning (LL) or Continual Learning (CL) approaches are built on the idea of continuous, open-ended accumulation of knowledge and skills. This tutorial bridges these paradigms, exploring how NEL principles can enhance modern LLMs through continual pre-training, self-reflective adaptation, and integration with dynamic knowledge sources such as knowledge graphs, databases, and multimodal inputs. We will survey the evolution of NEL and continual learning (from the Never-Ending Language Learner (NELL) to modern lifelong reinforcement and self-supervised learning systems) and position them within today’s agentic AI architectures. Through conceptual grounding, empirical demonstrations, and a running case study (e.g., continual NER), we will illustrate how to design, evaluate, and deploy LLM-based systems that learn forever.
Tutorial slides: click here
Tutorial Outline
Part I: Foundations & Taxonomy
- Evolution of NEL, Lifelong Learning, and Continual Learning.
- Positioning NEL vs. Semi-supervised and Reinforcement Learning.
- Survey of modern taxonomies (Gao et al. 2025, Wang et al. 2024).
Part II: From Concept to Practice
- Case Study: Continual Named-Entity Recognition (NER).
- Modeling feedback loops, memory updates, and drift control.
- Scaling to Compound AI Systems (ReAct, Reflexion).
Part III: The Future of Agentic AI
- Current bottlenecks and open research challenges.
- Pathways to self-improving systems for the Semantic Web.
About the Speaker
Estevam Hruschka is the Lab Director and Staff Research Scientist at Megagon Labs, in Mountain View, CA., the research arm of Recruit Holdings (the parent company of Indeed.com, Glassdoor.com among other companies). Before Megagon Labs, he was with Amazon (2017-2020) helping Alexa to learn to read the Web. Before joining Amazon, Estevam was co-founder and co-leader of the Carnegie Mellon Read the Web project –http://rtw.ml.cmu.edu/rtw/ (in which the Never-Ending Language Learner (NELL) System was designed, modeled and deployed), and the head of the Machine Learning Lab (MaLL) at Federal University of Sao Carlos, in Brazil (where he was associate professor 2004–2019). From 2016 through 2022, he was also adjunct professor the Machine Learning Department (http://www.ml.cmu.edu) at Carnegie Mellon University. He received the 2025 AAAI Classic paper Award and a Google Research Award (for Latin America) in 2017. He was a “young research fellow” at FAPESP (Sao Paulo State Research Agency, Brazil) and a “research fellow” at CNPq (Brazilian National Research Agency). His main research interests are related to never-ending learning, natural Language understanding and conversational learning. He has been working on these research topics with many international research teams, collaborating with research groups from companies and universities.
Selected References
- Akari Asai et al. 2023. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv:2310.11511.
- Yuntao Bai et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv:2204.05862.
- Andrew et al. Carlson. 2010. Toward an architecture for never-ending language learning. In AAAI-2010.
- Zhiyuan Chen and Bing Liu. 2016. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 10(3):1–145.
- Gheorghe Comanici et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv:2507.06261.
- Daya Guo et al. DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv:2501.12948.
- Abhimanyu Dubey et al. 2024. The llama 3 herd of models. arXiv:2407.21783.
- Huan-ang Gao, et al. 2025. A survey of self-evolving agents: On path to artificial super intelligence. arXiv preprint arXiv:2507.21046.
- Aaron Jaech et al. 2024. Openai o1 system card. arXiv:2412.16720.
- Leslie Pack Kaelbling et al. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285.
- Bing Liu. 2020. Learning on the job: Online lifelong and continual learning. In AAAI-2020.
- Aman Madaan et al. 2023. Self-refine: Iterative refinement with self-feedback. NeurIPS-2023, 36:46534-46594.
- Potsawee Manakul et al. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv:2303.08896.
- Tom Mitchell et al. 2018. Never-ending learning. Communications of the ACM, 61(5):103–115.
- Openai 2024. Openai. Openai o1 System Card. preprint. arXiv:2412.16720
- Rafael Rafailov et al. 2023. Direct preference optimization: Your language model is secretly a reward model. NeurIPS-2023, 36:53728–53741.
- Keshav Santhanam et al. 2024. Alto: An efficient net- work orchestrator for compound ai systems. In 4th Workshop on Machine Learning and Systems.
- Haizhou Shi et al. 2024. Continual learning of large language models: A comprehensive survey. ACM Computing Surveys.
- Noah Shinn et al. 2024. Reflexion: Language agents with verbal reinforcement learning. NeurIPS-2024.
- Shagun Sodhani, Sarath Chandar, and Yoshua Bengio. 2020. Toward training recurrent neural networks for lifelong learning. Neural computation, 32(1):1–35.
- Sebastian Thrun. 1998. Lifelong Learning Algorithms. In Learning to Learn, pages 181–209. Springer.
- Jesper E Van Engelen and Holger H Hoos. 2020. A survey on semi-supervised learning. Machine learning, 109(2):373–440.
- Liyuan Wang et al. 2024. A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on PAMI, 46(8):5362–5383.
- Jason Wei et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems.
- Shunyu Yao et al. 2022. React: Synergizing reasoning and acting in language models. arXiv:2210.03629.
