Never-ending Learning (NEL) systems are, in a very high-level, computer systems that learn over time to become better in solving one or more specific tasks. In the last decades, different Never-Ending Learning (NEL) approaches have been proposed and applied in different tasks involving Artificial Intelligence, Machine Learning, Natural Language Processing (NLP) and Natural Language Understanding (NLU). Results from NEL approaches are getting more frequently successful, thus encouraging us to address the problem of how to build computer systems that can take advantage of NEL principles. In addition, the variety of different names (never-ending learning, continuous learning, lifelong learning, etc.) used to describe systems and models that can keep learning in a continuous way, as well as the new achievements on self-supervised learning and multi-task learning (which are closely related to NEL principles) present in large pre-trained language models, make relevant to have a tutorial on NEL.

In this tutorial we will explore the Never-Ending Learning (NEL) ideas and principles, different approaches (and variations) that can be found in the literature, the similarities and differences from traditional ML approaches (such as semi-supervised learning, reinforcement learning, etc.) and the applications. Another interesting aspect to be explored is how to formulate a problem following a NEL approach. Thus, we will also show how to model a problem in a NEL fashion and help the audience to become familiar with such approaches.

In summary, this tutorial aims at enabling the attendees to:

• Learn about the current state-of-the art on NEL (and its variants);
• Understand the similarities and differences between the NEL principles and other approaches such as large pre-trained language models, reinforcement learning, and semi-supervised learning, as well as how they can be jointly employed in hybrid approaches.
• Learn how to model different problems using NEL principles;
• Be prepared to follow along NEL principles and to propose new approaches to boost performance of applications related to the NLP/NLU.