AI Dictionary › Fondamenti AI

Catastrophic Forgetting

Oblio catastrofico

Catastrophic forgetting is a neural network's tendency to suddenly forget what it had learned when trained on a new task. Because learning means modifying the same shared parameters, adapting them to new information can overwrite the old, erasing previously acquired skills. It's like a student who, focusing intensely on a new subject, forgets almost everything studied before, because they 'overwrote' the same memory instead of adding to it. The phenomenon is especially insidious in sequential learning, where tasks arrive one after another.

Definition

Managing catastrophic forgetting is central when updating or specializing a model without losing its general abilities, for instance during fine-tuning. Techniques like rehearsing old data, protecting the most important parameters, or adding small dedicated modules help the model learn new things while retaining what it already knew.

Related terms

• Transfer Learning
• Fine-tuning
• Parameters
• Neural Network
• Pre-training

More in Fondamenti AI

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.