AI Dictionary › AI Fundamentals

Self-Supervised Learning

Apprendimento auto-supervisionato

Self-supervised learning is a training approach in which the model generates its own supervision signals from the data itself, without needing labels manually created by people. The training task is built by hiding or altering part of the data and asking the model to predict it, using the rest of the data as context.

Definition

How it works

A typical example is masking some words in a text and training the model to predict them from the surrounding words, or training it to predict the next word given a sequence of text. This way, the data's own intrinsic structure is used as a source of supervision, without needing to collect external annotations.

Applications

It is the approach underlying the pre-training of most modern large language models, which learn from huge amounts of unlabeled text gathered from various sources. It is also used in areas such as computer vision, to learn useful representations from images or video without manual labels.

History & etymology

It has become central to training models at scale because it allows huge amounts of available data to be exploited without the cost of manual labeling required by traditional supervised learning.

Related terms

• Pre-training
• Supervised Learning
• Unsupervised learning
• Training Data
• Embedding

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.