AI Dictionary › Fondamenti AI

Pre-training

Pre-addestramento

Pre-training is the first and most expensive phase of building a large model, where it learns from enormous amounts of raw, unlabeled data. For a language model this means reading billions of sentences and learning to predict the next word, thereby absorbing grammar, facts, styles, and reasoning patterns. It's like a person's general education: before specializing in a trade, they accumulate broad background knowledge by reading and observing the world. This phase produces the foundation model, a reservoir of general knowledge not yet aimed at a specific task.

Definition

Pre-training takes months of computation across thousands of processors and is what makes models so versatile. Only afterward, with lighter phases like fine-tuning, is this base adapted to specific uses, far more efficient than training from scratch for every application.

Related terms

• Foundation Model
• Fine-tuning
• Transfer Learning
• Training Data
• Scaling Laws

More in Fondamenti AI

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.