AI Dictionary › AI Fundamentals

Scaling Laws

Leggi di scala

Scaling laws describe how a model's performance improves predictably as three ingredients grow: model size in parameters, the amount of training data, and the compute used. Empirical studies showed that error falls along smooth, almost mathematical curves as you scale. It's like a recipe where increasing ingredients, cooking time, and heat proportionally yields a reliably better result, letting you estimate in advance what to expect from a bigger model before even building it. Scaling laws also help balance data and parameters correctly for a given budget.

Definition

These regularities drove the race toward ever-larger models, justifying huge investments with reliable improvement forecasts. At the same time they raise questions about cost, physical limits, and sustainability, and whether scaling alone suffices to reach qualitatively new capabilities.

Related terms

• Emergent Abilities
• Parameters
• Pre-training
• Foundation Model
• Mixture of Experts (MoE)

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.