AI Dictionary › AI Models
Tasso di apprendimento
The learning rate is the hyperparameter that determines how much a model's parameters are changed at each training step, based on the gradient computed by the optimization algorithm. It is one of the most influential configuration values in training any neural network: it regulates the size of the "step" the model takes toward reducing error.
A learning rate that is too high can cause parameter updates so large that they overshoot the optimal point at every step, preventing the model from converging and making training unstable or even divergent. A learning rate that is too low, on the other hand, makes training extremely slow, requiring a great many steps for modest improvements. In practice, strategies that vary the learning rate over time are often used, gradually increasing it at the start and progressively reducing it toward the end of training.
It is a central parameter in training every type of neural network, from small classification models to large language models, where the choice of the initial learning rate and how it evolves over time is often the subject of extensive experimentation before launching a large-scale training run.
The term directly describes its role: it defines the "rate", meaning the speed, at which the model incorporates what it learns from the data at each iteration, a concept central since the earliest formulations of gradient descent algorithms applied to neural networks.
From our network
AGORÀ Intelligence: Enterprise AI Governance Platform
Govern AI at scale: policies, adoption and measurable results on your data. Built for boards and C-suite.
Visit agora-intelligence.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.