AI Dictionary › AI Models
Normalizzazione per batch
Batch normalization is a technique that rescales the values produced by a neural network layer by computing mean and variance across all examples in the same training batch, rather than over the values of a single example. Its purpose is to reduce unwanted variation in the distribution of values flowing through the network, making training more stable and often faster.
During training, for each mini-batch of examples processed together, the layer computes the mean and variance of the output values and normalizes them accordingly, then applies two learned parameters that let the network rescale and shift the values again if beneficial. During inference, when examples are often processed one at a time, running statistics accumulated during training are used instead of the current batch's statistics.
For years it was a standard component of convolutional neural networks used in computer vision, helping make much deeper networks trainable than before. It is less suited to models that process sequences of variable length or small batch sizes, contexts where techniques such as layer normalization are generally more effective.
The name describes the mechanism exactly: normalization is computed over the "batch", meaning the group of examples processed together in a single training step, an idea that had a significant impact on the speed and stability of training deep networks starting in the mid-2010s.
From our network
Magellano GPS: Fleet Tracking Made Simple
Real-time GPS tracking, remote engine lock, fuel and CO₂ reporting for your fleet.
Visit magellanogps.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.