AI Dictionary › AI Models
Normalizzazione dei livelli
Layer normalization is a technique that rescales the values within each individual layer of a neural network so they have controlled mean and variance before being passed to the next layer. Its purpose is to keep the flow of numerical values stable across very deep networks, preventing numbers from growing or shrinking uncontrollably as they pass through many layers.
Unlike other normalization techniques that compute statistics over an entire batch of examples, layer normalization computes mean and variance over the values of a single example within the same layer, making it independent of the batch size used during training or inference. After normalization, two learned parameters let the model rescale and shift the values again if needed.
It is present in every block of modern attention-based architectures, typically before or after the attention and feed-forward sublayers, and contributes decisively to the ability to train networks with dozens or hundreds of layers without training becoming unstable.
It was proposed as an alternative to batch normalization in cases where the latter proved less effective, particularly for processing sequences of variable length, and has become the standard in sequential attention-based architectures.
From our network
HSE Genius: AI for Safety Data Sheets
Extract SDS data, H phrases and ECHA compliance checks in seconds, powered by AI.
Visit hsegenius.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.