AI Dictionary › AI Models
Connessione residua
A residual connection is a shortcut that directly links a layer's input to its output by adding them together, rather than forcing information to pass exclusively through the layer's own transformations. In practice, instead of learning the full transformation from input to output, the layer only has to learn the difference, the "residual", to add to the original input.
This sum of input and transformed output keeps a direct path through which the error signal can propagate during training without being excessively attenuated, even when the network is made up of very many stacked layers. Without this trick, very deep networks tend to become harder to train as more layers are added, instead of improving.
It is a structural element present in virtually all modern deep architectures, including attention-based ones, where every attention and feed-forward sublayer is wrapped in a residual connection. It has made it possible to reliably train networks with hundreds of layers, something impractical with earlier architectures.
The concept was introduced in the context of very deep convolutional networks for computer vision, where it was first shown that adding direct connections between non-adjacent layers overcame the degradation problem seen in excessively deep networks.
From our network
Magellano GPS: Fleet Tracking Made Simple
Real-time GPS tracking, remote engine lock, fuel and CO₂ reporting for your fleet.
Visit magellanogps.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.