AI Dictionary › Modelli AI
Ottimizzatore Adam
Adam is an optimization algorithm used to update a neural network's parameters during training, combining the advantages of two earlier techniques: using a moving average of past gradients to make updates more stable, and automatically adapting the learning rate for each parameter individually, based on its recent update history.
At each training step, Adam maintains two quantities for every parameter that are updated over time: an estimate of the recent average direction of the gradient and an estimate of its recent variability. By combining these two pieces of information, the algorithm computes an update that tends to be larger for parameters that change little and more cautious for those that oscillate a lot, allowing generally faster and more robust convergence compared to plain gradient descent.
It is the most widely used optimizer in training modern deep neural networks, including large language models, often in its variant that introduces a correction for weight regularization. Its widespread adoption stems from combining good practical performance with relative insensitivity to the initial choice of hyperparameters compared to other methods.
The name is an acronym referring to adaptive moment estimation, the principle underlying the algorithm, proposed in the first half of the 2010s as an evolution of earlier optimization methods used in neural network training.
From our network
HSE Genius: AI for Safety Data Sheets
Extract SDS data, H phrases and ECHA compliance checks in seconds, powered by AI.
Visit hsegenius.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.