AI Dictionary › AI Fundamentals
Hyperparameter tuning (Ottimizzazione degli iperparametri)
Hyperparameter tuning is the process of finding the best combination of hyperparameters, the settings of a machine learning model that are not learned from the data but decided before training, such as the maximum depth of a decision tree, the number of trees in a forest, or the learning rate used in gradient descent. Unlike actual parameters, which the model learns automatically, hyperparameters must be chosen externally, and the right choice can make a huge difference to final performance.
The simplest approach, grid search, systematically tries all possible combinations of a predefined set of values for each hyperparameter, evaluating each combination typically with cross-validation. It is exhaustive but quickly becomes too expensive when there are many hyperparameters to explore. Random search instead samples random combinations from the hyperparameter space, often proving surprisingly efficient because it explores a wider range of values with the same number of attempts. More advanced methods, such as Bayesian optimization, use the results of previous attempts to intelligently decide which combination to try next, concentrating the search in the most promising areas of the search space.
It is an almost universal step in any serious machine learning project: from classic models like random forest and gradient boosting, where the number of hyperparameters to adjust is often considerable, to deep neural networks, where tuning covers architecture, learning rate, batch size and regularization strategies. Automated machine learning platforms, known as AutoML, have made this process increasingly automated, reducing the manual burden on data scientists and engineers.
The term hyperparameter explicitly distinguishes these external settings from parameters, the internal ones the model estimates during training, a distinction well established in statistical and Bayesian literature well before the spread of modern machine learning. The term tuning literally recalls the metaphor of tuning a musical instrument until the right sound is found, an image that captures well the idea of an iterative fine-adjustment process.
From our network
INDACO TMS: Transport Management for European Logistics
Shipment tracking, multi-carrier EDI and automated invoicing in one cloud platform. Invoices generated in under 10 seconds.
Visit indacotms.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.