AI Dictionary › AI Fundamentals
Compromesso bias-varianza
The bias-variance tradeoff describes a fundamental tension in machine learning: a model that is too simple tends to be systematically wrong because it cannot capture the true complexity of the data, a problem called bias, while a model that is too complex tends to chase every detail and random noise in the training data, a problem called variance, and in both cases performance on new data suffers.
Bias is the error that comes from overly rigid assumptions about the model: a linear regression applied to a strongly nonlinear relationship will always have high bias, regardless of how much data it receives. Variance is the error that comes from the model's excessive sensitivity to the specific training data: a very deep decision tree can change its predictions drastically if trained on a slightly different sample of the same data. The total error a model makes on new data is, roughly, the sum of squared bias, variance and an irreducible error due to the data's intrinsic noise: reducing one almost always tends to increase the other.
This tradeoff guides nearly every practical decision in machine learning: how complex to make a model, how much data to collect, how strongly to apply regularization, when to stop training. A model with high bias suffers from underfitting, not learning enough from the data; a model with high variance suffers from overfitting, learning too much, including the noise. Cross-validation is the most widely used practical tool for finding the optimal balance point between the two extremes for a specific problem.
The formal decomposition of prediction error into bias and variance belongs to classical statistics, but its explicit formulation in the context of machine learning and neural networks is particularly credited to an influential paper by Stuart Geman, Elizabeth Bienenstock and René Doursat published in 1992, which made it one of the most cited guiding concepts for understanding why models generalize well or poorly to unseen data.
From our network
INDACO TMS: Transport Management for European Logistics
Shipment tracking, multi-carrier EDI and automated invoicing in one cloud platform. Invoices generated in under 10 seconds.
Visit indacotms.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.