AI Dictionary › AI Fundamentals

Cross-validation

Cross-validation is a technique for estimating how well a machine learning model will generalize to unseen data, without having to sacrifice a large portion of the dataset to a single test. Instead of splitting the data into training and validation just once, it splits it multiple times in different ways and averages the results, producing a far more reliable estimate of the model's real performance.

Definition

How it works

The most common variant is k-fold cross-validation: the dataset is divided into k equally sized parts, called folds. On each round a different part is held out as the test set, the model trains on the remaining k-1 parts, and performance is measured on the one left out. Repeating the procedure k times means every observation ends up in the test set exactly once, and the average of the k performance scores is the final estimate, along with its variability, valuable information that a single train-test split would not provide.

Applications

It is an essential tool for fairly comparing different models, choosing the best hyperparameters, and checking that a model did not simply get lucky with one particular data split. It matters especially with small datasets, where a single test set risks giving an unstable and unreliable estimate of real performance. Variants such as stratified k-fold preserve class proportions in every fold, which is useful with imbalanced data.

History & etymology

The term cross-validation appears in the statistical literature starting in the 1930s, but it spread mainly from the 1960s and 70s onward through the work of statisticians such as Seymour Geisser and Mervyn Stone, who formalized its use for model selection. The name literally describes the central idea: validating the model by repeatedly crossing the roles of training and testing across the different portions of the data, instead of fixing them just once.

Related terms

• Dataset
• Train-test split
• Overfitting
• Hyperparameter tuning
• Evaluation metrics

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.