AI Dictionary › Fondamenti AI

K-Nearest Neighbors (KNN)

K-nearest neighbors, or KNN, is one of the conceptually simplest machine learning algorithms: to classify or predict a new point, it looks at the k most similar examples it already knows and takes their most common decision, or the average of their values if the task is regression. It builds no abstract model during training: it simply stores all the data and consults it every time it has to make a prediction.

Definition

How it works

Similarity between points is measured with a distance, often the Euclidean one, computed over the numeric features of each observation. To classify a new example, the algorithm finds the k nearest neighbors in the training dataset and assigns the most frequent class among them; k is a parameter that must be chosen carefully, too low a value makes the model sensitive to noise, too high a value flattens it and makes it lose detail. Precisely because there is no real learning phase, KNN is called a lazy algorithm: all the work happens at prediction time, not before.

Applications

It is used for recommendation systems based on similarity between users or products, pattern recognition in simple images, anomaly detection by comparing a new data point with its closest historical neighbors, and as a reference method for benchmarking more sophisticated algorithms. Its simplicity makes it an excellent teaching tool for introducing the concept of similarity in data.

History & etymology

The idea of classifying based on nearest neighbors was first formalized by Evelyn Fix and Joseph Hodges in a 1951 technical report prepared for the United States Air Force, within the field of nonparametric statistical analysis. The version with k neighbors, standard today, was later developed and made known by Thomas Cover and Peter Hart in a 1967 paper that formally analyzed its properties.

Related terms

• K-means
• Classification
• Clustering
• Feature engineering
• Outlier

More in Fondamenti AI

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.