AI Dictionary › AI Fundamentals

Naive Bayes

Naive Bayes is a family of classification algorithms based on Bayes' theorem, which computes the probability that an observation belongs to a given class by combining the baseline probability of that class with how typical, for that class, the observed features are. It is called naive because it assumes, for simplicity, that all features are independent of one another given the class value, an assumption that is almost always false in reality but works surprisingly well in practice.

Definition

How it works

The algorithm estimates, from the training data, how likely each class is overall and how likely it is to observe each individual feature within each class. To classify a new example, it multiplies these probabilities together for every possible class and picks the one with the highest overall value. Thanks to the independence assumption, the computation stays very simple even with a huge number of features, which makes it extremely fast to train and to use.

Applications

It has historically been one of the most widely used algorithms for text classification: spam filters, sentiment analysis, automatic categorization of documents and articles, recommendation systems based on textual content. It remains a very practical choice when a fast, lightweight and reasonably accurate model is needed, especially with high-dimensional text data where every word becomes a feature.

History & etymology

The name combines naive, referring to the simplifying independence assumption, and Bayes, in honor of the Reverend Thomas Bayes, an eighteenth-century British mathematician whose theorem on conditional probability underlies the calculation. Applying Bayes' theorem to automatic text classification, particularly spam filtering, spread widely starting in the 1990s, when it became one of the first statistical methods used at scale to distinguish unwanted email from legitimate messages.

Related terms

• Classification
• Feature engineering
• NLP (Natural Language Processing)
• Logistic regression
• Dataset

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.