AI Dictionary › AI Fundamentals

Softmax

The softmax function turns a set of raw numerical scores into a probability distribution: values that sum to one and represent how likely each option is. It takes the logits produced by the model, amplifies them exponentially, and normalizes them, so the highest score becomes the largest probability without fully zeroing out the alternatives. It's like converting a jury's raw votes into clear, comparable preference percentages. In a language model, the final-layer softmax gives the probability of each possible next word, guiding which token to generate.

Definition

Softmax is everywhere in AI: in classification to pick a category, in attention mechanisms to weigh token importance, in text generation. It's also where parameters like temperature act, making the distribution sharper or fuzzier and thus influencing how creative or predictable a model's responses are.

Related terms

• Logits
• Temperature
• Attention Mechanism
• Loss Function
• Inference

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.