AI Dictionary › AI Fundamentals

Benchmark

A benchmark is a standardized test used to measure and compare AI model performance. Well-known examples include MMLU (multidisciplinary knowledge), HumanEval (code writing), GSM8K (math) and SWE-bench (real software engineering bugs).

Definition

Benchmarks enable objective comparisons between models, but need to be read critically: a model can be optimized to "win" benchmarks without being genuinely better in daily use (a phenomenon called benchmark contamination, when the test questions end up in the training data). Leaderboards shift with every new release.

For those choosing a model for business use, benchmarks are a starting point, but testing on your own real use case, with your own prompts and data, remains the only decisive test.

Related terms

• LLM (Large Language Model)
• Score
• Inference
• Fine-tuning

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.