AI Dictionary › AI Fundamentals

Inference

Inference is the process through which an already-trained AI model generates a response from an input. Unlike training (the phase in which the model learns from data), inference is the operational phase: the model receives a prompt and produces an output by applying patterns learned during training.

Definition

Every time you write a message to ChatGPT, Gemini or any other LLM and receive a response, you are using inference. Computationally, inference requires far less processing power than training, but at scale (billions of requests a day) it still demands huge, costly infrastructure.

Understanding how inference works helps explain why AI models can feel slower at certain times (high demand), why response quality can vary (stochastic sampling and temperature), and why the prompt matters so much: during inference, the model has only the prompt's text to guide how it generates the response.

Related terms

• LLM (Large Language Model)
• Temperature
• Transformer
• Neural Network
• Fine-tuning

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.