AI Dictionary › AI Fundamentals

Latency

Latenza (Latency)

Latency is the time between sending a prompt and receiving the model's response. It is typically measured in two ways: time to first token (how long you wait before the response starts appearing) and total generation time.

Definition

It depends on several factors: model size (larger models are slower), prompt and response length, server load, and inference optimization techniques. That's why providers often offer multiple versions: a powerful but slower model for complex tasks, a fast one for real-time interactions.

In AI product design, latency is a decisive UX factor: a customer service chatbot must answer in 1-2 seconds, document analysis can afford 30. Prompt efficiency helps too: denser prompts generate faster responses.

Related terms

• Inference
• Token
• SLM (Small Language Model)
• API
• Token Efficiency

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.