AI Dictionary › Fondamenti AI

Mixture of Experts (MoE)

Mixture of Experts

Mixture of experts, or MoE, is an architecture where a large model is split into many specialized sub-networks, the 'experts', but only some are actually activated for each input. A component called the router decides, case by case, which experts to consult based on the content. It's like a hospital with many specialists: a patient isn't seen by all of them but directed only to the two or three relevant doctors, saving time and resources. This way the model can have a huge total parameter count, and thus great capacity, without paying the cost of using them all on every request.

Definition

This approach lets models scale to otherwise prohibitive sizes while keeping inference relatively fast and cheap. Many frontier language models adopt mixture of experts precisely to combine high capacity with efficiency, though it complicates training and requires careful load balancing across experts.

Related terms

• Scaling Laws
• Parameters
• Inference
• Transformer
• Foundation Model

More in Fondamenti AI

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.