AI Dictionary › AI Fundamentals

Transformer

The Transformer is the neural network architecture introduced in the paper "Attention is All You Need" (Google, 2017) that revolutionized the field of artificial intelligence and made modern language models possible. GPT, BERT, Claude, Gemini, Mistral, all major LLMs are based on the Transformer architecture.

Definition

The Transformer's breakthrough over previous architectures (RNN, LSTM) is the Attention mechanism: instead of processing text sequentially, the Transformer can analyze relationships between all words in parallel.

Related terms

• Neural Network
• Attention Mechanism
• LLM (Large Language Model)
• Embedding
• Inference

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.