AI Dictionary › AI Fundamentals

Transformer

The Transformer is the neural network architecture introduced in the paper "Attention is All You Need" (Google, 2017) that revolutionized the field of artificial intelligence and made modern language models possible. GPT (Generative Pre-trained Transformer), BERT, Claude, Gemini, Mistral, all major LLMs are based on the Transformer architecture.

Definition

The Transformer's breakthrough over previous architectures (RNN, LSTM) is the Attention mechanism: instead of processing text sequentially, the Transformer can analyze the relationships between all words in parallel, capturing long-range dependencies in the text with extreme efficiency. This makes it possible to handle much longer texts and capture complex semantic relationships.

Understanding the Transformer at a conceptual level, not necessarily the mathematical details, helps professionals see why AI models excel at some tasks (language reasoning, translation, summarization) and struggle with others (precise numerical calculation, spatial reasoning).

Related terms

• Neural Network
• Attention Mechanism
• LLM (Large Language Model)
• Embedding
• Inference

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.