AI Dictionary › AI Fundamentals

Positional Encoding

Codifica posizionale

Positional encoding is the mechanism that tells a transformer the order of words in a sequence. Unlike recurrent networks, a transformer processes all tokens simultaneously and on its own couldn't distinguish 'the dog bites the man' from 'the man bites the dog'. Positional encoding fixes this by adding to each token numerical information about its position, like putting an order number on each train car so the model knows which comes first and which later. This information, often based on periodic mathematical functions or learned during training, is added to the token representations.

Definition

Without positional encoding, a language model would lose all notion of sequence, and with it grammar and meaning. It's a simple but essential ingredient of transformers, and its modern variants affect how well a model handles long texts and generalizes to contexts larger than those seen in training.

Related terms

• Transformer
• Attention Mechanism
• Token
• Context Window
• Embedding

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.