AI Dictionary › Modelli AI
Architettura Decoder-only
A decoder-only architecture is a configuration of an attention-based model that uses exclusively the decoding part of the original encoder-decoder architecture, processing text in a single stream where each token can only look at the tokens preceding it, never those that follow. It is the architectural configuration underlying most modern generative language models.
Unlike an encoder-decoder architecture, which separates understanding an input sequence from generating an output sequence, a decoder-only model treats input and output as a single continuous stream of tokens, trained to predict each next token based on all the preceding ones. This constraint, called causal or masked attention, prevents a token from "seeing" future information during training, making the model naturally suited to both understanding and generating text.
It is the architecture chosen by the vast majority of large language models intended for generative tasks, from conversation to writing code, thanks to its conceptual simplicity and the ease with which it can be trained on enormous amounts of text simply by predicting the next token in a continuous stream.
The name distinguishes this configuration from the original encoder-decoder one and from encoder-only configurations, used mainly for text understanding tasks: "decoder-only" indicates that the model uses only the decoding block, the one designed to generate sequences one element at a time.
From our network
INDACO TMS: Transport Management for European Logistics
Shipment tracking, multi-carrier EDI and automated invoicing in one cloud platform. Invoices generated in under 10 seconds.
Visit indacotms.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.