AI Dictionary › AI Fundamentals

Adapter Layers

Adapter layers are small additional modules inserted inside a pre-trained model to specialize it for a new task, without modifying the model's original weights. They are an alternative to full fine-tuning, designed to drastically reduce the number of parameters that need training while still achieving good adaptation.

Definition

How it works

During training, the base model's weights stay frozen, and only the adapter parameters, which are much smaller in size, get updated. These modules are inserted at specific points in the architecture, typically after the main blocks, and learn targeted transformations that adjust the model's behavior for the new task.

Applications

They are useful when handling many different tasks with the same base model, since they allow a separate, lightweight adapter to be kept for each task instead of maintaining full copies of the model. They are related to more recent techniques such as LoRA, which share the same goal of parameter-efficient adaptation.

History & etymology

They were proposed as an efficient transfer learning method, adapting large-scale models to new tasks while reducing computational and memory costs compared to full fine-tuning.

Related terms

• LoRA (Low-Rank Adaptation)
• Transfer Learning
• Fine-tuning
• Parameters
• Foundation Model

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.