AI Dictionary › AI Fundamentals

Red teaming

Red teaming is the practice of deliberately attacking an AI system to uncover its weaknesses before malicious users or real incidents do. A group of people, the "red team", takes the adversary's role and tries every way to make the model fail: getting it to produce dangerous content, bypass its rules, or leak confidential information. It is like hiring professional thieves to test a bank's security: better to find the gaps in a controlled drill than in a real robbery. Techniques range from sneaky questions to disguised prompts to automated attacks generated by other models.

Definition

Red teaming matters because no system is secure by design: flaws surface only under hostile, creative pressure. It is now a mandatory step before releasing a model, and its findings guide later safeguards and fixes.

Related terms

• Jailbreak
• Alignment
• Guardrails
• Data poisoning
• Constitutional AI

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

📱 Download the Android app (beta) iOS coming soon

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.