AI Dictionary › Prompting
Prompt Avversariale
An adversarial prompt is an input deliberately crafted to confuse, deceive, or force an AI model to behave differently from what its safety constraints intend. The term is the umbrella under which many specific techniques fall, jailbreak, prompt injection, prompt leaking, united by the intent to exploit the model's weaknesses rather than use it as designed.
Adversarial attacks on language models exploit the statistical nature of responses: ambiguous phrasing, role-play framing, encapsulating malicious text inside innocuous contexts (translations, poems, code), or character sequences designed to confuse the tokenizer. Unlike classic adversarial attacks on image neural networks, which alter pixels imperceptible to the human eye, those on LLMs work almost entirely at the natural language level, making them harder to catch with simple automated filters.
In modern AI, resistance to adversarial prompts is a standard evaluation criterion before a model ships: red-teaming teams systematically generate thousands of variants to measure how easily protections can be bypassed. Public safety benchmarks also include suites of adversarial prompts to compare different models.
The concept of the adversarial example originated in machine learning applied to computer vision, before large language models became widespread; with the arrival of ChatGPT and similar systems the term naturally extended into the domain of text prompting.
From our network
AGORÀ Intelligence: Enterprise AI Governance Platform
Govern AI at scale: policies, adoption and measurable results on your data. Built for boards and C-suite.
Visit agora-intelligence.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.