loading
loading
Deliberately attacking your own system to find ways it can be made to misbehave before users or attackers do.
You play adversary against your agent — crafting prompts that try to leak the system prompt, bypass safety rules, exfiltrate data via tool calls, or produce harmful output — and log every break so you can patch it. It's testing the failure modes that normal evals never exercise because normal evals assume good-faith input. Example: feeding 200 jailbreak variants and prompt-injection payloads through your tool-using agent and counting how many succeed.
Plainly
Think of Red-teaming as a simple recipe for doing the work better. Deliberately attacking your own system to find ways it can be made to misbehave before users or attackers do.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.