Why AI red teaming matters in 2026
AI red teaming means deliberately trying to make an AI system misbehave so its weaknesses are found before attackers find them. Five open tools now make that work easier, and this guide compares Promptfoo, Garak, PyRIT, Augustus and DeepTeam for people who are new to the field.
Many AI systems read text from users, documents and web pages, and that text can be crafted to change what the system does.
If you’re new to AI security, the fastest way to start probing AI systems for weaknesses is to pick the red-teaming tool that matches how you already work — a command-line scanner for security-minded readers, a Python library for developers, or a framework for security teams — and this guide compares five open-source options so you can choose in minutes.
This comparison is based on official documentation, GitHub repositories, and published announcements as of October 2026 — we did not run these tools hands-on for this article.
Readers who want more head-to-head comparisons can start at the comparisons hub.
What is AI red teaming? (In plain English)
Red teaming is the practice of testing a system by trying to break it, on purpose and with permission. For AI, that means probing a model or AI app to see whether it leaks data, produces harmful content or ignores its rules. It is defensive work: the goal is to find weaknesses so the owner can fix them.
Some terms first. A jailbreak is a prompt that tries to push a model past its safety rules. Prompt injection hides instructions inside content an AI reads, such as a web page or an email, so the model follows the hidden text instead of the person using it. Data leakage means a model reveals information it should keep private.
Why does this matter? Once an AI system is connected to documents, tools or customer records, one successful attack can cause real damage. Security professionals therefore use shared reference lists. The OWASP Top 10 for LLM Applications names common risks, and the NIST AI Risk Management Framework describes how organisations can manage AI risk. You do not need to memorise them yet, but they are useful anchors when you read tool documentation.
Automation is the practical part. Rather than typing attack prompts by hand, a red-teaming tool sends packaged test inputs, called probes, to a target model and records which ones succeed. Everything in this guide assumes defensive, authorised use: you test systems you own or have permission to test. The ToolBrain tools directory lists the tools themselves.
The five tools at a glance
All five are open source under permissive licences, but they fit different people. Garak and Augustus lean toward scanner-style probe batteries, DeepTeam and PyRIT are code-first Python options, and Promptfoo sits closer to everyday prompt testing. The table lists the facts that matter most for a first decision.
AI red-teaming tools compared (2026)
| Tool | License | Language / install | Interface | Core approach | Best fit for a beginner |
|---|---|---|---|---|---|
| Promptfoo | MIT | TypeScript (Node); npm install -g promptfoo | Command line (per official docs) | Prompt and agent testing extended to red teaming and vulnerability scanning | Beginners comfortable in a terminal, and product teams |
| Garak | Apache-2.0 | Python; python -m pip install -U garak | Command line (per its README) | Probes for hallucination, data leakage, prompt injection, misinformation, toxicity and jailbreaks | Security-minded users who want a scanner-style workflow |
| PyRIT | MIT | Python library; repo archived; moved to microsoft/PyRIT | Python library and notebooks | Open-source framework for identifying risks in generative AI systems | Python users who want code-level control |
| Augustus | Apache-2.0 | Go, shipped as a single binary (see repository) | Command-line binary (per its repository) | LLM security testing for prompt injection, jailbreaks and adversarial attacks; 190+ built-in probes (per its own repository) | Terminal-comfortable security users who want broad built-in probe coverage |
| DeepTeam | Apache-2.0 | Python; pip install -U deepteam | Python code with a model callback | Framework to red team LLMs and AI agents across dozens of vulnerability types | Python developers building custom test pipelines |
All figures per official sources, as of October 2026. This comparison is based on official documentation, repositories, and published announcements. We did not run any of these tools hands-on.
Tool-by-tool breakdown
Each tool below gets a short profile covering what it is, the documented facts behind it, and who it suits. Star counts are GitHub figures read in October 2026 and are shown only where the repository reports them. Scores come later, in the verdict section, and reflect our judgement rather than measured results.
Promptfoo: the generalist that joined OpenAI
Promptfoo is an open-source project for testing prompts, agents and retrieval-augmented apps, which look up documents before answering. Its GitHub repository describes red teaming, pentesting and vulnerability scanning for AI, and reports 26.1k stars under an MIT licence. It is written in TypeScript and installs through npm with npm install -g promptfoo.
On 9 March 2026, OpenAI announced it was acquiring Promptfoo. Promptfoo’s GitHub README now says the project is part of OpenAI and remains open source under the MIT licence. TechCrunch and Promptfoo’s own post cover the same news, and Promptfoo says the open-source project continues. Start with the red-team guide and its quickstart.
Who it suits: non-coders and product teams who are comfortable with a terminal for setup.
Garak: the security-scanner mindset
Garak calls itself an LLM vulnerability scanner. Its README lists probes for hallucination, data leakage, prompt injection, misinformation, toxicity generation and jailbreaks. The project is maintained by NVIDIA, licensed under Apache-2.0, and installs with python -m pip install -U garak. The GitHub repository reports 9.6k stars.
The approach is easy to picture: run a battery of known attack probes against a target model and report what got through. The documentation site explains the options, and the project’s research paper is arXiv:2406.11036.
Who it suits: security-minded beginners who want a scanner-style workflow and are comfortable running Python commands.
PyRIT: Microsoft’s risk-identification framework (original repo now archived)
PyRIT, the Python Risk Identification Tool, is an open-source framework built to help security professionals and engineers find risks in generative AI systems. The original Azure/PyRIT repository was archived in March 2026, and the project moved to the microsoft/PyRIT repository, which reports 4.6k stars under an MIT licence.
PyRIT is library-first: you write Python and often work through notebooks, although the repository also includes frontend and Docker files. The official docs are the right starting point, and you should use the microsoft/PyRIT location rather than the archived one.
Who it suits: Python users who want code-level control and can handle a steep learning curve.
Augustus: the probe-heavy challenger
Augustus describes itself as an LLM security testing framework for detecting prompt injection, jailbreaks and adversarial attacks. Its repository reports 190+ probes (per its own repository), and it ships as a single Go binary under Apache-2.0. The GitHub repository states that the tool is designed for authorised security testing only.
Augustus is maintained by Praetorian, an offensive security vendor. It has 307 stars at the time of reading, the smallest count of the five, so expect fewer community answers and check the repository for current instructions before you start.
Who it suits: terminal-comfortable security users who want broad built-in probe coverage and accept a smaller community.
DeepTeam: the code-first framework for LLMs and agents
DeepTeam describes itself as a framework to red team LLMs and AI agents. Its README cites dozens of vulnerability types and a set of guardrails. The GitHub repository reports 3.0k stars under Apache-2.0, and the tool is built on DeepEval. It runs locally and installs with pip install -U deepteam.
Your first run is code-first: you write Python and supply a model callback, which is the function DeepTeam calls with test inputs. The getting-started guide is the official starting point. Confident AI sells a separate paid platform for managing results, but it is optional.
Who it suits: Python developers who want to script their own red-teaming checks inside a pipeline.
Verdict: which should you pick?
For most beginners, Promptfoo is the easiest starting point in our judgement, Garak is the next step for scanner-style checks, and the other three suit narrower needs. Choose by who you are and what you already use, then start small and deliberately.
Not a programmer, but comfortable in a terminal: Promptfoo. In our judgement, it is the most approachable of the five for someone who does not write code daily, and the OpenAI announcement plus the project’s stated continuation suggest it is well supported. Read the red-team guide before running anything, and only against systems you are authorised to test.
Python user: PyRIT or DeepTeam. Both give you code-level control. Use the current microsoft/PyRIT repository for PyRIT, because the original Azure/PyRIT repository is archived. DeepTeam is the quicker first step for a Python developer, because its getting-started guide runs a complete red_team() example after one pip install.
Security professional: Garak and Augustus. Both are scanner-style tools built around probe batteries. Augustus offers breadth, with 190+ probes (per its own repository), while Garak brings a workflow backed by NVIDIA maintenance and a published paper. Augustus’s lower beginner-fit score in the verdict reflects its smaller community, not its capability. Use either only on systems you are authorised to test.
Team or organisation: Promptfoo or Garak. Choose Promptfoo when your priority is product evaluations you repeat during development, and Garak when your priority is dedicated security scanning of models. Either way, agree in writing who is authorised to run tests before anyone starts.
Beginner-fit scores, ranked, reflect our judgement rather than measured results:
- Promptfoo: 8/10 (our judgement). The clearest starting point for most beginners.
- Garak: 6/10 (our judgement). A simple scanner concept, but it needs Python commands.
- DeepTeam: 5/10 (our judgement). Code-first, with a single-command install.
- Augustus: 4/10 (our judgement). Terminal-based, with a smaller community.
- PyRIT: 3/10 (our judgement). A framework with the highest learning curve of the five.
Whichever you choose, start with one tool, one model and one risk category, such as prompt injection.
FAQ
These four questions cover common starting points. The short answers below point back to the other sections of this guide; the official documentation remains the final reference for the tool you choose.
Is red teaming my own AI app legal?
Testing a system you own, or a system whose owner has given you written permission, is standard practice, and every tool here is built for that purpose. Never probe a system you are not authorised to test, because unauthorised testing can be illegal. When in doubt, ask the owner first and get the scope in writing.
Do I need to know how to code?
Not necessarily, but you should be comfortable in a terminal. Promptfoo is installed with npm and Garak with pip, so a few commands are unavoidable, while DeepTeam and PyRIT require writing Python. Start with the tool that matches your comfort level, and learn basic Python before moving to code-first options.
What’s the difference between a ‘probe’ and a ‘prompt’?
A prompt is any single input you send to a model, such as a question or instruction. A probe is a packaged test that sends one or more prompts designed to trigger a specific failure, such as leaking a secret or ignoring a rule.
Which tool should I install first?
Install Promptfoo first if you are new, following its quickstart with npm. Use one model you are allowed to test, pick one risk category such as prompt injection, and run only the tests for that category. Once you understand the output, read the Garak documentation for a scanner-style follow-up. The verdict section explains the other picks.
Related reading and next steps
Once you have picked a tool, these pages help you go further. Start with the comparison hub for more head-to-head reviews, then use the tools directory and roadmap to plan a learning path. The arena hub covers this matchup series, and the network site below covers adjacent topics for builders.
- More head-to-head comparisons: the rest of the comparison hub.
- A learning path for AI security: a sequenced route from basics to hands-on-safe practice in authorised settings.
- The ToolBrain Arena: the hub for this matchup series.
- CodeIntel: code-quality and CI perspectives on evals, if you want to take testing into your build pipeline.
📖 Related Reads
- ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
- NiteAgent — AI agent development, frameworks, and production patterns
Cross-links automatically generated from None.
Back to all posts