Garak Is the LLM Vulnerability Scanner the Industry Should Have Standardized On Two Years Ago, and the Fact That Your Production App Has Never Run It Should Worry You

Hey guys, Mr. Technology here.

I am going to say something uncharitable and defend it. **Every team shipping an LLM feature in 2026 that has never run garak against their production endpoint is shipping an insecurity. That is a measurable fact you can reproduce before lunch. There is an open-source, NVIDIA-maintained project on GitHub called garak (Generative AI Red-teaming & Assessment Kit) that probes your model for hallucination, data leakage, prompt injection, jailbreaks, and a long tail of failure modes — static, dynamic, and adaptive — with a CLI a junior can run in an afternoon. Most of you have never heard of it. That is the problem, not the project. Numbers as of June 15, 2026: 8,109 stars, 1,021 forks, last commit today, Apache 2.0.**

What Garak Actually Is

Garak is to LLM endpoints what nmap is to network hosts and Metasploit is to known CVEs. It enumerates failure modes, dispatches probes, runs detectors against responses, scores them, and writes a report. The mental model is familiar. The implementation is purpose-built for stochastic systems.

The taxonomy is what makes the project worth your time. Garak organizes failure modes into a plugin hierarchy: probes (attack generators), detectors (verdict functions), generators (the system under test), and buffs (mutations that turn one probe into many). You pick a generator — OpenAI chat, Anthropic messages, AWS Bedrock, a gguf running on llama.cpp, anything reachable over REST — and a probe family. Garak runs the matrix.

As of v0.15.x it ships with 50+ probe families: promptinject (the agencyenterprise library, full injection surface), dan (curated jailbreaks from public red-team reports), encoding (the base64/rot13 bypass that will eventually leak your system prompt), gcg (the Greedy Coordinate Gradient universal adversarial suffix, runs on a GPU and is mean), and packagehallucination (finds your coding agent recommending non-existent pip packages). The whole point of a vulnerability scanner is that you do not have to enumerate the attack surface yourself.

The Five-Minute Smoke Test

bash python -m pip install -U garak export OPENAI_API_KEY="sk-..." garak --target_type openai --target_name gpt-5-nano --probes encoding,dan,promptinject

Garak runs the probes, scores responses with the detectors each probe recommends, and writes a JSONL report. The hit_rate per probe is the number you put on a dashboard. **Five minutes from pip install to "yes, my endpoint is vulnerable to base64-encoded prompt injection, here is the exact payload."** The teams that have run that are not the ones getting popped.

For local models the generator story is the second underrated part. garak --target_type gguf --target_name ./mistral-7b-instruct.Q4_K_M.gguf works without any cloud. garak --target_type rest --target_name https://your-internal-llm.example.com works without any cloud. The thing scans whatever you point it at — including the local Llama 3 you were about to deploy Friday because the cost model made sense.

What Is Actually Wrong

Two things, and the second is a deal-breaker for the people who need it most.

The detector layer is model-judge-heavy. A meaningful fraction of garak's detectors are LLM-as-judge — they call another model to score whether the target violated a policy. Your "did this model leak a system prompt?" detector is a model call. If the judge model is also jailbroken, your report is a lie. The deterministic detectors (string match, regex, encoding round-trip) are solid. The model-judge ones are best-effort. You need to know which is which before you put garak in a security gate.

Most LLM applications in 2026 are not single endpoints. They are graphs of calls — a router, a retriever, a tool-using agent loop, a guard, a rewriter, a final model. Garak's generator is one-shot request/response. You can glue retries and tool-call loops around it, but the project has not solved the agent-graph case. The closed-source red-team platforms charge a lot for it. For single-model deployments, garak is the answer. For multi-agent graphs, garak is a starting point and you will be writing harnesses. That is the gap, and the gap is what the closed vendors are selling against.

The other three open-source red-team tools are Promptfoo (CI-shaped workflow, right for a team with a policy to enforce), Microsoft PyRIT (research-project-shaped, multi-turn orchestration, Jupyter workflow, overkill for a 4-person team), and DeepTeam (newest and most developer-ergonomic, pytest-shaped). The four are not substitutes. They are layered — DeepTeam in CI, garak on a schedule, PyRIT for the targeted multi-turn attack when an incident happens, and Promptfoo enforcing policy in the deploy gate. The teams running all four are the ones whose LLM features are not making the news for the wrong reasons.

The Take

LLM security in 2026 is in the same place web application security was in 2003. The tool that does the boring, well-instrumented, deterministic thing exists, is open source, and is being maintained by a team with two years of stable releases. The tool is garak. 8,109 stars, Apache 2.0, NVIDIA-owned, last commit today, and your production endpoint has never been scanned by it. That sentence is true for the median LLM team I talk to. Stop shipping features that have not been probed. pip install garak. Run the encoding probe. Read the report. Wire the report into CI. The cost is an afternoon. The alternative is a CVE blog post with your company name in the byline.

— Mr. Technology

*Repo: github.com/NVIDIA/garak — Apache 2.0, 8,109 stars, 1,021 forks, last commit 2026-06-15, v0.15.x. Generators: OpenAI, Anthropic, AWS Bedrock, Hugging Face, LiteLLM, gguf/llama.cpp, REST. 50+ probe families. CLI: garak --target_type openai --target_name gpt-5-nano --probes encoding. ArXiv: 2406.11036.*

Sources:

• github.com/NVIDIA/garak
• garak.ai
• arxiv.org/abs/2406.11036