- Garak and PyRIT help you discover adversarial inputs and conversations that expose risks in an AI application.
- A red-team result becomes useful release evidence after you define the expected safe behavior and rerun the case after relevant changes.
- Braintrust can collect cases from production logs, manual review, and external test results in versioned datasets for ongoing evaluation.
- Run high-severity cases in CI, then compare immutable experiment results to catch known-risk regressions before release.
Red team findings need a durable home
Red team campaigns expose risks at a point in time. A change to a prompt, model, retrieval pipeline, tool permission, or guardrail can make a previously fixed attack work again.
You need a durable record for each confirmed risk. That record should contain the adversarial input or transcript, the expected safe behavior, the source of the finding, and enough metadata to decide when the case must run again.
Discover risk vectors with open-source tools
Garak and PyRIT help you find different kinds of adversarial behavior. Use their results to identify cases that deserve ongoing ownership in your evaluation suite. If you're still choosing a discovery tool, compare the broader landscape in the best LLM red teaming tools for production.
Garak for broad vulnerability scans
Garak runs a broad set of probes against a target model or application and evaluates responses with detectors. Its plugin model helps you test categories such as prompt injection, data leakage, encoding attacks, and package hallucination without building every attack from scratch.
A Garak scan produces a JSONL report and a hit log for attempts scored as successful. Review those hits to distinguish a reproducible product risk from a probe result that needs more context, then add the confirmed case to your ongoing evaluation suite.
PyRIT for custom multi-turn attacks
PyRIT helps you build adversarial campaigns that match the behavior of your application. Its attack configuration can combine targets, prompt transformations, scorers, and adversarial chat targets for single-turn or multi-turn testing.
Multi-turn attacks let an adversarial model pursue an objective through a conversation instead of relying on one prompt. PyRIT documents that workflow and supports scoring the resulting responses, but you still need a separate place to retain the confirmed transcript as a release check.
Score a finding against expected safe behavior
A raw adversarial prompt does not define a useful regression test by itself. You need a scorer that judges the application response against the behavior you expect after the fix.
Use deterministic checks for events with an unambiguous signal, such as a secret in the output or an unauthorized tool call. Use a classifier or an LLM judge with a written rubric when the safe response depends on the conversation, retrieved content, or the actions an agent takes.
The scorer should test the behavior that created the risk. For example, a prompt-injection case should check more than whether the model refuses. It should also check whether the application withholds sensitive context and avoids the tool action the attack was intended to trigger.
Braintrust as the source of truth and ongoing eval runner
Braintrust gives confirmed risk vectors a shared home that persists after the original red team campaign. Datasets in Braintrust can include cases from production logs, manual curation, and imported files, and every change to a dataset is versioned.
Store the attack input or structured transcript with the expected safe behavior. Add metadata such as source, severity, affected feature, model, owner, and the condition that should trigger a rerun. Those fields help you filter the suite and explain why a case exists months after the original investigation.
Production evidence can contribute new cases as well. Braintrust logs let you browse and inspect production traces, while dataset curation turns the useful traces into tests that you can rerun offline.
Braintrust also runs the suite over time. Experiments preserve immutable evaluation results so you can compare a change against earlier runs instead of relying on a one-off scan report.
Turn a confirmed risk into a regression test
1. Record the risk. Add the adversarial input or full transcript to a Braintrust dataset. Include the expected safe behavior and metadata that identifies where the finding came from.
2. Define the scorer. Write a deterministic check, classification rule, LLM judge, or combination that evaluates the application behavior exposed by the attack.
3. Run an evaluation. Execute the case against the prompt, model, agent, retrieval pipeline, or application version that you want to assess. Preserve the result as an experiment so later runs have a concrete baseline.
4. Review production logs. Use production traces and human review to find failures that the original campaign missed. Promote confirmed edge cases into the same dataset rather than keeping a separate incident-only record.
5. Enforce the right cases in CI. Run the highest-severity cases on pull requests that affect the relevant behavior. Run the full known-risk suite for release candidates and larger changes.
Run known-risk evaluations before release
Braintrust can run evaluations in CI/CD and preserve each run as an experiment. A focused pull request suite gives you fast feedback, while a full suite can test broader changes before release.
Use the same scorer and dataset version when you need a direct comparison with an earlier result. When a case changes because the product requirement changed, update the expected behavior deliberately and retain the version history so reviewers can see why the release rule changed.
Divide responsibilities across the workflow
| Activity | Garak | PyRIT | Braintrust |
|---|---|---|---|
| Broad automated scanning | Runs plugin-based probes | Supports custom attack construction | Retains confirmed findings |
| Custom multi-turn attacks | Supports configured probes | Builds adaptive conversations | Stores the transcript as a case |
| Scoring during discovery | Uses detectors | Uses configurable scorers | Defines product-specific regression scorers |
| Ongoing evaluation | Produces scan reports | Produces campaign results | Runs versioned cases and compares experiments |
| Release checks | Requires a separate workflow | Requires a separate workflow | Runs targeted evaluations in CI |
Garak and PyRIT help you discover attacks. Braintrust gives the confirmed attacks a shared source of truth and runs them as ongoing evaluations throughout development.
Keep confirmed failures in the release process
Red teaming produces the most value when confirmed findings remain visible after the campaign ends. Each confirmed risk vector needs a versioned dataset, explicit scorer, and recurring evaluation to retain a durable role in the release process.
Start with one confirmed finding in Braintrust, then expand the suite as red team campaigns, production logs, and security reviews reveal new risks.
FAQs: Operationalizing LLM red team findings
Does Braintrust replace Garak or PyRIT?
Garak and PyRIT are red-teaming tools that generate and assess adversarial attacks. For a fuller comparison of scanners, frameworks, and managed platforms, see the best LLM red teaming tools for production. Braintrust retains confirmed results from either tool as versioned evaluation cases. You can keep your preferred discovery workflow while running the cases that matter as regression checks.
How do production logs contribute to red team evaluations?
Production logs record the requests, traces, and outcomes that occur in a live application. Braintrust lets you review production traces and add useful cases to a dataset for offline evaluation. You can turn a confirmed production failure into a test that runs before future releases.
How does regression testing differ from runtime guardrails?
Regression testing evaluates known cases before you release a change. Braintrust runs those cases against a candidate application version and records the result for comparison. Runtime guardrails handle requests in production, while regression tests show whether the guardrail or application behavior still holds before release.
How often should you rerun a known-risk suite?
Known-risk suites should run after changes that can affect the relevant behavior. Braintrust can run a focused set in CI for a pull request and a larger set for release candidates. Frequent runs help you catch a returning vulnerability when the change that caused it is still easy to review.