Multi-Agent AI Review & Evaluation Pipeline
I designed a review system in which independent Claude reviewers examine every change, a synthesis agent scores and verifies what they find, and the results feed back into how the system runs.
- AT A GLANCE
- Role
- Designed the lanes, scoring, model tiers, fix loop, and hooks
- Used on
- Every batch of a production campaign platform
- Stack
- Claude Code subagents · Sonnet · Haiku · Claude Code hooks · Git
- Outcome
- 12+ false positives stopped; real regressions caught from history
No reviewer grades itself.
Each lane sees a different slice of context and never sees the others’ output. Only the synthesis agent sees everything.
The problem
I build with AI coding agents, and a single agent reviewing its own work misses things, invents problems, or both. I needed reviews I could trust without re-reading every line myself, and a way to tell whether the review system was getting better or just producing more output.
What I designed
I designed the lanes, the context each one receives, the scoring rules, the model tiers, and the fix loop, and ran the system on every batch of a production campaign platform built spec-first with Claude Code.
Scoring findings
The synthesis agent scores every finding, so agreement and evidence matter more than volume.
| Factor | Effect |
|---|---|
| Each additional lane that flags the same issue | +25 |
| Regression, security, or data-loss risk | +20 to +40 |
| Exact file and line with a clear explanation | +10 |
| Style-only preference | capped at 20 |
Scores of 75 and above must be fixed before the next batch, 25–74 are fixed when feasible, and anything lower is logged or dropped. Fixes go through a lighter re-review, up to three passes.
Sizing the review to the change
Small changes run one combined reviewer plus synthesis. Typical features run three lanes plus synthesis. Large integrations and refactors add a Haiku second opinion on the final scored report, a cheap check on the Sonnet scoring.
Learning from the transcripts
I kept a running log of what each lane caught, what it got wrong, and where reviewers agreed, and changed the system based on it:
The synthesis step rechecks each finding against the actual diff. It stopped more than twelve false positives from becoming work.
Reviews of written summaries produced false positives, so reviewers always receive the actual diff.
The history lane added nothing on first batches and all-new files, where it duplicated the bug scan, so it’s skipped there and kept where history exists.
On the first large batch, Haiku agreed with Sonnet on every finding. I kept tracking it across more batches before deciding its role.
Guardrails
A Claude Code hook runs after every agent stop and checks for hard-coded API keys, components that have grown too large, and missing session handoff notes, so problems surface before a batch is called done.
The result
Reviews became something I could act on: fewer invented issues, real regressions caught from history, and a clear rule for what must be fixed. The same pattern carries into my other agent work, separating what a model suggests from what gets verified and accepted.