Cloudflare Security Audit Skill: evidence-led reviews for coding agents
Budgeting an AI-assisted security review
Measure useful coverage and verified records instead of counting raw candidates.
What you will learn
- Define the denominator
- Account for verification work
- Compare repeatable trials
Before you start
- Source review and JSON knowledge
- Explicit target authorization and isolation planning
A proposed learning project keeps source revisions, coverage gaps and verdicts visible.
Key takeaways
- Candidate count is not confirmed yield.
- Blocked coverage remains visible.
- Model and target changes affect comparisons.
Define the denominator
Record the target revision and the number of in-scope coverage units before comparing runs. A larger candidate list may reflect duplicated leads or a broader target rather than better review quality.
Track blocked and deferred units separately from covered work. A run that cannot safely execute a required check should not silently improve its coverage percentage by dropping that unit.
Account for verification work
Model calls, repeated source reading and fresh verification all consume time or provider budget. Keep candidate discovery separate from evidence review so costs can be attributed to actual stages.
Include rejected leads and exact unresolved facts in the evaluation. Reporting only confirmed records hides investigation cost; treating unresolved candidates as confirmed inflates the apparent yield.
Compare repeatable trials
Use the same authorized fixture, source revision and acceptance criteria across experiments. Preserve previous records so changed-source revalidation is distinguishable from genuinely new coverage.
No runtime, token bill or discovery-rate benchmark was measured here. The repository’s reported experience is not a performance guarantee for another model, codebase or environment.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Fix scope and coverage units.
- 2
Record discovery and verification costs separately.
- 3
Keep rejected and blocked work in the accounting.
Copy-ready example
run,source_revision,in_scope_units,covered,blocked,confirmed,rejected,unresolved,cost
trial,,,,,,,,Frequently asked questions
Can I estimate quality from report length?
No. Examine evidence, coverage and verification.
Are savings measured in this series?
No cost or discovery-rate experiment was executed.
Sources
- Security Audit Skill / README.mdSource checked 2026-09-23
- Security Audit Skill / skills/security-audit/validate-coverage-ledger.cjsSource checked 2026-09-23