FirstMate: agent crews, durable evidence and delivery authority
FirstMate cost and performance: measure supervision, crew work and integration separately
Interpret zero-token watcher supervision narrowly and design a useful trial that measures accepted work, model usage and human recovery effort.
What you will learn
- A sleeping shell watcher is not a free fleet
- Measure phases instead of publishing an invented speedup
- Optimize only the bottleneck supported by observations
Before you start
- Git worktrees and pull-request basics
- Understanding of terminal agents and credential scope
Define a delivery contract and inspect state and authority evidence without claiming untested runtime guarantees.
Key takeaways
- Zero-token waiting does not mean zero-token agent work.
- Measure accepted deliverables and human repair effort.
- Parallel elapsed time and aggregate resource use are different.
A sleeping shell watcher is not a free fleet
The README describes event-driven zero-token supervision because a Bash watcher can wait without making a model request. When the first mate wakes, reasons, dispatches or reviews work, model usage resumes. Crew tasks, retries and optional supervision-model calls also have their own usage; the watcher claim must not be expanded into zero-cost execution.
Count the work that actually reaches the agreed deliverable. Three simultaneously running agents are not three completed changes. A faster first draft can be offset by review, test contention or integration rework. The relevant outcome is accepted reports or safely delivered changes, not windows opened or tokens generated.
Measure phases instead of publishing an invented speedup
For a bounded trial, record intake, dispatch, active implementation, waiting, review and delivery timestamps. Keep failure and retry counts alongside model usage from the actual host. Separate elapsed duration from aggregate worker time: parallel workers can reduce one while increasing the other.
Use the same task definition and acceptance checks when comparing one worker with a crew. Start with independent tasks; changes to the same API create integration coupling. Record human inspection and recovery minutes as well as compute usage. No throughput, latency or monetary benchmark was executed in this series.
Optimize only the bottleneck supported by observations
If repeated model wakes dominate usage, inspect the wake reasons before reducing monitoring. If tests saturate the machine, adding agents can make completion slower. If integration dominates, improve task boundaries and review order. These are measurement-driven hypotheses, not claims that a particular fleet size is optimal.
The documented Pi supervision branch can select a model and effort separately from the captain conversation. That is a configuration capability, not a promise that a cheaper choice preserves supervision quality. Trial the exact configuration and retain escalation failures in the result rather than hiding them in an average.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Define one acceptance contract for both comparison runs.
- 2
Record phase times, usage, retries and human review.
- 3
Identify the dominant bottleneck from evidence.
- 4
Retest one configuration change at a time.
Copy-ready example
{
"trialProposal": true,
"acceptedDeliverables": null,
"elapsedMinutes": null,
"totalWorkerMinutes": null,
"modelUsage": null,
"humanReviewMinutes": null,
"benchmarkExecuted": false
}Frequently asked questions
How much money does FirstMate save?
This review measured no cost savings; record actual host usage and review effort in your own bounded trial.
Will more workers always finish sooner?
No. Shared tests, coupled changes and integration review can become bottlenecks.
Sources
- FirstMate / README.mdSource checked 2026-09-14
- FirstMate / docs/architecture.mdSource checked 2026-09-14
- FirstMate / docs/configuration.mdSource checked 2026-09-14