HumanLayer Skills: instruction preservation and reviewable control loops
HumanLayer loop throughput: size work to review capacity, not the clock
Measure accepted improvements, open-PR age, retries and human effort before increasing the cadence or batch size of a maintenance agent.
What you will learn
- A faster schedule can produce a slower review queue
- Measure each phase and keep unsuccessful attempts
- Tune cadence only after the loop behaves predictably
Before you start
- Repository instructions and basic GitHub Actions concepts
- Understanding of review scope and persistent agent context
Design a bounded, inspectable workflow and distinguish template assumptions from tested behavior.
Key takeaways
- Accepted improvements are more useful than raw PR counts.
- The template’s open-PR limit does not apply to every manual run.
- Cancellation and rollback are different outcomes.
A faster schedule can produce a slower review queue
The design skill recommends one open PR per loop as a default work-in-progress limit. Its template checks for an open PR with the loop label on scheduled runs and does nothing when one exists. Manual dispatch bypasses that check, so repeated manual runs can still create additional work.
This makes review capacity part of throughput. If new changes arrive faster than reviewers can accept them, a higher cadence grows the queue instead of improving the repository faster. Count accepted, validated changes and their time to review, not merely workflow starts or PRs opened.
Measure each phase and keep unsuccessful attempts
Record sensor time, controller selection, actuator usage, validation, review delay and rework. Keep the chosen scope and acceptance checks stable across comparisons. A low-cost sensor does not make the model actuator free, and a short agent response does not establish a low total operating cost.
For a bounded trial, report no-op runs, failed measurements, rejected changes and memory-related retries alongside successes. Separate elapsed time from total agent and human effort. This series measured no token savings, speedup, convergence rate or service price.
Tune cadence only after the loop behaves predictably
If review is the bottleneck, reduce batch size or pause new work rather than weakening checks. If measurement is unstable, stabilize the sensor before increasing frequency. If the controller repeatedly selects poor targets, update its actual selection policy; feedback seen only by the actuator may not repair the controller.
The skeleton’s concurrency group uses cancel-in-progress. Understand what cancellation means for an in-flight agent, artifact and partial branch before relying on it as a safety mechanism. A cancelled job is not automatically a clean rollback, and we did not run a cancellation experiment here.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Set a review-capacity limit before choosing cadence.
- 2
Measure no-ops, failures, review delay and usage.
- 3
Fix unstable measurement or poor selection first.
- 4
Increase one operating parameter only after a reviewed trial.
Copy-ready example
{
"trialProposal": true,
"scheduledOpenPrLimit": 1,
"manualBypassInTemplate": true,
"acceptedChanges": null,
"reviewDelayHours": null,
"rejectedChanges": null,
"modelUsage": null,
"benchmarkExecuted": false
}Frequently asked questions
Will hourly runs necessarily finish maintenance sooner?
No. Review and rework can dominate throughput.
Does cancelling an agent job undo its changes?
No rollback guarantee was established; inspect partial state and the chosen runner behavior.
Sources
- HumanLayer Skills / plugins/design-control-loop/skills/design-control-loop/SKILL.mdSource checked 2026-09-14
- HumanLayer Skills / plugins/design-control-loop/skills/design-control-loop/references/workflow-template.ymlSource checked 2026-09-14