microduck_rl
microduck_rl Performance and Cost: Measure Rollouts, Physics, and Review
Benchmark simulation throughput, GPU memory, parallel environments, checkpoint storage, and accepted-policy review.
What you will learn
- Map environment, simulator, and runner
- Run a reproducible simulation
- Identify sim-to-real gates
Before you start
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Key takeaways
- Throughput depends on physics and environment count.
- Accepted-policy cost includes evaluation and review.
- Pruning must preserve lineage and safety evidence.
Freeze the workload
Fix simulator revision, task, seed policy, environment count, physics steps, rollout length, algorithm, device, and evaluation protocol. Report training and evaluation separately.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Measure complete outcomes
Record steps/second, p50/p95 episode time, GPU/CPU memory, utilization, reset rate, crashes, checkpoint size, video cost, and reviewer minutes. Compare policies on equal disturbances and seeds.
Optimize carefully
Parallel environments improve throughput until memory or simulator contention dominates. Early stopping and checkpoint pruning save storage, but never remove the evidence needed to reproduce a safety decision.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Freeze task, simulator, seeds, and device.
- 2
Measure rollout, physics, reset, and memory metrics.
- 3
Compare equal evaluation protocols.
- 4
Set budget and regression thresholds.
Copy-ready example
envs x steps -> throughput/p50/p95
GPU/CPU + resets + crashes -> run cost
checkpoint + disturbance eval + review -> accepted policyFrequently asked questions
Is steps/second enough?
No. Include episode success, stability, resets, memory, evaluation, and review time.
Can more parallel environments hurt?
Yes. Contention, memory pressure, and unstable physics can offset throughput gains.
Sources
- microduck_rl repositorySource checked 2026-08-31