microduck_rl
microduck_rl Source-Code Analysis: Follow One Rollout to the Optimizer
Use deterministic fixtures to read environment registration, mjlab adapters, rewards, resets, runners, and checkpoint export.
What you will learn
- Map environment, simulator, and runner
- Run a reproducible simulation
- Identify sim-to-real gates
Before you start
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
Key takeaways
- A single rollout exposes most contracts.
- Seed and device behavior need tests.
- Simulation fixtures are safer than hardware regressions.
Read the repository as a graph
Pin the checkout and locate environment registration, configuration, task/reward definitions, simulator adapter, runner, logging, and export code. The README description is an entry point; verify call paths in source and tests.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Trace a rollout
Follow reset, observation construction, action clipping, physics stepping, reward accumulation, termination, buffer insertion, and optimizer update. Record tensor shapes, devices, and episode counters.
Safe contributions
Add fixtures for reward terms, reset determinism, action bounds, seed parity, checkpoint metadata, and simulator failures without requiring a physical robot.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Decision guide
| Criterion | Option A | Option B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
Implementation steps
- 1
Pin source and dependencies.
- 2
Trace reset-to-update symbols.
- 3
Inspect limits, rewards, and logs.
- 4
Add deterministic fixture tests.
Copy-ready example
fixture -> env registry -> reset/obs
-> policy/action clip -> mjlab step
-> reward/done -> buffer -> optimizer/exportFrequently asked questions
Can I inspect the code without GPU?
Usually yes with static analysis and tiny CPU fixtures; validate GPU-specific paths separately.
What is a useful first issue?
A deterministic test for reset, reward, action bounds, or checkpoint metadata.
Sources
- microduck_rl repositorySource checked 2026-08-31