microduck_rl
microduck_rl 架构：环境、mjlab 物理、奖励与 Runner
跟踪观察、动作、物理、奖励、终止和策略更新。
你将学会
- Map environment, simulator, and runner
- Run a reproducible simulation
- Identify sim-to-real gates
开始前需要
- Basic Git and command-line usage
- Comfort reading a project README
You can explain the project, run its documented first step, and decide what to verify before adopting it.
先看结论
- Environment and runner are different responsibilities.
- Reward/termination define the task semantics.
- Transfer requires a runtime adapter and safety controls.
The control loop
将环境适配、mjlab 物理、奖励/终止、rollout buffer 和优化器分层；任何奖励或重置改变都形成新的实验谱系。
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
Where behavior comes from
Task definitions, reset conditions, contacts, actuator limits, and randomization shape the data seen by the policy. Version each one because a reward change can invalidate a checkpoint comparison.
Transfer surface
Exported policies need a runtime adapter for control frequency, sensors, saturation, watchdogs, and fallback actions. Simulation and hardware telemetry should share a comparable schema.
Use the pinned source evidence, record versions and assumptions, and ask a qualified reviewer to validate the result before adopting it.
如何选择
| 比较维度 | 方案 A | 方案 B |
|---|---|---|
| Best when | You need predictable behavior and easy auditing | You need adaptive optimization and have reliable telemetry |
| Main risk | May leave performance on the table | Can become difficult to explain or debug |
实施步骤
- 1
Map environment and simulator interfaces.
- 2
Trace one action through physics and reward.
- 3
Version resets, randomization, and limits.
- 4
Define export and hardware adapter contracts.
可复制示例
obs -> policy -> action limits
-> mjlab physics -> reward + done
-> rollout buffer -> optimizer
-> export adapter -> watchdog/hardware常见问题
Can reward be changed after training?
Changing reward changes task semantics; treat it as a new experiment and checkpoint lineage.
Why keep termination explicit?
It affects data distribution, safety, and evaluation, so hidden termination logic makes runs incomparable.
资料来源
- microduck_rl repository来源核查 2026-08-31