microduck_rl explicado: entornos de RL para Microduck

The short answer

microduck_rl ofrece entornos de reinforcement learning para Microduck sobre mjlab; el éxito en simulación no sustituye la validación de sensores, actuadores y seguridad física.

The repository is most useful as a reproducible experiment surface: environment definitions, simulator settings, reward terms, seeds, and policy checkpoints can be inspected together. That makes it easier to teach the difference between a benchmark result and a control policy that is ready for hardware.

Start reproducibly

Pin the repository, simulator, CUDA/PyTorch stack, and seed. Run the smallest environment or evaluation script first, inspect observation/action shapes and reward terms, and save configuration, checkpoints, telemetry, and videos. Compare wall-clock cost only when parallel environments, physics steps, and rollout length match.

Architecture and deployment boundary

Inspect the environment adapter, mjlab physics loop, and RL runner separately. Keep reward shaping and termination explicit. A fourth deployment boundary covers policy export, control frequency, actuator limits, sensor dropout, watchdogs, and emergency stop; these must be tested before hardware.

When it fits

Use it to learn robot RL and prototype policies in simulation. Do not transfer a checkpoint directly to a robot without qualified operator review and sim-to-real validation under disturbances and saturation.