Microduck software: from command to action

1 · Start with the complete loop

A robot needs more than a neural network. Someone requests a movement, software reads sensors, the policy proposes joint targets, safety checks constrain what can be executed, and the motor interface sends commands. New sensor readings then close the loop.

The Microduck runtime is a Rust workspace: related libraries and services are maintained in one repository. A daemon is a program that runs continuously in the background. The robot daemon, robotd, owns the control loop; other services provide communication, media, configuration, and updates.

The architecture document is explicitly marked as a draft. Use it to understand intent, and follow the accompanying code links to inspect the implementation at the cited revision.

Trace one request through the system

Consider a student pushing a gamepad stick forward. The input service interprets the stick and sends a movement request. The robot service owns the decision about what can be applied. It reads current state, chooses the relevant behavior, runs the policy, and passes proposed targets through safety and hardware handling. The next sensor reading tells it what actually happened.

The user interface, policy, and motor bus are not interchangeable control points. A UI can show that a request was sent without proving that a motor moved. A policy can return valid numbers without knowing that a cable is disconnected. A motor can move while the robot travels in the wrong direction. Each stage needs its own evidence.

A useful debugging sequence is therefore: confirm the request, inspect the applied command, verify fresh observations, inspect policy output, and compare measured motion. Begin at the first disagreement instead of changing several layers at once. This preserves the ability to identify a cause.

Services are separated partly to contain failures. Configuration and recovery paths must remain usable when the movement service is unhealthy. Otherwise the very failure that needs repair could also prevent access to the repair mechanism.

Check your understanding

Explain why “run the neural network” describes only one part of operating the robot.

2 · From a user request to a motor target

A gamepad or application sends an intent, such as a requested forward velocity. It does not directly take ownership of the motor bus. robotd combines commands with current robot state and runs the movement logic.

The duck-control library contains the robot model, sensor interface, policy observations, inference, and safety components. Keeping these responsibilities together makes it possible to check a proposed action before it reaches the motors.

Follow the code in order: control-loop orchestration, observation construction, policy evaluation, safety checks, and bus access. Distinguish the requested command from the command actually applied after limits.

A concrete observation and action contract

The documented observation has 61 values: 3 gyroscope values, 3 projected-gravity values, 14 joint-position offsets, 14 joint velocities, 14 previous actions, and 13 command values. Adding them gives 3 + 3 + 14 + 14 + 14 + 13 = 61. The command block contains movement, head, and body-pose fields; it is not 13 arbitrary labels.

Joint positions are expressed relative to a home pose. Suppose a joint’s home angle is 0.30 radians and its measured angle is 0.35 radians. Its relative position is 0.05 radians. Supplying 0.35 instead changes the meaning of the input even though its numeric type and array length are valid.

The runtime converts a policy action into a target using the home pose and an action scale. For a teaching example, a home angle of 0.30 radians, an action value of 0.10, and scale 0.9 produce a target of 0.39 radians. That target is still a proposal: filtering, limits, and hardware response remain part of the execution path.

The reviewed runtime’s control module includes behavior-dependent scaling and optional filtering of head and leg targets. This matters for reproduction: executing the same ONNX file in a different wrapper does not automatically reproduce the same controller. Read the complete path from observations to applied targets, not just the network call.

Check your understanding

Where would you look if an application requests a speed but the robot applies a smaller one?

3 · Communication and perception are separate responsibilities

robotctl is a command-line client. padd reads gamepad input; btd provides a Bluetooth path. Services exchange structured requests using the shared interprocess communication contract. These clients can request behavior without implementing the control loop themselves.

mediad handles camera and media streaming, while the tof component serves range measurements. configd handles configuration and connectivity. Separating these services helps isolate failures: losing a video stream should not be confused with a policy inference failure.

A camera stream is a view of the robot’s surroundings. The training lab’s animated recording is different: it replays positions saved from a simulation. Neither should be described as live training unless it actually is.

A request is not the same as a measurement

Interprocess communication means exchanging information between separately running programs. A protocol defines the names, fields, and meanings of those messages. A Unix socket is one local communication endpoint; JSON-RPC is the structured request-and-response format described by the architecture.

For example, a movement request can contain a desired speed. A successful response might mean the request was accepted, not that the robot has already travelled a particular distance. To establish motion, inspect later measurements. This distinction also applies to starting training: accepting a job, finishing it, and passing a behavior test are separate events.

Streaming has another timing model. Video frames may arrive late or at a lower rate without changing the motor controller’s intended rate. Conversely, a smooth video stream can show a robot whose control loop is unhealthy. Do not use video smoothness as a proxy for control-loop health.

Shared message definitions reduce the risk that a phone, gamepad, and command-line tool interpret the same field differently. They do not eliminate the need for validation, timeouts, ownership rules, and clear errors when a service is unavailable.

Check your understanding

Which component would you inspect first if movement works but the camera stream does not?

4 · Training: where do model, data, and recipe come from?

Training lives in the separate microduck_rl repository. The model is the policy that maps observations to actions. Its data are generated online: simulated robots try actions, receive new observations and rewards, and supply experience to the learning algorithm.

The recipe defines the task, command distribution, rewards, physical randomization, and learning settings. PPO—proximal policy optimization—is the algorithm that updates the policy from that experience. A reward is a learning signal, not proof that the resulting behavior meets a user’s goal.

The walking task configuration is the best entry point for understanding what the robot practises. Read its reward functions and actuator model next. Increasing training time cannot by itself repair a task definition that rewards the wrong behavior.

What one learning iteration actually does

At each simulated time step, the actor—the policy network—receives observations and samples actions. The simulator advances the robot and returns rewards and new observations. A rollout is a collected sequence of these interactions. Many simulated robots can collect rollouts at the same time.

For a concrete arithmetic example using the lab’s configured rollout length, 4,096 environments each collecting 24 steps produce 98,304 transitions before an update: 4,096 × 24. A transition records one interaction, not an independent video or a complete walking episode. Samples from the same trajectory are correlated.

A critic is a second learned model that estimates future accumulated reward. PPO uses experience and these estimates to adjust the actor, limiting how much action probabilities change in an update. The critic helps training; it is not necessarily part of the exported movement policy. The actor must remain usable with the observations available at deployment.

Rewards express preferences, not hard physical guarantees. If standing earns enough reward and movement carries penalties, standing can be a learned shortcut. If the command sampler rarely asks for slow straight motion, a policy may have little relevant practice. Inspect the actual sampled commands as well as the declared configuration.

Training, development testing, and confirmation have different roles. Training updates weights. Development tests guide model selection and diagnosis. Confirmation tests evaluate a frozen choice on previously unused trials. Repeatedly choosing a model based on the same tests makes those tests development evidence, not independent confirmation.

Check your understanding

How could a policy earn reward while standing still when the intended task is slow walking?

5 · Export, test, and deploy: why the interfaces must match

Export converts the learned model into an ONNX file, a format the runtime can load. The documented movement interface has 61 input values and 14 outputs. Matching dimensions alone is insufficient: joint order, units, default pose, command meaning, and action scaling must agree too.

Test the saved policy before deployment. The upstream CPU inference script exercises a policy in MuJoCo. The daemon-level simulation instead runs robot services against a simulated body, testing more of the software system. Neither substitutes for testing physical drivers and hardware.

Sim-to-real means transferring behavior learned in simulation to a physical robot. It requires compatible interfaces and sufficiently realistic dynamics, followed by careful hardware validation. Our local experiments have not established reliable walking or hardware readiness.

Three tests that answer different questions

First, test the model interface: can the exported model accept the intended observation and return finite actions of the expected shape? Next, test behavior in simulation: does it move, balance, track commands, and stop? Finally, test the integrated runtime and physical hardware under controlled conditions. Passing an earlier test does not establish the later claims.

Imagine a 14-output policy whose first output was trained for the left hip, while deployment sends that value to the right hip. The file can load, inference can succeed, and all 14 numbers can be finite. The robot can still move incorrectly. The contract must specify meaning and ordering, not merely shape.

The simulator also supplies a model of contact, friction, motor response, and timing. A policy trained with delayed or filtered targets can behave differently when deployment applies them immediately. Likewise, a good result in one scene does not automatically transfer to another collision model or floor surface.

A disciplined comparison holds the policy and commands fixed while changing one execution layer at a time. Preserve the exact source revision, model checksum, runtime settings, and raw observations. If the released baseline does not reproduce, diagnose that mismatch before using custom training changes as an explanation.

For a walking test, distinguish requested speed, measured body-forward speed, world displacement, and heading. A robot travelling at the requested speed in its own forward direction may be circling. A proper stopping test must first establish that it moved, then measure whether speed and drift fall after the stop command.

Check your understanding

Why can a model load successfully yet control the wrong joints?

6 · Reproducibility and recovery: how do we know what ran?

Keep the model, robot description, software versions, training recipe, and test results together. Otherwise, a changed simulator or policy file can make two recordings incomparable even when their experiment names look the same.

The updater component manages releases and recovery. The design describes verifying a release, changing the active version, checking health, and rolling back when needed. This is distinct from training a new policy.

For students, the practical lesson is simple: label a recording with the policy and test conditions that produced it. “Training completed,” “software is healthy,” and “the robot walked correctly” answer three different questions.

Make every result traceable and every change recoverable

A source revision identifies a particular version of code. A checksum is a compact value computed from file contents; changing the file changes its checksum with overwhelming probability for the cryptographic hashes used here. Together with a configuration record, these identify what was actually tested more reliably than a descriptive experiment name.

Suppose two students use a file called walking.onnx, but one file was exported yesterday and the other was overwritten today. Their filenames match, yet they are testing different policies. Recording checksums exposes this discrepancy. The same reasoning applies to robot XML, actuator code, and inference settings.

A software update is a deployment operation, not a learning operation. The updater verifies a release and manages activation and recovery. A health check can establish that required services are running or responding; unless it explicitly runs a behavior test, it cannot establish that the new policy walks straight.

For a reproducible report, retain the model identity, source revisions, dependencies, robot and actuator configuration, commands, initial conditions, timing, metrics, and recordings. Record failures too. Selecting only attractive videos makes it impossible for another student to assess reliability.

Check your understanding

What records would another student need to reproduce the movement you just watched?