Sim-to-real and system identification

Stage 6 · Toward hardware · ~35 min · Before this: 04 Actuators, 15 PD control, 16 RL for walking · numbers from the repository as of 2026-11-02

Marvin Bot’s walking policies live in a simulator whose motors are educated guesses. Before they can run on metal, the guesses have to be measured, and the policy has to be trained to cope with whatever the measurements still get wrong. This page is about that gap and the two tools for closing it: system identification and domain randomisation.

You will learn

• What the sim-to-real gap is made of for a legged robot: actuator dynamics, latency, friction, contact, sensor noise, mass errors.
• How system identification turns bench data into model parameters with least squares, and why the test signal matters.
• How domain randomisation, actuator networks and privileged training cover what identification cannot.
• What Marvin Bot’s phase 2 bench will measure, and where each number goes back into the simulation.

The idea in one picture

A simulator is a set of equations with numbers in them: masses, inertias, friction, motor strength, delays. A policy trained in it learns to exploit those exact numbers. If the real robot’s numbers differ, the policy’s actions have different effects than it expects, and a gait that was balanced becomes unbalanced. That difference is the reality gap.^[1][4] Two things shrink its effect: measure the real numbers and put them in the simulator (system identification), and vary the numbers in training so the policy works across a range that contains the real ones (domain randomisation).^[2][3]

An everyday comparison: a learner who has only ever driven one car stalls a different car with a heavier clutch. You fix that either by telling them exactly how this clutch feels (measure it), or by having them practise on many different cars, so no clutch surprises them (randomise). Real projects do both. This is the step between the simulation work of pages 14–17 and building hardware (page 22, phase 2).

How to read the picture below: each panel is a range of values for one physical parameter (say, a motor’s friction). The blue shape is the range of values the simulator trains on; the orange line is the real robot’s value. Left: the guess misses reality. Middle: measuring moves the guess onto it. Right: a wider range covers it even if the measurement is a little off.

Playground: identify a motor, then deploy

You play the engineer at the phase 2 test bench. Hidden inside the page is a “true” motor (one joint with a test arm bolted to its output), standing in for the real hardware. In three steps:

1. Test it. The bench sends the motor a pattern of current (current makes torque) and records its speed, with sensor noise: the grey trace in the top chart. A chirp is a wiggle that gets faster and faster.
2. Match it. Move the five model sliders until your model’s speed (blue) lies on the measured one, or press Fit to let least squares do it (it also tries every delay from 0 to 20 ms and keeps the best). The red residual under the chart is the mismatch: you want it as small as the noise.
3. Use it. The bottom chart runs a leg-like motion on the joint with a PD controller that relies on your model to know how much torque to add. It compares what the simulator predicted with what the “real” joint does. A poor model shows up as a gap between the two.

Model and bench are a teaching toy: one joint, no gravity, no backlash; the true values are illustrative, not RobStride measurements.

Presets

Marvin Bot today (placeholders) Datasheet, no delay No friction model Fitted

Bench test

Test signal Steps then chirp (4 s)Current steps only (2 s)Chirp 0.5–8 Hz only (2 s)

The output turns freely; the motor is given a current directly, with no position control (k_p = k_d = 0).

Speed noise σ

How noisy the speed sensor is (typical size of the random error).

Fit (least squares)New hidden motor

Reveal the true values

Model parameters

Inertia J (output side)

How hard the joint is to speed up: the motor's spinning parts as felt through the gearbox (armature) plus the 0.012 kg·m² test arm.

Viscous friction b

Drag that grows with speed, in N·m per rad/s.

Coulomb friction τ_c

Constant drag whenever the joint moves (gear teeth, seals), in N·m.

Command delay d

Time from sending a command to the torque appearing, in ms.

Torque constant K_t

Output-side N·m per A; RobStride's RS04 figure is 2.1.

Deploy controller

k_p

Marvin Bot's training gains: 40 ankles, 150 hips and knees.

k_d

k_p is the joint's stiffness (spring), k_d its damping (see page 15).

Speed residual (RMS)

–

Noise floor

–

Torque residual (locked)

–

Worst parameter error

–

Tracking error, sim predicts

–

Tracking error, real

–

Sim-to-real gap (RMS)

–

Real peak current

–

measured (noisy)your modelreferencereal jointresidual / error

Reading the numbers: a good model has a speed residual close to the noise floor (you cannot do better than the sensor's own noise), a small worst parameter error, and a small sim-to-real gap, meaning the simulator's prediction and the real joint agree. The straight-line chart under the speed chart is a separate test with the output clamped, which measures K_t (torque per amp) from the slope.

Try this

1. Start from “Marvin Bot today”: the speed residual is many times the noise floor, and in the deploy panel the real joint lags and overshoots what the simulator promised. Press Fit: the residual drops close to the noise floor (the fit is slightly biased by the smoothing, as real fits are) and the gap nearly vanishes.
2. Choose “Current steps only” and press Fit, then “Chirp only” and Fit. Which test pins down the inertia better, and which the friction? (Steps reach steady speeds, where only friction balances the torque; the chirp keeps accelerating, which is where inertia shows.)
3. After fitting, raise k_p to 200 with k_d at 2 and watch the mini playground below with “Marvin Bot today” ranges: the worst corner, with a 10 ms delay, jumps from about 2° to about 11° of tracking error while the true joint (4 ms) stays near 0.1°. Delay matters more the stiffer the controller.
4. Raise the noise to 0.3 rad/s and fit several times with “New hidden motor”: estimates scatter more. That scatter is a fair width for the randomisation ranges below.

Mini playground: does your randomisation cover reality?

Pick a training range for each parameter. The bar shows the range, your model value (blue tick) and the hidden true value (orange; known only in this toy). If the truth lies outside, the policy has never trained on the real robot. The worst-case column runs the deploy controller on all 32 corners of the box.

Presets

Marvin Bot today Model ±10% Model ±30% Very wide

True values covered

–

Range width (geo. mean)

–

Worst corner tracking error

–

Real tracking error

–

randomised rangemodel valuetrue value

Try this

Press “Marvin Bot today”: only the delay is randomised (0–10 ms), so the four motor parameters are point values and the truth is outside all of them. Press “Model ±30%” with the placeholder model: friction is still not covered, because the placeholder has no viscous term at all. Now fit the model first and try “Model ±10%”: everything is covered with a narrow box. Wider boxes cover more but cost performance (the worst corner gets worse): a randomised policy is a compromise across its whole range.

From first principles

Where the gap comes from

Source	What the simulator assumes	What reality does
Actuator dynamics	an ideal torque source, or a DC-motor curve with fixed armature and friction	current loop bandwidth, gear friction that changes with load and temperature, backlash, torque constant off the datasheet, and torque that drops as the windings heat[5]
Latency	commands act instantly	inference time, CAN round trips and sensor filtering add milliseconds; a delay in a feedback loop removes phase margin[6]
Contact and friction	rigid bodies, one friction coefficient	soft soles, varying floors, slip[4]
Sensors	exact joint angles, exact orientation	noise, bias, IMU drift, encoder quantisation
Mass and geometry	CAD masses and inertias	cables, fasteners, glue, battery position: the real robot is heavier in places the CAD forgot

Tan et al. traced most of the gap on a small quadruped to two items: the actuator model and latency. Identifying the motor and adding the measured latency to the simulator was what made their gaits transfer.^[6] Hwangbo et al. found the actuator so hard to model analytically that they learned it instead.^[5]

A model of one joint

The playground’s actuator obeys Newton’s second law for rotation, the same model the bench step test fits:

J · dω/dt = K_t · i(t − d) − b · ω − τ_c · sign(ω)

• J (kg·m²): inertia at the output. A geared motor’s rotor inertia appears multiplied by the gear ratio squared: this “reflected inertia” is what simulators call armature. For a 9:1 gearbox a rotor of 4×10⁻⁴ kg·m² looks like 0.032 kg·m² at the output, larger than many leg links.
• K_t (N·m/A): torque constant at the output; current i times K_t is the motor torque.
• b (N·m·s/rad): viscous friction, proportional to speed (oil, eddy currents, back-EMF in the current loop).
• τ_c (N·m): Coulomb friction, a constant drag opposing motion (gear teeth, seals). Units check: every term is N·m.
• d (s): the delay between commanding a current and the torque appearing, including bus time.

System identification by least squares

The idea in plain words: write down the motor equation once for every sample of bench data, then pick the values of J, b and τ_c that make all those equations as nearly true as possible at once. That is what “least squares” means: the smallest total of squared mismatches.

The model is linear in its unknowns once the delay is fixed: rearranged, each sample k gives one equation

K_t i(t_k − d) = J · α_k + b · ω_k + τ_c · sign(ω_k)

where α_k is the acceleration estimated by differentiating the smoothed speed. Thousands of samples give thousands of equations in three unknowns; least squares picks the J, b, τ_c that minimise the sum of squared equation errors, by solving the 3 × 3 “normal equations” (AᵀA)x = Aᵀy.^[7] The delay enters non-linearly, so the Fit button repeats the solve for every delay from 0 to 20 ms in 0.5 ms steps and keeps the best. K_t cannot be separated from J, b and τ_c using speed data alone (doubling all four changes nothing), so it comes from a separate locked-output test: hold the output against a load cell, step the current, and fit the slope of torque against current.

Two practical lessons. First, the test signal must excite every parameter: steps reach steady speeds (where torque balances friction), chirps sweep frequencies (where inertia dominates). Second, noise matters little if you have many samples, but bias matters a lot: if the model lacks a term that reality has (backlash, say), least squares will distort the other parameters to compensate.

Mitigations, from cheapest to most involved

• Observation noise and delay modelling: add realistic noise and latency to the simulated sensors and commands.^[6]
• Domain randomisation: sample physics parameters per episode so the policy must work across all of them. Peng et al. randomised dynamics such as masses, friction, damping and latency and gave the policy memory so it could adapt within an episode^[2]; Tobin et al. introduced the idea for camera images^[3]. Wider is more robust but more conservative.^[8]
• System identification: measure, then centre the randomisation on the measurement.^[6][7]
• Actuator networks: record a real actuator’s response to many commands and train a small network that predicts torque from the recent history of position error and velocity; use it inside the simulator in place of the ideal motor.^[5]
• Privileged training and adaptation: a teacher policy sees the true parameters in simulation; a student learns to act from sensor history alone, implicitly estimating them.^[9][10] History-conditioned humanoid policies do this in context.^[11]
• Learning the residual from real data: for humanoids, ASAP trains a correction (“delta action”) model on real-world rollouts and fine-tunes the policy in the corrected simulator.^[12]
• Sim-to-sim checks: run the trained policy in a second, independent simulator before hardware; Humanoid-Gym uses MuJoCo for this.^[13] A bipedal stair policy trained with randomisation transferred to hardware blind in Siekmann et al.^[14]

How Marvin Bot does it

In short: today the simulator’s motor numbers are educated guesses, softened by some randomisation and a second-simulator check. Phase 2 buys two real motors, measures them on a bench and puts the measured numbers back into the simulator, before most of the motors are bought. That order means a surprise costs two motors, not twenty-three.

In the repo

Phase 2 in docs/plan.md: “One RS04 and one RS03, CAN adapter, bench supply with regen clamp. Measure torque constant, speed, heating, CAN timing, built-in position control; fit actuator models and feed them back into simulation”, pass check “manual figures confirmed; models match measurements; CAN at the planned rates works on the Orin”, 2–3 weeks. Scripts in bench/actuator/bench_robstride.py (untested until the motors arrive), procedure in bench/README.md, driver in robot/drivers/robstride.py.

Today: placeholders, a cross-check and some randomisation

• Motor model: robot_cfg.py uses a DC-motor model per joint (torque saturates at the peak rating and falls linearly to zero at no-load speed) with gains, armature and friction that its docstring calls “placeholders until the phase 2 bench test fits them”: hip and knee k_p 150, k_d 5, armature 0.03 kg·m², friction 0.3 N·m; ankles 40, 2, 0.02, 0.2.
• Delay: motor commands arrive 0, 5 or 10 ms late (DelayedDCMotor), sensor observations 0 or 20 ms late, sampled per episode.
• Noise: gyro ±0.2 rad/s, gravity vector ±0.05, joint angles ±0.01 rad, joint speeds ±0.5 rad/s; the height map gets drift, holes and 10 Hz updates.
• Randomisation: ground friction 0.4–1.0 (static), torso mass ±15% and centre of mass ±3 cm, pushes to 0.5 m/s, random upper-body motion. Joint armature, friction, gains and motor strength are not randomised.
• Asymmetric critic: the critic sees the true velocity and terrain; the actor only real sensors (page 16).
• Sim-to-sim: every policy is exported to ONNX and replayed in MuJoCo (sim/mujoco/check_policy.py) on the current URDF, a different physics engine than training (Newton).

Phase 2: the bench tests and where each result goes

Test	Command (from bench/README.md)	Measures	Feeds back into
Scan	bench_robstride.py scan	motor IDs, wiring	CAN map in description/robot.yaml
Latency	latency –id 1 –model RS04 –rate 500	command-to-feedback round trip at 500 Hz on the Orin: median, p99 (the delay 99% of round trips beat) and max	the delay range in robot_cfg.py; whether USB-CAN adapters suffice (roadmap items 4, 13)
Step	step –id 1 –model RS04 –kp 150 –kd 5	position steps of ±0.3 rad with the training gains, logging position, speed, torque	armature, friction and effective kp/kd in robot_cfg.py and the MuJoCo checks
Thermal	thermal –id 1 –model RS04 –torque 47 –minutes 10	temperature holding 47 N·m (the slow 18 cm stair knee load, 118% of rated) against a locked output; stops at 90 °C	the knee decision (roadmap item 11), the heat penalty’s time constant (sustained_torque)
By hand	dial indicator, scales, calipers	backlash, mass, flange dimensions	description/actuators.yaml (verified: bench)

The step test uses the motor’s own position mode (operation control with k_p and k_d, robstride.py type 1 frames), which is how the robot will run, so it identifies the motor and its built-in controller together. The playground uses pure torque commands instead because the maths stays linear; the driver supports that too (the thermal test sends k_p = k_d = 0 with a feed-forward torque). One caution: the torque in RobStride’s feedback frame is computed by the driver from current, so checking K_t needs an external reference such as a load cell or a known weight on a lever.^[15]

Open items that are sim-to-real questions

• Roadmap item 2: output-bearing type, backlash and moment ratings.
• Item 3: bus current and regen behaviour.
• Item 4 and 13: real CAN timing with 3–5 motors per bus on the Orin, and whether a dual-channel adapter carries two 500 Hz buses.
• Item 11: knee and hip-roll heating on slow stairs.
• Stair robustness: the stair policy (rough v9, trained on the earlier 1.29 m body) passes all 25 stair runs (5–20 cm, five noise seeds) in simulation on the current model, but two retrains on newer bodies did worse (rough v10, v11), so its margin is not well understood; the real robot will not do better than simulation (page 16).

Trade-offs and alternatives

Approach	For	Against	Marvin Bot
Analytic model + identification	few parameters, interpretable, cheap	misses effects the model lacks	phase 2 plan
Actuator network[5]	captures what the model misses	needs lots of bench data over many conditions; harder to run in Newton’s GPU kernels	not planned; an option if the fit leaves large residuals
Wide randomisation only[2]	no bench needed	conservative gaits, and the truth may still be outside	partly (contact, mass, delay)
Teacher-student / history[9][11]	adapts online	more training, more compute on the Orin	planned for a depth-camera stair student
Real-data residual (ASAP)[12]	fixes what remains after transfer	needs a robot that already roughly works	later, after phase 4

The bench also chose its own scope: one RS04 and one RS03 are bought singly, and the remaining RS03 are ordered only after the bench passes (docs/decisions.md, 2026-10-26), so the motor models are known before most of the money is spent. (An earlier decision named an RS06 as the second bench unit; it was later replaced by the RS03 when the ankles went back to RS03.)

Where sources and the repo disagree

• The README and decision log describe the delays as ranges (0–20 ms, 0–10 ms); the code samples discrete values (0 or 20 ms; 0, 5 or 10 ms). Published practice, for example Tan et al., models the measured latency itself^[6]; the bench latency test will say which values matter.
• Many published humanoid recipes randomise joint friction, armature or motor strength as well as contact and mass^[13]; Marvin Bot does not yet, by choice, until the bench gives it a centre to randomise around.

Decisions you can make here

• Which tests the bench must include to identify each parameter (a chirp for inertia, steady speeds for friction, a locked-output load cell for K_t).
• Whether the fitted motor model is good enough (residual near the noise floor) or an actuator network is needed.
• How wide to randomise each motor parameter around the fit: the fit’s scatter over repeated tests and over temperature is a reasonable minimum. Wider ranges make the policy more robust but more cautious (the worst-corner error in the mini playground grows).
• Which delay range to train with once the latency test reports median and p99 round trips.
• How stiff the deploy gains should be given the measured delay: stiffer joints track better when the delay is small, but a delay hurts a stiff joint more (Try this, step 3).

Your decision noteCopy

Built from your settings above. Paste it into an issue, a design discussion or an AI assistant session.

Further reading

• MIT 6.832 Underactuated Robotics, System Identification chapter: least squares on models linear in their parameters, and what goes wrong.
• Tan et al., Sim-to-Real: Learning Agile Locomotion for Quadruped Robots: the clearest account of which parts of the gap mattered.
• Hwangbo et al., Learning agile and dynamic motor skills: actuator networks, with the data collection described.
• Muratore et al., Robot Learning from Randomized Simulations: A Review: the design space of randomisation.
• He et al., ASAP: humanoid sim-to-real with a learned correction from real data.

Sources

1. W. Zhao, J. P. Queralta, T. Westerlund, Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey, arXiv 2009.13303, 2020 — used for: the reality gap and mitigation families.
2. X. B. Peng et al., Sim-to-Real Transfer of Robotic Control with Dynamics Randomization, ICRA 2018 (arXiv 1710.06537) — used for: dynamics randomisation with a memory policy.
3. J. Tobin et al., Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, IROS 2017 (arXiv 1703.06907) — used for: origin of domain randomisation.
4. N. Rudin et al., Learning to Walk in Minutes Using Massively Parallel Deep RL, CoRL 2021 (arXiv 2109.11978) — used for: friction, push and noise randomisation in GPU-parallel locomotion training.
5. J. Hwangbo et al., Learning agile and dynamic motor skills for legged robots, Science Robotics 2019 (arXiv 1901.08652) — used for: actuator networks, actuator dynamics as a main gap.
6. J. Tan et al., Sim-to-Real: Learning Agile Locomotion For Quadruped Robots, RSS 2018 (arXiv 1804.10332) — used for: actuator identification, latency modelling, randomisation.
7. R. Tedrake, Underactuated Robotics (MIT 6.832), chapter “System Identification” — used for: least-squares identification of models linear in parameters.
8. F. Muratore et al., Robot Learning from Randomized Simulations: A Review, Frontiers in Robotics and AI 2022 (arXiv 2111.00956) — used for: robustness versus conservatism of randomisation.
9. J. Lee et al., Learning Quadrupedal Locomotion over Challenging Terrain, Science Robotics 2020 (arXiv 2010.11251) — used for: privileged teacher-student training.
10. A. Kumar et al., RMA: Rapid Motor Adaptation for Legged Robots, RSS 2021 (arXiv 2107.04034) — used for: online adaptation from history.
11. I. Radosavovic et al., Real-World Humanoid Locomotion with Reinforcement Learning, Science Robotics 2024 (arXiv 2303.03381) — used for: humanoid sim-to-real with a history-conditioned policy.
12. T. He et al., ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills, arXiv 2502.01143, 2025 — used for: humanoid delta-action correction from real data.
13. X. Gu et al., Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer, arXiv 2404.05695, 2024 — used for: sim-to-sim validation in MuJoCo, humanoid randomisation practice.
14. J. Siekmann et al., Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning, RSS 2021 (arXiv 2105.08328) — used for: biped stair policy transferred with randomisation.
15. RobStride, Product_Information (user manuals), GitHub — used for: torque constant, CAN protocol and feedback frame.