Sim-to-Real: The Gap Between Practice and the World
Simulation grants a million cheap rehearsals — of a world subtly wrong in a hundred ways. The fix is deliberately sloppy practice: vary the wrongness until only robustness survives.
Builds on: 6.1 Capstone: Teach the Arm by Hand3.2 PID: Three Letters That Run the World
Why robots rehearse in software
Real-robot practice is expensive in every currency: hours per attempt, wear per fall, one robot per experimenter. A physics simulation runs thousands of attempts per minute, in parallel, for free, and a crashed simulated robot is a resurrected one at the next reset. Modern locomotion and manipulation training happens overwhelmingly in simulation — there is no other way to afford the millions of trials learning needs.
The gap
But the simulator is a model, and models are wrong in detail: real friction differs from modelled friction, real motors lag and sag, the real chassis flexes, sensor noise has texture no random-number generator quite matches. A controller tuned to perfection in simulation has effectively memorised the simulator’s particular lies — deployed on hardware, it meets different lies and stumbles. That performance cliff is the sim-to-real gap, and closing it is a discipline of its own.
Domain randomization: sloppy on purpose
The counterintuitive cure: make the simulation less consistent, not more accurate. Each training episode, randomly perturb the physics — friction ±30%, mass ±20%, sensor noise doubled, motor strength wobbled:
A policy that must succeed across a thousand differently-wrong worlds cannot lean on any single world’s quirks; it is forced to find strategies that work because they are robust, not because they exploit a simulator’s particular fiction. When the real world turns out to be one more sample from the spread, the policy shrugs and performs. The trade is honest: peak performance in any one world drops; performance in the unknown world — the only one that matters — soars.
The whole course, in one loop
Notice what closed the gap: not a truer model, but tolerance of model error — the same humility that made the complementary filter work (Unit 2), that made feedback beat open-loop (Unit 3), that inflated obstacles rather than trusting perfect paths (Unit 5). Robotics’ deepest habit is designing for being somewhat wrong. Machines built on that habit are the ones that keep working when the world declines to match the datasheet — which it always, always does.
The result that made this canon: a robot hand trained purely in randomized simulation — gravity, friction, even visual appearance scrambled per episode — solved a Rubik’s cube one-handed on the first hardware deployment. The simulator was never right; it was variously wrong, and that was enough.
⚗️ Lab — The Reality Check
Tune a cart controller in simulation, then hit DEPLOY: it runs on a ‘real’ cart whose hidden physics differ. A randomization slider varies the training worlds.
- Randomization at zero: tune to a perfect sim score, deploy — and watch the score cliff. You have met the gap.
- Raise randomization and retune: sim score drops, deployed score climbs. Find the sweet spot.
- Hit RE-ROLL REALITY a few times at your best setting — robustness means the deployed score stops caring which reality you get.