Skip to content
Hochschule für Informatik FHNW P9 — Mars Rover Autonomy

Demo · runs in your browser

SmolVLA + RECAP + guidance, client-side

The policy family from contribution 01, solving PushT with no backend at all: Chipmunk2D physics compiled to WASM, policy inference on WebGPU through onnxruntime-web. Watch it push the T onto the goal, drag the agent away and let the policy recover, and move the guidance weight to see what it changes — and what it does not.

Everything the frame below prints, and every figure on this page, was measured on pusht_recap6 — the checkpoint the demo serves. They are read from the evaluation outputs rather than copied, so the page cannot go on quoting a policy it has stopped running.

Before you start it

The demo runs on your machine, not on ours

Physics, rendering and policy inference all happen in this tab. There is no backend to queue behind and nothing of yours leaves the browser — but it is your hardware doing the work, so it is worth knowing what it is being asked for before the frame starts.

Nothing loads until you press it. Open it in its own tab instead, if you would rather have the full window.

Precheck

Graphics
The real policy wants WebGPU. Checking…
Download
1.58 GB of ONNX weights, from the Hugging Face Hub, and only if you pick the real policy. It is cached afterwards, so a second visit is quick.
Costs nothing
The stub policy, the physics, the arena and the drag-to-take-over. Enough to see what the task is before committing to the download.

The demo is a separate build on its own origin, embedded here. recap-demo.fhnw-rover.ch — so it can be opened standalone, and so a page of thesis prose never ships alongside a gigabyte and a half of model weights.

What you are looking at

A PD-controlled agent pushes a rigid T onto a target pose. The policy sees a 96×96 image and a state vector — its own position plus the eight corners of the T, read straight out of the physics engine. That last part is privileged information a real robot would not have; it is here because it is what this checkpoint was trained with, and because the browser is the simulator, so handing it over is honest in a way it would not be on hardware.

Its action is a target position — sixteen of them per chunk. That detail is what makes the demo legible: because an action is a position, a sampled action chunk can be drawn straight onto the arena as a trajectory. The fan you see is sixteen independent draws from the flow-matching policy, all conditioned on one frozen observation.

Dragging inside the arena is therefore a valid action, not a special case — take over mid-episode, put the agent somewhere awkward, release, and watch the policy resume from a state it never saw in training.

Worth being precise about

Guidance does not visibly narrow the fan. The spread of sampled headings is the same at high w as at zero — the intuition that guidance collapses a distribution comes from language-conditioned image models and does not carry over to a single advantage token.

What the fan does show is how uncertain the policy is at this particular state, which varies a great deal between starting positions. Read it that way rather than as a readout of the slider.

What it actually scores

Guidance weight, 100 pre-registered starting states

  • w = 0 0.524 ± 0.037 · cleared the bar 6/100 · paired against the shipped weight, −0.073 ± 0.041 (1.8 sem)
  • w = 1 shipped 0.598 ± 0.039 · cleared the bar 10/100
  • w = 1.5 0.550 ± 0.040 · cleared the bar 9/100 · paired against the shipped weight, −0.048 ± 0.037 (1.3 sem)
  • w = 2 0.523 ± 0.041 · cleared the bar 8/100 · paired against the shipped weight, −0.075 ± 0.041 (1.8 sem)

The curve is flat. Nothing in that table is two standard errors from anything else, so the ordering of the point estimates is not a result — it is what a sweep of this size looks like when the control does not move the policy. The demo starts at w = 1 because that is the weight this policy was measured at and the cheap end of the curve — w of zero or one needs one prefix branch through the model, anything between them needs two. Not because it is a peak.

Scores are best coverage of the target pose divided by PushT's 0.95 success bar. That bar is not one the human demonstrations reach reliably either, so a plain success rate would read as near zero for every policy in this literature — which is why the count of episodes that cleared it is beside each row rather than instead of it. 10% of starting states are ones this policy makes no progress from at all, and two of those are among the presets in the demo on purpose.

The comparison that matters

0.896 vs 0.598

MLP + flow matching, state only (0.57 M parameters) against the policy this page serves, on the same 100 starting states and the same demonstrations. It clears the 0.95 bar 49 times to this policy's 10, a paired difference of 0.298 ± 0.045 in its favour (6.6 sem).

It trains in minutes. The demo is worth opening for what the policy family does — language conditioning, advantage conditioning, a chunk you can watch being denoised — not because it wins PushT. It does not.

Browser against Python

On 50 starting states run identically in both, the browser reaches 0.478 against Python's 0.599 — −0.121 ± 0.058, or 2.1 standard errors.

More than sampling alone comfortably explains, and the page says so rather than rounding it away. The two sides draw their own denoising noise, so they follow different trajectories from the same start; what is checked exactly is the action chunk itself, which matches Python's to 9e-05 arena pixels on identical inputs. The browser ran on WebGPU.

The presets in the demo

The start dropdown is not a random sample. These are six of the evaluated episodes, picked to span the outcomes — including two the policy gets nowhere from — so a couple of minutes with the demo shows the range rather than whichever start the seed happened to give. Each carries the reward that evaluation run actually recorded.

Elsewhere