Stawamus · software

Stawamus Software Map

Every piece of the deck software, in plain words, with real code. As of 2026-10-19 evening. Code quoted from main (1e883b33); the RLSTREAM part from branch worktree-agent-aff90aca332d08452 (ee2198e6), marked with a tag.

1 · Overview

What the software does, in one paragraph

The deck is a grid of columns (44 × 50 on the 5 m² unit, 48 mm apart). Each column has a cap on top and moves up and down. Parcels lie on the caps. The software moves parcels by tilting the caps under them into a gentle slope, so the parcel slides where we want. 100 times a second it does the same five things: look (camera), work out where every parcel is (perception), decide where each parcel should go next (planner), compute a height for every column (controller), optionally add a small extra motion under soft bags (bag motion layer), and send the heights to the deck.

The same software/ code runs in the simulator and on the real deck. sim/ only replaces two things: the deck (MuJoCo physics instead of motors) and the camera (a simulated sensor instead of a real one). A test makes sure software/ never imports anything from sim/.

software/ (same code in sim and on the deck)sim/ stand-inreal hardware

The registry is filled when a parcel is inducted (scanned and put on a belt). The planner gets its size, mass and kind through flow.add(); the tracker only accepts camera measurements for ids that are in the registry. The controller also receives the tracked parcels directly (not drawn).

2 · Repo map

Where things live

One repository, stawamus. Six top folders. Python for the product and simulator, C++ for the firmware.

software/

The product. Runs unchanged on the real deck. Never imports the simulator.

  • interface/ the Deck contract: DeckInfo, ParcelState, Deck
  • perception/ camera image → parcel poses → Kalman tracker
  • planner/ where each parcel goes: flow.py, traffic/, stuck/, mapf/, planner.py, bulk.py
  • control/ column heights: controller.py, geometry.py, primitives.py, bag_motion.py
  • policies/ learned policies as plain numpy, plus frozen/ weight files
  • link/ PC ↔ firmware: RealDeck, bench tool, CAN, health

sim/

The simulator and everything around it. Replaces the deck and the camera.

  • deck/ SimDeck (MuJoCo), bag models, sensors, motor model
  • experiments/ run loop (harness.py) and scenarios (s10.py stream)
  • rl/ RL training: bag/, control/, routes/, traffic/ (stream/ on the RLSTREAM branch)
  • tools/ one script per study (runs, tables)
  • cloud/ launch, watch and collect cloud VMs
  • config/ deck configs (YAML)
  • tests/ pytest suite
  • viz/ interactive sandbox control panel

firmware/

C++ for the STM32G474. core/ is all behaviour (no chip-specific code), board/ the chip glue, host/ runs the same core on the Mac for tests.

hardware/

The Tile-20 board (hardware/tile20): KiCad files, BOM, layout audit, bring-up notes.

studies/

One folder per question: README with the answer, a rule written before the results, result tables. E.g. RLBAG, CAMERA, BAGFIX, CURRENT.

docs/

Pages for the docs site (docs/pages, e.g. bench tests and BOMs), the spec, the wire contract (CONTRACTS.md), research notes, print files.

3 · Layers

The Deck interface

The contract between "the brain" and "the deck". The planner and controller only ever talk to these types. SimDeck and RealDeck both implement them, so the brain cannot tell which deck it drives.

software/interface/__init__.py

ParcelState: what the camera pipeline says about one parcel (position, angle, speed, size). Never the simulator's truth.

software/interface/__init__.py:55-69
@dataclass(frozen=True)
class ParcelState:
    """Tracked planar parcel state, exactly what the real camera pipeline provides."""

    id: int
    x: float
    y: float
    yaw: float
    vx: float
    vy: float
    omega: float
    length: float  # along parcel local x
    width: float  # along parcel local y
    height: float
    mass_estimate: float

DeckInfo: the fixed facts about the deck (grid, pitch, stroke, chutes).

software/interface/__init__.py:102-113
@dataclass(frozen=True, eq=False)
class DeckInfo:
    """Static deck description handed to the controller and planner."""

    grid_shape: tuple[int, int]  # (nx, ny)
    pitch: float
    cap_size: float
    stroke: float
    rest_height: float
    max_speed: float  # nominal column speed (m/s)
    column_xy: np.ndarray  # (n, 2) column centres, index = ix * ny + iy
    chutes: tuple[Chute, ...] = ()

Deck: four calls. Send heights, read parcels, read the clock, advance time.

software/interface/__init__.py:196-210
    @abstractmethod
    def set_heights(self, heights_m: np.ndarray, t_apply: float) -> None:
        """Command target heights (n,) in metres, to be applied at deck time t_apply."""

    @abstractmethod
    def get_parcels(self) -> list[ParcelState]:
        """Tracked parcel states from perception (never ground truth)."""

    @abstractmethod
    def now(self) -> float:
        """Current deck time (s)."""

    @abstractmethod
    def step(self, dt: float) -> None:
        """Advance by dt seconds (simulation) or wait/no-op (real hardware)."""

The run loop

One function drives every tick: read parcels, ask the planner for goals, ask the controller for heights, send them, step. In the sim, "step" runs physics; on the real deck it waits for the next 10 ms.

sim/experiments/harness.py (run_episode)

sim/experiments/harness.py:199-215
    n_ticks = int(math.floor(duration / dt + _TIME_EPS)) if duration > 0 else 0
    for _ in range(n_ticks):
        t = deck.now()
        if on_tick is not None:
            on_tick(deck, planner, t)
        parcels = deck.get_parcels()
        goals = planner.update(t, parcels)
        if recorder is not None:
            recorder.capture(goals)
        out = controller.compute(t, parcels, goals)
        deck.set_heights(out.heights, t)
        deck.step(dt)
        log.record(deck, out, parcels, goals, planner)
        if until is not None and until(deck, planner, log):
            log.stopped_by_until = True
            break
    return log.finalize(planner)

Perception

Turns camera images into parcel positions. Two steps: find outlines in the image, then smooth them over time with a tracker.

software/perception/outline_pipeline.py, tracker.py, types.py (also apriltag_pipeline.py for tagged parcels)

Measurement: one sighting of one parcel in one frame, stamped with the time the picture was taken.

software/perception/types.py:13-27
@dataclass(frozen=True)
class Measurement:
    """One planar pose observation of a parcel, stamped with its capture time.

    Optional: the observed footprint (length along the measured yaw, width across it; None = the sensor does not
    see the outline, e.g. Mode A / AprilTags) and quality flags (FLAG_*)."""

    id: int
    t_capture: float  # s, deck time at which the image/sample was taken
    x: float
    y: float
    yaw: float  # rad, wrapped to (-pi, pi]
    length: float | None = None  # m, observed footprint along yaw
    width: float | None = None  # m, observed footprint across yaw
    flags: int = 0

Outline pipeline: image → mask (what is not empty deck) → blobs (rectangles) → match each blob to a known parcel.

software/perception/outline_pipeline.py:306-321
    def process(self, image: np.ndarray, t_capture: float, tracks: Iterable = (),
                hints: Iterable[Expect] = ()) -> list[Measurement]:
        """segment -> blobs -> associate on one camera image (`timing` accumulates the wall time per stage)."""
        t0 = time.perf_counter()
        mask = self.segment(image)
        t1 = time.perf_counter()
        self.last_blobs = self.blobs(mask)
        t2 = time.perf_counter()
        meas = self.associate(self.last_blobs, self.candidates(t_capture, tracks, hints), t_capture)
        t3 = time.perf_counter()
        tm = self.timing
        tm["frames"] += 1
        tm["segment_s"] += t1 - t0
        tm["blobs_s"] += t2 - t1
        tm["associate_s"] += t3 - t2
        return meas

Tracker: a Kalman filter per parcel. In plain words: it keeps a best guess of position and speed, plus how sure it is. Each new sighting nudges the guess; a noisy sighting nudges it less. Because each sighting carries its capture time, a late picture corrects the past, and the guess is then projected forward to "now".

software/perception/tracker.py:1-7
"""Per-parcel constant-velocity Kalman filter with timestamped latency compensation.

State s = [x, y, yaw, vx, vy, omega]. Process model: constant velocity driven by
piecewise-constant white acceleration (sigma `accel_noise` m/s² on x/y,
`yaw_accel_noise` rad/s² on yaw). Measurements observe (x, y, yaw) at `t_capture`,
so a delayed measurement corrects the state at the time it was taken and
`estimate(t)` predicts forward to any later time.
software/perception/tracker.py:279-296
    def update(self, measurements: list[Measurement]) -> None:
        """Fuse measurements (any order within the batch); stale or non-finite ones are skipped."""
        valid = [m for m in measurements if m.id in self.registry
                 and all(map(math.isfinite, (m.t_capture, m.x, m.y, m.yaw)))]
        for m in sorted(valid, key=lambda m: m.t_capture):
            tr = self._tracks.get(m.id)
            if tr is None or m.t_capture - tr.t > self.timeout:
                self._tracks[m.id] = self._init_track(m)
                continue
            if m.t_capture - tr.t < 0:
                continue
            if self.fp is not None and m.flags:
                self.stats["partial"] += bool(m.flags & FLAG_PARTIAL)
                self.stats["merged"] += bool(m.flags & FLAG_MERGED)
            new = self._fuse(tr, m)
            if new is None:
                self.stats["reinit"] += 1
            self._tracks[m.id] = self._init_track(m, tr) if new is None else new

Registry: the tracker only accepts ids it knows. The sizes come from the induction scan.

software/perception/types.py:30-38
@dataclass(frozen=True)
class ParcelInfo:
    """Parcel registry entry: dimensions known from the induction scan."""

    id: int
    length: float
    width: float
    height: float
    mass: float

Planner

Decides, for every parcel, what it should do this tick: move, hold, deliver to a chute or turn. It also runs and stops the infeed belts. Its output is one Goal per parcel.

software/planner/flow.py (FlowOrchestrator), planner/traffic/ (TrafficOrchestrator: route choice, locks, jam breaker, FOLLOW rule), planner/stuck/ (stuck parcels), planner/mapf/ (lattice path-finding variant), planner/planner.py (the original single-parcel planner), control/goals.py

software/control/goals.py:8-23
GoalMode = Literal["velocity", "goto", "hold", "deliver", "twist", "idle"]


@dataclass(frozen=True)
class Goal:
    mode: GoalMode
    # velocity: desired planar velocity (m/s), deck frame
    velocity: tuple[float, float] = (0.0, 0.0)
    # goto / hold: target position (m), deck frame
    target: tuple[float, float] | None = None
    # deliver: destination chute id (DeckInfo.chutes)
    chute_id: int | None = None
    # twist: +1 = counter-clockwise, -1 = clockwise
    twist_dir: int = 1
    # optional per-goal speed cap (m/s); None = control.max_speed
    max_speed: float | None = None

What it decides:

software/planner/flow.py:1-7
"""High-density flow orchestrator for singulated belt infeeds (stage B, reports/DESIGN/BRIEF.md 4.2).

Pure numpy (planner/ is checked by tests/test_purity.py). `FlowOrchestrator` is the episode's planner in S10
(experiments/s10.py): the harness calls `update(t, parcels) -> goals` every control tick, exactly as it calls
`Planner.update`. The scenario tells it which parcel is on which belt and where (`add`, `observe_belts`: what a
belt PLC knows from its photo-eyes and encoder), reads its belt commands (`belt_commands`: run or stop each belt
at its stop line), reports chute-mouth photo-eyes (`set_chute_blocked`) and removes parcels once they have left

A parcel enters the planner when it is loaded on a belt, with its registry size and kind:

software/planner/flow.py:1567-1582
    def add(self, pid: int, infeed: int, chute: int, length: float, width: float, height: float, mass: float,
            t: float, yaw: float | None = None, kind: str | None = None) -> Route:
        """A parcel was loaded on belt `infeed`, bound for `chute` (L >= W as it lies; yaw = its belt yaw; kind:
        'box' or a bag kind, used by the adaptive tubes' classes only)."""
        lane = self._choose_lane(int(infeed), int(chute))
        route = self.route_for(infeed, chute, length, width, height, yaw, lane=lane)
        f = _Flow(int(pid), int(infeed), int(chute), float(length), float(width), float(height), float(mass),
                  float(t), route, lane=lane, kind=kind)
        f.tube = self.prm.tube_of(kind, length, mass)
        f.heavy = self.is_heavy(kind, length, mass)
        if f.heavy and self.prm.heavy_a_lat is not None:
            set_clock(route, self.prm, corner_radius=self.prm.corner_radius, a_lat=self.prm.heavy_a_lat)
        f.hx, f.hy = half_extents(length, width, self.infeeds[int(infeed)].yaw if yaw is None else yaw,
                                  math.radians(self.prm.yaw_allow_deg))
        f.s = -float(self.infeeds[int(infeed)].length)
        self.flows[int(pid)] = f
software/planner/traffic/params.py:156-164
    # studies/CURRENT: the FOLLOW rule's liveness fix (studies/RLX FOLLOW, rl_routes mode "follow"; no effect without a
    # FOLLOW pair). A FOLLOW leader is not checked against its own follower's reservation while that reservation lies
    # entirely behind the leader's centre on the leader's first-leg line and the leader's remaining route only moves
    # ahead along that line and away from it: the leader advancing can then only widen the gap. The follower keeps its
    # full-envelope check against the leader (both placement orders), so the pair keeps the full crossing envelope as
    # before. Without it, a follower placed first (jam breaker, priority inversion, heavy first) and closer than the
    # full envelope (e.g. after the leader slid or tipped back) holds its leader for good: a wait-for 2-cycle the jam
    # breaker cannot open (the follower's rear is still on the belt). False (default): unchanged.
    follow_live: bool = False

Controller

Turns each Goal into column heights. For each parcel it builds a "field" (usually a tilted plane under the parcel), then merges all fields into one height per column.

software/control/controller.py, primitives.py (plane, cup, brake, twist, deliver), geometry.py (footprints, which caps lie under a parcel)

The tick: smooth each parcel's state, build its field, compose, then limit how fast columns change hands.

software/control/controller.py:665-687
        fields: list[_Field] = []
        smooth: list[ParcelState] = []
        for p in ps:
            st = self._states.setdefault(p.id, _State())
            pf = self._filter(p, st, dt)
            smooth.append(pf)
            goal = goals.get(p.id) or Goal("idle")
            fields.append(self._window_field(pf, goal, st, dt))
            if self.ball_pred:  # the tilt just commanded acts over the next delay (`_predict`)
                st.pred_hist.append(st.tau.copy())
                while len(st.pred_hist) > self.pred_n:
                    st.pred_hist.popleft()
            st.age += dt
        ps = smooth
        self._touching = None
        heights, owner, conflicts = self._compose(ps, fields)
        if self._lost:
            heights = self._lost_apply(ps, heights)
        if self.lost_hold > 0.0:
            self._last_ps = {p.id: p for p in ps}
        heights = self._transition_slew(ps, heights, owner, dt, self._touching)
        if self.rehome_sink_n > 0:
            heights = self._rehome_sink(t, ps, heights)

Tilt plane: the plane is centred on rest height (60 mm) and pushed back inside the 0-120 mm stroke when it would not fit. Each column's height is the plane's height at its cap corner.

software/control/primitives.py:360-373
    lo = float(base) + floor
    span_h = t * (s_hi - s_lo)
    room = info.stroke - lo
    if front_low:
        mid = lo + span_h / 2.0
    elif span_h > room:
        mid = lo + room / 2.0
    elif low:
        mid = lo + span_h / 2.0
    else:
        mid = float(np.clip(centre, lo + span_h / 2.0, info.stroke - span_h / 2.0))
    h0 = mid + t * (s_hi + s_lo) / 2.0 + float(offset)
    raw = h0 - t * s
    return np.clip(raw, 0.0, info.stroke) if clip else raw

Support, keep-out and owners: each column goes to the nearest parcel that claims it. A column inside one parcel's keep-out ring (15 mm in current_30v) is never given to a parcel farther away. The result owner says which parcel owns each column; the bag motion layer uses it.

software/control/controller.py:2336-2355
        U = np.array([support_mask(p, info, 0.0) for p in ps])  # caps under each footprint
        self._accommodate(ps, fields, U)
        W = np.array([f.w > 0 for f in fields])  # (P, n)
        D = np.array([point_rect_distance(info.column_xy, p) for p in ps])  # (P, n)
        near = np.array([support_mask(p, info, self.keepout) for p in ps])  # caps within keepout of footprint
        conflicts = int(np.count_nonzero(W.sum(axis=0) > 1))
        cand = W.copy()
        for i in range(len(ps)):
            for j in range(len(ps)):
                if i != j:
                    # a column in j's keep-out ring is never given to a parcel farther from it than j
                    cand[i] &= ~(near[j] & (D[j] < D[i]))
        Dm = np.where(cand, D, np.inf)
        best = np.argmin(Dm, axis=0)  # ties -> lowest parcel id (sorted order)
        has = cand.any(axis=0)
        cols = np.nonzero(has)[0]
        H = np.array([f.h for f in fields])
        heights[cols] = H[best[cols], cols]
        ids = np.array([p.id for p in ps])
        owner[cols] = ids[best[cols]]

Footprint: with the camera's tracked footprint on, a parcel is planned with its measured size plus a safety margin.

software/control/geometry.py:285-290
def tracked_footprint(p: ParcelState, k: float) -> tuple[float, float]:
    """(length, width) to plan with: the tracked footprint (fp_length / fp_width; the induction length / width when
    the tracker does not track it) grown by k x fp_margin on every side."""
    g = 2.0 * float(k) * float(p.fp_margin)
    return ((p.length if p.fp_length is None else p.fp_length) + g,
            (p.width if p.fp_width is None else p.fp_width) + g)

Re-homing guard (part of the camera collapse fix, off in today's configs):

software/control/controller.py:446-451
        # rehome_sink_n: a column that starts a re-homing run (a known dead column at or below rest height appearing
        # in the dead-column cache: a StallGuard / encoder homing) rehome_sink_n times within rehome_window s while a
        # parcel's footprint covers it is commanded to 0 (dead-low) until no footprint covers it: under a heavy box it
        # otherwise climbs back into the box after every homing, slips again and the box sinks into a pit of
        # re-homing columns (0 = off).
        self.rehome_sink_n = int(g("rehome_sink_n", 0) or 0)

Bag motion layer

An optional extra on top of the controller, only under soft bags. It adds a small up-and-down motion to the bag's own columns to break the bag loose. One config switch: control.bag_motion = off (default), dither or rl.

software/control/bag_motion.py; how-to in software/README.md

software/control/bag_motion.py:5-14
Modes
  off     the controller as it is.
  dither  a random height offset per cap, N(0, bag_motion_sd) (default 6 mm), new every decision (DECIDE_EVERY = 5
          control ticks: 50 ms at the 100 Hz loop), from a generator seeded with bag_motion_seed; on the 9 x 9 cap
          patch around the bag (policies.bagmotion.patch_cells), as the RLBAG "dither" arms.
  rl      the frozen skill pair of studies/RLBAG/FROZEN.json, deployed as trained: the landing skill
          (policies/frozen/rlbag_land.npz) while the bag's centre is less than LAND_IN = 0.35 m inside the edge of the
          belt it comes from, the moving skill (policies/frozen/rlbag.npz) after that; action = mean + learned std x
          N(0, 1) from a generator seeded with bag_motion_seed (policies.bagmotion.BagMotion, stochastic). The default
          files are checked against FROZEN.json's hashes when loaded.
software/control/bag_motion.py:276-281
    def from_params(cls, params: dict | None) -> "BagMotionParams":
        p = dict(params or {})
        raw = p.get("bag_motion", "off")
        mode = "off" if raw is None or raw is False else str(raw).strip().lower()
        if mode not in MODES:
            raise ValueError(f"control.bag_motion {raw!r}: one of {MODES}")

The "own" mask: it may only move columns the controller gave to this bag, never within one cap pitch of another parcel (now, and where that parcel will be in 0.25 s and 0.5 s).

software/control/bag_motion.py:165-171
def own_mask(info: DeckInfo, p, owner: np.ndarray, ko: dict[int, np.ndarray], tracked: bool) -> np.ndarray:
    """(n,) the columns the motion may move for bag p: the ones the controller assigned to it (a bag not tracked yet,
    still on its belt: the caps under its estimated footprint that nobody owns), minus every other parcel's keep-out."""
    mine = owner == p.id
    if not mine.any() and not tracked:
        mine = support_mask(p, info, 0.0) & (owner < 0)
    return mine & ~others_keepout(info.n, ko, p.id)

Limits: every column it moves stays within 0.4 m/s and 4 m/s² (the 30 V drive), and inside the stroke.

software/control/bag_motion.py:245-256
def limit_step(target: np.ndarray, prev: np.ndarray, vel: np.ndarray, dt: float, vmax: float, amax: float
               ) -> tuple[np.ndarray, np.ndarray]:
    """One tick of a velocity- and acceleration-limited move from `prev` (moving at `vel`) toward `target`: the speed
    that can still stop at the target (v dt + v^2 / 2a <= distance), capped at vmax, reached within amax dt of the
    previous speed. Returns (new targets, their speeds)."""
    e = np.asarray(target, float) - prev
    d = np.abs(e)
    v_stop = amax * (np.sqrt(dt * dt + 2.0 * d / amax) - dt)
    v_want = np.sign(e) * np.minimum(vmax, v_stop)
    v0 = np.clip(vel, -vmax, vmax)
    v = np.clip(np.clip(v_want, v0 - amax * dt, v0 + amax * dt), -vmax, vmax)
    return prev + v * dt, v

Wiring: with off it returns the controller unchanged, so nothing changes. Sim and real deck build it the same way.

software/control/bag_motion.py:465-480
def wrap_controller(ctl: Any, info: DeckInfo, params: dict | None, *, flow: Any = None,
                    measured: Callable[[], np.ndarray] | None = None, kind_of: Callable[[int], str | None] | None = None
                    ) -> Any:
    """The deck's controller with control.bag_motion applied: `ctl` itself when off (nothing constructed), else a
    BagMotionController around it. The real deck and the simulator both build it this way, from the same control
    params; `flow` (the orchestrator) gives the parcel kinds and the belt observations, `measured` the column heights.
    """
    prm = BagMotionParams.from_params(params)
    if prm.mode == "off":
        return ctl
    src = flow_sources(flow, info) if flow is not None else {"kind_of": kind_of or (lambda pid: None)}
    if kind_of is not None:
        src["kind_of"] = kind_of
    rate = float((params or {}).get("rate", 100.0))
    return BagMotionController(ctl, info, prm, src["kind_of"], src.get("belt_items"), src.get("belts", ()),
                               measured, rate)

RL policy runner

Runs a trained policy on the deck. Plain numpy, no training library. Every 50 ms it looks at a 9 × 9 patch of caps around the bag and outputs a small set of motion numbers (extra tilt, a travelling wave, edge lifts).

software/policies/bagmotion.py; frozen weights software/policies/frozen/rlbag.npz, rlbag_land.npz; record studies/RLBAG/FROZEN.json

software/policies/bagmotion.py:281-287
    def heights(self, t: float, info, p, is_bag: bool, measured: np.ndarray, commanded: np.ndarray,
                goal: Goal) -> np.ndarray:
        """One control tick for parcel p (the deployed loop)."""
        if self.due(p.id):
            obs = self.observe(t, info, p, is_bag, measured, commanded, goal)
            return self.commit(info, p, commanded, goal, t, self.act(obs))
        return self.hold(info, p, commanded, goal, t)

"As trained" means the policy adds its learned random spread on purpose. On the deck that spread is a small dither.

software/policies/bagmotion.py:237-243
    def act(self, obs: np.ndarray) -> np.ndarray:
        if self.mlp is None:
            return np.zeros(self.act_dim)
        a = self.mlp(obs)
        if self.std is not None:
            a = np.clip(a + self.std * self.rng.standard_normal(self.act_dim), -1.0, 1.0)
        return a

The deck refuses a weight file whose fingerprint (sha256) does not match the frozen record:

software/control/bag_motion.py:60-63
FROZEN_DIR = Path(__file__).resolve().parents[1] / "policies" / "frozen"
REPO = Path(__file__).resolve().parents[2]
# studies/RLBAG/FROZEN.json: the deployed skill pair (file, sha256[:16]); a default file that differs is refused
FROZEN = {"move": ("rlbag.npz", "0306ab8794216c45"), "land": ("rlbag_land.npz", "d667e57598287ea6")}

RLSTREAM branch: a new mode structd = the 8 motion numbers plus a 9th, a dither the policy can switch on and size. It also sees neighbours and who owns which column.

software/policies/bagmotion.py:55-70 RLSTREAM branch
ACT_DIMS = {"cap": PATCH * PATCH, "struct": 8, "structd": 9}
DITHER_MAX = 0.012  # m: structd's dither amplitude at a = 1 (sd of the per-cap N(0, 1) draw)
STREAM_CH = 5  # structd: patch channels of the stream block
N_STREAM = 8  # structd: scalars of the stream block
NEAR_M = 0.5  # m: "nearby" other parcels (the count; only these are drawn into the patch)
ACT_DIM = ACT_DIMS["cap"]
MEM_S = 1.0  # s of track history for the mean speed
MOVE_TOL = 0.015  # m: the track "moved" (the stream's jam watch)


def obs_dim(mode: str = "cap") -> int:
    """Patch channels, scalars, last action (struct: + the wave phase's cos / sin; structd: the struct observation +
    the stream block)."""
    if mode == "structd":
        return obs_dim("struct") + 1 + STREAM_CH * PATCH * PATCH + N_STREAM
    return 3 * PATCH * PATCH + N_EXTRA + ACT_DIMS[mode] + (2 if mode == "struct" else 0)

Firmware

Runs on the STM32G474 next to the drivers. It receives height frames, applies each at its "apply at" time, limits each column's speed, and drives the TMC5240 stepper drivers over SPI. It homes columns by StallGuard (feels the motor stall at the bottom stop), latches the E-stop, and goes flat if no valid frame arrives for 200 ms.

firmware/core/ (protocol, scheduler, motion, tmc5240, health), firmware/board/, firmware/README.txt

firmware/core/protocol.h:1-13
// Wire protocol between the Mac and the column firmware (USB CDC; 921600 baud on a UART).
//
// Two frame kinds share the host -> MCU byte stream:
//
//   height frame (binding, deck/real_deck.py, byte-identical):
//     0xAA 0x55 | seq u16 | t_apply_ms u32 | n u8 | heights u16[n] (0.01 mm) | crc16
//   command frame (bring-up / control channel):
//     0xAA 0x5A | cmd u8 | tag u8 | len u8 (<= 64) | payload[len] | crc16
//
// and one kind flows MCU -> host:
//
//   reply frame:
//     0xAA 0x5B | cmd u8 | tag u8 | status u8 | len u16 | payload[len] | crc16
firmware/core/scheduler.h:1-10
// Frame scheduler: the firmware side of deck.real_deck.FirmwareModel, decision for decision.
//
//   receive(frame, t_rx): accept iff n matches and seq is newer than the last accepted seq
//                         (uint16 serial arithmetic, RFC 1982); an accepted frame feeds the
//                         watchdog, clears the tripped state and is queued for t_apply.
//   update(now):          if no valid frame for watchdog_ms (or none yet): tripped, pending
//                         dropped, last seq forgotten (a restarted host may resume from any seq),
//                         target = rest. Otherwise the due frame (t_apply <= now) with the latest
//                         t_apply becomes the target (ties: the newest received); all due frames
//                         leave the queue.

Simulator

SimDeck implements the same Deck interface with MuJoCo physics. Each 10 ms tick it runs many small physics steps (1 ms; 0.5 ms with soft bags), feeds the motor model, and lets the simulated camera take pictures.

sim/deck/sim_deck.py (SimDeck), flexbag.py (soft bags), sim_sensor.py (camera modes), actuator_model.py (motors, lost steps)

sim/deck/sim_deck.py:1201-1224
        for _ in range(int(round(dt / ts))):
            if motor is None:
                d.ctrl[:n] = act.update(d.time, ts)
            else:
                self._motor_substep(act.update(d.time, ts), ts)
            if pusher:
                d.ctrl[n] = self._pusher_tick(ts)
            if belts:
                self._belt_tick()
            if kinetic:  # friction.kinetic: mj_step split around the per-contact friction (`_kinetic_contacts`)
                mujoco.mj_step1(m, d)
                if d.ncon:
                    self._kinetic_contacts()
                mujoco.mj_step2(m, d)
            else:
                mujoco.mj_step(m, d)
            if flex:
                self._flex_sync()
            if d.ncon:
                pen = self._contact_stats(pen, pairs)
            if pa is not None:
                self._liftoff_tick(ts)
            if not timed or d.time + _TIME_EPS >= sensor.next_capture_time:
                sensor.sample(d.time, self._visible_truth())

Shell bag: the newer bag model. One bendable sheet of beads at the bag's bottom that cannot stretch, like real film. It drapes over cap steps.

sim/deck/flexbag.py:95-108
    # structure (studies/BAGFIX, docs/research/bag_physics.md): "solid" (default: the RLFLEX model, a 2-layer solid
    # lattice whose modulus `young` sets in-plane, compression AND bending stiffness together: a 30 mm, 5 kPa solid bends
    # at ~1e-2 N m, 10-1000 x stiffer than a bag of garments, and stretches like rubber where real film is inextensible)
    # or "shell": ONE sheet of vertices at the bag's bottom face, a <flex dim=2> with bending only (elastic2d bend,
    # stiffness `bend` N m: the contents' bending, folded garments 1e-5 - 1e-3) and an edge-length equality (the
    # film's inextensibility, E t ~ 8-23 kN/m: no stretch); the whole mass on the sheet; the tracker reports the bag's
    # centre H / 2 above the sheet along its mean normal. With `fill` the content flex covers the content lump and the
    # film flaps around it are a second flex of film bending `bend_film` (PE film ~4e-6 N m), hinged to the lump by
    # point constraints at duplicated seam vertices. Contents compression and the bag's own thickness as a collision
    # volume are not modelled (thin tiles at the sheet).
    sphere_frac: float = 0.5  # tile "sphere": bead radius over the vertex spacing (0.5: neighbours touch)
    structure: str = "solid"
    bend: float = 1.0e-4
    bend_film: float = 4.0e-6

Camera modes: A = truth plus noise (fast, "ideal"). B = rendered images with AprilTags. C = a statistical outline camera feeding the real outline pipeline.

sim/deck/sim_sensor.py:1-4
"""Simulated sensing. Mode A (SimSensor): ground-truth planar poses + Gaussian noise, sampled at camera rate with latency.
Mode B (RenderedTagSensor): renders a MuJoCo camera and runs the product AprilTag pipeline (perception/) on each frame.
Mode C (OutlineSensor): a fast statistical overhead outline camera (no rendering) feeding the product outline pipeline's
blob and association code (perception.outline_pipeline); presets OUTLINE_PRESETS, `apply_outline_preset`."""
sim/deck/sim_sensor.py:52-75
    def sample(self, t: float, truth: list[tuple[int, float, float, float]]) -> None:
        """Capture (id, x, y, yaw) poses if a capture slot is due at time t."""
        if t + _EPS < self._k / self.rate:
            return
        # Next slot strictly after t: a long gap yields one capture, not a burst.
        self._k = math.floor((t + _EPS) * self.rate) + 1
        if not truth:
            return
        noise = self.rng.standard_normal((len(truth), 3))
        for (pid, x, y, yaw), (nx, ny, nyaw) in zip(truth, noise):
            self._queue.append(Measurement(
                id=int(pid),
                t_capture=float(t),
                x=float(x + self.noise_pos * nx),
                y=float(y + self.noise_pos * ny),
                yaw=_wrap(yaw + self.noise_yaw * nyaw),
            ))

    def poll(self, t: float) -> list[Measurement]:
        """Pop every queued measurement with t_capture + latency <= t, oldest first."""
        out: list[Measurement] = []
        while self._queue and self._queue[0].t_capture + self.latency <= t + _EPS:
            out.append(self._queue.popleft())
        return out

Configs

Each deck setup is one YAML file. sim/config/current_30v.yaml is today's best safe setup on the 30 V hardware: 3 belts, 11 chutes, about 5 m².

sim/config/current_30v.yaml (also current_48v.yaml)

sim/config/current_30v.yaml:12-25
name: current_30v
grid: {nx: 44, ny: 50, pitch: 0.048}
column: {cap_size: 0.0455, cap_thickness: 0.006, stroke: 0.12, rest_height: 0.06, mass: 0.0614, joint_damping: 0.0, armature: 1.237,
  cap_edge: round, cap_edge_size: 0.002, cap_crown: 0.001}
actuator:
  kp: 200000.0
  kv: 1019.2
  max_force: 49.9
  max_speed: 0.4
  latency: 0.005
  max_accel: 4.0
  motor: {model: nema17_40d|tmc2240_30v|tr8x12|120, sizing_factor: 1.0, stall: encoder, thermal: true, hold_current: 0.4,
    idle_current: 0.4, power_down_delay: 0.21, sg_latency: 0.005, sg_home_speed: 0.03, sg_home_dwell: 0.33, detect_delay: 0.002,
    detect_error: 0.00012, recover_time: 0.02}
sim/config/current_30v.yaml:62-76
sensing:
  mode: A
  rate: 200.0
  latency: 0.01
  noise_pos: 0.0015
  noise_yaw_deg: 0.75
  tag_size: 0.035
  image_size: [1280, 960]
tracker: {accel_noise: 3.0, yaw_accel_noise: 30.0, accel_noise_low: null, yaw_accel_noise_low: null, maneuver_rate: 0.5, quiet_rate: 10.0,
  min_meas_pos_sigma: 0.0002, min_meas_yaw_sigma_deg: 0.1, lookahead: null, timeout: 0.25}
control: {rate: 100.0, mu_nominal: 0.23, margin_pitches: 1.0, center_on_support: true, max_speed: 0.4, accel_limit: 1.5, kp_vel: 6.0,
  ki_vel: 3.0, pos_tolerance: 0.008, hold_speed: 0.01, brake_gain: 1.0, cup_depth: 0.02, keepout: 0.015, edge_approach: 0.06,
  twist_angle_deg: 6.0, deliver_walls: true, deliver_wall_height: null, deliver_wall_anchor: false, deliver_walls_under: false,
  deliver_entry_speed: null, accommodate: true, corridor_time: null, stuck_escape: true, mu_static: 0.23, mu_kinetic: 0.16}
planner: {gap_threshold: 0.03, gap_margin: 0.005, replan_period: 0.5, fine_res: null, waypoint_tolerance: 0.015}
sim/config/current_30v.yaml:103-111
  s10:
    control_preset: d2
    tube_calibration: sim/config/calibration/tubes_X2S_T99.json
    installed_footprint: true
    routing: nearest
    flow: {gate_slope: 0.19, gate_stall: 0.5, restart: true, giveup_retry: 8.0, stuck_footprint: true, jam_time: 3.0, route_fallback: true,
      gate_stall_belt: 2.0, homing_aware: true, homing_pause_max: 2.0, dead_clearance: 0.1, clear_stuck: true, clear_expire: true}
    control: {mu_anchor: 0.3, dead_raise_max: 0.02, rehome_sink_n: 0, kp_vel: 9.0, ki_vel: 4.5, cross_track_gain: 6.0, cross_track_max: 0.08,
      tilt_slew: 1.5}
sim/config/current_30v.yaml:112-128
    traffic:
      mode: tm
      k_routes: 6
      detour: 0.15
      commit_window: 2.0
      deck_cap: 9
      cycle_check: true
      trunk_groups: true
      fallback: true
      fb_joint: true
      rest_rule: true
      stuck: {detect: true, contain: true, mouth: true, clear_reach: 0.8, max_tries: 3, tight_box: true}
      soft_late: true
      late_relief: 4.0
      trains: false
      dead_obst: {floor: null, clearance: 0.024, dense: 0.096, refresh: true, confirm: 0.5, divert: true}
      follow_live: true

Studies and the cloud

Big runs go to Google Cloud VMs. One script launches them with fixed guards: only project volve-test-01, a hard time limit, and the VM stops (not deletes) when the limit hits. No service account. There is no budget alert on the project, so the time limit and the power-off are the guards.

sim/cloud/launch.sh, common.sh, vm.sh (status, sync, delete); each study's rule and results in studies/<NAME>/

sim/cloud/common.sh:11-20
PROJECT="volve-test-01"
REGION="us-central1"
ZONE="${ZONE:-us-central1-a}"
case "$ZONE" in
  "$REGION"-[a-f]) ;;
  *) echo "ZONE must be a zone of $REGION (got $ZONE)" >&2; exit 2 ;;
esac
KEY="$HOME/Documents/keys/volve-test-rl-runner.json"
GCLOUD="${GCLOUD:-gcloud}"
MAX_VMS="${MAX_VMS:-3}"
sim/cloud/launch.sh:94-101
    if run_gc compute instances create "$n" --project="$PROJECT" --zone="$z" \
        --machine-type="$MACHINE" \
        --provisioning-model="$PROVISIONING" --max-run-duration="${MAX_RUN_H}h" --instance-termination-action=STOP \
        --image-family=ubuntu-2404-lts-amd64 --image-project=ubuntu-os-cloud \
        --boot-disk-size="${DISK_GB}GB" --boot-disk-type=pd-balanced \
        --no-service-account --no-scopes --shielded-secure-boot \
        --labels=purpose="$PREFIX",owner=abyl,wave="$NAME" \
        --metadata="block-project-ssh-keys=TRUE,enable-oslogin=FALSE,stawamus-workers=$WORKERS,stawamus-commit=$COMMIT" \

Tests

The pytest suite lives in sim/tests/ (controller, planner, flow, bag motion, firmware core, determinism…) and software/link/tests/. One test guards the whole architecture: software/ must never import the simulator.

software/link/tests/test_layering.py

software/link/tests/test_layering.py:8-21
SOFTWARE = Path(__file__).resolve().parents[2]
SIM_PACKAGES = {"deck", "experiments", "rl", "viz", "tools", "mujoco"}


def test_software_never_imports_sim():
    bad = []
    for f in (p for p in SOFTWARE.rglob("*.py") if p.name != "test_layering.py"):
        for node in ast.walk(ast.parse(f.read_text(), str(f))):
            names = [a.name for a in node.names] if isinstance(node, ast.Import) else \
                [node.module or ""] if isinstance(node, ast.ImportFrom) and node.level == 0 else []
            bad += [f"{f.relative_to(SOFTWARE)}: {n}" for n in names if n.split(".")[0] in SIM_PACKAGES]
        if '"sim"' in f.read_text() and "sys.path" in f.read_text():
            bad.append(f"{f.relative_to(SOFTWARE)}: puts sim/ on sys.path")
    assert not bad, "software/ imports the simulator:\n" + "\n".join(bad)

4 · As of 2026-10-19 evening

What we are doing now

a · RLSTREAM: train the bag motion on the full deck running overnight

The single-bag policy did not help on the full deck. So now it trains on the full deck stream itself, in the cloud. Every soft bag with a moving goal is an agent; all share one policy. Reward: its own progress, plus a shared bonus for every correct delivery on the deck, minus a penalty if it stalls its neighbours.

Five arms, one VM each, 12 h of learning, then about 3 h of development rows. Estimated cost: about USD 390 at list price (5 VMs × ~15.5 h); the held-out run after it about USD 90 more.

sim/rl/stream/launch_train.sh:11-15 RLSTREAM branch
#   a  pair        frozen RLBAG skill pair (+ a dither output, off at the start), default reward
#   b  pairdither  the pair with the dither at 6 mm from the start
#   c  dither      one network starting as the plain 6 mm dither (struct outputs zero)
#   d  pairdither  stronger deck terms: team 3, neighbour stall -0.05
#   e  move        one network: the frozen moving skill everywhere (+ dither, off), team 2
sim/rl/stream/launch_train.sh:36-40 RLSTREAM branch
launch a pair base 5000000
launch b pairdither dither 105000000
launch c dither base 205000000
launch d pairdither dither 305000000 "--rew team=3,disturb=-0.05"
launch e move none 405000000 "--rew team=2"
sim/rl/stream/world.py:18-29 RLSTREAM branch
REWARD per decision (the ticks until the agent's next decision), weights R (JSON overrides --rew):
  own       progress   R.progress x the bag's true CoM progress along its goal direction (m; +-2 cm per tick cap)
            stall      R.stall per decision while the bag's true speed < 3 cm/s (0.3 s after its first decision)
            effort     R.effort x mean(a^2) + R.deffort x mean(da^2)
            terminal   R.delivered (correct bin), R.wrong, R.fallen, R.hung, R.giveup (the jam breaker gave it up)
  team      every open agent: R.team per correct delivery anywhere on the deck (box or bag) and R.team_loss per
            parcel lost anywhere (wrong bin, floor, hung): the deck's throughput, shared
  neighbours R.disturb per decision for each other parcel with a moving goal that stands still (< 3 cm/s) within
            NEAR_DILATE of the columns this bag's motion moved (every 5th tick); R.disturb_loss when a parcel this bag's
            motion came within NEAR_DILATE of in the last TOUCH_S is lost
Credit assignment: the own and neighbour terms are the agent's; the team term is the deck's (all open agents), so the
critic (per agent, with deck counts in its observation) learns what the deck makes of the bag's motion.

The learner updates while the workers keep simulating, so every core works all the time (RLBAG used about a third of each VM).

sim/rl/stream/appo.py:1-14 RLSTREAM branch
"""studies/RLSTREAM learner: asynchronous PPO on chunks from many rollout workers.

The workers act with a policy that may be a few versions old (behaviour mu). Each update ("decoupled PPO", Hilton et
al. 2021, with IMPALA's V-trace targets):
  - prox = the learner's policy at the start of the update; w = min(1, prox / mu) per decision (truncated importance
    weight of the stale behaviour);
  - V-trace over every chunk with rho = c / lam = min(1, prox / mu): value targets vs, advantages
    A = rho (r + gamma vs' - V); the chunk's end bootstraps from its next observation (0 when the agent's bag
    finished);
  - PPO's clipped objective on r = pi / prox, weighted by w; advantages normalised per minibatch; a Huber value loss
    on vs; epochs stop early once the KL to prox passes 1.5 x target_kl (summed over the 9 action numbers);
  - the first critic_warmup updates train the critic only (the warm-started actor is not moved by an untrained
    critic's advantages).
Rewards are scaled by rew_scale before any of this.

Then: the best candidate is chosen mechanically on development seeds 17001-17012 and frozen. One held-out run on fresh seeds 1800-1805 (4 streams each, 900 s windows) compares it with today's controller and the dither. Verdict YES only if it beats both at both loads without losing more parcels. Rule: studies/RLSTREAM/rule.txt on the branch.

b · Camera collapse fix: recheck VM

In 22 of 108 camera trials the deck nearly stopped. Cause: columns under a stuck bag kept re-homing, and every re-homing restarted the planner's watchdog, so nothing was ever given up. Two fixes: control.rehome_sink_n = 3 holds such columns low; flow.homing_pause_cumulative makes the watchdog count all pauses together. The recheck reruns the affected cells (66 trials). If clean, both become defaults. See studies/CAMERA/README.md, "The collapses".

c · Bench tests in Seattle waiting on you

docs/pages/BAG_SLIDE_TEST.html: friction, bending and the staircase test on real mailers. This decides whether the shell bag model is right. Then the dither on a real column: does it stay within the motor limits (no lost steps, no stalls), and how loud is it? Until then bag_motion stays off on the real deck.

Results so far (all simulation)

Single bag, RL (held-out)
35 → 71 % bags delivered (35.4 → 71.4 %). Dither alone 44.3 %.
Full-deck stream, studies 1 and 2
RL: NO (rate a TIE with today's controller). Dither: +140 to +174 /h over base, BETTER (study 1 at sat: +74 /h).
Shell bag model vs rigid
Today's controller, research mix: 153-167 /h with shell bags against 3,007-3,567 /h with rigid stand-ins (about −3,400 /h, −95 %). With the dither the stream studies reached 160-240 /h.
Camera
Delay matters, not frame rate: about 10 ms end to end; 60 fps is enough. Recommended: Basler a2A1920-160umBAS. Measure the delay on the bench.

5 · Next

Future: deep RL, and first a faster simulator

Today's policy is a small network that sees one moment, one bag and a 9 × 9 patch. Three steps make it smarter, in order:

  1. Memory. A recurrent or transformer policy remembers the last seconds. Then it can notice "this bag has not moved for a while" and change tactic, instead of repeating the same motion.
  2. Attention over nearby parcels. The policy looks at every parcel near its bag (position, speed, goal) and learns whom to give way to. That is coordination, which the full-deck study showed is missing.
  3. Whole-deck RL, later. One policy shaping the whole height field, or deciding routes and timing too. Only after the first two work, because it needs far more simulation.

First step: simulation speed

Every step above needs many more simulated hours. Today the full-deck stream runs about 19 times slower than real time on one CPU core, and physics is about 89 % of a worker's time (estimates in studies/RLSTREAM/README.md on the branch). Soft bags need 0.5 ms physics steps: 20 per control tick.

Profile first. Time one full-deck step and split it into: MuJoCo contact and solver, the shell bags' flex, the motor model, the controller, the planner, and Python glue. The 89 % figure is a rough estimate; the split inside physics is not measured yet. Then fix the biggest piece.

Candidate fixExpected gainEffortStatus
Profile one full-deck step (where does the time go?)tells us which row below matterslowproposed, first
CG contact solver instead of Newtonabout 10-12× on bag scenes, same outcomeslowdone (RLBAG, RLSTREAM)
Contact pruning: caps collide only near parcels, parked parcels inert (physics.fast)large on a big deckdonedone
Controller on a window of the grid around each parcel (geometry.DeckWindow)large on a big deckdonedone
Async workers on all coresabout 3× per VM (RLBAG used a third of the cores)mediumdone (RLSTREAM)
Solver settings: iterations, tolerance, timestep for bagsunknown until profiled; possibly 1.5-2×lowproposed
Coarser bag sheet, fewer bead contactspossibly large (contacts scale with beads)low-medium; must re-match the bench testsproposed
Local sub-deck worlds around one bag, started from saved full-deck snapshotslarge per bag decisionmediumproposed
Batch the controller (and planner) over many worldsneeded for any GPU pathhighproposed
Check MuJoCo Warp (GPU) support for our flex shell bagsvery large if supportedlow to check, high to adoptproposed (not checked)
Learned surrogate of the bag physicsvery largehigh; needs lots of data firstlater

Gains marked "possibly" or "large" are expectations, not measurements. Only the CG and core-use numbers come from studies.

Why a GPU needs the planner and controller batched too. A GPU simulator is fast because it steps thousands of decks at once. But every 10 ms each of those decks needs new column heights from the planner and controller, and ours run one deck at a time in Python on the CPU. The GPU would then sit idle waiting, and data would shuttle back and forth every tick. So the planner and controller must also run for all decks at once (array code for many worlds), or the GPU path must use a simplified stand-in for them.

6 · Words

Glossary

tick
One pass of the control loop: 10 ms (100 Hz).
stroke
How far a column can travel: 120 mm (0 to 0.12 m). Rest height is the middle, 60 mm.
pitch
Distance between column centres: 48 mm.
cap
The flat square top of a column (45.5 mm wide) that parcels rest on.
column
One motor + lead screw + cap. The deck is a grid of them.
footprint
The rectangle a parcel covers on the deck, seen from above.
induction
Putting a parcel into the system: scanned (size, mass, kind) and loaded on an infeed belt.
chute
An opening at the deck edge leading to a destination bin.
1x / sat
Load levels. 1x = 5,500 parcels/h offered to the feed; sat (saturation) = 3×, 16,500/h, so the belts are always full.
held-out seed
A random scenario kept unused until the final test, so the result is not tuned to it.
dither
Small random up-and-down cap motion (6 mm, new every 50 ms) to break a bag loose.
policy
The learned decision maker: observation in, action out. Here a small neural network.
PPO
The standard RL training method used here: improve the policy in small, safe steps.
flex / shell bag
MuJoCo's deformable "flex" bodies. The shell bag is one bendable, non-stretching sheet at the bag's bottom.
Mode A / B / C
Simulated camera types: A ideal (truth + noise), B rendered AprilTags, C statistical outline camera through the real pipeline.