1 · Overview
What the software does, in one paragraph
The deck is a grid of columns (44 × 50 on the 5 m² unit, 48 mm apart). Each column has a cap on top and moves up and down. Parcels lie on the caps. The software moves parcels by tilting the caps under them into a gentle slope, so the parcel slides where we want. 100 times a second it does the same five things: look (camera), work out where every parcel is (perception), decide where each parcel should go next (planner), compute a height for every column (controller), optionally add a small extra motion under soft bags (bag motion layer), and send the heights to the deck.
The same software/ code runs in the simulator and on the real deck. sim/ only replaces two things: the deck (MuJoCo physics instead of motors) and the camera (a simulated sensor instead of a real one). A test makes sure software/ never imports anything from sim/.
The registry is filled when a parcel is inducted (scanned and put on a belt). The planner gets its size, mass and kind through flow.add(); the tracker only accepts camera measurements for ids that are in the registry. The controller also receives the tracked parcels directly (not drawn).
2 · Repo map
Where things live
One repository, stawamus. Six top folders. Python for the product and simulator, C++ for the firmware.
software/
The product. Runs unchanged on the real deck. Never imports the simulator.
interface/the Deck contract: DeckInfo, ParcelState, Deckperception/camera image → parcel poses → Kalman trackerplanner/where each parcel goes:flow.py,traffic/,stuck/,mapf/,planner.py,bulk.pycontrol/column heights:controller.py,geometry.py,primitives.py,bag_motion.pypolicies/learned policies as plain numpy, plusfrozen/weight fileslink/PC ↔ firmware: RealDeck, bench tool, CAN, health
sim/
The simulator and everything around it. Replaces the deck and the camera.
deck/SimDeck (MuJoCo), bag models, sensors, motor modelexperiments/run loop (harness.py) and scenarios (s10.pystream)rl/RL training:bag/,control/,routes/,traffic/(stream/on the RLSTREAM branch)tools/one script per study (runs, tables)cloud/launch, watch and collect cloud VMsconfig/deck configs (YAML)tests/pytest suiteviz/interactive sandbox control panel
firmware/
C++ for the STM32G474. core/ is all behaviour (no chip-specific code), board/ the chip glue, host/ runs the same core on the Mac for tests.
hardware/
The Tile-20 board (hardware/tile20): KiCad files, BOM, layout audit, bring-up notes.
studies/
One folder per question: README with the answer, a rule written before the results, result tables. E.g. RLBAG, CAMERA, BAGFIX, CURRENT.
docs/
Pages for the docs site (docs/pages, e.g. bench tests and BOMs), the spec, the wire contract (CONTRACTS.md), research notes, print files.
3 · Layers
The Deck interface
The contract between "the brain" and "the deck". The planner and controller only ever talk to these types. SimDeck and RealDeck both implement them, so the brain cannot tell which deck it drives.
software/interface/__init__.py
ParcelState: what the camera pipeline says about one parcel (position, angle, speed, size). Never the simulator's truth.
software/interface/__init__.py:55-69@dataclass(frozen=True)
class ParcelState:
"""Tracked planar parcel state, exactly what the real camera pipeline provides."""
id: int
x: float
y: float
yaw: float
vx: float
vy: float
omega: float
length: float # along parcel local x
width: float # along parcel local y
height: float
mass_estimate: floatDeckInfo: the fixed facts about the deck (grid, pitch, stroke, chutes).
software/interface/__init__.py:102-113@dataclass(frozen=True, eq=False)
class DeckInfo:
"""Static deck description handed to the controller and planner."""
grid_shape: tuple[int, int] # (nx, ny)
pitch: float
cap_size: float
stroke: float
rest_height: float
max_speed: float # nominal column speed (m/s)
column_xy: np.ndarray # (n, 2) column centres, index = ix * ny + iy
chutes: tuple[Chute, ...] = ()Deck: four calls. Send heights, read parcels, read the clock, advance time.
software/interface/__init__.py:196-210 @abstractmethod
def set_heights(self, heights_m: np.ndarray, t_apply: float) -> None:
"""Command target heights (n,) in metres, to be applied at deck time t_apply."""
@abstractmethod
def get_parcels(self) -> list[ParcelState]:
"""Tracked parcel states from perception (never ground truth)."""
@abstractmethod
def now(self) -> float:
"""Current deck time (s)."""
@abstractmethod
def step(self, dt: float) -> None:
"""Advance by dt seconds (simulation) or wait/no-op (real hardware)."""The run loop
One function drives every tick: read parcels, ask the planner for goals, ask the controller for heights, send them, step. In the sim, "step" runs physics; on the real deck it waits for the next 10 ms.
sim/experiments/harness.py (run_episode)
sim/experiments/harness.py:199-215 n_ticks = int(math.floor(duration / dt + _TIME_EPS)) if duration > 0 else 0
for _ in range(n_ticks):
t = deck.now()
if on_tick is not None:
on_tick(deck, planner, t)
parcels = deck.get_parcels()
goals = planner.update(t, parcels)
if recorder is not None:
recorder.capture(goals)
out = controller.compute(t, parcels, goals)
deck.set_heights(out.heights, t)
deck.step(dt)
log.record(deck, out, parcels, goals, planner)
if until is not None and until(deck, planner, log):
log.stopped_by_until = True
break
return log.finalize(planner)Perception
Turns camera images into parcel positions. Two steps: find outlines in the image, then smooth them over time with a tracker.
software/perception/outline_pipeline.py, tracker.py, types.py (also apriltag_pipeline.py for tagged parcels)
Measurement: one sighting of one parcel in one frame, stamped with the time the picture was taken.
software/perception/types.py:13-27@dataclass(frozen=True)
class Measurement:
"""One planar pose observation of a parcel, stamped with its capture time.
Optional: the observed footprint (length along the measured yaw, width across it; None = the sensor does not
see the outline, e.g. Mode A / AprilTags) and quality flags (FLAG_*)."""
id: int
t_capture: float # s, deck time at which the image/sample was taken
x: float
y: float
yaw: float # rad, wrapped to (-pi, pi]
length: float | None = None # m, observed footprint along yaw
width: float | None = None # m, observed footprint across yaw
flags: int = 0Outline pipeline: image → mask (what is not empty deck) → blobs (rectangles) → match each blob to a known parcel.
software/perception/outline_pipeline.py:306-321 def process(self, image: np.ndarray, t_capture: float, tracks: Iterable = (),
hints: Iterable[Expect] = ()) -> list[Measurement]:
"""segment -> blobs -> associate on one camera image (`timing` accumulates the wall time per stage)."""
t0 = time.perf_counter()
mask = self.segment(image)
t1 = time.perf_counter()
self.last_blobs = self.blobs(mask)
t2 = time.perf_counter()
meas = self.associate(self.last_blobs, self.candidates(t_capture, tracks, hints), t_capture)
t3 = time.perf_counter()
tm = self.timing
tm["frames"] += 1
tm["segment_s"] += t1 - t0
tm["blobs_s"] += t2 - t1
tm["associate_s"] += t3 - t2
return measTracker: a Kalman filter per parcel. In plain words: it keeps a best guess of position and speed, plus how sure it is. Each new sighting nudges the guess; a noisy sighting nudges it less. Because each sighting carries its capture time, a late picture corrects the past, and the guess is then projected forward to "now".
software/perception/tracker.py:1-7"""Per-parcel constant-velocity Kalman filter with timestamped latency compensation.
State s = [x, y, yaw, vx, vy, omega]. Process model: constant velocity driven by
piecewise-constant white acceleration (sigma `accel_noise` m/s² on x/y,
`yaw_accel_noise` rad/s² on yaw). Measurements observe (x, y, yaw) at `t_capture`,
so a delayed measurement corrects the state at the time it was taken and
`estimate(t)` predicts forward to any later time.software/perception/tracker.py:279-296 def update(self, measurements: list[Measurement]) -> None:
"""Fuse measurements (any order within the batch); stale or non-finite ones are skipped."""
valid = [m for m in measurements if m.id in self.registry
and all(map(math.isfinite, (m.t_capture, m.x, m.y, m.yaw)))]
for m in sorted(valid, key=lambda m: m.t_capture):
tr = self._tracks.get(m.id)
if tr is None or m.t_capture - tr.t > self.timeout:
self._tracks[m.id] = self._init_track(m)
continue
if m.t_capture - tr.t < 0:
continue
if self.fp is not None and m.flags:
self.stats["partial"] += bool(m.flags & FLAG_PARTIAL)
self.stats["merged"] += bool(m.flags & FLAG_MERGED)
new = self._fuse(tr, m)
if new is None:
self.stats["reinit"] += 1
self._tracks[m.id] = self._init_track(m, tr) if new is None else newRegistry: the tracker only accepts ids it knows. The sizes come from the induction scan.
software/perception/types.py:30-38@dataclass(frozen=True)
class ParcelInfo:
"""Parcel registry entry: dimensions known from the induction scan."""
id: int
length: float
width: float
height: float
mass: floatPlanner
Decides, for every parcel, what it should do this tick: move, hold, deliver to a chute or turn. It also runs and stops the infeed belts. Its output is one Goal per parcel.
software/planner/flow.py (FlowOrchestrator), planner/traffic/ (TrafficOrchestrator: route choice, locks, jam breaker, FOLLOW rule), planner/stuck/ (stuck parcels), planner/mapf/ (lattice path-finding variant), planner/planner.py (the original single-parcel planner), control/goals.py
software/control/goals.py:8-23GoalMode = Literal["velocity", "goto", "hold", "deliver", "twist", "idle"]
@dataclass(frozen=True)
class Goal:
mode: GoalMode
# velocity: desired planar velocity (m/s), deck frame
velocity: tuple[float, float] = (0.0, 0.0)
# goto / hold: target position (m), deck frame
target: tuple[float, float] | None = None
# deliver: destination chute id (DeckInfo.chutes)
chute_id: int | None = None
# twist: +1 = counter-clockwise, -1 = clockwise
twist_dir: int = 1
# optional per-goal speed cap (m/s); None = control.max_speed
max_speed: float | None = NoneWhat it decides:
- Routes: one route per belt-to-chute pair, shaped like a U, L or Z, with rounded corners. With traffic on, up to 6 candidate routes per parcel; it picks the cheapest free one.
- Timing ("tubes"): every parcel reserves the space it will sweep, slot by slot, 0.1 s at a time. A parcel is slowed until its tube clears everyone else's.
- Belts: a parcel is only released from its belt when its tube is clear. Otherwise the belt stops at the stop line.
- FOLLOW rule: a parcel may follow its own belt's leader into the deck when its tube clears the leader's with the full safety margin.
follow_livefixes a freeze where a follower held its leader forever. - Column ownership is not decided here; the controller does that (next section).
software/planner/flow.py:1-7"""High-density flow orchestrator for singulated belt infeeds (stage B, reports/DESIGN/BRIEF.md 4.2).
Pure numpy (planner/ is checked by tests/test_purity.py). `FlowOrchestrator` is the episode's planner in S10
(experiments/s10.py): the harness calls `update(t, parcels) -> goals` every control tick, exactly as it calls
`Planner.update`. The scenario tells it which parcel is on which belt and where (`add`, `observe_belts`: what a
belt PLC knows from its photo-eyes and encoder), reads its belt commands (`belt_commands`: run or stop each belt
at its stop line), reports chute-mouth photo-eyes (`set_chute_blocked`) and removes parcels once they have leftA parcel enters the planner when it is loaded on a belt, with its registry size and kind:
software/planner/flow.py:1567-1582 def add(self, pid: int, infeed: int, chute: int, length: float, width: float, height: float, mass: float,
t: float, yaw: float | None = None, kind: str | None = None) -> Route:
"""A parcel was loaded on belt `infeed`, bound for `chute` (L >= W as it lies; yaw = its belt yaw; kind:
'box' or a bag kind, used by the adaptive tubes' classes only)."""
lane = self._choose_lane(int(infeed), int(chute))
route = self.route_for(infeed, chute, length, width, height, yaw, lane=lane)
f = _Flow(int(pid), int(infeed), int(chute), float(length), float(width), float(height), float(mass),
float(t), route, lane=lane, kind=kind)
f.tube = self.prm.tube_of(kind, length, mass)
f.heavy = self.is_heavy(kind, length, mass)
if f.heavy and self.prm.heavy_a_lat is not None:
set_clock(route, self.prm, corner_radius=self.prm.corner_radius, a_lat=self.prm.heavy_a_lat)
f.hx, f.hy = half_extents(length, width, self.infeeds[int(infeed)].yaw if yaw is None else yaw,
math.radians(self.prm.yaw_allow_deg))
f.s = -float(self.infeeds[int(infeed)].length)
self.flows[int(pid)] = fsoftware/planner/traffic/params.py:156-164 # studies/CURRENT: the FOLLOW rule's liveness fix (studies/RLX FOLLOW, rl_routes mode "follow"; no effect without a
# FOLLOW pair). A FOLLOW leader is not checked against its own follower's reservation while that reservation lies
# entirely behind the leader's centre on the leader's first-leg line and the leader's remaining route only moves
# ahead along that line and away from it: the leader advancing can then only widen the gap. The follower keeps its
# full-envelope check against the leader (both placement orders), so the pair keeps the full crossing envelope as
# before. Without it, a follower placed first (jam breaker, priority inversion, heavy first) and closer than the
# full envelope (e.g. after the leader slid or tipped back) holds its leader for good: a wait-for 2-cycle the jam
# breaker cannot open (the follower's rear is still on the belt). False (default): unchanged.
follow_live: bool = FalseController
Turns each Goal into column heights. For each parcel it builds a "field" (usually a tilted plane under the parcel), then merges all fields into one height per column.
software/control/controller.py, primitives.py (plane, cup, brake, twist, deliver), geometry.py (footprints, which caps lie under a parcel)
The tick: smooth each parcel's state, build its field, compose, then limit how fast columns change hands.
software/control/controller.py:665-687 fields: list[_Field] = []
smooth: list[ParcelState] = []
for p in ps:
st = self._states.setdefault(p.id, _State())
pf = self._filter(p, st, dt)
smooth.append(pf)
goal = goals.get(p.id) or Goal("idle")
fields.append(self._window_field(pf, goal, st, dt))
if self.ball_pred: # the tilt just commanded acts over the next delay (`_predict`)
st.pred_hist.append(st.tau.copy())
while len(st.pred_hist) > self.pred_n:
st.pred_hist.popleft()
st.age += dt
ps = smooth
self._touching = None
heights, owner, conflicts = self._compose(ps, fields)
if self._lost:
heights = self._lost_apply(ps, heights)
if self.lost_hold > 0.0:
self._last_ps = {p.id: p for p in ps}
heights = self._transition_slew(ps, heights, owner, dt, self._touching)
if self.rehome_sink_n > 0:
heights = self._rehome_sink(t, ps, heights)Tilt plane: the plane is centred on rest height (60 mm) and pushed back inside the 0-120 mm stroke when it would not fit. Each column's height is the plane's height at its cap corner.
software/control/primitives.py:360-373 lo = float(base) + floor
span_h = t * (s_hi - s_lo)
room = info.stroke - lo
if front_low:
mid = lo + span_h / 2.0
elif span_h > room:
mid = lo + room / 2.0
elif low:
mid = lo + span_h / 2.0
else:
mid = float(np.clip(centre, lo + span_h / 2.0, info.stroke - span_h / 2.0))
h0 = mid + t * (s_hi + s_lo) / 2.0 + float(offset)
raw = h0 - t * s
return np.clip(raw, 0.0, info.stroke) if clip else rawSupport, keep-out and owners: each column goes to the nearest parcel that claims it. A column inside one parcel's keep-out ring (15 mm in current_30v) is never given to a parcel farther away. The result owner says which parcel owns each column; the bag motion layer uses it.
software/control/controller.py:2336-2355 U = np.array([support_mask(p, info, 0.0) for p in ps]) # caps under each footprint
self._accommodate(ps, fields, U)
W = np.array([f.w > 0 for f in fields]) # (P, n)
D = np.array([point_rect_distance(info.column_xy, p) for p in ps]) # (P, n)
near = np.array([support_mask(p, info, self.keepout) for p in ps]) # caps within keepout of footprint
conflicts = int(np.count_nonzero(W.sum(axis=0) > 1))
cand = W.copy()
for i in range(len(ps)):
for j in range(len(ps)):
if i != j:
# a column in j's keep-out ring is never given to a parcel farther from it than j
cand[i] &= ~(near[j] & (D[j] < D[i]))
Dm = np.where(cand, D, np.inf)
best = np.argmin(Dm, axis=0) # ties -> lowest parcel id (sorted order)
has = cand.any(axis=0)
cols = np.nonzero(has)[0]
H = np.array([f.h for f in fields])
heights[cols] = H[best[cols], cols]
ids = np.array([p.id for p in ps])
owner[cols] = ids[best[cols]]Footprint: with the camera's tracked footprint on, a parcel is planned with its measured size plus a safety margin.
software/control/geometry.py:285-290def tracked_footprint(p: ParcelState, k: float) -> tuple[float, float]:
"""(length, width) to plan with: the tracked footprint (fp_length / fp_width; the induction length / width when
the tracker does not track it) grown by k x fp_margin on every side."""
g = 2.0 * float(k) * float(p.fp_margin)
return ((p.length if p.fp_length is None else p.fp_length) + g,
(p.width if p.fp_width is None else p.fp_width) + g)Re-homing guard (part of the camera collapse fix, off in today's configs):
software/control/controller.py:446-451 # rehome_sink_n: a column that starts a re-homing run (a known dead column at or below rest height appearing
# in the dead-column cache: a StallGuard / encoder homing) rehome_sink_n times within rehome_window s while a
# parcel's footprint covers it is commanded to 0 (dead-low) until no footprint covers it: under a heavy box it
# otherwise climbs back into the box after every homing, slips again and the box sinks into a pit of
# re-homing columns (0 = off).
self.rehome_sink_n = int(g("rehome_sink_n", 0) or 0)Bag motion layer
An optional extra on top of the controller, only under soft bags. It adds a small up-and-down motion to the bag's own columns to break the bag loose. One config switch: control.bag_motion = off (default), dither or rl.
software/control/bag_motion.py; how-to in software/README.md
software/control/bag_motion.py:5-14Modes
off the controller as it is.
dither a random height offset per cap, N(0, bag_motion_sd) (default 6 mm), new every decision (DECIDE_EVERY = 5
control ticks: 50 ms at the 100 Hz loop), from a generator seeded with bag_motion_seed; on the 9 x 9 cap
patch around the bag (policies.bagmotion.patch_cells), as the RLBAG "dither" arms.
rl the frozen skill pair of studies/RLBAG/FROZEN.json, deployed as trained: the landing skill
(policies/frozen/rlbag_land.npz) while the bag's centre is less than LAND_IN = 0.35 m inside the edge of the
belt it comes from, the moving skill (policies/frozen/rlbag.npz) after that; action = mean + learned std x
N(0, 1) from a generator seeded with bag_motion_seed (policies.bagmotion.BagMotion, stochastic). The default
files are checked against FROZEN.json's hashes when loaded.software/control/bag_motion.py:276-281 def from_params(cls, params: dict | None) -> "BagMotionParams":
p = dict(params or {})
raw = p.get("bag_motion", "off")
mode = "off" if raw is None or raw is False else str(raw).strip().lower()
if mode not in MODES:
raise ValueError(f"control.bag_motion {raw!r}: one of {MODES}")The "own" mask: it may only move columns the controller gave to this bag, never within one cap pitch of another parcel (now, and where that parcel will be in 0.25 s and 0.5 s).
software/control/bag_motion.py:165-171def own_mask(info: DeckInfo, p, owner: np.ndarray, ko: dict[int, np.ndarray], tracked: bool) -> np.ndarray:
"""(n,) the columns the motion may move for bag p: the ones the controller assigned to it (a bag not tracked yet,
still on its belt: the caps under its estimated footprint that nobody owns), minus every other parcel's keep-out."""
mine = owner == p.id
if not mine.any() and not tracked:
mine = support_mask(p, info, 0.0) & (owner < 0)
return mine & ~others_keepout(info.n, ko, p.id)Limits: every column it moves stays within 0.4 m/s and 4 m/s² (the 30 V drive), and inside the stroke.
software/control/bag_motion.py:245-256def limit_step(target: np.ndarray, prev: np.ndarray, vel: np.ndarray, dt: float, vmax: float, amax: float
) -> tuple[np.ndarray, np.ndarray]:
"""One tick of a velocity- and acceleration-limited move from `prev` (moving at `vel`) toward `target`: the speed
that can still stop at the target (v dt + v^2 / 2a <= distance), capped at vmax, reached within amax dt of the
previous speed. Returns (new targets, their speeds)."""
e = np.asarray(target, float) - prev
d = np.abs(e)
v_stop = amax * (np.sqrt(dt * dt + 2.0 * d / amax) - dt)
v_want = np.sign(e) * np.minimum(vmax, v_stop)
v0 = np.clip(vel, -vmax, vmax)
v = np.clip(np.clip(v_want, v0 - amax * dt, v0 + amax * dt), -vmax, vmax)
return prev + v * dt, vWiring: with off it returns the controller unchanged, so nothing changes. Sim and real deck build it the same way.
software/control/bag_motion.py:465-480def wrap_controller(ctl: Any, info: DeckInfo, params: dict | None, *, flow: Any = None,
measured: Callable[[], np.ndarray] | None = None, kind_of: Callable[[int], str | None] | None = None
) -> Any:
"""The deck's controller with control.bag_motion applied: `ctl` itself when off (nothing constructed), else a
BagMotionController around it. The real deck and the simulator both build it this way, from the same control
params; `flow` (the orchestrator) gives the parcel kinds and the belt observations, `measured` the column heights.
"""
prm = BagMotionParams.from_params(params)
if prm.mode == "off":
return ctl
src = flow_sources(flow, info) if flow is not None else {"kind_of": kind_of or (lambda pid: None)}
if kind_of is not None:
src["kind_of"] = kind_of
rate = float((params or {}).get("rate", 100.0))
return BagMotionController(ctl, info, prm, src["kind_of"], src.get("belt_items"), src.get("belts", ()),
measured, rate)RL policy runner
Runs a trained policy on the deck. Plain numpy, no training library. Every 50 ms it looks at a 9 × 9 patch of caps around the bag and outputs a small set of motion numbers (extra tilt, a travelling wave, edge lifts).
software/policies/bagmotion.py; frozen weights software/policies/frozen/rlbag.npz, rlbag_land.npz; record studies/RLBAG/FROZEN.json
software/policies/bagmotion.py:281-287 def heights(self, t: float, info, p, is_bag: bool, measured: np.ndarray, commanded: np.ndarray,
goal: Goal) -> np.ndarray:
"""One control tick for parcel p (the deployed loop)."""
if self.due(p.id):
obs = self.observe(t, info, p, is_bag, measured, commanded, goal)
return self.commit(info, p, commanded, goal, t, self.act(obs))
return self.hold(info, p, commanded, goal, t)"As trained" means the policy adds its learned random spread on purpose. On the deck that spread is a small dither.
software/policies/bagmotion.py:237-243 def act(self, obs: np.ndarray) -> np.ndarray:
if self.mlp is None:
return np.zeros(self.act_dim)
a = self.mlp(obs)
if self.std is not None:
a = np.clip(a + self.std * self.rng.standard_normal(self.act_dim), -1.0, 1.0)
return aThe deck refuses a weight file whose fingerprint (sha256) does not match the frozen record:
software/control/bag_motion.py:60-63FROZEN_DIR = Path(__file__).resolve().parents[1] / "policies" / "frozen"
REPO = Path(__file__).resolve().parents[2]
# studies/RLBAG/FROZEN.json: the deployed skill pair (file, sha256[:16]); a default file that differs is refused
FROZEN = {"move": ("rlbag.npz", "0306ab8794216c45"), "land": ("rlbag_land.npz", "d667e57598287ea6")}RLSTREAM branch: a new mode structd = the 8 motion numbers plus a 9th, a dither the policy can switch on and size. It also sees neighbours and who owns which column.
software/policies/bagmotion.py:55-70 RLSTREAM branchACT_DIMS = {"cap": PATCH * PATCH, "struct": 8, "structd": 9}
DITHER_MAX = 0.012 # m: structd's dither amplitude at a = 1 (sd of the per-cap N(0, 1) draw)
STREAM_CH = 5 # structd: patch channels of the stream block
N_STREAM = 8 # structd: scalars of the stream block
NEAR_M = 0.5 # m: "nearby" other parcels (the count; only these are drawn into the patch)
ACT_DIM = ACT_DIMS["cap"]
MEM_S = 1.0 # s of track history for the mean speed
MOVE_TOL = 0.015 # m: the track "moved" (the stream's jam watch)
def obs_dim(mode: str = "cap") -> int:
"""Patch channels, scalars, last action (struct: + the wave phase's cos / sin; structd: the struct observation +
the stream block)."""
if mode == "structd":
return obs_dim("struct") + 1 + STREAM_CH * PATCH * PATCH + N_STREAM
return 3 * PATCH * PATCH + N_EXTRA + ACT_DIMS[mode] + (2 if mode == "struct" else 0)Link to hardware
RealDeck packs the 100 Hz heights into small binary frames and sends them to the microcontroller, over USB serial or CAN FD (the Tile-20 board). The same folder holds the bench tool you use on the Mac.
software/link/real_deck.py, cli.py (python -m link), canlink.py, protocol.py, health/
Frame format: sync bytes, a sequence number, "apply at" time, the heights in 0.01 mm steps, a checksum.
software/link/real_deck.py:3-13Wire format (little-endian)::
0xAA 0x55 | seq u16 | t_apply_ms u32 | n u8 | heights u16[n] (0.01 mm) | crc16
The CRC is CRC-16/CCITT-FALSE over every byte after the sync, up to the end of the
heights. A 30-column frame is 71 bytes; at 100 Hz with 8N1 framing (10 bits/byte) that
is 71 kbit/s, about 7.7 % of a 921600 baud link.
`FirmwareModel` is the executable reference for the firmware contract: apply at
`t_apply`, rate-limit each column, ignore stale/duplicate seq (uint16 wrap-aware), and go
flat (toward `rest_height`) once no valid frame has arrived for 200 ms.software/link/real_deck.py:308-326 def set_heights(self, heights_m: np.ndarray, t_apply: float) -> None:
h = np.asarray(heights_m, dtype=float).ravel()
if h.shape[0] != self.info.n:
raise ValueError(f"expected {self.info.n} heights, got {h.shape[0]}")
if not np.all(np.isfinite(h)):
raise ValueError("heights must be finite")
t_ms = min(max(_ms(t_apply), 0), MAX_T_APPLY_MS)
frame = encode_frame(self._seq, t_ms, np.clip(h, 0.0, self.info.stroke))
self.transport.write(frame)
self._seq = (self._seq + 1) % SEQ_MOD
self.frames_sent += 1
def _pump_camera(self) -> None:
if self.camera is not None:
self.tracker.update(self.camera.poll(self.now()))
def get_parcels(self) -> list[ParcelState]:
self._pump_camera()
return self.tracker.estimate(self.now() + self.lookahead)Bench tool (first lines of its command list):
software/link/cli.py:3-15 python -m link ports list USB serial ports (best match first)
python -m link ping firmware version, columns, microsteps
python -m link regs [--col 0] TMC5240 register dump, decoded, with SPI-OK / SPI-FAIL
python -m link init (re)initialise the drivers (writes and verifies every register)
python -m link current 0.8 [--hold 0.8] run / hold current in A RMS (hard cap: param current_cap_ma)
python -m link enable | disable driver enable (EN pin)
python -m link spin [--revs 1 --rps 1] turn the motor (works unhomed, max 1 rev per command)
python -m link setpos 60 declare the carriage's height in mm (bench guide G)
python -m link move 80 [--speed 10] move to a height in mm (homed or setpos first)
python -m link home [--speed 30] StallGuard homing into the bottom stop
python -m link status [--watch 1] state, flags, positions, counters
python -m link estop-rearm re-initialise the drivers and clear the E-stop latch
python -m link stream --pattern sine 100 Hz height frames (the RealDeck protocol) with clock syncCAN FD: the same frames, cut into CAN messages. Each board has a node number 1-14.
software/link/canlink.py:7-13CAN id (11 bit) = function << 4 | node (node 1..14: the board's slot ID or its can_node parameter):
0x01 EVENT board -> host urgent event at once, 12 bytes: code, column, channel, 0, deck_ms u32, value i32
0x02 HEIGHT host -> board height frames, segmented
0x04 COMMAND host -> board command frames, segmented
0x05 REPLY board -> host reply frames (replies, telemetry, MOVE_STATS), segmented
0x07 HEARTBEAT board -> host every 100 ms, 24 bytes (layout 1), 32 (layout 2, firmware >= 0.3: + the failed
and passable column masks; decode_heartbeat)Firmware
Runs on the STM32G474 next to the drivers. It receives height frames, applies each at its "apply at" time, limits each column's speed, and drives the TMC5240 stepper drivers over SPI. It homes columns by StallGuard (feels the motor stall at the bottom stop), latches the E-stop, and goes flat if no valid frame arrives for 200 ms.
firmware/core/ (protocol, scheduler, motion, tmc5240, health), firmware/board/, firmware/README.txt
firmware/core/protocol.h:1-13// Wire protocol between the Mac and the column firmware (USB CDC; 921600 baud on a UART).
//
// Two frame kinds share the host -> MCU byte stream:
//
// height frame (binding, deck/real_deck.py, byte-identical):
// 0xAA 0x55 | seq u16 | t_apply_ms u32 | n u8 | heights u16[n] (0.01 mm) | crc16
// command frame (bring-up / control channel):
// 0xAA 0x5A | cmd u8 | tag u8 | len u8 (<= 64) | payload[len] | crc16
//
// and one kind flows MCU -> host:
//
// reply frame:
// 0xAA 0x5B | cmd u8 | tag u8 | status u8 | len u16 | payload[len] | crc16firmware/core/scheduler.h:1-10// Frame scheduler: the firmware side of deck.real_deck.FirmwareModel, decision for decision.
//
// receive(frame, t_rx): accept iff n matches and seq is newer than the last accepted seq
// (uint16 serial arithmetic, RFC 1982); an accepted frame feeds the
// watchdog, clears the tripped state and is queued for t_apply.
// update(now): if no valid frame for watchdog_ms (or none yet): tripped, pending
// dropped, last seq forgotten (a restarted host may resume from any seq),
// target = rest. Otherwise the due frame (t_apply <= now) with the latest
// t_apply becomes the target (ties: the newest received); all due frames
// leave the queue.Simulator
SimDeck implements the same Deck interface with MuJoCo physics. Each 10 ms tick it runs many small physics steps (1 ms; 0.5 ms with soft bags), feeds the motor model, and lets the simulated camera take pictures.
sim/deck/sim_deck.py (SimDeck), flexbag.py (soft bags), sim_sensor.py (camera modes), actuator_model.py (motors, lost steps)
sim/deck/sim_deck.py:1201-1224 for _ in range(int(round(dt / ts))):
if motor is None:
d.ctrl[:n] = act.update(d.time, ts)
else:
self._motor_substep(act.update(d.time, ts), ts)
if pusher:
d.ctrl[n] = self._pusher_tick(ts)
if belts:
self._belt_tick()
if kinetic: # friction.kinetic: mj_step split around the per-contact friction (`_kinetic_contacts`)
mujoco.mj_step1(m, d)
if d.ncon:
self._kinetic_contacts()
mujoco.mj_step2(m, d)
else:
mujoco.mj_step(m, d)
if flex:
self._flex_sync()
if d.ncon:
pen = self._contact_stats(pen, pairs)
if pa is not None:
self._liftoff_tick(ts)
if not timed or d.time + _TIME_EPS >= sensor.next_capture_time:
sensor.sample(d.time, self._visible_truth())Shell bag: the newer bag model. One bendable sheet of beads at the bag's bottom that cannot stretch, like real film. It drapes over cap steps.
sim/deck/flexbag.py:95-108 # structure (studies/BAGFIX, docs/research/bag_physics.md): "solid" (default: the RLFLEX model, a 2-layer solid
# lattice whose modulus `young` sets in-plane, compression AND bending stiffness together: a 30 mm, 5 kPa solid bends
# at ~1e-2 N m, 10-1000 x stiffer than a bag of garments, and stretches like rubber where real film is inextensible)
# or "shell": ONE sheet of vertices at the bag's bottom face, a <flex dim=2> with bending only (elastic2d bend,
# stiffness `bend` N m: the contents' bending, folded garments 1e-5 - 1e-3) and an edge-length equality (the
# film's inextensibility, E t ~ 8-23 kN/m: no stretch); the whole mass on the sheet; the tracker reports the bag's
# centre H / 2 above the sheet along its mean normal. With `fill` the content flex covers the content lump and the
# film flaps around it are a second flex of film bending `bend_film` (PE film ~4e-6 N m), hinged to the lump by
# point constraints at duplicated seam vertices. Contents compression and the bag's own thickness as a collision
# volume are not modelled (thin tiles at the sheet).
sphere_frac: float = 0.5 # tile "sphere": bead radius over the vertex spacing (0.5: neighbours touch)
structure: str = "solid"
bend: float = 1.0e-4
bend_film: float = 4.0e-6Camera modes: A = truth plus noise (fast, "ideal"). B = rendered images with AprilTags. C = a statistical outline camera feeding the real outline pipeline.
sim/deck/sim_sensor.py:1-4"""Simulated sensing. Mode A (SimSensor): ground-truth planar poses + Gaussian noise, sampled at camera rate with latency.
Mode B (RenderedTagSensor): renders a MuJoCo camera and runs the product AprilTag pipeline (perception/) on each frame.
Mode C (OutlineSensor): a fast statistical overhead outline camera (no rendering) feeding the product outline pipeline's
blob and association code (perception.outline_pipeline); presets OUTLINE_PRESETS, `apply_outline_preset`."""sim/deck/sim_sensor.py:52-75 def sample(self, t: float, truth: list[tuple[int, float, float, float]]) -> None:
"""Capture (id, x, y, yaw) poses if a capture slot is due at time t."""
if t + _EPS < self._k / self.rate:
return
# Next slot strictly after t: a long gap yields one capture, not a burst.
self._k = math.floor((t + _EPS) * self.rate) + 1
if not truth:
return
noise = self.rng.standard_normal((len(truth), 3))
for (pid, x, y, yaw), (nx, ny, nyaw) in zip(truth, noise):
self._queue.append(Measurement(
id=int(pid),
t_capture=float(t),
x=float(x + self.noise_pos * nx),
y=float(y + self.noise_pos * ny),
yaw=_wrap(yaw + self.noise_yaw * nyaw),
))
def poll(self, t: float) -> list[Measurement]:
"""Pop every queued measurement with t_capture + latency <= t, oldest first."""
out: list[Measurement] = []
while self._queue and self._queue[0].t_capture + self.latency <= t + _EPS:
out.append(self._queue.popleft())
return outConfigs
Each deck setup is one YAML file. sim/config/current_30v.yaml is today's best safe setup on the 30 V hardware: 3 belts, 11 chutes, about 5 m².
sim/config/current_30v.yaml (also current_48v.yaml)
grid,column,actuator: deck size, cap size, stroke, 0.4 m/s and 4 m/s² columns, motor and driver model.sensing,tracker: camera mode, rate, delay, noise; Kalman tuning.control,planner: controller and planner gains and margins.environment.s10: the stream: belts, chutes, flow and traffic keys.
sim/config/current_30v.yaml:12-25name: current_30v
grid: {nx: 44, ny: 50, pitch: 0.048}
column: {cap_size: 0.0455, cap_thickness: 0.006, stroke: 0.12, rest_height: 0.06, mass: 0.0614, joint_damping: 0.0, armature: 1.237,
cap_edge: round, cap_edge_size: 0.002, cap_crown: 0.001}
actuator:
kp: 200000.0
kv: 1019.2
max_force: 49.9
max_speed: 0.4
latency: 0.005
max_accel: 4.0
motor: {model: nema17_40d|tmc2240_30v|tr8x12|120, sizing_factor: 1.0, stall: encoder, thermal: true, hold_current: 0.4,
idle_current: 0.4, power_down_delay: 0.21, sg_latency: 0.005, sg_home_speed: 0.03, sg_home_dwell: 0.33, detect_delay: 0.002,
detect_error: 0.00012, recover_time: 0.02}sim/config/current_30v.yaml:62-76sensing:
mode: A
rate: 200.0
latency: 0.01
noise_pos: 0.0015
noise_yaw_deg: 0.75
tag_size: 0.035
image_size: [1280, 960]
tracker: {accel_noise: 3.0, yaw_accel_noise: 30.0, accel_noise_low: null, yaw_accel_noise_low: null, maneuver_rate: 0.5, quiet_rate: 10.0,
min_meas_pos_sigma: 0.0002, min_meas_yaw_sigma_deg: 0.1, lookahead: null, timeout: 0.25}
control: {rate: 100.0, mu_nominal: 0.23, margin_pitches: 1.0, center_on_support: true, max_speed: 0.4, accel_limit: 1.5, kp_vel: 6.0,
ki_vel: 3.0, pos_tolerance: 0.008, hold_speed: 0.01, brake_gain: 1.0, cup_depth: 0.02, keepout: 0.015, edge_approach: 0.06,
twist_angle_deg: 6.0, deliver_walls: true, deliver_wall_height: null, deliver_wall_anchor: false, deliver_walls_under: false,
deliver_entry_speed: null, accommodate: true, corridor_time: null, stuck_escape: true, mu_static: 0.23, mu_kinetic: 0.16}
planner: {gap_threshold: 0.03, gap_margin: 0.005, replan_period: 0.5, fine_res: null, waypoint_tolerance: 0.015}sim/config/current_30v.yaml:103-111 s10:
control_preset: d2
tube_calibration: sim/config/calibration/tubes_X2S_T99.json
installed_footprint: true
routing: nearest
flow: {gate_slope: 0.19, gate_stall: 0.5, restart: true, giveup_retry: 8.0, stuck_footprint: true, jam_time: 3.0, route_fallback: true,
gate_stall_belt: 2.0, homing_aware: true, homing_pause_max: 2.0, dead_clearance: 0.1, clear_stuck: true, clear_expire: true}
control: {mu_anchor: 0.3, dead_raise_max: 0.02, rehome_sink_n: 0, kp_vel: 9.0, ki_vel: 4.5, cross_track_gain: 6.0, cross_track_max: 0.08,
tilt_slew: 1.5}sim/config/current_30v.yaml:112-128 traffic:
mode: tm
k_routes: 6
detour: 0.15
commit_window: 2.0
deck_cap: 9
cycle_check: true
trunk_groups: true
fallback: true
fb_joint: true
rest_rule: true
stuck: {detect: true, contain: true, mouth: true, clear_reach: 0.8, max_tries: 3, tight_box: true}
soft_late: true
late_relief: 4.0
trains: false
dead_obst: {floor: null, clearance: 0.024, dense: 0.096, refresh: true, confirm: 0.5, divert: true}
follow_live: trueStudies and the cloud
Big runs go to Google Cloud VMs. One script launches them with fixed guards: only project volve-test-01, a hard time limit, and the VM stops (not deletes) when the limit hits. No service account. There is no budget alert on the project, so the time limit and the power-off are the guards.
sim/cloud/launch.sh, common.sh, vm.sh (status, sync, delete); each study's rule and results in studies/<NAME>/
sim/cloud/common.sh:11-20PROJECT="volve-test-01"
REGION="us-central1"
ZONE="${ZONE:-us-central1-a}"
case "$ZONE" in
"$REGION"-[a-f]) ;;
*) echo "ZONE must be a zone of $REGION (got $ZONE)" >&2; exit 2 ;;
esac
KEY="$HOME/Documents/keys/volve-test-rl-runner.json"
GCLOUD="${GCLOUD:-gcloud}"
MAX_VMS="${MAX_VMS:-3}"sim/cloud/launch.sh:94-101 if run_gc compute instances create "$n" --project="$PROJECT" --zone="$z" \
--machine-type="$MACHINE" \
--provisioning-model="$PROVISIONING" --max-run-duration="${MAX_RUN_H}h" --instance-termination-action=STOP \
--image-family=ubuntu-2404-lts-amd64 --image-project=ubuntu-os-cloud \
--boot-disk-size="${DISK_GB}GB" --boot-disk-type=pd-balanced \
--no-service-account --no-scopes --shielded-secure-boot \
--labels=purpose="$PREFIX",owner=abyl,wave="$NAME" \
--metadata="block-project-ssh-keys=TRUE,enable-oslogin=FALSE,stawamus-workers=$WORKERS,stawamus-commit=$COMMIT" \Tests
The pytest suite lives in sim/tests/ (controller, planner, flow, bag motion, firmware core, determinism…) and software/link/tests/. One test guards the whole architecture: software/ must never import the simulator.
software/link/tests/test_layering.py
software/link/tests/test_layering.py:8-21SOFTWARE = Path(__file__).resolve().parents[2]
SIM_PACKAGES = {"deck", "experiments", "rl", "viz", "tools", "mujoco"}
def test_software_never_imports_sim():
bad = []
for f in (p for p in SOFTWARE.rglob("*.py") if p.name != "test_layering.py"):
for node in ast.walk(ast.parse(f.read_text(), str(f))):
names = [a.name for a in node.names] if isinstance(node, ast.Import) else \
[node.module or ""] if isinstance(node, ast.ImportFrom) and node.level == 0 else []
bad += [f"{f.relative_to(SOFTWARE)}: {n}" for n in names if n.split(".")[0] in SIM_PACKAGES]
if '"sim"' in f.read_text() and "sys.path" in f.read_text():
bad.append(f"{f.relative_to(SOFTWARE)}: puts sim/ on sys.path")
assert not bad, "software/ imports the simulator:\n" + "\n".join(bad)4 · As of 2026-10-19 evening
What we are doing now
a · RLSTREAM: train the bag motion on the full deck running overnight
The single-bag policy did not help on the full deck. So now it trains on the full deck stream itself, in the cloud. Every soft bag with a moving goal is an agent; all share one policy. Reward: its own progress, plus a shared bonus for every correct delivery on the deck, minus a penalty if it stalls its neighbours.
Five arms, one VM each, 12 h of learning, then about 3 h of development rows. Estimated cost: about USD 390 at list price (5 VMs × ~15.5 h); the held-out run after it about USD 90 more.
sim/rl/stream/launch_train.sh:11-15 RLSTREAM branch# a pair frozen RLBAG skill pair (+ a dither output, off at the start), default reward
# b pairdither the pair with the dither at 6 mm from the start
# c dither one network starting as the plain 6 mm dither (struct outputs zero)
# d pairdither stronger deck terms: team 3, neighbour stall -0.05
# e move one network: the frozen moving skill everywhere (+ dither, off), team 2sim/rl/stream/launch_train.sh:36-40 RLSTREAM branchlaunch a pair base 5000000
launch b pairdither dither 105000000
launch c dither base 205000000
launch d pairdither dither 305000000 "--rew team=3,disturb=-0.05"
launch e move none 405000000 "--rew team=2"sim/rl/stream/world.py:18-29 RLSTREAM branchREWARD per decision (the ticks until the agent's next decision), weights R (JSON overrides --rew):
own progress R.progress x the bag's true CoM progress along its goal direction (m; +-2 cm per tick cap)
stall R.stall per decision while the bag's true speed < 3 cm/s (0.3 s after its first decision)
effort R.effort x mean(a^2) + R.deffort x mean(da^2)
terminal R.delivered (correct bin), R.wrong, R.fallen, R.hung, R.giveup (the jam breaker gave it up)
team every open agent: R.team per correct delivery anywhere on the deck (box or bag) and R.team_loss per
parcel lost anywhere (wrong bin, floor, hung): the deck's throughput, shared
neighbours R.disturb per decision for each other parcel with a moving goal that stands still (< 3 cm/s) within
NEAR_DILATE of the columns this bag's motion moved (every 5th tick); R.disturb_loss when a parcel this bag's
motion came within NEAR_DILATE of in the last TOUCH_S is lost
Credit assignment: the own and neighbour terms are the agent's; the team term is the deck's (all open agents), so the
critic (per agent, with deck counts in its observation) learns what the deck makes of the bag's motion.The learner updates while the workers keep simulating, so every core works all the time (RLBAG used about a third of each VM).
sim/rl/stream/appo.py:1-14 RLSTREAM branch"""studies/RLSTREAM learner: asynchronous PPO on chunks from many rollout workers.
The workers act with a policy that may be a few versions old (behaviour mu). Each update ("decoupled PPO", Hilton et
al. 2021, with IMPALA's V-trace targets):
- prox = the learner's policy at the start of the update; w = min(1, prox / mu) per decision (truncated importance
weight of the stale behaviour);
- V-trace over every chunk with rho = c / lam = min(1, prox / mu): value targets vs, advantages
A = rho (r + gamma vs' - V); the chunk's end bootstraps from its next observation (0 when the agent's bag
finished);
- PPO's clipped objective on r = pi / prox, weighted by w; advantages normalised per minibatch; a Huber value loss
on vs; epochs stop early once the KL to prox passes 1.5 x target_kl (summed over the 9 action numbers);
- the first critic_warmup updates train the critic only (the warm-started actor is not moved by an untrained
critic's advantages).
Rewards are scaled by rew_scale before any of this.Then: the best candidate is chosen mechanically on development seeds 17001-17012 and frozen. One held-out run on fresh seeds 1800-1805 (4 streams each, 900 s windows) compares it with today's controller and the dither. Verdict YES only if it beats both at both loads without losing more parcels. Rule: studies/RLSTREAM/rule.txt on the branch.
b · Camera collapse fix: recheck VM
In 22 of 108 camera trials the deck nearly stopped. Cause: columns under a stuck bag kept re-homing, and every re-homing restarted the planner's watchdog, so nothing was ever given up. Two fixes: control.rehome_sink_n = 3 holds such columns low; flow.homing_pause_cumulative makes the watchdog count all pauses together. The recheck reruns the affected cells (66 trials). If clean, both become defaults. See studies/CAMERA/README.md, "The collapses".
c · Bench tests in Seattle waiting on you
docs/pages/BAG_SLIDE_TEST.html: friction, bending and the staircase test on real mailers. This decides whether the shell bag model is right. Then the dither on a real column: does it stay within the motor limits (no lost steps, no stalls), and how loud is it? Until then bag_motion stays off on the real deck.
Results so far (all simulation)
- Single bag, RL (held-out)
- 35 → 71 % bags delivered (35.4 → 71.4 %). Dither alone 44.3 %.
- Full-deck stream, studies 1 and 2
- RL: NO (rate a TIE with today's controller). Dither: +140 to +174 /h over base, BETTER (study 1 at sat: +74 /h).
- Shell bag model vs rigid
- Today's controller, research mix: 153-167 /h with shell bags against 3,007-3,567 /h with rigid stand-ins (about −3,400 /h, −95 %). With the dither the stream studies reached 160-240 /h.
- Camera
- Delay matters, not frame rate: about 10 ms end to end; 60 fps is enough. Recommended: Basler a2A1920-160umBAS. Measure the delay on the bench.
5 · Next
Future: deep RL, and first a faster simulator
Today's policy is a small network that sees one moment, one bag and a 9 × 9 patch. Three steps make it smarter, in order:
- Memory. A recurrent or transformer policy remembers the last seconds. Then it can notice "this bag has not moved for a while" and change tactic, instead of repeating the same motion.
- Attention over nearby parcels. The policy looks at every parcel near its bag (position, speed, goal) and learns whom to give way to. That is coordination, which the full-deck study showed is missing.
- Whole-deck RL, later. One policy shaping the whole height field, or deciding routes and timing too. Only after the first two work, because it needs far more simulation.
First step: simulation speed
Every step above needs many more simulated hours. Today the full-deck stream runs about 19 times slower than real time on one CPU core, and physics is about 89 % of a worker's time (estimates in studies/RLSTREAM/README.md on the branch). Soft bags need 0.5 ms physics steps: 20 per control tick.
Profile first. Time one full-deck step and split it into: MuJoCo contact and solver, the shell bags' flex, the motor model, the controller, the planner, and Python glue. The 89 % figure is a rough estimate; the split inside physics is not measured yet. Then fix the biggest piece.
| Candidate fix | Expected gain | Effort | Status |
|---|---|---|---|
| Profile one full-deck step (where does the time go?) | tells us which row below matters | low | proposed, first |
| CG contact solver instead of Newton | about 10-12× on bag scenes, same outcomes | low | done (RLBAG, RLSTREAM) |
Contact pruning: caps collide only near parcels, parked parcels inert (physics.fast) | large on a big deck | done | done |
Controller on a window of the grid around each parcel (geometry.DeckWindow) | large on a big deck | done | done |
| Async workers on all cores | about 3× per VM (RLBAG used a third of the cores) | medium | done (RLSTREAM) |
| Solver settings: iterations, tolerance, timestep for bags | unknown until profiled; possibly 1.5-2× | low | proposed |
| Coarser bag sheet, fewer bead contacts | possibly large (contacts scale with beads) | low-medium; must re-match the bench tests | proposed |
| Local sub-deck worlds around one bag, started from saved full-deck snapshots | large per bag decision | medium | proposed |
| Batch the controller (and planner) over many worlds | needed for any GPU path | high | proposed |
| Check MuJoCo Warp (GPU) support for our flex shell bags | very large if supported | low to check, high to adopt | proposed (not checked) |
| Learned surrogate of the bag physics | very large | high; needs lots of data first | later |
Gains marked "possibly" or "large" are expectations, not measurements. Only the CG and core-use numbers come from studies.
Why a GPU needs the planner and controller batched too. A GPU simulator is fast because it steps thousands of decks at once. But every 10 ms each of those decks needs new column heights from the planner and controller, and ours run one deck at a time in Python on the CPU. The GPU would then sit idle waiting, and data would shuttle back and forth every tick. So the planner and controller must also run for all decks at once (array code for many worlds), or the GPU path must use a simplified stand-in for them.
6 · Words
Glossary
- tick
- One pass of the control loop: 10 ms (100 Hz).
- stroke
- How far a column can travel: 120 mm (0 to 0.12 m). Rest height is the middle, 60 mm.
- pitch
- Distance between column centres: 48 mm.
- cap
- The flat square top of a column (45.5 mm wide) that parcels rest on.
- column
- One motor + lead screw + cap. The deck is a grid of them.
- footprint
- The rectangle a parcel covers on the deck, seen from above.
- induction
- Putting a parcel into the system: scanned (size, mass, kind) and loaded on an infeed belt.
- chute
- An opening at the deck edge leading to a destination bin.
- 1x / sat
- Load levels. 1x = 5,500 parcels/h offered to the feed; sat (saturation) = 3×, 16,500/h, so the belts are always full.
- held-out seed
- A random scenario kept unused until the final test, so the result is not tuned to it.
- dither
- Small random up-and-down cap motion (6 mm, new every 50 ms) to break a bag loose.
- policy
- The learned decision maker: observation in, action out. Here a small neural network.
- PPO
- The standard RL training method used here: improve the policy in small, safe steps.
- flex / shell bag
- MuJoCo's deformable "flex" bodies. The shell bag is one bendable, non-stretching sheet at the bag's bottom.
- Mode A / B / C
- Simulated camera types: A ideal (truth + noise), B rendered AprilTags, C statistical outline camera through the real pipeline.