What this is: the day the drone started writing down what it understands about a room while it flies — which cells it has seen, which are still a mystery, and which are dangerously close to a wall — and the day the Player learned to draw all of that on top of the flight track.
Why it’s here: until now the replay showed only the bare facts of a flight (where the drone went, where it pinged). This entry connects the drone’s internal “mental map” to the viewer, so you can finally watch the agent’s sense of the room grow in real time instead of guessing at it.
Date: 2026-06-15 Ticket: semantic sidecar wiring (rl-lab dev-log 46)
Glossary
- Semantics (semantic map): a meaning-layer painted over the floor plan. Instead of “wall / not wall”, it answers softer questions: is this spot safe? is it hugging a wall? have I seen it yet, or is it still a blind spot? Think of a hiker coloring a paper map as they walk — green for “I’ve walked here and it’s fine”, red for “edge of a cliff”, grey for “no idea what’s over there”.
- Sidecar: a second file that travels right next to the flight recording, holding that meaning-layer indexed by time. Picture a movie and its subtitle file: the subtitles aren’t baked into the video, they sit alongside it and the player loads them on demand. The flight-log is the movie; the semantic file is the subtitles.
- Player / Map Studio: our replay viewer. It already knew how to show the flight; now it also pulls in the semantic sidecar and paints it.
- sensed, not visited: a cell gets colored the moment the drone sees it with a sensor beam — not when it physically flies into it. This is the “anti-vacuum-cleaner” rule: a robot vacuum only learns a spot by bumping into it; we want a drone that learns a room the way a person scanning a dark room with a flashlight does — you understand the far wall without walking up and touching it.
1. What I wanted
One screen, three truths laid over the same floor plan:
- the trajectory — the path the drone actually flew;
- the scan points — where its sensors pinged the world;
- the dynamic semantics — the labels the drone assigns to space for itself, on the fly, as the flight unfolds.
The third one is the new piece. The first two are just geometry; the third is the agent’s opinion about the room, and that’s the part I’d never been able to see before.
2. What I tried (and built)
The lucky part: the interface side had already taught the Player to understand my coloring format, and we’d cross-checked it earlier — it reads back one-to-one with what I write. So this entry wasn’t about inventing a viewer. It was about finally starting to write the coloring down.
Now, every time a flight is recorded, a companion coloring file is born next to it. Two things get painted:
- A growing map of knowledge. Cells the drone has seen via a sensor beam flip to explored. The frontier — the ragged edge between explored and the void — gets its own label, frontier, because that’s exactly where it’s worth flying next (you go to the edge of what you know to push it outward). Everything the drone hasn’t looked at stays unknown. It’s the fog-of-war from a strategy game, lifting as the drone scouts.
- Wall-proximity safety. Cells hugging a wall are painted danger; cells with plenty of open air around them are open; the band in between is the safe corridor. So the map doesn’t just say where the walls are — it says how comfortable each spot is to be in, the way a driver instinctively reads “tight squeeze” vs. “wide lane”.
The crucial design choice — and the one you asked for — is that all of this is keyed on visibility, not footprints. A cell colors because the drone saw it, not because the drone flew through it. No vacuum-cleaner behavior.
3. What happened
I ran a flight through the apartment map and watched the semantic layer fill in.
| Check | Result |
|---|---|
| Cells colored by ~40s of flight | ~3,400 |
| Coloring vs. Player’s own reader | Matches (verified with the same function the Player uses to assemble it) |
| Flight file ↔ coloring file cross-reference | Correct; Player picks up the sidecar automatically |
The headline number: by the 40-second mark, roughly 3,400 cells had been painted in. I verified it the honest way — by running the exact function the Player uses to gather the coloring, so I’m comparing against the real consumer, not a side script that might lie to me. The flight file and the coloring file point at each other correctly, so the Player resolves the sidecar on its own with nothing extra to wire up.
While I was in there, I also fixed an old broken test. It had been asserting the previous start position, which quietly shifted after the maps were rebuilt — a stale expectation rather than a real regression, but it was failing loudly so it’s now green again.
4. How to look at it yourself
Open the apartment episode’s flight-log in Map Studio:
interface/save/rl/a1_apartment/a1_apartment_ep1.flight.jsonl
You’ll see the full picture stacked together: the flight track, the drone’s tilt, the scan points, and — new this time — the colored dynamic semantics painting in as the replay plays.
5. Sources
- Apartment episode flight-log:
interface/save/rl/a1_apartment/a1_apartment_ep1.flight.jsonl(plus its companion semantic sidecar). - Map Studio / Player semantic reader (the same function used to verify the coloring round-trips correctly).
6. What’s next
The semantic layer is now visible, which is the precondition for everything downstream. The natural follow-ups: lean on the frontier labels to reason about where exploration should head, and keep validating that the sensed-not-visited rule holds up as flights get longer and rooms get more cluttered. For now, the win is simple and concrete — the drone’s understanding of a room is no longer invisible.