What this is: A cleanup of a word we kept overloading. “Semantic” was meaning three completely different things in our pipeline, and they were quietly getting tangled. This entry pulls them apart and, while I was in there, splits the drone’s live understanding of the world out of the raw flight file and into its own companion file.
Why it’s here: Because confusing the world’s geometry with the mission’s intent with the drone’s evolving belief is exactly the kind of muddle that produces bugs you can’t explain. Naming things correctly is half the engineering. The other half here was a small, satisfying plumbing change that makes replays cleaner.
Date: 2026-06-15 Ticket: rl-lab dev-log 39
Glossary
- Semantic artifacts — fancy words for “data that carries meaning, not just raw numbers.” A list of coordinates is data; a list of coordinates labeled “this is a wall, this is the goal, stay out of here” is semantic. The whole point of this entry is that we had three of these meaning-carrying files and were treating them as if they were one.
- Sidecar file — a companion file that travels right next to a main file, sharing its name. Think of a photo named
vacation.jpgwith a tinyvacation.txtsitting beside it holding the caption. You can open the photo alone, but if you want the caption, the program knows exactly where to look. The two are linked by name. - Split — here it just means “stop cramming two different things into one file; give each its own file.” Like separating your grocery receipt from your shopping list — both belong to the same trip, but they answer different questions.
- Keyframe — a complete, full snapshot saved once every few seconds, so you can jump (scrub) to any point in a replay without replaying everything from the start. Like chapter markers on a video.
1. What I wanted
I wanted to stop saying “the semantic map” as if it were one thing, because by this point it was secretly three things wearing the same coat. Every time someone said “semantic” in a code review or a message, we each pictured something slightly different, and that mismatch was starting to cost us.
So the first goal was purely conceptual: write down, plainly, the three distinct artifacts and who owns each one.
-
The simulation map. This is just the world with its walls — pure geometry. Where the rooms are, where the obstacles sit. The simulation produces this. It has no opinion about the mission; it’s the stage, not the play.
-
The semantic mission map. This is the training assignment: where the flight starts, where it should go, which zones are off-limits. This is an input. It’s authored by hand — by Aleks, and by me. (I might not draw mine in an editor; I can generate it programmatically, but I keep it in the exact same format so Aleks can open it and see what I meant.)
-
The flight snapshot. This is the painting the drone makes itself, live, while it flies and scans. It’s the drone’s evolving belief about the world — its “I think there’s a wall here, I’ve covered that area, this part is still unknown.” This exists purely for the Player, so a human can watch how the drone understood the world moment to moment.
Once those three are named, the second goal followed naturally: the flight snapshot (artifact 3) had been getting written into the wrong place, and I wanted it living on its own.
2. What I tried
The day before, I had made a sloppy call: I wrote the drone’s live painting (artifact 3) straight into the flight file itself. Aleks pushed back — that belongs in its own save. He was right; mixing the raw record of what happened with the drone’s interpretation of what happened is exactly the kind of overloading this whole entry is trying to kill.
So I split them into a pair:
- the flight file
<name>.flight.jsonl— the trajectory, the scans, the events. The objective record. - a sidecar beside it
<name>.semantic.jsonl— the drone’s painting over time. The interpretation.
The part Aleks actually asked for, and the part I’m happiest with, is the loading mechanism. You pick one file — the flight file — and the Player goes and finds the paired painting file on its own, loads it, and plays them back together in sync along the timeline. You never have to remember to open two things.
The link between the pair is held two ways, deliberately belt-and-suspenders:
- by shared name —
cleanroom_run_07.flight.jsonlandcleanroom_run_07.semantic.jsonlobviously belong together; - by an explicit reference — each file names its partner inside itself, so even if a file got renamed, the pairing isn’t lost to a typo.
3. What happened
It worked, and the conceptual cleanup turned out to matter as much as the code. Here’s how ownership and storage shook out:
| Artifact | What it is | Who authors it | Where it lives |
|---|---|---|---|
| Simulation map | The world’s walls (geometry) | The simulation | Sim output |
| Semantic mission map | Mission assignment (start / goal / no-go zones) | Authored by hand — Aleks and me | semantic_mission/ (Aleks → rl_manual, me → rl_auto) |
| Flight snapshot | Drone’s live painting of the world | My agent (the model’s own belief) | <name>.semantic.jsonl sidecar |
A few things fell out of this neatly:
- The flight snapshot (artifact 3) is written by my agent — it’s the model’s own painting, exactly as Aleks framed it. Nobody hand-authors this one; it’s a recording of the drone’s mind.
- The mission drafts (artifact 2) we both author, into one shared folder
semantic_mission/— Aleks’s go in therl_manualsubfolder, mine inrl_auto. Same format on both sides, so either of us can open any file the other made and immediately read it.
Crucially, none of this touches training itself. The painting is written out as a separate file purely for viewing — it’s an observability artifact, not a signal the learner consumes. The PPO loop runs exactly as before; I just made its replays easier for a human to inspect, and I tidied up what we call things so we stop talking past each other.
4. Sources
- The flight files and their new
.semantic.jsonlsidecars produced by recent rl-lab runs. - The
semantic_mission/folder, with therl_manualandrl_autosubfolders holding hand-authored mission drafts. - The Player’s loading path, now extended to discover and sync the paired sidecar.
5. What’s next
The pairing is in place and the vocabulary is settled, so the obvious follow-ups are about polish and reuse: keyframing the flight snapshot so scrubbing a long replay is instant rather than a slow replay-from-the-top, and making sure that when either of us drops a new mission draft into semantic_mission/, it’s trivially loadable on the other side. Mostly, though, the win here is quiet: three things now have three names, and the drone’s belief lives in its own file where it belongs.