What this is: a drop-one-input test on EXP-7, our best model so far. We checked which input sensors actually influence navigation by feeding the model noisy inputs — muting one “channel” at a time — and watching how much coverage fell. Why: before a long, expensive augmentation run, we want to know which sensors are worth augmenting. It also helps prepare for moving onto real hardware: fewer channels on the drone means fewer sources of noise.
Date: 10 May 2026 Machine: D2 (RTX 5070)
1. What we wanted to check
EXP-7 takes three input channels:
- distances — seven distance readings: six short-range ToF sensors arranged around the drone at even intervals, plus one long-range TF-Luna mounted on a servo.
- servo angle — where the long-range sensor is pointing, expressed as a value between 0 and 1.
- visited map — a grid that records “where I’ve already been” (marked if a cell was visited, empty otherwise).
Main question: are all of these channels actually needed? Or does the agent rely on only some of them, with the rest acting as noise?
We went in with a few expectations: maybe there’s a smaller, minimally-sufficient set we could keep without losing quality; maybe one or two channels are effectively empty and the agent ignores them; or maybe every channel matters and nothing can be dropped.
Why it fits the mission: a good model is also a simple model. Fewer inputs mean less noise on the real drone, simpler decision-making, and a clearer picture of what the agent is actually relying on.
2. What we did
Step A — sensitivity analysis
For each step we asked: how much does the model’s decision change if we nudge this channel’s value a little? The more it reacts, the more sensitive it is to that channel.
Step B — drop-channel sweep
One channel at a time, we muted it (replaced its values with nothing) and measured coverage across five maps, five episodes each, and three episode lengths (short, medium, long). No retraining — we simply checked how well the already-trained model held up when it lost a channel.
The cases we tested:
- mute the visited map
- mute all seven distance readings
- mute only the six short-range ToF sensors (TF-Luna still active)
- mute only the TF-Luna (the six short-range ToF still active)
- mute the servo angle
- mute nothing (baseline)
3. What we saw — the main table
Coverage drop (Δ = muted − baseline, in percentage points)
| Channel muted | Δ short | Δ medium | Δ long | Verdict |
|---|---|---|---|---|
| visited map | −29.9 | −52.4 | −55.3 | breaks the model |
| six short-range ToF | −41.3 | −58.6 | −58.8 | breaks the model |
| all distances | −42.9 | −59.0 | −55.4 | breaks the model |
| TF-Luna (long sensor) | −0.3 | −3.6 | −1.9 | within noise |
| servo angle | −1.4 | −1.9 | −0.4 | within noise |
Reading the table:
- Mute the visited map or the six short-range ToF and the model falls apart (a drop of roughly 30 to 59 points). These inputs carry the model.
- Mute the servo angle and almost nothing changes (drop of at most 1.9). This channel looks like a safe candidate to remove.
- Mute the TF-Luna and there’s only a small dip at medium length, with short and long both inside the noise. Borderline — droppable.
Sensitivity (how strongly the model reacts to changes in a channel)
Looked at per element of each channel, the distance readings drew by far the strongest reaction, the servo angle a much weaker one, and each individual cell of the visited map only a tiny one — though with thousands of cells the visited map’s cumulative signal is large.
The puzzle: the TF-Luna draws a strong reaction on par with the regular ToF sensors, yet removing it barely changes anything. The model is locally sensitive to the TF-Luna but doesn’t actually depend on it for basic navigation. It seems to treat the TF-Luna as a secondary signal it can do without.
A counter-intuitive observation
When we muted only the six short-range ToF (leaving TF-Luna active), coverage dropped sharply. When we muted all seven distances (including TF-Luna), coverage was actually a little better than that.
In other words, dropping only part of the group is worse than dropping all of it. The likely reason: the model learned to read all seven distances together. When six are blank but the seventh still reports a real number, that looks like corrupted data rather than missing data. When everything is blank, the model recognises the data is simply gone and falls back to navigating by the visited map.
What this means for hardware: one broken sensor is worse than the whole group going offline. If a single ToF on the real drone fails and starts sending garbage, the agent degrades badly. If all the ToF sensors go dark at once (a power loss, say), the agent still manages.
4. Main takeaways
-
The servo angle can be removed. Both the sensitivity analysis and the drop test agree. One fewer channel means less wiring on the drone and simpler maintenance.
-
TF-Luna is borderline. It could go, but we’d recommend keeping it as a small buffer. Removing it saves a sensor module at the cost of a slight regression at medium length.
-
The six short-range ToF and the visited map are essential — don’t touch them. They are the core inputs, and any future augmentation work should target them first.
-
Hardware failure mode: one broken sensor is worse than all of them broken. We need to account for this when moving to real hardware: test a “one sensor noisy” case separately from an “all sensors noisy” case.
5. What’s next
- A map-augmentation run is the top next step: augment the short-range ToF and the visited grid with rotations, flips, and crops. The servo angle and TF-Luna stay out of it.
- Then the move onto real hardware: noise calibration focused on the short-range ToF, with the servo angle ignored and TF-Luna treated as low priority. The single-sensor failure mode gets its own test case.
- An optional, low-priority follow-up: retrain EXP-7 on the leaner set of inputs (without the servo angle) to confirm parity holds under retraining, not just under after-the-fact muting.
Glossary
- Observation — everything the agent sees on each step. Ours has three parts: distances, servo angle, and the visited map.
- Channel — one part of the observation. For example, the visited map is a single channel.
- Drop-channel — muting a channel by blanking out its values.
- Sensitivity — how strongly the model reacts to a small change in an input. High sensitivity means the model is responsive to that input.
- Distortion vs absence — the model is more robust to absence (everything blank) than to distortion (some values blank, some real).
- Map augmentation — a planned experiment: rotating, flipping, and cropping maps during training so the agent generalises better.
- Sim-to-real — moving from simulation to real hardware.