What this is: A post-crash investigation into why our control bridge received zero position messages during its first Gazebo run, even though the publisher was clearly broadcasting them.
Why it’s here: The culprit was a subtle ROS2 messaging-policy mismatch that produces no error, no exception, and no log line — just silence. It is one of the most common traps when wiring new software into an existing robotics stack, and we want it documented so nobody loses another afternoon to it.
Date: 2026-05-19 Ticket: TASK-059 (first model-to-sim bridge run)
Glossary
- QoS (Quality of Service): A set of “delivery preferences” attached to every ROS2 connection. Think of it like choosing between registered mail and a regular postcard: registered mail guarantees delivery, a postcard is fast but might get lost. In ROS2 the sender and receiver each pick their own preferences, and if they don’t agree, the connection silently never forms.
- DDS: Data Distribution Service. The networking layer underneath ROS2 that actually moves messages between programs. QoS rules live here, which is why a mismatch fails quietly at the network level rather than crashing your code.
- Reliability: A QoS setting with two values.
RELIABLEmeans “resend until it arrives” (the registered mail).BEST_EFFORTmeans “send once, don’t worry if it’s lost” (the postcard) — perfect for fast sensor streams where the next reading is always seconds away. - Durability: A QoS setting deciding whether late-joining receivers get older messages.
VOLATILEmeans “you only get what’s sent after you connect.”TRANSIENT_LOCALmeans “you also get the last message I sent before you arrived” — useful for state you join into mid-stream. - History: How many past messages the system keeps in its buffer (for example, “keep the last 5”).
- Publisher / Subscriber: The sender and the receiver. A publisher broadcasts to a named topic; a subscriber listens on it.
- Topic: A named channel, like
/mavros/local_position/odom, that carries one kind of message.
1. The symptom: a confident drone flying into the wall
On the first live run of our control bridge, the drone took off and drove straight forward at full command — action “forward” — with no reaction to its surroundings. From the outside it looked like a logic bug. From the inside, the bridge thought it was doing everything right.
The chain of failure looked like this:
- The bridge subscribed to the drone’s position stream (odometry) but received zero messages.
- With no position data, its internal coverage map never updated and stayed at zero.
- The policy kept acting on the same stale, empty observation.
- With no feedback ever arriving, it just kept issuing “forward” forever.
Crucially, nothing crashed and nothing logged an error. The position publisher was running and broadcasting normally — we confirmed that. The bridge subscriber was alive and waiting. They simply never connected.
2. The audit: inspecting QoS on the live stack
Rather than guess, we inspected the running system before tearing it down. ROS2 ships a tool that prints the exact QoS settings each side of a topic is using:
ros2 topic info /mavros/local_position/odom --verbose
We ran this across every topic the bridge touches and saved the full dump as an artifact. Here is the publisher map we built from it — the QoS each producer was actually advertising:
| Topic | Publisher | Reliability | Durability | Bridge subscriber compatible? |
|---|---|---|---|---|
/mavros/local_position/odom |
mavros (local position) | BEST_EFFORT | VOLATILE | ❌ default RELIABLE — incompatible |
/drone/perimeter |
sensor_monitor | RELIABLE | VOLATILE | ✅ match |
/scan/sweep |
Gazebo bridge | RELIABLE | VOLATILE | ✅ match |
/drone/tf_luna_down |
Gazebo bridge | RELIABLE | VOLATILE | ✅ match |
/drone/vl53l0x/ch[0-5] |
Gazebo bridge | RELIABLE | VOLATILE | ✅ match |
joint_state (iris model) |
Gazebo bridge | RELIABLE | VOLATILE | ✅ match |
/mavros/state |
mavros (system) | RELIABLE | TRANSIENT_LOCAL | ⚠ needs TRANSIENT_LOCAL to read last cached state |
/clock |
Gazebo bridge (×2 pubs) | RELIABLE | VOLATILE | ✅ match |
One row stands out. The odometry topic — the single most important feed for closing the control loop — publishes as BEST_EFFORT. That is the right choice for mavros: position arrives many times a second, and an occasional dropped frame is harmless. But our bridge subscribed to it with the default reliability, which is RELIABLE.
3. The rule that bit us
ROS2’s documentation spells out the compatibility rule precisely. Reliability settings are compatible in every combination except one:
| Publisher Reliability | Subscriber Reliability | Compatible? |
|---|---|---|
| RELIABLE | RELIABLE | ✅ |
| RELIABLE | BEST_EFFORT | ✅ (subscriber still gets messages) |
| BEST_EFFORT | RELIABLE | ❌ incompatible — subscriber gets zero |
| BEST_EFFORT | BEST_EFFORT | ✅ |
The intuition: a RELIABLE subscriber is demanding a guarantee the BEST_EFFORT publisher refuses to make. Since the publisher won’t promise resends, the two cannot agree on terms, and DDS simply declines to connect them. No error — because from each side’s point of view it is behaving correctly. It is the receiver insisting on registered mail from a sender who only does postcards.
That single mismatch is the whole story of the flyaway. The bridge asked for guaranteed odometry; mavros offered best-effort odometry; DDS connected nothing; the bridge ran blind.
4. The fix
The subscriber needed to match the publisher’s terms. ROS2’s Python library ships a ready-made profile for exactly this kind of high-rate sensor feed:
node.create_subscription(
Odometry,
odom_topic,
callback,
qos_profile_sensor_data, # BEST_EFFORT + VOLATILE + keep-last buffer
)
qos_profile_sensor_data sets reliability to BEST_EFFORT with a small recent-message buffer — a perfect match for the mavros odometry publisher. The bridge’s other subscriptions (perimeter, sweep, joint state) were already on the default RELIABLE setting, which matches their RELIABLE publishers, so they were left untouched.
On the publishing side, the bridge’s own outputs (action, coverage, visited-grid, velocity command) all use the default RELIABLE setting. That is safe: a RELIABLE publisher feeding a BEST_EFFORT subscriber is one of the compatible combinations, so the bridge’s velocity command reaches mavros without issue.
5. Verification plan
The patch is in. Confirming it works has two stages:
- [x] Patch applied to the bridge’s observation builder.
- [ ] Restart the bridge against a mock feed and confirm the coverage map tracks the synthetic input.
- [ ] Restart against real Gazebo, confirm no “incompatible QoS” warning appears, and confirm the odometry topic echoes messages into the bridge.
6. Topics still in the risk zone
We only audited what the bridge currently touches. Several mavros feeds we may wire in later carry the same trap and will need matching QoS before use:
/mavros/imu/data—BEST_EFFORT/mavros/global_position/global—BEST_EFFORT/mavros/extended_state— possiblyTRANSIENT_LOCAL(relevant if we ever read landing state)
The pattern is consistent: nearly every /mavros/* sensor stream is BEST_EFFORT, and a default subscriber will fail silently against all of them.
7. Lessons for the next integration
Two habits would have caught this before it ever flew:
- Every
/mavros/*sensor topic isBEST_EFFORT. A default subscriber asks forRELIABLEand will quietly receive nothing. Use the sensor-data QoS profile when subscribing to them. - Audit QoS before you write the subscriber. When connecting a new program to an existing topic, run
ros2 topic info --verbose <topic>first and read the publisher’s reliability and durability. Match them deliberately rather than inheriting whatever default your code library hands you.
The deeper lesson is that the most expensive bugs in a robotics stack are often the ones that don’t announce themselves. A crash points you at the problem. Silence makes you doubt your own logic for an hour before you think to question the plumbing.
References
- Quality of Service settings — ROS2 Documentation — “all combinations are compatible except for when the publisher uses Best Effort and the subscriber uses Reliable”
- QoS profile for ROS2 in Python — Dronecode Forum — example of configuring a custom QoS profile
- Publishers and Subscriptions — rclpy reference — rationale behind
qos_profile_sensor_data