Files
firmware/src/mesh/ReliableRouter.cpp
T
22072c5f4b Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store

Reworks the TrafficManagementModule cache layer (policing behaviour unchanged
from upstream) and adds a routing-hint overflow store:

- Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed
  PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as
  WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible,
  and it removes a large amount of hashing/displacement complexity. The cache
  entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always
  means "empty" across every sub-store. Adds rebaseEpoch() so cached state
  survives the ~19 h relative-timestamp horizon instead of being flushed.

- Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte
  relay for a destination, written only from NextHopRouter's ACK-confirmed
  decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back
  to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail
  nodes keep routing after the node ages out of NodeInfoLite.

- Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted
  NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive
  across the maintenance sweep (no TTL) and never clobbered by a stale preload.

All packet-policing logic (rate limit, position dedup, unknown-packet drop,
NodeInfo direct response, hop exhaustion) is the existing upstream behaviour,
untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note).

Tests: upstream policing suite now actually runs (adds the MeshTypes.h include
that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles,
politeness, precision clamp, port-interval and mesh-radius gating — and the
rate-limit >255 saturation fix — are deferred to the advanced-TMM branch.

Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* TrafficManagement: fix cppcheck constVariablePointer warning

`node` in preloadNextHopsFromNodeDB() is never written through — mark
it const to satisfy cppcheck's constVariablePointer check in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add multi-hop NextHop recovery tests and unit tests for routing reliability

- Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop.
- Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering:
  - M1: Ambiguity-aware last-byte resolution.
  - M2: NextHopRouter's strict-neighbor gate and hop limit checks.
  - M3: Route-health freshness and failure decay.
- Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic.

* grafting fixed

* Address Copilot review for PR #10735 (NextHop improvements)

- docs/nexthop-routing-reliability.md: update status from "no code
  changes yet" to reflect that mitigations and tests are implemented

RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in
PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose
default=0) respectively; (0,0) sentinel fixed in PR2.5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* CI: fix cppcheck constVariablePointer and test include path

- NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only
  read for stale-route checks, never mutated through the pointer
- Router.cpp: qualify meshtastic_NodeInfoLite *node as const in
  shouldDecrementHopLimit — only read for favorite/role predicate
- test_position_module/test_main.cpp: change bare PositionModule.h to
  modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules,
  so the bare form fails to resolve in the native PlatformIO test env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* WarmStore: cache device role + protected category in last_heard low bits

Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's
device role (4 bits) and a protected category (2 bits) for the hop-trim path,
at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits
remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU
ordering of long-tail nodes.

- absorb() packs role/protectedCat; place()/ring replay store the raw word so
  metadata round-trips through flash. LRU compares masked time (warmTimeOf).
- take() rehydration masks the metadata bits and restores the cached role so a
  re-admitted node isn't stuck at CLIENT until its next NodeInfo.
- NodeDB classifies the category (favorite/ignored/verified -> Flag;
  tracker/sensor/tak_tracker -> Role) at each eviction site.
- WarmNodeStore::lookupMeta() exposes role/category to consumers.
- Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild;
  warm data is a non-critical evictee cache, so discard-on-upgrade is safe.

Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering);
NodeDB compiles (test_nodedb_blocked 4/4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: migrate v1 rings/files by discarding last_heard, not the data

Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all
warm entries — including the PKI public keys that let evicted nodes keep
decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's
identity + public key, discarding only last_heard (its low bits would otherwise
be misread as the new role/protected metadata). Records re-rank and re-learn
their role on next contact.

- Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via
  an out-param; replay zeroes last_heard for v1 records. If the active head page
  is v1, force a rotation so new v2 records never land in a v1-headered page
  (which would discard their freshly-set role on the next load). Legacy pages
  convert to v2 as the ring rotates.
- File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify
  CRC against the stored bytes, then discard last_heard and mark dirty so the
  next save rewrites as v2.

Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard:
key survives, role/protected reset).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: guard role bit-width + test eviction carries role/protected

- static_assert that the device role enum still fits the 4-bit warm metadata
  field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15
  rather than silently truncating role on eviction. (Max role today = 12.)
- Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in
  the warm tier with its key, role=TRACKER and protected category=Role; a demoted
  CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path +
  warmProtectedCategory classification (the warm-store unit tests only cover
  absorb() directly).

Tests: test_nodedb_blocked 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix copilot comments

* fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test

The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard
that PR1.5 introduced, leaving the #else/#endif tail orphaned and
causing compile errors on non-TMM builds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-06-19 19:52:58 -05:00

198 lines
9.9 KiB
C++

#include "ReliableRouter.h"
#include "Default.h"
#include "MeshTypes.h"
#include "NodeDB.h"
#include "configuration.h"
#include "memGet.h"
#include "mesh-pb-constants.h"
#include "modules/NodeInfoModule.h"
#include "modules/RoutingModule.h"
// ReliableRouter::ReliableRouter() {}
/**
* If the message is want_ack, then add it to a list of packets to retransmit.
* If we run out of retransmissions, send a nak packet towards the original client to indicate failure.
*/
ErrorCode ReliableRouter::send(meshtastic_MeshPacket *p)
{
if (p->want_ack) {
DEBUG_HEAP_BEFORE;
auto copy = packetPool.allocCopy(*p);
DEBUG_HEAP_AFTER("ReliableRouter::send", copy);
startRetransmission(copy, NUM_RELIABLE_RETX);
}
/* If we have pending retransmissions, add the airtime of this packet to it, because during that time we cannot receive an
(implicit) ACK. Otherwise, we might retransmit too early.
*/
for (auto i = pending.begin(); i != pending.end(); i++) {
if (i->first.id != p->id) {
i->second.nextTxMsec += iface->getPacketTime(p);
}
}
return isBroadcast(p->to) ? FloodingRouter::send(p) : NextHopRouter::send(p);
}
bool ReliableRouter::shouldFilterReceived(const meshtastic_MeshPacket *p)
{
// Note: do not use getFrom() here, because we want to ignore messages sent from phone
if (p->from == getNodeNum()) {
printPacket("Rx someone rebroadcasting for us", p);
// We are seeing someone rebroadcast one of our broadcast attempts.
// If this is the first time we saw this, cancel any retransmissions we have queued up and generate an internal ack for
// the original sending process.
// This "optimization", does save lots of airtime. For DMs, you also get a real ACK back
// from the intended recipient.
auto key = GlobalPacketId(getFrom(p), p->id);
auto old = findPendingPacket(key);
if (old) {
LOG_DEBUG("Generate implicit ack");
// NOTE: we do NOT check p->wantAck here because p is the INCOMING rebroadcast and that packet is not expected to be
// marked as wantAck
sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, old->packet->channel);
// Only stop retransmissions if the rebroadcast came via LoRa
if (p->transport_mechanism == meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA) {
stopRetransmission(key);
}
} else {
LOG_DEBUG("Didn't find pending packet");
}
}
/* At this point we have already deleted the pending retransmission if this packet was an (implicit) ACK to it.
Now for all other pending retransmissions, we have to add the airtime of this received packet to the retransmission timer,
because while receiving this packet, we could not have received an (implicit) ACK for it.
If we don't add this, we will likely retransmit too early.
*/
for (auto i = pending.begin(); i != pending.end(); i++) {
i->second.nextTxMsec += iface->getPacketTime(p, true);
}
return isBroadcast(p->to) ? FloodingRouter::shouldFilterReceived(p) : NextHopRouter::shouldFilterReceived(p);
}
/**
* If we receive a want_ack packet (do not check for wasSeenRecently), send back an ack (this might generate multiple ack sends in
* case the our first ack gets lost)
*
* If we receive an ack packet (do check wasSeenRecently), clear out any retransmissions and
* forward the ack to the application layer.
*
* If we receive a nak packet (do check wasSeenRecently), clear out any retransmissions
* and forward the nak to the application layer.
*
* Otherwise, let superclass handle it.
*/
void ReliableRouter::sniffReceived(const meshtastic_MeshPacket *p, const meshtastic_Routing *c)
{
if (isToUs(p)) { // ignore ack/nak/want_ack packets that are not address to us (we only handle 0 hop reliability)
if (!MeshModule::currentReply) {
if (p->want_ack) {
if (p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) {
/* A response may be set to want_ack for retransmissions, but we don't need to ACK a response if it received
an implicit ACK already. If we received it directly or via NextHopRouter, only ACK with a hop limit of 0 to
make sure the other side stops retransmitting. */
if (shouldSuccessAckWithWantAck(p)) {
// If this packet should always be ACKed reliably with want_ack back to the original sender, make sure we
// do that unconditionally.
sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, p->channel,
routingModule->getHopLimitForResponse(*p), true);
} else if (!p->decoded.request_id && !p->decoded.reply_id) {
// If it's not an ACK or a reply, send an ACK.
sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, p->channel,
routingModule->getHopLimitForResponse(*p));
} else if ((getHopsAway(*p) == 0) || p->next_hop != NO_NEXT_HOP_PREFERENCE) {
// If we received the packet directly from the original sender, send a 0-hop ACK since the original sender
// won't overhear any implicit ACKs. If we received the packet via NextHopRouter, also send a 0-hop ACK to
// stop the immediate relayer's retransmissions.
sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, p->channel, 0);
}
} else if (p->which_payload_variant == meshtastic_MeshPacket_encrypted_tag && p->channel == 0 &&
(nodeDB->getMeshNode(p->from) == nullptr || nodeDB->getMeshNode(p->from)->public_key.size == 0)) {
LOG_INFO("PKI packet from unknown node, send PKI_UNKNOWN_PUBKEY");
sendAckNak(meshtastic_Routing_Error_PKI_UNKNOWN_PUBKEY, getFrom(p), p->id, channels.getPrimaryIndex(),
routingModule->getHopLimitForResponse(*p));
} else {
// Send a 'NO_CHANNEL' error on the primary channel if want_ack packet destined for us cannot be decoded
sendAckNak(meshtastic_Routing_Error_NO_CHANNEL, getFrom(p), p->id, channels.getPrimaryIndex(),
routingModule->getHopLimitForResponse(*p));
}
} else if (p->next_hop == nodeDB->getLastByteOfNodeNum(getNodeNum()) && p->hop_limit > 0) {
// No wantAck, but we need to ACK with hop limit of 0 if we were the next hop to stop their retransmissions
sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, p->channel, 0);
}
} else {
LOG_DEBUG("Another module replied to this message, no need for 2nd ack");
}
if (p->which_payload_variant == meshtastic_MeshPacket_decoded_tag && c &&
c->error_reason == meshtastic_Routing_Error_PKI_UNKNOWN_PUBKEY) {
if (owner.public_key.size == 32) {
LOG_INFO("PKI decrypt failure, send a NodeInfo");
nodeInfoModule->sendOurNodeInfo(p->from, false, p->channel, true);
}
}
// We consider an ack to be either a !routing packet with a request ID or a routing packet with !error
PacketId ackId = ((c && c->error_reason == meshtastic_Routing_Error_NONE) || !c) ? p->decoded.request_id : 0;
// A nak is a routing packt that has an error code
PacketId nakId = (c && c->error_reason != meshtastic_Routing_Error_NONE) ? p->decoded.request_id : 0;
// We intentionally don't check wasSeenRecently, because it is harmless to delete non existent retransmission records
if ((ackId || nakId) &&
// Implicit ACKs from MQTT should not stop retransmissions
!(isFromUs(p) && p->transport_mechanism == meshtastic_MeshPacket_TransportMechanism_TRANSPORT_MQTT)) {
LOG_DEBUG("Received a %s for 0x%x, stopping retransmissions", ackId ? "ACK" : "NAK", ackId);
if (ackId) {
stopRetransmission(p->to, ackId);
// M3: an end-to-end ACK proves the directed route to the ACK's sender currently works,
// so clear its failure count and refresh freshness (keeps a good route pinned).
if (!isBroadcast(getFrom(p)))
noteRouteSuccess(getFrom(p), millis());
} else {
stopRetransmission(p->to, nakId);
}
}
}
// handle the packet as normal
isBroadcast(p->to) ? FloodingRouter::sniffReceived(p, c) : NextHopRouter::sniffReceived(p, c);
}
/**
* If we ACK this packet, should we set want_ack=true on the ACK for reliable delivery of the ACK packet?
*/
bool ReliableRouter::shouldSuccessAckWithWantAck(const meshtastic_MeshPacket *p)
{
// Don't ACK-with-want-ACK outgoing packets
if (isFromUs(p))
return false;
// Only ACK-with-want-ACK if the original packet asked for want_ack
if (!p->want_ack)
return false;
// Only ACK-with-want-ACK packets to us (not broadcast)
if (!isToUs(p))
return false;
// Special case for text message DMs:
bool isTextMessage =
(p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) &&
IS_ONE_OF(p->decoded.portnum, meshtastic_PortNum_TEXT_MESSAGE_APP, meshtastic_PortNum_TEXT_MESSAGE_COMPRESSED_APP);
if (isTextMessage) {
// If it's a non-broadcast text message, and the original asked for want_ack,
// let's send an ACK that is itself want_ack to improve reliability of confirming delivery back to the sender.
// This should include all DMs regardless of whether or not reply_id is set.
return true;
}
return false;
}