Commit Graph
36 Commits
Author SHA1 Message Date
ec5d230305 Feat/mesh beacon (#10618)
* Tips robot virtual node / relayer to different LoRa modes & channels

Note that this commit has details hardcoded for the Wellington (NZ)
mesh, and also requires the following patch to the protobufs:

-----
diff --git a/meshtastic/mesh.proto b/meshtastic/mesh.proto
index 03162d8..ec54c99 100644
--- a/meshtastic/mesh.proto
+++ b/meshtastic/mesh.proto
@@ -1393,6 +1393,21 @@ message MeshPacket {
    * Set by the firmware internally, clients are not supposed to set this.
    */
   uint32 tx_after = 20;
+
+  /*
+   * The modem preset to use fo rthis packet
+   */
+  uint32 modem_preset = 21;
+
+  /*
+   * The frequency slot to use for this packet
+   */
+  uint32 frequency_slot = 22;
+
+  /*
+   * Whether the packet has a nonstandard radio config
+   */
+  bool nonstandard_radio_config = 23;
 }

 /*
-----

* fix: repair mesh tips CI build

* feat: add MeshBeacon module (Phase 1 — proto + generated code + initial stub)

* feat(beacon): implement broadcaster + listener (phases 2-5)

* feat(beacon): wire RadioLibInterface hooks + admin validation (phases 6-7)

* fix(beacon): fix LocalModuleConfig flat access (no payload_variant), add localonly proto field

* feat(beacon): fix broadcaster inheritance, add preset/region validation + proto cache

- MeshBeaconBroadcastModule now inherits ProtobufModule<meshtastic_MeshBeacon>
  (alongside private MeshBeaconModule + OSThread), giving it allocDataPacket()
  and setStartDelay() without extra includes.

- Payload cache: rebuildCache() encodes the MeshBeacon protobuf once and stores
  it in payloadCache[]/payloadCacheSize; sendBeacon() only calls rebuildCache()
  when payloadCacheDirty==true. AdminModule calls invalidateCache() after saving
  new config so the next broadcast picks up changes.

- Region/preset validation in handleSetModuleConfig (mesh_beacon_tag):
  broadcast_on_preset is validated against the device's current region via
  RadioInterface::validateConfigLora(); broadcast_offer_region is validated via
  RadioInterface::validateConfigRegion(). Invalid values are zeroed with a
  LOG_WARN before saving.

* feat(beacon): add unit tests for MeshBeaconModule and AdminModule configuration validation

* remove old meshtips

* more  validation in NodeDB and AdminModule, and userprefs for baked in goodness

* copilot is my gravity

* mmmmm... beacon

* oops

* Enhance unit tests for MeshBeaconModule with detailed validation checks and output formatting

* new lines. Why not?

* finally

* legacy mode activate!

* Update protobufs (#17)

Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>

* better logic, fixed a test

* updated for packet signing
fixed a test
added guards for licensed/ham mode

* channel numbers

* beacon: encrypt on the beacon channel PSK; fix split note

When broadcast_on_channel overrides the primary channel's name/PSK, the
beacon was encrypted with the PRIMARY PSK: perhapsEncode keys encryption
off the primary slot, but the radio-thread channel switch happens only
after encryption. sendBeaconPacket() now installs the beacon channel into
the primary slot for the synchronous duration of send() (cooperative
threading => no interleaving) so encryption/hash use the beacon channel,
then restores it. A shared beaconChannelSettings() helper builds the
channel for both the encrypt-time swap and the RF-time swap so the
key+hash cannot drift.

Also: correct the legacy-split comments (both packets go out on the same
beacon radio settings, not the normal config) and merge the two
consecutive `if (hasText)` blocks in the listener (cppcheck
duplicateCondition).

Tests: add channelPskOverride_swapsBeaconChannelAndRestores and
noChannelOverride_doesNotSwapPrimary; MockRouter snapshots the primary
channel at send() time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test/beacon: drain toPhoneQueue in tearDown to fix LSan leak abort

The listener delivers received text via MeshService::sendToPhone(), which
enqueues the packet into toPhoneQueue and takes ownership. Nothing dequeues
it in tests, so the three listener tests carrying message text stranded a
MeshPacket each — 1272 bytes / 3 allocations that LeakSanitizer flagged at
process exit, aborting the coverage run (surfaced by pio as [ERRORED] /
SIGHUP even though all 40 assertions passed).

Drain the phone queue in tearDown (getForPhone()/releaseToPool) so the
packets return to packetPool. Suite is now GREEN with no sanitizer abort.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* legacy hop override for zero-hoppers

* ever more beacons

* beacon: comment out broadcast_send_as_node pending further review

Functionality preserved in comments with full signing/has_bitfield notes
for when it is re-enabled. Proto tag 3 retained on the wire.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test/beacon: fix fromIsCustomNodeWhenSet now that send-as-node is disabled

broadcast_send_as_node is commented out; from is always the local node.
Update the test assertion and doc comment to match current behaviour.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update protobufs (#21)

Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>

* flags for beacons

* beacon: do more with less — slot-index targets + validation

Multi-target beacons embedded a full ChannelSettings in every BroadcastTarget,
blowing ModuleConfig past the 512-byte BLE FromRadio budget so the firmware would
not compile. Targets now reference an existing channel-table slot by channel_index
and the broadcaster resolves it via channels.getByIndex() at TX time. Net effect:
the same multi-target capability for a fraction of the bytes —
FromRadio 609 -> 510 B, MeshBeaconConfig 596 -> 324 B, AdminMessage 615 -> 511 B.

- proto: BroadcastTarget.channel (embedded) -> channel_index (uint32 ref); regen all
  generated headers (size constants propagate to admin/localonly/deviceonly/mesh).
- broadcaster: resolve channel_index from the channel table; an out-of-range or blank
  slot falls back to the default channel for the target preset rather than borrowing
  the primary's name/PSK.
- AdminModule: validate broadcast_targets entries on write (region/preset sanitised
  like the single-target fields; channel_index range-checked).
- userPrefs: TARGET_<n>_CHANNEL_{NAME,NUM,PSK} collapse to a single CHANNEL_INDEX.
- docs: two-step (set_channel -> set_module_config) multi-target setup, inline-vs-
  reference distinction, and single-/multi-target are equal (not "legacy") options.
- tests: target validation + channel-index resolution incl. blank-slot fallback
  (47/47 green on `./bin/run-tests.sh -e native -f test_mesh_beacon`).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NRAF5csgsMn6p1zEcFL8Qz

* throttling after reboot

* address copilot review

* simplify

* fix(beacon): use 0x%08x for node/packet IDs in logs; register test suite

The %#08lx log specifiers passed uint32_t (NodeNum/PacketId) to a %lx
length modifier — undefined behaviour on 64-bit (native test) targets and
non-standard width. Switch to the project-standard 0x%08x. Also bump
test/native-suite-count to 25 for the added test_mesh_beacon suite.

clod helped too

* copilot & clarity
clod helped too

* refactor(beacon): use auto for the sanitized config copy

clod helped too

* fix(beacon): guard empty-payload sends; gate has_mesh_beacon on build flag; document ISR_TX pre-switch

clod helped too

---------

Co-authored-by: Steve Gilberd <steve@erayd.net>
Co-authored-by: Darafei Praliaskouski <me@komzpa.net>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 08:20:51 -05:00
TomandClaude Sonnet 4.6 c51c01607d Traffic Management module: dedup, rate limiting, role-aware policing (#10706)
Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and
next-hop foundations it builds on:

- Unified per-node cache (flat array, 8-bit relative ticks) shared by all
  features; role-aware throttles for tracker / lost-and-found.
- Position deduplication: drop unchanged position rebroadcasts within a
  configurable interval; precision driven off the channel ceiling (clamped to
  the public-key max on well-known channels). Enabled by default at 11h.
- Per-node rate limiting and unknown-packet filtering (config-driven; a
  non-zero companion field enables each feature -- no bool toggles).
- NodeInfo direct response from cache with role-based hop clamps.
- Persistent next-hop overflow store: confirmed hops have no TTL, are seeded
  from NodeInfoLite at boot, and survive hot-store eviction.
- Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM
  cache). Role is cached write-time (seeded on first track, refreshed from
  NodeInfo), pins its cache entry like a next-hop hint, and is evicted last.
- Warm store caches device role + protected category across reboot/eviction.
- PositionModule stationary floor for tracker / lost-and-found.
- PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded.

Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually
used; submodule repointed to protobufs develop.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 20:35:14 -05:00
22072c5f4b Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store

Reworks the TrafficManagementModule cache layer (policing behaviour unchanged
from upstream) and adds a routing-hint overflow store:

- Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed
  PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as
  WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible,
  and it removes a large amount of hashing/displacement complexity. The cache
  entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always
  means "empty" across every sub-store. Adds rebaseEpoch() so cached state
  survives the ~19 h relative-timestamp horizon instead of being flushed.

- Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte
  relay for a destination, written only from NextHopRouter's ACK-confirmed
  decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back
  to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail
  nodes keep routing after the node ages out of NodeInfoLite.

- Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted
  NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive
  across the maintenance sweep (no TTL) and never clobbered by a stale preload.

All packet-policing logic (rate limit, position dedup, unknown-packet drop,
NodeInfo direct response, hop exhaustion) is the existing upstream behaviour,
untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note).

Tests: upstream policing suite now actually runs (adds the MeshTypes.h include
that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles,
politeness, precision clamp, port-interval and mesh-radius gating — and the
rate-limit >255 saturation fix — are deferred to the advanced-TMM branch.

Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* TrafficManagement: fix cppcheck constVariablePointer warning

`node` in preloadNextHopsFromNodeDB() is never written through — mark
it const to satisfy cppcheck's constVariablePointer check in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add multi-hop NextHop recovery tests and unit tests for routing reliability

- Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop.
- Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering:
  - M1: Ambiguity-aware last-byte resolution.
  - M2: NextHopRouter's strict-neighbor gate and hop limit checks.
  - M3: Route-health freshness and failure decay.
- Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic.

* grafting fixed

* Address Copilot review for PR #10735 (NextHop improvements)

- docs/nexthop-routing-reliability.md: update status from "no code
  changes yet" to reflect that mitigations and tests are implemented

RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in
PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose
default=0) respectively; (0,0) sentinel fixed in PR2.5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* CI: fix cppcheck constVariablePointer and test include path

- NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only
  read for stale-route checks, never mutated through the pointer
- Router.cpp: qualify meshtastic_NodeInfoLite *node as const in
  shouldDecrementHopLimit — only read for favorite/role predicate
- test_position_module/test_main.cpp: change bare PositionModule.h to
  modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules,
  so the bare form fails to resolve in the native PlatformIO test env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* WarmStore: cache device role + protected category in last_heard low bits

Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's
device role (4 bits) and a protected category (2 bits) for the hop-trim path,
at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits
remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU
ordering of long-tail nodes.

- absorb() packs role/protectedCat; place()/ring replay store the raw word so
  metadata round-trips through flash. LRU compares masked time (warmTimeOf).
- take() rehydration masks the metadata bits and restores the cached role so a
  re-admitted node isn't stuck at CLIENT until its next NodeInfo.
- NodeDB classifies the category (favorite/ignored/verified -> Flag;
  tracker/sensor/tak_tracker -> Role) at each eviction site.
- WarmNodeStore::lookupMeta() exposes role/category to consumers.
- Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild;
  warm data is a non-critical evictee cache, so discard-on-upgrade is safe.

Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering);
NodeDB compiles (test_nodedb_blocked 4/4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: migrate v1 rings/files by discarding last_heard, not the data

Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all
warm entries — including the PKI public keys that let evicted nodes keep
decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's
identity + public key, discarding only last_heard (its low bits would otherwise
be misread as the new role/protected metadata). Records re-rank and re-learn
their role on next contact.

- Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via
  an out-param; replay zeroes last_heard for v1 records. If the active head page
  is v1, force a rotation so new v2 records never land in a v1-headered page
  (which would discard their freshly-set role on the next load). Legacy pages
  convert to v2 as the ring rotates.
- File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify
  CRC against the stored bytes, then discard last_heard and mark dirty so the
  next save rewrites as v2.

Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard:
key survives, role/protected reset).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: guard role bit-width + test eviction carries role/protected

- static_assert that the device role enum still fits the 4-bit warm metadata
  field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15
  rather than silently truncating role on eviction. (Max role today = 12.)
- Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in
  the warm tier with its key, role=TRACKER and protected category=Role; a demoted
  CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path +
  warmProtectedCategory classification (the warm-store unit tests only cover
  absorb() directly).

Tests: test_nodedb_blocked 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix copilot comments

* fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test

The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard
that PR1.5 introduced, leaving the #else/#endif tail orphaned and
causing compile errors on non-TMM builds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-06-19 19:52:58 -05:00
TomandCopilot de345939af Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf

* Implement SphereOfInfluenceModule for traffic management and eviction tracking

* Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor

* Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy

* Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule

* Enable variable hop limits and role-based hop floors in Sphere of Influence module

* Respond to copilot review

* Disable variable hop limits and role-based hop floors in Sphere of Influence module

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Implement adaptive sampling for unique node ID tracking in Sphere of Influence module

* Add state persistence for Sphere of Influence module

* Enhance Sphere of Influence module with state management and adaptive sampling adjustments

* Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule

- Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule.
- Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation.
- Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity.
- Introduced state persistence for hop scaling parameters to maintain continuity across reboots.
- Enhanced thread safety and modularity by utilizing concurrency features.

* Guard out STM32. Sowwy.

* Refactor HopScalingModule: enhance sampling logic and improve state management

* Add unit tests for HopScalingModule: implement mock database and various test scenarios

* Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity

* Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability

* Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity

* Remove unnecessary delay in setup function for improved test performance

* Add missing include for MeshTypes in test_main.cpp

* Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity

* Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases

* Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc.

* Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management

* Add sample traffic injection for HopScaling tests to enhance sampledEst visibility

* Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting

* Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior

* Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests

* Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output

* Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests

* Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation

* Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points

* Implement CompactHistogram for parallel hop scaling sampling

- Added CompactHistogram class to track node hop distances with bitwise sampling.
- Integrated CompactHistogram into HopScalingModule for independent packet sampling.
- Updated NodeDB to feed both the hop scaling module and the new histogram sampler.
- Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics.
- Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling.
- Updated existing tests to validate the integration of the new histogram sampling mechanism.

* Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior

* CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution

* Refactor HopScalingModule and CompactHistogram integration

- Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic.
- Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling.
- Enhanced logging in HopScalingModule to provide detailed histogram statistics.
- Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic.
- Improved node ID distribution in tests to better exercise sampling mechanisms.
- Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling.

* Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes

- Updated the bitfield structure to accommodate 13-hour seen tracking.
- Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation.
- Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios.
- Adjusted filtering and sampling logic to reflect the new 13-hour tracking period.
- Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes.

* Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making

- Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling.
- Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management.
- Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection.
- Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops.
- Enhanced unit tests to validate new sampling logic and memory layout changes.

* Expose hashNodeId for testing in CompactHistogram

* Add mesh trend statistics to CompactHistogram for enhanced activity tracking

* Implement histogram state persistence in CompactHistogram with save and load functions

* Refactor CompactHistogram to improve entry management and enhance rollHour logging

* feat: add HopScalingModule for adaptive hop limit recommendations

Introduces HopScalingModule, a sampled hop-distance histogram that
recommends the minimum hop limit needed to reach ~40 nodes, and
automatically reducing the hops as the mesh grows.

Key design:
- 512-byte packed histogram (128 × 4-byte Record entries) embedded
   in a new HopScalingModule.
- Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap
- Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept;
  denominator doubles on overflow and halves when utilisation is low
- Hourly rollHour(): tallies per-hop counts, walks scaled buckets to
  find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a
  politeness extension based on recent/older activity ratio, shifts all
  seen bitmaps, and persists state to /prefs/hopScalingState.bin
- Hop recommendation gated by bootstrap (requires >=1 rollHour before
  overriding HOP_MAX)
- NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet
- Module also estimates total mesh size and logs useful information about
  mesh characteristics.

Changes:
- src/modules/HopScalingModule.h/.cpp: new module
- src/mesh/NodeDB.cpp: wire up samplePacketForHistogram
- src/mesh/Router.cpp: consume getLastRequiredHop()
- test/test_hop_scaling/: 12-test suite covering all mesh topologies and
  anticipated operational requirements

* test: increase run iterations in sparse to dense transition test

* feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging

* feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions

* refactor: remove CompactHistogram module and related files

* address copilot review comments

* Tweak: packet sampling only lora

* ove role-based hop floor logic and related definitions into the module - keep it in one place.

* Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting

* Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact.

* Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency

* refactor: improve test output organization and clarity in hop scaling tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-06-04 05:59:49 -05:00
TomandCopilot 5e69bc6c3f Enable Narrow and Lite regions for EU (#10120)
* Enable Lite and Narrow regions and introduce getEffectiveDutyCycle for Lite profiles

* Add TrafficType enum and extend getConfiguredOrDefaultMsScaled to manage based on regionProfile settings

* Refactor telemetry modules to include TrafficType in getConfiguredOrDefaultMsScaled calls

* Update submodule protobufs to latest commit

* Add support for new region presets and modem presets in menu options

* Add new LoRa region codes and modem presets for EU bands

* boof

* Add modem presets for LITE and NARROW configurations

* Update subproject commit reference in protobufs

* Update protobufs

* Refactor modem preset definitions to use macro for consistency and clarity

* Refactor modem preset cases to use PRESET macro for consistency

* fix: update LoRa region code for EU 868 narrowband configuration

Co-authored-by: Copilot <copilot@github.com>

* Fix test suite failure

Co-authored-by: Copilot <copilot@github.com>

* Add override slot override - for when one override isn't enough.

Co-authored-by: Copilot <copilot@github.com>

* address copilot comments

---------

Co-authored-by: Copilot <copilot@github.com>
2026-05-21 10:20:09 -05:00
Tom b2c8cbb78d Enhance traffic management by adjusting position update interval and refining hop exhaustion logic based on channel congestion (#9921) 2026-04-10 10:53:04 -05:00
Ben Meadors fc030d2d19 Merge remote-tracking branch 'origin/master' into develop 2026-03-20 07:43:15 -05:00
Ben MeadorsandCopilot Autofix powered by AI 4fbd5c9f80 Ensure infrastructure role-based minimums are coerced since they don't have scaling (#9937)
* Ensure infrastructure role-based minimums are coerced since they don't have scaling

* Add test

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 10:28:23 -05:00
Clive BlackledgeandCopilot 016e68ec53 Traffic Management Module for packet forwarding logic (#9358)
* Add ESP32 Power Management lessons learned document

Documents our experimentation with ESP-IDF DFS and why it doesn't
work well for Meshtastic (RTOS locks, BLE locks, USB issues).

Proposes simpler alternative: manual setCpuFrequencyMhz() control
with explicit triggers for when to go fast vs slow.

* Addition of traffic management module

* Fixing compile issues, but may still need to update protobufs.

* Fixing log2Floor in cuckoo hash function

* Adding support for traffic management in PhoneAPI.

* Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE.

* Adding station-g2 and portduino varients to be able to use this module.

* Spoofing from address for nodeinfo cache

* Changing name and behavior for zero_hop_telemetry / zero_hop_position

* Name change for exhausting telemetry packets and setting hop_limit to 1 so it will be 0 when sent.

* Updated hop logic, including exhaustRequested flag to bypass some checks later in the code.

* Reducing memory on nrf52 nodes further to 12 bytes per entry, 12KB total using 8 bit hashes with 0.4% collision. Probably ok. Adding portduino to the platforms that don't need to worry about memory as much.

* Fixing hopsAway for nodeinfo responses.

* traffic_management.nodeinfo_direct_response_min_hops -> traffic_management.nodeinfo_direct_response_max_hops

* Removing dry run mode

* Updates to UnifiedCacheEntry to use a common cache, created defaults for some values, reduced a couple bytes per entry by using a resolution-scale time selection based on configuration value.

* Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants.

* Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails.

* Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit.

* Creating consistent log messages

* Remove docs/ESP32_Power_Management.md from traffic_module

* Add unit tests for Traffic Management Module functionality

* Fixing compile issues, but may still need to update protobufs.

* Adding support for traffic management in PhoneAPI.

* Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE.

* Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants.

* Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails.

* Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit.

* Add mock classes and unit tests for Traffic Management Module functionality.

* Refactor setup and loop functions in test_main.cpp to include extern "C" linkage

* Update comment to include reduced  memory requirements

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Re-arranging comments for programmers with the attention span of less than 5 lines of code.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update comments in TrafficManagementModule to reflect changes in timestamp epoch handling and memory optimization details.

* bug: Use node-wide config_ok_to_mqtt setting for cached NodeInfo replies.

* Better way to handle clearing the ok_to_mqtt bit

* Add bucketing to cuckoo hashing, allowing for 95% occupied rate before major eviction problems.

* Extend nodeinfo cache for psram devices.

* Refactor traffic management to make hop exhaustion packet-scoped. Nice catch.

* Implement better position precision sanitization in TrafficManagementModule.

* Added logic in TrafficManagementModule to invalidate stale traffic state. Also, added some tests to avoid future me from creating a regression here.

* Fixing tests for native

* Enhance TrafficManagementModule to improve NodeInfo response handling and position deduplication logic. Added tests to ensure local packets bypass transit filters and that NodeInfo requests correctly update the requester information in the cache. Updated deduplication checks to prevent dropping valid position packets under certain conditions.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-11 06:12:12 -05:00
Clive BlackledgeandBen Meadors 22031c5886 fix: treat ROUTER_LATE like ROUTER for power management and defaults (#9815)
ROUTER_LATE is an infrastructure relay that should use the same power
assumptions, interval defaults, and LoRa wake behavior as ROUTER.
Without this, ROUTER_LATE uses client-class defaults and will not
wake on LoRa activity from deep sleep, making it a broken relay.

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-03-05 06:22:12 -06:00
Ben Meadors 65bf1630b3 Add some lora parameter clamping logic to coalesce to defaults and enforce some bounds (#9808) 2026-03-04 07:55:34 -06:00
b7bf251798 Scaling tweaks (#9653)
* refactor: update throttling factor calculation and add unit tests for scaling behavior

* refactor: adjust throttling factor calculation for improved accuracy in different configurations

* Update src/mesh/Default.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/mesh/Default.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: enhance throttling factor calculation and introduce pow_of_2 utility function

* refactor: improve expected ms calculation in unit tests for Default::getConfiguredOrDefaultMsScaled

* refactor: improve scaling logic for routers and sensors in computeExpectedMs function

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-04 05:59:42 -06:00
Ben Meadors ff8d6aa9c3 Increase default position broadcast intervals and enforce minimums for default channels 2026-01-09 08:35:37 -06:00
Jorropo beb268ff25 Revert "add a .clang-format file (#9154)" (#9172)
I thought git would be smart enough to understand all the whitespace changes but even with all the flags I know to make it ignore theses it still blows up if there are identical changes on both sides.

I have a solution but it require creating a new commit at the merge base for each conflicting PR and merging it into develop.

I don't think blowing up all PRs is worth for now, maybe if we can coordinate this for V3 let's say.

This reverts commit 0d11331d18.
2026-01-04 05:15:53 -06:00
Jorropo 0d11331d18 add a .clang-format file (#9154) 2026-01-03 14:19:24 -06:00
Benjamin Faershtein 61e41a8beb Don't scale up the frequency of telemetry sending (#8664) 2025-12-02 13:59:05 -06:00
Xavier horwoodandTom Fifield 718fd118b0 Add IPv6 Support for esp32 (#6866)
* Update Default.h

* Update NodeDB.cpp

* Update WiFiAPClient.cpp

* Update userPrefs.jsonc

* set ipv6 to off by default

* Trunk fix

---------

Co-authored-by: Tom Fifield <tom@tomfifield.net>
2025-11-02 09:25:59 +11:00
Clive Blackledge 28f53d132a refactor: change node count variables from uint8_t to uint16_t (#8478)
This is a non-breaking change that increases the internal representation
of node counts from uint8_t (max 255) to uint16_t (max 65535) to support
larger mesh networks, particularly on ESP32-S3 devices with PSRAM.

Changes:
- NodeStatus: numOnline, numTotal, lastNumTotal (uint8_t -> uint16_t)
- ProtobufModule: numOnlineNodes (uint8_t -> uint16_t)
- MapApplet: loop counters changed to size_t for consistency with getNumMeshNodes()
- NodeStatus: Fixed log format to use %u for unsigned integers

Note: Default class methods keep uint32_t for numOnlineNodes parameter
to match the public API and allow flexibility, even though internal node
counts use uint16_t (max 65535 nodes).

This change does NOT affect protobuf definitions, maintaining wire
compatibility with existing clients and devices.
2025-10-28 15:32:08 -05:00
799cf0e8b3 Master --> develop (#8436)
* Issue: #7944 External notification module: Adjusted default nag timeout to 15s (from 60s) (#7946)

* External notification module: Adjusted default nag timeout to 5s (from 60s)

* Change nag to 15s

---------

Co-authored-by: Tom Fifield <tom@tomfifield.net>

* Add support for RAK WISMESH TAP V2 by enabling SDCARD_CS pin during deep sleep (#8429)

* Upgrade trunk (#8369)

Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>

* Don't assign negative SNR to unsigned int type

SNR-based contention windows are broken on systems with 64-bit long integers.
Fixes #8430

* Allow vibra or buzzer only notifications to obey cutoff (#8342)

* Allow vibra or buzzer only notifications to obey cutoff

* Update src/modules/ExternalNotificationModule.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* InkHUD crash fix when nodes get deleted from NodeDB (#8428)

* InkHUD crash fix

* trunk fix

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Onyx Clawe <58921814+OnyxClawe@users.noreply.github.com>
Co-authored-by: Daniel.Cao <144674500+DanielCao0@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
Co-authored-by: korbinianbauer <64415847+korbinianbauer@users.noreply.github.com>
Co-authored-by: Jason P <applewiz@mac.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com>
2025-10-24 10:37:38 +11:00
Austin 86be2ac12f userPrefs: Set default ringtone nag time (#7314) 2025-07-12 16:26:25 -05:00
Ben Meadors 18fbc2149d Fix iOS bluetooth crash: Ensure UINT32_MAX is not used (#7147) 2025-06-26 19:23:08 -05:00
Ben MeadorsandCopilot ad23c065f6 Rate limiting fix and added 2 second rate limiting to text messages (#7139)
* Rate limiting fix and added 1.5 second rate limiting to text messages

* Remove copy-pasta

* Update src/mesh/Default.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Two is more reasonable

* Two too

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-26 07:56:34 -05:00
Ben Meadors 91bcf072a0 Tweak interval trottling (#7113) 2025-06-24 05:27:40 -05:00
Ben Meadors 7cffd9ba70 Added new map report opt-in for compliance and limit map report (and default) to one hour (#6813)
* Added new map report opt-in for compliance and limit map report (and default) to one hour

* Trunk
2025-05-14 15:31:28 -05:00
Austin f16402dec1 MQTT userprefs (#6802) 2025-05-14 13:39:46 -05:00
Ben MeadorsandCopilot e0b1fdb5e8 Rate limit waypoints and alerts and increase to allow every 10 seconds instead of 5 (#6699)
* Rate limit waypoints and alerts and increase to allow every 10 seconds instead of 5.

* Update src/mesh/PhoneAPI.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Doot

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-30 06:08:10 -05:00
GUVWAF 2c2213ef9b Add setting to transmit NeighborInfo over LoRa (#5286)
* Add setting to transmit NeighborInfo over LoRa
Only if not using the default channel

* Bump minimum broadcast interval for NeighborInfo to 4 hours
2024-11-09 09:38:48 +08:00
Ben Meadors 73e2e25eb1 Smarter traffic scaling (#5264) 2024-11-06 15:00:53 -06:00
Ben Meadors ec9e562a77 Coerce minimum telemetry interval of 30 minutes on defaults and make new default interval one hour (#5086)
* Coerce minimum telemetry interval of 30 minutes on defaults and make new default interval one hour

* Smaller log messages
2024-10-17 13:33:52 -05:00
Ben Meadors 777bcf691a Initial PhoneAPI rate-limiting of messages on certain ports (#4756) 2024-09-18 10:13:07 -05:00
Ben Meadors 5ff1078c8c Move NeighborInfo interval default to 6 hours (double NodeInfo) (#4483)
* Move NeighborInfo in line with NodeInfo

* Set default to 6 hours and cap minimum at 2 hours
2024-08-17 11:51:53 -05:00
Ben MeadorsandJonathan Bennett 59cc57fc29 Event mode: Enforce reliable hop limit and disallow default public MQTT (#4343)
* Event mode: Enforce reliable hop limit

* Event mode: Short circuit wantsLink on MQTT for default broker address

* Just enforce at channels level since everything uses this

* For events never forward packets with excessive hop_limit

* In EVENT_MODE, don't respond with hop_limit set more then the configured max.

* Correct hop_start when correcting hop_limit in event mode.

* Make EVENT_MODE work from userPrefs.h

* Event mode: Disallow Router or Repeater roles

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2024-07-29 20:16:47 -05:00
Ben Meadors c5d747cd3e Scale default intervals based for *online* mesh size past 40 nodes (#4277)
* Add congestion scaling coefficient

* Added active mesh sized based interval scaling

* Moved back to bottom

* Format

* Add observers and use correct number of online nodes
2024-07-13 05:59:19 -05:00
Ben Meadors 699d37b04c Move up telemetry defaults to every 30 minutes (#4274) 2024-07-12 09:24:42 -05:00
Ben Meadors 425a715995 Added one minute throttling to NodeDB save to disk (#3648)
* Added one minute throttling to NodeDB

* Derp
2024-04-18 14:20:39 -05:00
Ben Meadors 0d1d79b6d1 Extract default intervals and coalesce methods into their own file / static class methods (#3425)
* Extract default intervals and coalesce methods into their own file / static class methods

* Missed pax

* Still managed to miss one
2024-03-17 08:18:30 -05:00