Merge branch 'develop' into thinknode-g3

Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>

.claude/commands/diagnose.md

@@ -49,11 +49,17 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
    - Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
    - Do the primary channel NAMES match? Mismatch = different PSK = no decode.
 
-7. **Suggest next actions only for specific, recognisable failure modes**:
+7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
+   - `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
+   - `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
+   - If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
+
+8. **Suggest next actions only for specific, recognisable failure modes**:
    - Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
    - Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
    - Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
    - CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
+   - Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
 
 ## What NOT to do
 

.claude/commands/leakhunt.md

@@ -0,0 +1,103 @@
+---
+description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
+argument-hint: [window=1h] [field=free_heap] [variant=local]
+---
+
+<!-- markdownlint-disable MD029 -->
+
+# `/leakhunt` — read the recorder, classify a memory leak
+
+Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
+
+## Two signal paths — pick the right one
+
+| Path                  | Build flag       | Cadence        | Per-thread attribution | Cost                      |
+| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
+| LocalStats packet     | (default)        | ~60 s          | No                     | Free — always on          |
+| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked)  | Bigger flash + log volume |
+
+Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
+
+## What to do
+
+1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
+
+2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
+   - `running == True`
+   - `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
+   - `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
+
+3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
+
+4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
+   - `samples` — how many raw points contributed
+   - `min`, `max` — total swing
+   - `slope_per_min` — units per minute (linear regression over the whole window)
+
+5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
+
+6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
+
+6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
+
+7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
+   - **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
+   - **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
+   - **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
+   - **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
+   - **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
+   - **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
+   - **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
+
+8. **Report**:
+
+   ```text
+   /leakhunt window=6h field=free_heap variant=local
+   ────────────────────────────────────────────────────
+   recorder        : running, telem last_ts 8s ago
+   build           : DEBUG_HEAP=ON (per-line prefix detected)
+   samples         : 14,200 over 6h (cadence ~1.5s, log-line synth)
+   free_heap       : min 92,344 / max 124,008 / range 31,664
+   slope           : -82 bytes/min (negative — heap declining)
+   reboots         : none in window
+   OOM events      : none
+   error lines     : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
+   thread leaks    : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
+                                  Router       -1,408 B over 4 events
+                                  others       -240 B
+   verdict         : SLOW LEAK — primary suspect MeshPacket thread
+   est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
+   evidence        : (3 log line citations with uptimes)
+   ```
+
+   Then: **what to do next.**
+   - SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
+     ```text
+     mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
+     mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
+     ```
+     After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
+   - SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
+   - FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
+   - OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
+   - STEADY → say so; stop. Don't invent problems.
+
+## What NOT to do
+
+- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
+- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
+- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
+- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
+
+## Companion: `mark_event` for stress runs
+
+If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
+
+```text
+mark_event("burst-start")
+… run the workload …
+mark_event("burst-end")
+/leakhunt window=15m
+```
+
+The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.

.claude/commands/repro.md

@@ -3,6 +3,8 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
 argument-hint: <test-node-id> [count=5]
 ---
 
+<!-- markdownlint-disable MD029 -->
+
 # `/repro` — flakiness triage for one test
 
 Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -40,6 +42,8 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
 
    Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
 
+5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
+
 6. **Classify the flake** into one of:
    - **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
    - **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.

.github/copilot-instructions.md

@@ -135,6 +135,95 @@ On top of authorization, any remote admin message that **mutates** state (not a
 - **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
 - **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
 
+## NodeDB Layout (v25)
+
+`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
+
+### Slim `NodeInfoLite`
+
+`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
+
+Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
+
+```cpp
+nodeInfoLiteHasUser(n)                  // bit 5 — user fields populated
+nodeInfoLiteIsFavorite(n)               // bit 3
+nodeInfoLiteIsIgnored(n)                // bit 4
+nodeInfoLiteIsMuted(n)                  // bit 1
+nodeInfoLiteIsLicensed(n)               // bit 6 — Ham mode peer
+nodeInfoLiteIsKeyManuallyVerified(n)    // bit 0
+nodeInfoLiteHasIsUnmessagable(n)        // bit 8 — "is_unmessagable was sent"
+nodeInfoLiteIsUnmessagable(n)           // bit 7
+// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
+
+nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);  // setter
+```
+
+### Satellite stores
+
+Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its own build flag:
+
+| Map               | Value type                      | Build flag                         |
+| ----------------- | ------------------------------- | ---------------------------------- |
+| `nodePositions`   | `meshtastic_PositionLite`       | `MESHTASTIC_EXCLUDE_POSITIONDB`    |
+| `nodeTelemetry`   | `meshtastic_DeviceMetrics`      | `MESHTASTIC_EXCLUDE_TELEMETRYDB`   |
+| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
+| `nodeStatus`      | `meshtastic_StatusMessage`      | `MESHTASTIC_EXCLUDE_STATUSDB`      |
+
+Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
+
+All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
+
+### Accessor convention
+
+**Never hand out pointers into the maps.** Use the copy-out accessors on `NodeDB`:
+
+```cpp
+bool copyNodePosition(NodeNum, meshtastic_PositionLite &out)       const;
+bool copyNodeTelemetry(NodeNum, meshtastic_DeviceMetrics &out)     const;
+bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
+bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out)        const;
+```
+
+Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
+
+Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
+
+### Eviction
+
+Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
+
+### Gradient sync (opt-in via special nonces)
+
+`client_capabilities` is **not** a thing in this branch. Phone clients opt into the new sync flow by sending one of two values in the `ToRadio.want_config_id`:
+
+- `SPECIAL_NONCE_GRADIENT_SYNC` (69422) — full config + thin NodeInfo + replay phases.
+- `SPECIAL_NONCE_GRADIENT_ONLY_NODES` (69423) — skip config segments, NodeInfo + replay only.
+
+`PhoneAPI::clientWantsGradientSync()` is the single switch. When true, `STATE_SEND_OTHER_NODEINFOS` is followed by:
+
+```text
+STATE_REPLAY_POSITIONS → STATE_REPLAY_TELEMETRY → STATE_REPLAY_ENVIRONMENT → STATE_REPLAY_STATUS
+```
+
+Each replay phase walks the corresponding satellite map and emits synthetic `MeshPacket`s on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` for both device + environment variants, `STATUS_MESSAGE_APP`). Legacy clients (no special nonce) get the bundled-NodeInfo path with position/device_metrics joined back in by `ConvertToNodeInfo(lite, pos*, dm*)` — wire bytes are byte-identical to pre-v25 for them.
+
+`ConvertToNodeInfoThin(lite)` is the gradient-sync emitter (no position/telemetry).
+
+### v24 → v25 migration
+
+The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not in `NodeDB.cpp`. It owns the `meshtastic_NodeDatabase_Legacy` callback and `NodeDB::migrateLegacyNodeDatabase()`. The legacy proto descriptor is `protobufs/meshtastic/deviceonly_legacy.proto` (only included by the migration TU). The boot path peeks the file's leading version tag, runs the migration if `version < 25`, then re-saves in v25 layout. The legacy descriptor is scheduled for removal once `DEVICESTATE_MIN_VER` is bumped.
+
+### Read-site rules of thumb
+
+- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
+- Never `node->user.long_name` — `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
+- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
+- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
+- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
+
+Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
+
 ## Project Structure
 
 ```
@@ -196,6 +285,8 @@ firmware/
 - Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
 - Use `assert()` for invariants that should never fail
 - C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ### Naming Conventions
 

.trunk/trunk.yaml

@@ -4,11 +4,11 @@ cli:
 plugins:
   sources:
     - id: trunk
-      ref: v1.8.0
+      ref: v1.9.0
       uri: https://github.com/trunk-io/plugins
 lint:
   enabled:
-    - checkov@3.2.526
+    - checkov@3.2.528
     - renovate@43.150.0
     - prettier@3.8.3
     - trufflehog@3.95.2

AGENTS.md

@@ -66,6 +66,8 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
 - **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
 - **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
 - **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ## Typical agent workflows
 

bin/org.meshtastic.meshtasticd.metainfo.xml

@@ -87,6 +87,9 @@
   </screenshots>
 
   <releases>
+    <release version="2.7.24" date="2026-05-08">
+      <url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.24</url>
+    </release>
     <release version="2.7.23" date="2026-04-14">
       <url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.23</url>
     </release>

boards/ThinkNode-M7.json

@@ -0,0 +1,42 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-D BOARD_HAS_PSRAM",
+      "-D ARDUINO_USB_CDC_ON_BOOT=0",
+      "-D ARDUINO_USB_MODE=0",
+      "-D ARDUINO_RUNNING_CORE=1",
+      "-D ARDUINO_EVENT_RUNNING_CORE=0"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "psram_type": "qio_opi",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "ELECROW-ThinkNode-M7"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "ELECROW ThinkNode M7",
+  "upload": {
+    "flash_size": "8MB",
+    "maximum_ram_size": 524288,
+    "maximum_size": 8388608,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "https://www.elecrow.com",
+  "vendor": "ELECROW"
+}

debian/changelog

@@ -1,3 +1,9 @@
+meshtasticd (2.7.24.0) unstable; urgency=medium
+
+  * Version 2.7.24
+
+ -- GitHub Actions <github-actions[bot]@users.noreply.github.com>  Fri, 08 May 2026 10:44:12 +0000
+
 meshtasticd (2.7.23.0) unstable; urgency=medium
 
   * Version 2.7.23

mcp-server/.gitignore

@@ -7,6 +7,12 @@ __pycache__/
 dist/
 build/
 
+# Persistent device-log capture (recorder + Datadog cursor).
+# Cross-session JSONL streams written by the autouse Recorder singleton
+# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
+# fixture truncate doesn't touch it.
+.mtlog/
+
 # Test harness artifacts
 tests/report.html
 tests/junit.xml

mcp-server/scripts/datadog-dashboard.json

@@ -0,0 +1,217 @@
+{
+  "title": "Meshtastic Firmware — Recorder Stream",
+  "description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
+  "widgets": [
+    {
+      "definition": {
+        "title": "Free heap (bytes)",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "free_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "label": "bytes" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Heap slope (bytes/min) — last 1h",
+        "type": "query_value",
+        "precision": 0,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "slope",
+                "data_source": "metrics",
+                "query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
+                "aggregator": "avg"
+              }
+            ],
+            "response_format": "scalar"
+          }
+        ],
+        "conditional_formats": [
+          { "comparator": "<", "value": -100, "palette": "white_on_red" },
+          { "comparator": "<", "value": 0, "palette": "white_on_yellow" },
+          { "comparator": ">=", "value": 0, "palette": "white_on_green" }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Total heap (bytes)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "total_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Battery level (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "battery",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "min": "0", "max": "105" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Air utilization (TX %)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "airutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Channel utilization (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "chutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Log volume by level",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "response_format": "timeseries",
+            "display_type": "bars",
+            "queries": [
+              {
+                "name": "log_count",
+                "data_source": "logs",
+                "indexes": ["*"],
+                "compute": { "aggregation": "count" },
+                "search": { "query": "service:meshtastic-firmware" },
+                "group_by": [
+                  {
+                    "facet": "@level",
+                    "limit": 10,
+                    "sort": { "order": "desc", "aggregation": "count" }
+                  }
+                ]
+              }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recent ERROR / CRIT firmware logs",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "@port", "width": "auto" },
+              { "field": "@level", "width": "auto" },
+              { "field": "@thread", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recorder marker events",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware @level:MARK",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    }
+  ],
+  "template_variables": [
+    {
+      "name": "port",
+      "prefix": "port",
+      "available_values": [],
+      "default": "*"
+    },
+    { "name": "host", "prefix": "host", "available_values": [], "default": "*" }
+  ],
+  "layout_type": "ordered",
+  "notify_list": [],
+  "reflow_type": "auto"
+}

mcp-server/scripts/mtlog_to_datadog.py

@@ -0,0 +1,389 @@
+#!/usr/bin/env python3
+"""Forward selected recorder JSONL streams to Datadog.
+
+Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
+Logs Intake API and telemetry numerics to the Metrics v2 series API.
+Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
+duplicate rows already shipped from the current live files.
+
+This forwarder does not currently backfill rotated `.jsonl.gz` archives.
+If the recorder rotates before this process drains the live file, or the
+forwarder is down across a rotation, those older rows are skipped.
+
+Usage:
+    DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
+    ./scripts/mtlog_to_datadog.py --once          # catch up + exit
+    ./scripts/mtlog_to_datadog.py --since 3600   # backfill last hour from start
+
+Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
+Override via `DD_SITE=...` env var or `--site` flag for one-offs.
+
+The forwarder is a separate process by design — a Datadog outage or
+auth failure must not backpressure the recorder. We exit non-zero on
+fatal config errors (missing API key) and keep retrying on transient
+network/HTTP errors.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import socket
+import sys
+import time
+from pathlib import Path
+from typing import Any, Iterator
+
+try:
+    import requests
+except ImportError:
+    print(
+        "requests is required. Install it in the mcp-server venv: "
+        "uv pip install requests",
+        file=sys.stderr,
+    )
+    sys.exit(2)
+
+
+_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
+_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
+_METRICS_TPL = "https://api.{site}/api/v2/series"
+_LOG_BATCH = 50
+_METRICS_BATCH = 100
+_MAX_RETRIES = 5
+_RETRY_BASE_S = 1.5
+
+
+# --- streaming JSONL with byte-position cursor -------------------------
+
+
+class _StreamReader:
+    """Reads a single rotating JSONL with cursor-based resume.
+
+    This tails only the live `.jsonl` file. The recorder rotates files
+    (live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
+    the live file shrinks abruptly. We detect that via inode change OR live
+    size < cursor position, and reset the live-file cursor to 0.
+    """
+
+    def __init__(self, path: Path, cursor: dict[str, Any]):
+        self.path = path
+        self.cursor = cursor
+
+    def _state(self) -> tuple[int, int]:
+        """Return (inode, size) for the live file. (0, 0) if missing."""
+        try:
+            st = self.path.stat()
+            return (st.st_ino, st.st_size)
+        except FileNotFoundError:
+            return (0, 0)
+
+    def iter_new_records(self) -> Iterator[dict[str, Any]]:
+        ino, size = self._state()
+        last_ino = self.cursor.get("ino")
+        last_pos = int(self.cursor.get("pos") or 0)
+        if ino == 0:
+            return
+        if last_ino is not None and last_ino != ino:
+            # Rotation happened. Start over.
+            last_pos = 0
+        if last_pos > size:
+            # Live file truncated/shrunk under us — recorder rotated.
+            last_pos = 0
+        try:
+            with self.path.open("r", encoding="utf-8") as fh:
+                fh.seek(last_pos)
+                for line in fh:
+                    line = line.rstrip("\n")
+                    if not line:
+                        continue
+                    try:
+                        yield json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    last_pos = fh.tell()
+        except FileNotFoundError:
+            return
+        self.cursor["ino"] = ino
+        self.cursor["pos"] = last_pos
+
+
+def _load_cursor(path: Path) -> dict[str, Any]:
+    if not path.exists():
+        return {}
+    try:
+        return json.loads(path.read_text())
+    except (OSError, json.JSONDecodeError):
+        return {}
+
+
+def _save_cursor(path: Path, data: dict[str, Any]) -> None:
+    tmp = path.with_suffix(".json.tmp")
+    tmp.write_text(json.dumps(data, separators=(",", ":")))
+    tmp.replace(path)
+
+
+# --- Datadog clients ---------------------------------------------------
+
+
+class _DDSession:
+    """Pool one HTTPS session, share retry logic."""
+
+    def __init__(self, api_key: str, site: str, hostname: str) -> None:
+        self.api_key = api_key
+        self.site = site
+        self.hostname = hostname
+        self.session = requests.Session()
+        self.session.headers.update(
+            {
+                "DD-API-KEY": api_key,
+                "Content-Type": "application/json",
+            }
+        )
+
+    def _post(self, url: str, payload: Any) -> bool:
+        for attempt in range(_MAX_RETRIES):
+            try:
+                resp = self.session.post(url, json=payload, timeout=30)
+            except requests.RequestException as e:
+                _wait_retry(attempt, f"network error: {e}")
+                continue
+            if 200 <= resp.status_code < 300:
+                return True
+            if resp.status_code in (408, 429, 500, 502, 503, 504):
+                _wait_retry(
+                    attempt,
+                    f"HTTP {resp.status_code} retrying",
+                )
+                continue
+            print(
+                f"datadog refused: {resp.status_code} {resp.text[:200]}",
+                file=sys.stderr,
+            )
+            return False
+        return False
+
+    def send_logs(self, records: list[dict[str, Any]]) -> int:
+        if not records:
+            return 0
+        url = _LOG_INTAKE_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(records), _LOG_BATCH):
+            batch = records[i : i + _LOG_BATCH]
+            if self._post(url, batch):
+                sent += len(batch)
+        return sent
+
+    def send_metrics(self, series: list[dict[str, Any]]) -> int:
+        if not series:
+            return 0
+        url = _METRICS_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(series), _METRICS_BATCH):
+            batch = series[i : i + _METRICS_BATCH]
+            if self._post(url, {"series": batch}):
+                sent += len(batch)
+        return sent
+
+
+def _wait_retry(attempt: int, reason: str) -> None:
+    wait = _RETRY_BASE_S * (2**attempt)
+    print(
+        f"  retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
+        file=sys.stderr,
+    )
+    time.sleep(wait)
+
+
+# --- record → datadog payload ------------------------------------------
+
+
+def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
+    line = rec.get("line") or ""
+    tags = [
+        f"role:{rec.get('role')}",
+        f"port:{rec.get('port')}",
+    ]
+    level = rec.get("level")
+    if level:
+        tags.append(f"level:{level}")
+    tag = rec.get("tag")
+    if tag:
+        tags.append(f"thread:{tag}")
+    return {
+        "ddsource": "meshtastic-firmware",
+        "service": "meshtastic-firmware",
+        "hostname": host,
+        "message": line,
+        "ddtags": ",".join(t for t in tags if t and "None" not in t),
+        "timestamp": int((rec.get("ts") or time.time()) * 1000),
+        "level": level,
+    }
+
+
+def _telemetry_record_to_metrics(
+    rec: dict[str, Any], host: str
+) -> list[dict[str, Any]]:
+    fields = rec.get("fields") or {}
+    if not isinstance(fields, dict):
+        return []
+    variant = rec.get("variant") or "unknown"
+    ts = int(rec.get("ts") or time.time())
+    out: list[dict[str, Any]] = []
+    tags = []
+    if rec.get("port"):
+        tags.append(f"port:{rec['port']}")
+    if rec.get("role"):
+        tags.append(f"role:{rec['role']}")
+    if rec.get("from_node"):
+        tags.append(f"from_node:{rec['from_node']}")
+    tags.append(f"variant:{variant}")
+    for field, value in fields.items():
+        if not isinstance(value, (int, float)) or isinstance(value, bool):
+            continue
+        metric = f"mesh.{variant}.{_metric_safe(field)}"
+        out.append(
+            {
+                "metric": metric,
+                "type": 3,  # GAUGE
+                "points": [{"timestamp": ts, "value": float(value)}],
+                "tags": tags,
+                "resources": [{"type": "host", "name": host}],
+            }
+        )
+    return out
+
+
+def _metric_safe(name: str) -> str:
+    # Lowercase, replace non-alnum with underscore for safe metric names.
+    return "".join(c.lower() if c.isalnum() else "_" for c in name)
+
+
+# --- main loop ---------------------------------------------------------
+
+
+def run(
+    log_dir: Path,
+    *,
+    once: bool,
+    since_seconds: float | None,
+    poll_interval: float,
+    dd: _DDSession,
+) -> int:
+    cursor_path = log_dir / ".dd-cursor.json"
+    cursors = _load_cursor(cursor_path)
+
+    # `--since` overrides cursor: rewind to (now-since) timestamp.
+    # We can't seek by timestamp directly (cursor is byte position), so
+    # we just reset cursors to 0 and let the time filter in iter_new
+    # drop older records.
+    cutoff_ts: float | None = None
+    if since_seconds is not None:
+        cursors = {}
+        cutoff_ts = time.time() - since_seconds
+
+    sent_total = {"logs": 0, "telemetry": 0}
+
+    while True:
+        # logs.jsonl → DD logs
+        log_cursor = cursors.setdefault("logs", {})
+        log_batch: list[dict[str, Any]] = []
+        for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            log_batch.append(_log_record_to_dd(rec, dd.hostname))
+        if log_batch:
+            n = dd.send_logs(log_batch)
+            sent_total["logs"] += n
+            print(f"logs: sent {n}/{len(log_batch)}")
+
+        # telemetry.jsonl → DD metrics
+        telem_cursor = cursors.setdefault("telemetry", {})
+        metric_series: list[dict[str, Any]] = []
+        for rec in _StreamReader(
+            log_dir / "telemetry.jsonl", telem_cursor
+        ).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
+        if metric_series:
+            n = dd.send_metrics(metric_series)
+            sent_total["telemetry"] += n
+            print(f"telemetry: sent {n}/{len(metric_series)} metric points")
+
+        _save_cursor(cursor_path, cursors)
+
+        if once:
+            print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
+            return 0
+        time.sleep(poll_interval)
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--log-dir",
+        default=str(_DEFAULT_LOG_DIR),
+        help="Path to .mtlog/ (default: mcp-server/.mtlog)",
+    )
+    mode = parser.add_mutually_exclusive_group()
+    mode.add_argument("--once", action="store_true", help="Catch up then exit")
+    mode.add_argument(
+        "--tail",
+        action="store_true",
+        help="Daemon: poll forever (default)",
+    )
+    parser.add_argument(
+        "--since",
+        type=float,
+        default=None,
+        help="Backfill last N seconds. Resets cursor.",
+    )
+    parser.add_argument(
+        "--poll-interval",
+        type=float,
+        default=5.0,
+        help="Seconds between tail polls (default 5)",
+    )
+    parser.add_argument(
+        "--site",
+        default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
+        help=(
+            "Datadog site. Default is the team's instance (us5.datadoghq.com). "
+            "Override via DD_SITE env var or this flag."
+        ),
+    )
+    parser.add_argument(
+        "--host",
+        default=socket.gethostname(),
+        help="Hostname tag (default: socket.gethostname())",
+    )
+    args = parser.parse_args(argv)
+
+    api_key = os.environ.get("DD_API_KEY")
+    if not api_key:
+        print("DD_API_KEY env var required.", file=sys.stderr)
+        return 2
+
+    log_dir = Path(args.log_dir)
+    if not log_dir.exists():
+        print(
+            f"log dir {log_dir} does not exist — start the mcp-server first.",
+            file=sys.stderr,
+        )
+        return 2
+
+    dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
+    once = args.once and not args.tail
+    return run(
+        log_dir,
+        once=once,
+        since_seconds=args.since,
+        poll_interval=args.poll_interval,
+        dd=dd,
+    )
+
+
+if __name__ == "__main__":
+    sys.exit(main())

mcp-server/src/meshtastic_mcp/flash.py

@@ -108,18 +108,33 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Run `pio run -e <env>` and return artifact paths.
 
     `userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
     into userPrefs.jsonc for this build only. File is restored byte-for-byte
     on exit. Use `userprefs_set()` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
+    this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
+    `{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
+    prefix on every log line. Combines with the recorder so heap shows
+    up at log cadence (much higher resolution than the ~60 s LocalStats
+    packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
+    expand to bare `-D<NAME>` (presence-only flags).
     """
     args = ["run", "-e", env]
     if with_manifest:
         args.extend(["-t", "mtjson"])
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
-        result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
+        result = pio.run(
+            args,
+            timeout=pio.TIMEOUT_BUILD,
+            check=False,
+            extra_env=extra_env,
+        )
     return {
         "exit_code": result.returncode,
         "artifacts": [str(p) for p in _artifacts_for(env)],
@@ -127,9 +142,27 @@ def build(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 
+def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
+    """Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
+    "-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
+    the flag entirely. Other types stringify."""
+    parts: list[str] = []
+    for key, value in build_flags.items():
+        if value is False or value is None:
+            continue
+        if value is True:
+            parts.append(f"-D{key}")
+        else:
+            parts.append(f"-D{key}={value}")
+    if not parts:
+        return {}
+    return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
+
+
 def clean(env: str) -> dict[str, Any]:
     """Run `pio run -e <env> -t clean`."""
     result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -146,20 +179,29 @@ def flash(
     port: str,
     confirm: bool = False,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """`pio run -e <env> -t upload --upload-port <port>`. All architectures.
 
     `userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
     that pio performs will pick up the injected values.
+
+    `build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
+    is exported for the rebuild-before-upload, so the uploaded firmware
+    actually carries the flags. Without this propagation, `pio run -t upload`
+    would relink without the env var and silently drop them. Common use:
+    `build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
     """
     _require_confirm(confirm, "flash")
     _reject_native_env(env, "flash")
     connection.reject_if_tcp(port, "flash")
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
         result = pio.run(
             ["run", "-e", env, "-t", "upload", "--upload-port", port],
             timeout=pio.TIMEOUT_UPLOAD,
             check=False,
+            extra_env=extra_env,
         )
     return {
         "exit_code": result.returncode,
@@ -167,6 +209,7 @@ def flash(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 

mcp-server/src/meshtastic_mcp/log_query.py

@@ -0,0 +1,410 @@
+"""Read-side queries over the recorder's JSONL streams.
+
+Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
+loads a whole file. Time-bound queries short-circuit as soon as `ts`
+exceeds the requested end. The recorder writes monotonically, so a
+forward scan is cheap; we don't need an index.
+
+All time arguments accept:
+  - epoch seconds (int/float)
+  - relative strings: "-15m", "-2h", "-3d", "now"
+  - ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
+    treated as UTC)
+
+Tools that return data ALWAYS cap their output (max_lines / max_points
+/ max), and report whether more matched than was returned.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import re
+import statistics
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Iterator
+
+from .recorder.recorder import get_recorder
+
+_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
+_REGEX_PREVIEW_MAX = 100
+_REGEX_PREVIEW_TRUNCATE = 97
+
+
+def _parse_time(value: Any, *, now: float | None = None) -> float:
+    """Coerce to epoch seconds. Defaults `now` to `time.time()`."""
+    if value is None:
+        return time.time()
+    if isinstance(value, (int, float)):
+        return float(value)
+    if not isinstance(value, str):
+        raise ValueError(f"invalid time: {value!r}")
+    s = value.strip().lower()
+    if s in ("", "now"):
+        return time.time() if now is None else now
+    m = _REL_RE.match(s)
+    if m:
+        n = float(m.group(1))
+        unit = m.group(2)
+        secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
+        base = time.time() if now is None else now
+        return base - secs
+    # Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
+    try:
+        if s.endswith("z"):
+            s = s[:-1] + "+00:00"
+        dt = datetime.fromisoformat(s)
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=timezone.utc)
+        return dt.timestamp()
+    except ValueError as e:
+        raise ValueError(f"unparseable time: {value!r}") from e
+
+
+def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
+    """Stream records in chronological order: rotated archives first
+    (oldest → newest by lex sort, which is chronological for our
+    `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
+    last. The "keep last N" pop-front logic in the window queries
+    relies on records arriving in time order across files.
+    """
+    files: list[Path] = []
+    # Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
+    for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
+        files.append(archive)
+    if path.exists():
+        files.append(path)
+    for f in files:
+        opener = gzip.open if f.suffix == ".gz" else open
+        try:
+            with opener(f, "rt", encoding="utf-8") as fh:  # type: ignore[arg-type]
+                for line in fh:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    try:
+                        rec = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    ts = rec.get("ts")
+                    if not isinstance(ts, (int, float)):
+                        continue
+                    if ts < since:
+                        continue
+                    if ts > until:
+                        # Records are append-monotonic within a file, so
+                        # the rest of this file is also past `until`.
+                        # Archives can still overlap each other, so only
+                        # short-circuit this file, not the whole scan.
+                        break
+                    yield rec
+        except (FileNotFoundError, OSError):
+            continue
+
+
+# -- queries ------------------------------------------------------------
+
+
+def logs_window(
+    start: Any = "-15m",
+    end: Any = "now",
+    *,
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines, filtered.
+
+    `level` accepts a single level name or pipe-separated set
+    ("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
+    `line` field. Returns the last `max_lines` matches.
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    levels = _split_set(level)
+    if grep:
+        try:
+            grep_re = re.compile(grep)
+        except re.error as exc:
+            preview = (
+                grep
+                if len(grep) <= _REGEX_PREVIEW_MAX
+                else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
+            )
+            raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
+    else:
+        grep_re = None
+
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
+        if levels and rec.get("level") not in levels:
+            continue
+        if tag and rec.get("tag") != tag:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        if grep_re and not grep_re.search(rec.get("line") or ""):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max_lines:
+            out.pop(0)  # keep the most recent N
+    return {
+        "lines": out,
+        "total_matched": matched,
+        "dropped": max(0, matched - max_lines),
+        "window": {"start": s, "end": e},
+    }
+
+
+def telemetry_timeline(
+    window: Any = "1h",
+    *,
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Timeseries of one telemetry field, downsampled.
+
+    `field` matches both the protobuf snake_case name (`free_heap`,
+    `heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
+    Server-side bucket-mean downsamples to ≤ `max_points`. Returns
+    `slope_per_min` (linear regression slope, units/min) so a leak
+    detector can read one number.
+    """
+    end = time.time()
+    if isinstance(window, (int, float)):
+        # Numeric `window` is a duration in seconds — "last N seconds".
+        # Without this branch, `_parse_time(-N)` would treat -N as an
+        # absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
+        # producing a wildly negative `start` and matching nothing.
+        start = end - float(window)
+    elif isinstance(window, str) and not window.startswith("-"):
+        # Bare string like "1h" is sugar for "-1h".
+        start = _parse_time(f"-{window}", now=end)
+    else:
+        start = _parse_time(window, now=end)
+
+    base = get_recorder().base_dir
+    raw: list[tuple[float, float]] = []
+    field_aliases = _field_aliases(field)
+    for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
+        if rec.get("variant") != variant:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        fields = rec.get("fields") or {}
+        value: Any = None
+        for alias in field_aliases:
+            if alias in fields:
+                value = fields[alias]
+                break
+        if not isinstance(value, (int, float)):
+            continue
+        raw.append((float(rec["ts"]), float(value)))
+
+    if not raw:
+        return {
+            "points": [],
+            "samples": 0,
+            "min": None,
+            "max": None,
+            "slope_per_min": None,
+            "window": {"start": start, "end": end, "variant": variant, "field": field},
+        }
+
+    points = _downsample(raw, max_points=max_points)
+    values = [v for _, v in raw]
+    return {
+        "points": [{"ts": ts, "value": v} for ts, v in points],
+        "samples": len(raw),
+        "min": min(values),
+        "max": max(values),
+        "slope_per_min": _slope_per_min(raw),
+        "window": {"start": start, "end": end, "variant": variant, "field": field},
+    }
+
+
+def packets_window(
+    start: Any = "-5m",
+    end: Any = "now",
+    *,
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    portnums = _split_set(portnum)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
+        if portnums and rec.get("portnum") not in portnums:
+            continue
+        if from_node and str(rec.get("from_node")) != str(from_node):
+            continue
+        if to_node and str(rec.get("to_node")) != str(to_node):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "packets": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def events_window(
+    start: Any = "-1h",
+    end: Any = "now",
+    *,
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    kinds = _split_set(kind)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
+        if kinds and rec.get("kind") not in kinds:
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "events": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def export(
+    start: Any,
+    end: Any,
+    dest_dir: str,
+    *,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of each requested stream into `dest_dir`.
+
+    For a notebook, a bug report, or a Datadog backfill. Output files
+    are uncompressed JSONL (callers gzip themselves if they want to).
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    selected = streams or ["logs", "telemetry", "packets", "events"]
+    dest = Path(dest_dir)
+    dest.mkdir(parents=True, exist_ok=True)
+
+    base = get_recorder().base_dir
+    paths: dict[str, str] = {}
+    for stream in selected:
+        src = base / f"{stream}.jsonl"
+        if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
+            continue
+        out_path = dest / f"{stream}.jsonl"
+        n = 0
+        with out_path.open("w", encoding="utf-8") as fh:
+            for rec in _iter_jsonl(src, since=s, until=e):
+                fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
+                n += 1
+        paths[stream] = str(out_path)
+        paths[f"{stream}_count"] = str(n)
+    return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
+
+
+# -- helpers ------------------------------------------------------------
+
+
+def _split_set(value: str | None) -> set[str] | None:
+    if not value:
+        return None
+    return {v.strip() for v in value.split("|") if v.strip()}
+
+
+def _field_aliases(field: str) -> list[str]:
+    """Accept snake_case OR camelCase, plus a few legacy aliases."""
+    snake = field
+    camel = _snake_to_camel(field)
+    aliases = {snake, camel}
+    # Old protobuf fields (pre-LocalStats) used different names
+    legacy = {
+        "free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
+        "heap_free_bytes": [
+            "heap_free_bytes",
+            "heapFreeBytes",
+            "free_heap",
+            "freeHeap",
+        ],
+        "total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
+        "heap_total_bytes": [
+            "heap_total_bytes",
+            "heapTotalBytes",
+            "total_heap",
+            "totalHeap",
+        ],
+    }
+    if field in legacy:
+        aliases.update(legacy[field])
+    return list(aliases)
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _downsample(
+    points: list[tuple[float, float]], *, max_points: int
+) -> list[tuple[float, float]]:
+    if len(points) <= max_points:
+        return points
+    # Even-bucket mean. Preserves shape better than nth-sample picking.
+    n = len(points)
+    bucket = n / max_points
+    out: list[tuple[float, float]] = []
+    i = 0
+    for k in range(max_points):
+        end = int((k + 1) * bucket)
+        end = min(end, n)
+        if end <= i:
+            continue
+        chunk = points[i:end]
+        ts = chunk[len(chunk) // 2][0]
+        val = statistics.fmean(v for _, v in chunk)
+        out.append((ts, val))
+        i = end
+    return out
+
+
+def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
+    """Least-squares slope (units per minute). None if too few points."""
+    if len(points) < 2:
+        return None
+    xs = [t for t, _ in points]
+    ys = [v for _, v in points]
+    n = len(xs)
+    mean_x = sum(xs) / n
+    mean_y = sum(ys) / n
+    num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
+    den = sum((x - mean_x) ** 2 for x in xs)
+    if den == 0:
+        return None
+    slope_per_sec = num / den
+    return slope_per_sec * 60.0

mcp-server/src/meshtastic_mcp/pio.py

@@ -92,6 +92,7 @@ def _run_capturing(
     cwd: Path | None = None,
     timeout: float | None = None,
     tee_header: str | None = None,
+    extra_env: dict[str, str] | None = None,
 ) -> tuple[int, str, str, float]:
     """Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
 
@@ -99,6 +100,9 @@ def _run_capturing(
     `subprocess.TimeoutExpired` on timeout (callers map this to their own
     domain-specific error).
 
+    `extra_env` merges into the subprocess environment (parent env stays
+    intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
+
     Fast path: `subprocess.run(capture_output=True)` when no flash log is
     configured (unchanged behavior).
 
@@ -110,6 +114,9 @@ def _run_capturing(
     """
     log_path = _flash_log_path()
     t0 = time.monotonic()
+    env = None
+    if extra_env:
+        env = {**os.environ, **extra_env}
 
     if log_path is None:
         # Fast path — unchanged.
@@ -119,6 +126,7 @@ def _run_capturing(
             capture_output=True,
             text=True,
             timeout=timeout,
+            env=env,
         )
         return (
             proc.returncode,
@@ -145,6 +153,7 @@ def _run_capturing(
         stderr=subprocess.PIPE,
         text=True,
         bufsize=1,  # line-buffered
+        env=env,
     )
     stdout_chunks: list[str] = []
     stderr_chunks: list[str] = []
@@ -232,12 +241,17 @@ def run(
     cwd: Path | None = None,
     timeout: float | None = TIMEOUT_DEFAULT,
     check: bool = True,
+    extra_env: dict[str, str] | None = None,
 ) -> PioResult:
     """Invoke `pio <args>` and return captured output.
 
     `cwd` defaults to the firmware root. `check=True` raises `PioError` on
     non-zero exit; set `check=False` to inspect `returncode` manually.
 
+    `extra_env` merges into the subprocess environment — used for
+    `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
+    toggles that can't be expressed as command-line args.
+
     If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
     line-by-line as it arrives (for live flash progress in the TUI).
     """
@@ -250,6 +264,7 @@ def run(
             cwd=work_dir,
             timeout=timeout,
             tee_header=f"pio {' '.join(args)}",
+            extra_env=extra_env,
         )
     except subprocess.TimeoutExpired as exc:
         raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc

mcp-server/src/meshtastic_mcp/recorder/__init__.py

@@ -0,0 +1,19 @@
+"""Persistent device-log capture.
+
+Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
+(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
+and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
+process-global so a single subscription captures every active interface
+(serial / TCP / BLE) without any per-connection bookkeeping.
+
+The recorder is opt-in-by-import: importing this package is a no-op; call
+`get_recorder().start()` (which `server.py` does at FastMCP app init) to
+begin writing. `pause()` / `resume()` exist for the rare case the user
+wants a clean stretch of file (e.g. capturing a known-good baseline).
+"""
+
+from __future__ import annotations
+
+from .recorder import Recorder, get_recorder
+
+__all__ = ["Recorder", "get_recorder"]

mcp-server/src/meshtastic_mcp/recorder/parsers.py

@@ -0,0 +1,309 @@
+"""Best-effort parsers for log lines and telemetry packets.
+
+Two flavors of log line cross our pubsub subscription:
+  1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
+     accumulates bytes between protobuf frames and emits the full
+     firmware-formatted line, e.g.
+         "INFO  | 12:34:56 12345 [Main] Booting"
+     — level, HH:MM:SS, uptime seconds, thread bracket, then message.
+  2. LogRecord protobuf path (debug_log_api enabled): the lib calls
+     `_handleLogLine(record.message)` with ONLY the message body. The
+     level/source/time fields on the LogRecord are dropped before
+     pubsub fan-out. We get e.g. just "Booting".
+
+Both arrive on `meshtastic.log.line`. The parser tries to recover a
+level + thread when the prefix is present and falls back to level=None
+otherwise. Consumers who want level filtering on protobuf-mode hosts
+should grep the raw `line` field instead.
+
+Telemetry: `meshtastic.receive.telemetry` packets carry one of several
+metric variants in `packet["decoded"]["telemetry"]`. We flatten the
+chosen variant into a {field: value} dict so callers don't have to
+know the protobuf shape.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
+# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
+# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
+# macros must stay in sync with these strings.
+_LINE_RE = re.compile(
+    r"""
+    ^
+    (?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
+    \s*\|\s*
+    (?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
+    \s+
+    (?P<uptime>\d+)
+    \s+
+    (?:\[(?P<thread>[^\]]+)\]\s+)?
+    (?P<msg>.*)
+    $
+    """,
+    re.VERBOSE,
+)
+
+# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
+# thread bracket. See src/RedirectablePrint.cpp:175.
+_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
+
+# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
+# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
+#         "++++++ Thread NAME freed heap A -> B (delta) ++++++"
+_THREAD_HEAP_RE = re.compile(
+    r"""
+    ^[\-+]+\s*
+    Thread\s+(?P<thread>\S+)\s+
+    (?P<kind>leaked|freed)\s+heap\s+
+    (?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
+    \((?P<delta>-?\d+)\)
+    """,
+    re.VERBOSE,
+)
+
+# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
+# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
+_HEAP_STATUS_RE = re.compile(
+    r"""
+    Heap\s+status:\s+
+    (?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
+    (?:\s+\((?P<delta>-?\d+)\))?
+    """,
+    re.VERBOSE,
+)
+
+
+_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
+_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
+
+
+def parse_log_line(line: str) -> dict[str, Any]:
+    """Best-effort decompose a raw firmware log line.
+
+    Returns a dict with at least `line` (the original, unmodified — ANSI
+    codes preserved for fidelity). Adds `level`, `tag`, `clock`,
+    `uptime_s`, and `msg` when the full prefix is present.
+
+    Handles two firmware quirks:
+      - LogRecord.message can carry ANSI color escapes from RedirectablePrint
+        (the BLE/StreamAPI path inherited the colored body in some builds).
+        We strip ANSI before regex matching so the prefix survives.
+      - DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
+        thread name is set, the heap takes the thread bracket position —
+        looks like `[heap 12345] msg`. We detect that shape and move it
+        out of `tag` and into `heap_free`.
+
+    DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
+    (bytes), and when a `Thread X leaked|freed heap` line is recognized,
+    `heap_event` = {kind, thread, before, after, delta}.
+
+    Never raises.
+    """
+    out: dict[str, Any] = {"line": line}
+    if not line:
+        return out
+
+    # Strip ANSI escapes BEFORE any regex matching. The original `line`
+    # stays in `out["line"]` for fidelity / future grep.
+    clean = _ANSI_RE.sub("", line)
+
+    m = _LINE_RE.match(clean)
+    msg: str | None = None
+    if m:
+        level = m.group("level").rstrip()
+        out["level"] = level
+        out["clock"] = m.group("clock")
+        try:
+            out["uptime_s"] = int(m.group("uptime"))
+        except (TypeError, ValueError):
+            out["uptime_s"] = None
+        thread = m.group("thread")
+        if thread:
+            # If "thread" is actually the heap prefix taking the bracket
+            # position (DEBUG_HEAP build, no thread set), capture heap
+            # and leave tag unset.
+            hb = _HEAP_BRACKET_RE.match(thread.strip())
+            if hb:
+                try:
+                    out["heap_free"] = int(hb.group("heap"))
+                except (TypeError, ValueError):
+                    pass
+            else:
+                out["tag"] = thread
+        msg = m.group("msg")
+        out["msg"] = msg
+    else:
+        # No prefix — bare LogRecord.message body. Inspect the whole
+        # line for DEBUG_HEAP-style content; the heap-prefix and
+        # thread-leak patterns can survive on either path.
+        msg = clean
+
+    # DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
+    # Sits AFTER the thread bracket and BEFORE the message body, but
+    # for bare LogRecord lines it's at the start. Match it at the
+    # head of `msg`.
+    if msg:
+        hp = _HEAP_PREFIX_RE.match(msg)
+        if hp:
+            try:
+                out["heap_free"] = int(hp.group("heap"))
+            except (TypeError, ValueError):
+                pass
+            else:
+                # Strip the prefix from `msg` so a grep on the message
+                # body doesn't have to know about it.
+                out["msg"] = hp.group("rest")
+                msg = hp.group("rest")
+
+        # Thread-level leak/free detection.
+        thr = _THREAD_HEAP_RE.search(msg)
+        if thr:
+            try:
+                out["heap_event"] = {
+                    "kind": thr.group("kind"),
+                    "thread": thr.group("thread"),
+                    "before": int(thr.group("before")),
+                    "after": int(thr.group("after")),
+                    "delta": int(thr.group("delta")),
+                }
+            except (TypeError, ValueError):
+                pass
+
+        # Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
+        hs = _HEAP_STATUS_RE.search(msg)
+        if hs:
+            try:
+                out["heap_free"] = int(hs.group("free"))
+                out["heap_total"] = int(hs.group("total"))
+                if hs.group("delta") is not None:
+                    out["heap_delta"] = int(hs.group("delta"))
+            except (TypeError, ValueError):
+                pass
+
+    return out
+
+
+# -- Telemetry ----------------------------------------------------------
+
+# Order matters: meshtastic-python decoded packets use the protobuf
+# `oneof variant` field name (snake_case) as the dict key.
+_TELEMETRY_VARIANTS = (
+    ("device_metrics", "device"),
+    ("local_stats", "local"),
+    ("environment_metrics", "environment"),
+    ("power_metrics", "power"),
+    ("air_quality_metrics", "airQuality"),
+    ("health_metrics", "health"),
+    ("host_metrics", "host"),
+)
+
+
+def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
+    """Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
+    packet. Returns None when the shape isn't what we expect — so the
+    caller can fall back to a generic packets.jsonl row.
+    """
+    if not isinstance(packet, dict):
+        return None
+    decoded = packet.get("decoded")
+    if not isinstance(decoded, dict):
+        return None
+    telem = decoded.get("telemetry")
+    if not isinstance(telem, dict):
+        return None
+    # The Python lib produces dict-of-camelCase keys via MessageToDict.
+    # Try both camelCase and snake_case to be robust to lib version drift.
+    for snake, label in _TELEMETRY_VARIANTS:
+        camel = _snake_to_camel(snake)
+        for key in (snake, camel):
+            value = telem.get(key)
+            if isinstance(value, dict):
+                return {
+                    "variant": label,
+                    "fields": {k: _scalarize(v) for k, v in value.items()},
+                    "time": telem.get("time"),
+                }
+    return None
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _scalarize(value: Any) -> Any:
+    """Keep telemetry fields JSON-friendly. Lists/dicts pass through
+    untouched; bytes -> hex string; protobuf enums occasionally arrive
+    as ints (fine) or strings (also fine)."""
+    if isinstance(value, (bytes, bytearray, memoryview)):
+        return bytes(value).hex()
+    return value
+
+
+# -- Generic packet summary ---------------------------------------------
+
+
+def summarize_packet(
+    packet: dict[str, Any], *, payload_hex_len: int = 64
+) -> dict[str, Any]:
+    """Reduce a packet dict to a stable, queryable summary. Drops the
+    full payload bytes — the recorder records summaries, not pcaps.
+    """
+    if not isinstance(packet, dict):
+        return {"raw_type": type(packet).__name__}
+    decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
+    portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
+    payload = decoded.get("payload") if isinstance(decoded, dict) else None
+    payload_hex = None
+    payload_size = None
+    if isinstance(payload, (bytes, bytearray, memoryview)):
+        b = bytes(payload)
+        payload_size = len(b)
+        payload_hex = b[:payload_hex_len].hex() if b else ""
+    elif isinstance(payload, str):
+        # Some decoded payloads (text messages) come as decoded strings.
+        payload_size = len(payload)
+        payload_hex = None  # not bytes
+    return {
+        "from_node": packet.get("fromId") or packet.get("from"),
+        "to_node": packet.get("toId") or packet.get("to"),
+        "portnum": portnum,
+        "hop_limit": packet.get("hopLimit"),
+        "want_ack": packet.get("wantAck"),
+        "rx_rssi": packet.get("rxRssi"),
+        "rx_snr": packet.get("rxSnr"),
+        "channel": packet.get("channel"),
+        "id": packet.get("id"),
+        "payload_size": payload_size,
+        "payload_hex_prefix": payload_hex,
+    }
+
+
+# -- Interface identification ------------------------------------------
+
+
+def interface_label(interface: Any) -> dict[str, Any]:
+    """Stable identifier for the meshtastic interface that emitted an event.
+
+    Used as the `port`/`role` tag on every recorded row. SerialInterface
+    has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
+    has `address`. Falls back to the class name when none of those exist.
+    """
+    if interface is None:
+        return {"port": None, "role": None}
+    dev_path = getattr(interface, "devPath", None)
+    if dev_path:
+        return {"port": str(dev_path), "role": "serial"}
+    hostname = getattr(interface, "hostname", None)
+    if hostname:
+        port_num = getattr(interface, "portNumber", None)
+        endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
+        return {"port": endpoint, "role": "tcp"}
+    address = getattr(interface, "address", None)
+    if address:
+        return {"port": str(address), "role": "ble"}
+    return {"port": type(interface).__name__, "role": None}

mcp-server/src/meshtastic_mcp/recorder/recorder.py

@@ -0,0 +1,467 @@
+"""Process-global recorder singleton.
+
+Subscribes once to the meshtastic pubsub fan-out and writes four append-only
+JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
+process-global — a single subscription captures every active interface
+without per-connection bookkeeping.
+
+Files:
+  logs.jsonl      — every `meshtastic.log.line` event (best-effort prefix
+                    parsed for level/tag/uptime; raw `line` always preserved)
+  telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
+                    variant (device / local / environment / power / etc.)
+  packets.jsonl   — every other `meshtastic.receive.*` packet, summarized
+                    (portnum, hops, RSSI/SNR, payload size + 64-byte hex)
+  events.jsonl    — connection lifecycle, node-DB updates, and manual
+                    `mark_event` rows. Lower volume; useful for aligning
+                    timelines.
+
+Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
+write methods short-circuit when paused, so we don't lose ordering when
+resumed (we just have a gap). No queueing.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+from pathlib import Path
+from typing import Any
+
+from . import parsers
+from .rotating import _RotatingJsonl
+
+_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
+log = logging.getLogger(__name__)
+
+
+class Recorder:
+    """Singleton write-side of the persistent log capture system."""
+
+    def __init__(self, base_dir: Path | None = None) -> None:
+        self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
+        self._lock = threading.RLock()
+        self._started = False
+        self._paused = False
+        self._pause_reason: str | None = None
+        self._started_at: float | None = None
+        self._handlers: list[tuple[str, Any]] = []
+        self._files: dict[str, _RotatingJsonl] = {}
+
+    # -- lifecycle ----------------------------------------------------
+
+    def start(self) -> None:
+        """Idempotent. Safe to call from FastMCP app startup."""
+        with self._lock:
+            if self._started:
+                return
+            self.base_dir.mkdir(parents=True, exist_ok=True)
+            self._files = {
+                "logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
+                "telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
+                "packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
+                "events": _RotatingJsonl(self.base_dir / "events.jsonl"),
+            }
+            self._wire_pubsub()
+            self._started = True
+            self._started_at = time.time()
+        # Write the recorder_start marker after the initialization block.
+        # `_write_event()` re-checks recorder state via `_files_snapshot()`,
+        # so keeping this out of the setup block avoids nested lifecycle work.
+        self._write_event(kind="recorder_start", label="recorder_started")
+
+    def stop(self) -> None:
+        with self._lock:
+            if not self._started:
+                return
+            self._unwire_pubsub()
+            for f in self._files.values():
+                f.close()
+            self._files = {}
+            self._started = False
+
+    def pause(self, reason: str | None = None) -> None:
+        # Write the pause marker BEFORE flipping the flag — `_write_event`
+        # short-circuits when paused, so the order matters for this event
+        # to actually land in events.jsonl.
+        self._write_event(
+            kind="recorder_pause",
+            label="paused",
+            note=reason,
+        )
+        with self._lock:
+            self._paused = True
+            self._pause_reason = reason
+
+    def resume(self) -> None:
+        # Mirror of `pause()`: clear the flag first, then write the marker
+        # so it isn't suppressed by the still-paused short-circuit.
+        with self._lock:
+            self._paused = False
+            self._pause_reason = None
+        self._write_event(kind="recorder_resume", label="resumed")
+
+    # -- pubsub wiring ------------------------------------------------
+
+    def _wire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        # Subscribers — one per topic. Each pubsub publisher sends
+        # keyword args matching its handler's signature; pubsub
+        # introspects the function signature to route args.
+        bindings = [
+            ("meshtastic.log.line", self._on_log_line),
+            ("meshtastic.serial.line", self._on_serial_line),
+            ("meshtastic.receive", self._on_receive),
+            ("meshtastic.receive.telemetry", self._on_telemetry),
+            ("meshtastic.connection.established", self._on_connection_established),
+            ("meshtastic.connection.lost", self._on_connection_lost),
+            ("meshtastic.node.updated", self._on_node_updated),
+        ]
+        for topic, handler in bindings:
+            try:
+                pub.subscribe(handler, topic)
+                self._handlers.append((topic, handler))
+            except Exception as exc:
+                # If pubsub refuses one binding (signature mismatch on
+                # an old lib version), log it and keep the rest.
+                log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
+
+    def _unwire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        for topic, handler in self._handlers:
+            try:
+                pub.unsubscribe(handler, topic)
+            except Exception:
+                pass
+        self._handlers.clear()
+
+    # -- handlers -----------------------------------------------------
+    #
+    # Pubsub callbacks must never raise. Every handler is wrapped in a
+    # try/except that swallows so a bug here can't take down the
+    # SerialInterface receive thread.
+    #
+    # Threading: handlers fire on whatever thread the meshtastic library
+    # dispatches from (varies by interface), while `stop()` clears
+    # `self._files` under `self._lock`. We snapshot `_files` under the
+    # lock at the top of each handler so a concurrent stop can't
+    # KeyError us mid-write. The actual file write goes through
+    # `_RotatingJsonl` which has its own lock.
+
+    def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
+        """Atomic-ish view of `self._files`. Returns None when the recorder
+        is paused or stopped, so handlers can early-exit cleanly without
+        racing `stop()`'s clear."""
+        with self._lock:
+            if not self._started or self._paused:
+                return None
+            return dict(self._files)
+
+    def _on_log_line(self, line: str, interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": tags["port"],
+                "role": tags["role"],
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            # DEBUG_HEAP enrichments (only present when the firmware
+            # was built with -DDEBUG_HEAP=1). Surface as first-class
+            # fields so logs_window can grep/filter on them and so
+            # heap_free synthesizes a telemetry point below.
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # If the line carried a heap snapshot, also write it as a
+            # synthesized LocalStats-shaped row so telemetry_timeline
+            # picks it up at log cadence (much higher resolution than
+            # the ~60 s LocalStats packet). Tagged source=debug_heap so
+            # consumers can filter if mixing scales is unwanted.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": tags["port"],
+                        "role": tags["role"],
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_serial_line(self, line: str, port: str | None = None) -> None:
+        """Text-mode passive tap. Fired from `serial_session._drain` when a
+        `pio device monitor` subprocess is running.
+
+        Same parse + heap-synthesis path as `_on_log_line`, but receives
+        the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
+        body). On DEBUG_HEAP builds in text mode this gives us per-log-line
+        heap data — far higher cadence than LocalStats, and works without
+        protobuf API mode (no SerialInterface required).
+        """
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": port,
+                "role": "serial_session",
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # Synthesize a heap_free telemetry sample whenever the line
+            # carries one — same logic as _on_log_line, tagged source so
+            # consumers can distinguish text-mode tap from protobuf path.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": port,
+                        "role": "serial_session",
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap_serial",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            extracted = parsers.extract_telemetry(packet)
+            if extracted is None:
+                # Couldn't extract a known variant — fall through to the
+                # generic `_on_receive` path, which will still fire for
+                # this packet via the parent topic.
+                return
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                "from_node": packet.get("fromId") or packet.get("from"),
+                "variant": extracted["variant"],
+                "fields": extracted["fields"],
+                "device_time": extracted.get("time"),
+            }
+            files["telemetry"].write(record)
+        except Exception:
+            pass
+
+    def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
+        # Generic-receive fires for EVERY packet. Telemetry packets get
+        # recorded twice (here and in _on_telemetry) — that's intentional:
+        # packets.jsonl is the universal record, telemetry.jsonl is the
+        # structured timeseries view.
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            summary = parsers.summarize_packet(packet)
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                **summary,
+            }
+            files["packets"].write(record)
+        except Exception:
+            pass
+
+    def _on_connection_established(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_established",
+            interface=interface,
+        )
+
+    def _on_connection_lost(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_lost",
+            interface=interface,
+        )
+
+    def _on_node_updated(
+        self, node: dict[str, Any] | None = None, interface: Any = None
+    ) -> None:
+        # Lower-volume than packets but informative — node ID, hops away,
+        # last heard. Skip the user dict if absent.
+        try:
+            user = (node or {}).get("user") if isinstance(node, dict) else None
+            self._write_event(
+                kind="node_updated",
+                interface=interface,
+                data={
+                    "num": (node or {}).get("num"),
+                    "id": (user or {}).get("id"),
+                    "short": (user or {}).get("shortName"),
+                    "long": (user or {}).get("longName"),
+                    "hops_away": (node or {}).get("hopsAway"),
+                    "snr": (node or {}).get("snr"),
+                    "last_heard": (node or {}).get("lastHeard"),
+                },
+            )
+        except Exception:
+            pass
+
+    # -- public write helpers -----------------------------------------
+
+    def mark_event(
+        self,
+        label: str,
+        note: str | None = None,
+        data: dict[str, Any] | None = None,
+    ) -> dict[str, Any]:
+        """User-facing marker. Writes to events.jsonl AND emits a
+        synthetic logs.jsonl row tagged level=MARK so timelines align.
+        """
+        ts = self._write_event(kind="mark", label=label, note=note, data=data)
+        # Mirror into logs so a single logs_window grep finds it.
+        files = self._files_snapshot()
+        if files is not None:
+            try:
+                files["logs"].write(
+                    {
+                        "ts": ts,
+                        "port": None,
+                        "role": "marker",
+                        "level": "MARK",
+                        "tag": "mark_event",
+                        "line": f"[mark] {label}" + (f" — {note}" if note else ""),
+                    }
+                )
+            except Exception:
+                pass
+        return {"ts": ts, "label": label}
+
+    def _write_event(
+        self,
+        *,
+        kind: str,
+        label: str | None = None,
+        note: str | None = None,
+        interface: Any = None,
+        data: dict[str, Any] | None = None,
+    ) -> float:
+        ts = time.time()
+        # Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
+        # arrive at choreographed moments — `pause()` writes BEFORE flipping
+        # the flag and `resume()` writes AFTER clearing it, so those calls
+        # see _paused=False here. Other event kinds short-circuit when
+        # paused via the snapshot guard below.
+        files = self._files_snapshot()
+        if files is None:
+            return ts
+        try:
+            tags = parsers.interface_label(interface)
+            files["events"].write(
+                {
+                    "ts": ts,
+                    "kind": kind,
+                    "label": label,
+                    "note": note,
+                    "port": tags["port"],
+                    "role": tags["role"],
+                    "data": data,
+                }
+            )
+        except Exception:
+            pass
+        return ts
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            return {
+                "running": self._started,
+                "paused": self._paused,
+                "pause_reason": self._pause_reason,
+                "started_at": self._started_at,
+                "base_dir": str(self.base_dir),
+                "files": {name: f.status() for name, f in self._files.items()},
+            }
+
+    def force_rotate_all(self) -> dict[str, Any]:
+        """Test/admin hook: rotate every stream right now."""
+        with self._lock:
+            files = list(self._files.values())
+        for f in files:
+            f.force_rotate()
+        # `status()` re-acquires `self._lock`; release before calling it.
+        return self.status()
+
+
+# -- module-level singleton accessor ------------------------------------
+
+_INSTANCE_LOCK = threading.Lock()
+_INSTANCE: Recorder | None = None
+
+
+def get_recorder() -> Recorder:
+    """Return the process-global Recorder. Created on first call.
+
+    Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
+    (used by tests to redirect to a tmpdir).
+    """
+    global _INSTANCE
+    with _INSTANCE_LOCK:
+        if _INSTANCE is None:
+            override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
+            base = Path(override) if override else None
+            _INSTANCE = Recorder(base_dir=base)
+        return _INSTANCE

mcp-server/src/meshtastic_mcp/recorder/rotating.py

@@ -0,0 +1,163 @@
+"""Append-only JSONL writer with size-capped rotation.
+
+A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
+JSON objects (one row per call). When the live file exceeds `max_bytes`,
+it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
+and the live file resets to empty. Old archives past `keep_archives` are
+unlinked oldest-first.
+
+Size check is amortized — `os.fstat` runs every `check_every` writes,
+not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
+
+Threading: every public method acquires `self._lock`. The recorder runs
+several pubsub handlers on whatever thread the meshtastic library
+dispatches from (varies by interface), and queries from MCP tool calls
+arrive on the FastMCP request thread, so this lock is not optional.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import os
+import shutil
+import threading
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+class _RotatingJsonl:
+    """Append-only JSONL with size rotation. Thread-safe."""
+
+    def __init__(
+        self,
+        path: Path,
+        *,
+        max_bytes: int = 100 * 1024 * 1024,
+        keep_archives: int = 5,
+        check_every: int = 1000,
+    ) -> None:
+        self.path = path
+        self.max_bytes = max_bytes
+        self.keep_archives = keep_archives
+        self.check_every = check_every
+        self._lock = threading.Lock()
+        self._fh: Any = None
+        self._writes_since_check = 0
+        self._rotations = 0
+        self._lines_written = 0
+        self._last_ts: float | None = None
+        self._open()
+
+    # -- lifecycle ----------------------------------------------------
+
+    def _open(self) -> None:
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        self._fh = self.path.open("a", encoding="utf-8")
+
+    def close(self) -> None:
+        with self._lock:
+            if self._fh is not None:
+                try:
+                    self._fh.close()
+                finally:
+                    self._fh = None
+
+    # -- write --------------------------------------------------------
+
+    def write(self, record: dict[str, Any]) -> None:
+        """Append one JSON object as a line. Triggers rotation if oversized."""
+        line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
+        with self._lock:
+            if self._fh is None:
+                return
+            try:
+                self._fh.write(line)
+                self._fh.flush()
+            except Exception:
+                # Best-effort: a failed write must not crash the pubsub
+                # handler. Caller has no way to react anyway.
+                return
+            self._lines_written += 1
+            ts = record.get("ts")
+            if isinstance(ts, (int, float)):
+                self._last_ts = float(ts)
+            self._writes_since_check += 1
+            if self._writes_since_check >= self.check_every:
+                self._writes_since_check = 0
+                self._maybe_rotate()
+
+    # -- rotation -----------------------------------------------------
+
+    def _maybe_rotate(self) -> None:
+        # Caller holds self._lock.
+        try:
+            size = os.fstat(self._fh.fileno()).st_size
+        except OSError:
+            return
+        if size < self.max_bytes:
+            return
+        self._rotate_locked()
+
+    def _rotate_locked(self) -> None:
+        # Close, gzip-rename, reopen empty, prune oldest archives.
+        try:
+            self._fh.close()
+        except Exception:
+            pass
+        self._fh = None
+        # Microsecond-resolution timestamp + per-instance counter so back-
+        # to-back rotations (small max_bytes, repeated `force_rotate()`,
+        # or chatty test loops) get unique archive filenames. The lex
+        # sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
+        # which `_prune_archives()` and `log_query._iter_jsonl()` both
+        # rely on.
+        stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
+        archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
+        try:
+            with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
+                shutil.copyfileobj(src, dst, length=1024 * 1024)
+            self.path.unlink()
+        except Exception:
+            # Rotation is best-effort. If gzip fails, leave the file
+            # in place and re-open it; we'll try again next check.
+            pass
+        self._open()
+        self._rotations += 1
+        self._prune_archives()
+
+    def _prune_archives(self) -> None:
+        # Match siblings of self.path.name with `.jsonl.gz` suffix.
+        prefix = self.path.stem  # "logs" for "logs.jsonl"
+        # Archive filenames are already lexicographically chronological.
+        # Prune by name, not mtime, so copied/restored files don't reorder.
+        archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
+        excess = len(archives) - self.keep_archives
+        for old in archives[: max(0, excess)]:
+            try:
+                old.unlink()
+            except OSError:
+                pass
+
+    def force_rotate(self) -> None:
+        """Test/admin hook: rotate immediately regardless of size."""
+        with self._lock:
+            if self._fh is not None:
+                self._rotate_locked()
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            try:
+                size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
+            except OSError:
+                size = 0
+            return {
+                "path": str(self.path),
+                "size": size,
+                "lines": self._lines_written,
+                "last_ts": self._last_ts,
+                "rotations": self._rotations,
+            }

mcp-server/src/meshtastic_mcp/serial_session.py

@@ -46,7 +46,23 @@ class SerialSession:
 
 
 def _drain(session: SerialSession) -> None:
-    """Reader thread: line-by-line pull stdout into buffer."""
+    """Reader thread: line-by-line pull stdout into buffer.
+
+    Each line is also published to the `meshtastic.serial.line` pubsub
+    topic so the persistent recorder can capture it without holding its
+    own port. This is the text-mode tap path: when no SerialInterface is
+    open, the firmware emits full formatted lines (level + clock + uptime
+    + thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
+    fan them out to whoever is listening. Pubsub is best-effort —
+    publish failures must never block the reader.
+    """
+    # Lazy import: pubsub isn't required just to import this module
+    # (e.g., during static analysis), and we want a clean test surface.
+    try:
+        from pubsub import pub  # type: ignore[import-untyped]
+    except Exception:  # pragma: no cover - defensive
+        pub = None
+
     assert session.proc.stdout is not None
     try:
         for line in session.proc.stdout:
@@ -54,6 +70,16 @@ def _drain(session: SerialSession) -> None:
             with session.lock:
                 session.buffer.append(line_stripped)
                 session.total_lines += 1
+            if pub is not None:
+                try:
+                    pub.sendMessage(
+                        "meshtastic.serial.line",
+                        line=line_stripped,
+                        port=session.port,
+                    )
+                except Exception:
+                    # A subscriber raising must not break the reader.
+                    pass
     except Exception:  # pragma: no cover - defensive
         pass
     finally:

mcp-server/src/meshtastic_mcp/server.py

@@ -6,6 +6,7 @@ etc.). Business logic does not live here.
 
 from __future__ import annotations
 
+import logging
 from typing import Any
 
 from mcp.server.fastmcp import FastMCP
@@ -17,14 +18,34 @@ from . import (
     flash,
     hw_tools,
     info,
+    log_query,
     registry,
     serial_session,
 )
 from . import userprefs as userprefs_mod
+from .recorder import get_recorder
+
+log = logging.getLogger(__name__)
 
 app = FastMCP("meshtastic-mcp")
 
 
+def _start_recorder() -> None:
+    # Persistent device-log capture. Starts on first import — pubsub fan-out
+    # is process-global, so subscribing here captures every active interface
+    # (whether opened by an MCP tool, a pytest fixture, or a serial_session).
+    # Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
+    # for the full design. Recorder startup is best-effort: an unwritable
+    # log dir or pubsub mismatch should not take the MCP server down.
+    try:
+        get_recorder().start()
+    except Exception as exc:
+        log.warning("Failed to start persistent recorder: %s", exc)
+
+
+_start_recorder()
+
+
 # ---------- Discovery & metadata ------------------------------------------
 
 
@@ -75,6 +96,7 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Build firmware for one env via `pio run -e <env>`.
 
@@ -86,8 +108,21 @@ def build(
     build via userPrefs.jsonc injection. The file is restored after the build
     completes. Use `userprefs_manifest` to discover available keys. Use
     `userprefs_set` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
+    only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
+    `build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
+    `[heap N]` prefix on every log line. The recorder picks the prefix up
+    automatically and synthesizes a high-resolution heap timeline that
+    `telemetry_timeline(field="free_heap")` can read alongside the normal
+    ~60 s LocalStats packets. Pair with `/leakhunt` for classification.
     """
-    return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
+    return flash.build(
+        env,
+        with_manifest=with_manifest,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -105,6 +140,7 @@ def pio_flash(
     port: str,
     confirm: bool = False,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
 
@@ -114,8 +150,19 @@ def pio_flash(
 
     `userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
     build via userPrefs.jsonc injection; restored after upload.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
+    rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
+    to actually land in the uploaded firmware — without it, the implicit
+    rebuild relinks without the env var and silently drops them.
     """
-    return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
+    return flash.flash(
+        env,
+        port,
+        confirm=confirm,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -734,3 +781,185 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
 def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
     """Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
     return hw_tools.picotool_raw(args, confirm=confirm)
+
+
+# ---------- Persistent device-log capture (recorder) ----------------------
+#
+# The recorder is autouse — it starts at server import and continuously
+# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
+# tools are query-only over those files, plus a few lifecycle controls.
+
+
+@app.tool()
+def logs_window(
+    start: str = "-15m",
+    end: str = "now",
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines from the persistent recorder.
+
+    Filters by time window, regex over the line, level (single or
+    pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
+    interface port. Returns up to max_lines most-recent matches.
+
+    Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
+
+    Note: lines arriving via the LogRecord protobuf path (when
+    set_debug_log_api(True) is on) come without level prefix — the
+    meshtastic Python lib drops record.level before fan-out. For those,
+    `level` filter won't match; use `grep` instead.
+    """
+    return log_query.logs_window(
+        start=start,
+        end=end,
+        grep=grep,
+        level=level,
+        tag=tag,
+        port=port,
+        max_lines=max_lines,
+    )
+
+
+@app.tool()
+def telemetry_timeline(
+    window: str = "1h",
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Time series of one telemetry field, downsampled to <= max_points.
+
+    `variant` ∈ device, local, environment, power, airQuality, health, host.
+    `field` accepts snake_case or camelCase; common aliases (free_heap ↔
+    heap_free_bytes) are normalized.
+
+    Returns slope_per_min (linear-regression slope, units/minute) so a
+    leak detector can read one number — negative slope on free_heap over
+    a long window indicates a real leak.
+
+    LocalStats variant ("local") cadence is ~60 s (whatever the device's
+    `device_update_interval` is set to), so a 1 h window gives ~60 raw
+    points. Bucket-mean downsampling preserves shape.
+    """
+    return log_query.telemetry_timeline(
+        window=window,
+        variant=variant,
+        field=field,
+        port=port,
+        max_points=max_points,
+    )
+
+
+@app.tool()
+def packets_window(
+    start: str = "-5m",
+    end: str = "now",
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Recent mesh packets recorded by the recorder.
+
+    Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
+    size + first 64 bytes hex) — full payload bytes are not stored.
+    `portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
+    """
+    return log_query.packets_window(
+        start=start,
+        end=end,
+        portnum=portnum,
+        from_node=from_node,
+        to_node=to_node,
+        max=max,
+    )
+
+
+@app.tool()
+def events_window(
+    start: str = "-1h",
+    end: str = "now",
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
+
+    `kind` ∈ recorder_start, recorder_pause, recorder_resume,
+    connection_established, connection_lost, node_updated, mark.
+    Pipe-separated sets ("connection_lost|connection_established") work.
+    """
+    return log_query.events_window(start=start, end=end, kind=kind, max=max)
+
+
+@app.tool()
+def mark_event(
+    label: str,
+    note: str | None = None,
+    data: dict[str, Any] | None = None,
+) -> dict[str, Any]:
+    """Drop a named marker into events.jsonl AND logs.jsonl.
+
+    Useful for aligning a timeline around a known stimulus: call before
+    and after a stress workload, then query telemetry_timeline /
+    logs_window with the markers' timestamps as bounds.
+
+    The marker also lands in logs.jsonl with level=MARK so a single
+    grep over logs picks it up.
+    """
+    return get_recorder().mark_event(label=label, note=note, data=data)
+
+
+@app.tool()
+def recorder_status() -> dict[str, Any]:
+    """Return recorder runtime info: running, paused, file sizes, last_ts per stream.
+
+    Use this to sanity-check that capture is working before you trust a
+    `logs_window` / `telemetry_timeline` result.
+    """
+    return get_recorder().status()
+
+
+@app.tool()
+def recorder_pause(reason: str | None = None) -> dict[str, Any]:
+    """Pause writes to all four streams. Pubsub subscriptions stay active —
+    we just drop events on the floor while paused. Resume with `recorder_resume`.
+
+    Use when capturing a known-good baseline that you don't want to
+    pollute with pre-test noise. Default state is recording; this is
+    rarely needed.
+    """
+    get_recorder().pause(reason=reason)
+    return {"ok": True, "paused": True, "reason": reason}
+
+
+@app.tool()
+def recorder_resume() -> dict[str, Any]:
+    """Resume writes after `recorder_pause`. No-op if already running."""
+    get_recorder().resume()
+    return {"ok": True, "paused": False}
+
+
+@app.tool()
+def recorder_export(
+    start: str,
+    end: str,
+    dest_dir: str,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of the recorder's streams into `dest_dir`.
+
+    Writes one uncompressed JSONL per requested stream (logs / telemetry /
+    packets / events). Useful for: attaching to a bug report, feeding a
+    notebook, or backfilling Datadog after the fact.
+    """
+    return log_query.export(
+        start=start,
+        end=end,
+        dest_dir=dest_dir,
+        streams=streams,
+    )

mcp-server/tests/unit/test_build_flags.py

@@ -0,0 +1,88 @@
+"""Unit tests for the `build_flags` injection on `flash.build()`.
+
+We don't actually run pio here — too slow, requires hardware-aware envs.
+We test the translation layer (`_build_flags_env`) and that the env vars
+are threaded through pio.run correctly via mock.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import patch
+
+from meshtastic_mcp import flash, pio
+
+
+class TestBuildFlagsEnv:
+    def test_simple_value(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
+
+    def test_string_value(self) -> None:
+        out = flash._build_flags_env({"FOO": "bar"})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
+
+    def test_bool_true_is_bare_flag(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": True})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
+
+    def test_bool_false_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
+
+    def test_none_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": None})
+        assert out == {}
+
+    def test_multiple_combined(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
+        # Order isn't guaranteed in dict iteration, so check membership.
+        flags = out["PLATFORMIO_BUILD_FLAGS"].split()
+        assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
+
+
+class TestBuildPropagatesFlags:
+    def test_extra_env_passed_to_pio_run(self) -> None:
+        # Mock pio.run so we don't actually invoke pio. Capture extra_env.
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["args"] = args
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                out = flash.build(
+                    "fake-env",
+                    with_manifest=False,
+                    build_flags={"DEBUG_HEAP": 1},
+                )
+        assert captured["args"] == ["run", "-e", "fake-env"]
+        assert captured["kwargs"]["extra_env"] == {
+            "PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
+        }
+        assert out["build_flags"] == {"DEBUG_HEAP": 1}
+
+    def test_no_flags_means_no_extra_env(self) -> None:
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                flash.build("fake-env", with_manifest=False)
+        assert captured["kwargs"]["extra_env"] is None

mcp-server/tests/unit/test_recorder.py

@@ -0,0 +1,548 @@
+"""Unit tests for the persistent device-log recorder.
+
+Hardware-free: drives the Recorder through its `_on_*` handlers with
+synthetic packet/line dicts, then queries via log_query. Validates
+prefix parsing, telemetry variant dispatch, marker round-trip, time
+window filtering, downsampling, slope estimation, and gzip rotation
++ archive pruning.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import logging
+import os
+import time
+from pathlib import Path
+
+import pubsub
+import pytest
+from meshtastic_mcp import log_query
+from meshtastic_mcp.recorder.parsers import (
+    extract_telemetry,
+    interface_label,
+    parse_log_line,
+    summarize_packet,
+)
+from meshtastic_mcp.recorder.recorder import Recorder
+from meshtastic_mcp.recorder.rotating import _RotatingJsonl
+
+# -- isolation: every test gets a fresh Recorder + tmp dir -----------
+
+
+@pytest.fixture
+def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
+    # Redirect both the Recorder and the module-level singleton lookup
+    # to the same tmp dir so log_query queries the same files we write.
+    monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
+    monkeypatch.setattr(
+        "meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
+    )
+    r = Recorder(base_dir=tmp_path)
+    r.start()
+    monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
+    yield r
+    r.stop()
+
+
+class _FakeIface:
+    devPath = "/dev/cu.fake"
+
+
+# -- parsers ---------------------------------------------------------
+
+
+class TestParseLogLine:
+    def test_full_prefix(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["uptime_s"] == 12345
+        assert out["msg"] == "Booting"
+        assert out["clock"] == "12:34:56"
+
+    def test_invalid_clock(self) -> None:
+        out = parse_log_line("WARN  | ??:??:?? 7 [SerialConsole] Boot")
+        assert out["level"] == "WARN"
+        assert out["clock"] == "??:??:??"
+        assert out["uptime_s"] == 7
+
+    def test_no_thread_bracket(self) -> None:
+        out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
+        assert out["level"] == "DEBUG"
+        assert out.get("tag") is None
+        assert out["msg"] == "raw message body"
+
+    def test_bare_message(self) -> None:
+        # LogRecord.message path — no level prefix at all.
+        out = parse_log_line("just a bare message")
+        assert "level" not in out or out.get("level") is None
+        assert out["line"] == "just a bare message"
+
+    def test_empty(self) -> None:
+        assert parse_log_line("") == {"line": ""}
+
+    def test_debug_heap_prefix_extracted(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] [heap 92344] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["heap_free"] == 92344
+        assert out["msg"] == "Booting"
+
+    def test_debug_heap_prefix_on_bare_line(self) -> None:
+        # LogRecord.message path: no level prefix but still has [heap N].
+        out = parse_log_line("[heap 12345] some message")
+        assert out["heap_free"] == 12345
+        assert out["msg"] == "some message"
+
+    def test_thread_leak_event(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
+        )
+        assert out["level"] == "HEAP"
+        assert out["heap_free"] == 90000
+        ev = out["heap_event"]
+        assert ev["kind"] == "leaked"
+        assert ev["thread"] == "MeshPacket"
+        assert ev["before"] == 92344
+        assert ev["after"] == 90000
+        assert ev["delta"] == -2344
+
+    def test_thread_freed_event(self) -> None:
+        out = parse_log_line(
+            "++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
+        )
+        ev = out["heap_event"]
+        assert ev["kind"] == "freed"
+        assert ev["thread"] == "Router"
+        assert ev["delta"] == 500
+
+    def test_heap_status_periodic(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
+        )
+        assert out["heap_free"] == 92344
+        assert out["heap_total"] == 200000
+        assert out["heap_delta"] == -128
+
+
+class TestRecorderDebugHeapSynthesis:
+    def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
+        # When a log line carries [heap N], the recorder should also
+        # emit a synthesized telemetry row tagged source=debug_heap.
+        recorder._on_log_line(
+            "INFO  | 00:00:00 1 [Main] [heap 88888] hello",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["variant"] == "local"
+
+    def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert synth[-1]["fields"]["heap_free_bytes"] == 50000
+        assert synth[-1]["fields"]["heap_total_bytes"] == 200000
+
+    def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
+        # Plain log line (no [heap N], no Heap status) — telemetry.jsonl
+        # should NOT gain a synth row.
+        before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        recorder._on_log_line("INFO  | 00:00:00 1 [Main] just a message", _FakeIface())
+        after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        assert after == before
+
+    def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
+            _FakeIface(),
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt_rows = [r for r in rows if r.get("heap_event")]
+        assert len(evt_rows) == 1
+        assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
+        assert evt_rows[0]["heap_event"]["delta"] == -2344
+
+
+class TestSerialTap:
+    def test_serial_line_records_log_and_synthesizes_heap(
+        self, recorder: "Recorder"
+    ) -> None:
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 5 [Main] [heap 88888] tap-line",
+            port="/dev/cu.tap",
+        )
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        log_rows = [json.loads(r) for r in logs if r]
+        # Find the row from this call (port=/dev/cu.tap, role=serial_session)
+        tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
+        assert len(tap_rows) == 1
+        assert tap_rows[0]["role"] == "serial_session"
+        assert tap_rows[0]["level"] == "INFO"
+        assert tap_rows[0]["tag"] == "Main"
+        assert tap_rows[0]["heap_free"] == 88888
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["role"] == "serial_session"
+
+    def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
+        recorder._on_serial_line(
+            "HEAP  | 00:00:30 30 [Power] [heap 53484] "
+            "------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
+            port="/dev/cu.tap",
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt = [r for r in rows if r.get("heap_event")]
+        assert len(evt) == 1
+        assert evt[0]["heap_event"]["thread"] == "Router"
+        assert evt[0]["heap_event"]["delta"] == -128
+        # Heap also synthesized.
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text()
+        assert '"source":"debug_heap_serial"' in telem
+
+    def test_serial_line_pause(self, recorder: "Recorder") -> None:
+        recorder.pause("baseline")
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 1 [t] [heap 1000] dropped",
+            port="/dev/cu.tap",
+        )
+        # Only the pause event row should exist; no tap row.
+        logs = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "dropped" not in logs
+
+    def test_serial_line_handler_swallows_exceptions(
+        self, recorder: "Recorder"
+    ) -> None:
+        # Hostile input — should not raise.
+        recorder._on_serial_line(None, port="/dev/cu.tap")  # type: ignore[arg-type]
+        recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap")  # type: ignore[arg-type]
+        # Survived.
+
+
+class TestExtractTelemetry:
+    def test_local_stats_camel(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {
+                    "localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
+                }
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "local"
+        assert out["fields"]["heap_free_bytes"] == 600
+
+    def test_device_metrics_snake(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "device"
+        assert out["fields"]["battery_level"] == 88
+
+    def test_unknown_variant_returns_none(self) -> None:
+        assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
+        assert extract_telemetry({}) is None
+        assert extract_telemetry({"decoded": "not-a-dict"}) is None
+
+
+class TestSummarizePacket:
+    def test_text_with_payload(self) -> None:
+        pkt = {
+            "fromId": "!abc",
+            "toId": "!def",
+            "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
+            "hopLimit": 3,
+        }
+        out = summarize_packet(pkt)
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] == "TEXT_MESSAGE_APP"
+        assert out["payload_size"] == 5
+        assert out["payload_hex_prefix"] == "68656c6c6f"
+
+    def test_no_decoded(self) -> None:
+        out = summarize_packet({"fromId": "!abc"})
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] is None
+
+
+class TestInterfaceLabel:
+    def test_serial(self) -> None:
+        assert interface_label(_FakeIface()) == {
+            "port": "/dev/cu.fake",
+            "role": "serial",
+        }
+
+    def test_tcp(self) -> None:
+        class T:
+            hostname = "node.lan"
+            portNumber = 4403
+
+        assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
+
+    def test_unknown(self) -> None:
+        assert interface_label(object()) == {"port": "object", "role": None}
+
+    def test_none(self) -> None:
+        assert interface_label(None) == {"port": None, "role": None}
+
+
+# -- recorder write side ---------------------------------------------
+
+
+class TestRecorderWrites:
+    def test_log_line_is_recorded(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:34:56 99 [T] hi", _FakeIface())
+        path = recorder.base_dir / "logs.jsonl"
+        rows = [json.loads(line) for line in path.read_text().splitlines() if line]
+        # First row is recorder_start_event mirror? No — that's events.jsonl only.
+        assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
+
+    def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
+        # _on_telemetry alone — only telemetry.jsonl
+        recorder._on_telemetry(
+            {
+                "fromId": "!abc",
+                "decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
+            },
+            _FakeIface(),
+        )
+        telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        assert any('"variant":"local"' in r for r in telem_rows)
+
+    def test_packets_summary(self, recorder: Recorder) -> None:
+        recorder._on_receive(
+            {
+                "fromId": "!abc",
+                "toId": "!def",
+                "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
+            },
+            _FakeIface(),
+        )
+        rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
+        assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
+
+    def test_mark_event_round_trip(self, recorder: Recorder) -> None:
+        out = recorder.mark_event("checkpoint", note="midpoint")
+        assert "ts" in out
+        events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
+        assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
+
+    def test_pause_drops_writes(self, recorder: Recorder) -> None:
+        before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        recorder.pause(reason="baseline")
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] swallowed", _FakeIface())
+        after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        assert after == before
+        recorder.resume()
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] kept", _FakeIface())
+        post_resume = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "kept" in post_resume
+
+    def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
+        # If the writer dies, the pubsub callback must NOT raise — that
+        # would crash the meshtastic receive thread.
+        bad_packet = object()  # not a dict
+        recorder._on_receive(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_telemetry(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_log_line(None, _FakeIface())  # type: ignore[arg-type]
+        # No assertion needed — survival is the test.
+
+
+# -- log_query read side ---------------------------------------------
+
+
+class TestLogQuery:
+    def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        recorder._on_log_line("WARN  | 12:00:01 2 [B] bravo failed", _FakeIface())
+        recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
+
+        out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
+        assert out["total_matched"] == 2
+        levels = {r["level"] for r in out["lines"]}
+        assert levels == {"WARN", "ERROR"}
+
+        out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
+        assert out2["total_matched"] == 2
+
+    def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        with pytest.raises(ValueError, match="invalid grep regex"):
+            log_query.logs_window(start="-1m", grep="(")
+
+    def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
+        # Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
+        base_ts = time.time() - 60
+        for i in range(100):
+            recorder._files["telemetry"].write(
+                {
+                    "ts": base_ts + i * 0.5,
+                    "port": "/dev/cu.fake",
+                    "role": "serial",
+                    "from_node": "!abc",
+                    "variant": "local",
+                    "fields": {"heap_free_bytes": 10000 - i},
+                }
+            )
+
+        out = log_query.telemetry_timeline(
+            window="2m", variant="local", field="free_heap", max_points=10
+        )
+        assert out["samples"] == 100
+        assert len(out["points"]) <= 10
+        # Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
+        assert out["slope_per_min"] is not None
+        assert out["slope_per_min"] < -100
+
+    def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] one", _FakeIface())
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] two", _FakeIface())
+        dest = tmp_path / "bundle"
+        out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
+        assert (dest / "logs.jsonl").exists()
+        assert "logs" in out["paths"]
+
+
+# -- time parser -----------------------------------------------------
+
+
+class TestParseTime:
+    def test_relative(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("-15m", now=now) == now - 900
+        assert log_query._parse_time("-2h", now=now) == now - 7200
+        assert log_query._parse_time("-1d", now=now) == now - 86400
+
+    def test_now_and_epoch(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("now", now=now) == now
+        assert log_query._parse_time(now) == now
+
+    def test_iso(self) -> None:
+        ts = log_query._parse_time("2026-01-01T00:00:00Z")
+        assert isinstance(ts, float) and ts > 1_700_000_000
+
+    def test_naive_iso_assumes_utc(self) -> None:
+        assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
+            "2026-01-01T00:00:00Z"
+        )
+
+    def test_invalid(self) -> None:
+        with pytest.raises(ValueError):
+            log_query._parse_time("not a time")
+
+
+# -- rotation --------------------------------------------------------
+
+
+class TestRotation:
+    def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
+        for i in range(100):
+            r.write({"ts": float(i), "i": i, "pad": "x" * 40})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives, "expected at least one rotation"
+        # Archive content is valid gzip + valid JSONL
+        with gzip.open(archives[0], "rt") as fh:
+            first = json.loads(fh.readline())
+            assert "ts" in first
+
+    def test_archive_pruning(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
+        # Force several rotations.
+        for _ in range(8):
+            for i in range(20):
+                r.write({"ts": float(i), "pad": "x" * 30})
+            r.force_rotate()
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
+
+    def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, keep_archives=2)
+        old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
+        mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
+        new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
+        for archive in (old, mid, new):
+            with gzip.open(archive, "wt", encoding="utf-8") as fh:
+                fh.write('{"ts":1}\n')
+        # Deliberately scramble mtimes so lexicographic filename order is
+        # the only stable chronological signal.
+        os.utime(old, (300, 300))
+        os.utime(mid, (100, 100))
+        os.utime(new, (200, 200))
+
+        r._prune_archives()
+        r.close()
+
+        archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives == [mid.name, new.name]
+
+    def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
+        r.write({"ts": 1.0, "msg": "tiny"})
+        r.force_rotate()
+        r.write({"ts": 2.0, "msg": "after-rotate"})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) == 1
+        assert path.exists()
+        assert "after-rotate" in path.read_text()
+
+
+class TestRecorderLocks:
+    def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
+        out = recorder.force_rotate_all()
+        assert out["running"] is True
+        assert out["files"]
+
+    def test_wire_pubsub_logs_subscription_failure(
+        self,
+        tmp_path: Path,
+        monkeypatch: pytest.MonkeyPatch,
+        caplog: pytest.LogCaptureFixture,
+    ) -> None:
+        class FailingPubSubMock:
+            def subscribe(self, callback: object, topic: str) -> None:
+                raise RuntimeError("boom")
+
+        monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
+        recorder = Recorder(base_dir=tmp_path)
+        with caplog.at_level(logging.WARNING):
+            recorder._wire_pubsub()
+        assert (
+            "Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
+        )

platformio.ini

@@ -170,8 +170,8 @@ lib_deps =
 	https://github.com/EmotiBit/EmotiBit_MLX90632/archive/refs/tags/v1.0.8.zip
 	# renovate: datasource=github-tags depName=Adafruit MLX90614 packageName=adafruit/Adafruit_MLX90614
 	https://github.com/adafruit/Adafruit-MLX90614-Library/archive/refs/tags/2.1.6.zip
-	# renovate: datasource=git-refs depName=INA3221 packageName=https://github.com/sgtwilko/INA3221 gitBranch=FixOverflow
-	https://github.com/sgtwilko/INA3221/archive/bb03d7e9bfcc74fc798838a54f4f99738f29fc6a.zip
+	# renovate: datasource=github-tags depName=INA3221_RT packageName=RobTillaart/INA3221_RT
+	https://github.com/RobTillaart/INA3221_RT/archive/refs/tags/0.4.2.zip
 	# renovate: datasource=github-tags depName=QMC5883L Compass packageName=mprograms/QMC5883LCompass
 	https://github.com/mprograms/QMC5883LCompass/archive/refs/tags/v1.2.3.zip
 	# renovate: datasource=github-tags depName=DFRobot_RTU packageName=dfrobot/DFRobot_RTU

src/DebugConfiguration.cpp

@@ -26,6 +26,8 @@ SOFTWARE.*/
 
 #include "DebugConfiguration.h"
 
+#include <memory>
+
 #ifdef ARCH_PORTDUINO
 #include "platform/portduino/PortduinoGlue.h"
 #endif
@@ -119,27 +121,22 @@ bool Syslog::vlogf(uint16_t pri, const char *fmt, va_list args)
 
 bool Syslog::vlogf(uint16_t pri, const char *appName, const char *fmt, va_list args)
 {
-    char *message;
-    size_t initialLen;
-    size_t len;
-    bool result;
+    // First measure the formatted length using a copy of args; passing args directly
+    // to vsnprintf consumes it, and reusing a consumed va_list is undefined behavior.
+    va_list args_measure;
+    va_copy(args_measure, args);
+    int needed = vsnprintf(nullptr, 0, fmt, args_measure);
+    va_end(args_measure);
 
-    initialLen = strlen(fmt);
+    if (needed < 0)
+        return false; // encoding error
 
-    message = new char[initialLen + 1];
+    auto message = std::unique_ptr<char[]>(new char[static_cast<size_t>(needed) + 1]);
+    int written = vsnprintf(message.get(), static_cast<size_t>(needed) + 1, fmt, args);
+    if (written < 0)
+        return false;
 
-    len = vsnprintf(message, initialLen + 1, fmt, args);
-    if (len > initialLen) {
-        delete[] message;
-        message = new char[len + 1];
-
-        vsnprintf(message, len + 1, fmt, args);
-    }
-
-    result = this->_sendLog(pri, appName, message);
-
-    delete[] message;
-    return result;
+    return this->_sendLog(pri, appName, message.get());
 }
 
 inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *message)
@@ -154,7 +151,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
     if (!this->_enabled)
         return false;
 
-    if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
+    if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
         return false;
 
     // Check priority against priMask values.

src/DebugConfiguration.h

@@ -13,6 +13,11 @@ extern MemGet memGet;
 #define LED_STATE_ON 1
 #endif
 
+// WIFI LED
+#ifndef WIFI_STATE_ON
+#define WIFI_STATE_ON 1
+#endif
+
 // -----------------------------------------------------------------------------
 // DEBUG
 // -----------------------------------------------------------------------------
@@ -147,7 +152,9 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
 // Default Bluetooth PIN
 #define defaultBLEPin 123456
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && defined(USE_CH390D)
+#include <ESP32_CH390.h>
+#elif HAS_ETHERNET && !defined(USE_WS5500)
 #include <RAK13800_W5100S.h>
 #endif // HAS_ETHERNET
 

src/GPSStatus.h

@@ -53,8 +53,7 @@ class GPSStatus : public Status
     int32_t getLatitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.latitude_i;
+            return localPosition.latitude_i;
         } else {
             return p.latitude_i;
         }
@@ -63,8 +62,7 @@ class GPSStatus : public Status
     int32_t getLongitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.longitude_i;
+            return localPosition.longitude_i;
         } else {
             return p.longitude_i;
         }
@@ -73,8 +71,7 @@ class GPSStatus : public Status
     int32_t getAltitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.altitude;
+            return localPosition.altitude;
         } else {
             return p.altitude;
         }

src/RedirectablePrint.cpp

@@ -7,7 +7,6 @@
 #include "memGet.h"
 #include "mesh/generated/meshtastic/mesh.pb.h"
 #include <assert.h>
-#include <atomic>
 #include <cstring>
 #include <memory>
 #include <stdexcept>
@@ -21,22 +20,6 @@
 #if HAS_NETWORKING
 extern meshtastic::Syslog syslog;
 #endif
-
-namespace
-{
-std::atomic<bool> serialHalLogSuppressed{false};
-}
-
-void RedirectablePrint::setSerialHalLogSuppressed(bool suppressed)
-{
-    serialHalLogSuppressed.store(suppressed);
-}
-
-bool RedirectablePrint::isSerialHalLogSuppressed()
-{
-    return serialHalLogSuppressed.load();
-}
-
 void RedirectablePrint::rpInit()
 {
 #ifdef HAS_FREE_RTOS
@@ -294,10 +277,6 @@ meshtastic_LogRecord_Level RedirectablePrint::getLogLevel(const char *logLevel)
 void RedirectablePrint::log(const char *logLevel, const char *format, ...)
 {
 
-    if (isSerialHalLogSuppressed()) {
-        return;
-    }
-
     // append \n to format
     size_t len = strlen(format);
     auto newFormat = std::unique_ptr<char[]>(new char[len + 2]);

src/RedirectablePrint.h

@@ -24,11 +24,6 @@ class RedirectablePrint : public Print
   public:
     explicit RedirectablePrint(Print *_dest) : dest(_dest) {}
 
-    /// Suppress all log output while a SerialHal transaction is in progress.
-    // Unclear if this is necessary, but it seems to help with response speeds.
-    static void setSerialHalLogSuppressed(bool suppressed);
-    static bool isSerialHalLogSuppressed();
-
     /**
      * Set a new destination
      */

src/concurrency/Lock.cpp

@@ -14,6 +14,11 @@ Lock::Lock() : handle(xSemaphoreCreateBinary())
     }
 }
 
+Lock::~Lock()
+{
+    vSemaphoreDelete(handle);
+}
+
 void Lock::lock()
 {
     if (xSemaphoreTake(handle, portMAX_DELAY) == false) {
@@ -30,6 +35,8 @@ void Lock::unlock()
 #else
 Lock::Lock() {}
 
+Lock::~Lock() {}
+
 void Lock::lock() {}
 
 void Lock::unlock() {}

src/concurrency/Lock.h

@@ -12,6 +12,7 @@ class Lock
 {
   public:
     Lock();
+    ~Lock();
 
     Lock(const Lock &) = delete;
     Lock &operator=(const Lock &) = delete;

src/detect/LoRaRadioType.h

@@ -11,7 +11,8 @@ enum LoRaRadioType {
     SX1280_RADIO,
     LR1110_RADIO,
     LR1120_RADIO,
-    LR1121_RADIO
+    LR1121_RADIO,
+    LR2021_RADIO
 };
 
 extern LoRaRadioType radioType;

src/gps/GPS.cpp

@@ -1025,10 +1025,13 @@ void GPS::up()
     setPowerState(GPS_ACTIVE);
 }
 
-// We've got a GPS lock. Enter a low power state, potentially.
+// We've finished a GPS search cycle (lock or timeout). Enter a low power state, potentially.
 void GPS::down()
 {
-    scheduling.informGotLock();
+    if (hasValidLocation)
+        scheduling.informGotLock();
+    else
+        scheduling.informSearchFailed();
     uint32_t predictedSearchDuration = scheduling.predictedSearchDurationMs();
     uint32_t sleepTime = scheduling.msUntilNextSearch();
     uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval);
@@ -1553,7 +1556,12 @@ std::unique_ptr<GPS> GPS::createGps()
         _en_gpio = PIN_GPS_EN;
 #endif
 #ifdef ARCH_PORTDUINO
-    if (!portduino_config.has_gps)
+    if (portduino_config.has_gps) {
+        // These need to set as flags so later checks will pass on native and GPS will work.
+        // They are not used for any hardware access.
+        _rx_gpio = 1;
+        _tx_gpio = 1;
+    } else
         return nullptr;
 #endif
     if (!_rx_gpio || !_serial_gps) // Configured to have no GPS at all

src/gps/GPSUpdateScheduling.cpp

@@ -15,6 +15,19 @@ void GPSUpdateScheduling::informGotLock()
     searchEndedMs = millis();
     LOG_DEBUG("Took %us to get lock", (searchEndedMs - searchStartedMs) / 1000);
     updateLockTimePrediction();
+    consecutiveFailures = 0; // Drop back to fast cadence as soon as we acquire any fix
+}
+
+// Search finished without obtaining a fix. We still need to mark the end time so
+// the next sleep is timed correctly, but we must not feed the timeout duration
+// into predictedMsToGetLock — doing so poisons msUntilNextSearch() and causes
+// down() to fall into GPS_IDLE, leaving the chip awake on subsequent indoor cycles.
+void GPSUpdateScheduling::informSearchFailed()
+{
+    searchEndedMs = millis();
+    consecutiveFailures++;
+    LOG_DEBUG("GPS search ended without fix after %us (consecutive failures: %u)", (searchEndedMs - searchStartedMs) / 1000,
+              consecutiveFailures);
 }
 
 // Clear old lock-time prediction data.
@@ -25,6 +38,7 @@ void GPSUpdateScheduling::reset()
     searchEndedMs = 0;
     searchCount = 0;
     predictedMsToGetLock = 0;
+    consecutiveFailures = 0;
 }
 
 // How many milliseconds before we should next search for GPS position
@@ -36,6 +50,20 @@ uint32_t GPSUpdateScheduling::msUntilNextSearch()
     // Target interval (seconds), between GPS updates
     uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval, default_gps_update_interval);
 
+    // After a failed search, back off: indoors / no-sky environments will keep failing,
+    // so wake at most once per broadcast interval rather than once per gps_update_interval.
+    // Capped at 1 hour so a user-configured very-long broadcast interval still retries
+    // periodically (in case conditions change). Reset on any successful lock.
+    if (consecutiveFailures > 0) {
+        constexpr uint32_t failureRetryCapMs = 60UL * 60UL * 1000UL; // 1 hour cap
+        uint32_t failureSleepMs =
+            Default::getConfiguredOrDefaultMs(config.position.position_broadcast_secs, default_broadcast_interval_secs);
+        if (failureSleepMs > failureRetryCapMs)
+            failureSleepMs = failureRetryCapMs;
+        if (updateInterval < failureSleepMs)
+            updateInterval = failureSleepMs;
+    }
+
     // Check how long until we should start searching, to hopefully hit our target interval
     uint32_t dueAtMs = searchEndedMs + updateInterval;
     uint32_t compensatedStart = dueAtMs - predictedMsToGetLock;
@@ -71,14 +99,18 @@ bool GPSUpdateScheduling::isUpdateDue()
 bool GPSUpdateScheduling::searchedTooLong()
 {
     constexpr uint32_t oneMinuteMs = 60UL * 1000UL;
-    constexpr uint32_t maxSearchClampMs = 15UL * oneMinuteMs; // Hard cap: 15 minutes is always too long
+    constexpr uint32_t maxSearchClampMs = 15UL * oneMinuteMs;   // Hard cap: 15 minutes is always too long
+    constexpr uint32_t postFailureSearchMs = 5UL * oneMinuteMs; // Tighter dwell once we know the environment is hostile
     uint32_t elapsed = elapsedSearchMs();
 
     // Anything over 15 minutes is too long, regardless of the broadcast interval.
-    // TODO: Make a smarter algorithm that backs off the search dwell time when not getting a lock.
     if (elapsed > maxSearchClampMs)
         return true;
 
+    // After a prior failed search, shorten the dwell
+    if (consecutiveFailures > 0 && elapsed > postFailureSearchMs)
+        return true;
+
     uint32_t minimumOrConfiguredSecs =
         Default::getConfiguredOrMinimumValue(config.position.position_broadcast_secs, default_broadcast_interval_secs);
     uint32_t maxSearchMs = Default::getConfiguredOrDefaultMs(minimumOrConfiguredSecs, default_broadcast_interval_secs);

src/gps/GPSUpdateScheduling.h

@@ -8,7 +8,8 @@ class GPSUpdateScheduling
   public:
     // Marks the time of these events, for calculation use
     void informSearching();
-    void informGotLock(); // Predicted lock-time is recalculated here
+    void informGotLock();      // Predicted lock-time is recalculated here
+    void informSearchFailed(); // Search ended without a fix; prediction is left untouched
 
     void reset();           // Reset the prediction - after GPS::disable() / GPS::enable()
     bool isUpdateDue();     // Is it time to begin searching for a GPS position?
@@ -24,6 +25,7 @@ class GPSUpdateScheduling
     uint32_t searchEndedMs = 0;
     uint32_t searchCount = 0;
     uint32_t predictedMsToGetLock = 0;
+    uint32_t consecutiveFailures = 0; // Count of search cycles that ended without a fix; reset on lock
 
     const float weighting = 0.2; // Controls exponential smoothing of lock-times prediction. 20% weighting of "latest lock-time".
 };

src/graphics/Screen.cpp

@@ -1285,7 +1285,7 @@ void Screen::setFrames(FrameFocus focus)
 
         for (size_t i = 0; i < nodeDB->getNumMeshNodes(); i++) {
             const meshtastic_NodeInfoLite *n = nodeDB->getMeshNodeByIndex(i);
-            if (n && n->num != nodeDB->getNodeNum() && n->is_favorite) {
+            if (n && n->num != nodeDB->getNodeNum() && nodeInfoLiteIsFavorite(n)) {
                 favoriteFrames.push_back(graphics::UIRenderer::drawFavoriteNode);
             }
         }
@@ -1670,7 +1670,7 @@ int Screen::handleTextMessage(const meshtastic_MeshPacket *packet)
             const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet->from);
             const meshtastic_Channel channel =
                 channels.getByIndex(packet->channel ? packet->channel : channels.getPrimaryIndex());
-            const char *longName = (node && node->has_user) ? node->user.long_name : nullptr;
+            const char *longName = nodeInfoLiteHasUser(node) ? node->long_name : nullptr;
 
             const char *msgRaw = reinterpret_cast<const char *>(packet->decoded.payload.bytes);
 

src/graphics/draw/MenuHandler.cpp

@@ -845,10 +845,10 @@ void menuHandler::messageViewModeMenu()
     // Encode peers
     for (size_t i = 0; i < uniquePeers.size(); ++i) {
         uint32_t peer = uniquePeers[i];
-        auto node = nodeDB->getMeshNode(peer);
+        const auto *node = nodeDB->getMeshNode(peer);
         std::string name;
-        if (node && node->has_user)
-            name = sanitizeString(node->user.long_name).substr(0, 15);
+        if (nodeInfoLiteHasUser(node))
+            name = sanitizeString(node->long_name).substr(0, 15);
         else {
             char buf[20];
             snprintf(buf, sizeof(buf), "Node %08X", peer);
@@ -1355,14 +1355,14 @@ void menuHandler::manageNodeMenu()
     static int optionsEnumArray[enumEnd] = {Back};
     int options = 1;
 
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         optionsArray[options] = "Unfavorite";
     } else {
         optionsArray[options] = "Favorite";
     }
     optionsEnumArray[options++] = Favorite;
 
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
     if (isMuted) {
         optionsArray[options] = "Unmute Notifications";
     } else {
@@ -1376,7 +1376,7 @@ void menuHandler::manageNodeMenu()
     optionsArray[options] = "Key Verification";
     optionsEnumArray[options++] = KeyVerification;
 
-    if (node->is_ignored) {
+    if (nodeInfoLiteIsIgnored(node)) {
         optionsArray[options] = "Unignore Node";
     } else {
         optionsArray[options] = "Ignore Node";
@@ -1386,8 +1386,8 @@ void menuHandler::manageNodeMenu()
     BannerOverlayOptions bannerOptions;
 
     std::string title = "";
-    if (node->has_user && node->user.long_name && node->user.long_name[0]) {
-        title += sanitizeString(node->user.long_name).substr(0, 15);
+    if (nodeInfoLiteHasUser(node) && node->long_name[0]) {
+        title += sanitizeString(node->long_name).substr(0, 15);
     } else {
         char buf[20];
         snprintf(buf, sizeof(buf), "%08X", (unsigned int)node->num);
@@ -1409,7 +1409,7 @@ void menuHandler::manageNodeMenu()
             if (!n) {
                 return;
             }
-            if (n->is_favorite) {
+            if (nodeInfoLiteIsFavorite(n)) {
                 LOG_INFO("Removing node %08X from favorites", menuHandler::pickedNodeNum);
                 nodeDB->set_favorite(false, menuHandler::pickedNodeNum);
             } else {
@@ -1426,13 +1426,9 @@ void menuHandler::manageNodeMenu()
                 return;
             }
 
-            if (n->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) {
-                n->bitfield &= ~NODEINFO_BITFIELD_IS_MUTED_MASK;
-                LOG_INFO("Unmuted node %08X", menuHandler::pickedNodeNum);
-            } else {
-                n->bitfield |= NODEINFO_BITFIELD_IS_MUTED_MASK;
-                LOG_INFO("Muted node %08X", menuHandler::pickedNodeNum);
-            }
+            const bool wasMuted = nodeInfoLiteIsMuted(n);
+            nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_MUTED_MASK, !wasMuted);
+            LOG_INFO(wasMuted ? "Unmuted node %08X" : "Muted node %08X", menuHandler::pickedNodeNum);
             nodeDB->notifyObservers(true);
             nodeDB->saveToDisk();
             screen->setFrames(graphics::Screen::FOCUS_PRESERVE);
@@ -1461,11 +1457,11 @@ void menuHandler::manageNodeMenu()
                 return;
             }
 
-            if (n->is_ignored) {
-                n->is_ignored = false;
+            if (nodeInfoLiteIsIgnored(n)) {
+                nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, false);
                 LOG_INFO("Unignoring node %08X", menuHandler::pickedNodeNum);
             } else {
-                n->is_ignored = true;
+                nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
                 LOG_INFO("Ignoring node %08X", menuHandler::pickedNodeNum);
             }
             nodeDB->notifyObservers(true);
@@ -2079,9 +2075,9 @@ void menuHandler::removeFavoriteMenu()
     static const char *optionsArray[] = {"Back", "Yes"};
     BannerOverlayOptions bannerOptions;
     std::string message = "Unfavorite This Node?\n";
-    auto node = nodeDB->getMeshNode(graphics::UIRenderer::currentFavoriteNodeNum);
-    if (node && node->has_user) {
-        message += sanitizeString(node->user.long_name).substr(0, 15);
+    const auto *node = nodeDB->getMeshNode(graphics::UIRenderer::currentFavoriteNodeNum);
+    if (nodeInfoLiteHasUser(node)) {
+        message += sanitizeString(node->long_name).substr(0, 15);
     }
     bannerOptions.message = message.c_str();
     bannerOptions.optionsArrayPtr = optionsArray;

src/graphics/draw/MessageRenderer.cpp

@@ -462,8 +462,8 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
     }
     case ThreadMode::DIRECT: {
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(currentPeer);
-        if (node && node->has_user && node->user.short_name[0]) {
-            snprintf(titleStr, sizeof(titleStr), "@%s", node->user.short_name);
+        if (nodeInfoLiteHasUser(node) && node->short_name[0]) {
+            snprintf(titleStr, sizeof(titleStr), "@%s", node->short_name);
         } else {
             snprintf(titleStr, sizeof(titleStr), "@%08x", currentPeer);
         }
@@ -585,11 +585,11 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
         meshtastic_NodeInfoLite *node_recipient = nodeDB->getMeshNode(m.dest);
 
         char senderName[64] = "";
-        if (node && node->has_user) {
-            if (node->user.long_name[0]) {
-                strncpy(senderName, node->user.long_name, sizeof(senderName) - 1);
-            } else if (node->user.short_name[0]) {
-                strncpy(senderName, node->user.short_name, sizeof(senderName) - 1);
+        if (nodeInfoLiteHasUser(node)) {
+            if (node->long_name[0]) {
+                strncpy(senderName, node->long_name, sizeof(senderName) - 1);
+            } else if (node->short_name[0]) {
+                strncpy(senderName, node->short_name, sizeof(senderName) - 1);
             }
             senderName[sizeof(senderName) - 1] = '\0';
         }
@@ -599,18 +599,17 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
 
         // If this is *our own* message, override senderName to who the recipient was
         bool mine = (m.sender == nodeDB->getNodeNum());
-        if (mine && node_recipient && node_recipient->has_user) {
-            if (node_recipient->user.long_name[0]) {
-                strncpy(senderName, node_recipient->user.long_name, sizeof(senderName) - 1);
+        if (mine && nodeInfoLiteHasUser(node_recipient)) {
+            if (node_recipient->long_name[0]) {
+                strncpy(senderName, node_recipient->long_name, sizeof(senderName) - 1);
                 senderName[sizeof(senderName) - 1] = '\0';
-            } else if (node_recipient->user.short_name[0]) {
-                strncpy(senderName, node_recipient->user.short_name, sizeof(senderName) - 1);
+            } else if (node_recipient->short_name[0]) {
+                strncpy(senderName, node_recipient->short_name, sizeof(senderName) - 1);
                 senderName[sizeof(senderName) - 1] = '\0';
             }
         }
         // If recipient info is missing/empty, prefer a recipient identifier for outbound messages.
-        if (mine && (!node_recipient || !node_recipient->has_user ||
-                     (!node_recipient->user.long_name[0] && !node_recipient->user.short_name[0]))) {
+        if (mine && (!nodeInfoLiteHasUser(node_recipient) || (!node_recipient->long_name[0] && !node_recipient->short_name[0]))) {
             snprintf(senderName, sizeof(senderName), "(%08x)", m.dest);
         }
 
@@ -1073,12 +1072,12 @@ void handleNewMessage(OLEDDisplay *display, const StoredMessage &sm, const mesht
         // Banner logic
         const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet.from);
         char longName[64] = "?";
-        if (node && node->has_user) {
-            if (node->user.long_name[0]) {
-                strncpy(longName, node->user.long_name, sizeof(longName) - 1);
+        if (nodeInfoLiteHasUser(node)) {
+            if (node->long_name[0]) {
+                strncpy(longName, node->long_name, sizeof(longName) - 1);
                 longName[sizeof(longName) - 1] = '\0';
-            } else if (node->user.short_name[0]) {
-                strncpy(longName, node->user.short_name, sizeof(longName) - 1);
+            } else if (node->short_name[0]) {
+                strncpy(longName, node->short_name, sizeof(longName) - 1);
                 longName[sizeof(longName) - 1] = '\0';
             }
         }

src/graphics/draw/NodeListRenderer.cpp

@@ -98,29 +98,23 @@ std::string getSafeNodeName(OLEDDisplay *display, meshtastic_NodeInfoLite *node,
     // 1) Choose target candidate (long vs short) only if present
     const char *raw = nullptr;
 
-#if !MESHTASTIC_EXCLUDE_STATUS
+#if !MESHTASTIC_EXCLUDE_STATUS && !MESHTASTIC_EXCLUDE_STATUSDB
     // If long-name mode is enabled, and we have a recent status for this node,
-    // prefer "(short_name) statusText" as the raw candidate.
+    // prefer "(short_name) statusText" as the raw candidate. Pull straight out
+    // of NodeDB's per-NodeNum cache instead of scanning a FIFO.
     std::string composedFromStatus;
-    if (config.display.use_long_node_name && node && node->has_user && statusMessageModule) {
-        const auto &recent = statusMessageModule->getRecentReceived();
-        const StatusMessageModule::RecentStatus *found = nullptr;
-        for (auto it = recent.rbegin(); it != recent.rend(); ++it) {
-            if (it->fromNodeId == node->num && !it->statusText.empty()) {
-                found = &(*it);
-                break;
-            }
-        }
-
-        if (found) {
-            const char *shortName = node->user.short_name;
-            composedFromStatus.reserve(4 + (shortName ? std::strlen(shortName) : 0) + 1 + found->statusText.size());
+    if (config.display.use_long_node_name && nodeInfoLiteHasUser(node) && nodeDB) {
+        meshtastic_StatusMessage cachedStatus;
+        if (nodeDB->copyNodeStatus(node->num, cachedStatus) && cachedStatus.status[0]) {
+            const char *shortName = node->short_name;
+            const size_t statusLen = std::strlen(cachedStatus.status);
+            composedFromStatus.reserve(4 + (shortName ? std::strlen(shortName) : 0) + 1 + statusLen);
             composedFromStatus += "(";
             if (shortName && *shortName) {
                 composedFromStatus += shortName;
             }
             composedFromStatus += ") ";
-            composedFromStatus += found->statusText;
+            composedFromStatus += cachedStatus.status;
 
             raw = composedFromStatus.c_str(); // safe for now; we'll sanitize immediately into std::string
         }
@@ -129,8 +123,8 @@ std::string getSafeNodeName(OLEDDisplay *display, meshtastic_NodeInfoLite *node,
 
     // If we didn't compose from status, use normal long/short selection
     if (!raw) {
-        if (node && node->has_user) {
-            raw = config.display.use_long_node_name ? node->user.long_name : node->user.short_name;
+        if (nodeInfoLiteHasUser(node)) {
+            raw = config.display.use_long_node_name ? node->long_name : node->short_name;
         }
     }
 
@@ -218,7 +212,7 @@ void drawScrollbar(OLEDDisplay *display, int visibleNodeRows, int totalEntries,
 static inline void applyFavoriteNodeNameColor(OLEDDisplay *display, const meshtastic_NodeInfoLite *node, const char *nodeName,
                                               int16_t nameX, int16_t y, int nameMaxWidth)
 {
-    if (!display || !node || !node->is_favorite || !isTFTColoringEnabled() || !nodeName) {
+    if (!display || !node || !nodeInfoLiteIsFavorite(node) || !isTFTColoringEnabled() || !nodeName) {
         return;
     }
 
@@ -259,7 +253,7 @@ void drawEntryLastHeard(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     char timeStr[10];
     uint32_t seconds = sinceLastSeen(node);
@@ -279,14 +273,14 @@ void drawEntryLastHeard(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -321,20 +315,20 @@ void drawEntryHopSignal(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
 
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -401,15 +395,19 @@ void drawNodeDistance(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
     char distStr[10] = "";
 
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node)) {
-        double lat1 = ourNode->position.latitude_i * 1e-7;
-        double lon1 = ourNode->position.longitude_i * 1e-7;
-        double lat2 = node->position.latitude_i * 1e-7;
-        double lon2 = node->position.longitude_i * 1e-7;
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourPos;
+    meshtastic_PositionLite theirPos;
+    const bool haveOurPos = ourNode && nodeDB->copyNodePosition(ourNode->num, ourPos);
+    const bool haveTheirPos = nodeDB->copyNodePosition(node->num, theirPos);
+    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node) && haveOurPos && haveTheirPos) {
+        double lat1 = ourPos.latitude_i * 1e-7;
+        double lon1 = ourPos.longitude_i * 1e-7;
+        double lat2 = theirPos.latitude_i * 1e-7;
+        double lon2 = theirPos.longitude_i * 1e-7;
 
         double earthRadiusKm = 6371.0;
         double dLat = (lat2 - lat1) * DEG_TO_RAD;
@@ -455,14 +453,14 @@ void drawNodeDistance(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -509,19 +507,19 @@ void drawEntryCompass(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -542,8 +540,11 @@ void drawCompassArrow(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
     int centerX = x + columnWidth - arrowXOffset;
     int centerY = y + FONT_HEIGHT_SMALL / 2;
 
-    double nodeLat = node->position.latitude_i * 1e-7;
-    double nodeLon = node->position.longitude_i * 1e-7;
+    meshtastic_PositionLite nodePos;
+    if (!nodeDB->copyNodePosition(node->num, nodePos))
+        return;
+    double nodeLat = nodePos.latitude_i * 1e-7;
+    double nodeLon = nodePos.longitude_i * 1e-7;
     float bearing = GeoCoord::bearing(userLat, userLon, nodeLat, nodeLon);
     float relativeBearing = CompassRenderer::adjustBearingForCompassMode(bearing, myHeadingRadian);
     float relativeBearingDeg = CompassRenderer::radiansToDegrees360(relativeBearing);
@@ -652,7 +653,7 @@ void drawNodeListScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t
             continue;
         if (n->num == nodeDB->getNodeNum())
             continue;
-        if (locationScreen && !n->has_position)
+        if (locationScreen && !nodeDB->hasNodePosition(n->num))
             continue;
 
         drawList.push_back(n->num);
@@ -883,14 +884,15 @@ void drawDistanceScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t
 void drawNodeListWithCompasses(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
 {
     float headingRadian = 0.0f;
-    auto ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (!ourNode || !nodeDB->hasValidPosition(ourNode)) {
+    const auto *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourSelfPos;
+    if (!ourNode || !nodeDB->hasValidPosition(ourNode) || !nodeDB->copyNodePosition(ourNode->num, ourSelfPos)) {
         drawNodeListScreen(display, state, x, y, "Bearings", drawEntryCompass, drawCompassUnknown, headingRadian, 0.0, 0.0);
         return;
     }
 
-    double lat = DegD(ourNode->position.latitude_i);
-    double lon = DegD(ourNode->position.longitude_i);
+    double lat = DegD(ourSelfPos.latitude_i);
+    double lon = DegD(ourSelfPos.longitude_i);
 
 #if defined(M5STACK_UNITC6L)
     display->clear();

src/graphics/draw/NotificationRenderer.cpp

@@ -317,12 +317,12 @@ void NotificationRenderer::drawNodePicker(OLEDDisplay *display, OLEDDisplayUiSta
     for (int i = firstOptionToShow; i < alertBannerOptions && linesShown < visibleTotalLines; i++, linesShown++) {
         char tempName[48] = {0};
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNodeByIndex(i + 1);
-        if (node && node->has_user) {
+        if (nodeInfoLiteHasUser(node)) {
             const char *rawName = nullptr;
-            if (node->user.long_name[0]) {
-                rawName = node->user.long_name;
-            } else if (node->user.short_name[0]) {
-                rawName = node->user.short_name;
+            if (node->long_name[0]) {
+                rawName = node->long_name;
+            } else if (node->short_name[0]) {
+                rawName = node->short_name;
             }
             if (rawName) {
                 const int arrowWidth = (currentResolution == ScreenResolution::High)

src/graphics/draw/UIRenderer.cpp

@@ -464,11 +464,11 @@ static bool computeBottomCompassPlacement(OLEDDisplay *display, int16_t xOffset,
     return true;
 }
 
-static void drawTruncatedStatusLine(OLEDDisplay *display, int16_t x, int16_t y, const std::string &statusText)
+static void drawTruncatedStatusLine(OLEDDisplay *display, int16_t x, int16_t y, const char *statusText)
 {
     // Fixed-buffer truncate helper replaces iterative std::string chopping to keep code size down.
     char rawStatus[96];
-    snprintf(rawStatus, sizeof(rawStatus), " Status: %s", statusText.c_str());
+    snprintf(rawStatus, sizeof(rawStatus), " Status: %s", statusText ? statusText : "");
 
     char clippedStatus[96];
     UIRenderer::truncateStringWithEmotes(display, rawStatus, clippedStatus, sizeof(clippedStatus), display->getWidth());
@@ -495,7 +495,7 @@ void graphics::UIRenderer::rebuildFavoritedNodes()
         meshtastic_NodeInfoLite *n = nodeDB->getMeshNodeByIndex(i);
         if (!n || n->num == nodeDB->getNodeNum())
             continue;
-        if (n->is_favorite)
+        if (nodeInfoLiteIsFavorite(n))
             favoritedNodes.push_back(n);
     }
 
@@ -750,7 +750,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
         return;
 
     meshtastic_NodeInfoLite *node = favoritedNodes[nodeIndex];
-    if (!node || node->num == nodeDB->getNodeNum() || !node->is_favorite)
+    if (!node || node->num == nodeDB->getNodeNum() || !nodeInfoLiteIsFavorite(node))
         return;
     display->clear();
 #if defined(M5STACK_UNITC6L)
@@ -763,7 +763,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
 #endif
     currentFavoriteNodeNum = node->num;
     // === Create the shortName and title string ===
-    const char *shortName = (node->has_user && node->user.short_name[0]) ? node->user.short_name : "Node";
+    const char *shortName = (nodeInfoLiteHasUser(node) && node->short_name[0]) ? node->short_name : "Node";
     char titlestr[40];
     snprintf(titlestr, sizeof(titlestr), "*%s*", shortName);
 
@@ -781,9 +781,9 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     // === 1. Long Name (always try to show first) ===
     const char *username;
     if (currentResolution == ScreenResolution::UltraLow) {
-        username = (node->has_user && node->user.long_name[0]) ? node->user.short_name : nullptr;
+        username = (nodeInfoLiteHasUser(node) && node->long_name[0]) ? node->short_name : nullptr;
     } else {
-        username = (node->has_user && node->user.long_name[0]) ? node->user.long_name : nullptr;
+        username = (nodeInfoLiteHasUser(node) && node->long_name[0]) ? node->long_name : nullptr;
     }
 
     // Print node's long name (e.g. "Backpack Node")
@@ -796,23 +796,14 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
         UIRenderer::drawStringWithEmotes(display, x, getTextPositions(display)[line++], username, FONT_HEIGHT_SMALL, 1, false);
     }
 
-#if !MESHTASTIC_EXCLUDE_STATUS
+#if !MESHTASTIC_EXCLUDE_STATUS && !MESHTASTIC_EXCLUDE_STATUSDB
     // === Optional: Last received StatusMessage line for this node ===
-    // Display it directly under the username line (if we have one).
-    if (statusMessageModule) {
-        const auto &recent = statusMessageModule->getRecentReceived();
-        const StatusMessageModule::RecentStatus *found = nullptr;
-
-        // Search newest-to-oldest
-        for (auto it = recent.rbegin(); it != recent.rend(); ++it) {
-            if (it->fromNodeId == node->num && !it->statusText.empty()) {
-                found = &(*it);
-                break;
-            }
-        }
-
-        if (found) {
-            drawTruncatedStatusLine(display, x, getTextPositions(display)[line++], found->statusText);
+    // Display it directly under the username line (if we have one). The cache
+    // lives on NodeDB now, keyed by NodeNum, so this is an O(1) lookup.
+    if (nodeDB) {
+        meshtastic_StatusMessage cachedStatus;
+        if (nodeDB->copyNodeStatus(node->num, cachedStatus) && cachedStatus.status[0]) {
+            drawTruncatedStatusLine(display, x, getTextPositions(display)[line++], cachedStatus.status);
         }
     }
 #endif
@@ -971,25 +962,30 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
 #if !defined(M5STACK_UNITC6L)
     // === 4. Uptime (only show if metric is present) ===
     char uptimeStr[32] = "";
-    if (node->has_device_metrics && node->device_metrics.has_uptime_seconds) {
+    meshtastic_DeviceMetrics nodeMetrics;
+    const bool haveNodeMetrics = nodeDB->copyNodeTelemetry(node->num, nodeMetrics);
+    if (haveNodeMetrics && nodeMetrics.has_uptime_seconds) {
         char upPrefix[12]; // enough for leftSideSpacing + "Up:"
         snprintf(upPrefix, sizeof(upPrefix), "%sUp:", leftSideSpacing);
-        getUptimeStr(node->device_metrics.uptime_seconds * 1000, upPrefix, uptimeStr, sizeof(uptimeStr));
+        getUptimeStr(nodeMetrics.uptime_seconds * 1000, upPrefix, uptimeStr, sizeof(uptimeStr));
     }
     if (uptimeStr[0]) {
         display->drawString(x, getTextPositions(display)[line++], uptimeStr);
     }
 
     // === 5. Distance (only if both nodes have GPS position) ===
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
     char distStr[24] = ""; // Make buffer big enough for any string
     bool haveDistance = false;
 
-    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node)) {
+    meshtastic_PositionLite nodePos;
+    meshtastic_PositionLite ourPos;
+    const bool haveNodePos = nodeDB->copyNodePosition(node->num, nodePos);
+    const bool haveOurPos = ourNode && nodeDB->copyNodePosition(ourNode->num, ourPos);
+    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node) && haveNodePos && haveOurPos) {
         // Use shared meter conversion, then format display units with lightweight integer rounding.
-        const float distanceMeters =
-            GeoCoord::latLongToMeter(DegD(node->position.latitude_i), DegD(node->position.longitude_i),
-                                     DegD(ourNode->position.latitude_i), DegD(ourNode->position.longitude_i));
+        const float distanceMeters = GeoCoord::latLongToMeter(DegD(nodePos.latitude_i), DegD(nodePos.longitude_i),
+                                                              DegD(ourPos.latitude_i), DegD(ourPos.longitude_i));
         if (config.display.units == meshtastic_Config_DisplayConfig_DisplayUnits_IMPERIAL) {
             const int feet = static_cast<int>((distanceMeters * METERS_TO_FEET) + 0.5f);
             if (feet > 0 && feet < 1000) {
@@ -1024,19 +1020,19 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     char batLine[32] = "";
     bool haveBatLine = false;
 
-    if (node->has_device_metrics) {
-        bool hasPct = node->device_metrics.has_battery_level;
-        bool hasVolt = node->device_metrics.has_voltage && node->device_metrics.voltage > 0.001f;
+    if (haveNodeMetrics) {
+        bool hasPct = nodeMetrics.has_battery_level;
+        bool hasVolt = nodeMetrics.has_voltage && nodeMetrics.voltage > 0.001f;
 
         int pct = 0;
         float volt = 0.0f;
 
         if (hasPct) {
-            pct = (int)node->device_metrics.battery_level;
+            pct = (int)nodeMetrics.battery_level;
         }
 
         if (hasVolt) {
-            volt = node->device_metrics.voltage;
+            volt = nodeMetrics.voltage;
         }
 
         if (hasPct && pct > 0 && pct <= 100) {
@@ -1076,11 +1072,11 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     const bool hasNodePositionFix = nodeDB->hasValidPosition(node);
     const char *statusLine1 = nullptr;
     const char *statusLine2 = nullptr;
-    if (hasOwnPositionFix && hasNodePositionFix) {
-        const auto &op = ourNode->position;
+    if (hasOwnPositionFix && hasNodePositionFix && haveOurPos && haveNodePos) {
+        const auto &op = ourPos;
         showCompass = CompassRenderer::getHeadingRadians(DegD(op.latitude_i), DegD(op.longitude_i), myHeading);
         if (showCompass) {
-            const auto &p = node->position;
+            const auto &p = nodePos;
             bearing = GeoCoord::bearing(DegD(op.latitude_i), DegD(op.longitude_i), DegD(p.latitude_i), DegD(p.longitude_i));
             bearing = CompassRenderer::adjustBearingForCompassMode(bearing, myHeading);
         } else {
@@ -1130,7 +1126,7 @@ void UIRenderer::drawDeviceFocused(OLEDDisplay *display, OLEDDisplayUiState *sta
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     int line = 1;
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
 
     // === Header ===
     if (currentResolution == ScreenResolution::UltraLow) {
@@ -1270,7 +1266,7 @@ void UIRenderer::drawDeviceFocused(OLEDDisplay *display, OLEDDisplayUiState *sta
     int textWidth = 0;
     int nameX = 0;
     int yOffset = (currentResolution == ScreenResolution::High) ? 0 : 5;
-    const char *longName = (ourNode && ourNode->has_user && ourNode->user.long_name[0]) ? ourNode->user.long_name : "";
+    const char *longName = (nodeInfoLiteHasUser(ourNode) && ourNode->long_name[0]) ? ourNode->long_name : "";
     const char *shortName = owner.short_name ? owner.short_name : "";
     char combinedName[96];
     if (longName[0] && shortName[0]) {
@@ -1597,7 +1593,7 @@ void UIRenderer::drawCompassAndLocationScreen(OLEDDisplay *display, OLEDDisplayU
     geoCoord.updateCoords(int32_t(gpsStatus->getLatitude()), int32_t(gpsStatus->getLongitude()),
                           int32_t(gpsStatus->getAltitude()));
 
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
     const bool hasOwnPositionFix = (ourNode && nodeDB->hasValidPosition(ourNode));
     const bool hasLiveGpsFix =
         (gpsStatus && gpsStatus->getHasLock() && (gpsStatus->getLatitude() != 0 || gpsStatus->getLongitude() != 0));
@@ -1613,9 +1609,11 @@ void UIRenderer::drawCompassAndLocationScreen(OLEDDisplay *display, OLEDDisplayU
             headingLat = DegD(gpsStatus->getLatitude());
             headingLon = DegD(gpsStatus->getLongitude());
         } else if (hasOwnPositionFix) {
-            const auto &op = ourNode->position;
-            headingLat = DegD(op.latitude_i);
-            headingLon = DegD(op.longitude_i);
+            meshtastic_PositionLite ownPos;
+            if (nodeDB->copyNodePosition(ourNode->num, ownPos)) {
+                headingLat = DegD(ownPos.latitude_i);
+                headingLon = DegD(ownPos.longitude_i);
+            }
         }
         validHeading = CompassRenderer::getHeadingRadians(headingLat, headingLon, heading);
     }

src/graphics/niche/InkHUD/Applet.cpp

@@ -362,8 +362,8 @@ std::string InkHUD::Applet::parseShortName(meshtastic_NodeInfoLite *node)
     assert(node);
 
     // Use the true shortname if known, and doesn't contain any unprintable characters (emoji, etc.)
-    if (node->has_user) {
-        std::string parsed = parse(node->user.short_name);
+    if (nodeInfoLiteHasUser(node)) {
+        std::string parsed = parse(node->short_name);
         if (isPrintable(parsed))
             return parsed;
     }

src/graphics/niche/InkHUD/Applets/Bases/Map/MapApplet.cpp

@@ -136,9 +136,10 @@ void InkHUD::MapApplet::onRender(bool full)
     printAt(vertBarX + (bottomLabelW / 2) + 1, bottomLabelY + (bottomLabelH / 2), vertBottomLabel, CENTER, MIDDLE);
 
     // Draw our node LAST with full white fill + outline
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (ourNode && nodeDB->hasValidPosition(ourNode)) {
-        Marker self = calculateMarker(ourNode->position.latitude_i * 1e-7, ourNode->position.longitude_i * 1e-7, false, 0);
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourSelfPos;
+    if (ourNode && nodeDB->hasValidPosition(ourNode) && nodeDB->copyNodePosition(ourNode->num, ourSelfPos)) {
+        Marker self = calculateMarker(ourSelfPos.latitude_i * 1e-7, ourSelfPos.longitude_i * 1e-7, false, 0);
 
         int16_t centerX = X(0.5) + (self.eastMeters * metersToPx);
         int16_t centerY = Y(0.5) - (self.northMeters * metersToPx);
@@ -168,10 +169,11 @@ void InkHUD::MapApplet::onRender(bool full)
 void InkHUD::MapApplet::getMapCenter(float *lat, float *lng)
 {
     // If we have a valid position for our own node, use that as the anchor
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (ourNode && nodeDB->hasValidPosition(ourNode)) {
-        *lat = ourNode->position.latitude_i * 1e-7;
-        *lng = ourNode->position.longitude_i * 1e-7;
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourSelfPos;
+    if (ourNode && nodeDB->hasValidPosition(ourNode) && nodeDB->copyNodePosition(ourNode->num, ourSelfPos)) {
+        *lat = ourSelfPos.latitude_i * 1e-7;
+        *lng = ourSelfPos.longitude_i * 1e-7;
     } else {
         // Find mean lat long coords
         // ============================
@@ -201,9 +203,13 @@ void InkHUD::MapApplet::getMapCenter(float *lat, float *lng)
             if (!shouldDrawNode(node))
                 continue;
 
+            meshtastic_PositionLite pos;
+            if (!nodeDB->copyNodePosition(node->num, pos))
+                continue;
+
             // Latitude and Longitude of node, in radians
-            float latRad = node->position.latitude_i * (1e-7) * DEG_TO_RAD;
-            float lngRad = node->position.longitude_i * (1e-7) * DEG_TO_RAD;
+            float latRad = pos.latitude_i * (1e-7) * DEG_TO_RAD;
+            float lngRad = pos.longitude_i * (1e-7) * DEG_TO_RAD;
 
             // Convert to cartesian points, with center of earth at 0, 0, 0
             // Exact distance from center is irrelevant, as we're only interested in the vector
@@ -300,13 +306,17 @@ void InkHUD::MapApplet::getMapCenter(float *lat, float *lng)
         if (!shouldDrawNode(node))
             continue;
 
+        meshtastic_PositionLite pos;
+        if (!nodeDB->copyNodePosition(node->num, pos))
+            continue;
+
         // Check for a new top or bottom latitude
-        float latNode = node->position.latitude_i * 1e-7;
+        float latNode = pos.latitude_i * 1e-7;
         northernmost = max(northernmost, latNode);
         southernmost = min(southernmost, latNode);
 
         // Longitude is trickier
-        float lngNode = node->position.longitude_i * 1e-7;
+        float lngNode = pos.longitude_i * 1e-7;
         float degEastward = fmod(((lngNode - lngCenter) + 360), 360);      // Degrees traveled east from lngCenter to reach node
         float degWestward = abs(fmod(((lngNode - lngCenter) - 360), 360)); // Degrees traveled west from lngCenter to reach node
         if (degEastward < degWestward)
@@ -372,10 +382,13 @@ void InkHUD::MapApplet::drawLabeledMarker(meshtastic_NodeInfoLite *node)
 {
     // Find x and y position based on node's position in nodeDB
     assert(nodeDB->hasValidPosition(node));
-    Marker m = calculateMarker(node->position.latitude_i * 1e-7,  // Lat, converted from Meshtastic's internal int32 style
-                               node->position.longitude_i * 1e-7, // Long, converted from Meshtastic's internal int32 style
-                               node->has_hops_away,               // Is the hopsAway number valid
-                               node->hops_away                    // Hops away
+    meshtastic_PositionLite pos;
+    const bool hasPos = nodeDB->copyNodePosition(node->num, pos);
+    assert(hasPos);
+    Marker m = calculateMarker(pos.latitude_i * 1e-7,  // Lat, converted from Meshtastic's internal int32 style
+                               pos.longitude_i * 1e-7, // Long, converted from Meshtastic's internal int32 style
+                               node->has_hops_away,    // Is the hopsAway number valid
+                               node->hops_away         // Hops away
     );
 
     // Convert to pixel coords
@@ -516,13 +529,16 @@ void InkHUD::MapApplet::calculateAllMarkers()
         if (node->num == nodeDB->getNodeNum())
             continue;
 
+        meshtastic_PositionLite pos;
+        if (!nodeDB->copyNodePosition(node->num, pos))
+            continue;
+
         // Calculate marker and store it
-        markers.push_back(
-            calculateMarker(node->position.latitude_i * 1e-7,  // Lat, converted from Meshtastic's internal int32 style
-                            node->position.longitude_i * 1e-7, // Long, converted from Meshtastic's internal int32 style
-                            node->has_hops_away,               // Is the hopsAway number valid
-                            node->hops_away                    // Hops away
-                            ));
+        markers.push_back(calculateMarker(pos.latitude_i * 1e-7,  // Lat, converted from Meshtastic's internal int32 style
+                                          pos.longitude_i * 1e-7, // Long, converted from Meshtastic's internal int32 style
+                                          node->has_hops_away,    // Is the hopsAway number valid
+                                          node->hops_away         // Hops away
+                                          ));
     }
 }
 

src/graphics/niche/InkHUD/Applets/Bases/NodeList/NodeListApplet.cpp

@@ -50,21 +50,23 @@ ProcessMessage InkHUD::NodeListApplet::handleReceived(const meshtastic_MeshPacke
     c.signal = getSignalStrength(mp.rx_snr, mp.rx_rssi);
 
     // Assemble info: from nodeDB (needed to detect changes)
-    meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(c.nodeNum);
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(c.nodeNum);
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
     if (node) {
         if (node->has_hops_away)
             c.hopsAway = node->hops_away;
 
         if (nodeDB->hasValidPosition(node) && nodeDB->hasValidPosition(ourNode)) {
-            // Get lat and long as float
-            // Meshtastic stores these as integers internally
-            float ourLat = ourNode->position.latitude_i * 1e-7;
-            float ourLong = ourNode->position.longitude_i * 1e-7;
-            float theirLat = node->position.latitude_i * 1e-7;
-            float theirLong = node->position.longitude_i * 1e-7;
+            meshtastic_PositionLite ourPos;
+            meshtastic_PositionLite theirPos;
+            if (nodeDB->copyNodePosition(ourNode->num, ourPos) && nodeDB->copyNodePosition(node->num, theirPos)) {
+                float ourLat = ourPos.latitude_i * 1e-7;
+                float ourLong = ourPos.longitude_i * 1e-7;
+                float theirLat = theirPos.latitude_i * 1e-7;
+                float theirLong = theirPos.longitude_i * 1e-7;
 
-            c.distanceMeters = (int32_t)GeoCoord::latLongToMeter(theirLat, theirLong, ourLat, ourLong);
+                c.distanceMeters = (int32_t)GeoCoord::latLongToMeter(theirLat, theirLong, ourLat, ourLong);
+            }
         }
     }
 
@@ -179,8 +181,8 @@ void InkHUD::NodeListApplet::onRender(bool full)
         // -- Longname --
         // Parse special chars in long name
         // Use node id if unknown
-        if (node && node->has_user)
-            longName = parse(node->user.long_name); // Found in nodeDB
+        if (nodeInfoLiteHasUser(node))
+            longName = parse(node->long_name); // Found in nodeDB
         else {
             // Not found in nodeDB, show a hex nodeid instead
             longName = hexifyNodeNum(nodeNum);

src/graphics/niche/InkHUD/Applets/System/Menu/MenuApplet.cpp

@@ -2090,7 +2090,7 @@ void InkHUD::MenuApplet::populateRecipientPage()
 
     // Count favorites
     for (uint32_t i = 0; i < nodeCount; i++) {
-        if (nodeDB->getMeshNodeByIndex(i)->is_favorite)
+        if (nodeInfoLiteIsFavorite(nodeDB->getMeshNodeByIndex(i)))
             favoriteCount++;
     }
 
@@ -2098,10 +2098,10 @@ void InkHUD::MenuApplet::populateRecipientPage()
     // Don't want some monstrous list that takes 100 clicks to reach exit
     if (favoriteCount < 20) {
         for (uint32_t i = 0; i < nodeCount; i++) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNodeByIndex(i);
+            const meshtastic_NodeInfoLite *node = nodeDB->getMeshNodeByIndex(i);
 
             // Skip node if not a favorite
-            if (!node->is_favorite)
+            if (!nodeInfoLiteIsFavorite(node))
                 continue;
 
             CannedMessages::RecipientItem r;
@@ -2111,8 +2111,8 @@ void InkHUD::MenuApplet::populateRecipientPage()
 
             // Set a label for the menu item
             r.label = "DM: ";
-            if (node->has_user)
-                r.label += parse(node->user.long_name);
+            if (nodeInfoLiteHasUser(node))
+                r.label += parse(node->long_name);
             else
                 r.label += hexifyNodeNum(node->num); // Unsure if it's possible to favorite a node without NodeInfo?
 

src/graphics/niche/InkHUD/Applets/System/Notification/NotificationApplet.cpp

@@ -241,7 +241,7 @@ std::string InkHUD::NotificationApplet::getNotificationText(uint16_t widthAvaila
         text += msgIsBroadcast ? "From:" : "DM: ";
 
         // Sender id
-        if (node && node->has_user)
+        if (nodeInfoLiteHasUser(node))
             text += parseShortName(node);
         else
             text += hexifyNodeNum(message->sender);
@@ -255,7 +255,7 @@ std::string InkHUD::NotificationApplet::getNotificationText(uint16_t widthAvaila
             text += msgIsBroadcast ? "Msg from " : "DM from ";
 
             // Sender id
-            if (node && node->has_user)
+            if (nodeInfoLiteHasUser(node))
                 text += parseShortName(node);
             else
                 text += hexifyNodeNum(message->sender);

src/graphics/niche/InkHUD/Applets/User/AllMessage/AllMessageApplet.cpp

@@ -70,10 +70,10 @@ void InkHUD::AllMessageApplet::onRender(bool full)
     // - short name and long name, if available, or
     // - node id
     meshtastic_NodeInfoLite *sender = nodeDB->getMeshNode(message->sender);
-    if (sender && sender->has_user) {
+    if (nodeInfoLiteHasUser(sender)) {
         header += parseShortName(sender); // May be last-four of node if unprintable (emoji, etc)
         header += " (";
-        header += parse(sender->user.long_name);
+        header += parse(sender->long_name);
         header += ")";
     } else
         header += hexifyNodeNum(message->sender);

src/graphics/niche/InkHUD/Applets/User/DM/DMApplet.cpp

@@ -66,10 +66,10 @@ void InkHUD::DMApplet::onRender(bool full)
     // - shortname and long name, if available, or
     // - node id
     meshtastic_NodeInfoLite *sender = nodeDB->getMeshNode(latestMessage->dm.sender);
-    if (sender && sender->has_user) {
+    if (nodeInfoLiteHasUser(sender)) {
         header += parseShortName(sender); // May be last-four of node if unprintable (emoji, etc)
         header += " (";
-        header += parse(sender->user.long_name);
+        header += parse(sender->long_name);
         header += ")";
     } else
         header += hexifyNodeNum(latestMessage->dm.sender);

src/graphics/niche/InkHUD/Applets/User/FavoritesMap/FavoritesMapApplet.cpp

@@ -8,7 +8,7 @@ using namespace NicheGraphics;
 bool InkHUD::FavoritesMapApplet::shouldDrawNode(meshtastic_NodeInfoLite *node)
 {
     // Keep our own node available as map anchor/center; all others must be favorited.
-    return node && (node->num == nodeDB->getNodeNum() || node->is_favorite);
+    return node && (node->num == nodeDB->getNodeNum() || nodeInfoLiteIsFavorite(node));
 }
 
 void InkHUD::FavoritesMapApplet::onRender(bool full)
@@ -25,7 +25,7 @@ void InkHUD::FavoritesMapApplet::onRender(bool full)
 
     // Draw our latest "node of interest" as a special marker.
     meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(lastFrom);
-    if (node && node->is_favorite && nodeDB->hasValidPosition(node) && enoughMarkers())
+    if (node && nodeInfoLiteIsFavorite(node) && nodeDB->hasValidPosition(node) && enoughMarkers())
         drawLabeledMarker(node);
 }
 
@@ -74,7 +74,7 @@ ProcessMessage InkHUD::FavoritesMapApplet::handleReceived(const meshtastic_MeshP
     } else {
         // For non-local packets, this applet only reacts to favorited nodes.
         const meshtastic_NodeInfoLite *sender = nodeDB->getMeshNode(mp.from);
-        if (!sender || !sender->is_favorite)
+        if (!nodeInfoLiteIsFavorite(sender))
             return ProcessMessage::CONTINUE;
 
         // Check if this position is from someone different than our previous position packet.

src/graphics/niche/InkHUD/Applets/User/Heard/HeardApplet.cpp

@@ -80,8 +80,8 @@ void InkHUD::HeardApplet::populateFromNodeDB()
         ordered.resize(maxCards());
 
     // Create card info for these (stale) node observations
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    for (meshtastic_NodeInfoLite *node : ordered) {
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    for (const meshtastic_NodeInfoLite *node : ordered) {
         CardInfo c;
         c.nodeNum = node->num;
 
@@ -89,14 +89,16 @@ void InkHUD::HeardApplet::populateFromNodeDB()
             c.hopsAway = node->hops_away;
 
         if (nodeDB->hasValidPosition(node) && nodeDB->hasValidPosition(ourNode)) {
-            // Get lat and long as float
-            // Meshtastic stores these as integers internally
-            float ourLat = ourNode->position.latitude_i * 1e-7;
-            float ourLong = ourNode->position.longitude_i * 1e-7;
-            float theirLat = node->position.latitude_i * 1e-7;
-            float theirLong = node->position.longitude_i * 1e-7;
+            meshtastic_PositionLite ourPos;
+            meshtastic_PositionLite theirPos;
+            if (nodeDB->copyNodePosition(ourNode->num, ourPos) && nodeDB->copyNodePosition(node->num, theirPos)) {
+                float ourLat = ourPos.latitude_i * 1e-7;
+                float ourLong = ourPos.longitude_i * 1e-7;
+                float theirLat = theirPos.latitude_i * 1e-7;
+                float theirLong = theirPos.longitude_i * 1e-7;
 
-            c.distanceMeters = (int32_t)GeoCoord::latLongToMeter(theirLat, theirLong, ourLat, ourLong);
+                c.distanceMeters = (int32_t)GeoCoord::latLongToMeter(theirLat, theirLong, ourLat, ourLong);
+            }
         }
 
         // Insert into the card collection (member of base class)
@@ -122,4 +124,4 @@ std::string InkHUD::HeardApplet::getHeaderText()
     return text;
 }
 
-#endif
+#endif

src/input/InputBroker.cpp

@@ -333,6 +333,12 @@ void InputBroker::Init()
             BaseType_t higherWake = 0;
             concurrency::mainDelay.interruptFromISR(&higherWake);
         };
+#if defined(ELECROW_ThinkNode_M7)
+        userConfigNoScreen.longLongPressTime = 15 * 1000;
+        userConfigNoScreen.longLongPress = INPUT_BROKER_FACTORY_RST;
+#else
+        userConfigNoScreen.longLongPress = INPUT_BROKER_SHUTDOWN;
+#endif
         userConfigNoScreen.singlePress = INPUT_BROKER_USER_PRESS;
         userConfigNoScreen.longPress = INPUT_BROKER_NONE;
         userConfigNoScreen.longPressTime = 500;

src/input/InputBroker.h

@@ -25,6 +25,7 @@ enum input_broker_event {
     INPUT_BROKER_USER_PRESS,
     INPUT_BROKER_ALT_PRESS,
     INPUT_BROKER_ALT_LONG,
+    INPUT_BROKER_FACTORY_RST = 0x9a,
     INPUT_BROKER_SHUTDOWN = 0x9b,
     INPUT_BROKER_GPS_TOGGLE = 0x9e,
     INPUT_BROKER_SEND_PING = 0xaf,

src/input/RotaryEncoderImpl.cpp

@@ -13,7 +13,6 @@ RotaryEncoderImpl *rotaryEncoderImpl;
 
 RotaryEncoderImpl::RotaryEncoderImpl()
 {
-    rotary = nullptr;
 #ifdef ARCH_ESP32
     isFirstInit = true;
 #endif
@@ -23,11 +22,6 @@ RotaryEncoderImpl::~RotaryEncoderImpl()
 {
     LOG_DEBUG("RotaryEncoderImpl destructor");
     detachRotaryEncoderInterrupts();
-
-    if (rotary != nullptr) {
-        delete rotary;
-        rotary = nullptr;
-    }
 }
 
 bool RotaryEncoderImpl::init()
@@ -43,8 +37,9 @@ bool RotaryEncoderImpl::init()
     eventPressed = static_cast<input_broker_event>(moduleConfig.canned_message.inputbroker_event_press);
 
     if (rotary == nullptr) {
-        rotary = new RotaryEncoder(moduleConfig.canned_message.inputbroker_pin_a, moduleConfig.canned_message.inputbroker_pin_b,
-                                   moduleConfig.canned_message.inputbroker_pin_press);
+        rotary.reset(new RotaryEncoder(moduleConfig.canned_message.inputbroker_pin_a,
+                                       moduleConfig.canned_message.inputbroker_pin_b,
+                                       moduleConfig.canned_message.inputbroker_pin_press));
     }
 
     attachRotaryEncoderInterrupts();

src/input/RotaryEncoderImpl.h

@@ -5,6 +5,7 @@
 #include "InputBroker.h"
 #include "concurrency/OSThread.h"
 #include "mesh/NodeDB.h"
+#include <memory>
 
 class RotaryEncoder;
 
@@ -28,7 +29,7 @@ class RotaryEncoderImpl final : public InputPollable
     input_broker_event eventCcw = INPUT_BROKER_NONE;
     input_broker_event eventPressed = INPUT_BROKER_NONE;
 
-    RotaryEncoder *rotary;
+    std::unique_ptr<RotaryEncoder> rotary;
 
   private:
 #ifdef ARCH_ESP32

src/main.cpp

@@ -59,12 +59,12 @@ NimbleBluetooth *nimbleBluetooth = nullptr;
 NRF52Bluetooth *nrf52Bluetooth = nullptr;
 #endif
 
-#if HAS_WIFI || defined(USE_WS5500)
+#if HAS_WIFI || defined(USE_WS5500) || defined(USE_CH390D)
 #include "mesh/api/WiFiServerAPI.h"
 #include "mesh/wifi/WiFiAPClient.h"
 #endif
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 #include "mesh/api/ethServerAPI.h"
 #include "mesh/eth/ethClient.h"
 #endif
@@ -335,7 +335,7 @@ void setup()
 
 #ifdef WIFI_LED
     pinMode(WIFI_LED, OUTPUT);
-    digitalWrite(WIFI_LED, LOW);
+    digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
 #endif
 
 #ifdef BLE_LED
@@ -731,8 +731,15 @@ void setup()
 #elif defined(USE_SH1107_128_64)
     screen_model = meshtastic_Config_DisplayConfig_OledType_OLED_SH1107; // keep dimension of 128x64
 #else
-    if (config.display.oled != meshtastic_Config_DisplayConfig_OledType_OLED_AUTO)
+    if (config.display.oled != meshtastic_Config_DisplayConfig_OledType_OLED_AUTO) {
         screen_model = config.display.oled;
+
+        // Fix: update geometry for SH1107 128x128 selected via menu
+        if (screen_model == meshtastic_Config_DisplayConfig_OledType_OLED_SH1107_128_128) {
+            screen_geometry = GEOMETRY_128_128;
+            screen_model = meshtastic_Config_DisplayConfig_OledType_OLED_SH1107; // normalize
+        }
+    }
 #endif
 #endif
 
@@ -951,12 +958,8 @@ void setup()
 #endif
 #endif
 
-    std::unique_ptr<RadioInterface> rIf;
-    if (!config.lora.serial_hal_only) {
-        rIf = initLoRa();
-    } else {
-        LOG_INFO("skipping LoRa radio init, for serialHal");
-    }
+    auto rIf = initLoRa();
+
     lateInitVariant(); // Do board specific init (see extra_variants/README.md for documentation)
 
 #if !MESHTASTIC_EXCLUDE_MQTT
@@ -1000,9 +1003,10 @@ void setup()
 
     // Start airtime logger thread.
     airTime = new AirTime();
-    if (!rIf && !config.lora.serial_hal_only)
+
+    if (!rIf)
         RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_NO_RADIO);
-    else if (rIf) {
+    else {
         // Log bit rate to debug output
         LOG_DEBUG("LoRA bitrate = %f bytes / sec", (float(meshtastic_Constants_DATA_PAYLOAD_LEN) /
                                                     (float(rIf->getPacketTime(meshtastic_Constants_DATA_PAYLOAD_LEN)))) *

src/mesh/CryptoEngine.cpp

@@ -112,7 +112,7 @@ bool CryptoEngine::ensurePkiKeys(meshtastic_Config_SecurityConfig &security, mes
  * @param bytes Buffer containing plaintext input.
  * @param bytesOut Output buffer to be populated with encrypted ciphertext.
  */
-bool CryptoEngine::encryptCurve25519(uint32_t toNode, uint32_t fromNode, meshtastic_UserLite_public_key_t remotePublic,
+bool CryptoEngine::encryptCurve25519(uint32_t toNode, uint32_t fromNode, meshtastic_NodeInfoLite_public_key_t remotePublic,
                                      uint64_t packetNum, size_t numBytes, const uint8_t *bytes, uint8_t *bytesOut)
 {
     uint8_t *auth;
@@ -152,7 +152,7 @@ bool CryptoEngine::encryptCurve25519(uint32_t toNode, uint32_t fromNode, meshtas
  * @param bytes Buffer containing ciphertext input.
  * @param bytesOut Output buffer to be populated with decrypted plaintext.
  */
-bool CryptoEngine::decryptCurve25519(uint32_t fromNode, meshtastic_UserLite_public_key_t remotePublic, uint64_t packetNum,
+bool CryptoEngine::decryptCurve25519(uint32_t fromNode, meshtastic_NodeInfoLite_public_key_t remotePublic, uint64_t packetNum,
                                      size_t numBytes, const uint8_t *bytes, uint8_t *bytesOut)
 {
     const uint8_t *auth = bytes + numBytes - 12; // set to last 8 bytes of text?

src/mesh/CryptoEngine.h

@@ -40,9 +40,12 @@ class CryptoEngine
 
 #endif
     void setDHPrivateKey(uint8_t *_private_key);
-    virtual bool encryptCurve25519(uint32_t toNode, uint32_t fromNode, meshtastic_UserLite_public_key_t remotePublic,
+    // The remotePublic key parameter takes the public_key bytes container from
+    // a stored node header. NodeInfoLite is the on-device storage type since
+    // the slim refactor flattened UserLite into it.
+    virtual bool encryptCurve25519(uint32_t toNode, uint32_t fromNode, meshtastic_NodeInfoLite_public_key_t remotePublic,
                                    uint64_t packetNum, size_t numBytes, const uint8_t *bytes, uint8_t *bytesOut);
-    virtual bool decryptCurve25519(uint32_t fromNode, meshtastic_UserLite_public_key_t remotePublic, uint64_t packetNum,
+    virtual bool decryptCurve25519(uint32_t fromNode, meshtastic_NodeInfoLite_public_key_t remotePublic, uint64_t packetNum,
                                    size_t numBytes, const uint8_t *bytes, uint8_t *bytesOut);
     virtual bool setDHPublicKey(uint8_t *publicKey);
     virtual void hash(uint8_t *bytes, size_t numBytes);

src/mesh/InterfacesTemplates.cpp

@@ -1,5 +1,9 @@
+#include "configuration.h"
+
 #include "LR11x0Interface.cpp"
 #include "LR11x0Interface.h"
+#include "LR20x0Interface.cpp"
+#include "LR20x0Interface.h"
 #include "SX126xInterface.cpp"
 #include "SX126xInterface.h"
 #include "SX128xInterface.cpp"
@@ -21,11 +25,14 @@ template class LR11x0Interface<LR1110>;
 template class LR11x0Interface<LR1120>;
 template class LR11x0Interface<LR1121>;
 #endif
+#if defined(USE_LR2021) && RADIOLIB_EXCLUDE_LR2021 != 1
+template class LR20x0Interface<LR2021>;
+#endif
 #ifdef ARCH_STM32WL
 template class SX126xInterface<STM32WLx>;
 #endif
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 #include "api/ethServerAPI.h"
 template class ServerAPI<EthernetClient>;
 template class APIServerPort<ethServerAPI, EthernetServer>;

src/mesh/LR11x0Interface.cpp

@@ -57,16 +57,19 @@ template <typename T> bool LR11x0Interface<T>::init()
 #if ARCH_PORTDUINO
     float tcxoVoltage = (float)portduino_config.dio3_tcxo_voltage / 1000;
 // FIXME: correct logic to default to not using TCXO if no voltage is specified for LR11x0_DIO3_TCXO_VOLTAGE
-#elif !defined(LR11X0_DIO3_TCXO_VOLTAGE)
+#elif defined(LR11X0_DIO3_TCXO_VOLTAGE)
+    float tcxoVoltage = LR11X0_DIO3_TCXO_VOLTAGE;
+    LOG_DEBUG("LR11X0_DIO3_TCXO_VOLTAGE defined, using DIO3 as TCXO reference voltage at %f V", LR11X0_DIO3_TCXO_VOLTAGE);
+    // (DIO3 is not free to be used as an IRQ)
+#elif defined(TCXO_OPTIONAL)
+    float tcxoVoltage = 1.6f; // TCXO_OPTIONAL: try default 1.6 V first, fall back to XTAL on failure
+    LOG_DEBUG("TCXO_OPTIONAL: no LR11X0_DIO3_TCXO_VOLTAGE defined, trying default TCXO Vref 1.6 V first");
+#else
     float tcxoVoltage =
         0; // "TCXO reference voltage to be set on DIO3. Defaults to 1.6 V, set to 0 to skip." per
            // https://github.com/jgromes/RadioLib/blob/690a050ebb46e6097c5d00c371e961c1caa3b52e/src/modules/LR11x0/LR11x0.h#L471C26-L471C104
     // (DIO3 is free to be used as an IRQ)
     LOG_DEBUG("LR11X0_DIO3_TCXO_VOLTAGE not defined, not using DIO3 as TCXO reference voltage");
-#else
-    float tcxoVoltage = LR11X0_DIO3_TCXO_VOLTAGE;
-    LOG_DEBUG("LR11X0_DIO3_TCXO_VOLTAGE defined, using DIO3 as TCXO reference voltage at %f V", LR11X0_DIO3_TCXO_VOLTAGE);
-    // (DIO3 is not free to be used as an IRQ)
 #endif
 
     RadioLibInterface::init();
@@ -101,6 +104,17 @@ template <typename T> bool LR11x0Interface<T>::init()
         res = lora.begin(getFreq(), bw, sf, cr, syncWord, power, preambleLength, tcxoVoltage);
     }
 
+#if defined(TCXO_OPTIONAL)
+    // If init failed for any reason other than chip not found, retry without TCXO (XTAL mode)
+    if (res != RADIOLIB_ERR_NONE && res != RADIOLIB_ERR_CHIP_NOT_FOUND && tcxoVoltage > 0) {
+        LOG_WARN("LR11x0 init failed with TCXO Vref %f V (err %d), retrying without TCXO", tcxoVoltage, res);
+        tcxoVoltage = 0;
+        res = lora.begin(getFreq(), bw, sf, cr, syncWord, power, preambleLength, tcxoVoltage);
+        if (res == RADIOLIB_ERR_NONE)
+            LOG_INFO("LR11x0 init success without TCXO (XTAL mode)");
+    }
+#endif
+
     // \todo Display actual typename of the adapter, not just `LR11x0`
     LOG_INFO("LR11x0 init result %d", res);
     if (res == RADIOLIB_ERR_CHIP_NOT_FOUND || res == RADIOLIB_ERR_SPI_CMD_FAILED)
@@ -198,7 +212,7 @@ template <typename T> bool LR11x0Interface<T>::reconfigure()
 
     startReceive(); // restart receiving
 
-    return RADIOLIB_ERR_NONE;
+    return true;
 }
 
 template <typename T> void LR11x0Interface<T>::disableInterrupt()

src/mesh/LR2021Interface.cpp

@@ -0,0 +1,18 @@
+#include "configuration.h"
+
+#if defined(USE_LR2021) && RADIOLIB_EXCLUDE_LR2021 != 1
+
+#include "LR2021Interface.h"
+#include "error.h"
+
+LR2021Interface::LR2021Interface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
+                                 RADIOLIB_PIN_TYPE busy)
+    : LR20x0Interface(hal, cs, irq, rst, busy)
+{
+}
+
+bool LR2021Interface::wideLora()
+{
+    return true;
+}
+#endif

src/mesh/LR2021Interface.h

@@ -0,0 +1,15 @@
+#pragma once
+#if RADIOLIB_EXCLUDE_LR2021 != 1
+#include "LR20x0Interface.h"
+
+/**
+ * Our adapter for LR2021 radios
+ */
+class LR2021Interface : public LR20x0Interface<LR2021>
+{
+  public:
+    LR2021Interface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
+                    RADIOLIB_PIN_TYPE busy);
+    bool wideLora() override;
+};
+#endif

src/mesh/LR20x0Interface.cpp

@@ -0,0 +1,399 @@
+#include "configuration.h"
+
+#if defined(USE_LR2021) && RADIOLIB_EXCLUDE_LR2021 != 1
+#include "LR20x0Interface.h"
+#include "error.h"
+#include "mesh/NodeDB.h"
+
+// Keep LR20x0 naming while RadioLib exposes LR2021 symbols.
+#ifndef LR20x0
+#define LR20x0 LR2021
+#endif
+
+#ifdef LR2021_DIO_AS_RF_SWITCH
+#include "rfswitch.h"
+#elif ARCH_PORTDUINO
+#include "PortduinoGlue.h"
+#define lr20x0_rfswitch_dio_pins portduino_config.rfswitch_dio_pins
+#define lr20x0_rfswitch_table portduino_config.rfswitch_table
+#else
+static const uint32_t lr20x0_rfswitch_dio_pins[] = {RADIOLIB_NC, RADIOLIB_NC, RADIOLIB_NC, RADIOLIB_NC, RADIOLIB_NC};
+static const Module::RfSwitchMode_t lr20x0_rfswitch_table[] = {
+    {LR20x0::MODE_STBY, {}},  {LR20x0::MODE_RX, {}},    {LR20x0::MODE_TX, {}},
+    {LR20x0::MODE_RX_HF, {}}, {LR20x0::MODE_TX_HF, {}}, END_OF_MODE_TABLE,
+};
+#endif
+
+// Particular boards might define a different max power based on what their hardware can do, default to max power output if not
+// specified (may be dangerous if using external PA and LR20x0 power config forgotten)
+#if ARCH_PORTDUINO
+#define LR2021_MAX_POWER portduino_config.lr2021_max_power
+#endif
+#ifndef LR2021_MAX_POWER
+#define LR2021_MAX_POWER 22
+#endif
+
+// the 2.4G part maxes at 12dBm
+
+#if ARCH_PORTDUINO
+#define LR2021_MAX_POWER_HF portduino_config.lr2021_max_power_hf
+#endif
+#ifndef LR2021_MAX_POWER_HF
+#define LR2021_MAX_POWER_HF 12
+#endif
+
+template <typename T>
+LR20x0Interface<T>::LR20x0Interface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
+                                    RADIOLIB_PIN_TYPE busy)
+    : RadioLibInterface(hal, cs, irq, rst, busy, &lora), lora(&module)
+{
+    LOG_WARN("LR20x0Interface(cs=%d, irq=%d, rst=%d, busy=%d)", cs, irq, rst, busy);
+}
+
+/// Initialise the Driver transport hardware and software.
+/// Make sure the Driver is properly configured before calling init().
+/// \return true if initialisation succeeded.
+template <typename T> bool LR20x0Interface<T>::init()
+{
+#ifdef LR2021_POWER_EN
+    pinMode(LR2021_POWER_EN, OUTPUT);
+    digitalWrite(LR2021_POWER_EN, HIGH);
+#endif
+
+#if ARCH_PORTDUINO
+    float tcxoVoltage = (float)portduino_config.dio3_tcxo_voltage / 1000;
+// FIXME: correct logic to default to not using TCXO if no voltage is specified for LR20x0_DIO3_TCXO_VOLTAGE
+#elif defined(LR2021_DIO3_TCXO_VOLTAGE)
+    float tcxoVoltage = LR2021_DIO3_TCXO_VOLTAGE;
+    LOG_DEBUG("LR2021_DIO3_TCXO_VOLTAGE defined, using DIO3 as TCXO reference voltage at %f V", LR2021_DIO3_TCXO_VOLTAGE);
+    // (DIO3 is not free to be used as an IRQ)
+#elif defined(TCXO_OPTIONAL)
+    float tcxoVoltage = 1.6f; // TCXO_OPTIONAL: try default 1.6 V first, fall back to XTAL on failure
+    LOG_DEBUG("TCXO_OPTIONAL: no LR2021_DIO3_TCXO_VOLTAGE defined, trying default TCXO Vref 1.6 V first");
+#else
+    float tcxoVoltage =
+        0; // "TCXO reference voltage to be set on DIO3. Defaults to 1.6 V, set to 0 to skip." per
+           // https://github.com/jgromes/RadioLib/blob/690a050ebb46e6097c5d00c371e961c1caa3b52e/src/modules/LR11x0/LR11x0.h#L471C26-L471C104
+    // (DIO3 is free to be used as an IRQ)
+    LOG_DEBUG("LR2021_DIO3_TCXO_VOLTAGE not defined, not using DIO3 as TCXO reference voltage");
+#endif
+
+    RadioLibInterface::init();
+
+#ifdef LR2021_IRQ_DIO_NUM
+    lora.irqDioNum = LR2021_IRQ_DIO_NUM;
+    LOG_DEBUG("Set irqDioNum %d", lora.irqDioNum);
+#elif defined(IRQ_DIO_NUM)
+    lora.irqDioNum = IRQ_DIO_NUM;
+    LOG_DEBUG("Set irqDioNum %d", lora.irqDioNum);
+#else
+    LOG_DEBUG("Use default irqDioNum %d", lora.irqDioNum);
+#endif
+
+    if (config.lora.region == meshtastic_Config_LoRaConfig_RegionCode_LORA_24) { // clamp if wide freq range
+        limitPower(LR2021_MAX_POWER_HF);
+    } else {
+        limitPower(LR2021_MAX_POWER); // default clamp for non-wide freq range
+    }
+
+#ifdef LR2021_RF_SWITCH_SUBGHZ
+    pinMode(LR2021_RF_SWITCH_SUBGHZ, OUTPUT);
+    digitalWrite(LR2021_RF_SWITCH_SUBGHZ, getFreq() < 1e9 ? HIGH : LOW);
+    LOG_DEBUG("Set RF0 switch to %s", getFreq() < 1e9 ? "SubGHz" : "2.4GHz");
+#endif
+
+#ifdef LR2021_RF_SWITCH_2_4GHZ
+    pinMode(LR2021_RF_SWITCH_2_4GHZ, OUTPUT);
+    digitalWrite(LR2021_RF_SWITCH_2_4GHZ, getFreq() < 1e9 ? LOW : HIGH);
+    LOG_DEBUG("Set RF1 switch to %s", getFreq() < 1e9 ? "SubGHz" : "2.4GHz");
+#endif
+
+    // Allow extra time for TCXO to stabilize after power-on
+    delay(10);
+
+    int res = lora.begin(getFreq(), bw, sf, cr, syncWord, power, preambleLength, tcxoVoltage);
+
+    // Retry if we get SPI command failed - some units need extra TCXO stabilization time
+    if (res == RADIOLIB_ERR_SPI_CMD_FAILED) {
+        LOG_WARN("LR20x0 init failed with %d (SPI_CMD_FAILED), retrying after delay...", res);
+        delay(100);
+        res = lora.begin(getFreq(), bw, sf, cr, syncWord, power, preambleLength, tcxoVoltage);
+    }
+
+#if defined(TCXO_OPTIONAL)
+    // If init failed for any reason other than chip not found, retry without TCXO (XTAL mode)
+    if (res != RADIOLIB_ERR_NONE && res != RADIOLIB_ERR_CHIP_NOT_FOUND && tcxoVoltage > 0) {
+        LOG_WARN("LR20x0 init failed with TCXO Vref %f V (err %d), retrying without TCXO", tcxoVoltage, res);
+        tcxoVoltage = 0;
+        res = lora.begin(getFreq(), bw, sf, cr, syncWord, power, preambleLength, tcxoVoltage);
+        if (res == RADIOLIB_ERR_NONE)
+            LOG_INFO("LR20x0 init success without TCXO (XTAL mode)");
+    }
+#endif
+
+    // \todo Display actual typename of the adapter, not just `LR20x0`
+    LOG_INFO("LR20x0 init result %d", res);
+    if (res == RADIOLIB_ERR_CHIP_NOT_FOUND || res == RADIOLIB_ERR_SPI_CMD_FAILED)
+        return false;
+
+        // Some basic info about the module's explicit firmware version - no other info available
+        // Currently requires radiolib godmode
+
+#if RADIOLIB_GODMODE
+    uint8_t fwMajor = 0;
+    uint8_t fwMinor = 0;
+    int versionRes = lora.getVersion(&fwMajor, &fwMinor);
+    if (versionRes == RADIOLIB_ERR_NONE)
+        LOG_DEBUG("LR20x0 FW %d.%d", fwMajor, fwMinor);
+#endif
+
+    LOG_INFO("Frequency set to %f", getFreq());
+    LOG_INFO("Bandwidth set to %f", bw);
+    LOG_INFO("Power output set to %d", power);
+
+    if (res == RADIOLIB_ERR_NONE)
+        res = lora.setCRC(2);
+
+        // Standard DCDC ramp timing from RadioLib workarounds (register 0x00F20024)
+        // Currently requires radiolib godmode
+#if RADIOLIB_GODMODE
+    if (res == RADIOLIB_ERR_NONE) {
+        uint8_t rampTimes[4] = {15, 15, 15, 15}; // Standard case for all conditions
+        res = lora.setRegMode(RADIOLIB_LR2021_REG_MODE_SIMO_NORMAL, rampTimes);
+        if (res != RADIOLIB_ERR_NONE)
+            LOG_WARN("LR2021 setRegMode failed: %d", res);
+    }
+#endif
+
+#ifdef LR2021_DIO_AS_RF_SWITCH
+    bool dioAsRfSwitch = true;
+#elif defined(ARCH_PORTDUINO)
+    bool dioAsRfSwitch = portduino_config.has_rfswitch_table;
+#else
+    bool dioAsRfSwitch = false;
+#endif
+
+    if (dioAsRfSwitch) {
+        lora.setRfSwitchTable(lr20x0_rfswitch_dio_pins, lr20x0_rfswitch_table);
+        LOG_DEBUG("Set DIO RF switch");
+    }
+
+    if (res == RADIOLIB_ERR_NONE) {
+        if (config.lora.sx126x_rx_boosted_gain) { // the name is unfortunate but historically accurate
+            res = lora.setRxBoostedGainMode(true);
+            LOG_INFO("Set RX gain to boosted mode; result: %d", res);
+        } else {
+            res = lora.setRxBoostedGainMode(false);
+            LOG_INFO("Set RX gain to power saving mode (boosted mode off); result: %d", res);
+        }
+    }
+
+    if (res == RADIOLIB_ERR_NONE)
+        startReceive(); // start receiving
+
+    return res == RADIOLIB_ERR_NONE;
+}
+
+template <typename T> bool LR20x0Interface<T>::reconfigure()
+{
+    RadioLibInterface::reconfigure();
+
+    // set mode to standby
+    setStandby();
+
+    // configure publicly accessible settings
+    int err = lora.setSpreadingFactor(sf);
+    if (err != RADIOLIB_ERR_NONE)
+        RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_INVALID_RADIO_SETTING);
+
+    err = lora.setBandwidth(bw); // different form than LR11xx
+    if (err != RADIOLIB_ERR_NONE)
+        RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_INVALID_RADIO_SETTING);
+
+    err = lora.setCodingRate(cr, cr != 7); // use long interleaving except if CR is 4/7 which doesn't support it
+    if (err != RADIOLIB_ERR_NONE)
+        RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_INVALID_RADIO_SETTING);
+
+    err = lora.setSyncWord(syncWord);
+    assert(err == RADIOLIB_ERR_NONE);
+
+    if (config.lora.region == meshtastic_Config_LoRaConfig_RegionCode_LORA_24) { // clamp if wide freq range
+        limitPower(LR2021_MAX_POWER_HF);
+    } else {
+        limitPower(LR2021_MAX_POWER); // default clamp for non-wide freq range
+    }
+
+    err = lora.setPreambleLength(preambleLength);
+    assert(err == RADIOLIB_ERR_NONE);
+
+    err = lora.setFrequency(getFreq());
+    if (err != RADIOLIB_ERR_NONE)
+        RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_INVALID_RADIO_SETTING);
+
+    err = lora.setOutputPower(power);
+    assert(err == RADIOLIB_ERR_NONE);
+
+    // Apply RX gain mode — valid in STDBY, matches resetAGC() pattern
+    err = lora.setRxBoostedGainMode(config.lora.sx126x_rx_boosted_gain);
+    if (err != RADIOLIB_ERR_NONE)
+        LOG_WARN("LR20x0 setRxBoostedGainMode %s%d", radioLibErr, err);
+
+    startReceive(); // restart receiving
+
+    return true;
+}
+
+template <typename T> void LR20x0Interface<T>::disableInterrupt()
+{
+    lora.clearIrqAction();
+}
+
+template <typename T> void LR20x0Interface<T>::setStandby()
+{
+    checkNotification(); // handle any pending interrupts before we force standby
+
+    int err = lora.standby();
+
+    if (err != RADIOLIB_ERR_NONE) {
+        LOG_DEBUG("LR20x0 standby failed with error %d", err);
+    }
+
+    assert(err == RADIOLIB_ERR_NONE);
+
+    isReceiving = false; // If we were receiving, not any more
+    activeReceiveStart = 0;
+    disableInterrupt();
+    completeSending(); // If we were sending, not anymore
+    RadioLibInterface::setStandby();
+}
+
+/**
+ * Add SNR data to received messages
+ */
+template <typename T> void LR20x0Interface<T>::addReceiveMetadata(meshtastic_MeshPacket *mp)
+{
+    // LOG_DEBUG("PacketStatus %x", lora.getPacketStatus());
+    mp->rx_snr = lora.getSNR();
+    mp->rx_rssi = lround(lora.getRSSI());
+    // LOG_DEBUG("Corrected frequency offset: %f", lora.getFrequencyError()); // not implemented for LR20x0, but noop for LR11x0
+    // too(!)
+}
+
+/** We override to turn on transmitter power as needed.
+ */
+template <typename T> void LR20x0Interface<T>::configHardwareForSend()
+{
+    RadioLibInterface::configHardwareForSend();
+}
+
+// For power draw measurements, helpful to force radio to stay sleeping
+// #define SLEEP_ONLY
+
+template <typename T> void LR20x0Interface<T>::startReceive()
+{
+#ifdef SLEEP_ONLY
+    sleep();
+#else
+
+    setStandby();
+
+    lora.setPreambleLength(preambleLength); // Solve RX ack fail after direct message sent.  Not sure why this is needed.
+
+    // We use a 16 bit preamble so this should save some power by letting radio sit in standby mostly.
+    int err =
+        lora.startReceive(RADIOLIB_LR2021_RX_TIMEOUT_INF, MESHTASTIC_RADIOLIB_IRQ_RX_FLAGS, RADIOLIB_IRQ_RX_DEFAULT_MASK, 0);
+    if (err)
+        LOG_ERROR("StartReceive error: %d", err);
+    assert(err == RADIOLIB_ERR_NONE);
+
+    RadioLibInterface::startReceive();
+
+    // Must be done AFTER starting receive, because startReceive clears (possibly stale) interrupt pending register bits
+    enableInterrupt(isrRxLevel0);
+    checkRxDoneIrqFlag();
+#endif
+}
+
+/** Is the channel currently active? */
+template <typename T> bool LR20x0Interface<T>::isChannelActive()
+{
+    // check if we can detect a LoRa preamble on the current channel
+    ChannelScanConfig_t cfg = {.cad = {.symNum = NUM_SYM_CAD,
+                                       .detPeak = RADIOLIB_LR2021_CAD_PARAM_DEFAULT,
+                                       .detMin = RADIOLIB_LR2021_CAD_PARAM_DEFAULT,
+                                       .exitMode = RADIOLIB_LR2021_CAD_PARAM_DEFAULT,
+                                       .timeout = 0,
+                                       .irqFlags = RADIOLIB_IRQ_CAD_DEFAULT_FLAGS,
+                                       .irqMask = RADIOLIB_IRQ_CAD_DEFAULT_MASK}};
+    int16_t result;
+
+    setStandby();
+    result = lora.scanChannel(cfg);
+    if (result == RADIOLIB_LORA_DETECTED)
+        return true;
+
+    assert(result != RADIOLIB_ERR_WRONG_MODEM);
+
+    return false;
+}
+
+/** Could we send right now (i.e. either not actively receiving or transmitting)? */
+template <typename T> bool LR20x0Interface<T>::isActivelyReceiving()
+{
+    // The IRQ status will be cleared when we start our read operation. Check if we've started a header, but haven't yet
+    // received and handled the interrupt for reading the packet/handling errors.
+    return receiveDetected(lora.getIrqStatus(), RADIOLIB_LR2021_IRQ_LORA_HEADER_VALID, RADIOLIB_LR2021_IRQ_PREAMBLE_DETECTED);
+}
+
+#ifdef LR20X0_AGC_RESET
+template <typename T> void LR20x0Interface<T>::resetAGC()
+{
+    // Safety: don't reset mid-packet
+    if (sendingPacket != NULL || (isReceiving && isActivelyReceiving()))
+        return;
+
+    LOG_DEBUG("LR20x0 AGC reset: warm sleep + Calibrate(0x3F)");
+
+    // 1. Warm sleep — powers down the analog frontend, resetting AGC state
+    lora.sleep(true, 0);
+
+    // 2. Wake to RC standby for stable calibration
+    lora.standby(RADIOLIB_LR20X0_STANDBY_RC, true);
+
+    // 3. Calibrate all blocks (PLL, ADC, image, RC oscillators)
+    //    calibrate() is protected on LR20x0, so use raw SPI (same as internal implementation)
+    uint8_t calData = RADIOLIB_LR20X0_CALIBRATE_ALL;
+    module.SPIwriteStream(RADIOLIB_LR20X0_CMD_CALIBRATE, &calData, 1, true, true);
+
+    // 4. Re-calibrate image rejection for actual operating frequency
+    //    Calibrate(0x3F) defaults to 902-928 MHz which is wrong for other regions.
+    lora.calibrateImageRejection(getFreq() - 4.0f, getFreq() + 4.0f);
+
+    // 5. Re-apply RX boosted gain mode
+    lora.setRxBoostedGainMode(config.lora.sx126x_rx_boosted_gain);
+
+    // 6. Resume receiving
+    startReceive();
+}
+#endif
+
+template <typename T> bool LR20x0Interface<T>::sleep()
+{
+    // \todo Display actual typename of the adapter, not just `LR20x0`
+    LOG_DEBUG("LR20x0 entering sleep mode");
+    setStandby(); // Stop any pending operations
+
+    // turn off TCXO if it was powered
+    lora.setTCXO(0);
+
+    // put chipset into sleep mode (we've already disabled interrupts by now)
+    bool keepConfig = false;
+    lora.sleep(keepConfig, 0); // Note: we do not keep the config, full reinit will be needed
+
+#ifdef LR2021_POWER_EN
+    digitalWrite(LR2021_POWER_EN, LOW);
+#endif
+
+    return true;
+}
+#endif

src/mesh/LR20x0Interface.h

@@ -0,0 +1,75 @@
+#pragma once
+#if RADIOLIB_EXCLUDE_LR2021 != 1
+#include "RadioLibInterface.h"
+
+/**
+ * \brief Adapter for LR20x0 radio family. Implements common logic for child classes.
+ * \tparam T RadioLib module type for LR20x0, e.g. LR2021.
+ */
+template <class T> class LR20x0Interface : public RadioLibInterface
+{
+  public:
+    LR20x0Interface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
+                    RADIOLIB_PIN_TYPE busy);
+
+    /// Initialise the Driver transport hardware and software.
+    /// Make sure the Driver is properly configured before calling init().
+    /// \return true if initialisation succeeded.
+    virtual bool init() override;
+
+    /// Apply any radio provisioning changes
+    /// Make sure the Driver is properly configured before calling init().
+    /// \return true if initialisation succeeded.
+    virtual bool reconfigure() override;
+
+    /// Prepare hardware for sleep.  Call this _only_ for deep sleep, not needed for light sleep.
+    virtual bool sleep() override;
+
+    bool isIRQPending() override { return lora.getIrqFlags() != 0; }
+
+#ifdef LR20X0_AGC_RESET
+    void resetAGC() override;
+#endif
+
+  protected:
+    /**
+     * Specific module instance
+     */
+    T lora;
+
+    /**
+     * Glue functions called from ISR land
+     */
+    virtual void disableInterrupt() override;
+
+    /**
+     * Enable a particular ISR callback glue function
+     */
+    virtual void enableInterrupt(void (*callback)()) { lora.setIrqAction(callback); }
+
+    /** can we detect a LoRa preamble on the current channel? */
+    virtual bool isChannelActive() override;
+
+    /** are we actively receiving a packet (only called during receiving state) */
+    virtual bool isActivelyReceiving() override;
+
+    /**
+     * Start waiting to receive a message
+     */
+    virtual void startReceive() override;
+
+    /**
+     *  We override to turn on transmitter power as needed.
+     */
+    virtual void configHardwareForSend() override;
+
+    /**
+     * Add SNR data to received messages
+     */
+    virtual void addReceiveMetadata(meshtastic_MeshPacket *mp) override;
+
+    virtual void setStandby() override;
+
+    uint32_t getPacketTime(uint32_t pl, bool received) override { return computePacketTime(lora, pl, received); }
+};
+#endif

src/mesh/MeshService.cpp

@@ -94,8 +94,9 @@ int MeshService::handleFromRadio(const meshtastic_MeshPacket *mp)
         mp->decoded.portnum == meshtastic_PortNum_TELEMETRY_APP && mp->decoded.request_id > 0) {
         LOG_DEBUG("Received telemetry response. Skip sending our NodeInfo");
         //  ignore our request for its NodeInfo
-    } else if (mp->which_payload_variant == meshtastic_MeshPacket_decoded_tag && !nodeDB->getMeshNode(mp->from)->has_user &&
-               nodeInfoModule && !isPreferredRebroadcaster && !nodeDB->isFull()) {
+    } else if (mp->which_payload_variant == meshtastic_MeshPacket_decoded_tag &&
+               !nodeInfoLiteHasUser(nodeDB->getMeshNode(mp->from)) && nodeInfoModule && !isPreferredRebroadcaster &&
+               !nodeDB->isFull()) {
         if (airTime->isTxAllowedChannelUtil(true)) {
             const int8_t hopsUsed = getHopsAway(*mp, config.lora.hop_limit);
             if (hopsUsed > (int32_t)(config.lora.hop_limit + 2)) {
@@ -392,19 +393,16 @@ meshtastic_NodeInfoLite *MeshService::refreshLocalMeshNode()
     meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
     assert(node);
 
-    // We might not have a position yet for our local node, in that case, at least try to send the time
-    if (!node->has_position) {
-        memset(&node->position, 0, sizeof(node->position));
-        node->has_position = true;
-    }
-
-    meshtastic_PositionLite &position = node->position;
-
     // Update our local node info with our time (even if we don't decide to update anyone else)
     node->last_heard =
         getValidTime(RTCQualityFromNet); // This nodedb timestamp might be stale, so update it if our clock is kinda valid
 
-    position.time = getValidTime(RTCQualityFromNet);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    // Make sure our own NodeNum has a slot in the position map so subsequent
+    // updates (and the bundled NodeInfo emission to the phone) have somewhere
+    // to read from. Insert a default-zero entry on first call.
+    nodeDB->touchNodePositionTime(node->num, getValidTime(RTCQualityFromNet));
+#endif
 
     if (powerStatus->getHasBattery() == 1) {
         updateBatteryLevel(powerStatus->getBatteryChargePercent());
@@ -432,7 +430,9 @@ int MeshService::onGPSChanged(const meshtastic::GPSStatus *newStatus)
     // Used fixed position if configured regardless of GPS lock
     if (config.position.fixed_position) {
         LOG_WARN("Use fixed position");
-        pos = TypeConversions::ConvertToPosition(node->position);
+        meshtastic_PositionLite fixedSlot;
+        if (nodeDB->copyNodePosition(node->num, fixedSlot))
+            pos = TypeConversions::ConvertToPosition(fixedSlot);
     }
 
     // Add a fresh timestamp

src/mesh/NodeDB.cpp

@@ -22,6 +22,7 @@
 #include "error.h"
 #include "main.h"
 #include "mesh-pb-constants.h"
+#include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
 #include "meshUtils.h"
 #include "modules/NeighborInfoModule.h"
 #include <ErriezCRC32.h>
@@ -151,24 +152,103 @@ uint32_t get_st7789_id(uint8_t cs, uint8_t sck, uint8_t mosi, uint8_t dc, uint8_
 
 #endif
 
-bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_iter_t *field)
+bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field)
 {
-    if (ostream) {
-        std::vector<meshtastic_NodeInfoLite> const *vec = (std::vector<meshtastic_NodeInfoLite> *)field->pData;
-        for (auto item : *vec) {
-            if (!pb_encode_tag_for_field(ostream, field))
-                return false;
-            pb_encode_submessage(ostream, meshtastic_NodeInfoLite_fields, &item);
+    const auto *iter = reinterpret_cast<const pb_field_iter_t *>(field);
+    switch (iter->tag) {
+    case meshtastic_NodeDatabase_nodes_tag: {
+        if (ostream) {
+            const auto *vec = static_cast<const std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
+            for (auto item : *vec) {
+                if (!pb_encode_tag_for_field(ostream, iter))
+                    return false;
+                if (!pb_encode_submessage(ostream, meshtastic_NodeInfoLite_fields, &item))
+                    return false;
+            }
         }
+        if (istream) {
+            meshtastic_NodeInfoLite node;
+            auto *vec = static_cast<std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
+            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
+                vec->push_back(node);
+        }
+        return true;
     }
-    if (istream) {
-        meshtastic_NodeInfoLite node; // this gets good data
-        std::vector<meshtastic_NodeInfoLite> *vec = (std::vector<meshtastic_NodeInfoLite> *)field->pData;
-
-        if (istream->bytes_left && pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
-            vec->push_back(node);
+    case meshtastic_NodeDatabase_positions_tag: {
+        if (ostream) {
+            const auto *vec = static_cast<const std::vector<meshtastic_NodePositionEntry> *>(iter->pData);
+            for (auto item : *vec) {
+                if (!pb_encode_tag_for_field(ostream, iter))
+                    return false;
+                if (!pb_encode_submessage(ostream, meshtastic_NodePositionEntry_fields, &item))
+                    return false;
+            }
+        }
+        if (istream) {
+            meshtastic_NodePositionEntry entry;
+            auto *vec = static_cast<std::vector<meshtastic_NodePositionEntry> *>(iter->pData);
+            if (istream->bytes_left && pb_decode(istream, meshtastic_NodePositionEntry_fields, &entry))
+                vec->push_back(entry);
+        }
+        return true;
+    }
+    case meshtastic_NodeDatabase_telemetry_tag: {
+        if (ostream) {
+            const auto *vec = static_cast<const std::vector<meshtastic_NodeTelemetryEntry> *>(iter->pData);
+            for (auto item : *vec) {
+                if (!pb_encode_tag_for_field(ostream, iter))
+                    return false;
+                if (!pb_encode_submessage(ostream, meshtastic_NodeTelemetryEntry_fields, &item))
+                    return false;
+            }
+        }
+        if (istream) {
+            meshtastic_NodeTelemetryEntry entry;
+            auto *vec = static_cast<std::vector<meshtastic_NodeTelemetryEntry> *>(iter->pData);
+            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeTelemetryEntry_fields, &entry))
+                vec->push_back(entry);
+        }
+        return true;
+    }
+    case meshtastic_NodeDatabase_status_tag: {
+        if (ostream) {
+            const auto *vec = static_cast<const std::vector<meshtastic_NodeStatusEntry> *>(iter->pData);
+            for (auto item : *vec) {
+                if (!pb_encode_tag_for_field(ostream, iter))
+                    return false;
+                if (!pb_encode_submessage(ostream, meshtastic_NodeStatusEntry_fields, &item))
+                    return false;
+            }
+        }
+        if (istream) {
+            meshtastic_NodeStatusEntry entry;
+            auto *vec = static_cast<std::vector<meshtastic_NodeStatusEntry> *>(iter->pData);
+            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeStatusEntry_fields, &entry))
+                vec->push_back(entry);
+        }
+        return true;
+    }
+    case meshtastic_NodeDatabase_environment_tag: {
+        if (ostream) {
+            const auto *vec = static_cast<const std::vector<meshtastic_NodeEnvironmentEntry> *>(iter->pData);
+            for (auto item : *vec) {
+                if (!pb_encode_tag_for_field(ostream, iter))
+                    return false;
+                if (!pb_encode_submessage(ostream, meshtastic_NodeEnvironmentEntry_fields, &item))
+                    return false;
+            }
+        }
+        if (istream) {
+            meshtastic_NodeEnvironmentEntry entry;
+            auto *vec = static_cast<std::vector<meshtastic_NodeEnvironmentEntry> *>(iter->pData);
+            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeEnvironmentEntry_fields, &entry))
+                vec->push_back(entry);
+        }
+        return true;
+    }
+    default:
+        return true;
     }
-    return true;
 }
 
 /** The current change # for radio settings.  Starts at 0 on boot and any time the radio settings
@@ -298,8 +378,7 @@ NodeDB::NodeDB()
 #endif
     // Include our owner in the node db under our nodenum
     meshtastic_NodeInfoLite *info = getOrCreateMeshNode(getNodeNum());
-    info->user = TypeConversions::ConvertToUserLite(owner);
-    info->has_user = true;
+    TypeConversions::CopyUserToNodeInfoLite(info, owner);
 
     // If node database has not been saved for the first time, save it now
 #ifdef FSCom
@@ -442,8 +521,12 @@ NodeDB::NodeDB()
 #if defined(USERPREFS_FIXED_GPS_LAT) && defined(USERPREFS_FIXED_GPS_LON)
         fixedGPS.location_source = meshtastic_Position_LocSource_LOC_MANUAL;
         config.has_position = true;
-        info->has_position = true;
-        info->position = TypeConversions::ConvertToPositionLite(fixedGPS);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+        {
+            concurrency::LockGuard guard(&satelliteMutex);
+            nodePositions[info->num] = TypeConversions::ConvertToPositionLite(fixedGPS);
+        }
+#endif
         nodeDB->setLocalPosition(fixedGPS);
         config.position.fixed_position = true;
 #endif
@@ -479,6 +562,29 @@ bool isBroadcast(uint32_t dest)
     return dest == NODENUM_BROADCAST || dest == NODENUM_BROADCAST_NO_LORA;
 }
 
+namespace
+{
+template <typename Map, typename Value> bool copySatelliteEntry(const Map &map, NodeNum n, Value &out)
+{
+    auto it = map.find(n);
+    if (it == map.end())
+        return false;
+    out = it->second;
+    return true;
+}
+
+template <typename Map> std::vector<NodeNum> snapshotSatelliteNodeNums(const Map &map, NodeNum exclude)
+{
+    std::vector<NodeNum> result;
+    result.reserve(map.size());
+    for (const auto &kv : map) {
+        if (kv.first != exclude)
+            result.push_back(kv.first);
+    }
+    return result;
+}
+} // namespace
+
 void NodeDB::resetRadioConfig(bool is_fresh_install)
 {
     if (is_fresh_install) {
@@ -548,6 +654,19 @@ void NodeDB::installDefaultNodeDatabase()
     nodeDatabase.nodes = std::vector<meshtastic_NodeInfoLite>(MAX_NUM_NODES);
     numMeshNodes = 0;
     meshNodes = &nodeDatabase.nodes;
+    concurrency::LockGuard satelliteGuard(&satelliteMutex);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    nodePositions.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    nodeTelemetry.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    nodeEnvironment.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    nodeStatus.clear();
+#endif
 }
 
 void NodeDB::installDefaultConfig(bool preserveKey = false)
@@ -1019,12 +1138,14 @@ void NodeDB::resetNodes(bool keepFavorites)
 {
     if (!config.position.fixed_position)
         clearLocalPosition();
+    NodeNum ourNum = getNodeNum();
     numMeshNodes = 1;
     if (keepFavorites) {
         LOG_INFO("Clearing node database - preserving favorites");
         for (size_t i = 0; i < meshNodes->size(); i++) {
             meshtastic_NodeInfoLite &node = meshNodes->at(i);
-            if (i > 0 && !node.is_favorite) {
+            if (i > 0 && !nodeInfoLiteIsFavorite(&node)) {
+                eraseNodeSatellites(node.num);
                 node = meshtastic_NodeInfoLite();
             } else {
                 numMeshNodes += 1;
@@ -1032,8 +1153,14 @@ void NodeDB::resetNodes(bool keepFavorites)
         };
     } else {
         LOG_INFO("Clearing node database - removing favorites");
+        for (size_t i = 1; i < meshNodes->size(); i++) {
+            const NodeNum gone = meshNodes->at(i).num;
+            if (gone)
+                eraseNodeSatellites(gone);
+        }
         std::fill(nodeDatabase.nodes.begin() + 1, nodeDatabase.nodes.end(), meshtastic_NodeInfoLite());
     }
+    (void)ourNum;
     devicestate.has_rx_text_message = false;
     devicestate.has_rx_waypoint = false;
     saveNodeDatabaseToDisk();
@@ -1054,36 +1181,173 @@ void NodeDB::removeNodeByNum(NodeNum nodeNum)
     numMeshNodes -= removed;
     std::fill(nodeDatabase.nodes.begin() + numMeshNodes, nodeDatabase.nodes.begin() + numMeshNodes + 1,
               meshtastic_NodeInfoLite());
+    if (removed)
+        eraseNodeSatellites(nodeNum);
     LOG_DEBUG("NodeDB::removeNodeByNum purged %d entries. Save changes", removed);
     saveNodeDatabaseToDisk();
 }
 
 void NodeDB::clearLocalPosition()
 {
-    meshtastic_NodeInfoLite *node = getMeshNode(nodeDB->getNodeNum());
-    node->position.latitude_i = 0;
-    node->position.longitude_i = 0;
-    node->position.altitude = 0;
-    node->position.time = 0;
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    concurrency::LockGuard guard(&satelliteMutex);
+    nodePositions.erase(getNodeNum());
+#endif
     setLocalPosition(meshtastic_Position_init_default);
     localPositionUpdatedSinceBoot = false;
 }
 
+bool NodeDB::copyNodePosition(NodeNum n, meshtastic_PositionLite &out) const
+{
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    (void)n;
+    (void)out;
+    return false;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return copySatelliteEntry(nodePositions, n, out);
+#endif
+}
+
+bool NodeDB::copyNodeTelemetry(NodeNum n, meshtastic_DeviceMetrics &out) const
+{
+#if MESHTASTIC_EXCLUDE_TELEMETRYDB
+    (void)n;
+    (void)out;
+    return false;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return copySatelliteEntry(nodeTelemetry, n, out);
+#endif
+}
+
+bool NodeDB::copyNodeEnvironment(NodeNum n, meshtastic_EnvironmentMetrics &out) const
+{
+#if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    (void)n;
+    (void)out;
+    return false;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return copySatelliteEntry(nodeEnvironment, n, out);
+#endif
+}
+
+bool NodeDB::copyNodeStatus(NodeNum n, meshtastic_StatusMessage &out) const
+{
+#if MESHTASTIC_EXCLUDE_STATUSDB
+    (void)n;
+    (void)out;
+    return false;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return copySatelliteEntry(nodeStatus, n, out);
+#endif
+}
+
+std::vector<NodeNum> NodeDB::snapshotPositionNodeNums(NodeNum exclude) const
+{
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    (void)exclude;
+    return {};
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return snapshotSatelliteNodeNums(nodePositions, exclude);
+#endif
+}
+
+std::vector<NodeNum> NodeDB::snapshotTelemetryNodeNums(NodeNum exclude) const
+{
+#if MESHTASTIC_EXCLUDE_TELEMETRYDB
+    (void)exclude;
+    return {};
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return snapshotSatelliteNodeNums(nodeTelemetry, exclude);
+#endif
+}
+
+std::vector<NodeNum> NodeDB::snapshotEnvironmentNodeNums(NodeNum exclude) const
+{
+#if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    (void)exclude;
+    return {};
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return snapshotSatelliteNodeNums(nodeEnvironment, exclude);
+#endif
+}
+
+std::vector<NodeNum> NodeDB::snapshotStatusNodeNums(NodeNum exclude) const
+{
+#if MESHTASTIC_EXCLUDE_STATUSDB
+    (void)exclude;
+    return {};
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    return snapshotSatelliteNodeNums(nodeStatus, exclude);
+#endif
+}
+
+void NodeDB::setNodeStatus(NodeNum n, const meshtastic_StatusMessage &status)
+{
+#if MESHTASTIC_EXCLUDE_STATUSDB
+    (void)n;
+    (void)status;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    nodeStatus[n] = status;
+#endif
+}
+
+void NodeDB::touchNodePositionTime(NodeNum n, uint32_t time)
+{
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    (void)n;
+    (void)time;
+#else
+    concurrency::LockGuard guard(&satelliteMutex);
+    nodePositions[n].time = time;
+#endif
+}
+
+void NodeDB::eraseNodeSatellites(NodeNum n)
+{
+    concurrency::LockGuard guard(&satelliteMutex);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    nodePositions.erase(n);
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    nodeTelemetry.erase(n);
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    nodeEnvironment.erase(n);
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    nodeStatus.erase(n);
+#endif
+}
+
 void NodeDB::cleanupMeshDB()
 {
     int newPos = 0, removed = 0;
     for (int i = 0; i < numMeshNodes; i++) {
-        if (meshNodes->at(i).has_user) {
-            if (meshNodes->at(i).user.public_key.size > 0) {
-                if (memfll(meshNodes->at(i).user.public_key.bytes, 0, meshNodes->at(i).user.public_key.size)) {
-                    meshNodes->at(i).user.public_key.size = 0;
+        meshtastic_NodeInfoLite &n = meshNodes->at(i);
+        if (nodeInfoLiteHasUser(&n)) {
+            if (n.public_key.size > 0) {
+                if (memfll(n.public_key.bytes, 0, n.public_key.size)) {
+                    n.public_key.size = 0;
                 }
             }
             if (newPos != i)
-                meshNodes->at(newPos++) = meshNodes->at(i);
+                meshNodes->at(newPos++) = n;
             else
                 newPos++;
         } else {
+            // No user info - drop this node and its satellites
+            const NodeNum gone = n.num;
+            if (gone)
+                eraseNodeSatellites(gone);
             removed++;
         }
     }
@@ -1141,14 +1405,22 @@ void NodeDB::pickNewNodeNum()
         nodeNum = (ourMacAddr[2] << 24) | (ourMacAddr[3] << 16) | (ourMacAddr[4] << 8) | ourMacAddr[5];
     }
 
+    // Identity check via public key (or "empty slot?" when no keys yet);
+    // macaddr no longer lives on the slim header.
+    auto isOurOwnEntry = [&](const meshtastic_NodeInfoLite *n) -> bool {
+        if (!n)
+            return false;
+        if (owner.public_key.size == 32 && n->public_key.size == 32)
+            return memcmp(n->public_key.bytes, owner.public_key.bytes, 32) == 0;
+        return !nodeInfoLiteHasUser(n);
+    };
+
     meshtastic_NodeInfoLite *found;
-    while (((found = getMeshNode(nodeNum)) && memcmp(found->user.macaddr, ourMacAddr, sizeof(ourMacAddr)) != 0) ||
+    while (((found = getMeshNode(nodeNum)) && !isOurOwnEntry(found)) ||
            (nodeNum == NODENUM_BROADCAST || nodeNum < NUM_RESERVED)) {
         NodeNum candidate = random(NUM_RESERVED, LONG_MAX); // try a new random choice
         if (found)
-            LOG_WARN("NOTE! Our desired nodenum 0x%x is invalid or in use, by MAC ending in 0x%02x%02x vs our 0x%02x%02x, so "
-                     "trying for 0x%x",
-                     nodeNum, found->user.macaddr[4], found->user.macaddr[5], ourMacAddr[4], ourMacAddr[5], candidate);
+            LOG_WARN("NOTE! Our desired nodenum 0x%x is invalid or in use, picking 0x%x", nodeNum, candidate);
         nodeNum = candidate;
     }
     LOG_DEBUG("Use nodenum 0x%x ", nodeNum);
@@ -1169,7 +1441,8 @@ LoadFileResult NodeDB::loadProto(const char *filename, size_t protoSize, size_t
     if (f) {
         LOG_INFO("Load %s", filename);
         pb_istream_t stream = {&readcb, &f, protoSize};
-        if (fields != &meshtastic_NodeDatabase_msg) // contains a vector object
+        if (fields != &meshtastic_NodeDatabase_msg &&
+            fields != &meshtastic_NodeDatabase_Legacy_msg) // both NodeDatabase descriptors contain std::vector members
             memset(dest_struct, 0, objSize);
         if (!pb_decode(&stream, fields, dest_struct)) {
             LOG_ERROR("Error: can't decode protobuf %s", PB_GET_ERROR(&stream));
@@ -1242,9 +1515,57 @@ void NodeDB::loadFromDisk()
     if (nodeDatabase.version < DEVICESTATE_MIN_VER) {
         LOG_WARN("NodeDatabase %d is old, discard", nodeDatabase.version);
         installDefaultNodeDatabase();
+    } else if (nodeDatabase.version < DEVICESTATE_CUR_VER) {
+        if (migrateLegacyNodeDatabase())
+            saveNodeDatabaseToDisk();
+        else
+            installDefaultNodeDatabase();
     } else {
         meshNodes = &nodeDatabase.nodes;
         numMeshNodes = nodeDatabase.nodes.size();
+        // Hydrate the satellite maps; the on-disk vectors stay empty in steady
+        // state and are repopulated only at save time.
+        concurrency::LockGuard guard(&satelliteMutex);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+        nodePositions.clear();
+        nodePositions.reserve(nodeDatabase.positions.size());
+        for (const auto &entry : nodeDatabase.positions) {
+            if (entry.has_position)
+                nodePositions[entry.num] = entry.position;
+        }
+        nodeDatabase.positions.clear();
+        nodeDatabase.positions.shrink_to_fit();
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+        nodeTelemetry.clear();
+        nodeTelemetry.reserve(nodeDatabase.telemetry.size());
+        for (const auto &entry : nodeDatabase.telemetry) {
+            if (entry.has_device_metrics)
+                nodeTelemetry[entry.num] = entry.device_metrics;
+        }
+        nodeDatabase.telemetry.clear();
+        nodeDatabase.telemetry.shrink_to_fit();
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+        nodeEnvironment.clear();
+        nodeEnvironment.reserve(nodeDatabase.environment.size());
+        for (const auto &entry : nodeDatabase.environment) {
+            if (entry.has_environment_metrics)
+                nodeEnvironment[entry.num] = entry.environment_metrics;
+        }
+        nodeDatabase.environment.clear();
+        nodeDatabase.environment.shrink_to_fit();
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+        nodeStatus.clear();
+        nodeStatus.reserve(nodeDatabase.status.size());
+        for (const auto &entry : nodeDatabase.status) {
+            if (entry.has_status)
+                nodeStatus[entry.num] = entry.status;
+        }
+        nodeDatabase.status.clear();
+        nodeDatabase.status.shrink_to_fit();
+#endif
         LOG_INFO("Loaded saved nodedatabase version %d, with nodes count: %d", nodeDatabase.version, nodeDatabase.nodes.size());
     }
 
@@ -1272,16 +1593,16 @@ void NodeDB::loadFromDisk()
 
         // Attempt recovery of owner fields from our own NodeDB entry if available.
         meshtastic_NodeInfoLite *us = getMeshNode(getNodeNum());
-        if (us && us->has_user) {
+        if (nodeInfoLiteHasUser(us)) {
             LOG_WARN("Restoring owner fields (long_name/short_name/is_licensed/is_unmessagable) from NodeDB for our node 0x%08x",
                      us->num);
-            memcpy(owner.long_name, us->user.long_name, sizeof(owner.long_name));
+            memcpy(owner.long_name, us->long_name, sizeof(owner.long_name));
             owner.long_name[sizeof(owner.long_name) - 1] = '\0';
-            memcpy(owner.short_name, us->user.short_name, sizeof(owner.short_name));
+            memcpy(owner.short_name, us->short_name, sizeof(owner.short_name));
             owner.short_name[sizeof(owner.short_name) - 1] = '\0';
-            owner.is_licensed = us->user.is_licensed;
-            owner.has_is_unmessagable = us->user.has_is_unmessagable;
-            owner.is_unmessagable = us->user.is_unmessagable;
+            owner.is_licensed = nodeInfoLiteIsLicensed(us);
+            owner.has_is_unmessagable = nodeInfoLiteHasIsUnmessagable(us);
+            owner.is_unmessagable = nodeInfoLiteIsUnmessagable(us);
 
             // Save the recovered owner to device state on disk
             saveToDisk(SEGMENT_DEVICESTATE);
@@ -1528,9 +1849,75 @@ bool NodeDB::saveNodeDatabaseToDisk()
     FSCom.mkdir("/prefs");
     spiLock->unlock();
 #endif
+    // Project the maps into the on-disk vectors just before encoding; cleared
+    // again on the way out so we don't carry duplicate state.
+    concurrency::LockGuard guard(&satelliteMutex);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    nodeDatabase.positions.clear();
+    nodeDatabase.positions.reserve(nodePositions.size());
+    for (const auto &kv : nodePositions) {
+        meshtastic_NodePositionEntry entry = meshtastic_NodePositionEntry_init_default;
+        entry.num = kv.first;
+        entry.has_position = true;
+        entry.position = kv.second;
+        nodeDatabase.positions.push_back(entry);
+    }
+#else
+    nodeDatabase.positions.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    nodeDatabase.telemetry.clear();
+    nodeDatabase.telemetry.reserve(nodeTelemetry.size());
+    for (const auto &kv : nodeTelemetry) {
+        meshtastic_NodeTelemetryEntry entry = meshtastic_NodeTelemetryEntry_init_default;
+        entry.num = kv.first;
+        entry.has_device_metrics = true;
+        entry.device_metrics = kv.second;
+        nodeDatabase.telemetry.push_back(entry);
+    }
+#else
+    nodeDatabase.telemetry.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    nodeDatabase.environment.clear();
+    nodeDatabase.environment.reserve(nodeEnvironment.size());
+    for (const auto &kv : nodeEnvironment) {
+        meshtastic_NodeEnvironmentEntry entry = meshtastic_NodeEnvironmentEntry_init_default;
+        entry.num = kv.first;
+        entry.has_environment_metrics = true;
+        entry.environment_metrics = kv.second;
+        nodeDatabase.environment.push_back(entry);
+    }
+#else
+    nodeDatabase.environment.clear();
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    nodeDatabase.status.clear();
+    nodeDatabase.status.reserve(nodeStatus.size());
+    for (const auto &kv : nodeStatus) {
+        meshtastic_NodeStatusEntry entry = meshtastic_NodeStatusEntry_init_default;
+        entry.num = kv.first;
+        entry.has_status = true;
+        entry.status = kv.second;
+        nodeDatabase.status.push_back(entry);
+    }
+#else
+    nodeDatabase.status.clear();
+#endif
+
     size_t nodeDatabaseSize;
     pb_get_encoded_size(&nodeDatabaseSize, meshtastic_NodeDatabase_fields, &nodeDatabase);
-    return saveProto(nodeDatabaseFileName, nodeDatabaseSize, &meshtastic_NodeDatabase_msg, &nodeDatabase, false);
+    bool ok = saveProto(nodeDatabaseFileName, nodeDatabaseSize, &meshtastic_NodeDatabase_msg, &nodeDatabase, false);
+
+    nodeDatabase.positions.clear();
+    nodeDatabase.positions.shrink_to_fit();
+    nodeDatabase.telemetry.clear();
+    nodeDatabase.telemetry.shrink_to_fit();
+    nodeDatabase.environment.clear();
+    nodeDatabase.environment.shrink_to_fit();
+    nodeDatabase.status.clear();
+    nodeDatabase.status.shrink_to_fit();
+    return ok;
 }
 
 bool NodeDB::saveToDiskNoRetry(int saveWhat)
@@ -1699,7 +2086,7 @@ size_t NodeDB::getNumOnlineMeshNodes(bool localOnly)
 
     // FIXME this implementation is kinda expensive
     for (int i = 0; i < numMeshNodes; i++) {
-        if (localOnly && meshNodes->at(i).via_mqtt)
+        if (localOnly && nodeInfoLiteViaMqtt(&meshNodes->at(i)))
             continue;
         if (sinceLastSeen(&meshNodes->at(i)) < NUM_ONLINE_SECS)
             numseen++;
@@ -1736,62 +2123,94 @@ void NodeDB::updatePosition(uint32_t nodeId, const meshtastic_Position &p, RxSou
         return;
     }
 
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    // Build flag opted out: header still tracks last_heard via updateFrom; we
+    // simply don't cache the position payload anywhere on this device.
     if (src == RX_SRC_LOCAL) {
-        // Local packet, fully authoritative
-        LOG_INFO("updatePosition LOCAL pos@%x time=%u lat=%d lon=%d alt=%d", p.timestamp, p.time, p.latitude_i, p.longitude_i,
+        LOG_INFO("updatePosition LOCAL (PositionDB excluded) time=%u lat=%d lon=%d alt=%d", p.time, p.latitude_i, p.longitude_i,
                  p.altitude);
-
         setLocalPosition(p);
-        info->position = TypeConversions::ConvertToPositionLite(p);
-    } else if ((p.time > 0) && !p.latitude_i && !p.longitude_i && !p.timestamp && !p.location_source) {
-        // FIXME SPECIAL TIME SETTING PACKET FROM EUD TO RADIO
-        // (stop-gap fix for issue #900)
-        LOG_DEBUG("updatePosition SPECIAL time setting time=%u", p.time);
-        info->position.time = p.time;
-    } else {
-        // Be careful to only update fields that have been set by the REMOTE sender
-        // A lot of position reports don't have time populated.  In that case, be careful to not blow away the time we
-        // recorded based on the packet rxTime
-        //
-        // FIXME perhaps handle RX_SRC_USER separately?
-        LOG_INFO("updatePosition REMOTE node=0x%x time=%u lat=%d lon=%d", nodeId, p.time, p.latitude_i, p.longitude_i);
-
-        // First, back up fields that we want to protect from overwrite
-        uint32_t tmp_time = info->position.time;
-
-        // Next, update atomically
-        info->position = TypeConversions::ConvertToPositionLite(p);
-
-        // Last, restore any fields that may have been overwritten
-        if (!info->position.time)
-            info->position.time = tmp_time;
     }
-    info->has_position = true;
+    (void)nodeId;
+    updateGUIforNode = info;
+    notifyObservers(true);
+#else
+    {
+        concurrency::LockGuard guard(&satelliteMutex);
+        meshtastic_PositionLite &slot = nodePositions[nodeId]; // creates default-zero entry if missing
+
+        if (src == RX_SRC_LOCAL) {
+            // Local packet, fully authoritative
+            LOG_INFO("updatePosition LOCAL pos@%x time=%u lat=%d lon=%d alt=%d", p.timestamp, p.time, p.latitude_i, p.longitude_i,
+                     p.altitude);
+
+            setLocalPosition(p);
+            slot = TypeConversions::ConvertToPositionLite(p);
+        } else if ((p.time > 0) && !p.latitude_i && !p.longitude_i && !p.timestamp && !p.location_source) {
+            // FIXME SPECIAL TIME SETTING PACKET FROM EUD TO RADIO
+            // (stop-gap fix for issue #900)
+            LOG_DEBUG("updatePosition SPECIAL time setting time=%u", p.time);
+            slot.time = p.time;
+        } else {
+            // Be careful to only update fields that have been set by the REMOTE sender
+            // A lot of position reports don't have time populated.  In that case, be careful to not blow away the time we
+            // recorded based on the packet rxTime
+            //
+            // FIXME perhaps handle RX_SRC_USER separately?
+            LOG_INFO("updatePosition REMOTE node=0x%x time=%u lat=%d lon=%d", nodeId, p.time, p.latitude_i, p.longitude_i);
+
+            // First, back up fields that we want to protect from overwrite
+            uint32_t tmp_time = slot.time;
+
+            // Next, update atomically
+            slot = TypeConversions::ConvertToPositionLite(p);
+
+            // Last, restore any fields that may have been overwritten
+            if (!slot.time)
+                slot.time = tmp_time;
+        }
+    }
     updateGUIforNode = info;
     notifyObservers(true); // Force an update whether or not our node counts have changed
+#endif
 }
 
-/** Update telemetry info for this node based on received metrics
- *  We only care about device telemetry here
+/** Update telemetry info for this node based on received metrics. Stores
+ *  device_metrics and environment_metrics into their respective satellite
+ *  maps; other variants (air_quality, power, local_stats, health) are
+ *  intentionally not retained per-node.
  */
 void NodeDB::updateTelemetry(uint32_t nodeId, const meshtastic_Telemetry &t, RxSource src)
 {
     meshtastic_NodeInfoLite *info = getOrCreateMeshNode(nodeId);
-    // Environment metrics should never go to NodeDb but we'll safegaurd anyway
-    if (!info || t.which_variant != meshtastic_Telemetry_device_metrics_tag) {
+    if (!info)
         return;
-    }
 
-    if (src == RX_SRC_LOCAL) {
-        // Local packet, fully authoritative
-        LOG_DEBUG("updateTelemetry LOCAL");
+    if (t.which_variant == meshtastic_Telemetry_device_metrics_tag) {
+        if (src == RX_SRC_LOCAL) {
+            LOG_DEBUG("updateTelemetry LOCAL device");
+        } else {
+            LOG_DEBUG("updateTelemetry REMOTE device node=0x%x", nodeId);
+        }
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+        concurrency::LockGuard guard(&satelliteMutex);
+        nodeTelemetry[nodeId] = t.variant.device_metrics;
+#endif
+    } else if (t.which_variant == meshtastic_Telemetry_environment_metrics_tag) {
+        if (src == RX_SRC_LOCAL) {
+            LOG_DEBUG("updateTelemetry LOCAL env");
+        } else {
+            LOG_DEBUG("updateTelemetry REMOTE env node=0x%x", nodeId);
+        }
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+        concurrency::LockGuard guard(&satelliteMutex);
+        nodeEnvironment[nodeId] = t.variant.environment_metrics;
+#endif
     } else {
-        LOG_DEBUG("updateTelemetry REMOTE node=0x%x ", nodeId);
+        return; // air_quality / power / local_stats / health: not stored per-node
     }
-    info->device_metrics = t.variant.device_metrics;
-    info->has_device_metrics = true;
     updateGUIforNode = info;
-    notifyObservers(true); // Force an update whether or not our node counts have changed
+    notifyObservers(true);
 }
 
 /**
@@ -1806,24 +2225,22 @@ void NodeDB::addFromContact(meshtastic_SharedContact contact)
     // If the local node has this node marked as manually verified
     // and the client does not, do not allow the client to update the
     // saved public key.
-    if ((info->bitfield & NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK) && !contact.manually_verified) {
-        if (contact.user.public_key.size != info->user.public_key.size ||
-            memcmp(contact.user.public_key.bytes, info->user.public_key.bytes, info->user.public_key.size) != 0) {
+    if (nodeInfoLiteIsKeyManuallyVerified(info) && !contact.manually_verified) {
+        if (contact.user.public_key.size != info->public_key.size ||
+            memcmp(contact.user.public_key.bytes, info->public_key.bytes, info->public_key.size) != 0) {
             return;
         }
     }
     info->num = contact.node_num;
-    info->has_user = true;
-    info->user = TypeConversions::ConvertToUserLite(contact.user);
+    TypeConversions::CopyUserToNodeInfoLite(info, contact.user);
     if (contact.should_ignore) {
         // If should_ignore is set,
         // we need to clear the public key and other cruft, in addition to setting the node as ignored
-        info->is_ignored = true;
-        info->is_favorite = false;
-        info->has_device_metrics = false;
-        info->has_position = false;
-        info->user.public_key.size = 0;
-        memset(info->user.public_key.bytes, 0, sizeof(info->user.public_key.bytes));
+        nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
+        nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_FAVORITE_MASK, false);
+        eraseNodeSatellites(contact.node_num);
+        info->public_key.size = 0;
+        memset(info->public_key.bytes, 0, sizeof(info->public_key.bytes));
     } else {
         /* Clients are sending add_contact before every text message DM (because clients may hold a larger node database with
          * public keys than the radio holds). However, we don't want to update last_heard just because we sent someone a DM!
@@ -1842,12 +2259,12 @@ void NodeDB::addFromContact(meshtastic_SharedContact contact)
         } else {
             // Normal case: set is_favorite to prevent expiration.
             // last_heard will remain as-is (or remain 0 if this entry wasn't in the nodeDB).
-            info->is_favorite = true;
+            nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
         }
 
         // As the clients will begin sending the contact with DMs, we want to strictly check if the node is manually verified
         if (contact.manually_verified) {
-            info->bitfield |= NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK;
+            nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK, true);
         }
         // Mark the node's key as manually verified to indicate trustworthiness.
         updateGUIforNode = info;
@@ -1881,15 +2298,15 @@ bool NodeDB::updateUser(uint32_t nodeId, meshtastic_User &p, uint8_t channelInde
                 meshtastic_ClientNotification *cn = clientNotificationPool.allocZeroed();
                 cn->level = meshtastic_LogRecord_Level_WARNING;
                 cn->time = getValidTime(RTCQualityFromNet);
-                sprintf(cn->message, warning, p.long_name);
+                snprintf(cn->message, sizeof(cn->message), warning, p.long_name);
                 service->sendClientNotification(cn);
             }
             return false;
         }
     }
-    if (info->user.public_key.size == 32) { // if we have a key for this user already, don't overwrite with a new one
+    if (info->public_key.size == 32) { // if we have a key for this user already, don't overwrite with a new one
         // if the key doesn't match, don't update nodeDB at all.
-        if (p.public_key.size != 32 || (memcmp(p.public_key.bytes, info->user.public_key.bytes, 32) != 0)) {
+        if (p.public_key.size != 32 || (memcmp(p.public_key.bytes, info->public_key.bytes, 32) != 0)) {
             LOG_WARN("Public Key mismatch, dropping NodeInfo");
             return false;
         }
@@ -1902,19 +2319,22 @@ bool NodeDB::updateUser(uint32_t nodeId, meshtastic_User &p, uint8_t channelInde
     // Always ensure user.id is derived from nodeId, regardless of what was received
     snprintf(p.id, sizeof(p.id), "!%08x", nodeId);
 
-    // Both of info->user and p start as filled with zero so I think this is okay
-    auto lite = TypeConversions::ConvertToUserLite(p);
-    bool changed = memcmp(&info->user, &lite, sizeof(info->user)) || (info->channel != channelIndex);
+    meshtastic_NodeInfoLite before = *info;
+    TypeConversions::CopyUserToNodeInfoLite(info, p);
+    bool changed =
+        (memcmp(before.long_name, info->long_name, sizeof(info->long_name)) != 0) ||
+        (memcmp(before.short_name, info->short_name, sizeof(info->short_name)) != 0) || (before.hw_model != info->hw_model) ||
+        (before.role != info->role) || (before.public_key.size != info->public_key.size) ||
+        (info->public_key.size > 0 && memcmp(before.public_key.bytes, info->public_key.bytes, info->public_key.size) != 0) ||
+        (before.bitfield != info->bitfield) || (info->channel != channelIndex);
 
-    info->user = lite;
-    if (info->user.public_key.size == 32) {
-        printBytes("Saved Pubkey: ", info->user.public_key.bytes, 32);
+    if (info->public_key.size == 32) {
+        printBytes("Saved Pubkey: ", info->public_key.bytes, 32);
     }
     if (nodeId != getNodeNum())
         info->channel = channelIndex; // Set channel we need to use to reach this node (but don't set our own channel)
-    LOG_DEBUG("Update changed=%d user %s/%s, id=0x%08x, channel=%d", changed, info->user.long_name, info->user.short_name, nodeId,
+    LOG_DEBUG("Update changed=%d user %s/%s, id=0x%08x, channel=%d", changed, info->long_name, info->short_name, nodeId,
               info->channel);
-    info->has_user = true;
 
     if (changed) {
         updateGUIforNode = info;
@@ -1956,7 +2376,8 @@ void NodeDB::updateFrom(const meshtastic_MeshPacket &mp)
         if (mp.rx_snr)
             info->snr = mp.rx_snr; // keep the most recent SNR we received for this node.
 
-        info->via_mqtt = mp.via_mqtt; // Store if we received this packet via MQTT
+        nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_VIA_MQTT_MASK,
+                           mp.via_mqtt); // Store if we received this packet via MQTT
 
         // If hopStart was set and there wasn't someone messing with the limit in the middle, add hopsAway
         const int8_t hopsAway = getHopsAway(mp);
@@ -1971,8 +2392,8 @@ void NodeDB::updateFrom(const meshtastic_MeshPacket &mp)
 void NodeDB::set_favorite(bool is_favorite, uint32_t nodeId)
 {
     meshtastic_NodeInfoLite *lite = getMeshNode(nodeId);
-    if (lite && lite->is_favorite != is_favorite) {
-        lite->is_favorite = is_favorite;
+    if (lite && nodeInfoLiteIsFavorite(lite) != is_favorite) {
+        nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_IS_FAVORITE_MASK, is_favorite);
         sortMeshDB();
         saveNodeDatabaseToDisk();
     }
@@ -1986,10 +2407,10 @@ bool NodeDB::isFavorite(uint32_t nodeId)
     if (nodeId == NODENUM_BROADCAST)
         return false;
 
-    meshtastic_NodeInfoLite *lite = getMeshNode(nodeId);
+    const meshtastic_NodeInfoLite *lite = getMeshNode(nodeId);
 
     if (lite) {
-        return lite->is_favorite;
+        return nodeInfoLiteIsFavorite(lite);
     }
     return false;
 }
@@ -2014,14 +2435,14 @@ bool NodeDB::isFromOrToFavoritedNode(const meshtastic_MeshPacket &p)
         lite = &meshNodes->at(i);
 
         if (lite->num == p.from) {
-            if (lite->is_favorite)
+            if (nodeInfoLiteIsFavorite(lite))
                 return true;
 
             seenFrom = true;
         }
 
         if (lite->num == p.to) {
-            if (lite->is_favorite)
+            if (nodeInfoLiteIsFavorite(lite))
                 return true;
 
             seenTo = true;
@@ -2057,10 +2478,10 @@ void NodeDB::sortMeshDB()
                     // TODO: Look for at(i-1) also matching own node num, and throw the DB in the trash
                     std::swap(meshNodes->at(i), meshNodes->at(i - 1));
                     changed = true;
-                } else if (meshNodes->at(i).is_favorite && !meshNodes->at(i - 1).is_favorite) {
+                } else if (nodeInfoLiteIsFavorite(&meshNodes->at(i)) && !nodeInfoLiteIsFavorite(&meshNodes->at(i - 1))) {
                     std::swap(meshNodes->at(i), meshNodes->at(i - 1));
                     changed = true;
-                } else if (!meshNodes->at(i).is_favorite && meshNodes->at(i - 1).is_favorite) {
+                } else if (!nodeInfoLiteIsFavorite(&meshNodes->at(i)) && nodeInfoLiteIsFavorite(&meshNodes->at(i - 1))) {
                     // noop
                 } else if (meshNodes->at(i).last_heard > meshNodes->at(i - 1).last_heard) {
                     std::swap(meshNodes->at(i), meshNodes->at(i - 1));
@@ -2120,17 +2541,18 @@ meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
             int oldestIndex = -1;
             int oldestBoringIndex = -1;
             for (int i = 1; i < numMeshNodes; i++) {
+                const meshtastic_NodeInfoLite *cand = &meshNodes->at(i);
+                const bool isFavoriteNode = nodeInfoLiteIsFavorite(cand);
+                const bool isIgnored = nodeInfoLiteIsIgnored(cand);
+                const bool isVerified = nodeInfoLiteIsKeyManuallyVerified(cand);
                 // Simply the oldest non-favorite, non-ignored, non-verified node
-                if (!meshNodes->at(i).is_favorite && !meshNodes->at(i).is_ignored &&
-                    !(meshNodes->at(i).bitfield & NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK) &&
-                    meshNodes->at(i).last_heard < oldest) {
-                    oldest = meshNodes->at(i).last_heard;
+                if (!isFavoriteNode && !isIgnored && !isVerified && cand->last_heard < oldest) {
+                    oldest = cand->last_heard;
                     oldestIndex = i;
                 }
                 // The oldest "boring" node
-                if (!meshNodes->at(i).is_favorite && !meshNodes->at(i).is_ignored && meshNodes->at(i).user.public_key.size == 0 &&
-                    meshNodes->at(i).last_heard < oldestBoring) {
-                    oldestBoring = meshNodes->at(i).last_heard;
+                if (!isFavoriteNode && !isIgnored && cand->public_key.size == 0 && cand->last_heard < oldestBoring) {
+                    oldestBoring = cand->last_heard;
                     oldestBoringIndex = i;
                 }
             }
@@ -2140,6 +2562,7 @@ meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
             }
 
             if (oldestIndex != -1) {
+                eraseNodeSatellites(meshNodes->at(oldestIndex).num);
                 // Shove the remaining nodes down the chain
                 for (int i = oldestIndex; i < numMeshNodes - 1; i++) {
                     meshNodes->at(i) = meshNodes->at(i + 1);
@@ -2163,18 +2586,23 @@ meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
 /// valid lat/lon
 bool NodeDB::hasValidPosition(const meshtastic_NodeInfoLite *n)
 {
-    return n->has_position && (n->position.latitude_i != 0 || n->position.longitude_i != 0);
+    if (!n)
+        return false;
+    if (n->num == getNodeNum()) {
+        return localPosition.latitude_i != 0 || localPosition.longitude_i != 0;
+    }
+    meshtastic_PositionLite pos;
+    return copyNodePosition(n->num, pos) && (pos.latitude_i != 0 || pos.longitude_i != 0);
 }
 
 /// If we have a node / user and they report is_licensed = true
 /// we consider them licensed
 UserLicenseStatus NodeDB::getLicenseStatus(uint32_t nodeNum)
 {
-    meshtastic_NodeInfoLite *info = getMeshNode(nodeNum);
-    if (!info || !info->has_user) {
+    const meshtastic_NodeInfoLite *info = getMeshNode(nodeNum);
+    if (!nodeInfoLiteHasUser(info))
         return UserLicenseStatus::NotKnown;
-    }
-    return info->user.is_licensed ? UserLicenseStatus::Licensed : UserLicenseStatus::NotLicensed;
+    return nodeInfoLiteIsLicensed(info) ? UserLicenseStatus::Licensed : UserLicenseStatus::NotLicensed;
 }
 
 #if !defined(MESHTASTIC_EXCLUDE_PKI)

src/mesh/NodeDB.h

@@ -6,10 +6,12 @@
 #include <assert.h>
 #include <pb_encode.h>
 #include <string>
+#include <unordered_map>
 #include <vector>
 
 #include "MeshTypes.h"
 #include "NodeStatus.h"
+#include "concurrency/Lock.h"
 #include "configuration.h"
 #include "mesh-pb-constants.h"
 #include "mesh/generated/meshtastic/mesh.pb.h" // For CriticalErrorCode
@@ -82,7 +84,9 @@ DeviceState versions used to be defined in the .proto file but really only this
 #define SEGMENT_CHANNELS 8
 #define SEGMENT_NODEDATABASE 16
 
-#define DEVICESTATE_CUR_VER 24
+#define DEVICESTATE_CUR_VER 25
+// Lowest on-disk version we still know how to load. v24 saves are migrated
+// at boot via the parallel deviceonly_legacy descriptor and re-saved as v25.
 #define DEVICESTATE_MIN_VER 24
 
 extern meshtastic_DeviceState devicestate;
@@ -166,6 +170,21 @@ class NodeDB
     Observable<const meshtastic::NodeStatus *> newStatus;
     pb_size_t numMeshNodes;
 
+    // Satellite per-NodeNum maps for data we used to inline into NodeInfoLite,
+    // gated by MESHTASTIC_EXCLUDE_*DB so STM32WL can omit them.
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    std::unordered_map<NodeNum, meshtastic_PositionLite> nodePositions;
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    std::unordered_map<NodeNum, meshtastic_DeviceMetrics> nodeTelemetry;
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    std::unordered_map<NodeNum, meshtastic_EnvironmentMetrics> nodeEnvironment;
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    std::unordered_map<NodeNum, meshtastic_StatusMessage> nodeStatus;
+#endif
+
     bool keyIsLowEntropy = false;
     bool hasWarned = false;
 
@@ -277,6 +296,43 @@ class NodeDB
     virtual meshtastic_NodeInfoLite *getMeshNode(NodeNum n);
     size_t getNumMeshNodes() { return numMeshNodes; }
 
+    // Thread-safe satellite-map accessors. Return false if absent or the
+    // corresponding DB is compiled out.
+    bool copyNodePosition(NodeNum n, meshtastic_PositionLite &out) const;
+    bool copyNodeTelemetry(NodeNum n, meshtastic_DeviceMetrics &out) const;
+    bool copyNodeEnvironment(NodeNum n, meshtastic_EnvironmentMetrics &out) const;
+    bool copyNodeStatus(NodeNum n, meshtastic_StatusMessage &out) const;
+    std::vector<NodeNum> snapshotPositionNodeNums(NodeNum exclude) const;
+    std::vector<NodeNum> snapshotTelemetryNodeNums(NodeNum exclude) const;
+    std::vector<NodeNum> snapshotEnvironmentNodeNums(NodeNum exclude) const;
+    std::vector<NodeNum> snapshotStatusNodeNums(NodeNum exclude) const;
+
+    void setNodeStatus(NodeNum n, const meshtastic_StatusMessage &status);
+    void touchNodePositionTime(NodeNum n, uint32_t time);
+
+    bool hasNodePosition(NodeNum n) const
+    {
+        meshtastic_PositionLite scratch;
+        return copyNodePosition(n, scratch);
+    }
+    bool hasNodeTelemetry(NodeNum n) const
+    {
+        meshtastic_DeviceMetrics scratch;
+        return copyNodeTelemetry(n, scratch);
+    }
+    bool hasNodeEnvironment(NodeNum n) const
+    {
+        meshtastic_EnvironmentMetrics scratch;
+        return copyNodeEnvironment(n, scratch);
+    }
+    bool hasNodeStatus(NodeNum n) const
+    {
+        meshtastic_StatusMessage scratch;
+        return copyNodeStatus(n, scratch);
+    }
+
+    void eraseNodeSatellites(NodeNum n);
+
     UserLicenseStatus getLicenseStatus(uint32_t nodeNum);
 
     size_t getMaxNodesAllocatedSize()
@@ -285,7 +341,11 @@ class NodeDB
         emptyNodeDatabase.version = DEVICESTATE_CUR_VER;
         size_t nodeDatabaseSize;
         pb_get_encoded_size(&nodeDatabaseSize, meshtastic_NodeDatabase_fields, &emptyNodeDatabase);
-        return nodeDatabaseSize + (MAX_NUM_NODES * meshtastic_NodeInfoLite_size);
+        // Always include satellite slots so backups from higher-cap peers
+        // decode without truncation, even when our build excludes the DBs.
+        return nodeDatabaseSize + (MAX_NUM_NODES * meshtastic_NodeInfoLite_size) +
+               (MAX_NUM_NODES * meshtastic_NodePositionEntry_size) + (MAX_NUM_NODES * meshtastic_NodeTelemetryEntry_size) +
+               (MAX_NUM_NODES * meshtastic_NodeEnvironmentEntry_size) + (MAX_NUM_NODES * meshtastic_NodeStatusEntry_size);
     }
 
     // returns true if the maximum number of nodes is reached or we are running low on memory
@@ -329,6 +389,7 @@ class NodeDB
     }
 
   private:
+    mutable concurrency::Lock satelliteMutex;
     bool duplicateWarned = false;
     bool localPositionUpdatedSinceBoot = false;
     uint32_t lastNodeDbSave = 0;    // when we last saved our db to flash
@@ -363,6 +424,11 @@ class NodeDB
     bool saveDeviceStateToDisk();
     bool saveNodeDatabaseToDisk();
     void sortMeshDB();
+
+    // Defined in NodeDBLegacyMigration.cpp. Decodes /prefs/nodes.proto via
+    // the legacy descriptor and copies entries into the v25 layout. Caller
+    // is responsible for save / install-default on the result.
+    bool migrateLegacyNodeDatabase();
 };
 
 extern NodeDB *nodeDB;
@@ -397,10 +463,74 @@ extern meshtastic_CriticalErrorCode error_code;
  * A numeric error address (nonzero if available)
  */
 extern uint32_t error_address;
+// Bit assignments for meshtastic_NodeInfoLite.bitfield.
 #define NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_SHIFT 0
-#define NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK (1 << NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_SHIFT)
+#define NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK (1u << NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_SHIFT)
 #define NODEINFO_BITFIELD_IS_MUTED_SHIFT 1
-#define NODEINFO_BITFIELD_IS_MUTED_MASK (1 << NODEINFO_BITFIELD_IS_MUTED_SHIFT)
+#define NODEINFO_BITFIELD_IS_MUTED_MASK (1u << NODEINFO_BITFIELD_IS_MUTED_SHIFT)
+#define NODEINFO_BITFIELD_VIA_MQTT_SHIFT 2
+#define NODEINFO_BITFIELD_VIA_MQTT_MASK (1u << NODEINFO_BITFIELD_VIA_MQTT_SHIFT)
+#define NODEINFO_BITFIELD_IS_FAVORITE_SHIFT 3
+#define NODEINFO_BITFIELD_IS_FAVORITE_MASK (1u << NODEINFO_BITFIELD_IS_FAVORITE_SHIFT)
+#define NODEINFO_BITFIELD_IS_IGNORED_SHIFT 4
+#define NODEINFO_BITFIELD_IS_IGNORED_MASK (1u << NODEINFO_BITFIELD_IS_IGNORED_SHIFT)
+#define NODEINFO_BITFIELD_HAS_USER_SHIFT 5
+#define NODEINFO_BITFIELD_HAS_USER_MASK (1u << NODEINFO_BITFIELD_HAS_USER_SHIFT)
+#define NODEINFO_BITFIELD_IS_LICENSED_SHIFT 6
+#define NODEINFO_BITFIELD_IS_LICENSED_MASK (1u << NODEINFO_BITFIELD_IS_LICENSED_SHIFT)
+#define NODEINFO_BITFIELD_IS_UNMESSAGABLE_SHIFT 7
+#define NODEINFO_BITFIELD_IS_UNMESSAGABLE_MASK (1u << NODEINFO_BITFIELD_IS_UNMESSAGABLE_SHIFT)
+#define NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_SHIFT 8
+#define NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_MASK (1u << NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_SHIFT)
+// Bits 9..31 reserved for future single-bit flags.
+
+// Convenience accessors so call sites read like the old struct fields.
+inline bool nodeInfoLiteHasUser(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_HAS_USER_MASK);
+}
+inline bool nodeInfoLiteViaMqtt(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_VIA_MQTT_MASK);
+}
+inline bool nodeInfoLiteIsFavorite(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_FAVORITE_MASK);
+}
+inline bool nodeInfoLiteIsIgnored(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_IGNORED_MASK);
+}
+inline bool nodeInfoLiteIsLicensed(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_LICENSED_MASK);
+}
+inline bool nodeInfoLiteHasIsUnmessagable(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_MASK);
+}
+inline bool nodeInfoLiteIsUnmessagable(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_UNMESSAGABLE_MASK);
+}
+inline bool nodeInfoLiteIsMuted(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK);
+}
+inline bool nodeInfoLiteIsKeyManuallyVerified(const meshtastic_NodeInfoLite *n)
+{
+    return n && (n->bitfield & NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK);
+}
+
+inline void nodeInfoLiteSetBit(meshtastic_NodeInfoLite *n, uint32_t mask, bool value)
+{
+    if (!n)
+        return;
+    if (value)
+        n->bitfield |= mask;
+    else
+        n->bitfield &= ~mask;
+}
 
 #define Module_Config_size                                                                                                       \
     (ModuleConfig_CannedMessageConfig_size + ModuleConfig_ExternalNotificationConfig_size + ModuleConfig_MQTTConfig_size +       \

src/mesh/NodeDBLegacyMigration.cpp

@@ -0,0 +1,129 @@
+// Migration from the legacy (pre-v25) NodeDatabase shape to the slim header +
+// satellite maps layout. Lives in its own translation unit so the bulk
+// fixed-shape decode + field-by-field copy doesn't clutter NodeDB.cpp.
+//
+// Caller (NodeDB::loadFromDisk) decides what to do with the result:
+//   - true  -> persist via saveNodeDatabaseToDisk()
+//   - false -> reset via installDefaultNodeDatabase()
+//
+// This file (and the deviceonly_legacy proto) can be removed once
+// DEVICESTATE_MIN_VER advances past 24.
+
+#include "NodeDB.h"
+#include "concurrency/LockGuard.h"
+#include "configuration.h"
+#include "mesh-pb-constants.h"
+#include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
+
+#include <algorithm>
+#include <cstring>
+#include <pb_decode.h>
+#include <pb_encode.h>
+
+bool meshtastic_NodeDatabase_Legacy_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field)
+{
+    const auto *iter = reinterpret_cast<const pb_field_iter_t *>(field);
+    if (ostream) {
+        const auto *vec = static_cast<const std::vector<meshtastic_NodeInfoLite_Legacy> *>(iter->pData);
+        for (auto item : *vec) {
+            if (!pb_encode_tag_for_field(ostream, iter))
+                return false;
+            if (!pb_encode_submessage(ostream, meshtastic_NodeInfoLite_Legacy_fields, &item))
+                return false;
+        }
+    }
+    if (istream) {
+        meshtastic_NodeInfoLite_Legacy node;
+        auto *vec = static_cast<std::vector<meshtastic_NodeInfoLite_Legacy> *>(iter->pData);
+        if (istream->bytes_left && pb_decode(istream, meshtastic_NodeInfoLite_Legacy_fields, &node))
+            vec->push_back(node);
+    }
+    return true;
+}
+
+bool NodeDB::migrateLegacyNodeDatabase()
+{
+    LOG_WARN("NodeDatabase v%u: migrating to v%u", nodeDatabase.version, DEVICESTATE_CUR_VER);
+
+    // _init_zero brace-inits the embedded std::vector via its explicit
+    // (size_type, allocator) ctor, so default-construct instead.
+    meshtastic_NodeDatabase_Legacy legacyDb{};
+    legacyDb.version = 0;
+    meshtastic_NodeDatabase_Legacy legacyEmpty{};
+    legacyEmpty.version = DEVICESTATE_CUR_VER;
+    size_t legacyEmptyEncoded = 0;
+    pb_get_encoded_size(&legacyEmptyEncoded, meshtastic_NodeDatabase_Legacy_fields, &legacyEmpty);
+    const size_t legacyBufSize = legacyEmptyEncoded + (MAX_NUM_NODES * meshtastic_NodeInfoLite_Legacy_size);
+    auto legacyState = loadProto(nodeDatabaseFileName, legacyBufSize, sizeof(meshtastic_NodeDatabase_Legacy),
+                                 &meshtastic_NodeDatabase_Legacy_msg, &legacyDb);
+    if (legacyState != LoadFileResult::LOAD_SUCCESS) {
+        LOG_ERROR("Failed to load NodeDatabase via legacy descriptor; installing default");
+        return false;
+    }
+
+    nodeDatabase.nodes.clear();
+    const size_t maxToMigrate = std::min<size_t>(legacyDb.nodes.size(), MAX_NUM_NODES);
+    nodeDatabase.nodes.reserve(maxToMigrate);
+    size_t posCount = 0, telCount = 0;
+    {
+        concurrency::LockGuard guard(&satelliteMutex);
+        for (size_t i = 0; i < maxToMigrate; ++i) {
+            const auto &legacy = legacyDb.nodes[i];
+            meshtastic_NodeInfoLite slim = meshtastic_NodeInfoLite_init_default;
+            slim.num = legacy.num;
+            slim.snr = legacy.snr;
+            slim.last_heard = legacy.last_heard;
+            slim.channel = legacy.channel;
+            slim.has_hops_away = legacy.has_hops_away;
+            slim.hops_away = legacy.hops_away;
+            slim.next_hop = legacy.next_hop;
+            slim.bitfield = legacy.bitfield;
+            if (legacy.via_mqtt)
+                slim.bitfield |= NODEINFO_BITFIELD_VIA_MQTT_MASK;
+            if (legacy.is_favorite)
+                slim.bitfield |= NODEINFO_BITFIELD_IS_FAVORITE_MASK;
+            if (legacy.is_ignored)
+                slim.bitfield |= NODEINFO_BITFIELD_IS_IGNORED_MASK;
+            if (legacy.has_user) {
+                slim.bitfield |= NODEINFO_BITFIELD_HAS_USER_MASK;
+                strncpy(slim.long_name, legacy.user.long_name, sizeof(slim.long_name));
+                slim.long_name[sizeof(slim.long_name) - 1] = '\0';
+                strncpy(slim.short_name, legacy.user.short_name, sizeof(slim.short_name));
+                slim.short_name[sizeof(slim.short_name) - 1] = '\0';
+                slim.hw_model = legacy.user.hw_model;
+                slim.role = legacy.user.role;
+                if (legacy.user.is_licensed)
+                    slim.bitfield |= NODEINFO_BITFIELD_IS_LICENSED_MASK;
+                slim.public_key.size = legacy.user.public_key.size;
+                memcpy(slim.public_key.bytes, legacy.user.public_key.bytes, sizeof(slim.public_key.bytes));
+                if (legacy.user.has_is_unmessagable) {
+                    slim.bitfield |= NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_MASK;
+                    if (legacy.user.is_unmessagable)
+                        slim.bitfield |= NODEINFO_BITFIELD_IS_UNMESSAGABLE_MASK;
+                }
+                // macaddr deprecated since 1.2.11 - dropped from slim header.
+            }
+            nodeDatabase.nodes.push_back(slim);
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+            if (legacy.has_position)
+                nodePositions[legacy.num] = legacy.position;
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+            if (legacy.has_device_metrics)
+                nodeTelemetry[legacy.num] = legacy.device_metrics;
+#endif
+        }
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+        posCount = nodePositions.size();
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+        telCount = nodeTelemetry.size();
+#endif
+    }
+    nodeDatabase.version = DEVICESTATE_CUR_VER;
+    meshNodes = &nodeDatabase.nodes;
+    numMeshNodes = nodeDatabase.nodes.size();
+    LOG_INFO("Migrated %u nodes from legacy -> v%u (positions: %u, telemetry: %u)", (unsigned)numMeshNodes, DEVICESTATE_CUR_VER,
+             (unsigned)posCount, (unsigned)telCount);
+    return true;
+}

src/mesh/PacketHistory.cpp

@@ -17,7 +17,7 @@
 #define VERBOSE_PACKET_HISTORY 0     // Set to 1 for verbose logging, 2 for heavy debugging
 #define PACKET_HISTORY_TRACE_AGING 1 // Set to 1 to enable logging of the age of re/used history slots
 
-PacketHistory::PacketHistory(uint32_t size) : recentPacketsCapacity(0), recentPackets(NULL) // Initialize members
+PacketHistory::PacketHistory(uint32_t size) : recentPacketsCapacity(0) // Initialize members
 {
     if (size < 4 || size > PACKETHISTORY_MAX) { // Copilot suggested - makes sense
         LOG_WARN("Packet History - Invalid size %d, using default %d", size, PACKETHISTORY_MAX);
@@ -34,7 +34,7 @@ PacketHistory::PacketHistory(uint32_t size) : recentPacketsCapacity(0), recentPa
 
     // Allocate memory for the recent packets array
     recentPacketsCapacity = size;
-    recentPackets = new PacketRecord[recentPacketsCapacity];
+    recentPackets.reset(new PacketRecord[recentPacketsCapacity]);
     if (!recentPackets) { // No logging here, console/log probably uninitialized yet.
         LOG_ERROR("Packet History - Memory allocation failed for size=%d entries / %d Bytes", size,
                   sizeof(PacketRecord) * recentPacketsCapacity);
@@ -43,33 +43,20 @@ PacketHistory::PacketHistory(uint32_t size) : recentPacketsCapacity(0), recentPa
     }
 
     // Initialize the recent packets array to zero
-    memset(recentPackets, 0, sizeof(PacketRecord) * recentPacketsCapacity);
+    memset(recentPackets.get(), 0, sizeof(PacketRecord) * recentPacketsCapacity);
 
 #if !MESHTASTIC_EXCLUDE_PKT_HISTORY_HASH
     // Allocate hash index with load factor <= 0.5 for short probe chains
     hashCapacity = nextPowerOf2(recentPacketsCapacity * 2);
     hashMask = hashCapacity - 1;
-    hashIndex = new uint16_t[hashCapacity];
+    hashIndex.reset(new uint16_t[hashCapacity]);
     if (!hashIndex) {
         LOG_ERROR("Packet History - Hash index allocation failed for %d entries", hashCapacity);
         hashCapacity = 0;
         hashMask = 0;
         return;
     }
-    memset(hashIndex, 0xFF, sizeof(uint16_t) * hashCapacity); // Fill with HASH_EMPTY (0xFFFF)
-#endif
-}
-
-PacketHistory::~PacketHistory()
-{
-    recentPacketsCapacity = 0;
-    delete[] recentPackets;
-    recentPackets = NULL;
-#if !MESHTASTIC_EXCLUDE_PKT_HISTORY_HASH
-    delete[] hashIndex;
-    hashIndex = NULL;
-    hashCapacity = 0;
-    hashMask = 0;
+    memset(hashIndex.get(), 0xFF, sizeof(uint16_t) * hashCapacity); // Fill with HASH_EMPTY (0xFFFF)
 #endif
 }
 
@@ -285,7 +272,7 @@ void PacketHistory::hashRebuild()
 {
     if (!hashIndex)
         return;
-    memset(hashIndex, 0xFF, sizeof(uint16_t) * hashCapacity);
+    memset(hashIndex.get(), 0xFF, sizeof(uint16_t) * hashCapacity);
     for (uint32_t i = 0; i < recentPacketsCapacity; i++) {
         if (recentPackets[i].rxTimeMsec != 0)
             hashInsert(recentPackets[i].sender, recentPackets[i].id, (uint16_t)i);
@@ -332,7 +319,8 @@ PacketHistory::PacketRecord *PacketHistory::find(NodeNum sender, PacketId id)
 #endif
 
     // Linear scan (sole path when hash excluded, fallback when hash allocation failed)
-    for (PacketRecord *it = recentPackets; it < (recentPackets + recentPacketsCapacity); ++it) {
+    PacketRecord *base = recentPackets.get();
+    for (PacketRecord *it = base; it < (base + recentPacketsCapacity); ++it) {
         if (it->id == id && it->sender == sender) {
             return it;
         }
@@ -346,39 +334,38 @@ void PacketHistory::insert(const PacketRecord &r)
 {
     uint32_t now_millis = millis(); // Should not jump with time changes
     uint32_t OldtrxTimeMsec = 0;
+    PacketRecord *base = recentPackets.get();
     PacketRecord *tu = NULL; // Will insert here.
     PacketRecord *it = NULL;
 
     // Find a free, matching or oldest used slot in the recentPackets array
-    for (it = recentPackets; it < (recentPackets + recentPacketsCapacity); ++it) {
+    for (it = base; it < (base + recentPacketsCapacity); ++it) {
         if (it->id == 0 && it->sender == 0 /*&& rxTimeMsec == 0*/) { // Record is empty
             tu = it;                                                 // Remember the free slot
 #if VERBOSE_PACKET_HISTORY >= 2
-            LOG_DEBUG("Packet History - insert: Free slot@ %d/%d", tu - recentPackets, recentPacketsCapacity);
+            LOG_DEBUG("Packet History - insert: Free slot@ %d/%d", tu - base, recentPacketsCapacity);
 #endif
             // We have that, Exit the loop
-            it = (recentPackets + recentPacketsCapacity);
+            it = (base + recentPacketsCapacity);
         } else if (it->id == r.id && it->sender == r.sender) { // Record matches the packet we want to insert
             tu = it;                                           // Remember the matching slot
             OldtrxTimeMsec = now_millis - it->rxTimeMsec;      // ..and save current entry's age
 #if VERBOSE_PACKET_HISTORY >= 2
-            LOG_DEBUG("Packet History - insert: Matched slot@ %d/%d age=%d", tu - recentPackets, recentPacketsCapacity,
-                      OldtrxTimeMsec);
+            LOG_DEBUG("Packet History - insert: Matched slot@ %d/%d age=%d", tu - base, recentPacketsCapacity, OldtrxTimeMsec);
 #endif
             // We have that, Exit the loop
-            it = (recentPackets + recentPacketsCapacity);
+            it = (base + recentPacketsCapacity);
         } else {
             if (it->rxTimeMsec == 0) {
                 LOG_WARN(
                     "Packet History - insert: Found packet s=%08x id=%08x with rxTimeMsec = 0, slot %d/%d. Should never happen!",
-                    it->sender, it->id, it - recentPackets, recentPacketsCapacity);
+                    it->sender, it->id, it - base, recentPacketsCapacity);
             }
             if ((now_millis - it->rxTimeMsec) > OldtrxTimeMsec) { // 49.7 days rollover friendly
                 OldtrxTimeMsec = now_millis - it->rxTimeMsec;
                 tu = it; // remember the oldest packet
 #if VERBOSE_PACKET_HISTORY >= 2
-                LOG_DEBUG("Packet History - insert: Older slot@ %d/%d age=%d", tu - recentPackets, recentPacketsCapacity,
-                          OldtrxTimeMsec);
+                LOG_DEBUG("Packet History - insert: Older slot@ %d/%d age=%d", tu - base, recentPacketsCapacity, OldtrxTimeMsec);
 #endif
             }
             // keep looking for oldest till entire array is checked
@@ -393,13 +380,11 @@ void PacketHistory::insert(const PacketRecord &r)
 
 #if VERBOSE_PACKET_HISTORY
     if (tu->id == 0 && tu->sender == 0) {
-        LOG_DEBUG("Packet History - insert: slot@ %d/%d is NEW", tu - recentPackets, recentPacketsCapacity);
+        LOG_DEBUG("Packet History - insert: slot@ %d/%d is NEW", tu - base, recentPacketsCapacity);
     } else if (tu->id == r.id && tu->sender == r.sender) {
-        LOG_DEBUG("Packet History - insert: slot@ %d/%d MATCHED, age=%d", tu - recentPackets, recentPacketsCapacity,
-                  OldtrxTimeMsec);
+        LOG_DEBUG("Packet History - insert: slot@ %d/%d MATCHED, age=%d", tu - base, recentPacketsCapacity, OldtrxTimeMsec);
     } else {
-        LOG_DEBUG("Packet History - insert: slot@ %d/%d REUSE OLDEST, age=%d", tu - recentPackets, recentPacketsCapacity,
-                  OldtrxTimeMsec);
+        LOG_DEBUG("Packet History - insert: slot@ %d/%d REUSE OLDEST, age=%d", tu - base, recentPacketsCapacity, OldtrxTimeMsec);
     }
 #endif
 
@@ -432,9 +417,9 @@ void PacketHistory::insert(const PacketRecord &r)
 #endif
 
 #if VERBOSE_PACKET_HISTORY
-    LOG_DEBUG("Packet History - insert: Store slot@ %d/%d s=%08x id=%08x nh=%02x rby=%02x %02x %02x rxT=%d BEFORE",
-              tu - recentPackets, recentPacketsCapacity, tu->sender, tu->id, tu->next_hop, tu->relayed_by[0], tu->relayed_by[1],
-              tu->relayed_by[2], tu->rxTimeMsec);
+    LOG_DEBUG("Packet History - insert: Store slot@ %d/%d s=%08x id=%08x nh=%02x rby=%02x %02x %02x rxT=%d BEFORE", tu - base,
+              recentPacketsCapacity, tu->sender, tu->id, tu->next_hop, tu->relayed_by[0], tu->relayed_by[1], tu->relayed_by[2],
+              tu->rxTimeMsec);
 #endif
 
     if (r.rxTimeMsec == 0) {
@@ -454,16 +439,16 @@ void PacketHistory::insert(const PacketRecord &r)
     *tu = r; // store the packet
 
     if (!isMatchingSlot) {
-        hashInsert(r.sender, r.id, (uint16_t)(tu - recentPackets));
+        hashInsert(r.sender, r.id, (uint16_t)(tu - base));
     }
 #else
     *tu = r; // store the packet
 #endif
 
 #if VERBOSE_PACKET_HISTORY
-    LOG_DEBUG("Packet History - insert: Store slot@ %d/%d s=%08x id=%08x nh=%02x rby=%02x %02x %02x rxT=%d AFTER",
-              tu - recentPackets, recentPacketsCapacity, tu->sender, tu->id, tu->next_hop, tu->relayed_by[0], tu->relayed_by[1],
-              tu->relayed_by[2], tu->rxTimeMsec);
+    LOG_DEBUG("Packet History - insert: Store slot@ %d/%d s=%08x id=%08x nh=%02x rby=%02x %02x %02x rxT=%d AFTER", tu - base,
+              recentPacketsCapacity, tu->sender, tu->id, tu->next_hop, tu->relayed_by[0], tu->relayed_by[1], tu->relayed_by[2],
+              tu->rxTimeMsec);
 #endif
 }
 

src/mesh/PacketHistory.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include "NodeDB.h"
+#include <memory>
 
 // Number of relayers we keep track of. Use 6 to be efficient with memory alignment of PacketRecord to 20 bytes
 #define NUM_RELAYERS 6
@@ -26,7 +27,7 @@ class PacketHistory
 
     uint32_t recentPacketsCapacity =
         0; // Can be set in constructor, no need to recompile. Used to allocate memory for mx_recentPackets.
-    PacketRecord *recentPackets = NULL; // Simple and fixed in size. Debloat.
+    std::unique_ptr<PacketRecord[]> recentPackets; // Simple and fixed in size. Debloat.
 
 #if !MESHTASTIC_EXCLUDE_PKT_HISTORY_HASH
     // Open-addressing hash table for O(1) lookup in find(), replacing the O(N) linear scan.
@@ -34,7 +35,7 @@ class PacketHistory
     // The load factor invariant holds permanently: hashCapacity = 2 * nextPowerOf2(recentPacketsCapacity),
     // and at most recentPacketsCapacity entries can ever be live (one per recentPackets[] slot).
     static constexpr uint16_t HASH_EMPTY = 0xFFFF;
-    uint16_t *hashIndex = NULL;
+    std::unique_ptr<uint16_t[]> hashIndex;
     uint32_t hashCapacity = 0; // Always a power of 2
     uint32_t hashMask = 0;     // hashCapacity - 1, for fast modular indexing
 
@@ -64,11 +65,8 @@ class PacketHistory
     uint8_t getOurTxHopLimit(const PacketRecord &r);
     void setOurTxHopLimit(PacketRecord &r, uint8_t hopLimit);
 
-    PacketHistory(const PacketHistory &);            // non construction-copyable
-    PacketHistory &operator=(const PacketHistory &); // non copyable
   public:
     explicit PacketHistory(uint32_t size = -1); // Constructor with size parameter, default is PACKETHISTORY_MAX
-    ~PacketHistory();
 
     /**
      * Update recentBroadcasts and return true if we have already seen this packet
@@ -100,5 +98,5 @@ class PacketHistory
     void removeRelayer(const uint8_t relayer, const uint32_t id, const NodeNum sender);
 
     // To check if the PacketHistory was initialized correctly by constructor
-    bool initOk(void) { return recentPackets != NULL && recentPacketsCapacity != 0; }
+    bool initOk(void) { return recentPackets != nullptr && recentPacketsCapacity != 0; }
 };

src/mesh/PhoneAPI.cpp

@@ -62,7 +62,7 @@ void PhoneAPI::handleStartConfig()
     onConfigStart();
 
     // even if we were already connected - restart our state machine
-    if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
+    if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
         // If client only wants node info, jump directly to sending nodes
         state = STATE_SEND_OWN_NODEINFO;
         LOG_INFO("Client only wants node info, skipping other config");
@@ -81,6 +81,15 @@ void PhoneAPI::handleStartConfig()
         concurrency::LockGuard guard(&nodeInfoMutex);
         nodeInfoForPhone = {};
         nodeInfoQueue.clear();
+        replayQueue.clear();
+        replayPositionOrder.clear();
+        replayTelemetryOrder.clear();
+        replayEnvironmentOrder.clear();
+        replayStatusOrder.clear();
+        replayPositionIndex = 0;
+        replayTelemetryIndex = 0;
+        replayEnvironmentIndex = 0;
+        replayStatusIndex = 0;
     }
     resetReadIndex();
 }
@@ -120,6 +129,15 @@ void PhoneAPI::close()
             concurrency::LockGuard guard(&nodeInfoMutex);
             nodeInfoForPhone = {};
             nodeInfoQueue.clear();
+            replayQueue.clear();
+            replayPositionOrder.clear();
+            replayTelemetryOrder.clear();
+            replayEnvironmentOrder.clear();
+            replayStatusOrder.clear();
+            replayPositionIndex = 0;
+            replayTelemetryIndex = 0;
+            replayEnvironmentIndex = 0;
+            replayStatusIndex = 0;
         }
         packetForPhone = NULL;
         filesManifest.clear();
@@ -302,7 +320,7 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
                 nodeInfoForPhone.num = 0;
             }
         }
-        if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
+        if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
             // If client only wants node info, jump directly to sending nodes
             state = STATE_SEND_OTHER_NODEINFOS;
             onNowHasData(0);
@@ -529,8 +547,124 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
             LOG_DEBUG("Done sending %d of %d nodeinfos millis=%u", readIndex, nodeDB->getNumMeshNodes(), millis());
             concurrency::LockGuard guard(&nodeInfoMutex);
             nodeInfoQueue.clear();
+            // Replay states no-op for legacy clients / excluded DBs.
+            state = STATE_REPLAY_POSITIONS;
+            return getFromRadio(buf);
+        }
+        break;
+    }
+
+    case STATE_REPLAY_POSITIONS: {
+        if (replayPositionOrder.empty() && replayPositionIndex == 0)
+            beginReplayPositions();
+        prefetchReplayPositions();
+
+        meshtastic_MeshPacket pkt = {};
+        bool havePkt = false;
+        {
+            concurrency::LockGuard guard(&nodeInfoMutex);
+            if (!replayQueue.empty()) {
+                pkt = replayQueue.front();
+                replayQueue.pop_front();
+                havePkt = true;
+            }
+        }
+
+        if (havePkt) {
+            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
+            fromRadioScratch.packet = pkt;
+        } else {
+            LOG_DEBUG("Done replaying positions count=%u millis=%u", (unsigned)replayPositionIndex, millis());
+            state = STATE_REPLAY_TELEMETRY;
+            return getFromRadio(buf);
+        }
+        break;
+    }
+
+    case STATE_REPLAY_TELEMETRY: {
+        if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
+            beginReplayTelemetry();
+        prefetchReplayTelemetry();
+
+        meshtastic_MeshPacket pkt = {};
+        bool havePkt = false;
+        {
+            concurrency::LockGuard guard(&nodeInfoMutex);
+            if (!replayQueue.empty()) {
+                pkt = replayQueue.front();
+                replayQueue.pop_front();
+                havePkt = true;
+            }
+        }
+
+        if (havePkt) {
+            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
+            fromRadioScratch.packet = pkt;
+        } else {
+            LOG_DEBUG("Done replaying telemetry count=%u millis=%u", (unsigned)replayTelemetryIndex, millis());
+            state = STATE_REPLAY_ENVIRONMENT;
+            return getFromRadio(buf);
+        }
+        break;
+    }
+
+    case STATE_REPLAY_ENVIRONMENT: {
+        if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
+            beginReplayEnvironment();
+        prefetchReplayEnvironment();
+
+        meshtastic_MeshPacket pkt = {};
+        bool havePkt = false;
+        {
+            concurrency::LockGuard guard(&nodeInfoMutex);
+            if (!replayQueue.empty()) {
+                pkt = replayQueue.front();
+                replayQueue.pop_front();
+                havePkt = true;
+            }
+        }
+
+        if (havePkt) {
+            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
+            fromRadioScratch.packet = pkt;
+        } else {
+            LOG_DEBUG("Done replaying environment count=%u millis=%u", (unsigned)replayEnvironmentIndex, millis());
+            state = STATE_REPLAY_STATUS;
+            return getFromRadio(buf);
+        }
+        break;
+    }
+
+    case STATE_REPLAY_STATUS: {
+        if (replayStatusOrder.empty() && replayStatusIndex == 0)
+            beginReplayStatus();
+        prefetchReplayStatus();
+
+        meshtastic_MeshPacket pkt = {};
+        bool havePkt = false;
+        {
+            concurrency::LockGuard guard(&nodeInfoMutex);
+            if (!replayQueue.empty()) {
+                pkt = replayQueue.front();
+                replayQueue.pop_front();
+                havePkt = true;
+            }
+        }
+
+        if (havePkt) {
+            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
+            fromRadioScratch.packet = pkt;
+        } else {
+            LOG_DEBUG("Done replaying status count=%u millis=%u", (unsigned)replayStatusIndex, millis());
+            replayPositionOrder.clear();
+            replayPositionOrder.shrink_to_fit();
+            replayTelemetryOrder.clear();
+            replayTelemetryOrder.shrink_to_fit();
+            replayEnvironmentOrder.clear();
+            replayEnvironmentOrder.shrink_to_fit();
+            replayStatusOrder.clear();
+            replayStatusOrder.shrink_to_fit();
             state = STATE_SEND_FILEMANIFEST;
-            // Go ahead and send that ID right now
             return getFromRadio(buf);
         }
         break;
@@ -538,9 +672,9 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
 
     case STATE_SEND_FILEMANIFEST: {
         LOG_DEBUG("FromRadio=STATE_SEND_FILEMANIFEST");
-        // last element
-        if (config_state == filesManifest.size() ||
-            config_nonce == SPECIAL_NONCE_ONLY_NODES) { // also handles an empty filesManifest
+        // ONLY_NODES variants skip the manifest.
+        if (config_state == filesManifest.size() || config_nonce == SPECIAL_NONCE_ONLY_NODES ||
+            config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
             config_state = 0;
             filesManifest.clear();
             // Skip to complete packet
@@ -653,6 +787,7 @@ void PhoneAPI::prefetchNodeInfos()
 {
     bool added = false;
     bool wasEmpty = false;
+    const bool gradient = clientWantsGradientSync();
     // Keep the queue topped up so BLE reads stay responsive even if DB fetches take a moment.
     {
         concurrency::LockGuard guard(&nodeInfoMutex);
@@ -662,7 +797,8 @@ void PhoneAPI::prefetchNodeInfos()
             if (!nextNode)
                 break;
 
-            auto info = TypeConversions::ConvertToNodeInfo(nextNode);
+            auto info =
+                gradient ? TypeConversions::ConvertToNodeInfoThin(nextNode) : TypeConversions::ConvertToNodeInfo(nextNode);
             bool isUs = info.num == nodeDB->getNodeNum();
             info.hops_away = isUs ? 0 : info.hops_away;
             info.last_heard = isUs ? getValidTime(RTCQualityFromNet) : info.last_heard;
@@ -682,6 +818,257 @@ void PhoneAPI::prefetchNodeInfos()
         onNowHasData(0);
 }
 
+meshtastic_MeshPacket PhoneAPI::makeReplayPositionPacket(NodeNum num, const meshtastic_PositionLite &pos)
+{
+    meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
+    pkt.from = num;
+    pkt.to = nodeDB->getNodeNum();
+    pkt.id = generatePacketId();
+    pkt.rx_time = pos.time;
+    pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
+    pkt.decoded.portnum = meshtastic_PortNum_POSITION_APP;
+    meshtastic_Position fullPos = TypeConversions::ConvertToPosition(pos);
+    size_t len =
+        pb_encode_to_bytes(pkt.decoded.payload.bytes, sizeof(pkt.decoded.payload.bytes), &meshtastic_Position_msg, &fullPos);
+    pkt.decoded.payload.size = (pb_size_t)len;
+    return pkt;
+}
+
+meshtastic_MeshPacket PhoneAPI::makeReplayTelemetryPacket(NodeNum num, const meshtastic_DeviceMetrics &metrics)
+{
+    meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
+    pkt.from = num;
+    pkt.to = nodeDB->getNodeNum();
+    pkt.id = generatePacketId();
+    // No native timestamp on telemetry packets here; use last_heard.
+    const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
+    pkt.rx_time = header ? header->last_heard : 0;
+    pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
+    pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
+    meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
+    fullTel.time = pkt.rx_time;
+    fullTel.which_variant = meshtastic_Telemetry_device_metrics_tag;
+    fullTel.variant.device_metrics = metrics;
+    size_t len =
+        pb_encode_to_bytes(pkt.decoded.payload.bytes, sizeof(pkt.decoded.payload.bytes), &meshtastic_Telemetry_msg, &fullTel);
+    pkt.decoded.payload.size = (pb_size_t)len;
+    return pkt;
+}
+
+void PhoneAPI::beginReplayPositions()
+{
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    // Build excluded entirely - leave the order list empty so the state arm
+    // immediately drains and advances.
+    replayPositionOrder.clear();
+    replayPositionIndex = 0;
+#else
+    if (!clientWantsGradientSync()) {
+        replayPositionOrder.clear();
+        replayPositionIndex = 0;
+        return;
+    }
+    // Snapshot the keyset at phase start so concurrent inserts/erases on the
+    // map don't invalidate iteration. Skip our own node - the phone already
+    // got our position bundled in STATE_SEND_OWN_NODEINFO.
+    replayPositionOrder = nodeDB->snapshotPositionNodeNums(nodeDB->getNodeNum());
+    replayPositionIndex = 0;
+    LOG_INFO("Begin position replay: %u entries millis=%u", (unsigned)replayPositionOrder.size(), millis());
+#endif
+}
+
+void PhoneAPI::prefetchReplayPositions()
+{
+#if MESHTASTIC_EXCLUDE_POSITIONDB
+    return;
+#else
+    if (!clientWantsGradientSync())
+        return;
+    bool added = false;
+    bool wasEmpty = false;
+    {
+        concurrency::LockGuard guard(&nodeInfoMutex);
+        wasEmpty = replayQueue.empty();
+        while (replayQueue.size() < kReplayPrefetchDepth && replayPositionIndex < replayPositionOrder.size()) {
+            NodeNum num = replayPositionOrder[replayPositionIndex++];
+            meshtastic_PositionLite pos;
+            if (!nodeDB->copyNodePosition(num, pos))
+                continue; // entry was evicted between snapshot and now
+            replayQueue.push_back(makeReplayPositionPacket(num, pos));
+            added = true;
+        }
+    }
+    if (added && wasEmpty)
+        onNowHasData(0);
+#endif
+}
+
+void PhoneAPI::beginReplayTelemetry()
+{
+#if MESHTASTIC_EXCLUDE_TELEMETRYDB
+    replayTelemetryOrder.clear();
+    replayTelemetryIndex = 0;
+#else
+    if (!clientWantsGradientSync()) {
+        replayTelemetryOrder.clear();
+        replayTelemetryIndex = 0;
+        return;
+    }
+    replayTelemetryOrder = nodeDB->snapshotTelemetryNodeNums(nodeDB->getNodeNum());
+    replayTelemetryIndex = 0;
+    LOG_INFO("Begin telemetry replay: %u entries millis=%u", (unsigned)replayTelemetryOrder.size(), millis());
+#endif
+}
+
+void PhoneAPI::prefetchReplayTelemetry()
+{
+#if MESHTASTIC_EXCLUDE_TELEMETRYDB
+    return;
+#else
+    if (!clientWantsGradientSync())
+        return;
+    bool added = false;
+    bool wasEmpty = false;
+    {
+        concurrency::LockGuard guard(&nodeInfoMutex);
+        wasEmpty = replayQueue.empty();
+        while (replayQueue.size() < kReplayPrefetchDepth && replayTelemetryIndex < replayTelemetryOrder.size()) {
+            NodeNum num = replayTelemetryOrder[replayTelemetryIndex++];
+            meshtastic_DeviceMetrics dm;
+            if (!nodeDB->copyNodeTelemetry(num, dm))
+                continue;
+            replayQueue.push_back(makeReplayTelemetryPacket(num, dm));
+            added = true;
+        }
+    }
+    if (added && wasEmpty)
+        onNowHasData(0);
+#endif
+}
+
+meshtastic_MeshPacket PhoneAPI::makeReplayEnvironmentPacket(uint32_t num, const meshtastic_EnvironmentMetrics &env)
+{
+    meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
+    pkt.from = num;
+    pkt.to = nodeDB->getNodeNum();
+    pkt.id = generatePacketId();
+    const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
+    pkt.rx_time = header ? header->last_heard : 0;
+    pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
+    pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
+    meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
+    fullTel.time = pkt.rx_time;
+    fullTel.which_variant = meshtastic_Telemetry_environment_metrics_tag;
+    fullTel.variant.environment_metrics = env;
+    size_t len =
+        pb_encode_to_bytes(pkt.decoded.payload.bytes, sizeof(pkt.decoded.payload.bytes), &meshtastic_Telemetry_msg, &fullTel);
+    pkt.decoded.payload.size = (pb_size_t)len;
+    return pkt;
+}
+
+void PhoneAPI::beginReplayEnvironment()
+{
+#if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    replayEnvironmentOrder.clear();
+    replayEnvironmentIndex = 0;
+#else
+    if (!clientWantsGradientSync()) {
+        replayEnvironmentOrder.clear();
+        replayEnvironmentIndex = 0;
+        return;
+    }
+    replayEnvironmentOrder = nodeDB->snapshotEnvironmentNodeNums(nodeDB->getNodeNum());
+    replayEnvironmentIndex = 0;
+    LOG_INFO("Begin environment replay: %u entries millis=%u", (unsigned)replayEnvironmentOrder.size(), millis());
+#endif
+}
+
+void PhoneAPI::prefetchReplayEnvironment()
+{
+#if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    return;
+#else
+    if (!clientWantsGradientSync())
+        return;
+    bool added = false;
+    bool wasEmpty = false;
+    {
+        concurrency::LockGuard guard(&nodeInfoMutex);
+        wasEmpty = replayQueue.empty();
+        while (replayQueue.size() < kReplayPrefetchDepth && replayEnvironmentIndex < replayEnvironmentOrder.size()) {
+            NodeNum num = replayEnvironmentOrder[replayEnvironmentIndex++];
+            meshtastic_EnvironmentMetrics env;
+            if (!nodeDB->copyNodeEnvironment(num, env))
+                continue;
+            replayQueue.push_back(makeReplayEnvironmentPacket(num, env));
+            added = true;
+        }
+    }
+    if (added && wasEmpty)
+        onNowHasData(0);
+#endif
+}
+
+meshtastic_MeshPacket PhoneAPI::makeReplayStatusPacket(uint32_t num, const meshtastic_StatusMessage &status)
+{
+    meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
+    pkt.from = num;
+    pkt.to = nodeDB->getNodeNum();
+    pkt.id = generatePacketId();
+    // StatusMessage has no native timestamp; use last_heard.
+    const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
+    pkt.rx_time = header ? header->last_heard : 0;
+    pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
+    pkt.decoded.portnum = meshtastic_PortNum_NODE_STATUS_APP;
+    size_t len =
+        pb_encode_to_bytes(pkt.decoded.payload.bytes, sizeof(pkt.decoded.payload.bytes), &meshtastic_StatusMessage_msg, &status);
+    pkt.decoded.payload.size = (pb_size_t)len;
+    return pkt;
+}
+
+void PhoneAPI::beginReplayStatus()
+{
+#if MESHTASTIC_EXCLUDE_STATUSDB
+    replayStatusOrder.clear();
+    replayStatusIndex = 0;
+#else
+    if (!clientWantsGradientSync()) {
+        replayStatusOrder.clear();
+        replayStatusIndex = 0;
+        return;
+    }
+    replayStatusOrder = nodeDB->snapshotStatusNodeNums(nodeDB->getNodeNum());
+    replayStatusIndex = 0;
+    LOG_INFO("Begin status replay: %u entries millis=%u", (unsigned)replayStatusOrder.size(), millis());
+#endif
+}
+
+void PhoneAPI::prefetchReplayStatus()
+{
+#if MESHTASTIC_EXCLUDE_STATUSDB
+    return;
+#else
+    if (!clientWantsGradientSync())
+        return;
+    bool added = false;
+    bool wasEmpty = false;
+    {
+        concurrency::LockGuard guard(&nodeInfoMutex);
+        wasEmpty = replayQueue.empty();
+        while (replayQueue.size() < kReplayPrefetchDepth && replayStatusIndex < replayStatusOrder.size()) {
+            NodeNum num = replayStatusOrder[replayStatusIndex++];
+            meshtastic_StatusMessage status;
+            if (!nodeDB->copyNodeStatus(num, status) || status.status[0] == '\0')
+                continue;
+            replayQueue.push_back(makeReplayStatusPacket(num, status));
+            added = true;
+        }
+    }
+    if (added && wasEmpty)
+        onNowHasData(0);
+#endif
+}
+
 void PhoneAPI::releaseMqttClientProxyPhonePacket()
 {
     if (mqttClientProxyMessageForPhone) {
@@ -728,6 +1115,31 @@ bool PhoneAPI::available()
     PREFETCH_NODEINFO:
         prefetchNodeInfos();
         return true;
+    case STATE_REPLAY_POSITIONS: {
+        // Prime the iterator if we haven't yet, then top up the queue.
+        if (replayPositionOrder.empty() && replayPositionIndex == 0)
+            beginReplayPositions();
+        prefetchReplayPositions();
+        return true; // Always advance state machine; arm itself transitions when drained
+    }
+    case STATE_REPLAY_TELEMETRY: {
+        if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
+            beginReplayTelemetry();
+        prefetchReplayTelemetry();
+        return true;
+    }
+    case STATE_REPLAY_ENVIRONMENT: {
+        if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
+            beginReplayEnvironment();
+        prefetchReplayEnvironment();
+        return true;
+    }
+    case STATE_REPLAY_STATUS: {
+        if (replayStatusOrder.empty() && replayStatusIndex == 0)
+            beginReplayStatus();
+        prefetchReplayStatus();
+        return true;
+    }
     case STATE_SEND_PACKETS: {
         if (!queueStatusPacketForPhone)
             queueStatusPacketForPhone = service->getQueueStatusForPhone();

src/mesh/PhoneAPI.h

@@ -4,12 +4,16 @@
 #include "concurrency/Lock.h"
 #include "mesh-pb-constants.h"
 #include "meshtastic/portnums.pb.h"
+#include <cstdint>
 #include <deque>
 #include <iterator>
 #include <string>
 #include <unordered_map>
 #include <vector>
 
+// NodeNum stored as raw uint32_t below; including MeshTypes.h here breaks the
+// LOG_WARN include order via Arduino.h/MemoryPool.h.
+
 // Make sure that we never let our packets grow too large for one BLE packet
 #define MAX_TO_FROM_RADIO_SIZE 512
 
@@ -22,6 +26,9 @@
 
 #define SPECIAL_NONCE_ONLY_CONFIG 69420
 #define SPECIAL_NONCE_ONLY_NODES 69421 // ( ͡° ͜ʖ ͡°)
+// Gradient sync: phone sends one of these to opt into thin-header + replay.
+#define SPECIAL_NONCE_GRADIENT_SYNC 69422
+#define SPECIAL_NONCE_GRADIENT_ONLY_NODES 69423
 
 /**
  * Provides our protobuf based API which phone/PC clients can use to talk to our device
@@ -45,7 +52,13 @@ class PhoneAPI
         STATE_SEND_CONFIG,          // Replacement for the old Radioconfig
         STATE_SEND_MODULECONFIG,    // Send Module specific config
         STATE_SEND_OTHER_NODEINFOS, // states progress in this order as the device sends to to the client
-        STATE_SEND_FILEMANIFEST,    // Send file manifest
+        // Drain satellite DBs as synthetic POSITION_APP / TELEMETRY_APP /
+        // NODE_STATUS_APP packets when the phone opted into gradient sync.
+        STATE_REPLAY_POSITIONS,
+        STATE_REPLAY_TELEMETRY,
+        STATE_REPLAY_ENVIRONMENT,
+        STATE_REPLAY_STATUS,
+        STATE_SEND_FILEMANIFEST, // Send file manifest
         STATE_SEND_COMPLETE_ID,
         STATE_SEND_PACKETS // send packets or debug strings
     };
@@ -88,6 +101,20 @@ class PhoneAPI
     // Protect nodeInfoForPhone + nodeInfoQueue because NimBLE callbacks run in a separate FreeRTOS task.
     concurrency::Lock nodeInfoMutex;
 
+    // Synthetic-packet replay queue (paced via prefetch).
+    std::deque<meshtastic_MeshPacket> replayQueue;
+    static constexpr size_t kReplayPrefetchDepth = 4;
+    // NodeNum snapshots taken at each replay phase start so concurrent
+    // satellite-map mutations don't invalidate iteration.
+    std::vector<uint32_t> replayPositionOrder;
+    std::vector<uint32_t> replayTelemetryOrder;
+    std::vector<uint32_t> replayEnvironmentOrder;
+    std::vector<uint32_t> replayStatusOrder;
+    size_t replayPositionIndex = 0;
+    size_t replayTelemetryIndex = 0;
+    size_t replayEnvironmentIndex = 0;
+    size_t replayStatusIndex = 0;
+
     meshtastic_ToRadio toRadioScratch = {
         0}; // this is a static scratch object, any data must be copied elsewhere before returning
 
@@ -137,6 +164,10 @@ class PhoneAPI
 
     bool isConnected() { return state != STATE_SEND_NOTHING; }
     bool isSendingPackets() { return state == STATE_SEND_PACKETS; }
+    bool clientWantsGradientSync() const
+    {
+        return config_nonce == SPECIAL_NONCE_GRADIENT_SYNC || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES;
+    }
 
   protected:
     /// Our fromradio packet while it is being assembled
@@ -185,6 +216,18 @@ class PhoneAPI
     void releaseQueueStatusPhonePacket();
 
     void prefetchNodeInfos();
+    void beginReplayPositions();
+    void prefetchReplayPositions();
+    void beginReplayTelemetry();
+    void prefetchReplayTelemetry();
+    void beginReplayEnvironment();
+    void prefetchReplayEnvironment();
+    void beginReplayStatus();
+    void prefetchReplayStatus();
+    meshtastic_MeshPacket makeReplayPositionPacket(uint32_t num, const meshtastic_PositionLite &pos);
+    meshtastic_MeshPacket makeReplayTelemetryPacket(uint32_t num, const meshtastic_DeviceMetrics &metrics);
+    meshtastic_MeshPacket makeReplayEnvironmentPacket(uint32_t num, const meshtastic_EnvironmentMetrics &env);
+    meshtastic_MeshPacket makeReplayStatusPacket(uint32_t num, const meshtastic_StatusMessage &status);
 
     void releaseMqttClientProxyPhonePacket();
 

src/mesh/ProtobufModule.h

@@ -58,7 +58,7 @@ template <class T> class ProtobufModule : protected SinglePortModule
     const char *getSenderShortName(const meshtastic_MeshPacket &mp)
     {
         auto node = nodeDB->getMeshNode(getFrom(&mp));
-        const char *sender = (node) ? node->user.short_name : "???";
+        const char *sender = (node && nodeInfoLiteHasUser(node) && node->short_name[0]) ? node->short_name : "???";
         return sender;
     }
 
@@ -121,4 +121,4 @@ template <class T> class ProtobufModule : protected SinglePortModule
             return alterReceivedProtobuf(mp, decoded);
         }
     }
-};
+};

src/mesh/RF95Interface.cpp

@@ -252,7 +252,7 @@ bool RF95Interface::reconfigure()
 
     startReceive(); // restart receiving
 
-    return RADIOLIB_ERR_NONE;
+    return true;
 }
 
 /**

src/mesh/RadioInterface.cpp

@@ -5,6 +5,7 @@
 #include "LR1110Interface.h"
 #include "LR1120Interface.h"
 #include "LR1121Interface.h"
+#include "LR2021Interface.h"
 #include "MeshRadio.h"
 #include "MeshService.h"
 #include "NodeDB.h"
@@ -25,7 +26,6 @@
 
 #ifdef ARCH_PORTDUINO
 #include "platform/portduino/PortduinoGlue.h"
-#include "platform/portduino/SerialHal.h"
 #include "platform/portduino/SimRadio.h"
 #include "platform/portduino/USBHal.h"
 #endif
@@ -293,9 +293,6 @@ std::unique_ptr<RadioInterface> initLoRa()
               portduino_config.lora_spi_dev.c_str());
     if (portduino_config.lora_spi_dev == "ch341") {
         RadioLibHAL = ch341Hal;
-    } else if (portduino_config.lora_spi_dev == "serial") {
-        RadioLibHAL = new SerialHal(portduino_config.lora_serial_device, portduino_config.lora_serial_baud,
-                                    (uint32_t)portduino_config.lora_serial_timeout_ms);
     } else {
         if (RadioLibHAL != nullptr) {
             delete RadioLibHAL;
@@ -482,6 +479,20 @@ std::unique_ptr<RadioInterface> initLoRa()
     }
 #endif
 
+#if defined(USE_LR2021) && RADIOLIB_EXCLUDE_LR2021 != 1
+    if (!rIf) {
+        rIf = std::unique_ptr<LR2021Interface>(
+            new LR2021Interface(loraHal, LR2021_SPI_NSS_PIN, LR2021_IRQ_PIN, LR2021_NRESET_PIN, LR2021_BUSY_PIN));
+        if (!rIf->init()) {
+            LOG_WARN("No LR2021 radio");
+            rIf = nullptr;
+        } else {
+            LOG_INFO("LR2021 init success");
+            radioType = LR2021_RADIO;
+        }
+    }
+#endif
+
 #if defined(USE_SX1280) && RADIOLIB_EXCLUDE_SX128X != 1
     if (!rIf) {
         rIf = std::unique_ptr<SX1280Interface>(new SX1280Interface(loraHal, SX128X_CS, SX128X_DIO1, SX128X_RESET, SX128X_BUSY));

src/mesh/ReliableRouter.cpp

@@ -115,7 +115,7 @@ void ReliableRouter::sniffReceived(const meshtastic_MeshPacket *p, const meshtas
                         sendAckNak(meshtastic_Routing_Error_NONE, getFrom(p), p->id, p->channel, 0);
                     }
                 } else if (p->which_payload_variant == meshtastic_MeshPacket_encrypted_tag && p->channel == 0 &&
-                           (nodeDB->getMeshNode(p->from) == nullptr || nodeDB->getMeshNode(p->from)->user.public_key.size == 0)) {
+                           (nodeDB->getMeshNode(p->from) == nullptr || nodeDB->getMeshNode(p->from)->public_key.size == 0)) {
                     LOG_INFO("PKI packet from unknown node, send PKI_UNKNOWN_PUBKEY");
                     sendAckNak(meshtastic_Routing_Error_PKI_UNKNOWN_PUBKEY, getFrom(p), p->id, channels.getPrimaryIndex(),
                                routingModule->getHopLimitForResponse(*p));

src/mesh/Router.cpp

@@ -120,17 +120,17 @@ bool Router::shouldDecrementHopLimit(const meshtastic_MeshPacket *p)
         if (!node)
             continue;
 
-        // Check 1: is_favorite (cheapest - single bool)
-        if (!node->is_favorite)
+        // Check 1: is_favorite (cheapest - single bit test)
+        if (!nodeInfoLiteIsFavorite(node))
             continue;
 
-        // Check 2: has_user (cheap - single bool)
-        if (!node->has_user)
+        // Check 2: has_user (cheap - single bit test)
+        if (!nodeInfoLiteHasUser(node))
             continue;
 
         // Check 3: role check (moderate cost - multiple comparisons)
-        if (!IS_ONE_OF(node->user.role, meshtastic_Config_DeviceConfig_Role_ROUTER,
-                       meshtastic_Config_DeviceConfig_Role_ROUTER_LATE, meshtastic_Config_DeviceConfig_Role_CLIENT_BASE)) {
+        if (!IS_ONE_OF(node->role, meshtastic_Config_DeviceConfig_Role_ROUTER, meshtastic_Config_DeviceConfig_Role_ROUTER_LATE,
+                       meshtastic_Config_DeviceConfig_Role_CLIENT_BASE)) {
             continue;
         }
 
@@ -329,7 +329,7 @@ ErrorCode Router::send(meshtastic_MeshPacket *p)
             cn->reply_id = p->id;
             cn->level = meshtastic_LogRecord_Level_WARNING;
             cn->time = getValidTime(RTCQualityFromNet);
-            sprintf(cn->message, "Duty cycle limit exceeded. You can send again in %d mins", silentMinutes);
+            snprintf(cn->message, sizeof(cn->message), "Duty cycle limit exceeded. You can send again in %d mins", silentMinutes);
             service->sendClientNotification(cn);
 
             meshtastic_Routing_Error err = meshtastic_Routing_Error_DUTY_CYCLE_LIMIT;
@@ -433,7 +433,7 @@ DecodeState perhapsDecode(meshtastic_MeshPacket *p)
     concurrency::LockGuard g(cryptLock);
 
     if (config.device.rebroadcast_mode == meshtastic_Config_DeviceConfig_RebroadcastMode_KNOWN_ONLY &&
-        (nodeDB->getMeshNode(p->from) == NULL || !nodeDB->getMeshNode(p->from)->has_user)) {
+        !nodeInfoLiteHasUser(nodeDB->getMeshNode(p->from))) {
         LOG_DEBUG("Node 0x%x not in nodeDB-> Rebroadcast mode KNOWN_ONLY will ignore packet", p->from);
         return DecodeState::DECODE_FAILURE;
     }
@@ -451,11 +451,11 @@ DecodeState perhapsDecode(meshtastic_MeshPacket *p)
 #if !(MESHTASTIC_EXCLUDE_PKI)
     // Attempt PKI decryption first
     if (p->channel == 0 && isToUs(p) && p->to > 0 && !isBroadcast(p->to) && nodeDB->getMeshNode(p->from) != nullptr &&
-        nodeDB->getMeshNode(p->from)->user.public_key.size > 0 && nodeDB->getMeshNode(p->to)->user.public_key.size > 0 &&
-        rawSize > MESHTASTIC_PKC_OVERHEAD) {
+        nodeDB->getMeshNode(p->from)->public_key.size > 0 && nodeDB->getMeshNode(p->to) != nullptr &&
+        nodeDB->getMeshNode(p->to)->public_key.size > 0 && rawSize > MESHTASTIC_PKC_OVERHEAD) {
         LOG_DEBUG("Attempt PKI decryption");
 
-        if (crypto->decryptCurve25519(p->from, nodeDB->getMeshNode(p->from)->user.public_key, p->id, rawSize, p->encrypted.bytes,
+        if (crypto->decryptCurve25519(p->from, nodeDB->getMeshNode(p->from)->public_key, p->id, rawSize, p->encrypted.bytes,
                                       bytes)) {
             LOG_INFO("PKI Decryption worked!");
 
@@ -467,7 +467,7 @@ DecodeState perhapsDecode(meshtastic_MeshPacket *p)
                 decrypted = true;
                 LOG_INFO("Packet decrypted using PKI!");
                 p->pki_encrypted = true;
-                memcpy(&p->public_key.bytes, nodeDB->getMeshNode(p->from)->user.public_key.bytes, 32);
+                memcpy(&p->public_key.bytes, nodeDB->getMeshNode(p->from)->public_key.bytes, 32);
                 p->public_key.size = 32;
                 p->decoded = decodedtmp;
                 p->which_payload_variant = meshtastic_MeshPacket_decoded_tag; // change type to decoded
@@ -665,18 +665,18 @@ meshtastic_Routing_Error perhapsEncode(meshtastic_MeshPacket *p)
             if (numbytes + MESHTASTIC_HEADER_LENGTH + MESHTASTIC_PKC_OVERHEAD > MAX_LORA_PAYLOAD_LEN)
                 return meshtastic_Routing_Error_TOO_LARGE;
             // Check for a known public key for the destination
-            if (node == nullptr || node->user.public_key.size != 32) {
+            if (node == nullptr || node->public_key.size != 32) {
                 LOG_WARN("Unknown public key for destination node 0x%08x (portnum %d), refusing to send legacy DM", p->to,
                          p->decoded.portnum);
                 return meshtastic_Routing_Error_PKI_SEND_FAIL_PUBLIC_KEY;
             }
             if (p->pki_encrypted && !memfll(p->public_key.bytes, 0, 32) &&
-                memcmp(p->public_key.bytes, node->user.public_key.bytes, 32) != 0) {
+                memcmp(p->public_key.bytes, node->public_key.bytes, 32) != 0) {
                 LOG_WARN("Client public key differs from requested: 0x%02x, stored key begins 0x%02x", *p->public_key.bytes,
-                         *node->user.public_key.bytes);
+                         *node->public_key.bytes);
                 return meshtastic_Routing_Error_PKI_FAILED;
             }
-            crypto->encryptCurve25519(p->to, getFrom(p), node->user.public_key, p->id, numbytes, bytes, p->encrypted.bytes);
+            crypto->encryptCurve25519(p->to, getFrom(p), node->public_key, p->id, numbytes, bytes, p->encrypted.bytes);
             numbytes += MESHTASTIC_PKC_OVERHEAD;
             p->channel = 0;
             p->pki_encrypted = true;
@@ -860,7 +860,7 @@ void Router::perhapsHandleReceived(meshtastic_MeshPacket *p)
     }
 
     meshtastic_NodeInfoLite const *node = nodeDB->getMeshNode(p->from);
-    if (node != NULL && node->is_ignored) {
+    if (nodeInfoLiteIsIgnored(node)) {
         LOG_DEBUG("Ignore msg, 0x%x is ignored", p->from);
         packetPool.release(p);
         return;

src/mesh/SX126xInterface.cpp

@@ -262,7 +262,7 @@ template <typename T> bool SX126xInterface<T>::reconfigure()
 
     startReceive(); // restart receiving
 
-    return RADIOLIB_ERR_NONE;
+    return true;
 }
 
 template <typename T> void SX126xInterface<T>::disableInterrupt()

src/mesh/SX128xInterface.cpp

@@ -153,7 +153,7 @@ template <typename T> bool SX128xInterface<T>::reconfigure()
 
     startReceive(); // restart receiving
 
-    return RADIOLIB_ERR_NONE;
+    return true;
 }
 
 template <typename T> void SX128xInterface<T>::disableInterrupt()

src/mesh/SerialHalDevice.cpp

@@ -1,385 +0,0 @@
-#include "mesh/SerialHalDevice.h"
-#include "NodeDB.h"
-#include "SPILock.h"
-#include "concurrency/Periodic.h"
-#include "configuration.h"
-#include "mesh/StreamAPI.h"
-#include "mesh/generated/meshtastic/config.pb.h"
-#include <Arduino.h>
-#include <SPI.h>
-#include <cstring>
-#include <stdint.h>
-
-#if defined(ARCH_ESP32)
-#if defined(HW_SPI1_DEVICE)
-extern SPIClass SPI1;
-#endif
-#endif
-
-namespace
-{
-constexpr uint32_t SERIAL_PI_RISING = 1;
-constexpr uint32_t SERIAL_PI_FALLING = 2;
-constexpr uint32_t SERIAL_PI_INPUT = 0;
-constexpr uint32_t SERIAL_PI_OUTPUT = 1;
-constexpr size_t MAX_INTERRUPT_SLOTS = 8;
-constexpr int32_t INTERRUPT_POLL_MS = 5;
-
-struct InterruptSlot {
-    bool used = false;
-    uint32_t pin = 0;
-    uint32_t mode = 0;
-    volatile bool pending = false;
-};
-
-concurrency::Lock interruptMutex;
-InterruptSlot interruptSlots[MAX_INTERRUPT_SLOTS];
-StreamAPI *interruptStreamApi = nullptr;
-concurrency::Periodic *interruptEmitter = nullptr;
-
-int findSlotByPinLocked(uint32_t pin)
-{
-    for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-        if (interruptSlots[i].used && interruptSlots[i].pin == pin) {
-            return (int)i;
-        }
-    }
-    return -1;
-}
-
-int allocateSlotLocked()
-{
-    for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-        if (!interruptSlots[i].used) {
-            return (int)i;
-        }
-    }
-    return -1;
-}
-
-#if defined(ARCH_PORTDUINO) || defined(ARCH_RP2040)
-PinStatus toInterruptMode(uint32_t serialMode)
-{
-    if (serialMode == SERIAL_PI_RISING) {
-        return PinStatus::RISING;
-    }
-    if (serialMode == SERIAL_PI_FALLING) {
-        return PinStatus::FALLING;
-    }
-    return PinStatus::CHANGE;
-}
-#else
-int toInterruptMode(uint32_t serialMode)
-{
-    if (serialMode == SERIAL_PI_RISING) {
-        return RISING;
-    }
-    if (serialMode == SERIAL_PI_FALLING) {
-        return FALLING;
-    }
-    return CHANGE;
-}
-#endif
-
-int32_t pumpInterruptEvents();
-
-void ensureInterruptEmitter()
-{
-    if (!interruptEmitter) {
-        interruptEmitter = new concurrency::Periodic("SerialHalIrqEmitter", pumpInterruptEvents);
-    }
-}
-
-void emitInterruptEvent(uint32_t pin, StreamAPI *streamApi)
-{
-    if (streamApi == nullptr) {
-        return;
-    }
-
-    meshtastic_SerialHalResponse event = meshtastic_SerialHalResponse_init_zero;
-    event.transaction_id = 0; // asynchronous interrupt notification
-    event.result = meshtastic_SerialHalResponse_Result_OK;
-    event.value = pin; // host-side SerialHal treats value as interrupt pin
-    SerialHalDevice::emitResponse(event, streamApi);
-}
-
-void markPendingBySlot(uint8_t slot)
-{
-    if (slot < MAX_INTERRUPT_SLOTS && interruptSlots[slot].used) {
-        interruptSlots[slot].pending = true;
-    }
-}
-
-void isr0()
-{
-    markPendingBySlot(0);
-}
-void isr1()
-{
-    markPendingBySlot(1);
-}
-void isr2()
-{
-    markPendingBySlot(2);
-}
-void isr3()
-{
-    markPendingBySlot(3);
-}
-void isr4()
-{
-    markPendingBySlot(4);
-}
-void isr5()
-{
-    markPendingBySlot(5);
-}
-void isr6()
-{
-    markPendingBySlot(6);
-}
-void isr7()
-{
-    markPendingBySlot(7);
-}
-
-void (*const isrTable[MAX_INTERRUPT_SLOTS])() = {isr0, isr1, isr2, isr3, isr4, isr5, isr6, isr7};
-
-int32_t pumpInterruptEvents()
-{
-    uint32_t toEmit[MAX_INTERRUPT_SLOTS] = {0};
-    size_t emitCount = 0;
-    StreamAPI *streamApi = nullptr;
-
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        streamApi = interruptStreamApi;
-        for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-            if (interruptSlots[i].used && interruptSlots[i].pending) {
-                interruptSlots[i].pending = false;
-                toEmit[emitCount++] = interruptSlots[i].pin;
-            }
-        }
-    }
-
-    for (size_t i = 0; i < emitCount; ++i) {
-        emitInterruptEvent(toEmit[i], streamApi);
-    }
-
-    return INTERRUPT_POLL_MS;
-}
-} // namespace
-
-// Helper to safely set response result
-static inline void setResponseError(meshtastic_SerialHalResponse &response, meshtastic_SerialHalResponse_Result result,
-                                    const char *error = nullptr)
-{
-    response.result = result;
-    if (error != nullptr) {
-        snprintf(response.error, sizeof(response.error), "%s", error);
-    }
-}
-
-void SerialHalDevice::handleCommand(const uint8_t *buf, size_t len, StreamAPI *streamApi)
-{
-    if (buf == nullptr || streamApi == nullptr) {
-        return;
-    }
-
-    // Validate role - SerialHal commands only handled when config.lora.serial_hal_only
-    if (!config.lora.serial_hal_only) {
-        meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-        response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-        snprintf(response.error, sizeof(response.error), "SerialHal not enabled for this role");
-        emitResponse(response, streamApi);
-        return;
-    }
-
-    // Decode the command
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    if (!pb_decode_from_bytes(buf, len, &meshtastic_SerialHalCommand_msg, &cmd)) {
-        meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-        response.result = meshtastic_SerialHalResponse_Result_BAD_REQUEST;
-        snprintf(response.error, sizeof(response.error), "Failed to decode SerialHalCommand");
-        emitResponse(response, streamApi);
-        return;
-    }
-
-    // Initialize response with matching transaction_id
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    response.transaction_id = cmd.transaction_id;
-    response.result = meshtastic_SerialHalResponse_Result_OK;
-
-    // Dispatch to operation handler
-    switch (cmd.type) {
-    case meshtastic_SerialHalCommand_Type_PIN_MODE:
-        handlePinMode(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DIGITAL_WRITE:
-        handleDigitalWrite(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DIGITAL_READ:
-        handleDigitalRead(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_ATTACH_INTERRUPT:
-        handleAttachInterrupt(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DETACH_INTERRUPT:
-        handleDetachInterrupt(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_SPI_TRANSFER:
-        handleSpiTransfer(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_NOOP:
-        // NOOP: just return OK
-        break;
-    default:
-        response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-        snprintf(response.error, sizeof(response.error), "Unknown SerialHal operation type");
-        break;
-    }
-
-    emitResponse(response, streamApi);
-}
-
-void SerialHalDevice::handlePinMode(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: pinMode pin=%u mode=%u", cmd.pin, cmd.mode);
-    if (cmd.mode == SERIAL_PI_INPUT) {
-        pinMode((int)cmd.pin, INPUT);
-    } else if (cmd.mode == SERIAL_PI_OUTPUT) {
-        pinMode((int)cmd.pin, OUTPUT);
-    } else {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_BAD_REQUEST, "Unsupported pin mode");
-    }
-}
-
-void SerialHalDevice::handleDigitalWrite(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: digitalWrite pin=%u value=%u", cmd.pin, cmd.value);
-    digitalWrite((int)cmd.pin, cmd.value ? HIGH : LOW);
-}
-
-void SerialHalDevice::handleDigitalRead(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: digitalRead pin=%u", cmd.pin);
-    response.value = (uint32_t)digitalRead((int)cmd.pin);
-}
-
-void SerialHalDevice::handleAttachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: attachInterrupt pin=%u mode=%u", cmd.pin, cmd.mode);
-
-    ensureInterruptEmitter();
-
-    int slot = -1;
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        slot = findSlotByPinLocked(cmd.pin);
-        if (slot < 0) {
-            slot = allocateSlotLocked();
-        }
-
-        if (slot >= 0) {
-            interruptSlots[slot].used = true;
-            interruptSlots[slot].pin = cmd.pin;
-            interruptSlots[slot].mode = cmd.mode;
-            interruptSlots[slot].pending = false;
-        }
-    }
-
-    if (slot < 0) {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_ERROR, "No interrupt slots available");
-        return;
-    }
-
-    ::attachInterrupt((int)cmd.pin, isrTable[slot], toInterruptMode(cmd.mode));
-}
-
-void SerialHalDevice::handleDetachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: detachInterrupt pin=%u", cmd.pin);
-
-    ::detachInterrupt((int)cmd.pin);
-
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        const int slot = findSlotByPinLocked(cmd.pin);
-        if (slot >= 0) {
-            interruptSlots[slot] = InterruptSlot{};
-        }
-    }
-}
-
-void SerialHalDevice::handleSpiTransfer(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    if (cmd.data.size == 0) {
-        return;
-    }
-
-#if !ARCH_PORTDUINO
-    if (spiLock == nullptr) {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_ERROR, "SPI lock not initialized");
-        return;
-    }
-
-#if defined(HW_SPI1_DEVICE)
-    SPIClass &spiBus = SPI1;
-#else
-    SPIClass &spiBus = SPI;
-#endif
-
-    response.data.size = cmd.data.size;
-
-    {
-        concurrency::LockGuard guard(spiLock);
-        spiBus.beginTransaction(SPISettings(4000000, MSBFIRST, SPI_MODE0));
-#ifdef ARCH_ESP32
-        spiBus.transferBytes(cmd.data.bytes, response.data.bytes, cmd.data.size);
-#else
-        spiBus.transfer(cmd.data.bytes, response.data.bytes, cmd.data.size);
-#endif
-        spiBus.endTransaction();
-    }
-#else
-    // SPI wiring is board/radio-specific; keep this explicit for now.
-    response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-    snprintf(response.error, sizeof(response.error), "SPI not supported on this platform");
-#endif
-}
-
-void SerialHalDevice::emitResponse(const meshtastic_SerialHalResponse &response, StreamAPI *streamApi)
-{
-    if (streamApi == nullptr) {
-        return;
-    }
-
-    // Encode the response
-    uint8_t encoded[meshtastic_SerialHalResponse_size] = {0};
-    const size_t responseLen =
-        pb_encode_to_bytes(encoded, sizeof(encoded), &meshtastic_SerialHalResponse_msg, static_cast<const void *>(&response));
-
-    if (responseLen == 0 || responseLen > 0xFFFF) {
-        LOG_ERROR("SerialHalDevice: Failed to encode response (len=%zu)", responseLen);
-        return;
-    }
-
-    // Build frame with StreamAPI framing: START1 SERIALHAL_MAGIC LEN_H LEN_L [payload]
-    constexpr uint8_t START1 = 0x94;
-    constexpr uint8_t SERIALHAL_MAGIC = 0xA5;
-
-    uint8_t hdr[4];
-    hdr[0] = START1;
-    hdr[1] = SERIALHAL_MAGIC;
-    hdr[2] = (uint8_t)((responseLen >> 8) & 0xFF); // LEN_H
-    hdr[3] = (uint8_t)(responseLen & 0xFF);        // LEN_L
-
-    // Emit via StreamAPI (this uses the internal txBuf + framing)
-    streamApi->emitSerialHalResponse(hdr, sizeof(hdr), encoded, responseLen);
-
-    // Keep a recent stream instance so async interrupt events can be emitted.
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        interruptStreamApi = streamApi;
-    }
-}

src/mesh/SerialHalDevice.h

@@ -1,90 +0,0 @@
-#pragma once
-
-#include "mesh/generated/meshtastic/serial_hal.pb.h"
-#include <cstdint>
-
-/**
- * @brief Device-side handler for SerialHal GPIO/SPI operations over StreamAPI framing.
- *
- * This module decodes SerialHalCommand protobufs received from a host and executes
- * the requested GPIO (pinMode, digitalWrite, digitalRead, attach/detachInterrupt) or
- * SPI operations, then returns results via SerialHalResponse.
- *
- * Usage:
- *  1. Override StreamAPI::handleSerialHalCommand() in a subclass
- *  2. Call SerialHalDevice::handleCommand(buf, len, streamApi)
- *  3. SerialHalDevice will decode, execute, and emit the response
- *
- * The handler is only active when config.lora.serial_hal_only is true.
- */
-
-class StreamAPI; // forward declaration
-
-class SerialHalDevice
-{
-  public:
-    /**
-     * @brief Process a SerialHalCommand and emit a response.
-     *
-     * Decodes the protobuf, validates the operation, executes it on the device,
-     * and writes the response back via the StreamAPI instance.
-     *
-     * @param buf     Pointer to the encoded SerialHalCommand protobuf payload (not including framing)
-     * @param len     Length of the encoded payload
-     * @param streamApi Pointer to the StreamAPI instance (used for emitting responses)
-     */
-    static void handleCommand(const uint8_t *buf, size_t len, StreamAPI *streamApi);
-
-    /**
-     * @brief Emit a SerialHalResponse back to the host via StreamAPI framing.
-     *
-     * Encodes the response protobuf and sends it with proper framing (START1 SERIALHAL_MAGIC LEN_H LEN_L payload).
-     *
-     * @param response The response to send
-     * @param streamApi Pointer to the StreamAPI instance
-     */
-    static void emitResponse(const meshtastic_SerialHalResponse &response, StreamAPI *streamApi);
-
-  private:
-    /**
-     * @brief Execute a GPIO pinMode operation.
-     * @param cmd Decoded SerialHalCommand with PIN_MODE type
-     * @param response Response object to fill with result
-     */
-    static void handlePinMode(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a GPIO digitalWrite operation.
-     * @param cmd Decoded SerialHalCommand with DIGITAL_WRITE type
-     * @param response Response object to fill with result
-     */
-    static void handleDigitalWrite(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a GPIO digitalRead operation.
-     * @param cmd Decoded SerialHalCommand with DIGITAL_READ type
-     * @param response Response object to fill with result (value field contains read result)
-     */
-    static void handleDigitalRead(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute an attachInterrupt operation.
-     * @param cmd Decoded SerialHalCommand with ATTACH_INTERRUPT type
-     * @param response Response object to fill with result
-     */
-    static void handleAttachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a detachInterrupt operation.
-     * @param cmd Decoded SerialHalCommand with DETACH_INTERRUPT type
-     * @param response Response object to fill with result
-     */
-    static void handleDetachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute an SPI transfer operation.
-     * @param cmd Decoded SerialHalCommand with SPI_TRANSFER type and data to send
-     * @param response Response object to fill with result (data field contains received bytes)
-     */
-    static void handleSpiTransfer(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-};

src/mesh/StreamAPI.cpp

@@ -1,15 +1,12 @@
 #include "StreamAPI.h"
 #include "PowerFSM.h"
 #include "RTC.h"
-#include "RedirectablePrint.h"
-#include "SerialHalDevice.h"
 #include "Throttle.h"
 #include "concurrency/LockGuard.h"
 #include "configuration.h"
 
 #define START1 0x94
 #define START2 0xc3
-#define SERIALHAL_MAGIC 0xa5 // second framing byte for SerialHal frames (START1 SH_MAGIC LEN_H LEN_L PAYLOAD)
 #define HEADER_LEN 4
 
 int32_t StreamAPI::runOncePart()
@@ -82,29 +79,18 @@ int32_t StreamAPI::handleRecStream(const char *buf, uint16_t bufLen)
         if (ptr == 0) { // looking for START1
             if (c != START1)
                 rxPtr = 0;     // failed to find framing
-        } else if (ptr == 1) { // discriminate frame type on second byte
-            if (c == START2) {
-                rxIsSerialHal = false; // standard ToRadio frame
-                serialHalRxActive.store(false);
-                RedirectablePrint::setSerialHalLogSuppressed(false);
-            } else if (c == SERIALHAL_MAGIC) {
-                rxIsSerialHal = true; // SerialHal command frame
-                serialHalRxActive.store(true);
-                RedirectablePrint::setSerialHalLogSuppressed(true);
-            } else {
-                rxPtr = 0; // unrecognised second byte — not our frame
-                serialHalRxActive.store(false);
-                RedirectablePrint::setSerialHalLogSuppressed(false);
-            }
+        } else if (ptr == 1) { // looking for START2
+            if (c != START2)
+                rxPtr = 0;                             // failed to find framing
         } else if (ptr >= HEADER_LEN - 1) {            // we have at least read our 4 byte framing
             uint32_t len = (rxBuf[2] << 8) + rxBuf[3]; // big endian 16 bit length follows framing
 
             // console->printf("len %d\n", len);
 
             if (ptr == HEADER_LEN - 1) {
-                // we _just_ finished our 4 byte header, validate length now
-                uint32_t maxLen = rxIsSerialHal ? (uint32_t)meshtastic_SerialHalCommand_size : MAX_TO_FROM_RADIO_SIZE;
-                if (len > maxLen)
+                // we _just_ finished our 4 byte header, validate length now (note: a length of zero is a valid
+                // protobuf also)
+                if (len > MAX_TO_FROM_RADIO_SIZE)
                     rxPtr = 0; // length is bogus, restart search for framing
             }
 
@@ -112,16 +98,8 @@ int32_t StreamAPI::handleRecStream(const char *buf, uint16_t bufLen)
                 if (ptr + 1 >= len + HEADER_LEN) { // have we received all of the payload?
                     rxPtr = 0;                     // start over again on the next packet
 
-                    // Dispatch based on which frame type we identified at byte 1
-                    if (rxIsSerialHal)
-                        handleSerialHalCommand(rxBuf + HEADER_LEN, len);
-                    else
-                        handleToRadio(rxBuf + HEADER_LEN, len);
-
-                    if (rxIsSerialHal)
-                        serialHalRxActive.store(false);
-                    if (rxIsSerialHal)
-                        RedirectablePrint::setSerialHalLogSuppressed(false);
+                    // If we didn't just fail the packet and we now have the right # of bytes, parse it
+                    handleToRadio(rxBuf + HEADER_LEN, len);
                 }
         }
     }
@@ -136,12 +114,7 @@ int32_t StreamAPI::readStream()
     if (!stream->available()) {
         // Nothing available this time, if the computer has talked to us recently, poll often, otherwise let CPU sleep a long time
         bool recentRx = Throttle::isWithinTimespanMs(lastRxMsec, 2000);
-        if (!recentRx)
-            return 250; // Sleep a long time if we haven't heard from the computer in a while
-        if (serialHalRxActive.load())
-            return 0; // If we are in the middle of a SerialHal transaction, don't sleep at all because we want to be as
-                      // responsive as possible to incoming SerialHal bytes
-        return 5;     // Otherwise, poll frequently for new data
+        return recentRx ? 5 : 250;
     } else {
         while (stream->available()) { // Currently we never want to block
             int cInt = stream->read();
@@ -162,30 +135,18 @@ int32_t StreamAPI::readStream()
             if (ptr == 0) { // looking for START1
                 if (c != START1)
                     rxPtr = 0;     // failed to find framing
-            } else if (ptr == 1) { // discriminate frame type on second byte
-                if (c == START2) {
-                    rxIsSerialHal = false; // standard ToRadio frame
-                    serialHalRxActive.store(false);
-                    RedirectablePrint::setSerialHalLogSuppressed(false);
-                } else if (c == SERIALHAL_MAGIC) {
-                    rxIsSerialHal = true; // SerialHal command frame
-                    serialHalRxActive.store(true);
-                    RedirectablePrint::setSerialHalLogSuppressed(true);
-                    LOG_WARN("StreamAPI: Detected SerialHal command frame");
-                } else {
-                    rxPtr = 0; // unrecognised second byte — not our frame
-                    serialHalRxActive.store(false);
-                    RedirectablePrint::setSerialHalLogSuppressed(false);
-                }
+            } else if (ptr == 1) { // looking for START2
+                if (c != START2)
+                    rxPtr = 0;                             // failed to find framing
             } else if (ptr >= HEADER_LEN - 1) {            // we have at least read our 4 byte framing
                 uint32_t len = (rxBuf[2] << 8) + rxBuf[3]; // big endian 16 bit length follows framing
 
                 // console->printf("len %d\n", len);
 
                 if (ptr == HEADER_LEN - 1) {
-                    // we _just_ finished our 4 byte header, validate length now
-                    uint32_t maxLen = rxIsSerialHal ? (uint32_t)meshtastic_SerialHalCommand_size : MAX_TO_FROM_RADIO_SIZE;
-                    if (len > maxLen)
+                    // we _just_ finished our 4 byte header, validate length now (note: a length of zero is a valid
+                    // protobuf also)
+                    if (len > MAX_TO_FROM_RADIO_SIZE)
                         rxPtr = 0; // length is bogus, restart search for framing
                 }
 
@@ -193,16 +154,8 @@ int32_t StreamAPI::readStream()
                     if (ptr + 1 >= len + HEADER_LEN) { // have we received all of the payload?
                         rxPtr = 0;                     // start over again on the next packet
 
-                        // Dispatch based on which frame type we identified at byte 1
-                        if (rxIsSerialHal)
-                            handleSerialHalCommand(rxBuf + HEADER_LEN, len);
-                        else
-                            handleToRadio(rxBuf + HEADER_LEN, len);
-
-                        if (rxIsSerialHal)
-                            serialHalRxActive.store(false);
-                        if (rxIsSerialHal)
-                            RedirectablePrint::setSerialHalLogSuppressed(false);
+                        // If we didn't just fail the packet and we now have the right # of bytes, parse it
+                        handleToRadio(rxBuf + HEADER_LEN, len);
                     }
             }
         }
@@ -246,10 +199,6 @@ void StreamAPI::emitRebooted()
 
 void StreamAPI::emitLogRecord(meshtastic_LogRecord_Level level, const char *src, const char *format, va_list arg)
 {
-    if (serialHalRxActive.load()) {
-        return;
-    }
-
     // IMPORTANT: do NOT touch `fromRadioScratch` or `txBuf` here — those
     // belong to the main packet-emission path and a LOG_ firing during
     // `writeStream()` would corrupt an in-flight encode. We keep a
@@ -300,31 +249,4 @@ void StreamAPI::onConnectionChanged(bool connected)
         // received a packet in a while
         powerFSM.trigger(EVENT_SERIAL_DISCONNECTED);
     }
-}
-
-void StreamAPI::handleSerialHalCommand(const uint8_t *buf, size_t len)
-{
-    // Default implementation: dispatch to SerialHalDevice for GPIO/SPI handling
-    SerialHalDevice::handleCommand(buf, len, this);
-}
-
-void StreamAPI::emitSerialHalResponse(const uint8_t *hdr, size_t hdrLen, const uint8_t *payload, size_t payloadLen)
-{
-    if (hdr == nullptr || hdrLen != 4 || payload == nullptr || payloadLen > meshtastic_SerialHalResponse_size) {
-        LOG_ERROR("StreamAPI: Invalid SerialHal response parameters");
-        return;
-    }
-
-    // Build complete frame in a temporary buffer
-    uint8_t frame[4 + meshtastic_SerialHalResponse_size];
-    memcpy(frame, hdr, hdrLen);
-    memcpy(frame + hdrLen, payload, payloadLen);
-
-    size_t totalLen = hdrLen + payloadLen;
-
-    // Serialize stream writes against other emit operations via streamLock
-    concurrency::LockGuard guard(&streamLock);
-    stream->write(frame, totalLen);
-    stream->flush();
-    LOG_WARN("StreamAPI: Emitted SerialHal response frame (len=%zu)", totalLen);
 }

src/mesh/StreamAPI.h

@@ -4,15 +4,10 @@
 #include "Stream.h"
 #include "concurrency/Lock.h"
 #include "concurrency/OSThread.h"
-#include "generated/meshtastic/serial_hal.pb.h"
-#include <atomic>
 #include <cstdarg>
 
-// Buffer sized for the larger of a full ToRadio/FromRadio payload or a full SerialHalCommand payload, plus header.
-#define MAX_STREAM_PAYLOAD_SIZE                                                                                                  \
-    (MAX_TO_FROM_RADIO_SIZE > (int)meshtastic_SerialHalCommand_size ? MAX_TO_FROM_RADIO_SIZE                                     \
-                                                                    : (int)meshtastic_SerialHalCommand_size)
-#define MAX_STREAM_BUF_SIZE (MAX_STREAM_PAYLOAD_SIZE + (int)sizeof(uint32_t))
+// A To/FromRadio packet + our 32 bit header
+#define MAX_STREAM_BUF_SIZE (MAX_TO_FROM_RADIO_SIZE + sizeof(uint32_t))
 
 /**
  * A version of our 'phone' API that talks over a Stream.  So therefore well suited to use with serial links
@@ -44,8 +39,6 @@ class StreamAPI : public PhoneAPI
 
     uint8_t rxBuf[MAX_STREAM_BUF_SIZE] = {0};
     size_t rxPtr = 0;
-    bool rxIsSerialHal = false; ///< true when the current in-progress frame is a SerialHal frame (START1 SH_MAGIC ...)
-    std::atomic<bool> serialHalRxActive{false};
 
     /// time of last rx, used, to slow down our polling if we haven't heard from anyone
     uint32_t lastRxMsec = 0;
@@ -63,17 +56,6 @@ class StreamAPI : public PhoneAPI
     /// Check the current underlying physical link to see if the client is currently connected
     virtual bool checkIsConnected() override = 0;
 
-    /**
-     * Emit a SerialHal response frame with proper framing (START1 SERIALHAL_MAGIC LEN_H LEN_L payload).
-     * Called by SerialHalDevice to send responses back to the host.
-     *
-     * @param hdr     4-byte header (START1 SERIALHAL_MAGIC LEN_H LEN_L)
-     * @param hdrLen  Length of header (should be 4)
-     * @param payload Encoded SerialHalResponse protobuf payload
-     * @param payloadLen Length of payload
-     */
-    void emitSerialHalResponse(const uint8_t *hdr, size_t hdrLen, const uint8_t *payload, size_t payloadLen);
-
   private:
     /**
      * Read any rx chars from the link and call handleToRadio
@@ -93,12 +75,6 @@ class StreamAPI : public PhoneAPI
      */
     void emitRebooted();
 
-    /**
-     * Called when a complete SerialHal-framed packet has been received.
-     * Default implementation dispatches to SerialHalDevice for GPIO/SPI handling.
-     */
-    virtual void handleSerialHalCommand(const uint8_t *buf, size_t len);
-
     virtual void onConnectionChanged(bool connected) override;
 
     /**

src/mesh/TypeConversions.cpp

@@ -3,7 +3,9 @@
 #include "mesh/generated/meshtastic/mesh.pb.h"
 #include "meshUtils.h"
 
-meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfoLite *lite)
+meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfoLite *lite,
+                                                       const meshtastic_PositionLite *position,
+                                                       const meshtastic_DeviceMetrics *deviceMetrics)
 {
     meshtastic_NodeInfo info = meshtastic_NodeInfo_init_default;
 
@@ -11,42 +13,56 @@ meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfo
     info.snr = lite->snr;
     info.last_heard = lite->last_heard;
     info.channel = lite->channel;
-    info.via_mqtt = lite->via_mqtt;
-    info.is_favorite = lite->is_favorite;
-    info.is_ignored = lite->is_ignored;
-    info.is_key_manually_verified = lite->bitfield & NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK;
-    info.is_muted = lite->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK;
+    info.via_mqtt = nodeInfoLiteViaMqtt(lite);
+    info.is_favorite = nodeInfoLiteIsFavorite(lite);
+    info.is_ignored = nodeInfoLiteIsIgnored(lite);
+    info.is_key_manually_verified = nodeInfoLiteIsKeyManuallyVerified(lite);
+    info.is_muted = nodeInfoLiteIsMuted(lite);
 
     if (lite->has_hops_away) {
         info.has_hops_away = true;
         info.hops_away = lite->hops_away;
     }
 
-    if (lite->has_position) {
+    if (position) {
         info.has_position = true;
-        if (lite->position.latitude_i != 0)
+        if (position->latitude_i != 0)
             info.position.has_latitude_i = true;
-        info.position.latitude_i = lite->position.latitude_i;
-        if (lite->position.longitude_i != 0)
+        info.position.latitude_i = position->latitude_i;
+        if (position->longitude_i != 0)
             info.position.has_longitude_i = true;
-        info.position.longitude_i = lite->position.longitude_i;
-        if (lite->position.altitude != 0)
+        info.position.longitude_i = position->longitude_i;
+        if (position->altitude != 0)
             info.position.has_altitude = true;
-        info.position.altitude = lite->position.altitude;
-        info.position.location_source = lite->position.location_source;
-        info.position.time = lite->position.time;
+        info.position.altitude = position->altitude;
+        info.position.location_source = position->location_source;
+        info.position.time = position->time;
     }
-    if (lite->has_user) {
+    if (nodeInfoLiteHasUser(lite)) {
         info.has_user = true;
-        info.user = ConvertToUser(lite->num, lite->user);
+        info.user = ConvertToUser(lite);
     }
-    if (lite->has_device_metrics) {
+    if (deviceMetrics) {
         info.has_device_metrics = true;
-        info.device_metrics = lite->device_metrics;
+        info.device_metrics = *deviceMetrics;
     }
     return info;
 }
 
+meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfoLite *lite)
+{
+    meshtastic_PositionLite posScratch;
+    meshtastic_DeviceMetrics dmScratch;
+    const meshtastic_PositionLite *pos = (nodeDB && nodeDB->copyNodePosition(lite->num, posScratch)) ? &posScratch : nullptr;
+    const meshtastic_DeviceMetrics *dm = (nodeDB && nodeDB->copyNodeTelemetry(lite->num, dmScratch)) ? &dmScratch : nullptr;
+    return ConvertToNodeInfo(lite, pos, dm);
+}
+
+meshtastic_NodeInfo TypeConversions::ConvertToNodeInfoThin(const meshtastic_NodeInfoLite *lite)
+{
+    return ConvertToNodeInfo(lite, nullptr, nullptr);
+}
+
 meshtastic_PositionLite TypeConversions::ConvertToPositionLite(meshtastic_Position position)
 {
     meshtastic_PositionLite lite = meshtastic_PositionLite_init_default;
@@ -77,46 +93,50 @@ meshtastic_Position TypeConversions::ConvertToPosition(meshtastic_PositionLite l
     return position;
 }
 
-meshtastic_UserLite TypeConversions::ConvertToUserLite(meshtastic_User user)
+void TypeConversions::CopyUserToNodeInfoLite(meshtastic_NodeInfoLite *lite, const meshtastic_User &user)
 {
-    meshtastic_UserLite lite = meshtastic_UserLite_init_default;
+    if (!lite)
+        return;
 
-    strncpy(lite.long_name, user.long_name, sizeof(lite.long_name));
-    lite.long_name[sizeof(lite.long_name) - 1] = '\0';
-    sanitizeUtf8(lite.long_name, sizeof(lite.long_name));
-    strncpy(lite.short_name, user.short_name, sizeof(lite.short_name));
-    lite.short_name[sizeof(lite.short_name) - 1] = '\0';
-    sanitizeUtf8(lite.short_name, sizeof(lite.short_name));
-    lite.hw_model = user.hw_model;
-    lite.role = user.role;
-    lite.is_licensed = user.is_licensed;
-    memcpy(lite.macaddr, user.macaddr, sizeof(lite.macaddr));
-    memcpy(lite.public_key.bytes, user.public_key.bytes, sizeof(lite.public_key.bytes));
-    lite.public_key.size = user.public_key.size;
-    lite.has_is_unmessagable = user.has_is_unmessagable;
-    lite.is_unmessagable = user.is_unmessagable;
-    return lite;
+    strncpy(lite->long_name, user.long_name, sizeof(lite->long_name));
+    lite->long_name[sizeof(lite->long_name) - 1] = '\0';
+    sanitizeUtf8(lite->long_name, sizeof(lite->long_name));
+    strncpy(lite->short_name, user.short_name, sizeof(lite->short_name));
+    lite->short_name[sizeof(lite->short_name) - 1] = '\0';
+    sanitizeUtf8(lite->short_name, sizeof(lite->short_name));
+    lite->hw_model = user.hw_model;
+    lite->role = user.role;
+    memcpy(lite->public_key.bytes, user.public_key.bytes, sizeof(lite->public_key.bytes));
+    lite->public_key.size = user.public_key.size;
+
+    nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_IS_LICENSED_MASK, user.is_licensed);
+    nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_HAS_IS_UNMESSAGABLE_MASK, user.has_is_unmessagable);
+    nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_IS_UNMESSAGABLE_MASK, user.has_is_unmessagable && user.is_unmessagable);
+    nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_HAS_USER_MASK, true);
 }
 
-meshtastic_User TypeConversions::ConvertToUser(uint32_t nodeNum, meshtastic_UserLite lite)
+meshtastic_User TypeConversions::ConvertToUser(const meshtastic_NodeInfoLite *lite)
 {
     meshtastic_User user = meshtastic_User_init_default;
+    if (!lite)
+        return user;
 
-    snprintf(user.id, sizeof(user.id), "!%08x", nodeNum);
-    strncpy(user.long_name, lite.long_name, sizeof(user.long_name));
+    snprintf(user.id, sizeof(user.id), "!%08x", lite->num);
+    strncpy(user.long_name, lite->long_name, sizeof(user.long_name));
     user.long_name[sizeof(user.long_name) - 1] = '\0';
     sanitizeUtf8(user.long_name, sizeof(user.long_name));
-    strncpy(user.short_name, lite.short_name, sizeof(user.short_name));
+    strncpy(user.short_name, lite->short_name, sizeof(user.short_name));
     user.short_name[sizeof(user.short_name) - 1] = '\0';
     sanitizeUtf8(user.short_name, sizeof(user.short_name));
-    user.hw_model = lite.hw_model;
-    user.role = lite.role;
-    user.is_licensed = lite.is_licensed;
-    memcpy(user.macaddr, lite.macaddr, sizeof(user.macaddr));
-    memcpy(user.public_key.bytes, lite.public_key.bytes, sizeof(user.public_key.bytes));
-    user.public_key.size = lite.public_key.size;
-    user.has_is_unmessagable = lite.has_is_unmessagable;
-    user.is_unmessagable = lite.is_unmessagable;
+    user.hw_model = lite->hw_model;
+    user.role = lite->role;
+    user.is_licensed = nodeInfoLiteIsLicensed(lite);
+    memcpy(user.public_key.bytes, lite->public_key.bytes, sizeof(user.public_key.bytes));
+    user.public_key.size = lite->public_key.size;
+    user.has_is_unmessagable = nodeInfoLiteHasIsUnmessagable(lite);
+    user.is_unmessagable = nodeInfoLiteIsUnmessagable(lite);
+    // macaddr is gone from the slim header; zero-fill for old clients that read it.
+    memset(user.macaddr, 0, sizeof(user.macaddr));
 
     return user;
-}
+}

src/mesh/TypeConversions.h

@@ -7,9 +7,16 @@
 class TypeConversions
 {
   public:
+    // Either pointer may be null; the corresponding has_* fields stay unset.
+    static meshtastic_NodeInfo ConvertToNodeInfo(const meshtastic_NodeInfoLite *lite, const meshtastic_PositionLite *position,
+                                                 const meshtastic_DeviceMetrics *deviceMetrics);
     static meshtastic_NodeInfo ConvertToNodeInfo(const meshtastic_NodeInfoLite *lite);
+    // Identity + link-state only; satellite payloads are replayed afterward.
+    static meshtastic_NodeInfo ConvertToNodeInfoThin(const meshtastic_NodeInfoLite *lite);
+
     static meshtastic_PositionLite ConvertToPositionLite(meshtastic_Position position);
     static meshtastic_Position ConvertToPosition(meshtastic_PositionLite lite);
-    static meshtastic_UserLite ConvertToUserLite(meshtastic_User user);
-    static meshtastic_User ConvertToUser(uint32_t nodeNum, meshtastic_UserLite lite);
+
+    static void CopyUserToNodeInfoLite(meshtastic_NodeInfoLite *lite, const meshtastic_User &user);
+    static meshtastic_User ConvertToUser(const meshtastic_NodeInfoLite *lite);
 };

src/mesh/api/ethServerAPI.cpp

@@ -1,7 +1,7 @@
 #include "configuration.h"
 #include <Arduino.h>
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 
 #include "ethServerAPI.h"
 

src/mesh/api/ethServerAPI.h

@@ -1,7 +1,7 @@
 #pragma once
 
 #include "ServerAPI.h"
-#ifndef USE_WS5500
+#if !defined(USE_WS5500) && !defined(USE_CH390D)
 #include <RAK13800_W5100S.h>
 
 /**

src/mesh/eth/ethClient.cpp

@@ -9,7 +9,7 @@
 #include <RAK13800_W5100S.h>
 #include <SPI.h>
 
-#if HAS_NETWORKING
+#if HAS_NETWORKING && !defined(USE_WS5500) && !defined(USE_CH390D)
 
 #ifndef DISABLE_NTP
 #include <NTPClient.h>

src/mesh/generated/meshtastic/deviceonly.pb.cpp

@@ -18,6 +18,18 @@ PB_BIND(meshtastic_NodeInfoLite, meshtastic_NodeInfoLite, AUTO)
 PB_BIND(meshtastic_DeviceState, meshtastic_DeviceState, 2)
 
 
+PB_BIND(meshtastic_NodePositionEntry, meshtastic_NodePositionEntry, AUTO)
+
+
+PB_BIND(meshtastic_NodeTelemetryEntry, meshtastic_NodeTelemetryEntry, AUTO)
+
+
+PB_BIND(meshtastic_NodeEnvironmentEntry, meshtastic_NodeEnvironmentEntry, AUTO)
+
+
+PB_BIND(meshtastic_NodeStatusEntry, meshtastic_NodeStatusEntry, AUTO)
+
+
 PB_BIND(meshtastic_NodeDatabase, meshtastic_NodeDatabase, AUTO)
 
 

src/mesh/generated/meshtastic/deviceonly.pb.h

@@ -63,43 +63,35 @@ typedef struct _meshtastic_UserLite {
     bool is_unmessagable;
 } meshtastic_UserLite;
 
+typedef PB_BYTES_ARRAY_T(32) meshtastic_NodeInfoLite_public_key_t;
 typedef struct _meshtastic_NodeInfoLite {
     /* The node number */
     uint32_t num;
-    /* The user info for this node */
-    bool has_user;
-    meshtastic_UserLite user;
-    /* This position data. Note: before 1.2.14 we would also store the last time we've heard from this node in position.time, that is no longer true.
- Position.time now indicates the last time we received a POSITION from that node. */
-    bool has_position;
-    meshtastic_PositionLite position;
     /* Returns the Signal-to-noise ratio (SNR) of the last received message,
  as measured by the receiver. Return SNR of the last received message in dB */
     float snr;
     /* Set to indicate the last time we received a packet from this node */
     uint32_t last_heard;
-    /* The latest device metrics for the node. */
-    bool has_device_metrics;
-    meshtastic_DeviceMetrics device_metrics;
     /* local channel index we heard that node on. Only populated if its not the default channel. */
     uint8_t channel;
-    /* True if we witnessed the node over MQTT instead of LoRA transport */
-    bool via_mqtt;
     /* Number of hops away from us this node is (0 if direct neighbor) */
     bool has_hops_away;
     uint8_t hops_away;
-    /* True if node is in our favorites list
- Persists between NodeDB internal clean ups */
-    bool is_favorite;
-    /* True if node is in our ignored list
- Persists between NodeDB internal clean ups */
-    bool is_ignored;
     /* Last byte of the node number of the node that should be used as the next hop to reach this node. */
     uint8_t next_hop;
-    /* Bitfield for storing booleans.
- LSB 0 is_key_manually_verified
- LSB 1 is_muted */
+    /* Bitfield for storing booleans. See NODEINFO_BITFIELD_* in src/mesh/NodeDB.h. */
     uint32_t bitfield;
+    /* A full name for this user, i.e. "Kevin Hester". */
+    char long_name[25];
+    /* A VERY short name, ideally two characters or an emoji.
+ Suitable for a tiny OLED screen. */
+    char short_name[5];
+    /* Hardware model the user's device is running. */
+    meshtastic_HardwareModel hw_model;
+    /* The user's role in the mesh. */
+    meshtastic_Config_DeviceConfig_Role role;
+    /* The public key of the user's device, for PKI-based encrypted DMs. */
+    meshtastic_NodeInfoLite_public_key_t public_key;
 } meshtastic_NodeInfoLite;
 
 /* This message is never sent over the wire, but it is used for serializing DB
@@ -143,6 +135,30 @@ typedef struct _meshtastic_DeviceState {
     meshtastic_NodeRemoteHardwarePin node_remote_hardware_pins[12];
 } meshtastic_DeviceState;
 
+typedef struct _meshtastic_NodePositionEntry {
+    uint32_t num;
+    bool has_position;
+    meshtastic_PositionLite position;
+} meshtastic_NodePositionEntry;
+
+typedef struct _meshtastic_NodeTelemetryEntry {
+    uint32_t num;
+    bool has_device_metrics;
+    meshtastic_DeviceMetrics device_metrics;
+} meshtastic_NodeTelemetryEntry;
+
+typedef struct _meshtastic_NodeEnvironmentEntry {
+    uint32_t num;
+    bool has_environment_metrics;
+    meshtastic_EnvironmentMetrics environment_metrics;
+} meshtastic_NodeEnvironmentEntry;
+
+typedef struct _meshtastic_NodeStatusEntry {
+    uint32_t num;
+    bool has_status;
+    meshtastic_StatusMessage status;
+} meshtastic_NodeStatusEntry;
+
 typedef struct _meshtastic_NodeDatabase {
     /* A version integer used to invalidate old save files when we make
  incompatible changes This integer is set at build time and is private to
@@ -150,6 +166,12 @@ typedef struct _meshtastic_NodeDatabase {
     uint32_t version;
     /* New lite version of NodeDB to decrease memory footprint */
     std::vector<meshtastic_NodeInfoLite> nodes;
+    /* Per-NodeNum satellite arrays. Constrained platforms (e.g. STM32WL) omit
+ these via MESHTASTIC_EXCLUDE_*DB build flags. */
+    std::vector<meshtastic_NodePositionEntry> positions;
+    std::vector<meshtastic_NodeTelemetryEntry> telemetry;
+    std::vector<meshtastic_NodeStatusEntry> status;
+    std::vector<meshtastic_NodeEnvironmentEntry> environment;
 } meshtastic_NodeDatabase;
 
 /* The on-disk saved channels */
@@ -191,16 +213,24 @@ extern "C" {
 /* Initializer values for message structs */
 #define meshtastic_PositionLite_init_default     {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN}
 #define meshtastic_UserLite_init_default         {{0}, "", "", _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
-#define meshtastic_NodeInfoLite_init_default     {0, false, meshtastic_UserLite_init_default, false, meshtastic_PositionLite_init_default, 0, 0, false, meshtastic_DeviceMetrics_init_default, 0, 0, false, 0, 0, 0, 0, 0}
+#define meshtastic_NodeInfoLite_init_default     {0, 0, 0, 0, false, 0, 0, 0, "", "", _meshtastic_HardwareModel_MIN, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}}
 #define meshtastic_DeviceState_init_default      {false, meshtastic_MyNodeInfo_init_default, false, meshtastic_User_init_default, 0, {meshtastic_MeshPacket_init_default}, false, meshtastic_MeshPacket_init_default, 0, 0, 0, false, meshtastic_MeshPacket_init_default, 0, {meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default}}
-#define meshtastic_NodeDatabase_init_default     {0, {0}}
+#define meshtastic_NodePositionEntry_init_default {0, false, meshtastic_PositionLite_init_default}
+#define meshtastic_NodeTelemetryEntry_init_default {0, false, meshtastic_DeviceMetrics_init_default}
+#define meshtastic_NodeEnvironmentEntry_init_default {0, false, meshtastic_EnvironmentMetrics_init_default}
+#define meshtastic_NodeStatusEntry_init_default  {0, false, meshtastic_StatusMessage_init_default}
+#define meshtastic_NodeDatabase_init_default     {0, {0}, {0}, {0}, {0}, {0}}
 #define meshtastic_ChannelFile_init_default      {0, {meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default}, 0}
 #define meshtastic_BackupPreferences_init_default {0, 0, false, meshtastic_LocalConfig_init_default, false, meshtastic_LocalModuleConfig_init_default, false, meshtastic_ChannelFile_init_default, false, meshtastic_User_init_default}
 #define meshtastic_PositionLite_init_zero        {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN}
 #define meshtastic_UserLite_init_zero            {{0}, "", "", _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
-#define meshtastic_NodeInfoLite_init_zero        {0, false, meshtastic_UserLite_init_zero, false, meshtastic_PositionLite_init_zero, 0, 0, false, meshtastic_DeviceMetrics_init_zero, 0, 0, false, 0, 0, 0, 0, 0}
+#define meshtastic_NodeInfoLite_init_zero        {0, 0, 0, 0, false, 0, 0, 0, "", "", _meshtastic_HardwareModel_MIN, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}}
 #define meshtastic_DeviceState_init_zero         {false, meshtastic_MyNodeInfo_init_zero, false, meshtastic_User_init_zero, 0, {meshtastic_MeshPacket_init_zero}, false, meshtastic_MeshPacket_init_zero, 0, 0, 0, false, meshtastic_MeshPacket_init_zero, 0, {meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero}}
-#define meshtastic_NodeDatabase_init_zero        {0, {0}}
+#define meshtastic_NodePositionEntry_init_zero   {0, false, meshtastic_PositionLite_init_zero}
+#define meshtastic_NodeTelemetryEntry_init_zero  {0, false, meshtastic_DeviceMetrics_init_zero}
+#define meshtastic_NodeEnvironmentEntry_init_zero {0, false, meshtastic_EnvironmentMetrics_init_zero}
+#define meshtastic_NodeStatusEntry_init_zero     {0, false, meshtastic_StatusMessage_init_zero}
+#define meshtastic_NodeDatabase_init_zero        {0, {0}, {0}, {0}, {0}, {0}}
 #define meshtastic_ChannelFile_init_zero         {0, {meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero, meshtastic_Channel_init_zero}, 0}
 #define meshtastic_BackupPreferences_init_zero   {0, 0, false, meshtastic_LocalConfig_init_zero, false, meshtastic_LocalModuleConfig_init_zero, false, meshtastic_ChannelFile_init_zero, false, meshtastic_User_init_zero}
 
@@ -219,18 +249,17 @@ extern "C" {
 #define meshtastic_UserLite_public_key_tag       7
 #define meshtastic_UserLite_is_unmessagable_tag  9
 #define meshtastic_NodeInfoLite_num_tag          1
-#define meshtastic_NodeInfoLite_user_tag         2
-#define meshtastic_NodeInfoLite_position_tag     3
 #define meshtastic_NodeInfoLite_snr_tag          4
 #define meshtastic_NodeInfoLite_last_heard_tag   5
-#define meshtastic_NodeInfoLite_device_metrics_tag 6
 #define meshtastic_NodeInfoLite_channel_tag      7
-#define meshtastic_NodeInfoLite_via_mqtt_tag     8
 #define meshtastic_NodeInfoLite_hops_away_tag    9
-#define meshtastic_NodeInfoLite_is_favorite_tag  10
-#define meshtastic_NodeInfoLite_is_ignored_tag   11
 #define meshtastic_NodeInfoLite_next_hop_tag     12
 #define meshtastic_NodeInfoLite_bitfield_tag     13
+#define meshtastic_NodeInfoLite_long_name_tag    14
+#define meshtastic_NodeInfoLite_short_name_tag   15
+#define meshtastic_NodeInfoLite_hw_model_tag     16
+#define meshtastic_NodeInfoLite_role_tag         17
+#define meshtastic_NodeInfoLite_public_key_tag   18
 #define meshtastic_DeviceState_my_node_tag       2
 #define meshtastic_DeviceState_owner_tag         3
 #define meshtastic_DeviceState_receive_queue_tag 5
@@ -240,8 +269,20 @@ extern "C" {
 #define meshtastic_DeviceState_did_gps_reset_tag 11
 #define meshtastic_DeviceState_rx_waypoint_tag   12
 #define meshtastic_DeviceState_node_remote_hardware_pins_tag 13
+#define meshtastic_NodePositionEntry_num_tag     1
+#define meshtastic_NodePositionEntry_position_tag 2
+#define meshtastic_NodeTelemetryEntry_num_tag    1
+#define meshtastic_NodeTelemetryEntry_device_metrics_tag 2
+#define meshtastic_NodeEnvironmentEntry_num_tag  1
+#define meshtastic_NodeEnvironmentEntry_environment_metrics_tag 2
+#define meshtastic_NodeStatusEntry_num_tag       1
+#define meshtastic_NodeStatusEntry_status_tag    2
 #define meshtastic_NodeDatabase_version_tag      1
 #define meshtastic_NodeDatabase_nodes_tag        2
+#define meshtastic_NodeDatabase_positions_tag    3
+#define meshtastic_NodeDatabase_telemetry_tag    4
+#define meshtastic_NodeDatabase_status_tag       5
+#define meshtastic_NodeDatabase_environment_tag  6
 #define meshtastic_ChannelFile_channels_tag      1
 #define meshtastic_ChannelFile_version_tag       2
 #define meshtastic_BackupPreferences_version_tag 1
@@ -275,23 +316,19 @@ X(a, STATIC,   OPTIONAL, BOOL,     is_unmessagable,   9)
 
 #define meshtastic_NodeInfoLite_FIELDLIST(X, a) \
 X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
-X(a, STATIC,   OPTIONAL, MESSAGE,  user,              2) \
-X(a, STATIC,   OPTIONAL, MESSAGE,  position,          3) \
 X(a, STATIC,   SINGULAR, FLOAT,    snr,               4) \
 X(a, STATIC,   SINGULAR, FIXED32,  last_heard,        5) \
-X(a, STATIC,   OPTIONAL, MESSAGE,  device_metrics,    6) \
 X(a, STATIC,   SINGULAR, UINT32,   channel,           7) \
-X(a, STATIC,   SINGULAR, BOOL,     via_mqtt,          8) \
 X(a, STATIC,   OPTIONAL, UINT32,   hops_away,         9) \
-X(a, STATIC,   SINGULAR, BOOL,     is_favorite,      10) \
-X(a, STATIC,   SINGULAR, BOOL,     is_ignored,       11) \
 X(a, STATIC,   SINGULAR, UINT32,   next_hop,         12) \
-X(a, STATIC,   SINGULAR, UINT32,   bitfield,         13)
+X(a, STATIC,   SINGULAR, UINT32,   bitfield,         13) \
+X(a, STATIC,   SINGULAR, STRING,   long_name,        14) \
+X(a, STATIC,   SINGULAR, STRING,   short_name,       15) \
+X(a, STATIC,   SINGULAR, UENUM,    hw_model,         16) \
+X(a, STATIC,   SINGULAR, UENUM,    role,             17) \
+X(a, STATIC,   SINGULAR, BYTES,    public_key,       18)
 #define meshtastic_NodeInfoLite_CALLBACK NULL
 #define meshtastic_NodeInfoLite_DEFAULT NULL
-#define meshtastic_NodeInfoLite_user_MSGTYPE meshtastic_UserLite
-#define meshtastic_NodeInfoLite_position_MSGTYPE meshtastic_PositionLite
-#define meshtastic_NodeInfoLite_device_metrics_MSGTYPE meshtastic_DeviceMetrics
 
 #define meshtastic_DeviceState_FIELDLIST(X, a) \
 X(a, STATIC,   OPTIONAL, MESSAGE,  my_node,           2) \
@@ -312,13 +349,49 @@ X(a, STATIC,   REPEATED, MESSAGE,  node_remote_hardware_pins,  13)
 #define meshtastic_DeviceState_rx_waypoint_MSGTYPE meshtastic_MeshPacket
 #define meshtastic_DeviceState_node_remote_hardware_pins_MSGTYPE meshtastic_NodeRemoteHardwarePin
 
+#define meshtastic_NodePositionEntry_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  position,          2)
+#define meshtastic_NodePositionEntry_CALLBACK NULL
+#define meshtastic_NodePositionEntry_DEFAULT NULL
+#define meshtastic_NodePositionEntry_position_MSGTYPE meshtastic_PositionLite
+
+#define meshtastic_NodeTelemetryEntry_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  device_metrics,    2)
+#define meshtastic_NodeTelemetryEntry_CALLBACK NULL
+#define meshtastic_NodeTelemetryEntry_DEFAULT NULL
+#define meshtastic_NodeTelemetryEntry_device_metrics_MSGTYPE meshtastic_DeviceMetrics
+
+#define meshtastic_NodeEnvironmentEntry_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  environment_metrics,   2)
+#define meshtastic_NodeEnvironmentEntry_CALLBACK NULL
+#define meshtastic_NodeEnvironmentEntry_DEFAULT NULL
+#define meshtastic_NodeEnvironmentEntry_environment_metrics_MSGTYPE meshtastic_EnvironmentMetrics
+
+#define meshtastic_NodeStatusEntry_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  status,            2)
+#define meshtastic_NodeStatusEntry_CALLBACK NULL
+#define meshtastic_NodeStatusEntry_DEFAULT NULL
+#define meshtastic_NodeStatusEntry_status_MSGTYPE meshtastic_StatusMessage
+
 #define meshtastic_NodeDatabase_FIELDLIST(X, a) \
 X(a, STATIC,   SINGULAR, UINT32,   version,           1) \
-X(a, CALLBACK, REPEATED, MESSAGE,  nodes,             2)
+X(a, CALLBACK, REPEATED, MESSAGE,  nodes,             2) \
+X(a, CALLBACK, REPEATED, MESSAGE,  positions,         3) \
+X(a, CALLBACK, REPEATED, MESSAGE,  telemetry,         4) \
+X(a, CALLBACK, REPEATED, MESSAGE,  status,            5) \
+X(a, CALLBACK, REPEATED, MESSAGE,  environment,       6)
 extern bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field);
 #define meshtastic_NodeDatabase_CALLBACK meshtastic_NodeDatabase_callback
 #define meshtastic_NodeDatabase_DEFAULT NULL
 #define meshtastic_NodeDatabase_nodes_MSGTYPE meshtastic_NodeInfoLite
+#define meshtastic_NodeDatabase_positions_MSGTYPE meshtastic_NodePositionEntry
+#define meshtastic_NodeDatabase_telemetry_MSGTYPE meshtastic_NodeTelemetryEntry
+#define meshtastic_NodeDatabase_status_MSGTYPE meshtastic_NodeStatusEntry
+#define meshtastic_NodeDatabase_environment_MSGTYPE meshtastic_NodeEnvironmentEntry
 
 #define meshtastic_ChannelFile_FIELDLIST(X, a) \
 X(a, STATIC,   REPEATED, MESSAGE,  channels,          1) \
@@ -345,6 +418,10 @@ extern const pb_msgdesc_t meshtastic_PositionLite_msg;
 extern const pb_msgdesc_t meshtastic_UserLite_msg;
 extern const pb_msgdesc_t meshtastic_NodeInfoLite_msg;
 extern const pb_msgdesc_t meshtastic_DeviceState_msg;
+extern const pb_msgdesc_t meshtastic_NodePositionEntry_msg;
+extern const pb_msgdesc_t meshtastic_NodeTelemetryEntry_msg;
+extern const pb_msgdesc_t meshtastic_NodeEnvironmentEntry_msg;
+extern const pb_msgdesc_t meshtastic_NodeStatusEntry_msg;
 extern const pb_msgdesc_t meshtastic_NodeDatabase_msg;
 extern const pb_msgdesc_t meshtastic_ChannelFile_msg;
 extern const pb_msgdesc_t meshtastic_BackupPreferences_msg;
@@ -354,6 +431,10 @@ extern const pb_msgdesc_t meshtastic_BackupPreferences_msg;
 #define meshtastic_UserLite_fields &meshtastic_UserLite_msg
 #define meshtastic_NodeInfoLite_fields &meshtastic_NodeInfoLite_msg
 #define meshtastic_DeviceState_fields &meshtastic_DeviceState_msg
+#define meshtastic_NodePositionEntry_fields &meshtastic_NodePositionEntry_msg
+#define meshtastic_NodeTelemetryEntry_fields &meshtastic_NodeTelemetryEntry_msg
+#define meshtastic_NodeEnvironmentEntry_fields &meshtastic_NodeEnvironmentEntry_msg
+#define meshtastic_NodeStatusEntry_fields &meshtastic_NodeStatusEntry_msg
 #define meshtastic_NodeDatabase_fields &meshtastic_NodeDatabase_msg
 #define meshtastic_ChannelFile_fields &meshtastic_ChannelFile_msg
 #define meshtastic_BackupPreferences_fields &meshtastic_BackupPreferences_msg
@@ -364,7 +445,11 @@ extern const pb_msgdesc_t meshtastic_BackupPreferences_msg;
 #define meshtastic_BackupPreferences_size        2432
 #define meshtastic_ChannelFile_size              718
 #define meshtastic_DeviceState_size              1737
-#define meshtastic_NodeInfoLite_size             196
+#define meshtastic_NodeEnvironmentEntry_size     170
+#define meshtastic_NodeInfoLite_size             105
+#define meshtastic_NodePositionEntry_size        36
+#define meshtastic_NodeStatusEntry_size          89
+#define meshtastic_NodeTelemetryEntry_size       35
 #define meshtastic_PositionLite_size             28
 #define meshtastic_UserLite_size                 98
 

src/mesh/generated/meshtastic/deviceonly_legacy.pb.cpp

@@ -0,0 +1,15 @@
+/* Automatically generated nanopb constant definitions */
+/* Generated by nanopb-0.4.9.1 */
+
+#include "meshtastic/deviceonly_legacy.pb.h"
+#if PB_PROTO_HEADER_VERSION != 40
+#error Regenerate this file with the current version of nanopb generator.
+#endif
+
+PB_BIND(meshtastic_NodeInfoLite_Legacy, meshtastic_NodeInfoLite_Legacy, AUTO)
+
+
+PB_BIND(meshtastic_NodeDatabase_Legacy, meshtastic_NodeDatabase_Legacy, AUTO)
+
+
+

src/mesh/generated/meshtastic/deviceonly_legacy.pb.h

@@ -0,0 +1,124 @@
+/* Automatically generated nanopb header */
+/* Generated by nanopb-0.4.9.1 */
+
+#ifndef PB_MESHTASTIC_MESHTASTIC_DEVICEONLY_LEGACY_PB_H_INCLUDED
+#define PB_MESHTASTIC_MESHTASTIC_DEVICEONLY_LEGACY_PB_H_INCLUDED
+#include <pb.h>
+#include <vector>
+#include "meshtastic/deviceonly.pb.h"
+#include "meshtastic/telemetry.pb.h"
+
+#if PB_PROTO_HEADER_VERSION != 40
+#error Regenerate this file with the current version of nanopb generator.
+#endif
+
+/* Struct definitions */
+/* Legacy NodeInfoLite descriptor used only to decode pre-split
+ /prefs/nodes.proto saves during the v24 -> v25 migration boot.
+ This preserves the original NodeInfoLite-compatible field numbers needed
+ to parse old wire bytes cleanly, including user (2), position (3),
+ device_metrics (6), and the legacy-only compatibility fields via_mqtt (8),
+ is_favorite (10), and is_ignored (11). Steady-state code does not use
+ this struct; it is dropped after migration completes. This file should be
+ removed once DEVICESTATE_MIN_VER advances past 24. */
+typedef struct _meshtastic_NodeInfoLite_Legacy {
+    uint32_t num;
+    bool has_user;
+    meshtastic_UserLite user;
+    bool has_position;
+    meshtastic_PositionLite position;
+    float snr;
+    uint32_t last_heard;
+    bool has_device_metrics;
+    meshtastic_DeviceMetrics device_metrics;
+    uint8_t channel;
+    bool via_mqtt;
+    bool has_hops_away;
+    uint8_t hops_away;
+    bool is_favorite;
+    bool is_ignored;
+    uint8_t next_hop;
+    uint32_t bitfield;
+} meshtastic_NodeInfoLite_Legacy;
+
+/* Legacy NodeDatabase shape: one repeated array of fat NodeInfoLite_Legacy
+ with no satellite position/telemetry arrays. */
+typedef struct _meshtastic_NodeDatabase_Legacy {
+    uint32_t version;
+    std::vector<meshtastic_NodeInfoLite_Legacy> nodes;
+} meshtastic_NodeDatabase_Legacy;
+
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Initializer values for message structs */
+#define meshtastic_NodeInfoLite_Legacy_init_default {0, false, meshtastic_UserLite_init_default, false, meshtastic_PositionLite_init_default, 0, 0, false, meshtastic_DeviceMetrics_init_default, 0, 0, false, 0, 0, 0, 0, 0}
+#define meshtastic_NodeDatabase_Legacy_init_default {0, {0}}
+#define meshtastic_NodeInfoLite_Legacy_init_zero {0, false, meshtastic_UserLite_init_zero, false, meshtastic_PositionLite_init_zero, 0, 0, false, meshtastic_DeviceMetrics_init_zero, 0, 0, false, 0, 0, 0, 0, 0}
+#define meshtastic_NodeDatabase_Legacy_init_zero {0, {0}}
+
+/* Field tags (for use in manual encoding/decoding) */
+#define meshtastic_NodeInfoLite_Legacy_num_tag   1
+#define meshtastic_NodeInfoLite_Legacy_user_tag  2
+#define meshtastic_NodeInfoLite_Legacy_position_tag 3
+#define meshtastic_NodeInfoLite_Legacy_snr_tag   4
+#define meshtastic_NodeInfoLite_Legacy_last_heard_tag 5
+#define meshtastic_NodeInfoLite_Legacy_device_metrics_tag 6
+#define meshtastic_NodeInfoLite_Legacy_channel_tag 7
+#define meshtastic_NodeInfoLite_Legacy_via_mqtt_tag 8
+#define meshtastic_NodeInfoLite_Legacy_hops_away_tag 9
+#define meshtastic_NodeInfoLite_Legacy_is_favorite_tag 10
+#define meshtastic_NodeInfoLite_Legacy_is_ignored_tag 11
+#define meshtastic_NodeInfoLite_Legacy_next_hop_tag 12
+#define meshtastic_NodeInfoLite_Legacy_bitfield_tag 13
+#define meshtastic_NodeDatabase_Legacy_version_tag 1
+#define meshtastic_NodeDatabase_Legacy_nodes_tag 2
+
+/* Struct field encoding specification for nanopb */
+#define meshtastic_NodeInfoLite_Legacy_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   num,               1) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  user,              2) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  position,          3) \
+X(a, STATIC,   SINGULAR, FLOAT,    snr,               4) \
+X(a, STATIC,   SINGULAR, FIXED32,  last_heard,        5) \
+X(a, STATIC,   OPTIONAL, MESSAGE,  device_metrics,    6) \
+X(a, STATIC,   SINGULAR, UINT32,   channel,           7) \
+X(a, STATIC,   SINGULAR, BOOL,     via_mqtt,          8) \
+X(a, STATIC,   OPTIONAL, UINT32,   hops_away,         9) \
+X(a, STATIC,   SINGULAR, BOOL,     is_favorite,      10) \
+X(a, STATIC,   SINGULAR, BOOL,     is_ignored,       11) \
+X(a, STATIC,   SINGULAR, UINT32,   next_hop,         12) \
+X(a, STATIC,   SINGULAR, UINT32,   bitfield,         13)
+#define meshtastic_NodeInfoLite_Legacy_CALLBACK NULL
+#define meshtastic_NodeInfoLite_Legacy_DEFAULT NULL
+#define meshtastic_NodeInfoLite_Legacy_user_MSGTYPE meshtastic_UserLite
+#define meshtastic_NodeInfoLite_Legacy_position_MSGTYPE meshtastic_PositionLite
+#define meshtastic_NodeInfoLite_Legacy_device_metrics_MSGTYPE meshtastic_DeviceMetrics
+
+#define meshtastic_NodeDatabase_Legacy_FIELDLIST(X, a) \
+X(a, STATIC,   SINGULAR, UINT32,   version,           1) \
+X(a, CALLBACK, REPEATED, MESSAGE,  nodes,             2)
+extern bool meshtastic_NodeDatabase_Legacy_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field);
+#define meshtastic_NodeDatabase_Legacy_CALLBACK meshtastic_NodeDatabase_Legacy_callback
+#define meshtastic_NodeDatabase_Legacy_DEFAULT NULL
+#define meshtastic_NodeDatabase_Legacy_nodes_MSGTYPE meshtastic_NodeInfoLite_Legacy
+
+extern const pb_msgdesc_t meshtastic_NodeInfoLite_Legacy_msg;
+extern const pb_msgdesc_t meshtastic_NodeDatabase_Legacy_msg;
+
+/* Defines for backwards compatibility with code written before nanopb-0.4.0 */
+#define meshtastic_NodeInfoLite_Legacy_fields &meshtastic_NodeInfoLite_Legacy_msg
+#define meshtastic_NodeDatabase_Legacy_fields &meshtastic_NodeDatabase_Legacy_msg
+
+/* Maximum encoded size of messages (where known) */
+/* meshtastic_NodeDatabase_Legacy_size depends on runtime parameters */
+#define MESHTASTIC_MESHTASTIC_DEVICEONLY_LEGACY_PB_H_MAX_SIZE meshtastic_NodeInfoLite_Legacy_size
+#define meshtastic_NodeInfoLite_Legacy_size      196
+
+#ifdef __cplusplus
+} /* extern "C" */
+#endif
+
+#endif

src/mesh/generated/meshtastic/mesh.pb.h

@@ -317,6 +317,8 @@ typedef enum _meshtastic_HardwareModel {
     meshtastic_HardwareModel_THINKNODE_M9 = 131,
     /* The Heltec-V4-R8 uses an ESP32S3R8 chip, plus an SX1262. */
     meshtastic_HardwareModel_HELTEC_V4_R8 = 132,
+    /* The HELTEC_MESH_NODE_T1 uses an NRF52840 chip, plus an SX1262. */
+    meshtastic_HardwareModel_HELTEC_MESH_NODE_T1 = 133,
     /* ------------------------------------------------------------------------------------------------------------------------------------------
  Reserved ID For developing private Ports. These will show up in live traffic sparsely, so we can use a high number. Keep it within 8 bits.
  ------------------------------------------------------------------------------------------------------------------------------------------ */

src/mesh/generated/meshtastic/telemetry.pb.h

@@ -115,7 +115,11 @@ typedef enum _meshtastic_TelemetrySensorType {
     /* SHT family of sensors for temperature and humidity */
     meshtastic_TelemetrySensorType_SHTXX = 50,
     /* DS248X Bridge for one-wire temperature sensors */
-    meshtastic_TelemetrySensorType_DS248X = 51
+    meshtastic_TelemetrySensorType_DS248X = 51,
+    /* MMC5983MA 3-Axis Digital Magnetic Sensor */
+    meshtastic_TelemetrySensorType_MMC5983MA = 52,
+    /* ICM-42607-P 6‑Axis IMU */
+    meshtastic_TelemetrySensorType_ICM42607P = 53
 } meshtastic_TelemetrySensorType;
 
 /* Struct definitions */
@@ -496,8 +500,8 @@ extern "C" {
 
 /* Helper constants for enums */
 #define _meshtastic_TelemetrySensorType_MIN meshtastic_TelemetrySensorType_SENSOR_UNSET
-#define _meshtastic_TelemetrySensorType_MAX meshtastic_TelemetrySensorType_DS248X
-#define _meshtastic_TelemetrySensorType_ARRAYSIZE ((meshtastic_TelemetrySensorType)(meshtastic_TelemetrySensorType_DS248X+1))
+#define _meshtastic_TelemetrySensorType_MAX meshtastic_TelemetrySensorType_ICM42607P
+#define _meshtastic_TelemetrySensorType_ARRAYSIZE ((meshtastic_TelemetrySensorType)(meshtastic_TelemetrySensorType_ICM42607P+1))
 
 
 

src/mesh/http/ContentHandler.cpp

@@ -729,7 +729,7 @@ void handleNodes(HTTPRequest *req, HTTPResponse *res)
     uint32_t readIndex = 0;
     const meshtastic_NodeInfoLite *tempNodeInfo = nodeDB->readNextMeshNode(readIndex);
     while (tempNodeInfo != NULL) {
-        if (tempNodeInfo->has_user) {
+        if (nodeInfoLiteHasUser(tempNodeInfo)) {
             JSONObject node;
 
             char id[16];
@@ -737,26 +737,26 @@ void handleNodes(HTTPRequest *req, HTTPResponse *res)
 
             node["id"] = new JSONValue(id);
             node["snr"] = new JSONValue(tempNodeInfo->snr);
-            node["via_mqtt"] = new JSONValue(BoolToString(tempNodeInfo->via_mqtt));
+            node["via_mqtt"] = new JSONValue(BoolToString(nodeInfoLiteViaMqtt(tempNodeInfo)));
             node["last_heard"] = new JSONValue((int)tempNodeInfo->last_heard);
             node["position"] = new JSONValue();
 
             if (nodeDB->hasValidPosition(tempNodeInfo)) {
-                JSONObject position;
-                position["latitude"] = new JSONValue((float)tempNodeInfo->position.latitude_i * 1e-7);
-                position["longitude"] = new JSONValue((float)tempNodeInfo->position.longitude_i * 1e-7);
-                position["altitude"] = new JSONValue((int)tempNodeInfo->position.altitude);
-                node["position"] = new JSONValue(position);
+                meshtastic_PositionLite posLite;
+                if (nodeDB->copyNodePosition(tempNodeInfo->num, posLite)) {
+                    JSONObject position;
+                    position["latitude"] = new JSONValue((float)posLite.latitude_i * 1e-7);
+                    position["longitude"] = new JSONValue((float)posLite.longitude_i * 1e-7);
+                    position["altitude"] = new JSONValue((int)posLite.altitude);
+                    node["position"] = new JSONValue(position);
+                }
             }
 
-            node["long_name"] = new JSONValue(tempNodeInfo->user.long_name);
-            node["short_name"] = new JSONValue(tempNodeInfo->user.short_name);
-            char macStr[18];
-            snprintf(macStr, sizeof(macStr), "%02X:%02X:%02X:%02X:%02X:%02X", tempNodeInfo->user.macaddr[0],
-                     tempNodeInfo->user.macaddr[1], tempNodeInfo->user.macaddr[2], tempNodeInfo->user.macaddr[3],
-                     tempNodeInfo->user.macaddr[4], tempNodeInfo->user.macaddr[5]);
-            node["mac_address"] = new JSONValue(macStr);
-            node["hw_model"] = new JSONValue(tempNodeInfo->user.hw_model);
+            node["long_name"] = new JSONValue(tempNodeInfo->long_name);
+            node["short_name"] = new JSONValue(tempNodeInfo->short_name);
+            // mac_address dropped from NodeInfoLite as part of the slim refactor; emit zeros.
+            node["mac_address"] = new JSONValue("00:00:00:00:00:00");
+            node["hw_model"] = new JSONValue(tempNodeInfo->hw_model);
 
             nodesArray.push_back(new JSONValue(node));
         }
@@ -930,4 +930,4 @@ void handleScanNetworks(HTTPRequest *req, HTTPResponse *res)
     res->print(jsonString.c_str());
     delete value;
 }
-#endif
+#endif

src/mesh/mesh-pb-constants.h

@@ -37,9 +37,43 @@
 #define MAX_RX_NOTIFICATION_TOPHONE 2
 #endif
 
-/// Verify baseline assumption of node size. If it increases, we need to reevaluate
-/// the impact of its memory footprint, notably on MAX_NUM_NODES.
-static_assert(sizeof(meshtastic_NodeInfoLite) <= 200, "NodeInfoLite size increased. Reconsider impact on MAX_NUM_NODES.");
+/// Tighten this when the slim header shrinks; loosen only with deliberate
+/// awareness of MAX_NUM_NODES impact per platform.
+static_assert(sizeof(meshtastic_NodeInfoLite) <= 130, "NodeInfoLite size increased. Reconsider impact on MAX_NUM_NODES.");
+
+// Compile satellite NodeDBs out on STM32WL (and the status DB also follows
+// MESHTASTIC_EXCLUDE_STATUS).
+#ifndef MESHTASTIC_EXCLUDE_POSITIONDB
+#if defined(ARCH_STM32WL)
+#define MESHTASTIC_EXCLUDE_POSITIONDB 1
+#else
+#define MESHTASTIC_EXCLUDE_POSITIONDB 0
+#endif
+#endif
+
+#ifndef MESHTASTIC_EXCLUDE_TELEMETRYDB
+#if defined(ARCH_STM32WL)
+#define MESHTASTIC_EXCLUDE_TELEMETRYDB 1
+#else
+#define MESHTASTIC_EXCLUDE_TELEMETRYDB 0
+#endif
+#endif
+
+#ifndef MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+#if defined(ARCH_STM32WL)
+#define MESHTASTIC_EXCLUDE_ENVIRONMENTDB 1
+#else
+#define MESHTASTIC_EXCLUDE_ENVIRONMENTDB 0
+#endif
+#endif
+
+#ifndef MESHTASTIC_EXCLUDE_STATUSDB
+#if defined(ARCH_STM32WL) || defined(MESHTASTIC_EXCLUDE_STATUS)
+#define MESHTASTIC_EXCLUDE_STATUSDB 1
+#else
+#define MESHTASTIC_EXCLUDE_STATUSDB 0
+#endif
+#endif
 
 /// max number of nodes allowed in the nodeDB
 #ifndef MAX_NUM_NODES

src/mesh/wifi/WiFiAPClient.cpp

@@ -15,6 +15,12 @@
 #define ETH ETH2
 #endif // HAS_ETHERNET
 
+#if HAS_ETHERNET && defined(USE_CH390D)
+#include "ESP32_CH390.h"
+#include "hal/spi_types.h"
+#define ETH CH390
+#endif // HAS_ETHERNET
+
 #include <WiFiUdp.h>
 #ifdef ARCH_ESP32
 #if !MESHTASTIC_EXCLUDE_WEBSERVER
@@ -56,12 +62,43 @@ unsigned long lastrun_ntp = 0;
 
 bool needReconnect = true;   // If we create our reconnector, run it once at the beginning
 bool isReconnecting = false; // If we are currently reconnecting
+#if defined(USE_WS5500) || defined(USE_CH390D)
+static volatile bool ethNetworkConnectedPending = false;
+#endif
 
 WiFiUDP syslogClient;
 meshtastic::Syslog syslog(syslogClient);
 
 Periodic *wifiReconnect;
 
+#if defined(USE_WS5500) || defined(USE_CH390D)
+static void onNetworkConnected();
+static uint32_t lastEthIP = 0;
+static int32_t ethNetworkConnectedPoll()
+{
+    if (ethNetworkConnectedPending) {
+        ethNetworkConnectedPending = false;
+        uint32_t ip = (uint32_t)ETH.localIP();
+        bool ipChanged = APStartupComplete && ip != 0 && ip != lastEthIP;
+        onNetworkConnected();
+        if (ipChanged) {
+            LOG_INFO("Ethernet IP changed (%u.%u.%u.%u), restarting mDNS", ip & 0xff, (ip >> 8) & 0xff, (ip >> 16) & 0xff,
+                     (ip >> 24) & 0xff);
+            MDNS.end();
+            if (MDNS.begin("Meshtastic")) {
+                MDNS.addService("meshtastic", "tcp", SERVER_API_DEFAULT_PORT);
+                MDNS.addServiceTxt("meshtastic", "tcp", "shortname", String(owner.short_name));
+                MDNS.addServiceTxt("meshtastic", "tcp", "id", String(nodeDB->getNodeId().c_str()));
+                MDNS.addServiceTxt("meshtastic", "tcp", "pio_env", optstr(APP_ENV));
+            }
+        }
+        if (ip != 0)
+            lastEthIP = ip;
+    }
+    return 500;
+}
+#endif
+
 #ifdef USE_WS5500
 // Startup Ethernet
 bool initEthernet()
@@ -72,6 +109,38 @@ bool initEthernet()
 #if !MESHTASTIC_EXCLUDE_WEBSERVER
         createSSLCert(); // For WebServer
 #endif
+        new concurrency::Periodic("EthConnect", ethNetworkConnectedPoll);
+        return true;
+    }
+
+    return false;
+}
+#endif
+
+#ifdef USE_CH390D
+// Startup Ethernet
+bool initEthernet()
+{
+    // Configure CH390
+    ch390_config_t ch390_conf = CH390_DEFAULT_CONFIG();
+    ch390_conf.spi_host = SPI3_HOST;
+    ch390_conf.spi_cs_gpio = ETH_CS_PIN;
+    ch390_conf.spi_sck_gpio = ETH_SCLK_PIN;
+    ch390_conf.spi_mosi_gpio = ETH_MOSI_PIN;
+    ch390_conf.spi_miso_gpio = ETH_MISO_PIN;
+    ch390_conf.int_gpio = ETH_INT_PIN;
+#ifdef ETH_RST_PIN
+    ch390_conf.reset_gpio = ETH_RST_PIN;
+#else
+    ch390_conf.reset_gpio = -1;
+#endif
+    ch390_conf.spi_clock_mhz = 20;
+    if ((config.network.eth_enabled) && (ETH.begin(ch390_conf))) {
+        WiFi.onEvent(WiFiEvent);
+#if !MESHTASTIC_EXCLUDE_WEBSERVER
+        createSSLCert(); // For WebServer
+#endif
+        new concurrency::Periodic("EthConnect", ethNetworkConnectedPoll);
         return true;
     }
 
@@ -234,7 +303,7 @@ bool isWifiAvailable()
 
     if (config.network.wifi_enabled && (config.network.wifi_ssid[0])) {
         return true;
-#ifdef USE_WS5500
+#if defined(USE_WS5500) || defined(USE_CH390D)
     } else if (config.network.eth_enabled) {
         return true;
 #endif
@@ -384,13 +453,13 @@ static void WiFiEvent(WiFiEvent_t event)
 #endif
         }
 #ifdef WIFI_LED
-        digitalWrite(WIFI_LED, HIGH);
+        digitalWrite(WIFI_LED, LOW ^ WIFI_STATE_ON);
 #endif
         break;
     case ARDUINO_EVENT_WIFI_STA_DISCONNECTED:
         LOG_INFO("Disconnected from WiFi access point");
 #ifdef WIFI_LED
-        digitalWrite(WIFI_LED, LOW);
+        digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
 #endif
 #if HAS_UDP_MULTICAST
         if (udpHandler) {
@@ -452,13 +521,13 @@ static void WiFiEvent(WiFiEvent_t event)
     case ARDUINO_EVENT_WIFI_AP_START:
         LOG_INFO("WiFi access point started");
 #ifdef WIFI_LED
-        digitalWrite(WIFI_LED, HIGH);
+        digitalWrite(WIFI_LED, LOW ^ WIFI_STATE_ON);
 #endif
         break;
     case ARDUINO_EVENT_WIFI_AP_STOP:
         LOG_INFO("WiFi access point stopped");
 #ifdef WIFI_LED
-        digitalWrite(WIFI_LED, LOW);
+        digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
 #endif
         break;
     case ARDUINO_EVENT_WIFI_AP_STACONNECTED:
@@ -494,18 +563,18 @@ static void WiFiEvent(WiFiEvent_t event)
         LOG_INFO("Ethernet disconnected");
         break;
     case ARDUINO_EVENT_ETH_GOT_IP:
-#ifdef USE_WS5500
+#if defined(USE_WS5500) || defined(USE_CH390D)
         LOG_INFO("Obtained IP address: %s, %u Mbps, %s", ETH.localIP().toString().c_str(), ETH.linkSpeed(),
                  ETH.fullDuplex() ? "FULL_DUPLEX" : "HALF_DUPLEX");
-        onNetworkConnected();
+        ethNetworkConnectedPending = true;
 #endif
         break;
     case ARDUINO_EVENT_ETH_GOT_IP6:
-#ifdef USE_WS5500
+#if defined(USE_WS5500) || defined(USE_CH390D)
 #if ESP_ARDUINO_VERSION >= ESP_ARDUINO_VERSION_VAL(3, 0, 0)
         LOG_INFO("Obtained Local IP6 address: %s", ETH.linkLocalIPv6().toString().c_str());
         LOG_INFO("Obtained GlobalIP6 address: %s", ETH.globalIPv6().toString().c_str());
-#else
+#elif defined(USE_WS5500)
         LOG_INFO("Obtained IP6 address: %s", ETH.localIPv6().toString().c_str());
 #endif
 #endif

src/mesh/wifi/WiFiAPClient.h

@@ -26,7 +26,7 @@ bool isWifiAvailable();
 
 uint8_t getWifiDisconnectReason();
 
-#ifdef USE_WS5500
+#if defined(USE_WS5500) || defined(USE_CH390D)
 // Startup Ethernet
 bool initEthernet();
 #endif

src/modules/AdminModule.cpp

@@ -107,14 +107,14 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
 
             // Automatically favorite the node that is using the admin key
             auto remoteNode = nodeDB->getMeshNode(mp.from);
-            if (remoteNode && !remoteNode->is_favorite) {
+            if (remoteNode && !nodeInfoLiteIsFavorite(remoteNode)) {
                 if (config.device.role == meshtastic_Config_DeviceConfig_Role_CLIENT_BASE) {
                     // Special case for CLIENT_BASE: is_favorite has special meaning, and we don't want to automatically set it
                     // without the user doing so deliberately.
                     LOG_INFO("PKC admin valid, but not auto-favoriting node %x because role==CLIENT_BASE", mp.from);
                 } else {
                     LOG_INFO("PKC admin valid. Auto-favoriting node %x", mp.from);
-                    remoteNode->is_favorite = true;
+                    nodeInfoLiteSetBit(remoteNode, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
                 }
             }
         } else {
@@ -393,7 +393,7 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
         LOG_INFO("Client received set_favorite_node command");
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->set_favorite_node);
         if (node != NULL) {
-            node->is_favorite = true;
+            nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
             saveChanges(SEGMENT_NODEDATABASE, false);
             if (screen)
                 screen->setFrames(graphics::Screen::FOCUS_PRESERVE); // <-- Rebuild screens
@@ -404,7 +404,7 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
         LOG_INFO("Client received remove_favorite_node command");
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->remove_favorite_node);
         if (node != NULL) {
-            node->is_favorite = false;
+            nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_FAVORITE_MASK, false);
             saveChanges(SEGMENT_NODEDATABASE, false);
             if (screen)
                 screen->setFrames(graphics::Screen::FOCUS_PRESERVE); // <-- Rebuild screens
@@ -415,11 +415,10 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
         LOG_INFO("Client received set_ignored_node command");
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->set_ignored_node);
         if (node != NULL) {
-            node->is_ignored = true;
-            node->has_device_metrics = false;
-            node->has_position = false;
-            node->user.public_key.size = 0;
-            memset(node->user.public_key.bytes, 0, sizeof(node->user.public_key.bytes));
+            nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
+            nodeDB->eraseNodeSatellites(node->num);
+            node->public_key.size = 0;
+            memset(node->public_key.bytes, 0, sizeof(node->public_key.bytes));
             saveChanges(SEGMENT_NODEDATABASE, false);
         }
         break;
@@ -428,7 +427,7 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
         LOG_INFO("Client received remove_ignored_node command");
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->remove_ignored_node);
         if (node != NULL) {
-            node->is_ignored = false;
+            nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_IGNORED_MASK, false);
             saveChanges(SEGMENT_NODEDATABASE, false);
         }
         break;
@@ -437,7 +436,7 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
         LOG_INFO("Client received toggle_muted_node command");
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->toggle_muted_node);
         if (node != NULL) {
-            node->bitfield ^= (1 << NODEINFO_BITFIELD_IS_MUTED_SHIFT);
+            nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_MUTED_MASK, !nodeInfoLiteIsMuted(node));
             saveChanges(SEGMENT_NODEDATABASE, false);
         }
         break;
@@ -445,9 +444,11 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
 
     case meshtastic_AdminMessage_set_fixed_position_tag: {
         LOG_INFO("Client received set_fixed_position command");
-        meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-        node->has_position = true;
-        node->position = TypeConversions::ConvertToPositionLite(r->set_fixed_position);
+        const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
+        // Route the fixed position through updatePosition so it lands in the
+        // satellite map (or, on builds with PositionDB excluded, just sets
+        // localPosition for the local broadcast path).
+        nodeDB->updatePosition(node->num, r->set_fixed_position, RX_SRC_LOCAL);
         nodeDB->setLocalPosition(r->set_fixed_position);
         config.position.fixed_position = true;
         saveChanges(SEGMENT_NODEDATABASE | SEGMENT_CONFIG, false);
@@ -1317,7 +1318,7 @@ void AdminModule::handleGetDeviceConnectionStatus(const meshtastic_MeshPacket &r
     }
 #endif
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
     conn.has_ethernet = true;
     conn.ethernet.has_status = true;
     if (Ethernet.linkStatus() == LinkON) {

src/modules/CannedMessageModule.cpp

@@ -141,7 +141,7 @@ void CannedMessageModule::LaunchFreetextWithDestination(NodeNum newDest, uint8_t
 static bool returnToCannedList = false;
 bool hasKeyForNode(const meshtastic_NodeInfoLite *node)
 {
-    return node && node->has_user && node->user.public_key.size > 0;
+    return nodeInfoLiteHasUser(node) && node->public_key.size > 0;
 }
 /**
  * @brief Items in array this->messages will be set to be pointing on the right
@@ -262,10 +262,10 @@ void CannedMessageModule::updateDestinationSelectionList()
 
     for (size_t i = 0; i < numMeshNodes; ++i) {
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNodeByIndex(i);
-        if (!node || node->num == myNodeNum || !node->has_user || node->user.public_key.size != 32)
+        if (!node || node->num == myNodeNum || !nodeInfoLiteHasUser(node) || node->public_key.size != 32)
             continue;
 
-        const String &nodeName = node->user.long_name;
+        const String &nodeName = node->long_name;
 
         if (searchQuery.length() == 0) {
             this->filteredNodes.push_back({node, sinceLastSeen(node)});
@@ -1054,7 +1054,7 @@ void CannedMessageModule::sendText(NodeNum dest, ChannelIndex channel, const cha
     }
 
     NodeNum myNodeNum = nodeDB->getNodeNum();
-    if (node && node->num != myNodeNum && node->has_user && node->user.public_key.size == 32) {
+    if (node && node->num != myNodeNum && nodeInfoLiteHasUser(node) && node->public_key.size == 32) {
         p->pki_encrypted = true;
         p->channel = 0; // force PKI
     }
@@ -1415,8 +1415,8 @@ const char *CannedMessageModule::getNodeName(NodeNum node)
         return "Broadcast";
 
     meshtastic_NodeInfoLite *info = nodeDB->getMeshNode(node);
-    if (info && info->has_user && strlen(info->user.long_name) > 0) {
-        return info->user.long_name;
+    if (nodeInfoLiteHasUser(info) && strlen(info->long_name) > 0) {
+        return info->long_name;
     }
 
     static char fallback[12];
@@ -1723,17 +1723,17 @@ void CannedMessageModule::drawDestinationSelectionScreen(OLEDDisplay *display, O
                 meshtastic_NodeInfoLite *node = this->filteredNodes[nodeIndex].node;
                 if (node) {
                     if (display->getWidth() <= 64) {
-                        entryText = node->user.short_name;
-                    } else if (node->user.long_name[0]) {
-                        entryText = node->user.long_name;
+                        entryText = node->short_name;
+                    } else if (node->long_name[0]) {
+                        entryText = node->long_name;
                     } else {
-                        entryText = node->user.short_name;
+                        entryText = node->short_name;
                     }
                 }
 
                 int availWidth = display->getWidth() -
                                  ((graphics::currentResolution == graphics::ScreenResolution::High) ? 40 : 20) -
-                                 ((node && node->is_favorite) ? 10 : 0);
+                                 ((nodeInfoLiteIsFavorite(node)) ? 10 : 0);
                 if (availWidth < 0)
                     availWidth = 0;
                 char truncatedEntry[96];
@@ -1742,7 +1742,7 @@ void CannedMessageModule::drawDestinationSelectionScreen(OLEDDisplay *display, O
                 entryText = truncatedEntry;
 
                 // Prepend "* " if this is a favorite
-                if (node && node->is_favorite) {
+                if (nodeInfoLiteIsFavorite(node)) {
                     entryText = "* " + entryText;
                 }
                 graphics::UIRenderer::truncateStringWithEmotes(display, entryText.c_str(), truncatedEntry, sizeof(truncatedEntry),