Merge branch 'develop' into pioarduino

Develop is 2.8 WIP now

.claude/commands/diagnose.md

@@ -49,11 +49,17 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
    - Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
    - Do the primary channel NAMES match? Mismatch = different PSK = no decode.
 
-7. **Suggest next actions only for specific, recognisable failure modes**:
+7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
+   - `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
+   - `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
+   - If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
+
+8. **Suggest next actions only for specific, recognisable failure modes**:
    - Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
    - Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
    - Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
    - CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
+   - Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
 
 ## What NOT to do
 

.claude/commands/leakhunt.md

@@ -0,0 +1,103 @@
+---
+description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
+argument-hint: [window=1h] [field=free_heap] [variant=local]
+---
+
+<!-- markdownlint-disable MD029 -->
+
+# `/leakhunt` — read the recorder, classify a memory leak
+
+Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
+
+## Two signal paths — pick the right one
+
+| Path                  | Build flag       | Cadence        | Per-thread attribution | Cost                      |
+| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
+| LocalStats packet     | (default)        | ~60 s          | No                     | Free — always on          |
+| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked)  | Bigger flash + log volume |
+
+Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
+
+## What to do
+
+1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
+
+2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
+   - `running == True`
+   - `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
+   - `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
+
+3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
+
+4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
+   - `samples` — how many raw points contributed
+   - `min`, `max` — total swing
+   - `slope_per_min` — units per minute (linear regression over the whole window)
+
+5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
+
+6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
+
+6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
+
+7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
+   - **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
+   - **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
+   - **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
+   - **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
+   - **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
+   - **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
+   - **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
+
+8. **Report**:
+
+   ```text
+   /leakhunt window=6h field=free_heap variant=local
+   ────────────────────────────────────────────────────
+   recorder        : running, telem last_ts 8s ago
+   build           : DEBUG_HEAP=ON (per-line prefix detected)
+   samples         : 14,200 over 6h (cadence ~1.5s, log-line synth)
+   free_heap       : min 92,344 / max 124,008 / range 31,664
+   slope           : -82 bytes/min (negative — heap declining)
+   reboots         : none in window
+   OOM events      : none
+   error lines     : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
+   thread leaks    : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
+                                  Router       -1,408 B over 4 events
+                                  others       -240 B
+   verdict         : SLOW LEAK — primary suspect MeshPacket thread
+   est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
+   evidence        : (3 log line citations with uptimes)
+   ```
+
+   Then: **what to do next.**
+   - SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
+     ```text
+     mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
+     mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
+     ```
+     After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
+   - SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
+   - FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
+   - OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
+   - STEADY → say so; stop. Don't invent problems.
+
+## What NOT to do
+
+- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
+- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
+- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
+- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
+
+## Companion: `mark_event` for stress runs
+
+If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
+
+```text
+mark_event("burst-start")
+… run the workload …
+mark_event("burst-end")
+/leakhunt window=15m
+```
+
+The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.

.claude/commands/repro.md

@@ -3,6 +3,8 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
 argument-hint: <test-node-id> [count=5]
 ---
 
+<!-- markdownlint-disable MD029 -->
+
 # `/repro` — flakiness triage for one test
 
 Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -40,6 +42,8 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
 
    Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
 
+5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
+
 6. **Classify the flake** into one of:
    - **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
    - **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.

.devcontainer/devcontainer.json

@@ -8,17 +8,21 @@
   "features": {
     "ghcr.io/devcontainers/features/python:1": {
       "installTools": true,
-      "version": "3.14"
+      "version": "3.13"
     }
   },
   "customizations": {
     "vscode": {
       "extensions": [
         "ms-vscode.cpptools",
-        "platformio.platformio-ide",
+        "Jason2866.esp-decoder",
+        "pioarduino.pioarduino-ide",
         "Trunk.io"
       ],
-      "unwantedRecommendations": ["ms-azuretools.vscode-docker"],
+      "unwantedRecommendations": [
+        "ms-azuretools.vscode-docker",
+        "platformio.platformio-ide"
+      ],
       "settings": {
         "extensions.ignoreRecommendations": true
       }

.github/copilot-instructions.md

@@ -13,6 +13,7 @@ Meshtastic is an open-source LoRa mesh networking project for long-range, low-po
 - **RP2040/RP2350** - Raspberry Pi Pico variants
 - **STM32WL** - STM32 with integrated LoRa
 - **Linux/Portduino** - Native Linux builds (Raspberry Pi, etc.)
+- **macOS native** - Headless `meshtasticd` on Apple Silicon / x86_64; see `variants/native/portduino/platformio.ini` for Homebrew prereqs + CH341 LoRa setup
 
 ### Supported Radio Chips
 
@@ -134,6 +135,99 @@ On top of authorization, any remote admin message that **mutates** state (not a
 - **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
 - **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
 
+## NodeDB Layout (v25)
+
+`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
+
+### Slim `NodeInfoLite`
+
+`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
+
+Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
+
+```cpp
+nodeInfoLiteHasUser(n)                  // bit 5 — user fields populated
+nodeInfoLiteIsFavorite(n)               // bit 3
+nodeInfoLiteIsIgnored(n)                // bit 4
+nodeInfoLiteIsMuted(n)                  // bit 1
+nodeInfoLiteIsLicensed(n)               // bit 6 — Ham mode peer
+nodeInfoLiteIsKeyManuallyVerified(n)    // bit 0
+nodeInfoLiteHasIsUnmessagable(n)        // bit 8 — "is_unmessagable was sent"
+nodeInfoLiteIsUnmessagable(n)           // bit 7
+// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
+
+nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);  // setter
+```
+
+### Satellite stores
+
+Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its own build flag:
+
+| Map               | Value type                      | Build flag                         |
+| ----------------- | ------------------------------- | ---------------------------------- |
+| `nodePositions`   | `meshtastic_PositionLite`       | `MESHTASTIC_EXCLUDE_POSITIONDB`    |
+| `nodeTelemetry`   | `meshtastic_DeviceMetrics`      | `MESHTASTIC_EXCLUDE_TELEMETRYDB`   |
+| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
+| `nodeStatus`      | `meshtastic_StatusMessage`      | `MESHTASTIC_EXCLUDE_STATUSDB`      |
+
+Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
+
+All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
+
+### Accessor convention
+
+**Never hand out pointers into the maps.** Use the copy-out accessors on `NodeDB`:
+
+```cpp
+bool copyNodePosition(NodeNum, meshtastic_PositionLite &out)       const;
+bool copyNodeTelemetry(NodeNum, meshtastic_DeviceMetrics &out)     const;
+bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
+bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out)        const;
+```
+
+Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
+
+Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
+
+### Eviction
+
+Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
+
+### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
+
+There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
+
+1. Config / module-config / channel / metadata segments (same as before).
+2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
+3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
+4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
+5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
+
+`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
+
+STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
+
+Special nonces that still mean something:
+
+- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
+- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
+
+There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
+
+### v24 → v25 migration
+
+The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not in `NodeDB.cpp`. It owns the `meshtastic_NodeDatabase_Legacy` callback and `NodeDB::migrateLegacyNodeDatabase()`. The legacy proto descriptor is `protobufs/meshtastic/deviceonly_legacy.proto` (only included by the migration TU). The boot path peeks the file's leading version tag, runs the migration if `version < 25`, then re-saves in v25 layout. The legacy descriptor is scheduled for removal once `DEVICESTATE_MIN_VER` is bumped.
+
+### Read-site rules of thumb
+
+- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
+- Never `node->user.long_name` — `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
+- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
+- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
+- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
+
+Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
+
 ## Project Structure
 
 ```
@@ -195,6 +289,8 @@ firmware/
 - Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
 - Use `assert()` for invariants that should never fail
 - C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ### Naming Conventions
 
@@ -369,7 +465,7 @@ To reduce avoidable agent mistakes, assume these tools are available (or install
 - **Required CLI basics**: `bash`, `git`, `find`, `grep`, `sed`, `awk`, `xargs`
 - **Strongly recommended**: `rg` (ripgrep) for fast file/text search, `jq` for JSON processing
 - **Build/test tools**: `python3`, `pip`, virtualenv (`python3 -m venv`), `platformio` (`pio`)
-- **Containerized native testing**: `docker` (especially important on macOS / non-Linux hosts)
+- **Containerized native testing**: `docker` (fallback for non-Linux hosts; macOS can also build natively via `pio run -e native-macos`)
 
 Fallback expectations for agents:
 
@@ -388,6 +484,7 @@ Build commands:
 pio run -e tbeam              # Build specific target
 pio run -e tbeam -t upload    # Build and upload
 pio run -e native             # Build native/Linux version
+pio run -e native-macos       # Build headless macOS meshtasticd (Homebrew prereqs in variants/native/portduino/platformio.ini)
 ```
 
 ### Build Manifest
@@ -573,6 +670,8 @@ Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-v
 
 `confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve — it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
 
+**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step — use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
+
 ### Hardware test suite (`mcp-server/run-tests.sh`)
 
 The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.

.github/workflows/build_debian_src.yml

@@ -32,10 +32,15 @@ jobs:
         shell: bash
         working-directory: meshtasticd
         run: |
+          # Build-tools (notably platformio) come from the Meshtastic project
+          # on the OpenSUSE Build Service:
+          # https://build.opensuse.org/project/show/network:Meshtastic:build-tools
+          echo 'deb http://download.opensuse.org/repositories/network:/Meshtastic:/build-tools/xUbuntu_24.04/ /' \
+            | sudo tee /etc/apt/sources.list.d/network:Meshtastic:build-tools.list
+          curl -fsSL https://download.opensuse.org/repositories/network:Meshtastic:build-tools/xUbuntu_24.04/Release.key \
+            | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/network_Meshtastic_build-tools.gpg >/dev/null
           sudo apt-get update -y --fix-missing
-          sudo apt-get install -y software-properties-common build-essential devscripts equivs
-          sudo add-apt-repository ppa:meshtastic/build-tools -y
-          sudo apt-get update -y --fix-missing
+          sudo apt-get install -y build-essential devscripts equivs
           sudo mk-build-deps --install --remove --tool='apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes' debian/control
 
       - name: Import GPG key

.github/workflows/build_macos_bin.yml

@@ -0,0 +1,51 @@
+name: Build MacOS Binary
+
+on:
+  workflow_call:
+    inputs:
+      macos_ver:
+        required: false
+        default: "26" # ARM64
+        type: string
+
+permissions:
+  contents: read
+
+jobs:
+  build-MacOS:
+    runs-on: macos-${{ inputs.macos_ver }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v6
+        with:
+          submodules: recursive
+
+      - name: Install deps
+        shell: bash
+        run: |
+          brew update
+          brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config ulfius
+
+      - name: Get release version string
+        run: |
+          echo "long=$(./bin/buildinfo.py long)" >> $GITHUB_OUTPUT
+        id: version
+
+      - name: Build for MacOS
+        run: |
+          platformio run -e native-macos
+        env:
+          PKG_VERSION: ${{ steps.version.outputs.long }}
+        # Errors in this step should not fail the entire workflow while MacOS support is in development.
+        continue-on-error: true
+
+      - name: List output files
+        run: ls -lah .pio/build/native-macos/
+
+      - name: Store binaries as an artifact
+        uses: actions/upload-artifact@v7
+        with:
+          name: firmware-macos-${{ inputs.macos_ver }}-${{ steps.version.outputs.long }}
+          overwrite: true
+          path: |
+            .pio/build/native-macos/meshtasticd

.github/workflows/docker_build.yml

@@ -73,7 +73,9 @@ jobs:
       - name: Sanitize platform string
         id: sanitize_platform
         # Replace slashes with underscores
-        run: echo "cleaned_platform=${{ inputs.platform }}" | sed 's/\//_/g' >> $GITHUB_OUTPUT
+        env:
+          plat: ${{ inputs.platform }}
+        run: echo "cleaned_platform=${plat}" | sed 's/\//_/g' >> $GITHUB_OUTPUT
 
       - name: Docker login
         if: ${{ inputs.push }}

.github/workflows/docker_manifest.yml

@@ -43,6 +43,15 @@ jobs:
       push: true
     secrets: inherit
 
+  docker-debian-riscv64:
+    uses: ./.github/workflows/docker_build.yml
+    with:
+      distro: debian
+      platform: linux/riscv64
+      runs-on: ubuntu-24.04-arm
+      push: true
+    secrets: inherit
+
   docker-alpine-amd64:
     uses: ./.github/workflows/docker_build.yml
     with:
@@ -70,16 +79,27 @@ jobs:
       push: true
     secrets: inherit
 
+  docker-alpine-riscv64:
+    uses: ./.github/workflows/docker_build.yml
+    with:
+      distro: alpine
+      platform: linux/riscv64
+      runs-on: ubuntu-24.04-arm
+      push: true
+    secrets: inherit
+
   docker-manifest:
     needs:
       # Debian
       - docker-debian-amd64
       - docker-debian-arm64
       - docker-debian-armv7
+      - docker-debian-riscv64
       # Alpine
       - docker-alpine-amd64
       - docker-alpine-arm64
       - docker-alpine-armv7
+      - docker-alpine-riscv64
     runs-on: ubuntu-24.04
     steps:
       - name: Checkout code
@@ -162,6 +182,7 @@ jobs:
             meshtastic/meshtasticd@${{ needs.docker-debian-amd64.outputs.digest }}
             meshtastic/meshtasticd@${{ needs.docker-debian-arm64.outputs.digest }}
             meshtastic/meshtasticd@${{ needs.docker-debian-armv7.outputs.digest }}
+            meshtastic/meshtasticd@${{ needs.docker-debian-riscv64.outputs.digest }}
 
       - name: Docker meta (Alpine)
         id: meta_alpine
@@ -182,3 +203,4 @@ jobs:
             meshtastic/meshtasticd@${{ needs.docker-alpine-amd64.outputs.digest }}
             meshtastic/meshtasticd@${{ needs.docker-alpine-arm64.outputs.digest }}
             meshtastic/meshtasticd@${{ needs.docker-alpine-armv7.outputs.digest }}
+            meshtastic/meshtasticd@${{ needs.docker-alpine-riscv64.outputs.digest }}

.github/workflows/main_matrix.yml

@@ -116,6 +116,20 @@ jobs:
       build_location: local
     secrets: inherit
 
+  MacOS:
+    strategy:
+      fail-fast: false
+      matrix:
+        macos_ver:
+          - "26" # ARM64
+          # - '26-intel' # x86_64
+          - "15" # ARM64
+          # - '15-intel' # x86_64
+    uses: ./.github/workflows/build_macos_bin.yml
+    with:
+      macos_ver: ${{ matrix.macos_ver }}
+    # secrets: inherit
+
   package-pio-deps-native-tft:
     if: ${{ github.repository == 'meshtastic/firmware' && github.event_name == 'workflow_dispatch' }}
     uses: ./.github/workflows/package_pio_deps.yml
@@ -286,6 +300,7 @@ jobs:
       - gather-artifacts
       - build-debian-src
       - package-pio-deps-native-tft
+      # - MacOS
     steps:
       - name: Checkout
         uses: actions/checkout@v6

.gitignore

@@ -56,3 +56,9 @@ CMakeLists.txt
 .python3
 .claude/scheduled_tasks.lock
 userPrefs.jsonc.mcp-session-bak
+
+# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
+# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
+# compiled .proto outputs are ephemeral build artifacts.
+build/fixtures/
+bin/_generated/

.trunk/trunk.yaml

@@ -4,19 +4,19 @@ cli:
 plugins:
   sources:
     - id: trunk
-      ref: v1.7.6
+      ref: v1.9.0
       uri: https://github.com/trunk-io/plugins
 lint:
   enabled:
-    - checkov@3.2.524
-    - renovate@43.141.0
+    - checkov@3.2.528
+    - renovate@43.150.0
     - prettier@3.8.3
-    - trufflehog@3.95.2
+    - trufflehog@3.95.3
     - yamllint@1.38.0
     - bandit@1.9.4
     - trivy@0.70.0
     - taplo@0.10.0
-    - ruff@0.15.11
+    - ruff@0.15.12
     - isort@8.0.1
     - markdownlint@0.48.0
     - oxipng@10.1.1
@@ -34,9 +34,16 @@ lint:
     - linters: [ALL]
       paths:
         - bin/**
+    # Fake-NodeDB fixture JSONL files contain deterministic synthetic
+    # public_key_hex (64-char hex) values that gitleaks misidentifies as
+    # generic-api-key. These are not secrets — they're test fixtures
+    # produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
+    - linters: [gitleaks]
+      paths:
+        - test/fixtures/nodedb/seed_v25_*.jsonl
 runtimes:
   enabled:
-    - python@3.10.8
+    - python@3.14.4
     - go@1.21.0
     - node@22.16.0
 actions:

.vscode/extensions.json

@@ -1,10 +1,10 @@
 {
-    // See http://go.microsoft.com/fwlink/?LinkId=827846
-    // for the documentation about the extensions.json format
     "recommendations": [
-        "platformio.platformio-ide"
+        "Jason2866.esp-decoder",
+        "pioarduino.pioarduino-ide"
     ],
     "unwantedRecommendations": [
-        "ms-vscode.cpptools-extension-pack"
+        "ms-vscode.cpptools-extension-pack",
+        "platformio.platformio-ide"
     ]
 }

AGENTS.md

@@ -10,17 +10,18 @@ This file (`AGENTS.md`) is a short pointer + quick reference for agents that don
 
 ## Quick command reference
 
-| Action                           | Command                                                                             |
-| -------------------------------- | ----------------------------------------------------------------------------------- |
-| Build a firmware variant         | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`)              |
-| Clean + rebuild                  | `pio run -e <env> -t clean && pio run -e <env>`                                     |
-| Flash a device                   | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
-| Run firmware unit tests (native) | `pio test -e native`                                                                |
-| Run MCP hardware tests           | `./mcp-server/run-tests.sh`                                                         |
-| Live TUI test runner             | `mcp-server/.venv/bin/meshtastic-mcp-test-tui`                                      |
-| Format before commit             | `trunk fmt`                                                                         |
-| Regenerate protobuf bindings     | `bin/regen-protos.sh`                                                               |
-| Generate CI matrix               | `./bin/generate_ci_matrix.py all [--level pr]`                                      |
+| Action                           | Command                                                                                                       |
+| -------------------------------- | ------------------------------------------------------------------------------------------------------------- |
+| Build a firmware variant         | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`)                                        |
+| Build native macOS host binary   | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`) |
+| Clean + rebuild                  | `pio run -e <env> -t clean && pio run -e <env>`                                                               |
+| Flash a device                   | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool)                           |
+| Run firmware unit tests (native) | `pio test -e native`                                                                                          |
+| Run MCP hardware tests           | `./mcp-server/run-tests.sh`                                                                                   |
+| Live TUI test runner             | `mcp-server/.venv/bin/meshtastic-mcp-test-tui`                                                                |
+| Format before commit             | `trunk fmt`                                                                                                   |
+| Regenerate protobuf bindings     | `bin/regen-protos.sh`                                                                                         |
+| Generate CI matrix               | `./bin/generate_ci_matrix.py all [--level pr]`                                                                |
 
 ## MCP server (device + test automation)
 
@@ -65,6 +66,8 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
 - **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
 - **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
 - **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ## Typical agent workflows
 
@@ -121,19 +124,21 @@ Sequence these; don't parallelize on the same port.
 - **Device fully wedged (no DFU)?** `mcp__meshtastic__uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles it via USB hub PPPS. Needs `uhubctl` installed (`brew install uhubctl` / `apt install uhubctl`); on Linux without udev rules, permission errors fail fast, so use `sudo uhubctl` yourself or configure udev access.
 - **Port busy?** `lsof <port>` to find the holder. Usually a stale `pio device monitor` or zombie `meshtastic_mcp` process. Kill it.
 - **Multiple MCP servers running?** `ps aux | grep meshtastic_mcp` — zombies hold ports. Kill all but the one your host spawned.
+- **macOS: `LIBUSB_ERROR_BUSY` on a CH341 LoRa adapter?** A third-party WCH `CH34xVCPDriver` is claiming interface 0. Find the bundle ID with `ioreg -p IOUSB -l -w 0 | grep -B2 -A30 0x5512`, then `sudo kmutil unload -b <bundleID>`. Apple's bundled CH34x kext targets the CH340 UART (PID 0x7523), not the SPI bridge — it's never the culprit.
 
 ## Environment variables (test harness)
 
-| Var                                  | Purpose                                                                                                                                        |
-| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
-| `MESHTASTIC_MCP_ENV_<ROLE>`          | Override PlatformIO env for a role (e.g. `MESHTASTIC_MCP_ENV_NRF52=rak4631-dap`). Default map: `nrf52→rak4631`, `esp32s3→heltec-v3`.           |
-| `MESHTASTIC_MCP_SEED`                | PSK seed for the session test profile. Defaults to `mcp-<user>-<host>`.                                                                        |
-| `MESHTASTIC_MCP_FLASH_LOG`           | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress. |
-| `MESHTASTIC_UHUBCTL_BIN`             | Absolute path to `uhubctl` binary. Default: PATH lookup.                                                                                       |
-| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID.            |
-| `MESHTASTIC_UHUBCTL_PORT_<ROLE>`     | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`.                                                                |
-| `MESHTASTIC_UI_CAMERA_BACKEND`       | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default).                                           |
-| `MESHTASTIC_UI_CAMERA_DEVICE`        | Generic camera device (index or path). Used by the UI tier when no per-role var is set.                                                        |
-| `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` | Per-role camera pinning (e.g. `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3).                                         |
-| `MESHTASTIC_UI_OCR_BACKEND`          | OCR engine selection: `easyocr` / `pytesseract` / `null` / `auto` (default).                                                                   |
-| `MESHTASTIC_UI_TUI_CAMERA`           | Set to `1` to mount the live camera-feed panel in `meshtastic-mcp-test-tui`.                                                                   |
+| Var                                  | Purpose                                                                                                                                                                                                    |
+| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `MESHTASTIC_MCP_ENV_<ROLE>`          | Override PlatformIO env for a role (e.g. `MESHTASTIC_MCP_ENV_NRF52=rak4631-dap`). Default map: `nrf52→rak4631`, `esp32s3→heltec-v3`.                                                                       |
+| `MESHTASTIC_MCP_SEED`                | PSK seed for the session test profile. Defaults to `mcp-<user>-<host>`.                                                                                                                                    |
+| `MESHTASTIC_MCP_FLASH_LOG`           | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress.                                                             |
+| `MESHTASTIC_MCP_TCP_HOST`            | `host` or `host:port` of a `meshtasticd` daemon (e.g. the `native-macos` build). Surfaces it in `list_devices` as `tcp://host:port` so `connect()`-based tools target it transparently. Default port 4403. |
+| `MESHTASTIC_UHUBCTL_BIN`             | Absolute path to `uhubctl` binary. Default: PATH lookup.                                                                                                                                                   |
+| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID.                                                                        |
+| `MESHTASTIC_UHUBCTL_PORT_<ROLE>`     | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`.                                                                                                                            |
+| `MESHTASTIC_UI_CAMERA_BACKEND`       | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default).                                                                                                       |
+| `MESHTASTIC_UI_CAMERA_DEVICE`        | Generic camera device (index or path). Used by the UI tier when no per-role var is set.                                                                                                                    |
+| `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` | Per-role camera pinning (e.g. `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3).                                                                                                     |
+| `MESHTASTIC_UI_OCR_BACKEND`          | OCR engine selection: `easyocr` / `pytesseract` / `null` / `auto` (default).                                                                                                                               |
+| `MESHTASTIC_UI_TUI_CAMERA`           | Set to `1` to mount the live camera-feed panel in `meshtastic-mcp-test-tui`.                                                                                                                               |

Dockerfile

@@ -3,15 +3,17 @@
 # trunk-ignore-all(hadolint/DL3008): Do not pin apt package versions
 # trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
 
-FROM python:3.14-slim-trixie AS builder
+FROM debian:trixie AS builder
 ARG PIO_ENV=native
 ENV DEBIAN_FRONTEND=noninteractive
 ENV TZ=Etc/UTC
 
 # Install Dependencies
 ENV PIP_ROOT_USER_ACTION=ignore
+ENV PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apt-get update && apt-get install --no-install-recommends -y \
         curl wget g++ zip git ca-certificates pkg-config \
+        python3-pip python3-grpc-tools \
         libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
         libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
         libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev libsdl2-dev \

alpine.Dockerfile

@@ -3,12 +3,19 @@
 # trunk-ignore-all(hadolint/DL3018): Do not pin apk package versions
 # trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
 
-FROM python:3.14-alpine3.22 AS builder
+# Ensure the Alpine version is updated in both stages of the container!
+FROM alpine:3.23 AS builder
 ARG PIO_ENV=native
-ENV PIP_ROOT_USER_ACTION=ignore
 
+# Enable Alpine community repository (for 'py3-grpcio-tools')
+RUN echo "https://dl-cdn.alpinelinux.org/alpine/v$(cut -d. -f1,2 /etc/alpine-release)/community" >> /etc/apk/repositories
+
+# Install Dependencies
+ENV PIP_ROOT_USER_ACTION=ignore
+ENV PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apk --no-cache add \
         bash g++ libstdc++-dev linux-headers zip git ca-certificates libbsd-dev \
+        py3-pip py3-grpcio-tools \
         libgpiod-dev yaml-cpp-dev bluez-dev \
         libusb-dev i2c-tools-dev libuv-dev openssl-dev pkgconf argp-standalone \
         libx11-dev libinput-dev libxkbcommon-dev sqlite-dev sdl2-dev \
@@ -60,4 +67,4 @@ EXPOSE 4403
 
 CMD [ "sh",  "-cx", "meshtasticd --fsdir=/var/lib/meshtasticd" ]
 
-HEALTHCHECK NONE
+HEALTHCHECK NONE

bin/_rewrite_proto_namespace.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Post-process protoc-generated Python files to live under a local namespace.
+
+Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
+target directory and rewrites every `meshtastic` reference (imports, dotted
+attribute access) to use the new namespace (e.g., `meshtastic_v25`).
+
+Why: the .proto files declare `package meshtastic;`, so protoc emits
+`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
+`meshtastic` package which other parts of the mcp-server depend on. Renaming
+to a local namespace keeps both available.
+
+Usage:
+    _rewrite_proto_namespace.py <generated_dir> <new_namespace>
+"""
+
+from __future__ import annotations
+
+import pathlib
+import re
+import sys
+
+
+def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
+    # Standard protoc import forms:
+    #   from meshtastic.X_pb2 import ...     (rare, for direct symbol pulls)
+    #   from meshtastic import X_pb2 as ...  (common, the cross-file ref)
+    #   import meshtastic.X_pb2              (also possible)
+    pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
+    pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
+    pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
+
+    count = 0
+    for p in dir_path.glob("*.py"):
+        text = p.read_text(encoding="utf-8")
+        new = pattern_dotted_from.sub(f"from {new_ns}.", text)
+        new = pattern_bare_from.sub(f"from {new_ns} import ", new)
+        new = pattern_dotted_import.sub(f"import {new_ns}.", new)
+        # NOTE: we deliberately leave `meshtastic/X.proto` source-filename
+        # references inside descriptor strings alone. The descriptor pool is
+        # keyed by source filename (independent of Python package layout), so
+        # those don't collide with the PyPI package's descriptors.
+        if new != text:
+            p.write_text(new, encoding="utf-8")
+            count += 1
+    return count
+
+
+def main(argv: list[str]) -> int:
+    if len(argv) != 2:
+        print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
+        return 2
+    dir_path = pathlib.Path(argv[0])
+    new_ns = argv[1]
+    if not dir_path.is_dir():
+        print(f"directory not found: {dir_path}", file=sys.stderr)
+        return 2
+    n = rewrite(dir_path, new_ns)
+    print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

bin/build-esp32.sh

@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
 cp bin/device-update.* $OUTDIR/
 
 echo "Copying manifest"
-cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true
+cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json

bin/build-native.sh

@@ -31,5 +31,6 @@ basename=meshtasticd-$1-$VERSION
 pio pkg install --environment "$PIO_ENV" || platformioFailed
 pio run --environment "$PIO_ENV" || platformioFailed
 
-cp "$BUILDDIR/meshtasticd" "$OUTDIR/meshtasticd_linux_$(uname -m)"
+os_name=$(uname -s | tr '[:upper:]' '[:lower:]')
+cp "$BUILDDIR/meshtasticd" "$OUTDIR/meshtasticd_${os_name}_$(uname -m)"
 cp bin/native-install.* $OUTDIR/

bin/config.d/lora-station-g3.yaml

@@ -0,0 +1,18 @@
+# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
+# Verify spidev / I2C device paths for your OS — they may differ.
+Meta:
+  name: Station G3
+  support: community
+  compatible:
+    - raspberry-pi
+
+Lora:
+  Module: sx1262
+  IRQ: 22                       # BCM pin — wiki spec
+  Reset: 16                     # BCM pin — wiki spec
+  Busy: 24                      # BCM pin — wiki spec
+  # CS: 8                       # BCM 8 = SPI0 CE0 (default); uncomment only to override
+  DIO2_AS_RF_SWITCH: true
+  DIO3_TCXO_VOLTAGE: true
+  spidev: spidev0.0
+  # SX126X_MAX_POWER: 19        # matches Station G2 firmware cap; raise carefully per PA jumper mode

bin/gen-fake-nodedb-seed.py

@@ -0,0 +1,439 @@
+#!/usr/bin/env python3
+"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
+
+Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
+The output is hand-editable and committed; a sibling compile step
+(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
+v25 protobuf with fresh "now-relative" timestamps.
+
+Determinism contract:
+  Same --seed -> byte-identical JSONL output, regardless of wall clock.
+  All timestamps are stored as `*_offset_sec` (seconds before "now"); the
+  compile step resolves them to absolute epochs at compile time.
+
+Structural fields covered:
+  * NodeInfoLite header: num, long_name, short_name, hw_model, role,
+    public_key, snr, channel, hops_away, next_hop, bitfield flags
+  * PositionLite: lat/long Gaussian around --centroid, altitude, source
+  * DeviceMetrics: battery/voltage/util/uptime
+  * EnvironmentMetrics: temp/humidity/pressure/iaq
+  * StatusMessage: error_code (usually zero)
+
+Active-board allow-list:
+  hw_model values are restricted to the intersection of
+    (a) variants with `custom_meshtastic_support_level = 1` in
+        variants/*/*/platformio.ini, AND
+    (b) values present in the `HardwareModel` enum in mesh.proto.
+  See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
+  classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
+  (PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
+
+Active-role allow-list:
+  Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
+"""
+
+from __future__ import annotations
+
+import argparse
+import datetime as _dt
+import json
+import math
+import pathlib
+import random
+import sys
+
+# --------------------------------------------------------------------------
+# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
+# Refresh by running:
+#   for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
+#       grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
+#   done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
+# --------------------------------------------------------------------------
+HW_MODEL_WEIGHTS: dict[str, float] = {
+    "HELTEC_V3": 14.0,
+    "T_DECK": 9.0,
+    "HELTEC_V4": 8.0,
+    "RAK4631": 8.0,
+    "HELTEC_MESH_POCKET": 6.0,
+    "TRACKER_T1000_E": 5.0,
+    "HELTEC_MESH_NODE_T114": 5.0,
+    "T_DECK_PRO": 5.0,
+    "LILYGO_TBEAM_S3_CORE": 4.0,
+    "HELTEC_WIRELESS_PAPER": 4.0,
+    "HELTEC_WSL_V3": 3.0,
+    "T_ECHO": 3.0,
+    "HELTEC_WIRELESS_TRACKER": 3.0,
+    "HELTEC_WIRELESS_TRACKER_V2": 2.0,
+    "HELTEC_VISION_MASTER_E290": 2.0,
+    "HELTEC_MESH_SOLAR": 2.0,
+    "SEEED_WIO_TRACKER_L1": 2.0,
+    "T_LORA_PAGER": 1.5,
+    "HELTEC_VISION_MASTER_E213": 1.5,
+    "T_ECHO_PLUS": 1.0,
+    "MUZI_BASE": 1.0,
+    "WISMESH_TAP_V2": 1.0,
+    "THINKNODE_M2": 1.0,
+    "THINKNODE_M5": 1.0,
+    "TLORA_T3_S3": 1.0,
+    # Long tail (uniform low weight across remaining tier-1 boards):
+    "HELTEC_V4_R8": 0.3,
+    "HELTEC_VISION_MASTER_T190": 0.3,
+    "HELTEC_HT62": 0.3,
+    "HELTEC_MESH_NODE_T096": 0.3,
+    "M5STACK_C6L": 0.3,
+    "MINI_EPAPER_S3": 0.3,
+    "MUZI_R1_NEO": 0.3,
+    "NOMADSTAR_METEOR_PRO": 0.3,
+    "RAK3312": 0.3,
+    "RAK3401": 0.3,
+    "SEEED_SOLAR_NODE": 0.3,
+    "SEEED_WIO_TRACKER_L1_EINK": 0.3,
+    "SENSECAP_INDICATOR": 0.3,
+    "TBEAM_1_WATT": 0.3,
+    "THINKNODE_M1": 0.3,
+    "THINKNODE_M3": 0.3,
+    "THINKNODE_M6": 0.3,
+    "T_ECHO_LITE": 0.3,
+    "WISMESH_TAG": 0.3,
+    "WISMESH_TAP": 0.3,
+    "XIAO_NRF52_KIT": 0.3,
+    "CROWPANEL": 0.3,
+}
+
+# Non-deprecated roles only.
+ROLE_WEIGHTS: dict[str, float] = {
+    "CLIENT": 75.0,
+    "CLIENT_MUTE": 5.0,
+    "ROUTER": 7.0,
+    "TRACKER": 3.0,
+    "SENSOR": 2.0,
+    "CLIENT_HIDDEN": 2.0,
+    "ROUTER_LATE": 2.0,
+    "CLIENT_BASE": 2.0,
+    "TAK": 1.0,
+    "TAK_TRACKER": 0.5,
+    "LOST_AND_FOUND": 0.5,
+}
+
+# Name pools — 60 firsts × 60 lasts = 3600 combinations.
+FIRSTS = [
+    "Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
+    "Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
+    "Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
+    "Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
+    "Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
+    "Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
+    "Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
+    "Burning", "Frozen", "Whispering", "Howling",
+]
+LASTS = [
+    "Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
+    "Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
+    "Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
+    "Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
+    "Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
+    "Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
+    "Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
+    "Bluff", "Mesa", "Arroyo", "Ridge",
+]
+
+# Brief callsign pool for licensed-looking suffixes.
+CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
+
+# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
+# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
+# 6 bytes due to U+FE0F variation selector — explicitly excluded.
+EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
+                    "🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
+                    "🦇", "🦋"]
+
+# --------------------------------------------------------------------------
+# Helpers
+# --------------------------------------------------------------------------
+
+NUM_RESERVED = 4         # firmware reserves 0..3 (per NodeDB constants)
+NUM_MAX_EXCLUSIVE = 0x80000000  # restrict to positive int32 range for readability
+
+
+def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
+    """Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
+    keys = sorted(weights.keys())
+    totals = [weights[k] for k in keys]
+    return rng.choices(keys, weights=totals, k=1)[0]
+
+
+def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
+    base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
+    if is_licensed:
+        prefix = rng.choice(CALLSIGN_PREFIXES)
+        # Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
+        suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
+        # nanopb max_size:25 means C string fits 24 bytes + NUL.
+        if len(base) + len(suffix) <= 24:
+            base = base + suffix
+    # Hard cap to 24 chars (nanopb max_size:25 minus NUL).
+    return base[:24]
+
+
+def _gen_short_name(rng: random.Random, long_name: str) -> str:
+    # 10% emoji-only short_name
+    if rng.random() < 0.10:
+        return rng.choice(EMOJI_SHORTNAMES)
+    first_char = long_name[0].upper() if long_name else "X"
+    alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
+    return first_char + "".join(rng.choices(alphanums, k=3))
+
+
+def _gen_hops_away(rng: random.Random) -> int:
+    # Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
+    r = rng.random()
+    if r < 0.55:
+        return 0
+    if r < 0.80:
+        return 1
+    if r < 0.92:
+        return 2
+    if r < 0.97:
+        return 3
+    if r < 0.99:
+        return 4
+    return rng.randint(5, 7)
+
+
+def _gen_position(
+    rng: random.Random,
+    centroid_lat: float,
+    centroid_lon: float,
+    spread_km: float,
+    last_heard_offset_sec: int,
+) -> dict:
+    # 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
+    lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
+    lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
+    altitude = max(0, round(rng.gauss(1376.0, 250.0)))  # T or C valley floor + relief
+    # Position was reported up to 300s before last_heard.
+    time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
+    return {
+        "latitude": round(lat, 6),
+        "longitude": round(lon, 6),
+        "altitude": altitude,
+        "time_offset_sec": time_offset_sec,
+        "location_source": "LOC_INTERNAL",
+    }
+
+
+def _gen_telemetry(rng: random.Random) -> dict:
+    # 5% plugged-in (battery_level == 101); rest uniform [10..100].
+    if rng.random() < 0.05:
+        battery_level = 101
+        voltage = 4.20
+    else:
+        battery_level = rng.randint(10, 100)
+        voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
+    # Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
+    def _beta(a: float, b: float) -> float:
+        x = rng.gammavariate(a, 1.0)
+        y = rng.gammavariate(b, 1.0)
+        return x / (x + y)
+    channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
+    air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
+    uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
+    return {
+        "battery_level": battery_level,
+        "voltage": voltage,
+        "channel_utilization": channel_utilization,
+        "air_util_tx": air_util_tx,
+        "uptime_seconds": uptime_seconds,
+    }
+
+
+def _gen_environment(rng: random.Random) -> dict:
+    return {
+        "temperature": round(rng.gauss(22.0, 8.0), 2),
+        "relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
+        "barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
+        "iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
+    }
+
+
+def _gen_status(rng: random.Random) -> dict:
+    # `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
+    # Most peers report a healthy short status; occasional alert string.
+    healthy = ["OK", "online", "active", "running", "ready", "nominal"]
+    alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
+    if rng.random() < 0.92:
+        return {"status": rng.choice(healthy)}
+    return {"status": rng.choice(alert)}
+
+
+def _gen_node(
+    rng: random.Random,
+    num: int,
+    centroid_lat: float,
+    centroid_lon: float,
+    spread_km: float,
+    coverage: dict[str, float],
+    last_heard_mean_sec: int,
+    last_heard_max_sec: int,
+) -> dict:
+    is_licensed = rng.random() < 0.05
+    long_name = _gen_long_name(rng, is_licensed)
+    short_name = _gen_short_name(rng, long_name)
+    hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
+    role = _weighted_choice(rng, ROLE_WEIGHTS)
+    has_public_key = rng.random() < 0.92
+    public_key_hex = (
+        "".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
+    )
+    snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
+    channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
+    hops_away = _gen_hops_away(rng)
+    next_hop = rng.randint(0, 255) if hops_away > 0 else 0
+    last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
+
+    bitfield = {
+        "has_user": True,
+        "is_favorite": rng.random() < 0.08,
+        "is_muted": rng.random() < 0.03,
+        "via_mqtt": rng.random() < 0.12,
+        "is_ignored": rng.random() < 0.01,
+        "is_licensed": is_licensed,
+        "has_is_unmessagable": True,
+        "is_unmessagable": rng.random() < 0.02,
+        "is_key_manually_verified": rng.random() < 0.04,
+    }
+
+    node: dict = {
+        "num": f"0x{num:08x}",
+        "long_name": long_name,
+        "short_name": short_name,
+        "hw_model": hw_model,
+        "role": role,
+        "public_key_hex": public_key_hex,
+        "snr": snr,
+        "channel": channel,
+        "hops_away": hops_away,
+        "next_hop": next_hop,
+        "last_heard_offset_sec": last_heard_offset_sec,
+        "bitfield": bitfield,
+        "position": (
+            _gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
+            if rng.random() < coverage["position"]
+            else None
+        ),
+        "telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
+        "environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
+        "status": _gen_status(rng) if rng.random() < coverage["status"] else None,
+    }
+    return node
+
+
+def _parse_my_node_num(s: str | None) -> int | None:
+    if s is None:
+        return None
+    s = s.strip()
+    if s.startswith("0x") or s.startswith("0X"):
+        return int(s, 16)
+    return int(s)
+
+
+def main(argv: list[str]) -> int:
+    p = argparse.ArgumentParser(
+        description="Deterministic JSONL seed for the fake NodeDB fixture.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
+    p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
+    p.add_argument("--out", required=True, help="Output JSONL path.")
+    p.add_argument(
+        "--centroid",
+        default="33.1284,-107.2528",
+        help="LAT,LON centroid (default: Truth or Consequences, NM).",
+    )
+    p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
+    p.add_argument("--position-coverage", type=float, default=0.85)
+    p.add_argument("--telemetry-coverage", type=float, default=0.70)
+    p.add_argument("--environment-coverage", type=float, default=0.25)
+    p.add_argument("--status-coverage", type=float, default=0.40)
+    p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
+    p.add_argument("--last-heard-mean-sec", type=int, default=3600)
+    p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
+    args = p.parse_args(argv)
+
+    if args.count <= 0:
+        print("--count must be positive", file=sys.stderr)
+        return 2
+
+    try:
+        centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
+    except ValueError:
+        print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
+        return 2
+
+    my_node_num = _parse_my_node_num(args.my_node_num)
+
+    rng = random.Random(args.seed)
+
+    # 1) Generate a unique deterministic set of NodeNums.
+    nums: set[int] = set()
+    while len(nums) < args.count:
+        n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
+        if my_node_num is not None and n == my_node_num:
+            continue
+        nums.add(n)
+    ordered_nums = sorted(nums)  # sort to fix output order independent of set hash
+
+    # 2) Per-node generation (in num order, single RNG continues).
+    coverage = {
+        "position": args.position_coverage,
+        "telemetry": args.telemetry_coverage,
+        "environment": args.environment_coverage,
+        "status": args.status_coverage,
+    }
+    nodes = [
+        _gen_node(
+            rng,
+            n,
+            centroid_lat,
+            centroid_lon,
+            args.spread_km,
+            coverage,
+            args.last_heard_mean_sec,
+            args.last_heard_max_sec,
+        )
+        for n in ordered_nums
+    ]
+
+    # 3) Write JSONL.
+    out_path = pathlib.Path(args.out)
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    # `generated_at_iso` is informational; it does NOT affect determinism because
+    # we derive it from the seed, not from wall clock. (Same seed -> same string.)
+    generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
+    meta = {
+        "_meta": {
+            "version": 25,
+            "seed": args.seed,
+            "count": args.count,
+            "centroid": [centroid_lat, centroid_lon],
+            "spread_km": args.spread_km,
+            "generated_at_iso": generated_at,
+            "my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
+            "coverage": coverage,
+            "last_heard_mean_sec": args.last_heard_mean_sec,
+            "last_heard_max_sec": args.last_heard_max_sec,
+        }
+    }
+    with out_path.open("w", encoding="utf-8") as f:
+        # `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
+        # determinism (insertion order varies by Python version otherwise).
+        f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
+        for node in nodes:
+            f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
+
+    print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

bin/org.meshtastic.meshtasticd.metainfo.xml

@@ -87,6 +87,9 @@
   </screenshots>
 
   <releases>
+    <release version="2.7.24" date="2026-05-08">
+      <url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.24</url>
+    </release>
     <release version="2.7.23" date="2026-04-14">
       <url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.23</url>
     </release>

bin/platformio-custom.py

@@ -293,9 +293,12 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
 board_arch = infer_architecture(env.BoardConfig())
 should_skip_manifest = board_arch is None
 
-# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
-mtjson_deps = [] if should_skip_manifest else ["buildprog"]
-if not should_skip_manifest and platform.name == "espressif32":
+# Most platforms can generate the manifest as part of the default 'buildprog' target.
+# Typically this passes success/failure properly.
+mtjson_deps = ["buildprog"]
+if platform.name == "espressif32":
+    # On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
+    mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
     # Build littlefs image as part of mtjson target
     # Equivalent to `pio run -t buildfs`
     target_lfs = env.DataToBin(
@@ -309,7 +312,8 @@ if should_skip_manifest:
 
     env.AddCustomTarget(
         name="mtjson",
-        dependencies=mtjson_deps,
+        # For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
+        dependencies=[],
         actions=[skip_manifest],
         title="Meshtastic Manifest (skipped)",
         description="mtjson generation is skipped for native environments",

bin/regen-fake-nodedbs.sh

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
+# JSONL seed files + their compiled v25 protobufs.
+#
+# Layout:
+#   test/fixtures/nodedb/seed_v25_<N>.jsonl   — COMMITTED, hand-editable.
+#   build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
+#                                               Drop into /prefs/nodes.proto.
+#
+# Daily use:  ./bin/regen-fake-nodedbs.sh
+#   - Recompiles protos from committed seeds (fresh wall-clock timestamps).
+# Intentional seed bump:  REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
+#   - Overwrites the committed JSONL files with freshly-seeded data.
+
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
+if [[ ! -d bin/_generated/meshtastic ]]; then
+    echo "regenerating Python protobuf bindings (one-time)..."
+    ./bin/regen-py-protos.sh
+fi
+
+# 2) Pick a Python interpreter that has the meshtastic deps installed.
+#    Prefer the mcp-server venv (most likely to be set up by the operator).
+PY="python3"
+for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
+    if [[ -x "$cand" ]]; then
+        PY="$cand"
+        break
+    fi
+done
+
+# 3) Pinned seeds per size — bump only when you intentionally want different
+#    structural data committed. Parallel arrays so the script works on
+#    macOS bash 3.2 (no `declare -A`).
+SIZES=(250 500 1000 2000)
+SEEDS=(20260511 20260512 20260513 20260514)
+
+REGEN_SEEDS="${REGEN_SEEDS:-no}"
+
+mkdir -p build/fixtures/nodedb test/fixtures/nodedb
+
+for i in 0 1 2 3; do
+    n="${SIZES[$i]}"
+    seed="${SEEDS[$i]}"
+    jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
+    proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
+
+    if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
+        $PY bin/gen-fake-nodedb-seed.py \
+            --count "$n" \
+            --seed "$seed" \
+            --out "$jsonl" \
+            --centroid 33.1284,-107.2528 \
+            --spread-km 60 \
+            --position-coverage 0.85 \
+            --telemetry-coverage 0.70 \
+            --environment-coverage 0.25 \
+            --status-coverage 0.40
+        echo "  seed:   $jsonl ($(wc -c < "$jsonl") bytes)"
+    fi
+
+    $PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
+    echo "  proto:  $proto ($(wc -c < "$proto") bytes)"
+done
+
+echo ""
+echo "Done. To load on Portduino native:"
+echo "  cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
+echo ""
+echo "To push to a hardware device:"
+echo "  Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"

bin/regen-py-protos.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
+# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
+# a standalone refresh after any change to a .proto file.
+#
+# Output is .gitignored — bindings are a build artifact.
+#
+# Namespace rewrite:
+# The .proto files declare `package meshtastic;`, which makes protoc emit
+# imports like `from meshtastic import mesh_pb2`. That conflicts with the
+# PyPI `meshtastic` package (which the mcp-server relies on for its
+# SerialInterface/BLEInterface transport). We post-process the generated
+# files to live under `meshtastic_v25` instead — both the directory layout
+# and all internal imports — so they coexist cleanly with the PyPI package.
+
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+if ! command -v protoc >/dev/null 2>&1; then
+    echo "ERROR: protoc not found in PATH." >&2
+    echo "  macOS: brew install protobuf" >&2
+    echo "  Ubuntu/Debian: apt install protobuf-compiler" >&2
+    exit 1
+fi
+
+OUT=bin/_generated
+LOCAL_NS=meshtastic_v25
+
+rm -rf "$OUT"
+mkdir -p "$OUT"
+
+# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
+#    is available for the [(nanopb).*] options on other messages.
+protoc \
+    --proto_path=protobufs \
+    --python_out="$OUT" \
+    protobufs/nanopb.proto \
+    protobufs/meshtastic/*.proto
+
+# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
+mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
+
+# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
+#    `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
+python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
+
+# 4) Make the package importable.
+touch "$OUT/__init__.py"
+touch "$OUT/$LOCAL_NS/__init__.py"
+
+echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2

bin/seed-json-to-proto.py

@@ -0,0 +1,342 @@
+#!/usr/bin/env python3
+"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
+
+The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
+are stored as `*_offset_sec` (seconds before "now"); this script resolves them
+to absolute epochs using `--now-epoch` (default: current wall clock).
+
+Output is a raw `pb_encode`-compatible binary that can be dropped at
+`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
+XModem) and loaded by `NodeDB::loadFromDisk` at boot.
+
+Wire format reference:
+  protobufs/meshtastic/deviceonly.proto  (NodeDatabase, NodeInfoLite, sat entries)
+  src/mesh/NodeDB.h:467-484              (bitfield bit positions)
+  src/mesh/NodeDB.cpp:1523-1524          (pb_decode entry point)
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import pathlib
+import sys
+import time
+from typing import Any
+
+# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
+# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
+# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
+# ships. Run `bin/regen-py-protos.sh` to (re)generate.
+#
+# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
+# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
+# post-processes the protoc output to rename the package.
+_HERE = pathlib.Path(__file__).resolve().parent
+_LOCAL_PROTO_DIR = _HERE / "_generated"
+if _LOCAL_PROTO_DIR.is_dir():
+    sys.path.insert(0, str(_LOCAL_PROTO_DIR))
+
+try:
+    from meshtastic_v25.deviceonly_pb2 import (  # type: ignore[import-not-found]
+        NodeDatabase,
+        NodeInfoLite,
+        NodePositionEntry,
+        NodeTelemetryEntry,
+        NodeEnvironmentEntry,
+        NodeStatusEntry,
+        PositionLite,
+    )
+    from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage  # type: ignore[import-not-found]
+    from meshtastic_v25.config_pb2 import Config  # type: ignore[import-not-found]
+    from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics  # type: ignore[import-not-found]
+except ImportError as local_err:
+    # Fall back to the PyPI package if in-tree bindings haven't been generated.
+    # Will fail the v25 assertion below if the PyPI package predates the
+    # satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
+    # error message instead of an opaque ImportError.
+    try:
+        from meshtastic.protobuf.deviceonly_pb2 import (
+            NodeDatabase,
+            NodeInfoLite,
+            NodePositionEntry,
+            NodeTelemetryEntry,
+            NodeEnvironmentEntry,
+            NodeStatusEntry,
+            PositionLite,
+        )
+        from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
+        from meshtastic.protobuf.config_pb2 import Config
+        from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
+    except ImportError as pypi_err:
+        print(
+            "ERROR: could not import meshtastic protobuf bindings.\n"
+            "  In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
+            "  PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
+            f"  local error (meshtastic_v25): {local_err}\n"
+            f"  pypi error  (meshtastic.protobuf): {pypi_err}",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+# Fail loudly if bindings predate v25 (no satellite arrays).
+assert (
+    hasattr(NodeDatabase, "DESCRIPTOR")
+    and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
+), (
+    "Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
+    "Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
+)
+
+# ---------------------------------------------------------------------------
+# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
+# ---------------------------------------------------------------------------
+BIT_IS_KEY_MANUALLY_VERIFIED = 0
+BIT_IS_MUTED = 1
+BIT_VIA_MQTT = 2
+BIT_IS_FAVORITE = 3
+BIT_IS_IGNORED = 4
+BIT_HAS_USER = 5
+BIT_IS_LICENSED = 6
+BIT_IS_UNMESSAGABLE = 7
+BIT_HAS_IS_UNMESSAGABLE = 8
+
+BITFIELD_LAYOUT = (
+    # JSON key            bit position
+    ("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
+    ("is_muted", BIT_IS_MUTED),
+    ("via_mqtt", BIT_VIA_MQTT),
+    ("is_favorite", BIT_IS_FAVORITE),
+    ("is_ignored", BIT_IS_IGNORED),
+    ("has_user", BIT_HAS_USER),
+    ("is_licensed", BIT_IS_LICENSED),
+    ("is_unmessagable", BIT_IS_UNMESSAGABLE),
+    ("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
+)
+
+
+def _pack_bitfield(bf: dict[str, bool]) -> int:
+    out = 0
+    for key, shift in BITFIELD_LAYOUT:
+        if bf.get(key, False):
+            out |= (1 << shift)
+    return out
+
+
+def _validate_node(node: dict[str, Any]) -> None:
+    """Friendly errors so hand-editors get clear feedback."""
+    if "num" not in node or not isinstance(node["num"], str):
+        raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
+    if "long_name" not in node:
+        raise ValueError(f"node {node['num']}: missing 'long_name'")
+    if len(node["long_name"]) > 24:
+        raise ValueError(
+            f"node {node['num']}: long_name {node['long_name']!r} is "
+            f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
+        )
+    if "short_name" in node:
+        # short_name max_size:5 (incl. NUL) → 4 bytes of content.
+        # Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
+        # would slip past a `len(str) > 4` check. Always measure bytes.
+        b = node["short_name"].encode("utf-8")
+        if len(b) > 4:
+            raise ValueError(
+                f"node {node['num']}: short_name {node['short_name']!r} is "
+                f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
+            )
+    pk = node.get("public_key_hex", "")
+    if pk and len(pk) != 64:
+        raise ValueError(
+            f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
+            f"got {len(pk)} chars"
+        )
+    if pk:
+        try:
+            bytes.fromhex(pk)
+        except ValueError as e:
+            raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
+
+
+def _resolve_time(
+    node: dict[str, Any],
+    field_absolute: str,
+    field_offset: str,
+    now_epoch: int,
+) -> int:
+    """If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
+    if field_absolute in node and node[field_absolute] is not None:
+        return int(node[field_absolute])
+    offset = node.get(field_offset, 0)
+    return max(0, int(now_epoch) - int(offset))
+
+
+def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
+    _validate_node(node)
+    info = NodeInfoLite()
+    info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
+    info.long_name = node.get("long_name", "")
+    info.short_name = node.get("short_name", "")
+    # Enum lookups will raise ValueError on unknown names — that's exactly what we want.
+    info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
+    info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
+    pk_hex = node.get("public_key_hex", "")
+    if pk_hex:
+        info.public_key = bytes.fromhex(pk_hex)
+    info.snr = float(node.get("snr", 0.0))
+    info.channel = int(node.get("channel", 0))
+    if "hops_away" in node:
+        # `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
+        # field implicitly sets HasField("hops_away") to True. No has_hops_away
+        # setter exists (unlike the C++ nanopb-generated header).
+        info.hops_away = int(node["hops_away"])
+    info.next_hop = int(node.get("next_hop", 0))
+    info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
+    info.bitfield = _pack_bitfield(node.get("bitfield", {}))
+    return info
+
+
+def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
+    entry = NodePositionEntry()
+    entry.num = num
+    pl = PositionLite()
+    # Firmware stores lat/long as int32 in 1e-7 degrees.
+    pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
+    pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
+    pl.altitude = int(pos.get("altitude", 0))
+    pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
+    pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
+    entry.position.CopyFrom(pl)
+    return entry
+
+
+def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
+    entry = NodeTelemetryEntry()
+    entry.num = num
+    dm = DeviceMetrics()
+    if "battery_level" in tel:
+        dm.battery_level = int(tel["battery_level"])
+    if "voltage" in tel:
+        dm.voltage = float(tel["voltage"])
+    if "channel_utilization" in tel:
+        dm.channel_utilization = float(tel["channel_utilization"])
+    if "air_util_tx" in tel:
+        dm.air_util_tx = float(tel["air_util_tx"])
+    if "uptime_seconds" in tel:
+        dm.uptime_seconds = int(tel["uptime_seconds"])
+    entry.device_metrics.CopyFrom(dm)
+    return entry
+
+
+def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
+    entry = NodeEnvironmentEntry()
+    entry.num = num
+    em = EnvironmentMetrics()
+    if "temperature" in env:
+        em.temperature = float(env["temperature"])
+    if "relative_humidity" in env:
+        em.relative_humidity = float(env["relative_humidity"])
+    if "barometric_pressure" in env:
+        em.barometric_pressure = float(env["barometric_pressure"])
+    if "iaq" in env:
+        em.iaq = int(env["iaq"])
+    entry.environment_metrics.CopyFrom(em)
+    return entry
+
+
+def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
+    # `StatusMessage` (mesh.proto:1445) has a single `string status` field.
+    entry = NodeStatusEntry()
+    entry.num = num
+    sm = StatusMessage()
+    if "status" in status:
+        sm.status = str(status["status"])
+    entry.status.CopyFrom(sm)
+    return entry
+
+
+def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
+    """Read a seed JSONL and return the encoded NodeDatabase bytes."""
+    lines = jsonl_path.read_text(encoding="utf-8").splitlines()
+    if not lines:
+        raise ValueError(f"{jsonl_path} is empty")
+    meta_line = lines[0]
+    meta_obj = json.loads(meta_line)
+    meta = meta_obj.get("_meta", {})
+    version = meta.get("version")
+    if version != 25:
+        raise ValueError(
+            f"{jsonl_path}: meta version is {version!r}; this compiler "
+            f"requires version=25. Regenerate the seed with the matching tooling."
+        )
+
+    db = NodeDatabase()
+    db.version = 25
+
+    for ln, raw in enumerate(lines[1:], start=2):
+        raw = raw.strip()
+        if not raw:
+            continue
+        try:
+            node = json.loads(raw)
+        except json.JSONDecodeError as e:
+            raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
+
+        num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
+
+        # Header
+        info = _build_node_info_lite(node, now_epoch)
+        db.nodes.append(info)
+
+        # Satellites (nullable)
+        if node.get("position"):
+            db.positions.append(_build_position_entry(num, node["position"], now_epoch))
+        if node.get("telemetry"):
+            db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
+        if node.get("environment"):
+            db.environment.append(_build_environment_entry(num, node["environment"]))
+        if node.get("status"):
+            db.status.append(_build_status_entry(num, node["status"]))
+
+    return db.SerializeToString()
+
+
+def main(argv: list[str]) -> int:
+    p = argparse.ArgumentParser(
+        description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
+    p.add_argument("--out", required=True, help="Output binary .proto path.")
+    p.add_argument(
+        "--now-epoch",
+        type=int,
+        default=None,
+        help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
+    )
+    args = p.parse_args(argv)
+
+    in_path = pathlib.Path(args.in_path)
+    if not in_path.is_file():
+        print(f"input not found: {in_path}", file=sys.stderr)
+        return 2
+
+    now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
+
+    try:
+        encoded = compile_jsonl_to_proto(in_path, now_epoch)
+    except ValueError as e:
+        print(f"ERROR: {e}", file=sys.stderr)
+        return 3
+
+    out_path = pathlib.Path(args.out)
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    out_path.write_bytes(encoded)
+    print(
+        f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
+        file=sys.stderr,
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

boards/CDEBYTE_EoRa-S3.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-D CDEBYTE_EORA_S3",
       "-D ARDUINO_USB_CDC_ON_BOOT=1",
-      "-D ARDUINO_USB_MODE=0",
+      "-D ARDUINO_USB_MODE=1",
       "-D ARDUINO_RUNNING_CORE=1",
       "-D ARDUINO_EVENT_RUNNING_CORE=1",
       "-D BOARD_HAS_PSRAM"

boards/ThinkNode-M7.json

@@ -0,0 +1,42 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-D BOARD_HAS_PSRAM",
+      "-D ARDUINO_USB_CDC_ON_BOOT=0",
+      "-D ARDUINO_USB_MODE=0",
+      "-D ARDUINO_RUNNING_CORE=1",
+      "-D ARDUINO_EVENT_RUNNING_CORE=0"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "psram_type": "qio_opi",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "ELECROW-ThinkNode-M7"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "ELECROW ThinkNode M7",
+  "upload": {
+    "flash_size": "8MB",
+    "maximum_ram_size": 524288,
+    "maximum_size": 8388608,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "https://www.elecrow.com",
+  "vendor": "ELECROW"
+}

boards/bpi_picow_esp32_s3.json

@@ -6,7 +6,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/hackaday-communicator.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_v4_r8.json

@@ -0,0 +1,43 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "partitions": "default_16MB.csv",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-DBOARD_HAS_PSRAM",
+      "-DARDUINO_USB_CDC_ON_BOOT=1",
+      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_RUNNING_CORE=1",
+      "-DARDUINO_EVENT_RUNNING_CORE=1"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "psram_type": "opi",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "heltec_v4_r8"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "heltec_wifi_lora_32 v4 r8 (16 MB FLASH, 8 MB PSRAM)",
+  "upload": {
+    "flash_size": "16MB",
+    "maximum_ram_size": 327680,
+    "maximum_size": 16777216,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "https://heltec.org/",
+  "vendor": "heltec"
+}

boards/heltec_vision_master_e213.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_e290.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_t190.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DHELTEC_WIRELESS_TRACKER",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker_v2.json

@@ -7,7 +7,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/seeed-xiao-s3.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=0"
     ],

boards/station-g3.json

@@ -0,0 +1,41 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-DBOARD_HAS_PSRAM",
+      "-DARDUINO_USB_CDC_ON_BOOT=1",
+      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_RUNNING_CORE=1",
+      "-DARDUINO_EVENT_RUNNING_CORE=0"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "station-g3"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "BQ Station G3",
+  "upload": {
+    "flash_size": "16MB",
+    "maximum_ram_size": 327680,
+    "maximum_size": 16777216,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "",
+  "vendor": "BQ Consulting"
+}

boards/t-beam-1w.json

@@ -9,7 +9,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_1W",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/t-deck.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tbeam-s3-core.json

@@ -8,7 +8,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_S3_CORE",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tlora-t3s3-v1.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-DLILYGO_T3S3_V1",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/unphone.json

@@ -10,7 +10,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DUNPHONE_SPIN=9",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

debian/changelog

@@ -1,3 +1,9 @@
+meshtasticd (2.7.24.0) unstable; urgency=medium
+
+  * Version 2.7.24
+
+ -- GitHub Actions <github-actions[bot]@users.noreply.github.com>  Fri, 08 May 2026 10:44:12 +0000
+
 meshtasticd (2.7.23.0) unstable; urgency=medium
 
   * Version 2.7.23

debian/ci_pack_sdeb.sh

@@ -1,4 +1,5 @@
 #!/usr/bin/bash
+set -e
 export DEBEMAIL="jbennett@incomsystems.biz"
 export PLATFORMIO_LIBDEPS_DIR=pio/libdeps
 export PLATFORMIO_PACKAGES_DIR=pio/packages

default_16MB.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x640000,
+app1,     app,  ota_1,   0x650000,0x640000,
+spiffs,   data, spiffs,  0xc90000,0x360000,
+coredump, data, coredump,0xFF0000,0x10000,

default_8MB.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x330000,
+app1,     app,  ota_1,   0x340000,0x330000,
+spiffs,   data, spiffs,  0x670000,0x180000,
+coredump, data, coredump,0x7F0000,0x10000,

extra_scripts/esp32_extra.py

@@ -70,17 +70,6 @@ def esp32_create_combined_bin(source, target, env):
 
 env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
 
-esp32_kind = env.GetProjectOption("custom_esp32_kind")
-if esp32_kind == "esp32":
-    # Free up some IRAM by removing auxiliary SPI flash chip drivers.
-    # Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
-    env.Append(
-        LINKFLAGS=[
-            "-Wl,--wrap=esp_flash_chip_gd",
-            "-Wl,--wrap=esp_flash_chip_issi",
-            "-Wl,--wrap=esp_flash_chip_winbond",
-        ]
-    )
-else:
-    # For newer ESP32 targets, using newlib nano works better.
-    env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
+# Enable Newlib Nano formatting to save space
+# ...but allow printf float support (compromise)
+env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])

extra_scripts/ld_response_file.py

@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+# trunk-ignore-all(ruff/F821)
+# trunk-ignore-all(flake8/F821): For SConstruct imports
+
+# force linker response file instead of command line arguments
+
+Import("env")
+
+
+def wrap_with_tempfile(command_key):
+    command = env.get(command_key)
+    if not command or not isinstance(command, str):
+        return
+    if "TEMPFILE(" in command:
+        return
+    env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
+
+
+# Force SCons to spill long commands into response files on this target.
+env.Replace(MAXLINELENGTH=8192)
+
+for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
+    wrap_with_tempfile(key)

mcp-server/.gitignore

@@ -7,6 +7,12 @@ __pycache__/
 dist/
 build/
 
+# Persistent device-log capture (recorder + Datadog cursor).
+# Cross-session JSONL streams written by the autouse Recorder singleton
+# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
+# fixture truncate doesn't touch it.
+.mtlog/
+
 # Test harness artifacts
 tests/report.html
 tests/junit.xml

mcp-server/README.md

@@ -166,15 +166,73 @@ rather than auto-`sudo`'ing mid-run.
 
 ## Environment variables
 
-| Var                        | Default                                                     | Purpose                                                             |
-| -------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------- |
-| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini`                      | Pin the firmware repo                                               |
-| `MESHTASTIC_PIO_BIN`       | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location                                             |
-| `MESHTASTIC_ESPTOOL_BIN`   | `<firmware>/.venv/bin/esptool` → `$PATH`                    | Override esptool                                                    |
-| `MESHTASTIC_NRFUTIL_BIN`   | `$PATH`                                                     | Override nrfutil                                                    |
-| `MESHTASTIC_PICOTOOL_BIN`  | `$PATH`                                                     | Override picotool                                                   |
-| `MESHTASTIC_MCP_SEED`      | `mcp-<user>-<host>`                                         | PSK seed for test-harness session (CI override)                     |
-| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log`                              | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it) |
+| Var                        | Default                                                     | Purpose                                                                                                          |
+| -------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
+| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini`                      | Pin the firmware repo                                                                                            |
+| `MESHTASTIC_PIO_BIN`       | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location                                                                                          |
+| `MESHTASTIC_ESPTOOL_BIN`   | `<firmware>/.venv/bin/esptool` → `$PATH`                    | Override esptool                                                                                                 |
+| `MESHTASTIC_NRFUTIL_BIN`   | `$PATH`                                                     | Override nrfutil                                                                                                 |
+| `MESHTASTIC_PICOTOOL_BIN`  | `$PATH`                                                     | Override picotool                                                                                                |
+| `MESHTASTIC_MCP_SEED`      | `mcp-<user>-<host>`                                         | PSK seed for test-harness session (CI override)                                                                  |
+| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log`                              | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it)                                              |
+| `MESHTASTIC_MCP_TCP_HOST`  | unset                                                       | `host` or `host:port` of a `meshtasticd` daemon to surface as a TCP device (see "TCP / native-host nodes" below) |
+
+## TCP / native-host nodes
+
+The `native-macos` and `native` PlatformIO envs build a headless `meshtasticd`
+binary that runs on the host (Apple Silicon / Intel macOS, or Linux Portduino).
+The daemon exposes the meshtastic TCP API on port `4403` rather than a USB
+serial endpoint — point the MCP server at it via `MESHTASTIC_MCP_TCP_HOST`:
+
+```bash
+# 1. Build + run a daemon on this host (see variants/native/portduino/platformio.ini
+#    for full Homebrew prereqs and CH341 LoRa-adapter setup).
+pio run -e native-macos
+~/.meshtasticd/meshtasticd
+
+# 2. Point the MCP server at it.
+export MESHTASTIC_MCP_TCP_HOST=localhost     # or host:port, default port 4403
+```
+
+**First-run gotcha — MAC address.** `meshtasticd` derives its MAC from the
+USB adapter's serial-number / product strings. Many cheap CH341 dongles
+(MeshStick included — VID 0x1A86 / PID 0x5512) ship with `iSerialNumber=0`
+and `iProduct=0`, so the daemon aborts on boot with `*** Blank MAC Address
+not allowed!`. Set the MAC explicitly in `config.yaml`:
+
+```yaml
+# Under General:
+MACAddress: 02:CA:FE:BA:BE:01
+```
+
+Use a locally-administered address (first byte's second-LSB set, e.g.
+`02:*` / `06:*` / `0A:*` / `0E:*`) to avoid colliding with a real OUI.
+
+There is also a `--hwid AA:BB:CC:DD:EE:FF` CLI flag visible in
+`meshtasticd --help`, but it is **currently broken** in
+`MAC_from_string()` (`src/platform/portduino/PortduinoGlue.cpp`): the
+function strips colons from its parameter but then reads bytes from the
+global `portduino_config.mac_address`, so `--hwid` is silently overridden
+when `MACAddress:` is also set, and crashes the daemon (uncaught
+`std::invalid_argument: stoi: no conversion`) when it isn't. Use the YAML
+form until that's fixed upstream.
+
+`list_devices` will surface the daemon as `tcp://localhost:4403` with
+`likely_meshtastic=True`, so `device_info`, `list_nodes`, `get_config`,
+`set_config`, `set_owner`, `send_text`, `userprefs_*`, and the admin RPCs
+auto-select it when no `port` is passed. Pass `port="tcp://other-host:9999"`
+explicitly to target a different daemon.
+
+**Tools that don't apply to a TCP/native node** (no USB hardware to operate
+on) raise a clear `ConnectionError` rather than failing mysteriously:
+`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`,
+`serial_open` (use info/admin tools directly), and the vendor escape hatches
+`esptool_*`, `nrfutil_*`, `picotool_*`. `pio_flash` against a `native*` env
+similarly raises — there's no upload step; use `build` and run the binary
+directly.
+
+The pytest harness in `tests/` still assumes USB-attached devices per role —
+TCP-aware fixtures are not part of this surface yet.
 
 ## Hardware Test Suite
 

mcp-server/scripts/datadog-dashboard.json

@@ -0,0 +1,217 @@
+{
+  "title": "Meshtastic Firmware — Recorder Stream",
+  "description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
+  "widgets": [
+    {
+      "definition": {
+        "title": "Free heap (bytes)",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "free_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "label": "bytes" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Heap slope (bytes/min) — last 1h",
+        "type": "query_value",
+        "precision": 0,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "slope",
+                "data_source": "metrics",
+                "query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
+                "aggregator": "avg"
+              }
+            ],
+            "response_format": "scalar"
+          }
+        ],
+        "conditional_formats": [
+          { "comparator": "<", "value": -100, "palette": "white_on_red" },
+          { "comparator": "<", "value": 0, "palette": "white_on_yellow" },
+          { "comparator": ">=", "value": 0, "palette": "white_on_green" }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Total heap (bytes)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "total_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Battery level (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "battery",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "min": "0", "max": "105" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Air utilization (TX %)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "airutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Channel utilization (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "chutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Log volume by level",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "response_format": "timeseries",
+            "display_type": "bars",
+            "queries": [
+              {
+                "name": "log_count",
+                "data_source": "logs",
+                "indexes": ["*"],
+                "compute": { "aggregation": "count" },
+                "search": { "query": "service:meshtastic-firmware" },
+                "group_by": [
+                  {
+                    "facet": "@level",
+                    "limit": 10,
+                    "sort": { "order": "desc", "aggregation": "count" }
+                  }
+                ]
+              }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recent ERROR / CRIT firmware logs",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "@port", "width": "auto" },
+              { "field": "@level", "width": "auto" },
+              { "field": "@thread", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recorder marker events",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware @level:MARK",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    }
+  ],
+  "template_variables": [
+    {
+      "name": "port",
+      "prefix": "port",
+      "available_values": [],
+      "default": "*"
+    },
+    { "name": "host", "prefix": "host", "available_values": [], "default": "*" }
+  ],
+  "layout_type": "ordered",
+  "notify_list": [],
+  "reflow_type": "auto"
+}

mcp-server/scripts/mtlog_to_datadog.py

@@ -0,0 +1,389 @@
+#!/usr/bin/env python3
+"""Forward selected recorder JSONL streams to Datadog.
+
+Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
+Logs Intake API and telemetry numerics to the Metrics v2 series API.
+Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
+duplicate rows already shipped from the current live files.
+
+This forwarder does not currently backfill rotated `.jsonl.gz` archives.
+If the recorder rotates before this process drains the live file, or the
+forwarder is down across a rotation, those older rows are skipped.
+
+Usage:
+    DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
+    ./scripts/mtlog_to_datadog.py --once          # catch up + exit
+    ./scripts/mtlog_to_datadog.py --since 3600   # backfill last hour from start
+
+Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
+Override via `DD_SITE=...` env var or `--site` flag for one-offs.
+
+The forwarder is a separate process by design — a Datadog outage or
+auth failure must not backpressure the recorder. We exit non-zero on
+fatal config errors (missing API key) and keep retrying on transient
+network/HTTP errors.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import socket
+import sys
+import time
+from pathlib import Path
+from typing import Any, Iterator
+
+try:
+    import requests
+except ImportError:
+    print(
+        "requests is required. Install it in the mcp-server venv: "
+        "uv pip install requests",
+        file=sys.stderr,
+    )
+    sys.exit(2)
+
+
+_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
+_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
+_METRICS_TPL = "https://api.{site}/api/v2/series"
+_LOG_BATCH = 50
+_METRICS_BATCH = 100
+_MAX_RETRIES = 5
+_RETRY_BASE_S = 1.5
+
+
+# --- streaming JSONL with byte-position cursor -------------------------
+
+
+class _StreamReader:
+    """Reads a single rotating JSONL with cursor-based resume.
+
+    This tails only the live `.jsonl` file. The recorder rotates files
+    (live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
+    the live file shrinks abruptly. We detect that via inode change OR live
+    size < cursor position, and reset the live-file cursor to 0.
+    """
+
+    def __init__(self, path: Path, cursor: dict[str, Any]):
+        self.path = path
+        self.cursor = cursor
+
+    def _state(self) -> tuple[int, int]:
+        """Return (inode, size) for the live file. (0, 0) if missing."""
+        try:
+            st = self.path.stat()
+            return (st.st_ino, st.st_size)
+        except FileNotFoundError:
+            return (0, 0)
+
+    def iter_new_records(self) -> Iterator[dict[str, Any]]:
+        ino, size = self._state()
+        last_ino = self.cursor.get("ino")
+        last_pos = int(self.cursor.get("pos") or 0)
+        if ino == 0:
+            return
+        if last_ino is not None and last_ino != ino:
+            # Rotation happened. Start over.
+            last_pos = 0
+        if last_pos > size:
+            # Live file truncated/shrunk under us — recorder rotated.
+            last_pos = 0
+        try:
+            with self.path.open("r", encoding="utf-8") as fh:
+                fh.seek(last_pos)
+                for line in fh:
+                    line = line.rstrip("\n")
+                    if not line:
+                        continue
+                    try:
+                        yield json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    last_pos = fh.tell()
+        except FileNotFoundError:
+            return
+        self.cursor["ino"] = ino
+        self.cursor["pos"] = last_pos
+
+
+def _load_cursor(path: Path) -> dict[str, Any]:
+    if not path.exists():
+        return {}
+    try:
+        return json.loads(path.read_text())
+    except (OSError, json.JSONDecodeError):
+        return {}
+
+
+def _save_cursor(path: Path, data: dict[str, Any]) -> None:
+    tmp = path.with_suffix(".json.tmp")
+    tmp.write_text(json.dumps(data, separators=(",", ":")))
+    tmp.replace(path)
+
+
+# --- Datadog clients ---------------------------------------------------
+
+
+class _DDSession:
+    """Pool one HTTPS session, share retry logic."""
+
+    def __init__(self, api_key: str, site: str, hostname: str) -> None:
+        self.api_key = api_key
+        self.site = site
+        self.hostname = hostname
+        self.session = requests.Session()
+        self.session.headers.update(
+            {
+                "DD-API-KEY": api_key,
+                "Content-Type": "application/json",
+            }
+        )
+
+    def _post(self, url: str, payload: Any) -> bool:
+        for attempt in range(_MAX_RETRIES):
+            try:
+                resp = self.session.post(url, json=payload, timeout=30)
+            except requests.RequestException as e:
+                _wait_retry(attempt, f"network error: {e}")
+                continue
+            if 200 <= resp.status_code < 300:
+                return True
+            if resp.status_code in (408, 429, 500, 502, 503, 504):
+                _wait_retry(
+                    attempt,
+                    f"HTTP {resp.status_code} retrying",
+                )
+                continue
+            print(
+                f"datadog refused: {resp.status_code} {resp.text[:200]}",
+                file=sys.stderr,
+            )
+            return False
+        return False
+
+    def send_logs(self, records: list[dict[str, Any]]) -> int:
+        if not records:
+            return 0
+        url = _LOG_INTAKE_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(records), _LOG_BATCH):
+            batch = records[i : i + _LOG_BATCH]
+            if self._post(url, batch):
+                sent += len(batch)
+        return sent
+
+    def send_metrics(self, series: list[dict[str, Any]]) -> int:
+        if not series:
+            return 0
+        url = _METRICS_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(series), _METRICS_BATCH):
+            batch = series[i : i + _METRICS_BATCH]
+            if self._post(url, {"series": batch}):
+                sent += len(batch)
+        return sent
+
+
+def _wait_retry(attempt: int, reason: str) -> None:
+    wait = _RETRY_BASE_S * (2**attempt)
+    print(
+        f"  retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
+        file=sys.stderr,
+    )
+    time.sleep(wait)
+
+
+# --- record → datadog payload ------------------------------------------
+
+
+def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
+    line = rec.get("line") or ""
+    tags = [
+        f"role:{rec.get('role')}",
+        f"port:{rec.get('port')}",
+    ]
+    level = rec.get("level")
+    if level:
+        tags.append(f"level:{level}")
+    tag = rec.get("tag")
+    if tag:
+        tags.append(f"thread:{tag}")
+    return {
+        "ddsource": "meshtastic-firmware",
+        "service": "meshtastic-firmware",
+        "hostname": host,
+        "message": line,
+        "ddtags": ",".join(t for t in tags if t and "None" not in t),
+        "timestamp": int((rec.get("ts") or time.time()) * 1000),
+        "level": level,
+    }
+
+
+def _telemetry_record_to_metrics(
+    rec: dict[str, Any], host: str
+) -> list[dict[str, Any]]:
+    fields = rec.get("fields") or {}
+    if not isinstance(fields, dict):
+        return []
+    variant = rec.get("variant") or "unknown"
+    ts = int(rec.get("ts") or time.time())
+    out: list[dict[str, Any]] = []
+    tags = []
+    if rec.get("port"):
+        tags.append(f"port:{rec['port']}")
+    if rec.get("role"):
+        tags.append(f"role:{rec['role']}")
+    if rec.get("from_node"):
+        tags.append(f"from_node:{rec['from_node']}")
+    tags.append(f"variant:{variant}")
+    for field, value in fields.items():
+        if not isinstance(value, (int, float)) or isinstance(value, bool):
+            continue
+        metric = f"mesh.{variant}.{_metric_safe(field)}"
+        out.append(
+            {
+                "metric": metric,
+                "type": 3,  # GAUGE
+                "points": [{"timestamp": ts, "value": float(value)}],
+                "tags": tags,
+                "resources": [{"type": "host", "name": host}],
+            }
+        )
+    return out
+
+
+def _metric_safe(name: str) -> str:
+    # Lowercase, replace non-alnum with underscore for safe metric names.
+    return "".join(c.lower() if c.isalnum() else "_" for c in name)
+
+
+# --- main loop ---------------------------------------------------------
+
+
+def run(
+    log_dir: Path,
+    *,
+    once: bool,
+    since_seconds: float | None,
+    poll_interval: float,
+    dd: _DDSession,
+) -> int:
+    cursor_path = log_dir / ".dd-cursor.json"
+    cursors = _load_cursor(cursor_path)
+
+    # `--since` overrides cursor: rewind to (now-since) timestamp.
+    # We can't seek by timestamp directly (cursor is byte position), so
+    # we just reset cursors to 0 and let the time filter in iter_new
+    # drop older records.
+    cutoff_ts: float | None = None
+    if since_seconds is not None:
+        cursors = {}
+        cutoff_ts = time.time() - since_seconds
+
+    sent_total = {"logs": 0, "telemetry": 0}
+
+    while True:
+        # logs.jsonl → DD logs
+        log_cursor = cursors.setdefault("logs", {})
+        log_batch: list[dict[str, Any]] = []
+        for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            log_batch.append(_log_record_to_dd(rec, dd.hostname))
+        if log_batch:
+            n = dd.send_logs(log_batch)
+            sent_total["logs"] += n
+            print(f"logs: sent {n}/{len(log_batch)}")
+
+        # telemetry.jsonl → DD metrics
+        telem_cursor = cursors.setdefault("telemetry", {})
+        metric_series: list[dict[str, Any]] = []
+        for rec in _StreamReader(
+            log_dir / "telemetry.jsonl", telem_cursor
+        ).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
+        if metric_series:
+            n = dd.send_metrics(metric_series)
+            sent_total["telemetry"] += n
+            print(f"telemetry: sent {n}/{len(metric_series)} metric points")
+
+        _save_cursor(cursor_path, cursors)
+
+        if once:
+            print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
+            return 0
+        time.sleep(poll_interval)
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--log-dir",
+        default=str(_DEFAULT_LOG_DIR),
+        help="Path to .mtlog/ (default: mcp-server/.mtlog)",
+    )
+    mode = parser.add_mutually_exclusive_group()
+    mode.add_argument("--once", action="store_true", help="Catch up then exit")
+    mode.add_argument(
+        "--tail",
+        action="store_true",
+        help="Daemon: poll forever (default)",
+    )
+    parser.add_argument(
+        "--since",
+        type=float,
+        default=None,
+        help="Backfill last N seconds. Resets cursor.",
+    )
+    parser.add_argument(
+        "--poll-interval",
+        type=float,
+        default=5.0,
+        help="Seconds between tail polls (default 5)",
+    )
+    parser.add_argument(
+        "--site",
+        default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
+        help=(
+            "Datadog site. Default is the team's instance (us5.datadoghq.com). "
+            "Override via DD_SITE env var or this flag."
+        ),
+    )
+    parser.add_argument(
+        "--host",
+        default=socket.gethostname(),
+        help="Hostname tag (default: socket.gethostname())",
+    )
+    args = parser.parse_args(argv)
+
+    api_key = os.environ.get("DD_API_KEY")
+    if not api_key:
+        print("DD_API_KEY env var required.", file=sys.stderr)
+        return 2
+
+    log_dir = Path(args.log_dir)
+    if not log_dir.exists():
+        print(
+            f"log dir {log_dir} does not exist — start the mcp-server first.",
+            file=sys.stderr,
+        )
+        return 2
+
+    dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
+    once = args.once and not args.tail
+    return run(
+        log_dir,
+        once=once,
+        since_seconds=args.since,
+        poll_interval=args.poll_interval,
+        dd=dd,
+    )
+
+
+if __name__ == "__main__":
+    sys.exit(main())

mcp-server/src/meshtastic_mcp/connection.py

@@ -1,4 +1,4 @@
-"""Context manager for meshtastic.SerialInterface connections.
+"""Context manager for meshtastic interface connections (serial + TCP).
 
 Every info/admin tool goes through `connect(port)` so we have a single place
 that:
@@ -6,8 +6,16 @@ that:
   - fails fast if a serial_session is already holding the port,
   - guarantees `.close()` is called, even on exception.
 
-The `SerialInterface` blocks on construction waiting for the node database;
-that's fine for v1 since every tool is a short-lived request.
+Two transports:
+  - Serial: USB-attached firmware on `/dev/cu.*` / `/dev/ttyUSB*` / `COM*`.
+  - TCP: a `meshtasticd` daemon (e.g. the native macOS / Linux Portduino
+    headless build) addressed as `tcp://host[:port]` (default port 4403).
+    Surfaced by `devices.list_devices()` when `MESHTASTIC_MCP_TCP_HOST` is
+    set, so `resolve_port(None)` auto-selects it like a USB candidate.
+
+Both `SerialInterface` and `TCPInterface` block on construction waiting for
+the node database; that's fine for v1 since every tool is a short-lived
+request.
 """
 
 from __future__ import annotations
@@ -17,20 +25,107 @@ from typing import Iterator
 
 from . import devices, registry
 
+DEFAULT_TCP_PORT = 4403
+TCP_SCHEME = "tcp://"
+TCP_HOST_ENV = "MESHTASTIC_MCP_TCP_HOST"
+
 
 class ConnectionError(RuntimeError):
     pass
 
 
+def is_tcp_port(port: str | None) -> bool:
+    return bool(port) and port.startswith(TCP_SCHEME)
+
+
+def parse_tcp_port(port: str) -> tuple[str, int]:
+    """Parse `tcp://host[:port]` → (host, port). Defaults to 4403.
+
+    Validates host shape (non-empty, no path separators) and port range
+    (1..65535). Raises `ConnectionError` on malformed input — never lets
+    a raw `ValueError` bubble up to a tool surface.
+    """
+    if not port.startswith(TCP_SCHEME):
+        raise ConnectionError(
+            f"Invalid TCP endpoint {port!r}: expected '{TCP_SCHEME}host[:port]'."
+        )
+    rest = port[len(TCP_SCHEME) :]
+    if ":" in rest:
+        host, port_str = rest.rsplit(":", 1)
+        try:
+            tcp_port = int(port_str)
+        except ValueError as e:
+            raise ConnectionError(
+                f"Invalid TCP endpoint {port!r}: port {port_str!r} is not an integer."
+            ) from e
+    else:
+        host, tcp_port = rest, DEFAULT_TCP_PORT
+    if not host:
+        raise ConnectionError(f"Invalid TCP endpoint {port!r}: empty host.")
+    if any(c in host for c in ("/", "\\")):
+        raise ConnectionError(
+            f"Invalid TCP endpoint {port!r}: host {host!r} contains a path "
+            "separator. TCP hostnames cannot contain '/' or '\\' — did you "
+            "pass a serial port path or a Windows drive path by mistake?"
+        )
+    if not (1 <= tcp_port <= 65535):
+        raise ConnectionError(
+            f"Invalid TCP endpoint {port!r}: port {tcp_port} out of range "
+            "(must be 1..65535)."
+        )
+    return host, tcp_port
+
+
+def normalize_tcp_endpoint(endpoint: str) -> str:
+    r"""Normalize `host`, `host:port`, or `tcp://host[:port]` → canonical
+    `tcp://host:port` form. One place that owns the lock-key shape.
+
+    Defers all validation to `parse_tcp_port`, so path-like inputs
+    (`/dev/cu.foo`, `C:\Windows\…`), empty hosts, non-integer ports,
+    and out-of-range ports raise `ConnectionError` here too.
+    """
+    if endpoint.startswith(TCP_SCHEME):
+        canonical = endpoint
+    elif ":" in endpoint:
+        canonical = f"{TCP_SCHEME}{endpoint}"
+    else:
+        canonical = f"{TCP_SCHEME}{endpoint}:{DEFAULT_TCP_PORT}"
+    host, port = parse_tcp_port(canonical)
+    return f"{TCP_SCHEME}{host}:{port}"
+
+
+def reject_if_tcp(port: str | None, tool_name: str) -> None:
+    """Raise if `port` is a TCP endpoint — for tools that need real USB
+    hardware (flash, bootloader, vendor escape hatches, serial monitor).
+
+    Only checks the explicit arg; auto-selection via env var is the caller's
+    responsibility to handle if it matters.
+    """
+    if is_tcp_port(port):
+        raise ConnectionError(
+            f"{tool_name} is not applicable to TCP/native nodes ({port}). "
+            "This tool requires USB-attached hardware."
+        )
+
+
 def resolve_port(port: str | None) -> str:
-    """Pick a port: explicit > sole likely_meshtastic candidate > error."""
+    """Pick a port: explicit > sole likely_meshtastic candidate > error.
+
+    A `tcp://` string passes through (after canonicalization). When `port`
+    is None and no USB candidates are present, `MESHTASTIC_MCP_TCP_HOST`
+    is consulted via `devices.list_devices()`.
+    """
     if port:
+        if is_tcp_port(port):
+            return normalize_tcp_endpoint(port)
         return port
     candidates = [d for d in devices.list_devices() if d["likely_meshtastic"]]
     if not candidates:
         raise ConnectionError(
-            "No Meshtastic devices detected. Plug one in or pass `port` explicitly. "
-            "Run `list_devices` with include_unknown=True to see all serial ports."
+            "No Meshtastic devices detected. Plug one in, set "
+            f"{TCP_HOST_ENV}=<host[:port]> for a meshtasticd daemon, "
+            "or pass `port` explicitly. Run `list_devices` with "
+            "include_unknown=True to see all serial ports."
         )
     if len(candidates) > 1:
         ports = ", ".join(c["port"] for c in candidates)
@@ -43,17 +138,62 @@ def resolve_port(port: str | None) -> str:
 
 @contextmanager
 def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
-    """Open a `meshtastic.SerialInterface` and always close it.
+    """Open a meshtastic interface (serial or TCP) and always close it.
 
-    Raises `ConnectionError` immediately if another serial session holds the
-    port (a `pio device monitor` in `serial_sessions/`, for instance).
+    For serial: raises `ConnectionError` immediately if another serial
+    session holds the port (a `pio device monitor` in `serial_sessions/`).
+    For TCP: no exclusive-access requirement, so the serial-session check
+    is skipped — but the `port_lock` still serializes parallel `connect()`
+    calls to the same daemon endpoint.
+
+    `timeout_s` is plumbed through to both `SerialInterface(timeout=...)`
+    and `TCPInterface(timeout=...)`. The meshtastic library uses the value
+    as the reply-wait deadline for `localNode.waitForConfig()` during
+    construction and for any subsequent admin RPC. `int()`-converted at
+    the boundary because the upstream API expects whole seconds.
     """
+    resolved = resolve_port(port)
+    timeout = int(timeout_s)
+
+    if is_tcp_port(resolved):
+        from meshtastic.tcp_interface import (
+            TCPInterface,  # type: ignore[import-untyped]
+        )
+
+        host, tcp_port = parse_tcp_port(resolved)
+        lock = registry.port_lock(resolved)
+        if not lock.acquire(blocking=False):
+            raise ConnectionError(
+                f"TCP endpoint {resolved} is busy — another device operation "
+                "is in flight. Retry shortly."
+            )
+
+        iface = None
+        try:
+            iface = TCPInterface(
+                hostname=host,
+                portNumber=tcp_port,
+                connectNow=True,
+                noProto=False,
+                timeout=timeout,
+            )
+            yield iface
+        finally:
+            if iface is not None:
+                try:
+                    iface.close()
+                except Exception:
+                    pass
+            try:
+                lock.release()
+            except RuntimeError:
+                pass
+        return
+
     from meshtastic.serial_interface import (
         SerialInterface,  # type: ignore[import-untyped]
     )
 
-    resolved = resolve_port(port)
-
     active = registry.active_session_for_port(resolved)
     if active is not None:
         raise ConnectionError(
@@ -70,7 +210,12 @@ def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
 
     iface = None
     try:
-        iface = SerialInterface(devPath=resolved, connectNow=True, noProto=False)
+        iface = SerialInterface(
+            devPath=resolved,
+            connectNow=True,
+            noProto=False,
+            timeout=timeout,
+        )
         yield iface
     finally:
         if iface is not None:

mcp-server/src/meshtastic_mcp/devices.py

@@ -1,13 +1,18 @@
-"""USB/serial device discovery.
+"""USB/serial + TCP device discovery.
 
 Combines the canonical `meshtastic.util.findPorts()` allowlist/blocklist with
 the richer metadata (`serial.tools.list_ports.comports()`) so callers see
 VID/PID, descriptions, and manufacturer strings alongside the "is this likely
 a Meshtastic device" signal.
+
+If `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` is set, a synthetic entry for the
+`meshtasticd` daemon at that endpoint is prepended to the result, so
+`resolve_port(None)` auto-selects it like a USB candidate.
 """
 
 from __future__ import annotations
 
+import os
 from typing import Any
 
 from serial.tools import list_ports
@@ -19,6 +24,45 @@ def _to_hex(value: int | None) -> str | None:
     return f"0x{value:04x}"
 
 
+def _tcp_endpoint_from_env() -> dict[str, Any] | None:
+    """Synthesize a TCP device entry from MESHTASTIC_MCP_TCP_HOST, if set.
+
+    If the env var is malformed (non-integer port, path-like host, etc.),
+    return an entry with `likely_meshtastic=False` and the parser error in
+    the description, rather than raising — `list_devices` is the diagnostic
+    tool a user reaches for when their env var isn't working, so it must
+    not crash on misconfiguration.
+    """
+    host = os.environ.get("MESHTASTIC_MCP_TCP_HOST")
+    if not host:
+        return None
+    # Lazy import to avoid a circular dependency (connection imports devices).
+    from . import connection
+
+    try:
+        port = connection.normalize_tcp_endpoint(host)
+        description = "meshtasticd (TCP)"
+        likely = True
+    except connection.ConnectionError as e:
+        # Surface the raw env-var value plus the parser's reason so the
+        # user can see exactly what they set and why it was rejected.
+        # Don't double the scheme if the user already prefixed `tcp://`.
+        port = host if host.startswith(connection.TCP_SCHEME) else f"tcp://{host}"
+        description = f"meshtasticd (TCP) — invalid MESHTASTIC_MCP_TCP_HOST: {e}"
+        likely = False
+    return {
+        "port": port,
+        "vid": None,
+        "pid": None,
+        "description": description,
+        "manufacturer": None,
+        "product": None,
+        "serial_number": None,
+        "likely_meshtastic": likely,
+        "blacklisted": False,
+    }
+
+
 def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
     """Return enriched info for serial ports, flagging Meshtastic candidates.
 
@@ -70,6 +114,22 @@ def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
             }
         )
 
-    # Stable ordering: likely_meshtastic first, then by port path
-    results.sort(key=lambda r: (not r["likely_meshtastic"], r["port"]))
+    # Append the TCP endpoint (if env var set) and sort everything together.
+    tcp_entry = _tcp_endpoint_from_env()
+    if tcp_entry is not None:
+        results.append(tcp_entry)
+
+    # Stable ordering: likely_meshtastic first; within rank, TCP wins over
+    # USB (explicit env-var configuration takes precedence over USB
+    # enumeration); then by port path. A misconfigured TCP entry has
+    # likely_meshtastic=False and lands among the other ignored entries —
+    # it does NOT pre-empt real USB devices at the top of the list.
+    results.sort(
+        key=lambda r: (
+            not r["likely_meshtastic"],
+            not r["port"].startswith("tcp://"),
+            r["port"],
+        )
+    )
+
     return results

mcp-server/src/meshtastic_mcp/fixtures.py

@@ -0,0 +1,382 @@
+"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
+
+The fixture pipeline is two-stage:
+  1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
+     fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
+  2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
+     protobuf with fresh wall-clock timestamps.
+
+This module exposes `push_fake_nodedb(...)`, the MCP tool that:
+  - target="portduino": compiles the JSONL into the device's prefs dir on
+    the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
+  - target="hardware":  compiles to a temp file, then streams it over the
+    XModem protocol (via the meshtastic SerialInterface/BLEInterface +
+    `meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
+    device. Triggers a reboot so the firmware loads the new state on next
+    boot.
+
+XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
+  * 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
+  * CRC16-CCITT (poly 0x1021, init 0x0000).
+  * SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
+    `FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
+  * SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
+  * EOT after the last chunk flushes + closes the file on-device.
+
+Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
+in the .github/copilot-instructions.md "never do these without asking" list).
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import hashlib
+import pathlib
+import queue
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+from typing import Any, Literal
+
+from .connection import connect, is_tcp_port
+
+# Resolve repo root so the tool works regardless of mcp-server cwd.
+_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
+_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
+_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
+
+_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
+_XMODEM_CHUNK = 128
+_XMODEM_SUB = 0x1A
+_ACK_TIMEOUT_INIT_S = 5.0
+_ACK_TIMEOUT_CHUNK_S = 2.0
+_MAX_CHUNK_RETRIES = 5
+
+_VALID_SIZES = (250, 500, 1000, 2000)
+
+
+class FixtureError(RuntimeError):
+    """Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
+
+
+# ---------------------------------------------------------------------------
+# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
+# Hand-rolled to avoid the optional `crcmod` dep.
+# ---------------------------------------------------------------------------
+def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
+    crc = init
+    for b in data:
+        crc ^= b << 8
+        for _ in range(8):
+            if crc & 0x8000:
+                crc = ((crc << 1) ^ 0x1021) & 0xFFFF
+            else:
+                crc = (crc << 1) & 0xFFFF
+    return crc
+
+
+# ---------------------------------------------------------------------------
+# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
+# doesn't have to duplicate the proto-encoding logic.
+# ---------------------------------------------------------------------------
+def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
+    if not _COMPILE_SCRIPT.is_file():
+        raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
+    cmd = [
+        sys.executable,
+        str(_COMPILE_SCRIPT),
+        "--in",
+        str(jsonl_path),
+        "--out",
+        str(out_path),
+    ]
+    try:
+        subprocess.run(cmd, check=True, capture_output=True, text=True)
+    except subprocess.CalledProcessError as exc:
+        raise FixtureError(
+            f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
+            f"  stdout: {exc.stdout}\n  stderr: {exc.stderr}"
+        ) from exc
+
+
+def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
+    if custom is not None:
+        p = pathlib.Path(custom).expanduser().resolve()
+        if not p.is_file():
+            raise FixtureError(f"custom_seed_jsonl not found: {p}")
+        return p
+    p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
+    if not p.is_file():
+        raise FixtureError(
+            f"missing committed seed at {p}. "
+            f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
+        )
+    return p
+
+
+# ---------------------------------------------------------------------------
+# Portduino push — file copy into ~/.portduino/<config>/prefs/
+# ---------------------------------------------------------------------------
+def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
+    home = pathlib.Path.home()
+    return home / ".portduino" / config_name / "prefs"
+
+
+def _push_portduino(
+    size: int,
+    jsonl: pathlib.Path,
+    portduino_config: str,
+    backup_existing: bool,
+) -> dict[str, Any]:
+    prefs = _portduino_prefs_dir(portduino_config)
+    prefs.mkdir(parents=True, exist_ok=True)
+    target = prefs / "nodes.proto"
+    backed_up_to: str | None = None
+    if backup_existing and target.is_file():
+        ts = int(time.time())
+        backup = prefs / f"nodes.proto.bak.{ts}"
+        shutil.move(str(target), str(backup))
+        backed_up_to = str(backup)
+    _compile_proto(jsonl, target)
+    raw = target.read_bytes()
+    return {
+        "transport": "portduino",
+        "path": str(target),
+        "bytes": len(raw),
+        "sha256": hashlib.sha256(raw).hexdigest(),
+        "jsonl_source": str(jsonl),
+        "backed_up_to": backed_up_to,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
+# ---------------------------------------------------------------------------
+@dataclasses.dataclass
+class _AckEvent:
+    control: int
+    seq: int
+
+
+def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
+    try:
+        return q.get(timeout=timeout_s)
+    except queue.Empty as exc:
+        raise FixtureError(
+            f"XModem response timeout after {timeout_s:.1f}s — device not responding"
+        ) from exc
+
+
+def _push_hardware(
+    size: int,
+    jsonl: pathlib.Path,
+    port: str | None,
+    reboot_after: bool,
+) -> dict[str, Any]:
+    # Lazy imports so the module loads even when the meshtastic deps aren't
+    # available (e.g. CI in a Python env without the package installed).
+    try:
+        from meshtastic.protobuf import mesh_pb2, xmodem_pb2
+        from pubsub import pub
+    except ImportError as exc:  # pragma: no cover — dep missing
+        raise FixtureError(
+            f"hardware push requires the meshtastic + pypubsub packages: {exc}"
+        ) from exc
+
+    if is_tcp_port(port):
+        raise FixtureError(
+            "hardware push over TCP/portduino is not supported — use "
+            "target='portduino' to drop the fixture directly into the prefs dir."
+        )
+
+    # Compile the fixture to a temp file with fresh timestamps.
+    with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
+        proto_path = pathlib.Path(tf.name)
+    try:
+        _compile_proto(jsonl, proto_path)
+        payload = proto_path.read_bytes()
+    finally:
+        proto_path.unlink(missing_ok=True)
+
+    sha256 = hashlib.sha256(payload).hexdigest()
+    total_bytes = len(payload)
+
+    # Subscribe to XModem responses BEFORE we open the interface, so we don't
+    # race the first ACK that arrives during the SOH/seq=0 handshake.
+    #
+    # NB: the signature MUST declare every kwarg pypubsub will see for this
+    # topic, or pubsub locks the topic spec to a smaller set (whichever
+    # subscribe arrives first) and then *rejects* the meshtastic library's
+    # publish call with `SenderUnknownMsgDataError: unknown ... interface`.
+    # The meshtastic lib publishes both `packet=` and `interface=`
+    # (mesh_interface.py:1389-1395), so both must appear here.
+    response_q: "queue.Queue[_AckEvent]" = queue.Queue()
+
+    def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
+        if packet is None:
+            return
+        response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
+
+    pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
+
+    chunks_sent = 0
+    retried = 0
+    rebooted = False
+
+    XMC = xmodem_pb2.XModem.Control
+    try:
+        with connect(port=port) as iface:
+            # 1) Send the filename (SOH, seq=0).
+            init_pkt = xmodem_pb2.XModem(
+                control=XMC.Value("SOH"),
+                seq=0,
+                buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
+            )
+            iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
+            ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
+            if ack.control != XMC.Value("ACK"):
+                raise FixtureError(
+                    f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
+                    f"(got control={ack.control}, expected ACK). "
+                    f"Filesystem full or permissions issue?"
+                )
+
+            # 2) Stream the payload in 128 B chunks.
+            for offset in range(0, total_bytes, _XMODEM_CHUNK):
+                chunk = payload[offset : offset + _XMODEM_CHUNK]
+                if len(chunk) < _XMODEM_CHUNK:
+                    # Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
+                    # become part of the file on-device, but nanopb ignores
+                    # bytes past the end of the top-level message.
+                    chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
+                seq = ((offset // _XMODEM_CHUNK) + 1) % 256
+                # Retry loop on NAK / timeout.
+                attempts = 0
+                while True:
+                    pkt = xmodem_pb2.XModem(
+                        control=XMC.Value("SOH"),
+                        seq=seq,
+                        buffer=chunk,
+                        crc16=_crc16_ccitt(chunk),
+                    )
+                    iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
+                    ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
+                    if ack.control == XMC.Value("ACK"):
+                        chunks_sent += 1
+                        break
+                    if ack.control == XMC.Value("NAK"):
+                        attempts += 1
+                        retried += 1
+                        if attempts >= _MAX_CHUNK_RETRIES:
+                            # Abort: send CAN so the firmware removes the half-
+                            # written file via FSCom.remove(filename).
+                            iface._sendToRadio(
+                                mesh_pb2.ToRadio(
+                                    xmodemPacket=xmodem_pb2.XModem(
+                                        control=XMC.Value("CAN")
+                                    )
+                                )
+                            )
+                            raise FixtureError(
+                                f"chunk seq={seq} NAK'd {attempts} times; "
+                                f"aborted transfer (file removed on-device)."
+                            )
+                        continue  # retry the same chunk
+                    raise FixtureError(
+                        f"unexpected XModem control={ack.control} on seq={seq}"
+                    )
+
+            # 3) Tell the device we're done.
+            iface._sendToRadio(
+                mesh_pb2.ToRadio(
+                    xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
+                )
+            )
+            ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
+            if ack.control != XMC.Value("ACK"):
+                raise FixtureError(f"EOT not ACKed (got control={ack.control})")
+
+            # 4) Reboot so loadFromDisk picks up the new file.
+            if reboot_after:
+                iface.localNode.reboot(secs=1)
+                rebooted = True
+    finally:
+        try:
+            pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
+        except Exception:
+            pass
+
+    return {
+        "transport": "hardware",
+        "port": port,
+        "filename_on_device": _DEFAULT_NODES_FILENAME,
+        "bytes": total_bytes,
+        "chunks_sent": chunks_sent,
+        "retried": retried,
+        "sha256": sha256,
+        "jsonl_source": str(jsonl),
+        "rebooted": rebooted,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Public entry point — registered as an MCP tool in server.py.
+# ---------------------------------------------------------------------------
+def push_fake_nodedb(
+    size: int,
+    target: Literal["portduino", "hardware"] = "portduino",
+    *,
+    port: str | None = None,
+    portduino_config: str = "default",
+    backup_existing: bool = True,
+    confirm: bool = False,
+    reboot_after: bool = True,
+    custom_seed_jsonl: str | None = None,
+) -> dict[str, Any]:
+    """Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
+
+    Args:
+      size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
+      target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
+              "hardware" (XModem upload to /prefs/nodes.proto + reboot).
+      port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
+            or BLE identifier. TCP endpoints are rejected — use target="portduino"
+            instead.
+      portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
+      backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
+                        if present, so you can roll back.
+      confirm: required True for target="hardware" (writes flash + reboots).
+      reboot_after: hardware only. If True, send a 1-second reboot after the
+                    final ACK so loadFromDisk picks up the new file at next boot.
+      custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
+                         test scenario.
+
+    Returns:
+        dict with transport, bytes, sha256, etc. — depends on target.
+
+    """
+    if size not in _VALID_SIZES:
+        raise FixtureError(
+            f"size must be one of {_VALID_SIZES}; got {size!r}. "
+            f"Add a new committed seed if you need a different cardinality."
+        )
+
+    jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
+
+    if target == "portduino":
+        return _push_portduino(size, jsonl, portduino_config, backup_existing)
+
+    if target == "hardware":
+        if not confirm:
+            raise FixtureError(
+                "hardware push writes flash and triggers a reboot — pass confirm=True."
+            )
+        if not port:
+            raise FixtureError(
+                "target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
+            )
+        return _push_hardware(size, jsonl, port, reboot_after)
+
+    raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")

mcp-server/src/meshtastic_mcp/flash.py

@@ -17,7 +17,7 @@ from typing import Any
 
 import serial
 
-from . import boards, config, devices, pio, userprefs
+from . import boards, config, connection, devices, pio, userprefs
 
 # Meshtastic variants use both `esp32s3` and `esp32-s3` style names across
 # variants/*/platformio.ini (no consistency enforced). Accept both spellings.
@@ -46,6 +46,18 @@ def _require_confirm(confirm: bool, operation: str) -> None:
         )
 
 
+def _reject_native_env(env: str, operation: str) -> None:
+    """`native*` envs build a host executable, not firmware — there's no
+    upload step. The user wants `build` (or just runs the binary directly).
+    """
+    if env.startswith("native"):
+        raise FlashError(
+            f"{operation} is not applicable for env {env!r}: native envs "
+            "produce a host executable, not flashable firmware. Use `build` "
+            "instead, then run the resulting binary directly."
+        )
+
+
 def _artifacts_for(env: str) -> list[Path]:
     build_dir = config.firmware_root() / ".pio" / "build" / env
     if not build_dir.is_dir():
@@ -96,18 +108,33 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Run `pio run -e <env>` and return artifact paths.
 
     `userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
     into userPrefs.jsonc for this build only. File is restored byte-for-byte
     on exit. Use `userprefs_set()` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
+    this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
+    `{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
+    prefix on every log line. Combines with the recorder so heap shows
+    up at log cadence (much higher resolution than the ~60 s LocalStats
+    packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
+    expand to bare `-D<NAME>` (presence-only flags).
     """
     args = ["run", "-e", env]
     if with_manifest:
         args.extend(["-t", "mtjson"])
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
-        result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
+        result = pio.run(
+            args,
+            timeout=pio.TIMEOUT_BUILD,
+            check=False,
+            extra_env=extra_env,
+        )
     return {
         "exit_code": result.returncode,
         "artifacts": [str(p) for p in _artifacts_for(env)],
@@ -115,9 +142,27 @@ def build(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 
+def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
+    """Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
+    "-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
+    the flag entirely. Other types stringify."""
+    parts: list[str] = []
+    for key, value in build_flags.items():
+        if value is False or value is None:
+            continue
+        if value is True:
+            parts.append(f"-D{key}")
+        else:
+            parts.append(f"-D{key}={value}")
+    if not parts:
+        return {}
+    return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
+
+
 def clean(env: str) -> dict[str, Any]:
     """Run `pio run -e <env> -t clean`."""
     result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -134,18 +179,29 @@ def flash(
     port: str,
     confirm: bool = False,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """`pio run -e <env> -t upload --upload-port <port>`. All architectures.
 
     `userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
     that pio performs will pick up the injected values.
+
+    `build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
+    is exported for the rebuild-before-upload, so the uploaded firmware
+    actually carries the flags. Without this propagation, `pio run -t upload`
+    would relink without the env var and silently drop them. Common use:
+    `build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
     """
     _require_confirm(confirm, "flash")
+    _reject_native_env(env, "flash")
+    connection.reject_if_tcp(port, "flash")
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
         result = pio.run(
             ["run", "-e", env, "-t", "upload", "--upload-port", port],
             timeout=pio.TIMEOUT_UPLOAD,
             check=False,
+            extra_env=extra_env,
         )
     return {
         "exit_code": result.returncode,
@@ -153,6 +209,7 @@ def flash(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 
@@ -200,6 +257,7 @@ def erase_and_flash(
     in that case) since a cached factory.bin would not reflect the new prefs.
     """
     _require_confirm(confirm, "erase_and_flash")
+    connection.reject_if_tcp(port, "erase_and_flash")
     _check_esp32_env(env)
 
     if userprefs_overrides and skip_build:
@@ -257,6 +315,7 @@ def update_flash(
     overrides are provided we always force a rebuild.
     """
     _require_confirm(confirm, "update_flash")
+    connection.reject_if_tcp(port, "update_flash")
     _check_esp32_env(env)
 
     if userprefs_overrides and skip_build:
@@ -391,6 +450,7 @@ def touch_1200bps(
 
     Returns `{ok, former_port, new_port, new_port_vid_pid, attempts}`.
     """
+    connection.reject_if_tcp(port, "touch_1200bps")
     before_list = devices.list_devices(include_unknown=True)
     before_ports = {d["port"] for d in before_list}
 

mcp-server/src/meshtastic_mcp/hw_tools.py

@@ -16,7 +16,7 @@ import subprocess
 from pathlib import Path
 from typing import Any, Sequence
 
-from . import config, pio
+from . import config, connection, pio
 
 _TIMEOUT_SHORT = 30
 _TIMEOUT_LONG = 600
@@ -102,6 +102,7 @@ def _parse_esptool_chip_info(stdout: str) -> dict[str, Any]:
 
 
 def esptool_chip_info(port: str) -> dict[str, Any]:
+    connection.reject_if_tcp(port, "esptool_chip_info")
     binary = config.esptool_bin()
     # `chip_id` prints chip + mac + crystal + features. `flash_id` adds flash.
     combined = _run(binary, ["--port", port, "flash_id"], timeout=_TIMEOUT_SHORT)
@@ -116,6 +117,7 @@ def esptool_chip_info(port: str) -> dict[str, Any]:
 def esptool_erase_flash(port: str, confirm: bool = False) -> dict[str, Any]:
     """Full-chip erase. Leaves the device unbootable until reflashed."""
     _require_confirm(confirm, "esptool_erase_flash")
+    connection.reject_if_tcp(port, "esptool_erase_flash")
     binary = config.esptool_bin()
     # esptool v5 uses `erase-flash`, older uses `erase_flash`. Try the new name
     # first; if it fails with unknown command, retry old.
@@ -134,6 +136,7 @@ def esptool_raw(
     """Raw esptool passthrough. Destructive subcommands require confirm=True."""
     if not args:
         raise ToolError("args must not be empty")
+    connection.reject_if_tcp(port, "esptool_raw")
     # Find the first non-flag arg (the subcommand).
     subcommand = next((a for a in args if not a.startswith("-")), None)
     if subcommand and subcommand.replace("-", "_") in {
@@ -156,6 +159,7 @@ NRFUTIL_DESTRUCTIVE = {"dfu", "settings"}
 
 def nrfutil_dfu(port: str, package_path: str, confirm: bool = False) -> dict[str, Any]:
     _require_confirm(confirm, "nrfutil_dfu")
+    connection.reject_if_tcp(port, "nrfutil_dfu")
     pkg = Path(package_path).expanduser()
     if not pkg.is_file():
         raise ToolError(f"Package not found: {pkg}")
@@ -213,6 +217,7 @@ def _parse_picotool_info(stdout: str) -> dict[str, Any]:
 def picotool_info(port: str | None = None) -> dict[str, Any]:
     """Read device info from a Pico in BOOTSEL mode. `port` is informational
     only — picotool auto-detects."""
+    connection.reject_if_tcp(port, "picotool_info")
     binary = config.picotool_bin()
     res = _run(binary, ["info", "-a"], timeout=_TIMEOUT_SHORT)
     if res["exit_code"] != 0:

mcp-server/src/meshtastic_mcp/log_query.py

@@ -0,0 +1,410 @@
+"""Read-side queries over the recorder's JSONL streams.
+
+Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
+loads a whole file. Time-bound queries short-circuit as soon as `ts`
+exceeds the requested end. The recorder writes monotonically, so a
+forward scan is cheap; we don't need an index.
+
+All time arguments accept:
+  - epoch seconds (int/float)
+  - relative strings: "-15m", "-2h", "-3d", "now"
+  - ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
+    treated as UTC)
+
+Tools that return data ALWAYS cap their output (max_lines / max_points
+/ max), and report whether more matched than was returned.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import re
+import statistics
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Iterator
+
+from .recorder.recorder import get_recorder
+
+_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
+_REGEX_PREVIEW_MAX = 100
+_REGEX_PREVIEW_TRUNCATE = 97
+
+
+def _parse_time(value: Any, *, now: float | None = None) -> float:
+    """Coerce to epoch seconds. Defaults `now` to `time.time()`."""
+    if value is None:
+        return time.time()
+    if isinstance(value, (int, float)):
+        return float(value)
+    if not isinstance(value, str):
+        raise ValueError(f"invalid time: {value!r}")
+    s = value.strip().lower()
+    if s in ("", "now"):
+        return time.time() if now is None else now
+    m = _REL_RE.match(s)
+    if m:
+        n = float(m.group(1))
+        unit = m.group(2)
+        secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
+        base = time.time() if now is None else now
+        return base - secs
+    # Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
+    try:
+        if s.endswith("z"):
+            s = s[:-1] + "+00:00"
+        dt = datetime.fromisoformat(s)
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=timezone.utc)
+        return dt.timestamp()
+    except ValueError as e:
+        raise ValueError(f"unparseable time: {value!r}") from e
+
+
+def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
+    """Stream records in chronological order: rotated archives first
+    (oldest → newest by lex sort, which is chronological for our
+    `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
+    last. The "keep last N" pop-front logic in the window queries
+    relies on records arriving in time order across files.
+    """
+    files: list[Path] = []
+    # Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
+    for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
+        files.append(archive)
+    if path.exists():
+        files.append(path)
+    for f in files:
+        opener = gzip.open if f.suffix == ".gz" else open
+        try:
+            with opener(f, "rt", encoding="utf-8") as fh:  # type: ignore[arg-type]
+                for line in fh:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    try:
+                        rec = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    ts = rec.get("ts")
+                    if not isinstance(ts, (int, float)):
+                        continue
+                    if ts < since:
+                        continue
+                    if ts > until:
+                        # Records are append-monotonic within a file, so
+                        # the rest of this file is also past `until`.
+                        # Archives can still overlap each other, so only
+                        # short-circuit this file, not the whole scan.
+                        break
+                    yield rec
+        except (FileNotFoundError, OSError):
+            continue
+
+
+# -- queries ------------------------------------------------------------
+
+
+def logs_window(
+    start: Any = "-15m",
+    end: Any = "now",
+    *,
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines, filtered.
+
+    `level` accepts a single level name or pipe-separated set
+    ("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
+    `line` field. Returns the last `max_lines` matches.
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    levels = _split_set(level)
+    if grep:
+        try:
+            grep_re = re.compile(grep)
+        except re.error as exc:
+            preview = (
+                grep
+                if len(grep) <= _REGEX_PREVIEW_MAX
+                else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
+            )
+            raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
+    else:
+        grep_re = None
+
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
+        if levels and rec.get("level") not in levels:
+            continue
+        if tag and rec.get("tag") != tag:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        if grep_re and not grep_re.search(rec.get("line") or ""):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max_lines:
+            out.pop(0)  # keep the most recent N
+    return {
+        "lines": out,
+        "total_matched": matched,
+        "dropped": max(0, matched - max_lines),
+        "window": {"start": s, "end": e},
+    }
+
+
+def telemetry_timeline(
+    window: Any = "1h",
+    *,
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Timeseries of one telemetry field, downsampled.
+
+    `field` matches both the protobuf snake_case name (`free_heap`,
+    `heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
+    Server-side bucket-mean downsamples to ≤ `max_points`. Returns
+    `slope_per_min` (linear regression slope, units/min) so a leak
+    detector can read one number.
+    """
+    end = time.time()
+    if isinstance(window, (int, float)):
+        # Numeric `window` is a duration in seconds — "last N seconds".
+        # Without this branch, `_parse_time(-N)` would treat -N as an
+        # absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
+        # producing a wildly negative `start` and matching nothing.
+        start = end - float(window)
+    elif isinstance(window, str) and not window.startswith("-"):
+        # Bare string like "1h" is sugar for "-1h".
+        start = _parse_time(f"-{window}", now=end)
+    else:
+        start = _parse_time(window, now=end)
+
+    base = get_recorder().base_dir
+    raw: list[tuple[float, float]] = []
+    field_aliases = _field_aliases(field)
+    for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
+        if rec.get("variant") != variant:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        fields = rec.get("fields") or {}
+        value: Any = None
+        for alias in field_aliases:
+            if alias in fields:
+                value = fields[alias]
+                break
+        if not isinstance(value, (int, float)):
+            continue
+        raw.append((float(rec["ts"]), float(value)))
+
+    if not raw:
+        return {
+            "points": [],
+            "samples": 0,
+            "min": None,
+            "max": None,
+            "slope_per_min": None,
+            "window": {"start": start, "end": end, "variant": variant, "field": field},
+        }
+
+    points = _downsample(raw, max_points=max_points)
+    values = [v for _, v in raw]
+    return {
+        "points": [{"ts": ts, "value": v} for ts, v in points],
+        "samples": len(raw),
+        "min": min(values),
+        "max": max(values),
+        "slope_per_min": _slope_per_min(raw),
+        "window": {"start": start, "end": end, "variant": variant, "field": field},
+    }
+
+
+def packets_window(
+    start: Any = "-5m",
+    end: Any = "now",
+    *,
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    portnums = _split_set(portnum)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
+        if portnums and rec.get("portnum") not in portnums:
+            continue
+        if from_node and str(rec.get("from_node")) != str(from_node):
+            continue
+        if to_node and str(rec.get("to_node")) != str(to_node):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "packets": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def events_window(
+    start: Any = "-1h",
+    end: Any = "now",
+    *,
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    kinds = _split_set(kind)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
+        if kinds and rec.get("kind") not in kinds:
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "events": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def export(
+    start: Any,
+    end: Any,
+    dest_dir: str,
+    *,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of each requested stream into `dest_dir`.
+
+    For a notebook, a bug report, or a Datadog backfill. Output files
+    are uncompressed JSONL (callers gzip themselves if they want to).
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    selected = streams or ["logs", "telemetry", "packets", "events"]
+    dest = Path(dest_dir)
+    dest.mkdir(parents=True, exist_ok=True)
+
+    base = get_recorder().base_dir
+    paths: dict[str, str] = {}
+    for stream in selected:
+        src = base / f"{stream}.jsonl"
+        if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
+            continue
+        out_path = dest / f"{stream}.jsonl"
+        n = 0
+        with out_path.open("w", encoding="utf-8") as fh:
+            for rec in _iter_jsonl(src, since=s, until=e):
+                fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
+                n += 1
+        paths[stream] = str(out_path)
+        paths[f"{stream}_count"] = str(n)
+    return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
+
+
+# -- helpers ------------------------------------------------------------
+
+
+def _split_set(value: str | None) -> set[str] | None:
+    if not value:
+        return None
+    return {v.strip() for v in value.split("|") if v.strip()}
+
+
+def _field_aliases(field: str) -> list[str]:
+    """Accept snake_case OR camelCase, plus a few legacy aliases."""
+    snake = field
+    camel = _snake_to_camel(field)
+    aliases = {snake, camel}
+    # Old protobuf fields (pre-LocalStats) used different names
+    legacy = {
+        "free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
+        "heap_free_bytes": [
+            "heap_free_bytes",
+            "heapFreeBytes",
+            "free_heap",
+            "freeHeap",
+        ],
+        "total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
+        "heap_total_bytes": [
+            "heap_total_bytes",
+            "heapTotalBytes",
+            "total_heap",
+            "totalHeap",
+        ],
+    }
+    if field in legacy:
+        aliases.update(legacy[field])
+    return list(aliases)
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _downsample(
+    points: list[tuple[float, float]], *, max_points: int
+) -> list[tuple[float, float]]:
+    if len(points) <= max_points:
+        return points
+    # Even-bucket mean. Preserves shape better than nth-sample picking.
+    n = len(points)
+    bucket = n / max_points
+    out: list[tuple[float, float]] = []
+    i = 0
+    for k in range(max_points):
+        end = int((k + 1) * bucket)
+        end = min(end, n)
+        if end <= i:
+            continue
+        chunk = points[i:end]
+        ts = chunk[len(chunk) // 2][0]
+        val = statistics.fmean(v for _, v in chunk)
+        out.append((ts, val))
+        i = end
+    return out
+
+
+def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
+    """Least-squares slope (units per minute). None if too few points."""
+    if len(points) < 2:
+        return None
+    xs = [t for t, _ in points]
+    ys = [v for _, v in points]
+    n = len(xs)
+    mean_x = sum(xs) / n
+    mean_y = sum(ys) / n
+    num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
+    den = sum((x - mean_x) ** 2 for x in xs)
+    if den == 0:
+        return None
+    slope_per_sec = num / den
+    return slope_per_sec * 60.0

mcp-server/src/meshtastic_mcp/pio.py

@@ -92,6 +92,7 @@ def _run_capturing(
     cwd: Path | None = None,
     timeout: float | None = None,
     tee_header: str | None = None,
+    extra_env: dict[str, str] | None = None,
 ) -> tuple[int, str, str, float]:
     """Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
 
@@ -99,6 +100,9 @@ def _run_capturing(
     `subprocess.TimeoutExpired` on timeout (callers map this to their own
     domain-specific error).
 
+    `extra_env` merges into the subprocess environment (parent env stays
+    intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
+
     Fast path: `subprocess.run(capture_output=True)` when no flash log is
     configured (unchanged behavior).
 
@@ -110,6 +114,9 @@ def _run_capturing(
     """
     log_path = _flash_log_path()
     t0 = time.monotonic()
+    env = None
+    if extra_env:
+        env = {**os.environ, **extra_env}
 
     if log_path is None:
         # Fast path — unchanged.
@@ -119,6 +126,7 @@ def _run_capturing(
             capture_output=True,
             text=True,
             timeout=timeout,
+            env=env,
         )
         return (
             proc.returncode,
@@ -145,6 +153,7 @@ def _run_capturing(
         stderr=subprocess.PIPE,
         text=True,
         bufsize=1,  # line-buffered
+        env=env,
     )
     stdout_chunks: list[str] = []
     stderr_chunks: list[str] = []
@@ -232,12 +241,17 @@ def run(
     cwd: Path | None = None,
     timeout: float | None = TIMEOUT_DEFAULT,
     check: bool = True,
+    extra_env: dict[str, str] | None = None,
 ) -> PioResult:
     """Invoke `pio <args>` and return captured output.
 
     `cwd` defaults to the firmware root. `check=True` raises `PioError` on
     non-zero exit; set `check=False` to inspect `returncode` manually.
 
+    `extra_env` merges into the subprocess environment — used for
+    `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
+    toggles that can't be expressed as command-line args.
+
     If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
     line-by-line as it arrives (for live flash progress in the TUI).
     """
@@ -250,6 +264,7 @@ def run(
             cwd=work_dir,
             timeout=timeout,
             tee_header=f"pio {' '.join(args)}",
+            extra_env=extra_env,
         )
     except subprocess.TimeoutExpired as exc:
         raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc

mcp-server/src/meshtastic_mcp/recorder/__init__.py

@@ -0,0 +1,19 @@
+"""Persistent device-log capture.
+
+Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
+(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
+and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
+process-global so a single subscription captures every active interface
+(serial / TCP / BLE) without any per-connection bookkeeping.
+
+The recorder is opt-in-by-import: importing this package is a no-op; call
+`get_recorder().start()` (which `server.py` does at FastMCP app init) to
+begin writing. `pause()` / `resume()` exist for the rare case the user
+wants a clean stretch of file (e.g. capturing a known-good baseline).
+"""
+
+from __future__ import annotations
+
+from .recorder import Recorder, get_recorder
+
+__all__ = ["Recorder", "get_recorder"]

mcp-server/src/meshtastic_mcp/recorder/parsers.py

@@ -0,0 +1,309 @@
+"""Best-effort parsers for log lines and telemetry packets.
+
+Two flavors of log line cross our pubsub subscription:
+  1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
+     accumulates bytes between protobuf frames and emits the full
+     firmware-formatted line, e.g.
+         "INFO  | 12:34:56 12345 [Main] Booting"
+     — level, HH:MM:SS, uptime seconds, thread bracket, then message.
+  2. LogRecord protobuf path (debug_log_api enabled): the lib calls
+     `_handleLogLine(record.message)` with ONLY the message body. The
+     level/source/time fields on the LogRecord are dropped before
+     pubsub fan-out. We get e.g. just "Booting".
+
+Both arrive on `meshtastic.log.line`. The parser tries to recover a
+level + thread when the prefix is present and falls back to level=None
+otherwise. Consumers who want level filtering on protobuf-mode hosts
+should grep the raw `line` field instead.
+
+Telemetry: `meshtastic.receive.telemetry` packets carry one of several
+metric variants in `packet["decoded"]["telemetry"]`. We flatten the
+chosen variant into a {field: value} dict so callers don't have to
+know the protobuf shape.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
+# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
+# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
+# macros must stay in sync with these strings.
+_LINE_RE = re.compile(
+    r"""
+    ^
+    (?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
+    \s*\|\s*
+    (?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
+    \s+
+    (?P<uptime>\d+)
+    \s+
+    (?:\[(?P<thread>[^\]]+)\]\s+)?
+    (?P<msg>.*)
+    $
+    """,
+    re.VERBOSE,
+)
+
+# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
+# thread bracket. See src/RedirectablePrint.cpp:175.
+_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
+
+# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
+# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
+#         "++++++ Thread NAME freed heap A -> B (delta) ++++++"
+_THREAD_HEAP_RE = re.compile(
+    r"""
+    ^[\-+]+\s*
+    Thread\s+(?P<thread>\S+)\s+
+    (?P<kind>leaked|freed)\s+heap\s+
+    (?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
+    \((?P<delta>-?\d+)\)
+    """,
+    re.VERBOSE,
+)
+
+# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
+# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
+_HEAP_STATUS_RE = re.compile(
+    r"""
+    Heap\s+status:\s+
+    (?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
+    (?:\s+\((?P<delta>-?\d+)\))?
+    """,
+    re.VERBOSE,
+)
+
+
+_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
+_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
+
+
+def parse_log_line(line: str) -> dict[str, Any]:
+    """Best-effort decompose a raw firmware log line.
+
+    Returns a dict with at least `line` (the original, unmodified — ANSI
+    codes preserved for fidelity). Adds `level`, `tag`, `clock`,
+    `uptime_s`, and `msg` when the full prefix is present.
+
+    Handles two firmware quirks:
+      - LogRecord.message can carry ANSI color escapes from RedirectablePrint
+        (the BLE/StreamAPI path inherited the colored body in some builds).
+        We strip ANSI before regex matching so the prefix survives.
+      - DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
+        thread name is set, the heap takes the thread bracket position —
+        looks like `[heap 12345] msg`. We detect that shape and move it
+        out of `tag` and into `heap_free`.
+
+    DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
+    (bytes), and when a `Thread X leaked|freed heap` line is recognized,
+    `heap_event` = {kind, thread, before, after, delta}.
+
+    Never raises.
+    """
+    out: dict[str, Any] = {"line": line}
+    if not line:
+        return out
+
+    # Strip ANSI escapes BEFORE any regex matching. The original `line`
+    # stays in `out["line"]` for fidelity / future grep.
+    clean = _ANSI_RE.sub("", line)
+
+    m = _LINE_RE.match(clean)
+    msg: str | None = None
+    if m:
+        level = m.group("level").rstrip()
+        out["level"] = level
+        out["clock"] = m.group("clock")
+        try:
+            out["uptime_s"] = int(m.group("uptime"))
+        except (TypeError, ValueError):
+            out["uptime_s"] = None
+        thread = m.group("thread")
+        if thread:
+            # If "thread" is actually the heap prefix taking the bracket
+            # position (DEBUG_HEAP build, no thread set), capture heap
+            # and leave tag unset.
+            hb = _HEAP_BRACKET_RE.match(thread.strip())
+            if hb:
+                try:
+                    out["heap_free"] = int(hb.group("heap"))
+                except (TypeError, ValueError):
+                    pass
+            else:
+                out["tag"] = thread
+        msg = m.group("msg")
+        out["msg"] = msg
+    else:
+        # No prefix — bare LogRecord.message body. Inspect the whole
+        # line for DEBUG_HEAP-style content; the heap-prefix and
+        # thread-leak patterns can survive on either path.
+        msg = clean
+
+    # DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
+    # Sits AFTER the thread bracket and BEFORE the message body, but
+    # for bare LogRecord lines it's at the start. Match it at the
+    # head of `msg`.
+    if msg:
+        hp = _HEAP_PREFIX_RE.match(msg)
+        if hp:
+            try:
+                out["heap_free"] = int(hp.group("heap"))
+            except (TypeError, ValueError):
+                pass
+            else:
+                # Strip the prefix from `msg` so a grep on the message
+                # body doesn't have to know about it.
+                out["msg"] = hp.group("rest")
+                msg = hp.group("rest")
+
+        # Thread-level leak/free detection.
+        thr = _THREAD_HEAP_RE.search(msg)
+        if thr:
+            try:
+                out["heap_event"] = {
+                    "kind": thr.group("kind"),
+                    "thread": thr.group("thread"),
+                    "before": int(thr.group("before")),
+                    "after": int(thr.group("after")),
+                    "delta": int(thr.group("delta")),
+                }
+            except (TypeError, ValueError):
+                pass
+
+        # Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
+        hs = _HEAP_STATUS_RE.search(msg)
+        if hs:
+            try:
+                out["heap_free"] = int(hs.group("free"))
+                out["heap_total"] = int(hs.group("total"))
+                if hs.group("delta") is not None:
+                    out["heap_delta"] = int(hs.group("delta"))
+            except (TypeError, ValueError):
+                pass
+
+    return out
+
+
+# -- Telemetry ----------------------------------------------------------
+
+# Order matters: meshtastic-python decoded packets use the protobuf
+# `oneof variant` field name (snake_case) as the dict key.
+_TELEMETRY_VARIANTS = (
+    ("device_metrics", "device"),
+    ("local_stats", "local"),
+    ("environment_metrics", "environment"),
+    ("power_metrics", "power"),
+    ("air_quality_metrics", "airQuality"),
+    ("health_metrics", "health"),
+    ("host_metrics", "host"),
+)
+
+
+def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
+    """Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
+    packet. Returns None when the shape isn't what we expect — so the
+    caller can fall back to a generic packets.jsonl row.
+    """
+    if not isinstance(packet, dict):
+        return None
+    decoded = packet.get("decoded")
+    if not isinstance(decoded, dict):
+        return None
+    telem = decoded.get("telemetry")
+    if not isinstance(telem, dict):
+        return None
+    # The Python lib produces dict-of-camelCase keys via MessageToDict.
+    # Try both camelCase and snake_case to be robust to lib version drift.
+    for snake, label in _TELEMETRY_VARIANTS:
+        camel = _snake_to_camel(snake)
+        for key in (snake, camel):
+            value = telem.get(key)
+            if isinstance(value, dict):
+                return {
+                    "variant": label,
+                    "fields": {k: _scalarize(v) for k, v in value.items()},
+                    "time": telem.get("time"),
+                }
+    return None
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _scalarize(value: Any) -> Any:
+    """Keep telemetry fields JSON-friendly. Lists/dicts pass through
+    untouched; bytes -> hex string; protobuf enums occasionally arrive
+    as ints (fine) or strings (also fine)."""
+    if isinstance(value, (bytes, bytearray, memoryview)):
+        return bytes(value).hex()
+    return value
+
+
+# -- Generic packet summary ---------------------------------------------
+
+
+def summarize_packet(
+    packet: dict[str, Any], *, payload_hex_len: int = 64
+) -> dict[str, Any]:
+    """Reduce a packet dict to a stable, queryable summary. Drops the
+    full payload bytes — the recorder records summaries, not pcaps.
+    """
+    if not isinstance(packet, dict):
+        return {"raw_type": type(packet).__name__}
+    decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
+    portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
+    payload = decoded.get("payload") if isinstance(decoded, dict) else None
+    payload_hex = None
+    payload_size = None
+    if isinstance(payload, (bytes, bytearray, memoryview)):
+        b = bytes(payload)
+        payload_size = len(b)
+        payload_hex = b[:payload_hex_len].hex() if b else ""
+    elif isinstance(payload, str):
+        # Some decoded payloads (text messages) come as decoded strings.
+        payload_size = len(payload)
+        payload_hex = None  # not bytes
+    return {
+        "from_node": packet.get("fromId") or packet.get("from"),
+        "to_node": packet.get("toId") or packet.get("to"),
+        "portnum": portnum,
+        "hop_limit": packet.get("hopLimit"),
+        "want_ack": packet.get("wantAck"),
+        "rx_rssi": packet.get("rxRssi"),
+        "rx_snr": packet.get("rxSnr"),
+        "channel": packet.get("channel"),
+        "id": packet.get("id"),
+        "payload_size": payload_size,
+        "payload_hex_prefix": payload_hex,
+    }
+
+
+# -- Interface identification ------------------------------------------
+
+
+def interface_label(interface: Any) -> dict[str, Any]:
+    """Stable identifier for the meshtastic interface that emitted an event.
+
+    Used as the `port`/`role` tag on every recorded row. SerialInterface
+    has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
+    has `address`. Falls back to the class name when none of those exist.
+    """
+    if interface is None:
+        return {"port": None, "role": None}
+    dev_path = getattr(interface, "devPath", None)
+    if dev_path:
+        return {"port": str(dev_path), "role": "serial"}
+    hostname = getattr(interface, "hostname", None)
+    if hostname:
+        port_num = getattr(interface, "portNumber", None)
+        endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
+        return {"port": endpoint, "role": "tcp"}
+    address = getattr(interface, "address", None)
+    if address:
+        return {"port": str(address), "role": "ble"}
+    return {"port": type(interface).__name__, "role": None}

mcp-server/src/meshtastic_mcp/recorder/recorder.py

@@ -0,0 +1,467 @@
+"""Process-global recorder singleton.
+
+Subscribes once to the meshtastic pubsub fan-out and writes four append-only
+JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
+process-global — a single subscription captures every active interface
+without per-connection bookkeeping.
+
+Files:
+  logs.jsonl      — every `meshtastic.log.line` event (best-effort prefix
+                    parsed for level/tag/uptime; raw `line` always preserved)
+  telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
+                    variant (device / local / environment / power / etc.)
+  packets.jsonl   — every other `meshtastic.receive.*` packet, summarized
+                    (portnum, hops, RSSI/SNR, payload size + 64-byte hex)
+  events.jsonl    — connection lifecycle, node-DB updates, and manual
+                    `mark_event` rows. Lower volume; useful for aligning
+                    timelines.
+
+Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
+write methods short-circuit when paused, so we don't lose ordering when
+resumed (we just have a gap). No queueing.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+from pathlib import Path
+from typing import Any
+
+from . import parsers
+from .rotating import _RotatingJsonl
+
+_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
+log = logging.getLogger(__name__)
+
+
+class Recorder:
+    """Singleton write-side of the persistent log capture system."""
+
+    def __init__(self, base_dir: Path | None = None) -> None:
+        self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
+        self._lock = threading.RLock()
+        self._started = False
+        self._paused = False
+        self._pause_reason: str | None = None
+        self._started_at: float | None = None
+        self._handlers: list[tuple[str, Any]] = []
+        self._files: dict[str, _RotatingJsonl] = {}
+
+    # -- lifecycle ----------------------------------------------------
+
+    def start(self) -> None:
+        """Idempotent. Safe to call from FastMCP app startup."""
+        with self._lock:
+            if self._started:
+                return
+            self.base_dir.mkdir(parents=True, exist_ok=True)
+            self._files = {
+                "logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
+                "telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
+                "packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
+                "events": _RotatingJsonl(self.base_dir / "events.jsonl"),
+            }
+            self._wire_pubsub()
+            self._started = True
+            self._started_at = time.time()
+        # Write the recorder_start marker after the initialization block.
+        # `_write_event()` re-checks recorder state via `_files_snapshot()`,
+        # so keeping this out of the setup block avoids nested lifecycle work.
+        self._write_event(kind="recorder_start", label="recorder_started")
+
+    def stop(self) -> None:
+        with self._lock:
+            if not self._started:
+                return
+            self._unwire_pubsub()
+            for f in self._files.values():
+                f.close()
+            self._files = {}
+            self._started = False
+
+    def pause(self, reason: str | None = None) -> None:
+        # Write the pause marker BEFORE flipping the flag — `_write_event`
+        # short-circuits when paused, so the order matters for this event
+        # to actually land in events.jsonl.
+        self._write_event(
+            kind="recorder_pause",
+            label="paused",
+            note=reason,
+        )
+        with self._lock:
+            self._paused = True
+            self._pause_reason = reason
+
+    def resume(self) -> None:
+        # Mirror of `pause()`: clear the flag first, then write the marker
+        # so it isn't suppressed by the still-paused short-circuit.
+        with self._lock:
+            self._paused = False
+            self._pause_reason = None
+        self._write_event(kind="recorder_resume", label="resumed")
+
+    # -- pubsub wiring ------------------------------------------------
+
+    def _wire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        # Subscribers — one per topic. Each pubsub publisher sends
+        # keyword args matching its handler's signature; pubsub
+        # introspects the function signature to route args.
+        bindings = [
+            ("meshtastic.log.line", self._on_log_line),
+            ("meshtastic.serial.line", self._on_serial_line),
+            ("meshtastic.receive", self._on_receive),
+            ("meshtastic.receive.telemetry", self._on_telemetry),
+            ("meshtastic.connection.established", self._on_connection_established),
+            ("meshtastic.connection.lost", self._on_connection_lost),
+            ("meshtastic.node.updated", self._on_node_updated),
+        ]
+        for topic, handler in bindings:
+            try:
+                pub.subscribe(handler, topic)
+                self._handlers.append((topic, handler))
+            except Exception as exc:
+                # If pubsub refuses one binding (signature mismatch on
+                # an old lib version), log it and keep the rest.
+                log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
+
+    def _unwire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        for topic, handler in self._handlers:
+            try:
+                pub.unsubscribe(handler, topic)
+            except Exception:
+                pass
+        self._handlers.clear()
+
+    # -- handlers -----------------------------------------------------
+    #
+    # Pubsub callbacks must never raise. Every handler is wrapped in a
+    # try/except that swallows so a bug here can't take down the
+    # SerialInterface receive thread.
+    #
+    # Threading: handlers fire on whatever thread the meshtastic library
+    # dispatches from (varies by interface), while `stop()` clears
+    # `self._files` under `self._lock`. We snapshot `_files` under the
+    # lock at the top of each handler so a concurrent stop can't
+    # KeyError us mid-write. The actual file write goes through
+    # `_RotatingJsonl` which has its own lock.
+
+    def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
+        """Atomic-ish view of `self._files`. Returns None when the recorder
+        is paused or stopped, so handlers can early-exit cleanly without
+        racing `stop()`'s clear."""
+        with self._lock:
+            if not self._started or self._paused:
+                return None
+            return dict(self._files)
+
+    def _on_log_line(self, line: str, interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": tags["port"],
+                "role": tags["role"],
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            # DEBUG_HEAP enrichments (only present when the firmware
+            # was built with -DDEBUG_HEAP=1). Surface as first-class
+            # fields so logs_window can grep/filter on them and so
+            # heap_free synthesizes a telemetry point below.
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # If the line carried a heap snapshot, also write it as a
+            # synthesized LocalStats-shaped row so telemetry_timeline
+            # picks it up at log cadence (much higher resolution than
+            # the ~60 s LocalStats packet). Tagged source=debug_heap so
+            # consumers can filter if mixing scales is unwanted.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": tags["port"],
+                        "role": tags["role"],
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_serial_line(self, line: str, port: str | None = None) -> None:
+        """Text-mode passive tap. Fired from `serial_session._drain` when a
+        `pio device monitor` subprocess is running.
+
+        Same parse + heap-synthesis path as `_on_log_line`, but receives
+        the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
+        body). On DEBUG_HEAP builds in text mode this gives us per-log-line
+        heap data — far higher cadence than LocalStats, and works without
+        protobuf API mode (no SerialInterface required).
+        """
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": port,
+                "role": "serial_session",
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # Synthesize a heap_free telemetry sample whenever the line
+            # carries one — same logic as _on_log_line, tagged source so
+            # consumers can distinguish text-mode tap from protobuf path.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": port,
+                        "role": "serial_session",
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap_serial",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            extracted = parsers.extract_telemetry(packet)
+            if extracted is None:
+                # Couldn't extract a known variant — fall through to the
+                # generic `_on_receive` path, which will still fire for
+                # this packet via the parent topic.
+                return
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                "from_node": packet.get("fromId") or packet.get("from"),
+                "variant": extracted["variant"],
+                "fields": extracted["fields"],
+                "device_time": extracted.get("time"),
+            }
+            files["telemetry"].write(record)
+        except Exception:
+            pass
+
+    def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
+        # Generic-receive fires for EVERY packet. Telemetry packets get
+        # recorded twice (here and in _on_telemetry) — that's intentional:
+        # packets.jsonl is the universal record, telemetry.jsonl is the
+        # structured timeseries view.
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            summary = parsers.summarize_packet(packet)
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                **summary,
+            }
+            files["packets"].write(record)
+        except Exception:
+            pass
+
+    def _on_connection_established(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_established",
+            interface=interface,
+        )
+
+    def _on_connection_lost(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_lost",
+            interface=interface,
+        )
+
+    def _on_node_updated(
+        self, node: dict[str, Any] | None = None, interface: Any = None
+    ) -> None:
+        # Lower-volume than packets but informative — node ID, hops away,
+        # last heard. Skip the user dict if absent.
+        try:
+            user = (node or {}).get("user") if isinstance(node, dict) else None
+            self._write_event(
+                kind="node_updated",
+                interface=interface,
+                data={
+                    "num": (node or {}).get("num"),
+                    "id": (user or {}).get("id"),
+                    "short": (user or {}).get("shortName"),
+                    "long": (user or {}).get("longName"),
+                    "hops_away": (node or {}).get("hopsAway"),
+                    "snr": (node or {}).get("snr"),
+                    "last_heard": (node or {}).get("lastHeard"),
+                },
+            )
+        except Exception:
+            pass
+
+    # -- public write helpers -----------------------------------------
+
+    def mark_event(
+        self,
+        label: str,
+        note: str | None = None,
+        data: dict[str, Any] | None = None,
+    ) -> dict[str, Any]:
+        """User-facing marker. Writes to events.jsonl AND emits a
+        synthetic logs.jsonl row tagged level=MARK so timelines align.
+        """
+        ts = self._write_event(kind="mark", label=label, note=note, data=data)
+        # Mirror into logs so a single logs_window grep finds it.
+        files = self._files_snapshot()
+        if files is not None:
+            try:
+                files["logs"].write(
+                    {
+                        "ts": ts,
+                        "port": None,
+                        "role": "marker",
+                        "level": "MARK",
+                        "tag": "mark_event",
+                        "line": f"[mark] {label}" + (f" — {note}" if note else ""),
+                    }
+                )
+            except Exception:
+                pass
+        return {"ts": ts, "label": label}
+
+    def _write_event(
+        self,
+        *,
+        kind: str,
+        label: str | None = None,
+        note: str | None = None,
+        interface: Any = None,
+        data: dict[str, Any] | None = None,
+    ) -> float:
+        ts = time.time()
+        # Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
+        # arrive at choreographed moments — `pause()` writes BEFORE flipping
+        # the flag and `resume()` writes AFTER clearing it, so those calls
+        # see _paused=False here. Other event kinds short-circuit when
+        # paused via the snapshot guard below.
+        files = self._files_snapshot()
+        if files is None:
+            return ts
+        try:
+            tags = parsers.interface_label(interface)
+            files["events"].write(
+                {
+                    "ts": ts,
+                    "kind": kind,
+                    "label": label,
+                    "note": note,
+                    "port": tags["port"],
+                    "role": tags["role"],
+                    "data": data,
+                }
+            )
+        except Exception:
+            pass
+        return ts
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            return {
+                "running": self._started,
+                "paused": self._paused,
+                "pause_reason": self._pause_reason,
+                "started_at": self._started_at,
+                "base_dir": str(self.base_dir),
+                "files": {name: f.status() for name, f in self._files.items()},
+            }
+
+    def force_rotate_all(self) -> dict[str, Any]:
+        """Test/admin hook: rotate every stream right now."""
+        with self._lock:
+            files = list(self._files.values())
+        for f in files:
+            f.force_rotate()
+        # `status()` re-acquires `self._lock`; release before calling it.
+        return self.status()
+
+
+# -- module-level singleton accessor ------------------------------------
+
+_INSTANCE_LOCK = threading.Lock()
+_INSTANCE: Recorder | None = None
+
+
+def get_recorder() -> Recorder:
+    """Return the process-global Recorder. Created on first call.
+
+    Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
+    (used by tests to redirect to a tmpdir).
+    """
+    global _INSTANCE
+    with _INSTANCE_LOCK:
+        if _INSTANCE is None:
+            override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
+            base = Path(override) if override else None
+            _INSTANCE = Recorder(base_dir=base)
+        return _INSTANCE

mcp-server/src/meshtastic_mcp/recorder/rotating.py

@@ -0,0 +1,163 @@
+"""Append-only JSONL writer with size-capped rotation.
+
+A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
+JSON objects (one row per call). When the live file exceeds `max_bytes`,
+it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
+and the live file resets to empty. Old archives past `keep_archives` are
+unlinked oldest-first.
+
+Size check is amortized — `os.fstat` runs every `check_every` writes,
+not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
+
+Threading: every public method acquires `self._lock`. The recorder runs
+several pubsub handlers on whatever thread the meshtastic library
+dispatches from (varies by interface), and queries from MCP tool calls
+arrive on the FastMCP request thread, so this lock is not optional.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import os
+import shutil
+import threading
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+class _RotatingJsonl:
+    """Append-only JSONL with size rotation. Thread-safe."""
+
+    def __init__(
+        self,
+        path: Path,
+        *,
+        max_bytes: int = 100 * 1024 * 1024,
+        keep_archives: int = 5,
+        check_every: int = 1000,
+    ) -> None:
+        self.path = path
+        self.max_bytes = max_bytes
+        self.keep_archives = keep_archives
+        self.check_every = check_every
+        self._lock = threading.Lock()
+        self._fh: Any = None
+        self._writes_since_check = 0
+        self._rotations = 0
+        self._lines_written = 0
+        self._last_ts: float | None = None
+        self._open()
+
+    # -- lifecycle ----------------------------------------------------
+
+    def _open(self) -> None:
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        self._fh = self.path.open("a", encoding="utf-8")
+
+    def close(self) -> None:
+        with self._lock:
+            if self._fh is not None:
+                try:
+                    self._fh.close()
+                finally:
+                    self._fh = None
+
+    # -- write --------------------------------------------------------
+
+    def write(self, record: dict[str, Any]) -> None:
+        """Append one JSON object as a line. Triggers rotation if oversized."""
+        line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
+        with self._lock:
+            if self._fh is None:
+                return
+            try:
+                self._fh.write(line)
+                self._fh.flush()
+            except Exception:
+                # Best-effort: a failed write must not crash the pubsub
+                # handler. Caller has no way to react anyway.
+                return
+            self._lines_written += 1
+            ts = record.get("ts")
+            if isinstance(ts, (int, float)):
+                self._last_ts = float(ts)
+            self._writes_since_check += 1
+            if self._writes_since_check >= self.check_every:
+                self._writes_since_check = 0
+                self._maybe_rotate()
+
+    # -- rotation -----------------------------------------------------
+
+    def _maybe_rotate(self) -> None:
+        # Caller holds self._lock.
+        try:
+            size = os.fstat(self._fh.fileno()).st_size
+        except OSError:
+            return
+        if size < self.max_bytes:
+            return
+        self._rotate_locked()
+
+    def _rotate_locked(self) -> None:
+        # Close, gzip-rename, reopen empty, prune oldest archives.
+        try:
+            self._fh.close()
+        except Exception:
+            pass
+        self._fh = None
+        # Microsecond-resolution timestamp + per-instance counter so back-
+        # to-back rotations (small max_bytes, repeated `force_rotate()`,
+        # or chatty test loops) get unique archive filenames. The lex
+        # sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
+        # which `_prune_archives()` and `log_query._iter_jsonl()` both
+        # rely on.
+        stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
+        archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
+        try:
+            with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
+                shutil.copyfileobj(src, dst, length=1024 * 1024)
+            self.path.unlink()
+        except Exception:
+            # Rotation is best-effort. If gzip fails, leave the file
+            # in place and re-open it; we'll try again next check.
+            pass
+        self._open()
+        self._rotations += 1
+        self._prune_archives()
+
+    def _prune_archives(self) -> None:
+        # Match siblings of self.path.name with `.jsonl.gz` suffix.
+        prefix = self.path.stem  # "logs" for "logs.jsonl"
+        # Archive filenames are already lexicographically chronological.
+        # Prune by name, not mtime, so copied/restored files don't reorder.
+        archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
+        excess = len(archives) - self.keep_archives
+        for old in archives[: max(0, excess)]:
+            try:
+                old.unlink()
+            except OSError:
+                pass
+
+    def force_rotate(self) -> None:
+        """Test/admin hook: rotate immediately regardless of size."""
+        with self._lock:
+            if self._fh is not None:
+                self._rotate_locked()
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            try:
+                size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
+            except OSError:
+                size = 0
+            return {
+                "path": str(self.path),
+                "size": size,
+                "lines": self._lines_written,
+                "last_ts": self._last_ts,
+                "rotations": self._rotations,
+            }

mcp-server/src/meshtastic_mcp/serial_session.py

@@ -46,7 +46,23 @@ class SerialSession:
 
 
 def _drain(session: SerialSession) -> None:
-    """Reader thread: line-by-line pull stdout into buffer."""
+    """Reader thread: line-by-line pull stdout into buffer.
+
+    Each line is also published to the `meshtastic.serial.line` pubsub
+    topic so the persistent recorder can capture it without holding its
+    own port. This is the text-mode tap path: when no SerialInterface is
+    open, the firmware emits full formatted lines (level + clock + uptime
+    + thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
+    fan them out to whoever is listening. Pubsub is best-effort —
+    publish failures must never block the reader.
+    """
+    # Lazy import: pubsub isn't required just to import this module
+    # (e.g., during static analysis), and we want a clean test surface.
+    try:
+        from pubsub import pub  # type: ignore[import-untyped]
+    except Exception:  # pragma: no cover - defensive
+        pub = None
+
     assert session.proc.stdout is not None
     try:
         for line in session.proc.stdout:
@@ -54,6 +70,16 @@ def _drain(session: SerialSession) -> None:
             with session.lock:
                 session.buffer.append(line_stripped)
                 session.total_lines += 1
+            if pub is not None:
+                try:
+                    pub.sendMessage(
+                        "meshtastic.serial.line",
+                        line=line_stripped,
+                        port=session.port,
+                    )
+                except Exception:
+                    # A subscriber raising must not break the reader.
+                    pass
     except Exception:  # pragma: no cover - defensive
         pass
     finally:
@@ -71,6 +97,10 @@ def open_session(
     If `env` is supplied, pio resolves baud and filters from platformio.ini.
     Otherwise uses the supplied `baud` and `filters` (default `['direct']`).
     """
+    # Lazy import to avoid circular: registry imports serial_session.
+    from . import connection
+
+    connection.reject_if_tcp(port, "serial_open")
     args = ["device", "monitor", "--port", port, "--no-reconnect"]
     effective_filters: list[str]
     effective_baud: int = baud

mcp-server/src/meshtastic_mcp/server.py

@@ -6,6 +6,7 @@ etc.). Business logic does not live here.
 
 from __future__ import annotations
 
+import logging
 from typing import Any
 
 from mcp.server.fastmcp import FastMCP
@@ -14,17 +15,38 @@ from . import (
     admin,
     boards,
     devices,
+    fixtures,
     flash,
     hw_tools,
     info,
+    log_query,
     registry,
     serial_session,
 )
 from . import userprefs as userprefs_mod
+from .recorder import get_recorder
+
+log = logging.getLogger(__name__)
 
 app = FastMCP("meshtastic-mcp")
 
 
+def _start_recorder() -> None:
+    # Persistent device-log capture. Starts on first import — pubsub fan-out
+    # is process-global, so subscribing here captures every active interface
+    # (whether opened by an MCP tool, a pytest fixture, or a serial_session).
+    # Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
+    # for the full design. Recorder startup is best-effort: an unwritable
+    # log dir or pubsub mismatch should not take the MCP server down.
+    try:
+        get_recorder().start()
+    except Exception as exc:
+        log.warning("Failed to start persistent recorder: %s", exc)
+
+
+_start_recorder()
+
+
 # ---------- Discovery & metadata ------------------------------------------
 
 
@@ -75,6 +97,7 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Build firmware for one env via `pio run -e <env>`.
 
@@ -86,8 +109,21 @@ def build(
     build via userPrefs.jsonc injection. The file is restored after the build
     completes. Use `userprefs_manifest` to discover available keys. Use
     `userprefs_set` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
+    only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
+    `build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
+    `[heap N]` prefix on every log line. The recorder picks the prefix up
+    automatically and synthesizes a high-resolution heap timeline that
+    `telemetry_timeline(field="free_heap")` can read alongside the normal
+    ~60 s LocalStats packets. Pair with `/leakhunt` for classification.
     """
-    return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
+    return flash.build(
+        env,
+        with_manifest=with_manifest,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -105,6 +141,7 @@ def pio_flash(
     port: str,
     confirm: bool = False,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
 
@@ -114,8 +151,19 @@ def pio_flash(
 
     `userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
     build via userPrefs.jsonc injection; restored after upload.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
+    rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
+    to actually land in the uploaded firmware — without it, the implicit
+    rebuild relinks without the env var and silently drops them.
     """
-    return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
+    return flash.flash(
+        env,
+        port,
+        confirm=confirm,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -734,3 +782,227 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
 def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
     """Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
     return hw_tools.picotool_raw(args, confirm=confirm)
+
+
+# ---------- Persistent device-log capture (recorder) ----------------------
+#
+# The recorder is autouse — it starts at server import and continuously
+# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
+# tools are query-only over those files, plus a few lifecycle controls.
+
+
+@app.tool()
+def logs_window(
+    start: str = "-15m",
+    end: str = "now",
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines from the persistent recorder.
+
+    Filters by time window, regex over the line, level (single or
+    pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
+    interface port. Returns up to max_lines most-recent matches.
+
+    Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
+
+    Note: lines arriving via the LogRecord protobuf path (when
+    set_debug_log_api(True) is on) come without level prefix — the
+    meshtastic Python lib drops record.level before fan-out. For those,
+    `level` filter won't match; use `grep` instead.
+    """
+    return log_query.logs_window(
+        start=start,
+        end=end,
+        grep=grep,
+        level=level,
+        tag=tag,
+        port=port,
+        max_lines=max_lines,
+    )
+
+
+@app.tool()
+def telemetry_timeline(
+    window: str = "1h",
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Time series of one telemetry field, downsampled to <= max_points.
+
+    `variant` ∈ device, local, environment, power, airQuality, health, host.
+    `field` accepts snake_case or camelCase; common aliases (free_heap ↔
+    heap_free_bytes) are normalized.
+
+    Returns slope_per_min (linear-regression slope, units/minute) so a
+    leak detector can read one number — negative slope on free_heap over
+    a long window indicates a real leak.
+
+    LocalStats variant ("local") cadence is ~60 s (whatever the device's
+    `device_update_interval` is set to), so a 1 h window gives ~60 raw
+    points. Bucket-mean downsampling preserves shape.
+    """
+    return log_query.telemetry_timeline(
+        window=window,
+        variant=variant,
+        field=field,
+        port=port,
+        max_points=max_points,
+    )
+
+
+@app.tool()
+def packets_window(
+    start: str = "-5m",
+    end: str = "now",
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Recent mesh packets recorded by the recorder.
+
+    Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
+    size + first 64 bytes hex) — full payload bytes are not stored.
+    `portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
+    """
+    return log_query.packets_window(
+        start=start,
+        end=end,
+        portnum=portnum,
+        from_node=from_node,
+        to_node=to_node,
+        max=max,
+    )
+
+
+@app.tool()
+def events_window(
+    start: str = "-1h",
+    end: str = "now",
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
+
+    `kind` ∈ recorder_start, recorder_pause, recorder_resume,
+    connection_established, connection_lost, node_updated, mark.
+    Pipe-separated sets ("connection_lost|connection_established") work.
+    """
+    return log_query.events_window(start=start, end=end, kind=kind, max=max)
+
+
+@app.tool()
+def mark_event(
+    label: str,
+    note: str | None = None,
+    data: dict[str, Any] | None = None,
+) -> dict[str, Any]:
+    """Drop a named marker into events.jsonl AND logs.jsonl.
+
+    Useful for aligning a timeline around a known stimulus: call before
+    and after a stress workload, then query telemetry_timeline /
+    logs_window with the markers' timestamps as bounds.
+
+    The marker also lands in logs.jsonl with level=MARK so a single
+    grep over logs picks it up.
+    """
+    return get_recorder().mark_event(label=label, note=note, data=data)
+
+
+@app.tool()
+def recorder_status() -> dict[str, Any]:
+    """Return recorder runtime info: running, paused, file sizes, last_ts per stream.
+
+    Use this to sanity-check that capture is working before you trust a
+    `logs_window` / `telemetry_timeline` result.
+    """
+    return get_recorder().status()
+
+
+@app.tool()
+def recorder_pause(reason: str | None = None) -> dict[str, Any]:
+    """Pause writes to all four streams. Pubsub subscriptions stay active —
+    we just drop events on the floor while paused. Resume with `recorder_resume`.
+
+    Use when capturing a known-good baseline that you don't want to
+    pollute with pre-test noise. Default state is recording; this is
+    rarely needed.
+    """
+    get_recorder().pause(reason=reason)
+    return {"ok": True, "paused": True, "reason": reason}
+
+
+@app.tool()
+def recorder_resume() -> dict[str, Any]:
+    """Resume writes after `recorder_pause`. No-op if already running."""
+    get_recorder().resume()
+    return {"ok": True, "paused": False}
+
+
+@app.tool()
+def recorder_export(
+    start: str,
+    end: str,
+    dest_dir: str,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of the recorder's streams into `dest_dir`.
+
+    Writes one uncompressed JSONL per requested stream (logs / telemetry /
+    packets / events). Useful for: attaching to a bug report, feeding a
+    notebook, or backfilling Datadog after the fact.
+    """
+    return log_query.export(
+        start=start,
+        end=end,
+        dest_dir=dest_dir,
+        streams=streams,
+    )
+
+
+# ---------- Fixture / test-data push --------------------------------------
+
+
+@app.tool()
+def push_fake_nodedb(
+    size: int,
+    target: str = "portduino",
+    port: str | None = None,
+    portduino_config: str = "default",
+    backup_existing: bool = True,
+    confirm: bool = False,
+    reboot_after: bool = True,
+    custom_seed_jsonl: str | None = None,
+) -> dict[str, Any]:
+    """Push a fake-NodeDB v25 fixture (250/500/1000/2000 nodes) onto a device.
+
+    Two transports:
+      target="portduino" — file copy to ~/.portduino/<portduino_config>/prefs/nodes.proto.
+                            Fast, no device connection needed.
+      target="hardware"  — XModem upload over serial/BLE to /prefs/nodes.proto.
+                            Requires `port` + `confirm=True`. Triggers a reboot
+                            so loadFromDisk picks up the new file at next boot.
+
+    Compiles a fresh-timestamp proto from the committed JSONL seed under
+    test/fixtures/nodedb/seed_v25_<N>.jsonl each invocation, so the loaded
+    NodeDB always looks "recent" to the connecting phone. Structural data
+    (names, IDs, positions, telemetries) is deterministic per the seed.
+
+    Override the JSONL via `custom_seed_jsonl` to push a hand-edited scenario.
+    """
+    return fixtures.push_fake_nodedb(
+        size=size,
+        target=target,  # type: ignore[arg-type]
+        port=port,
+        portduino_config=portduino_config,
+        backup_existing=backup_existing,
+        confirm=confirm,
+        reboot_after=reboot_after,
+        custom_seed_jsonl=custom_seed_jsonl,
+    )

mcp-server/tests/unit/test_build_flags.py

@@ -0,0 +1,88 @@
+"""Unit tests for the `build_flags` injection on `flash.build()`.
+
+We don't actually run pio here — too slow, requires hardware-aware envs.
+We test the translation layer (`_build_flags_env`) and that the env vars
+are threaded through pio.run correctly via mock.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import patch
+
+from meshtastic_mcp import flash, pio
+
+
+class TestBuildFlagsEnv:
+    def test_simple_value(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
+
+    def test_string_value(self) -> None:
+        out = flash._build_flags_env({"FOO": "bar"})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
+
+    def test_bool_true_is_bare_flag(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": True})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
+
+    def test_bool_false_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
+
+    def test_none_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": None})
+        assert out == {}
+
+    def test_multiple_combined(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
+        # Order isn't guaranteed in dict iteration, so check membership.
+        flags = out["PLATFORMIO_BUILD_FLAGS"].split()
+        assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
+
+
+class TestBuildPropagatesFlags:
+    def test_extra_env_passed_to_pio_run(self) -> None:
+        # Mock pio.run so we don't actually invoke pio. Capture extra_env.
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["args"] = args
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                out = flash.build(
+                    "fake-env",
+                    with_manifest=False,
+                    build_flags={"DEBUG_HEAP": 1},
+                )
+        assert captured["args"] == ["run", "-e", "fake-env"]
+        assert captured["kwargs"]["extra_env"] == {
+            "PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
+        }
+        assert out["build_flags"] == {"DEBUG_HEAP": 1}
+
+    def test_no_flags_means_no_extra_env(self) -> None:
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                flash.build("fake-env", with_manifest=False)
+        assert captured["kwargs"]["extra_env"] is None

mcp-server/tests/unit/test_connection_tcp.py

@@ -0,0 +1,383 @@
+"""TCP transport plumbing in connection.py + devices.py.
+
+Pure-Python tests — no real device or daemon required. Mocks `TCPInterface`
+when exercising `connect()`.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import patch
+
+import pytest
+from meshtastic_mcp import connection, devices
+
+# ---------- helpers --------------------------------------------------------
+
+
+class TestIsTcpPort:
+    def test_tcp_scheme(self) -> None:
+        assert connection.is_tcp_port("tcp://localhost") is True
+        assert connection.is_tcp_port("tcp://localhost:4403") is True
+        assert connection.is_tcp_port("tcp://192.168.1.50:9999") is True
+
+    def test_serial_paths(self) -> None:
+        assert connection.is_tcp_port("/dev/cu.usbmodem1234") is False
+        assert connection.is_tcp_port("/dev/ttyUSB0") is False
+        assert connection.is_tcp_port("COM3") is False
+
+    def test_empty_or_none(self) -> None:
+        assert connection.is_tcp_port(None) is False
+        assert connection.is_tcp_port("") is False
+
+
+class TestParseTcpPort:
+    def test_default_port(self) -> None:
+        assert connection.parse_tcp_port("tcp://localhost") == ("localhost", 4403)
+
+    def test_explicit_port(self) -> None:
+        assert connection.parse_tcp_port("tcp://localhost:9999") == (
+            "localhost",
+            9999,
+        )
+
+    def test_ip_with_port(self) -> None:
+        assert connection.parse_tcp_port("tcp://192.168.1.50:4403") == (
+            "192.168.1.50",
+            4403,
+        )
+
+
+class TestNormalizeTcpEndpoint:
+    def test_bare_host(self) -> None:
+        assert connection.normalize_tcp_endpoint("localhost") == "tcp://localhost:4403"
+
+    def test_host_port(self) -> None:
+        assert (
+            connection.normalize_tcp_endpoint("localhost:5000")
+            == "tcp://localhost:5000"
+        )
+
+    def test_full_url(self) -> None:
+        assert (
+            connection.normalize_tcp_endpoint("tcp://1.2.3.4") == "tcp://1.2.3.4:4403"
+        )
+        assert (
+            connection.normalize_tcp_endpoint("tcp://1.2.3.4:9999")
+            == "tcp://1.2.3.4:9999"
+        )
+
+    def test_idempotent(self) -> None:
+        once = connection.normalize_tcp_endpoint("localhost:4403")
+        twice = connection.normalize_tcp_endpoint(once)
+        assert once == twice == "tcp://localhost:4403"
+
+    def test_path_like_endpoint_rejected(self) -> None:
+        # Serial port paths and Windows drive paths are common config typos
+        # (someone passes a serial path to MESHTASTIC_MCP_TCP_HOST). Reject
+        # rather than producing a nonsense `tcp:///dev/cu.foo:4403` URL.
+        with pytest.raises(connection.ConnectionError, match="path separator"):
+            connection.normalize_tcp_endpoint("/dev/cu.foo")
+        with pytest.raises(connection.ConnectionError):
+            connection.normalize_tcp_endpoint("tcp:///dev/cu.foo:4403")
+        with pytest.raises(connection.ConnectionError):
+            connection.normalize_tcp_endpoint(r"C:\Windows\System32")
+
+    def test_non_integer_port_rejected(self) -> None:
+        with pytest.raises(connection.ConnectionError, match="not an integer"):
+            connection.normalize_tcp_endpoint("tcp://host:notaport")
+        with pytest.raises(connection.ConnectionError, match="not an integer"):
+            connection.normalize_tcp_endpoint("host:notaport")
+
+    def test_empty_host_rejected(self) -> None:
+        with pytest.raises(connection.ConnectionError, match="empty host"):
+            connection.normalize_tcp_endpoint("tcp://:4403")
+
+    def test_port_out_of_range_rejected(self) -> None:
+        with pytest.raises(connection.ConnectionError, match="out of range"):
+            connection.normalize_tcp_endpoint("tcp://host:0")
+        with pytest.raises(connection.ConnectionError, match="out of range"):
+            connection.normalize_tcp_endpoint("tcp://host:65536")
+        with pytest.raises(connection.ConnectionError, match="out of range"):
+            connection.normalize_tcp_endpoint("host:99999")
+
+
+class TestParseTcpPortValidation:
+    def test_missing_scheme_rejected(self) -> None:
+        # parse_tcp_port is a low-level helper that requires the scheme.
+        # Misuse should fail loudly rather than silently mis-parsing.
+        with pytest.raises(connection.ConnectionError, match="expected"):
+            connection.parse_tcp_port("localhost:4403")
+
+    def test_negative_port_rejected(self) -> None:
+        with pytest.raises(connection.ConnectionError, match="out of range"):
+            connection.parse_tcp_port("tcp://host:-1")
+
+
+# ---------- reject_if_tcp --------------------------------------------------
+
+
+class TestRejectIfTcp:
+    def test_rejects_tcp(self) -> None:
+        with pytest.raises(connection.ConnectionError, match="not applicable"):
+            connection.reject_if_tcp("tcp://localhost", "esptool_chip_info")
+
+    def test_passes_through_serial(self) -> None:
+        connection.reject_if_tcp("/dev/cu.usbmodem1", "esptool_chip_info")  # no raise
+
+    def test_passes_through_none(self) -> None:
+        # None means "auto-detect"; not the explicit-arg case we guard.
+        connection.reject_if_tcp(None, "esptool_chip_info")  # no raise
+
+
+# ---------- resolve_port ---------------------------------------------------
+
+
+class TestResolvePort:
+    def test_explicit_serial_passthrough(self) -> None:
+        assert connection.resolve_port("/dev/cu.usbmodem999") == "/dev/cu.usbmodem999"
+
+    def test_explicit_tcp_normalized(self) -> None:
+        assert connection.resolve_port("tcp://localhost") == "tcp://localhost:4403"
+
+    def test_no_port_no_devices_errors(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+        with patch.object(devices, "list_devices", return_value=[]):
+            with pytest.raises(
+                connection.ConnectionError, match="No Meshtastic devices"
+            ):
+                connection.resolve_port(None)
+
+    def test_no_port_one_candidate_selected(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+        fake = [{"port": "/dev/cu.usbmodem1", "likely_meshtastic": True}]
+        with patch.object(devices, "list_devices", return_value=fake):
+            assert connection.resolve_port(None) == "/dev/cu.usbmodem1"
+
+    def test_no_port_multiple_candidates_errors(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+        fake = [
+            {"port": "/dev/cu.usbmodem1", "likely_meshtastic": True},
+            {"port": "/dev/cu.usbmodem2", "likely_meshtastic": True},
+        ]
+        with patch.object(devices, "list_devices", return_value=fake):
+            with pytest.raises(connection.ConnectionError, match="Multiple"):
+                connection.resolve_port(None)
+
+    def test_env_var_surfaces_tcp_via_devices(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost")
+        # Don't patch list_devices — let the real env-var path run, but stub
+        # the USB enumeration to keep the test hermetic.
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            assert connection.resolve_port(None) == "tcp://localhost:4403"
+
+
+# ---------- devices.list_devices TCP entry --------------------------------
+
+
+class TestDevicesTcpEntry:
+    def test_no_env_var_no_tcp_entry(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            ds = devices.list_devices()
+        assert all(not d["port"].startswith("tcp://") for d in ds)
+
+    def test_env_var_adds_tcp_entry(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "myhost:9999")
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            ds = devices.list_devices()
+        tcp = [d for d in ds if d["port"].startswith("tcp://")]
+        assert len(tcp) == 1
+        assert tcp[0]["port"] == "tcp://myhost:9999"
+        assert tcp[0]["likely_meshtastic"] is True
+        assert tcp[0]["description"] == "meshtasticd (TCP)"
+
+    def test_tcp_entry_first_in_results(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost")
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            ds = devices.list_devices()
+        assert ds, "expected at least the TCP entry"
+        assert ds[0]["port"].startswith("tcp://")
+
+    def test_invalid_env_var_does_not_break_list_devices(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        # `list_devices` is the diagnostic tool reached for when an env var
+        # isn't working — it must not throw on misconfiguration.
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            ds = devices.list_devices(include_unknown=True)
+        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
+        assert len(tcp) == 1
+        assert tcp[0]["likely_meshtastic"] is False
+        assert "invalid MESHTASTIC_MCP_TCP_HOST" in tcp[0]["description"]
+        assert "not an integer" in tcp[0]["description"]
+
+    def test_invalid_env_var_excluded_from_resolve_port_autodetect(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        # `likely_meshtastic=False` keeps the bad TCP entry out of the
+        # auto-select path — `resolve_port(None)` should still report
+        # "no Meshtastic devices" rather than picking a broken endpoint.
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            with pytest.raises(connection.ConnectionError, match="No Meshtastic"):
+                connection.resolve_port(None)
+
+    def test_invalid_env_var_does_not_double_tcp_scheme(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        # If a user mistakenly sets `MESHTASTIC_MCP_TCP_HOST=tcp://host:bad`,
+        # the diagnostic entry must surface the raw value as-is rather than
+        # producing `tcp://tcp://host:bad`.
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "tcp://host:notaport")
+        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
+            ds = devices.list_devices(include_unknown=True)
+        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
+        assert len(tcp) == 1
+        assert tcp[0]["port"] == "tcp://host:notaport"
+        assert "tcp://tcp://" not in tcp[0]["port"]
+
+    def test_invalid_env_var_does_not_pre_empt_real_usb_devices(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        # Sort ordering: a misconfigured TCP env var must NOT take position 0
+        # ahead of real USB candidates. Position 0 is reserved for the highest
+        # rank (likely_meshtastic=True), with TCP-before-USB as a tiebreaker
+        # within rank.
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
+
+        # Stub a USB Meshtastic candidate (Espressif VID, port present in
+        # findPorts).
+        class FakeInfo:
+            def __init__(self, device: str, vid: int, pid: int) -> None:
+                self.device = device
+                self.vid = vid
+                self.pid = pid
+                self.description = "Heltec V3"
+                self.manufacturer = "Espressif"
+                self.product = "USB JTAG/serial"
+                self.serial_number = "abc"
+
+        fake_port = FakeInfo("/dev/cu.usbmodem4201", 0x303A, 0x1001)
+        with patch(
+            "meshtastic_mcp.devices.list_ports.comports", return_value=[fake_port]
+        ), patch(
+            "meshtastic.util.findPorts",
+            return_value=["/dev/cu.usbmodem4201"],
+        ):
+            ds = devices.list_devices(include_unknown=True)
+
+        assert ds, "expected at least the USB + TCP entries"
+        # Real USB candidate must be at position 0 — it's likely_meshtastic.
+        assert ds[0]["port"] == "/dev/cu.usbmodem4201"
+        assert ds[0]["likely_meshtastic"] is True
+        # The malformed TCP entry exists but lands among the unlikely entries.
+        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
+        assert len(tcp) == 1
+        assert tcp[0]["likely_meshtastic"] is False
+        assert ds.index(tcp[0]) > 0
+
+    def test_likely_tcp_entry_wins_tiebreak_over_usb(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        # Conversely, a *valid* TCP env var should sort ahead of USB
+        # candidates of equal likely_meshtastic rank — explicit env-var
+        # configuration is a precedence signal.
+        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost:4403")
+
+        class FakeInfo:
+            def __init__(self, device: str, vid: int, pid: int) -> None:
+                self.device = device
+                self.vid = vid
+                self.pid = pid
+                self.description = "Heltec V3"
+                self.manufacturer = "Espressif"
+                self.product = "USB JTAG/serial"
+                self.serial_number = "abc"
+
+        fake_port = FakeInfo("/dev/cu.usbmodem4201", 0x303A, 0x1001)
+        with patch(
+            "meshtastic_mcp.devices.list_ports.comports", return_value=[fake_port]
+        ), patch(
+            "meshtastic.util.findPorts",
+            return_value=["/dev/cu.usbmodem4201"],
+        ):
+            ds = devices.list_devices()
+
+        assert ds[0]["port"] == "tcp://localhost:4403"
+        assert ds[0]["likely_meshtastic"] is True
+
+
+# ---------- connect() routing ---------------------------------------------
+
+
+class TestConnectRoutesTcp:
+    def test_connect_uses_tcp_interface_for_tcp_port(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        """Verify the TCP branch instantiates `TCPInterface(hostname, portNumber)`
+        and never touches `SerialInterface`."""
+        # Make sure the env var doesn't leak in and confuse resolve_port.
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+
+        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp, patch(
+            "meshtastic.serial_interface.SerialInterface"
+        ) as mock_serial:
+            mock_tcp.return_value.close.return_value = None
+            with connection.connect(port="tcp://example.com:1234", timeout_s=12.0):
+                pass
+
+        mock_tcp.assert_called_once_with(
+            hostname="example.com",
+            portNumber=1234,
+            connectNow=True,
+            noProto=False,
+            timeout=12,
+        )
+        mock_serial.assert_not_called()
+
+    def test_connect_plumbs_timeout_to_serial_interface(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        """Verify the serial branch also propagates `timeout_s` so callers
+        passing a custom timeout to `device_info` / `list_nodes` / etc. don't
+        silently get the library default."""
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+
+        with patch("meshtastic.serial_interface.SerialInterface") as mock_serial, patch(
+            "meshtastic.tcp_interface.TCPInterface"
+        ) as mock_tcp:
+            mock_serial.return_value.close.return_value = None
+            with connection.connect(port="/dev/cu.fake", timeout_s=20.0):
+                pass
+
+        mock_serial.assert_called_once_with(
+            devPath="/dev/cu.fake",
+            connectNow=True,
+            noProto=False,
+            timeout=20,
+        )
+        mock_tcp.assert_not_called()
+
+    def test_connect_releases_lock_on_tcp_failure(
+        self, monkeypatch: pytest.MonkeyPatch
+    ) -> None:
+        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
+        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp:
+            mock_tcp.side_effect = RuntimeError("boom")
+            with pytest.raises(RuntimeError, match="boom"):
+                with connection.connect(port="tcp://locktest:4403"):
+                    pass
+
+        # Lock should be released — a second connect attempt must not fail
+        # with "busy".
+        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp:
+            mock_tcp.return_value.close.return_value = None
+            with connection.connect(port="tcp://locktest:4403"):
+                pass

mcp-server/tests/unit/test_fake_nodedb_generator.py

@@ -0,0 +1,364 @@
+"""Tests for the fake-NodeDB fixture pipeline (bin/gen-fake-nodedb-seed.py
++ bin/seed-json-to-proto.py + mcp-server fixtures.push_fake_nodedb).
+
+Lives under tests/unit/ because none of these touch real hardware — they
+shell out to the bin/ scripts and decode the resulting protobufs in-process.
+"""
+
+from __future__ import annotations
+
+import json
+import pathlib
+import subprocess
+import sys
+import time
+
+import pytest
+
+REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
+SEED_GEN = REPO_ROOT / "bin" / "gen-fake-nodedb-seed.py"
+COMPILE = REPO_ROOT / "bin" / "seed-json-to-proto.py"
+FIXTURES_DIR = REPO_ROOT / "test" / "fixtures" / "nodedb"
+
+# Ensure the locally-generated Python protobuf bindings are importable.
+# These live under `meshtastic_v25` (not `meshtastic`) so they don't shadow
+# the PyPI `meshtastic` package that the rest of the mcp-server depends on.
+_BINDINGS_DIR = REPO_ROOT / "bin" / "_generated"
+if _BINDINGS_DIR.is_dir() and str(_BINDINGS_DIR) not in sys.path:
+    sys.path.insert(0, str(_BINDINGS_DIR))
+
+try:
+    from meshtastic_v25.deviceonly_pb2 import (
+        NodeDatabase,  # type: ignore[import-not-found]
+    )
+except ImportError:
+    NodeDatabase = None  # type: ignore[assignment]
+
+
+def _require_v25_bindings() -> None:
+    if NodeDatabase is None:
+        pytest.skip(
+            "v25 Python protobuf bindings missing; run `./bin/regen-py-protos.sh`."
+        )
+    if "positions" not in NodeDatabase.DESCRIPTOR.fields_by_name:
+        pytest.skip(
+            "Loaded NodeDatabase predates v25 — run `./bin/regen-py-protos.sh`."
+        )
+
+
+def _run(cmd: list[str]) -> None:
+    subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+
+
+# ---------------------------------------------------------------------------
+# Seed generator: deterministic for given --seed (no wall-clock dependence).
+# ---------------------------------------------------------------------------
+def test_seed_generator_is_deterministic(tmp_path: pathlib.Path) -> None:
+    a = tmp_path / "a.jsonl"
+    b = tmp_path / "b.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "100",
+            "--seed",
+            "42",
+            "--out",
+            str(a),
+        ]
+    )
+    # Sleep so any sneaky wall-clock leak in the generator would surface as
+    # a byte diff between the two runs.
+    time.sleep(0.8)
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "100",
+            "--seed",
+            "42",
+            "--out",
+            str(b),
+        ]
+    )
+    assert a.read_bytes() == b.read_bytes()
+
+
+def test_seed_generator_meta_line(tmp_path: pathlib.Path) -> None:
+    out = tmp_path / "seed.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "50",
+            "--seed",
+            "1",
+            "--out",
+            str(out),
+        ]
+    )
+    lines = out.read_text(encoding="utf-8").splitlines()
+    assert len(lines) == 51  # 1 meta + 50 nodes
+    meta = json.loads(lines[0])
+    assert "_meta" in meta
+    assert meta["_meta"]["version"] == 25
+    assert meta["_meta"]["count"] == 50
+    assert meta["_meta"]["seed"] == 1
+
+
+def test_seed_only_uses_active_hardware_and_roles(tmp_path: pathlib.Path) -> None:
+    """Confirm no deprecated roles + no off-list HW models leak through."""
+    out = tmp_path / "seed.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "500",
+            "--seed",
+            "7",
+            "--out",
+            str(out),
+        ]
+    )
+    forbidden_roles = {"ROUTER_CLIENT", "REPEATER"}
+    forbidden_hw = {
+        "TLORA_V1",
+        "TLORA_V2",
+        "TLORA_V1_1P3",
+        "TLORA_V2_1_1P6",
+        "TLORA_V2_1_1P8",
+        "HELTEC_V1",
+        "HELTEC_V2_0",
+        "HELTEC_V2_1",
+        "TBEAM",
+        "TBEAM_V0P7",
+        "NANO_G1",
+        "NANO_G1_EXPLORER",
+        "NANO_G2_ULTRA",
+        "STATION_G1",
+        "STATION_G2",
+        "PORTDUINO",
+        "ANDROID_SIM",
+        "DIY_V1",
+        "LORA_RELAY_V1",
+        "NRF52840_PCA10059",
+        "NRF52_UNKNOWN",
+        "DR_DEV",
+        "GENIEBLOCKS",
+        "M5STACK",
+        "RP2040_LORA",
+        "PPR",
+    }
+    for raw in out.read_text(encoding="utf-8").splitlines()[1:]:
+        node = json.loads(raw)
+        assert node["role"] not in forbidden_roles, f"deprecated role: {node['role']}"
+        assert (
+            node["hw_model"] not in forbidden_hw
+        ), f"non-tier-1 HW: {node['hw_model']}"
+
+
+# ---------------------------------------------------------------------------
+# Compile step + committed seeds.
+# ---------------------------------------------------------------------------
+@pytest.mark.parametrize("size", [250, 500, 1000, 2000])
+def test_committed_seed_compiles_and_decodes(size: int, tmp_path: pathlib.Path) -> None:
+    _require_v25_bindings()
+    proto = tmp_path / "out.proto"
+    jsonl = FIXTURES_DIR / f"seed_v25_{size:04d}.jsonl"
+    if not jsonl.is_file():
+        pytest.skip(f"{jsonl} not present — run ./bin/regen-fake-nodedbs.sh")
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(proto)])
+
+    db = NodeDatabase()
+    db.ParseFromString(proto.read_bytes())
+    assert db.version == 25
+    assert len(db.nodes) == size
+    nums = {n.num for n in db.nodes}
+    assert len(nums) == size, "node numbers must be unique"
+    assert all(n.long_name and n.short_name for n in db.nodes)
+    assert all(len(n.long_name) <= 24 for n in db.nodes)  # max_size:25 - NUL
+
+    # Coverage sanity (±10pp tolerance for binomial fluctuation).
+    def in_range(actual: int, expected_ratio: float, tol_pp: float = 0.10) -> bool:
+        lo = max(0, int((expected_ratio - tol_pp) * size))
+        hi = min(size, int((expected_ratio + tol_pp) * size))
+        return lo <= actual <= hi
+
+    assert in_range(len(db.positions), 0.85)
+    assert in_range(len(db.telemetry), 0.70)
+    assert in_range(len(db.environment), 0.25)
+    assert in_range(len(db.status), 0.40)
+
+
+def test_compile_freshens_timestamps(tmp_path: pathlib.Path) -> None:
+    """Same JSONL compiled twice → identical structure, different timestamps."""
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present — run ./bin/regen-fake-nodedbs.sh")
+    a = tmp_path / "a.proto"
+    b = tmp_path / "b.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(a)])
+    time.sleep(1.2)
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(b)])
+
+    da = NodeDatabase()
+    db_ = NodeDatabase()
+    da.ParseFromString(a.read_bytes())
+    db_.ParseFromString(b.read_bytes())
+
+    # Zero out timestamp fields and confirm everything else is byte-identical.
+    for d in (da, db_):
+        for n in d.nodes:
+            n.last_heard = 0
+        for p in d.positions:
+            p.position.time = 0
+    assert da.SerializeToString() == db_.SerializeToString()
+
+    # Re-load fresh copies to confirm timestamps actually moved.
+    aa = NodeDatabase()
+    bb = NodeDatabase()
+    aa.ParseFromString(a.read_bytes())
+    bb.ParseFromString(b.read_bytes())
+    aa_max = max(n.last_heard for n in aa.nodes if n.last_heard)
+    bb_max = max(n.last_heard for n in bb.nodes if n.last_heard)
+    assert bb_max >= aa_max
+    assert bb_max - aa_max < 5  # within a few seconds
+
+
+def test_compile_pinned_now_epoch_is_byte_identical(tmp_path: pathlib.Path) -> None:
+    """With --now-epoch pinned, two compiles produce identical bytes."""
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present")
+    a = tmp_path / "a.proto"
+    b = tmp_path / "b.proto"
+    for o in (a, b):
+        _run(
+            [
+                sys.executable,
+                str(COMPILE),
+                "--in",
+                str(jsonl),
+                "--now-epoch",
+                "1700000000",
+                "--out",
+                str(o),
+            ]
+        )
+    assert a.read_bytes() == b.read_bytes()
+
+
+def test_compile_timestamps_are_recent(tmp_path: pathlib.Path) -> None:
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present")
+    out = tmp_path / "out.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(out)])
+    db = NodeDatabase()
+    db.ParseFromString(out.read_bytes())
+    now = int(time.time())
+    # No timestamp older than 7 days, none in the future.
+    for n in db.nodes:
+        if n.last_heard:
+            assert now - 7 * 86400 <= n.last_heard <= now
+    # At least half should be within the last hour
+    # (matches expovariate(mean=3600s)).
+    recent = sum(1 for n in db.nodes if n.last_heard and n.last_heard >= now - 3600)
+    assert recent >= 0.4 * len(db.nodes)
+
+
+def test_compile_hand_edit_round_trip(tmp_path: pathlib.Path) -> None:
+    """Edit one JSONL line, recompile, confirm edit appears in the proto."""
+    _require_v25_bindings()
+    src = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not src.is_file():
+        pytest.skip("250-node seed not present")
+    dst = tmp_path / "edited.jsonl"
+    lines = src.read_text(encoding="utf-8").splitlines()
+
+    # Find a node that already has telemetry so the index relationship is
+    # easy to assert on the other side.
+    edit_idx = None
+    for i, raw in enumerate(lines[1:], start=1):
+        node = json.loads(raw)
+        if node.get("telemetry") is not None:
+            edit_idx = i
+            break
+    assert edit_idx is not None, "expected at least one node with telemetry"
+
+    node = json.loads(lines[edit_idx])
+    target_num = int(node["num"], 16)
+    node["long_name"] = "Hand Edited Node"
+    node["telemetry"] = {
+        "battery_level": 42,
+        "voltage": 3.71,
+        "channel_utilization": 0.0,
+        "air_util_tx": 0.0,
+        "uptime_seconds": 1,
+    }
+    lines[edit_idx] = json.dumps(node, ensure_ascii=False, sort_keys=True)
+    dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+    out = tmp_path / "out.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(dst), "--out", str(out)])
+    db = NodeDatabase()
+    db.ParseFromString(out.read_bytes())
+    edited = next((n for n in db.nodes if n.num == target_num), None)
+    assert edited is not None
+    assert edited.long_name == "Hand Edited Node"
+    tel = next((t for t in db.telemetry if t.num == target_num), None)
+    assert tel is not None
+    assert tel.device_metrics.battery_level == 42
+
+
+# ---------------------------------------------------------------------------
+# Misc smoke checks on the module surface.
+# ---------------------------------------------------------------------------
+def test_crc16_ccitt_matches_known_vectors() -> None:
+    """Sanity-check the hand-rolled CRC16-CCITT matches well-known vectors.
+
+    Test vectors from the XModem-CRC spec (init=0, poly=0x1021):
+      crc16("123456789") = 0x31C3
+      crc16("")          = 0x0000
+    """
+    from meshtastic_mcp.fixtures import _crc16_ccitt
+
+    assert _crc16_ccitt(b"") == 0x0000
+    assert _crc16_ccitt(b"123456789") == 0x31C3
+
+
+def test_push_fake_nodedb_rejects_invalid_size() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="size must be one of"):
+        push_fake_nodedb(size=999, target="portduino")  # type: ignore[arg-type]
+
+
+def test_push_fake_nodedb_hardware_requires_confirm() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="confirm=True"):
+        push_fake_nodedb(size=250, target="hardware", port="/dev/cu.fake")
+
+
+def test_push_fake_nodedb_hardware_requires_port() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="requires a port"):
+        push_fake_nodedb(size=250, target="hardware", confirm=True)
+
+
+def test_push_fake_nodedb_hardware_rejects_tcp_port() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="not supported"):
+        push_fake_nodedb(
+            size=250, target="hardware", confirm=True, port="tcp://localhost:4403"
+        )

mcp-server/tests/unit/test_recorder.py

@@ -0,0 +1,548 @@
+"""Unit tests for the persistent device-log recorder.
+
+Hardware-free: drives the Recorder through its `_on_*` handlers with
+synthetic packet/line dicts, then queries via log_query. Validates
+prefix parsing, telemetry variant dispatch, marker round-trip, time
+window filtering, downsampling, slope estimation, and gzip rotation
++ archive pruning.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import logging
+import os
+import time
+from pathlib import Path
+
+import pubsub
+import pytest
+from meshtastic_mcp import log_query
+from meshtastic_mcp.recorder.parsers import (
+    extract_telemetry,
+    interface_label,
+    parse_log_line,
+    summarize_packet,
+)
+from meshtastic_mcp.recorder.recorder import Recorder
+from meshtastic_mcp.recorder.rotating import _RotatingJsonl
+
+# -- isolation: every test gets a fresh Recorder + tmp dir -----------
+
+
+@pytest.fixture
+def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
+    # Redirect both the Recorder and the module-level singleton lookup
+    # to the same tmp dir so log_query queries the same files we write.
+    monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
+    monkeypatch.setattr(
+        "meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
+    )
+    r = Recorder(base_dir=tmp_path)
+    r.start()
+    monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
+    yield r
+    r.stop()
+
+
+class _FakeIface:
+    devPath = "/dev/cu.fake"
+
+
+# -- parsers ---------------------------------------------------------
+
+
+class TestParseLogLine:
+    def test_full_prefix(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["uptime_s"] == 12345
+        assert out["msg"] == "Booting"
+        assert out["clock"] == "12:34:56"
+
+    def test_invalid_clock(self) -> None:
+        out = parse_log_line("WARN  | ??:??:?? 7 [SerialConsole] Boot")
+        assert out["level"] == "WARN"
+        assert out["clock"] == "??:??:??"
+        assert out["uptime_s"] == 7
+
+    def test_no_thread_bracket(self) -> None:
+        out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
+        assert out["level"] == "DEBUG"
+        assert out.get("tag") is None
+        assert out["msg"] == "raw message body"
+
+    def test_bare_message(self) -> None:
+        # LogRecord.message path — no level prefix at all.
+        out = parse_log_line("just a bare message")
+        assert "level" not in out or out.get("level") is None
+        assert out["line"] == "just a bare message"
+
+    def test_empty(self) -> None:
+        assert parse_log_line("") == {"line": ""}
+
+    def test_debug_heap_prefix_extracted(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] [heap 92344] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["heap_free"] == 92344
+        assert out["msg"] == "Booting"
+
+    def test_debug_heap_prefix_on_bare_line(self) -> None:
+        # LogRecord.message path: no level prefix but still has [heap N].
+        out = parse_log_line("[heap 12345] some message")
+        assert out["heap_free"] == 12345
+        assert out["msg"] == "some message"
+
+    def test_thread_leak_event(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
+        )
+        assert out["level"] == "HEAP"
+        assert out["heap_free"] == 90000
+        ev = out["heap_event"]
+        assert ev["kind"] == "leaked"
+        assert ev["thread"] == "MeshPacket"
+        assert ev["before"] == 92344
+        assert ev["after"] == 90000
+        assert ev["delta"] == -2344
+
+    def test_thread_freed_event(self) -> None:
+        out = parse_log_line(
+            "++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
+        )
+        ev = out["heap_event"]
+        assert ev["kind"] == "freed"
+        assert ev["thread"] == "Router"
+        assert ev["delta"] == 500
+
+    def test_heap_status_periodic(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
+        )
+        assert out["heap_free"] == 92344
+        assert out["heap_total"] == 200000
+        assert out["heap_delta"] == -128
+
+
+class TestRecorderDebugHeapSynthesis:
+    def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
+        # When a log line carries [heap N], the recorder should also
+        # emit a synthesized telemetry row tagged source=debug_heap.
+        recorder._on_log_line(
+            "INFO  | 00:00:00 1 [Main] [heap 88888] hello",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["variant"] == "local"
+
+    def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert synth[-1]["fields"]["heap_free_bytes"] == 50000
+        assert synth[-1]["fields"]["heap_total_bytes"] == 200000
+
+    def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
+        # Plain log line (no [heap N], no Heap status) — telemetry.jsonl
+        # should NOT gain a synth row.
+        before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        recorder._on_log_line("INFO  | 00:00:00 1 [Main] just a message", _FakeIface())
+        after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        assert after == before
+
+    def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
+            _FakeIface(),
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt_rows = [r for r in rows if r.get("heap_event")]
+        assert len(evt_rows) == 1
+        assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
+        assert evt_rows[0]["heap_event"]["delta"] == -2344
+
+
+class TestSerialTap:
+    def test_serial_line_records_log_and_synthesizes_heap(
+        self, recorder: "Recorder"
+    ) -> None:
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 5 [Main] [heap 88888] tap-line",
+            port="/dev/cu.tap",
+        )
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        log_rows = [json.loads(r) for r in logs if r]
+        # Find the row from this call (port=/dev/cu.tap, role=serial_session)
+        tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
+        assert len(tap_rows) == 1
+        assert tap_rows[0]["role"] == "serial_session"
+        assert tap_rows[0]["level"] == "INFO"
+        assert tap_rows[0]["tag"] == "Main"
+        assert tap_rows[0]["heap_free"] == 88888
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["role"] == "serial_session"
+
+    def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
+        recorder._on_serial_line(
+            "HEAP  | 00:00:30 30 [Power] [heap 53484] "
+            "------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
+            port="/dev/cu.tap",
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt = [r for r in rows if r.get("heap_event")]
+        assert len(evt) == 1
+        assert evt[0]["heap_event"]["thread"] == "Router"
+        assert evt[0]["heap_event"]["delta"] == -128
+        # Heap also synthesized.
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text()
+        assert '"source":"debug_heap_serial"' in telem
+
+    def test_serial_line_pause(self, recorder: "Recorder") -> None:
+        recorder.pause("baseline")
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 1 [t] [heap 1000] dropped",
+            port="/dev/cu.tap",
+        )
+        # Only the pause event row should exist; no tap row.
+        logs = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "dropped" not in logs
+
+    def test_serial_line_handler_swallows_exceptions(
+        self, recorder: "Recorder"
+    ) -> None:
+        # Hostile input — should not raise.
+        recorder._on_serial_line(None, port="/dev/cu.tap")  # type: ignore[arg-type]
+        recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap")  # type: ignore[arg-type]
+        # Survived.
+
+
+class TestExtractTelemetry:
+    def test_local_stats_camel(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {
+                    "localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
+                }
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "local"
+        assert out["fields"]["heap_free_bytes"] == 600
+
+    def test_device_metrics_snake(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "device"
+        assert out["fields"]["battery_level"] == 88
+
+    def test_unknown_variant_returns_none(self) -> None:
+        assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
+        assert extract_telemetry({}) is None
+        assert extract_telemetry({"decoded": "not-a-dict"}) is None
+
+
+class TestSummarizePacket:
+    def test_text_with_payload(self) -> None:
+        pkt = {
+            "fromId": "!abc",
+            "toId": "!def",
+            "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
+            "hopLimit": 3,
+        }
+        out = summarize_packet(pkt)
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] == "TEXT_MESSAGE_APP"
+        assert out["payload_size"] == 5
+        assert out["payload_hex_prefix"] == "68656c6c6f"
+
+    def test_no_decoded(self) -> None:
+        out = summarize_packet({"fromId": "!abc"})
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] is None
+
+
+class TestInterfaceLabel:
+    def test_serial(self) -> None:
+        assert interface_label(_FakeIface()) == {
+            "port": "/dev/cu.fake",
+            "role": "serial",
+        }
+
+    def test_tcp(self) -> None:
+        class T:
+            hostname = "node.lan"
+            portNumber = 4403
+
+        assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
+
+    def test_unknown(self) -> None:
+        assert interface_label(object()) == {"port": "object", "role": None}
+
+    def test_none(self) -> None:
+        assert interface_label(None) == {"port": None, "role": None}
+
+
+# -- recorder write side ---------------------------------------------
+
+
+class TestRecorderWrites:
+    def test_log_line_is_recorded(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:34:56 99 [T] hi", _FakeIface())
+        path = recorder.base_dir / "logs.jsonl"
+        rows = [json.loads(line) for line in path.read_text().splitlines() if line]
+        # First row is recorder_start_event mirror? No — that's events.jsonl only.
+        assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
+
+    def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
+        # _on_telemetry alone — only telemetry.jsonl
+        recorder._on_telemetry(
+            {
+                "fromId": "!abc",
+                "decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
+            },
+            _FakeIface(),
+        )
+        telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        assert any('"variant":"local"' in r for r in telem_rows)
+
+    def test_packets_summary(self, recorder: Recorder) -> None:
+        recorder._on_receive(
+            {
+                "fromId": "!abc",
+                "toId": "!def",
+                "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
+            },
+            _FakeIface(),
+        )
+        rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
+        assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
+
+    def test_mark_event_round_trip(self, recorder: Recorder) -> None:
+        out = recorder.mark_event("checkpoint", note="midpoint")
+        assert "ts" in out
+        events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
+        assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
+
+    def test_pause_drops_writes(self, recorder: Recorder) -> None:
+        before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        recorder.pause(reason="baseline")
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] swallowed", _FakeIface())
+        after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        assert after == before
+        recorder.resume()
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] kept", _FakeIface())
+        post_resume = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "kept" in post_resume
+
+    def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
+        # If the writer dies, the pubsub callback must NOT raise — that
+        # would crash the meshtastic receive thread.
+        bad_packet = object()  # not a dict
+        recorder._on_receive(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_telemetry(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_log_line(None, _FakeIface())  # type: ignore[arg-type]
+        # No assertion needed — survival is the test.
+
+
+# -- log_query read side ---------------------------------------------
+
+
+class TestLogQuery:
+    def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        recorder._on_log_line("WARN  | 12:00:01 2 [B] bravo failed", _FakeIface())
+        recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
+
+        out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
+        assert out["total_matched"] == 2
+        levels = {r["level"] for r in out["lines"]}
+        assert levels == {"WARN", "ERROR"}
+
+        out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
+        assert out2["total_matched"] == 2
+
+    def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        with pytest.raises(ValueError, match="invalid grep regex"):
+            log_query.logs_window(start="-1m", grep="(")
+
+    def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
+        # Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
+        base_ts = time.time() - 60
+        for i in range(100):
+            recorder._files["telemetry"].write(
+                {
+                    "ts": base_ts + i * 0.5,
+                    "port": "/dev/cu.fake",
+                    "role": "serial",
+                    "from_node": "!abc",
+                    "variant": "local",
+                    "fields": {"heap_free_bytes": 10000 - i},
+                }
+            )
+
+        out = log_query.telemetry_timeline(
+            window="2m", variant="local", field="free_heap", max_points=10
+        )
+        assert out["samples"] == 100
+        assert len(out["points"]) <= 10
+        # Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
+        assert out["slope_per_min"] is not None
+        assert out["slope_per_min"] < -100
+
+    def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] one", _FakeIface())
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] two", _FakeIface())
+        dest = tmp_path / "bundle"
+        out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
+        assert (dest / "logs.jsonl").exists()
+        assert "logs" in out["paths"]
+
+
+# -- time parser -----------------------------------------------------
+
+
+class TestParseTime:
+    def test_relative(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("-15m", now=now) == now - 900
+        assert log_query._parse_time("-2h", now=now) == now - 7200
+        assert log_query._parse_time("-1d", now=now) == now - 86400
+
+    def test_now_and_epoch(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("now", now=now) == now
+        assert log_query._parse_time(now) == now
+
+    def test_iso(self) -> None:
+        ts = log_query._parse_time("2026-01-01T00:00:00Z")
+        assert isinstance(ts, float) and ts > 1_700_000_000
+
+    def test_naive_iso_assumes_utc(self) -> None:
+        assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
+            "2026-01-01T00:00:00Z"
+        )
+
+    def test_invalid(self) -> None:
+        with pytest.raises(ValueError):
+            log_query._parse_time("not a time")
+
+
+# -- rotation --------------------------------------------------------
+
+
+class TestRotation:
+    def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
+        for i in range(100):
+            r.write({"ts": float(i), "i": i, "pad": "x" * 40})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives, "expected at least one rotation"
+        # Archive content is valid gzip + valid JSONL
+        with gzip.open(archives[0], "rt") as fh:
+            first = json.loads(fh.readline())
+            assert "ts" in first
+
+    def test_archive_pruning(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
+        # Force several rotations.
+        for _ in range(8):
+            for i in range(20):
+                r.write({"ts": float(i), "pad": "x" * 30})
+            r.force_rotate()
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
+
+    def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, keep_archives=2)
+        old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
+        mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
+        new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
+        for archive in (old, mid, new):
+            with gzip.open(archive, "wt", encoding="utf-8") as fh:
+                fh.write('{"ts":1}\n')
+        # Deliberately scramble mtimes so lexicographic filename order is
+        # the only stable chronological signal.
+        os.utime(old, (300, 300))
+        os.utime(mid, (100, 100))
+        os.utime(new, (200, 200))
+
+        r._prune_archives()
+        r.close()
+
+        archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives == [mid.name, new.name]
+
+    def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
+        r.write({"ts": 1.0, "msg": "tiny"})
+        r.force_rotate()
+        r.write({"ts": 2.0, "msg": "after-rotate"})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) == 1
+        assert path.exists()
+        assert "after-rotate" in path.read_text()
+
+
+class TestRecorderLocks:
+    def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
+        out = recorder.force_rotate_all()
+        assert out["running"] is True
+        assert out["files"]
+
+    def test_wire_pubsub_logs_subscription_failure(
+        self,
+        tmp_path: Path,
+        monkeypatch: pytest.MonkeyPatch,
+        caplog: pytest.LogCaptureFixture,
+    ) -> None:
+        class FailingPubSubMock:
+            def subscribe(self, callback: object, topic: str) -> None:
+                raise RuntimeError("boom")
+
+        monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
+        recorder = Recorder(base_dir=tmp_path)
+        with caplog.at_level(logging.WARNING):
+            recorder._wire_pubsub()
+        assert (
+            "Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
+        )

platformio.ini

@@ -2,7 +2,7 @@
 ; https://docs.platformio.org/page/projectconf.html
 
 [platformio]
-default_envs = tbeam
+default_envs = heltec-v3
 
 extra_configs =
 	variants/*/*.ini
@@ -49,6 +49,7 @@ build_flags = -Wno-missing-field-initializers
 	-DRADIOLIB_EXCLUDE_PAGER=1
 	-DRADIOLIB_EXCLUDE_FSK4=1
 	-DRADIOLIB_EXCLUDE_APRS=1
+	-DRADIOLIB_EXCLUDE_ADSB=1
 	-DRADIOLIB_EXCLUDE_LORAWAN=1
 	-DMESHTASTIC_EXCLUDE_DROPZONE=1
 	-DMESHTASTIC_EXCLUDE_REPLYBOT=1
@@ -120,12 +121,12 @@ lib_deps =
 [radiolib_base]
 lib_deps =
 	# renovate: datasource=github-tags depName=RadioLib packageName=jgromes/RadioLib
-	https://github.com/jgromes/RadioLib/archive/refs/tags/7.6.0.zip
+	https://github.com/jgromes/RadioLib/archive/afe72ae46a343e15e3cac7f26ac585c7f98bffe5.zip
 
 [device-ui_base]
 lib_deps =
 	# renovate: datasource=git-refs depName=meshtastic/device-ui packageName=https://github.com/meshtastic/device-ui gitBranch=master
-	https://github.com/meshtastic/device-ui/archive/56e1da4e7d30abcd746a2092a30e422f8cf5fc2b.zip
+	https://github.com/meshtastic/device-ui/archive/4bf593a82100b911ff816dddf7158ffdee2114cd.zip
 
 ; Common libs for environmental measurements in telemetry module
 [environmental_base]
@@ -137,7 +138,7 @@ lib_deps =
 	# renovate: datasource=github-tags depName=Adafruit GFX packageName=adafruit/Adafruit-GFX-Library
 	https://github.com/adafruit/Adafruit-GFX-Library/archive/refs/tags/1.12.6.zip
 	# renovate: datasource=github-tags depName=NeoPixel packageName=adafruit/Adafruit_NeoPixel
-	https://github.com/adafruit/Adafruit_NeoPixel/archive/refs/tags/1.15.4.zip
+	https://github.com/adafruit/Adafruit_NeoPixel/archive/1.15.5.zip
 	# renovate: datasource=github-tags depName=Adafruit SSD1306 packageName=adafruit/Adafruit_SSD1306
 	https://github.com/adafruit/Adafruit_SSD1306/archive/refs/tags/2.5.16.zip
 	# renovate: datasource=github-tags depName=Adafruit BMP280 packageName=adafruit/Adafruit_BMP280_Library
@@ -170,8 +171,8 @@ lib_deps =
 	https://github.com/EmotiBit/EmotiBit_MLX90632/archive/refs/tags/v1.0.8.zip
 	# renovate: datasource=github-tags depName=Adafruit MLX90614 packageName=adafruit/Adafruit_MLX90614
 	https://github.com/adafruit/Adafruit-MLX90614-Library/archive/refs/tags/2.1.6.zip
-	# renovate: datasource=git-refs depName=INA3221 packageName=https://github.com/sgtwilko/INA3221 gitBranch=FixOverflow
-	https://github.com/sgtwilko/INA3221/archive/bb03d7e9bfcc74fc798838a54f4f99738f29fc6a.zip
+	# renovate: datasource=github-tags depName=INA3221_RT packageName=RobTillaart/INA3221_RT
+	https://github.com/RobTillaart/INA3221_RT/archive/refs/tags/0.4.2.zip
 	# renovate: datasource=github-tags depName=QMC5883L Compass packageName=mprograms/QMC5883LCompass
 	https://github.com/mprograms/QMC5883LCompass/archive/refs/tags/v1.2.3.zip
 	# renovate: datasource=github-tags depName=DFRobot_RTU packageName=dfrobot/DFRobot_RTU

src/DebugConfiguration.cpp

@@ -26,6 +26,8 @@ SOFTWARE.*/
 
 #include "DebugConfiguration.h"
 
+#include <memory>
+
 #ifdef ARCH_PORTDUINO
 #include "platform/portduino/PortduinoGlue.h"
 #endif
@@ -119,27 +121,22 @@ bool Syslog::vlogf(uint16_t pri, const char *fmt, va_list args)
 
 bool Syslog::vlogf(uint16_t pri, const char *appName, const char *fmt, va_list args)
 {
-    char *message;
-    size_t initialLen;
-    size_t len;
-    bool result;
+    // First measure the formatted length using a copy of args; passing args directly
+    // to vsnprintf consumes it, and reusing a consumed va_list is undefined behavior.
+    va_list args_measure;
+    va_copy(args_measure, args);
+    int needed = vsnprintf(nullptr, 0, fmt, args_measure);
+    va_end(args_measure);
 
-    initialLen = strlen(fmt);
+    if (needed < 0)
+        return false; // encoding error
 
-    message = new char[initialLen + 1];
+    auto message = std::unique_ptr<char[]>(new char[static_cast<size_t>(needed) + 1]);
+    int written = vsnprintf(message.get(), static_cast<size_t>(needed) + 1, fmt, args);
+    if (written < 0)
+        return false;
 
-    len = vsnprintf(message, initialLen + 1, fmt, args);
-    if (len > initialLen) {
-        delete[] message;
-        message = new char[len + 1];
-
-        vsnprintf(message, len + 1, fmt, args);
-    }
-
-    result = this->_sendLog(pri, appName, message);
-
-    delete[] message;
-    return result;
+    return this->_sendLog(pri, appName, message.get());
 }
 
 inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *message)
@@ -154,7 +151,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
     if (!this->_enabled)
         return false;
 
-    if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
+    if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
         return false;
 
     // Check priority against priMask values.

src/DebugConfiguration.h

@@ -13,6 +13,11 @@ extern MemGet memGet;
 #define LED_STATE_ON 1
 #endif
 
+// WIFI LED
+#ifndef WIFI_STATE_ON
+#define WIFI_STATE_ON 1
+#endif
+
 // -----------------------------------------------------------------------------
 // DEBUG
 // -----------------------------------------------------------------------------
@@ -147,13 +152,14 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
 // Default Bluetooth PIN
 #define defaultBLEPin 123456
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && defined(USE_CH390D)
+#include <ESP32_CH390.h>
+#elif HAS_ETHERNET && !defined(USE_WS5500)
 #include <RAK13800_W5100S.h>
 #endif // HAS_ETHERNET
 
-#if HAS_ETHERNET && defined(USE_WS5500)
-#include <ETHClass2.h>
-#define ETH ETH2
+#if HAS_ETHERNET && defined(ARCH_ESP32)
+#include <ETH.h>
 #endif // HAS_ETHERNET
 
 #if HAS_WIFI

src/FSCommon.cpp

@@ -79,28 +79,46 @@ bool copyFile(const char *from, const char *to)
 bool renameFile(const char *pathFrom, const char *pathTo)
 {
 #ifdef FSCom
-
-#ifdef ARCH_ESP32
-    // take SPI Lock
     spiLock->lock();
-    // rename was fixed for ESP32 IDF LittleFS in April
     bool result = FSCom.rename(pathFrom, pathTo);
     spiLock->unlock();
     return result;
 #else
-    // copyFile does its own locking.
-    if (copyFile(pathFrom, pathTo) && FSCom.remove(pathFrom)) {
-        return true;
-    } else {
-        return false;
-    }
-#endif
-
+    return false;
 #endif
 }
 
 #include <vector>
 
+/**
+ * @brief Platform-agnostic filesystem format / wipe.
+ *
+ * On embedded targets (ESP32, NRF52, STM32WL, RP2040) this calls the
+ * native FSCom.format() which erases and reinitialises the LittleFS
+ * partition.
+ *
+ * On Portduino the fs::FS backend has no format() method. We instead
+ * delete /prefs (the only meshtastic data directory written at runtime)
+ * and return. rmDir("/prefs") is already called unconditionally by
+ * factoryReset() so this is a proven primitive on Portduino.
+ * FSBegin() is a no-op (#define FSBegin() true) on Portduino.
+ *
+ * @return true on success, false on failure or if no filesystem is configured.
+ */
+bool fsFormat()
+{
+#ifdef FSCom
+#if defined(ARCH_PORTDUINO)
+    rmDir("/prefs");
+    return FSBegin();
+#else
+    return FSCom.format();
+#endif
+#else
+    return false;
+#endif
+}
+
 /**
  * @brief Get the list of files in a directory.
  *
@@ -123,23 +141,21 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
 
     File file = root.openNextFile();
     while (file) {
+#ifdef ARCH_ESP32
+        const char *filepath = file.path();
+#else
+        const char *filepath = file.name();
+#endif
         if (file.isDirectory() && !String(file.name()).endsWith(".")) {
             if (levels) {
-#ifdef ARCH_ESP32
-                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.path(), levels - 1);
-#else
-                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.name(), levels - 1);
-#endif
+                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(filepath, levels - 1);
                 filenames.insert(filenames.end(), subDirFilenames.begin(), subDirFilenames.end());
                 file.close();
             }
         } else {
             meshtastic_FileInfo fileInfo = {"", static_cast<uint32_t>(file.size())};
-#ifdef ARCH_ESP32
-            strcpy(fileInfo.file_name, file.path());
-#else
-            strcpy(fileInfo.file_name, file.name());
-#endif
+            strncpy(fileInfo.file_name, filepath, sizeof(fileInfo.file_name) - 1);
+            fileInfo.file_name[sizeof(fileInfo.file_name) - 1] = '\0';
             if (!String(fileInfo.file_name).endsWith(".")) {
                 filenames.push_back(fileInfo);
             }
@@ -163,98 +179,59 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
 void listDir(const char *dirname, uint8_t levels, bool del)
 {
 #ifdef FSCom
-#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
     char buffer[255];
-#endif
     File root = FSCom.open(dirname, FILE_O_READ);
-    if (!root) {
+    if (!root || !root.isDirectory())
         return;
-    }
-    if (!root.isDirectory()) {
-        return;
-    }
 
     File file = root.openNextFile();
-    while (
-        file &&
-        file.name()[0]) { // This file.name() check is a workaround for a bug in the Adafruit LittleFS nrf52 glue (see issue 4395)
+    while (file && file.name()[0]) { // file.name()[0] check: workaround for Adafruit LittleFS nRF52 bug #4395
+#ifdef ARCH_ESP32
+        const char *filepath = file.path();
+#else
+        const char *filepath = file.name();
+#endif
         if (file.isDirectory() && !String(file.name()).endsWith(".")) {
             if (levels) {
-#ifdef ARCH_ESP32
-                listDir(file.path(), levels - 1, del);
+                listDir(filepath, levels - 1, del);
                 if (del) {
-                    LOG_DEBUG("Remove %s", file.path());
-                    strncpy(buffer, file.path(), sizeof(buffer));
+                    LOG_DEBUG("Remove %s", filepath);
+                    strncpy(buffer, filepath, sizeof(buffer) - 1);
+                    buffer[sizeof(buffer) - 1] = '\0';
                     file.close();
                     FSCom.rmdir(buffer);
                 } else {
                     file.close();
                 }
-#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
-                listDir(file.name(), levels - 1, del);
-                if (del) {
-                    LOG_DEBUG("Remove %s", file.name());
-                    strncpy(buffer, file.name(), sizeof(buffer));
-                    file.close();
-                    FSCom.rmdir(buffer);
-                } else {
-                    file.close();
-                }
-#else
-                LOG_DEBUG(" %s (directory)", file.name());
-                listDir(file.name(), levels - 1, del);
-                file.close();
-#endif
             }
         } else {
-#ifdef ARCH_ESP32
             if (del) {
-                LOG_DEBUG("Delete %s", file.path());
-                strncpy(buffer, file.path(), sizeof(buffer));
+                LOG_DEBUG("Delete %s", filepath);
+                strncpy(buffer, filepath, sizeof(buffer) - 1);
+                buffer[sizeof(buffer) - 1] = '\0';
                 file.close();
                 FSCom.remove(buffer);
             } else {
-                LOG_DEBUG(" %s (%i Bytes)", file.path(), file.size());
+                LOG_DEBUG(" %s (%i Bytes)", filepath, file.size());
                 file.close();
             }
-#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
-            if (del) {
-                LOG_DEBUG("Delete %s", file.name());
-                strncpy(buffer, file.name(), sizeof(buffer));
-                file.close();
-                FSCom.remove(buffer);
-            } else {
-                LOG_DEBUG(" %s (%i Bytes)", file.name(), file.size());
-                file.close();
-            }
-#else
-            LOG_DEBUG("   %s (%i Bytes)", file.name(), file.size());
-            file.close();
-#endif
         }
         file = root.openNextFile();
     }
 #ifdef ARCH_ESP32
-    if (del) {
-        LOG_DEBUG("Remove %s", root.path());
-        strncpy(buffer, root.path(), sizeof(buffer));
-        root.close();
-        FSCom.rmdir(buffer);
-    } else {
-        root.close();
-    }
-#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
-    if (del) {
-        LOG_DEBUG("Remove %s", root.name());
-        strncpy(buffer, root.name(), sizeof(buffer));
-        root.close();
-        FSCom.rmdir(buffer);
-    } else {
-        root.close();
-    }
+    const char *rootpath = root.path();
 #else
-    root.close();
+    const char *rootpath = root.name();
 #endif
+    if (del) {
+        LOG_DEBUG("Remove %s", rootpath);
+        strncpy(buffer, rootpath, sizeof(buffer) - 1);
+        buffer[sizeof(buffer) - 1] = '\0';
+        root.close();
+        FSCom.rmdir(buffer);
+    } else {
+        root.close();
+    }
 #endif
 }
 
@@ -268,14 +245,7 @@ void listDir(const char *dirname, uint8_t levels, bool del)
 void rmDir(const char *dirname)
 {
 #ifdef FSCom
-
-#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
     listDir(dirname, 10, true);
-#elif defined(ARCH_NRF52)
-    // nRF52 implementation of LittleFS has a recursive delete function
-    FSCom.rmdir_r(dirname);
-#endif
-
 #endif
 }
 
@@ -307,7 +277,7 @@ void fsInit()
  */
 void setupSDCard()
 {
-#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI)
+#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI) && !defined(HAS_SD_MMC)
     concurrency::LockGuard g(spiLock);
     SDHandler.begin(SPI_SCK, SPI_MISO, SPI_MOSI);
     if (!SD.begin(SDCARD_CS, SDHandler, SD_SPI_FREQUENCY)) {

src/FSCommon.h

@@ -52,6 +52,7 @@ void fsInit();
 void fsListFiles();
 bool copyFile(const char *from, const char *to);
 bool renameFile(const char *pathFrom, const char *pathTo);
+bool fsFormat();
 std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels);
 void listDir(const char *dirname, uint8_t levels, bool del = false);
 void rmDir(const char *dirname);

src/GPSStatus.h

@@ -53,8 +53,7 @@ class GPSStatus : public Status
     int32_t getLatitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.latitude_i;
+            return localPosition.latitude_i;
         } else {
             return p.latitude_i;
         }
@@ -63,8 +62,7 @@ class GPSStatus : public Status
     int32_t getLongitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.longitude_i;
+            return localPosition.longitude_i;
         } else {
             return p.longitude_i;
         }
@@ -73,8 +71,7 @@ class GPSStatus : public Status
     int32_t getAltitude() const
     {
         if (config.position.fixed_position) {
-            meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
-            return node->position.altitude;
+            return localPosition.altitude;
         } else {
             return p.altitude;
         }

src/MessageStore.cpp

@@ -6,7 +6,6 @@
 #include "SPILock.h"
 #include "SafeFile.h"
 #include "gps/RTC.h"
-#include "graphics/draw/MessageRenderer.h"
 #include <cstring> // memcpy
 
 #ifndef MESSAGE_TEXT_POOL_SIZE
@@ -181,13 +180,8 @@ const StoredMessage &MessageStore::addFromPacket(const meshtastic_MeshPacket &pa
 
     bool isDM = (sm.dest != 0 && sm.dest != NODENUM_BROADCAST);
 
-    if (packet.from == 0) {
-        sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
-        sm.ackStatus = AckStatus::NONE;
-    } else {
-        sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
-        sm.ackStatus = AckStatus::ACKED;
-    }
+    sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
+    sm.ackStatus = (packet.from == 0) ? AckStatus::NONE : AckStatus::ACKED;
 
     addLiveMessage(sm);
 
@@ -372,26 +366,25 @@ void MessageStore::clearAllMessages()
 #endif
 }
 
-// Internal helper: erase first or last message matching a predicate
-template <typename Predicate> static void eraseIf(std::deque<StoredMessage> &deque, Predicate pred, bool fromBack = false)
+// Internal helpers for targeted erasure.
+template <typename Predicate> static bool eraseFirstMatch(std::deque<StoredMessage> &deque, Predicate pred)
 {
-    if (fromBack) {
-        // Iterate from the back and erase all matches from the end
-        for (auto it = deque.rbegin(); it != deque.rend();) {
-            if (pred(*it)) {
-                it = std::deque<StoredMessage>::reverse_iterator(deque.erase(std::next(it).base()));
-            } else {
-                ++it;
-            }
+    for (auto it = deque.begin(); it != deque.end(); ++it) {
+        if (pred(*it)) {
+            deque.erase(it);
+            return true;
         }
-    } else {
-        // Manual forward search to erase all matches
-        for (auto it = deque.begin(); it != deque.end();) {
-            if (pred(*it)) {
-                it = deque.erase(it);
-            } else {
-                ++it;
-            }
+    }
+    return false;
+}
+
+template <typename Predicate> static void eraseAllMatches(std::deque<StoredMessage> &deque, Predicate pred)
+{
+    for (auto it = deque.begin(); it != deque.end();) {
+        if (pred(*it)) {
+            it = deque.erase(it);
+        } else {
+            ++it;
         }
     }
 }
@@ -399,7 +392,9 @@ template <typename Predicate> static void eraseIf(std::deque<StoredMessage> &deq
 // Delete oldest message (RAM + persisted queue)
 void MessageStore::deleteOldestMessage()
 {
-    eraseIf(liveMessages, [](StoredMessage &) { return true; });
+    if (!liveMessages.empty()) {
+        liveMessages.pop_front();
+    }
     saveToFlash();
 }
 
@@ -407,14 +402,14 @@ void MessageStore::deleteOldestMessage()
 void MessageStore::deleteOldestMessageInChannel(uint8_t channel)
 {
     auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
-    eraseIf(liveMessages, pred);
+    eraseFirstMatch(liveMessages, pred);
     saveToFlash();
 }
 
 void MessageStore::deleteAllMessagesInChannel(uint8_t channel)
 {
     auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
-    eraseIf(liveMessages, pred, false /* delete ALL, not just first */);
+    eraseAllMatches(liveMessages, pred);
     saveToFlash();
 }
 
@@ -427,7 +422,7 @@ void MessageStore::deleteAllMessagesWithPeer(uint32_t peer)
         uint32_t other = (m.sender == local) ? m.dest : m.sender;
         return other == peer;
     };
-    eraseIf(liveMessages, pred, false);
+    eraseAllMatches(liveMessages, pred);
     saveToFlash();
 }
 
@@ -440,7 +435,7 @@ void MessageStore::deleteOldestMessageWithPeer(uint32_t peer)
         uint32_t other = (m.sender == nodeDB->getNodeNum()) ? m.dest : m.sender;
         return other == peer;
     };
-    eraseIf(liveMessages, pred);
+    eraseFirstMatch(liveMessages, pred);
     saveToFlash();
 }
 

src/MessageStore.h

@@ -124,9 +124,6 @@ class MessageStore
     // Allocate text into pool (used by sender-side code)
     static uint16_t storeText(const char *src, size_t len);
 
-    // Used when loading from flash to rebuild the text pool
-    static uint16_t rebuildTextFromFlash(const char *src, size_t len);
-
   private:
     std::deque<StoredMessage> liveMessages; // Single in-RAM message buffer (also used for persistence)
     std::string filename;                   // Flash filename for persistence

src/Power.cpp

@@ -24,6 +24,13 @@
 #include "meshUtils.h"
 #include "power/PowerHAL.h"
 #include "sleep.h"
+#ifdef ARCH_ESP32
+// #include <driver/adc.h>
+#include <esp_adc/adc_cali.h>
+#include <esp_adc/adc_cali_scheme.h>
+#include <esp_adc/adc_oneshot.h>
+#include <esp_err.h>
+#endif
 
 #if defined(ARCH_PORTDUINO)
 #include "api/WiFiServerAPI.h"
@@ -63,9 +70,8 @@
 #include <WiFi.h>
 #endif
 
-#if HAS_ETHERNET && defined(USE_WS5500)
-#include <ETHClass2.h>
-#define ETH ETH2
+#if HAS_ETHERNET && defined(ARCH_ESP32)
+#include <ETH.h>
 #endif // HAS_ETHERNET
 
 #endif
@@ -77,21 +83,86 @@
 #if defined(BATTERY_PIN) && defined(ARCH_ESP32)
 
 #ifndef BAT_MEASURE_ADC_UNIT // ADC1 is default
-static const adc1_channel_t adc_channel = ADC_CHANNEL;
+static const adc_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_1;
-#else // ADC2
-static const adc2_channel_t adc_channel = ADC_CHANNEL;
+#else  // ADC2
+static const adc_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_2;
-RTC_NOINIT_ATTR uint64_t RTC_reg_b;
-
 #endif // BAT_MEASURE_ADC_UNIT
 
-esp_adc_cal_characteristics_t *adc_characs = (esp_adc_cal_characteristics_t *)calloc(1, sizeof(esp_adc_cal_characteristics_t));
+static adc_oneshot_unit_handle_t adc_handle = nullptr;
+static adc_cali_handle_t adc_cali_handle = nullptr;
+static bool adc_calibrated = false;
 #ifndef ADC_ATTENUATION
 static const adc_atten_t atten = ADC_ATTEN_DB_12;
 #else
 static const adc_atten_t atten = ADC_ATTENUATION;
 #endif
+#ifdef ADC_BITWIDTH
+static const adc_bitwidth_t adc_width = ADC_BITWIDTH;
+#else
+static const adc_bitwidth_t adc_width = ADC_BITWIDTH_DEFAULT;
+#endif
+
+static int adcBitWidthToBits(adc_bitwidth_t width)
+{
+    switch (width) {
+    case ADC_BITWIDTH_9:
+        return 9;
+    case ADC_BITWIDTH_10:
+        return 10;
+    case ADC_BITWIDTH_11:
+        return 11;
+    case ADC_BITWIDTH_12:
+        return 12;
+#ifdef ADC_BITWIDTH_13
+    case ADC_BITWIDTH_13:
+        return 13;
+#endif
+    default:
+        return 12;
+    }
+}
+
+static bool initAdcCalibration()
+{
+#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED
+    adc_cali_curve_fitting_config_t cali_config = {
+        .unit_id = unit,
+        .atten = atten,
+        .bitwidth = adc_width,
+    };
+    esp_err_t ret = adc_cali_create_scheme_curve_fitting(&cali_config, &adc_cali_handle);
+    if (ret == ESP_OK) {
+        LOG_INFO("ADC calibration: curve fitting enabled");
+        return true;
+    }
+    if (ret != ESP_ERR_NOT_SUPPORTED) {
+        LOG_WARN("ADC calibration: curve fitting failed: %s", esp_err_to_name(ret));
+    }
+#endif
+
+#if ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED
+    adc_cali_line_fitting_config_t cali_config = {
+        .unit_id = unit,
+        .atten = atten,
+        .bitwidth = adc_width,
+        .default_vref = DEFAULT_VREF,
+    };
+    esp_err_t ret = adc_cali_create_scheme_line_fitting(&cali_config, &adc_cali_handle);
+    if (ret == ESP_OK) {
+        LOG_INFO("ADC calibration: line fitting enabled");
+        return true;
+    }
+    if (ret != ESP_ERR_NOT_SUPPORTED) {
+        LOG_WARN("ADC calibration: line fitting failed: %s", esp_err_to_name(ret));
+    }
+#endif
+
+    LOG_INFO("ADC calibration not supported; using approximate scaling");
+    return false;
+}
+
 #endif // BATTERY_PIN && ARCH_ESP32
 
 #ifdef EXT_PWR_DETECT
@@ -367,8 +438,20 @@ class AnalogBatteryLevel : public HasBatteryLevel
             scaled *= operativeAdcMultiplier;
 #elif defined(ARCH_ESP32) // ADC block for espressif platforms
             raw = espAdcRead();
-            scaled = esp_adc_cal_raw_to_voltage(raw, adc_characs);
-            scaled *= operativeAdcMultiplier;
+            int voltage_mv = 0;
+            if (adc_calibrated && adc_cali_handle) {
+                if (adc_cali_raw_to_voltage(adc_cali_handle, raw, &voltage_mv) != ESP_OK) {
+                    LOG_WARN("ADC calibration read failed; using raw value");
+                    voltage_mv = 0;
+                }
+            }
+            if (voltage_mv == 0) {
+                // Fallback approximate conversion without calibration
+                const int bits = adcBitWidthToBits(adc_width);
+                const float max_code = powf(2.0f, bits) - 1.0f;
+                voltage_mv = (int)((raw / max_code) * DEFAULT_VREF);
+            }
+            scaled = voltage_mv * operativeAdcMultiplier;
 #else                     // block for all other platforms
 #ifdef ARCH_NRF52
             concurrency::LockGuard saadcGuard(concurrency::nrf52SaadcLock);
@@ -410,51 +493,22 @@ class AnalogBatteryLevel : public HasBatteryLevel
         uint32_t raw = 0;
         uint8_t raw_c = 0; // raw reading counter
 
-#ifndef BAT_MEASURE_ADC_UNIT // ADC1
+        if (!adc_handle) {
+            LOG_ERROR("ADC oneshot handle not initialized");
+            return 0;
+        }
+
         for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            int val_ = adc1_get_raw(adc_channel);
-            if (val_ >= 0) { // save only valid readings
-                raw += val_;
+            int val = 0;
+            esp_err_t err = adc_oneshot_read(adc_handle, adc_channel, &val);
+            if (err == ESP_OK) {
+                raw += val;
                 raw_c++;
-            }
-            // delayMicroseconds(100);
-        }
-#else                            // ADC2
-#ifdef CONFIG_IDF_TARGET_ESP32S3 // ESP32S3
-        // ADC2 wifi bug workaround not required, breaks compile
-        // On ESP32S3, ADC2 can take turns with Wifi (?)
-
-        int32_t adc_buf;
-        esp_err_t read_result;
-
-        // Multiple samples
-        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            adc_buf = 0;
-            read_result = -1;
-
-            read_result = adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
-            if (read_result == ESP_OK) {
-                raw += adc_buf;
-                raw_c++; // Count valid samples
             } else {
-                LOG_DEBUG("An attempt to sample ADC2 failed");
+                LOG_DEBUG("ADC read failed: %s", esp_err_to_name(err));
             }
         }
 
-#else  // Other ESP32
-        int32_t adc_buf = 0;
-        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            // ADC2 wifi bug workaround, see
-            // https://github.com/espressif/arduino-esp32/issues/102
-            WRITE_PERI_REG(SENS_SAR_READ_CTRL2_REG, RTC_reg_b);
-            SET_PERI_REG_MASK(SENS_SAR_READ_CTRL2_REG, SENS_SAR2_DATA_INV);
-            adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
-            raw += adc_buf;
-            raw_c++;
-        }
-#endif // BAT_MEASURE_ADC_UNIT
-
-#endif // End BAT_MEASURE_ADC_UNIT
         return (raw / (raw_c < 1 ? 1 : raw_c));
     }
 #endif
@@ -666,42 +720,31 @@ bool Power::analogInit()
 #ifdef ARCH_STM32WL
     analogReadResolution(BATTERY_SENSE_RESOLUTION_BITS);
 #elif defined(ARCH_ESP32) // ESP32 needs special analog stuff
+    adc_oneshot_unit_init_cfg_t init_config = {
+        .unit_id = unit,
+    };
 
-#ifndef ADC_WIDTH // max resolution by default
-    static const adc_bits_width_t width = ADC_WIDTH_BIT_12;
-#else
-    static const adc_bits_width_t width = ADC_WIDTH;
-#endif
-#ifndef BAT_MEASURE_ADC_UNIT // ADC1
-    adc1_config_width(width);
-    adc1_config_channel_atten(adc_channel, atten);
-#else                        // ADC2
-    adc2_config_channel_atten(adc_channel, atten);
-#ifndef CONFIG_IDF_TARGET_ESP32S3
-    // ADC2 wifi bug workaround
-    // Not required with ESP32S3, breaks compile
-    RTC_reg_b = READ_PERI_REG(SENS_SAR_READ_CTRL2_REG);
-#endif
-#endif
-    // calibrate ADC
-    esp_adc_cal_value_t val_type = esp_adc_cal_characterize(unit, atten, width, DEFAULT_VREF, adc_characs);
-    // show ADC characterization base
-    if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP) {
-        LOG_INFO("ADC config based on Two Point values stored in eFuse");
-    } else if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
-        LOG_INFO("ADC config based on reference voltage stored in eFuse");
+    if (!adc_handle) {
+        esp_err_t err = adc_oneshot_new_unit(&init_config, &adc_handle);
+        if (err != ESP_OK) {
+            LOG_ERROR("ADC oneshot init failed: %s", esp_err_to_name(err));
+            return false;
+        }
     }
-#ifdef CONFIG_IDF_TARGET_ESP32S3
-    // ESP32S3
-    else if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP_FIT) {
-        LOG_INFO("ADC config based on Two Point values and fitting curve "
-                 "coefficients stored in eFuse");
+
+    adc_oneshot_chan_cfg_t chan_cfg = {
+        .atten = atten,
+        .bitwidth = adc_width,
+    };
+
+    esp_err_t err = adc_oneshot_config_channel(adc_handle, adc_channel, &chan_cfg);
+    if (err != ESP_OK) {
+        LOG_ERROR("ADC channel config failed: %s", esp_err_to_name(err));
+        return false;
     }
-#endif
-    else {
-        LOG_INFO("ADC config based on default reference voltage");
-    }
-#endif // ARCH_ESP32
+
+    adc_calibrated = initAdcCalibration();
+#endif                    // ARCH_ESP32
 
     // NRF52 ADC init moved to powerHAL_init in nrf52 platform
 
@@ -781,8 +824,10 @@ void Power::reboot()
     rp2040.reboot();
 #elif defined(ARCH_PORTDUINO)
     deInitApiServer();
+#ifdef __linux__
     if (aLinuxInputImpl)
         aLinuxInputImpl->deInit();
+#endif
     SPI.end();
     Wire.end();
     Serial1.end();
@@ -902,7 +947,16 @@ void Power::readPowerStatus()
 
     // Notify any status instances that are observing us
     const PowerStatus powerStatus2 = PowerStatus(hasBattery, usbPowered, isChargingNow, batteryVoltageMv, batteryChargePercent);
-    if (millis() > lastLogTime + 50 * 1000) {
+
+    // Log battery-presence transitions once; skip OptUnknown so we don't lie before the first probe.
+    static OptionalBool prevHasBattery = OptUnknown;
+    if (hasBattery != OptUnknown && hasBattery != prevHasBattery) {
+        LOG_INFO("Power: battery hardware %s", hasBattery == OptTrue ? "detected" : "absent (USB-only)");
+        prevHasBattery = hasBattery;
+    }
+
+    // Periodic telemetry only emits when a battery is actually present (otherwise values are constant -1/0).
+    if (hasBattery == OptTrue && !Throttle::isWithinTimespanMs(lastLogTime, 50 * 1000)) {
         LOG_DEBUG("Battery: usbPower=%d, isCharging=%d, batMv=%d, batPct=%d", powerStatus2.getHasUSB(),
                   powerStatus2.getIsCharging(), powerStatus2.getBatteryVoltageMv(), powerStatus2.getBatteryChargePercent());
         lastLogTime = millis();

src/RedirectablePrint.cpp

@@ -230,9 +230,9 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
             auto thread = concurrency::OSThread::currentThread;
             meshtastic_LogRecord logRecord = meshtastic_LogRecord_init_zero;
             logRecord.level = getLogLevel(logLevel);
-            vsprintf(logRecord.message, format, arg);
+            vsnprintf(logRecord.message, sizeof(logRecord.message), format, arg);
             if (thread)
-                strcpy(logRecord.source, thread->ThreadName.c_str());
+                strlcpy(logRecord.source, thread->ThreadName.c_str(), sizeof(logRecord.source));
             logRecord.time = getValidTime(RTCQuality::RTCQualityDevice, true);
 
             auto buffer = std::unique_ptr<uint8_t[]>(new uint8_t[meshtastic_LogRecord_size]);

src/RedirectablePrint.h

@@ -1,8 +1,8 @@
 #pragma once
 
 #include "../freertosinc.h"
+#include "Print.h"
 #include "mesh/generated/meshtastic/mesh.pb.h"
-#include <Print.h>
 #include <stdarg.h>
 #include <string>
 

src/SafeFile.cpp

@@ -7,10 +7,6 @@ static File openFile(const char *filename, bool fullAtomic)
 {
     concurrency::LockGuard g(spiLock);
     LOG_DEBUG("Opening %s, fullAtomic=%d", filename, fullAtomic);
-#ifdef ARCH_NRF52
-    FSCom.remove(filename);
-    return FSCom.open(filename, FILE_O_WRITE);
-#endif
     if (!fullAtomic) {
         FSCom.remove(filename); // Nuke the old file to make space (ignore if it !exists)
     }
@@ -67,9 +63,6 @@ bool SafeFile::close()
     f.close();
     spiLock->unlock();
 
-#ifdef ARCH_NRF52
-    return true;
-#endif
     if (!testReadback())
         return false;
 

src/concurrency/Lock.cpp

@@ -14,6 +14,11 @@ Lock::Lock() : handle(xSemaphoreCreateBinary())
     }
 }
 
+Lock::~Lock()
+{
+    vSemaphoreDelete(handle);
+}
+
 void Lock::lock()
 {
     if (xSemaphoreTake(handle, portMAX_DELAY) == false) {
@@ -30,6 +35,8 @@ void Lock::unlock()
 #else
 Lock::Lock() {}
 
+Lock::~Lock() {}
+
 void Lock::lock() {}
 
 void Lock::unlock() {}

src/concurrency/Lock.h

@@ -12,6 +12,7 @@ class Lock
 {
   public:
     Lock();
+    ~Lock();
 
     Lock(const Lock &) = delete;
     Lock &operator=(const Lock &) = delete;

src/configuration.h

@@ -80,7 +80,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 // Pre-hop drop handling (compile-time flag).
 #ifndef MESHTASTIC_PREHOP_DROP
-#define MESHTASTIC_PREHOP_DROP 0
+#define MESHTASTIC_PREHOP_DROP 1
 #endif
 
 /// Convert a preprocessor name into a quoted string
@@ -162,6 +162,9 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #elif defined(HELTEC_MESH_NODE_T096)
 #define NUM_PA_POINTS 22
 #define TX_GAIN_LORA 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 13, 13, 13, 12, 11, 10, 9, 8, 7
+#elif defined(HELTEC_V4_R8)
+#define NUM_PA_POINTS 22
+#define TX_GAIN_LORA 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 12, 12, 11, 11, 10, 9, 8, 7
 #else
 // If a board enables USE_KCT8103L_PA but does not match a known variant and has
 // not already provided a PA curve, fail at compile time to avoid unsafe defaults.
@@ -558,5 +561,9 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define HAS_SCREEN 0
 #endif
 
+#ifndef USE_ETHERNET_DEFAULT
+#define USE_ETHERNET_DEFAULT 0
+#endif
+
 #include "DebugConfiguration.h"
 #include "RF95Configuration.h"

src/detect/LoRaRadioType.h

@@ -11,7 +11,8 @@ enum LoRaRadioType {
     SX1280_RADIO,
     LR1110_RADIO,
     LR1120_RADIO,
-    LR1121_RADIO
+    LR1121_RADIO,
+    LR2021_RADIO
 };
 
 extern LoRaRadioType radioType;

src/gps/GPS.cpp

@@ -1025,10 +1025,13 @@ void GPS::up()
     setPowerState(GPS_ACTIVE);
 }
 
-// We've got a GPS lock. Enter a low power state, potentially.
+// We've finished a GPS search cycle (lock or timeout). Enter a low power state, potentially.
 void GPS::down()
 {
-    scheduling.informGotLock();
+    if (hasValidLocation)
+        scheduling.informGotLock();
+    else
+        scheduling.informSearchFailed();
     uint32_t predictedSearchDuration = scheduling.predictedSearchDurationMs();
     uint32_t sleepTime = scheduling.msUntilNextSearch();
     uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval);
@@ -1553,7 +1556,12 @@ std::unique_ptr<GPS> GPS::createGps()
         _en_gpio = PIN_GPS_EN;
 #endif
 #ifdef ARCH_PORTDUINO
-    if (!portduino_config.has_gps)
+    if (portduino_config.has_gps) {
+        // These need to set as flags so later checks will pass on native and GPS will work.
+        // They are not used for any hardware access.
+        _rx_gpio = 1;
+        _tx_gpio = 1;
+    } else
         return nullptr;
 #endif
     if (!_rx_gpio || !_serial_gps) // Configured to have no GPS at all

src/gps/GPSUpdateScheduling.cpp

@@ -15,6 +15,19 @@ void GPSUpdateScheduling::informGotLock()
     searchEndedMs = millis();
     LOG_DEBUG("Took %us to get lock", (searchEndedMs - searchStartedMs) / 1000);
     updateLockTimePrediction();
+    consecutiveFailures = 0; // Drop back to fast cadence as soon as we acquire any fix
+}
+
+// Search finished without obtaining a fix. We still need to mark the end time so
+// the next sleep is timed correctly, but we must not feed the timeout duration
+// into predictedMsToGetLock — doing so poisons msUntilNextSearch() and causes
+// down() to fall into GPS_IDLE, leaving the chip awake on subsequent indoor cycles.
+void GPSUpdateScheduling::informSearchFailed()
+{
+    searchEndedMs = millis();
+    consecutiveFailures++;
+    LOG_DEBUG("GPS search ended without fix after %us (consecutive failures: %u)", (searchEndedMs - searchStartedMs) / 1000,
+              consecutiveFailures);
 }
 
 // Clear old lock-time prediction data.
@@ -25,6 +38,7 @@ void GPSUpdateScheduling::reset()
     searchEndedMs = 0;
     searchCount = 0;
     predictedMsToGetLock = 0;
+    consecutiveFailures = 0;
 }
 
 // How many milliseconds before we should next search for GPS position
@@ -36,6 +50,20 @@ uint32_t GPSUpdateScheduling::msUntilNextSearch()
     // Target interval (seconds), between GPS updates
     uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval, default_gps_update_interval);
 
+    // After a failed search, back off: indoors / no-sky environments will keep failing,
+    // so wake at most once per broadcast interval rather than once per gps_update_interval.
+    // Capped at 1 hour so a user-configured very-long broadcast interval still retries
+    // periodically (in case conditions change). Reset on any successful lock.
+    if (consecutiveFailures > 0) {
+        constexpr uint32_t failureRetryCapMs = 60UL * 60UL * 1000UL; // 1 hour cap
+        uint32_t failureSleepMs =
+            Default::getConfiguredOrDefaultMs(config.position.position_broadcast_secs, default_broadcast_interval_secs);
+        if (failureSleepMs > failureRetryCapMs)
+            failureSleepMs = failureRetryCapMs;
+        if (updateInterval < failureSleepMs)
+            updateInterval = failureSleepMs;
+    }
+
     // Check how long until we should start searching, to hopefully hit our target interval
     uint32_t dueAtMs = searchEndedMs + updateInterval;
     uint32_t compensatedStart = dueAtMs - predictedMsToGetLock;
@@ -70,20 +98,29 @@ bool GPSUpdateScheduling::isUpdateDue()
 // Have we been searching for a GPS position for too long?
 bool GPSUpdateScheduling::searchedTooLong()
 {
+    constexpr uint32_t oneMinuteMs = 60UL * 1000UL;
+    constexpr uint32_t maxSearchClampMs = 15UL * oneMinuteMs;   // Hard cap: 15 minutes is always too long
+    constexpr uint32_t postFailureSearchMs = 5UL * oneMinuteMs; // Tighter dwell once we know the environment is hostile
+    uint32_t elapsed = elapsedSearchMs();
+
+    // Anything over 15 minutes is too long, regardless of the broadcast interval.
+    if (elapsed > maxSearchClampMs)
+        return true;
+
+    // After a prior failed search, shorten the dwell
+    if (consecutiveFailures > 0 && elapsed > postFailureSearchMs)
+        return true;
+
     uint32_t minimumOrConfiguredSecs =
         Default::getConfiguredOrMinimumValue(config.position.position_broadcast_secs, default_broadcast_interval_secs);
     uint32_t maxSearchMs = Default::getConfiguredOrDefaultMs(minimumOrConfiguredSecs, default_broadcast_interval_secs);
-    // If broadcast interval set to max, no such thing as "too long"
-    if (maxSearchMs == UINT32_MAX)
-        return false;
 
     // If we've been searching longer than our position broadcast interval: that's too long
-    else if (elapsedSearchMs() > maxSearchMs)
+    if (elapsed > maxSearchMs)
         return true;
 
     // Otherwise, not too long yet!
-    else
-        return false;
+    return false;
 }
 
 // Updates the predicted time-to-get-lock, by exponentially smoothing the latest observation

src/gps/GPSUpdateScheduling.h

@@ -8,7 +8,8 @@ class GPSUpdateScheduling
   public:
     // Marks the time of these events, for calculation use
     void informSearching();
-    void informGotLock(); // Predicted lock-time is recalculated here
+    void informGotLock();      // Predicted lock-time is recalculated here
+    void informSearchFailed(); // Search ended without a fix; prediction is left untouched
 
     void reset();           // Reset the prediction - after GPS::disable() / GPS::enable()
     bool isUpdateDue();     // Is it time to begin searching for a GPS position?
@@ -24,6 +25,7 @@ class GPSUpdateScheduling
     uint32_t searchEndedMs = 0;
     uint32_t searchCount = 0;
     uint32_t predictedMsToGetLock = 0;
+    uint32_t consecutiveFailures = 0; // Count of search cycles that ended without a fix; reset on lock
 
     const float weighting = 0.2; // Controls exponential smoothing of lock-times prediction. 20% weighting of "latest lock-time".
 };

src/graphics/Screen.cpp

@@ -65,7 +65,6 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #include "mesh/Default.h"
 #include "mesh/generated/meshtastic/deviceonly.pb.h"
 #include "modules/ExternalNotificationModule.h"
-#include "modules/TextMessageModule.h"
 #include "modules/WaypointModule.h"
 #include "sleep.h"
 #include "target_specific.h"
@@ -231,6 +230,7 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
 #endif
     // Store the message and set the expiration timestamp
     strncpy(NotificationRenderer::alertBannerMessage, banner_overlay_options.message, 255);
+    NotificationRenderer::parseBannerMessageWithFonts(NotificationRenderer::alertBannerMessage);
     NotificationRenderer::alertBannerMessage[255] = '\0'; // Ensure null termination
     NotificationRenderer::alertBannerUntil =
         (banner_overlay_options.durationMs == 0) ? 0 : millis() + banner_overlay_options.durationMs;
@@ -240,9 +240,9 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
     NotificationRenderer::alertBannerCallback = banner_overlay_options.bannerCallback;
     NotificationRenderer::curSelected = banner_overlay_options.InitialSelected;
     NotificationRenderer::pauseBanner = false;
-    NotificationRenderer::current_notification_type = notificationTypeEnum::selection_picker;
+    NotificationRenderer::current_notification_type = banner_overlay_options.notificationType;
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -264,7 +264,7 @@ void Screen::showNodePicker(const char *message, uint32_t durationMs, std::funct
     NotificationRenderer::current_notification_type = notificationTypeEnum::node_picker;
 
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -288,7 +288,7 @@ void Screen::showNumberPicker(const char *message, uint32_t durationMs, uint8_t
     NotificationRenderer::currentNumber = 0;
 
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -311,7 +311,7 @@ void Screen::showTextInput(const char *header, const char *initialText, uint32_t
 
     // Set the overlay using the same pattern as other notification types
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -524,6 +524,11 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
             delay(100);
 #endif
 #if !ARCH_PORTDUINO
+#if defined(USE_ST7789) && defined(VTFT_CTRL)
+            // Ensure panel power rail is enabled before sending wake commands.
+            pinMode(VTFT_CTRL, OUTPUT);
+            digitalWrite(VTFT_CTRL, LOW);
+#endif
             dispdev->displayOn();
 #endif
 
@@ -545,10 +550,6 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
             ui->init();
 #endif
 #if defined(USE_ST7789) && defined(VTFT_LEDA)
-#ifdef VTFT_CTRL
-            pinMode(VTFT_CTRL, OUTPUT);
-            digitalWrite(VTFT_CTRL, LOW);
-#endif
             ui->init();
 #ifdef ESP_PLATFORM
             analogWrite(VTFT_LEDA, BRIGHTNESS_DEFAULT);
@@ -589,23 +590,22 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
 #endif
 #ifdef USE_ST7789
             SPI1.end();
-#if defined(ARCH_ESP32)
+            // Keep TFT control pins in deterministic states while timed-off.
+            // Floating/default pin states can corrupt panel edge rows on wake.
 #ifdef VTFT_LEDA
-            pinMode(VTFT_LEDA, ANALOG);
+            pinMode(VTFT_LEDA, OUTPUT);
+            digitalWrite(VTFT_LEDA, !TFT_BACKLIGHT_ON);
 #endif
 #ifdef VTFT_CTRL
-            pinMode(VTFT_CTRL, ANALOG);
-#endif
-            pinMode(ST7789_RESET, ANALOG);
-            pinMode(ST7789_RS, ANALOG);
-            pinMode(ST7789_NSS, ANALOG);
-#else
-            nrf_gpio_cfg_default(VTFT_LEDA);
-            nrf_gpio_cfg_default(VTFT_CTRL);
-            nrf_gpio_cfg_default(ST7789_RESET);
-            nrf_gpio_cfg_default(ST7789_RS);
-            nrf_gpio_cfg_default(ST7789_NSS);
+            pinMode(VTFT_CTRL, OUTPUT);
+            digitalWrite(VTFT_CTRL, HIGH);
 #endif
+            pinMode(ST7789_RESET, OUTPUT);
+            digitalWrite(ST7789_RESET, HIGH);
+            pinMode(ST7789_RS, OUTPUT);
+            digitalWrite(ST7789_RS, HIGH);
+            pinMode(ST7789_NSS, OUTPUT);
+            digitalWrite(ST7789_NSS, HIGH);
 #endif
 #ifdef USE_ST7796
             SPI1.end();
@@ -696,7 +696,7 @@ void Screen::setup()
     static OverlayCallback overlays[] = {
         graphics::UIRenderer::drawNavigationBar // Custom indicator icons for each frame
     };
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 1);
 
     // Enable UTF-8 to display mapping
     dispdev->setFontTableLookupFunction(customFontTableLookup);
@@ -1285,7 +1285,7 @@ void Screen::setFrames(FrameFocus focus)
 
         for (size_t i = 0; i < nodeDB->getNumMeshNodes(); i++) {
             const meshtastic_NodeInfoLite *n = nodeDB->getMeshNodeByIndex(i);
-            if (n && n->num != nodeDB->getNodeNum() && n->is_favorite) {
+            if (n && n->num != nodeDB->getNodeNum() && nodeInfoLiteIsFavorite(n)) {
                 favoriteFrames.push_back(graphics::UIRenderer::drawFavoriteNode);
             }
         }
@@ -1313,7 +1313,7 @@ void Screen::setFrames(FrameFocus focus)
 
     // Add overlays: frame icons and alert banner)
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
 
     prevFrame = -1; // Force drawFavoriteNode to pick a new node (because our list just changed)
 
@@ -1643,138 +1643,6 @@ int Screen::handleStatusUpdate(const meshtastic::Status *arg)
     return 0;
 }
 
-// Handles when message is received; will jump to text message frame.
-int Screen::handleTextMessage(const meshtastic_MeshPacket *packet)
-{
-    if (showingNormalScreen) {
-        if (packet->from == 0) {
-            // Outgoing message (likely sent from phone)
-            devicestate.has_rx_text_message = false;
-            memset(&devicestate.rx_text_message, 0, sizeof(devicestate.rx_text_message));
-            hiddenFrames.textMessage = true;
-            hasUnreadMessage = false; // Clear unread state when user replies
-
-            setFrames(FOCUS_PRESERVE); // Stay on same frame, silently update frame list
-        } else {
-            // Incoming message
-            devicestate.has_rx_text_message = true; // Needed to include the message frame
-            hasUnreadMessage = true;                // Enables mail icon in the header
-            setFrames(FOCUS_PRESERVE);              // Refresh frame list without switching view (no-op during text_input)
-
-            // Only wake/force display if the configuration allows it
-            if (shouldWakeOnReceivedMessage()) {
-                setOn(true);    // Wake up the screen first
-                forceDisplay(); // Forces screen redraw
-            }
-            // === Prepare banner/popup content ===
-            const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet->from);
-            const meshtastic_Channel channel =
-                channels.getByIndex(packet->channel ? packet->channel : channels.getPrimaryIndex());
-            const char *longName = (node && node->has_user) ? node->user.long_name : nullptr;
-
-            const char *msgRaw = reinterpret_cast<const char *>(packet->decoded.payload.bytes);
-
-            char banner[256];
-
-            bool isAlert = false;
-
-            if (moduleConfig.external_notification.alert_bell || moduleConfig.external_notification.alert_bell_vibra ||
-                moduleConfig.external_notification.alert_bell_buzzer)
-                // Check for bell character to determine if this message is an alert
-                for (size_t i = 0; i < packet->decoded.payload.size && i < 100; i++) {
-                    if (msgRaw[i] == ASCII_BELL) {
-                        isAlert = true;
-                        break;
-                    }
-                }
-
-            // Unlike generic messages, alerts (when enabled via the ext notif module) ignore any
-            // 'mute' preferences set to any specific node or channel.
-            // If on-screen keyboard is active, show a transient popup over keyboard instead of interrupting it
-            if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
-                // Wake and force redraw so popup is visible immediately
-                if (shouldWakeOnReceivedMessage()) {
-                    setOn(true);
-                    forceDisplay();
-                }
-
-                // Build popup: title = message source name, content = message text (sanitized)
-                // Title
-                char titleBuf[64] = {0};
-                if (longName && longName[0]) {
-                    // Sanitize sender name
-                    std::string t = sanitizeString(longName);
-                    strncpy(titleBuf, t.c_str(), sizeof(titleBuf) - 1);
-                } else {
-                    strncpy(titleBuf, "Message", sizeof(titleBuf) - 1);
-                }
-
-                // Content: payload bytes may not be null-terminated, remove ASCII_BELL and sanitize
-                char content[256] = {0};
-                {
-                    std::string raw;
-                    raw.reserve(packet->decoded.payload.size);
-                    for (size_t i = 0; i < packet->decoded.payload.size; ++i) {
-                        char c = msgRaw[i];
-                        if (c == ASCII_BELL)
-                            continue; // strip bell
-                        raw.push_back(c);
-                    }
-                    std::string sanitized = sanitizeString(raw);
-                    strncpy(content, sanitized.c_str(), sizeof(content) - 1);
-                }
-
-                NotificationRenderer::showKeyboardMessagePopupWithTitle(titleBuf, content, 3000);
-
-// Maintain existing buzzer behavior on M5 if applicable
-#if defined(M5STACK_UNITC6L)
-                if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
-                    (isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
-                    (!isBroadcast(packet->to) && isToUs(packet))) {
-                    playLongBeep();
-                }
-#endif
-            } else {
-                // No keyboard active: use regular banner flow, respecting mute settings
-                if (isAlert) {
-                    if (longName && longName[0]) {
-                        snprintf(banner, sizeof(banner), "Alert Received from\n%s", longName);
-                    } else {
-                        strcpy(banner, "Alert Received");
-                    }
-                    screen->showSimpleBanner(banner, 3000);
-                } else if (!channel.settings.has_module_settings || !channel.settings.module_settings.is_muted) {
-                    if (longName && longName[0]) {
-                        if (currentResolution == ScreenResolution::UltraLow) {
-                            strcpy(banner, "New Message");
-                        } else {
-                            snprintf(banner, sizeof(banner), "New Message from\n%s", longName);
-                        }
-                    } else {
-                        strcpy(banner, "New Message");
-                    }
-#if defined(M5STACK_UNITC6L)
-                    screen->setOn(true);
-                    screen->showSimpleBanner(banner, 1500);
-                    if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
-                        (isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
-                        (!isBroadcast(packet->to) && isToUs(packet))) {
-                        // Beep if not in DIRECT_MSG_ONLY mode or if in DIRECT_MSG_ONLY mode and either
-                        // - packet contains an alert and alert bell buzzer is enabled
-                        // - packet is a non-broadcast that is addressed to this node
-                        playLongBeep();
-                    }
-#else
-                    screen->showSimpleBanner(banner, 3000);
-#endif
-                }
-            }
-        }
-    }
-
-    return 0;
-}
-
 // Triggered by MeshModules
 int Screen::handleUIFrameEvent(const UIFrameEvent *event)
 {
@@ -1820,7 +1688,7 @@ int Screen::handleInputEvent(const InputEvent *event)
     if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
         NotificationRenderer::inEvent = *event;
         static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-        ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+        ui->setOverlays(overlays, 2);
         setFastFramerate(); // Draw ASAP
         updateUiFrame(ui);
         return 0;
@@ -1835,7 +1703,7 @@ int Screen::handleInputEvent(const InputEvent *event)
     if (NotificationRenderer::isOverlayBannerShowing()) {
         NotificationRenderer::inEvent = *event;
         static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-        ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+        ui->setOverlays(overlays, 2);
         setFastFramerate(); // Draw ASAP
         updateUiFrame(ui);
 

src/graphics/Screen.h

@@ -609,7 +609,6 @@ class Screen : public concurrency::OSThread
 
     // Handle observer events
     int handleStatusUpdate(const meshtastic::Status *arg);
-    int handleTextMessage(const meshtastic_MeshPacket *packet);
     int handleUIFrameEvent(const UIFrameEvent *arg);
     int handleInputEvent(const InputEvent *arg);
     int handleAdminMessage(AdminModule_ObserverData *arg);

src/graphics/ScreenFonts.h

@@ -88,6 +88,16 @@
 #endif
 #endif
 
+// nRF52 flash optimization: re-route FONT_LARGE_LOCAL to the 16pt glyph so
+// the display-tier dispatch below picks up 16pt everywhere it would have used
+// 24pt. Drops the ~9.6 KB ArialMT_Plain_24 table from the linked binary.
+// Set MESHTASTIC_LARGE_FONT_24PT=1 in build_flags to opt out per variant
+// (undefined or 0 keeps the optimization on).
+#if defined(ARCH_NRF52) && (!defined(MESHTASTIC_LARGE_FONT_24PT) || MESHTASTIC_LARGE_FONT_24PT == 0)
+#undef FONT_LARGE_LOCAL
+#define FONT_LARGE_LOCAL FONT_MEDIUM_LOCAL
+#endif
+
 #if (defined(USE_EINK) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || defined(ST7701_CS) || defined(ST7735_CS) ||      \
      defined(ST7789_CS) || defined(USE_ST7789) || defined(HX8357_CS) || defined(ILI9488_CS) || defined(ST7796_CS) ||             \
      defined(USE_ST7796) || defined(HACKADAY_COMMUNICATOR)) &&                                                                   \

src/graphics/TFTDisplay.cpp

@@ -422,7 +422,7 @@ static LGFX *tft = nullptr;
 
 #elif defined(ST7789_CS)
 #include <LovyanGFX.hpp> // Graphics and font library for ST7735 driver chip
-#ifdef HELTEC_V4_TFT
+#if defined(HELTEC_V4_TFT) || defined(HELTEC_V4_R8_TFT)
 #include "chsc6x.h"
 #include "lgfx/v1/Touch.hpp"
 namespace lgfx
@@ -444,7 +444,11 @@ class TOUCH_CHSC6X : public ITouch
     bool init(void) override
     {
         if (chsc6xTouch == nullptr) {
+#if (TOUCH_I2C_PORT == 1)
             chsc6xTouch = new chsc6x(&Wire1, TOUCH_SDA_PIN, TOUCH_SCL_PIN, TOUCH_INT_PIN, TOUCH_RST_PIN);
+#else
+            chsc6xTouch = new chsc6x(&Wire, TOUCH_SDA_PIN, TOUCH_SCL_PIN, TOUCH_INT_PIN, TOUCH_RST_PIN);
+#endif
         }
         chsc6xTouch->chsc6x_init();
         return true;
@@ -481,7 +485,7 @@ class LGFX : public lgfx::LGFX_Device
 #if HAS_TOUCHSCREEN
 #if defined(T_WATCH_S3) || defined(ELECROW)
     lgfx::Touch_FT5x06 _touch_instance;
-#elif defined(HELTEC_V4_TFT)
+#elif defined(HELTEC_V4_TFT) || defined(HELTEC_V4_R8_TFT)
     lgfx::TOUCH_CHSC6X _touch_instance;
 #else
     lgfx::Touch_GT911 _touch_instance;
@@ -500,7 +504,11 @@ class LGFX : public lgfx::LGFX_Device
             cfg.freq_write = SPI_FREQUENCY; // SPI clock for transmission (up to 80MHz, rounded to the value obtained by dividing
                                             // 80MHz by an integer)
             cfg.freq_read = SPI_READ_FREQUENCY; // SPI clock when receiving
-            cfg.spi_3wire = false;
+#ifdef SPI_3_WIRE
+            cfg.spi_3wire = SPI_3_WIRE;
+#else
+            cfg.spi_3wire = true;                      // Set to true if reception is done on the MOSI pin
+#endif
             cfg.use_lock = true;               // Set to true to use transaction locking
             cfg.dma_channel = SPI_DMA_CH_AUTO; // SPI_DMA_CH_AUTO; // Set DMA channel to use (0=not use DMA / 1=1ch / 2=ch /
                                                // SPI_DMA_CH_AUTO=auto setting)
@@ -550,8 +558,11 @@ class LGFX : public lgfx::LGFX_Device
             cfg.rgb_order = false;                        // Set to true if the panel's red and blue are swapped
             cfg.dlen_16bit =
                 false;             // Set to true for panels that transmit data length in 16-bit units with 16-bit parallel or SPI
+#if defined(HAS_SDCARD)
             cfg.bus_shared = true; // If the bus is shared with the SD card, set to true (bus control with drawJpgFile etc.)
-
+#else
+            cfg.bus_shared = false;
+#endif
             // Set the following only when the display is shifted with a driver with a variable number of pixels, such as the
             // ST7735 or ILI9163.
             // cfg.memory_width = TFT_WIDTH;   // Maximum width supported by the driver IC

src/graphics/draw/MenuHandler.cpp

@@ -57,6 +57,70 @@ BannerOverlayOptions createStaticBannerOptions(const char *message, const MenuOp
     return bannerOptions;
 }
 
+const StoredMessage *getNewestMessageForActiveThread()
+{
+    const auto &messages = messageStore.getMessages();
+    if (messages.empty()) {
+        return nullptr;
+    }
+
+    const auto mode = graphics::MessageRenderer::getThreadMode();
+    const int channel = graphics::MessageRenderer::getThreadChannel();
+    const uint32_t peer = graphics::MessageRenderer::getThreadPeer();
+    const uint32_t localNode = nodeDB->getNodeNum();
+
+    if (mode == graphics::MessageRenderer::ThreadMode::ALL) {
+        return &messages.back();
+    }
+
+    for (auto it = messages.rbegin(); it != messages.rend(); ++it) {
+        const StoredMessage &m = *it;
+
+        if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
+            if (m.type == MessageType::BROADCAST && static_cast<int>(m.channelIndex) == channel) {
+                return &m;
+            }
+            continue;
+        }
+
+        if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
+            if (m.type != MessageType::DM_TO_US) {
+                continue;
+            }
+            const uint32_t other = (m.sender == localNode) ? m.dest : m.sender;
+            if (other == peer) {
+                return &m;
+            }
+        }
+    }
+
+    return nullptr;
+}
+
+void launchReplyForMessage(const StoredMessage &message, bool freetext)
+{
+    if (message.type == MessageType::BROADCAST || message.dest == NODENUM_BROADCAST) {
+        if (freetext) {
+            cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, message.channelIndex);
+        } else {
+            cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, message.channelIndex);
+        }
+        return;
+    }
+
+    const uint32_t localNode = nodeDB->getNodeNum();
+    const uint32_t peer = (message.sender == localNode) ? message.dest : message.sender;
+    if (peer == 0 || peer == NODENUM_BROADCAST) {
+        return;
+    }
+
+    if (freetext) {
+        cannedMessageModule->LaunchFreetextWithDestination(peer);
+    } else {
+        cannedMessageModule->LaunchWithDestination(peer);
+    }
+}
+
 } // namespace
 
 menuHandler::screenMenus menuHandler::menuQueue = MenuNone;
@@ -594,9 +658,12 @@ void menuHandler::messageResponseMenu()
 
 #ifdef HAS_I2S
         } else if (selected == Aloud) {
-            const meshtastic_MeshPacket &mp = devicestate.rx_text_message;
-            const char *msg = reinterpret_cast<const char *>(mp.decoded.payload.bytes);
-            audioThread->readAloud(msg);
+            if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
+                const char *msg = MessageStore::getText(*latest);
+                if (msg && msg[0]) {
+                    audioThread->readAloud(msg);
+                }
+            }
 #endif
         }
     };
@@ -656,20 +723,12 @@ void menuHandler::replyMenu()
 
         // Preset reply
         if (selected == ReplyPreset) {
-
             if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
                 cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, ch);
-
             } else if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
                 cannedMessageModule->LaunchWithDestination(peer);
-
-            } else {
-                // Fallback for last received message
-                if (devicestate.rx_text_message.to == NODENUM_BROADCAST) {
-                    cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, devicestate.rx_text_message.channel);
-                } else {
-                    cannedMessageModule->LaunchWithDestination(devicestate.rx_text_message.from);
-                }
+            } else if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
+                launchReplyForMessage(*latest, false);
             }
 
             return;
@@ -677,20 +736,12 @@ void menuHandler::replyMenu()
 
         // Freetext reply
         if (selected == ReplyFreetext) {
-
             if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
                 cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, ch);
-
             } else if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
                 cannedMessageModule->LaunchFreetextWithDestination(peer);
-
-            } else {
-                // Fallback for last received message
-                if (devicestate.rx_text_message.to == NODENUM_BROADCAST) {
-                    cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, devicestate.rx_text_message.channel);
-                } else {
-                    cannedMessageModule->LaunchFreetextWithDestination(devicestate.rx_text_message.from);
-                }
+            } else if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
+                launchReplyForMessage(*latest, true);
             }
 
             return;
@@ -845,10 +896,10 @@ void menuHandler::messageViewModeMenu()
     // Encode peers
     for (size_t i = 0; i < uniquePeers.size(); ++i) {
         uint32_t peer = uniquePeers[i];
-        auto node = nodeDB->getMeshNode(peer);
+        const auto *node = nodeDB->getMeshNode(peer);
         std::string name;
-        if (node && node->has_user)
-            name = sanitizeString(node->user.long_name).substr(0, 15);
+        if (nodeInfoLiteHasUser(node))
+            name = sanitizeString(node->long_name).substr(0, 15);
         else {
             char buf[20];
             snprintf(buf, sizeof(buf), "Node %08X", peer);
@@ -1355,14 +1406,14 @@ void menuHandler::manageNodeMenu()
     static int optionsEnumArray[enumEnd] = {Back};
     int options = 1;
 
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         optionsArray[options] = "Unfavorite";
     } else {
         optionsArray[options] = "Favorite";
     }
     optionsEnumArray[options++] = Favorite;
 
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
     if (isMuted) {
         optionsArray[options] = "Unmute Notifications";
     } else {
@@ -1376,7 +1427,7 @@ void menuHandler::manageNodeMenu()
     optionsArray[options] = "Key Verification";
     optionsEnumArray[options++] = KeyVerification;
 
-    if (node->is_ignored) {
+    if (nodeInfoLiteIsIgnored(node)) {
         optionsArray[options] = "Unignore Node";
     } else {
         optionsArray[options] = "Ignore Node";
@@ -1386,8 +1437,8 @@ void menuHandler::manageNodeMenu()
     BannerOverlayOptions bannerOptions;
 
     std::string title = "";
-    if (node->has_user && node->user.long_name && node->user.long_name[0]) {
-        title += sanitizeString(node->user.long_name).substr(0, 15);
+    if (nodeInfoLiteHasUser(node) && node->long_name[0]) {
+        title += sanitizeString(node->long_name).substr(0, 15);
     } else {
         char buf[20];
         snprintf(buf, sizeof(buf), "%08X", (unsigned int)node->num);
@@ -1409,7 +1460,7 @@ void menuHandler::manageNodeMenu()
             if (!n) {
                 return;
             }
-            if (n->is_favorite) {
+            if (nodeInfoLiteIsFavorite(n)) {
                 LOG_INFO("Removing node %08X from favorites", menuHandler::pickedNodeNum);
                 nodeDB->set_favorite(false, menuHandler::pickedNodeNum);
             } else {
@@ -1426,13 +1477,9 @@ void menuHandler::manageNodeMenu()
                 return;
             }
 
-            if (n->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) {
-                n->bitfield &= ~NODEINFO_BITFIELD_IS_MUTED_MASK;
-                LOG_INFO("Unmuted node %08X", menuHandler::pickedNodeNum);
-            } else {
-                n->bitfield |= NODEINFO_BITFIELD_IS_MUTED_MASK;
-                LOG_INFO("Muted node %08X", menuHandler::pickedNodeNum);
-            }
+            const bool wasMuted = nodeInfoLiteIsMuted(n);
+            nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_MUTED_MASK, !wasMuted);
+            LOG_INFO(wasMuted ? "Unmuted node %08X" : "Muted node %08X", menuHandler::pickedNodeNum);
             nodeDB->notifyObservers(true);
             nodeDB->saveToDisk();
             screen->setFrames(graphics::Screen::FOCUS_PRESERVE);
@@ -1461,11 +1508,11 @@ void menuHandler::manageNodeMenu()
                 return;
             }
 
-            if (n->is_ignored) {
-                n->is_ignored = false;
+            if (nodeInfoLiteIsIgnored(n)) {
+                nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, false);
                 LOG_INFO("Unignoring node %08X", menuHandler::pickedNodeNum);
             } else {
-                n->is_ignored = true;
+                nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
                 LOG_INFO("Ignoring node %08X", menuHandler::pickedNodeNum);
             }
             nodeDB->notifyObservers(true);
@@ -2079,9 +2126,9 @@ void menuHandler::removeFavoriteMenu()
     static const char *optionsArray[] = {"Back", "Yes"};
     BannerOverlayOptions bannerOptions;
     std::string message = "Unfavorite This Node?\n";
-    auto node = nodeDB->getMeshNode(graphics::UIRenderer::currentFavoriteNodeNum);
-    if (node && node->has_user) {
-        message += sanitizeString(node->user.long_name).substr(0, 15);
+    const auto *node = nodeDB->getMeshNode(graphics::UIRenderer::currentFavoriteNodeNum);
+    if (nodeInfoLiteHasUser(node)) {
+        message += sanitizeString(node->long_name).substr(0, 15);
     }
     bannerOptions.message = message.c_str();
     bannerOptions.optionsArrayPtr = optionsArray;

src/graphics/draw/MessageRenderer.cpp

@@ -462,8 +462,8 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
     }
     case ThreadMode::DIRECT: {
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(currentPeer);
-        if (node && node->has_user && node->user.short_name[0]) {
-            snprintf(titleStr, sizeof(titleStr), "@%s", node->user.short_name);
+        if (nodeInfoLiteHasUser(node) && node->short_name[0]) {
+            snprintf(titleStr, sizeof(titleStr), "@%s", node->short_name);
         } else {
             snprintf(titleStr, sizeof(titleStr), "@%08x", currentPeer);
         }
@@ -585,11 +585,11 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
         meshtastic_NodeInfoLite *node_recipient = nodeDB->getMeshNode(m.dest);
 
         char senderName[64] = "";
-        if (node && node->has_user) {
-            if (node->user.long_name[0]) {
-                strncpy(senderName, node->user.long_name, sizeof(senderName) - 1);
-            } else if (node->user.short_name[0]) {
-                strncpy(senderName, node->user.short_name, sizeof(senderName) - 1);
+        if (nodeInfoLiteHasUser(node)) {
+            if (node->long_name[0]) {
+                strncpy(senderName, node->long_name, sizeof(senderName) - 1);
+            } else if (node->short_name[0]) {
+                strncpy(senderName, node->short_name, sizeof(senderName) - 1);
             }
             senderName[sizeof(senderName) - 1] = '\0';
         }
@@ -599,18 +599,17 @@ void drawTextMessageFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
 
         // If this is *our own* message, override senderName to who the recipient was
         bool mine = (m.sender == nodeDB->getNodeNum());
-        if (mine && node_recipient && node_recipient->has_user) {
-            if (node_recipient->user.long_name[0]) {
-                strncpy(senderName, node_recipient->user.long_name, sizeof(senderName) - 1);
+        if (mine && nodeInfoLiteHasUser(node_recipient)) {
+            if (node_recipient->long_name[0]) {
+                strncpy(senderName, node_recipient->long_name, sizeof(senderName) - 1);
                 senderName[sizeof(senderName) - 1] = '\0';
-            } else if (node_recipient->user.short_name[0]) {
-                strncpy(senderName, node_recipient->user.short_name, sizeof(senderName) - 1);
+            } else if (node_recipient->short_name[0]) {
+                strncpy(senderName, node_recipient->short_name, sizeof(senderName) - 1);
                 senderName[sizeof(senderName) - 1] = '\0';
             }
         }
         // If recipient info is missing/empty, prefer a recipient identifier for outbound messages.
-        if (mine && (!node_recipient || !node_recipient->has_user ||
-                     (!node_recipient->user.long_name[0] && !node_recipient->user.short_name[0]))) {
+        if (mine && (!nodeInfoLiteHasUser(node_recipient) || (!node_recipient->long_name[0] && !node_recipient->short_name[0]))) {
             snprintf(senderName, sizeof(senderName), "(%08x)", m.dest);
         }
 
@@ -1073,12 +1072,12 @@ void handleNewMessage(OLEDDisplay *display, const StoredMessage &sm, const mesht
         // Banner logic
         const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet.from);
         char longName[64] = "?";
-        if (node && node->has_user) {
-            if (node->user.long_name[0]) {
-                strncpy(longName, node->user.long_name, sizeof(longName) - 1);
+        if (nodeInfoLiteHasUser(node)) {
+            if (node->long_name[0]) {
+                strncpy(longName, node->long_name, sizeof(longName) - 1);
                 longName[sizeof(longName) - 1] = '\0';
-            } else if (node->user.short_name[0]) {
-                strncpy(longName, node->user.short_name, sizeof(longName) - 1);
+            } else if (node->short_name[0]) {
+                strncpy(longName, node->short_name, sizeof(longName) - 1);
                 longName[sizeof(longName) - 1] = '\0';
             }
         }

src/graphics/draw/NodeListRenderer.cpp

@@ -98,29 +98,23 @@ std::string getSafeNodeName(OLEDDisplay *display, meshtastic_NodeInfoLite *node,
     // 1) Choose target candidate (long vs short) only if present
     const char *raw = nullptr;
 
-#if !MESHTASTIC_EXCLUDE_STATUS
+#if !MESHTASTIC_EXCLUDE_STATUS && !MESHTASTIC_EXCLUDE_STATUSDB
     // If long-name mode is enabled, and we have a recent status for this node,
-    // prefer "(short_name) statusText" as the raw candidate.
+    // prefer "(short_name) statusText" as the raw candidate. Pull straight out
+    // of NodeDB's per-NodeNum cache instead of scanning a FIFO.
     std::string composedFromStatus;
-    if (config.display.use_long_node_name && node && node->has_user && statusMessageModule) {
-        const auto &recent = statusMessageModule->getRecentReceived();
-        const StatusMessageModule::RecentStatus *found = nullptr;
-        for (auto it = recent.rbegin(); it != recent.rend(); ++it) {
-            if (it->fromNodeId == node->num && !it->statusText.empty()) {
-                found = &(*it);
-                break;
-            }
-        }
-
-        if (found) {
-            const char *shortName = node->user.short_name;
-            composedFromStatus.reserve(4 + (shortName ? std::strlen(shortName) : 0) + 1 + found->statusText.size());
+    if (config.display.use_long_node_name && nodeInfoLiteHasUser(node) && nodeDB) {
+        meshtastic_StatusMessage cachedStatus;
+        if (nodeDB->copyNodeStatus(node->num, cachedStatus) && cachedStatus.status[0]) {
+            const char *shortName = node->short_name;
+            const size_t statusLen = std::strlen(cachedStatus.status);
+            composedFromStatus.reserve(4 + (shortName ? std::strlen(shortName) : 0) + 1 + statusLen);
             composedFromStatus += "(";
             if (shortName && *shortName) {
                 composedFromStatus += shortName;
             }
             composedFromStatus += ") ";
-            composedFromStatus += found->statusText;
+            composedFromStatus += cachedStatus.status;
 
             raw = composedFromStatus.c_str(); // safe for now; we'll sanitize immediately into std::string
         }
@@ -129,8 +123,8 @@ std::string getSafeNodeName(OLEDDisplay *display, meshtastic_NodeInfoLite *node,
 
     // If we didn't compose from status, use normal long/short selection
     if (!raw) {
-        if (node && node->has_user) {
-            raw = config.display.use_long_node_name ? node->user.long_name : node->user.short_name;
+        if (nodeInfoLiteHasUser(node)) {
+            raw = config.display.use_long_node_name ? node->long_name : node->short_name;
         }
     }
 
@@ -218,7 +212,7 @@ void drawScrollbar(OLEDDisplay *display, int visibleNodeRows, int totalEntries,
 static inline void applyFavoriteNodeNameColor(OLEDDisplay *display, const meshtastic_NodeInfoLite *node, const char *nodeName,
                                               int16_t nameX, int16_t y, int nameMaxWidth)
 {
-    if (!display || !node || !node->is_favorite || !isTFTColoringEnabled() || !nodeName) {
+    if (!display || !node || !nodeInfoLiteIsFavorite(node) || !isTFTColoringEnabled() || !nodeName) {
         return;
     }
 
@@ -259,7 +253,7 @@ void drawEntryLastHeard(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     char timeStr[10];
     uint32_t seconds = sinceLastSeen(node);
@@ -279,14 +273,14 @@ void drawEntryLastHeard(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -321,20 +315,20 @@ void drawEntryHopSignal(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
 
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -401,15 +395,19 @@ void drawNodeDistance(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
     char distStr[10] = "";
 
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node)) {
-        double lat1 = ourNode->position.latitude_i * 1e-7;
-        double lon1 = ourNode->position.longitude_i * 1e-7;
-        double lat2 = node->position.latitude_i * 1e-7;
-        double lon2 = node->position.longitude_i * 1e-7;
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourPos;
+    meshtastic_PositionLite theirPos;
+    const bool haveOurPos = ourNode && nodeDB->copyNodePosition(ourNode->num, ourPos);
+    const bool haveTheirPos = nodeDB->copyNodePosition(node->num, theirPos);
+    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node) && haveOurPos && haveTheirPos) {
+        double lat1 = ourPos.latitude_i * 1e-7;
+        double lon1 = ourPos.longitude_i * 1e-7;
+        double lat2 = theirPos.latitude_i * 1e-7;
+        double lon2 = theirPos.longitude_i * 1e-7;
 
         double earthRadiusKm = 6371.0;
         double dLat = (lat2 - lat1) * DEG_TO_RAD;
@@ -455,14 +453,14 @@ void drawNodeDistance(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -509,19 +507,19 @@ void drawEntryCompass(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
 #if GRAPHICS_TFT_COLORING_ENABLED
     applyFavoriteNodeNameColor(display, node, nodeName, nameX, y, nameMaxWidth);
 #endif
-    bool isMuted = (node->bitfield & NODEINFO_BITFIELD_IS_MUTED_MASK) != 0;
+    bool isMuted = nodeInfoLiteIsMuted(node);
 
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     UIRenderer::drawStringWithEmotes(display, nameX, y, nodeName, FONT_HEIGHT_SMALL, 1, false);
-    if (node->is_favorite) {
+    if (nodeInfoLiteIsFavorite(node)) {
         if (currentResolution == ScreenResolution::High) {
             drawScaledXBitmap16x16(x, y + 6, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint, display);
         } else {
             display->drawXbm(x, y + 5, smallbulletpoint_width, smallbulletpoint_height, smallbulletpoint);
         }
     }
-    if (node->is_ignored || isMuted) {
+    if (nodeInfoLiteIsIgnored(node) || isMuted) {
         if (currentResolution == ScreenResolution::High) {
             display->drawLine(x + 8, y + 8, (isLeftCol ? 0 : x - 4) + nameMaxWidth - 17, y + 8);
         } else {
@@ -542,8 +540,11 @@ void drawCompassArrow(OLEDDisplay *display, meshtastic_NodeInfoLite *node, int16
     int centerX = x + columnWidth - arrowXOffset;
     int centerY = y + FONT_HEIGHT_SMALL / 2;
 
-    double nodeLat = node->position.latitude_i * 1e-7;
-    double nodeLon = node->position.longitude_i * 1e-7;
+    meshtastic_PositionLite nodePos;
+    if (!nodeDB->copyNodePosition(node->num, nodePos))
+        return;
+    double nodeLat = nodePos.latitude_i * 1e-7;
+    double nodeLon = nodePos.longitude_i * 1e-7;
     float bearing = GeoCoord::bearing(userLat, userLon, nodeLat, nodeLon);
     float relativeBearing = CompassRenderer::adjustBearingForCompassMode(bearing, myHeadingRadian);
     float relativeBearingDeg = CompassRenderer::radiansToDegrees360(relativeBearing);
@@ -652,7 +653,7 @@ void drawNodeListScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t
             continue;
         if (n->num == nodeDB->getNodeNum())
             continue;
-        if (locationScreen && !n->has_position)
+        if (locationScreen && !nodeDB->hasNodePosition(n->num))
             continue;
 
         drawList.push_back(n->num);
@@ -883,14 +884,15 @@ void drawDistanceScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t
 void drawNodeListWithCompasses(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
 {
     float headingRadian = 0.0f;
-    auto ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
-    if (!ourNode || !nodeDB->hasValidPosition(ourNode)) {
+    const auto *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    meshtastic_PositionLite ourSelfPos;
+    if (!ourNode || !nodeDB->hasValidPosition(ourNode) || !nodeDB->copyNodePosition(ourNode->num, ourSelfPos)) {
         drawNodeListScreen(display, state, x, y, "Bearings", drawEntryCompass, drawCompassUnknown, headingRadian, 0.0, 0.0);
         return;
     }
 
-    double lat = DegD(ourNode->position.latitude_i);
-    double lon = DegD(ourNode->position.longitude_i);
+    double lat = DegD(ourSelfPos.latitude_i);
+    double lon = DegD(ourSelfPos.longitude_i);
 
 #if defined(M5STACK_UNITC6L)
     display->clear();

src/graphics/draw/NotificationRenderer.cpp

@@ -66,6 +66,110 @@ uint32_t pow_of_10(uint32_t n)
     return ret;
 }
 
+char graphics::NotificationRenderer::alertBannerLines[MAX_LINES + 1][64] = {};
+uint8_t graphics::NotificationRenderer::alertBannerLineCount = 0;
+graphics::NotificationRenderer::BannerFont graphics::NotificationRenderer::alertBannerLineFonts[MAX_LINES + 1] = {};
+
+static inline graphics::NotificationRenderer::BannerFont parseFontTagPrefix(const char *&p)
+{
+    // Tags must be at the start of the line:
+    // [S] small, [M] medium, [L] large
+    if (p && p[0] == '[' && p[2] == ']' && p[1] != '\0') {
+        char t = p[1];
+        if (t == 'S') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_SMALL;
+        }
+        if (t == 'M') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_MEDIUM;
+        }
+        if (t == 'L') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_LARGE;
+        }
+    }
+    return graphics::NotificationRenderer::BANNER_FONT_DEFAULT;
+}
+
+static inline const uint8_t *fontForBannerLine(graphics::NotificationRenderer::BannerFont f)
+{
+    switch (f) {
+    case graphics::NotificationRenderer::BANNER_FONT_SMALL:
+        return FONT_SMALL;
+    case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
+        return FONT_MEDIUM;
+    case graphics::NotificationRenderer::BANNER_FONT_LARGE:
+        return FONT_LARGE;
+    case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
+    default:
+        return FONT_SMALL;
+    }
+}
+
+static inline uint8_t effectiveLineHeightForBannerLine(graphics::NotificationRenderer::BannerFont f)
+{
+    uint8_t height = FONT_HEIGHT_SMALL;
+    switch (f) {
+    case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
+        height = FONT_HEIGHT_MEDIUM;
+        break;
+    case graphics::NotificationRenderer::BANNER_FONT_LARGE:
+        height = FONT_HEIGHT_LARGE;
+        break;
+    case graphics::NotificationRenderer::BANNER_FONT_SMALL:
+    case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
+    default:
+        height = FONT_HEIGHT_SMALL;
+        break;
+    }
+    return (height > 3) ? (height - 3) : height;
+}
+
+void graphics::NotificationRenderer::parseBannerMessageWithFonts(const char *message)
+{
+    alertBannerLineCount = 0;
+    for (uint8_t i = 0; i < (MAX_LINES + 1); i++) {
+        alertBannerLines[i][0] = '\0';
+        alertBannerLineFonts[i] = BANNER_FONT_DEFAULT;
+    }
+
+    if (!message || !message[0]) {
+        return;
+    }
+
+    const char *p = message;
+
+    while (*p && alertBannerLineCount < (MAX_LINES + 1)) {
+        const char *lineStart = p;
+        while (*p && *p != '\n') {
+            p++;
+        }
+
+        char tmp[64] = {0};
+        size_t len = (size_t)(p - lineStart);
+        if (len > (sizeof(tmp) - 1)) {
+            len = sizeof(tmp) - 1;
+        }
+        memcpy(tmp, lineStart, len);
+        tmp[len] = '\0';
+
+        // Tag at start
+        const char *tp = tmp;
+        BannerFont f = parseFontTagPrefix(tp);
+        alertBannerLineFonts[alertBannerLineCount] = f;
+
+        // Store stripped text
+        strncpy(alertBannerLines[alertBannerLineCount], tp, sizeof(alertBannerLines[0]) - 1);
+        alertBannerLines[alertBannerLineCount][sizeof(alertBannerLines[0]) - 1] = '\0';
+        alertBannerLineCount++;
+
+        if (*p == '\n') {
+            p++;
+        }
+    }
+}
+
 // Used on boot when a certificate is being created
 void NotificationRenderer::drawSSLScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
 {
@@ -317,12 +421,12 @@ void NotificationRenderer::drawNodePicker(OLEDDisplay *display, OLEDDisplayUiSta
     for (int i = firstOptionToShow; i < alertBannerOptions && linesShown < visibleTotalLines; i++, linesShown++) {
         char tempName[48] = {0};
         meshtastic_NodeInfoLite *node = nodeDB->getMeshNodeByIndex(i + 1);
-        if (node && node->has_user) {
+        if (nodeInfoLiteHasUser(node)) {
             const char *rawName = nullptr;
-            if (node->user.long_name[0]) {
-                rawName = node->user.long_name;
-            } else if (node->user.short_name[0]) {
-                rawName = node->user.short_name;
+            if (node->long_name[0]) {
+                rawName = node->long_name;
+            } else if (node->short_name[0]) {
+                rawName = node->short_name;
             }
             if (rawName) {
                 const int arrowWidth = (currentResolution == ScreenResolution::High)
@@ -375,23 +479,37 @@ void NotificationRenderer::drawAlertBannerOverlay(OLEDDisplay *display, OLEDDisp
     const char *lineStarts[MAX_LINES + 1] = {0};
     uint16_t lineCount = 0;
     char lineBuffer[40] = {0};
+    bool useTaggedTextBanner =
+        (current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0 && alertBannerLineCount > 0);
 
-    // Parse lines
-    char *alertEnd = alertBannerMessage + strnlen(alertBannerMessage, sizeof(alertBannerMessage));
-    lineStarts[lineCount] = alertBannerMessage;
+    if (useTaggedTextBanner) {
+        lineCount = std::min<uint8_t>(alertBannerLineCount, MAX_LINES);
+        for (uint16_t i = 0; i < lineCount; i++) {
+            lineStarts[i] = alertBannerLines[i];
+            lineLengths[i] = strlen(lineStarts[i]);
+            display->setFont(fontForBannerLine(alertBannerLineFonts[i]));
+            lineWidths[i] = display->getStringWidth(lineStarts[i], lineLengths[i], true);
+            if (lineWidths[i] > maxWidth)
+                maxWidth = lineWidths[i];
+        }
+    } else {
+        char *alertEnd = alertBannerMessage + strnlen(alertBannerMessage, sizeof(alertBannerMessage));
+        lineStarts[lineCount] = alertBannerMessage;
 
-    while ((lineCount < MAX_LINES) && (lineStarts[lineCount] < alertEnd)) {
-        lineStarts[lineCount + 1] = std::find((char *)lineStarts[lineCount], alertEnd, '\n');
-        lineLengths[lineCount] = lineStarts[lineCount + 1] - lineStarts[lineCount];
-        if (lineStarts[lineCount + 1][0] == '\n')
-            lineStarts[lineCount + 1] += 1;
-        lineWidths[lineCount] = display->getStringWidth(lineStarts[lineCount], lineLengths[lineCount], true);
-        if (lineWidths[lineCount] > maxWidth)
-            maxWidth = lineWidths[lineCount];
-        lineCount++;
+        while ((lineCount < MAX_LINES) && (lineStarts[lineCount] < alertEnd)) {
+            lineStarts[lineCount + 1] = std::find((char *)lineStarts[lineCount], alertEnd, '\n');
+            lineLengths[lineCount] = lineStarts[lineCount + 1] - lineStarts[lineCount];
+            if (lineStarts[lineCount + 1][0] == '\n')
+                lineStarts[lineCount + 1] += 1;
+            lineWidths[lineCount] = display->getStringWidth(lineStarts[lineCount], lineLengths[lineCount], true);
+            if (lineWidths[lineCount] > maxWidth)
+                maxWidth = lineWidths[lineCount];
+            lineCount++;
+        }
     }
 
     // Measure option widths
+    display->setFont(FONT_SMALL);
     for (int i = 0; i < alertBannerOptions; i++) {
         optionWidths[i] = display->getStringWidth(optionsArrayPtr[i], strlen(optionsArrayPtr[i]), true);
         if (optionWidths[i] > maxWidth)
@@ -505,6 +623,10 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
     bool needs_bell = false;
     uint16_t lineWidths[totalLines] = {0};
     uint16_t lineLengths[totalLines] = {0};
+    BannerFont lineFonts[totalLines] = {};
+    uint8_t lineEffectiveHeights[totalLines] = {0};
+    const char *renderLines[totalLines] = {0};
+    bool useTaggedBannerFonts = (current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0);
 
     if (maxWidth != 0)
         is_picker = true;
@@ -517,11 +639,22 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
     uint16_t widestLineWithBars = 0;
 
     while (lines[lineCount] != nullptr) {
-        auto newlinePointer = strchr(lines[lineCount], '\n');
+        const char *renderText = lines[lineCount];
+        BannerFont lineFont = BANNER_FONT_DEFAULT;
+        if (useTaggedBannerFonts && lineCount < alertBannerLineCount) {
+            renderText = alertBannerLines[lineCount];
+            lineFont = alertBannerLineFonts[lineCount];
+        }
+        renderLines[lineCount] = renderText;
+        lineFonts[lineCount] = lineFont;
+        lineEffectiveHeights[lineCount] = effectiveLineHeightForBannerLine(lineFont);
+        display->setFont(fontForBannerLine(lineFont));
+
+        auto newlinePointer = strchr(renderText, '\n');
         if (newlinePointer)
-            lineLengths[lineCount] = (newlinePointer - lines[lineCount]); // Check for newlines first
-        else // if the newline wasn't found, then pull string length from strlen
-            lineLengths[lineCount] = strlen(lines[lineCount]);
+            lineLengths[lineCount] = (newlinePointer - renderText);
+        else
+            lineLengths[lineCount] = strlen(renderText);
 
         if (current_notification_type == notificationTypeEnum::node_picker) {
             char measureBuffer[64] = {0};
@@ -533,7 +666,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
         // Consider extra width for signal bars on lines that contain "Signal:"
         uint16_t potentialWidth = lineWidths[lineCount];
-        if (graphics::bannerSignalBars >= 0 && strncmp(lines[lineCount], "Signal:", 7) == 0) {
+        if (graphics::bannerSignalBars >= 0 && strncmp(renderText, "Signal:", 7) == 0) {
             const int totalBars = 5;
             const int barWidth = 3;
             const int barSpacing = 2;
@@ -569,8 +702,21 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
     uint16_t screenHeight = display->height();
     uint8_t effectiveLineHeight = FONT_HEIGHT_SMALL - 3;
-    uint8_t visibleTotalLines = std::min<uint8_t>(lineCount, (screenHeight - vPadding * 2) / effectiveLineHeight);
-    uint16_t contentHeight = visibleTotalLines * effectiveLineHeight;
+    uint8_t visibleTotalLines = 0;
+    uint16_t contentHeight = 0;
+    const uint16_t availableHeight = (screenHeight > (vPadding * 2)) ? (screenHeight - vPadding * 2) : 0;
+    for (uint8_t i = 0; i < lineCount; i++) {
+        uint8_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
+        if (contentHeight + thisLineHeight > availableHeight) {
+            break;
+        }
+        contentHeight += thisLineHeight;
+        visibleTotalLines++;
+    }
+    if (visibleTotalLines == 0 && lineCount > 0) {
+        visibleTotalLines = 1;
+        contentHeight = lineEffectiveHeights[0] ? lineEffectiveHeights[0] : effectiveLineHeight;
+    }
     uint16_t boxHeight = contentHeight + vPadding * 2;
     if (visibleTotalLines == 1) {
         boxHeight += (currentResolution == ScreenResolution::High) ? 4 : 3;
@@ -616,15 +762,18 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
     // Draw Content
     int16_t lineY = boxTop + vPadding;
-    for (int i = 0; i < lineCount; i++) {
+    for (int i = 0; i < visibleTotalLines; i++) {
+        display->setFont(fontForBannerLine(lineFonts[i]));
+        int16_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
         int16_t textX = boxLeft + (boxWidth - lineWidths[i]) / 2;
         if (needs_bell && i == 0) {
-            int bellY = lineY + (FONT_HEIGHT_SMALL - 8) / 2;
+            int fontHeight = thisLineHeight + 3;
+            int bellY = lineY + (fontHeight - 8) / 2;
             display->drawXbm(textX - 10, bellY, 8, 8, bell_alert);
             display->drawXbm(textX + lineWidths[i] + 2, bellY, 8, 8, bell_alert);
         }
         char lineBuffer[lineLengths[i] + 1];
-        strncpy(lineBuffer, lines[i], lineLengths[i]);
+        strncpy(lineBuffer, renderLines[i], lineLengths[i]);
         lineBuffer[lineLengths[i]] = '\0';
         // Determine if this is a pop-up or a pick list
         if (alertBannerOptions > 0 && i == 0) {
@@ -658,7 +807,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
                 display->drawString(textX, lineY - yOffset, lineBuffer);
             }
             display->setColor(WHITE);
-            lineY += (effectiveLineHeight - 2 - background_yOffset);
+            lineY += (thisLineHeight - 2 - background_yOffset);
         } else {
             // Pop-up
             // If this is the Signal line, center text + bars as one group
@@ -716,7 +865,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
                     display->drawString(textX, lineY, lineBuffer);
                 }
             }
-            lineY += (effectiveLineHeight);
+            lineY += thisLineHeight;
         }
     }
 

src/graphics/draw/NotificationRenderer.h

@@ -31,6 +31,12 @@ class NotificationRenderer
 
     static bool pauseBanner;
 
+    enum BannerFont : uint8_t { BANNER_FONT_DEFAULT = 0, BANNER_FONT_SMALL, BANNER_FONT_MEDIUM, BANNER_FONT_LARGE };
+
+    static char alertBannerLines[MAX_LINES + 1][64]; // parsed text per line
+    static uint8_t alertBannerLineCount;
+    static BannerFont alertBannerLineFonts[MAX_LINES + 1];
+    static void parseBannerMessageWithFonts(const char *message);
     static void resetBanner();
     static void showKeyboardMessagePopupWithTitle(const char *title, const char *content, uint32_t durationMs);
     static void drawBannercallback(OLEDDisplay *display, OLEDDisplayUiState *state);

src/graphics/draw/UIRenderer.cpp

@@ -464,11 +464,11 @@ static bool computeBottomCompassPlacement(OLEDDisplay *display, int16_t xOffset,
     return true;
 }
 
-static void drawTruncatedStatusLine(OLEDDisplay *display, int16_t x, int16_t y, const std::string &statusText)
+static void drawTruncatedStatusLine(OLEDDisplay *display, int16_t x, int16_t y, const char *statusText)
 {
     // Fixed-buffer truncate helper replaces iterative std::string chopping to keep code size down.
     char rawStatus[96];
-    snprintf(rawStatus, sizeof(rawStatus), " Status: %s", statusText.c_str());
+    snprintf(rawStatus, sizeof(rawStatus), " Status: %s", statusText ? statusText : "");
 
     char clippedStatus[96];
     UIRenderer::truncateStringWithEmotes(display, rawStatus, clippedStatus, sizeof(clippedStatus), display->getWidth());
@@ -495,7 +495,7 @@ void graphics::UIRenderer::rebuildFavoritedNodes()
         meshtastic_NodeInfoLite *n = nodeDB->getMeshNodeByIndex(i);
         if (!n || n->num == nodeDB->getNodeNum())
             continue;
-        if (n->is_favorite)
+        if (nodeInfoLiteIsFavorite(n))
             favoritedNodes.push_back(n);
     }
 
@@ -750,7 +750,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
         return;
 
     meshtastic_NodeInfoLite *node = favoritedNodes[nodeIndex];
-    if (!node || node->num == nodeDB->getNodeNum() || !node->is_favorite)
+    if (!node || node->num == nodeDB->getNodeNum() || !nodeInfoLiteIsFavorite(node))
         return;
     display->clear();
 #if defined(M5STACK_UNITC6L)
@@ -763,7 +763,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
 #endif
     currentFavoriteNodeNum = node->num;
     // === Create the shortName and title string ===
-    const char *shortName = (node->has_user && node->user.short_name[0]) ? node->user.short_name : "Node";
+    const char *shortName = (nodeInfoLiteHasUser(node) && node->short_name[0]) ? node->short_name : "Node";
     char titlestr[40];
     snprintf(titlestr, sizeof(titlestr), "*%s*", shortName);
 
@@ -781,9 +781,9 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     // === 1. Long Name (always try to show first) ===
     const char *username;
     if (currentResolution == ScreenResolution::UltraLow) {
-        username = (node->has_user && node->user.long_name[0]) ? node->user.short_name : nullptr;
+        username = (nodeInfoLiteHasUser(node) && node->long_name[0]) ? node->short_name : nullptr;
     } else {
-        username = (node->has_user && node->user.long_name[0]) ? node->user.long_name : nullptr;
+        username = (nodeInfoLiteHasUser(node) && node->long_name[0]) ? node->long_name : nullptr;
     }
 
     // Print node's long name (e.g. "Backpack Node")
@@ -796,23 +796,14 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
         UIRenderer::drawStringWithEmotes(display, x, getTextPositions(display)[line++], username, FONT_HEIGHT_SMALL, 1, false);
     }
 
-#if !MESHTASTIC_EXCLUDE_STATUS
+#if !MESHTASTIC_EXCLUDE_STATUS && !MESHTASTIC_EXCLUDE_STATUSDB
     // === Optional: Last received StatusMessage line for this node ===
-    // Display it directly under the username line (if we have one).
-    if (statusMessageModule) {
-        const auto &recent = statusMessageModule->getRecentReceived();
-        const StatusMessageModule::RecentStatus *found = nullptr;
-
-        // Search newest-to-oldest
-        for (auto it = recent.rbegin(); it != recent.rend(); ++it) {
-            if (it->fromNodeId == node->num && !it->statusText.empty()) {
-                found = &(*it);
-                break;
-            }
-        }
-
-        if (found) {
-            drawTruncatedStatusLine(display, x, getTextPositions(display)[line++], found->statusText);
+    // Display it directly under the username line (if we have one). The cache
+    // lives on NodeDB now, keyed by NodeNum, so this is an O(1) lookup.
+    if (nodeDB) {
+        meshtastic_StatusMessage cachedStatus;
+        if (nodeDB->copyNodeStatus(node->num, cachedStatus) && cachedStatus.status[0]) {
+            drawTruncatedStatusLine(display, x, getTextPositions(display)[line++], cachedStatus.status);
         }
     }
 #endif
@@ -971,25 +962,30 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
 #if !defined(M5STACK_UNITC6L)
     // === 4. Uptime (only show if metric is present) ===
     char uptimeStr[32] = "";
-    if (node->has_device_metrics && node->device_metrics.has_uptime_seconds) {
+    meshtastic_DeviceMetrics nodeMetrics;
+    const bool haveNodeMetrics = nodeDB->copyNodeTelemetry(node->num, nodeMetrics);
+    if (haveNodeMetrics && nodeMetrics.has_uptime_seconds) {
         char upPrefix[12]; // enough for leftSideSpacing + "Up:"
         snprintf(upPrefix, sizeof(upPrefix), "%sUp:", leftSideSpacing);
-        getUptimeStr(node->device_metrics.uptime_seconds * 1000, upPrefix, uptimeStr, sizeof(uptimeStr));
+        getUptimeStr(nodeMetrics.uptime_seconds * 1000, upPrefix, uptimeStr, sizeof(uptimeStr));
     }
     if (uptimeStr[0]) {
         display->drawString(x, getTextPositions(display)[line++], uptimeStr);
     }
 
     // === 5. Distance (only if both nodes have GPS position) ===
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
     char distStr[24] = ""; // Make buffer big enough for any string
     bool haveDistance = false;
 
-    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node)) {
+    meshtastic_PositionLite nodePos;
+    meshtastic_PositionLite ourPos;
+    const bool haveNodePos = nodeDB->copyNodePosition(node->num, nodePos);
+    const bool haveOurPos = ourNode && nodeDB->copyNodePosition(ourNode->num, ourPos);
+    if (nodeDB->hasValidPosition(ourNode) && nodeDB->hasValidPosition(node) && haveNodePos && haveOurPos) {
         // Use shared meter conversion, then format display units with lightweight integer rounding.
-        const float distanceMeters =
-            GeoCoord::latLongToMeter(DegD(node->position.latitude_i), DegD(node->position.longitude_i),
-                                     DegD(ourNode->position.latitude_i), DegD(ourNode->position.longitude_i));
+        const float distanceMeters = GeoCoord::latLongToMeter(DegD(nodePos.latitude_i), DegD(nodePos.longitude_i),
+                                                              DegD(ourPos.latitude_i), DegD(ourPos.longitude_i));
         if (config.display.units == meshtastic_Config_DisplayConfig_DisplayUnits_IMPERIAL) {
             const int feet = static_cast<int>((distanceMeters * METERS_TO_FEET) + 0.5f);
             if (feet > 0 && feet < 1000) {
@@ -1024,19 +1020,19 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     char batLine[32] = "";
     bool haveBatLine = false;
 
-    if (node->has_device_metrics) {
-        bool hasPct = node->device_metrics.has_battery_level;
-        bool hasVolt = node->device_metrics.has_voltage && node->device_metrics.voltage > 0.001f;
+    if (haveNodeMetrics) {
+        bool hasPct = nodeMetrics.has_battery_level;
+        bool hasVolt = nodeMetrics.has_voltage && nodeMetrics.voltage > 0.001f;
 
         int pct = 0;
         float volt = 0.0f;
 
         if (hasPct) {
-            pct = (int)node->device_metrics.battery_level;
+            pct = (int)nodeMetrics.battery_level;
         }
 
         if (hasVolt) {
-            volt = node->device_metrics.voltage;
+            volt = nodeMetrics.voltage;
         }
 
         if (hasPct && pct > 0 && pct <= 100) {
@@ -1076,11 +1072,11 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     const bool hasNodePositionFix = nodeDB->hasValidPosition(node);
     const char *statusLine1 = nullptr;
     const char *statusLine2 = nullptr;
-    if (hasOwnPositionFix && hasNodePositionFix) {
-        const auto &op = ourNode->position;
+    if (hasOwnPositionFix && hasNodePositionFix && haveOurPos && haveNodePos) {
+        const auto &op = ourPos;
         showCompass = CompassRenderer::getHeadingRadians(DegD(op.latitude_i), DegD(op.longitude_i), myHeading);
         if (showCompass) {
-            const auto &p = node->position;
+            const auto &p = nodePos;
             bearing = GeoCoord::bearing(DegD(op.latitude_i), DegD(op.longitude_i), DegD(p.latitude_i), DegD(p.longitude_i));
             bearing = CompassRenderer::adjustBearingForCompassMode(bearing, myHeading);
         } else {
@@ -1130,7 +1126,7 @@ void UIRenderer::drawDeviceFocused(OLEDDisplay *display, OLEDDisplayUiState *sta
     display->setTextAlignment(TEXT_ALIGN_LEFT);
     display->setFont(FONT_SMALL);
     int line = 1;
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
 
     // === Header ===
     if (currentResolution == ScreenResolution::UltraLow) {
@@ -1270,7 +1266,7 @@ void UIRenderer::drawDeviceFocused(OLEDDisplay *display, OLEDDisplayUiState *sta
     int textWidth = 0;
     int nameX = 0;
     int yOffset = (currentResolution == ScreenResolution::High) ? 0 : 5;
-    const char *longName = (ourNode && ourNode->has_user && ourNode->user.long_name[0]) ? ourNode->user.long_name : "";
+    const char *longName = (nodeInfoLiteHasUser(ourNode) && ourNode->long_name[0]) ? ourNode->long_name : "";
     const char *shortName = owner.short_name ? owner.short_name : "";
     char combinedName[96];
     if (longName[0] && shortName[0]) {
@@ -1597,7 +1593,7 @@ void UIRenderer::drawCompassAndLocationScreen(OLEDDisplay *display, OLEDDisplayU
     geoCoord.updateCoords(int32_t(gpsStatus->getLatitude()), int32_t(gpsStatus->getLongitude()),
                           int32_t(gpsStatus->getAltitude()));
 
-    meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
+    const meshtastic_NodeInfoLite *ourNode = nodeDB->getMeshNode(nodeDB->getNodeNum());
     const bool hasOwnPositionFix = (ourNode && nodeDB->hasValidPosition(ourNode));
     const bool hasLiveGpsFix =
         (gpsStatus && gpsStatus->getHasLock() && (gpsStatus->getLatitude() != 0 || gpsStatus->getLongitude() != 0));
@@ -1613,9 +1609,11 @@ void UIRenderer::drawCompassAndLocationScreen(OLEDDisplay *display, OLEDDisplayU
             headingLat = DegD(gpsStatus->getLatitude());
             headingLon = DegD(gpsStatus->getLongitude());
         } else if (hasOwnPositionFix) {
-            const auto &op = ourNode->position;
-            headingLat = DegD(op.latitude_i);
-            headingLon = DegD(op.longitude_i);
+            meshtastic_PositionLite ownPos;
+            if (nodeDB->copyNodePosition(ourNode->num, ownPos)) {
+                headingLat = DegD(ownPos.latitude_i);
+                headingLon = DegD(ownPos.longitude_i);
+            }
         }
         validHeading = CompassRenderer::getHeadingRadians(headingLat, headingLon, heading);
     }