ElixirForge/firmware
1
0
Fork 0
mirror of https://github.com/alexhopeoconnor/firmware.git synced 2026-10-04 03:18:10 +10:00
Code Releases Activity

Merge branch 'develop' into t-display-s3-pro

Browse Source
This commit is contained in:
Manuel
2026-05-17 21:23:40 +01:00
committed by GitHub
parent f1be5c4ff3 767a748188
commit e112c3e682
161 changed files with 15141 additions and 758 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/commands/diagnose.md
+7 -1
View File
@@ -49,11 +49,17 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
7. **Suggest next actions only for specific, recognisable failure modes**:
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
8. **Suggest next actions only for specific, recognisable failure modes**:
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
## What NOT to do
.claude/commands/leakhunt.md
+103
View File
@@ -0,0 +1,103 @@
---
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
argument-hint: [window=1h] [field=free_heap] [variant=local]
---
<!-- markdownlint-disable MD029 -->
# `/leakhunt` — read the recorder, classify a memory leak
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
## Two signal paths — pick the right one
| Path | Build flag | Cadence | Per-thread attribution | Cost |
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
| LocalStats packet | (default) | ~60 s | No | Free — always on |
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
## What to do
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
- `running == True`
- `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
- `samples` — how many raw points contributed
- `min`, `max` — total swing
- `slope_per_min` — units per minute (linear regression over the whole window)
5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
- **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
8. **Report**:
```text
/leakhunt window=6h field=free_heap variant=local
────────────────────────────────────────────────────
recorder : running, telem last_ts 8s ago
build : DEBUG_HEAP=ON (per-line prefix detected)
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
free_heap : min 92,344 / max 124,008 / range 31,664
slope : -82 bytes/min (negative — heap declining)
reboots : none in window
OOM events : none
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
Router -1,408 B over 4 events
others -240 B
verdict : SLOW LEAK — primary suspect MeshPacket thread
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
evidence : (3 log line citations with uptimes)
```
Then: **what to do next.**
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
```text
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
```
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
- STEADY → say so; stop. Don't invent problems.
## What NOT to do
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
## Companion: `mark_event` for stress runs
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
```text
mark_event("burst-start")
… run the workload …
mark_event("burst-end")
/leakhunt window=15m
```
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
.claude/commands/repro.md
+4
View File
@@ -3,6 +3,8 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
argument-hint: <test-node-id> [count=5]
---
<!-- markdownlint-disable MD029 -->
# `/repro` — flakiness triage for one test
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -40,6 +42,8 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
6. **Classify the flake** into one of:
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
.devcontainer/devcontainer.json
+7 -3
View File
@@ -8,17 +8,21 @@
"features": {
"ghcr.io/devcontainers/features/python:1": {
"installTools": true,
"version": "3.14"
"version": "3.13"
}
},
"customizations": {
"vscode": {
"extensions": [
"ms-vscode.cpptools",
"platformio.platformio-ide",
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide",
"Trunk.io"
],
"unwantedRecommendations": ["ms-azuretools.vscode-docker"],
"unwantedRecommendations": [
"ms-azuretools.vscode-docker",
"platformio.platformio-ide"
],
"settings": {
"extensions.ignoreRecommendations": true
}
.github/copilot-instructions.md
+16 -10
View File
@@ -193,22 +193,26 @@ Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which d
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
### Gradient sync (opt-in via special nonces)
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
`client_capabilities` is **not** a thing in this branch. Phone clients opt into the new sync flow by sending one of two values in the `ToRadio.want_config_id`:
There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
- `SPECIAL_NONCE_GRADIENT_SYNC` (69422) — full config + thin NodeInfo + replay phases.
- `SPECIAL_NONCE_GRADIENT_ONLY_NODES` (69423) — skip config segments, NodeInfo + replay only.
1. Config / module-config / channel / metadata segments (same as before).
2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
`PhoneAPI::clientWantsGradientSync()` is the single switch. When true, `STATE_SEND_OTHER_NODEINFOS` is followed by:
`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
```text
STATE_REPLAY_POSITIONS → STATE_REPLAY_TELEMETRY → STATE_REPLAY_ENVIRONMENT → STATE_REPLAY_STATUS
```
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
Each replay phase walks the corresponding satellite map and emits synthetic `MeshPacket`s on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` for both device + environment variants, `STATUS_MESSAGE_APP`). Legacy clients (no special nonce) get the bundled-NodeInfo path with position/device_metrics joined back in by `ConvertToNodeInfo(lite, pos*, dm*)` — wire bytes are byte-identical to pre-v25 for them.
Special nonces that still mean something:
`ConvertToNodeInfoThin(lite)` is the gradient-sync emitter (no position/telemetry).
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
### v24 → v25 migration
@@ -285,6 +289,8 @@ firmware/
- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
- Use `assert()` for invariants that should never fail
- C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
### Naming Conventions
.gitignore
+10
View File
@@ -47,6 +47,10 @@ data/boot/logo.*
managed_components/*
arduino-lib-builder*
dependencies.lock
# JLink / RTT debug artifacts (nRF SoCs)
flash.jlink
rtt_*.txt
idf_component.yml
CMakeLists.txt
/sdkconfig.*
@@ -56,3 +60,9 @@ CMakeLists.txt
.python3
.claude/scheduled_tasks.lock
userPrefs.jsonc.mcp-session-bak
# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
# compiled .proto outputs are ephemeral build artifacts.
build/fixtures/
bin/_generated/
.trunk/trunk.yaml
+10 -3
View File
@@ -4,14 +4,14 @@ cli:
plugins:
sources:
- id: trunk
ref: v1.8.0
ref: v1.9.0
uri: https://github.com/trunk-io/plugins
lint:
enabled:
- checkov@3.2.526
- checkov@3.2.528
- renovate@43.150.0
- prettier@3.8.3
- trufflehog@3.95.2
- trufflehog@3.95.3
- yamllint@1.38.0
- bandit@1.9.4
- trivy@0.70.0
@@ -34,6 +34,13 @@ lint:
- linters: [ALL]
paths:
- bin/**
# Fake-NodeDB fixture JSONL files contain deterministic synthetic
# public_key_hex (64-char hex) values that gitleaks misidentifies as
# generic-api-key. These are not secrets — they're test fixtures
# produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
- linters: [gitleaks]
paths:
- test/fixtures/nodedb/seed_v25_*.jsonl
runtimes:
enabled:
- python@3.14.4
.vscode/extensions.json
Vendored
+4 -4
View File
@@ -1,10 +1,10 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"platformio.platformio-ide"
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide"
],
"unwantedRecommendations": [
"ms-vscode.cpptools-extension-pack"
"ms-vscode.cpptools-extension-pack",
"platformio.platformio-ide"
]
}
AGENTS.md
+2
View File
@@ -66,6 +66,8 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
- **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
- **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
## Typical agent workflows
bin/_rewrite_proto_namespace.py
Executable
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env python3
"""Post-process protoc-generated Python files to live under a local namespace.
Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
target directory and rewrites every `meshtastic` reference (imports, dotted
attribute access) to use the new namespace (e.g., `meshtastic_v25`).
Why: the .proto files declare `package meshtastic;`, so protoc emits
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
`meshtastic` package which other parts of the mcp-server depend on. Renaming
to a local namespace keeps both available.
Usage:
_rewrite_proto_namespace.py <generated_dir> <new_namespace>
"""
from __future__ import annotations
import pathlib
import re
import sys
def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
# Standard protoc import forms:
# from meshtastic.X_pb2 import ... (rare, for direct symbol pulls)
# from meshtastic import X_pb2 as ... (common, the cross-file ref)
# import meshtastic.X_pb2 (also possible)
pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
count = 0
for p in dir_path.glob("*.py"):
text = p.read_text(encoding="utf-8")
new = pattern_dotted_from.sub(f"from {new_ns}.", text)
new = pattern_bare_from.sub(f"from {new_ns} import ", new)
new = pattern_dotted_import.sub(f"import {new_ns}.", new)
# NOTE: we deliberately leave `meshtastic/X.proto` source-filename
# references inside descriptor strings alone. The descriptor pool is
# keyed by source filename (independent of Python package layout), so
# those don't collide with the PyPI package's descriptors.
if new != text:
p.write_text(new, encoding="utf-8")
count += 1
return count
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
return 2
dir_path = pathlib.Path(argv[0])
new_ns = argv[1]
if not dir_path.is_dir():
print(f"directory not found: {dir_path}", file=sys.stderr)
return 2
n = rewrite(dir_path, new_ns)
print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
bin/config.d/lora-station-g3.yaml
+18
View File
@@ -0,0 +1,18 @@
# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
# Verify spidev / I2C device paths for your OS — they may differ.
Meta:
name: Station G3
support: community
compatible:
- raspberry-pi
Lora:
Module: sx1262
IRQ: 22 # BCM pin — wiki spec
Reset: 16 # BCM pin — wiki spec
Busy: 24 # BCM pin — wiki spec
# CS: 8 # BCM 8 = SPI0 CE0 (default); uncomment only to override
DIO2_AS_RF_SWITCH: true
DIO3_TCXO_VOLTAGE: true
spidev: spidev0.0
# SX126X_MAX_POWER: 19 # matches Station G2 firmware cap; raise carefully per PA jumper mode
bin/gen-fake-nodedb-seed.py
Executable
+439
View File
@@ -0,0 +1,439 @@
#!/usr/bin/env python3
"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
The output is hand-editable and committed; a sibling compile step
(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
v25 protobuf with fresh "now-relative" timestamps.
Determinism contract:
Same --seed -> byte-identical JSONL output, regardless of wall clock.
All timestamps are stored as `*_offset_sec` (seconds before "now"); the
compile step resolves them to absolute epochs at compile time.
Structural fields covered:
* NodeInfoLite header: num, long_name, short_name, hw_model, role,
public_key, snr, channel, hops_away, next_hop, bitfield flags
* PositionLite: lat/long Gaussian around --centroid, altitude, source
* DeviceMetrics: battery/voltage/util/uptime
* EnvironmentMetrics: temp/humidity/pressure/iaq
* StatusMessage: error_code (usually zero)
Active-board allow-list:
hw_model values are restricted to the intersection of
(a) variants with `custom_meshtastic_support_level = 1` in
variants/*/*/platformio.ini, AND
(b) values present in the `HardwareModel` enum in mesh.proto.
See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
(PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
Active-role allow-list:
Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
"""
from __future__ import annotations
import argparse
import datetime as _dt
import json
import math
import pathlib
import random
import sys
# --------------------------------------------------------------------------
# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
# Refresh by running:
# for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
# grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
# done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
# --------------------------------------------------------------------------
HW_MODEL_WEIGHTS: dict[str, float] = {
"HELTEC_V3": 14.0,
"T_DECK": 9.0,
"HELTEC_V4": 8.0,
"RAK4631": 8.0,
"HELTEC_MESH_POCKET": 6.0,
"TRACKER_T1000_E": 5.0,
"HELTEC_MESH_NODE_T114": 5.0,
"T_DECK_PRO": 5.0,
"LILYGO_TBEAM_S3_CORE": 4.0,
"HELTEC_WIRELESS_PAPER": 4.0,
"HELTEC_WSL_V3": 3.0,
"T_ECHO": 3.0,
"HELTEC_WIRELESS_TRACKER": 3.0,
"HELTEC_WIRELESS_TRACKER_V2": 2.0,
"HELTEC_VISION_MASTER_E290": 2.0,
"HELTEC_MESH_SOLAR": 2.0,
"SEEED_WIO_TRACKER_L1": 2.0,
"T_LORA_PAGER": 1.5,
"HELTEC_VISION_MASTER_E213": 1.5,
"T_ECHO_PLUS": 1.0,
"MUZI_BASE": 1.0,
"WISMESH_TAP_V2": 1.0,
"THINKNODE_M2": 1.0,
"THINKNODE_M5": 1.0,
"TLORA_T3_S3": 1.0,
# Long tail (uniform low weight across remaining tier-1 boards):
"HELTEC_V4_R8": 0.3,
"HELTEC_VISION_MASTER_T190": 0.3,
"HELTEC_HT62": 0.3,
"HELTEC_MESH_NODE_T096": 0.3,
"M5STACK_C6L": 0.3,
"MINI_EPAPER_S3": 0.3,
"MUZI_R1_NEO": 0.3,
"NOMADSTAR_METEOR_PRO": 0.3,
"RAK3312": 0.3,
"RAK3401": 0.3,
"SEEED_SOLAR_NODE": 0.3,
"SEEED_WIO_TRACKER_L1_EINK": 0.3,
"SENSECAP_INDICATOR": 0.3,
"TBEAM_1_WATT": 0.3,
"THINKNODE_M1": 0.3,
"THINKNODE_M3": 0.3,
"THINKNODE_M6": 0.3,
"T_ECHO_LITE": 0.3,
"WISMESH_TAG": 0.3,
"WISMESH_TAP": 0.3,
"XIAO_NRF52_KIT": 0.3,
"CROWPANEL": 0.3,
}
# Non-deprecated roles only.
ROLE_WEIGHTS: dict[str, float] = {
"CLIENT": 75.0,
"CLIENT_MUTE": 5.0,
"ROUTER": 7.0,
"TRACKER": 3.0,
"SENSOR": 2.0,
"CLIENT_HIDDEN": 2.0,
"ROUTER_LATE": 2.0,
"CLIENT_BASE": 2.0,
"TAK": 1.0,
"TAK_TRACKER": 0.5,
"LOST_AND_FOUND": 0.5,
}
# Name pools — 60 firsts × 60 lasts = 3600 combinations.
FIRSTS = [
"Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
"Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
"Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
"Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
"Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
"Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
"Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
"Burning", "Frozen", "Whispering", "Howling",
]
LASTS = [
"Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
"Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
"Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
"Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
"Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
"Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
"Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
"Bluff", "Mesa", "Arroyo", "Ridge",
]
# Brief callsign pool for licensed-looking suffixes.
CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
# 6 bytes due to U+FE0F variation selector — explicitly excluded.
EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
"🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
"🦇", "🦋"]
# --------------------------------------------------------------------------
# Helpers
# --------------------------------------------------------------------------
NUM_RESERVED = 4 # firmware reserves 0..3 (per NodeDB constants)
NUM_MAX_EXCLUSIVE = 0x80000000 # restrict to positive int32 range for readability
def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
"""Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
keys = sorted(weights.keys())
totals = [weights[k] for k in keys]
return rng.choices(keys, weights=totals, k=1)[0]
def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
if is_licensed:
prefix = rng.choice(CALLSIGN_PREFIXES)
# Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
# nanopb max_size:25 means C string fits 24 bytes + NUL.
if len(base) + len(suffix) <= 24:
base = base + suffix
# Hard cap to 24 chars (nanopb max_size:25 minus NUL).
return base[:24]
def _gen_short_name(rng: random.Random, long_name: str) -> str:
# 10% emoji-only short_name
if rng.random() < 0.10:
return rng.choice(EMOJI_SHORTNAMES)
first_char = long_name[0].upper() if long_name else "X"
alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
return first_char + "".join(rng.choices(alphanums, k=3))
def _gen_hops_away(rng: random.Random) -> int:
# Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
r = rng.random()
if r < 0.55:
return 0
if r < 0.80:
return 1
if r < 0.92:
return 2
if r < 0.97:
return 3
if r < 0.99:
return 4
return rng.randint(5, 7)
def _gen_position(
rng: random.Random,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
last_heard_offset_sec: int,
) -> dict:
# 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
altitude = max(0, round(rng.gauss(1376.0, 250.0))) # T or C valley floor + relief
# Position was reported up to 300s before last_heard.
time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
return {
"latitude": round(lat, 6),
"longitude": round(lon, 6),
"altitude": altitude,
"time_offset_sec": time_offset_sec,
"location_source": "LOC_INTERNAL",
}
def _gen_telemetry(rng: random.Random) -> dict:
# 5% plugged-in (battery_level == 101); rest uniform [10..100].
if rng.random() < 0.05:
battery_level = 101
voltage = 4.20
else:
battery_level = rng.randint(10, 100)
voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
# Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
def _beta(a: float, b: float) -> float:
x = rng.gammavariate(a, 1.0)
y = rng.gammavariate(b, 1.0)
return x / (x + y)
channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
return {
"battery_level": battery_level,
"voltage": voltage,
"channel_utilization": channel_utilization,
"air_util_tx": air_util_tx,
"uptime_seconds": uptime_seconds,
}
def _gen_environment(rng: random.Random) -> dict:
return {
"temperature": round(rng.gauss(22.0, 8.0), 2),
"relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
"barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
"iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
}
def _gen_status(rng: random.Random) -> dict:
# `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
# Most peers report a healthy short status; occasional alert string.
healthy = ["OK", "online", "active", "running", "ready", "nominal"]
alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
if rng.random() < 0.92:
return {"status": rng.choice(healthy)}
return {"status": rng.choice(alert)}
def _gen_node(
rng: random.Random,
num: int,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
coverage: dict[str, float],
last_heard_mean_sec: int,
last_heard_max_sec: int,
) -> dict:
is_licensed = rng.random() < 0.05
long_name = _gen_long_name(rng, is_licensed)
short_name = _gen_short_name(rng, long_name)
hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
role = _weighted_choice(rng, ROLE_WEIGHTS)
has_public_key = rng.random() < 0.92
public_key_hex = (
"".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
)
snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
hops_away = _gen_hops_away(rng)
next_hop = rng.randint(0, 255) if hops_away > 0 else 0
last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
bitfield = {
"has_user": True,
"is_favorite": rng.random() < 0.08,
"is_muted": rng.random() < 0.03,
"via_mqtt": rng.random() < 0.12,
"is_ignored": rng.random() < 0.01,
"is_licensed": is_licensed,
"has_is_unmessagable": True,
"is_unmessagable": rng.random() < 0.02,
"is_key_manually_verified": rng.random() < 0.04,
}
node: dict = {
"num": f"0x{num:08x}",
"long_name": long_name,
"short_name": short_name,
"hw_model": hw_model,
"role": role,
"public_key_hex": public_key_hex,
"snr": snr,
"channel": channel,
"hops_away": hops_away,
"next_hop": next_hop,
"last_heard_offset_sec": last_heard_offset_sec,
"bitfield": bitfield,
"position": (
_gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
if rng.random() < coverage["position"]
else None
),
"telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
"environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
"status": _gen_status(rng) if rng.random() < coverage["status"] else None,
}
return node
def _parse_my_node_num(s: str | None) -> int | None:
if s is None:
return None
s = s.strip()
if s.startswith("0x") or s.startswith("0X"):
return int(s, 16)
return int(s)
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Deterministic JSONL seed for the fake NodeDB fixture.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
p.add_argument("--out", required=True, help="Output JSONL path.")
p.add_argument(
"--centroid",
default="33.1284,-107.2528",
help="LAT,LON centroid (default: Truth or Consequences, NM).",
)
p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
p.add_argument("--position-coverage", type=float, default=0.85)
p.add_argument("--telemetry-coverage", type=float, default=0.70)
p.add_argument("--environment-coverage", type=float, default=0.25)
p.add_argument("--status-coverage", type=float, default=0.40)
p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
p.add_argument("--last-heard-mean-sec", type=int, default=3600)
p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
args = p.parse_args(argv)
if args.count <= 0:
print("--count must be positive", file=sys.stderr)
return 2
try:
centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
except ValueError:
print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
return 2
my_node_num = _parse_my_node_num(args.my_node_num)
rng = random.Random(args.seed)
# 1) Generate a unique deterministic set of NodeNums.
nums: set[int] = set()
while len(nums) < args.count:
n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
if my_node_num is not None and n == my_node_num:
continue
nums.add(n)
ordered_nums = sorted(nums) # sort to fix output order independent of set hash
# 2) Per-node generation (in num order, single RNG continues).
coverage = {
"position": args.position_coverage,
"telemetry": args.telemetry_coverage,
"environment": args.environment_coverage,
"status": args.status_coverage,
}
nodes = [
_gen_node(
rng,
n,
centroid_lat,
centroid_lon,
args.spread_km,
coverage,
args.last_heard_mean_sec,
args.last_heard_max_sec,
)
for n in ordered_nums
]
# 3) Write JSONL.
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
# `generated_at_iso` is informational; it does NOT affect determinism because
# we derive it from the seed, not from wall clock. (Same seed -> same string.)
generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
meta = {
"_meta": {
"version": 25,
"seed": args.seed,
"count": args.count,
"centroid": [centroid_lat, centroid_lon],
"spread_km": args.spread_km,
"generated_at_iso": generated_at,
"my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
"coverage": coverage,
"last_heard_mean_sec": args.last_heard_mean_sec,
"last_heard_max_sec": args.last_heard_max_sec,
}
}
with out_path.open("w", encoding="utf-8") as f:
# `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
# determinism (insertion order varies by Python version otherwise).
f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
for node in nodes:
f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
bin/regen-fake-nodedbs.sh
Executable
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
# JSONL seed files + their compiled v25 protobufs.
#
# Layout:
# test/fixtures/nodedb/seed_v25_<N>.jsonl — COMMITTED, hand-editable.
# build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
# Drop into /prefs/nodes.proto.
#
# Daily use: ./bin/regen-fake-nodedbs.sh
# - Recompiles protos from committed seeds (fresh wall-clock timestamps).
# Intentional seed bump: REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
# - Overwrites the committed JSONL files with freshly-seeded data.
set -euo pipefail
cd "$(dirname "$0")/.."
# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
if [[ ! -d bin/_generated/meshtastic ]]; then
echo "regenerating Python protobuf bindings (one-time)..."
./bin/regen-py-protos.sh
fi
# 2) Pick a Python interpreter that has the meshtastic deps installed.
# Prefer the mcp-server venv (most likely to be set up by the operator).
PY="python3"
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
if [[ -x "$cand" ]]; then
PY="$cand"
break
fi
done
# 3) Pinned seeds per size — bump only when you intentionally want different
# structural data committed. Parallel arrays so the script works on
# macOS bash 3.2 (no `declare -A`).
SIZES=(250 500 1000 2000)
SEEDS=(20260511 20260512 20260513 20260514)
REGEN_SEEDS="${REGEN_SEEDS:-no}"
mkdir -p build/fixtures/nodedb test/fixtures/nodedb
for i in 0 1 2 3; do
n="${SIZES[$i]}"
seed="${SEEDS[$i]}"
jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
$PY bin/gen-fake-nodedb-seed.py \
--count "$n" \
--seed "$seed" \
--out "$jsonl" \
--centroid 33.1284,-107.2528 \
--spread-km 60 \
--position-coverage 0.85 \
--telemetry-coverage 0.70 \
--environment-coverage 0.25 \
--status-coverage 0.40
echo " seed: $jsonl ($(wc -c < "$jsonl") bytes)"
fi
$PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
echo " proto: $proto ($(wc -c < "$proto") bytes)"
done
echo ""
echo "Done. To load on Portduino native:"
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
echo ""
echo "To push to a hardware device:"
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
bin/regen-py-protos.sh
Executable
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
# a standalone refresh after any change to a .proto file.
#
# Output is .gitignored — bindings are a build artifact.
#
# Namespace rewrite:
# The .proto files declare `package meshtastic;`, which makes protoc emit
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
# PyPI `meshtastic` package (which the mcp-server relies on for its
# SerialInterface/BLEInterface transport). We post-process the generated
# files to live under `meshtastic_v25` instead — both the directory layout
# and all internal imports — so they coexist cleanly with the PyPI package.
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v protoc >/dev/null 2>&1; then
echo "ERROR: protoc not found in PATH." >&2
echo " macOS: brew install protobuf" >&2
echo " Ubuntu/Debian: apt install protobuf-compiler" >&2
exit 1
fi
OUT=bin/_generated
LOCAL_NS=meshtastic_v25
rm -rf "$OUT"
mkdir -p "$OUT"
# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
# is available for the [(nanopb).*] options on other messages.
protoc \
--proto_path=protobufs \
--python_out="$OUT" \
protobufs/nanopb.proto \
protobufs/meshtastic/*.proto
# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
# `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
# 4) Make the package importable.
touch "$OUT/__init__.py"
touch "$OUT/$LOCAL_NS/__init__.py"
echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2
bin/seed-json-to-proto.py
Executable
+342
View File
@@ -0,0 +1,342 @@
#!/usr/bin/env python3
"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
are stored as `*_offset_sec` (seconds before "now"); this script resolves them
to absolute epochs using `--now-epoch` (default: current wall clock).
Output is a raw `pb_encode`-compatible binary that can be dropped at
`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
XModem) and loaded by `NodeDB::loadFromDisk` at boot.
Wire format reference:
protobufs/meshtastic/deviceonly.proto (NodeDatabase, NodeInfoLite, sat entries)
src/mesh/NodeDB.h:467-484 (bitfield bit positions)
src/mesh/NodeDB.cpp:1523-1524 (pb_decode entry point)
"""
from __future__ import annotations
import argparse
import json
import pathlib
import sys
import time
from typing import Any
# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
# ships. Run `bin/regen-py-protos.sh` to (re)generate.
#
# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
# post-processes the protoc output to rename the package.
_HERE = pathlib.Path(__file__).resolve().parent
_LOCAL_PROTO_DIR = _HERE / "_generated"
if _LOCAL_PROTO_DIR.is_dir():
sys.path.insert(0, str(_LOCAL_PROTO_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import ( # type: ignore[import-not-found]
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage # type: ignore[import-not-found]
from meshtastic_v25.config_pb2 import Config # type: ignore[import-not-found]
from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics # type: ignore[import-not-found]
except ImportError as local_err:
# Fall back to the PyPI package if in-tree bindings haven't been generated.
# Will fail the v25 assertion below if the PyPI package predates the
# satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
# error message instead of an opaque ImportError.
try:
from meshtastic.protobuf.deviceonly_pb2 import (
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
from meshtastic.protobuf.config_pb2 import Config
from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
except ImportError as pypi_err:
print(
"ERROR: could not import meshtastic protobuf bindings.\n"
" In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
" PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
f" local error (meshtastic_v25): {local_err}\n"
f" pypi error (meshtastic.protobuf): {pypi_err}",
file=sys.stderr,
)
sys.exit(1)
# Fail loudly if bindings predate v25 (no satellite arrays).
assert (
hasattr(NodeDatabase, "DESCRIPTOR")
and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
), (
"Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
"Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
)
# ---------------------------------------------------------------------------
# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
# ---------------------------------------------------------------------------
BIT_IS_KEY_MANUALLY_VERIFIED = 0
BIT_IS_MUTED = 1
BIT_VIA_MQTT = 2
BIT_IS_FAVORITE = 3
BIT_IS_IGNORED = 4
BIT_HAS_USER = 5
BIT_IS_LICENSED = 6
BIT_IS_UNMESSAGABLE = 7
BIT_HAS_IS_UNMESSAGABLE = 8
BITFIELD_LAYOUT = (
# JSON key bit position
("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
("is_muted", BIT_IS_MUTED),
("via_mqtt", BIT_VIA_MQTT),
("is_favorite", BIT_IS_FAVORITE),
("is_ignored", BIT_IS_IGNORED),
("has_user", BIT_HAS_USER),
("is_licensed", BIT_IS_LICENSED),
("is_unmessagable", BIT_IS_UNMESSAGABLE),
("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
)
def _pack_bitfield(bf: dict[str, bool]) -> int:
out = 0
for key, shift in BITFIELD_LAYOUT:
if bf.get(key, False):
out |= (1 << shift)
return out
def _validate_node(node: dict[str, Any]) -> None:
"""Friendly errors so hand-editors get clear feedback."""
if "num" not in node or not isinstance(node["num"], str):
raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
if "long_name" not in node:
raise ValueError(f"node {node['num']}: missing 'long_name'")
if len(node["long_name"]) > 24:
raise ValueError(
f"node {node['num']}: long_name {node['long_name']!r} is "
f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
)
if "short_name" in node:
# short_name max_size:5 (incl. NUL) → 4 bytes of content.
# Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
# would slip past a `len(str) > 4` check. Always measure bytes.
b = node["short_name"].encode("utf-8")
if len(b) > 4:
raise ValueError(
f"node {node['num']}: short_name {node['short_name']!r} is "
f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
)
pk = node.get("public_key_hex", "")
if pk and len(pk) != 64:
raise ValueError(
f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
f"got {len(pk)} chars"
)
if pk:
try:
bytes.fromhex(pk)
except ValueError as e:
raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
def _resolve_time(
node: dict[str, Any],
field_absolute: str,
field_offset: str,
now_epoch: int,
) -> int:
"""If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
if field_absolute in node and node[field_absolute] is not None:
return int(node[field_absolute])
offset = node.get(field_offset, 0)
return max(0, int(now_epoch) - int(offset))
def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
_validate_node(node)
info = NodeInfoLite()
info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
info.long_name = node.get("long_name", "")
info.short_name = node.get("short_name", "")
# Enum lookups will raise ValueError on unknown names — that's exactly what we want.
info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
pk_hex = node.get("public_key_hex", "")
if pk_hex:
info.public_key = bytes.fromhex(pk_hex)
info.snr = float(node.get("snr", 0.0))
info.channel = int(node.get("channel", 0))
if "hops_away" in node:
# `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
# field implicitly sets HasField("hops_away") to True. No has_hops_away
# setter exists (unlike the C++ nanopb-generated header).
info.hops_away = int(node["hops_away"])
info.next_hop = int(node.get("next_hop", 0))
info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
info.bitfield = _pack_bitfield(node.get("bitfield", {}))
return info
def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
entry = NodePositionEntry()
entry.num = num
pl = PositionLite()
# Firmware stores lat/long as int32 in 1e-7 degrees.
pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
pl.altitude = int(pos.get("altitude", 0))
pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
entry.position.CopyFrom(pl)
return entry
def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
entry = NodeTelemetryEntry()
entry.num = num
dm = DeviceMetrics()
if "battery_level" in tel:
dm.battery_level = int(tel["battery_level"])
if "voltage" in tel:
dm.voltage = float(tel["voltage"])
if "channel_utilization" in tel:
dm.channel_utilization = float(tel["channel_utilization"])
if "air_util_tx" in tel:
dm.air_util_tx = float(tel["air_util_tx"])
if "uptime_seconds" in tel:
dm.uptime_seconds = int(tel["uptime_seconds"])
entry.device_metrics.CopyFrom(dm)
return entry
def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
entry = NodeEnvironmentEntry()
entry.num = num
em = EnvironmentMetrics()
if "temperature" in env:
em.temperature = float(env["temperature"])
if "relative_humidity" in env:
em.relative_humidity = float(env["relative_humidity"])
if "barometric_pressure" in env:
em.barometric_pressure = float(env["barometric_pressure"])
if "iaq" in env:
em.iaq = int(env["iaq"])
entry.environment_metrics.CopyFrom(em)
return entry
def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
# `StatusMessage` (mesh.proto:1445) has a single `string status` field.
entry = NodeStatusEntry()
entry.num = num
sm = StatusMessage()
if "status" in status:
sm.status = str(status["status"])
entry.status.CopyFrom(sm)
return entry
def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
"""Read a seed JSONL and return the encoded NodeDatabase bytes."""
lines = jsonl_path.read_text(encoding="utf-8").splitlines()
if not lines:
raise ValueError(f"{jsonl_path} is empty")
meta_line = lines[0]
meta_obj = json.loads(meta_line)
meta = meta_obj.get("_meta", {})
version = meta.get("version")
if version != 25:
raise ValueError(
f"{jsonl_path}: meta version is {version!r}; this compiler "
f"requires version=25. Regenerate the seed with the matching tooling."
)
db = NodeDatabase()
db.version = 25
for ln, raw in enumerate(lines[1:], start=2):
raw = raw.strip()
if not raw:
continue
try:
node = json.loads(raw)
except json.JSONDecodeError as e:
raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
# Header
info = _build_node_info_lite(node, now_epoch)
db.nodes.append(info)
# Satellites (nullable)
if node.get("position"):
db.positions.append(_build_position_entry(num, node["position"], now_epoch))
if node.get("telemetry"):
db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
if node.get("environment"):
db.environment.append(_build_environment_entry(num, node["environment"]))
if node.get("status"):
db.status.append(_build_status_entry(num, node["status"]))
return db.SerializeToString()
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
p.add_argument("--out", required=True, help="Output binary .proto path.")
p.add_argument(
"--now-epoch",
type=int,
default=None,
help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
)
args = p.parse_args(argv)
in_path = pathlib.Path(args.in_path)
if not in_path.is_file():
print(f"input not found: {in_path}", file=sys.stderr)
return 2
now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
try:
encoded = compile_jsonl_to_proto(in_path, now_epoch)
except ValueError as e:
print(f"ERROR: {e}", file=sys.stderr)
return 3
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(encoded)
print(
f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
boards/ThinkNode-M7.json
+42
View File
@@ -0,0 +1,42 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-D BOARD_HAS_PSRAM",
"-D ARDUINO_USB_CDC_ON_BOOT=0",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"psram_type": "qio_opi",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "ELECROW-ThinkNode-M7"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "ELECROW ThinkNode M7",
"upload": {
"flash_size": "8MB",
"maximum_ram_size": 524288,
"maximum_size": 8388608,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "https://www.elecrow.com",
"vendor": "ELECROW"
}
boards/nrf54l15dk.json
+26
View File
@@ -0,0 +1,26 @@
{
"build": {
"cpu": "cortex-m33",
"f_cpu": "128000000L",
"mcu": "nrf54l15",
"zephyr": {
"variant": "nrf54l15dk/nrf54l15/cpuapp"
}
},
"connectivity": ["bluetooth"],
"debug": {
"default_tools": ["jlink"],
"jlink_device": "nRF54L15_M33",
"svd_path": "nrf54l15.svd"
},
"frameworks": ["zephyr"],
"name": "Nordic nRF54L15-DK (PCA10156)",
"upload": {
"maximum_ram_size": 262144,
"maximum_size": 1572864,
"protocol": "jlink",
"protocols": ["jlink"]
},
"url": "https://www.nordicsemi.com/Products/nRF54L15",
"vendor": "Nordic Semiconductor"
}
boards/station-g3.json
+41
View File
@@ -0,0 +1,41 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "station-g3"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "BQ Station G3",
"upload": {
"flash_size": "16MB",
"maximum_ram_size": 327680,
"maximum_size": 16777216,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "",
"vendor": "BQ Consulting"
}
extra_scripts/nrf54l15_linker.py
+140
View File
@@ -0,0 +1,140 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
#
# post:extra_scripts/nrf54l15_linker.py
#
# Fix for Zephyr two-pass link on nRF54L15:
# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
# the SCons dependency chain is broken (final_ld_script Command never runs).
# This script adds a PreAction on the final firmware binary that runs the gcc
# preprocessing command directly (extracted from build.ninja) to generate
# zephyr/linker.cmd before the link step.
#
# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
import os
import re
import subprocess
Import("env")
if env.get("PIOENV") != "nrf54l15dk":
pass # Only for the nrf54l15dk environment
else:
def _extract_gcc_command(ninja_build):
"""Parse build.ninja to find the gcc -E command that generates linker.cmd.
The rule format depends on the host:
Windows (CMake's RunCMake wraps every command):
COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
POSIX (Linux/macOS — no wrapper):
COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
"""
in_rule = False
with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
for line in f:
# Detect start of the linker.cmd custom command rule
if not in_rule:
if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
in_rule = True
continue
stripped = line.strip()
if not stripped.startswith("COMMAND = "):
continue
command_val = stripped[len("COMMAND = ") :]
# On Windows the value is wrapped in `cmd.exe /C "..."` — strip
# the wrapper. On POSIX hosts the inner sequence is the value
# itself (no quoting layer).
m = re.search(r'/C\s+"(.*)"\s*$', command_val)
inner = m.group(1) if m else command_val
parts = inner.split(" && ")
cwd = None
gcc_cmd = None
for part in parts:
part = part.strip()
if part.startswith("cd /D "): # Windows form
cwd = part[len("cd /D ") :]
elif part.startswith("cd "): # POSIX form
cwd = part[len("cd ") :]
elif "arm-none-eabi-gcc" in part:
gcc_cmd = part
if not gcc_cmd:
raise RuntimeError(
"nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
% inner[:400]
)
return gcc_cmd, cwd
raise RuntimeError(
"nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
)
def _generate_linker_cmd(target, source, env):
"""Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
build_dir = env.subst("$BUILD_DIR")
zephyr_dir = os.path.join(build_dir, "zephyr")
linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
if os.path.exists(linker_cmd):
return # Already present — nothing to do
ninja_build = os.path.join(build_dir, "build.ninja")
if not os.path.exists(ninja_build):
raise RuntimeError(
"nRF54L15 linker fix: build.ninja not found at %s\n"
"Run a full build first so CMake generates the Ninja files."
% ninja_build
)
gcc_cmd, cwd = _extract_gcc_command(ninja_build)
run_cwd = cwd if cwd else zephyr_dir
print(
"==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
)
# gcc_cmd comes verbatim from our own build.ninja (never user input) and
# contains Windows-style paths with spaces that cannot be safely argv-split
# with shlex, so we run it via the platform shell. nosec/nosemgrep below
# acknowledge this deliberate, scoped use of shell=True.
result = subprocess.run( # nosec B602
gcc_cmd,
shell=True, # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
cwd=run_cwd,
capture_output=True,
text=True,
)
if result.returncode != 0:
print("GCC stdout:", result.stdout[:2000])
print("GCC stderr:", result.stderr[:2000])
raise RuntimeError(
"nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
% result.returncode
)
if not os.path.exists(linker_cmd):
raise RuntimeError(
"nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
% linker_cmd
)
print("==> linker.cmd generated successfully")
# Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
prog = env.get("PIOMAINPROG")
if prog:
env.AddPreAction(prog, _generate_linker_cmd)
else:
print(
"[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
)
env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)
mcp-server/.gitignore
+6
View File
@@ -7,6 +7,12 @@ __pycache__/
dist/
build/
# Persistent device-log capture (recorder + Datadog cursor).
# Cross-session JSONL streams written by the autouse Recorder singleton
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
# fixture truncate doesn't touch it.
.mtlog/
# Test harness artifacts
tests/report.html
tests/junit.xml
mcp-server/scripts/datadog-dashboard.json
+217
View File
@@ -0,0 +1,217 @@
{
"title": "Meshtastic Firmware — Recorder Stream",
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
"widgets": [
{
"definition": {
"title": "Free heap (bytes)",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"queries": [
{
"name": "free_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "label": "bytes" }
}
},
{
"definition": {
"title": "Heap slope (bytes/min) — last 1h",
"type": "query_value",
"precision": 0,
"requests": [
{
"queries": [
{
"name": "slope",
"data_source": "metrics",
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
"aggregator": "avg"
}
],
"response_format": "scalar"
}
],
"conditional_formats": [
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
]
}
},
{
"definition": {
"title": "Total heap (bytes)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "total_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Battery level (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "battery",
"data_source": "metrics",
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "min": "0", "max": "105" }
}
},
{
"definition": {
"title": "Air utilization (TX %)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "airutil",
"data_source": "metrics",
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Channel utilization (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "chutil",
"data_source": "metrics",
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Log volume by level",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"response_format": "timeseries",
"display_type": "bars",
"queries": [
{
"name": "log_count",
"data_source": "logs",
"indexes": ["*"],
"compute": { "aggregation": "count" },
"search": { "query": "service:meshtastic-firmware" },
"group_by": [
{
"facet": "@level",
"limit": 10,
"sort": { "order": "desc", "aggregation": "count" }
}
]
}
]
}
]
}
},
{
"definition": {
"title": "Recent ERROR / CRIT firmware logs",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "@port", "width": "auto" },
{ "field": "@level", "width": "auto" },
{ "field": "@thread", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
},
{
"definition": {
"title": "Recorder marker events",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware @level:MARK",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
}
],
"template_variables": [
{
"name": "port",
"prefix": "port",
"available_values": [],
"default": "*"
},
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
],
"layout_type": "ordered",
"notify_list": [],
"reflow_type": "auto"
}
mcp-server/scripts/mtlog_to_datadog.py
Executable
+389
View File
@@ -0,0 +1,389 @@
#!/usr/bin/env python3
"""Forward selected recorder JSONL streams to Datadog.
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
Logs Intake API and telemetry numerics to the Metrics v2 series API.
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
duplicate rows already shipped from the current live files.
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
If the recorder rotates before this process drains the live file, or the
forwarder is down across a rotation, those older rows are skipped.
Usage:
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
./scripts/mtlog_to_datadog.py --once # catch up + exit
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
The forwarder is a separate process by design — a Datadog outage or
auth failure must not backpressure the recorder. We exit non-zero on
fatal config errors (missing API key) and keep retrying on transient
network/HTTP errors.
"""
from __future__ import annotations
import argparse
import json
import os
import socket
import sys
import time
from pathlib import Path
from typing import Any, Iterator
try:
import requests
except ImportError:
print(
"requests is required. Install it in the mcp-server venv: "
"uv pip install requests",
file=sys.stderr,
)
sys.exit(2)
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
_METRICS_TPL = "https://api.{site}/api/v2/series"
_LOG_BATCH = 50
_METRICS_BATCH = 100
_MAX_RETRIES = 5
_RETRY_BASE_S = 1.5
# --- streaming JSONL with byte-position cursor -------------------------
class _StreamReader:
"""Reads a single rotating JSONL with cursor-based resume.
This tails only the live `.jsonl` file. The recorder rotates files
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
the live file shrinks abruptly. We detect that via inode change OR live
size < cursor position, and reset the live-file cursor to 0.
"""
def __init__(self, path: Path, cursor: dict[str, Any]):
self.path = path
self.cursor = cursor
def _state(self) -> tuple[int, int]:
"""Return (inode, size) for the live file. (0, 0) if missing."""
try:
st = self.path.stat()
return (st.st_ino, st.st_size)
except FileNotFoundError:
return (0, 0)
def iter_new_records(self) -> Iterator[dict[str, Any]]:
ino, size = self._state()
last_ino = self.cursor.get("ino")
last_pos = int(self.cursor.get("pos") or 0)
if ino == 0:
return
if last_ino is not None and last_ino != ino:
# Rotation happened. Start over.
last_pos = 0
if last_pos > size:
# Live file truncated/shrunk under us — recorder rotated.
last_pos = 0
try:
with self.path.open("r", encoding="utf-8") as fh:
fh.seek(last_pos)
for line in fh:
line = line.rstrip("\n")
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
last_pos = fh.tell()
except FileNotFoundError:
return
self.cursor["ino"] = ino
self.cursor["pos"] = last_pos
def _load_cursor(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except (OSError, json.JSONDecodeError):
return {}
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(data, separators=(",", ":")))
tmp.replace(path)
# --- Datadog clients ---------------------------------------------------
class _DDSession:
"""Pool one HTTPS session, share retry logic."""
def __init__(self, api_key: str, site: str, hostname: str) -> None:
self.api_key = api_key
self.site = site
self.hostname = hostname
self.session = requests.Session()
self.session.headers.update(
{
"DD-API-KEY": api_key,
"Content-Type": "application/json",
}
)
def _post(self, url: str, payload: Any) -> bool:
for attempt in range(_MAX_RETRIES):
try:
resp = self.session.post(url, json=payload, timeout=30)
except requests.RequestException as e:
_wait_retry(attempt, f"network error: {e}")
continue
if 200 <= resp.status_code < 300:
return True
if resp.status_code in (408, 429, 500, 502, 503, 504):
_wait_retry(
attempt,
f"HTTP {resp.status_code} retrying",
)
continue
print(
f"datadog refused: {resp.status_code} {resp.text[:200]}",
file=sys.stderr,
)
return False
return False
def send_logs(self, records: list[dict[str, Any]]) -> int:
if not records:
return 0
url = _LOG_INTAKE_TPL.format(site=self.site)
sent = 0
for i in range(0, len(records), _LOG_BATCH):
batch = records[i : i + _LOG_BATCH]
if self._post(url, batch):
sent += len(batch)
return sent
def send_metrics(self, series: list[dict[str, Any]]) -> int:
if not series:
return 0
url = _METRICS_TPL.format(site=self.site)
sent = 0
for i in range(0, len(series), _METRICS_BATCH):
batch = series[i : i + _METRICS_BATCH]
if self._post(url, {"series": batch}):
sent += len(batch)
return sent
def _wait_retry(attempt: int, reason: str) -> None:
wait = _RETRY_BASE_S * (2**attempt)
print(
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
file=sys.stderr,
)
time.sleep(wait)
# --- record → datadog payload ------------------------------------------
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
line = rec.get("line") or ""
tags = [
f"role:{rec.get('role')}",
f"port:{rec.get('port')}",
]
level = rec.get("level")
if level:
tags.append(f"level:{level}")
tag = rec.get("tag")
if tag:
tags.append(f"thread:{tag}")
return {
"ddsource": "meshtastic-firmware",
"service": "meshtastic-firmware",
"hostname": host,
"message": line,
"ddtags": ",".join(t for t in tags if t and "None" not in t),
"timestamp": int((rec.get("ts") or time.time()) * 1000),
"level": level,
}
def _telemetry_record_to_metrics(
rec: dict[str, Any], host: str
) -> list[dict[str, Any]]:
fields = rec.get("fields") or {}
if not isinstance(fields, dict):
return []
variant = rec.get("variant") or "unknown"
ts = int(rec.get("ts") or time.time())
out: list[dict[str, Any]] = []
tags = []
if rec.get("port"):
tags.append(f"port:{rec['port']}")
if rec.get("role"):
tags.append(f"role:{rec['role']}")
if rec.get("from_node"):
tags.append(f"from_node:{rec['from_node']}")
tags.append(f"variant:{variant}")
for field, value in fields.items():
if not isinstance(value, (int, float)) or isinstance(value, bool):
continue
metric = f"mesh.{variant}.{_metric_safe(field)}"
out.append(
{
"metric": metric,
"type": 3, # GAUGE
"points": [{"timestamp": ts, "value": float(value)}],
"tags": tags,
"resources": [{"type": "host", "name": host}],
}
)
return out
def _metric_safe(name: str) -> str:
# Lowercase, replace non-alnum with underscore for safe metric names.
return "".join(c.lower() if c.isalnum() else "_" for c in name)
# --- main loop ---------------------------------------------------------
def run(
log_dir: Path,
*,
once: bool,
since_seconds: float | None,
poll_interval: float,
dd: _DDSession,
) -> int:
cursor_path = log_dir / ".dd-cursor.json"
cursors = _load_cursor(cursor_path)
# `--since` overrides cursor: rewind to (now-since) timestamp.
# We can't seek by timestamp directly (cursor is byte position), so
# we just reset cursors to 0 and let the time filter in iter_new
# drop older records.
cutoff_ts: float | None = None
if since_seconds is not None:
cursors = {}
cutoff_ts = time.time() - since_seconds
sent_total = {"logs": 0, "telemetry": 0}
while True:
# logs.jsonl → DD logs
log_cursor = cursors.setdefault("logs", {})
log_batch: list[dict[str, Any]] = []
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
log_batch.append(_log_record_to_dd(rec, dd.hostname))
if log_batch:
n = dd.send_logs(log_batch)
sent_total["logs"] += n
print(f"logs: sent {n}/{len(log_batch)}")
# telemetry.jsonl → DD metrics
telem_cursor = cursors.setdefault("telemetry", {})
metric_series: list[dict[str, Any]] = []
for rec in _StreamReader(
log_dir / "telemetry.jsonl", telem_cursor
).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
if metric_series:
n = dd.send_metrics(metric_series)
sent_total["telemetry"] += n
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
_save_cursor(cursor_path, cursors)
if once:
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
return 0
time.sleep(poll_interval)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--log-dir",
default=str(_DEFAULT_LOG_DIR),
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--once", action="store_true", help="Catch up then exit")
mode.add_argument(
"--tail",
action="store_true",
help="Daemon: poll forever (default)",
)
parser.add_argument(
"--since",
type=float,
default=None,
help="Backfill last N seconds. Resets cursor.",
)
parser.add_argument(
"--poll-interval",
type=float,
default=5.0,
help="Seconds between tail polls (default 5)",
)
parser.add_argument(
"--site",
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
help=(
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
"Override via DD_SITE env var or this flag."
),
)
parser.add_argument(
"--host",
default=socket.gethostname(),
help="Hostname tag (default: socket.gethostname())",
)
args = parser.parse_args(argv)
api_key = os.environ.get("DD_API_KEY")
if not api_key:
print("DD_API_KEY env var required.", file=sys.stderr)
return 2
log_dir = Path(args.log_dir)
if not log_dir.exists():
print(
f"log dir {log_dir} does not exist — start the mcp-server first.",
file=sys.stderr,
)
return 2
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
once = args.once and not args.tail
return run(
log_dir,
once=once,
since_seconds=args.since,
poll_interval=args.poll_interval,
dd=dd,
)
if __name__ == "__main__":
sys.exit(main())
mcp-server/src/meshtastic_mcp/fixtures.py
+382
View File
@@ -0,0 +1,382 @@
"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
The fixture pipeline is two-stage:
1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
protobuf with fresh wall-clock timestamps.
This module exposes `push_fake_nodedb(...)`, the MCP tool that:
- target="portduino": compiles the JSONL into the device's prefs dir on
the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
- target="hardware": compiles to a temp file, then streams it over the
XModem protocol (via the meshtastic SerialInterface/BLEInterface +
`meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
device. Triggers a reboot so the firmware loads the new state on next
boot.
XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
* 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
* CRC16-CCITT (poly 0x1021, init 0x0000).
* SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
`FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
* SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
* EOT after the last chunk flushes + closes the file on-device.
Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
in the .github/copilot-instructions.md "never do these without asking" list).
"""
from __future__ import annotations
import dataclasses
import hashlib
import pathlib
import queue
import shutil
import subprocess
import sys
import tempfile
import time
from typing import Any, Literal
from .connection import connect, is_tcp_port
# Resolve repo root so the tool works regardless of mcp-server cwd.
_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
_XMODEM_CHUNK = 128
_XMODEM_SUB = 0x1A
_ACK_TIMEOUT_INIT_S = 5.0
_ACK_TIMEOUT_CHUNK_S = 2.0
_MAX_CHUNK_RETRIES = 5
_VALID_SIZES = (250, 500, 1000, 2000)
class FixtureError(RuntimeError):
"""Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
# ---------------------------------------------------------------------------
# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
# Hand-rolled to avoid the optional `crcmod` dep.
# ---------------------------------------------------------------------------
def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
crc = init
for b in data:
crc ^= b << 8
for _ in range(8):
if crc & 0x8000:
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
else:
crc = (crc << 1) & 0xFFFF
return crc
# ---------------------------------------------------------------------------
# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
# doesn't have to duplicate the proto-encoding logic.
# ---------------------------------------------------------------------------
def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
if not _COMPILE_SCRIPT.is_file():
raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
cmd = [
sys.executable,
str(_COMPILE_SCRIPT),
"--in",
str(jsonl_path),
"--out",
str(out_path),
]
try:
subprocess.run(cmd, check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as exc:
raise FixtureError(
f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
f" stdout: {exc.stdout}\n stderr: {exc.stderr}"
) from exc
def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
if custom is not None:
p = pathlib.Path(custom).expanduser().resolve()
if not p.is_file():
raise FixtureError(f"custom_seed_jsonl not found: {p}")
return p
p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
if not p.is_file():
raise FixtureError(
f"missing committed seed at {p}. "
f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
)
return p
# ---------------------------------------------------------------------------
# Portduino push — file copy into ~/.portduino/<config>/prefs/
# ---------------------------------------------------------------------------
def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
home = pathlib.Path.home()
return home / ".portduino" / config_name / "prefs"
def _push_portduino(
size: int,
jsonl: pathlib.Path,
portduino_config: str,
backup_existing: bool,
) -> dict[str, Any]:
prefs = _portduino_prefs_dir(portduino_config)
prefs.mkdir(parents=True, exist_ok=True)
target = prefs / "nodes.proto"
backed_up_to: str | None = None
if backup_existing and target.is_file():
ts = int(time.time())
backup = prefs / f"nodes.proto.bak.{ts}"
shutil.move(str(target), str(backup))
backed_up_to = str(backup)
_compile_proto(jsonl, target)
raw = target.read_bytes()
return {
"transport": "portduino",
"path": str(target),
"bytes": len(raw),
"sha256": hashlib.sha256(raw).hexdigest(),
"jsonl_source": str(jsonl),
"backed_up_to": backed_up_to,
}
# ---------------------------------------------------------------------------
# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
# ---------------------------------------------------------------------------
@dataclasses.dataclass
class _AckEvent:
control: int
seq: int
def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
try:
return q.get(timeout=timeout_s)
except queue.Empty as exc:
raise FixtureError(
f"XModem response timeout after {timeout_s:.1f}s — device not responding"
) from exc
def _push_hardware(
size: int,
jsonl: pathlib.Path,
port: str | None,
reboot_after: bool,
) -> dict[str, Any]:
# Lazy imports so the module loads even when the meshtastic deps aren't
# available (e.g. CI in a Python env without the package installed).
try:
from meshtastic.protobuf import mesh_pb2, xmodem_pb2
from pubsub import pub
except ImportError as exc: # pragma: no cover — dep missing
raise FixtureError(
f"hardware push requires the meshtastic + pypubsub packages: {exc}"
) from exc
if is_tcp_port(port):
raise FixtureError(
"hardware push over TCP/portduino is not supported — use "
"target='portduino' to drop the fixture directly into the prefs dir."
)
# Compile the fixture to a temp file with fresh timestamps.
with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
proto_path = pathlib.Path(tf.name)
try:
_compile_proto(jsonl, proto_path)
payload = proto_path.read_bytes()
finally:
proto_path.unlink(missing_ok=True)
sha256 = hashlib.sha256(payload).hexdigest()
total_bytes = len(payload)
# Subscribe to XModem responses BEFORE we open the interface, so we don't
# race the first ACK that arrives during the SOH/seq=0 handshake.
#
# NB: the signature MUST declare every kwarg pypubsub will see for this
# topic, or pubsub locks the topic spec to a smaller set (whichever
# subscribe arrives first) and then *rejects* the meshtastic library's
# publish call with `SenderUnknownMsgDataError: unknown ... interface`.
# The meshtastic lib publishes both `packet=` and `interface=`
# (mesh_interface.py:1389-1395), so both must appear here.
response_q: "queue.Queue[_AckEvent]" = queue.Queue()
def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
if packet is None:
return
response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
chunks_sent = 0
retried = 0
rebooted = False
XMC = xmodem_pb2.XModem.Control
try:
with connect(port=port) as iface:
# 1) Send the filename (SOH, seq=0).
init_pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=0,
buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(
f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
f"(got control={ack.control}, expected ACK). "
f"Filesystem full or permissions issue?"
)
# 2) Stream the payload in 128 B chunks.
for offset in range(0, total_bytes, _XMODEM_CHUNK):
chunk = payload[offset : offset + _XMODEM_CHUNK]
if len(chunk) < _XMODEM_CHUNK:
# Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
# become part of the file on-device, but nanopb ignores
# bytes past the end of the top-level message.
chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
seq = ((offset // _XMODEM_CHUNK) + 1) % 256
# Retry loop on NAK / timeout.
attempts = 0
while True:
pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=seq,
buffer=chunk,
crc16=_crc16_ccitt(chunk),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control == XMC.Value("ACK"):
chunks_sent += 1
break
if ack.control == XMC.Value("NAK"):
attempts += 1
retried += 1
if attempts >= _MAX_CHUNK_RETRIES:
# Abort: send CAN so the firmware removes the half-
# written file via FSCom.remove(filename).
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(
control=XMC.Value("CAN")
)
)
)
raise FixtureError(
f"chunk seq={seq} NAK'd {attempts} times; "
f"aborted transfer (file removed on-device)."
)
continue # retry the same chunk
raise FixtureError(
f"unexpected XModem control={ack.control} on seq={seq}"
)
# 3) Tell the device we're done.
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
)
)
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(f"EOT not ACKed (got control={ack.control})")
# 4) Reboot so loadFromDisk picks up the new file.
if reboot_after:
iface.localNode.reboot(secs=1)
rebooted = True
finally:
try:
pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
except Exception:
pass
return {
"transport": "hardware",
"port": port,
"filename_on_device": _DEFAULT_NODES_FILENAME,
"bytes": total_bytes,
"chunks_sent": chunks_sent,
"retried": retried,
"sha256": sha256,
"jsonl_source": str(jsonl),
"rebooted": rebooted,
}
# ---------------------------------------------------------------------------
# Public entry point — registered as an MCP tool in server.py.
# ---------------------------------------------------------------------------
def push_fake_nodedb(
size: int,
target: Literal["portduino", "hardware"] = "portduino",
*,
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
Args:
size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
"hardware" (XModem upload to /prefs/nodes.proto + reboot).
port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
or BLE identifier. TCP endpoints are rejected — use target="portduino"
instead.
portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
if present, so you can roll back.
confirm: required True for target="hardware" (writes flash + reboots).
reboot_after: hardware only. If True, send a 1-second reboot after the
final ACK so loadFromDisk picks up the new file at next boot.
custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
test scenario.
Returns:
dict with transport, bytes, sha256, etc. — depends on target.
"""
if size not in _VALID_SIZES:
raise FixtureError(
f"size must be one of {_VALID_SIZES}; got {size!r}. "
f"Add a new committed seed if you need a different cardinality."
)
jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
if target == "portduino":
return _push_portduino(size, jsonl, portduino_config, backup_existing)
if target == "hardware":
if not confirm:
raise FixtureError(
"hardware push writes flash and triggers a reboot — pass confirm=True."
)
if not port:
raise FixtureError(
"target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
)
return _push_hardware(size, jsonl, port, reboot_after)
raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")
mcp-server/src/meshtastic_mcp/flash.py
+44 -1
View File
@@ -108,18 +108,33 @@ def build(
env: str,
with_manifest: bool = True,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Run `pio run -e <env>` and return artifact paths.
`userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
into userPrefs.jsonc for this build only. File is restored byte-for-byte
on exit. Use `userprefs_set()` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
`{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
prefix on every log line. Combines with the recorder so heap shows
up at log cadence (much higher resolution than the ~60 s LocalStats
packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
expand to bare `-D<NAME>` (presence-only flags).
"""
args = ["run", "-e", env]
if with_manifest:
args.extend(["-t", "mtjson"])
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
result = pio.run(
args,
timeout=pio.TIMEOUT_BUILD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
"artifacts": [str(p) for p in _artifacts_for(env)],
@@ -127,9 +142,27 @@ def build(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
"""Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
"-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
the flag entirely. Other types stringify."""
parts: list[str] = []
for key, value in build_flags.items():
if value is False or value is None:
continue
if value is True:
parts.append(f"-D{key}")
else:
parts.append(f"-D{key}={value}")
if not parts:
return {}
return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
def clean(env: str) -> dict[str, Any]:
"""Run `pio run -e <env> -t clean`."""
result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -146,20 +179,29 @@ def flash(
port: str,
confirm: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""`pio run -e <env> -t upload --upload-port <port>`. All architectures.
`userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
that pio performs will pick up the injected values.
`build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
is exported for the rebuild-before-upload, so the uploaded firmware
actually carries the flags. Without this propagation, `pio run -t upload`
would relink without the env var and silently drop them. Common use:
`build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
"""
_require_confirm(confirm, "flash")
_reject_native_env(env, "flash")
connection.reject_if_tcp(port, "flash")
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
["run", "-e", env, "-t", "upload", "--upload-port", port],
timeout=pio.TIMEOUT_UPLOAD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
@@ -167,6 +209,7 @@ def flash(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
mcp-server/src/meshtastic_mcp/log_query.py
+410
View File
@@ -0,0 +1,410 @@
"""Read-side queries over the recorder's JSONL streams.
Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
loads a whole file. Time-bound queries short-circuit as soon as `ts`
exceeds the requested end. The recorder writes monotonically, so a
forward scan is cheap; we don't need an index.
All time arguments accept:
- epoch seconds (int/float)
- relative strings: "-15m", "-2h", "-3d", "now"
- ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
treated as UTC)
Tools that return data ALWAYS cap their output (max_lines / max_points
/ max), and report whether more matched than was returned.
"""
from __future__ import annotations
import gzip
import json
import re
import statistics
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterator
from .recorder.recorder import get_recorder
_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
_REGEX_PREVIEW_MAX = 100
_REGEX_PREVIEW_TRUNCATE = 97
def _parse_time(value: Any, *, now: float | None = None) -> float:
"""Coerce to epoch seconds. Defaults `now` to `time.time()`."""
if value is None:
return time.time()
if isinstance(value, (int, float)):
return float(value)
if not isinstance(value, str):
raise ValueError(f"invalid time: {value!r}")
s = value.strip().lower()
if s in ("", "now"):
return time.time() if now is None else now
m = _REL_RE.match(s)
if m:
n = float(m.group(1))
unit = m.group(2)
secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
base = time.time() if now is None else now
return base - secs
# Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
try:
if s.endswith("z"):
s = s[:-1] + "+00:00"
dt = datetime.fromisoformat(s)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.timestamp()
except ValueError as e:
raise ValueError(f"unparseable time: {value!r}") from e
def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
"""Stream records in chronological order: rotated archives first
(oldest → newest by lex sort, which is chronological for our
`YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
last. The "keep last N" pop-front logic in the window queries
relies on records arriving in time order across files.
"""
files: list[Path] = []
# Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
files.append(archive)
if path.exists():
files.append(path)
for f in files:
opener = gzip.open if f.suffix == ".gz" else open
try:
with opener(f, "rt", encoding="utf-8") as fh: # type: ignore[arg-type]
for line in fh:
line = line.strip()
if not line:
continue
try:
rec = json.loads(line)
except json.JSONDecodeError:
continue
ts = rec.get("ts")
if not isinstance(ts, (int, float)):
continue
if ts < since:
continue
if ts > until:
# Records are append-monotonic within a file, so
# the rest of this file is also past `until`.
# Archives can still overlap each other, so only
# short-circuit this file, not the whole scan.
break
yield rec
except (FileNotFoundError, OSError):
continue
# -- queries ------------------------------------------------------------
def logs_window(
start: Any = "-15m",
end: Any = "now",
*,
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines, filtered.
`level` accepts a single level name or pipe-separated set
("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
`line` field. Returns the last `max_lines` matches.
"""
s = _parse_time(start)
e = _parse_time(end)
levels = _split_set(level)
if grep:
try:
grep_re = re.compile(grep)
except re.error as exc:
preview = (
grep
if len(grep) <= _REGEX_PREVIEW_MAX
else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
)
raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
else:
grep_re = None
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
if levels and rec.get("level") not in levels:
continue
if tag and rec.get("tag") != tag:
continue
if port and rec.get("port") != port:
continue
if grep_re and not grep_re.search(rec.get("line") or ""):
continue
matched += 1
out.append(rec)
if len(out) > max_lines:
out.pop(0) # keep the most recent N
return {
"lines": out,
"total_matched": matched,
"dropped": max(0, matched - max_lines),
"window": {"start": s, "end": e},
}
def telemetry_timeline(
window: Any = "1h",
*,
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Timeseries of one telemetry field, downsampled.
`field` matches both the protobuf snake_case name (`free_heap`,
`heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
Server-side bucket-mean downsamples to ≤ `max_points`. Returns
`slope_per_min` (linear regression slope, units/min) so a leak
detector can read one number.
"""
end = time.time()
if isinstance(window, (int, float)):
# Numeric `window` is a duration in seconds — "last N seconds".
# Without this branch, `_parse_time(-N)` would treat -N as an
# absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
# producing a wildly negative `start` and matching nothing.
start = end - float(window)
elif isinstance(window, str) and not window.startswith("-"):
# Bare string like "1h" is sugar for "-1h".
start = _parse_time(f"-{window}", now=end)
else:
start = _parse_time(window, now=end)
base = get_recorder().base_dir
raw: list[tuple[float, float]] = []
field_aliases = _field_aliases(field)
for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
if rec.get("variant") != variant:
continue
if port and rec.get("port") != port:
continue
fields = rec.get("fields") or {}
value: Any = None
for alias in field_aliases:
if alias in fields:
value = fields[alias]
break
if not isinstance(value, (int, float)):
continue
raw.append((float(rec["ts"]), float(value)))
if not raw:
return {
"points": [],
"samples": 0,
"min": None,
"max": None,
"slope_per_min": None,
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
points = _downsample(raw, max_points=max_points)
values = [v for _, v in raw]
return {
"points": [{"ts": ts, "value": v} for ts, v in points],
"samples": len(raw),
"min": min(values),
"max": max(values),
"slope_per_min": _slope_per_min(raw),
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
def packets_window(
start: Any = "-5m",
end: Any = "now",
*,
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
portnums = _split_set(portnum)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
if portnums and rec.get("portnum") not in portnums:
continue
if from_node and str(rec.get("from_node")) != str(from_node):
continue
if to_node and str(rec.get("to_node")) != str(to_node):
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"packets": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def events_window(
start: Any = "-1h",
end: Any = "now",
*,
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
kinds = _split_set(kind)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
if kinds and rec.get("kind") not in kinds:
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"events": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def export(
start: Any,
end: Any,
dest_dir: str,
*,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of each requested stream into `dest_dir`.
For a notebook, a bug report, or a Datadog backfill. Output files
are uncompressed JSONL (callers gzip themselves if they want to).
"""
s = _parse_time(start)
e = _parse_time(end)
selected = streams or ["logs", "telemetry", "packets", "events"]
dest = Path(dest_dir)
dest.mkdir(parents=True, exist_ok=True)
base = get_recorder().base_dir
paths: dict[str, str] = {}
for stream in selected:
src = base / f"{stream}.jsonl"
if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
continue
out_path = dest / f"{stream}.jsonl"
n = 0
with out_path.open("w", encoding="utf-8") as fh:
for rec in _iter_jsonl(src, since=s, until=e):
fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
n += 1
paths[stream] = str(out_path)
paths[f"{stream}_count"] = str(n)
return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
# -- helpers ------------------------------------------------------------
def _split_set(value: str | None) -> set[str] | None:
if not value:
return None
return {v.strip() for v in value.split("|") if v.strip()}
def _field_aliases(field: str) -> list[str]:
"""Accept snake_case OR camelCase, plus a few legacy aliases."""
snake = field
camel = _snake_to_camel(field)
aliases = {snake, camel}
# Old protobuf fields (pre-LocalStats) used different names
legacy = {
"free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
"heap_free_bytes": [
"heap_free_bytes",
"heapFreeBytes",
"free_heap",
"freeHeap",
],
"total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
"heap_total_bytes": [
"heap_total_bytes",
"heapTotalBytes",
"total_heap",
"totalHeap",
],
}
if field in legacy:
aliases.update(legacy[field])
return list(aliases)
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _downsample(
points: list[tuple[float, float]], *, max_points: int
) -> list[tuple[float, float]]:
if len(points) <= max_points:
return points
# Even-bucket mean. Preserves shape better than nth-sample picking.
n = len(points)
bucket = n / max_points
out: list[tuple[float, float]] = []
i = 0
for k in range(max_points):
end = int((k + 1) * bucket)
end = min(end, n)
if end <= i:
continue
chunk = points[i:end]
ts = chunk[len(chunk) // 2][0]
val = statistics.fmean(v for _, v in chunk)
out.append((ts, val))
i = end
return out
def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
"""Least-squares slope (units per minute). None if too few points."""
if len(points) < 2:
return None
xs = [t for t, _ in points]
ys = [v for _, v in points]
n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
den = sum((x - mean_x) ** 2 for x in xs)
if den == 0:
return None
slope_per_sec = num / den
return slope_per_sec * 60.0
mcp-server/src/meshtastic_mcp/pio.py
+15
View File
@@ -92,6 +92,7 @@ def _run_capturing(
cwd: Path | None = None,
timeout: float | None = None,
tee_header: str | None = None,
extra_env: dict[str, str] | None = None,
) -> tuple[int, str, str, float]:
"""Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
@@ -99,6 +100,9 @@ def _run_capturing(
`subprocess.TimeoutExpired` on timeout (callers map this to their own
domain-specific error).
`extra_env` merges into the subprocess environment (parent env stays
intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
Fast path: `subprocess.run(capture_output=True)` when no flash log is
configured (unchanged behavior).
@@ -110,6 +114,9 @@ def _run_capturing(
"""
log_path = _flash_log_path()
t0 = time.monotonic()
env = None
if extra_env:
env = {**os.environ, **extra_env}
if log_path is None:
# Fast path — unchanged.
@@ -119,6 +126,7 @@ def _run_capturing(
capture_output=True,
text=True,
timeout=timeout,
env=env,
)
return (
proc.returncode,
@@ -145,6 +153,7 @@ def _run_capturing(
stderr=subprocess.PIPE,
text=True,
bufsize=1, # line-buffered
env=env,
)
stdout_chunks: list[str] = []
stderr_chunks: list[str] = []
@@ -232,12 +241,17 @@ def run(
cwd: Path | None = None,
timeout: float | None = TIMEOUT_DEFAULT,
check: bool = True,
extra_env: dict[str, str] | None = None,
) -> PioResult:
"""Invoke `pio <args>` and return captured output.
`cwd` defaults to the firmware root. `check=True` raises `PioError` on
non-zero exit; set `check=False` to inspect `returncode` manually.
`extra_env` merges into the subprocess environment — used for
`PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
toggles that can't be expressed as command-line args.
If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
line-by-line as it arrives (for live flash progress in the TUI).
"""
@@ -250,6 +264,7 @@ def run(
cwd=work_dir,
timeout=timeout,
tee_header=f"pio {' '.join(args)}",
extra_env=extra_env,
)
except subprocess.TimeoutExpired as exc:
raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc
mcp-server/src/meshtastic_mcp/recorder/__init__.py
+19
View File
@@ -0,0 +1,19 @@
"""Persistent device-log capture.
Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
process-global so a single subscription captures every active interface
(serial / TCP / BLE) without any per-connection bookkeeping.
The recorder is opt-in-by-import: importing this package is a no-op; call
`get_recorder().start()` (which `server.py` does at FastMCP app init) to
begin writing. `pause()` / `resume()` exist for the rare case the user
wants a clean stretch of file (e.g. capturing a known-good baseline).
"""
from __future__ import annotations
from .recorder import Recorder, get_recorder
__all__ = ["Recorder", "get_recorder"]
mcp-server/src/meshtastic_mcp/recorder/parsers.py
+309
View File
@@ -0,0 +1,309 @@
"""Best-effort parsers for log lines and telemetry packets.
Two flavors of log line cross our pubsub subscription:
1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
accumulates bytes between protobuf frames and emits the full
firmware-formatted line, e.g.
"INFO | 12:34:56 12345 [Main] Booting"
— level, HH:MM:SS, uptime seconds, thread bracket, then message.
2. LogRecord protobuf path (debug_log_api enabled): the lib calls
`_handleLogLine(record.message)` with ONLY the message body. The
level/source/time fields on the LogRecord are dropped before
pubsub fan-out. We get e.g. just "Booting".
Both arrive on `meshtastic.log.line`. The parser tries to recover a
level + thread when the prefix is present and falls back to level=None
otherwise. Consumers who want level filtering on protobuf-mode hosts
should grep the raw `line` field instead.
Telemetry: `meshtastic.receive.telemetry` packets carry one of several
metric variants in `packet["decoded"]["telemetry"]`. We flatten the
chosen variant into a {field: value} dict so callers don't have to
know the protobuf shape.
"""
from __future__ import annotations
import re
from typing import Any
# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
# macros must stay in sync with these strings.
_LINE_RE = re.compile(
r"""
^
(?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
\s*\|\s*
(?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
\s+
(?P<uptime>\d+)
\s+
(?:\[(?P<thread>[^\]]+)\]\s+)?
(?P<msg>.*)
$
""",
re.VERBOSE,
)
# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
# thread bracket. See src/RedirectablePrint.cpp:175.
_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
# "++++++ Thread NAME freed heap A -> B (delta) ++++++"
_THREAD_HEAP_RE = re.compile(
r"""
^[\-+]+\s*
Thread\s+(?P<thread>\S+)\s+
(?P<kind>leaked|freed)\s+heap\s+
(?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
\((?P<delta>-?\d+)\)
""",
re.VERBOSE,
)
# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
_HEAP_STATUS_RE = re.compile(
r"""
Heap\s+status:\s+
(?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
(?:\s+\((?P<delta>-?\d+)\))?
""",
re.VERBOSE,
)
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
def parse_log_line(line: str) -> dict[str, Any]:
"""Best-effort decompose a raw firmware log line.
Returns a dict with at least `line` (the original, unmodified — ANSI
codes preserved for fidelity). Adds `level`, `tag`, `clock`,
`uptime_s`, and `msg` when the full prefix is present.
Handles two firmware quirks:
- LogRecord.message can carry ANSI color escapes from RedirectablePrint
(the BLE/StreamAPI path inherited the colored body in some builds).
We strip ANSI before regex matching so the prefix survives.
- DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
thread name is set, the heap takes the thread bracket position —
looks like `[heap 12345] msg`. We detect that shape and move it
out of `tag` and into `heap_free`.
DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
(bytes), and when a `Thread X leaked|freed heap` line is recognized,
`heap_event` = {kind, thread, before, after, delta}.
Never raises.
"""
out: dict[str, Any] = {"line": line}
if not line:
return out
# Strip ANSI escapes BEFORE any regex matching. The original `line`
# stays in `out["line"]` for fidelity / future grep.
clean = _ANSI_RE.sub("", line)
m = _LINE_RE.match(clean)
msg: str | None = None
if m:
level = m.group("level").rstrip()
out["level"] = level
out["clock"] = m.group("clock")
try:
out["uptime_s"] = int(m.group("uptime"))
except (TypeError, ValueError):
out["uptime_s"] = None
thread = m.group("thread")
if thread:
# If "thread" is actually the heap prefix taking the bracket
# position (DEBUG_HEAP build, no thread set), capture heap
# and leave tag unset.
hb = _HEAP_BRACKET_RE.match(thread.strip())
if hb:
try:
out["heap_free"] = int(hb.group("heap"))
except (TypeError, ValueError):
pass
else:
out["tag"] = thread
msg = m.group("msg")
out["msg"] = msg
else:
# No prefix — bare LogRecord.message body. Inspect the whole
# line for DEBUG_HEAP-style content; the heap-prefix and
# thread-leak patterns can survive on either path.
msg = clean
# DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
# Sits AFTER the thread bracket and BEFORE the message body, but
# for bare LogRecord lines it's at the start. Match it at the
# head of `msg`.
if msg:
hp = _HEAP_PREFIX_RE.match(msg)
if hp:
try:
out["heap_free"] = int(hp.group("heap"))
except (TypeError, ValueError):
pass
else:
# Strip the prefix from `msg` so a grep on the message
# body doesn't have to know about it.
out["msg"] = hp.group("rest")
msg = hp.group("rest")
# Thread-level leak/free detection.
thr = _THREAD_HEAP_RE.search(msg)
if thr:
try:
out["heap_event"] = {
"kind": thr.group("kind"),
"thread": thr.group("thread"),
"before": int(thr.group("before")),
"after": int(thr.group("after")),
"delta": int(thr.group("delta")),
}
except (TypeError, ValueError):
pass
# Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
hs = _HEAP_STATUS_RE.search(msg)
if hs:
try:
out["heap_free"] = int(hs.group("free"))
out["heap_total"] = int(hs.group("total"))
if hs.group("delta") is not None:
out["heap_delta"] = int(hs.group("delta"))
except (TypeError, ValueError):
pass
return out
# -- Telemetry ----------------------------------------------------------
# Order matters: meshtastic-python decoded packets use the protobuf
# `oneof variant` field name (snake_case) as the dict key.
_TELEMETRY_VARIANTS = (
("device_metrics", "device"),
("local_stats", "local"),
("environment_metrics", "environment"),
("power_metrics", "power"),
("air_quality_metrics", "airQuality"),
("health_metrics", "health"),
("host_metrics", "host"),
)
def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
"""Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
packet. Returns None when the shape isn't what we expect — so the
caller can fall back to a generic packets.jsonl row.
"""
if not isinstance(packet, dict):
return None
decoded = packet.get("decoded")
if not isinstance(decoded, dict):
return None
telem = decoded.get("telemetry")
if not isinstance(telem, dict):
return None
# The Python lib produces dict-of-camelCase keys via MessageToDict.
# Try both camelCase and snake_case to be robust to lib version drift.
for snake, label in _TELEMETRY_VARIANTS:
camel = _snake_to_camel(snake)
for key in (snake, camel):
value = telem.get(key)
if isinstance(value, dict):
return {
"variant": label,
"fields": {k: _scalarize(v) for k, v in value.items()},
"time": telem.get("time"),
}
return None
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _scalarize(value: Any) -> Any:
"""Keep telemetry fields JSON-friendly. Lists/dicts pass through
untouched; bytes -> hex string; protobuf enums occasionally arrive
as ints (fine) or strings (also fine)."""
if isinstance(value, (bytes, bytearray, memoryview)):
return bytes(value).hex()
return value
# -- Generic packet summary ---------------------------------------------
def summarize_packet(
packet: dict[str, Any], *, payload_hex_len: int = 64
) -> dict[str, Any]:
"""Reduce a packet dict to a stable, queryable summary. Drops the
full payload bytes — the recorder records summaries, not pcaps.
"""
if not isinstance(packet, dict):
return {"raw_type": type(packet).__name__}
decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
payload = decoded.get("payload") if isinstance(decoded, dict) else None
payload_hex = None
payload_size = None
if isinstance(payload, (bytes, bytearray, memoryview)):
b = bytes(payload)
payload_size = len(b)
payload_hex = b[:payload_hex_len].hex() if b else ""
elif isinstance(payload, str):
# Some decoded payloads (text messages) come as decoded strings.
payload_size = len(payload)
payload_hex = None # not bytes
return {
"from_node": packet.get("fromId") or packet.get("from"),
"to_node": packet.get("toId") or packet.get("to"),
"portnum": portnum,
"hop_limit": packet.get("hopLimit"),
"want_ack": packet.get("wantAck"),
"rx_rssi": packet.get("rxRssi"),
"rx_snr": packet.get("rxSnr"),
"channel": packet.get("channel"),
"id": packet.get("id"),
"payload_size": payload_size,
"payload_hex_prefix": payload_hex,
}
# -- Interface identification ------------------------------------------
def interface_label(interface: Any) -> dict[str, Any]:
"""Stable identifier for the meshtastic interface that emitted an event.
Used as the `port`/`role` tag on every recorded row. SerialInterface
has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
has `address`. Falls back to the class name when none of those exist.
"""
if interface is None:
return {"port": None, "role": None}
dev_path = getattr(interface, "devPath", None)
if dev_path:
return {"port": str(dev_path), "role": "serial"}
hostname = getattr(interface, "hostname", None)
if hostname:
port_num = getattr(interface, "portNumber", None)
endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
return {"port": endpoint, "role": "tcp"}
address = getattr(interface, "address", None)
if address:
return {"port": str(address), "role": "ble"}
return {"port": type(interface).__name__, "role": None}
mcp-server/src/meshtastic_mcp/recorder/recorder.py
+467
View File
@@ -0,0 +1,467 @@
"""Process-global recorder singleton.
Subscribes once to the meshtastic pubsub fan-out and writes four append-only
JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
process-global — a single subscription captures every active interface
without per-connection bookkeeping.
Files:
logs.jsonl — every `meshtastic.log.line` event (best-effort prefix
parsed for level/tag/uptime; raw `line` always preserved)
telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
variant (device / local / environment / power / etc.)
packets.jsonl — every other `meshtastic.receive.*` packet, summarized
(portnum, hops, RSSI/SNR, payload size + 64-byte hex)
events.jsonl — connection lifecycle, node-DB updates, and manual
`mark_event` rows. Lower volume; useful for aligning
timelines.
Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
write methods short-circuit when paused, so we don't lose ordering when
resumed (we just have a gap). No queueing.
"""
from __future__ import annotations
import logging
import os
import threading
import time
from pathlib import Path
from typing import Any
from . import parsers
from .rotating import _RotatingJsonl
_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
log = logging.getLogger(__name__)
class Recorder:
"""Singleton write-side of the persistent log capture system."""
def __init__(self, base_dir: Path | None = None) -> None:
self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
self._lock = threading.RLock()
self._started = False
self._paused = False
self._pause_reason: str | None = None
self._started_at: float | None = None
self._handlers: list[tuple[str, Any]] = []
self._files: dict[str, _RotatingJsonl] = {}
# -- lifecycle ----------------------------------------------------
def start(self) -> None:
"""Idempotent. Safe to call from FastMCP app startup."""
with self._lock:
if self._started:
return
self.base_dir.mkdir(parents=True, exist_ok=True)
self._files = {
"logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
"telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
"packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
"events": _RotatingJsonl(self.base_dir / "events.jsonl"),
}
self._wire_pubsub()
self._started = True
self._started_at = time.time()
# Write the recorder_start marker after the initialization block.
# `_write_event()` re-checks recorder state via `_files_snapshot()`,
# so keeping this out of the setup block avoids nested lifecycle work.
self._write_event(kind="recorder_start", label="recorder_started")
def stop(self) -> None:
with self._lock:
if not self._started:
return
self._unwire_pubsub()
for f in self._files.values():
f.close()
self._files = {}
self._started = False
def pause(self, reason: str | None = None) -> None:
# Write the pause marker BEFORE flipping the flag — `_write_event`
# short-circuits when paused, so the order matters for this event
# to actually land in events.jsonl.
self._write_event(
kind="recorder_pause",
label="paused",
note=reason,
)
with self._lock:
self._paused = True
self._pause_reason = reason
def resume(self) -> None:
# Mirror of `pause()`: clear the flag first, then write the marker
# so it isn't suppressed by the still-paused short-circuit.
with self._lock:
self._paused = False
self._pause_reason = None
self._write_event(kind="recorder_resume", label="resumed")
# -- pubsub wiring ------------------------------------------------
def _wire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
# Subscribers — one per topic. Each pubsub publisher sends
# keyword args matching its handler's signature; pubsub
# introspects the function signature to route args.
bindings = [
("meshtastic.log.line", self._on_log_line),
("meshtastic.serial.line", self._on_serial_line),
("meshtastic.receive", self._on_receive),
("meshtastic.receive.telemetry", self._on_telemetry),
("meshtastic.connection.established", self._on_connection_established),
("meshtastic.connection.lost", self._on_connection_lost),
("meshtastic.node.updated", self._on_node_updated),
]
for topic, handler in bindings:
try:
pub.subscribe(handler, topic)
self._handlers.append((topic, handler))
except Exception as exc:
# If pubsub refuses one binding (signature mismatch on
# an old lib version), log it and keep the rest.
log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
def _unwire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
for topic, handler in self._handlers:
try:
pub.unsubscribe(handler, topic)
except Exception:
pass
self._handlers.clear()
# -- handlers -----------------------------------------------------
#
# Pubsub callbacks must never raise. Every handler is wrapped in a
# try/except that swallows so a bug here can't take down the
# SerialInterface receive thread.
#
# Threading: handlers fire on whatever thread the meshtastic library
# dispatches from (varies by interface), while `stop()` clears
# `self._files` under `self._lock`. We snapshot `_files` under the
# lock at the top of each handler so a concurrent stop can't
# KeyError us mid-write. The actual file write goes through
# `_RotatingJsonl` which has its own lock.
def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
"""Atomic-ish view of `self._files`. Returns None when the recorder
is paused or stopped, so handlers can early-exit cleanly without
racing `stop()`'s clear."""
with self._lock:
if not self._started or self._paused:
return None
return dict(self._files)
def _on_log_line(self, line: str, interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
# DEBUG_HEAP enrichments (only present when the firmware
# was built with -DDEBUG_HEAP=1). Surface as first-class
# fields so logs_window can grep/filter on them and so
# heap_free synthesizes a telemetry point below.
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# If the line carried a heap snapshot, also write it as a
# synthesized LocalStats-shaped row so telemetry_timeline
# picks it up at log cadence (much higher resolution than
# the ~60 s LocalStats packet). Tagged source=debug_heap so
# consumers can filter if mixing scales is unwanted.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap",
}
)
except Exception:
pass
def _on_serial_line(self, line: str, port: str | None = None) -> None:
"""Text-mode passive tap. Fired from `serial_session._drain` when a
`pio device monitor` subprocess is running.
Same parse + heap-synthesis path as `_on_log_line`, but receives
the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
body). On DEBUG_HEAP builds in text mode this gives us per-log-line
heap data — far higher cadence than LocalStats, and works without
protobuf API mode (no SerialInterface required).
"""
files = self._files_snapshot()
if files is None:
return
try:
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": port,
"role": "serial_session",
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# Synthesize a heap_free telemetry sample whenever the line
# carries one — same logic as _on_log_line, tagged source so
# consumers can distinguish text-mode tap from protobuf path.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": port,
"role": "serial_session",
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap_serial",
}
)
except Exception:
pass
def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
extracted = parsers.extract_telemetry(packet)
if extracted is None:
# Couldn't extract a known variant — fall through to the
# generic `_on_receive` path, which will still fire for
# this packet via the parent topic.
return
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
"from_node": packet.get("fromId") or packet.get("from"),
"variant": extracted["variant"],
"fields": extracted["fields"],
"device_time": extracted.get("time"),
}
files["telemetry"].write(record)
except Exception:
pass
def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
# Generic-receive fires for EVERY packet. Telemetry packets get
# recorded twice (here and in _on_telemetry) — that's intentional:
# packets.jsonl is the universal record, telemetry.jsonl is the
# structured timeseries view.
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
summary = parsers.summarize_packet(packet)
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
**summary,
}
files["packets"].write(record)
except Exception:
pass
def _on_connection_established(self, interface: Any = None) -> None:
self._write_event(
kind="connection_established",
interface=interface,
)
def _on_connection_lost(self, interface: Any = None) -> None:
self._write_event(
kind="connection_lost",
interface=interface,
)
def _on_node_updated(
self, node: dict[str, Any] | None = None, interface: Any = None
) -> None:
# Lower-volume than packets but informative — node ID, hops away,
# last heard. Skip the user dict if absent.
try:
user = (node or {}).get("user") if isinstance(node, dict) else None
self._write_event(
kind="node_updated",
interface=interface,
data={
"num": (node or {}).get("num"),
"id": (user or {}).get("id"),
"short": (user or {}).get("shortName"),
"long": (user or {}).get("longName"),
"hops_away": (node or {}).get("hopsAway"),
"snr": (node or {}).get("snr"),
"last_heard": (node or {}).get("lastHeard"),
},
)
except Exception:
pass
# -- public write helpers -----------------------------------------
def mark_event(
self,
label: str,
note: str | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""User-facing marker. Writes to events.jsonl AND emits a
synthetic logs.jsonl row tagged level=MARK so timelines align.
"""
ts = self._write_event(kind="mark", label=label, note=note, data=data)
# Mirror into logs so a single logs_window grep finds it.
files = self._files_snapshot()
if files is not None:
try:
files["logs"].write(
{
"ts": ts,
"port": None,
"role": "marker",
"level": "MARK",
"tag": "mark_event",
"line": f"[mark] {label}" + (f" — {note}" if note else ""),
}
)
except Exception:
pass
return {"ts": ts, "label": label}
def _write_event(
self,
*,
kind: str,
label: str | None = None,
note: str | None = None,
interface: Any = None,
data: dict[str, Any] | None = None,
) -> float:
ts = time.time()
# Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
# arrive at choreographed moments — `pause()` writes BEFORE flipping
# the flag and `resume()` writes AFTER clearing it, so those calls
# see _paused=False here. Other event kinds short-circuit when
# paused via the snapshot guard below.
files = self._files_snapshot()
if files is None:
return ts
try:
tags = parsers.interface_label(interface)
files["events"].write(
{
"ts": ts,
"kind": kind,
"label": label,
"note": note,
"port": tags["port"],
"role": tags["role"],
"data": data,
}
)
except Exception:
pass
return ts
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
return {
"running": self._started,
"paused": self._paused,
"pause_reason": self._pause_reason,
"started_at": self._started_at,
"base_dir": str(self.base_dir),
"files": {name: f.status() for name, f in self._files.items()},
}
def force_rotate_all(self) -> dict[str, Any]:
"""Test/admin hook: rotate every stream right now."""
with self._lock:
files = list(self._files.values())
for f in files:
f.force_rotate()
# `status()` re-acquires `self._lock`; release before calling it.
return self.status()
# -- module-level singleton accessor ------------------------------------
_INSTANCE_LOCK = threading.Lock()
_INSTANCE: Recorder | None = None
def get_recorder() -> Recorder:
"""Return the process-global Recorder. Created on first call.
Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
(used by tests to redirect to a tmpdir).
"""
global _INSTANCE
with _INSTANCE_LOCK:
if _INSTANCE is None:
override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
base = Path(override) if override else None
_INSTANCE = Recorder(base_dir=base)
return _INSTANCE
mcp-server/src/meshtastic_mcp/recorder/rotating.py
+163
View File
@@ -0,0 +1,163 @@
"""Append-only JSONL writer with size-capped rotation.
A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
JSON objects (one row per call). When the live file exceeds `max_bytes`,
it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
and the live file resets to empty. Old archives past `keep_archives` are
unlinked oldest-first.
Size check is amortized — `os.fstat` runs every `check_every` writes,
not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
Threading: every public method acquires `self._lock`. The recorder runs
several pubsub handlers on whatever thread the meshtastic library
dispatches from (varies by interface), and queries from MCP tool calls
arrive on the FastMCP request thread, so this lock is not optional.
"""
from __future__ import annotations
import gzip
import json
import os
import shutil
import threading
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
class _RotatingJsonl:
"""Append-only JSONL with size rotation. Thread-safe."""
def __init__(
self,
path: Path,
*,
max_bytes: int = 100 * 1024 * 1024,
keep_archives: int = 5,
check_every: int = 1000,
) -> None:
self.path = path
self.max_bytes = max_bytes
self.keep_archives = keep_archives
self.check_every = check_every
self._lock = threading.Lock()
self._fh: Any = None
self._writes_since_check = 0
self._rotations = 0
self._lines_written = 0
self._last_ts: float | None = None
self._open()
# -- lifecycle ----------------------------------------------------
def _open(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
self._fh = self.path.open("a", encoding="utf-8")
def close(self) -> None:
with self._lock:
if self._fh is not None:
try:
self._fh.close()
finally:
self._fh = None
# -- write --------------------------------------------------------
def write(self, record: dict[str, Any]) -> None:
"""Append one JSON object as a line. Triggers rotation if oversized."""
line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
with self._lock:
if self._fh is None:
return
try:
self._fh.write(line)
self._fh.flush()
except Exception:
# Best-effort: a failed write must not crash the pubsub
# handler. Caller has no way to react anyway.
return
self._lines_written += 1
ts = record.get("ts")
if isinstance(ts, (int, float)):
self._last_ts = float(ts)
self._writes_since_check += 1
if self._writes_since_check >= self.check_every:
self._writes_since_check = 0
self._maybe_rotate()
# -- rotation -----------------------------------------------------
def _maybe_rotate(self) -> None:
# Caller holds self._lock.
try:
size = os.fstat(self._fh.fileno()).st_size
except OSError:
return
if size < self.max_bytes:
return
self._rotate_locked()
def _rotate_locked(self) -> None:
# Close, gzip-rename, reopen empty, prune oldest archives.
try:
self._fh.close()
except Exception:
pass
self._fh = None
# Microsecond-resolution timestamp + per-instance counter so back-
# to-back rotations (small max_bytes, repeated `force_rotate()`,
# or chatty test loops) get unique archive filenames. The lex
# sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
# which `_prune_archives()` and `log_query._iter_jsonl()` both
# rely on.
stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
try:
with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
shutil.copyfileobj(src, dst, length=1024 * 1024)
self.path.unlink()
except Exception:
# Rotation is best-effort. If gzip fails, leave the file
# in place and re-open it; we'll try again next check.
pass
self._open()
self._rotations += 1
self._prune_archives()
def _prune_archives(self) -> None:
# Match siblings of self.path.name with `.jsonl.gz` suffix.
prefix = self.path.stem # "logs" for "logs.jsonl"
# Archive filenames are already lexicographically chronological.
# Prune by name, not mtime, so copied/restored files don't reorder.
archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
excess = len(archives) - self.keep_archives
for old in archives[: max(0, excess)]:
try:
old.unlink()
except OSError:
pass
def force_rotate(self) -> None:
"""Test/admin hook: rotate immediately regardless of size."""
with self._lock:
if self._fh is not None:
self._rotate_locked()
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
try:
size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
except OSError:
size = 0
return {
"path": str(self.path),
"size": size,
"lines": self._lines_written,
"last_ts": self._last_ts,
"rotations": self._rotations,
}
mcp-server/src/meshtastic_mcp/serial_session.py
+27 -1
View File
@@ -46,7 +46,23 @@ class SerialSession:
def _drain(session: SerialSession) -> None:
"""Reader thread: line-by-line pull stdout into buffer."""
"""Reader thread: line-by-line pull stdout into buffer.
Each line is also published to the `meshtastic.serial.line` pubsub
topic so the persistent recorder can capture it without holding its
own port. This is the text-mode tap path: when no SerialInterface is
open, the firmware emits full formatted lines (level + clock + uptime
+ thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
fan them out to whoever is listening. Pubsub is best-effort —
publish failures must never block the reader.
"""
# Lazy import: pubsub isn't required just to import this module
# (e.g., during static analysis), and we want a clean test surface.
try:
from pubsub import pub # type: ignore[import-untyped]
except Exception: # pragma: no cover - defensive
pub = None
assert session.proc.stdout is not None
try:
for line in session.proc.stdout:
@@ -54,6 +70,16 @@ def _drain(session: SerialSession) -> None:
with session.lock:
session.buffer.append(line_stripped)
session.total_lines += 1
if pub is not None:
try:
pub.sendMessage(
"meshtastic.serial.line",
line=line_stripped,
port=session.port,
)
except Exception:
# A subscriber raising must not break the reader.
pass
except Exception: # pragma: no cover - defensive
pass
finally:
mcp-server/src/meshtastic_mcp/server.py
+274 -2
View File
@@ -6,6 +6,7 @@ etc.). Business logic does not live here.
from __future__ import annotations
import logging
from typing import Any
from mcp.server.fastmcp import FastMCP
@@ -14,17 +15,38 @@ from . import (
admin,
boards,
devices,
fixtures,
flash,
hw_tools,
info,
log_query,
registry,
serial_session,
)
from . import userprefs as userprefs_mod
from .recorder import get_recorder
log = logging.getLogger(__name__)
app = FastMCP("meshtastic-mcp")
def _start_recorder() -> None:
# Persistent device-log capture. Starts on first import — pubsub fan-out
# is process-global, so subscribing here captures every active interface
# (whether opened by an MCP tool, a pytest fixture, or a serial_session).
# Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
# for the full design. Recorder startup is best-effort: an unwritable
# log dir or pubsub mismatch should not take the MCP server down.
try:
get_recorder().start()
except Exception as exc:
log.warning("Failed to start persistent recorder: %s", exc)
_start_recorder()
# ---------- Discovery & metadata ------------------------------------------
@@ -75,6 +97,7 @@ def build(
env: str,
with_manifest: bool = True,
userprefs: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Build firmware for one env via `pio run -e <env>`.
@@ -86,8 +109,21 @@ def build(
build via userPrefs.jsonc injection. The file is restored after the build
completes. Use `userprefs_manifest` to discover available keys. Use
`userprefs_set` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
`build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
`[heap N]` prefix on every log line. The recorder picks the prefix up
automatically and synthesizes a high-resolution heap timeline that
`telemetry_timeline(field="free_heap")` can read alongside the normal
~60 s LocalStats packets. Pair with `/leakhunt` for classification.
"""
return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
return flash.build(
env,
with_manifest=with_manifest,
userprefs_overrides=userprefs,
build_flags=build_flags,
)
@app.tool()
@@ -105,6 +141,7 @@ def pio_flash(
port: str,
confirm: bool = False,
userprefs: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
@@ -114,8 +151,19 @@ def pio_flash(
`userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
build via userPrefs.jsonc injection; restored after upload.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
to actually land in the uploaded firmware — without it, the implicit
rebuild relinks without the env var and silently drops them.
"""
return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
return flash.flash(
env,
port,
confirm=confirm,
userprefs_overrides=userprefs,
build_flags=build_flags,
)
@app.tool()
@@ -734,3 +782,227 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
"""Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
return hw_tools.picotool_raw(args, confirm=confirm)
# ---------- Persistent device-log capture (recorder) ----------------------
#
# The recorder is autouse — it starts at server import and continuously
# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
# tools are query-only over those files, plus a few lifecycle controls.
@app.tool()
def logs_window(
start: str = "-15m",
end: str = "now",
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines from the persistent recorder.
Filters by time window, regex over the line, level (single or
pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
interface port. Returns up to max_lines most-recent matches.
Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
Note: lines arriving via the LogRecord protobuf path (when
set_debug_log_api(True) is on) come without level prefix — the
meshtastic Python lib drops record.level before fan-out. For those,
`level` filter won't match; use `grep` instead.
"""
return log_query.logs_window(
start=start,
end=end,
grep=grep,
level=level,
tag=tag,
port=port,
max_lines=max_lines,
)
@app.tool()
def telemetry_timeline(
window: str = "1h",
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Time series of one telemetry field, downsampled to <= max_points.
`variant` ∈ device, local, environment, power, airQuality, health, host.
`field` accepts snake_case or camelCase; common aliases (free_heap ↔
heap_free_bytes) are normalized.
Returns slope_per_min (linear-regression slope, units/minute) so a
leak detector can read one number — negative slope on free_heap over
a long window indicates a real leak.
LocalStats variant ("local") cadence is ~60 s (whatever the device's
`device_update_interval` is set to), so a 1 h window gives ~60 raw
points. Bucket-mean downsampling preserves shape.
"""
return log_query.telemetry_timeline(
window=window,
variant=variant,
field=field,
port=port,
max_points=max_points,
)
@app.tool()
def packets_window(
start: str = "-5m",
end: str = "now",
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
"""Recent mesh packets recorded by the recorder.
Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
size + first 64 bytes hex) — full payload bytes are not stored.
`portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
"""
return log_query.packets_window(
start=start,
end=end,
portnum=portnum,
from_node=from_node,
to_node=to_node,
max=max,
)
@app.tool()
def events_window(
start: str = "-1h",
end: str = "now",
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
"""Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
`kind` ∈ recorder_start, recorder_pause, recorder_resume,
connection_established, connection_lost, node_updated, mark.
Pipe-separated sets ("connection_lost|connection_established") work.
"""
return log_query.events_window(start=start, end=end, kind=kind, max=max)
@app.tool()
def mark_event(
label: str,
note: str | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Drop a named marker into events.jsonl AND logs.jsonl.
Useful for aligning a timeline around a known stimulus: call before
and after a stress workload, then query telemetry_timeline /
logs_window with the markers' timestamps as bounds.
The marker also lands in logs.jsonl with level=MARK so a single
grep over logs picks it up.
"""
return get_recorder().mark_event(label=label, note=note, data=data)
@app.tool()
def recorder_status() -> dict[str, Any]:
"""Return recorder runtime info: running, paused, file sizes, last_ts per stream.
Use this to sanity-check that capture is working before you trust a
`logs_window` / `telemetry_timeline` result.
"""
return get_recorder().status()
@app.tool()
def recorder_pause(reason: str | None = None) -> dict[str, Any]:
"""Pause writes to all four streams. Pubsub subscriptions stay active —
we just drop events on the floor while paused. Resume with `recorder_resume`.
Use when capturing a known-good baseline that you don't want to
pollute with pre-test noise. Default state is recording; this is
rarely needed.
"""
get_recorder().pause(reason=reason)
return {"ok": True, "paused": True, "reason": reason}
@app.tool()
def recorder_resume() -> dict[str, Any]:
"""Resume writes after `recorder_pause`. No-op if already running."""
get_recorder().resume()
return {"ok": True, "paused": False}
@app.tool()
def recorder_export(
start: str,
end: str,
dest_dir: str,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of the recorder's streams into `dest_dir`.
Writes one uncompressed JSONL per requested stream (logs / telemetry /
packets / events). Useful for: attaching to a bug report, feeding a
notebook, or backfilling Datadog after the fact.
"""
return log_query.export(
start=start,
end=end,
dest_dir=dest_dir,
streams=streams,
)
# ---------- Fixture / test-data push --------------------------------------
@app.tool()
def push_fake_nodedb(
size: int,
target: str = "portduino",
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Push a fake-NodeDB v25 fixture (250/500/1000/2000 nodes) onto a device.
Two transports:
target="portduino" — file copy to ~/.portduino/<portduino_config>/prefs/nodes.proto.
Fast, no device connection needed.
target="hardware" — XModem upload over serial/BLE to /prefs/nodes.proto.
Requires `port` + `confirm=True`. Triggers a reboot
so loadFromDisk picks up the new file at next boot.
Compiles a fresh-timestamp proto from the committed JSONL seed under
test/fixtures/nodedb/seed_v25_<N>.jsonl each invocation, so the loaded
NodeDB always looks "recent" to the connecting phone. Structural data
(names, IDs, positions, telemetries) is deterministic per the seed.
Override the JSONL via `custom_seed_jsonl` to push a hand-edited scenario.
"""
return fixtures.push_fake_nodedb(
size=size,
target=target, # type: ignore[arg-type]
port=port,
portduino_config=portduino_config,
backup_existing=backup_existing,
confirm=confirm,
reboot_after=reboot_after,
custom_seed_jsonl=custom_seed_jsonl,
)
mcp-server/tests/unit/test_build_flags.py
+88
View File
@@ -0,0 +1,88 @@
"""Unit tests for the `build_flags` injection on `flash.build()`.
We don't actually run pio here — too slow, requires hardware-aware envs.
We test the translation layer (`_build_flags_env`) and that the env vars
are threaded through pio.run correctly via mock.
"""
from __future__ import annotations
from unittest.mock import patch
from meshtastic_mcp import flash, pio
class TestBuildFlagsEnv:
def test_simple_value(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": 1})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
def test_string_value(self) -> None:
out = flash._build_flags_env({"FOO": "bar"})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
def test_bool_true_is_bare_flag(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": True})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
def test_bool_false_dropped(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
def test_none_dropped(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": None})
assert out == {}
def test_multiple_combined(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
# Order isn't guaranteed in dict iteration, so check membership.
flags = out["PLATFORMIO_BUILD_FLAGS"].split()
assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
class TestBuildPropagatesFlags:
def test_extra_env_passed_to_pio_run(self) -> None:
# Mock pio.run so we don't actually invoke pio. Capture extra_env.
captured = {}
class _StubResult:
returncode = 0
stdout = ""
stderr = ""
duration_s = 0.1
def _stub(args, **kwargs):
captured["args"] = args
captured["kwargs"] = kwargs
return _StubResult()
with patch.object(pio, "run", side_effect=_stub):
with patch.object(flash, "_artifacts_for", return_value=[]):
out = flash.build(
"fake-env",
with_manifest=False,
build_flags={"DEBUG_HEAP": 1},
)
assert captured["args"] == ["run", "-e", "fake-env"]
assert captured["kwargs"]["extra_env"] == {
"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
}
assert out["build_flags"] == {"DEBUG_HEAP": 1}
def test_no_flags_means_no_extra_env(self) -> None:
captured = {}
class _StubResult:
returncode = 0
stdout = ""
stderr = ""
duration_s = 0.1
def _stub(args, **kwargs):
captured["kwargs"] = kwargs
return _StubResult()
with patch.object(pio, "run", side_effect=_stub):
with patch.object(flash, "_artifacts_for", return_value=[]):
flash.build("fake-env", with_manifest=False)
assert captured["kwargs"]["extra_env"] is None
mcp-server/tests/unit/test_fake_nodedb_generator.py
+364
View File
@@ -0,0 +1,364 @@
"""Tests for the fake-NodeDB fixture pipeline (bin/gen-fake-nodedb-seed.py
+ bin/seed-json-to-proto.py + mcp-server fixtures.push_fake_nodedb).
Lives under tests/unit/ because none of these touch real hardware — they
shell out to the bin/ scripts and decode the resulting protobufs in-process.
"""
from __future__ import annotations
import json
import pathlib
import subprocess
import sys
import time
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
SEED_GEN = REPO_ROOT / "bin" / "gen-fake-nodedb-seed.py"
COMPILE = REPO_ROOT / "bin" / "seed-json-to-proto.py"
FIXTURES_DIR = REPO_ROOT / "test" / "fixtures" / "nodedb"
# Ensure the locally-generated Python protobuf bindings are importable.
# These live under `meshtastic_v25` (not `meshtastic`) so they don't shadow
# the PyPI `meshtastic` package that the rest of the mcp-server depends on.
_BINDINGS_DIR = REPO_ROOT / "bin" / "_generated"
if _BINDINGS_DIR.is_dir() and str(_BINDINGS_DIR) not in sys.path:
sys.path.insert(0, str(_BINDINGS_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import (
NodeDatabase, # type: ignore[import-not-found]
)
except ImportError:
NodeDatabase = None # type: ignore[assignment]
def _require_v25_bindings() -> None:
if NodeDatabase is None:
pytest.skip(
"v25 Python protobuf bindings missing; run `./bin/regen-py-protos.sh`."
)
if "positions" not in NodeDatabase.DESCRIPTOR.fields_by_name:
pytest.skip(
"Loaded NodeDatabase predates v25 — run `./bin/regen-py-protos.sh`."
)
def _run(cmd: list[str]) -> None:
subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
# ---------------------------------------------------------------------------
# Seed generator: deterministic for given --seed (no wall-clock dependence).
# ---------------------------------------------------------------------------
def test_seed_generator_is_deterministic(tmp_path: pathlib.Path) -> None:
a = tmp_path / "a.jsonl"
b = tmp_path / "b.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"100",
"--seed",
"42",
"--out",
str(a),
]
)
# Sleep so any sneaky wall-clock leak in the generator would surface as
# a byte diff between the two runs.
time.sleep(0.8)
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"100",
"--seed",
"42",
"--out",
str(b),
]
)
assert a.read_bytes() == b.read_bytes()
def test_seed_generator_meta_line(tmp_path: pathlib.Path) -> None:
out = tmp_path / "seed.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"50",
"--seed",
"1",
"--out",
str(out),
]
)
lines = out.read_text(encoding="utf-8").splitlines()
assert len(lines) == 51 # 1 meta + 50 nodes
meta = json.loads(lines[0])
assert "_meta" in meta
assert meta["_meta"]["version"] == 25
assert meta["_meta"]["count"] == 50
assert meta["_meta"]["seed"] == 1
def test_seed_only_uses_active_hardware_and_roles(tmp_path: pathlib.Path) -> None:
"""Confirm no deprecated roles + no off-list HW models leak through."""
out = tmp_path / "seed.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"500",
"--seed",
"7",
"--out",
str(out),
]
)
forbidden_roles = {"ROUTER_CLIENT", "REPEATER"}
forbidden_hw = {
"TLORA_V1",
"TLORA_V2",
"TLORA_V1_1P3",
"TLORA_V2_1_1P6",
"TLORA_V2_1_1P8",
"HELTEC_V1",
"HELTEC_V2_0",
"HELTEC_V2_1",
"TBEAM",
"TBEAM_V0P7",
"NANO_G1",
"NANO_G1_EXPLORER",
"NANO_G2_ULTRA",
"STATION_G1",
"STATION_G2",
"PORTDUINO",
"ANDROID_SIM",
"DIY_V1",
"LORA_RELAY_V1",
"NRF52840_PCA10059",
"NRF52_UNKNOWN",
"DR_DEV",
"GENIEBLOCKS",
"M5STACK",
"RP2040_LORA",
"PPR",
}
for raw in out.read_text(encoding="utf-8").splitlines()[1:]:
node = json.loads(raw)
assert node["role"] not in forbidden_roles, f"deprecated role: {node['role']}"
assert (
node["hw_model"] not in forbidden_hw
), f"non-tier-1 HW: {node['hw_model']}"
# ---------------------------------------------------------------------------
# Compile step + committed seeds.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("size", [250, 500, 1000, 2000])
def test_committed_seed_compiles_and_decodes(size: int, tmp_path: pathlib.Path) -> None:
_require_v25_bindings()
proto = tmp_path / "out.proto"
jsonl = FIXTURES_DIR / f"seed_v25_{size:04d}.jsonl"
if not jsonl.is_file():
pytest.skip(f"{jsonl} not present — run ./bin/regen-fake-nodedbs.sh")
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(proto)])
db = NodeDatabase()
db.ParseFromString(proto.read_bytes())
assert db.version == 25
assert len(db.nodes) == size
nums = {n.num for n in db.nodes}
assert len(nums) == size, "node numbers must be unique"
assert all(n.long_name and n.short_name for n in db.nodes)
assert all(len(n.long_name) <= 24 for n in db.nodes) # max_size:25 - NUL
# Coverage sanity (±10pp tolerance for binomial fluctuation).
def in_range(actual: int, expected_ratio: float, tol_pp: float = 0.10) -> bool:
lo = max(0, int((expected_ratio - tol_pp) * size))
hi = min(size, int((expected_ratio + tol_pp) * size))
return lo <= actual <= hi
assert in_range(len(db.positions), 0.85)
assert in_range(len(db.telemetry), 0.70)
assert in_range(len(db.environment), 0.25)
assert in_range(len(db.status), 0.40)
def test_compile_freshens_timestamps(tmp_path: pathlib.Path) -> None:
"""Same JSONL compiled twice → identical structure, different timestamps."""
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present — run ./bin/regen-fake-nodedbs.sh")
a = tmp_path / "a.proto"
b = tmp_path / "b.proto"
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(a)])
time.sleep(1.2)
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(b)])
da = NodeDatabase()
db_ = NodeDatabase()
da.ParseFromString(a.read_bytes())
db_.ParseFromString(b.read_bytes())
# Zero out timestamp fields and confirm everything else is byte-identical.
for d in (da, db_):
for n in d.nodes:
n.last_heard = 0
for p in d.positions:
p.position.time = 0
assert da.SerializeToString() == db_.SerializeToString()
# Re-load fresh copies to confirm timestamps actually moved.
aa = NodeDatabase()
bb = NodeDatabase()
aa.ParseFromString(a.read_bytes())
bb.ParseFromString(b.read_bytes())
aa_max = max(n.last_heard for n in aa.nodes if n.last_heard)
bb_max = max(n.last_heard for n in bb.nodes if n.last_heard)
assert bb_max >= aa_max
assert bb_max - aa_max < 5 # within a few seconds
def test_compile_pinned_now_epoch_is_byte_identical(tmp_path: pathlib.Path) -> None:
"""With --now-epoch pinned, two compiles produce identical bytes."""
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present")
a = tmp_path / "a.proto"
b = tmp_path / "b.proto"
for o in (a, b):
_run(
[
sys.executable,
str(COMPILE),
"--in",
str(jsonl),
"--now-epoch",
"1700000000",
"--out",
str(o),
]
)
assert a.read_bytes() == b.read_bytes()
def test_compile_timestamps_are_recent(tmp_path: pathlib.Path) -> None:
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present")
out = tmp_path / "out.proto"
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(out)])
db = NodeDatabase()
db.ParseFromString(out.read_bytes())
now = int(time.time())
# No timestamp older than 7 days, none in the future.
for n in db.nodes:
if n.last_heard:
assert now - 7 * 86400 <= n.last_heard <= now
# At least half should be within the last hour
# (matches expovariate(mean=3600s)).
recent = sum(1 for n in db.nodes if n.last_heard and n.last_heard >= now - 3600)
assert recent >= 0.4 * len(db.nodes)
def test_compile_hand_edit_round_trip(tmp_path: pathlib.Path) -> None:
"""Edit one JSONL line, recompile, confirm edit appears in the proto."""
_require_v25_bindings()
src = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not src.is_file():
pytest.skip("250-node seed not present")
dst = tmp_path / "edited.jsonl"
lines = src.read_text(encoding="utf-8").splitlines()
# Find a node that already has telemetry so the index relationship is
# easy to assert on the other side.
edit_idx = None
for i, raw in enumerate(lines[1:], start=1):
node = json.loads(raw)
if node.get("telemetry") is not None:
edit_idx = i
break
assert edit_idx is not None, "expected at least one node with telemetry"
node = json.loads(lines[edit_idx])
target_num = int(node["num"], 16)
node["long_name"] = "Hand Edited Node"
node["telemetry"] = {
"battery_level": 42,
"voltage": 3.71,
"channel_utilization": 0.0,
"air_util_tx": 0.0,
"uptime_seconds": 1,
}
lines[edit_idx] = json.dumps(node, ensure_ascii=False, sort_keys=True)
dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
out = tmp_path / "out.proto"
_run([sys.executable, str(COMPILE), "--in", str(dst), "--out", str(out)])
db = NodeDatabase()
db.ParseFromString(out.read_bytes())
edited = next((n for n in db.nodes if n.num == target_num), None)
assert edited is not None
assert edited.long_name == "Hand Edited Node"
tel = next((t for t in db.telemetry if t.num == target_num), None)
assert tel is not None
assert tel.device_metrics.battery_level == 42
# ---------------------------------------------------------------------------
# Misc smoke checks on the module surface.
# ---------------------------------------------------------------------------
def test_crc16_ccitt_matches_known_vectors() -> None:
"""Sanity-check the hand-rolled CRC16-CCITT matches well-known vectors.
Test vectors from the XModem-CRC spec (init=0, poly=0x1021):
crc16("123456789") = 0x31C3
crc16("") = 0x0000
"""
from meshtastic_mcp.fixtures import _crc16_ccitt
assert _crc16_ccitt(b"") == 0x0000
assert _crc16_ccitt(b"123456789") == 0x31C3
def test_push_fake_nodedb_rejects_invalid_size() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="size must be one of"):
push_fake_nodedb(size=999, target="portduino") # type: ignore[arg-type]
def test_push_fake_nodedb_hardware_requires_confirm() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="confirm=True"):
push_fake_nodedb(size=250, target="hardware", port="/dev/cu.fake")
def test_push_fake_nodedb_hardware_requires_port() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="requires a port"):
push_fake_nodedb(size=250, target="hardware", confirm=True)
def test_push_fake_nodedb_hardware_rejects_tcp_port() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="not supported"):
push_fake_nodedb(
size=250, target="hardware", confirm=True, port="tcp://localhost:4403"
)
mcp-server/tests/unit/test_recorder.py
+548
View File
@@ -0,0 +1,548 @@
"""Unit tests for the persistent device-log recorder.
Hardware-free: drives the Recorder through its `_on_*` handlers with
synthetic packet/line dicts, then queries via log_query. Validates
prefix parsing, telemetry variant dispatch, marker round-trip, time
window filtering, downsampling, slope estimation, and gzip rotation
+ archive pruning.
"""
from __future__ import annotations
import gzip
import json
import logging
import os
import time
from pathlib import Path
import pubsub
import pytest
from meshtastic_mcp import log_query
from meshtastic_mcp.recorder.parsers import (
extract_telemetry,
interface_label,
parse_log_line,
summarize_packet,
)
from meshtastic_mcp.recorder.recorder import Recorder
from meshtastic_mcp.recorder.rotating import _RotatingJsonl
# -- isolation: every test gets a fresh Recorder + tmp dir -----------
@pytest.fixture
def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
# Redirect both the Recorder and the module-level singleton lookup
# to the same tmp dir so log_query queries the same files we write.
monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
monkeypatch.setattr(
"meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
)
r = Recorder(base_dir=tmp_path)
r.start()
monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
yield r
r.stop()
class _FakeIface:
devPath = "/dev/cu.fake"
# -- parsers ---------------------------------------------------------
class TestParseLogLine:
def test_full_prefix(self) -> None:
out = parse_log_line("INFO | 12:34:56 12345 [Main] Booting")
assert out["level"] == "INFO"
assert out["tag"] == "Main"
assert out["uptime_s"] == 12345
assert out["msg"] == "Booting"
assert out["clock"] == "12:34:56"
def test_invalid_clock(self) -> None:
out = parse_log_line("WARN | ??:??:?? 7 [SerialConsole] Boot")
assert out["level"] == "WARN"
assert out["clock"] == "??:??:??"
assert out["uptime_s"] == 7
def test_no_thread_bracket(self) -> None:
out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
assert out["level"] == "DEBUG"
assert out.get("tag") is None
assert out["msg"] == "raw message body"
def test_bare_message(self) -> None:
# LogRecord.message path — no level prefix at all.
out = parse_log_line("just a bare message")
assert "level" not in out or out.get("level") is None
assert out["line"] == "just a bare message"
def test_empty(self) -> None:
assert parse_log_line("") == {"line": ""}
def test_debug_heap_prefix_extracted(self) -> None:
out = parse_log_line("INFO | 12:34:56 12345 [Main] [heap 92344] Booting")
assert out["level"] == "INFO"
assert out["tag"] == "Main"
assert out["heap_free"] == 92344
assert out["msg"] == "Booting"
def test_debug_heap_prefix_on_bare_line(self) -> None:
# LogRecord.message path: no level prefix but still has [heap N].
out = parse_log_line("[heap 12345] some message")
assert out["heap_free"] == 12345
assert out["msg"] == "some message"
def test_thread_leak_event(self) -> None:
out = parse_log_line(
"HEAP | 00:00:01 100 [Power] [heap 90000] "
"------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
)
assert out["level"] == "HEAP"
assert out["heap_free"] == 90000
ev = out["heap_event"]
assert ev["kind"] == "leaked"
assert ev["thread"] == "MeshPacket"
assert ev["before"] == 92344
assert ev["after"] == 90000
assert ev["delta"] == -2344
def test_thread_freed_event(self) -> None:
out = parse_log_line(
"++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
)
ev = out["heap_event"]
assert ev["kind"] == "freed"
assert ev["thread"] == "Router"
assert ev["delta"] == 500
def test_heap_status_periodic(self) -> None:
out = parse_log_line(
"HEAP | 00:00:30 30 [Power] "
"Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
)
assert out["heap_free"] == 92344
assert out["heap_total"] == 200000
assert out["heap_delta"] == -128
class TestRecorderDebugHeapSynthesis:
def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
# When a log line carries [heap N], the recorder should also
# emit a synthesized telemetry row tagged source=debug_heap.
recorder._on_log_line(
"INFO | 00:00:00 1 [Main] [heap 88888] hello",
_FakeIface(),
)
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
assert len(synth) == 1
assert synth[0]["fields"]["heap_free_bytes"] == 88888
assert synth[0]["variant"] == "local"
def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
recorder._on_log_line(
"HEAP | 00:00:30 30 [Power] "
"Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
_FakeIface(),
)
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
assert synth[-1]["fields"]["heap_free_bytes"] == 50000
assert synth[-1]["fields"]["heap_total_bytes"] == 200000
def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
# Plain log line (no [heap N], no Heap status) — telemetry.jsonl
# should NOT gain a synth row.
before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
recorder._on_log_line("INFO | 00:00:00 1 [Main] just a message", _FakeIface())
after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
assert after == before
def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
recorder._on_log_line(
"HEAP | 00:00:01 100 [Power] [heap 90000] "
"------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
_FakeIface(),
)
rows = [
json.loads(r)
for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
if r
]
evt_rows = [r for r in rows if r.get("heap_event")]
assert len(evt_rows) == 1
assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
assert evt_rows[0]["heap_event"]["delta"] == -2344
class TestSerialTap:
def test_serial_line_records_log_and_synthesizes_heap(
self, recorder: "Recorder"
) -> None:
recorder._on_serial_line(
"INFO | 00:00:00 5 [Main] [heap 88888] tap-line",
port="/dev/cu.tap",
)
logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
log_rows = [json.loads(r) for r in logs if r]
# Find the row from this call (port=/dev/cu.tap, role=serial_session)
tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
assert len(tap_rows) == 1
assert tap_rows[0]["role"] == "serial_session"
assert tap_rows[0]["level"] == "INFO"
assert tap_rows[0]["tag"] == "Main"
assert tap_rows[0]["heap_free"] == 88888
synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
assert len(synth) == 1
assert synth[0]["fields"]["heap_free_bytes"] == 88888
assert synth[0]["role"] == "serial_session"
def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
recorder._on_serial_line(
"HEAP | 00:00:30 30 [Power] [heap 53484] "
"------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
port="/dev/cu.tap",
)
rows = [
json.loads(r)
for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
if r
]
evt = [r for r in rows if r.get("heap_event")]
assert len(evt) == 1
assert evt[0]["heap_event"]["thread"] == "Router"
assert evt[0]["heap_event"]["delta"] == -128
# Heap also synthesized.
telem = (recorder.base_dir / "telemetry.jsonl").read_text()
assert '"source":"debug_heap_serial"' in telem
def test_serial_line_pause(self, recorder: "Recorder") -> None:
recorder.pause("baseline")
recorder._on_serial_line(
"INFO | 00:00:00 1 [t] [heap 1000] dropped",
port="/dev/cu.tap",
)
# Only the pause event row should exist; no tap row.
logs = (recorder.base_dir / "logs.jsonl").read_text()
assert "dropped" not in logs
def test_serial_line_handler_swallows_exceptions(
self, recorder: "Recorder"
) -> None:
# Hostile input — should not raise.
recorder._on_serial_line(None, port="/dev/cu.tap") # type: ignore[arg-type]
recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap") # type: ignore[arg-type]
# Survived.
class TestExtractTelemetry:
def test_local_stats_camel(self) -> None:
pkt = {
"decoded": {
"telemetry": {
"localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
}
}
}
out = extract_telemetry(pkt)
assert out is not None
assert out["variant"] == "local"
assert out["fields"]["heap_free_bytes"] == 600
def test_device_metrics_snake(self) -> None:
pkt = {
"decoded": {
"telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
}
}
out = extract_telemetry(pkt)
assert out is not None
assert out["variant"] == "device"
assert out["fields"]["battery_level"] == 88
def test_unknown_variant_returns_none(self) -> None:
assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
assert extract_telemetry({}) is None
assert extract_telemetry({"decoded": "not-a-dict"}) is None
class TestSummarizePacket:
def test_text_with_payload(self) -> None:
pkt = {
"fromId": "!abc",
"toId": "!def",
"decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
"hopLimit": 3,
}
out = summarize_packet(pkt)
assert out["from_node"] == "!abc"
assert out["portnum"] == "TEXT_MESSAGE_APP"
assert out["payload_size"] == 5
assert out["payload_hex_prefix"] == "68656c6c6f"
def test_no_decoded(self) -> None:
out = summarize_packet({"fromId": "!abc"})
assert out["from_node"] == "!abc"
assert out["portnum"] is None
class TestInterfaceLabel:
def test_serial(self) -> None:
assert interface_label(_FakeIface()) == {
"port": "/dev/cu.fake",
"role": "serial",
}
def test_tcp(self) -> None:
class T:
hostname = "node.lan"
portNumber = 4403
assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
def test_unknown(self) -> None:
assert interface_label(object()) == {"port": "object", "role": None}
def test_none(self) -> None:
assert interface_label(None) == {"port": None, "role": None}
# -- recorder write side ---------------------------------------------
class TestRecorderWrites:
def test_log_line_is_recorded(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:34:56 99 [T] hi", _FakeIface())
path = recorder.base_dir / "logs.jsonl"
rows = [json.loads(line) for line in path.read_text().splitlines() if line]
# First row is recorder_start_event mirror? No — that's events.jsonl only.
assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
# _on_telemetry alone — only telemetry.jsonl
recorder._on_telemetry(
{
"fromId": "!abc",
"decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
},
_FakeIface(),
)
telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
assert any('"variant":"local"' in r for r in telem_rows)
def test_packets_summary(self, recorder: Recorder) -> None:
recorder._on_receive(
{
"fromId": "!abc",
"toId": "!def",
"decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
},
_FakeIface(),
)
rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
def test_mark_event_round_trip(self, recorder: Recorder) -> None:
out = recorder.mark_event("checkpoint", note="midpoint")
assert "ts" in out
events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
def test_pause_drops_writes(self, recorder: Recorder) -> None:
before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
recorder.pause(reason="baseline")
recorder._on_log_line("INFO | 00:00:00 1 [t] swallowed", _FakeIface())
after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
assert after == before
recorder.resume()
recorder._on_log_line("INFO | 00:00:00 2 [t] kept", _FakeIface())
post_resume = (recorder.base_dir / "logs.jsonl").read_text()
assert "kept" in post_resume
def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
# If the writer dies, the pubsub callback must NOT raise — that
# would crash the meshtastic receive thread.
bad_packet = object() # not a dict
recorder._on_receive(bad_packet, _FakeIface()) # type: ignore[arg-type]
recorder._on_telemetry(bad_packet, _FakeIface()) # type: ignore[arg-type]
recorder._on_log_line(None, _FakeIface()) # type: ignore[arg-type]
# No assertion needed — survival is the test.
# -- log_query read side ---------------------------------------------
class TestLogQuery:
def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:00:00 1 [A] alpha", _FakeIface())
recorder._on_log_line("WARN | 12:00:01 2 [B] bravo failed", _FakeIface())
recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
assert out["total_matched"] == 2
levels = {r["level"] for r in out["lines"]}
assert levels == {"WARN", "ERROR"}
out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
assert out2["total_matched"] == 2
def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:00:00 1 [A] alpha", _FakeIface())
with pytest.raises(ValueError, match="invalid grep regex"):
log_query.logs_window(start="-1m", grep="(")
def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
# Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
base_ts = time.time() - 60
for i in range(100):
recorder._files["telemetry"].write(
{
"ts": base_ts + i * 0.5,
"port": "/dev/cu.fake",
"role": "serial",
"from_node": "!abc",
"variant": "local",
"fields": {"heap_free_bytes": 10000 - i},
}
)
out = log_query.telemetry_timeline(
window="2m", variant="local", field="free_heap", max_points=10
)
assert out["samples"] == 100
assert len(out["points"]) <= 10
# Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
assert out["slope_per_min"] is not None
assert out["slope_per_min"] < -100
def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
recorder._on_log_line("INFO | 00:00:00 1 [t] one", _FakeIface())
recorder._on_log_line("INFO | 00:00:00 2 [t] two", _FakeIface())
dest = tmp_path / "bundle"
out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
assert (dest / "logs.jsonl").exists()
assert "logs" in out["paths"]
# -- time parser -----------------------------------------------------
class TestParseTime:
def test_relative(self) -> None:
now = 1_000_000.0
assert log_query._parse_time("-15m", now=now) == now - 900
assert log_query._parse_time("-2h", now=now) == now - 7200
assert log_query._parse_time("-1d", now=now) == now - 86400
def test_now_and_epoch(self) -> None:
now = 1_000_000.0
assert log_query._parse_time("now", now=now) == now
assert log_query._parse_time(now) == now
def test_iso(self) -> None:
ts = log_query._parse_time("2026-01-01T00:00:00Z")
assert isinstance(ts, float) and ts > 1_700_000_000
def test_naive_iso_assumes_utc(self) -> None:
assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
"2026-01-01T00:00:00Z"
)
def test_invalid(self) -> None:
with pytest.raises(ValueError):
log_query._parse_time("not a time")
# -- rotation --------------------------------------------------------
class TestRotation:
def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
for i in range(100):
r.write({"ts": float(i), "i": i, "pad": "x" * 40})
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert archives, "expected at least one rotation"
# Archive content is valid gzip + valid JSONL
with gzip.open(archives[0], "rt") as fh:
first = json.loads(fh.readline())
assert "ts" in first
def test_archive_pruning(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
# Force several rotations.
for _ in range(8):
for i in range(20):
r.write({"ts": float(i), "pad": "x" * 30})
r.force_rotate()
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, keep_archives=2)
old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
for archive in (old, mid, new):
with gzip.open(archive, "wt", encoding="utf-8") as fh:
fh.write('{"ts":1}\n')
# Deliberately scramble mtimes so lexicographic filename order is
# the only stable chronological signal.
os.utime(old, (300, 300))
os.utime(mid, (100, 100))
os.utime(new, (200, 200))
r._prune_archives()
r.close()
archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
assert archives == [mid.name, new.name]
def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
r.write({"ts": 1.0, "msg": "tiny"})
r.force_rotate()
r.write({"ts": 2.0, "msg": "after-rotate"})
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert len(archives) == 1
assert path.exists()
assert "after-rotate" in path.read_text()
class TestRecorderLocks:
def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
out = recorder.force_rotate_all()
assert out["running"] is True
assert out["files"]
def test_wire_pubsub_logs_subscription_failure(
self,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
class FailingPubSubMock:
def subscribe(self, callback: object, topic: str) -> None:
raise RuntimeError("boom")
monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
recorder = Recorder(base_dir=tmp_path)
with caplog.at_level(logging.WARNING):
recorder._wire_pubsub()
assert (
"Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
)
platformio.ini
+3 -1
View File
@@ -17,6 +17,7 @@ test_build_src = true
extra_scripts =
pre:bin/platformio-pre.py
bin/platformio-custom.py
post:extra_scripts/nrf54l15_linker.py
; note: we add src to our include search path so that lmic_project_config can override
; note: TINYGPS_OPTION_NO_CUSTOM_FIELDS is VERY important. We don't use custom fields and somewhere in that pile
; of code is a heap corruption bug!
@@ -49,6 +50,7 @@ build_flags = -Wno-missing-field-initializers
-DRADIOLIB_EXCLUDE_PAGER=1
-DRADIOLIB_EXCLUDE_FSK4=1
-DRADIOLIB_EXCLUDE_APRS=1
-DRADIOLIB_EXCLUDE_ADSB=1
-DRADIOLIB_EXCLUDE_LORAWAN=1
-DMESHTASTIC_EXCLUDE_DROPZONE=1
-DMESHTASTIC_EXCLUDE_REPLYBOT=1
@@ -137,7 +139,7 @@ lib_deps =
# renovate: datasource=github-tags depName=Adafruit GFX packageName=adafruit/Adafruit-GFX-Library
https://github.com/adafruit/Adafruit-GFX-Library/archive/refs/tags/1.12.6.zip
# renovate: datasource=github-tags depName=NeoPixel packageName=adafruit/Adafruit_NeoPixel
https://github.com/adafruit/Adafruit_NeoPixel/archive/refs/tags/1.15.4.zip
https://github.com/adafruit/Adafruit_NeoPixel/archive/1.15.5.zip
# renovate: datasource=github-tags depName=Adafruit SSD1306 packageName=adafruit/Adafruit_SSD1306
https://github.com/adafruit/Adafruit_SSD1306/archive/refs/tags/2.5.16.zip
# renovate: datasource=github-tags depName=Adafruit BMP280 packageName=adafruit/Adafruit_BMP280_Library
protobufs
+1 -1
Submodule protobufs updated: 149586802f...7ffb4bb60d
src/DebugConfiguration.cpp
+1 -1
View File
@@ -151,7 +151,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
if (!this->_enabled)
return false;
if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
return false;
// Check priority against priMask values.
src/DebugConfiguration.h
+8 -1
View File
@@ -13,6 +13,11 @@ extern MemGet memGet;
#define LED_STATE_ON 1
#endif
// WIFI LED
#ifndef WIFI_STATE_ON
#define WIFI_STATE_ON 1
#endif
// -----------------------------------------------------------------------------
// DEBUG
// -----------------------------------------------------------------------------
@@ -147,7 +152,9 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
// Default Bluetooth PIN
#define defaultBLEPin 123456
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && defined(USE_CH390D)
#include <ESP32_CH390.h>
#elif HAS_ETHERNET && !defined(USE_WS5500)
#include <RAK13800_W5100S.h>
#endif // HAS_ETHERNET
src/FSCommon.h
+8
View File
@@ -48,6 +48,14 @@ using namespace STM32_LittleFS_Namespace;
using namespace Adafruit_LittleFS_Namespace;
#endif
#if defined(ARCH_NRF54L15)
// nRF54L15 — Zephyr LittleFS on 36 KB storage_partition (internal RRAM)
#include "InternalFileSystem.h"
#define FSCom InternalFS
#define FSBegin() FSCom.begin()
using namespace Adafruit_LittleFS_Namespace;
#endif
void fsInit();
void fsListFiles();
bool copyFile(const char *from, const char *to);
src/MessageStore.cpp
+18 -23
View File
@@ -6,7 +6,6 @@
#include "SPILock.h"
#include "SafeFile.h"
#include "gps/RTC.h"
#include "graphics/draw/MessageRenderer.h"
#include <cstring> // memcpy
#ifndef MESSAGE_TEXT_POOL_SIZE
@@ -181,13 +180,8 @@ const StoredMessage &MessageStore::addFromPacket(const meshtastic_MeshPacket &pa
bool isDM = (sm.dest != 0 && sm.dest != NODENUM_BROADCAST);
if (packet.from == 0) {
sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
sm.ackStatus = AckStatus::NONE;
} else {
sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
sm.ackStatus = AckStatus::ACKED;
}
sm.ackStatus = (packet.from == 0) ? AckStatus::NONE : AckStatus::ACKED;
addLiveMessage(sm);
@@ -372,20 +366,20 @@ void MessageStore::clearAllMessages()
#endif
}
// Internal helper: erase first or last message matching a predicate
template <typename Predicate> static void eraseIf(std::deque<StoredMessage> &deque, Predicate pred, bool fromBack = false)
// Internal helpers for targeted erasure.
template <typename Predicate> static bool eraseFirstMatch(std::deque<StoredMessage> &deque, Predicate pred)
{
if (fromBack) {
// Iterate from the back and erase all matches from the end
for (auto it = deque.rbegin(); it != deque.rend();) {
for (auto it = deque.begin(); it != deque.end(); ++it) {
if (pred(*it)) {
it = std::deque<StoredMessage>::reverse_iterator(deque.erase(std::next(it).base()));
} else {
++it;
deque.erase(it);
return true;
}
}
} else {
// Manual forward search to erase all matches
return false;
}
template <typename Predicate> static void eraseAllMatches(std::deque<StoredMessage> &deque, Predicate pred)
{
for (auto it = deque.begin(); it != deque.end();) {
if (pred(*it)) {
it = deque.erase(it);
@@ -394,12 +388,13 @@ template <typename Predicate> static void eraseIf(std::deque<StoredMessage> &deq
}
}
}
}
// Delete oldest message (RAM + persisted queue)
void MessageStore::deleteOldestMessage()
{
eraseIf(liveMessages, [](StoredMessage &) { return true; });
if (!liveMessages.empty()) {
liveMessages.pop_front();
}
saveToFlash();
}
@@ -407,14 +402,14 @@ void MessageStore::deleteOldestMessage()
void MessageStore::deleteOldestMessageInChannel(uint8_t channel)
{
auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
eraseIf(liveMessages, pred);
eraseFirstMatch(liveMessages, pred);
saveToFlash();
}
void MessageStore::deleteAllMessagesInChannel(uint8_t channel)
{
auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
eraseIf(liveMessages, pred, false /* delete ALL, not just first */);
eraseAllMatches(liveMessages, pred);
saveToFlash();
}
@@ -427,7 +422,7 @@ void MessageStore::deleteAllMessagesWithPeer(uint32_t peer)
uint32_t other = (m.sender == local) ? m.dest : m.sender;
return other == peer;
};
eraseIf(liveMessages, pred, false);
eraseAllMatches(liveMessages, pred);
saveToFlash();
}
@@ -440,7 +435,7 @@ void MessageStore::deleteOldestMessageWithPeer(uint32_t peer)
uint32_t other = (m.sender == nodeDB->getNodeNum()) ? m.dest : m.sender;
return other == peer;
};
eraseIf(liveMessages, pred);
eraseFirstMatch(liveMessages, pred);
saveToFlash();
}
src/MessageStore.h
-3
View File
@@ -124,9 +124,6 @@ class MessageStore
// Allocate text into pool (used by sender-side code)
static uint16_t storeText(const char *src, size_t len);
// Used when loading from flash to rebuild the text pool
static uint16_t rebuildTextFromFlash(const char *src, size_t len);
private:
std::deque<StoredMessage> liveMessages; // Single in-RAM message buffer (also used for persistence)
std::string filename; // Flash filename for persistence
src/Power.cpp
+10 -1
View File
@@ -910,7 +910,16 @@ void Power::readPowerStatus()
// Notify any status instances that are observing us
const PowerStatus powerStatus2 = PowerStatus(hasBattery, usbPowered, isChargingNow, batteryVoltageMv, batteryChargePercent);
if (millis() > lastLogTime + 50 * 1000) {
// Log battery-presence transitions once; skip OptUnknown so we don't lie before the first probe.
static OptionalBool prevHasBattery = OptUnknown;
if (hasBattery != OptUnknown && hasBattery != prevHasBattery) {
LOG_INFO("Power: battery hardware %s", hasBattery == OptTrue ? "detected" : "absent (USB-only)");
prevHasBattery = hasBattery;
}
// Periodic telemetry only emits when a battery is actually present (otherwise values are constant -1/0).
if (hasBattery == OptTrue && !Throttle::isWithinTimespanMs(lastLogTime, 50 * 1000)) {
LOG_DEBUG("Battery: usbPower=%d, isCharging=%d, batMv=%d, batPct=%d", powerStatus2.getHasUSB(),
powerStatus2.getIsCharging(), powerStatus2.getBatteryVoltageMv(), powerStatus2.getBatteryChargePercent());
lastLogTime = millis();
src/RedirectablePrint.cpp
+6 -2
View File
@@ -225,14 +225,16 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
isBleConnected = nimbleBluetooth && nimbleBluetooth->isActive() && nimbleBluetooth->isConnected();
#elif defined(ARCH_NRF52)
isBleConnected = nrf52Bluetooth != nullptr && nrf52Bluetooth->isConnected();
#elif defined(ARCH_NRF54L15)
isBleConnected = nrf54l15Bluetooth != nullptr && nrf54l15Bluetooth->isConnected();
#endif
if (isBleConnected) {
auto thread = concurrency::OSThread::currentThread;
meshtastic_LogRecord logRecord = meshtastic_LogRecord_init_zero;
logRecord.level = getLogLevel(logLevel);
vsprintf(logRecord.message, format, arg);
vsnprintf(logRecord.message, sizeof(logRecord.message), format, arg);
if (thread)
strcpy(logRecord.source, thread->ThreadName.c_str());
strlcpy(logRecord.source, thread->ThreadName.c_str(), sizeof(logRecord.source));
logRecord.time = getValidTime(RTCQuality::RTCQualityDevice, true);
auto buffer = std::unique_ptr<uint8_t[]>(new uint8_t[meshtastic_LogRecord_size]);
@@ -241,6 +243,8 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
nimbleBluetooth->sendLog(buffer.get(), size);
#elif defined(ARCH_NRF52)
nrf52Bluetooth->sendLog(buffer.get(), size);
#elif defined(ARCH_NRF54L15)
nrf54l15Bluetooth->sendLog(buffer.get(), size);
#endif
}
}
src/graphics/Screen.cpp
+10 -142
View File
@@ -65,7 +65,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#include "mesh/Default.h"
#include "mesh/generated/meshtastic/deviceonly.pb.h"
#include "modules/ExternalNotificationModule.h"
#include "modules/TextMessageModule.h"
#include "modules/WaypointModule.h"
#include "sleep.h"
#include "target_specific.h"
@@ -231,6 +230,7 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
#endif
// Store the message and set the expiration timestamp
strncpy(NotificationRenderer::alertBannerMessage, banner_overlay_options.message, 255);
NotificationRenderer::parseBannerMessageWithFonts(NotificationRenderer::alertBannerMessage);
NotificationRenderer::alertBannerMessage[255] = '\0'; // Ensure null termination
NotificationRenderer::alertBannerUntil =
(banner_overlay_options.durationMs == 0) ? 0 : millis() + banner_overlay_options.durationMs;
@@ -240,9 +240,9 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
NotificationRenderer::alertBannerCallback = banner_overlay_options.bannerCallback;
NotificationRenderer::curSelected = banner_overlay_options.InitialSelected;
NotificationRenderer::pauseBanner = false;
NotificationRenderer::current_notification_type = notificationTypeEnum::selection_picker;
NotificationRenderer::current_notification_type = banner_overlay_options.notificationType;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
ui->setTargetFPS(60);
updateUiFrame(ui);
}
@@ -264,7 +264,7 @@ void Screen::showNodePicker(const char *message, uint32_t durationMs, std::funct
NotificationRenderer::current_notification_type = notificationTypeEnum::node_picker;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
ui->setTargetFPS(60);
updateUiFrame(ui);
}
@@ -288,7 +288,7 @@ void Screen::showNumberPicker(const char *message, uint32_t durationMs, uint8_t
NotificationRenderer::currentNumber = 0;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
ui->setTargetFPS(60);
updateUiFrame(ui);
}
@@ -311,7 +311,7 @@ void Screen::showTextInput(const char *header, const char *initialText, uint32_t
// Set the overlay using the same pattern as other notification types
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
ui->setTargetFPS(60);
updateUiFrame(ui);
}
@@ -696,7 +696,7 @@ void Screen::setup()
static OverlayCallback overlays[] = {
graphics::UIRenderer::drawNavigationBar // Custom indicator icons for each frame
};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 1);
// Enable UTF-8 to display mapping
dispdev->setFontTableLookupFunction(customFontTableLookup);
@@ -1313,7 +1313,7 @@ void Screen::setFrames(FrameFocus focus)
// Add overlays: frame icons and alert banner)
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
prevFrame = -1; // Force drawFavoriteNode to pick a new node (because our list just changed)
@@ -1643,138 +1643,6 @@ int Screen::handleStatusUpdate(const meshtastic::Status *arg)
return 0;
}
// Handles when message is received; will jump to text message frame.
int Screen::handleTextMessage(const meshtastic_MeshPacket *packet)
{
if (showingNormalScreen) {
if (packet->from == 0) {
// Outgoing message (likely sent from phone)
devicestate.has_rx_text_message = false;
memset(&devicestate.rx_text_message, 0, sizeof(devicestate.rx_text_message));
hiddenFrames.textMessage = true;
hasUnreadMessage = false; // Clear unread state when user replies
setFrames(FOCUS_PRESERVE); // Stay on same frame, silently update frame list
} else {
// Incoming message
devicestate.has_rx_text_message = true; // Needed to include the message frame
hasUnreadMessage = true; // Enables mail icon in the header
setFrames(FOCUS_PRESERVE); // Refresh frame list without switching view (no-op during text_input)
// Only wake/force display if the configuration allows it
if (shouldWakeOnReceivedMessage()) {
setOn(true); // Wake up the screen first
forceDisplay(); // Forces screen redraw
}
// === Prepare banner/popup content ===
const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet->from);
const meshtastic_Channel channel =
channels.getByIndex(packet->channel ? packet->channel : channels.getPrimaryIndex());
const char *longName = nodeInfoLiteHasUser(node) ? node->long_name : nullptr;
const char *msgRaw = reinterpret_cast<const char *>(packet->decoded.payload.bytes);
char banner[256];
bool isAlert = false;
if (moduleConfig.external_notification.alert_bell || moduleConfig.external_notification.alert_bell_vibra ||
moduleConfig.external_notification.alert_bell_buzzer)
// Check for bell character to determine if this message is an alert
for (size_t i = 0; i < packet->decoded.payload.size && i < 100; i++) {
if (msgRaw[i] == ASCII_BELL) {
isAlert = true;
break;
}
}
// Unlike generic messages, alerts (when enabled via the ext notif module) ignore any
// 'mute' preferences set to any specific node or channel.
// If on-screen keyboard is active, show a transient popup over keyboard instead of interrupting it
if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
// Wake and force redraw so popup is visible immediately
if (shouldWakeOnReceivedMessage()) {
setOn(true);
forceDisplay();
}
// Build popup: title = message source name, content = message text (sanitized)
// Title
char titleBuf[64] = {0};
if (longName && longName[0]) {
// Sanitize sender name
std::string t = sanitizeString(longName);
strncpy(titleBuf, t.c_str(), sizeof(titleBuf) - 1);
} else {
strncpy(titleBuf, "Message", sizeof(titleBuf) - 1);
}
// Content: payload bytes may not be null-terminated, remove ASCII_BELL and sanitize
char content[256] = {0};
{
std::string raw;
raw.reserve(packet->decoded.payload.size);
for (size_t i = 0; i < packet->decoded.payload.size; ++i) {
char c = msgRaw[i];
if (c == ASCII_BELL)
continue; // strip bell
raw.push_back(c);
}
std::string sanitized = sanitizeString(raw);
strncpy(content, sanitized.c_str(), sizeof(content) - 1);
}
NotificationRenderer::showKeyboardMessagePopupWithTitle(titleBuf, content, 3000);
// Maintain existing buzzer behavior on M5 if applicable
#if defined(M5STACK_UNITC6L)
if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
(isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
(!isBroadcast(packet->to) && isToUs(packet))) {
playLongBeep();
}
#endif
} else {
// No keyboard active: use regular banner flow, respecting mute settings
if (isAlert) {
if (longName && longName[0]) {
snprintf(banner, sizeof(banner), "Alert Received from\n%s", longName);
} else {
strcpy(banner, "Alert Received");
}
screen->showSimpleBanner(banner, 3000);
} else if (!channel.settings.has_module_settings || !channel.settings.module_settings.is_muted) {
if (longName && longName[0]) {
if (currentResolution == ScreenResolution::UltraLow) {
strcpy(banner, "New Message");
} else {
snprintf(banner, sizeof(banner), "New Message from\n%s", longName);
}
} else {
strcpy(banner, "New Message");
}
#if defined(M5STACK_UNITC6L)
screen->setOn(true);
screen->showSimpleBanner(banner, 1500);
if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
(isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
(!isBroadcast(packet->to) && isToUs(packet))) {
// Beep if not in DIRECT_MSG_ONLY mode or if in DIRECT_MSG_ONLY mode and either
// - packet contains an alert and alert bell buzzer is enabled
// - packet is a non-broadcast that is addressed to this node
playLongBeep();
}
#else
screen->showSimpleBanner(banner, 3000);
#endif
}
}
}
}
return 0;
}
// Triggered by MeshModules
int Screen::handleUIFrameEvent(const UIFrameEvent *event)
{
@@ -1820,7 +1688,7 @@ int Screen::handleInputEvent(const InputEvent *event)
if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
NotificationRenderer::inEvent = *event;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
setFastFramerate(); // Draw ASAP
updateUiFrame(ui);
return 0;
@@ -1835,7 +1703,7 @@ int Screen::handleInputEvent(const InputEvent *event)
if (NotificationRenderer::isOverlayBannerShowing()) {
NotificationRenderer::inEvent = *event;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setOverlays(overlays, 2);
setFastFramerate(); // Draw ASAP
updateUiFrame(ui);
src/graphics/Screen.h
-1
View File
@@ -609,7 +609,6 @@ class Screen : public concurrency::OSThread
// Handle observer events
int handleStatusUpdate(const meshtastic::Status *arg);
int handleTextMessage(const meshtastic_MeshPacket *packet);
int handleUIFrameEvent(const UIFrameEvent *arg);
int handleInputEvent(const InputEvent *arg);
int handleAdminMessage(AdminModule_ObserverData *arg);
src/graphics/ScreenFonts.h
+10
View File
@@ -88,6 +88,16 @@
#endif
#endif
// nRF52 flash optimization: re-route FONT_LARGE_LOCAL to the 16pt glyph so
// the display-tier dispatch below picks up 16pt everywhere it would have used
// 24pt. Drops the ~9.6 KB ArialMT_Plain_24 table from the linked binary.
// Set MESHTASTIC_LARGE_FONT_24PT=1 in build_flags to opt out per variant
// (undefined or 0 keeps the optimization on).
#if defined(ARCH_NRF52) && (!defined(MESHTASTIC_LARGE_FONT_24PT) || MESHTASTIC_LARGE_FONT_24PT == 0)
#undef FONT_LARGE_LOCAL
#define FONT_LARGE_LOCAL FONT_MEDIUM_LOCAL
#endif
#if (defined(USE_EINK) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || defined(ST7701_CS) || defined(ST7735_CS) || \
defined(ST7789_CS) || defined(USE_ST7789) || defined(HX8357_CS) || defined(ILI9488_CS) || defined(ST7796_CS) || \
defined(USE_ST7796) || defined(HACKADAY_COMMUNICATOR)) && \
src/graphics/draw/MenuHandler.cpp
+73 -22
View File
@@ -57,6 +57,70 @@ BannerOverlayOptions createStaticBannerOptions(const char *message, const MenuOp
return bannerOptions;
}
const StoredMessage *getNewestMessageForActiveThread()
{
const auto &messages = messageStore.getMessages();
if (messages.empty()) {
return nullptr;
}
const auto mode = graphics::MessageRenderer::getThreadMode();
const int channel = graphics::MessageRenderer::getThreadChannel();
const uint32_t peer = graphics::MessageRenderer::getThreadPeer();
const uint32_t localNode = nodeDB->getNodeNum();
if (mode == graphics::MessageRenderer::ThreadMode::ALL) {
return &messages.back();
}
for (auto it = messages.rbegin(); it != messages.rend(); ++it) {
const StoredMessage &m = *it;
if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
if (m.type == MessageType::BROADCAST && static_cast<int>(m.channelIndex) == channel) {
return &m;
}
continue;
}
if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
if (m.type != MessageType::DM_TO_US) {
continue;
}
const uint32_t other = (m.sender == localNode) ? m.dest : m.sender;
if (other == peer) {
return &m;
}
}
}
return nullptr;
}
void launchReplyForMessage(const StoredMessage &message, bool freetext)
{
if (message.type == MessageType::BROADCAST || message.dest == NODENUM_BROADCAST) {
if (freetext) {
cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, message.channelIndex);
} else {
cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, message.channelIndex);
}
return;
}
const uint32_t localNode = nodeDB->getNodeNum();
const uint32_t peer = (message.sender == localNode) ? message.dest : message.sender;
if (peer == 0 || peer == NODENUM_BROADCAST) {
return;
}
if (freetext) {
cannedMessageModule->LaunchFreetextWithDestination(peer);
} else {
cannedMessageModule->LaunchWithDestination(peer);
}
}
} // namespace
menuHandler::screenMenus menuHandler::menuQueue = MenuNone;
@@ -594,9 +658,12 @@ void menuHandler::messageResponseMenu()
#ifdef HAS_I2S
} else if (selected == Aloud) {
const meshtastic_MeshPacket &mp = devicestate.rx_text_message;
const char *msg = reinterpret_cast<const char *>(mp.decoded.payload.bytes);
if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
const char *msg = MessageStore::getText(*latest);
if (msg && msg[0]) {
audioThread->readAloud(msg);
}
}
#endif
}
};
@@ -656,20 +723,12 @@ void menuHandler::replyMenu()
// Preset reply
if (selected == ReplyPreset) {
if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, ch);
} else if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
cannedMessageModule->LaunchWithDestination(peer);
} else {
// Fallback for last received message
if (devicestate.rx_text_message.to == NODENUM_BROADCAST) {
cannedMessageModule->LaunchWithDestination(NODENUM_BROADCAST, devicestate.rx_text_message.channel);
} else {
cannedMessageModule->LaunchWithDestination(devicestate.rx_text_message.from);
}
} else if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
launchReplyForMessage(*latest, false);
}
return;
@@ -677,20 +736,12 @@ void menuHandler::replyMenu()
// Freetext reply
if (selected == ReplyFreetext) {
if (mode == graphics::MessageRenderer::ThreadMode::CHANNEL) {
cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, ch);
} else if (mode == graphics::MessageRenderer::ThreadMode::DIRECT) {
cannedMessageModule->LaunchFreetextWithDestination(peer);
} else {
// Fallback for last received message
if (devicestate.rx_text_message.to == NODENUM_BROADCAST) {
cannedMessageModule->LaunchFreetextWithDestination(NODENUM_BROADCAST, devicestate.rx_text_message.channel);
} else {
cannedMessageModule->LaunchFreetextWithDestination(devicestate.rx_text_message.from);
}
} else if (const StoredMessage *latest = getNewestMessageForActiveThread()) {
launchReplyForMessage(*latest, true);
}
return;
src/graphics/draw/NotificationRenderer.cpp
+162 -13
View File
@@ -66,6 +66,110 @@ uint32_t pow_of_10(uint32_t n)
return ret;
}
char graphics::NotificationRenderer::alertBannerLines[MAX_LINES + 1][64] = {};
uint8_t graphics::NotificationRenderer::alertBannerLineCount = 0;
graphics::NotificationRenderer::BannerFont graphics::NotificationRenderer::alertBannerLineFonts[MAX_LINES + 1] = {};
static inline graphics::NotificationRenderer::BannerFont parseFontTagPrefix(const char *&p)
{
// Tags must be at the start of the line:
// [S] small, [M] medium, [L] large
if (p && p[0] == '[' && p[2] == ']' && p[1] != '\0') {
char t = p[1];
if (t == 'S') {
p += 3;
return graphics::NotificationRenderer::BANNER_FONT_SMALL;
}
if (t == 'M') {
p += 3;
return graphics::NotificationRenderer::BANNER_FONT_MEDIUM;
}
if (t == 'L') {
p += 3;
return graphics::NotificationRenderer::BANNER_FONT_LARGE;
}
}
return graphics::NotificationRenderer::BANNER_FONT_DEFAULT;
}
static inline const uint8_t *fontForBannerLine(graphics::NotificationRenderer::BannerFont f)
{
switch (f) {
case graphics::NotificationRenderer::BANNER_FONT_SMALL:
return FONT_SMALL;
case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
return FONT_MEDIUM;
case graphics::NotificationRenderer::BANNER_FONT_LARGE:
return FONT_LARGE;
case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
default:
return FONT_SMALL;
}
}
static inline uint8_t effectiveLineHeightForBannerLine(graphics::NotificationRenderer::BannerFont f)
{
uint8_t height = FONT_HEIGHT_SMALL;
switch (f) {
case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
height = FONT_HEIGHT_MEDIUM;
break;
case graphics::NotificationRenderer::BANNER_FONT_LARGE:
height = FONT_HEIGHT_LARGE;
break;
case graphics::NotificationRenderer::BANNER_FONT_SMALL:
case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
default:
height = FONT_HEIGHT_SMALL;
break;
}
return (height > 3) ? (height - 3) : height;
}
void graphics::NotificationRenderer::parseBannerMessageWithFonts(const char *message)
{
alertBannerLineCount = 0;
for (uint8_t i = 0; i < (MAX_LINES + 1); i++) {
alertBannerLines[i][0] = '\0';
alertBannerLineFonts[i] = BANNER_FONT_DEFAULT;
}
if (!message || !message[0]) {
return;
}
const char *p = message;
while (*p && alertBannerLineCount < (MAX_LINES + 1)) {
const char *lineStart = p;
while (*p && *p != '\n') {
p++;
}
char tmp[64] = {0};
size_t len = (size_t)(p - lineStart);
if (len > (sizeof(tmp) - 1)) {
len = sizeof(tmp) - 1;
}
memcpy(tmp, lineStart, len);
tmp[len] = '\0';
// Tag at start
const char *tp = tmp;
BannerFont f = parseFontTagPrefix(tp);
alertBannerLineFonts[alertBannerLineCount] = f;
// Store stripped text
strncpy(alertBannerLines[alertBannerLineCount], tp, sizeof(alertBannerLines[0]) - 1);
alertBannerLines[alertBannerLineCount][sizeof(alertBannerLines[0]) - 1] = '\0';
alertBannerLineCount++;
if (*p == '\n') {
p++;
}
}
}
// Used on boot when a certificate is being created
void NotificationRenderer::drawSSLScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
{
@@ -375,8 +479,20 @@ void NotificationRenderer::drawAlertBannerOverlay(OLEDDisplay *display, OLEDDisp
const char *lineStarts[MAX_LINES + 1] = {0};
uint16_t lineCount = 0;
char lineBuffer[40] = {0};
bool useTaggedTextBanner =
(current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0 && alertBannerLineCount > 0);
// Parse lines
if (useTaggedTextBanner) {
lineCount = std::min<uint8_t>(alertBannerLineCount, MAX_LINES);
for (uint16_t i = 0; i < lineCount; i++) {
lineStarts[i] = alertBannerLines[i];
lineLengths[i] = strlen(lineStarts[i]);
display->setFont(fontForBannerLine(alertBannerLineFonts[i]));
lineWidths[i] = display->getStringWidth(lineStarts[i], lineLengths[i], true);
if (lineWidths[i] > maxWidth)
maxWidth = lineWidths[i];
}
} else {
char *alertEnd = alertBannerMessage + strnlen(alertBannerMessage, sizeof(alertBannerMessage));
lineStarts[lineCount] = alertBannerMessage;
@@ -390,8 +506,10 @@ void NotificationRenderer::drawAlertBannerOverlay(OLEDDisplay *display, OLEDDisp
maxWidth = lineWidths[lineCount];
lineCount++;
}
}
// Measure option widths
display->setFont(FONT_SMALL);
for (int i = 0; i < alertBannerOptions; i++) {
optionWidths[i] = display->getStringWidth(optionsArrayPtr[i], strlen(optionsArrayPtr[i]), true);
if (optionWidths[i] > maxWidth)
@@ -505,6 +623,10 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
bool needs_bell = false;
uint16_t lineWidths[totalLines] = {0};
uint16_t lineLengths[totalLines] = {0};
BannerFont lineFonts[totalLines] = {};
uint8_t lineEffectiveHeights[totalLines] = {0};
const char *renderLines[totalLines] = {0};
bool useTaggedBannerFonts = (current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0);
if (maxWidth != 0)
is_picker = true;
@@ -517,11 +639,22 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
uint16_t widestLineWithBars = 0;
while (lines[lineCount] != nullptr) {
auto newlinePointer = strchr(lines[lineCount], '\n');
const char *renderText = lines[lineCount];
BannerFont lineFont = BANNER_FONT_DEFAULT;
if (useTaggedBannerFonts && lineCount < alertBannerLineCount) {
renderText = alertBannerLines[lineCount];
lineFont = alertBannerLineFonts[lineCount];
}
renderLines[lineCount] = renderText;
lineFonts[lineCount] = lineFont;
lineEffectiveHeights[lineCount] = effectiveLineHeightForBannerLine(lineFont);
display->setFont(fontForBannerLine(lineFont));
auto newlinePointer = strchr(renderText, '\n');
if (newlinePointer)
lineLengths[lineCount] = (newlinePointer - lines[lineCount]); // Check for newlines first
else // if the newline wasn't found, then pull string length from strlen
lineLengths[lineCount] = strlen(lines[lineCount]);
lineLengths[lineCount] = (newlinePointer - renderText);
else
lineLengths[lineCount] = strlen(renderText);
if (current_notification_type == notificationTypeEnum::node_picker) {
char measureBuffer[64] = {0};
@@ -533,7 +666,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
// Consider extra width for signal bars on lines that contain "Signal:"
uint16_t potentialWidth = lineWidths[lineCount];
if (graphics::bannerSignalBars >= 0 && strncmp(lines[lineCount], "Signal:", 7) == 0) {
if (graphics::bannerSignalBars >= 0 && strncmp(renderText, "Signal:", 7) == 0) {
const int totalBars = 5;
const int barWidth = 3;
const int barSpacing = 2;
@@ -569,8 +702,21 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
uint16_t screenHeight = display->height();
uint8_t effectiveLineHeight = FONT_HEIGHT_SMALL - 3;
uint8_t visibleTotalLines = std::min<uint8_t>(lineCount, (screenHeight - vPadding * 2) / effectiveLineHeight);
uint16_t contentHeight = visibleTotalLines * effectiveLineHeight;
uint8_t visibleTotalLines = 0;
uint16_t contentHeight = 0;
const uint16_t availableHeight = (screenHeight > (vPadding * 2)) ? (screenHeight - vPadding * 2) : 0;
for (uint8_t i = 0; i < lineCount; i++) {
uint8_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
if (contentHeight + thisLineHeight > availableHeight) {
break;
}
contentHeight += thisLineHeight;
visibleTotalLines++;
}
if (visibleTotalLines == 0 && lineCount > 0) {
visibleTotalLines = 1;
contentHeight = lineEffectiveHeights[0] ? lineEffectiveHeights[0] : effectiveLineHeight;
}
uint16_t boxHeight = contentHeight + vPadding * 2;
if (visibleTotalLines == 1) {
boxHeight += (currentResolution == ScreenResolution::High) ? 4 : 3;
@@ -616,15 +762,18 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
// Draw Content
int16_t lineY = boxTop + vPadding;
for (int i = 0; i < lineCount; i++) {
for (int i = 0; i < visibleTotalLines; i++) {
display->setFont(fontForBannerLine(lineFonts[i]));
int16_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
int16_t textX = boxLeft + (boxWidth - lineWidths[i]) / 2;
if (needs_bell && i == 0) {
int bellY = lineY + (FONT_HEIGHT_SMALL - 8) / 2;
int fontHeight = thisLineHeight + 3;
int bellY = lineY + (fontHeight - 8) / 2;
display->drawXbm(textX - 10, bellY, 8, 8, bell_alert);
display->drawXbm(textX + lineWidths[i] + 2, bellY, 8, 8, bell_alert);
}
char lineBuffer[lineLengths[i] + 1];
strncpy(lineBuffer, lines[i], lineLengths[i]);
strncpy(lineBuffer, renderLines[i], lineLengths[i]);
lineBuffer[lineLengths[i]] = '\0';
// Determine if this is a pop-up or a pick list
if (alertBannerOptions > 0 && i == 0) {
@@ -658,7 +807,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
display->drawString(textX, lineY - yOffset, lineBuffer);
}
display->setColor(WHITE);
lineY += (effectiveLineHeight - 2 - background_yOffset);
lineY += (thisLineHeight - 2 - background_yOffset);
} else {
// Pop-up
// If this is the Signal line, center text + bars as one group
@@ -716,7 +865,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
display->drawString(textX, lineY, lineBuffer);
}
}
lineY += (effectiveLineHeight);
lineY += thisLineHeight;
}
}
src/graphics/draw/NotificationRenderer.h
+6
View File
@@ -31,6 +31,12 @@ class NotificationRenderer
static bool pauseBanner;
enum BannerFont : uint8_t { BANNER_FONT_DEFAULT = 0, BANNER_FONT_SMALL, BANNER_FONT_MEDIUM, BANNER_FONT_LARGE };
static char alertBannerLines[MAX_LINES + 1][64]; // parsed text per line
static uint8_t alertBannerLineCount;
static BannerFont alertBannerLineFonts[MAX_LINES + 1];
static void parseBannerMessageWithFonts(const char *message);
static void resetBanner();
static void showKeyboardMessagePopupWithTitle(const char *title, const char *content, uint32_t durationMs);
static void drawBannercallback(OLEDDisplay *display, OLEDDisplayUiState *state);
src/graphics/niche/InkHUD/Applets/User/AllMessage/AllMessageApplet.h
+1 -2
View File
@@ -5,9 +5,8 @@
Shows the latest incoming text message, as well as sender.
Both broadcast and direct messages will be shown here, from all channels.
This module doesn't doesn't use the devicestate.rx_text_message,' as this is overwritten to contain outgoing messages
This module doesn't collect its own text message. Instead, the WindowManager stores the most recent incoming text message.
This is available to any interested modules (SingeMessageApplet, NotificationApplet etc.) via InkHUD::latestMessage
This is available to any interested modules (SingleMessageApplet, NotificationApplet etc.) via InkHUD::latestMessage
We do still receive notifications from the text message module though,
to know when a new message has arrived, and trigger the update.
src/graphics/niche/InkHUD/Applets/User/DM/DMApplet.h
+2 -3
View File
@@ -3,11 +3,10 @@
/*
Shows the latest incoming *Direct Message* (DM), as well as sender.
This compliments the threaded message applets
This complements the threaded message applets
This module doesn't doesn't use the devicestate.rx_text_message,' as this is overwritten to contain outgoing messages
This module doesn't collect its own text message. Instead, the WindowManager stores the most recent incoming text message.
This is available to any interested modules (SingeMessageApplet, NotificationApplet etc.) via InkHUD::latestMessage
This is available to any interested modules (SingleMessageApplet, NotificationApplet etc.) via InkHUD::latestMessage
We do still receive notifications from the text message module though,
to know when a new message has arrived, and trigger the update.
src/graphics/niche/InkHUD/Events.cpp
+1 -1
View File
@@ -525,7 +525,7 @@ int InkHUD::Events::beforeReboot(void *unused)
// Callback when a new text message is received
// Caches the most recently received message, for use by applets
// Rx does not trigger a save to flash, however the data *will* be saved alongside other during shutdown, etc.
// Note: this is different from devicestate.rx_text_message, which may contain an *outgoing* message
// Note: this is intentionally separate from device-state message fields.
int InkHUD::Events::onReceiveTextMessage(const meshtastic_MeshPacket *packet)
{
// Short circuit: don't store outgoing messages
src/graphics/niche/InkHUD/Persistence.h
+1 -2
View File
@@ -121,8 +121,7 @@ class Persistence
// Most recently received text message
// Value is updated by InkHUD::WindowManager, as a courtesy to applets
// Note: different from devicestate.rx_text_message,
// which may contain an *outgoing message* to broadcast
// InkHUD keeps its own latest-message cache for applets.
struct LatestMessage {
MessageStore::Message broadcast; // Most recent message received broadcast
MessageStore::Message dm; // Most recent received DM
src/graphics/niche/InkHUD/docs/README.md
+1 -1
View File
@@ -464,7 +464,7 @@ Most recently received text message
Collected here, so various user applets don't all have to store their own copy of this info.
We are unable to use `devicestate.rx_text_message` for this purpose, because:
We keep this separate latest-message cache for this purpose, because:
- it is cleared by an outgoing text message
- we want to store both a recent broadcast and a recent DM
src/input/InputBroker.cpp
+6
View File
@@ -333,6 +333,12 @@ void InputBroker::Init()
BaseType_t higherWake = 0;
concurrency::mainDelay.interruptFromISR(&higherWake);
};
#if defined(ELECROW_ThinkNode_M7)
userConfigNoScreen.longLongPressTime = 15 * 1000;
userConfigNoScreen.longLongPress = INPUT_BROKER_FACTORY_RST;
#else
userConfigNoScreen.longLongPress = INPUT_BROKER_SHUTDOWN;
#endif
userConfigNoScreen.singlePress = INPUT_BROKER_USER_PRESS;
userConfigNoScreen.longPress = INPUT_BROKER_NONE;
userConfigNoScreen.longPressTime = 500;
src/input/InputBroker.h
+1
View File
@@ -25,6 +25,7 @@ enum input_broker_event {
INPUT_BROKER_USER_PRESS,
INPUT_BROKER_ALT_PRESS,
INPUT_BROKER_ALT_LONG,
INPUT_BROKER_FACTORY_RST = 0x9a,
INPUT_BROKER_SHUTDOWN = 0x9b,
INPUT_BROKER_GPS_TOGGLE = 0x9e,
INPUT_BROKER_SEND_PING = 0xaf,
src/main.cpp
+19 -4
View File
@@ -2,6 +2,9 @@
#if !MESHTASTIC_EXCLUDE_GPS
#include "GPS.h"
#endif
#if !MESHTASTIC_EXCLUDE_INPUTBROKER
#include "input/InputBroker.h"
#endif
#include "MeshRadio.h"
#include "MeshService.h"
#include "NodeDB.h"
@@ -59,12 +62,18 @@ NimbleBluetooth *nimbleBluetooth = nullptr;
NRF52Bluetooth *nrf52Bluetooth = nullptr;
#endif
#if HAS_WIFI || defined(USE_WS5500)
#ifdef ARCH_NRF54L15
void nrf54l15Setup();
void nrf54l15Loop();
NRF54L15Bluetooth *nrf54l15Bluetooth = nullptr;
#endif
#if HAS_WIFI || defined(USE_WS5500) || defined(USE_CH390D)
#include "mesh/api/WiFiServerAPI.h"
#include "mesh/wifi/WiFiAPClient.h"
#endif
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
#include "mesh/api/ethServerAPI.h"
#include "mesh/eth/ethClient.h"
#endif
@@ -245,7 +254,7 @@ const char *getDeviceName()
uint32_t timeLastPowered = 0;
static OSThread *powerFSMthread;
OSThread *ambientLightingThread;
AmbientLightingThread *ambientLightingThread;
RadioLibHal *RadioLibHAL = NULL;
@@ -335,7 +344,7 @@ void setup()
#ifdef WIFI_LED
pinMode(WIFI_LED, OUTPUT);
digitalWrite(WIFI_LED, LOW);
digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
#endif
#ifdef BLE_LED
@@ -696,6 +705,9 @@ void setup()
#ifdef ARCH_NRF52
nrf52Setup();
#endif
#ifdef ARCH_NRF54L15
nrf54l15Setup();
#endif
#ifdef ARCH_RP2040
rp2040Setup();
@@ -1126,6 +1138,9 @@ void loop()
#endif
#ifdef ARCH_NRF52
nrf52Loop();
#endif
#ifdef ARCH_NRF54L15
nrf54l15Loop();
#endif
power->powerCommandsCheck();
src/main.h
+4
View File
@@ -19,6 +19,10 @@ extern NimbleBluetooth *nimbleBluetooth;
#include "NRF52Bluetooth.h"
extern NRF52Bluetooth *nrf52Bluetooth;
#endif
#ifdef ARCH_NRF54L15
#include "NRF54L15Bluetooth.h"
extern NRF54L15Bluetooth *nrf54l15Bluetooth;
#endif
#if !MESHTASTIC_EXCLUDE_I2C
#include "detect/ScanI2CTwoWire.h"
#endif
src/mesh/Channels.cpp
+17
View File
@@ -93,6 +93,23 @@ void Channels::initDefaultLoraConfig()
#ifdef USERPREFS_LORACONFIG_CHANNEL_NUM
loraConfig.channel_num = USERPREFS_LORACONFIG_CHANNEL_NUM;
#endif
// Apply any hardcoded USERPREFS overrides for custom modem config (e.g. region-locked boards)
#ifdef USERPREFS_LORACONFIG_USE_PRESET
loraConfig.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
#endif
#ifdef USERPREFS_LORACONFIG_BANDWIDTH
loraConfig.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
#endif
#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
loraConfig.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
#endif
#ifdef USERPREFS_LORACONFIG_CODING_RATE
loraConfig.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
#endif
#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
loraConfig.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
#endif
}
bool Channels::ensureLicensedOperation()
src/mesh/InterfacesTemplates.cpp
+1 -1
View File
@@ -32,7 +32,7 @@ template class LR20x0Interface<LR2021>;
template class SX126xInterface<STM32WLx>;
#endif
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
#include "api/ethServerAPI.h"
template class ServerAPI<EthernetClient>;
template class APIServerPort<ethServerAPI, EthernetServer>;
src/mesh/MeshService.cpp
+7 -3
View File
@@ -334,7 +334,9 @@ void MeshService::sendToPhone(meshtastic_MeshPacket *p)
if (toPhoneQueue.enqueue(p, 0) == false) {
LOG_CRIT("Failed to queue a packet into toPhoneQueue!");
abort();
releaseToPool(p);
fromNum++; // notify observers so phone can resync
return;
}
fromNum++;
}
@@ -351,7 +353,8 @@ void MeshService::sendMqttMessageToClientProxy(meshtastic_MqttClientProxyMessage
if (toPhoneMqttProxyQueue.enqueue(m, 0) == false) {
LOG_CRIT("Failed to queue a packet into toPhoneMqttProxyQueue!");
abort();
releaseMqttClientProxyMessageToPool(m);
return;
}
fromNum++;
}
@@ -383,7 +386,8 @@ void MeshService::sendClientNotification(meshtastic_ClientNotification *n)
if (toPhoneClientNotificationQueue.enqueue(n, 0) == false) {
LOG_CRIT("Failed to queue a notification into toPhoneClientNotificationQueue!");
abort();
releaseClientNotificationToPool(n);
return;
}
fromNum++;
}
src/mesh/NodeDB.cpp
+207 -53
View File
@@ -25,6 +25,7 @@
#include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
#include "meshUtils.h"
#include "modules/NeighborInfoModule.h"
#include "xmodem.h"
#include <ErriezCRC32.h>
#include <algorithm>
#include <pb_decode.h>
@@ -81,6 +82,14 @@ static unsigned char userprefs_admin_key_1[] = USERPREFS_USE_ADMIN_KEY_1;
static unsigned char userprefs_admin_key_2[] = USERPREFS_USE_ADMIN_KEY_2;
#endif
// Weak empty variant initialization function.
// May be redefined by variant files.
void variantDefaultConfig() __attribute__((weak));
void variantDefaultConfig() {}
void variantDefaultModuleConfig() __attribute__((weak));
void variantDefaultModuleConfig() {}
#ifdef HELTEC_MESH_NODE_T114
uint32_t read8(uint8_t bits, uint8_t dummy, uint8_t cs, uint8_t sck, uint8_t mosi, uint8_t dc, uint8_t rst)
@@ -152,6 +161,25 @@ uint32_t get_st7789_id(uint8_t cs, uint8_t sck, uint8_t mosi, uint8_t dc, uint8_
#endif
// When armed by loadFromDisk, the decode callback writes satellite entries
// straight into these maps instead of the temp vectors. Nullptr = legacy
// push_back-to-vector path for backup/restore and other decoders.
namespace
{
#if !MESHTASTIC_EXCLUDE_POSITIONDB
std::map<NodeNum, meshtastic_PositionLite> *s_decodePositionsTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
std::map<NodeNum, meshtastic_DeviceMetrics> *s_decodeTelemetryTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
std::map<NodeNum, meshtastic_EnvironmentMetrics> *s_decodeEnvironmentTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
std::map<NodeNum, meshtastic_StatusMessage> *s_decodeStatusTarget = nullptr;
#endif
} // namespace
bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field)
{
const auto *iter = reinterpret_cast<const pb_field_iter_t *>(field);
@@ -166,10 +194,10 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
return false;
}
}
if (istream) {
meshtastic_NodeInfoLite node;
if (istream && istream->bytes_left) {
meshtastic_NodeInfoLite node = meshtastic_NodeInfoLite_init_zero;
auto *vec = static_cast<std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
if (istream->bytes_left && pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
if (pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
vec->push_back(node);
}
return true;
@@ -184,12 +212,20 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
return false;
}
}
if (istream) {
meshtastic_NodePositionEntry entry;
if (istream && istream->bytes_left) {
meshtastic_NodePositionEntry entry = meshtastic_NodePositionEntry_init_zero;
if (pb_decode(istream, meshtastic_NodePositionEntry_fields, &entry)) {
#if !MESHTASTIC_EXCLUDE_POSITIONDB
if (s_decodePositionsTarget) {
if (entry.has_position)
(*s_decodePositionsTarget)[entry.num] = entry.position;
return true;
}
#endif
auto *vec = static_cast<std::vector<meshtastic_NodePositionEntry> *>(iter->pData);
if (istream->bytes_left && pb_decode(istream, meshtastic_NodePositionEntry_fields, &entry))
vec->push_back(entry);
}
}
return true;
}
case meshtastic_NodeDatabase_telemetry_tag: {
@@ -202,12 +238,20 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
return false;
}
}
if (istream) {
meshtastic_NodeTelemetryEntry entry;
if (istream && istream->bytes_left) {
meshtastic_NodeTelemetryEntry entry = meshtastic_NodeTelemetryEntry_init_zero;
if (pb_decode(istream, meshtastic_NodeTelemetryEntry_fields, &entry)) {
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
if (s_decodeTelemetryTarget) {
if (entry.has_device_metrics)
(*s_decodeTelemetryTarget)[entry.num] = entry.device_metrics;
return true;
}
#endif
auto *vec = static_cast<std::vector<meshtastic_NodeTelemetryEntry> *>(iter->pData);
if (istream->bytes_left && pb_decode(istream, meshtastic_NodeTelemetryEntry_fields, &entry))
vec->push_back(entry);
}
}
return true;
}
case meshtastic_NodeDatabase_status_tag: {
@@ -220,12 +264,20 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
return false;
}
}
if (istream) {
meshtastic_NodeStatusEntry entry;
if (istream && istream->bytes_left) {
meshtastic_NodeStatusEntry entry = meshtastic_NodeStatusEntry_init_zero;
if (pb_decode(istream, meshtastic_NodeStatusEntry_fields, &entry)) {
#if !MESHTASTIC_EXCLUDE_STATUSDB
if (s_decodeStatusTarget) {
if (entry.has_status)
(*s_decodeStatusTarget)[entry.num] = entry.status;
return true;
}
#endif
auto *vec = static_cast<std::vector<meshtastic_NodeStatusEntry> *>(iter->pData);
if (istream->bytes_left && pb_decode(istream, meshtastic_NodeStatusEntry_fields, &entry))
vec->push_back(entry);
}
}
return true;
}
case meshtastic_NodeDatabase_environment_tag: {
@@ -238,12 +290,20 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
return false;
}
}
if (istream) {
meshtastic_NodeEnvironmentEntry entry;
if (istream && istream->bytes_left) {
meshtastic_NodeEnvironmentEntry entry = meshtastic_NodeEnvironmentEntry_init_zero;
if (pb_decode(istream, meshtastic_NodeEnvironmentEntry_fields, &entry)) {
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
if (s_decodeEnvironmentTarget) {
if (entry.has_environment_metrics)
(*s_decodeEnvironmentTarget)[entry.num] = entry.environment_metrics;
return true;
}
#endif
auto *vec = static_cast<std::vector<meshtastic_NodeEnvironmentEntry> *>(iter->pData);
if (istream->bytes_left && pb_decode(istream, meshtastic_NodeEnvironmentEntry_fields, &entry))
vec->push_back(entry);
}
}
return true;
}
default:
@@ -251,6 +311,42 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
}
}
void NodeDB::armNodeDatabaseDecodeTargets()
{
#if !MESHTASTIC_EXCLUDE_POSITIONDB
nodePositions.clear();
s_decodePositionsTarget = &nodePositions;
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
nodeTelemetry.clear();
s_decodeTelemetryTarget = &nodeTelemetry;
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
nodeEnvironment.clear();
s_decodeEnvironmentTarget = &nodeEnvironment;
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
nodeStatus.clear();
s_decodeStatusTarget = &nodeStatus;
#endif
}
void NodeDB::disarmNodeDatabaseDecodeTargets()
{
#if !MESHTASTIC_EXCLUDE_POSITIONDB
s_decodePositionsTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
s_decodeTelemetryTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
s_decodeEnvironmentTarget = nullptr;
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
s_decodeStatusTarget = nullptr;
#endif
}
/** The current change # for radio settings. Starts at 0 on boot and any time the radio settings
* might have changed is incremented. Allows others to detect they might now be on a new channel.
*/
@@ -297,6 +393,13 @@ NodeDB::NodeDB()
} else {
LOG_WARN("Failed to read unique id from efuse");
}
#elif defined(ARCH_NRF54L15)
// nRF54L15: DEVICEID is under FICR->INFO sub-struct (not top-level as on nRF52)
uint64_t device_id_start = ((uint64_t)NRF_FICR->INFO.DEVICEID[1] << 32) | NRF_FICR->INFO.DEVICEID[0];
uint64_t device_id_end = ((uint64_t)NRF_FICR->DEVICEADDR[1] << 32) | NRF_FICR->DEVICEADDR[0];
memcpy(myNodeInfo.device_id.bytes, &device_id_start, sizeof(device_id_start));
memcpy(myNodeInfo.device_id.bytes + sizeof(device_id_start), &device_id_end, sizeof(device_id_end));
myNodeInfo.device_id.size = 16;
#elif defined(ARCH_NRF52)
// Nordic applies a FIPS compliant Random ID to each chip at the factory
// We concatenate the device address to the Random ID to create a unique ID for now
@@ -730,6 +833,23 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
config.lora.modem_preset = USERPREFS_LORACONFIG_MODEM_PRESET;
#else
config.lora.modem_preset = meshtastic_Config_LoRaConfig_ModemPreset_LONG_FAST;
#endif
#ifdef USERPREFS_LORACONFIG_USE_PRESET
config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
#else
config.lora.use_preset = true;
#endif
#ifdef USERPREFS_LORACONFIG_BANDWIDTH
config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
#endif
#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
#endif
#ifdef USERPREFS_LORACONFIG_CODING_RATE
config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
#endif
#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
#endif
config.lora.hop_limit = HOP_RELIABLE;
#ifdef USERPREFS_CONFIG_LORA_IGNORE_MQTT
@@ -904,6 +1024,8 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
#endif
initConfigIntervals();
variantDefaultConfig();
variantDefaultModuleConfig();
}
void NodeDB::initConfigIntervals()
@@ -1119,7 +1241,14 @@ void NodeDB::initModuleConfigIntervals()
#else
moduleConfig.telemetry.device_update_interval = MAX_INTERVAL;
#endif
#ifdef USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL
moduleConfig.telemetry.environment_update_interval = USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL;
#else
moduleConfig.telemetry.environment_update_interval = 0;
#endif
#ifdef USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED
moduleConfig.telemetry.environment_measurement_enabled = USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED;
#endif
moduleConfig.telemetry.air_quality_interval = 0;
moduleConfig.telemetry.power_update_interval = 0;
moduleConfig.telemetry.health_update_interval = 0;
@@ -1161,7 +1290,6 @@ void NodeDB::resetNodes(bool keepFavorites)
std::fill(nodeDatabase.nodes.begin() + 1, nodeDatabase.nodes.end(), meshtastic_NodeInfoLite());
}
(void)ourNum;
devicestate.has_rx_text_message = false;
devicestate.has_rx_waypoint = false;
saveNodeDatabaseToDisk();
saveDeviceStateToDisk();
@@ -1368,7 +1496,6 @@ void NodeDB::installDefaultDeviceState()
devicestate.version = DEVICESTATE_CUR_VER;
devicestate.receive_queue_count = 0; // Not yet implemented FIXME
devicestate.has_rx_waypoint = false;
devicestate.has_rx_text_message = false;
generatePacketId(); // FIXME - ugly way to init current_packet_id;
@@ -1510,6 +1637,19 @@ void NodeDB::loadFromDisk()
}
#endif
// Arm the direct-into-map decode so satellite entries skip the temp vectors.
{
concurrency::LockGuard guard(&satelliteMutex);
armNodeDatabaseDecodeTargets();
}
struct Disarm {
NodeDB &self;
~Disarm() { self.disarmNodeDatabaseDecodeTargets(); }
} disarm{*this};
// Avoid push_back's power-of-2 capacity growth wasting RAM at small N.
nodeDatabase.nodes.reserve(MAX_NUM_NODES);
auto state = loadProto(nodeDatabaseFileName, getMaxNodesAllocatedSize(), sizeof(meshtastic_NodeDatabase),
&meshtastic_NodeDatabase_msg, &nodeDatabase);
if (nodeDatabase.version < DEVICESTATE_MIN_VER) {
@@ -1523,50 +1663,33 @@ void NodeDB::loadFromDisk()
} else {
meshNodes = &nodeDatabase.nodes;
numMeshNodes = nodeDatabase.nodes.size();
// Hydrate the satellite maps; the on-disk vectors stay empty in steady
// state and are repopulated only at save time.
concurrency::LockGuard guard(&satelliteMutex);
// Counts computed outside LOG_INFO() so cppcheck doesn't choke on #if in macro args.
const unsigned posCount =
#if !MESHTASTIC_EXCLUDE_POSITIONDB
nodePositions.clear();
nodePositions.reserve(nodeDatabase.positions.size());
for (const auto &entry : nodeDatabase.positions) {
if (entry.has_position)
nodePositions[entry.num] = entry.position;
}
nodeDatabase.positions.clear();
nodeDatabase.positions.shrink_to_fit();
(unsigned)nodePositions.size();
#else
0u;
#endif
const unsigned telCount =
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
nodeTelemetry.clear();
nodeTelemetry.reserve(nodeDatabase.telemetry.size());
for (const auto &entry : nodeDatabase.telemetry) {
if (entry.has_device_metrics)
nodeTelemetry[entry.num] = entry.device_metrics;
}
nodeDatabase.telemetry.clear();
nodeDatabase.telemetry.shrink_to_fit();
(unsigned)nodeTelemetry.size();
#else
0u;
#endif
const unsigned envCount =
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
nodeEnvironment.clear();
nodeEnvironment.reserve(nodeDatabase.environment.size());
for (const auto &entry : nodeDatabase.environment) {
if (entry.has_environment_metrics)
nodeEnvironment[entry.num] = entry.environment_metrics;
}
nodeDatabase.environment.clear();
nodeDatabase.environment.shrink_to_fit();
(unsigned)nodeEnvironment.size();
#else
0u;
#endif
const unsigned statusCount =
#if !MESHTASTIC_EXCLUDE_STATUSDB
nodeStatus.clear();
nodeStatus.reserve(nodeDatabase.status.size());
for (const auto &entry : nodeDatabase.status) {
if (entry.has_status)
nodeStatus[entry.num] = entry.status;
}
nodeDatabase.status.clear();
nodeDatabase.status.shrink_to_fit();
(unsigned)nodeStatus.size();
#else
0u;
#endif
LOG_INFO("Loaded saved nodedatabase version %d, with nodes count: %d", nodeDatabase.version, nodeDatabase.nodes.size());
LOG_INFO("Loaded saved nodedatabase v%d: %d nodes, %u pos, %u tel, %u env, %u status", nodeDatabase.version,
nodeDatabase.nodes.size(), posCount, telCount, envCount, statusCount);
}
if (numMeshNodes > MAX_NUM_NODES) {
@@ -1634,6 +1757,28 @@ void NodeDB::loadFromDisk()
config.lora.tx_enabled = false;
#endif
// Always-apply LoRa overrides: applied after loading saved config so they
// take effect even when NVS already has a valid config (e.g. region-locked
// dev boards with no BLE/serial to set the region at runtime).
#ifdef USERPREFS_CONFIG_LORA_REGION
config.lora.region = USERPREFS_CONFIG_LORA_REGION;
#endif
#ifdef USERPREFS_LORACONFIG_USE_PRESET
config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
#endif
#ifdef USERPREFS_LORACONFIG_BANDWIDTH
config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
#endif
#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
#endif
#ifdef USERPREFS_LORACONFIG_CODING_RATE
config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
#endif
#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
#endif
if (backupSecurity.private_key.size > 0) {
LOG_DEBUG("Restoring backup of security config");
config.security = backupSecurity;
@@ -1844,6 +1989,15 @@ bool NodeDB::saveNodeDatabaseToDisk()
return false;
}
// Defer (don't fail) while xmodem holds the prefs file handle. Returning false
// would propagate through saveToDisk() and trigger fsFormat() mid-transfer.
#ifdef FSCom
if (xModem.isBusy()) {
LOG_DEBUG("Deferring NodeDB save: xmodem transfer in progress");
return true;
}
#endif
#ifdef FSCom
spiLock->lock();
FSCom.mkdir("/prefs");
src/mesh/NodeDB.h
+12 -7
View File
@@ -4,9 +4,9 @@
#include <Arduino.h>
#include <algorithm>
#include <assert.h>
#include <map>
#include <pb_encode.h>
#include <string>
#include <unordered_map>
#include <vector>
#include "MeshTypes.h"
@@ -170,19 +170,19 @@ class NodeDB
Observable<const meshtastic::NodeStatus *> newStatus;
pb_size_t numMeshNodes;
// Satellite per-NodeNum maps for data we used to inline into NodeInfoLite,
// gated by MESHTASTIC_EXCLUDE_*DB so STM32WL can omit them.
// Satellite per-NodeNum maps. std::map avoids unordered_map's bucket-array
// preallocation; O(log N) lookup is fine at these sizes.
#if !MESHTASTIC_EXCLUDE_POSITIONDB
std::unordered_map<NodeNum, meshtastic_PositionLite> nodePositions;
std::map<NodeNum, meshtastic_PositionLite> nodePositions;
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
std::unordered_map<NodeNum, meshtastic_DeviceMetrics> nodeTelemetry;
std::map<NodeNum, meshtastic_DeviceMetrics> nodeTelemetry;
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
std::unordered_map<NodeNum, meshtastic_EnvironmentMetrics> nodeEnvironment;
std::map<NodeNum, meshtastic_EnvironmentMetrics> nodeEnvironment;
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
std::unordered_map<NodeNum, meshtastic_StatusMessage> nodeStatus;
std::map<NodeNum, meshtastic_StatusMessage> nodeStatus;
#endif
bool keyIsLowEntropy = false;
@@ -429,6 +429,11 @@ class NodeDB
// the legacy descriptor and copies entries into the v25 layout. Caller
// is responsible for save / install-default on the result.
bool migrateLegacyNodeDatabase();
// Route satellite-store decode entries straight into our maps instead of
// temp vectors. Must be paired — disarm before any other NodeDatabase decode.
void armNodeDatabaseDecodeTargets();
void disarmNodeDatabaseDecodeTargets();
};
extern NodeDB *nodeDB;
src/mesh/PacketHistory.cpp
+2 -2
View File
@@ -9,8 +9,8 @@
#include "Throttle.h"
#define PACKETHISTORY_MAX \
max((u_int32_t)(MAX_NUM_NODES * 2.0), \
(u_int32_t)100) // x2..3 Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100
max((uint32_t)(MAX_NUM_NODES * 2.0), \
(uint32_t)100) // x2..3 Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100
#define RECENT_WARN_AGE (10 * 60 * 1000L) // Warn if the packet that gets removed was more recent than 10 min
src/mesh/PhoneAPI.cpp
+166 -189
View File
@@ -62,7 +62,7 @@ void PhoneAPI::handleStartConfig()
onConfigStart();
// even if we were already connected - restart our state machine
if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
// If client only wants node info, jump directly to sending nodes
state = STATE_SEND_OWN_NODEINFO;
LOG_INFO("Client only wants node info, skipping other config");
@@ -71,7 +71,15 @@ void PhoneAPI::handleStartConfig()
}
pauseBluetoothLogging = true;
spiLock->lock();
#if defined(MESHTASTIC_EXCLUDE_FILES_MANIFEST)
// Skip the recursive FS walk. Used by platforms whose Zephyr LittleFS
// backend can't safely traverse a deep tree (e.g. nRF54L15) and platforms
// that don't support OTA browsing — the manifest is only consumed by
// companion apps for those flows.
filesManifest.clear();
#else
filesManifest = getFiles("/", 10);
#endif
spiLock->unlock();
LOG_DEBUG("Got %d files in manifest", filesManifest.size());
@@ -138,6 +146,7 @@ void PhoneAPI::close()
replayTelemetryIndex = 0;
replayEnvironmentIndex = 0;
replayStatusIndex = 0;
replayPhase = REPLAY_PHASE_IDLE;
}
packetForPhone = NULL;
filesManifest.clear();
@@ -320,7 +329,7 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
nodeInfoForPhone.num = 0;
}
}
if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
// If client only wants node info, jump directly to sending nodes
state = STATE_SEND_OTHER_NODEINFOS;
onNowHasData(0);
@@ -535,135 +544,16 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
// Just in case we stored a different user.id in the past, but should never happen going forward
sprintf(infoToSend.user.id, "!%08x", infoToSend.num);
// Logging this really slows down sending nodes on initial connection because the serial console is so slow, so only
// uncomment if you really need to:
// LOG_INFO("nodeinfo: num=0x%x, lastseen=%u, id=%s, name=%s", nodeInfoForPhone.num, nodeInfoForPhone.last_heard,
// nodeInfoForPhone.user.id, nodeInfoForPhone.user.long_name);
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_node_info_tag;
fromRadioScratch.node_info = infoToSend;
prefetchNodeInfos();
} else {
LOG_DEBUG("Done sending %d of %d nodeinfos millis=%u", readIndex, nodeDB->getNumMeshNodes(), millis());
concurrency::LockGuard guard(&nodeInfoMutex);
nodeInfoMutex.lock();
nodeInfoQueue.clear();
// Replay states no-op for legacy clients / excluded DBs.
state = STATE_REPLAY_POSITIONS;
return getFromRadio(buf);
}
break;
}
case STATE_REPLAY_POSITIONS: {
if (replayPositionOrder.empty() && replayPositionIndex == 0)
beginReplayPositions();
prefetchReplayPositions();
meshtastic_MeshPacket pkt = {};
bool havePkt = false;
{
concurrency::LockGuard guard(&nodeInfoMutex);
if (!replayQueue.empty()) {
pkt = replayQueue.front();
replayQueue.pop_front();
havePkt = true;
}
}
if (havePkt) {
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = pkt;
} else {
LOG_DEBUG("Done replaying positions count=%u millis=%u", (unsigned)replayPositionIndex, millis());
state = STATE_REPLAY_TELEMETRY;
return getFromRadio(buf);
}
break;
}
case STATE_REPLAY_TELEMETRY: {
if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
beginReplayTelemetry();
prefetchReplayTelemetry();
meshtastic_MeshPacket pkt = {};
bool havePkt = false;
{
concurrency::LockGuard guard(&nodeInfoMutex);
if (!replayQueue.empty()) {
pkt = replayQueue.front();
replayQueue.pop_front();
havePkt = true;
}
}
if (havePkt) {
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = pkt;
} else {
LOG_DEBUG("Done replaying telemetry count=%u millis=%u", (unsigned)replayTelemetryIndex, millis());
state = STATE_REPLAY_ENVIRONMENT;
return getFromRadio(buf);
}
break;
}
case STATE_REPLAY_ENVIRONMENT: {
if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
beginReplayEnvironment();
prefetchReplayEnvironment();
meshtastic_MeshPacket pkt = {};
bool havePkt = false;
{
concurrency::LockGuard guard(&nodeInfoMutex);
if (!replayQueue.empty()) {
pkt = replayQueue.front();
replayQueue.pop_front();
havePkt = true;
}
}
if (havePkt) {
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = pkt;
} else {
LOG_DEBUG("Done replaying environment count=%u millis=%u", (unsigned)replayEnvironmentIndex, millis());
state = STATE_REPLAY_STATUS;
return getFromRadio(buf);
}
break;
}
case STATE_REPLAY_STATUS: {
if (replayStatusOrder.empty() && replayStatusIndex == 0)
beginReplayStatus();
prefetchReplayStatus();
meshtastic_MeshPacket pkt = {};
bool havePkt = false;
{
concurrency::LockGuard guard(&nodeInfoMutex);
if (!replayQueue.empty()) {
pkt = replayQueue.front();
replayQueue.pop_front();
havePkt = true;
}
}
if (havePkt) {
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = pkt;
} else {
LOG_DEBUG("Done replaying status count=%u millis=%u", (unsigned)replayStatusIndex, millis());
replayPositionOrder.clear();
replayPositionOrder.shrink_to_fit();
replayTelemetryOrder.clear();
replayTelemetryOrder.shrink_to_fit();
replayEnvironmentOrder.clear();
replayEnvironmentOrder.shrink_to_fit();
replayStatusOrder.clear();
replayStatusOrder.shrink_to_fit();
nodeInfoMutex.unlock();
// Satellite-DB replay (positions/telemetry/environment/status) now happens
// *after* config_complete_id, interleaved with live traffic in STATE_SEND_PACKETS.
state = STATE_SEND_FILEMANIFEST;
return getFromRadio(buf);
}
@@ -673,8 +563,7 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
case STATE_SEND_FILEMANIFEST: {
LOG_DEBUG("FromRadio=STATE_SEND_FILEMANIFEST");
// ONLY_NODES variants skip the manifest.
if (config_state == filesManifest.size() || config_nonce == SPECIAL_NONCE_ONLY_NODES ||
config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
if (config_state == filesManifest.size() || config_nonce == SPECIAL_NONCE_ONLY_NODES) {
config_state = 0;
filesManifest.clear();
// Skip to complete packet
@@ -719,6 +608,16 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = *packetForPhone;
releasePhonePacket();
} else if (replayPending()) {
// No live packet pending — feed the phone one cached satellite-DB packet.
// popReplayPacket advances through positions->telemetry->environment->status,
// and flips replayPhase back to IDLE when everything has been drained.
meshtastic_MeshPacket replayPkt;
if (popReplayPacket(replayPkt)) {
printPacket("replay packet to phone", &replayPkt);
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
fromRadioScratch.packet = replayPkt;
}
}
break;
@@ -743,10 +642,19 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
void PhoneAPI::sendConfigComplete()
{
LOG_INFO("Config Send Complete millis=%u", millis());
const bool shouldReplaySatellites = (config_nonce != SPECIAL_NONCE_ONLY_CONFIG);
// The phone sees config_complete_id first (treats sync as done), then the cached
// satellite-DB packets (positions / telemetry / environment / status) trickle in
// afterward as ordinary mesh packets (except SPECIAL_NONCE_ONLY_CONFIG, which
// skips node/satellite sync entirely). Any client that handles live POSITION_APP /
// TELEMETRY_APP / NODE_STATUS_APP packets handles these identically. STM32WL and
// other builds that compile the satellite DBs out produce no replay packets and
// the phase advances to IDLE in microseconds.
fromRadioScratch.which_payload_variant = meshtastic_FromRadio_config_complete_id_tag;
fromRadioScratch.config_complete_id = config_nonce;
config_nonce = 0;
state = STATE_SEND_PACKETS;
replayPhase = shouldReplaySatellites ? REPLAY_PHASE_POSITIONS : REPLAY_PHASE_IDLE;
if (api_type == TYPE_BLE) {
service->api_state = service->STATE_BLE;
} else if (api_type == TYPE_WIFI) {
@@ -787,7 +695,8 @@ void PhoneAPI::prefetchNodeInfos()
{
bool added = false;
bool wasEmpty = false;
const bool gradient = clientWantsGradientSync();
// Other-node NodeInfos always go out thin (no bundled position/device_metrics).
// The post-config_complete_id replay drain delivers those as ordinary mesh packets.
// Keep the queue topped up so BLE reads stay responsive even if DB fetches take a moment.
{
concurrency::LockGuard guard(&nodeInfoMutex);
@@ -797,8 +706,7 @@ void PhoneAPI::prefetchNodeInfos()
if (!nextNode)
break;
auto info =
gradient ? TypeConversions::ConvertToNodeInfoThin(nextNode) : TypeConversions::ConvertToNodeInfo(nextNode);
auto info = TypeConversions::ConvertToNodeInfoThin(nextNode);
bool isUs = info.num == nodeDB->getNodeNum();
info.hops_away = isUs ? 0 : info.hops_away;
info.last_heard = isUs ? getValidTime(RTCQualityFromNet) : info.last_heard;
@@ -820,11 +728,20 @@ void PhoneAPI::prefetchNodeInfos()
meshtastic_MeshPacket PhoneAPI::makeReplayPositionPacket(NodeNum num, const meshtastic_PositionLite &pos)
{
// Shape this exactly like a fresh live broadcast Position from the peer so the
// phone runs it through its normal "live position broadcast" handler path.
// to=ourNum would read as a DM-from-peer and never lands in node detail UI.
meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
pkt.from = num;
pkt.to = nodeDB->getNodeNum();
pkt.to = NODENUM_BROADCAST;
pkt.id = generatePacketId();
pkt.rx_time = pos.time;
pkt.channel = 0;
pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
pkt.hop_start = pkt.hop_limit;
pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
// Mark as if heard over the air, not internally generated
pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
pkt.decoded.portnum = meshtastic_PortNum_POSITION_APP;
meshtastic_Position fullPos = TypeConversions::ConvertToPosition(pos);
@@ -838,11 +755,18 @@ meshtastic_MeshPacket PhoneAPI::makeReplayTelemetryPacket(NodeNum num, const mes
{
meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
pkt.from = num;
pkt.to = nodeDB->getNodeNum();
pkt.to = NODENUM_BROADCAST;
pkt.id = generatePacketId();
// No native timestamp on telemetry packets here; use last_heard.
const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
pkt.rx_time = header ? header->last_heard : 0;
pkt.channel = 0;
pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
pkt.hop_start = pkt.hop_limit;
pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
// Mark as if heard over the air, not internally generated — iOS client filters
// TRANSPORT_INTERNAL packets out of broadcast peer state updates.
pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
@@ -858,16 +782,12 @@ meshtastic_MeshPacket PhoneAPI::makeReplayTelemetryPacket(NodeNum num, const mes
void PhoneAPI::beginReplayPositions()
{
#if MESHTASTIC_EXCLUDE_POSITIONDB
// Build excluded entirely - leave the order list empty so the state arm
// Build excluded entirely - leave the order list empty so the phase
// immediately drains and advances.
replayPositionOrder.clear();
replayPositionIndex = 0;
#else
if (!clientWantsGradientSync()) {
replayPositionOrder.clear();
replayPositionIndex = 0;
return;
}
// Caller (popReplayPacket) only invokes us when replayPhase is armed.
// Snapshot the keyset at phase start so concurrent inserts/erases on the
// map don't invalidate iteration. Skip our own node - the phone already
// got our position bundled in STATE_SEND_OWN_NODEINFO.
@@ -882,8 +802,6 @@ void PhoneAPI::prefetchReplayPositions()
#if MESHTASTIC_EXCLUDE_POSITIONDB
return;
#else
if (!clientWantsGradientSync())
return;
bool added = false;
bool wasEmpty = false;
{
@@ -909,11 +827,6 @@ void PhoneAPI::beginReplayTelemetry()
replayTelemetryOrder.clear();
replayTelemetryIndex = 0;
#else
if (!clientWantsGradientSync()) {
replayTelemetryOrder.clear();
replayTelemetryIndex = 0;
return;
}
replayTelemetryOrder = nodeDB->snapshotTelemetryNodeNums(nodeDB->getNodeNum());
replayTelemetryIndex = 0;
LOG_INFO("Begin telemetry replay: %u entries millis=%u", (unsigned)replayTelemetryOrder.size(), millis());
@@ -925,8 +838,6 @@ void PhoneAPI::prefetchReplayTelemetry()
#if MESHTASTIC_EXCLUDE_TELEMETRYDB
return;
#else
if (!clientWantsGradientSync())
return;
bool added = false;
bool wasEmpty = false;
{
@@ -950,10 +861,17 @@ meshtastic_MeshPacket PhoneAPI::makeReplayEnvironmentPacket(uint32_t num, const
{
meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
pkt.from = num;
pkt.to = nodeDB->getNodeNum();
pkt.to = NODENUM_BROADCAST;
pkt.id = generatePacketId();
const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
pkt.rx_time = header ? header->last_heard : 0;
pkt.channel = 0;
pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
pkt.hop_start = pkt.hop_limit;
pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
// Mark as if heard over the air, not internally generated — iOS client filters
// TRANSPORT_INTERNAL packets out of broadcast peer state updates.
pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
@@ -972,11 +890,6 @@ void PhoneAPI::beginReplayEnvironment()
replayEnvironmentOrder.clear();
replayEnvironmentIndex = 0;
#else
if (!clientWantsGradientSync()) {
replayEnvironmentOrder.clear();
replayEnvironmentIndex = 0;
return;
}
replayEnvironmentOrder = nodeDB->snapshotEnvironmentNodeNums(nodeDB->getNodeNum());
replayEnvironmentIndex = 0;
LOG_INFO("Begin environment replay: %u entries millis=%u", (unsigned)replayEnvironmentOrder.size(), millis());
@@ -988,8 +901,6 @@ void PhoneAPI::prefetchReplayEnvironment()
#if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
return;
#else
if (!clientWantsGradientSync())
return;
bool added = false;
bool wasEmpty = false;
{
@@ -1013,11 +924,17 @@ meshtastic_MeshPacket PhoneAPI::makeReplayStatusPacket(uint32_t num, const mesht
{
meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
pkt.from = num;
pkt.to = nodeDB->getNodeNum();
pkt.to = NODENUM_BROADCAST;
pkt.id = generatePacketId();
// StatusMessage has no native timestamp; use last_heard.
const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
pkt.rx_time = header ? header->last_heard : 0;
pkt.channel = 0;
pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
pkt.hop_start = pkt.hop_limit;
pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
// Mark as if heard over the air, not internally generated — client filters
pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
pkt.decoded.portnum = meshtastic_PortNum_NODE_STATUS_APP;
size_t len =
@@ -1032,11 +949,6 @@ void PhoneAPI::beginReplayStatus()
replayStatusOrder.clear();
replayStatusIndex = 0;
#else
if (!clientWantsGradientSync()) {
replayStatusOrder.clear();
replayStatusIndex = 0;
return;
}
replayStatusOrder = nodeDB->snapshotStatusNodeNums(nodeDB->getNodeNum());
replayStatusIndex = 0;
LOG_INFO("Begin status replay: %u entries millis=%u", (unsigned)replayStatusOrder.size(), millis());
@@ -1048,8 +960,6 @@ void PhoneAPI::prefetchReplayStatus()
#if MESHTASTIC_EXCLUDE_STATUSDB
return;
#else
if (!clientWantsGradientSync())
return;
bool added = false;
bool wasEmpty = false;
{
@@ -1069,6 +979,94 @@ void PhoneAPI::prefetchReplayStatus()
#endif
}
// Pop one cached satellite-DB packet from the active replay phase.
// Phases drain in order: positions -> telemetry -> environment -> status.
// When the current phase's cursor is exhausted (queue empty AND no more entries
// to snapshot), advance to the next phase. When all four phases are done,
// flip replayPhase back to IDLE and release the snapshot vectors.
//
// Returns true if a packet was placed in `out`; false if everything is drained.
bool PhoneAPI::popReplayPacket(meshtastic_MeshPacket &out)
{
while (replayPhase != REPLAY_PHASE_IDLE) {
// Prime the active phase: seed the snapshot vector on first entry,
// top up replayQueue from the snapshot up to kReplayPrefetchDepth.
switch (replayPhase) {
case REPLAY_PHASE_POSITIONS:
if (replayPositionOrder.empty() && replayPositionIndex == 0)
beginReplayPositions();
prefetchReplayPositions();
break;
case REPLAY_PHASE_TELEMETRY:
if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
beginReplayTelemetry();
prefetchReplayTelemetry();
break;
case REPLAY_PHASE_ENVIRONMENT:
if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
beginReplayEnvironment();
prefetchReplayEnvironment();
break;
case REPLAY_PHASE_STATUS:
if (replayStatusOrder.empty() && replayStatusIndex == 0)
beginReplayStatus();
prefetchReplayStatus();
break;
default:
break;
}
{
concurrency::LockGuard guard(&nodeInfoMutex);
if (!replayQueue.empty()) {
out = replayQueue.front();
replayQueue.pop_front();
return true;
}
}
// Queue empty AND no more entries to feed it — phase is exhausted.
advanceReplayPhase();
}
return false;
}
void PhoneAPI::advanceReplayPhase()
{
switch (replayPhase) {
case REPLAY_PHASE_POSITIONS:
LOG_DEBUG("Replay drain: positions done (count=%u) millis=%u", (unsigned)replayPositionIndex, millis());
replayPhase = REPLAY_PHASE_TELEMETRY;
break;
case REPLAY_PHASE_TELEMETRY:
LOG_DEBUG("Replay drain: telemetry done (count=%u) millis=%u", (unsigned)replayTelemetryIndex, millis());
replayPhase = REPLAY_PHASE_ENVIRONMENT;
break;
case REPLAY_PHASE_ENVIRONMENT:
LOG_DEBUG("Replay drain: environment done (count=%u) millis=%u", (unsigned)replayEnvironmentIndex, millis());
replayPhase = REPLAY_PHASE_STATUS;
break;
case REPLAY_PHASE_STATUS:
LOG_INFO("Replay drain complete (status count=%u) millis=%u", (unsigned)replayStatusIndex, millis());
replayPositionOrder.clear();
replayPositionOrder.shrink_to_fit();
replayTelemetryOrder.clear();
replayTelemetryOrder.shrink_to_fit();
replayEnvironmentOrder.clear();
replayEnvironmentOrder.shrink_to_fit();
replayStatusOrder.clear();
replayStatusOrder.shrink_to_fit();
replayPositionIndex = 0;
replayTelemetryIndex = 0;
replayEnvironmentIndex = 0;
replayStatusIndex = 0;
replayPhase = REPLAY_PHASE_IDLE;
break;
default:
break;
}
}
void PhoneAPI::releaseMqttClientProxyPhonePacket()
{
if (mqttClientProxyMessageForPhone) {
@@ -1115,31 +1113,6 @@ bool PhoneAPI::available()
PREFETCH_NODEINFO:
prefetchNodeInfos();
return true;
case STATE_REPLAY_POSITIONS: {
// Prime the iterator if we haven't yet, then top up the queue.
if (replayPositionOrder.empty() && replayPositionIndex == 0)
beginReplayPositions();
prefetchReplayPositions();
return true; // Always advance state machine; arm itself transitions when drained
}
case STATE_REPLAY_TELEMETRY: {
if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
beginReplayTelemetry();
prefetchReplayTelemetry();
return true;
}
case STATE_REPLAY_ENVIRONMENT: {
if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
beginReplayEnvironment();
prefetchReplayEnvironment();
return true;
}
case STATE_REPLAY_STATUS: {
if (replayStatusOrder.empty() && replayStatusIndex == 0)
beginReplayStatus();
prefetchReplayStatus();
return true;
}
case STATE_SEND_PACKETS: {
if (!queueStatusPacketForPhone)
queueStatusPacketForPhone = service->getQueueStatusForPhone();
@@ -1171,7 +1144,11 @@ bool PhoneAPI::available()
if (!packetForPhone)
packetForPhone = service->getForPhone();
hasPacket = !!packetForPhone;
return hasPacket;
if (hasPacket)
return true;
// Trailing replay drain — feeds cached satellite-DB packets alongside
// (lower priority than) live traffic.
return replayPending();
}
default:
LOG_ERROR("PhoneAPI::available unexpected state %d", state);
src/mesh/PhoneAPI.h
+21 -14
View File
@@ -26,9 +26,6 @@
#define SPECIAL_NONCE_ONLY_CONFIG 69420
#define SPECIAL_NONCE_ONLY_NODES 69421 // ( ͡° ͜ʖ ͡°)
// Gradient sync: phone sends one of these to opt into thin-header + replay.
#define SPECIAL_NONCE_GRADIENT_SYNC 69422
#define SPECIAL_NONCE_GRADIENT_ONLY_NODES 69423
/**
* Provides our protobuf based API which phone/PC clients can use to talk to our device
@@ -52,15 +49,22 @@ class PhoneAPI
STATE_SEND_CONFIG, // Replacement for the old Radioconfig
STATE_SEND_MODULECONFIG, // Send Module specific config
STATE_SEND_OTHER_NODEINFOS, // states progress in this order as the device sends to to the client
// Drain satellite DBs as synthetic POSITION_APP / TELEMETRY_APP /
// NODE_STATUS_APP packets when the phone opted into gradient sync.
STATE_REPLAY_POSITIONS,
STATE_REPLAY_TELEMETRY,
STATE_REPLAY_ENVIRONMENT,
STATE_REPLAY_STATUS,
STATE_SEND_FILEMANIFEST, // Send file manifest
STATE_SEND_COMPLETE_ID,
STATE_SEND_PACKETS // send packets or debug strings
STATE_SEND_PACKETS // live mesh packets + any cached satellite-DB replay that trails sync completion
};
// Satellite-DB replay (positions / telemetry / environment / status) used to live
// as four top-level states between STATE_SEND_OTHER_NODEINFOS and STATE_SEND_FILEMANIFEST.
// It now drains *after* config_complete_id has been emitted: the phone considers the
// initial sync done as soon as headers + manifest are delivered, and the cached
// position/telemetry/etc. trickle in alongside live mesh traffic inside STATE_SEND_PACKETS.
enum ReplayPhase : uint8_t {
REPLAY_PHASE_IDLE = 0, // not replaying (legacy clients, no-op DBs, or replay finished)
REPLAY_PHASE_POSITIONS,
REPLAY_PHASE_TELEMETRY,
REPLAY_PHASE_ENVIRONMENT,
REPLAY_PHASE_STATUS,
};
State state = STATE_SEND_NOTHING;
@@ -114,6 +118,7 @@ class PhoneAPI
size_t replayTelemetryIndex = 0;
size_t replayEnvironmentIndex = 0;
size_t replayStatusIndex = 0;
ReplayPhase replayPhase = REPLAY_PHASE_IDLE; // armed by sendConfigComplete() for full/default sync
meshtastic_ToRadio toRadioScratch = {
0}; // this is a static scratch object, any data must be copied elsewhere before returning
@@ -164,10 +169,6 @@ class PhoneAPI
bool isConnected() { return state != STATE_SEND_NOTHING; }
bool isSendingPackets() { return state == STATE_SEND_PACKETS; }
bool clientWantsGradientSync() const
{
return config_nonce == SPECIAL_NONCE_GRADIENT_SYNC || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES;
}
protected:
/// Our fromradio packet while it is being assembled
@@ -229,6 +230,12 @@ class PhoneAPI
meshtastic_MeshPacket makeReplayEnvironmentPacket(uint32_t num, const meshtastic_EnvironmentMetrics &env);
meshtastic_MeshPacket makeReplayStatusPacket(uint32_t num, const meshtastic_StatusMessage &status);
// Post-sync replay drain: pop one cached packet from the active phase, advancing
// through positions -> telemetry -> environment -> status until everything is drained.
bool popReplayPacket(meshtastic_MeshPacket &out);
void advanceReplayPhase();
bool replayPending() const { return replayPhase != REPLAY_PHASE_IDLE; }
void releaseMqttClientProxyPhonePacket();
void releaseClientNotification();
src/mesh/PositionPrecision.cpp
+75
View File
@@ -0,0 +1,75 @@
#include "PositionPrecision.h"
#include "Channels.h"
#include "mesh-pb-constants.h"
#include <Arduino.h>
uint32_t getPositionPrecisionForChannel(uint8_t channelIndex)
{
const meshtastic_Channel &channel = channels.getByIndex(channelIndex);
if (channel.settings.has_module_settings) {
return channel.settings.module_settings.position_precision;
} else if (channel.role == meshtastic_Channel_Role_PRIMARY) {
return 32;
} else {
return 0;
}
}
static int32_t truncateCoordinate(int32_t coordinate, uint32_t precision)
{
uint32_t coordinateBits = static_cast<uint32_t>(coordinate);
uint32_t truncated = coordinateBits & (UINT32_MAX << (32 - precision));
// Use the middle of the possible location, not the low edge of the bucket.
truncated += (1UL << (31 - precision));
return static_cast<int32_t>(truncated);
}
void applyPositionPrecision(meshtastic_Position &position, uint32_t precision)
{
if (precision == 0) {
uint32_t time = position.time;
position = meshtastic_Position_init_default;
position.time = time;
return;
}
uint32_t effectivePrecision = precision > 32 ? 32 : precision;
position.precision_bits = effectivePrecision;
if (effectivePrecision < 32) {
position.latitude_i = truncateCoordinate(position.latitude_i, effectivePrecision);
position.longitude_i = truncateCoordinate(position.longitude_i, effectivePrecision);
}
}
bool applyPositionPrecision(meshtastic_MeshPacket &packet, uint32_t precision)
{
if (packet.which_payload_variant != meshtastic_MeshPacket_decoded_tag ||
packet.decoded.portnum != meshtastic_PortNum_POSITION_APP) {
return true;
}
meshtastic_Position position = meshtastic_Position_init_default;
if (!pb_decode_from_bytes(packet.decoded.payload.bytes, packet.decoded.payload.size, &meshtastic_Position_msg, &position)) {
return false;
}
applyPositionPrecision(position, precision);
packet.decoded.payload.size = pb_encode_to_bytes(packet.decoded.payload.bytes, sizeof(packet.decoded.payload.bytes),
&meshtastic_Position_msg, &position);
return true;
}
bool applyPositionPrecisionForChannel(meshtastic_MeshPacket &packet, uint8_t channelIndex)
{
if (packet.which_payload_variant != meshtastic_MeshPacket_decoded_tag ||
packet.decoded.portnum != meshtastic_PortNum_POSITION_APP) {
return true;
}
return applyPositionPrecision(packet, getPositionPrecisionForChannel(channelIndex));
}
src/mesh/PositionPrecision.h
+9
View File
@@ -0,0 +1,9 @@
#pragma once
#include "meshtastic/mesh.pb.h"
#include <stdint.h>
uint32_t getPositionPrecisionForChannel(uint8_t channelIndex);
void applyPositionPrecision(meshtastic_Position &position, uint32_t precision);
bool applyPositionPrecision(meshtastic_MeshPacket &packet, uint32_t precision);
bool applyPositionPrecisionForChannel(meshtastic_MeshPacket &packet, uint8_t channelIndex);
src/mesh/RadioLibInterface.cpp
+17 -10
View File
@@ -46,16 +46,6 @@ RadioLibInterface::RadioLibInterface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE c
#endif
}
RadioLibInterface::~RadioLibInterface()
{
// If the static `instance` pointer still references us, clear it.
// A later successful init() may have replaced `instance` with a newer
// interface — don't clobber that case.
if (instance == this) {
instance = nullptr;
}
}
#ifdef ARCH_ESP32
// ESP32 doesn't use that flag
#define YIELD_FROM_ISR(x) portYIELD_FROM_ISR()
@@ -434,6 +424,9 @@ void RadioLibInterface::completeSending()
// that can take a long time
auto p = sendingPacket;
sendingPacket = NULL;
#ifdef LED_LORA
digitalWrite(LED_LORA, LED_STATE_OFF);
#endif
if (p) {
// Packet has been sent, count it toward our TX airtime utilization.
@@ -555,6 +548,9 @@ void RadioLibInterface::pollMissedIrqs()
if (isReceiving) {
checkRxDoneIrqFlag();
}
if (sendingPacket) {
checkTxDoneIrqFlag();
}
}
void RadioLibInterface::resetAGC()
@@ -570,6 +566,14 @@ void RadioLibInterface::checkRxDoneIrqFlag()
}
}
void RadioLibInterface::checkTxDoneIrqFlag()
{
if (iface->checkIrq(RADIOLIB_IRQ_TX_DONE)) {
LOG_WARN("caught missed TX_DONE");
notify(ISR_TX, true);
}
}
void RadioLibInterface::configHardwareForSend()
{
powerMon->setState(meshtastic_PowerMon_State_Lora_TXOn);
@@ -611,6 +615,9 @@ bool RadioLibInterface::startSend(meshtastic_MeshPacket *txp)
enableInterrupt(isrTxLevel0);
lastTxStart = millis();
printPacket("Started Tx", txp);
#ifdef LED_LORA
digitalWrite(LED_LORA, LED_STATE_ON);
#endif
}
return res == RADIOLIB_ERR_NONE;
src/mesh/RadioLibInterface.h
+8 -7
View File
@@ -104,6 +104,13 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
*/
static RadioLibInterface *instance;
/** Clear instance on destruction so stale pointer checks in loop() are safe */
virtual ~RadioLibInterface()
{
if (instance == this)
instance = nullptr;
}
/**
* Glue functions called from ISR land
*/
@@ -136,13 +143,6 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
RadioLibInterface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
RADIOLIB_PIN_TYPE busy, PhysicalLayer *iface = NULL);
/**
* Clear the static `instance` pointer if it still points at us, so callers
* that check `RadioLibInterface::instance != nullptr` don't dereference a
* freed object after a failed init() + unique_ptr reset.
*/
virtual ~RadioLibInterface();
virtual ErrorCode send(meshtastic_MeshPacket *p) override;
/**
@@ -293,4 +293,5 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
bool removePendingTXPacket(NodeNum from, PacketId id, uint32_t hop_limit_lt) override;
void checkRxDoneIrqFlag();
void checkTxDoneIrqFlag();
};
src/mesh/Router.cpp
+6
View File
@@ -4,6 +4,7 @@
#include "MeshRadio.h"
#include "MeshService.h"
#include "NodeDB.h"
#include "PositionPrecision.h"
#include "RTC.h"
#include "configuration.h"
@@ -367,6 +368,11 @@ ErrorCode Router::send(meshtastic_MeshPacket *p)
}
fixPriority(p); // Before encryption, fix the priority if it's unset
if (!applyPositionPrecisionForChannel(*p, p->channel)) {
LOG_ERROR("Dropping malformed position packet before send");
packetPool.release(p);
return meshtastic_Routing_Error_BAD_REQUEST;
}
// If the packet is not yet encrypted, do so now
if (p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) {
src/mesh/TypeConversions.cpp
+7
View File
@@ -37,6 +37,7 @@ meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfo
info.position.altitude = position->altitude;
info.position.location_source = position->location_source;
info.position.time = position->time;
info.position.precision_bits = position->precision_bits;
}
if (nodeInfoLiteHasUser(lite)) {
info.has_user = true;
@@ -71,6 +72,7 @@ meshtastic_PositionLite TypeConversions::ConvertToPositionLite(meshtastic_Positi
lite.altitude = position.altitude;
lite.location_source = position.location_source;
lite.time = position.time;
lite.precision_bits = position.precision_bits;
return lite;
}
@@ -89,6 +91,11 @@ meshtastic_Position TypeConversions::ConvertToPosition(meshtastic_PositionLite l
position.altitude = lite.altitude;
position.location_source = lite.location_source;
position.time = lite.time;
// Preserve the peer's broadcast precision; falls back to 0 for entries cached
// before the precision_bits field existed in PositionLite (pre-migration data).
// iOS treats 0 as "unspecified precision" and won't render the pin — so for
// unset values, declare full precision so the stored lat/lon renders as a point.
position.precision_bits = lite.precision_bits == 0 ? 32 : lite.precision_bits;
return position;
}
src/mesh/api/ethServerAPI.cpp
+1 -1
View File
@@ -1,7 +1,7 @@
#include "configuration.h"
#include <Arduino.h>
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
#include "ethServerAPI.h"
src/mesh/api/ethServerAPI.h
+1 -1
View File
@@ -1,7 +1,7 @@
#pragma once
#include "ServerAPI.h"
#ifndef USE_WS5500
#if !defined(USE_WS5500) && !defined(USE_CH390D)
#include <RAK13800_W5100S.h>
/**
src/mesh/eth/ethClient.cpp
+1 -1
View File
@@ -9,7 +9,7 @@
#include <RAK13800_W5100S.h>
#include <SPI.h>
#if HAS_NETWORKING
#if HAS_NETWORKING && !defined(USE_WS5500) && !defined(USE_CH390D)
#ifndef DISABLE_NTP
#include <NTPClient.h>
src/mesh/generated/meshtastic/admin.pb.cpp
+3
View File
@@ -15,6 +15,9 @@ PB_BIND(meshtastic_AdminMessage_InputEvent, meshtastic_AdminMessage_InputEvent,
PB_BIND(meshtastic_AdminMessage_OTAEvent, meshtastic_AdminMessage_OTAEvent, AUTO)
PB_BIND(meshtastic_LockdownAuth, meshtastic_LockdownAuth, AUTO)
PB_BIND(meshtastic_HamParameters, meshtastic_HamParameters, AUTO)
src/mesh/generated/meshtastic/admin.pb.h
+103 -1
View File
@@ -130,6 +130,76 @@ typedef struct _meshtastic_AdminMessage_OTAEvent {
meshtastic_AdminMessage_OTAEvent_ota_hash_t ota_hash;
} meshtastic_AdminMessage_OTAEvent;
typedef PB_BYTES_ARRAY_T(32) meshtastic_LockdownAuth_passphrase_t;
/* Lockdown passphrase delivery payload.
One message handles three operations distinguished by content:
- Provision (first-time): passphrase set, lock_now=false. Firmware
generates DEK, wraps with passphrase-derived KEK, persists.
- Unlock: passphrase set, lock_now=false. Firmware verifies
passphrase against stored DEK, unlocks storage, authorizes the
connection that delivered this packet.
- Lock now: lock_now=true, passphrase ignored. Firmware revokes
all client auth and reboots into the locked state.
Firmware decides between provision and unlock based on its own state
(whether a DEK file already exists). Clients do not need to track
which case applies. */
typedef struct _meshtastic_LockdownAuth {
/* Passphrase bytes (1-32). Empty when lock_now is true.
Capped to 32 to match the proto cap on related security fields. */
meshtastic_LockdownAuth_passphrase_t passphrase;
/* Optional override of the boot-count token TTL granted on success.
0 = use firmware default (TOKEN_DEFAULT_BOOTS).
On reboot the firmware decrements this; when it reaches 0 the
device boots fully locked and requires a fresh passphrase. */
uint32_t boots_remaining;
/* Optional wall-clock expiry for the unlock token, as absolute
Unix-epoch seconds. 0 = no time limit (only the boot-count TTL
applies). On boot, if the device RTC is set and now > this value,
the token is treated as expired. */
uint32_t valid_until_epoch;
/* If true, ignore passphrase fields, immediately revoke all
connection-level admin authorization, and reboot the device into
the locked state. Always honoured regardless of current lock state. */
bool lock_now;
/* Optional per-boot uptime cap on the unlocked session, in seconds.
0 = unlimited (token-only enforcement, suitable for unattended
tower / infrastructure nodes).
When non-zero, the firmware arms an uptime timer at unlock. On
each expiry, while there is still boot-count budget, the firmware
decrements the on-flash boot count in place, revokes per-
connection admin auth (clients must re-authenticate to see
content), re-engages the screen lock, and re-arms the timer
without rebooting. Mesh routing keeps running across session
boundaries; only when the boot-count budget reaches zero does
the device hard-lock and reboot.
Total exposure ceiling = ((resolved boot count) + 1) * max_session_seconds.
The +1 accounts for the initial passphrase-unlocked session
itself, since boots_remaining is the number of subsequent
session rolls (each consuming one boot from the rollback ledger).
The resolved boot count is the value the firmware writes into the
token at unlock time: the client-supplied boots_remaining when
non-zero, otherwise the firmware default (TOKEN_DEFAULT_BOOTS).
Note that boots_remaining == 0 in this message means "use firmware
default", NOT "zero boots" — a client computing the ceiling for
display should mirror that resolution rather than multiplying the
raw request value.
The cap is persisted in the token, so it survives token-based
auto-unlock across reboots. Explicit operator Lock Now still
deletes the token and forces passphrase re-entry.
Uses millis() (CPU uptime), not wall-clock time, so the cap is
immune to GPS spoofing, RTC backup-battery removal, and Faraday
cage isolation — none of those move the uptime counter. The only
way to reset the session clock is a reboot, which costs a boot
from the on-flash, HMAC-bound counter. */
uint32_t max_session_seconds;
} meshtastic_LockdownAuth;
/* Parameters for setting up Meshtastic for ameteur radio usage */
typedef struct _meshtastic_HamParameters {
/* Amateur radio call sign, eg. KD2ABC */
@@ -384,6 +454,15 @@ typedef struct _meshtastic_AdminMessage {
meshtastic_AdminMessage_OTAEvent ota_request;
/* Parameters and sensor configuration */
meshtastic_SensorConfig sensor_config;
/* Lockdown passphrase delivery / unlock / lock-now command for hardened
firmware builds (see MESHTASTIC_LOCKDOWN). Used to provision the
passphrase on first boot, unlock encrypted storage on subsequent
reboots, re-verify on already-unlocked devices to authorize a new
client connection, or immediately re-lock the device.
Replaces the earlier scheme that repurposed SecurityConfig.private_key
to carry passphrase bytes; that hack is retired. */
meshtastic_LockdownAuth lockdown_auth;
};
/* The node generates this key and sends it with any get_x_response packets.
The client MUST include the same key with any set_x commands. Key expires after 300 seconds.
@@ -429,6 +508,7 @@ extern "C" {
#define meshtastic_KeyVerificationAdmin_message_type_ENUMTYPE meshtastic_KeyVerificationAdmin_MessageType
@@ -441,6 +521,7 @@ extern "C" {
#define meshtastic_AdminMessage_init_default {0, {0}, {0, {0}}}
#define meshtastic_AdminMessage_InputEvent_init_default {0, 0, 0, 0}
#define meshtastic_AdminMessage_OTAEvent_init_default {_meshtastic_OTAMode_MIN, {0, {0}}}
#define meshtastic_LockdownAuth_init_default {{0, {0}}, 0, 0, 0, 0}
#define meshtastic_HamParameters_init_default {"", 0, 0, ""}
#define meshtastic_NodeRemoteHardwarePinsResponse_init_default {0, {meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default}}
#define meshtastic_SharedContact_init_default {0, false, meshtastic_User_init_default, 0, 0}
@@ -453,6 +534,7 @@ extern "C" {
#define meshtastic_AdminMessage_init_zero {0, {0}, {0, {0}}}
#define meshtastic_AdminMessage_InputEvent_init_zero {0, 0, 0, 0}
#define meshtastic_AdminMessage_OTAEvent_init_zero {_meshtastic_OTAMode_MIN, {0, {0}}}
#define meshtastic_LockdownAuth_init_zero {{0, {0}}, 0, 0, 0, 0}
#define meshtastic_HamParameters_init_zero {"", 0, 0, ""}
#define meshtastic_NodeRemoteHardwarePinsResponse_init_zero {0, {meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero}}
#define meshtastic_SharedContact_init_zero {0, false, meshtastic_User_init_zero, 0, 0}
@@ -470,6 +552,11 @@ extern "C" {
#define meshtastic_AdminMessage_InputEvent_touch_y_tag 4
#define meshtastic_AdminMessage_OTAEvent_reboot_ota_mode_tag 1
#define meshtastic_AdminMessage_OTAEvent_ota_hash_tag 2
#define meshtastic_LockdownAuth_passphrase_tag 1
#define meshtastic_LockdownAuth_boots_remaining_tag 2
#define meshtastic_LockdownAuth_valid_until_epoch_tag 3
#define meshtastic_LockdownAuth_lock_now_tag 4
#define meshtastic_LockdownAuth_max_session_seconds_tag 5
#define meshtastic_HamParameters_call_sign_tag 1
#define meshtastic_HamParameters_tx_power_tag 2
#define meshtastic_HamParameters_frequency_tag 3
@@ -560,6 +647,7 @@ extern "C" {
#define meshtastic_AdminMessage_nodedb_reset_tag 100
#define meshtastic_AdminMessage_ota_request_tag 102
#define meshtastic_AdminMessage_sensor_config_tag 103
#define meshtastic_AdminMessage_lockdown_auth_tag 104
#define meshtastic_AdminMessage_session_passkey_tag 101
/* Struct field encoding specification for nanopb */
@@ -621,7 +709,8 @@ X(a, STATIC, ONEOF, INT32, (payload_variant,factory_reset_config,factory
X(a, STATIC, ONEOF, BOOL, (payload_variant,nodedb_reset,nodedb_reset), 100) \
X(a, STATIC, SINGULAR, BYTES, session_passkey, 101) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,ota_request,ota_request), 102) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,sensor_config,sensor_config), 103)
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,sensor_config,sensor_config), 103) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,lockdown_auth,lockdown_auth), 104)
#define meshtastic_AdminMessage_CALLBACK NULL
#define meshtastic_AdminMessage_DEFAULT NULL
#define meshtastic_AdminMessage_payload_variant_get_channel_response_MSGTYPE meshtastic_Channel
@@ -644,6 +733,7 @@ X(a, STATIC, ONEOF, MESSAGE, (payload_variant,sensor_config,sensor_config)
#define meshtastic_AdminMessage_payload_variant_key_verification_MSGTYPE meshtastic_KeyVerificationAdmin
#define meshtastic_AdminMessage_payload_variant_ota_request_MSGTYPE meshtastic_AdminMessage_OTAEvent
#define meshtastic_AdminMessage_payload_variant_sensor_config_MSGTYPE meshtastic_SensorConfig
#define meshtastic_AdminMessage_payload_variant_lockdown_auth_MSGTYPE meshtastic_LockdownAuth
#define meshtastic_AdminMessage_InputEvent_FIELDLIST(X, a) \
X(a, STATIC, SINGULAR, UINT32, event_code, 1) \
@@ -659,6 +749,15 @@ X(a, STATIC, SINGULAR, BYTES, ota_hash, 2)
#define meshtastic_AdminMessage_OTAEvent_CALLBACK NULL
#define meshtastic_AdminMessage_OTAEvent_DEFAULT NULL
#define meshtastic_LockdownAuth_FIELDLIST(X, a) \
X(a, STATIC, SINGULAR, BYTES, passphrase, 1) \
X(a, STATIC, SINGULAR, UINT32, boots_remaining, 2) \
X(a, STATIC, SINGULAR, UINT32, valid_until_epoch, 3) \
X(a, STATIC, SINGULAR, BOOL, lock_now, 4) \
X(a, STATIC, SINGULAR, UINT32, max_session_seconds, 5)
#define meshtastic_LockdownAuth_CALLBACK NULL
#define meshtastic_LockdownAuth_DEFAULT NULL
#define meshtastic_HamParameters_FIELDLIST(X, a) \
X(a, STATIC, SINGULAR, STRING, call_sign, 1) \
X(a, STATIC, SINGULAR, INT32, tx_power, 2) \
@@ -737,6 +836,7 @@ X(a, STATIC, OPTIONAL, UINT32, set_accuracy, 1)
extern const pb_msgdesc_t meshtastic_AdminMessage_msg;
extern const pb_msgdesc_t meshtastic_AdminMessage_InputEvent_msg;
extern const pb_msgdesc_t meshtastic_AdminMessage_OTAEvent_msg;
extern const pb_msgdesc_t meshtastic_LockdownAuth_msg;
extern const pb_msgdesc_t meshtastic_HamParameters_msg;
extern const pb_msgdesc_t meshtastic_NodeRemoteHardwarePinsResponse_msg;
extern const pb_msgdesc_t meshtastic_SharedContact_msg;
@@ -751,6 +851,7 @@ extern const pb_msgdesc_t meshtastic_SHTXX_config_msg;
#define meshtastic_AdminMessage_fields &meshtastic_AdminMessage_msg
#define meshtastic_AdminMessage_InputEvent_fields &meshtastic_AdminMessage_InputEvent_msg
#define meshtastic_AdminMessage_OTAEvent_fields &meshtastic_AdminMessage_OTAEvent_msg
#define meshtastic_LockdownAuth_fields &meshtastic_LockdownAuth_msg
#define meshtastic_HamParameters_fields &meshtastic_HamParameters_msg
#define meshtastic_NodeRemoteHardwarePinsResponse_fields &meshtastic_NodeRemoteHardwarePinsResponse_msg
#define meshtastic_SharedContact_fields &meshtastic_SharedContact_msg
@@ -768,6 +869,7 @@ extern const pb_msgdesc_t meshtastic_SHTXX_config_msg;
#define meshtastic_AdminMessage_size 511
#define meshtastic_HamParameters_size 31
#define meshtastic_KeyVerificationAdmin_size 25
#define meshtastic_LockdownAuth_size 54
#define meshtastic_NodeRemoteHardwarePinsResponse_size 496
#define meshtastic_SCD30_config_size 27
#define meshtastic_SCD4X_config_size 29
src/mesh/generated/meshtastic/deviceonly.pb.h
+10 -6
View File
@@ -33,6 +33,8 @@ typedef struct _meshtastic_PositionLite {
uint32_t time;
/* TODO: REPLACE */
meshtastic_Position_LocSource location_source;
/* Indicates the bits of precision set by the sending node */
uint32_t precision_bits;
} meshtastic_PositionLite;
typedef PB_BYTES_ARRAY_T(32) meshtastic_UserLite_public_key_t;
@@ -211,7 +213,7 @@ extern "C" {
#endif
/* Initializer values for message structs */
#define meshtastic_PositionLite_init_default {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN}
#define meshtastic_PositionLite_init_default {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN, 0}
#define meshtastic_UserLite_init_default {{0}, "", "", _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
#define meshtastic_NodeInfoLite_init_default {0, 0, 0, 0, false, 0, 0, 0, "", "", _meshtastic_HardwareModel_MIN, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}}
#define meshtastic_DeviceState_init_default {false, meshtastic_MyNodeInfo_init_default, false, meshtastic_User_init_default, 0, {meshtastic_MeshPacket_init_default}, false, meshtastic_MeshPacket_init_default, 0, 0, 0, false, meshtastic_MeshPacket_init_default, 0, {meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default, meshtastic_NodeRemoteHardwarePin_init_default}}
@@ -222,7 +224,7 @@ extern "C" {
#define meshtastic_NodeDatabase_init_default {0, {0}, {0}, {0}, {0}, {0}}
#define meshtastic_ChannelFile_init_default {0, {meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default, meshtastic_Channel_init_default}, 0}
#define meshtastic_BackupPreferences_init_default {0, 0, false, meshtastic_LocalConfig_init_default, false, meshtastic_LocalModuleConfig_init_default, false, meshtastic_ChannelFile_init_default, false, meshtastic_User_init_default}
#define meshtastic_PositionLite_init_zero {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN}
#define meshtastic_PositionLite_init_zero {0, 0, 0, 0, _meshtastic_Position_LocSource_MIN, 0}
#define meshtastic_UserLite_init_zero {{0}, "", "", _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
#define meshtastic_NodeInfoLite_init_zero {0, 0, 0, 0, false, 0, 0, 0, "", "", _meshtastic_HardwareModel_MIN, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}}
#define meshtastic_DeviceState_init_zero {false, meshtastic_MyNodeInfo_init_zero, false, meshtastic_User_init_zero, 0, {meshtastic_MeshPacket_init_zero}, false, meshtastic_MeshPacket_init_zero, 0, 0, 0, false, meshtastic_MeshPacket_init_zero, 0, {meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero, meshtastic_NodeRemoteHardwarePin_init_zero}}
@@ -240,6 +242,7 @@ extern "C" {
#define meshtastic_PositionLite_altitude_tag 3
#define meshtastic_PositionLite_time_tag 4
#define meshtastic_PositionLite_location_source_tag 5
#define meshtastic_PositionLite_precision_bits_tag 6
#define meshtastic_UserLite_macaddr_tag 1
#define meshtastic_UserLite_long_name_tag 2
#define meshtastic_UserLite_short_name_tag 3
@@ -298,7 +301,8 @@ X(a, STATIC, SINGULAR, SFIXED32, latitude_i, 1) \
X(a, STATIC, SINGULAR, SFIXED32, longitude_i, 2) \
X(a, STATIC, SINGULAR, INT32, altitude, 3) \
X(a, STATIC, SINGULAR, FIXED32, time, 4) \
X(a, STATIC, SINGULAR, UENUM, location_source, 5)
X(a, STATIC, SINGULAR, UENUM, location_source, 5) \
X(a, STATIC, SINGULAR, UINT32, precision_bits, 6)
#define meshtastic_PositionLite_CALLBACK NULL
#define meshtastic_PositionLite_DEFAULT NULL
@@ -444,13 +448,13 @@ extern const pb_msgdesc_t meshtastic_BackupPreferences_msg;
#define MESHTASTIC_MESHTASTIC_DEVICEONLY_PB_H_MAX_SIZE meshtastic_BackupPreferences_size
#define meshtastic_BackupPreferences_size 2432
#define meshtastic_ChannelFile_size 718
#define meshtastic_DeviceState_size 1737
#define meshtastic_DeviceState_size 1944
#define meshtastic_NodeEnvironmentEntry_size 170
#define meshtastic_NodeInfoLite_size 105
#define meshtastic_NodePositionEntry_size 36
#define meshtastic_NodePositionEntry_size 42
#define meshtastic_NodeStatusEntry_size 89
#define meshtastic_NodeTelemetryEntry_size 35
#define meshtastic_PositionLite_size 28
#define meshtastic_PositionLite_size 34
#define meshtastic_UserLite_size 98
#ifdef __cplusplus
src/mesh/generated/meshtastic/deviceonly_legacy.pb.h
+1 -1
View File
@@ -115,7 +115,7 @@ extern const pb_msgdesc_t meshtastic_NodeDatabase_Legacy_msg;
/* Maximum encoded size of messages (where known) */
/* meshtastic_NodeDatabase_Legacy_size depends on runtime parameters */
#define MESHTASTIC_MESHTASTIC_DEVICEONLY_LEGACY_PB_H_MAX_SIZE meshtastic_NodeInfoLite_Legacy_size
#define meshtastic_NodeInfoLite_Legacy_size 196
#define meshtastic_NodeInfoLite_Legacy_size 202
#ifdef __cplusplus
} /* extern "C" */
src/mesh/generated/meshtastic/mesh.pb.cpp
+5
View File
@@ -57,6 +57,9 @@ PB_BIND(meshtastic_QueueStatus, meshtastic_QueueStatus, AUTO)
PB_BIND(meshtastic_FromRadio, meshtastic_FromRadio, 2)
PB_BIND(meshtastic_LockdownStatus, meshtastic_LockdownStatus, AUTO)
PB_BIND(meshtastic_ClientNotification, meshtastic_ClientNotification, 2)
@@ -134,6 +137,8 @@ PB_BIND(meshtastic_ChunkedPayloadResponse, meshtastic_ChunkedPayloadResponse, AU
src/mesh/generated/meshtastic/mesh.pb.h
+117 -13
View File
@@ -317,6 +317,10 @@ typedef enum _meshtastic_HardwareModel {
meshtastic_HardwareModel_THINKNODE_M9 = 131,
/* The Heltec-V4-R8 uses an ESP32S3R8 chip, plus an SX1262. */
meshtastic_HardwareModel_HELTEC_V4_R8 = 132,
/* The HELTEC_MESH_NODE_T1 uses an NRF52840 chip, plus an SX1262. */
meshtastic_HardwareModel_HELTEC_MESH_NODE_T1 = 133,
/* B&Q Consulting Station G3: TBD */
meshtastic_HardwareModel_STATION_G3 = 134,
/* ------------------------------------------------------------------------------------------------------------------------------------------
Reserved ID For developing private Ports. These will show up in live traffic sparsely, so we can use a high number. Keep it within 8 bits.
------------------------------------------------------------------------------------------------------------------------------------------ */
@@ -634,6 +638,25 @@ typedef enum _meshtastic_LogRecord_Level {
meshtastic_LogRecord_Level_TRACE = 5
} meshtastic_LogRecord_Level;
typedef enum _meshtastic_LockdownStatus_State {
/* Default; should not be sent. */
meshtastic_LockdownStatus_State_STATE_UNSPECIFIED = 0,
/* No passphrase has ever been provisioned on this device.
Client should prompt the operator to set one. */
meshtastic_LockdownStatus_State_NEEDS_PROVISION = 1,
/* Storage is locked or this client has not authenticated yet.
lock_reason carries a machine-readable detail string.
Client should present (or auto-replay) a passphrase via
AdminMessage.lockdown_auth. */
meshtastic_LockdownStatus_State_LOCKED = 2,
/* Passphrase accepted; client is now authorized for this connection.
boots_remaining and valid_until_epoch describe the active session
token's TTL. */
meshtastic_LockdownStatus_State_UNLOCKED = 3,
/* Passphrase rejected. backoff_seconds is non-zero when rate-limited. */
meshtastic_LockdownStatus_State_UNLOCK_FAILED = 4
} meshtastic_LockdownStatus_State;
/* Struct definitions */
/* A GPS Position */
typedef struct _meshtastic_Position {
@@ -797,6 +820,7 @@ typedef struct _meshtastic_Routing {
} meshtastic_Routing;
typedef PB_BYTES_ARRAY_T(233) meshtastic_Data_payload_t;
typedef PB_BYTES_ARRAY_T(64) meshtastic_Data_xeddsa_signature_t;
/* (Formerly called SubPacket)
The payload portion fo a packet, this is the actual bytes that are sent
inside a radio packet (because from/to are broken out by the comms library) */
@@ -830,6 +854,8 @@ typedef struct _meshtastic_Data {
/* Bitfield for extra flags. First use is to indicate that user approves the packet being uploaded to MQTT. */
bool has_bitfield;
uint8_t bitfield;
/* XEdDSA signature for the payload */
meshtastic_Data_xeddsa_signature_t xeddsa_signature;
} meshtastic_Data;
typedef PB_BYTES_ARRAY_T(32) meshtastic_KeyVerification_hash1_t;
@@ -1034,6 +1060,8 @@ typedef struct _meshtastic_MeshPacket {
uint32_t tx_after;
/* Indicates which transport mechanism this packet arrived over */
meshtastic_MeshPacket_TransportMechanism transport_mechanism;
/* Indicates whether the packet has a valid signature */
bool xeddsa_signed;
} meshtastic_MeshPacket;
/* The bluetooth to device link:
@@ -1090,6 +1118,10 @@ typedef struct _meshtastic_NodeInfo {
/* True if node has been muted
Persistes between NodeDB internal clean ups */
bool is_muted;
/* True if node is signing its packets via XEdDSA
Persists between NodeDB internal clean ups
LSB 1 of the bitfield */
bool has_xeddsa_signed;
} meshtastic_NodeInfo;
typedef PB_BYTES_ARRAY_T(16) meshtastic_MyNodeInfo_device_id_t;
@@ -1144,6 +1176,38 @@ typedef struct _meshtastic_QueueStatus {
uint32_t mesh_packet_id;
} meshtastic_QueueStatus;
/* Lockdown state report from firmware to client (for hardened builds
with MESHTASTIC_LOCKDOWN). Sent immediately after config_complete_id
to inform a freshly-connected unauthorized client what it must do,
and again in response to each LockdownAuth admin command. */
typedef struct _meshtastic_LockdownStatus {
/* Current lockdown state being reported. */
meshtastic_LockdownStatus_State state;
/* For LOCKED: machine-readable reason. Known values:
"needs_auth" — storage already unlocked, client must auth
"token_missing" — no boot token on flash
"token_expired" — boot token wall-clock TTL elapsed
"token_boots_zero" — boot token boot-count TTL exhausted
"token_hmac_fail" — token tampered or wrong device
"token_dek_fail" — token DEK decrypt failed
"token_wrong_size" — token file corrupted
"token_bad_magic" — token file corrupted
"not_provisioned" — should generally use NEEDS_PROVISION state instead
Other values may be added; clients should treat unknown values as
"locked, ask for passphrase". */
char lock_reason[32];
/* For UNLOCKED: remaining boots on the issued session token.
Decrements by 1 on each subsequent boot. */
uint32_t boots_remaining;
/* For UNLOCKED: wall-clock expiry of the issued session token,
absolute Unix-epoch seconds. 0 = no time limit. */
uint32_t valid_until_epoch;
/* For UNLOCK_FAILED: seconds the client must wait before another
passphrase attempt will be accepted. 0 = wrong passphrase, no
backoff (immediate retry allowed but advisable to prompt user). */
uint32_t backoff_seconds;
} meshtastic_LockdownStatus;
typedef struct _meshtastic_KeyVerificationNumberInform {
uint64_t nonce;
char remote_longname[40];
@@ -1317,6 +1381,12 @@ typedef struct _meshtastic_FromRadio {
meshtastic_ClientNotification clientNotification;
/* Persistent data for device-ui */
meshtastic_DeviceUIConfig deviceuiConfig;
/* Lockdown state notification for hardened firmware builds.
Sent post-config (so unauthorized clients learn they must
provision/unlock) and after each LockdownAuth admin command
to report success or failure. Replaces the earlier scheme of
encoding state as magic-string prefixes inside ClientNotification. */
meshtastic_LockdownStatus lockdown_status;
};
} meshtastic_FromRadio;
@@ -1458,6 +1528,10 @@ extern "C" {
#define _meshtastic_LogRecord_Level_MAX meshtastic_LogRecord_Level_CRITICAL
#define _meshtastic_LogRecord_Level_ARRAYSIZE ((meshtastic_LogRecord_Level)(meshtastic_LogRecord_Level_CRITICAL+1))
#define _meshtastic_LockdownStatus_State_MIN meshtastic_LockdownStatus_State_STATE_UNSPECIFIED
#define _meshtastic_LockdownStatus_State_MAX meshtastic_LockdownStatus_State_UNLOCK_FAILED
#define _meshtastic_LockdownStatus_State_ARRAYSIZE ((meshtastic_LockdownStatus_State)(meshtastic_LockdownStatus_State_UNLOCK_FAILED+1))
#define meshtastic_Position_location_source_ENUMTYPE meshtastic_Position_LocSource
#define meshtastic_Position_altitude_source_ENUMTYPE meshtastic_Position_AltSource
@@ -1488,6 +1562,8 @@ extern "C" {
#define meshtastic_LockdownStatus_state_ENUMTYPE meshtastic_LockdownStatus_State
#define meshtastic_ClientNotification_level_ENUMTYPE meshtastic_LogRecord_Level
@@ -1515,19 +1591,20 @@ extern "C" {
#define meshtastic_User_init_default {"", "", "", {0}, _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
#define meshtastic_RouteDiscovery_init_default {0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}}
#define meshtastic_Routing_init_default {0, {meshtastic_RouteDiscovery_init_default}}
#define meshtastic_Data_init_default {_meshtastic_PortNum_MIN, {0, {0}}, 0, 0, 0, 0, 0, 0, false, 0}
#define meshtastic_Data_init_default {_meshtastic_PortNum_MIN, {0, {0}}, 0, 0, 0, 0, 0, 0, false, 0, {0, {0}}}
#define meshtastic_KeyVerification_init_default {0, {0, {0}}, {0, {0}}}
#define meshtastic_StoreForwardPlusPlus_init_default {_meshtastic_StoreForwardPlusPlus_SFPP_message_type_MIN, {0, {0}}, {0, {0}}, {0, {0}}, {0, {0}}, 0, 0, 0, 0, 0}
#define meshtastic_RemoteShell_init_default {_meshtastic_RemoteShell_OpCode_MIN, 0, 0, 0, {0, {0}}, 0, 0, 0, 0, 0}
#define meshtastic_Waypoint_init_default {0, false, 0, false, 0, 0, 0, "", "", 0}
#define meshtastic_StatusMessage_init_default {""}
#define meshtastic_MqttClientProxyMessage_init_default {"", 0, {{0, {0}}}, 0}
#define meshtastic_MeshPacket_init_default {0, 0, 0, 0, {meshtastic_Data_init_default}, 0, 0, 0, 0, 0, _meshtastic_MeshPacket_Priority_MIN, 0, _meshtastic_MeshPacket_Delayed_MIN, 0, 0, {0, {0}}, 0, 0, 0, 0, _meshtastic_MeshPacket_TransportMechanism_MIN}
#define meshtastic_NodeInfo_init_default {0, false, meshtastic_User_init_default, false, meshtastic_Position_init_default, 0, 0, false, meshtastic_DeviceMetrics_init_default, 0, 0, false, 0, 0, 0, 0, 0}
#define meshtastic_MeshPacket_init_default {0, 0, 0, 0, {meshtastic_Data_init_default}, 0, 0, 0, 0, 0, _meshtastic_MeshPacket_Priority_MIN, 0, _meshtastic_MeshPacket_Delayed_MIN, 0, 0, {0, {0}}, 0, 0, 0, 0, _meshtastic_MeshPacket_TransportMechanism_MIN, 0}
#define meshtastic_NodeInfo_init_default {0, false, meshtastic_User_init_default, false, meshtastic_Position_init_default, 0, 0, false, meshtastic_DeviceMetrics_init_default, 0, 0, false, 0, 0, 0, 0, 0, 0}
#define meshtastic_MyNodeInfo_init_default {0, 0, 0, {0, {0}}, "", _meshtastic_FirmwareEdition_MIN, 0}
#define meshtastic_LogRecord_init_default {"", 0, "", _meshtastic_LogRecord_Level_MIN}
#define meshtastic_QueueStatus_init_default {0, 0, 0, 0}
#define meshtastic_FromRadio_init_default {0, 0, {meshtastic_MeshPacket_init_default}}
#define meshtastic_LockdownStatus_init_default {_meshtastic_LockdownStatus_State_MIN, "", 0, 0, 0}
#define meshtastic_ClientNotification_init_default {false, 0, 0, _meshtastic_LogRecord_Level_MIN, "", 0, {meshtastic_KeyVerificationNumberInform_init_default}}
#define meshtastic_KeyVerificationNumberInform_init_default {0, "", 0}
#define meshtastic_KeyVerificationNumberRequest_init_default {0, ""}
@@ -1549,19 +1626,20 @@ extern "C" {
#define meshtastic_User_init_zero {"", "", "", {0}, _meshtastic_HardwareModel_MIN, 0, _meshtastic_Config_DeviceConfig_Role_MIN, {0, {0}}, false, 0}
#define meshtastic_RouteDiscovery_init_zero {0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}, 0, {0, 0, 0, 0, 0, 0, 0, 0}}
#define meshtastic_Routing_init_zero {0, {meshtastic_RouteDiscovery_init_zero}}
#define meshtastic_Data_init_zero {_meshtastic_PortNum_MIN, {0, {0}}, 0, 0, 0, 0, 0, 0, false, 0}
#define meshtastic_Data_init_zero {_meshtastic_PortNum_MIN, {0, {0}}, 0, 0, 0, 0, 0, 0, false, 0, {0, {0}}}
#define meshtastic_KeyVerification_init_zero {0, {0, {0}}, {0, {0}}}
#define meshtastic_StoreForwardPlusPlus_init_zero {_meshtastic_StoreForwardPlusPlus_SFPP_message_type_MIN, {0, {0}}, {0, {0}}, {0, {0}}, {0, {0}}, 0, 0, 0, 0, 0}
#define meshtastic_RemoteShell_init_zero {_meshtastic_RemoteShell_OpCode_MIN, 0, 0, 0, {0, {0}}, 0, 0, 0, 0, 0}
#define meshtastic_Waypoint_init_zero {0, false, 0, false, 0, 0, 0, "", "", 0}
#define meshtastic_StatusMessage_init_zero {""}
#define meshtastic_MqttClientProxyMessage_init_zero {"", 0, {{0, {0}}}, 0}
#define meshtastic_MeshPacket_init_zero {0, 0, 0, 0, {meshtastic_Data_init_zero}, 0, 0, 0, 0, 0, _meshtastic_MeshPacket_Priority_MIN, 0, _meshtastic_MeshPacket_Delayed_MIN, 0, 0, {0, {0}}, 0, 0, 0, 0, _meshtastic_MeshPacket_TransportMechanism_MIN}
#define meshtastic_NodeInfo_init_zero {0, false, meshtastic_User_init_zero, false, meshtastic_Position_init_zero, 0, 0, false, meshtastic_DeviceMetrics_init_zero, 0, 0, false, 0, 0, 0, 0, 0}
#define meshtastic_MeshPacket_init_zero {0, 0, 0, 0, {meshtastic_Data_init_zero}, 0, 0, 0, 0, 0, _meshtastic_MeshPacket_Priority_MIN, 0, _meshtastic_MeshPacket_Delayed_MIN, 0, 0, {0, {0}}, 0, 0, 0, 0, _meshtastic_MeshPacket_TransportMechanism_MIN, 0}
#define meshtastic_NodeInfo_init_zero {0, false, meshtastic_User_init_zero, false, meshtastic_Position_init_zero, 0, 0, false, meshtastic_DeviceMetrics_init_zero, 0, 0, false, 0, 0, 0, 0, 0, 0}
#define meshtastic_MyNodeInfo_init_zero {0, 0, 0, {0, {0}}, "", _meshtastic_FirmwareEdition_MIN, 0}
#define meshtastic_LogRecord_init_zero {"", 0, "", _meshtastic_LogRecord_Level_MIN}
#define meshtastic_QueueStatus_init_zero {0, 0, 0, 0}
#define meshtastic_FromRadio_init_zero {0, 0, {meshtastic_MeshPacket_init_zero}}
#define meshtastic_LockdownStatus_init_zero {_meshtastic_LockdownStatus_State_MIN, "", 0, 0, 0}
#define meshtastic_ClientNotification_init_zero {false, 0, 0, _meshtastic_LogRecord_Level_MIN, "", 0, {meshtastic_KeyVerificationNumberInform_init_zero}}
#define meshtastic_KeyVerificationNumberInform_init_zero {0, "", 0}
#define meshtastic_KeyVerificationNumberRequest_init_zero {0, ""}
@@ -1629,6 +1707,7 @@ extern "C" {
#define meshtastic_Data_reply_id_tag 7
#define meshtastic_Data_emoji_tag 8
#define meshtastic_Data_bitfield_tag 9
#define meshtastic_Data_xeddsa_signature_tag 10
#define meshtastic_KeyVerification_nonce_tag 1
#define meshtastic_KeyVerification_hash1_tag 2
#define meshtastic_KeyVerification_hash2_tag 3
@@ -1686,6 +1765,7 @@ extern "C" {
#define meshtastic_MeshPacket_relay_node_tag 19
#define meshtastic_MeshPacket_tx_after_tag 20
#define meshtastic_MeshPacket_transport_mechanism_tag 21
#define meshtastic_MeshPacket_xeddsa_signed_tag 22
#define meshtastic_NodeInfo_num_tag 1
#define meshtastic_NodeInfo_user_tag 2
#define meshtastic_NodeInfo_position_tag 3
@@ -1699,6 +1779,7 @@ extern "C" {
#define meshtastic_NodeInfo_is_ignored_tag 11
#define meshtastic_NodeInfo_is_key_manually_verified_tag 12
#define meshtastic_NodeInfo_is_muted_tag 13
#define meshtastic_NodeInfo_has_xeddsa_signed_tag 14
#define meshtastic_MyNodeInfo_my_node_num_tag 1
#define meshtastic_MyNodeInfo_reboot_count_tag 8
#define meshtastic_MyNodeInfo_min_app_version_tag 11
@@ -1714,6 +1795,11 @@ extern "C" {
#define meshtastic_QueueStatus_free_tag 2
#define meshtastic_QueueStatus_maxlen_tag 3
#define meshtastic_QueueStatus_mesh_packet_id_tag 4
#define meshtastic_LockdownStatus_state_tag 1
#define meshtastic_LockdownStatus_lock_reason_tag 2
#define meshtastic_LockdownStatus_boots_remaining_tag 3
#define meshtastic_LockdownStatus_valid_until_epoch_tag 4
#define meshtastic_LockdownStatus_backoff_seconds_tag 5
#define meshtastic_KeyVerificationNumberInform_nonce_tag 1
#define meshtastic_KeyVerificationNumberInform_remote_longname_tag 2
#define meshtastic_KeyVerificationNumberInform_security_number_tag 3
@@ -1773,6 +1859,7 @@ extern "C" {
#define meshtastic_FromRadio_fileInfo_tag 15
#define meshtastic_FromRadio_clientNotification_tag 16
#define meshtastic_FromRadio_deviceuiConfig_tag 17
#define meshtastic_FromRadio_lockdown_status_tag 18
#define meshtastic_Heartbeat_nonce_tag 1
#define meshtastic_ToRadio_packet_tag 1
#define meshtastic_ToRadio_want_config_id_tag 3
@@ -1859,7 +1946,8 @@ X(a, STATIC, SINGULAR, FIXED32, source, 5) \
X(a, STATIC, SINGULAR, FIXED32, request_id, 6) \
X(a, STATIC, SINGULAR, FIXED32, reply_id, 7) \
X(a, STATIC, SINGULAR, FIXED32, emoji, 8) \
X(a, STATIC, OPTIONAL, UINT32, bitfield, 9)
X(a, STATIC, OPTIONAL, UINT32, bitfield, 9) \
X(a, STATIC, SINGULAR, BYTES, xeddsa_signature, 10)
#define meshtastic_Data_CALLBACK NULL
#define meshtastic_Data_DEFAULT NULL
@@ -1944,7 +2032,8 @@ X(a, STATIC, SINGULAR, BOOL, pki_encrypted, 17) \
X(a, STATIC, SINGULAR, UINT32, next_hop, 18) \
X(a, STATIC, SINGULAR, UINT32, relay_node, 19) \
X(a, STATIC, SINGULAR, UINT32, tx_after, 20) \
X(a, STATIC, SINGULAR, UENUM, transport_mechanism, 21)
X(a, STATIC, SINGULAR, UENUM, transport_mechanism, 21) \
X(a, STATIC, SINGULAR, BOOL, xeddsa_signed, 22)
#define meshtastic_MeshPacket_CALLBACK NULL
#define meshtastic_MeshPacket_DEFAULT NULL
#define meshtastic_MeshPacket_payload_variant_decoded_MSGTYPE meshtastic_Data
@@ -1962,7 +2051,8 @@ X(a, STATIC, OPTIONAL, UINT32, hops_away, 9) \
X(a, STATIC, SINGULAR, BOOL, is_favorite, 10) \
X(a, STATIC, SINGULAR, BOOL, is_ignored, 11) \
X(a, STATIC, SINGULAR, BOOL, is_key_manually_verified, 12) \
X(a, STATIC, SINGULAR, BOOL, is_muted, 13)
X(a, STATIC, SINGULAR, BOOL, is_muted, 13) \
X(a, STATIC, SINGULAR, BOOL, has_xeddsa_signed, 14)
#define meshtastic_NodeInfo_CALLBACK NULL
#define meshtastic_NodeInfo_DEFAULT NULL
#define meshtastic_NodeInfo_user_MSGTYPE meshtastic_User
@@ -2013,7 +2103,8 @@ X(a, STATIC, ONEOF, MESSAGE, (payload_variant,metadata,metadata), 13) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,mqttClientProxyMessage,mqttClientProxyMessage), 14) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,fileInfo,fileInfo), 15) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,clientNotification,clientNotification), 16) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,deviceuiConfig,deviceuiConfig), 17)
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,deviceuiConfig,deviceuiConfig), 17) \
X(a, STATIC, ONEOF, MESSAGE, (payload_variant,lockdown_status,lockdown_status), 18)
#define meshtastic_FromRadio_CALLBACK NULL
#define meshtastic_FromRadio_DEFAULT NULL
#define meshtastic_FromRadio_payload_variant_packet_MSGTYPE meshtastic_MeshPacket
@@ -2030,6 +2121,16 @@ X(a, STATIC, ONEOF, MESSAGE, (payload_variant,deviceuiConfig,deviceuiConfi
#define meshtastic_FromRadio_payload_variant_fileInfo_MSGTYPE meshtastic_FileInfo
#define meshtastic_FromRadio_payload_variant_clientNotification_MSGTYPE meshtastic_ClientNotification
#define meshtastic_FromRadio_payload_variant_deviceuiConfig_MSGTYPE meshtastic_DeviceUIConfig
#define meshtastic_FromRadio_payload_variant_lockdown_status_MSGTYPE meshtastic_LockdownStatus
#define meshtastic_LockdownStatus_FIELDLIST(X, a) \
X(a, STATIC, SINGULAR, UENUM, state, 1) \
X(a, STATIC, SINGULAR, STRING, lock_reason, 2) \
X(a, STATIC, SINGULAR, UINT32, boots_remaining, 3) \
X(a, STATIC, SINGULAR, UINT32, valid_until_epoch, 4) \
X(a, STATIC, SINGULAR, UINT32, backoff_seconds, 5)
#define meshtastic_LockdownStatus_CALLBACK NULL
#define meshtastic_LockdownStatus_DEFAULT NULL
#define meshtastic_ClientNotification_FIELDLIST(X, a) \
X(a, STATIC, OPTIONAL, UINT32, reply_id, 1) \
@@ -2190,6 +2291,7 @@ extern const pb_msgdesc_t meshtastic_MyNodeInfo_msg;
extern const pb_msgdesc_t meshtastic_LogRecord_msg;
extern const pb_msgdesc_t meshtastic_QueueStatus_msg;
extern const pb_msgdesc_t meshtastic_FromRadio_msg;
extern const pb_msgdesc_t meshtastic_LockdownStatus_msg;
extern const pb_msgdesc_t meshtastic_ClientNotification_msg;
extern const pb_msgdesc_t meshtastic_KeyVerificationNumberInform_msg;
extern const pb_msgdesc_t meshtastic_KeyVerificationNumberRequest_msg;
@@ -2226,6 +2328,7 @@ extern const pb_msgdesc_t meshtastic_ChunkedPayloadResponse_msg;
#define meshtastic_LogRecord_fields &meshtastic_LogRecord_msg
#define meshtastic_QueueStatus_fields &meshtastic_QueueStatus_msg
#define meshtastic_FromRadio_fields &meshtastic_FromRadio_msg
#define meshtastic_LockdownStatus_fields &meshtastic_LockdownStatus_msg
#define meshtastic_ClientNotification_fields &meshtastic_ClientNotification_msg
#define meshtastic_KeyVerificationNumberInform_fields &meshtastic_KeyVerificationNumberInform_msg
#define meshtastic_KeyVerificationNumberRequest_fields &meshtastic_KeyVerificationNumberRequest_msg
@@ -2251,7 +2354,7 @@ extern const pb_msgdesc_t meshtastic_ChunkedPayloadResponse_msg;
#define meshtastic_ChunkedPayload_size 245
#define meshtastic_ClientNotification_size 482
#define meshtastic_Compressed_size 239
#define meshtastic_Data_size 269
#define meshtastic_Data_size 335
#define meshtastic_DeviceMetadata_size 54
#define meshtastic_DuplicatedPublicKey_size 0
#define meshtastic_FileInfo_size 236
@@ -2261,14 +2364,15 @@ extern const pb_msgdesc_t meshtastic_ChunkedPayloadResponse_msg;
#define meshtastic_KeyVerificationNumberInform_size 58
#define meshtastic_KeyVerificationNumberRequest_size 52
#define meshtastic_KeyVerification_size 79
#define meshtastic_LockdownStatus_size 53
#define meshtastic_LogRecord_size 426
#define meshtastic_LowEntropyKey_size 0
#define meshtastic_MeshPacket_size 381
#define meshtastic_MeshPacket_size 450
#define meshtastic_MqttClientProxyMessage_size 501
#define meshtastic_MyNodeInfo_size 83
#define meshtastic_NeighborInfo_size 258
#define meshtastic_Neighbor_size 22
#define meshtastic_NodeInfo_size 325
#define meshtastic_NodeInfo_size 327
#define meshtastic_NodeRemoteHardwarePin_size 29
#define meshtastic_Position_size 144
#define meshtastic_QueueStatus_size 23
src/mesh/generated/meshtastic/telemetry.pb.h
+7 -3
View File
@@ -115,7 +115,11 @@ typedef enum _meshtastic_TelemetrySensorType {
/* SHT family of sensors for temperature and humidity */
meshtastic_TelemetrySensorType_SHTXX = 50,
/* DS248X Bridge for one-wire temperature sensors */
meshtastic_TelemetrySensorType_DS248X = 51
meshtastic_TelemetrySensorType_DS248X = 51,
/* MMC5983MA 3-Axis Digital Magnetic Sensor */
meshtastic_TelemetrySensorType_MMC5983MA = 52,
/* ICM-42607-P 6‑Axis IMU */
meshtastic_TelemetrySensorType_ICM42607P = 53
} meshtastic_TelemetrySensorType;
/* Struct definitions */
@@ -496,8 +500,8 @@ extern "C" {
/* Helper constants for enums */
#define _meshtastic_TelemetrySensorType_MIN meshtastic_TelemetrySensorType_SENSOR_UNSET
#define _meshtastic_TelemetrySensorType_MAX meshtastic_TelemetrySensorType_DS248X
#define _meshtastic_TelemetrySensorType_ARRAYSIZE ((meshtastic_TelemetrySensorType)(meshtastic_TelemetrySensorType_DS248X+1))
#define _meshtastic_TelemetrySensorType_MAX meshtastic_TelemetrySensorType_ICM42607P
#define _meshtastic_TelemetrySensorType_ARRAYSIZE ((meshtastic_TelemetrySensorType)(meshtastic_TelemetrySensorType_ICM42607P+1))
src/mesh/mesh-pb-constants.h
+1 -1
View File
@@ -80,7 +80,7 @@ static_assert(sizeof(meshtastic_NodeInfoLite) <= 130, "NodeInfoLite size increas
#if defined(ARCH_STM32WL)
#define MAX_NUM_NODES 10
#elif defined(ARCH_NRF52)
#define MAX_NUM_NODES 80
#define MAX_NUM_NODES 150
#elif defined(CONFIG_IDF_TARGET_ESP32S3)
#include "Esp.h"
static inline int get_max_num_nodes()
src/mesh/wifi/WiFiAPClient.cpp
+88 -15
View File
@@ -15,6 +15,12 @@
#define ETH ETH2
#endif // HAS_ETHERNET
#if HAS_ETHERNET && defined(USE_CH390D)
#include "ESP32_CH390.h"
#include "hal/spi_types.h"
#define ETH CH390
#endif // HAS_ETHERNET
#include <WiFiUdp.h>
#ifdef ARCH_ESP32
#if !MESHTASTIC_EXCLUDE_WEBSERVER
@@ -56,12 +62,43 @@ unsigned long lastrun_ntp = 0;
bool needReconnect = true; // If we create our reconnector, run it once at the beginning
bool isReconnecting = false; // If we are currently reconnecting
#if defined(USE_WS5500) || defined(USE_CH390D)
static volatile bool ethNetworkConnectedPending = false;
#endif
WiFiUDP syslogClient;
meshtastic::Syslog syslog(syslogClient);
Periodic *wifiReconnect;
#if defined(USE_WS5500) || defined(USE_CH390D)
static void onNetworkConnected();
static uint32_t lastEthIP = 0;
static int32_t ethNetworkConnectedPoll()
{
if (ethNetworkConnectedPending) {
ethNetworkConnectedPending = false;
uint32_t ip = (uint32_t)ETH.localIP();
bool ipChanged = APStartupComplete && ip != 0 && ip != lastEthIP;
onNetworkConnected();
if (ipChanged) {
LOG_INFO("Ethernet IP changed (%u.%u.%u.%u), restarting mDNS", ip & 0xff, (ip >> 8) & 0xff, (ip >> 16) & 0xff,
(ip >> 24) & 0xff);
MDNS.end();
if (MDNS.begin("Meshtastic")) {
MDNS.addService("meshtastic", "tcp", SERVER_API_DEFAULT_PORT);
MDNS.addServiceTxt("meshtastic", "tcp", "shortname", String(owner.short_name));
MDNS.addServiceTxt("meshtastic", "tcp", "id", String(nodeDB->getNodeId().c_str()));
MDNS.addServiceTxt("meshtastic", "tcp", "pio_env", optstr(APP_ENV));
}
}
if (ip != 0)
lastEthIP = ip;
}
return 500;
}
#endif
#ifdef USE_WS5500
// Startup Ethernet
bool initEthernet()
@@ -72,6 +109,38 @@ bool initEthernet()
#if !MESHTASTIC_EXCLUDE_WEBSERVER
createSSLCert(); // For WebServer
#endif
new concurrency::Periodic("EthConnect", ethNetworkConnectedPoll);
return true;
}
return false;
}
#endif
#ifdef USE_CH390D
// Startup Ethernet
bool initEthernet()
{
// Configure CH390
ch390_config_t ch390_conf = CH390_DEFAULT_CONFIG();
ch390_conf.spi_host = SPI3_HOST;
ch390_conf.spi_cs_gpio = ETH_CS_PIN;
ch390_conf.spi_sck_gpio = ETH_SCLK_PIN;
ch390_conf.spi_mosi_gpio = ETH_MOSI_PIN;
ch390_conf.spi_miso_gpio = ETH_MISO_PIN;
ch390_conf.int_gpio = ETH_INT_PIN;
#ifdef ETH_RST_PIN
ch390_conf.reset_gpio = ETH_RST_PIN;
#else
ch390_conf.reset_gpio = -1;
#endif
ch390_conf.spi_clock_mhz = 20;
if ((config.network.eth_enabled) && (ETH.begin(ch390_conf))) {
WiFi.onEvent(WiFiEvent);
#if !MESHTASTIC_EXCLUDE_WEBSERVER
createSSLCert(); // For WebServer
#endif
new concurrency::Periodic("EthConnect", ethNetworkConnectedPoll);
return true;
}
@@ -222,9 +291,7 @@ static int32_t reconnectWiFi()
#endif
return 1000; // check once per second
} else {
#ifdef ARCH_RP2040
onNetworkConnected(); // will only do anything once
#endif
onNetworkConnected(); // will only do anything once (guarded by APStartupComplete)
return 300000; // every 5 minutes
}
}
@@ -234,7 +301,7 @@ bool isWifiAvailable()
if (config.network.wifi_enabled && (config.network.wifi_ssid[0])) {
return true;
#ifdef USE_WS5500
#if defined(USE_WS5500) || defined(USE_CH390D)
} else if (config.network.eth_enabled) {
return true;
#endif
@@ -274,9 +341,6 @@ bool initWifi()
const char *wifiPsw = config.network.wifi_psk;
#ifndef ARCH_RP2040
#if !MESHTASTIC_EXCLUDE_WEBSERVER
createSSLCert(); // For WebServer
#endif
WiFi.persistent(false); // Disable flash storage for WiFi credentials
#endif
if (!*wifiPsw) // Treat empty password as no password
@@ -301,6 +365,9 @@ bool initWifi()
#endif
}
#ifdef ARCH_ESP32
// Register WiFi event handler BEFORE createSSLCert() to prevent race condition:
// Without this, WiFi can auto-reconnect during cert generation and fire GOT_IP
// before the handler is registered, causing onNetworkConnected() to never run.
WiFi.onEvent(WiFiEvent);
WiFi.setAutoReconnect(true);
WiFi.setSleep(false);
@@ -322,6 +389,12 @@ bool initWifi()
wifiDisconnectReason = info.wifi_sta_disconnected.reason;
},
WiFiEvent_t::ARDUINO_EVENT_WIFI_STA_DISCONNECTED);
#endif
#ifndef ARCH_RP2040
#if !MESHTASTIC_EXCLUDE_WEBSERVER
createSSLCert(); // For WebServer - called after WiFi.onEvent() to avoid race condition
#endif
#endif
LOG_DEBUG("JOINING WIFI soon: ssid=%s", wifiName);
wifiReconnect = new Periodic("WifiConnect", reconnectWiFi);
@@ -384,13 +457,13 @@ static void WiFiEvent(WiFiEvent_t event)
#endif
}
#ifdef WIFI_LED
digitalWrite(WIFI_LED, HIGH);
digitalWrite(WIFI_LED, LOW ^ WIFI_STATE_ON);
#endif
break;
case ARDUINO_EVENT_WIFI_STA_DISCONNECTED:
LOG_INFO("Disconnected from WiFi access point");
#ifdef WIFI_LED
digitalWrite(WIFI_LED, LOW);
digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
#endif
#if HAS_UDP_MULTICAST
if (udpHandler) {
@@ -452,13 +525,13 @@ static void WiFiEvent(WiFiEvent_t event)
case ARDUINO_EVENT_WIFI_AP_START:
LOG_INFO("WiFi access point started");
#ifdef WIFI_LED
digitalWrite(WIFI_LED, HIGH);
digitalWrite(WIFI_LED, LOW ^ WIFI_STATE_ON);
#endif
break;
case ARDUINO_EVENT_WIFI_AP_STOP:
LOG_INFO("WiFi access point stopped");
#ifdef WIFI_LED
digitalWrite(WIFI_LED, LOW);
digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
#endif
break;
case ARDUINO_EVENT_WIFI_AP_STACONNECTED:
@@ -494,18 +567,18 @@ static void WiFiEvent(WiFiEvent_t event)
LOG_INFO("Ethernet disconnected");
break;
case ARDUINO_EVENT_ETH_GOT_IP:
#ifdef USE_WS5500
#if defined(USE_WS5500) || defined(USE_CH390D)
LOG_INFO("Obtained IP address: %s, %u Mbps, %s", ETH.localIP().toString().c_str(), ETH.linkSpeed(),
ETH.fullDuplex() ? "FULL_DUPLEX" : "HALF_DUPLEX");
onNetworkConnected();
ethNetworkConnectedPending = true;
#endif
break;
case ARDUINO_EVENT_ETH_GOT_IP6:
#ifdef USE_WS5500
#if defined(USE_WS5500) || defined(USE_CH390D)
#if ESP_ARDUINO_VERSION >= ESP_ARDUINO_VERSION_VAL(3, 0, 0)
LOG_INFO("Obtained Local IP6 address: %s", ETH.linkLocalIPv6().toString().c_str());
LOG_INFO("Obtained GlobalIP6 address: %s", ETH.globalIPv6().toString().c_str());
#else
#elif defined(USE_WS5500)
LOG_INFO("Obtained IP6 address: %s", ETH.localIPv6().toString().c_str());
#endif
#endif
src/mesh/wifi/WiFiAPClient.h
+1 -1
View File
@@ -26,7 +26,7 @@ bool isWifiAvailable();
uint8_t getWifiDisconnectReason();
#ifdef USE_WS5500
#if defined(USE_WS5500) || defined(USE_CH390D)
// Startup Ethernet
bool initEthernet();
#endif
src/modules/AdminModule.cpp
+10 -2
View File
@@ -666,7 +666,7 @@ void AdminModule::handleSetConfig(const meshtastic_Config &c, bool fromOthers)
bool requiresReboot = true;
switch (c.which_payload_variant) {
case meshtastic_Config_device_tag:
case meshtastic_Config_device_tag: {
LOG_INFO("Set config: Device");
config.has_device = true;
#if !defined(ARCH_PORTDUINO) && !defined(ARCH_STM32WL) && !MESHTASTIC_EXCLUDE_ENVIRONMENTAL_SENSOR && \
@@ -720,6 +720,7 @@ void AdminModule::handleSetConfig(const meshtastic_Config &c, bool fromOthers)
}
#endif
break;
} // case meshtastic_Config_device_tag
case meshtastic_Config_position_tag:
LOG_INFO("Set config: Position");
config.has_position = true;
@@ -1318,7 +1319,7 @@ void AdminModule::handleGetDeviceConnectionStatus(const meshtastic_MeshPacket &r
}
#endif
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
conn.has_ethernet = true;
conn.ethernet.has_status = true;
if (Ethernet.linkStatus() == LinkON) {
@@ -1345,6 +1346,10 @@ void AdminModule::handleGetDeviceConnectionStatus(const meshtastic_MeshPacket &r
if (config.bluetooth.enabled && nrf52Bluetooth) {
conn.bluetooth.is_connected = nrf52Bluetooth->isConnected();
}
#elif defined(ARCH_NRF54L15)
if (config.bluetooth.enabled && nrf54l15Bluetooth) {
conn.bluetooth.is_connected = nrf54l15Bluetooth->isConnected();
}
#endif
#endif
conn.has_serial = true; // No serial-less devices
@@ -1605,6 +1610,9 @@ void disableBluetooth()
#elif defined(ARCH_NRF52)
if (nrf52Bluetooth)
nrf52Bluetooth->shutdown();
#elif defined(ARCH_NRF54L15)
if (nrf54l15Bluetooth)
nrf54l15Bluetooth->shutdown();
#endif
#endif
}
src/modules/PositionModule.cpp
+11 -31
View File
@@ -4,6 +4,7 @@
#include "GPS.h"
#include "MeshService.h"
#include "NodeDB.h"
#include "PositionPrecision.h"
#include "RTC.h"
#include "Router.h"
#include "TransmitHistory.h"
@@ -107,13 +108,7 @@ bool PositionModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, mes
}
nodeDB->updatePosition(getFrom(&mp), p);
if (channels.getByIndex(mp.channel).settings.has_module_settings) {
precision = channels.getByIndex(mp.channel).settings.module_settings.position_precision;
} else if (channels.getByIndex(mp.channel).role == meshtastic_Channel_Role_PRIMARY) {
precision = 32;
} else {
precision = 0;
}
precision = getPositionPrecisionForChannel(mp.channel);
return false; // Let others look at this message also if they want
}
@@ -121,15 +116,12 @@ bool PositionModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, mes
void PositionModule::alterReceivedProtobuf(meshtastic_MeshPacket &mp, meshtastic_Position *p)
{
// Phone position packets need to be truncated to the channel precision
if (isFromUs(&mp) && (precision < 32 && precision > 0)) {
if (isFromUs(&mp)) {
if (precision == 0)
LOG_DEBUG("Strip phone position due to channel precision 0");
else if (precision < 32)
LOG_DEBUG("Truncate phone position to channel precision %i", precision);
p->latitude_i = p->latitude_i & (UINT32_MAX << (32 - precision));
p->longitude_i = p->longitude_i & (UINT32_MAX << (32 - precision));
// We want the imprecise position to be the middle of the possible location, not
p->latitude_i += (1 << (31 - precision));
p->longitude_i += (1 << (31 - precision));
applyPositionPrecision(*p, precision);
mp.decoded.payload.size =
pb_encode_to_bytes(mp.decoded.payload.bytes, sizeof(mp.decoded.payload.bytes), &meshtastic_Position_msg, p);
}
@@ -206,20 +198,11 @@ meshtastic_MeshPacket *PositionModule::allocPositionPacket()
// lat/lon are unconditionally included - IF AVAILABLE!
LOG_DEBUG("Send location with precision %i", precision);
if (precision < 32 && precision > 0) {
p.latitude_i = localPosition.latitude_i & (UINT32_MAX << (32 - precision));
p.longitude_i = localPosition.longitude_i & (UINT32_MAX << (32 - precision));
// We want the imprecise position to be the middle of the possible location, not
p.latitude_i += (1 << (31 - precision));
p.longitude_i += (1 << (31 - precision));
} else {
p.latitude_i = localPosition.latitude_i;
p.longitude_i = localPosition.longitude_i;
}
p.precision_bits = precision;
p.has_latitude_i = true;
p.has_longitude_i = true;
applyPositionPrecision(p, precision);
// Always use NTP / GPS time if available
if (getValidTime(RTCQualityNTP) > 0) {
p.time = getValidTime(RTCQualityNTP);
@@ -350,8 +333,7 @@ void PositionModule::sendOurPosition()
// If we changed channels, ask everyone else for their latest info
LOG_INFO("Send pos@%x:6 to mesh (wantReplies=%d)", localPosition.timestamp, requestReplies);
for (uint8_t channelNum = 0; channelNum < 8; channelNum++) {
if (channels.getByIndex(channelNum).settings.has_module_settings &&
channels.getByIndex(channelNum).settings.module_settings.position_precision != 0) {
if (getPositionPrecisionForChannel(channelNum) != 0) {
sendOurPosition(NODENUM_BROADCAST, requestReplies, channelNum);
return;
}
@@ -369,10 +351,8 @@ void PositionModule::sendOurPosition(NodeNum dest, bool wantReplies, uint8_t cha
if (prevPacketId) // if we wrap around to zero, we'll simply fail to cancel in that rare case (no big deal)
service->cancelSending(prevPacketId);
// Set's the class precision value for this particular packet
if (channels.getByIndex(channel).settings.has_module_settings) {
precision = channels.getByIndex(channel).settings.module_settings.position_precision;
}
// Set the class precision value for this particular packet.
precision = getPositionPrecisionForChannel(channel);
meshtastic_MeshPacket *p = allocPositionPacket();
if (p == nullptr) {
src/modules/StatusLEDModule.cpp
+18 -3
View File
@@ -95,8 +95,23 @@ int32_t StatusLEDModule::runOnce()
}
}
}
if (power_state != charging && power_state != charged && !doing_fast_blink) {
// If we want a LED to be dedicated to the simple hearbeat, we can use that instead of the charge LED
#if defined(LED_HEARTBEAT)
if (power_state != charging && power_state != charged && !doing_fast_blink && !config.device.led_heartbeat_disabled) {
if (HEARTBEAT_LED_state == LED_STATE_ON) {
HEARTBEAT_LED_state = LED_STATE_OFF;
my_interval = 999;
} else {
HEARTBEAT_LED_state = LED_STATE_ON;
my_interval = 1;
}
digitalWrite(LED_HEARTBEAT, HEARTBEAT_LED_state);
} else {
HEARTBEAT_LED_state = LED_STATE_OFF;
digitalWrite(LED_HEARTBEAT, HEARTBEAT_LED_state);
}
#else
if (power_state != charging && power_state != charged && !doing_fast_blink && !config.device.led_heartbeat_disabled) {
if (CHARGE_LED_state == LED_STATE_ON) {
CHARGE_LED_state = LED_STATE_OFF;
my_interval = 999;
@@ -105,7 +120,7 @@ int32_t StatusLEDModule::runOnce()
my_interval = 1;
}
}
#endif
if (!config.bluetooth.enabled || PAIRING_LED_starttime + 30 * 1000 < millis() || doing_fast_blink) {
PAIRING_LED_state = LED_STATE_OFF;
} else if (ble_state == unpaired) {
src/modules/StatusLEDModule.h
+3
View File
@@ -43,6 +43,9 @@ class StatusLEDModule : private concurrency::OSThread
private:
bool CHARGE_LED_state = LED_STATE_OFF;
bool PAIRING_LED_state = LED_STATE_OFF;
#if defined(LED_HEARTBEAT)
bool HEARTBEAT_LED_state = LED_STATE_OFF;
#endif
uint32_t PAIRING_LED_starttime = 0;
uint32_t lastUserbuttonTime = 0;
src/modules/SystemCommandsModule.cpp
+11
View File
@@ -115,6 +115,17 @@ int SystemCommandsModule::handleInputEvent(const InputEvent *event)
case INPUT_BROKER_SHUTDOWN:
shutdownAtMsec = millis();
return true;
// factory reset
case INPUT_BROKER_FACTORY_RST:
disableBluetooth();
LOG_INFO("Initiate full factory reset");
nodeDB->factoryReset(true);
// reboot(DEFAULT_REBOOT_SECONDS);
LOG_INFO("Reboot in %d seconds", DEFAULT_REBOOT_SECONDS);
if (screen)
screen->showSimpleBanner("Rebooting...", 0); // stays on screen
rebootAtMsec = (DEFAULT_REBOOT_SECONDS < 0) ? 0 : (millis() + DEFAULT_REBOOT_SECONDS * 1000);
return true;
default:
// No other input events handled here
src/modules/TextMessageModule.cpp
-3
View File
@@ -21,9 +21,6 @@ ProcessMessage TextMessageModule::handleReceived(const meshtastic_MeshPacket &mp
textPacketList[textPacketListIndex] = mp.id;
textPacketListIndex = (textPacketListIndex + 1) % TEXT_PACKET_LIST_SIZE;
// We only store/display messages destined for us.
devicestate.rx_text_message = mp;
devicestate.has_rx_text_message = true;
IF_SCREEN(
// Guard against running in MeshtasticUI or with no screen
if (config.display.displaymode != meshtastic_Config_DisplayConfig_DisplayMode_COLOR) {
src/modules/TextMessageModule.h
+2 -2
View File
@@ -7,8 +7,8 @@
* Text message handling for Meshtastic.
*
* This module is responsible for receiving and storing incoming text messages
* from the mesh. It updates device state and notifies observers so that other
* components (such as the MessageRenderer) can later display or process them.
* from the mesh. It notifies observers so that other components (such as the
* MessageRenderer) can later display or process them.
*
* Rendering of messages on screen is no longer done here.
*/
src/mqtt/MQTT.cpp
+6
View File
@@ -22,6 +22,9 @@
#if HAS_ETHERNET && defined(USE_WS5500)
#include <ETHClass2.h>
#define ETH ETH2
#elif HAS_ETHERNET && defined(USE_CH390D)
#include "ESP32_CH390.h"
#define ETH CH390
#endif // HAS_ETHERNET
#include "Default.h"
#if !defined(ARCH_NRF52) || NRF52_USE_JSON
@@ -344,6 +347,9 @@ inline bool isConnectedToNetwork()
#ifdef USE_WS5500
if (ETH.connected())
return true;
#elif defined(USE_CH390D)
if (ETH.isConnected())
return true;
#endif
#if HAS_WIFI
src/mqtt/MQTT.h
+1 -1
View File
@@ -15,7 +15,7 @@
#include <WiFiClientSecure.h>
#endif
#endif
#if HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
#include <EthernetClient.h>
#endif
src/nimble/NimbleBluetooth.cpp
+8 -29
View File
@@ -587,51 +587,30 @@ class NimbleBluetoothServerCallback : public NimBLEServerCallbacks
virtual uint32_t onPassKeyRequest() override
#endif
{
uint32_t passkey = config.bluetooth.fixed_pin;
uint32_t configuredPasskey = config.bluetooth.fixed_pin;
if (config.bluetooth.mode == meshtastic_Config_BluetoothConfig_PairingMode_RANDOM_PIN) {
LOG_INFO("Use random passkey");
// This is the passkey to be entered on peer - we pick a number >100,000 to ensure 6 digits
passkey = random(100000, 999999);
configuredPasskey = random(100000, 999999);
}
LOG_INFO("*** Enter passkey %d on the peer side ***", passkey);
LOG_INFO("*** Enter passkey %d on the peer side ***", configuredPasskey);
powerFSM.trigger(EVENT_BLUETOOTH_PAIR);
meshtastic::BluetoothStatus newStatus(std::to_string(passkey));
std::string passkey = std::to_string(configuredPasskey);
meshtastic::BluetoothStatus newStatus(passkey);
bluetoothStatus->updateStatus(&newStatus);
#if HAS_SCREEN // Todo: migrate this display code back into Screen class, and observe bluetoothStatus
if (screen) {
screen->startAlert([passkey](OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y) -> void {
char btPIN[16] = "888888";
snprintf(btPIN, sizeof(btPIN), "%06u", passkey);
int x_offset = display->width() / 2;
int y_offset = display->height() <= 80 ? 0 : 12;
display->setTextAlignment(TEXT_ALIGN_CENTER);
display->setFont(FONT_MEDIUM);
display->drawString(x_offset + x, y_offset + y, "Bluetooth");
#if !defined(M5STACK_UNITC6L)
display->setFont(FONT_SMALL);
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_MEDIUM - 4 : y_offset + FONT_HEIGHT_MEDIUM + 5;
display->drawString(x_offset + x, y_offset + y, "Enter this code");
#endif
display->setFont(FONT_LARGE);
char pin[8];
snprintf(pin, sizeof(pin), "%.3s %.3s", btPIN, btPIN + 3);
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_SMALL - 5 : y_offset + FONT_HEIGHT_SMALL + 5;
display->drawString(x_offset + x, y_offset + y, pin);
display->setFont(FONT_SMALL);
char deviceName[64];
snprintf(deviceName, sizeof(deviceName), "Name: %s", getDeviceName());
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_LARGE - 6 : y_offset + FONT_HEIGHT_LARGE + 5;
display->drawString(x_offset + x, y_offset + y, deviceName);
});
std::string ble_message = "Bluetooth\nPIN\n[M]" + passkey.substr(0, 3) + " " + passkey.substr(3, 6);
screen->showSimpleBanner(ble_message.c_str(), 30000);
}
#endif
passkeyShowing = true;
return passkey;
return configuredPasskey;
}
#ifdef NIMBLE_TWO
src/platform/esp32/architecture.h
+4
View File
@@ -146,6 +146,8 @@
#define HW_VENDOR meshtastic_HardwareModel_THINKNODE_M2
#elif defined(ELECROW_ThinkNode_M5)
#define HW_VENDOR meshtastic_HardwareModel_THINKNODE_M5
#elif defined(ELECROW_ThinkNode_M7)
#define HW_VENDOR meshtastic_HardwareModel_THINKNODE_M7
#elif defined(ESP32_S3_PICO)
#define HW_VENDOR meshtastic_HardwareModel_ESP32_S3_PICO
#elif defined(SENSELORA_S3)
@@ -156,6 +158,8 @@
#define HW_VENDOR meshtastic_HardwareModel_CHATTER_2
#elif defined(STATION_G2)
#define HW_VENDOR meshtastic_HardwareModel_STATION_G2
#elif defined(STATION_G3)
#define HW_VENDOR meshtastic_HardwareModel_STATION_G3
#elif defined(UNPHONE)
#define HW_VENDOR meshtastic_HardwareModel_UNPHONE
#elif defined(WIPHONE)
src/platform/esp32/main-esp32.cpp
+1 -1
View File
@@ -32,7 +32,7 @@ void variant_shutdown() {}
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !MESHTASTIC_EXCLUDE_BLUETOOTH
void setBluetoothEnable(bool enable)
{
#ifdef USE_WS5500
#if defined(USE_WS5500) || defined(USE_CH390D)
if ((config.bluetooth.enabled == true) && (config.network.wifi_enabled == false))
#elif HAS_WIFI
if (!isWifiAvailable() && config.bluetooth.enabled == true)
src/platform/nrf52/NRF52Bluetooth.cpp
+5 -27
View File
@@ -386,34 +386,12 @@ bool NRF52Bluetooth::onPairingPasskey(uint16_t conn_handle, uint8_t const passke
meshtastic::BluetoothStatus newStatus(textkey);
bluetoothStatus->updateStatus(&newStatus);
#if HAS_SCREEN && \
!defined(MESHTASTIC_EXCLUDE_SCREEN) // Todo: migrate this display code back into Screen class, and observe bluetoothStatus
#if HAS_SCREEN && !defined(MESHTASTIC_EXCLUDE_SCREEN)
if (screen) {
screen->startAlert([](OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y) -> void {
char btPIN[16] = "888888";
snprintf(btPIN, sizeof(btPIN), "%06u", configuredPasskey);
int x_offset = display->width() / 2;
int y_offset = display->height() <= 80 ? 0 : 12;
display->setTextAlignment(TEXT_ALIGN_CENTER);
display->setFont(FONT_MEDIUM);
display->drawString(x_offset + x, y_offset + y, "Bluetooth");
display->setFont(FONT_SMALL);
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_MEDIUM - 4 : y_offset + FONT_HEIGHT_MEDIUM + 5;
display->drawString(x_offset + x, y_offset + y, "Enter this code");
display->setFont(FONT_LARGE);
String displayPin(btPIN);
String pin = displayPin.substring(0, 3) + " " + displayPin.substring(3, 6);
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_SMALL - 5 : y_offset + FONT_HEIGHT_SMALL + 5;
display->drawString(x_offset + x, y_offset + y, pin);
display->setFont(FONT_SMALL);
String deviceName = "Name: ";
deviceName.concat(getDeviceName());
y_offset = display->height() == 64 ? y_offset + FONT_HEIGHT_LARGE - 6 : y_offset + FONT_HEIGHT_LARGE + 5;
display->drawString(x_offset + x, y_offset + y, deviceName);
});
std::string configuredPasskeyText = std::to_string(configuredPasskey);
std::string ble_message =
"Bluetooth\nPIN\n[M]" + configuredPasskeyText.substr(0, 3) + " " + configuredPasskeyText.substr(3, 6);
screen->showSimpleBanner(ble_message.c_str(), 30000);
}
#endif
passkeyShowing = true;
src/platform/nrf54l15/Arduino.h
+824
View File
@@ -0,0 +1,824 @@
/**
* Arduino.h — Zephyr compatibility shim for nRF54L15
*
* Provides the Arduino API surface expected by Meshtastic, backed by
* Zephyr primitives. Only the subset actually used by Meshtastic is
* implemented; the rest compiles as no-ops / stubs for now.
*
* Phase 2: compile only. Real GPIO / SPI / Wire implementations follow
* in Phase 3 once the build is clean.
*/
#pragma once
#ifndef Arduino_h
#define Arduino_h
// ── C standard headers ───────────────────────────────────────────────────────
#include <math.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h> /* strcasecmp, strncasecmp */
// ── Zephyr kernel ────────────────────────────────────────────────────────────
#include <zephyr/kernel.h>
#include <zephyr/sys/reboot.h>
// ── Basic Arduino types ──────────────────────────────────────────────────────
typedef bool boolean;
typedef uint8_t byte;
typedef uint16_t word;
// ── Pin / digital constants ──────────────────────────────────────────────────
#define INPUT 0u
#define OUTPUT 1u
#define INPUT_PULLUP 2u
#define INPUT_PULLDOWN 3u
#define OUTPUT_OPENDRAIN 4u
#define HIGH 1u
#define LOW 0u
#define CHANGE 1
#define FALLING 2
#define RISING 3
#ifndef LED_BUILTIN
#define LED_BUILTIN -1
#endif
// ── Math / trig constants ────────────────────────────────────────────────────
#ifndef PI
#define PI 3.14159265358979323846
#endif
#define HALF_PI 1.57079632679489661923
#define TWO_PI 6.28318530717958647693
#define DEG_TO_RAD 0.01745329251994329576
#define RAD_TO_DEG 57.2957795130823208767
#define EULER 2.71828182845904523536
// ── Bit utilities ────────────────────────────────────────────────────────────
#define bitRead(v, b) (((v) >> (b)) & 1)
#define bitSet(v, b) ((v) |= (1UL << (b)))
#define bitClear(v, b) ((v) &= ~(1UL << (b)))
#define bitToggle(v, b) ((v) ^= (1UL << (b)))
#define bitWrite(v, b, x) ((x) ? bitSet(v, b) : bitClear(v, b))
#define bit(b) (1UL << (b))
#define lowByte(w) ((uint8_t)((w)&0xff))
#define highByte(w) ((uint8_t)((w) >> 8))
// word(h,l) — only define if not already defined (conflicts with typedef above)
#undef word
#define word(h, l) ((uint16_t)(((h) << 8) | (l)))
// ── UART config constants ─────────────────────────────────────────────────────
#define SERIAL_8N1 0x800001cu
#define SERIAL_8N2 0x8000001eu
#define SERIAL_8E1 0x8000001eu
#define SERIAL_7E1 0x8000001cu
// ── Integer order ────────────────────────────────────────────────────────────
// Adafruit BusIO's SPIDevice.h has `typedef BitOrder BusIOBitOrder;` which
// requires BitOrder to be a *type*, not a macro. Mirror the ArduinoCore-API
// enum definition rather than #defines.
enum BitOrder : uint8_t {
LSBFIRST = 0,
MSBFIRST = 1,
};
// ── pgmspace compatibility (no-ops on Cortex-M) ──────────────────────────────
#define PROGMEM
#define PSTR(s) (s)
#define F(s) (s)
#define pgm_read_byte(addr) (*((const uint8_t *)(addr)))
#define pgm_read_word(addr) (*((const uint16_t *)(addr)))
#define pgm_read_dword(addr) (*((const uint32_t *)(addr)))
#define pgm_read_float(addr) (*((const float *)(addr)))
#define pgm_read_ptr(addr) (*((const void **)(addr)))
#define strlen_P(s) strlen(s)
#define strcpy_P(d, s) strcpy(d, s)
#define strncpy_P(d, s, n) strncpy(d, s, n)
#define strcmp_P(a, b) strcmp(a, b)
#define memcpy_P(d, s, n) memcpy(d, s, n)
#define sprintf_P sprintf
typedef const char *PGM_P;
typedef const char *PGM_VOID_P;
// ── Arduino numeric base constants (used by Print, RadioLib, etc.) ───────────
#define DEC 10
#define HEX 16
#define OCT 8
#define BIN 2
// ── ulong / uint typedef (used by RadioLibInterface, etc.) ───────────────────
typedef unsigned long ulong;
typedef unsigned int uint;
// ── Interrupt stubs ──────────────────────────────────────────────────────────
static inline void interrupts() {}
static inline void noInterrupts() {}
#define digitalPinToInterrupt(p) (p)
// ── portMAX_DELAY — freertosinc.h also defines this; let it win ──────────────
// We intentionally do NOT define portMAX_DELAY here. freertosinc.h defines
// it for the FreeRTOS / Meshtastic threading layer and must not be overridden.
// ── Timing & system functions — declared with C linkage ──────────────────────
// buzz.cpp and others forward-declare delay() as extern "C"; keep linkage
// consistent by wrapping in extern "C" here.
#ifdef __cplusplus
extern "C" {
#endif
void NVIC_SystemReset(void);
uint32_t millis(void);
uint32_t micros(void);
void delay(uint32_t ms);
void delayMicroseconds(uint32_t us);
void yield(void);
#ifdef __cplusplus
}
#endif
#ifdef __cplusplus
#include <cctype>
#include <cstdarg>
// ── C++ STL — include BEFORE defining any min/max helpers ───────────────────
// Include algorithm first so its min/max templates are in scope.
// We must NOT define min/max as function-like macros: the C++ STL uses
// 3-argument versions (min(a,b,comp)) that the preprocessor would treat as
// calling a 2-arg macro with 3 args.
#include <algorithm>
// Bring 2-arg std::min / std::max into the global namespace as unqualified
// names so that Arduino code calling min(a,b) continues to compile.
// (Arduino convention; kept minimal to avoid surprises.)
#undef min
#undef max
using std::max;
using std::min;
// ── Arduino math helpers (macros safe for mixed-type / C calls) ──────────────
#ifndef abs
#define abs(x) ((x) >= 0 ? (x) : -(x))
#endif
#define constrain(x, l, h) ((x) < (l) ? (l) : ((x) > (h) ? (h) : (x)))
#define round(x) ((x) >= 0 ? (long)((x) + 0.5) : (long)((x)-0.5))
#define radians(d) ((d)*DEG_TO_RAD)
#define degrees(r) ((r)*RAD_TO_DEG)
#define sq(x) ((x) * (x))
// ── Random ───────────────────────────────────────────────────────────────────
static inline void randomSeed(unsigned long seed)
{
srand((unsigned int)seed);
}
static inline long random(void)
{
return (long)rand();
}
static inline long random(long bound)
{
return bound > 0 ? (rand() % bound) : 0;
}
static inline long random(long lo, long hi)
{
return hi > lo ? lo + rand() % (hi - lo) : lo;
}
// ── GPIO — real Zephyr implementation (Phase 3) ──────────────────────────────
// Implemented in nrf54l15_arduino.cpp using Zephyr GPIO/SPI APIs.
// Pin numbering: P0.n = n, P1.n = 16+n, P2.n = 32+n
void pinMode(uint32_t pin, uint32_t mode);
void digitalWrite(uint32_t pin, uint32_t value);
int digitalRead(uint32_t pin);
static inline void digitalToggle(uint32_t pin)
{
digitalWrite(pin, !digitalRead(pin));
}
static inline uint32_t analogRead(uint32_t)
{
return 0;
}
static inline void analogWrite(uint32_t, uint32_t) {}
static inline void analogReadResolution(int) {}
static inline void analogWriteResolution(int) {}
// ── __WFI — provided by CMSIS core_cm33.h; do NOT redefine here ─────────────
// ── __FlashStringHelper — Arduino PROGMEM string class (no-op on Cortex-M) ──
class __FlashStringHelper;
// ── attachInterrupt / detachInterrupt — real Zephyr GPIO interrupt impl ──────
typedef void (*voidFuncPtr)(void);
void attachInterrupt(uint32_t pin, voidFuncPtr cb, int mode);
void detachInterrupt(uint32_t pin);
// ── Forward declaration of String (needed by Print / Stream) ─────────────────
class String;
// ── Print base class ─────────────────────────────────────────────────────────
class Print
{
public:
virtual size_t write(uint8_t c) = 0;
virtual size_t write(const uint8_t *buf, size_t n)
{
size_t written = 0;
while (n--)
written += write(*buf++);
return written;
}
size_t write(const char *s) { return s ? write((const uint8_t *)s, strlen(s)) : 0; }
size_t write(const char *s, size_t n) { return write((const uint8_t *)s, n); }
size_t print(const char *s) { return s ? write((const uint8_t *)s, strlen(s)) : 0; }
int printf(const char *fmt, ...) __attribute__((format(printf, 2, 3)));
size_t print(char c) { return write((uint8_t)c); }
size_t print(const String &s);
size_t print(unsigned char n, int base = 10);
size_t print(int n, int base = 10);
size_t print(long n, int base = 10);
size_t print(unsigned int n, int base = 10);
size_t print(unsigned long n, int base = 10);
size_t print(float n, int digits = 2);
size_t print(double n, int digits = 2);
size_t print(bool b) { return print(b ? "true" : "false"); }
size_t println() { return write((uint8_t)'\n'); }
size_t println(const char *s)
{
size_t r = print(s);
return r + println();
}
size_t println(char c)
{
size_t r = print(c);
return r + println();
}
size_t println(const String &s);
size_t println(int n, int base = 10)
{
size_t r = print(n, base);
return r + println();
}
size_t println(long n, int base = 10)
{
size_t r = print(n, base);
return r + println();
}
size_t println(unsigned long n, int base = 10)
{
size_t r = print(n, base);
return r + println();
}
size_t println(unsigned int n, int base = 10)
{
size_t r = print(n, base);
return r + println();
}
size_t println(float n, int d = 2)
{
size_t r = print(n, d);
return r + println();
}
size_t println(double n, int d = 2)
{
size_t r = print(n, d);
return r + println();
}
size_t println(bool b)
{
size_t r = print(b);
return r + println();
}
virtual void flush() {}
};
// ── Stream base class ────────────────────────────────────────────────────────
class Stream : public Print
{
public:
virtual int available() = 0;
virtual int read() = 0;
virtual int peek() = 0;
virtual void setTimeout(unsigned long) {}
virtual bool find(const char *) { return false; }
String readString();
String readStringUntil(char terminator);
};
// ── Minimal Arduino String class (backed by a char buffer) ───────────────────
class String
{
public:
String() : _buf(nullptr), _len(0), _cap(0) {}
// Implicit conversion is part of the Arduino String contract, used pervasively across the codebase.
// cppcheck-suppress noExplicitConstructor
String(const char *cstr) : _buf(nullptr), _len(0), _cap(0)
{
if (cstr)
assign(cstr, strlen(cstr));
}
// cppcheck-suppress noExplicitConstructor
String(const String &s) : _buf(nullptr), _len(0), _cap(0) { assign(s._buf ? s._buf : "", s._len); }
// cppcheck-suppress noExplicitConstructor
String(char c) : _buf(nullptr), _len(0), _cap(0)
{
const char tmp[2] = {c, 0};
assign(tmp, 1);
}
// cppcheck-suppress noExplicitConstructor
String(int n) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[16];
snprintf(tmp, 16, "%d", n);
assign(tmp, strlen(tmp));
}
// cppcheck-suppress noExplicitConstructor
String(unsigned int n) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[16];
snprintf(tmp, 16, "%u", n);
assign(tmp, strlen(tmp));
}
// cppcheck-suppress noExplicitConstructor
String(long n) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[24];
snprintf(tmp, 24, "%ld", n);
assign(tmp, strlen(tmp));
}
// cppcheck-suppress noExplicitConstructor
String(unsigned long n) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[24];
snprintf(tmp, 24, "%lu", n);
assign(tmp, strlen(tmp));
}
// cppcheck-suppress noExplicitConstructor
String(float n, int d = 2) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[32];
snprintf(tmp, 32, "%.*f", d, n);
assign(tmp, strlen(tmp));
}
// cppcheck-suppress noExplicitConstructor
String(double n, int d = 2) : _buf(nullptr), _len(0), _cap(0)
{
char tmp[32];
snprintf(tmp, 32, "%.*f", d, (double)n);
assign(tmp, strlen(tmp));
}
~String() { free(_buf); }
String &operator=(const String &s)
{
assign(s._buf ? s._buf : "", s._len);
return *this;
}
String &operator=(const char *s)
{
assign(s ? s : "", s ? strlen(s) : 0);
return *this;
}
String &operator=(char c)
{
const char tmp[2] = {c, 0};
assign(tmp, 1);
return *this;
}
String &operator+=(const String &s)
{
concat(s._buf ? s._buf : "", s._len);
return *this;
}
String &operator+=(const char *s)
{
if (s)
concat(s, strlen(s));
return *this;
}
String &operator+=(char c)
{
concat(&c, 1);
return *this;
}
String &operator+=(int n) { return *this += String(n); }
String &operator+=(unsigned long n) { return *this += String(n); }
String operator+(const String &rhs) const
{
String r(*this);
r += rhs;
return r;
}
String operator+(const char *rhs) const
{
String r(*this);
r += rhs;
return r;
}
String operator+(char rhs) const
{
String r(*this);
r += rhs;
return r;
}
bool operator==(const String &s) const { return _len == s._len && (_len == 0 || strcmp(_buf, s._buf) == 0); }
bool operator==(const char *s) const { return s && strcmp(c_str(), s) == 0; }
bool operator!=(const String &s) const { return !(*this == s); }
bool operator!=(const char *s) const { return !(*this == s); }
bool operator<(const String &s) const { return strcmp(c_str(), s.c_str()) < 0; }
bool operator>(const String &s) const { return strcmp(c_str(), s.c_str()) > 0; }
char operator[](unsigned int i) const { return (_buf && i < _len) ? _buf[i] : 0; }
char &operator[](unsigned int i)
{
static char dummy = 0;
return (_buf && i < _len) ? _buf[i] : dummy;
}
const char *c_str() const { return _buf ? _buf : ""; }
unsigned int length() const { return _len; }
bool isEmpty() const { return _len == 0; }
bool equals(const String &s) const { return *this == s; }
bool equals(const char *s) const { return *this == s; }
bool equalsIgnoreCase(const String &s) const
{
if (_len != s._len)
return false;
for (unsigned i = 0; i < _len; i++)
if (std::tolower(_buf[i]) != std::tolower(s._buf[i]))
return false;
return true;
}
bool startsWith(const String &pfx) const
{
if (pfx._len > _len)
return false;
return strncmp(c_str(), pfx.c_str(), pfx._len) == 0;
}
bool startsWith(const char *pfx) const
{
if (!pfx)
return false;
size_t pl = strlen(pfx);
return pl <= _len && strncmp(c_str(), pfx, pl) == 0;
}
bool endsWith(const String &sfx) const
{
if (sfx._len > _len)
return false;
return strcmp(c_str() + _len - sfx._len, sfx.c_str()) == 0;
}
int indexOf(char c, unsigned from = 0) const
{
if (!_buf)
return -1;
const char *p = strchr(_buf + from, c);
return p ? (int)(p - _buf) : -1;
}
int indexOf(const String &s, unsigned from = 0) const
{
if (!_buf)
return -1;
const char *p = strstr(_buf + from, s.c_str());
return p ? (int)(p - _buf) : -1;
}
int lastIndexOf(char c) const
{
if (!_buf)
return -1;
const char *p = strrchr(_buf, c);
return p ? (int)(p - _buf) : -1;
}
String substring(unsigned beginIndex) const
{
if (!_buf || beginIndex >= _len)
return String();
return String(_buf + beginIndex);
}
String substring(unsigned beginIndex, unsigned endIndex) const
{
if (!_buf || beginIndex >= _len)
return String();
if (endIndex > _len)
endIndex = _len;
if (endIndex <= beginIndex)
return String();
String r;
r.assign(_buf + beginIndex, endIndex - beginIndex);
return r;
}
void toUpperCase()
{
if (_buf)
for (unsigned i = 0; i < _len; i++)
_buf[i] = (char)std::toupper(_buf[i]);
}
void toLowerCase()
{
if (_buf)
for (unsigned i = 0; i < _len; i++)
_buf[i] = (char)std::tolower(_buf[i]);
}
void trim()
{
if (!_buf || _len == 0)
return;
unsigned s = 0;
while (s < _len && std::isspace(_buf[s]))
s++;
unsigned e = _len;
while (e > s && std::isspace(_buf[e - 1]))
e--;
if (s > 0 || e < _len) {
memmove(_buf, _buf + s, e - s);
_len = e - s;
_buf[_len] = 0;
}
}
void replace(char from, char to)
{
if (_buf)
for (unsigned i = 0; i < _len; i++)
if (_buf[i] == from)
_buf[i] = to;
}
void replace(const String &from, const String &to);
bool remove(unsigned index, unsigned count = 1)
{
if (!_buf || index >= _len)
return false;
if (index + count > _len)
count = _len - index;
memmove(_buf + index, _buf + index + count, _len - index - count + 1);
_len -= count;
return true;
}
void clear()
{
_len = 0;
if (_buf)
_buf[0] = 0;
}
char charAt(unsigned i) const { return (*this)[i]; }
void setCharAt(unsigned i, char c)
{
if (_buf && i < _len)
_buf[i] = c;
}
void toCharArray(char *buf, unsigned int bufsize, unsigned int index = 0) const
{
if (!buf || bufsize == 0)
return;
unsigned int avail = (_buf && _len > index) ? (_len - index) : 0;
unsigned int copy = avail < bufsize - 1 ? avail : bufsize - 1;
if (copy > 0)
memcpy(buf, _buf + index, copy);
buf[copy] = '\0';
}
void concat(const String &s) { *this += s; }
void concat(const char *s) { *this += s; }
long toInt() const { return _buf ? atol(_buf) : 0; }
float toFloat() const { return _buf ? (float)atof(_buf) : 0.0f; }
double toDouble() const { return _buf ? atof(_buf) : 0.0; }
private:
char *_buf;
unsigned int _len;
unsigned int _cap;
void assign(const char *s, unsigned int n)
{
if (n >= _cap)
reserve(n + 1);
if (_buf) {
memcpy(_buf, s, n);
_buf[n] = 0;
_len = n;
}
}
void concat(const char *s, unsigned int n)
{
if (!s || n == 0)
return;
unsigned newlen = _len + n;
if (newlen >= _cap)
reserve(newlen + 1);
if (_buf) {
memcpy(_buf + _len, s, n);
_len = newlen;
_buf[_len] = 0;
}
}
void reserve(unsigned int n)
{
char *b = (char *)realloc(_buf, n);
if (b) {
_buf = b;
_cap = n;
}
}
};
inline String operator+(const char *lhs, const String &rhs)
{
return String(lhs) + rhs;
}
inline String operator+(char lhs, const String &rhs)
{
return String(lhs) + rhs;
}
// ── Print inline definitions that need String ────────────────────────────────
inline size_t Print::print(const String &s)
{
return write((const uint8_t *)s.c_str(), s.length());
}
inline size_t Print::println(const String &s)
{
size_t r = print(s);
return r + println();
}
// ── Stream inline definitions that need String ───────────────────────────────
inline String Stream::readString()
{
return String();
}
inline String Stream::readStringUntil(char)
{
return String();
}
// ── HardwareSerial ───────────────────────────────────────────────────────────
class HardwareSerial : public Stream
{
public:
void begin(unsigned long) {}
void begin(unsigned long, uint16_t) {}
void end() {}
void setPins(int rx, int tx) {}
void setPinout(int tx, int rx) {}
void setFIFOSize(size_t) {}
void setRxBufferSize(size_t) {}
void begin(unsigned long baud, uint32_t config, int8_t rx = -1, int8_t tx = -1, bool invert = false) {}
int available() override { return 0; }
int read() override { return -1; }
int peek() override { return -1; }
size_t write(uint8_t c) override;
size_t write(const uint8_t *buf, size_t n) override;
using Print::write; // un-hide base class write(const char*)
size_t readBytes(uint8_t *buf, size_t len) { return 0; }
size_t readBytes(char *buf, size_t len) { return 0; }
operator bool() const { return true; }
void flush() override {}
String readString() { return String(); }
String readStringUntil(char) { return String(); }
};
// Uart — nRF52 BSP alias for HardwareSerial (used by GPS.h when ARCH_NRF52)
typedef HardwareSerial Uart;
extern HardwareSerial Serial;
extern HardwareSerial Serial1;
extern HardwareSerial Serial2;
// ── map() utility ────────────────────────────────────────────────────────────
static inline long map(long x, long in_min, long in_max, long out_min, long out_max)
{
return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}
// ── shiftIn / shiftOut stubs ─────────────────────────────────────────────────
static inline uint8_t shiftIn(uint8_t, uint8_t, uint8_t)
{
return 0;
}
static inline void shiftOut(uint8_t, uint8_t, uint8_t, uint8_t) {}
// ── tone / noTone stubs ──────────────────────────────────────────────────────
static inline void tone(uint8_t, unsigned int, unsigned long = 0) {}
static inline void noTone(uint8_t) {}
// ── pulseIn stub ─────────────────────────────────────────────────────────────
static inline unsigned long pulseIn(uint8_t, uint8_t, unsigned long = 1000000UL)
{
return 0;
}
// ── strdup / stpcpy — POSIX extensions not in Zephyr newlib ─────────────────
#ifndef strdup
static inline char *strdup(const char *s)
{
size_t n = strlen(s) + 1;
char *d = (char *)malloc(n);
if (d)
memcpy(d, s, n);
return d;
}
#endif
#ifndef stpcpy
static inline char *stpcpy(char *dst, const char *src)
{
while ((*dst++ = *src++) != '\0') {
}
return dst - 1;
}
#endif
// ── strnstr — BSD extension not in Zephyr libc; defined in meshUtils.cpp ─────
// Declare here so callers (GPS.cpp etc.) don't need ARCH_PORTDUINO.
#ifndef STRNSTR
#define STRNSTR
char *strnstr(const char *s, const char *find, size_t slen);
#endif
// ── strlcpy — BSD extension; implementation in nrf54l15_arduino.cpp ──────────
#ifndef HAVE_STRLCPY
#define HAVE_STRLCPY
#ifdef __cplusplus
extern "C" {
#endif
size_t strlcpy(char *dst, const char *src, size_t size);
#ifdef __cplusplus
}
#endif
#endif
// ── setenv / getenv / tzset — Zephyr stubs for timezone support ──────────────
#include <stdlib.h>
static inline int setenv(const char *, const char *, int)
{
return 0;
}
static inline void tzset(void) {}
// ── dbgHeapFree / dbgHeapTotal — nRF52 BSP heap diagnostics ─────────────────
// Used by memGet.cpp when ARCH_NRF52 is defined. Return 0 for Phase 2.
static inline uint32_t dbgHeapFree(void)
{
return 0;
}
static inline uint32_t dbgHeapTotal(void)
{
return 0;
}
// ── WCharacter helpers ───────────────────────────────────────────────────────
static inline bool isAlpha(char c)
{
return std::isalpha((unsigned char)c) != 0;
}
static inline bool isAlphaNumeric(char c)
{
return std::isalnum((unsigned char)c) != 0;
}
static inline bool isDigit(char c)
{
return std::isdigit((unsigned char)c) != 0;
}
static inline bool isSpace(char c)
{
return std::isspace((unsigned char)c) != 0;
}
static inline bool isUpperCase(char c)
{
return std::isupper((unsigned char)c) != 0;
}
static inline bool isLowerCase(char c)
{
return std::islower((unsigned char)c) != 0;
}
static inline char toUpperCase(char c)
{
return (char)std::toupper((unsigned char)c);
}
static inline char toLowerCase(char c)
{
return (char)std::tolower((unsigned char)c);
}
#else /* C only */
#ifndef min
#define min(a, b) ((a) < (b) ? (a) : (b))
#endif
#ifndef max
#define max(a, b) ((a) > (b) ? (a) : (b))
#endif
#ifndef abs
#define abs(x) ((x) >= 0 ? (x) : -(x))
#endif
#define constrain(x, l, h) ((x) < (l) ? (l) : ((x) > (h) ? (h) : (x)))
#define round(x) ((x) >= 0 ? (long)((x) + 0.5) : (long)((x)-0.5))
#endif /* __cplusplus */
#endif /* Arduino_h */

Some files were not shown because too many files have changed in this diff Show More