Compare commits

..
Author SHA1 Message Date
Jason P 7b912cecd8 Update PhoneAPI.cpp to reduce chattiness 2026-05-01 07:39:06 -05:00
504 changed files with 4351 additions and 33728 deletions
+1 -7
View File
@@ -49,17 +49,11 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
8. **Suggest next actions only for specific, recognisable failure modes**:
7. **Suggest next actions only for specific, recognisable failure modes**:
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
## What NOT to do
-103
View File
@@ -1,103 +0,0 @@
---
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
argument-hint: [window=1h] [field=free_heap] [variant=local]
---
<!-- markdownlint-disable MD029 -->
# `/leakhunt` — read the recorder, classify a memory leak
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
## Two signal paths — pick the right one
| Path | Build flag | Cadence | Per-thread attribution | Cost |
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
| LocalStats packet | (default) | ~60 s | No | Free — always on |
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
## What to do
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
- `running == True`
- `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
- `samples` — how many raw points contributed
- `min`, `max` — total swing
- `slope_per_min` — units per minute (linear regression over the whole window)
5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
- **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
8. **Report**:
```text
/leakhunt window=6h field=free_heap variant=local
────────────────────────────────────────────────────
recorder : running, telem last_ts 8s ago
build : DEBUG_HEAP=ON (per-line prefix detected)
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
free_heap : min 92,344 / max 124,008 / range 31,664
slope : -82 bytes/min (negative — heap declining)
reboots : none in window
OOM events : none
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
Router -1,408 B over 4 events
others -240 B
verdict : SLOW LEAK — primary suspect MeshPacket thread
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
evidence : (3 log line citations with uptimes)
```
Then: **what to do next.**
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
```text
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
```
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
- STEADY → say so; stop. Don't invent problems.
## What NOT to do
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
## Companion: `mark_event` for stress runs
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
```text
mark_event("burst-start")
… run the workload …
mark_event("burst-end")
/leakhunt window=15m
```
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
-4
View File
@@ -3,8 +3,6 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
argument-hint: <test-node-id> [count=5]
---
<!-- markdownlint-disable MD029 -->
# `/repro` — flakiness triage for one test
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -42,8 +40,6 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
6. **Classify the flake** into one of:
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
+3 -7
View File
@@ -8,21 +8,17 @@
"features": {
"ghcr.io/devcontainers/features/python:1": {
"installTools": true,
"version": "3.13"
"version": "3.14"
}
},
"customizations": {
"vscode": {
"extensions": [
"ms-vscode.cpptools",
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide",
"platformio.platformio-ide",
"Trunk.io"
],
"unwantedRecommendations": [
"ms-azuretools.vscode-docker",
"platformio.platformio-ide"
],
"unwantedRecommendations": ["ms-azuretools.vscode-docker"],
"settings": {
"extensions.ignoreRecommendations": true
}
-95
View File
@@ -135,99 +135,6 @@ On top of authorization, any remote admin message that **mutates** state (not a
- **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
- **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
## NodeDB Layout (v25)
`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
### Slim `NodeInfoLite`
`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
```cpp
nodeInfoLiteHasUser(n) // bit 5 — user fields populated
nodeInfoLiteIsFavorite(n) // bit 3
nodeInfoLiteIsIgnored(n) // bit 4
nodeInfoLiteIsMuted(n) // bit 1
nodeInfoLiteIsLicensed(n) // bit 6 — Ham mode peer
nodeInfoLiteIsKeyManuallyVerified(n) // bit 0
nodeInfoLiteHasIsUnmessagable(n) // bit 8 — "is_unmessagable was sent"
nodeInfoLiteIsUnmessagable(n) // bit 7
// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true); // setter
```
### Satellite stores
Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its own build flag:
| Map | Value type | Build flag |
| ----------------- | ------------------------------- | ---------------------------------- |
| `nodePositions` | `meshtastic_PositionLite` | `MESHTASTIC_EXCLUDE_POSITIONDB` |
| `nodeTelemetry` | `meshtastic_DeviceMetrics` | `MESHTASTIC_EXCLUDE_TELEMETRYDB` |
| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
| `nodeStatus` | `meshtastic_StatusMessage` | `MESHTASTIC_EXCLUDE_STATUSDB` |
Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
### Accessor convention
**Never hand out pointers into the maps.** Use the copy-out accessors on `NodeDB`:
```cpp
bool copyNodePosition(NodeNum, meshtastic_PositionLite &out) const;
bool copyNodeTelemetry(NodeNum, meshtastic_DeviceMetrics &out) const;
bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out) const;
```
Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
### Eviction
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
1. Config / module-config / channel / metadata segments (same as before).
2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
Special nonces that still mean something:
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
### v24 → v25 migration
The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not in `NodeDB.cpp`. It owns the `meshtastic_NodeDatabase_Legacy` callback and `NodeDB::migrateLegacyNodeDatabase()`. The legacy proto descriptor is `protobufs/meshtastic/deviceonly_legacy.proto` (only included by the migration TU). The boot path peeks the file's leading version tag, runs the migration if `version < 25`, then re-saves in v25 layout. The legacy descriptor is scheduled for removal once `DEVICESTATE_MIN_VER` is bumped.
### Read-site rules of thumb
- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
- Never `node->user.long_name` — `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
## Project Structure
```
@@ -289,8 +196,6 @@ firmware/
- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
- Use `assert()` for invariants that should never fail
- C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
### Naming Conventions
+3 -8
View File
@@ -32,15 +32,10 @@ jobs:
shell: bash
working-directory: meshtasticd
run: |
# Build-tools (notably platformio) come from the Meshtastic project
# on the OpenSUSE Build Service:
# https://build.opensuse.org/project/show/network:Meshtastic:build-tools
echo 'deb http://download.opensuse.org/repositories/network:/Meshtastic:/build-tools/xUbuntu_24.04/ /' \
| sudo tee /etc/apt/sources.list.d/network:Meshtastic:build-tools.list
curl -fsSL https://download.opensuse.org/repositories/network:Meshtastic:build-tools/xUbuntu_24.04/Release.key \
| gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/network_Meshtastic_build-tools.gpg >/dev/null
sudo apt-get update -y --fix-missing
sudo apt-get install -y build-essential devscripts equivs
sudo apt-get install -y software-properties-common build-essential devscripts equivs
sudo add-apt-repository ppa:meshtastic/build-tools -y
sudo apt-get update -y --fix-missing
sudo mk-build-deps --install --remove --tool='apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes' debian/control
- name: Import GPG key
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
shell: bash
run: |
brew update
brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config ulfius
brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config
- name: Get release version string
run: |
+25 -39
View File
@@ -4,14 +4,9 @@ on:
workflow_dispatch:
inputs:
# trunk-ignore(checkov/CKV_GHA_7)
target:
type: string
required: false
description: Choose the target board, e.g. nrf52_promicro_diy_tcxo. If blank, will find available targets.
arch:
type: choice
options:
- all
- esp32
- esp32s3
- esp32c3
@@ -20,18 +15,32 @@ on:
- rp2040
- rp2350
- stm32
description: Choose an arch to limit the search, or 'all' to search all architectures.
default: all
target:
type: string
required: false
description: Choose the target board, e.g. nrf52_promicro_diy_tcxo. If blank, will find available targets.
# find-target:
# type: boolean
# default: true
# description: 'Find the available targets'
permissions: read-all
jobs:
find-targets:
if: ${{ inputs.target == '' }}
strategy:
fail-fast: false
matrix:
arch:
- all
- esp32
- esp32s3
- esp32c3
- esp32c6
- nrf52840
- rp2040
- rp2350
- stm32
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
@@ -42,37 +51,14 @@ jobs:
- run: pip install -U platformio
- name: Generate matrix
id: jsonStep
env:
BUILDTARGET: ${{ inputs.target }}
MATRIXARCH: ${{ inputs.arch }}
run: |
TARGETS=$(./bin/generate_ci_matrix.py ${{matrix.arch}} --level extra)
if [ "$BUILDTARGET" = "" ]; then
echo "Name: $GITHUB_REF_NAME" >> $GITHUB_STEP_SUMMARY
echo "Base: $GITHUB_BASE_REF" >> $GITHUB_STEP_SUMMARY
echo "Arch: $MATRIXARCH" >> $GITHUB_STEP_SUMMARY
echo "Ref: $GITHUB_REF" >> $GITHUB_STEP_SUMMARY
echo "## 🎯 The following target boards are available to build:" >> $GITHUB_STEP_SUMMARY
echo "| Platform | Board |" >> $GITHUB_STEP_SUMMARY
echo "| -------- | ----- |" >> $GITHUB_STEP_SUMMARY
echo $TARGETS | jq -r 'sort_by(.board) | sort_by(.platform) |.[] | "| " + .platform + " | " + .board + " |" ' >> $GITHUB_STEP_SUMMARY
else
echo "We build this one:" >> $GITHUB_STEP_SUMMARY
ARCH=$(echo "$TARGETS" | jq --arg BUILDTARGET "$BUILDTARGET" -r '.[] | select(.board==$BUILDTARGET) | .platform')
echo "| Platform | Board |" >> $GITHUB_STEP_SUMMARY
echo "| -------- | ----- |" >> $GITHUB_STEP_SUMMARY
echo "| $ARCH | "$BUILDTARGET" |" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [[ "$ARCH" == "" ]]; then
echo "## ❌ Error: Target "$BUILDTARGET" not found!" >> $GITHUB_STEP_SUMMARY
else
echo "## ✅ Target "$BUILDTARGET" found, proceeding to build." >> $GITHUB_STEP_SUMMARY
fi
echo "You may need to refresh this page to make the built firmware appear below." >> $GITHUB_STEP_SUMMARY
echo "arch=$ARCH" >> $GITHUB_OUTPUT
fi
outputs:
arch: ${{ steps.jsonStep.outputs.arch }}
echo "Name: $GITHUB_REF_NAME" >> $GITHUB_STEP_SUMMARY
echo "Base: $GITHUB_BASE_REF" >> $GITHUB_STEP_SUMMARY
echo "Arch: ${{matrix.arch}}" >> $GITHUB_STEP_SUMMARY
echo "Ref: $GITHUB_REF" >> $GITHUB_STEP_SUMMARY
echo "Targets:" >> $GITHUB_STEP_SUMMARY
echo $TARGETS | jq -r 'sort_by(.board) |.[] | "- " + .board' >> $GITHUB_STEP_SUMMARY
version:
if: ${{ inputs.target != '' }}
@@ -92,12 +78,12 @@ jobs:
build:
if: ${{ inputs.target != '' && inputs.arch != 'native' }}
needs: [version, find-targets]
needs: [version]
uses: ./.github/workflows/build_firmware.yml
with:
version: ${{ needs.version.outputs.long }}
pio_env: ${{ inputs.target }}
platform: ${{ needs.find-targets.outputs.arch }}
platform: ${{ inputs.arch }}
gather-artifacts:
permissions:
+1 -7
View File
@@ -86,13 +86,7 @@ jobs:
run: sed -i 's/-DBUILD_EPOCH=$UNIX_TIME/#-DBUILD_EPOCH=$UNIX_TIME/' platformio.ini
- name: PlatformIO Tests
run: |
set -o pipefail
# Filter out SKIPPED summary rows for hardware variants that can't run on the
# native host. They flood the log and make it harder to spot real failures.
# The JUnit XML is written directly to testreport.xml before the pipe, so
# the test artifact is unaffected.
platformio test -e coverage -v --junit-output-path testreport.xml 2>&1 | grep -v "[[:space:]]SKIPPED$"
run: platformio test -e coverage -v --junit-output-path testreport.xml
- name: Save test results
if: always() # run this step even if previous step failed
-10
View File
@@ -47,10 +47,6 @@ data/boot/logo.*
managed_components/*
arduino-lib-builder*
dependencies.lock
# JLink / RTT debug artifacts (nRF SoCs)
flash.jlink
rtt_*.txt
idf_component.yml
CMakeLists.txt
/sdkconfig.*
@@ -60,9 +56,3 @@ CMakeLists.txt
.python3
.claude/scheduled_tasks.lock
userPrefs.jsonc.mcp-session-bak
# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
# compiled .proto outputs are ephemeral build artifacts.
build/fixtures/
bin/_generated/
+5 -12
View File
@@ -4,19 +4,19 @@ cli:
plugins:
sources:
- id: trunk
ref: v1.10.0
ref: v1.8.0
uri: https://github.com/trunk-io/plugins
lint:
enabled:
- checkov@3.2.529
- checkov@3.2.525
- renovate@43.150.0
- prettier@3.8.3
- trufflehog@3.95.3
- trufflehog@3.95.2
- yamllint@1.38.0
- bandit@1.9.4
- trivy@0.70.0
- taplo@0.10.0
- ruff@0.15.13
- ruff@0.15.12
- isort@8.0.1
- markdownlint@0.48.0
- oxipng@10.1.1
@@ -26,7 +26,7 @@ lint:
- hadolint@2.14.0
- shfmt@3.6.0
- shellcheck@0.11.0
- black@26.5.1
- black@26.3.1
- git-diff-check
- gitleaks@8.30.1
- clang-format@16.0.3
@@ -34,13 +34,6 @@ lint:
- linters: [ALL]
paths:
- bin/**
# Fake-NodeDB fixture JSONL files contain deterministic synthetic
# public_key_hex (64-char hex) values that gitleaks misidentifies as
# generic-api-key. These are not secrets — they're test fixtures
# produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
- linters: [gitleaks]
paths:
- test/fixtures/nodedb/seed_v25_*.jsonl
runtimes:
enabled:
- python@3.14.4
+4 -4
View File
@@ -1,10 +1,10 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide"
"platformio.platformio-ide"
],
"unwantedRecommendations": [
"ms-vscode.cpptools-extension-pack",
"platformio.platformio-ide"
"ms-vscode.cpptools-extension-pack"
]
}
-2
View File
@@ -66,8 +66,6 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
- **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
- **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
## Typical agent workflows
-64
View File
@@ -1,64 +0,0 @@
#!/usr/bin/env python3
"""Post-process protoc-generated Python files to live under a local namespace.
Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
target directory and rewrites every `meshtastic` reference (imports, dotted
attribute access) to use the new namespace (e.g., `meshtastic_v25`).
Why: the .proto files declare `package meshtastic;`, so protoc emits
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
`meshtastic` package which other parts of the mcp-server depend on. Renaming
to a local namespace keeps both available.
Usage:
_rewrite_proto_namespace.py <generated_dir> <new_namespace>
"""
from __future__ import annotations
import pathlib
import re
import sys
def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
# Standard protoc import forms:
# from meshtastic.X_pb2 import ... (rare, for direct symbol pulls)
# from meshtastic import X_pb2 as ... (common, the cross-file ref)
# import meshtastic.X_pb2 (also possible)
pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
count = 0
for p in dir_path.glob("*.py"):
text = p.read_text(encoding="utf-8")
new = pattern_dotted_from.sub(f"from {new_ns}.", text)
new = pattern_bare_from.sub(f"from {new_ns} import ", new)
new = pattern_dotted_import.sub(f"import {new_ns}.", new)
# NOTE: we deliberately leave `meshtastic/X.proto` source-filename
# references inside descriptor strings alone. The descriptor pool is
# keyed by source filename (independent of Python package layout), so
# those don't collide with the PyPI package's descriptors.
if new != text:
p.write_text(new, encoding="utf-8")
count += 1
return count
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
return 2
dir_path = pathlib.Path(argv[0])
new_ns = argv[1]
if not dir_path.is_dir():
print(f"directory not found: {dir_path}", file=sys.stderr)
return 2
n = rewrite(dir_path, new_ns)
print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+1 -1
View File
@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
cp bin/device-update.* $OUTDIR/
echo "Copying manifest"
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true
@@ -1,30 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins: [DIO5, DIO6, DIO7]
MODE_STBY: [LOW, LOW, LOW]
MODE_RX: [LOW, HIGH, LOW]
MODE_TX: [HIGH, HIGH, LOW]
MODE_TX_HP: [HIGH, LOW, LOW]
MODE_TX_HF: [LOW, LOW, LOW]
MODE_GNSS: [LOW, LOW, HIGH]
MODE_WIFI: [LOW, LOW, LOW]
General:
MACAddressSource: eth0
@@ -1,46 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins:
- DIO5
- DIO6
MODE_STBY:
- LOW
- LOW
MODE_RX:
- HIGH
- LOW
MODE_TX:
- HIGH
- HIGH
MODE_TX_HP:
- LOW
- HIGH
MODE_TX_HF:
- LOW
- LOW
MODE_GNSS:
- LOW
- LOW
MODE_WIFI:
- LOW
- LOW
General:
MACAddressSource: eth0
@@ -1,30 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins: [DIO5, DIO6, DIO7]
MODE_STBY: [LOW, LOW, LOW]
MODE_RX: [LOW, LOW, LOW]
MODE_TX: [LOW, HIGH, LOW]
MODE_TX_HP: [HIGH, LOW, LOW]
# MODE_TX_HF: []
# MODE_GNSS: []
MODE_WIFI: [LOW, LOW, LOW]
General:
MACAddressSource: eth0
-18
View File
@@ -1,18 +0,0 @@
# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
# Verify spidev / I2C device paths for your OS — they may differ.
Meta:
name: Station G3
support: community
compatible:
- raspberry-pi
Lora:
Module: sx1262
IRQ: 22 # BCM pin — wiki spec
Reset: 16 # BCM pin — wiki spec
Busy: 24 # BCM pin — wiki spec
# CS: 8 # BCM 8 = SPI0 CE0 (default); uncomment only to override
DIO2_AS_RF_SWITCH: true
DIO3_TCXO_VOLTAGE: true
spidev: spidev0.0
# SX126X_MAX_POWER: 19 # matches Station G2 firmware cap; raise carefully per PA jumper mode
-439
View File
@@ -1,439 +0,0 @@
#!/usr/bin/env python3
"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
The output is hand-editable and committed; a sibling compile step
(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
v25 protobuf with fresh "now-relative" timestamps.
Determinism contract:
Same --seed -> byte-identical JSONL output, regardless of wall clock.
All timestamps are stored as `*_offset_sec` (seconds before "now"); the
compile step resolves them to absolute epochs at compile time.
Structural fields covered:
* NodeInfoLite header: num, long_name, short_name, hw_model, role,
public_key, snr, channel, hops_away, next_hop, bitfield flags
* PositionLite: lat/long Gaussian around --centroid, altitude, source
* DeviceMetrics: battery/voltage/util/uptime
* EnvironmentMetrics: temp/humidity/pressure/iaq
* StatusMessage: error_code (usually zero)
Active-board allow-list:
hw_model values are restricted to the intersection of
(a) variants with `custom_meshtastic_support_level = 1` in
variants/*/*/platformio.ini, AND
(b) values present in the `HardwareModel` enum in mesh.proto.
See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
(PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
Active-role allow-list:
Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
"""
from __future__ import annotations
import argparse
import datetime as _dt
import json
import math
import pathlib
import random
import sys
# --------------------------------------------------------------------------
# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
# Refresh by running:
# for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
# grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
# done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
# --------------------------------------------------------------------------
HW_MODEL_WEIGHTS: dict[str, float] = {
"HELTEC_V3": 14.0,
"T_DECK": 9.0,
"HELTEC_V4": 8.0,
"RAK4631": 8.0,
"HELTEC_MESH_POCKET": 6.0,
"TRACKER_T1000_E": 5.0,
"HELTEC_MESH_NODE_T114": 5.0,
"T_DECK_PRO": 5.0,
"LILYGO_TBEAM_S3_CORE": 4.0,
"HELTEC_WIRELESS_PAPER": 4.0,
"HELTEC_WSL_V3": 3.0,
"T_ECHO": 3.0,
"HELTEC_WIRELESS_TRACKER": 3.0,
"HELTEC_WIRELESS_TRACKER_V2": 2.0,
"HELTEC_VISION_MASTER_E290": 2.0,
"HELTEC_MESH_SOLAR": 2.0,
"SEEED_WIO_TRACKER_L1": 2.0,
"T_LORA_PAGER": 1.5,
"HELTEC_VISION_MASTER_E213": 1.5,
"T_ECHO_PLUS": 1.0,
"MUZI_BASE": 1.0,
"WISMESH_TAP_V2": 1.0,
"THINKNODE_M2": 1.0,
"THINKNODE_M5": 1.0,
"TLORA_T3_S3": 1.0,
# Long tail (uniform low weight across remaining tier-1 boards):
"HELTEC_V4_R8": 0.3,
"HELTEC_VISION_MASTER_T190": 0.3,
"HELTEC_HT62": 0.3,
"HELTEC_MESH_NODE_T096": 0.3,
"M5STACK_C6L": 0.3,
"MINI_EPAPER_S3": 0.3,
"MUZI_R1_NEO": 0.3,
"NOMADSTAR_METEOR_PRO": 0.3,
"RAK3312": 0.3,
"RAK3401": 0.3,
"SEEED_SOLAR_NODE": 0.3,
"SEEED_WIO_TRACKER_L1_EINK": 0.3,
"SENSECAP_INDICATOR": 0.3,
"TBEAM_1_WATT": 0.3,
"THINKNODE_M1": 0.3,
"THINKNODE_M3": 0.3,
"THINKNODE_M6": 0.3,
"T_ECHO_LITE": 0.3,
"WISMESH_TAG": 0.3,
"WISMESH_TAP": 0.3,
"XIAO_NRF52_KIT": 0.3,
"CROWPANEL": 0.3,
}
# Non-deprecated roles only.
ROLE_WEIGHTS: dict[str, float] = {
"CLIENT": 75.0,
"CLIENT_MUTE": 5.0,
"ROUTER": 7.0,
"TRACKER": 3.0,
"SENSOR": 2.0,
"CLIENT_HIDDEN": 2.0,
"ROUTER_LATE": 2.0,
"CLIENT_BASE": 2.0,
"TAK": 1.0,
"TAK_TRACKER": 0.5,
"LOST_AND_FOUND": 0.5,
}
# Name pools — 60 firsts × 60 lasts = 3600 combinations.
FIRSTS = [
"Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
"Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
"Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
"Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
"Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
"Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
"Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
"Burning", "Frozen", "Whispering", "Howling",
]
LASTS = [
"Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
"Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
"Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
"Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
"Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
"Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
"Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
"Bluff", "Mesa", "Arroyo", "Ridge",
]
# Brief callsign pool for licensed-looking suffixes.
CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
# 6 bytes due to U+FE0F variation selector — explicitly excluded.
EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
"🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
"🦇", "🦋"]
# --------------------------------------------------------------------------
# Helpers
# --------------------------------------------------------------------------
NUM_RESERVED = 4 # firmware reserves 0..3 (per NodeDB constants)
NUM_MAX_EXCLUSIVE = 0x80000000 # restrict to positive int32 range for readability
def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
"""Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
keys = sorted(weights.keys())
totals = [weights[k] for k in keys]
return rng.choices(keys, weights=totals, k=1)[0]
def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
if is_licensed:
prefix = rng.choice(CALLSIGN_PREFIXES)
# Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
# nanopb max_size:25 means C string fits 24 bytes + NUL.
if len(base) + len(suffix) <= 24:
base = base + suffix
# Hard cap to 24 chars (nanopb max_size:25 minus NUL).
return base[:24]
def _gen_short_name(rng: random.Random, long_name: str) -> str:
# 10% emoji-only short_name
if rng.random() < 0.10:
return rng.choice(EMOJI_SHORTNAMES)
first_char = long_name[0].upper() if long_name else "X"
alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
return first_char + "".join(rng.choices(alphanums, k=3))
def _gen_hops_away(rng: random.Random) -> int:
# Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
r = rng.random()
if r < 0.55:
return 0
if r < 0.80:
return 1
if r < 0.92:
return 2
if r < 0.97:
return 3
if r < 0.99:
return 4
return rng.randint(5, 7)
def _gen_position(
rng: random.Random,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
last_heard_offset_sec: int,
) -> dict:
# 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
altitude = max(0, round(rng.gauss(1376.0, 250.0))) # T or C valley floor + relief
# Position was reported up to 300s before last_heard.
time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
return {
"latitude": round(lat, 6),
"longitude": round(lon, 6),
"altitude": altitude,
"time_offset_sec": time_offset_sec,
"location_source": "LOC_INTERNAL",
}
def _gen_telemetry(rng: random.Random) -> dict:
# 5% plugged-in (battery_level == 101); rest uniform [10..100].
if rng.random() < 0.05:
battery_level = 101
voltage = 4.20
else:
battery_level = rng.randint(10, 100)
voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
# Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
def _beta(a: float, b: float) -> float:
x = rng.gammavariate(a, 1.0)
y = rng.gammavariate(b, 1.0)
return x / (x + y)
channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
return {
"battery_level": battery_level,
"voltage": voltage,
"channel_utilization": channel_utilization,
"air_util_tx": air_util_tx,
"uptime_seconds": uptime_seconds,
}
def _gen_environment(rng: random.Random) -> dict:
return {
"temperature": round(rng.gauss(22.0, 8.0), 2),
"relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
"barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
"iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
}
def _gen_status(rng: random.Random) -> dict:
# `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
# Most peers report a healthy short status; occasional alert string.
healthy = ["OK", "online", "active", "running", "ready", "nominal"]
alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
if rng.random() < 0.92:
return {"status": rng.choice(healthy)}
return {"status": rng.choice(alert)}
def _gen_node(
rng: random.Random,
num: int,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
coverage: dict[str, float],
last_heard_mean_sec: int,
last_heard_max_sec: int,
) -> dict:
is_licensed = rng.random() < 0.05
long_name = _gen_long_name(rng, is_licensed)
short_name = _gen_short_name(rng, long_name)
hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
role = _weighted_choice(rng, ROLE_WEIGHTS)
has_public_key = rng.random() < 0.92
public_key_hex = (
"".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
)
snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
hops_away = _gen_hops_away(rng)
next_hop = rng.randint(0, 255) if hops_away > 0 else 0
last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
bitfield = {
"has_user": True,
"is_favorite": rng.random() < 0.08,
"is_muted": rng.random() < 0.03,
"via_mqtt": rng.random() < 0.12,
"is_ignored": rng.random() < 0.01,
"is_licensed": is_licensed,
"has_is_unmessagable": True,
"is_unmessagable": rng.random() < 0.02,
"is_key_manually_verified": rng.random() < 0.04,
}
node: dict = {
"num": f"0x{num:08x}",
"long_name": long_name,
"short_name": short_name,
"hw_model": hw_model,
"role": role,
"public_key_hex": public_key_hex,
"snr": snr,
"channel": channel,
"hops_away": hops_away,
"next_hop": next_hop,
"last_heard_offset_sec": last_heard_offset_sec,
"bitfield": bitfield,
"position": (
_gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
if rng.random() < coverage["position"]
else None
),
"telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
"environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
"status": _gen_status(rng) if rng.random() < coverage["status"] else None,
}
return node
def _parse_my_node_num(s: str | None) -> int | None:
if s is None:
return None
s = s.strip()
if s.startswith("0x") or s.startswith("0X"):
return int(s, 16)
return int(s)
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Deterministic JSONL seed for the fake NodeDB fixture.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
p.add_argument("--out", required=True, help="Output JSONL path.")
p.add_argument(
"--centroid",
default="33.1284,-107.2528",
help="LAT,LON centroid (default: Truth or Consequences, NM).",
)
p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
p.add_argument("--position-coverage", type=float, default=0.85)
p.add_argument("--telemetry-coverage", type=float, default=0.70)
p.add_argument("--environment-coverage", type=float, default=0.25)
p.add_argument("--status-coverage", type=float, default=0.40)
p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
p.add_argument("--last-heard-mean-sec", type=int, default=3600)
p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
args = p.parse_args(argv)
if args.count <= 0:
print("--count must be positive", file=sys.stderr)
return 2
try:
centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
except ValueError:
print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
return 2
my_node_num = _parse_my_node_num(args.my_node_num)
rng = random.Random(args.seed)
# 1) Generate a unique deterministic set of NodeNums.
nums: set[int] = set()
while len(nums) < args.count:
n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
if my_node_num is not None and n == my_node_num:
continue
nums.add(n)
ordered_nums = sorted(nums) # sort to fix output order independent of set hash
# 2) Per-node generation (in num order, single RNG continues).
coverage = {
"position": args.position_coverage,
"telemetry": args.telemetry_coverage,
"environment": args.environment_coverage,
"status": args.status_coverage,
}
nodes = [
_gen_node(
rng,
n,
centroid_lat,
centroid_lon,
args.spread_km,
coverage,
args.last_heard_mean_sec,
args.last_heard_max_sec,
)
for n in ordered_nums
]
# 3) Write JSONL.
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
# `generated_at_iso` is informational; it does NOT affect determinism because
# we derive it from the seed, not from wall clock. (Same seed -> same string.)
generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
meta = {
"_meta": {
"version": 25,
"seed": args.seed,
"count": args.count,
"centroid": [centroid_lat, centroid_lon],
"spread_km": args.spread_km,
"generated_at_iso": generated_at,
"my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
"coverage": coverage,
"last_heard_mean_sec": args.last_heard_mean_sec,
"last_heard_max_sec": args.last_heard_max_sec,
}
}
with out_path.open("w", encoding="utf-8") as f:
# `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
# determinism (insertion order varies by Python version otherwise).
f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
for node in nodes:
f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -87,9 +87,6 @@
</screenshots>
<releases>
<release version="2.7.24" date="2026-05-08">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.24</url>
</release>
<release version="2.7.23" date="2026-04-14">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.23</url>
</release>
+4 -8
View File
@@ -293,12 +293,9 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
board_arch = infer_architecture(env.BoardConfig())
should_skip_manifest = board_arch is None
# Most platforms can generate the manifest as part of the default 'buildprog' target.
# Typically this passes success/failure properly.
mtjson_deps = ["buildprog"]
if platform.name == "espressif32":
# On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
mtjson_deps = [] if should_skip_manifest else ["buildprog"]
if not should_skip_manifest and platform.name == "espressif32":
# Build littlefs image as part of mtjson target
# Equivalent to `pio run -t buildfs`
target_lfs = env.DataToBin(
@@ -312,8 +309,7 @@ if should_skip_manifest:
env.AddCustomTarget(
name="mtjson",
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
dependencies=[],
dependencies=mtjson_deps,
actions=[skip_manifest],
title="Meshtastic Manifest (skipped)",
description="mtjson generation is skipped for native environments",
-73
View File
@@ -1,73 +0,0 @@
#!/usr/bin/env bash
# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
# JSONL seed files + their compiled v25 protobufs.
#
# Layout:
# test/fixtures/nodedb/seed_v25_<N>.jsonl — COMMITTED, hand-editable.
# build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
# Drop into /prefs/nodes.proto.
#
# Daily use: ./bin/regen-fake-nodedbs.sh
# - Recompiles protos from committed seeds (fresh wall-clock timestamps).
# Intentional seed bump: REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
# - Overwrites the committed JSONL files with freshly-seeded data.
set -euo pipefail
cd "$(dirname "$0")/.."
# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
if [[ ! -d bin/_generated/meshtastic ]]; then
echo "regenerating Python protobuf bindings (one-time)..."
./bin/regen-py-protos.sh
fi
# 2) Pick a Python interpreter that has the meshtastic deps installed.
# Prefer the mcp-server venv (most likely to be set up by the operator).
PY="python3"
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
if [[ -x "$cand" ]]; then
PY="$cand"
break
fi
done
# 3) Pinned seeds per size — bump only when you intentionally want different
# structural data committed. Parallel arrays so the script works on
# macOS bash 3.2 (no `declare -A`).
SIZES=(250 500 1000 2000)
SEEDS=(20260511 20260512 20260513 20260514)
REGEN_SEEDS="${REGEN_SEEDS:-no}"
mkdir -p build/fixtures/nodedb test/fixtures/nodedb
for i in 0 1 2 3; do
n="${SIZES[$i]}"
seed="${SEEDS[$i]}"
jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
$PY bin/gen-fake-nodedb-seed.py \
--count "$n" \
--seed "$seed" \
--out "$jsonl" \
--centroid 33.1284,-107.2528 \
--spread-km 60 \
--position-coverage 0.85 \
--telemetry-coverage 0.70 \
--environment-coverage 0.25 \
--status-coverage 0.40
echo " seed: $jsonl ($(wc -c < "$jsonl") bytes)"
fi
$PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
echo " proto: $proto ($(wc -c < "$proto") bytes)"
done
echo ""
echo "Done. To load on Portduino native:"
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
echo ""
echo "To push to a hardware device:"
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
-51
View File
@@ -1,51 +0,0 @@
#!/usr/bin/env bash
# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
# a standalone refresh after any change to a .proto file.
#
# Output is .gitignored — bindings are a build artifact.
#
# Namespace rewrite:
# The .proto files declare `package meshtastic;`, which makes protoc emit
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
# PyPI `meshtastic` package (which the mcp-server relies on for its
# SerialInterface/BLEInterface transport). We post-process the generated
# files to live under `meshtastic_v25` instead — both the directory layout
# and all internal imports — so they coexist cleanly with the PyPI package.
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v protoc >/dev/null 2>&1; then
echo "ERROR: protoc not found in PATH." >&2
echo " macOS: brew install protobuf" >&2
echo " Ubuntu/Debian: apt install protobuf-compiler" >&2
exit 1
fi
OUT=bin/_generated
LOCAL_NS=meshtastic_v25
rm -rf "$OUT"
mkdir -p "$OUT"
# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
# is available for the [(nanopb).*] options on other messages.
protoc \
--proto_path=protobufs \
--python_out="$OUT" \
protobufs/nanopb.proto \
protobufs/meshtastic/*.proto
# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
# `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
# 4) Make the package importable.
touch "$OUT/__init__.py"
touch "$OUT/$LOCAL_NS/__init__.py"
echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2
-342
View File
@@ -1,342 +0,0 @@
#!/usr/bin/env python3
"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
are stored as `*_offset_sec` (seconds before "now"); this script resolves them
to absolute epochs using `--now-epoch` (default: current wall clock).
Output is a raw `pb_encode`-compatible binary that can be dropped at
`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
XModem) and loaded by `NodeDB::loadFromDisk` at boot.
Wire format reference:
protobufs/meshtastic/deviceonly.proto (NodeDatabase, NodeInfoLite, sat entries)
src/mesh/NodeDB.h:467-484 (bitfield bit positions)
src/mesh/NodeDB.cpp:1523-1524 (pb_decode entry point)
"""
from __future__ import annotations
import argparse
import json
import pathlib
import sys
import time
from typing import Any
# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
# ships. Run `bin/regen-py-protos.sh` to (re)generate.
#
# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
# post-processes the protoc output to rename the package.
_HERE = pathlib.Path(__file__).resolve().parent
_LOCAL_PROTO_DIR = _HERE / "_generated"
if _LOCAL_PROTO_DIR.is_dir():
sys.path.insert(0, str(_LOCAL_PROTO_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import ( # type: ignore[import-not-found]
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage # type: ignore[import-not-found]
from meshtastic_v25.config_pb2 import Config # type: ignore[import-not-found]
from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics # type: ignore[import-not-found]
except ImportError as local_err:
# Fall back to the PyPI package if in-tree bindings haven't been generated.
# Will fail the v25 assertion below if the PyPI package predates the
# satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
# error message instead of an opaque ImportError.
try:
from meshtastic.protobuf.deviceonly_pb2 import (
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
from meshtastic.protobuf.config_pb2 import Config
from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
except ImportError as pypi_err:
print(
"ERROR: could not import meshtastic protobuf bindings.\n"
" In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
" PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
f" local error (meshtastic_v25): {local_err}\n"
f" pypi error (meshtastic.protobuf): {pypi_err}",
file=sys.stderr,
)
sys.exit(1)
# Fail loudly if bindings predate v25 (no satellite arrays).
assert (
hasattr(NodeDatabase, "DESCRIPTOR")
and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
), (
"Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
"Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
)
# ---------------------------------------------------------------------------
# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
# ---------------------------------------------------------------------------
BIT_IS_KEY_MANUALLY_VERIFIED = 0
BIT_IS_MUTED = 1
BIT_VIA_MQTT = 2
BIT_IS_FAVORITE = 3
BIT_IS_IGNORED = 4
BIT_HAS_USER = 5
BIT_IS_LICENSED = 6
BIT_IS_UNMESSAGABLE = 7
BIT_HAS_IS_UNMESSAGABLE = 8
BITFIELD_LAYOUT = (
# JSON key bit position
("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
("is_muted", BIT_IS_MUTED),
("via_mqtt", BIT_VIA_MQTT),
("is_favorite", BIT_IS_FAVORITE),
("is_ignored", BIT_IS_IGNORED),
("has_user", BIT_HAS_USER),
("is_licensed", BIT_IS_LICENSED),
("is_unmessagable", BIT_IS_UNMESSAGABLE),
("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
)
def _pack_bitfield(bf: dict[str, bool]) -> int:
out = 0
for key, shift in BITFIELD_LAYOUT:
if bf.get(key, False):
out |= (1 << shift)
return out
def _validate_node(node: dict[str, Any]) -> None:
"""Friendly errors so hand-editors get clear feedback."""
if "num" not in node or not isinstance(node["num"], str):
raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
if "long_name" not in node:
raise ValueError(f"node {node['num']}: missing 'long_name'")
if len(node["long_name"]) > 24:
raise ValueError(
f"node {node['num']}: long_name {node['long_name']!r} is "
f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
)
if "short_name" in node:
# short_name max_size:5 (incl. NUL) → 4 bytes of content.
# Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
# would slip past a `len(str) > 4` check. Always measure bytes.
b = node["short_name"].encode("utf-8")
if len(b) > 4:
raise ValueError(
f"node {node['num']}: short_name {node['short_name']!r} is "
f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
)
pk = node.get("public_key_hex", "")
if pk and len(pk) != 64:
raise ValueError(
f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
f"got {len(pk)} chars"
)
if pk:
try:
bytes.fromhex(pk)
except ValueError as e:
raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
def _resolve_time(
node: dict[str, Any],
field_absolute: str,
field_offset: str,
now_epoch: int,
) -> int:
"""If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
if field_absolute in node and node[field_absolute] is not None:
return int(node[field_absolute])
offset = node.get(field_offset, 0)
return max(0, int(now_epoch) - int(offset))
def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
_validate_node(node)
info = NodeInfoLite()
info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
info.long_name = node.get("long_name", "")
info.short_name = node.get("short_name", "")
# Enum lookups will raise ValueError on unknown names — that's exactly what we want.
info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
pk_hex = node.get("public_key_hex", "")
if pk_hex:
info.public_key = bytes.fromhex(pk_hex)
info.snr = float(node.get("snr", 0.0))
info.channel = int(node.get("channel", 0))
if "hops_away" in node:
# `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
# field implicitly sets HasField("hops_away") to True. No has_hops_away
# setter exists (unlike the C++ nanopb-generated header).
info.hops_away = int(node["hops_away"])
info.next_hop = int(node.get("next_hop", 0))
info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
info.bitfield = _pack_bitfield(node.get("bitfield", {}))
return info
def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
entry = NodePositionEntry()
entry.num = num
pl = PositionLite()
# Firmware stores lat/long as int32 in 1e-7 degrees.
pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
pl.altitude = int(pos.get("altitude", 0))
pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
entry.position.CopyFrom(pl)
return entry
def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
entry = NodeTelemetryEntry()
entry.num = num
dm = DeviceMetrics()
if "battery_level" in tel:
dm.battery_level = int(tel["battery_level"])
if "voltage" in tel:
dm.voltage = float(tel["voltage"])
if "channel_utilization" in tel:
dm.channel_utilization = float(tel["channel_utilization"])
if "air_util_tx" in tel:
dm.air_util_tx = float(tel["air_util_tx"])
if "uptime_seconds" in tel:
dm.uptime_seconds = int(tel["uptime_seconds"])
entry.device_metrics.CopyFrom(dm)
return entry
def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
entry = NodeEnvironmentEntry()
entry.num = num
em = EnvironmentMetrics()
if "temperature" in env:
em.temperature = float(env["temperature"])
if "relative_humidity" in env:
em.relative_humidity = float(env["relative_humidity"])
if "barometric_pressure" in env:
em.barometric_pressure = float(env["barometric_pressure"])
if "iaq" in env:
em.iaq = int(env["iaq"])
entry.environment_metrics.CopyFrom(em)
return entry
def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
# `StatusMessage` (mesh.proto:1445) has a single `string status` field.
entry = NodeStatusEntry()
entry.num = num
sm = StatusMessage()
if "status" in status:
sm.status = str(status["status"])
entry.status.CopyFrom(sm)
return entry
def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
"""Read a seed JSONL and return the encoded NodeDatabase bytes."""
lines = jsonl_path.read_text(encoding="utf-8").splitlines()
if not lines:
raise ValueError(f"{jsonl_path} is empty")
meta_line = lines[0]
meta_obj = json.loads(meta_line)
meta = meta_obj.get("_meta", {})
version = meta.get("version")
if version != 25:
raise ValueError(
f"{jsonl_path}: meta version is {version!r}; this compiler "
f"requires version=25. Regenerate the seed with the matching tooling."
)
db = NodeDatabase()
db.version = 25
for ln, raw in enumerate(lines[1:], start=2):
raw = raw.strip()
if not raw:
continue
try:
node = json.loads(raw)
except json.JSONDecodeError as e:
raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
# Header
info = _build_node_info_lite(node, now_epoch)
db.nodes.append(info)
# Satellites (nullable)
if node.get("position"):
db.positions.append(_build_position_entry(num, node["position"], now_epoch))
if node.get("telemetry"):
db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
if node.get("environment"):
db.environment.append(_build_environment_entry(num, node["environment"]))
if node.get("status"):
db.status.append(_build_status_entry(num, node["status"]))
return db.SerializeToString()
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
p.add_argument("--out", required=True, help="Output binary .proto path.")
p.add_argument(
"--now-epoch",
type=int,
default=None,
help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
)
args = p.parse_args(argv)
in_path = pathlib.Path(args.in_path)
if not in_path.is_file():
print(f"input not found: {in_path}", file=sys.stderr)
return 2
now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
try:
encoded = compile_jsonl_to_proto(in_path, now_epoch)
except ValueError as e:
print(f"ERROR: {e}", file=sys.stderr)
return 3
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(encoded)
print(
f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-D CDEBYTE_EORA_S3",
"-D ARDUINO_USB_CDC_ON_BOOT=1",
"-D ARDUINO_USB_MODE=1",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=1",
"-D BOARD_HAS_PSRAM"
-42
View File
@@ -1,42 +0,0 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-D BOARD_HAS_PSRAM",
"-D ARDUINO_USB_CDC_ON_BOOT=0",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"psram_type": "qio_opi",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "ELECROW-ThinkNode-M7"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "ELECROW ThinkNode M7",
"upload": {
"flash_size": "8MB",
"maximum_ram_size": 524288,
"maximum_size": 8388608,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "https://www.elecrow.com",
"vendor": "ELECROW"
}
+1 -1
View File
@@ -6,7 +6,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DHELTEC_WIRELESS_TRACKER",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
-26
View File
@@ -1,26 +0,0 @@
{
"build": {
"cpu": "cortex-m33",
"f_cpu": "128000000L",
"mcu": "nrf54l15",
"zephyr": {
"variant": "nrf54l15dk/nrf54l15/cpuapp"
}
},
"connectivity": ["bluetooth"],
"debug": {
"default_tools": ["jlink"],
"jlink_device": "nRF54L15_M33",
"svd_path": "nrf54l15.svd"
},
"frameworks": ["zephyr"],
"name": "Nordic nRF54L15-DK (PCA10156)",
"upload": {
"maximum_ram_size": 262144,
"maximum_size": 1572864,
"protocol": "jlink",
"protocols": ["jlink"]
},
"url": "https://www.nordicsemi.com/Products/nRF54L15",
"vendor": "Nordic Semiconductor"
}
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
-41
View File
@@ -1,41 +0,0 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "station-g3"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "BQ Station G3",
"upload": {
"flash_size": "16MB",
"maximum_ram_size": 327680,
"maximum_size": 16777216,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "",
"vendor": "BQ Consulting"
}
+1 -1
View File
@@ -9,7 +9,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_1W",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_S3_CORE",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-DLILYGO_T3S3_V1",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -10,7 +10,7 @@
"-DBOARD_HAS_PSRAM",
"-DUNPHONE_SPIN=9",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
-6
View File
@@ -1,9 +1,3 @@
meshtasticd (2.7.24.0) unstable; urgency=medium
* Version 2.7.24
-- GitHub Actions <github-actions[bot]@users.noreply.github.com> Fri, 08 May 2026 10:44:12 +0000
meshtasticd (2.7.23.0) unstable; urgency=medium
* Version 2.7.23
-7
View File
@@ -1,7 +0,0 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x640000,
app1, app, ota_1, 0x650000,0x640000,
spiffs, data, spiffs, 0xc90000,0x360000,
coredump, data, coredump,0xFF0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x640000
5 app1 app ota_1 0x650000 0x640000
6 spiffs data spiffs 0xc90000 0x360000
7 coredump data coredump 0xFF0000 0x10000
-7
View File
@@ -1,7 +0,0 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x330000,
app1, app, ota_1, 0x340000,0x330000,
spiffs, data, spiffs, 0x670000,0x180000,
coredump, data, coredump,0x7F0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x330000
5 app1 app ota_1 0x340000 0x330000
6 spiffs data spiffs 0x670000 0x180000
7 coredump data coredump 0x7F0000 0x10000
+14 -3
View File
@@ -70,6 +70,17 @@ def esp32_create_combined_bin(source, target, env):
env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
# Enable Newlib Nano formatting to save space
# ...but allow printf float support (compromise)
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
esp32_kind = env.GetProjectOption("custom_esp32_kind")
if esp32_kind == "esp32":
# Free up some IRAM by removing auxiliary SPI flash chip drivers.
# Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
env.Append(
LINKFLAGS=[
"-Wl,--wrap=esp_flash_chip_gd",
"-Wl,--wrap=esp_flash_chip_issi",
"-Wl,--wrap=esp_flash_chip_winbond",
]
)
else:
# For newer ESP32 targets, using newlib nano works better.
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
# force linker response file instead of command line arguments
Import("env")
def wrap_with_tempfile(command_key):
command = env.get(command_key)
if not command or not isinstance(command, str):
return
if "TEMPFILE(" in command:
return
env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
# Force SCons to spill long commands into response files on this target.
env.Replace(MAXLINELENGTH=8192)
for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
wrap_with_tempfile(key)
-140
View File
@@ -1,140 +0,0 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
#
# post:extra_scripts/nrf54l15_linker.py
#
# Fix for Zephyr two-pass link on nRF54L15:
# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
# the SCons dependency chain is broken (final_ld_script Command never runs).
# This script adds a PreAction on the final firmware binary that runs the gcc
# preprocessing command directly (extracted from build.ninja) to generate
# zephyr/linker.cmd before the link step.
#
# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
import os
import re
import subprocess
Import("env")
if env.get("PIOENV") != "nrf54l15dk":
pass # Only for the nrf54l15dk environment
else:
def _extract_gcc_command(ninja_build):
"""Parse build.ninja to find the gcc -E command that generates linker.cmd.
The rule format depends on the host:
Windows (CMake's RunCMake wraps every command):
COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
POSIX (Linux/macOS — no wrapper):
COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
"""
in_rule = False
with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
for line in f:
# Detect start of the linker.cmd custom command rule
if not in_rule:
if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
in_rule = True
continue
stripped = line.strip()
if not stripped.startswith("COMMAND = "):
continue
command_val = stripped[len("COMMAND = ") :]
# On Windows the value is wrapped in `cmd.exe /C "..."` — strip
# the wrapper. On POSIX hosts the inner sequence is the value
# itself (no quoting layer).
m = re.search(r'/C\s+"(.*)"\s*$', command_val)
inner = m.group(1) if m else command_val
parts = inner.split(" && ")
cwd = None
gcc_cmd = None
for part in parts:
part = part.strip()
if part.startswith("cd /D "): # Windows form
cwd = part[len("cd /D ") :]
elif part.startswith("cd "): # POSIX form
cwd = part[len("cd ") :]
elif "arm-none-eabi-gcc" in part:
gcc_cmd = part
if not gcc_cmd:
raise RuntimeError(
"nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
% inner[:400]
)
return gcc_cmd, cwd
raise RuntimeError(
"nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
)
def _generate_linker_cmd(target, source, env):
"""Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
build_dir = env.subst("$BUILD_DIR")
zephyr_dir = os.path.join(build_dir, "zephyr")
linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
if os.path.exists(linker_cmd):
return # Already present — nothing to do
ninja_build = os.path.join(build_dir, "build.ninja")
if not os.path.exists(ninja_build):
raise RuntimeError(
"nRF54L15 linker fix: build.ninja not found at %s\n"
"Run a full build first so CMake generates the Ninja files."
% ninja_build
)
gcc_cmd, cwd = _extract_gcc_command(ninja_build)
run_cwd = cwd if cwd else zephyr_dir
print(
"==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
)
# gcc_cmd comes verbatim from our own build.ninja (never user input) and
# contains Windows-style paths with spaces that cannot be safely argv-split
# with shlex, so we run it via the platform shell. nosec/nosemgrep below
# acknowledge this deliberate, scoped use of shell=True.
result = subprocess.run( # nosec B602
gcc_cmd,
shell=True, # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
cwd=run_cwd,
capture_output=True,
text=True,
)
if result.returncode != 0:
print("GCC stdout:", result.stdout[:2000])
print("GCC stderr:", result.stderr[:2000])
raise RuntimeError(
"nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
% result.returncode
)
if not os.path.exists(linker_cmd):
raise RuntimeError(
"nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
% linker_cmd
)
print("==> linker.cmd generated successfully")
# Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
prog = env.get("PIOMAINPROG")
if prog:
env.AddPreAction(prog, _generate_linker_cmd)
else:
print(
"[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
)
env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)
-6
View File
@@ -7,12 +7,6 @@ __pycache__/
dist/
build/
# Persistent device-log capture (recorder + Datadog cursor).
# Cross-session JSONL streams written by the autouse Recorder singleton
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
# fixture truncate doesn't touch it.
.mtlog/
# Test harness artifacts
tests/report.html
tests/junit.xml
-217
View File
@@ -1,217 +0,0 @@
{
"title": "Meshtastic Firmware — Recorder Stream",
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
"widgets": [
{
"definition": {
"title": "Free heap (bytes)",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"queries": [
{
"name": "free_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "label": "bytes" }
}
},
{
"definition": {
"title": "Heap slope (bytes/min) — last 1h",
"type": "query_value",
"precision": 0,
"requests": [
{
"queries": [
{
"name": "slope",
"data_source": "metrics",
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
"aggregator": "avg"
}
],
"response_format": "scalar"
}
],
"conditional_formats": [
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
]
}
},
{
"definition": {
"title": "Total heap (bytes)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "total_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Battery level (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "battery",
"data_source": "metrics",
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "min": "0", "max": "105" }
}
},
{
"definition": {
"title": "Air utilization (TX %)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "airutil",
"data_source": "metrics",
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Channel utilization (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "chutil",
"data_source": "metrics",
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Log volume by level",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"response_format": "timeseries",
"display_type": "bars",
"queries": [
{
"name": "log_count",
"data_source": "logs",
"indexes": ["*"],
"compute": { "aggregation": "count" },
"search": { "query": "service:meshtastic-firmware" },
"group_by": [
{
"facet": "@level",
"limit": 10,
"sort": { "order": "desc", "aggregation": "count" }
}
]
}
]
}
]
}
},
{
"definition": {
"title": "Recent ERROR / CRIT firmware logs",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "@port", "width": "auto" },
{ "field": "@level", "width": "auto" },
{ "field": "@thread", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
},
{
"definition": {
"title": "Recorder marker events",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware @level:MARK",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
}
],
"template_variables": [
{
"name": "port",
"prefix": "port",
"available_values": [],
"default": "*"
},
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
],
"layout_type": "ordered",
"notify_list": [],
"reflow_type": "auto"
}
-389
View File
@@ -1,389 +0,0 @@
#!/usr/bin/env python3
"""Forward selected recorder JSONL streams to Datadog.
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
Logs Intake API and telemetry numerics to the Metrics v2 series API.
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
duplicate rows already shipped from the current live files.
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
If the recorder rotates before this process drains the live file, or the
forwarder is down across a rotation, those older rows are skipped.
Usage:
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
./scripts/mtlog_to_datadog.py --once # catch up + exit
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
The forwarder is a separate process by design — a Datadog outage or
auth failure must not backpressure the recorder. We exit non-zero on
fatal config errors (missing API key) and keep retrying on transient
network/HTTP errors.
"""
from __future__ import annotations
import argparse
import json
import os
import socket
import sys
import time
from pathlib import Path
from typing import Any, Iterator
try:
import requests
except ImportError:
print(
"requests is required. Install it in the mcp-server venv: "
"uv pip install requests",
file=sys.stderr,
)
sys.exit(2)
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
_METRICS_TPL = "https://api.{site}/api/v2/series"
_LOG_BATCH = 50
_METRICS_BATCH = 100
_MAX_RETRIES = 5
_RETRY_BASE_S = 1.5
# --- streaming JSONL with byte-position cursor -------------------------
class _StreamReader:
"""Reads a single rotating JSONL with cursor-based resume.
This tails only the live `.jsonl` file. The recorder rotates files
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
the live file shrinks abruptly. We detect that via inode change OR live
size < cursor position, and reset the live-file cursor to 0.
"""
def __init__(self, path: Path, cursor: dict[str, Any]):
self.path = path
self.cursor = cursor
def _state(self) -> tuple[int, int]:
"""Return (inode, size) for the live file. (0, 0) if missing."""
try:
st = self.path.stat()
return (st.st_ino, st.st_size)
except FileNotFoundError:
return (0, 0)
def iter_new_records(self) -> Iterator[dict[str, Any]]:
ino, size = self._state()
last_ino = self.cursor.get("ino")
last_pos = int(self.cursor.get("pos") or 0)
if ino == 0:
return
if last_ino is not None and last_ino != ino:
# Rotation happened. Start over.
last_pos = 0
if last_pos > size:
# Live file truncated/shrunk under us — recorder rotated.
last_pos = 0
try:
with self.path.open("r", encoding="utf-8") as fh:
fh.seek(last_pos)
for line in fh:
line = line.rstrip("\n")
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
last_pos = fh.tell()
except FileNotFoundError:
return
self.cursor["ino"] = ino
self.cursor["pos"] = last_pos
def _load_cursor(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except (OSError, json.JSONDecodeError):
return {}
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(data, separators=(",", ":")))
tmp.replace(path)
# --- Datadog clients ---------------------------------------------------
class _DDSession:
"""Pool one HTTPS session, share retry logic."""
def __init__(self, api_key: str, site: str, hostname: str) -> None:
self.api_key = api_key
self.site = site
self.hostname = hostname
self.session = requests.Session()
self.session.headers.update(
{
"DD-API-KEY": api_key,
"Content-Type": "application/json",
}
)
def _post(self, url: str, payload: Any) -> bool:
for attempt in range(_MAX_RETRIES):
try:
resp = self.session.post(url, json=payload, timeout=30)
except requests.RequestException as e:
_wait_retry(attempt, f"network error: {e}")
continue
if 200 <= resp.status_code < 300:
return True
if resp.status_code in (408, 429, 500, 502, 503, 504):
_wait_retry(
attempt,
f"HTTP {resp.status_code} retrying",
)
continue
print(
f"datadog refused: {resp.status_code} {resp.text[:200]}",
file=sys.stderr,
)
return False
return False
def send_logs(self, records: list[dict[str, Any]]) -> int:
if not records:
return 0
url = _LOG_INTAKE_TPL.format(site=self.site)
sent = 0
for i in range(0, len(records), _LOG_BATCH):
batch = records[i : i + _LOG_BATCH]
if self._post(url, batch):
sent += len(batch)
return sent
def send_metrics(self, series: list[dict[str, Any]]) -> int:
if not series:
return 0
url = _METRICS_TPL.format(site=self.site)
sent = 0
for i in range(0, len(series), _METRICS_BATCH):
batch = series[i : i + _METRICS_BATCH]
if self._post(url, {"series": batch}):
sent += len(batch)
return sent
def _wait_retry(attempt: int, reason: str) -> None:
wait = _RETRY_BASE_S * (2**attempt)
print(
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
file=sys.stderr,
)
time.sleep(wait)
# --- record → datadog payload ------------------------------------------
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
line = rec.get("line") or ""
tags = [
f"role:{rec.get('role')}",
f"port:{rec.get('port')}",
]
level = rec.get("level")
if level:
tags.append(f"level:{level}")
tag = rec.get("tag")
if tag:
tags.append(f"thread:{tag}")
return {
"ddsource": "meshtastic-firmware",
"service": "meshtastic-firmware",
"hostname": host,
"message": line,
"ddtags": ",".join(t for t in tags if t and "None" not in t),
"timestamp": int((rec.get("ts") or time.time()) * 1000),
"level": level,
}
def _telemetry_record_to_metrics(
rec: dict[str, Any], host: str
) -> list[dict[str, Any]]:
fields = rec.get("fields") or {}
if not isinstance(fields, dict):
return []
variant = rec.get("variant") or "unknown"
ts = int(rec.get("ts") or time.time())
out: list[dict[str, Any]] = []
tags = []
if rec.get("port"):
tags.append(f"port:{rec['port']}")
if rec.get("role"):
tags.append(f"role:{rec['role']}")
if rec.get("from_node"):
tags.append(f"from_node:{rec['from_node']}")
tags.append(f"variant:{variant}")
for field, value in fields.items():
if not isinstance(value, (int, float)) or isinstance(value, bool):
continue
metric = f"mesh.{variant}.{_metric_safe(field)}"
out.append(
{
"metric": metric,
"type": 3, # GAUGE
"points": [{"timestamp": ts, "value": float(value)}],
"tags": tags,
"resources": [{"type": "host", "name": host}],
}
)
return out
def _metric_safe(name: str) -> str:
# Lowercase, replace non-alnum with underscore for safe metric names.
return "".join(c.lower() if c.isalnum() else "_" for c in name)
# --- main loop ---------------------------------------------------------
def run(
log_dir: Path,
*,
once: bool,
since_seconds: float | None,
poll_interval: float,
dd: _DDSession,
) -> int:
cursor_path = log_dir / ".dd-cursor.json"
cursors = _load_cursor(cursor_path)
# `--since` overrides cursor: rewind to (now-since) timestamp.
# We can't seek by timestamp directly (cursor is byte position), so
# we just reset cursors to 0 and let the time filter in iter_new
# drop older records.
cutoff_ts: float | None = None
if since_seconds is not None:
cursors = {}
cutoff_ts = time.time() - since_seconds
sent_total = {"logs": 0, "telemetry": 0}
while True:
# logs.jsonl → DD logs
log_cursor = cursors.setdefault("logs", {})
log_batch: list[dict[str, Any]] = []
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
log_batch.append(_log_record_to_dd(rec, dd.hostname))
if log_batch:
n = dd.send_logs(log_batch)
sent_total["logs"] += n
print(f"logs: sent {n}/{len(log_batch)}")
# telemetry.jsonl → DD metrics
telem_cursor = cursors.setdefault("telemetry", {})
metric_series: list[dict[str, Any]] = []
for rec in _StreamReader(
log_dir / "telemetry.jsonl", telem_cursor
).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
if metric_series:
n = dd.send_metrics(metric_series)
sent_total["telemetry"] += n
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
_save_cursor(cursor_path, cursors)
if once:
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
return 0
time.sleep(poll_interval)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--log-dir",
default=str(_DEFAULT_LOG_DIR),
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--once", action="store_true", help="Catch up then exit")
mode.add_argument(
"--tail",
action="store_true",
help="Daemon: poll forever (default)",
)
parser.add_argument(
"--since",
type=float,
default=None,
help="Backfill last N seconds. Resets cursor.",
)
parser.add_argument(
"--poll-interval",
type=float,
default=5.0,
help="Seconds between tail polls (default 5)",
)
parser.add_argument(
"--site",
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
help=(
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
"Override via DD_SITE env var or this flag."
),
)
parser.add_argument(
"--host",
default=socket.gethostname(),
help="Hostname tag (default: socket.gethostname())",
)
args = parser.parse_args(argv)
api_key = os.environ.get("DD_API_KEY")
if not api_key:
print("DD_API_KEY env var required.", file=sys.stderr)
return 2
log_dir = Path(args.log_dir)
if not log_dir.exists():
print(
f"log dir {log_dir} does not exist — start the mcp-server first.",
file=sys.stderr,
)
return 2
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
once = args.once and not args.tail
return run(
log_dir,
once=once,
since_seconds=args.since,
poll_interval=args.poll_interval,
dd=dd,
)
if __name__ == "__main__":
sys.exit(main())
-382
View File
@@ -1,382 +0,0 @@
"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
The fixture pipeline is two-stage:
1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
protobuf with fresh wall-clock timestamps.
This module exposes `push_fake_nodedb(...)`, the MCP tool that:
- target="portduino": compiles the JSONL into the device's prefs dir on
the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
- target="hardware": compiles to a temp file, then streams it over the
XModem protocol (via the meshtastic SerialInterface/BLEInterface +
`meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
device. Triggers a reboot so the firmware loads the new state on next
boot.
XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
* 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
* CRC16-CCITT (poly 0x1021, init 0x0000).
* SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
`FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
* SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
* EOT after the last chunk flushes + closes the file on-device.
Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
in the .github/copilot-instructions.md "never do these without asking" list).
"""
from __future__ import annotations
import dataclasses
import hashlib
import pathlib
import queue
import shutil
import subprocess
import sys
import tempfile
import time
from typing import Any, Literal
from .connection import connect, is_tcp_port
# Resolve repo root so the tool works regardless of mcp-server cwd.
_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
_XMODEM_CHUNK = 128
_XMODEM_SUB = 0x1A
_ACK_TIMEOUT_INIT_S = 5.0
_ACK_TIMEOUT_CHUNK_S = 2.0
_MAX_CHUNK_RETRIES = 5
_VALID_SIZES = (250, 500, 1000, 2000)
class FixtureError(RuntimeError):
"""Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
# ---------------------------------------------------------------------------
# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
# Hand-rolled to avoid the optional `crcmod` dep.
# ---------------------------------------------------------------------------
def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
crc = init
for b in data:
crc ^= b << 8
for _ in range(8):
if crc & 0x8000:
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
else:
crc = (crc << 1) & 0xFFFF
return crc
# ---------------------------------------------------------------------------
# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
# doesn't have to duplicate the proto-encoding logic.
# ---------------------------------------------------------------------------
def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
if not _COMPILE_SCRIPT.is_file():
raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
cmd = [
sys.executable,
str(_COMPILE_SCRIPT),
"--in",
str(jsonl_path),
"--out",
str(out_path),
]
try:
subprocess.run(cmd, check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as exc:
raise FixtureError(
f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
f" stdout: {exc.stdout}\n stderr: {exc.stderr}"
) from exc
def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
if custom is not None:
p = pathlib.Path(custom).expanduser().resolve()
if not p.is_file():
raise FixtureError(f"custom_seed_jsonl not found: {p}")
return p
p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
if not p.is_file():
raise FixtureError(
f"missing committed seed at {p}. "
f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
)
return p
# ---------------------------------------------------------------------------
# Portduino push — file copy into ~/.portduino/<config>/prefs/
# ---------------------------------------------------------------------------
def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
home = pathlib.Path.home()
return home / ".portduino" / config_name / "prefs"
def _push_portduino(
size: int,
jsonl: pathlib.Path,
portduino_config: str,
backup_existing: bool,
) -> dict[str, Any]:
prefs = _portduino_prefs_dir(portduino_config)
prefs.mkdir(parents=True, exist_ok=True)
target = prefs / "nodes.proto"
backed_up_to: str | None = None
if backup_existing and target.is_file():
ts = int(time.time())
backup = prefs / f"nodes.proto.bak.{ts}"
shutil.move(str(target), str(backup))
backed_up_to = str(backup)
_compile_proto(jsonl, target)
raw = target.read_bytes()
return {
"transport": "portduino",
"path": str(target),
"bytes": len(raw),
"sha256": hashlib.sha256(raw).hexdigest(),
"jsonl_source": str(jsonl),
"backed_up_to": backed_up_to,
}
# ---------------------------------------------------------------------------
# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
# ---------------------------------------------------------------------------
@dataclasses.dataclass
class _AckEvent:
control: int
seq: int
def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
try:
return q.get(timeout=timeout_s)
except queue.Empty as exc:
raise FixtureError(
f"XModem response timeout after {timeout_s:.1f}s — device not responding"
) from exc
def _push_hardware(
size: int,
jsonl: pathlib.Path,
port: str | None,
reboot_after: bool,
) -> dict[str, Any]:
# Lazy imports so the module loads even when the meshtastic deps aren't
# available (e.g. CI in a Python env without the package installed).
try:
from meshtastic.protobuf import mesh_pb2, xmodem_pb2
from pubsub import pub
except ImportError as exc: # pragma: no cover — dep missing
raise FixtureError(
f"hardware push requires the meshtastic + pypubsub packages: {exc}"
) from exc
if is_tcp_port(port):
raise FixtureError(
"hardware push over TCP/portduino is not supported — use "
"target='portduino' to drop the fixture directly into the prefs dir."
)
# Compile the fixture to a temp file with fresh timestamps.
with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
proto_path = pathlib.Path(tf.name)
try:
_compile_proto(jsonl, proto_path)
payload = proto_path.read_bytes()
finally:
proto_path.unlink(missing_ok=True)
sha256 = hashlib.sha256(payload).hexdigest()
total_bytes = len(payload)
# Subscribe to XModem responses BEFORE we open the interface, so we don't
# race the first ACK that arrives during the SOH/seq=0 handshake.
#
# NB: the signature MUST declare every kwarg pypubsub will see for this
# topic, or pubsub locks the topic spec to a smaller set (whichever
# subscribe arrives first) and then *rejects* the meshtastic library's
# publish call with `SenderUnknownMsgDataError: unknown ... interface`.
# The meshtastic lib publishes both `packet=` and `interface=`
# (mesh_interface.py:1389-1395), so both must appear here.
response_q: "queue.Queue[_AckEvent]" = queue.Queue()
def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
if packet is None:
return
response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
chunks_sent = 0
retried = 0
rebooted = False
XMC = xmodem_pb2.XModem.Control
try:
with connect(port=port) as iface:
# 1) Send the filename (SOH, seq=0).
init_pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=0,
buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(
f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
f"(got control={ack.control}, expected ACK). "
f"Filesystem full or permissions issue?"
)
# 2) Stream the payload in 128 B chunks.
for offset in range(0, total_bytes, _XMODEM_CHUNK):
chunk = payload[offset : offset + _XMODEM_CHUNK]
if len(chunk) < _XMODEM_CHUNK:
# Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
# become part of the file on-device, but nanopb ignores
# bytes past the end of the top-level message.
chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
seq = ((offset // _XMODEM_CHUNK) + 1) % 256
# Retry loop on NAK / timeout.
attempts = 0
while True:
pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=seq,
buffer=chunk,
crc16=_crc16_ccitt(chunk),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control == XMC.Value("ACK"):
chunks_sent += 1
break
if ack.control == XMC.Value("NAK"):
attempts += 1
retried += 1
if attempts >= _MAX_CHUNK_RETRIES:
# Abort: send CAN so the firmware removes the half-
# written file via FSCom.remove(filename).
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(
control=XMC.Value("CAN")
)
)
)
raise FixtureError(
f"chunk seq={seq} NAK'd {attempts} times; "
f"aborted transfer (file removed on-device)."
)
continue # retry the same chunk
raise FixtureError(
f"unexpected XModem control={ack.control} on seq={seq}"
)
# 3) Tell the device we're done.
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
)
)
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(f"EOT not ACKed (got control={ack.control})")
# 4) Reboot so loadFromDisk picks up the new file.
if reboot_after:
iface.localNode.reboot(secs=1)
rebooted = True
finally:
try:
pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
except Exception:
pass
return {
"transport": "hardware",
"port": port,
"filename_on_device": _DEFAULT_NODES_FILENAME,
"bytes": total_bytes,
"chunks_sent": chunks_sent,
"retried": retried,
"sha256": sha256,
"jsonl_source": str(jsonl),
"rebooted": rebooted,
}
# ---------------------------------------------------------------------------
# Public entry point — registered as an MCP tool in server.py.
# ---------------------------------------------------------------------------
def push_fake_nodedb(
size: int,
target: Literal["portduino", "hardware"] = "portduino",
*,
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
Args:
size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
"hardware" (XModem upload to /prefs/nodes.proto + reboot).
port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
or BLE identifier. TCP endpoints are rejected — use target="portduino"
instead.
portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
if present, so you can roll back.
confirm: required True for target="hardware" (writes flash + reboots).
reboot_after: hardware only. If True, send a 1-second reboot after the
final ACK so loadFromDisk picks up the new file at next boot.
custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
test scenario.
Returns:
dict with transport, bytes, sha256, etc. — depends on target.
"""
if size not in _VALID_SIZES:
raise FixtureError(
f"size must be one of {_VALID_SIZES}; got {size!r}. "
f"Add a new committed seed if you need a different cardinality."
)
jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
if target == "portduino":
return _push_portduino(size, jsonl, portduino_config, backup_existing)
if target == "hardware":
if not confirm:
raise FixtureError(
"hardware push writes flash and triggers a reboot — pass confirm=True."
)
if not port:
raise FixtureError(
"target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
)
return _push_hardware(size, jsonl, port, reboot_after)
raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")
+1 -44
View File
@@ -108,33 +108,18 @@ def build(
env: str,
with_manifest: bool = True,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Run `pio run -e <env>` and return artifact paths.
`userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
into userPrefs.jsonc for this build only. File is restored byte-for-byte
on exit. Use `userprefs_set()` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
`{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
prefix on every log line. Combines with the recorder so heap shows
up at log cadence (much higher resolution than the ~60 s LocalStats
packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
expand to bare `-D<NAME>` (presence-only flags).
"""
args = ["run", "-e", env]
if with_manifest:
args.extend(["-t", "mtjson"])
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
args,
timeout=pio.TIMEOUT_BUILD,
check=False,
extra_env=extra_env,
)
result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
return {
"exit_code": result.returncode,
"artifacts": [str(p) for p in _artifacts_for(env)],
@@ -142,27 +127,9 @@ def build(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
"""Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
"-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
the flag entirely. Other types stringify."""
parts: list[str] = []
for key, value in build_flags.items():
if value is False or value is None:
continue
if value is True:
parts.append(f"-D{key}")
else:
parts.append(f"-D{key}={value}")
if not parts:
return {}
return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
def clean(env: str) -> dict[str, Any]:
"""Run `pio run -e <env> -t clean`."""
result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -179,29 +146,20 @@ def flash(
port: str,
confirm: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""`pio run -e <env> -t upload --upload-port <port>`. All architectures.
`userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
that pio performs will pick up the injected values.
`build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
is exported for the rebuild-before-upload, so the uploaded firmware
actually carries the flags. Without this propagation, `pio run -t upload`
would relink without the env var and silently drop them. Common use:
`build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
"""
_require_confirm(confirm, "flash")
_reject_native_env(env, "flash")
connection.reject_if_tcp(port, "flash")
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
["run", "-e", env, "-t", "upload", "--upload-port", port],
timeout=pio.TIMEOUT_UPLOAD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
@@ -209,7 +167,6 @@ def flash(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
-410
View File
@@ -1,410 +0,0 @@
"""Read-side queries over the recorder's JSONL streams.
Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
loads a whole file. Time-bound queries short-circuit as soon as `ts`
exceeds the requested end. The recorder writes monotonically, so a
forward scan is cheap; we don't need an index.
All time arguments accept:
- epoch seconds (int/float)
- relative strings: "-15m", "-2h", "-3d", "now"
- ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
treated as UTC)
Tools that return data ALWAYS cap their output (max_lines / max_points
/ max), and report whether more matched than was returned.
"""
from __future__ import annotations
import gzip
import json
import re
import statistics
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterator
from .recorder.recorder import get_recorder
_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
_REGEX_PREVIEW_MAX = 100
_REGEX_PREVIEW_TRUNCATE = 97
def _parse_time(value: Any, *, now: float | None = None) -> float:
"""Coerce to epoch seconds. Defaults `now` to `time.time()`."""
if value is None:
return time.time()
if isinstance(value, (int, float)):
return float(value)
if not isinstance(value, str):
raise ValueError(f"invalid time: {value!r}")
s = value.strip().lower()
if s in ("", "now"):
return time.time() if now is None else now
m = _REL_RE.match(s)
if m:
n = float(m.group(1))
unit = m.group(2)
secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
base = time.time() if now is None else now
return base - secs
# Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
try:
if s.endswith("z"):
s = s[:-1] + "+00:00"
dt = datetime.fromisoformat(s)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.timestamp()
except ValueError as e:
raise ValueError(f"unparseable time: {value!r}") from e
def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
"""Stream records in chronological order: rotated archives first
(oldest → newest by lex sort, which is chronological for our
`YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
last. The "keep last N" pop-front logic in the window queries
relies on records arriving in time order across files.
"""
files: list[Path] = []
# Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
files.append(archive)
if path.exists():
files.append(path)
for f in files:
opener = gzip.open if f.suffix == ".gz" else open
try:
with opener(f, "rt", encoding="utf-8") as fh: # type: ignore[arg-type]
for line in fh:
line = line.strip()
if not line:
continue
try:
rec = json.loads(line)
except json.JSONDecodeError:
continue
ts = rec.get("ts")
if not isinstance(ts, (int, float)):
continue
if ts < since:
continue
if ts > until:
# Records are append-monotonic within a file, so
# the rest of this file is also past `until`.
# Archives can still overlap each other, so only
# short-circuit this file, not the whole scan.
break
yield rec
except (FileNotFoundError, OSError):
continue
# -- queries ------------------------------------------------------------
def logs_window(
start: Any = "-15m",
end: Any = "now",
*,
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines, filtered.
`level` accepts a single level name or pipe-separated set
("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
`line` field. Returns the last `max_lines` matches.
"""
s = _parse_time(start)
e = _parse_time(end)
levels = _split_set(level)
if grep:
try:
grep_re = re.compile(grep)
except re.error as exc:
preview = (
grep
if len(grep) <= _REGEX_PREVIEW_MAX
else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
)
raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
else:
grep_re = None
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
if levels and rec.get("level") not in levels:
continue
if tag and rec.get("tag") != tag:
continue
if port and rec.get("port") != port:
continue
if grep_re and not grep_re.search(rec.get("line") or ""):
continue
matched += 1
out.append(rec)
if len(out) > max_lines:
out.pop(0) # keep the most recent N
return {
"lines": out,
"total_matched": matched,
"dropped": max(0, matched - max_lines),
"window": {"start": s, "end": e},
}
def telemetry_timeline(
window: Any = "1h",
*,
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Timeseries of one telemetry field, downsampled.
`field` matches both the protobuf snake_case name (`free_heap`,
`heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
Server-side bucket-mean downsamples to ≤ `max_points`. Returns
`slope_per_min` (linear regression slope, units/min) so a leak
detector can read one number.
"""
end = time.time()
if isinstance(window, (int, float)):
# Numeric `window` is a duration in seconds — "last N seconds".
# Without this branch, `_parse_time(-N)` would treat -N as an
# absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
# producing a wildly negative `start` and matching nothing.
start = end - float(window)
elif isinstance(window, str) and not window.startswith("-"):
# Bare string like "1h" is sugar for "-1h".
start = _parse_time(f"-{window}", now=end)
else:
start = _parse_time(window, now=end)
base = get_recorder().base_dir
raw: list[tuple[float, float]] = []
field_aliases = _field_aliases(field)
for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
if rec.get("variant") != variant:
continue
if port and rec.get("port") != port:
continue
fields = rec.get("fields") or {}
value: Any = None
for alias in field_aliases:
if alias in fields:
value = fields[alias]
break
if not isinstance(value, (int, float)):
continue
raw.append((float(rec["ts"]), float(value)))
if not raw:
return {
"points": [],
"samples": 0,
"min": None,
"max": None,
"slope_per_min": None,
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
points = _downsample(raw, max_points=max_points)
values = [v for _, v in raw]
return {
"points": [{"ts": ts, "value": v} for ts, v in points],
"samples": len(raw),
"min": min(values),
"max": max(values),
"slope_per_min": _slope_per_min(raw),
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
def packets_window(
start: Any = "-5m",
end: Any = "now",
*,
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
portnums = _split_set(portnum)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
if portnums and rec.get("portnum") not in portnums:
continue
if from_node and str(rec.get("from_node")) != str(from_node):
continue
if to_node and str(rec.get("to_node")) != str(to_node):
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"packets": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def events_window(
start: Any = "-1h",
end: Any = "now",
*,
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
kinds = _split_set(kind)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
if kinds and rec.get("kind") not in kinds:
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"events": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def export(
start: Any,
end: Any,
dest_dir: str,
*,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of each requested stream into `dest_dir`.
For a notebook, a bug report, or a Datadog backfill. Output files
are uncompressed JSONL (callers gzip themselves if they want to).
"""
s = _parse_time(start)
e = _parse_time(end)
selected = streams or ["logs", "telemetry", "packets", "events"]
dest = Path(dest_dir)
dest.mkdir(parents=True, exist_ok=True)
base = get_recorder().base_dir
paths: dict[str, str] = {}
for stream in selected:
src = base / f"{stream}.jsonl"
if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
continue
out_path = dest / f"{stream}.jsonl"
n = 0
with out_path.open("w", encoding="utf-8") as fh:
for rec in _iter_jsonl(src, since=s, until=e):
fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
n += 1
paths[stream] = str(out_path)
paths[f"{stream}_count"] = str(n)
return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
# -- helpers ------------------------------------------------------------
def _split_set(value: str | None) -> set[str] | None:
if not value:
return None
return {v.strip() for v in value.split("|") if v.strip()}
def _field_aliases(field: str) -> list[str]:
"""Accept snake_case OR camelCase, plus a few legacy aliases."""
snake = field
camel = _snake_to_camel(field)
aliases = {snake, camel}
# Old protobuf fields (pre-LocalStats) used different names
legacy = {
"free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
"heap_free_bytes": [
"heap_free_bytes",
"heapFreeBytes",
"free_heap",
"freeHeap",
],
"total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
"heap_total_bytes": [
"heap_total_bytes",
"heapTotalBytes",
"total_heap",
"totalHeap",
],
}
if field in legacy:
aliases.update(legacy[field])
return list(aliases)
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _downsample(
points: list[tuple[float, float]], *, max_points: int
) -> list[tuple[float, float]]:
if len(points) <= max_points:
return points
# Even-bucket mean. Preserves shape better than nth-sample picking.
n = len(points)
bucket = n / max_points
out: list[tuple[float, float]] = []
i = 0
for k in range(max_points):
end = int((k + 1) * bucket)
end = min(end, n)
if end <= i:
continue
chunk = points[i:end]
ts = chunk[len(chunk) // 2][0]
val = statistics.fmean(v for _, v in chunk)
out.append((ts, val))
i = end
return out
def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
"""Least-squares slope (units per minute). None if too few points."""
if len(points) < 2:
return None
xs = [t for t, _ in points]
ys = [v for _, v in points]
n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
den = sum((x - mean_x) ** 2 for x in xs)
if den == 0:
return None
slope_per_sec = num / den
return slope_per_sec * 60.0
-15
View File
@@ -92,7 +92,6 @@ def _run_capturing(
cwd: Path | None = None,
timeout: float | None = None,
tee_header: str | None = None,
extra_env: dict[str, str] | None = None,
) -> tuple[int, str, str, float]:
"""Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
@@ -100,9 +99,6 @@ def _run_capturing(
`subprocess.TimeoutExpired` on timeout (callers map this to their own
domain-specific error).
`extra_env` merges into the subprocess environment (parent env stays
intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
Fast path: `subprocess.run(capture_output=True)` when no flash log is
configured (unchanged behavior).
@@ -114,9 +110,6 @@ def _run_capturing(
"""
log_path = _flash_log_path()
t0 = time.monotonic()
env = None
if extra_env:
env = {**os.environ, **extra_env}
if log_path is None:
# Fast path — unchanged.
@@ -126,7 +119,6 @@ def _run_capturing(
capture_output=True,
text=True,
timeout=timeout,
env=env,
)
return (
proc.returncode,
@@ -153,7 +145,6 @@ def _run_capturing(
stderr=subprocess.PIPE,
text=True,
bufsize=1, # line-buffered
env=env,
)
stdout_chunks: list[str] = []
stderr_chunks: list[str] = []
@@ -241,17 +232,12 @@ def run(
cwd: Path | None = None,
timeout: float | None = TIMEOUT_DEFAULT,
check: bool = True,
extra_env: dict[str, str] | None = None,
) -> PioResult:
"""Invoke `pio <args>` and return captured output.
`cwd` defaults to the firmware root. `check=True` raises `PioError` on
non-zero exit; set `check=False` to inspect `returncode` manually.
`extra_env` merges into the subprocess environment — used for
`PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
toggles that can't be expressed as command-line args.
If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
line-by-line as it arrives (for live flash progress in the TUI).
"""
@@ -264,7 +250,6 @@ def run(
cwd=work_dir,
timeout=timeout,
tee_header=f"pio {' '.join(args)}",
extra_env=extra_env,
)
except subprocess.TimeoutExpired as exc:
raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc
@@ -1,19 +0,0 @@
"""Persistent device-log capture.
Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
process-global so a single subscription captures every active interface
(serial / TCP / BLE) without any per-connection bookkeeping.
The recorder is opt-in-by-import: importing this package is a no-op; call
`get_recorder().start()` (which `server.py` does at FastMCP app init) to
begin writing. `pause()` / `resume()` exist for the rare case the user
wants a clean stretch of file (e.g. capturing a known-good baseline).
"""
from __future__ import annotations
from .recorder import Recorder, get_recorder
__all__ = ["Recorder", "get_recorder"]
@@ -1,309 +0,0 @@
"""Best-effort parsers for log lines and telemetry packets.
Two flavors of log line cross our pubsub subscription:
1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
accumulates bytes between protobuf frames and emits the full
firmware-formatted line, e.g.
"INFO | 12:34:56 12345 [Main] Booting"
— level, HH:MM:SS, uptime seconds, thread bracket, then message.
2. LogRecord protobuf path (debug_log_api enabled): the lib calls
`_handleLogLine(record.message)` with ONLY the message body. The
level/source/time fields on the LogRecord are dropped before
pubsub fan-out. We get e.g. just "Booting".
Both arrive on `meshtastic.log.line`. The parser tries to recover a
level + thread when the prefix is present and falls back to level=None
otherwise. Consumers who want level filtering on protobuf-mode hosts
should grep the raw `line` field instead.
Telemetry: `meshtastic.receive.telemetry` packets carry one of several
metric variants in `packet["decoded"]["telemetry"]`. We flatten the
chosen variant into a {field: value} dict so callers don't have to
know the protobuf shape.
"""
from __future__ import annotations
import re
from typing import Any
# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
# macros must stay in sync with these strings.
_LINE_RE = re.compile(
r"""
^
(?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
\s*\|\s*
(?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
\s+
(?P<uptime>\d+)
\s+
(?:\[(?P<thread>[^\]]+)\]\s+)?
(?P<msg>.*)
$
""",
re.VERBOSE,
)
# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
# thread bracket. See src/RedirectablePrint.cpp:175.
_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
# "++++++ Thread NAME freed heap A -> B (delta) ++++++"
_THREAD_HEAP_RE = re.compile(
r"""
^[\-+]+\s*
Thread\s+(?P<thread>\S+)\s+
(?P<kind>leaked|freed)\s+heap\s+
(?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
\((?P<delta>-?\d+)\)
""",
re.VERBOSE,
)
# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
_HEAP_STATUS_RE = re.compile(
r"""
Heap\s+status:\s+
(?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
(?:\s+\((?P<delta>-?\d+)\))?
""",
re.VERBOSE,
)
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
def parse_log_line(line: str) -> dict[str, Any]:
"""Best-effort decompose a raw firmware log line.
Returns a dict with at least `line` (the original, unmodified — ANSI
codes preserved for fidelity). Adds `level`, `tag`, `clock`,
`uptime_s`, and `msg` when the full prefix is present.
Handles two firmware quirks:
- LogRecord.message can carry ANSI color escapes from RedirectablePrint
(the BLE/StreamAPI path inherited the colored body in some builds).
We strip ANSI before regex matching so the prefix survives.
- DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
thread name is set, the heap takes the thread bracket position —
looks like `[heap 12345] msg`. We detect that shape and move it
out of `tag` and into `heap_free`.
DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
(bytes), and when a `Thread X leaked|freed heap` line is recognized,
`heap_event` = {kind, thread, before, after, delta}.
Never raises.
"""
out: dict[str, Any] = {"line": line}
if not line:
return out
# Strip ANSI escapes BEFORE any regex matching. The original `line`
# stays in `out["line"]` for fidelity / future grep.
clean = _ANSI_RE.sub("", line)
m = _LINE_RE.match(clean)
msg: str | None = None
if m:
level = m.group("level").rstrip()
out["level"] = level
out["clock"] = m.group("clock")
try:
out["uptime_s"] = int(m.group("uptime"))
except (TypeError, ValueError):
out["uptime_s"] = None
thread = m.group("thread")
if thread:
# If "thread" is actually the heap prefix taking the bracket
# position (DEBUG_HEAP build, no thread set), capture heap
# and leave tag unset.
hb = _HEAP_BRACKET_RE.match(thread.strip())
if hb:
try:
out["heap_free"] = int(hb.group("heap"))
except (TypeError, ValueError):
pass
else:
out["tag"] = thread
msg = m.group("msg")
out["msg"] = msg
else:
# No prefix — bare LogRecord.message body. Inspect the whole
# line for DEBUG_HEAP-style content; the heap-prefix and
# thread-leak patterns can survive on either path.
msg = clean
# DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
# Sits AFTER the thread bracket and BEFORE the message body, but
# for bare LogRecord lines it's at the start. Match it at the
# head of `msg`.
if msg:
hp = _HEAP_PREFIX_RE.match(msg)
if hp:
try:
out["heap_free"] = int(hp.group("heap"))
except (TypeError, ValueError):
pass
else:
# Strip the prefix from `msg` so a grep on the message
# body doesn't have to know about it.
out["msg"] = hp.group("rest")
msg = hp.group("rest")
# Thread-level leak/free detection.
thr = _THREAD_HEAP_RE.search(msg)
if thr:
try:
out["heap_event"] = {
"kind": thr.group("kind"),
"thread": thr.group("thread"),
"before": int(thr.group("before")),
"after": int(thr.group("after")),
"delta": int(thr.group("delta")),
}
except (TypeError, ValueError):
pass
# Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
hs = _HEAP_STATUS_RE.search(msg)
if hs:
try:
out["heap_free"] = int(hs.group("free"))
out["heap_total"] = int(hs.group("total"))
if hs.group("delta") is not None:
out["heap_delta"] = int(hs.group("delta"))
except (TypeError, ValueError):
pass
return out
# -- Telemetry ----------------------------------------------------------
# Order matters: meshtastic-python decoded packets use the protobuf
# `oneof variant` field name (snake_case) as the dict key.
_TELEMETRY_VARIANTS = (
("device_metrics", "device"),
("local_stats", "local"),
("environment_metrics", "environment"),
("power_metrics", "power"),
("air_quality_metrics", "airQuality"),
("health_metrics", "health"),
("host_metrics", "host"),
)
def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
"""Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
packet. Returns None when the shape isn't what we expect — so the
caller can fall back to a generic packets.jsonl row.
"""
if not isinstance(packet, dict):
return None
decoded = packet.get("decoded")
if not isinstance(decoded, dict):
return None
telem = decoded.get("telemetry")
if not isinstance(telem, dict):
return None
# The Python lib produces dict-of-camelCase keys via MessageToDict.
# Try both camelCase and snake_case to be robust to lib version drift.
for snake, label in _TELEMETRY_VARIANTS:
camel = _snake_to_camel(snake)
for key in (snake, camel):
value = telem.get(key)
if isinstance(value, dict):
return {
"variant": label,
"fields": {k: _scalarize(v) for k, v in value.items()},
"time": telem.get("time"),
}
return None
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _scalarize(value: Any) -> Any:
"""Keep telemetry fields JSON-friendly. Lists/dicts pass through
untouched; bytes -> hex string; protobuf enums occasionally arrive
as ints (fine) or strings (also fine)."""
if isinstance(value, (bytes, bytearray, memoryview)):
return bytes(value).hex()
return value
# -- Generic packet summary ---------------------------------------------
def summarize_packet(
packet: dict[str, Any], *, payload_hex_len: int = 64
) -> dict[str, Any]:
"""Reduce a packet dict to a stable, queryable summary. Drops the
full payload bytes — the recorder records summaries, not pcaps.
"""
if not isinstance(packet, dict):
return {"raw_type": type(packet).__name__}
decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
payload = decoded.get("payload") if isinstance(decoded, dict) else None
payload_hex = None
payload_size = None
if isinstance(payload, (bytes, bytearray, memoryview)):
b = bytes(payload)
payload_size = len(b)
payload_hex = b[:payload_hex_len].hex() if b else ""
elif isinstance(payload, str):
# Some decoded payloads (text messages) come as decoded strings.
payload_size = len(payload)
payload_hex = None # not bytes
return {
"from_node": packet.get("fromId") or packet.get("from"),
"to_node": packet.get("toId") or packet.get("to"),
"portnum": portnum,
"hop_limit": packet.get("hopLimit"),
"want_ack": packet.get("wantAck"),
"rx_rssi": packet.get("rxRssi"),
"rx_snr": packet.get("rxSnr"),
"channel": packet.get("channel"),
"id": packet.get("id"),
"payload_size": payload_size,
"payload_hex_prefix": payload_hex,
}
# -- Interface identification ------------------------------------------
def interface_label(interface: Any) -> dict[str, Any]:
"""Stable identifier for the meshtastic interface that emitted an event.
Used as the `port`/`role` tag on every recorded row. SerialInterface
has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
has `address`. Falls back to the class name when none of those exist.
"""
if interface is None:
return {"port": None, "role": None}
dev_path = getattr(interface, "devPath", None)
if dev_path:
return {"port": str(dev_path), "role": "serial"}
hostname = getattr(interface, "hostname", None)
if hostname:
port_num = getattr(interface, "portNumber", None)
endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
return {"port": endpoint, "role": "tcp"}
address = getattr(interface, "address", None)
if address:
return {"port": str(address), "role": "ble"}
return {"port": type(interface).__name__, "role": None}
@@ -1,467 +0,0 @@
"""Process-global recorder singleton.
Subscribes once to the meshtastic pubsub fan-out and writes four append-only
JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
process-global — a single subscription captures every active interface
without per-connection bookkeeping.
Files:
logs.jsonl — every `meshtastic.log.line` event (best-effort prefix
parsed for level/tag/uptime; raw `line` always preserved)
telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
variant (device / local / environment / power / etc.)
packets.jsonl — every other `meshtastic.receive.*` packet, summarized
(portnum, hops, RSSI/SNR, payload size + 64-byte hex)
events.jsonl — connection lifecycle, node-DB updates, and manual
`mark_event` rows. Lower volume; useful for aligning
timelines.
Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
write methods short-circuit when paused, so we don't lose ordering when
resumed (we just have a gap). No queueing.
"""
from __future__ import annotations
import logging
import os
import threading
import time
from pathlib import Path
from typing import Any
from . import parsers
from .rotating import _RotatingJsonl
_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
log = logging.getLogger(__name__)
class Recorder:
"""Singleton write-side of the persistent log capture system."""
def __init__(self, base_dir: Path | None = None) -> None:
self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
self._lock = threading.RLock()
self._started = False
self._paused = False
self._pause_reason: str | None = None
self._started_at: float | None = None
self._handlers: list[tuple[str, Any]] = []
self._files: dict[str, _RotatingJsonl] = {}
# -- lifecycle ----------------------------------------------------
def start(self) -> None:
"""Idempotent. Safe to call from FastMCP app startup."""
with self._lock:
if self._started:
return
self.base_dir.mkdir(parents=True, exist_ok=True)
self._files = {
"logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
"telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
"packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
"events": _RotatingJsonl(self.base_dir / "events.jsonl"),
}
self._wire_pubsub()
self._started = True
self._started_at = time.time()
# Write the recorder_start marker after the initialization block.
# `_write_event()` re-checks recorder state via `_files_snapshot()`,
# so keeping this out of the setup block avoids nested lifecycle work.
self._write_event(kind="recorder_start", label="recorder_started")
def stop(self) -> None:
with self._lock:
if not self._started:
return
self._unwire_pubsub()
for f in self._files.values():
f.close()
self._files = {}
self._started = False
def pause(self, reason: str | None = None) -> None:
# Write the pause marker BEFORE flipping the flag — `_write_event`
# short-circuits when paused, so the order matters for this event
# to actually land in events.jsonl.
self._write_event(
kind="recorder_pause",
label="paused",
note=reason,
)
with self._lock:
self._paused = True
self._pause_reason = reason
def resume(self) -> None:
# Mirror of `pause()`: clear the flag first, then write the marker
# so it isn't suppressed by the still-paused short-circuit.
with self._lock:
self._paused = False
self._pause_reason = None
self._write_event(kind="recorder_resume", label="resumed")
# -- pubsub wiring ------------------------------------------------
def _wire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
# Subscribers — one per topic. Each pubsub publisher sends
# keyword args matching its handler's signature; pubsub
# introspects the function signature to route args.
bindings = [
("meshtastic.log.line", self._on_log_line),
("meshtastic.serial.line", self._on_serial_line),
("meshtastic.receive", self._on_receive),
("meshtastic.receive.telemetry", self._on_telemetry),
("meshtastic.connection.established", self._on_connection_established),
("meshtastic.connection.lost", self._on_connection_lost),
("meshtastic.node.updated", self._on_node_updated),
]
for topic, handler in bindings:
try:
pub.subscribe(handler, topic)
self._handlers.append((topic, handler))
except Exception as exc:
# If pubsub refuses one binding (signature mismatch on
# an old lib version), log it and keep the rest.
log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
def _unwire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
for topic, handler in self._handlers:
try:
pub.unsubscribe(handler, topic)
except Exception:
pass
self._handlers.clear()
# -- handlers -----------------------------------------------------
#
# Pubsub callbacks must never raise. Every handler is wrapped in a
# try/except that swallows so a bug here can't take down the
# SerialInterface receive thread.
#
# Threading: handlers fire on whatever thread the meshtastic library
# dispatches from (varies by interface), while `stop()` clears
# `self._files` under `self._lock`. We snapshot `_files` under the
# lock at the top of each handler so a concurrent stop can't
# KeyError us mid-write. The actual file write goes through
# `_RotatingJsonl` which has its own lock.
def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
"""Atomic-ish view of `self._files`. Returns None when the recorder
is paused or stopped, so handlers can early-exit cleanly without
racing `stop()`'s clear."""
with self._lock:
if not self._started or self._paused:
return None
return dict(self._files)
def _on_log_line(self, line: str, interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
# DEBUG_HEAP enrichments (only present when the firmware
# was built with -DDEBUG_HEAP=1). Surface as first-class
# fields so logs_window can grep/filter on them and so
# heap_free synthesizes a telemetry point below.
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# If the line carried a heap snapshot, also write it as a
# synthesized LocalStats-shaped row so telemetry_timeline
# picks it up at log cadence (much higher resolution than
# the ~60 s LocalStats packet). Tagged source=debug_heap so
# consumers can filter if mixing scales is unwanted.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap",
}
)
except Exception:
pass
def _on_serial_line(self, line: str, port: str | None = None) -> None:
"""Text-mode passive tap. Fired from `serial_session._drain` when a
`pio device monitor` subprocess is running.
Same parse + heap-synthesis path as `_on_log_line`, but receives
the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
body). On DEBUG_HEAP builds in text mode this gives us per-log-line
heap data — far higher cadence than LocalStats, and works without
protobuf API mode (no SerialInterface required).
"""
files = self._files_snapshot()
if files is None:
return
try:
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": port,
"role": "serial_session",
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# Synthesize a heap_free telemetry sample whenever the line
# carries one — same logic as _on_log_line, tagged source so
# consumers can distinguish text-mode tap from protobuf path.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": port,
"role": "serial_session",
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap_serial",
}
)
except Exception:
pass
def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
extracted = parsers.extract_telemetry(packet)
if extracted is None:
# Couldn't extract a known variant — fall through to the
# generic `_on_receive` path, which will still fire for
# this packet via the parent topic.
return
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
"from_node": packet.get("fromId") or packet.get("from"),
"variant": extracted["variant"],
"fields": extracted["fields"],
"device_time": extracted.get("time"),
}
files["telemetry"].write(record)
except Exception:
pass
def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
# Generic-receive fires for EVERY packet. Telemetry packets get
# recorded twice (here and in _on_telemetry) — that's intentional:
# packets.jsonl is the universal record, telemetry.jsonl is the
# structured timeseries view.
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
summary = parsers.summarize_packet(packet)
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
**summary,
}
files["packets"].write(record)
except Exception:
pass
def _on_connection_established(self, interface: Any = None) -> None:
self._write_event(
kind="connection_established",
interface=interface,
)
def _on_connection_lost(self, interface: Any = None) -> None:
self._write_event(
kind="connection_lost",
interface=interface,
)
def _on_node_updated(
self, node: dict[str, Any] | None = None, interface: Any = None
) -> None:
# Lower-volume than packets but informative — node ID, hops away,
# last heard. Skip the user dict if absent.
try:
user = (node or {}).get("user") if isinstance(node, dict) else None
self._write_event(
kind="node_updated",
interface=interface,
data={
"num": (node or {}).get("num"),
"id": (user or {}).get("id"),
"short": (user or {}).get("shortName"),
"long": (user or {}).get("longName"),
"hops_away": (node or {}).get("hopsAway"),
"snr": (node or {}).get("snr"),
"last_heard": (node or {}).get("lastHeard"),
},
)
except Exception:
pass
# -- public write helpers -----------------------------------------
def mark_event(
self,
label: str,
note: str | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""User-facing marker. Writes to events.jsonl AND emits a
synthetic logs.jsonl row tagged level=MARK so timelines align.
"""
ts = self._write_event(kind="mark", label=label, note=note, data=data)
# Mirror into logs so a single logs_window grep finds it.
files = self._files_snapshot()
if files is not None:
try:
files["logs"].write(
{
"ts": ts,
"port": None,
"role": "marker",
"level": "MARK",
"tag": "mark_event",
"line": f"[mark] {label}" + (f" — {note}" if note else ""),
}
)
except Exception:
pass
return {"ts": ts, "label": label}
def _write_event(
self,
*,
kind: str,
label: str | None = None,
note: str | None = None,
interface: Any = None,
data: dict[str, Any] | None = None,
) -> float:
ts = time.time()
# Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
# arrive at choreographed moments — `pause()` writes BEFORE flipping
# the flag and `resume()` writes AFTER clearing it, so those calls
# see _paused=False here. Other event kinds short-circuit when
# paused via the snapshot guard below.
files = self._files_snapshot()
if files is None:
return ts
try:
tags = parsers.interface_label(interface)
files["events"].write(
{
"ts": ts,
"kind": kind,
"label": label,
"note": note,
"port": tags["port"],
"role": tags["role"],
"data": data,
}
)
except Exception:
pass
return ts
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
return {
"running": self._started,
"paused": self._paused,
"pause_reason": self._pause_reason,
"started_at": self._started_at,
"base_dir": str(self.base_dir),
"files": {name: f.status() for name, f in self._files.items()},
}
def force_rotate_all(self) -> dict[str, Any]:
"""Test/admin hook: rotate every stream right now."""
with self._lock:
files = list(self._files.values())
for f in files:
f.force_rotate()
# `status()` re-acquires `self._lock`; release before calling it.
return self.status()
# -- module-level singleton accessor ------------------------------------
_INSTANCE_LOCK = threading.Lock()
_INSTANCE: Recorder | None = None
def get_recorder() -> Recorder:
"""Return the process-global Recorder. Created on first call.
Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
(used by tests to redirect to a tmpdir).
"""
global _INSTANCE
with _INSTANCE_LOCK:
if _INSTANCE is None:
override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
base = Path(override) if override else None
_INSTANCE = Recorder(base_dir=base)
return _INSTANCE
@@ -1,163 +0,0 @@
"""Append-only JSONL writer with size-capped rotation.
A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
JSON objects (one row per call). When the live file exceeds `max_bytes`,
it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
and the live file resets to empty. Old archives past `keep_archives` are
unlinked oldest-first.
Size check is amortized — `os.fstat` runs every `check_every` writes,
not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
Threading: every public method acquires `self._lock`. The recorder runs
several pubsub handlers on whatever thread the meshtastic library
dispatches from (varies by interface), and queries from MCP tool calls
arrive on the FastMCP request thread, so this lock is not optional.
"""
from __future__ import annotations
import gzip
import json
import os
import shutil
import threading
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
class _RotatingJsonl:
"""Append-only JSONL with size rotation. Thread-safe."""
def __init__(
self,
path: Path,
*,
max_bytes: int = 100 * 1024 * 1024,
keep_archives: int = 5,
check_every: int = 1000,
) -> None:
self.path = path
self.max_bytes = max_bytes
self.keep_archives = keep_archives
self.check_every = check_every
self._lock = threading.Lock()
self._fh: Any = None
self._writes_since_check = 0
self._rotations = 0
self._lines_written = 0
self._last_ts: float | None = None
self._open()
# -- lifecycle ----------------------------------------------------
def _open(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
self._fh = self.path.open("a", encoding="utf-8")
def close(self) -> None:
with self._lock:
if self._fh is not None:
try:
self._fh.close()
finally:
self._fh = None
# -- write --------------------------------------------------------
def write(self, record: dict[str, Any]) -> None:
"""Append one JSON object as a line. Triggers rotation if oversized."""
line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
with self._lock:
if self._fh is None:
return
try:
self._fh.write(line)
self._fh.flush()
except Exception:
# Best-effort: a failed write must not crash the pubsub
# handler. Caller has no way to react anyway.
return
self._lines_written += 1
ts = record.get("ts")
if isinstance(ts, (int, float)):
self._last_ts = float(ts)
self._writes_since_check += 1
if self._writes_since_check >= self.check_every:
self._writes_since_check = 0
self._maybe_rotate()
# -- rotation -----------------------------------------------------
def _maybe_rotate(self) -> None:
# Caller holds self._lock.
try:
size = os.fstat(self._fh.fileno()).st_size
except OSError:
return
if size < self.max_bytes:
return
self._rotate_locked()
def _rotate_locked(self) -> None:
# Close, gzip-rename, reopen empty, prune oldest archives.
try:
self._fh.close()
except Exception:
pass
self._fh = None
# Microsecond-resolution timestamp + per-instance counter so back-
# to-back rotations (small max_bytes, repeated `force_rotate()`,
# or chatty test loops) get unique archive filenames. The lex
# sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
# which `_prune_archives()` and `log_query._iter_jsonl()` both
# rely on.
stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
try:
with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
shutil.copyfileobj(src, dst, length=1024 * 1024)
self.path.unlink()
except Exception:
# Rotation is best-effort. If gzip fails, leave the file
# in place and re-open it; we'll try again next check.
pass
self._open()
self._rotations += 1
self._prune_archives()
def _prune_archives(self) -> None:
# Match siblings of self.path.name with `.jsonl.gz` suffix.
prefix = self.path.stem # "logs" for "logs.jsonl"
# Archive filenames are already lexicographically chronological.
# Prune by name, not mtime, so copied/restored files don't reorder.
archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
excess = len(archives) - self.keep_archives
for old in archives[: max(0, excess)]:
try:
old.unlink()
except OSError:
pass
def force_rotate(self) -> None:
"""Test/admin hook: rotate immediately regardless of size."""
with self._lock:
if self._fh is not None:
self._rotate_locked()
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
try:
size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
except OSError:
size = 0
return {
"path": str(self.path),
"size": size,
"lines": self._lines_written,
"last_ts": self._last_ts,
"rotations": self._rotations,
}
@@ -46,23 +46,7 @@ class SerialSession:
def _drain(session: SerialSession) -> None:
"""Reader thread: line-by-line pull stdout into buffer.
Each line is also published to the `meshtastic.serial.line` pubsub
topic so the persistent recorder can capture it without holding its
own port. This is the text-mode tap path: when no SerialInterface is
open, the firmware emits full formatted lines (level + clock + uptime
+ thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
fan them out to whoever is listening. Pubsub is best-effort —
publish failures must never block the reader.
"""
# Lazy import: pubsub isn't required just to import this module
# (e.g., during static analysis), and we want a clean test surface.
try:
from pubsub import pub # type: ignore[import-untyped]
except Exception: # pragma: no cover - defensive
pub = None
"""Reader thread: line-by-line pull stdout into buffer."""
assert session.proc.stdout is not None
try:
for line in session.proc.stdout:
@@ -70,16 +54,6 @@ def _drain(session: SerialSession) -> None:
with session.lock:
session.buffer.append(line_stripped)
session.total_lines += 1
if pub is not None:
try:
pub.sendMessage(
"meshtastic.serial.line",
line=line_stripped,
port=session.port,
)
except Exception:
# A subscriber raising must not break the reader.
pass
except Exception: # pragma: no cover - defensive
pass
finally:
+2 -274
View File
@@ -6,7 +6,6 @@ etc.). Business logic does not live here.
from __future__ import annotations
import logging
from typing import Any
from mcp.server.fastmcp import FastMCP
@@ -15,38 +14,17 @@ from . import (
admin,
boards,
devices,
fixtures,
flash,
hw_tools,
info,
log_query,
registry,
serial_session,
)
from . import userprefs as userprefs_mod
from .recorder import get_recorder
log = logging.getLogger(__name__)
app = FastMCP("meshtastic-mcp")
def _start_recorder() -> None:
# Persistent device-log capture. Starts on first import — pubsub fan-out
# is process-global, so subscribing here captures every active interface
# (whether opened by an MCP tool, a pytest fixture, or a serial_session).
# Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
# for the full design. Recorder startup is best-effort: an unwritable
# log dir or pubsub mismatch should not take the MCP server down.
try:
get_recorder().start()
except Exception as exc:
log.warning("Failed to start persistent recorder: %s", exc)
_start_recorder()
# ---------- Discovery & metadata ------------------------------------------
@@ -97,7 +75,6 @@ def build(
env: str,
with_manifest: bool = True,
userprefs: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Build firmware for one env via `pio run -e <env>`.
@@ -109,21 +86,8 @@ def build(
build via userPrefs.jsonc injection. The file is restored after the build
completes. Use `userprefs_manifest` to discover available keys. Use
`userprefs_set` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
`build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
`[heap N]` prefix on every log line. The recorder picks the prefix up
automatically and synthesizes a high-resolution heap timeline that
`telemetry_timeline(field="free_heap")` can read alongside the normal
~60 s LocalStats packets. Pair with `/leakhunt` for classification.
"""
return flash.build(
env,
with_manifest=with_manifest,
userprefs_overrides=userprefs,
build_flags=build_flags,
)
return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
@app.tool()
@@ -141,7 +105,6 @@ def pio_flash(
port: str,
confirm: bool = False,
userprefs: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
@@ -151,19 +114,8 @@ def pio_flash(
`userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
build via userPrefs.jsonc injection; restored after upload.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
to actually land in the uploaded firmware — without it, the implicit
rebuild relinks without the env var and silently drops them.
"""
return flash.flash(
env,
port,
confirm=confirm,
userprefs_overrides=userprefs,
build_flags=build_flags,
)
return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
@app.tool()
@@ -782,227 +734,3 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
"""Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
return hw_tools.picotool_raw(args, confirm=confirm)
# ---------- Persistent device-log capture (recorder) ----------------------
#
# The recorder is autouse — it starts at server import and continuously
# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
# tools are query-only over those files, plus a few lifecycle controls.
@app.tool()
def logs_window(
start: str = "-15m",
end: str = "now",
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines from the persistent recorder.
Filters by time window, regex over the line, level (single or
pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
interface port. Returns up to max_lines most-recent matches.
Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
Note: lines arriving via the LogRecord protobuf path (when
set_debug_log_api(True) is on) come without level prefix — the
meshtastic Python lib drops record.level before fan-out. For those,
`level` filter won't match; use `grep` instead.
"""
return log_query.logs_window(
start=start,
end=end,
grep=grep,
level=level,
tag=tag,
port=port,
max_lines=max_lines,
)
@app.tool()
def telemetry_timeline(
window: str = "1h",
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Time series of one telemetry field, downsampled to <= max_points.
`variant` ∈ device, local, environment, power, airQuality, health, host.
`field` accepts snake_case or camelCase; common aliases (free_heap ↔
heap_free_bytes) are normalized.
Returns slope_per_min (linear-regression slope, units/minute) so a
leak detector can read one number — negative slope on free_heap over
a long window indicates a real leak.
LocalStats variant ("local") cadence is ~60 s (whatever the device's
`device_update_interval` is set to), so a 1 h window gives ~60 raw
points. Bucket-mean downsampling preserves shape.
"""
return log_query.telemetry_timeline(
window=window,
variant=variant,
field=field,
port=port,
max_points=max_points,
)
@app.tool()
def packets_window(
start: str = "-5m",
end: str = "now",
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
"""Recent mesh packets recorded by the recorder.
Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
size + first 64 bytes hex) — full payload bytes are not stored.
`portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
"""
return log_query.packets_window(
start=start,
end=end,
portnum=portnum,
from_node=from_node,
to_node=to_node,
max=max,
)
@app.tool()
def events_window(
start: str = "-1h",
end: str = "now",
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
"""Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
`kind` ∈ recorder_start, recorder_pause, recorder_resume,
connection_established, connection_lost, node_updated, mark.
Pipe-separated sets ("connection_lost|connection_established") work.
"""
return log_query.events_window(start=start, end=end, kind=kind, max=max)
@app.tool()
def mark_event(
label: str,
note: str | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Drop a named marker into events.jsonl AND logs.jsonl.
Useful for aligning a timeline around a known stimulus: call before
and after a stress workload, then query telemetry_timeline /
logs_window with the markers' timestamps as bounds.
The marker also lands in logs.jsonl with level=MARK so a single
grep over logs picks it up.
"""
return get_recorder().mark_event(label=label, note=note, data=data)
@app.tool()
def recorder_status() -> dict[str, Any]:
"""Return recorder runtime info: running, paused, file sizes, last_ts per stream.
Use this to sanity-check that capture is working before you trust a
`logs_window` / `telemetry_timeline` result.
"""
return get_recorder().status()
@app.tool()
def recorder_pause(reason: str | None = None) -> dict[str, Any]:
"""Pause writes to all four streams. Pubsub subscriptions stay active —
we just drop events on the floor while paused. Resume with `recorder_resume`.
Use when capturing a known-good baseline that you don't want to
pollute with pre-test noise. Default state is recording; this is
rarely needed.
"""
get_recorder().pause(reason=reason)
return {"ok": True, "paused": True, "reason": reason}
@app.tool()
def recorder_resume() -> dict[str, Any]:
"""Resume writes after `recorder_pause`. No-op if already running."""
get_recorder().resume()
return {"ok": True, "paused": False}
@app.tool()
def recorder_export(
start: str,
end: str,
dest_dir: str,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of the recorder's streams into `dest_dir`.
Writes one uncompressed JSONL per requested stream (logs / telemetry /
packets / events). Useful for: attaching to a bug report, feeding a
notebook, or backfilling Datadog after the fact.
"""
return log_query.export(
start=start,
end=end,
dest_dir=dest_dir,
streams=streams,
)
# ---------- Fixture / test-data push --------------------------------------
@app.tool()
def push_fake_nodedb(
size: int,
target: str = "portduino",
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Push a fake-NodeDB v25 fixture (250/500/1000/2000 nodes) onto a device.
Two transports:
target="portduino" — file copy to ~/.portduino/<portduino_config>/prefs/nodes.proto.
Fast, no device connection needed.
target="hardware" — XModem upload over serial/BLE to /prefs/nodes.proto.
Requires `port` + `confirm=True`. Triggers a reboot
so loadFromDisk picks up the new file at next boot.
Compiles a fresh-timestamp proto from the committed JSONL seed under
test/fixtures/nodedb/seed_v25_<N>.jsonl each invocation, so the loaded
NodeDB always looks "recent" to the connecting phone. Structural data
(names, IDs, positions, telemetries) is deterministic per the seed.
Override the JSONL via `custom_seed_jsonl` to push a hand-edited scenario.
"""
return fixtures.push_fake_nodedb(
size=size,
target=target, # type: ignore[arg-type]
port=port,
portduino_config=portduino_config,
backup_existing=backup_existing,
confirm=confirm,
reboot_after=reboot_after,
custom_seed_jsonl=custom_seed_jsonl,
)
-88
View File
@@ -1,88 +0,0 @@
"""Unit tests for the `build_flags` injection on `flash.build()`.
We don't actually run pio here — too slow, requires hardware-aware envs.
We test the translation layer (`_build_flags_env`) and that the env vars
are threaded through pio.run correctly via mock.
"""
from __future__ import annotations
from unittest.mock import patch
from meshtastic_mcp import flash, pio
class TestBuildFlagsEnv:
def test_simple_value(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": 1})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
def test_string_value(self) -> None:
out = flash._build_flags_env({"FOO": "bar"})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
def test_bool_true_is_bare_flag(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": True})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
def test_bool_false_dropped(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
def test_none_dropped(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": None})
assert out == {}
def test_multiple_combined(self) -> None:
out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
# Order isn't guaranteed in dict iteration, so check membership.
flags = out["PLATFORMIO_BUILD_FLAGS"].split()
assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
class TestBuildPropagatesFlags:
def test_extra_env_passed_to_pio_run(self) -> None:
# Mock pio.run so we don't actually invoke pio. Capture extra_env.
captured = {}
class _StubResult:
returncode = 0
stdout = ""
stderr = ""
duration_s = 0.1
def _stub(args, **kwargs):
captured["args"] = args
captured["kwargs"] = kwargs
return _StubResult()
with patch.object(pio, "run", side_effect=_stub):
with patch.object(flash, "_artifacts_for", return_value=[]):
out = flash.build(
"fake-env",
with_manifest=False,
build_flags={"DEBUG_HEAP": 1},
)
assert captured["args"] == ["run", "-e", "fake-env"]
assert captured["kwargs"]["extra_env"] == {
"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
}
assert out["build_flags"] == {"DEBUG_HEAP": 1}
def test_no_flags_means_no_extra_env(self) -> None:
captured = {}
class _StubResult:
returncode = 0
stdout = ""
stderr = ""
duration_s = 0.1
def _stub(args, **kwargs):
captured["kwargs"] = kwargs
return _StubResult()
with patch.object(pio, "run", side_effect=_stub):
with patch.object(flash, "_artifacts_for", return_value=[]):
flash.build("fake-env", with_manifest=False)
assert captured["kwargs"]["extra_env"] is None
@@ -1,364 +0,0 @@
"""Tests for the fake-NodeDB fixture pipeline (bin/gen-fake-nodedb-seed.py
+ bin/seed-json-to-proto.py + mcp-server fixtures.push_fake_nodedb).
Lives under tests/unit/ because none of these touch real hardware — they
shell out to the bin/ scripts and decode the resulting protobufs in-process.
"""
from __future__ import annotations
import json
import pathlib
import subprocess
import sys
import time
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
SEED_GEN = REPO_ROOT / "bin" / "gen-fake-nodedb-seed.py"
COMPILE = REPO_ROOT / "bin" / "seed-json-to-proto.py"
FIXTURES_DIR = REPO_ROOT / "test" / "fixtures" / "nodedb"
# Ensure the locally-generated Python protobuf bindings are importable.
# These live under `meshtastic_v25` (not `meshtastic`) so they don't shadow
# the PyPI `meshtastic` package that the rest of the mcp-server depends on.
_BINDINGS_DIR = REPO_ROOT / "bin" / "_generated"
if _BINDINGS_DIR.is_dir() and str(_BINDINGS_DIR) not in sys.path:
sys.path.insert(0, str(_BINDINGS_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import (
NodeDatabase, # type: ignore[import-not-found]
)
except ImportError:
NodeDatabase = None # type: ignore[assignment]
def _require_v25_bindings() -> None:
if NodeDatabase is None:
pytest.skip(
"v25 Python protobuf bindings missing; run `./bin/regen-py-protos.sh`."
)
if "positions" not in NodeDatabase.DESCRIPTOR.fields_by_name:
pytest.skip(
"Loaded NodeDatabase predates v25 — run `./bin/regen-py-protos.sh`."
)
def _run(cmd: list[str]) -> None:
subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
# ---------------------------------------------------------------------------
# Seed generator: deterministic for given --seed (no wall-clock dependence).
# ---------------------------------------------------------------------------
def test_seed_generator_is_deterministic(tmp_path: pathlib.Path) -> None:
a = tmp_path / "a.jsonl"
b = tmp_path / "b.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"100",
"--seed",
"42",
"--out",
str(a),
]
)
# Sleep so any sneaky wall-clock leak in the generator would surface as
# a byte diff between the two runs.
time.sleep(0.8)
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"100",
"--seed",
"42",
"--out",
str(b),
]
)
assert a.read_bytes() == b.read_bytes()
def test_seed_generator_meta_line(tmp_path: pathlib.Path) -> None:
out = tmp_path / "seed.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"50",
"--seed",
"1",
"--out",
str(out),
]
)
lines = out.read_text(encoding="utf-8").splitlines()
assert len(lines) == 51 # 1 meta + 50 nodes
meta = json.loads(lines[0])
assert "_meta" in meta
assert meta["_meta"]["version"] == 25
assert meta["_meta"]["count"] == 50
assert meta["_meta"]["seed"] == 1
def test_seed_only_uses_active_hardware_and_roles(tmp_path: pathlib.Path) -> None:
"""Confirm no deprecated roles + no off-list HW models leak through."""
out = tmp_path / "seed.jsonl"
_run(
[
sys.executable,
str(SEED_GEN),
"--count",
"500",
"--seed",
"7",
"--out",
str(out),
]
)
forbidden_roles = {"ROUTER_CLIENT", "REPEATER"}
forbidden_hw = {
"TLORA_V1",
"TLORA_V2",
"TLORA_V1_1P3",
"TLORA_V2_1_1P6",
"TLORA_V2_1_1P8",
"HELTEC_V1",
"HELTEC_V2_0",
"HELTEC_V2_1",
"TBEAM",
"TBEAM_V0P7",
"NANO_G1",
"NANO_G1_EXPLORER",
"NANO_G2_ULTRA",
"STATION_G1",
"STATION_G2",
"PORTDUINO",
"ANDROID_SIM",
"DIY_V1",
"LORA_RELAY_V1",
"NRF52840_PCA10059",
"NRF52_UNKNOWN",
"DR_DEV",
"GENIEBLOCKS",
"M5STACK",
"RP2040_LORA",
"PPR",
}
for raw in out.read_text(encoding="utf-8").splitlines()[1:]:
node = json.loads(raw)
assert node["role"] not in forbidden_roles, f"deprecated role: {node['role']}"
assert (
node["hw_model"] not in forbidden_hw
), f"non-tier-1 HW: {node['hw_model']}"
# ---------------------------------------------------------------------------
# Compile step + committed seeds.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("size", [250, 500, 1000, 2000])
def test_committed_seed_compiles_and_decodes(size: int, tmp_path: pathlib.Path) -> None:
_require_v25_bindings()
proto = tmp_path / "out.proto"
jsonl = FIXTURES_DIR / f"seed_v25_{size:04d}.jsonl"
if not jsonl.is_file():
pytest.skip(f"{jsonl} not present — run ./bin/regen-fake-nodedbs.sh")
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(proto)])
db = NodeDatabase()
db.ParseFromString(proto.read_bytes())
assert db.version == 25
assert len(db.nodes) == size
nums = {n.num for n in db.nodes}
assert len(nums) == size, "node numbers must be unique"
assert all(n.long_name and n.short_name for n in db.nodes)
assert all(len(n.long_name) <= 24 for n in db.nodes) # max_size:25 - NUL
# Coverage sanity (±10pp tolerance for binomial fluctuation).
def in_range(actual: int, expected_ratio: float, tol_pp: float = 0.10) -> bool:
lo = max(0, int((expected_ratio - tol_pp) * size))
hi = min(size, int((expected_ratio + tol_pp) * size))
return lo <= actual <= hi
assert in_range(len(db.positions), 0.85)
assert in_range(len(db.telemetry), 0.70)
assert in_range(len(db.environment), 0.25)
assert in_range(len(db.status), 0.40)
def test_compile_freshens_timestamps(tmp_path: pathlib.Path) -> None:
"""Same JSONL compiled twice → identical structure, different timestamps."""
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present — run ./bin/regen-fake-nodedbs.sh")
a = tmp_path / "a.proto"
b = tmp_path / "b.proto"
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(a)])
time.sleep(1.2)
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(b)])
da = NodeDatabase()
db_ = NodeDatabase()
da.ParseFromString(a.read_bytes())
db_.ParseFromString(b.read_bytes())
# Zero out timestamp fields and confirm everything else is byte-identical.
for d in (da, db_):
for n in d.nodes:
n.last_heard = 0
for p in d.positions:
p.position.time = 0
assert da.SerializeToString() == db_.SerializeToString()
# Re-load fresh copies to confirm timestamps actually moved.
aa = NodeDatabase()
bb = NodeDatabase()
aa.ParseFromString(a.read_bytes())
bb.ParseFromString(b.read_bytes())
aa_max = max(n.last_heard for n in aa.nodes if n.last_heard)
bb_max = max(n.last_heard for n in bb.nodes if n.last_heard)
assert bb_max >= aa_max
assert bb_max - aa_max < 5 # within a few seconds
def test_compile_pinned_now_epoch_is_byte_identical(tmp_path: pathlib.Path) -> None:
"""With --now-epoch pinned, two compiles produce identical bytes."""
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present")
a = tmp_path / "a.proto"
b = tmp_path / "b.proto"
for o in (a, b):
_run(
[
sys.executable,
str(COMPILE),
"--in",
str(jsonl),
"--now-epoch",
"1700000000",
"--out",
str(o),
]
)
assert a.read_bytes() == b.read_bytes()
def test_compile_timestamps_are_recent(tmp_path: pathlib.Path) -> None:
_require_v25_bindings()
jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not jsonl.is_file():
pytest.skip("250-node seed not present")
out = tmp_path / "out.proto"
_run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(out)])
db = NodeDatabase()
db.ParseFromString(out.read_bytes())
now = int(time.time())
# No timestamp older than 7 days, none in the future.
for n in db.nodes:
if n.last_heard:
assert now - 7 * 86400 <= n.last_heard <= now
# At least half should be within the last hour
# (matches expovariate(mean=3600s)).
recent = sum(1 for n in db.nodes if n.last_heard and n.last_heard >= now - 3600)
assert recent >= 0.4 * len(db.nodes)
def test_compile_hand_edit_round_trip(tmp_path: pathlib.Path) -> None:
"""Edit one JSONL line, recompile, confirm edit appears in the proto."""
_require_v25_bindings()
src = FIXTURES_DIR / "seed_v25_0250.jsonl"
if not src.is_file():
pytest.skip("250-node seed not present")
dst = tmp_path / "edited.jsonl"
lines = src.read_text(encoding="utf-8").splitlines()
# Find a node that already has telemetry so the index relationship is
# easy to assert on the other side.
edit_idx = None
for i, raw in enumerate(lines[1:], start=1):
node = json.loads(raw)
if node.get("telemetry") is not None:
edit_idx = i
break
assert edit_idx is not None, "expected at least one node with telemetry"
node = json.loads(lines[edit_idx])
target_num = int(node["num"], 16)
node["long_name"] = "Hand Edited Node"
node["telemetry"] = {
"battery_level": 42,
"voltage": 3.71,
"channel_utilization": 0.0,
"air_util_tx": 0.0,
"uptime_seconds": 1,
}
lines[edit_idx] = json.dumps(node, ensure_ascii=False, sort_keys=True)
dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
out = tmp_path / "out.proto"
_run([sys.executable, str(COMPILE), "--in", str(dst), "--out", str(out)])
db = NodeDatabase()
db.ParseFromString(out.read_bytes())
edited = next((n for n in db.nodes if n.num == target_num), None)
assert edited is not None
assert edited.long_name == "Hand Edited Node"
tel = next((t for t in db.telemetry if t.num == target_num), None)
assert tel is not None
assert tel.device_metrics.battery_level == 42
# ---------------------------------------------------------------------------
# Misc smoke checks on the module surface.
# ---------------------------------------------------------------------------
def test_crc16_ccitt_matches_known_vectors() -> None:
"""Sanity-check the hand-rolled CRC16-CCITT matches well-known vectors.
Test vectors from the XModem-CRC spec (init=0, poly=0x1021):
crc16("123456789") = 0x31C3
crc16("") = 0x0000
"""
from meshtastic_mcp.fixtures import _crc16_ccitt
assert _crc16_ccitt(b"") == 0x0000
assert _crc16_ccitt(b"123456789") == 0x31C3
def test_push_fake_nodedb_rejects_invalid_size() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="size must be one of"):
push_fake_nodedb(size=999, target="portduino") # type: ignore[arg-type]
def test_push_fake_nodedb_hardware_requires_confirm() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="confirm=True"):
push_fake_nodedb(size=250, target="hardware", port="/dev/cu.fake")
def test_push_fake_nodedb_hardware_requires_port() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="requires a port"):
push_fake_nodedb(size=250, target="hardware", confirm=True)
def test_push_fake_nodedb_hardware_rejects_tcp_port() -> None:
from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
with pytest.raises(FixtureError, match="not supported"):
push_fake_nodedb(
size=250, target="hardware", confirm=True, port="tcp://localhost:4403"
)
-548
View File
@@ -1,548 +0,0 @@
"""Unit tests for the persistent device-log recorder.
Hardware-free: drives the Recorder through its `_on_*` handlers with
synthetic packet/line dicts, then queries via log_query. Validates
prefix parsing, telemetry variant dispatch, marker round-trip, time
window filtering, downsampling, slope estimation, and gzip rotation
+ archive pruning.
"""
from __future__ import annotations
import gzip
import json
import logging
import os
import time
from pathlib import Path
import pubsub
import pytest
from meshtastic_mcp import log_query
from meshtastic_mcp.recorder.parsers import (
extract_telemetry,
interface_label,
parse_log_line,
summarize_packet,
)
from meshtastic_mcp.recorder.recorder import Recorder
from meshtastic_mcp.recorder.rotating import _RotatingJsonl
# -- isolation: every test gets a fresh Recorder + tmp dir -----------
@pytest.fixture
def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
# Redirect both the Recorder and the module-level singleton lookup
# to the same tmp dir so log_query queries the same files we write.
monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
monkeypatch.setattr(
"meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
)
r = Recorder(base_dir=tmp_path)
r.start()
monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
yield r
r.stop()
class _FakeIface:
devPath = "/dev/cu.fake"
# -- parsers ---------------------------------------------------------
class TestParseLogLine:
def test_full_prefix(self) -> None:
out = parse_log_line("INFO | 12:34:56 12345 [Main] Booting")
assert out["level"] == "INFO"
assert out["tag"] == "Main"
assert out["uptime_s"] == 12345
assert out["msg"] == "Booting"
assert out["clock"] == "12:34:56"
def test_invalid_clock(self) -> None:
out = parse_log_line("WARN | ??:??:?? 7 [SerialConsole] Boot")
assert out["level"] == "WARN"
assert out["clock"] == "??:??:??"
assert out["uptime_s"] == 7
def test_no_thread_bracket(self) -> None:
out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
assert out["level"] == "DEBUG"
assert out.get("tag") is None
assert out["msg"] == "raw message body"
def test_bare_message(self) -> None:
# LogRecord.message path — no level prefix at all.
out = parse_log_line("just a bare message")
assert "level" not in out or out.get("level") is None
assert out["line"] == "just a bare message"
def test_empty(self) -> None:
assert parse_log_line("") == {"line": ""}
def test_debug_heap_prefix_extracted(self) -> None:
out = parse_log_line("INFO | 12:34:56 12345 [Main] [heap 92344] Booting")
assert out["level"] == "INFO"
assert out["tag"] == "Main"
assert out["heap_free"] == 92344
assert out["msg"] == "Booting"
def test_debug_heap_prefix_on_bare_line(self) -> None:
# LogRecord.message path: no level prefix but still has [heap N].
out = parse_log_line("[heap 12345] some message")
assert out["heap_free"] == 12345
assert out["msg"] == "some message"
def test_thread_leak_event(self) -> None:
out = parse_log_line(
"HEAP | 00:00:01 100 [Power] [heap 90000] "
"------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
)
assert out["level"] == "HEAP"
assert out["heap_free"] == 90000
ev = out["heap_event"]
assert ev["kind"] == "leaked"
assert ev["thread"] == "MeshPacket"
assert ev["before"] == 92344
assert ev["after"] == 90000
assert ev["delta"] == -2344
def test_thread_freed_event(self) -> None:
out = parse_log_line(
"++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
)
ev = out["heap_event"]
assert ev["kind"] == "freed"
assert ev["thread"] == "Router"
assert ev["delta"] == 500
def test_heap_status_periodic(self) -> None:
out = parse_log_line(
"HEAP | 00:00:30 30 [Power] "
"Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
)
assert out["heap_free"] == 92344
assert out["heap_total"] == 200000
assert out["heap_delta"] == -128
class TestRecorderDebugHeapSynthesis:
def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
# When a log line carries [heap N], the recorder should also
# emit a synthesized telemetry row tagged source=debug_heap.
recorder._on_log_line(
"INFO | 00:00:00 1 [Main] [heap 88888] hello",
_FakeIface(),
)
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
assert len(synth) == 1
assert synth[0]["fields"]["heap_free_bytes"] == 88888
assert synth[0]["variant"] == "local"
def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
recorder._on_log_line(
"HEAP | 00:00:30 30 [Power] "
"Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
_FakeIface(),
)
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
assert synth[-1]["fields"]["heap_free_bytes"] == 50000
assert synth[-1]["fields"]["heap_total_bytes"] == 200000
def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
# Plain log line (no [heap N], no Heap status) — telemetry.jsonl
# should NOT gain a synth row.
before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
recorder._on_log_line("INFO | 00:00:00 1 [Main] just a message", _FakeIface())
after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
assert after == before
def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
recorder._on_log_line(
"HEAP | 00:00:01 100 [Power] [heap 90000] "
"------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
_FakeIface(),
)
rows = [
json.loads(r)
for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
if r
]
evt_rows = [r for r in rows if r.get("heap_event")]
assert len(evt_rows) == 1
assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
assert evt_rows[0]["heap_event"]["delta"] == -2344
class TestSerialTap:
def test_serial_line_records_log_and_synthesizes_heap(
self, recorder: "Recorder"
) -> None:
recorder._on_serial_line(
"INFO | 00:00:00 5 [Main] [heap 88888] tap-line",
port="/dev/cu.tap",
)
logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
log_rows = [json.loads(r) for r in logs if r]
# Find the row from this call (port=/dev/cu.tap, role=serial_session)
tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
assert len(tap_rows) == 1
assert tap_rows[0]["role"] == "serial_session"
assert tap_rows[0]["level"] == "INFO"
assert tap_rows[0]["tag"] == "Main"
assert tap_rows[0]["heap_free"] == 88888
synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
assert len(synth) == 1
assert synth[0]["fields"]["heap_free_bytes"] == 88888
assert synth[0]["role"] == "serial_session"
def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
recorder._on_serial_line(
"HEAP | 00:00:30 30 [Power] [heap 53484] "
"------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
port="/dev/cu.tap",
)
rows = [
json.loads(r)
for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
if r
]
evt = [r for r in rows if r.get("heap_event")]
assert len(evt) == 1
assert evt[0]["heap_event"]["thread"] == "Router"
assert evt[0]["heap_event"]["delta"] == -128
# Heap also synthesized.
telem = (recorder.base_dir / "telemetry.jsonl").read_text()
assert '"source":"debug_heap_serial"' in telem
def test_serial_line_pause(self, recorder: "Recorder") -> None:
recorder.pause("baseline")
recorder._on_serial_line(
"INFO | 00:00:00 1 [t] [heap 1000] dropped",
port="/dev/cu.tap",
)
# Only the pause event row should exist; no tap row.
logs = (recorder.base_dir / "logs.jsonl").read_text()
assert "dropped" not in logs
def test_serial_line_handler_swallows_exceptions(
self, recorder: "Recorder"
) -> None:
# Hostile input — should not raise.
recorder._on_serial_line(None, port="/dev/cu.tap") # type: ignore[arg-type]
recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap") # type: ignore[arg-type]
# Survived.
class TestExtractTelemetry:
def test_local_stats_camel(self) -> None:
pkt = {
"decoded": {
"telemetry": {
"localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
}
}
}
out = extract_telemetry(pkt)
assert out is not None
assert out["variant"] == "local"
assert out["fields"]["heap_free_bytes"] == 600
def test_device_metrics_snake(self) -> None:
pkt = {
"decoded": {
"telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
}
}
out = extract_telemetry(pkt)
assert out is not None
assert out["variant"] == "device"
assert out["fields"]["battery_level"] == 88
def test_unknown_variant_returns_none(self) -> None:
assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
assert extract_telemetry({}) is None
assert extract_telemetry({"decoded": "not-a-dict"}) is None
class TestSummarizePacket:
def test_text_with_payload(self) -> None:
pkt = {
"fromId": "!abc",
"toId": "!def",
"decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
"hopLimit": 3,
}
out = summarize_packet(pkt)
assert out["from_node"] == "!abc"
assert out["portnum"] == "TEXT_MESSAGE_APP"
assert out["payload_size"] == 5
assert out["payload_hex_prefix"] == "68656c6c6f"
def test_no_decoded(self) -> None:
out = summarize_packet({"fromId": "!abc"})
assert out["from_node"] == "!abc"
assert out["portnum"] is None
class TestInterfaceLabel:
def test_serial(self) -> None:
assert interface_label(_FakeIface()) == {
"port": "/dev/cu.fake",
"role": "serial",
}
def test_tcp(self) -> None:
class T:
hostname = "node.lan"
portNumber = 4403
assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
def test_unknown(self) -> None:
assert interface_label(object()) == {"port": "object", "role": None}
def test_none(self) -> None:
assert interface_label(None) == {"port": None, "role": None}
# -- recorder write side ---------------------------------------------
class TestRecorderWrites:
def test_log_line_is_recorded(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:34:56 99 [T] hi", _FakeIface())
path = recorder.base_dir / "logs.jsonl"
rows = [json.loads(line) for line in path.read_text().splitlines() if line]
# First row is recorder_start_event mirror? No — that's events.jsonl only.
assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
# _on_telemetry alone — only telemetry.jsonl
recorder._on_telemetry(
{
"fromId": "!abc",
"decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
},
_FakeIface(),
)
telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
assert any('"variant":"local"' in r for r in telem_rows)
def test_packets_summary(self, recorder: Recorder) -> None:
recorder._on_receive(
{
"fromId": "!abc",
"toId": "!def",
"decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
},
_FakeIface(),
)
rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
def test_mark_event_round_trip(self, recorder: Recorder) -> None:
out = recorder.mark_event("checkpoint", note="midpoint")
assert "ts" in out
events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
def test_pause_drops_writes(self, recorder: Recorder) -> None:
before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
recorder.pause(reason="baseline")
recorder._on_log_line("INFO | 00:00:00 1 [t] swallowed", _FakeIface())
after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
assert after == before
recorder.resume()
recorder._on_log_line("INFO | 00:00:00 2 [t] kept", _FakeIface())
post_resume = (recorder.base_dir / "logs.jsonl").read_text()
assert "kept" in post_resume
def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
# If the writer dies, the pubsub callback must NOT raise — that
# would crash the meshtastic receive thread.
bad_packet = object() # not a dict
recorder._on_receive(bad_packet, _FakeIface()) # type: ignore[arg-type]
recorder._on_telemetry(bad_packet, _FakeIface()) # type: ignore[arg-type]
recorder._on_log_line(None, _FakeIface()) # type: ignore[arg-type]
# No assertion needed — survival is the test.
# -- log_query read side ---------------------------------------------
class TestLogQuery:
def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:00:00 1 [A] alpha", _FakeIface())
recorder._on_log_line("WARN | 12:00:01 2 [B] bravo failed", _FakeIface())
recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
assert out["total_matched"] == 2
levels = {r["level"] for r in out["lines"]}
assert levels == {"WARN", "ERROR"}
out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
assert out2["total_matched"] == 2
def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
recorder._on_log_line("INFO | 12:00:00 1 [A] alpha", _FakeIface())
with pytest.raises(ValueError, match="invalid grep regex"):
log_query.logs_window(start="-1m", grep="(")
def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
# Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
base_ts = time.time() - 60
for i in range(100):
recorder._files["telemetry"].write(
{
"ts": base_ts + i * 0.5,
"port": "/dev/cu.fake",
"role": "serial",
"from_node": "!abc",
"variant": "local",
"fields": {"heap_free_bytes": 10000 - i},
}
)
out = log_query.telemetry_timeline(
window="2m", variant="local", field="free_heap", max_points=10
)
assert out["samples"] == 100
assert len(out["points"]) <= 10
# Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
assert out["slope_per_min"] is not None
assert out["slope_per_min"] < -100
def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
recorder._on_log_line("INFO | 00:00:00 1 [t] one", _FakeIface())
recorder._on_log_line("INFO | 00:00:00 2 [t] two", _FakeIface())
dest = tmp_path / "bundle"
out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
assert (dest / "logs.jsonl").exists()
assert "logs" in out["paths"]
# -- time parser -----------------------------------------------------
class TestParseTime:
def test_relative(self) -> None:
now = 1_000_000.0
assert log_query._parse_time("-15m", now=now) == now - 900
assert log_query._parse_time("-2h", now=now) == now - 7200
assert log_query._parse_time("-1d", now=now) == now - 86400
def test_now_and_epoch(self) -> None:
now = 1_000_000.0
assert log_query._parse_time("now", now=now) == now
assert log_query._parse_time(now) == now
def test_iso(self) -> None:
ts = log_query._parse_time("2026-01-01T00:00:00Z")
assert isinstance(ts, float) and ts > 1_700_000_000
def test_naive_iso_assumes_utc(self) -> None:
assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
"2026-01-01T00:00:00Z"
)
def test_invalid(self) -> None:
with pytest.raises(ValueError):
log_query._parse_time("not a time")
# -- rotation --------------------------------------------------------
class TestRotation:
def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
for i in range(100):
r.write({"ts": float(i), "i": i, "pad": "x" * 40})
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert archives, "expected at least one rotation"
# Archive content is valid gzip + valid JSONL
with gzip.open(archives[0], "rt") as fh:
first = json.loads(fh.readline())
assert "ts" in first
def test_archive_pruning(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
# Force several rotations.
for _ in range(8):
for i in range(20):
r.write({"ts": float(i), "pad": "x" * 30})
r.force_rotate()
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, keep_archives=2)
old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
for archive in (old, mid, new):
with gzip.open(archive, "wt", encoding="utf-8") as fh:
fh.write('{"ts":1}\n')
# Deliberately scramble mtimes so lexicographic filename order is
# the only stable chronological signal.
os.utime(old, (300, 300))
os.utime(mid, (100, 100))
os.utime(new, (200, 200))
r._prune_archives()
r.close()
archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
assert archives == [mid.name, new.name]
def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
path = tmp_path / "rot.jsonl"
r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
r.write({"ts": 1.0, "msg": "tiny"})
r.force_rotate()
r.write({"ts": 2.0, "msg": "after-rotate"})
r.close()
archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
assert len(archives) == 1
assert path.exists()
assert "after-rotate" in path.read_text()
class TestRecorderLocks:
def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
out = recorder.force_rotate_all()
assert out["running"] is True
assert out["files"]
def test_wire_pubsub_logs_subscription_failure(
self,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
class FailingPubSubMock:
def subscribe(self, callback: object, topic: str) -> None:
raise RuntimeError("boom")
monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
recorder = Recorder(base_dir=tmp_path)
with caplog.at_level(logging.WARNING):
recorder._wire_pubsub()
assert (
"Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
)
+6 -9
View File
@@ -2,7 +2,7 @@
; https://docs.platformio.org/page/projectconf.html
[platformio]
default_envs = heltec-v3
default_envs = tbeam
extra_configs =
variants/*/*.ini
@@ -17,7 +17,6 @@ test_build_src = true
extra_scripts =
pre:bin/platformio-pre.py
bin/platformio-custom.py
post:extra_scripts/nrf54l15_linker.py
; note: we add src to our include search path so that lmic_project_config can override
; note: TINYGPS_OPTION_NO_CUSTOM_FIELDS is VERY important. We don't use custom fields and somewhere in that pile
; of code is a heap corruption bug!
@@ -50,7 +49,6 @@ build_flags = -Wno-missing-field-initializers
-DRADIOLIB_EXCLUDE_PAGER=1
-DRADIOLIB_EXCLUDE_FSK4=1
-DRADIOLIB_EXCLUDE_APRS=1
-DRADIOLIB_EXCLUDE_ADSB=1
-DRADIOLIB_EXCLUDE_LORAWAN=1
-DMESHTASTIC_EXCLUDE_DROPZONE=1
-DMESHTASTIC_EXCLUDE_REPLYBOT=1
@@ -59,6 +57,7 @@ build_flags = -Wno-missing-field-initializers
-DMESHTASTIC_EXCLUDE_POWERSTRESS=1 ; exclude power stress test module from main firmware
-DMESHTASTIC_EXCLUDE_GENERIC_THREAD_MODULE=1
-DMESHTASTIC_EXCLUDE_POWERMON=1
-DMESHTASTIC_EXCLUDE_STATUS=1
-D MAX_THREADS=40 ; As we've split modules, we have more threads to manage
#-DBUILD_EPOCH=$UNIX_TIME ; set in platformio-custom.py now
#-D OLED_PL=1
@@ -122,7 +121,7 @@ lib_deps =
[radiolib_base]
lib_deps =
# renovate: datasource=github-tags depName=RadioLib packageName=jgromes/RadioLib
https://github.com/jgromes/RadioLib/archive/afe72ae46a343e15e3cac7f26ac585c7f98bffe5.zip
https://github.com/jgromes/RadioLib/archive/refs/tags/7.6.0.zip
[device-ui_base]
lib_deps =
@@ -139,7 +138,7 @@ lib_deps =
# renovate: datasource=github-tags depName=Adafruit GFX packageName=adafruit/Adafruit-GFX-Library
https://github.com/adafruit/Adafruit-GFX-Library/archive/refs/tags/1.12.6.zip
# renovate: datasource=github-tags depName=NeoPixel packageName=adafruit/Adafruit_NeoPixel
https://github.com/adafruit/Adafruit_NeoPixel/archive/1.15.5.zip
https://github.com/adafruit/Adafruit_NeoPixel/archive/refs/tags/1.15.4.zip
# renovate: datasource=github-tags depName=Adafruit SSD1306 packageName=adafruit/Adafruit_SSD1306
https://github.com/adafruit/Adafruit_SSD1306/archive/refs/tags/2.5.16.zip
# renovate: datasource=github-tags depName=Adafruit BMP280 packageName=adafruit/Adafruit_BMP280_Library
@@ -172,8 +171,8 @@ lib_deps =
https://github.com/EmotiBit/EmotiBit_MLX90632/archive/refs/tags/v1.0.8.zip
# renovate: datasource=github-tags depName=Adafruit MLX90614 packageName=adafruit/Adafruit_MLX90614
https://github.com/adafruit/Adafruit-MLX90614-Library/archive/refs/tags/2.1.6.zip
# renovate: datasource=github-tags depName=INA3221_RT packageName=RobTillaart/INA3221_RT
https://github.com/RobTillaart/INA3221_RT/archive/refs/tags/0.4.2.zip
# renovate: datasource=git-refs depName=INA3221 packageName=https://github.com/sgtwilko/INA3221 gitBranch=FixOverflow
https://github.com/sgtwilko/INA3221/archive/bb03d7e9bfcc74fc798838a54f4f99738f29fc6a.zip
# renovate: datasource=github-tags depName=QMC5883L Compass packageName=mprograms/QMC5883LCompass
https://github.com/mprograms/QMC5883LCompass/archive/refs/tags/v1.2.3.zip
# renovate: datasource=github-tags depName=DFRobot_RTU packageName=dfrobot/DFRobot_RTU
@@ -228,8 +227,6 @@ lib_deps =
https://github.com/Sensirion/arduino-i2c-sfa3x/archive/refs/tags/1.0.0.zip
# renovate: datasource=github-tags depName=Sensirion I2C SCD30 packageName=sensirion/arduino-i2c-scd30
https://github.com/Sensirion/arduino-i2c-scd30/archive/refs/tags/1.0.0.zip
# renovate: datasource=github-tags depName=arduino-sht packageName=sensirion/arduino-sht
https://github.com/Sensirion/arduino-sht/archive/refs/tags/v1.2.6.zip
; Environmental sensors with BSEC2 (Bosch proprietary IAQ)
[environmental_extra]
+19 -16
View File
@@ -26,8 +26,6 @@ SOFTWARE.*/
#include "DebugConfiguration.h"
#include <memory>
#ifdef ARCH_PORTDUINO
#include "platform/portduino/PortduinoGlue.h"
#endif
@@ -121,22 +119,27 @@ bool Syslog::vlogf(uint16_t pri, const char *fmt, va_list args)
bool Syslog::vlogf(uint16_t pri, const char *appName, const char *fmt, va_list args)
{
// First measure the formatted length using a copy of args; passing args directly
// to vsnprintf consumes it, and reusing a consumed va_list is undefined behavior.
va_list args_measure;
va_copy(args_measure, args);
int needed = vsnprintf(nullptr, 0, fmt, args_measure);
va_end(args_measure);
char *message;
size_t initialLen;
size_t len;
bool result;
if (needed < 0)
return false; // encoding error
initialLen = strlen(fmt);
auto message = std::unique_ptr<char[]>(new char[static_cast<size_t>(needed) + 1]);
int written = vsnprintf(message.get(), static_cast<size_t>(needed) + 1, fmt, args);
if (written < 0)
return false;
message = new char[initialLen + 1];
return this->_sendLog(pri, appName, message.get());
len = vsnprintf(message, initialLen + 1, fmt, args);
if (len > initialLen) {
delete[] message;
message = new char[len + 1];
vsnprintf(message, len + 1, fmt, args);
}
result = this->_sendLog(pri, appName, message);
delete[] message;
return result;
}
inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *message)
@@ -151,7 +154,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
if (!this->_enabled)
return false;
if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
return false;
// Check priority against priMask values.
+4 -10
View File
@@ -13,11 +13,6 @@ extern MemGet memGet;
#define LED_STATE_ON 1
#endif
// WIFI LED
#ifndef WIFI_STATE_ON
#define WIFI_STATE_ON 1
#endif
// -----------------------------------------------------------------------------
// DEBUG
// -----------------------------------------------------------------------------
@@ -152,14 +147,13 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
// Default Bluetooth PIN
#define defaultBLEPin 123456
#if HAS_ETHERNET && defined(USE_CH390D)
#include <ESP32_CH390.h>
#elif HAS_ETHERNET && !defined(USE_WS5500)
#if HAS_ETHERNET && !defined(USE_WS5500)
#include <RAK13800_W5100S.h>
#endif // HAS_ETHERNET
#if HAS_ETHERNET && defined(ARCH_ESP32)
#include <ETH.h>
#if HAS_ETHERNET && defined(USE_WS5500)
#include <ETHClass2.h>
#define ETH ETH2
#endif // HAS_ETHERNET
#if HAS_WIFI
+1 -2
View File
@@ -4,8 +4,7 @@ const char *DisplayFormatters::getModemPresetDisplayName(meshtastic_Config_LoRaC
bool usePreset)
{
// If use_preset is false, always return "Custom" — callers such as RadioInterface and Channels
// rely on this being a stable literal for channel-name hashing and default-channel detection.
// If use_preset is false, always return "Custom"
if (!usePreset) {
return "Custom";
}
+91 -61
View File
@@ -79,46 +79,28 @@ bool copyFile(const char *from, const char *to)
bool renameFile(const char *pathFrom, const char *pathTo)
{
#ifdef FSCom
#ifdef ARCH_ESP32
// take SPI Lock
spiLock->lock();
// rename was fixed for ESP32 IDF LittleFS in April
bool result = FSCom.rename(pathFrom, pathTo);
spiLock->unlock();
return result;
#else
return false;
// copyFile does its own locking.
if (copyFile(pathFrom, pathTo) && FSCom.remove(pathFrom)) {
return true;
} else {
return false;
}
#endif
#endif
}
#include <vector>
/**
* @brief Platform-agnostic filesystem format / wipe.
*
* On embedded targets (ESP32, NRF52, STM32WL, RP2040) this calls the
* native FSCom.format() which erases and reinitialises the LittleFS
* partition.
*
* On Portduino the fs::FS backend has no format() method. We instead
* delete /prefs (the only meshtastic data directory written at runtime)
* and return. rmDir("/prefs") is already called unconditionally by
* factoryReset() so this is a proven primitive on Portduino.
* FSBegin() is a no-op (#define FSBegin() true) on Portduino.
*
* @return true on success, false on failure or if no filesystem is configured.
*/
bool fsFormat()
{
#ifdef FSCom
#if defined(ARCH_PORTDUINO)
rmDir("/prefs");
return FSBegin();
#else
return FSCom.format();
#endif
#else
return false;
#endif
}
/**
* @brief Get the list of files in a directory.
*
@@ -141,21 +123,23 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
File file = root.openNextFile();
while (file) {
#ifdef ARCH_ESP32
const char *filepath = file.path();
#else
const char *filepath = file.name();
#endif
if (file.isDirectory() && !String(file.name()).endsWith(".")) {
if (levels) {
std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(filepath, levels - 1);
#ifdef ARCH_ESP32
std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.path(), levels - 1);
#else
std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.name(), levels - 1);
#endif
filenames.insert(filenames.end(), subDirFilenames.begin(), subDirFilenames.end());
file.close();
}
} else {
meshtastic_FileInfo fileInfo = {"", static_cast<uint32_t>(file.size())};
strncpy(fileInfo.file_name, filepath, sizeof(fileInfo.file_name) - 1);
fileInfo.file_name[sizeof(fileInfo.file_name) - 1] = '\0';
#ifdef ARCH_ESP32
strcpy(fileInfo.file_name, file.path());
#else
strcpy(fileInfo.file_name, file.name());
#endif
if (!String(fileInfo.file_name).endsWith(".")) {
filenames.push_back(fileInfo);
}
@@ -179,59 +163,98 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
void listDir(const char *dirname, uint8_t levels, bool del)
{
#ifdef FSCom
#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
char buffer[255];
#endif
File root = FSCom.open(dirname, FILE_O_READ);
if (!root || !root.isDirectory())
if (!root) {
return;
}
if (!root.isDirectory()) {
return;
}
File file = root.openNextFile();
while (file && file.name()[0]) { // file.name()[0] check: workaround for Adafruit LittleFS nRF52 bug #4395
#ifdef ARCH_ESP32
const char *filepath = file.path();
#else
const char *filepath = file.name();
#endif
while (
file &&
file.name()[0]) { // This file.name() check is a workaround for a bug in the Adafruit LittleFS nrf52 glue (see issue 4395)
if (file.isDirectory() && !String(file.name()).endsWith(".")) {
if (levels) {
listDir(filepath, levels - 1, del);
#ifdef ARCH_ESP32
listDir(file.path(), levels - 1, del);
if (del) {
LOG_DEBUG("Remove %s", filepath);
strncpy(buffer, filepath, sizeof(buffer) - 1);
buffer[sizeof(buffer) - 1] = '\0';
LOG_DEBUG("Remove %s", file.path());
strncpy(buffer, file.path(), sizeof(buffer));
file.close();
FSCom.rmdir(buffer);
} else {
file.close();
}
#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
listDir(file.name(), levels - 1, del);
if (del) {
LOG_DEBUG("Remove %s", file.name());
strncpy(buffer, file.name(), sizeof(buffer));
file.close();
FSCom.rmdir(buffer);
} else {
file.close();
}
#else
LOG_DEBUG(" %s (directory)", file.name());
listDir(file.name(), levels - 1, del);
file.close();
#endif
}
} else {
#ifdef ARCH_ESP32
if (del) {
LOG_DEBUG("Delete %s", filepath);
strncpy(buffer, filepath, sizeof(buffer) - 1);
buffer[sizeof(buffer) - 1] = '\0';
LOG_DEBUG("Delete %s", file.path());
strncpy(buffer, file.path(), sizeof(buffer));
file.close();
FSCom.remove(buffer);
} else {
LOG_DEBUG(" %s (%i Bytes)", filepath, file.size());
LOG_DEBUG(" %s (%i Bytes)", file.path(), file.size());
file.close();
}
#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
if (del) {
LOG_DEBUG("Delete %s", file.name());
strncpy(buffer, file.name(), sizeof(buffer));
file.close();
FSCom.remove(buffer);
} else {
LOG_DEBUG(" %s (%i Bytes)", file.name(), file.size());
file.close();
}
#else
LOG_DEBUG(" %s (%i Bytes)", file.name(), file.size());
file.close();
#endif
}
file = root.openNextFile();
}
#ifdef ARCH_ESP32
const char *rootpath = root.path();
#else
const char *rootpath = root.name();
#endif
if (del) {
LOG_DEBUG("Remove %s", rootpath);
strncpy(buffer, rootpath, sizeof(buffer) - 1);
buffer[sizeof(buffer) - 1] = '\0';
LOG_DEBUG("Remove %s", root.path());
strncpy(buffer, root.path(), sizeof(buffer));
root.close();
FSCom.rmdir(buffer);
} else {
root.close();
}
#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
if (del) {
LOG_DEBUG("Remove %s", root.name());
strncpy(buffer, root.name(), sizeof(buffer));
root.close();
FSCom.rmdir(buffer);
} else {
root.close();
}
#else
root.close();
#endif
#endif
}
@@ -245,7 +268,14 @@ void listDir(const char *dirname, uint8_t levels, bool del)
void rmDir(const char *dirname)
{
#ifdef FSCom
#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
listDir(dirname, 10, true);
#elif defined(ARCH_NRF52)
// nRF52 implementation of LittleFS has a recursive delete function
FSCom.rmdir_r(dirname);
#endif
#endif
}
@@ -277,7 +307,7 @@ void fsInit()
*/
void setupSDCard()
{
#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI) && !defined(HAS_SD_MMC)
#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI)
concurrency::LockGuard g(spiLock);
SDHandler.begin(SPI_SCK, SPI_MISO, SPI_MOSI);
if (!SD.begin(SDCARD_CS, SDHandler, SD_SPI_FREQUENCY)) {
-9
View File
@@ -48,19 +48,10 @@ using namespace STM32_LittleFS_Namespace;
using namespace Adafruit_LittleFS_Namespace;
#endif
#if defined(ARCH_NRF54L15)
// nRF54L15 — Zephyr LittleFS on 36 KB storage_partition (internal RRAM)
#include "InternalFileSystem.h"
#define FSCom InternalFS
#define FSBegin() FSCom.begin()
using namespace Adafruit_LittleFS_Namespace;
#endif
void fsInit();
void fsListFiles();
bool copyFile(const char *from, const char *to);
bool renameFile(const char *pathFrom, const char *pathTo);
bool fsFormat();
std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels);
void listDir(const char *dirname, uint8_t levels, bool del = false);
void rmDir(const char *dirname);
+6 -3
View File
@@ -53,7 +53,8 @@ class GPSStatus : public Status
int32_t getLatitude() const
{
if (config.position.fixed_position) {
return localPosition.latitude_i;
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
return node->position.latitude_i;
} else {
return p.latitude_i;
}
@@ -62,7 +63,8 @@ class GPSStatus : public Status
int32_t getLongitude() const
{
if (config.position.fixed_position) {
return localPosition.longitude_i;
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
return node->position.longitude_i;
} else {
return p.longitude_i;
}
@@ -71,7 +73,8 @@ class GPSStatus : public Status
int32_t getAltitude() const
{
if (config.position.fixed_position) {
return localPosition.altitude;
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(nodeDB->getNodeNum());
return node->position.altitude;
} else {
return p.altitude;
}
+31 -26
View File
@@ -6,6 +6,7 @@
#include "SPILock.h"
#include "SafeFile.h"
#include "gps/RTC.h"
#include "graphics/draw/MessageRenderer.h"
#include <cstring> // memcpy
#ifndef MESSAGE_TEXT_POOL_SIZE
@@ -180,8 +181,13 @@ const StoredMessage &MessageStore::addFromPacket(const meshtastic_MeshPacket &pa
bool isDM = (sm.dest != 0 && sm.dest != NODENUM_BROADCAST);
sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
sm.ackStatus = (packet.from == 0) ? AckStatus::NONE : AckStatus::ACKED;
if (packet.from == 0) {
sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
sm.ackStatus = AckStatus::NONE;
} else {
sm.type = isDM ? MessageType::DM_TO_US : MessageType::BROADCAST;
sm.ackStatus = AckStatus::ACKED;
}
addLiveMessage(sm);
@@ -366,25 +372,26 @@ void MessageStore::clearAllMessages()
#endif
}
// Internal helpers for targeted erasure.
template <typename Predicate> static bool eraseFirstMatch(std::deque<StoredMessage> &deque, Predicate pred)
// Internal helper: erase first or last message matching a predicate
template <typename Predicate> static void eraseIf(std::deque<StoredMessage> &deque, Predicate pred, bool fromBack = false)
{
for (auto it = deque.begin(); it != deque.end(); ++it) {
if (pred(*it)) {
deque.erase(it);
return true;
if (fromBack) {
// Iterate from the back and erase all matches from the end
for (auto it = deque.rbegin(); it != deque.rend();) {
if (pred(*it)) {
it = std::deque<StoredMessage>::reverse_iterator(deque.erase(std::next(it).base()));
} else {
++it;
}
}
}
return false;
}
template <typename Predicate> static void eraseAllMatches(std::deque<StoredMessage> &deque, Predicate pred)
{
for (auto it = deque.begin(); it != deque.end();) {
if (pred(*it)) {
it = deque.erase(it);
} else {
++it;
} else {
// Manual forward search to erase all matches
for (auto it = deque.begin(); it != deque.end();) {
if (pred(*it)) {
it = deque.erase(it);
} else {
++it;
}
}
}
}
@@ -392,9 +399,7 @@ template <typename Predicate> static void eraseAllMatches(std::deque<StoredMessa
// Delete oldest message (RAM + persisted queue)
void MessageStore::deleteOldestMessage()
{
if (!liveMessages.empty()) {
liveMessages.pop_front();
}
eraseIf(liveMessages, [](StoredMessage &) { return true; });
saveToFlash();
}
@@ -402,14 +407,14 @@ void MessageStore::deleteOldestMessage()
void MessageStore::deleteOldestMessageInChannel(uint8_t channel)
{
auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
eraseFirstMatch(liveMessages, pred);
eraseIf(liveMessages, pred);
saveToFlash();
}
void MessageStore::deleteAllMessagesInChannel(uint8_t channel)
{
auto pred = [channel](const StoredMessage &m) { return m.type == MessageType::BROADCAST && m.channelIndex == channel; };
eraseAllMatches(liveMessages, pred);
eraseIf(liveMessages, pred, false /* delete ALL, not just first */);
saveToFlash();
}
@@ -422,7 +427,7 @@ void MessageStore::deleteAllMessagesWithPeer(uint32_t peer)
uint32_t other = (m.sender == local) ? m.dest : m.sender;
return other == peer;
};
eraseAllMatches(liveMessages, pred);
eraseIf(liveMessages, pred, false);
saveToFlash();
}
@@ -435,7 +440,7 @@ void MessageStore::deleteOldestMessageWithPeer(uint32_t peer)
uint32_t other = (m.sender == nodeDB->getNodeNum()) ? m.dest : m.sender;
return other == peer;
};
eraseFirstMatch(liveMessages, pred);
eraseIf(liveMessages, pred);
saveToFlash();
}
+3
View File
@@ -124,6 +124,9 @@ class MessageStore
// Allocate text into pool (used by sender-side code)
static uint16_t storeText(const char *src, size_t len);
// Used when loading from flash to rebuild the text pool
static uint16_t rebuildTextFromFlash(const char *src, size_t len);
private:
std::deque<StoredMessage> liveMessages; // Single in-RAM message buffer (also used for persistence)
std::string filename; // Flash filename for persistence
+141 -217
View File
@@ -24,13 +24,6 @@
#include "meshUtils.h"
#include "power/PowerHAL.h"
#include "sleep.h"
#ifdef ARCH_ESP32
// #include <driver/adc.h>
#include <esp_adc/adc_cali.h>
#include <esp_adc/adc_cali_scheme.h>
#include <esp_adc/adc_oneshot.h>
#include <esp_err.h>
#endif
#if defined(ARCH_PORTDUINO)
#include "api/WiFiServerAPI.h"
@@ -47,22 +40,6 @@
#include "concurrency/LockGuard.h"
#endif
#if defined(ARCH_STM32WL) && defined(BATTERY_PIN)
#include "stm32yyxx_ll_adc.h"
/* Analog read resolution */
#if defined(LL_ADC_RESOLUTION_12B)
#define LL_ADC_RESOLUTION LL_ADC_RESOLUTION_12B
#define BATTERY_SENSE_RESOLUTION_BITS 12
#elif defined(LL_ADC_DS_DATA_WIDTH_12_BIT)
#define LL_ADC_RESOLUTION LL_ADC_DS_DATA_WIDTH_12_BIT
#define BATTERY_SENSE_RESOLUTION_BITS 12
#else
#error "ADC resolution could not be defined!"
#endif
#define ADC_RANGE (1 << BATTERY_SENSE_RESOLUTION_BITS)
#endif
#if defined(DEBUG_HEAP_MQTT) && !MESHTASTIC_EXCLUDE_MQTT
#include "mqtt/MQTT.h"
#include "target_specific.h"
@@ -70,8 +47,9 @@
#include <WiFi.h>
#endif
#if HAS_ETHERNET && defined(ARCH_ESP32)
#include <ETH.h>
#if HAS_ETHERNET && defined(USE_WS5500)
#include <ETHClass2.h>
#define ETH ETH2
#endif // HAS_ETHERNET
#endif
@@ -83,113 +61,33 @@
#if defined(BATTERY_PIN) && defined(ARCH_ESP32)
#ifndef BAT_MEASURE_ADC_UNIT // ADC1 is default
static const adc_channel_t adc_channel = ADC_CHANNEL;
static const adc1_channel_t adc_channel = ADC_CHANNEL;
static const adc_unit_t unit = ADC_UNIT_1;
#else // ADC2
static const adc_channel_t adc_channel = ADC_CHANNEL;
#else // ADC2
static const adc2_channel_t adc_channel = ADC_CHANNEL;
static const adc_unit_t unit = ADC_UNIT_2;
RTC_NOINIT_ATTR uint64_t RTC_reg_b;
#endif // BAT_MEASURE_ADC_UNIT
static adc_oneshot_unit_handle_t adc_handle = nullptr;
static adc_cali_handle_t adc_cali_handle = nullptr;
static bool adc_calibrated = false;
esp_adc_cal_characteristics_t *adc_characs = (esp_adc_cal_characteristics_t *)calloc(1, sizeof(esp_adc_cal_characteristics_t));
#ifndef ADC_ATTENUATION
static const adc_atten_t atten = ADC_ATTEN_DB_12;
#else
static const adc_atten_t atten = ADC_ATTENUATION;
#endif
#ifdef ADC_BITWIDTH
static const adc_bitwidth_t adc_width = ADC_BITWIDTH;
#else
static const adc_bitwidth_t adc_width = ADC_BITWIDTH_DEFAULT;
#endif
static int adcBitWidthToBits(adc_bitwidth_t width)
{
switch (width) {
case ADC_BITWIDTH_9:
return 9;
case ADC_BITWIDTH_10:
return 10;
case ADC_BITWIDTH_11:
return 11;
case ADC_BITWIDTH_12:
return 12;
#ifdef ADC_BITWIDTH_13
case ADC_BITWIDTH_13:
return 13;
#endif
default:
return 12;
}
}
static bool initAdcCalibration()
{
#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED
adc_cali_curve_fitting_config_t cali_config = {
.unit_id = unit,
.atten = atten,
.bitwidth = adc_width,
};
esp_err_t ret = adc_cali_create_scheme_curve_fitting(&cali_config, &adc_cali_handle);
if (ret == ESP_OK) {
LOG_INFO("ADC calibration: curve fitting enabled");
return true;
}
if (ret != ESP_ERR_NOT_SUPPORTED) {
LOG_WARN("ADC calibration: curve fitting failed: %s", esp_err_to_name(ret));
}
#endif
#if ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED
adc_cali_line_fitting_config_t cali_config = {
.unit_id = unit,
.atten = atten,
.bitwidth = adc_width,
.default_vref = DEFAULT_VREF,
};
esp_err_t ret = adc_cali_create_scheme_line_fitting(&cali_config, &adc_cali_handle);
if (ret == ESP_OK) {
LOG_INFO("ADC calibration: line fitting enabled");
return true;
}
if (ret != ESP_ERR_NOT_SUPPORTED) {
LOG_WARN("ADC calibration: line fitting failed: %s", esp_err_to_name(ret));
}
#endif
LOG_INFO("ADC calibration not supported; using approximate scaling");
return false;
}
#endif // BATTERY_PIN && ARCH_ESP32
#ifdef EXT_PWR_DETECT
#ifndef EXT_PWR_DETECT_MODE
#define EXT_PWR_DETECT_MODE INPUT
// If using internal pull resistors, we can infer EXT_PWR_DETECT_VALUE
#elif EXT_PWR_DETECT_MODE == INPUT_PULLUP
#define EXT_PWR_DETECT_VALUE LOW
#elif EXT_PWR_DETECT_MODE == INPUT_PULLDOWN
#define EXT_PWR_DETECT_VALUE HIGH
#endif
#ifndef EXT_PWR_DETECT_VALUE
#define EXT_PWR_DETECT_VALUE HIGH
#endif
#endif
#ifdef EXT_CHRG_DETECT
#ifndef EXT_CHRG_DETECT_MODE
#define EXT_CHRG_DETECT_MODE INPUT
// If using internal pull resistors, we can infer EXT_CHRG_DETECT_VALUE
#elif EXT_CHRG_DETECT_MODE == INPUT_PULLUP
#define EXT_CHRG_DETECT_VALUE LOW
#elif EXT_CHRG_DETECT_MODE == INPUT_PULLDOWN
#define EXT_CHRG_DETECT_VALUE HIGH
static const uint8_t ext_chrg_detect_mode = INPUT;
#else
static const uint8_t ext_chrg_detect_mode = EXT_CHRG_DETECT_MODE;
#endif
#ifndef EXT_CHRG_DETECT_VALUE
#define EXT_CHRG_DETECT_VALUE HIGH
static const uint8_t ext_chrg_detect_value = HIGH;
#else
static const uint8_t ext_chrg_detect_value = EXT_CHRG_DETECT_VALUE;
#endif
#endif
@@ -430,29 +328,11 @@ class AnalogBatteryLevel : public HasBatteryLevel
float scaled = 0;
battery_adcEnable();
#ifdef ARCH_STM32WL
// STM32 ADC with VREFINT runtime calibration
Vref = __LL_ADC_CALC_VREFANALOG_VOLTAGE(analogRead(AVREF), LL_ADC_RESOLUTION);
raw = analogRead(BATTERY_PIN);
scaled = __LL_ADC_CALC_DATA_TO_VOLTAGE(Vref, raw, LL_ADC_RESOLUTION);
scaled *= operativeAdcMultiplier;
#elif defined(ARCH_ESP32) // ADC block for espressif platforms
#ifdef ARCH_ESP32 // ADC block for espressif platforms
raw = espAdcRead();
int voltage_mv = 0;
if (adc_calibrated && adc_cali_handle) {
if (adc_cali_raw_to_voltage(adc_cali_handle, raw, &voltage_mv) != ESP_OK) {
LOG_WARN("ADC calibration read failed; using raw value");
voltage_mv = 0;
}
}
if (voltage_mv == 0) {
// Fallback approximate conversion without calibration
const int bits = adcBitWidthToBits(adc_width);
const float max_code = powf(2.0f, bits) - 1.0f;
voltage_mv = (int)((raw / max_code) * DEFAULT_VREF);
}
scaled = voltage_mv * operativeAdcMultiplier;
#else // block for all other platforms
scaled = esp_adc_cal_raw_to_voltage(raw, adc_characs);
scaled *= operativeAdcMultiplier;
#else // block for all other platforms
#ifdef ARCH_NRF52
concurrency::LockGuard saadcGuard(concurrency::nrf52SaadcLock);
#endif
@@ -493,22 +373,51 @@ class AnalogBatteryLevel : public HasBatteryLevel
uint32_t raw = 0;
uint8_t raw_c = 0; // raw reading counter
if (!adc_handle) {
LOG_ERROR("ADC oneshot handle not initialized");
return 0;
}
#ifndef BAT_MEASURE_ADC_UNIT // ADC1
for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
int val = 0;
esp_err_t err = adc_oneshot_read(adc_handle, adc_channel, &val);
if (err == ESP_OK) {
raw += val;
int val_ = adc1_get_raw(adc_channel);
if (val_ >= 0) { // save only valid readings
raw += val_;
raw_c++;
}
// delayMicroseconds(100);
}
#else // ADC2
#ifdef CONFIG_IDF_TARGET_ESP32S3 // ESP32S3
// ADC2 wifi bug workaround not required, breaks compile
// On ESP32S3, ADC2 can take turns with Wifi (?)
int32_t adc_buf;
esp_err_t read_result;
// Multiple samples
for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
adc_buf = 0;
read_result = -1;
read_result = adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
if (read_result == ESP_OK) {
raw += adc_buf;
raw_c++; // Count valid samples
} else {
LOG_DEBUG("ADC read failed: %s", esp_err_to_name(err));
LOG_DEBUG("An attempt to sample ADC2 failed");
}
}
#else // Other ESP32
int32_t adc_buf = 0;
for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
// ADC2 wifi bug workaround, see
// https://github.com/espressif/arduino-esp32/issues/102
WRITE_PERI_REG(SENS_SAR_READ_CTRL2_REG, RTC_reg_b);
SET_PERI_REG_MASK(SENS_SAR_READ_CTRL2_REG, SENS_SAR2_DATA_INV);
adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
raw += adc_buf;
raw_c++;
}
#endif // BAT_MEASURE_ADC_UNIT
#endif // End BAT_MEASURE_ADC_UNIT
return (raw / (raw_c < 1 ? 1 : raw_c));
}
#endif
@@ -538,14 +447,28 @@ class AnalogBatteryLevel : public HasBatteryLevel
virtual bool isBatteryConnect() override { return getBatteryPercent() != -1; }
#endif
// Detect if an external power source is connected if we don’t have a PMIC;
// Firstly prefer EXT_PWR_DETECT GPIO if available,
// secondly try an nRF52-specific routine on some variants,
// lastly provide a fallback to indicate external power when fully charged.
/// If we see a battery voltage higher than physics allows - assume charger is
/// pumping in power On some boards we don't have the power management chip
/// (like AXPxxxx) so we use EXT_PWR_DETECT GPIO pin to detect external power
/// source
virtual bool isVbusIn() override
{
#ifdef EXT_PWR_DETECT
return digitalRead(EXT_PWR_DETECT) == EXT_PWR_DETECT_VALUE;
#if defined(HELTEC_CAPSULE_SENSOR_V3) || defined(HELTEC_SENSOR_HUB)
// if external powered that pin will be pulled down
if (digitalRead(EXT_PWR_DETECT) == LOW) {
return true;
}
// if it's not LOW - check the battery
#else
// if external powered that pin will be pulled up
if (digitalRead(EXT_PWR_DETECT) == HIGH) {
return true;
}
// if it's not HIGH - check the battery
#endif
// If we have an EXT_PWR_DETECT pin and it indicates no external power, believe it.
return false;
// technically speaking this should work for all(?) NRF52 boards
// but needs testing across multiple devices. NRF52 USB would not even work if
@@ -566,9 +489,9 @@ class AnalogBatteryLevel : public HasBatteryLevel
}
#endif
#if defined(ELECROW_ThinkNode_M6)
return digitalRead(EXT_CHRG_DETECT) == EXT_CHRG_DETECT_VALUE || isVbusIn();
return digitalRead(EXT_CHRG_DETECT) == ext_chrg_detect_value || isVbusIn();
#elif EXT_CHRG_DETECT
return digitalRead(EXT_CHRG_DETECT) == EXT_CHRG_DETECT_VALUE;
return digitalRead(EXT_CHRG_DETECT) == ext_chrg_detect_value;
#elif defined(BATTERY_CHARGING_INV)
return !digitalRead(BATTERY_CHARGING_INV);
#else
@@ -607,11 +530,6 @@ class AnalogBatteryLevel : public HasBatteryLevel
bool initial_read_done = false;
float last_read_value = (OCV[NUM_OCV_POINTS - 1] * NUM_CELLS);
uint32_t last_read_time_ms = 0;
#ifdef ARCH_STM32WL
// 3300mV placeholder for STM32 errata where VREFINT factory calibration may be missing
// (e.g. STM32U0, see DS14756 Rev 3 §2.4.1 "VREFINT offset")
uint32_t Vref = 3300;
#endif
#if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_ENVIRONMENTAL_SENSOR && defined(HAS_RAKPROT)
@@ -701,10 +619,14 @@ Power::Power() : OSThread("Power")
bool Power::analogInit()
{
#ifdef EXT_PWR_DETECT
pinMode(EXT_PWR_DETECT, EXT_PWR_DETECT_MODE);
#if defined(HELTEC_CAPSULE_SENSOR_V3) || defined(HELTEC_SENSOR_HUB)
pinMode(EXT_PWR_DETECT, INPUT_PULLUP);
#else
pinMode(EXT_PWR_DETECT, INPUT);
#endif
#endif
#ifdef EXT_CHRG_DETECT
pinMode(EXT_CHRG_DETECT, EXT_CHRG_DETECT_MODE);
pinMode(EXT_CHRG_DETECT, ext_chrg_detect_mode);
#endif
#ifdef BATTERY_PIN
@@ -717,38 +639,47 @@ bool Power::analogInit()
#define BATTERY_SENSE_RESOLUTION_BITS 10
#endif
#ifdef ARCH_STM32WL
analogReadResolution(BATTERY_SENSE_RESOLUTION_BITS);
#elif defined(ARCH_ESP32) // ESP32 needs special analog stuff
adc_oneshot_unit_init_cfg_t init_config = {
.unit_id = unit,
};
#ifdef ARCH_ESP32 // ESP32 needs special analog stuff
if (!adc_handle) {
esp_err_t err = adc_oneshot_new_unit(&init_config, &adc_handle);
if (err != ESP_OK) {
LOG_ERROR("ADC oneshot init failed: %s", esp_err_to_name(err));
return false;
}
#ifndef ADC_WIDTH // max resolution by default
static const adc_bits_width_t width = ADC_WIDTH_BIT_12;
#else
static const adc_bits_width_t width = ADC_WIDTH;
#endif
#ifndef BAT_MEASURE_ADC_UNIT // ADC1
adc1_config_width(width);
adc1_config_channel_atten(adc_channel, atten);
#else // ADC2
adc2_config_channel_atten(adc_channel, atten);
#ifndef CONFIG_IDF_TARGET_ESP32S3
// ADC2 wifi bug workaround
// Not required with ESP32S3, breaks compile
RTC_reg_b = READ_PERI_REG(SENS_SAR_READ_CTRL2_REG);
#endif
#endif
// calibrate ADC
esp_adc_cal_value_t val_type = esp_adc_cal_characterize(unit, atten, width, DEFAULT_VREF, adc_characs);
// show ADC characterization base
if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP) {
LOG_INFO("ADC config based on Two Point values stored in eFuse");
} else if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
LOG_INFO("ADC config based on reference voltage stored in eFuse");
}
adc_oneshot_chan_cfg_t chan_cfg = {
.atten = atten,
.bitwidth = adc_width,
};
esp_err_t err = adc_oneshot_config_channel(adc_handle, adc_channel, &chan_cfg);
if (err != ESP_OK) {
LOG_ERROR("ADC channel config failed: %s", esp_err_to_name(err));
return false;
#ifdef CONFIG_IDF_TARGET_ESP32S3
// ESP32S3
else if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP_FIT) {
LOG_INFO("ADC config based on Two Point values and fitting curve "
"coefficients stored in eFuse");
}
adc_calibrated = initAdcCalibration();
#endif // ARCH_ESP32
#endif
else {
LOG_INFO("ADC config based on default reference voltage");
}
#endif // ARCH_ESP32
// NRF52 ADC init moved to powerHAL_init in nrf52 platform
#if !defined(ARCH_ESP32) && !defined(ARCH_STM32WL)
#ifndef ARCH_ESP32
analogReadResolution(BATTERY_SENSE_RESOLUTION_BITS);
#endif
@@ -947,16 +878,7 @@ void Power::readPowerStatus()
// Notify any status instances that are observing us
const PowerStatus powerStatus2 = PowerStatus(hasBattery, usbPowered, isChargingNow, batteryVoltageMv, batteryChargePercent);
// Log battery-presence transitions once; skip OptUnknown so we don't lie before the first probe.
static OptionalBool prevHasBattery = OptUnknown;
if (hasBattery != OptUnknown && hasBattery != prevHasBattery) {
LOG_INFO("Power: battery hardware %s", hasBattery == OptTrue ? "detected" : "absent (USB-only)");
prevHasBattery = hasBattery;
}
// Periodic telemetry only emits when a battery is actually present (otherwise values are constant -1/0).
if (hasBattery == OptTrue && !Throttle::isWithinTimespanMs(lastLogTime, 50 * 1000)) {
if (millis() > lastLogTime + 50 * 1000) {
LOG_DEBUG("Battery: usbPower=%d, isCharging=%d, batMv=%d, batPct=%d", powerStatus2.getHasUSB(),
powerStatus2.getIsCharging(), powerStatus2.getBatteryVoltageMv(), powerStatus2.getBatteryChargePercent());
lastLogTime = millis();
@@ -1054,14 +976,6 @@ int32_t Power::runOnce()
powerFSM.trigger(EVENT_POWER_CONNECTED);
}
#ifdef PMU_POWER_BUTTON_IS_CANCEL
// cancel action also turns the screen on and off.
if (PMU->isPekeyShortPressIrq()) {
LOG_INFO("Input: Corona Button Click");
InputEvent event = {.inputEvent = (input_broker_event)INPUT_BROKER_CANCEL, .kbchar = 0, .touchX = 0, .touchY = 0};
inputBroker->injectInputEvent(&event);
}
#endif
/*
Other things we could check if we cared...
@@ -1078,6 +992,13 @@ int32_t Power::runOnce()
LOG_DEBUG("Battery removed");
}
*/
#ifndef T_WATCH_S3 // FIXME - why is this triggering on the T-Watch S3?
if (PMU->isPekeyLongPressIrq()) {
LOG_DEBUG("PEK long button press");
if (screen)
screen->setOn(false);
}
#endif
PMU->clearIrqStatus();
}
@@ -1146,7 +1067,7 @@ void Power::attachPowerInterrupts()
if (PMU) {
attachInterrupt(
PMU_IRQ,
[]() {
[] {
pmu_irq = true;
power->setIntervalFromNow(0);
runASAP = true;
@@ -1449,16 +1370,19 @@ bool Power::axpChipInit()
uint64_t pmuIrqMask = 0;
if (PMU->getChipModel() == XPOWERS_AXP192) {
pmuIrqMask = XPOWERS_AXP192_VBUS_INSERT_IRQ | XPOWERS_AXP192_VBUS_REMOVE_IRQ | XPOWERS_AXP192_PKEY_SHORT_IRQ;
pmuIrqMask = XPOWERS_AXP192_VBUS_INSERT_IRQ | XPOWERS_AXP192_BAT_INSERT_IRQ | XPOWERS_AXP192_PKEY_SHORT_IRQ;
} else if (PMU->getChipModel() == XPOWERS_AXP2101) {
pmuIrqMask = XPOWERS_AXP2101_VBUS_INSERT_IRQ | XPOWERS_AXP2101_VBUS_REMOVE_IRQ | XPOWERS_AXP2101_PKEY_SHORT_IRQ;
pmuIrqMask = XPOWERS_AXP2101_VBUS_INSERT_IRQ | XPOWERS_AXP2101_BAT_INSERT_IRQ | XPOWERS_AXP2101_PKEY_SHORT_IRQ;
}
pinMode(PMU_IRQ, INPUT);
// We wake on IRQ, so only enable the IRQs that we care about.
// we want USB plug and unplug to update the screen and LED status,
// and short press on the power button to trigger the "cancel" action in the UI (which also turns the screen on and off).
// we do not look for AXPXXX_CHARGING_FINISHED_IRQ & AXPXXX_CHARGING_IRQ
// because it occurs repeatedly while there is no battery also it could cause
// inadvertent waking from light sleep just because the battery filled we
// don't look for AXPXXX_BATT_REMOVED_IRQ because it occurs repeatedly while
// no battery installed we don't look at AXPXXX_VBUS_REMOVED_IRQ because we
// don't have anything hooked to vbus
PMU->enableIRQ(pmuIrqMask);
PMU->clearIrqStatus();
@@ -1924,7 +1848,7 @@ class SerialBatteryLevel : public HasBatteryLevel
{
#if defined(EXT_CHRG_DETECT)
return digitalRead(EXT_CHRG_DETECT) == EXT_CHRG_DETECT_VALUE;
return digitalRead(EXT_CHRG_DETECT) == ext_chrg_detect_value;
#endif
return false;
@@ -1933,7 +1857,7 @@ class SerialBatteryLevel : public HasBatteryLevel
virtual bool isCharging() override
{
#ifdef EXT_CHRG_DETECT
return digitalRead(EXT_CHRG_DETECT) == EXT_CHRG_DETECT_VALUE;
return digitalRead(EXT_CHRG_DETECT) == ext_chrg_detect_value;
#endif
// by default, we check the battery voltage only
@@ -1955,10 +1879,10 @@ SerialBatteryLevel serialBatteryLevel;
bool Power::serialBatteryInit()
{
#ifdef EXT_PWR_DETECT
pinMode(EXT_PWR_DETECT, EXT_PWR_DETECT_MODE);
pinMode(EXT_PWR_DETECT, INPUT);
#endif
#ifdef EXT_CHRG_DETECT
pinMode(EXT_CHRG_DETECT, EXT_CHRG_DETECT_MODE);
pinMode(EXT_CHRG_DETECT, ext_chrg_detect_mode);
#endif
bool result = serialBatteryLevel.runOnce();
+11 -48
View File
@@ -58,35 +58,6 @@ static bool isPowered()
return !isPowerSavingMode && powerStatus && (!powerStatus->getHasBattery() || powerStatus->getHasUSB());
}
#if defined(T5_S3_EPAPER_PRO)
static void t5BacklightOffForSleep()
{
t5BacklightSetForcedBySleep(true);
}
static void t5BacklightWakeFromSleep()
{
t5BacklightSetForcedBySleep(false);
}
static void t5BacklightOffForTimeout()
{
t5BacklightSetForcedByTimeout(true);
t5TouchSetForcedByTimeout(true);
}
static void t5BacklightOnFromUserInput()
{
t5BacklightHandleUserInput();
t5TouchHandleUserInput();
}
#else
static void t5BacklightOffForSleep() {}
static void t5BacklightWakeFromSleep() {}
static void t5BacklightOffForTimeout() {}
static void t5BacklightOnFromUserInput() {}
#endif
static void sdsEnter()
{
LOG_POWERFSM("State: SDS");
@@ -116,7 +87,6 @@ static void lsEnter()
LOG_POWERFSM("lsEnter begin, ls_secs=%u", config.power.ls_secs);
if (screen)
screen->setOn(false);
t5BacklightOffForSleep();
secsSlept = 0; // How long have we been sleeping this time
// LOG_INFO("lsEnter end");
@@ -189,8 +159,6 @@ static void lsIdle()
static void lsExit()
{
LOG_POWERFSM("State: lsExit");
// Lift the light-sleep force-off gate when leaving LS.
t5BacklightWakeFromSleep();
}
static void nbEnter()
@@ -212,8 +180,6 @@ static void darkEnter()
setBluetoothEnable(true);
if (screen)
screen->setOn(false);
// Screen timeout enters DARK; ensure backlight also turns off.
t5BacklightOffForTimeout();
}
static void serialEnter()
@@ -323,13 +289,12 @@ void PowerFSM_setup()
powerFSM.add_transition(&stateNB, &stateNB, EVENT_PACKET_FOR_PHONE, NULL, "Received packet, resetting win wake");
// Handle press events - note: we ignore button presses when in API mode
powerFSM.add_transition(&stateLS, &stateON, EVENT_PRESS, t5BacklightOnFromUserInput, "Press");
powerFSM.add_transition(&stateNB, &stateON, EVENT_PRESS, t5BacklightOnFromUserInput, "Press");
powerFSM.add_transition(&stateDARK, isPowered() ? &statePOWER : &stateON, EVENT_PRESS, t5BacklightOnFromUserInput, "Press");
powerFSM.add_transition(&statePOWER, &statePOWER, EVENT_PRESS, t5BacklightOnFromUserInput, "Press");
powerFSM.add_transition(&stateON, &stateON, EVENT_PRESS, t5BacklightOnFromUserInput,
"Press"); // reenter On to restart our timers
powerFSM.add_transition(&stateSERIAL, &stateSERIAL, EVENT_PRESS, t5BacklightOnFromUserInput,
powerFSM.add_transition(&stateLS, &stateON, EVENT_PRESS, NULL, "Press");
powerFSM.add_transition(&stateNB, &stateON, EVENT_PRESS, NULL, "Press");
powerFSM.add_transition(&stateDARK, isPowered() ? &statePOWER : &stateON, EVENT_PRESS, NULL, "Press");
powerFSM.add_transition(&statePOWER, &statePOWER, EVENT_PRESS, NULL, "Press");
powerFSM.add_transition(&stateON, &stateON, EVENT_PRESS, NULL, "Press"); // reenter On to restart our timers
powerFSM.add_transition(&stateSERIAL, &stateSERIAL, EVENT_PRESS, NULL,
"Press"); // Allow button to work while in serial API
// Handle critically low power battery by forcing deep sleep
@@ -349,13 +314,11 @@ void PowerFSM_setup()
powerFSM.add_transition(&stateSERIAL, &stateSHUTDOWN, EVENT_SHUTDOWN, NULL, "Shutdown");
// Inputbroker
powerFSM.add_transition(&stateLS, &stateON, EVENT_INPUT, t5BacklightOnFromUserInput, "Input Device");
powerFSM.add_transition(&stateNB, &stateON, EVENT_INPUT, t5BacklightOnFromUserInput, "Input Device");
powerFSM.add_transition(&stateDARK, &stateON, EVENT_INPUT, t5BacklightOnFromUserInput, "Input Device");
powerFSM.add_transition(&stateON, &stateON, EVENT_INPUT, t5BacklightOnFromUserInput,
"Input Device"); // restarts the sleep timer
powerFSM.add_transition(&statePOWER, &statePOWER, EVENT_INPUT, t5BacklightOnFromUserInput,
"Input Device"); // restarts the sleep timer
powerFSM.add_transition(&stateLS, &stateON, EVENT_INPUT, NULL, "Input Device");
powerFSM.add_transition(&stateNB, &stateON, EVENT_INPUT, NULL, "Input Device");
powerFSM.add_transition(&stateDARK, &stateON, EVENT_INPUT, NULL, "Input Device");
powerFSM.add_transition(&stateON, &stateON, EVENT_INPUT, NULL, "Input Device"); // restarts the sleep timer
powerFSM.add_transition(&statePOWER, &statePOWER, EVENT_INPUT, NULL, "Input Device"); // restarts the sleep timer
powerFSM.add_transition(&stateDARK, &stateON, EVENT_BLUETOOTH_PAIR, NULL, "Bluetooth pairing");
powerFSM.add_transition(&stateON, &stateON, EVENT_BLUETOOTH_PAIR, NULL, "Bluetooth pairing");
+4 -8
View File
@@ -137,7 +137,7 @@ void RedirectablePrint::log_to_serial(const char *logLevel, const char *format,
if (color) {
::printf("\u001b[0m");
}
::printf("| %02d:%02d:%02d %u.%03u ", hour, min, sec, millis() / 1000, millis() % 1000);
::printf("| %02d:%02d:%02d %u ", hour, min, sec, millis() / 1000);
#else
printf("%s ", logLevel);
if (color) {
@@ -151,7 +151,7 @@ void RedirectablePrint::log_to_serial(const char *logLevel, const char *format,
if (color) {
::printf("\u001b[0m");
}
::printf("| ??:??:?? %u.%03u ", millis() / 1000, millis() % 1000);
::printf("| ??:??:?? %u ", millis() / 1000);
#else
printf("%s ", logLevel);
if (color) {
@@ -225,16 +225,14 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
isBleConnected = nimbleBluetooth && nimbleBluetooth->isActive() && nimbleBluetooth->isConnected();
#elif defined(ARCH_NRF52)
isBleConnected = nrf52Bluetooth != nullptr && nrf52Bluetooth->isConnected();
#elif defined(ARCH_NRF54L15)
isBleConnected = nrf54l15Bluetooth != nullptr && nrf54l15Bluetooth->isConnected();
#endif
if (isBleConnected) {
auto thread = concurrency::OSThread::currentThread;
meshtastic_LogRecord logRecord = meshtastic_LogRecord_init_zero;
logRecord.level = getLogLevel(logLevel);
vsnprintf(logRecord.message, sizeof(logRecord.message), format, arg);
vsprintf(logRecord.message, format, arg);
if (thread)
strlcpy(logRecord.source, thread->ThreadName.c_str(), sizeof(logRecord.source));
strcpy(logRecord.source, thread->ThreadName.c_str());
logRecord.time = getValidTime(RTCQuality::RTCQualityDevice, true);
auto buffer = std::unique_ptr<uint8_t[]>(new uint8_t[meshtastic_LogRecord_size]);
@@ -243,8 +241,6 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
nimbleBluetooth->sendLog(buffer.get(), size);
#elif defined(ARCH_NRF52)
nrf52Bluetooth->sendLog(buffer.get(), size);
#elif defined(ARCH_NRF54L15)
nrf54l15Bluetooth->sendLog(buffer.get(), size);
#endif
}
}
+7
View File
@@ -7,6 +7,10 @@ static File openFile(const char *filename, bool fullAtomic)
{
concurrency::LockGuard g(spiLock);
LOG_DEBUG("Opening %s, fullAtomic=%d", filename, fullAtomic);
#ifdef ARCH_NRF52
FSCom.remove(filename);
return FSCom.open(filename, FILE_O_WRITE);
#endif
if (!fullAtomic) {
FSCom.remove(filename); // Nuke the old file to make space (ignore if it !exists)
}
@@ -63,6 +67,9 @@ bool SafeFile::close()
f.close();
spiLock->unlock();
#ifdef ARCH_NRF52
return true;
#endif
if (!testReadback())
return false;
-3
View File
@@ -30,9 +30,6 @@ SerialConsole *console;
void consoleInit()
{
if (console) {
return;
}
auto sc = new SerialConsole(); // Must be dynamically allocated because we are now inheriting from thread
#if defined(SERIAL_HAS_ON_RECEIVE)
+2 -2
View File
@@ -19,8 +19,8 @@
TX_LOG + RX_LOG = Total air time for a particular meshtastic channel.
TX_LOG + RX_ALL_LOG = Total air time for a particular meshtastic channel, including
other lora radios.
TX_LOG + RX_LOG = Total air time for a particular meshtastic channel, including
other lora radios.
RX_ALL_LOG - RX_LOG = Other lora radios on our frequency channel.
*/
-7
View File
@@ -14,11 +14,6 @@ Lock::Lock() : handle(xSemaphoreCreateBinary())
}
}
Lock::~Lock()
{
vSemaphoreDelete(handle);
}
void Lock::lock()
{
if (xSemaphoreTake(handle, portMAX_DELAY) == false) {
@@ -35,8 +30,6 @@ void Lock::unlock()
#else
Lock::Lock() {}
Lock::~Lock() {}
void Lock::lock() {}
void Lock::unlock() {}
-1
View File
@@ -12,7 +12,6 @@ class Lock
{
public:
Lock();
~Lock();
Lock(const Lock &) = delete;
Lock &operator=(const Lock &) = delete;
+3 -14
View File
@@ -78,11 +78,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
// Configuration
// -----------------------------------------------------------------------------
// Pre-hop drop handling (compile-time flag).
#ifndef MESHTASTIC_PREHOP_DROP
#define MESHTASTIC_PREHOP_DROP 1
#endif
/// Convert a preprocessor name into a quoted string
#define xstr(s) ystr(s)
#define ystr(s) #s
@@ -234,7 +229,7 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#define BME_ADDR 0x76
#define BME_ADDR_ALTERNATE 0x77
#define MCP9808_ADDR 0x18
#define INA_ADDR 0x40 // same as SHT2X
#define INA_ADDR 0x40
#define INA_ADDR_ALTERNATE 0x41
#define INA_ADDR_WAVESHARE_UPS 0x43
#define INA3221_ADDR 0x42
@@ -247,8 +242,8 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#define LPS22HB_ADDR 0x5C
#define LPS22HB_ADDR_ALT 0x5D
#define SFA30_ADDR 0x5D
#define SHTXX_ADDR 0x44
#define SHTXX_ADDR_ALT 0x45
#define SHT31_4x_ADDR 0x44
#define SHT31_4x_ADDR_ALT 0x45
#define PMSA003I_ADDR 0x12
#define QMA6100P_ADDR 0x12
#define AHT10_ADDR 0x38
@@ -502,7 +497,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#define MESHTASTIC_EXCLUDE_PKI 1
#define MESHTASTIC_EXCLUDE_POWER_FSM 1
#define MESHTASTIC_EXCLUDE_TZ 1
#define MESHTASTIC_EXCLUDE_PKT_HISTORY_HASH 1
#endif
// Turn off all optional modules
@@ -519,7 +513,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#define MESHTASTIC_EXCLUDE_REMOTEHARDWARE 1
#define MESHTASTIC_EXCLUDE_STOREFORWARD 1
#define MESHTASTIC_EXCLUDE_TEXTMESSAGE 1
#define MESHTASTIC_EXCLUDE_TRAFFIC_MANAGEMENT 1
#define MESHTASTIC_EXCLUDE_ATAK 1
#define MESHTASTIC_EXCLUDE_CANNEDMESSAGES 1
#define MESHTASTIC_EXCLUDE_NEIGHBORINFO 1
@@ -561,9 +554,5 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#define HAS_SCREEN 0
#endif
#ifndef USE_ETHERNET_DEFAULT
#define USE_ETHERNET_DEFAULT 0
#endif
#include "DebugConfiguration.h"
#include "RF95Configuration.h"
+1 -2
View File
@@ -11,8 +11,7 @@ enum LoRaRadioType {
SX1280_RADIO,
LR1110_RADIO,
LR1120_RADIO,
LR1121_RADIO,
LR2021_RADIO
LR1121_RADIO
};
extern LoRaRadioType radioType;
+20 -81
View File
@@ -136,9 +136,7 @@ bool ScanI2CTwoWire::i2cCommandResponseLength(ScanI2C::DeviceAddress addr, uint1
return match;
}
#if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_AIR_QUALITY_SENSOR
// FIXME Move to a separate file for detection of sensors that require more complex interactions?
// For SEN5X detection
/// for SEN5X detection
// Note, this code needs to be called before setting the I2C bus speed
// for the screen at high speed. The speed needs to be at 100kHz, otherwise
// detection will not work
@@ -176,46 +174,6 @@ String readSEN5xProductName(TwoWire *i2cBus, uint8_t address)
return String(productName);
}
#endif
#if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_ENVIRONMENTAL_SENSOR
bool detectSHT21SerialNumber(TwoWire *i2cBus, uint8_t address)
{
i2cBus->beginTransmission(address);
i2cBus->write(0xFA);
i2cBus->write(0x0F);
if (i2cBus->endTransmission() != 0)
return false;
if (i2cBus->requestFrom(address, (uint8_t)8) != 8)
return false;
// Just flush the data
while (i2cBus->available() < 8) {
i2cBus->read();
}
i2cBus->beginTransmission(address);
i2cBus->write(0xFC);
i2cBus->write(0xC9);
if (i2cBus->endTransmission() != 0)
return false;
if (i2cBus->requestFrom(address, (uint8_t)6) != 6)
return false;
// Just flush the data
while (i2cBus->available() < 6) {
i2cBus->read();
}
// Assume we detect the SHT21 if something came back from the request
return true;
}
#endif
#define SCAN_SIMPLE_CASE(ADDR, T, ...) \
case ADDR: \
@@ -413,47 +371,27 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
break;
#endif
#if !defined(M5STACK_UNITC6L)
case INA_ADDR: // Same as SHT2X
case INA_ADDR:
case INA_ADDR_ALTERNATE:
case INA_ADDR_WAVESHARE_UPS: {
uint16_t mfg = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFE), 2);
case INA_ADDR_WAVESHARE_UPS:
registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFE), 2);
LOG_DEBUG("Register MFG_UID: 0x%x", registerValue);
if (registerValue == 0x5449) {
registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFF), 2);
LOG_DEBUG("Register DIE_UID: 0x%x", registerValue);
LOG_DEBUG("Register MFG_UID: 0x%x", mfg);
// Only read DIE_UID for vendors we recognize as INA-compatible to avoid
// an extra I2C transaction + delay on other devices sharing this address.
if (mfg == 0x5449 || mfg == 0x190F) {
uint16_t die = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFF), 2);
LOG_DEBUG("Register DIE_UID: 0x%x", die);
// TI INA226 or fully compatible clones (e.g. TPA626)
if (mfg == 0x5449 && die == 0x2260) {
if (registerValue == 0x2260) {
logFoundDevice("INA226", (uint8_t)addr.address);
type = INA226;
}
// Silergy SQ52201 (INA226-compatible with different IDs)
else if (mfg == 0x190F && die == 0x0000) {
logFoundDevice("INA226 (SQ52201)", (uint8_t)addr.address);
type = INA226;
}
// TI INA260
else if (mfg == 0x5449) {
} else {
logFoundDevice("INA260", (uint8_t)addr.address);
type = INA260;
}
}
#if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_ENVIRONMENTAL_SENSOR
if (type == NONE && detectSHT21SerialNumber(i2cBus, (uint8_t)addr.address)) {
logFoundDevice("SHTXX (SHT2X)", (uint8_t)addr.address);
type = SHTXX;
}
#endif
else { // Assume INA219 if none of the above ones are found
} else { // Assume INA219 if INA260 ID is not found
logFoundDevice("INA219", (uint8_t)addr.address);
type = INA219;
}
break;
}
case INA3221_ADDR:
registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFE), 2);
LOG_DEBUG("Register MFG_UID FE: 0x%x", registerValue);
@@ -510,19 +448,22 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
}
break;
}
case SHTXX_ADDR: // same as OPT3001_ADDR_ALT
case SHTXX_ADDR_ALT: // same as OPT3001_ADDR
case SHT31_4x_ADDR: // same as OPT3001_ADDR_ALT
case SHT31_4x_ADDR_ALT: // same as OPT3001_ADDR
if (getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0x7E), 2) == 0x5449) {
type = OPT3001;
logFoundDevice("OPT3001", (uint8_t)addr.address);
} else { // SHTXX
type = SHTXX;
logFoundDevice("SHTXX", (uint8_t)addr.address);
} else if (i2cCommandResponseLength(addr, 0x89, 6)) { // SHT4x serial number (6 bytes inc. CRC)
type = SHT4X;
logFoundDevice("SHT4X", (uint8_t)addr.address);
} else {
type = SHT31;
logFoundDevice("SHT31", (uint8_t)addr.address);
}
break;
SCAN_SIMPLE_CASE(SHTC3_ADDR, SHTXX, "SHTXX", (uint8_t)addr.address)
SCAN_SIMPLE_CASE(SHTC3_ADDR, SHTC3, "SHTC3", (uint8_t)addr.address)
case RCWL9620_ADDR:
// get MAX30102 PARTID
registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0xFF), 1);
@@ -758,7 +699,6 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
logFoundDevice("BMX160", (uint8_t)addr.address);
break;
} else {
#if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_AIR_QUALITY_SENSOR
String prod = "";
prod = readSEN5xProductName(i2cBus, addr.address);
if (prod.startsWith("SEN55")) {
@@ -774,7 +714,6 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
logFoundDevice("Sensirion SEN50", addr.address);
break;
}
#endif
if (addr.address == BMX160_ADDR) {
type = BMX160;
logFoundDevice("BMX160", (uint8_t)addr.address);
+3 -19
View File
@@ -103,14 +103,6 @@ static int32_t gpsSwitch()
if (gps) {
int currentState = digitalRead(PIN_GPS_SWITCH);
// Respect explicit NOT_PRESENT mode and do not let the hardware switch re-enable GPS.
if (config.position.gps_mode == meshtastic_Config_PositionConfig_GpsMode_NOT_PRESENT) {
gps->disable();
lastState = currentState;
firstrun = false;
return 1000;
}
// if the switch is set to zero, disable the GPS Thread
if (firstrun)
if (currentState == LOW)
@@ -1025,13 +1017,10 @@ void GPS::up()
setPowerState(GPS_ACTIVE);
}
// We've finished a GPS search cycle (lock or timeout). Enter a low power state, potentially.
// We've got a GPS lock. Enter a low power state, potentially.
void GPS::down()
{
if (hasValidLocation)
scheduling.informGotLock();
else
scheduling.informSearchFailed();
scheduling.informGotLock();
uint32_t predictedSearchDuration = scheduling.predictedSearchDurationMs();
uint32_t sleepTime = scheduling.msUntilNextSearch();
uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval);
@@ -1556,12 +1545,7 @@ std::unique_ptr<GPS> GPS::createGps()
_en_gpio = PIN_GPS_EN;
#endif
#ifdef ARCH_PORTDUINO
if (portduino_config.has_gps) {
// These need to set as flags so later checks will pass on native and GPS will work.
// They are not used for any hardware access.
_rx_gpio = 1;
_tx_gpio = 1;
} else
if (!portduino_config.has_gps)
return nullptr;
#endif
if (!_rx_gpio || !_serial_gps) // Configured to have no GPS at all
+2 -34
View File
@@ -15,19 +15,6 @@ void GPSUpdateScheduling::informGotLock()
searchEndedMs = millis();
LOG_DEBUG("Took %us to get lock", (searchEndedMs - searchStartedMs) / 1000);
updateLockTimePrediction();
consecutiveFailures = 0; // Drop back to fast cadence as soon as we acquire any fix
}
// Search finished without obtaining a fix. We still need to mark the end time so
// the next sleep is timed correctly, but we must not feed the timeout duration
// into predictedMsToGetLock — doing so poisons msUntilNextSearch() and causes
// down() to fall into GPS_IDLE, leaving the chip awake on subsequent indoor cycles.
void GPSUpdateScheduling::informSearchFailed()
{
searchEndedMs = millis();
consecutiveFailures++;
LOG_DEBUG("GPS search ended without fix after %us (consecutive failures: %u)", (searchEndedMs - searchStartedMs) / 1000,
consecutiveFailures);
}
// Clear old lock-time prediction data.
@@ -38,7 +25,6 @@ void GPSUpdateScheduling::reset()
searchEndedMs = 0;
searchCount = 0;
predictedMsToGetLock = 0;
consecutiveFailures = 0;
}
// How many milliseconds before we should next search for GPS position
@@ -50,20 +36,6 @@ uint32_t GPSUpdateScheduling::msUntilNextSearch()
// Target interval (seconds), between GPS updates
uint32_t updateInterval = Default::getConfiguredOrDefaultMs(config.position.gps_update_interval, default_gps_update_interval);
// After a failed search, back off: indoors / no-sky environments will keep failing,
// so wake at most once per broadcast interval rather than once per gps_update_interval.
// Capped at 1 hour so a user-configured very-long broadcast interval still retries
// periodically (in case conditions change). Reset on any successful lock.
if (consecutiveFailures > 0) {
constexpr uint32_t failureRetryCapMs = 60UL * 60UL * 1000UL; // 1 hour cap
uint32_t failureSleepMs =
Default::getConfiguredOrDefaultMs(config.position.position_broadcast_secs, default_broadcast_interval_secs);
if (failureSleepMs > failureRetryCapMs)
failureSleepMs = failureRetryCapMs;
if (updateInterval < failureSleepMs)
updateInterval = failureSleepMs;
}
// Check how long until we should start searching, to hopefully hit our target interval
uint32_t dueAtMs = searchEndedMs + updateInterval;
uint32_t compensatedStart = dueAtMs - predictedMsToGetLock;
@@ -99,18 +71,14 @@ bool GPSUpdateScheduling::isUpdateDue()
bool GPSUpdateScheduling::searchedTooLong()
{
constexpr uint32_t oneMinuteMs = 60UL * 1000UL;
constexpr uint32_t maxSearchClampMs = 15UL * oneMinuteMs; // Hard cap: 15 minutes is always too long
constexpr uint32_t postFailureSearchMs = 5UL * oneMinuteMs; // Tighter dwell once we know the environment is hostile
constexpr uint32_t maxSearchClampMs = 15UL * oneMinuteMs; // Hard cap: 15 minutes is always too long
uint32_t elapsed = elapsedSearchMs();
// Anything over 15 minutes is too long, regardless of the broadcast interval.
// TODO: Make a smarter algorithm that backs off the search dwell time when not getting a lock.
if (elapsed > maxSearchClampMs)
return true;
// After a prior failed search, shorten the dwell
if (consecutiveFailures > 0 && elapsed > postFailureSearchMs)
return true;
uint32_t minimumOrConfiguredSecs =
Default::getConfiguredOrMinimumValue(config.position.position_broadcast_secs, default_broadcast_interval_secs);
uint32_t maxSearchMs = Default::getConfiguredOrDefaultMs(minimumOrConfiguredSecs, default_broadcast_interval_secs);
+1 -3
View File
@@ -8,8 +8,7 @@ class GPSUpdateScheduling
public:
// Marks the time of these events, for calculation use
void informSearching();
void informGotLock(); // Predicted lock-time is recalculated here
void informSearchFailed(); // Search ended without a fix; prediction is left untouched
void informGotLock(); // Predicted lock-time is recalculated here
void reset(); // Reset the prediction - after GPS::disable() / GPS::enable()
bool isUpdateDue(); // Is it time to begin searching for a GPS position?
@@ -25,7 +24,6 @@ class GPSUpdateScheduling
uint32_t searchEndedMs = 0;
uint32_t searchCount = 0;
uint32_t predictedMsToGetLock = 0;
uint32_t consecutiveFailures = 0; // Count of search cycles that ended without a fix; reset on lock
const float weighting = 0.2; // Controls exponential smoothing of lock-times prediction. 20% weighting of "latest lock-time".
};
+223 -194
View File
@@ -39,7 +39,6 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#include "draw/NodeListRenderer.h"
#include "draw/NotificationRenderer.h"
#include "draw/UIRenderer.h"
#include "graphics/TFTColorRegions.h"
#include "modules/CannedMessageModule.h"
#if !MESHTASTIC_EXCLUDE_GPS
@@ -55,21 +54,26 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
#include "gps/RTC.h"
#include "graphics/ScreenFonts.h"
#include "graphics/SharedUIDisplay.h"
#include "graphics/TFTPalette.h"
#include "graphics/emotes.h"
#include "graphics/images.h"
#include "input/TouchScreenImpl1.h"
#include "main.h"
#include "mesh-pb-constants.h"
#include "mesh/Channels.h"
#include "mesh/Default.h"
#include "mesh/generated/meshtastic/deviceonly.pb.h"
#include "modules/ExternalNotificationModule.h"
#include "modules/TextMessageModule.h"
#include "modules/WaypointModule.h"
#include "sleep.h"
#include "target_specific.h"
extern MessageStore messageStore;
#if USE_TFTDISPLAY
extern uint16_t TFT_MESH;
#else
uint16_t TFT_MESH = COLOR565(0x67, 0xEA, 0x94);
#endif
#if HAS_WIFI && !defined(ARCH_PORTDUINO)
#include "mesh/wifi/WiFiAPClient.h"
#endif
@@ -94,7 +98,6 @@ namespace graphics
// This means the *visible* area (sh1106 can address 132, but shows 128 for example)
#define IDLE_FRAMERATE 1 // in fps
#define COMPASS_ACTIVE_FRAMERATE 20
// DEBUG
#define NUM_EXTRA_FRAMES 3 // text message and debug frame
@@ -104,27 +107,6 @@ namespace graphics
// A text message frame + debug frame + all the node infos
FrameCallback *normalFrames;
static uint32_t targetFramerate = IDLE_FRAMERATE;
#if GRAPHICS_TFT_COLORING_ENABLED
static inline void prepareFrameColorRegions()
{
#if GRAPHICS_TFT_COLORING_ENABLED
clearTFTColorRegions();
// Full-frame FrameMono inversion for themes that need it (e.g. light themes).
if (isThemeFullFrameInvert()) {
setAndRegisterTFTColorRole(TFTColorRole::FrameMono, getThemeBodyFg(), getThemeBodyBg(), 0, 0, screen->getWidth(),
screen->getHeight());
}
#endif
}
#endif
static inline void updateUiFrame(OLEDDisplayUi *ui)
{
#if GRAPHICS_TFT_COLORING_ENABLED
prepareFrameColorRegions();
#endif
ui->update();
}
// Global variables for alert banner - explicitly define with extern "C" linkage to prevent optimization
uint32_t logo_timeout = 5000; // 4 seconds for EACH logo
@@ -153,60 +135,6 @@ static bool heartbeat = false;
extern bool hasUnreadMessage;
static inline float wrapHeading360(float heading)
{
if (heading < 0.0f) {
heading += 360.0f;
} else if (heading >= 360.0f) {
heading -= 360.0f;
}
return heading;
}
void Screen::setHeading(float heading)
{
const float wrappedHeading = wrapHeading360(heading);
if (!hasCompass) {
hasCompass = true;
compassHeading = wrappedHeading;
return;
}
// Interpolate using shortest-path angular delta to avoid jumps around 0/360.
float delta = wrappedHeading - compassHeading;
if (delta > 180.0f) {
delta -= 360.0f;
} else if (delta < -180.0f) {
delta += 360.0f;
}
// Adaptive filtering:
// - Strong damping for tiny deltas (jitter)
// - Faster response for larger turns
const float absDelta = (delta >= 0.0f) ? delta : -delta;
if (absDelta < 1.0f) {
return;
}
float alpha = 0.35f;
if (absDelta > 25.0f) {
alpha = 0.85f;
} else if (absDelta > 10.0f) {
alpha = 0.65f;
}
float step = delta * alpha;
const float maxStep = 12.0f;
if (step > maxStep) {
step = maxStep;
} else if (step < -maxStep) {
step = -maxStep;
}
compassHeading = wrapHeading360(compassHeading + step);
}
// ==============================
// Overlay Alert Banner Renderer
// ==============================
@@ -230,7 +158,6 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
#endif
// Store the message and set the expiration timestamp
strncpy(NotificationRenderer::alertBannerMessage, banner_overlay_options.message, 255);
NotificationRenderer::parseBannerMessageWithFonts(NotificationRenderer::alertBannerMessage);
NotificationRenderer::alertBannerMessage[255] = '\0'; // Ensure null termination
NotificationRenderer::alertBannerUntil =
(banner_overlay_options.durationMs == 0) ? 0 : millis() + banner_overlay_options.durationMs;
@@ -240,11 +167,11 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
NotificationRenderer::alertBannerCallback = banner_overlay_options.bannerCallback;
NotificationRenderer::curSelected = banner_overlay_options.InitialSelected;
NotificationRenderer::pauseBanner = false;
NotificationRenderer::current_notification_type = banner_overlay_options.notificationType;
NotificationRenderer::current_notification_type = notificationTypeEnum::selection_picker;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setTargetFPS(60);
updateUiFrame(ui);
ui->update();
}
// Called to trigger a banner with custom message and duration
@@ -264,9 +191,9 @@ void Screen::showNodePicker(const char *message, uint32_t durationMs, std::funct
NotificationRenderer::current_notification_type = notificationTypeEnum::node_picker;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setTargetFPS(60);
updateUiFrame(ui);
ui->update();
}
// Called to trigger a banner with custom message and duration
@@ -288,9 +215,9 @@ void Screen::showNumberPicker(const char *message, uint32_t durationMs, uint8_t
NotificationRenderer::currentNumber = 0;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setTargetFPS(60);
updateUiFrame(ui);
ui->update();
}
void Screen::showTextInput(const char *header, const char *initialText, uint32_t durationMs,
@@ -311,9 +238,9 @@ void Screen::showTextInput(const char *header, const char *initialText, uint32_t
// Set the overlay using the same pattern as other notification types
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
ui->setTargetFPS(60);
updateUiFrame(ui);
ui->update();
}
static void drawModuleFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
@@ -345,25 +272,10 @@ static void drawModuleFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int
float Screen::estimatedHeading(double lat, double lon)
{
static double oldLat, oldLon;
static float b = -1.0f;
static uint32_t lastHeadingAtMs = 0;
const uint32_t now = millis();
const uint32_t gpsUpdateIntervalSecs =
Default::getConfiguredOrDefault(config.position.gps_update_interval, default_gps_update_interval);
uint32_t effectiveUpdateIntervalSecs = gpsUpdateIntervalSecs;
if (config.position.position_broadcast_smart_enabled) {
const uint32_t smartMinIntervalSecs = Default::getConfiguredOrDefault(
config.position.broadcast_smart_minimum_interval_secs, default_broadcast_smart_minimum_interval_secs);
if (smartMinIntervalSecs > effectiveUpdateIntervalSecs) {
effectiveUpdateIntervalSecs = smartMinIntervalSecs;
}
}
// Two expected update windows; keep arithmetic 32-bit to avoid pulling in larger 64-bit helpers.
const uint32_t headingStaleMs =
(effectiveUpdateIntervalSecs > (UINT32_MAX / 2000U)) ? UINT32_MAX : (effectiveUpdateIntervalSecs * 2000U);
static float b;
if (oldLat == 0) {
// Need at least two position points before we can infer heading.
// just prepare for next time
oldLat = lat;
oldLon = lon;
@@ -371,20 +283,12 @@ float Screen::estimatedHeading(double lat, double lon)
}
float d = GeoCoord::latLongToMeter(oldLat, oldLon, lat, lon);
if (d < 10) { // haven't moved enough, keep previous heading (invalid until first real movement)
if (lastHeadingAtMs != 0 && (now - lastHeadingAtMs) >= headingStaleMs) {
// Heading is stale after prolonged no-movement; force reacquire.
b = -1.0f;
oldLat = lat;
oldLon = lon;
}
if (d < 10) // haven't moved enough, just keep current bearing
return b;
}
b = GeoCoord::bearing(oldLat, oldLon, lat, lon) * RAD_TO_DEG;
oldLat = lat;
oldLon = lon;
lastHeadingAtMs = now;
return b;
}
@@ -405,6 +309,30 @@ Screen::Screen(ScanI2C::DeviceAddress address, meshtastic_Config_DisplayConfig_O
{
graphics::normalFrames = new FrameCallback[MAX_NUM_NODES + NUM_EXTRA_FRAMES];
int32_t rawRGB = uiconfig.screen_rgb_color;
// Only validate the combined value once
if (rawRGB > 0 && rawRGB <= 255255255) {
LOG_INFO("Setting screen RGB color to user chosen: 0x%06X", rawRGB);
// Extract each component as a normal int first
int r = (rawRGB >> 16) & 0xFF;
int g = (rawRGB >> 8) & 0xFF;
int b = rawRGB & 0xFF;
if (r >= 0 && r <= 255 && g >= 0 && g <= 255 && b >= 0 && b <= 255) {
TFT_MESH = COLOR565(static_cast<uint8_t>(r), static_cast<uint8_t>(g), static_cast<uint8_t>(b));
}
#ifdef TFT_MESH_OVERRIDE
} else if (rawRGB == 0) {
LOG_INFO("Setting screen RGB color to TFT_MESH_OVERRIDE: 0x%04X", TFT_MESH_OVERRIDE);
// Default to TFT_MESH_OVERRIDE if available
TFT_MESH = TFT_MESH_OVERRIDE;
#endif
} else {
// Default best readable yellow color
LOG_INFO("Setting screen RGB color to default: (255,255,128)");
TFT_MESH = COLOR565(255, 255, 128);
}
#if defined(USE_SH1106) || defined(USE_SH1107) || defined(USE_SH1107_128_64)
dispdev = new SH1106Wire(address.address, -1, -1, geometry,
(address.port == ScanI2C::I2CPort::WIRE1) ? HW_I2C::I2C_TWO : HW_I2C::I2C_ONE);
@@ -425,11 +353,6 @@ Screen::Screen(ScanI2C::DeviceAddress address, meshtastic_Config_DisplayConfig_O
#elif defined(USE_SSD1306)
dispdev = new SSD1306Wire(address.address, -1, -1, geometry,
(address.port == ScanI2C::I2CPort::WIRE1) ? HW_I2C::I2C_TWO : HW_I2C::I2C_ONE);
#if defined(OLED_Y_OFFSET_PAGES)
// Panels whose active window does not start at GDDRAM row 0 (e.g. 72x40
// modules on pages 3..7) need a fixed vertical page shift on every write.
static_cast<SSD1306Wire *>(dispdev)->setYOffset(OLED_Y_OFFSET_PAGES);
#endif
#elif defined(USE_SPISSD1306)
dispdev = new SSD1306Spi(SSD1306_RESET, SSD1306_RS, SSD1306_NSS, GEOMETRY_64_48);
if (!dispdev->init()) {
@@ -472,13 +395,9 @@ Screen::Screen(ScanI2C::DeviceAddress address, meshtastic_Config_DisplayConfig_O
#endif
#if defined(USE_ST7789)
// Keep firmware and ST7789 driver region structs layout-compatible:
// we pass `graphics::colorRegions` through a type cast below.
static_assert(sizeof(graphics::TFTColorRegion) == sizeof(::TFTColorRegion),
"graphics::TFTColorRegion layout must match ST7789 TFTColorRegion");
static_cast<ST7789Spi *>(dispdev)->setRGB(TFTPalette::White, (::TFTColorRegion *)colorRegions);
static_cast<ST7789Spi *>(dispdev)->setRGB(TFT_MESH);
#elif defined(USE_ST7796)
static_cast<ST7796Spi *>(dispdev)->setRGB(TFTPalette::White);
static_cast<ST7796Spi *>(dispdev)->setRGB(TFT_MESH);
#endif
ui = new OLEDDisplayUi(dispdev);
@@ -529,11 +448,6 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
delay(100);
#endif
#if !ARCH_PORTDUINO
#if defined(USE_ST7789) && defined(VTFT_CTRL)
// Ensure panel power rail is enabled before sending wake commands.
pinMode(VTFT_CTRL, OUTPUT);
digitalWrite(VTFT_CTRL, LOW);
#endif
dispdev->displayOn();
#endif
@@ -555,6 +469,10 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
ui->init();
#endif
#if defined(USE_ST7789) && defined(VTFT_LEDA)
#ifdef VTFT_CTRL
pinMode(VTFT_CTRL, OUTPUT);
digitalWrite(VTFT_CTRL, LOW);
#endif
ui->init();
#ifdef ESP_PLATFORM
analogWrite(VTFT_LEDA, BRIGHTNESS_DEFAULT);
@@ -595,22 +513,23 @@ void Screen::handleSetOn(bool on, FrameCallback einkScreensaver)
#endif
#ifdef USE_ST7789
SPI1.end();
// Keep TFT control pins in deterministic states while timed-off.
// Floating/default pin states can corrupt panel edge rows on wake.
#if defined(ARCH_ESP32)
#ifdef VTFT_LEDA
pinMode(VTFT_LEDA, OUTPUT);
digitalWrite(VTFT_LEDA, !TFT_BACKLIGHT_ON);
pinMode(VTFT_LEDA, ANALOG);
#endif
#ifdef VTFT_CTRL
pinMode(VTFT_CTRL, OUTPUT);
digitalWrite(VTFT_CTRL, HIGH);
pinMode(VTFT_CTRL, ANALOG);
#endif
pinMode(ST7789_RESET, ANALOG);
pinMode(ST7789_RS, ANALOG);
pinMode(ST7789_NSS, ANALOG);
#else
nrf_gpio_cfg_default(VTFT_LEDA);
nrf_gpio_cfg_default(VTFT_CTRL);
nrf_gpio_cfg_default(ST7789_RESET);
nrf_gpio_cfg_default(ST7789_RS);
nrf_gpio_cfg_default(ST7789_NSS);
#endif
pinMode(ST7789_RESET, OUTPUT);
digitalWrite(ST7789_RESET, HIGH);
pinMode(ST7789_RS, OUTPUT);
digitalWrite(ST7789_RS, HIGH);
pinMode(ST7789_NSS, OUTPUT);
digitalWrite(ST7789_NSS, HIGH);
#endif
#ifdef USE_ST7796
SPI1.end();
@@ -665,16 +584,16 @@ void Screen::setup()
static_cast<SH1106Wire *>(dispdev)->setSubtype(7);
#endif
#if defined(USE_ST7789)
static_assert(sizeof(graphics::TFTColorRegion) == sizeof(::TFTColorRegion),
"graphics::TFTColorRegion layout must match ST7789 TFTColorRegion");
static_cast<ST7789Spi *>(dispdev)->setRGB(TFTPalette::White, (::TFTColorRegion *)colorRegions);
#if defined(USE_ST7789) && defined(TFT_MESH)
// Apply custom RGB color (e.g. Heltec T114/T190)
static_cast<ST7789Spi *>(dispdev)->setRGB(TFT_MESH);
#endif
#if defined(MUZI_BASE)
dispdev->delayPoweron = true;
#endif
#if defined(USE_ST7796)
static_cast<ST7796Spi *>(dispdev)->setRGB(TFTPalette::White);
#if defined(USE_ST7796) && defined(TFT_MESH)
// Custom text color, if defined in variant.h
static_cast<ST7796Spi *>(dispdev)->setRGB(TFT_MESH);
#endif
// Initialize display and UI system
@@ -701,7 +620,7 @@ void Screen::setup()
static OverlayCallback overlays[] = {
graphics::UIRenderer::drawNavigationBar // Custom indicator icons for each frame
};
ui->setOverlays(overlays, 1);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
// Enable UTF-8 to display mapping
dispdev->setFontTableLookupFunction(customFontTableLookup);
@@ -720,7 +639,7 @@ void Screen::setup()
#endif
{
const char *region = myRegion ? myRegion->name : nullptr;
graphics::UIRenderer::drawBootIconScreen(region, display, state, x, y);
graphics::UIRenderer::drawIconScreen(region, display, state, x, y);
}
};
ui->setFrames(alertFrames, 1);
@@ -759,9 +678,9 @@ void Screen::setup()
// Turn on display and trigger first draw
handleSetOn(true);
graphics::currentResolution = graphics::determineScreenResolution(dispdev->height(), dispdev->width());
updateUiFrame(ui);
ui->update();
#ifndef USE_EINK
updateUiFrame(ui); // Some SSD1306 clones drop the first draw, so run twice
ui->update(); // Some SSD1306 clones drop the first draw, so run twice
#endif
serialSinceMsec = millis();
@@ -834,7 +753,7 @@ void Screen::forceDisplay(bool forceUiUpdate)
do {
startUpdate = millis(); // Handle impossibly unlikely corner case of a millis() overflow..
delay(10);
updateUiFrame(ui);
ui->update();
} while (ui->getUiState()->lastUpdate < startUpdate);
// Return to normal frame rate
@@ -905,9 +824,9 @@ int32_t Screen::runOnce()
static FrameCallback bootOEMFrames[] = {graphics::UIRenderer::drawOEMBootScreen};
static const int bootOEMFrameCount = sizeof(bootOEMFrames) / sizeof(bootOEMFrames[0]);
ui->setFrames(bootOEMFrames, bootOEMFrameCount);
updateUiFrame(ui);
ui->update();
#ifndef USE_EINK
updateUiFrame(ui);
ui->update();
#endif
showingOEMBootScreen = false;
}
@@ -915,7 +834,7 @@ int32_t Screen::runOnce()
#ifndef DISABLE_WELCOME_UNSET
if (!NotificationRenderer::isOverlayBannerShowing() && config.lora.region == meshtastic_Config_LoRaConfig_RegionCode_UNSET) {
#if defined(OLED_TINY)
#if defined(M5STACK_UNITC6L)
menuHandler::LoraRegionPicker();
#else
menuHandler::OnboardMessage();
@@ -998,28 +917,15 @@ int32_t Screen::runOnce()
// this must be before the frameState == FIXED check, because we always
// want to draw at least one FIXED frame before doing forceDisplay
updateUiFrame(ui);
ui->update();
// Switch to a low framerate (to save CPU) when we are not in transition
// but we should only call setTargetFPS when framestate changes, because
// otherwise that breaks animations.
uint32_t desiredFramerate = IDLE_FRAMERATE;
#if HAS_GPS && !defined(USE_EINK)
if (showingNormalScreen && hasCompass) {
const uint8_t currentFrame = ui->getUiState()->currentFrame;
if ((framesetInfo.positions.gps != 255 && currentFrame == framesetInfo.positions.gps) ||
(framesetInfo.positions.waypoint != 255 && currentFrame == framesetInfo.positions.waypoint) ||
(framesetInfo.positions.firstFavorite != 255 && currentFrame >= framesetInfo.positions.firstFavorite &&
currentFrame <= framesetInfo.positions.lastFavorite)) {
desiredFramerate = COMPASS_ACTIVE_FRAMERATE;
}
}
#endif
if (targetFramerate != desiredFramerate && ui->getUiState()->frameState == FIXED) {
if (targetFramerate != IDLE_FRAMERATE && ui->getUiState()->frameState == FIXED) {
// oldFrameState = ui->getUiState()->frameState;
targetFramerate = desiredFramerate;
targetFramerate = IDLE_FRAMERATE;
ui->setTargetFPS(targetFramerate);
forceDisplay();
@@ -1060,7 +966,7 @@ void Screen::setSSLFrames()
// LOG_DEBUG("Show SSL frames");
static FrameCallback sslFrames[] = {NotificationRenderer::drawSSLScreen};
ui->setFrames(sslFrames, 1);
updateUiFrame(ui);
ui->update();
}
}
@@ -1096,7 +1002,7 @@ void Screen::setScreensaverFrames(FrameCallback einkScreensaver)
do {
startUpdate = millis(); // Handle impossibly unlikely corner case of a millis() overflow..
delay(1);
updateUiFrame(ui);
ui->update();
} while (ui->getUiState()->lastUpdate < startUpdate);
#if defined(USE_EINK_PARALLELDISPLAY)
@@ -1152,7 +1058,7 @@ void Screen::setFrames(FrameFocus focus)
#if defined(DISPLAY_CLOCK_FRAME)
if (!hiddenFrames.clock) {
fsi.positions.clock = numframes;
#if defined(OLED_TINY)
#if defined(M5STACK_UNITC6L)
normalFrames[numframes++] = graphics::ClockRenderer::drawAnalogClockFrame;
#else
normalFrames[numframes++] = uiconfig.is_clockface_analog ? graphics::ClockRenderer::drawAnalogClockFrame
@@ -1290,8 +1196,8 @@ void Screen::setFrames(FrameFocus focus)
for (size_t i = 0; i < nodeDB->getNumMeshNodes(); i++) {
const meshtastic_NodeInfoLite *n = nodeDB->getMeshNodeByIndex(i);
if (n && n->num != nodeDB->getNodeNum() && nodeInfoLiteIsFavorite(n)) {
favoriteFrames.push_back(graphics::UIRenderer::drawFavoriteNode);
if (n && n->num != nodeDB->getNodeNum() && n->is_favorite) {
favoriteFrames.push_back(graphics::UIRenderer::drawNodeInfo);
}
}
@@ -1318,9 +1224,9 @@ void Screen::setFrames(FrameFocus focus)
// Add overlays: frame icons and alert banner)
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
prevFrame = -1; // Force drawFavoriteNode to pick a new node (because our list just changed)
prevFrame = -1; // Force drawNodeInfo to pick a new node (because our list just changed)
// Focus on a specific frame, in the frame set we just created
switch (focus) {
@@ -1471,15 +1377,9 @@ void Screen::blink()
dispdev->setBrightness(254);
while (count > 0) {
dispdev->fillRect(0, 0, dispdev->getWidth(), dispdev->getHeight());
#if GRAPHICS_TFT_COLORING_ENABLED
prepareFrameColorRegions();
#endif
dispdev->display();
delay(50);
dispdev->clear();
#if GRAPHICS_TFT_COLORING_ENABLED
prepareFrameColorRegions();
#endif
dispdev->display();
delay(50);
count = count - 1;
@@ -1611,11 +1511,8 @@ void Screen::showFrame(FrameDirection direction)
void Screen::setFastFramerate()
{
#if defined(OLED_TINY)
#if defined(M5STACK_UNITC6L)
dispdev->clear();
#if GRAPHICS_TFT_COLORING_ENABLED
prepareFrameColorRegions();
#endif
dispdev->display();
#endif
// We are about to start a transition so speed up fps
@@ -1648,6 +1545,138 @@ int Screen::handleStatusUpdate(const meshtastic::Status *arg)
return 0;
}
// Handles when message is received; will jump to text message frame.
int Screen::handleTextMessage(const meshtastic_MeshPacket *packet)
{
if (showingNormalScreen) {
if (packet->from == 0) {
// Outgoing message (likely sent from phone)
devicestate.has_rx_text_message = false;
memset(&devicestate.rx_text_message, 0, sizeof(devicestate.rx_text_message));
hiddenFrames.textMessage = true;
hasUnreadMessage = false; // Clear unread state when user replies
setFrames(FOCUS_PRESERVE); // Stay on same frame, silently update frame list
} else {
// Incoming message
devicestate.has_rx_text_message = true; // Needed to include the message frame
hasUnreadMessage = true; // Enables mail icon in the header
setFrames(FOCUS_PRESERVE); // Refresh frame list without switching view (no-op during text_input)
// Only wake/force display if the configuration allows it
if (shouldWakeOnReceivedMessage()) {
setOn(true); // Wake up the screen first
forceDisplay(); // Forces screen redraw
}
// === Prepare banner/popup content ===
const meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(packet->from);
const meshtastic_Channel channel =
channels.getByIndex(packet->channel ? packet->channel : channels.getPrimaryIndex());
const char *longName = (node && node->has_user) ? node->user.long_name : nullptr;
const char *msgRaw = reinterpret_cast<const char *>(packet->decoded.payload.bytes);
char banner[256];
bool isAlert = false;
if (moduleConfig.external_notification.alert_bell || moduleConfig.external_notification.alert_bell_vibra ||
moduleConfig.external_notification.alert_bell_buzzer)
// Check for bell character to determine if this message is an alert
for (size_t i = 0; i < packet->decoded.payload.size && i < 100; i++) {
if (msgRaw[i] == ASCII_BELL) {
isAlert = true;
break;
}
}
// Unlike generic messages, alerts (when enabled via the ext notif module) ignore any
// 'mute' preferences set to any specific node or channel.
// If on-screen keyboard is active, show a transient popup over keyboard instead of interrupting it
if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
// Wake and force redraw so popup is visible immediately
if (shouldWakeOnReceivedMessage()) {
setOn(true);
forceDisplay();
}
// Build popup: title = message source name, content = message text (sanitized)
// Title
char titleBuf[64] = {0};
if (longName && longName[0]) {
// Sanitize sender name
std::string t = sanitizeString(longName);
strncpy(titleBuf, t.c_str(), sizeof(titleBuf) - 1);
} else {
strncpy(titleBuf, "Message", sizeof(titleBuf) - 1);
}
// Content: payload bytes may not be null-terminated, remove ASCII_BELL and sanitize
char content[256] = {0};
{
std::string raw;
raw.reserve(packet->decoded.payload.size);
for (size_t i = 0; i < packet->decoded.payload.size; ++i) {
char c = msgRaw[i];
if (c == ASCII_BELL)
continue; // strip bell
raw.push_back(c);
}
std::string sanitized = sanitizeString(raw);
strncpy(content, sanitized.c_str(), sizeof(content) - 1);
}
NotificationRenderer::showKeyboardMessagePopupWithTitle(titleBuf, content, 3000);
// Maintain existing buzzer behavior on M5 if applicable
#if defined(M5STACK_UNITC6L)
if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
(isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
(!isBroadcast(packet->to) && isToUs(packet))) {
playLongBeep();
}
#endif
} else {
// No keyboard active: use regular banner flow, respecting mute settings
if (isAlert) {
if (longName && longName[0]) {
snprintf(banner, sizeof(banner), "Alert Received from\n%s", longName);
} else {
strcpy(banner, "Alert Received");
}
screen->showSimpleBanner(banner, 3000);
} else if (!channel.settings.has_module_settings || !channel.settings.module_settings.is_muted) {
if (longName && longName[0]) {
if (currentResolution == ScreenResolution::UltraLow) {
strcpy(banner, "New Message");
} else {
snprintf(banner, sizeof(banner), "New Message from\n%s", longName);
}
} else {
strcpy(banner, "New Message");
}
#if defined(M5STACK_UNITC6L)
screen->setOn(true);
screen->showSimpleBanner(banner, 1500);
if (config.device.buzzer_mode != meshtastic_Config_DeviceConfig_BuzzerMode_DIRECT_MSG_ONLY ||
(isAlert && moduleConfig.external_notification.alert_bell_buzzer) ||
(!isBroadcast(packet->to) && isToUs(packet))) {
// Beep if not in DIRECT_MSG_ONLY mode or if in DIRECT_MSG_ONLY mode and either
// - packet contains an alert and alert bell buzzer is enabled
// - packet is a non-broadcast that is addressed to this node
playLongBeep();
}
#else
screen->showSimpleBanner(banner, 3000);
#endif
}
}
}
}
return 0;
}
// Triggered by MeshModules
int Screen::handleUIFrameEvent(const UIFrameEvent *event)
{
@@ -1693,9 +1722,9 @@ int Screen::handleInputEvent(const InputEvent *event)
if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
NotificationRenderer::inEvent = *event;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
setFastFramerate(); // Draw ASAP
updateUiFrame(ui);
ui->update();
return 0;
}
@@ -1708,9 +1737,9 @@ int Screen::handleInputEvent(const InputEvent *event)
if (NotificationRenderer::isOverlayBannerShowing()) {
NotificationRenderer::inEvent = *event;
static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
ui->setOverlays(overlays, 2);
ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
setFastFramerate(); // Draw ASAP
updateUiFrame(ui);
ui->update();
menuHandler::handleMenuSwitch(dispdev);
return 0;
+8 -3
View File
@@ -330,11 +330,15 @@ class Screen : public concurrency::OSThread
// Function to allow the AccelerometerThread to set the heading if a sensor provides it
// Mutex needed?
void setHeading(float heading);
void setHeading(long _heading)
{
hasCompass = true;
compassHeading = fmod(_heading, 360);
}
bool hasHeading() { return hasCompass; }
float getHeading() { return compassHeading; }
long getHeading() { return compassHeading; }
void setEndCalibration(uint32_t _endCalibrationAt) { endCalibrationAt = _endCalibrationAt; }
uint32_t getEndCalibration() { return endCalibrationAt; }
@@ -609,6 +613,7 @@ class Screen : public concurrency::OSThread
// Handle observer events
int handleStatusUpdate(const meshtastic::Status *arg);
int handleTextMessage(const meshtastic_MeshPacket *packet);
int handleUIFrameEvent(const UIFrameEvent *arg);
int handleInputEvent(const InputEvent *arg);
int handleAdminMessage(AdminModule_ObserverData *arg);
@@ -787,4 +792,4 @@ extern std::vector<std::string> functionSymbol;
extern std::string functionSymbolString;
extern graphics::Screen *screen;
#endif
#endif
+1 -11
View File
@@ -88,16 +88,6 @@
#endif
#endif
// nRF52 flash optimization: re-route FONT_LARGE_LOCAL to the 16pt glyph so
// the display-tier dispatch below picks up 16pt everywhere it would have used
// 24pt. Drops the ~9.6 KB ArialMT_Plain_24 table from the linked binary.
// Set MESHTASTIC_LARGE_FONT_24PT=1 in build_flags to opt out per variant
// (undefined or 0 keeps the optimization on).
#if defined(ARCH_NRF52) && (!defined(MESHTASTIC_LARGE_FONT_24PT) || MESHTASTIC_LARGE_FONT_24PT == 0)
#undef FONT_LARGE_LOCAL
#define FONT_LARGE_LOCAL FONT_MEDIUM_LOCAL
#endif
#if (defined(USE_EINK) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || defined(ST7701_CS) || defined(ST7735_CS) || \
defined(ST7789_CS) || defined(USE_ST7789) || defined(HX8357_CS) || defined(ILI9488_CS) || defined(ST7796_CS) || \
defined(USE_ST7796) || defined(HACKADAY_COMMUNICATOR)) && \
@@ -106,7 +96,7 @@
#define FONT_SMALL FONT_MEDIUM_LOCAL // Height: 19
#define FONT_MEDIUM FONT_LARGE_LOCAL // Height: 28
#define FONT_LARGE FONT_LARGE_LOCAL // Height: 28
#elif defined(OLED_TINY)
#elif defined(M5STACK_UNITC6L)
#define FONT_SMALL FONT_SMALL_LOCAL // Height: 13
#define FONT_MEDIUM FONT_SMALL_LOCAL // Height: 13
#define FONT_LARGE FONT_SMALL_LOCAL // Height: 13
+61 -267
View File
@@ -1,20 +1,16 @@
#include "configuration.h"
#if HAS_SCREEN
#include "MeshService.h"
#include "NodeDB.h"
#include "RTC.h"
#include "draw/NodeListRenderer.h"
#include "graphics/ScreenFonts.h"
#include "graphics/SharedUIDisplay.h"
#include "graphics/TFTColorRegions.h"
#include "graphics/TFTPalette.h"
#include "graphics/draw/UIRenderer.h"
#include "main.h"
#include "meshtastic/config.pb.h"
#include "modules/ExternalNotificationModule.h"
#include "power.h"
#include <OLEDDisplay.h>
#include <cctype>
#include <graphics/images.h>
namespace graphics
@@ -31,12 +27,6 @@ ScreenResolution determineScreenResolution(int16_t screenheight, int16_t screenw
return ScreenResolution::UltraLow;
}
#ifdef DISPLAY_FORCE_SMALL_FONTS
if (screenwidth <= 160 && screenheight <= 80) {
return ScreenResolution::Low;
}
#endif
// Standard OLED screens
if (screenwidth > 128 && screenheight <= 64) {
return ScreenResolution::Low;
@@ -75,12 +65,6 @@ uint32_t lastBlinkShared = 0;
bool isMailIconVisible = true;
uint32_t lastMailBlink = 0;
static inline bool useClockHeaderAccentTheme(uint32_t themeId)
{
return themeId == ThemeID::Pink || themeId == ThemeID::Creamsicle || themeId == ThemeID::MeshtasticGreen ||
themeId == ThemeID::ClassicRed || themeId == ThemeID::MonochromeWhite;
}
// *********************************
// * Rounded Header when inverted *
// *********************************
@@ -101,8 +85,7 @@ void drawRoundedHighlight(OLEDDisplay *display, int16_t x, int16_t y, int16_t w,
// *************************
// * Common Header Drawing *
// *************************
void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *titleStr, bool force_no_invert, bool show_date,
bool transparent_background, bool use_title_color_override, uint16_t title_color_override)
void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *titleStr, bool force_no_invert, bool show_date)
{
constexpr int HEADER_OFFSET_Y = 1;
y += HEADER_OFFSET_Y;
@@ -117,93 +100,30 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
const int screenW = display->getWidth();
const int screenH = display->getHeight();
const int headerHeight = highlightHeight + 2;
const uint16_t headerColorForRoles = getThemeHeaderBg();
// Color TFT headers use a fixed dark background + white glyphs.
// Keep legacy inverted bitmap behavior only for monochrome displays.
const bool useInvertedHeaderStyle = (isInverted && !force_no_invert && !isTFTColoringEnabled() && !transparent_background);
#if GRAPHICS_TFT_COLORING_ENABLED
int statusLeftEndX = 0;
int statusRightStartX = screenW;
const bool isClockHeader = transparent_background && show_date && (!titleStr || titleStr[0] == '\0');
const auto activeThemeId = getActiveTheme().id;
const bool useClockHeaderAccent = isClockHeader && useClockHeaderAccentTheme(activeThemeId);
#endif
{
const uint16_t headerColor = getThemeHeaderBg();
const uint16_t headerTextColor = getThemeHeaderText();
const uint16_t headerTitleColorForRole = use_title_color_override ? title_color_override : headerTextColor;
uint16_t headerStatusColor = getThemeHeaderStatus();
#if GRAPHICS_TFT_COLORING_ENABLED
// Clock frame uses transparent header + date + empty title.
// For accent clock themes (Pink/Creamsicle + classic monochrome), tint
// status items (battery outline, %, date, mail icon) to the header accent.
if (useClockHeaderAccent) {
headerStatusColor = getThemeHeaderBg();
}
if (transparent_background) {
// Transparent clock headers should inherit whatever body off-color is
// already active under the header (important for light/inverted themes).
const uint16_t transparentBgColor = resolveTFTOffColorAt(0, headerHeight + 1, getThemeBodyBg());
setAndRegisterTFTColorRole(TFTColorRole::HeaderBackground, transparentBgColor, transparentBgColor, 0, 0, screenW,
headerHeight);
setTFTColorRole(TFTColorRole::HeaderTitle, headerTitleColorForRole, transparentBgColor);
setTFTColorRole(TFTColorRole::HeaderStatus, headerStatusColor, transparentBgColor);
} else if (useInvertedHeaderStyle) {
setAndRegisterTFTColorRole(TFTColorRole::HeaderBackground, headerColor, TFTPalette::Black, 0, 0, screenW,
headerHeight);
setTFTColorRole(TFTColorRole::HeaderTitle, headerColor, headerTitleColorForRole);
setTFTColorRole(TFTColorRole::HeaderStatus, headerColor, headerStatusColor);
} else {
setAndRegisterTFTColorRole(TFTColorRole::HeaderBackground, TFTPalette::Black, headerColor, 0, 0, screenW,
headerHeight);
setTFTColorRole(TFTColorRole::HeaderTitle, headerTitleColorForRole, headerColor);
setTFTColorRole(TFTColorRole::HeaderStatus, headerStatusColor, headerColor);
}
#endif
if (!force_no_invert) {
// === Inverted Header Background ===
if (useInvertedHeaderStyle) {
if (isInverted) {
display->setColor(BLACK);
display->fillRect(0, 0, screenW, headerHeight);
display->fillRect(0, 0, screenW, highlightHeight + 2);
display->setColor(WHITE);
drawRoundedHighlight(display, x, y, screenW, highlightHeight, 2);
display->setColor(BLACK);
} else {
display->setColor(BLACK);
display->fillRect(0, 0, screenW, headerHeight);
// Keep the legacy white separator for monochrome displays only when header background is visible.
#if !GRAPHICS_TFT_COLORING_ENABLED
if (!transparent_background) {
display->setColor(WHITE);
if (currentResolution == ScreenResolution::High) {
display->drawLine(0, 20, screenW, 20);
} else {
display->drawLine(0, 14, screenW, 14);
}
}
#endif
}
if (transparent_background) {
display->fillRect(0, 0, screenW, highlightHeight + 2);
display->setColor(WHITE);
if (currentResolution == ScreenResolution::High) {
display->drawLine(0, 20, screenW, 20);
} else {
display->drawLine(0, 14, screenW, 14);
}
}
#if GRAPHICS_TFT_COLORING_ENABLED
// TFT role coloring expects foreground glyph bits to be "set".
display->setColor(WHITE);
#endif
// === Screen Title ===
const char *headerTitle = titleStr ? titleStr : "";
const int titleWidth = UIRenderer::measureStringWithEmotes(display, headerTitle);
const int titleX = (SCREEN_WIDTH - titleWidth) / 2;
#if GRAPHICS_TFT_COLORING_ENABLED
const int titleRegionWidth = titleWidth + (config.display.heading_bold ? 3 : 2);
registerTFTColorRegion(TFTColorRole::HeaderTitle, titleX - 1, y, titleRegionWidth, FONT_HEIGHT_SMALL);
#endif
UIRenderer::drawStringWithEmotes(display, titleX, y, headerTitle, FONT_HEIGHT_SMALL, 1, config.display.heading_bold);
}
display->setTextAlignment(TEXT_ALIGN_LEFT);
@@ -232,21 +152,10 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
bool useHorizontalBattery = (currentResolution == ScreenResolution::High && screenW >= screenH);
const int textY = y + (highlightHeight - FONT_HEIGHT_SMALL) / 2;
bool hasBatteryFillRegion = false;
int16_t batteryFillRegionX = 0;
int16_t batteryFillRegionY = 0;
int16_t batteryFillRegionW = 0;
int16_t batteryFillRegionH = 0;
#if GRAPHICS_TFT_COLORING_ENABLED
uint16_t batteryFillColor = getThemeBatteryFillColor(chargePercent);
if (useClockHeaderAccent) {
batteryFillColor = getThemeHeaderBg();
}
#endif
int batteryX = 1;
int batteryY = HEADER_OFFSET_Y + 1;
#if !defined(OLED_TINY)
#if !defined(M5STACK_UNITC6L)
// === Battery Icons ===
if (usbPowered && !isCharging) { // This is a basic check to determine USB Powered is flagged but not charging
batteryX += 1;
@@ -271,15 +180,6 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
display->drawLine(batteryX + 5, batteryY + 12, batteryX + 10, batteryY + 12);
int fillWidth = 14 * chargePercent / 100;
display->fillRect(batteryX + 1, batteryY + 1, fillWidth, 11);
#if GRAPHICS_TFT_COLORING_ENABLED
if (fillWidth > 0) {
hasBatteryFillRegion = true;
batteryFillRegionX = batteryX + 1;
batteryFillRegionY = batteryY + 1;
batteryFillRegionW = fillWidth;
batteryFillRegionH = 11;
}
#endif
}
batteryX += 18; // Icon + 2 pixels
} else {
@@ -294,41 +194,21 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
int fillHeight = 8 * chargePercent / 100;
int fillY = batteryY - fillHeight;
display->fillRect(batteryX + 1, fillY + 10, 5, fillHeight);
#if GRAPHICS_TFT_COLORING_ENABLED
if (fillHeight > 0) {
hasBatteryFillRegion = true;
batteryFillRegionX = batteryX + 1;
batteryFillRegionY = fillY + 10;
batteryFillRegionW = 5;
batteryFillRegionH = fillHeight;
}
#endif
}
batteryX += 9; // Icon + 2 pixels
}
}
#if GRAPHICS_TFT_COLORING_ENABLED
statusLeftEndX = batteryX + 2;
#endif
if (chargePercent != 101) {
// === Battery % Display ===
char chargeStr[4];
snprintf(chargeStr, sizeof(chargeStr), "%d", chargePercent);
int chargeNumWidth = display->getStringWidth(chargeStr);
const int percentWidth = display->getStringWidth("%");
const int percentX = batteryX + chargeNumWidth - 1;
display->drawString(batteryX, textY, chargeStr);
display->drawString(percentX, textY, "%");
#if GRAPHICS_TFT_COLORING_ENABLED
statusLeftEndX = percentX + percentWidth + 2;
#endif
display->drawString(batteryX + chargeNumWidth - 1, textY, "%");
if (isBold) {
display->drawString(batteryX + 1, textY, chargeStr);
display->drawString(percentX + 1, textY, "%");
#if GRAPHICS_TFT_COLORING_ENABLED
statusLeftEndX = percentX + percentWidth + 3;
#endif
display->drawString(batteryX + chargeNumWidth, textY, "%");
}
}
@@ -373,9 +253,6 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
timeStrWidth = display->getStringWidth(timeStr);
}
timeX = screenW - xOffset - timeStrWidth + 3;
#if GRAPHICS_TFT_COLORING_ENABLED
statusRightStartX = timeX - (useHorizontalBattery ? 22 : 16);
#endif
// === Show Mail or Mute Icon to the Left of Time ===
int iconRightEdge = timeX - 2;
@@ -401,7 +278,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
int iconW = 16, iconH = 12;
int iconX = iconRightEdge - iconW;
int iconY = textY + (FONT_HEIGHT_SMALL - iconH) / 2 - 1;
if (useInvertedHeaderStyle) {
if (isInverted && !force_no_invert) {
display->setColor(WHITE);
display->fillRect(iconX - 1, iconY - 1, iconW + 3, iconH + 2);
display->setColor(BLACK);
@@ -416,7 +293,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
} else {
int iconX = iconRightEdge - (mail_width - 2);
int iconY = textY + (FONT_HEIGHT_SMALL - mail_height) / 2;
if (useInvertedHeaderStyle) {
if (isInverted && !force_no_invert) {
display->setColor(WHITE);
display->fillRect(iconX - 1, iconY - 1, mail_width + 2, mail_height + 2);
display->setColor(BLACK);
@@ -432,7 +309,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
int iconX = iconRightEdge - mute_symbol_big_width;
int iconY = textY + (FONT_HEIGHT_SMALL - mute_symbol_big_height) / 2;
if (useInvertedHeaderStyle) {
if (isInverted && !force_no_invert) {
display->setColor(WHITE);
display->fillRect(iconX - 1, iconY - 1, mute_symbol_big_width + 2, mute_symbol_big_height + 2);
display->setColor(BLACK);
@@ -446,7 +323,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
int iconX = iconRightEdge - mute_symbol_width;
int iconY = textY + (FONT_HEIGHT_SMALL - mail_height) / 2;
if (useInvertedHeaderStyle) {
if (isInverted && !force_no_invert) {
display->setColor(WHITE);
display->fillRect(iconX - 1, iconY - 1, mute_symbol_width + 2, mute_symbol_height + 2);
display->setColor(BLACK);
@@ -474,9 +351,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
} else {
// === No Time Available: Mail/Mute Icon Moves to Far Right ===
int iconRightEdge = screenW - xOffset;
#if GRAPHICS_TFT_COLORING_ENABLED
statusRightStartX = screenW - (useHorizontalBattery ? 22 : 12);
#endif
bool showMail = false;
#ifndef USE_EINK
@@ -518,16 +393,6 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
}
}
}
#endif
#if GRAPHICS_TFT_COLORING_ENABLED
registerTFTColorRegion(TFTColorRole::HeaderStatus, 0, 0, statusLeftEndX, headerHeight);
if (statusRightStartX < screenW) {
registerTFTColorRegion(TFTColorRole::HeaderStatus, statusRightStartX, 0, screenW - statusRightStartX, headerHeight);
}
if (hasBatteryFillRegion) {
registerTFTColorRegionDirect(batteryFillRegionX, batteryFillRegionY, batteryFillRegionW, batteryFillRegionH,
batteryFillColor, headerColorForRoles);
}
#endif
display->setColor(WHITE); // Reset for other UI
}
@@ -565,23 +430,14 @@ void drawCommonFooter(OLEDDisplay *display, int16_t x, int16_t y)
return;
const int scale = (currentResolution == ScreenResolution::High) ? 2 : 1;
const int footerY = SCREEN_HEIGHT - (1 * scale) - (connection_icon_height * scale);
const int footerH = (connection_icon_height * scale) + (2 * scale);
const int iconX = 0;
const int iconY = SCREEN_HEIGHT - (connection_icon_height * scale);
const int iconW = connection_icon_width * scale;
const int iconH = connection_icon_height * scale;
#if GRAPHICS_TFT_COLORING_ENABLED
// Only tint the link glyph itself on TFT; keep the footer background black.
setAndRegisterTFTColorRole(TFTColorRole::ConnectionIcon, TFTPalette::Blue, TFTPalette::Black, iconX, iconY, iconW, iconH);
#endif
display->setColor(BLACK);
display->fillRect(0, footerY, SCREEN_WIDTH, footerH);
display->fillRect(0, SCREEN_HEIGHT - (1 * scale) - (connection_icon_height * scale), (connection_icon_width * scale),
(connection_icon_height * scale) + (2 * scale));
display->setColor(WHITE);
if (currentResolution == ScreenResolution::High) {
const int bytesPerRow = (connection_icon_width + 7) / 8;
int iconX = 0;
int iconY = SCREEN_HEIGHT - (connection_icon_height * 2);
for (int yy = 0; yy < connection_icon_height; ++yy) {
const uint8_t *rowPtr = connection_icon + yy * bytesPerRow;
@@ -595,127 +451,65 @@ void drawCommonFooter(OLEDDisplay *display, int16_t x, int16_t y)
}
} else {
display->drawXbm(iconX, iconY, connection_icon_width, connection_icon_height, connection_icon);
display->drawXbm(0, SCREEN_HEIGHT - connection_icon_height, connection_icon_width, connection_icon_height,
connection_icon);
}
}
bool isAllowedPunctuation(char c)
{
switch (c) {
case '.':
case ',':
case '!':
case '?':
case ';':
case ':':
case '-':
case '_':
case '(':
case ')':
case '[':
case ']':
case '{':
case '}':
case '\'':
case '"':
case '@':
case '#':
case '$':
case '/':
case '\\':
case '&':
case '+':
case '=':
case '%':
case '~':
case '^':
case ' ':
return true;
default:
return false;
}
const std::string allowed = ".,!?;:-_()[]{}'\"@#$/\\&+=%~^ ";
return allowed.find(c) != std::string::npos;
}
static inline size_t utf8CodePointLength(unsigned char lead)
static void replaceAll(std::string &s, const std::string &from, const std::string &to)
{
if ((lead & 0x80) == 0x00) {
return 1;
if (from.empty())
return;
size_t pos = 0;
while ((pos = s.find(from, pos)) != std::string::npos) {
s.replace(pos, from.size(), to);
pos += to.size();
}
if ((lead & 0xE0) == 0xC0) {
return 2;
}
if ((lead & 0xF0) == 0xE0) {
return 3;
}
if ((lead & 0xF8) == 0xF0) {
return 4;
}
return 1;
}
std::string sanitizeString(const std::string &input)
{
static constexpr char kReplacementChar = static_cast<char>(0xBF); // Inverted question mark in ISO-8859-1.
std::string output;
output.reserve(input.size());
bool inReplacement = false;
const size_t inputSize = input.size();
size_t i = 0;
while (i < inputSize) {
const unsigned char byte0 = static_cast<unsigned char>(input[i]);
char normalized = '\0';
size_t consumed = 0;
if (byte0 < 0x80) {
normalized = static_cast<char>(byte0);
consumed = 1;
} else if ((i + 2) < inputSize && byte0 == 0xE2 && static_cast<unsigned char>(input[i + 1]) == 0x80) {
// Smart punctuation: ' ' \" \" - -
switch (static_cast<unsigned char>(input[i + 2])) {
case 0x98:
case 0x99:
normalized = '\'';
consumed = 3;
break;
case 0x9C:
case 0x9D:
normalized = '\"';
consumed = 3;
break;
case 0x93:
case 0x94:
normalized = '-';
consumed = 3;
break;
default:
break;
}
} else if ((i + 1) < inputSize && byte0 == 0xC2 && static_cast<unsigned char>(input[i + 1]) == 0xA0) {
// Non-breaking space.
normalized = ' ';
consumed = 2;
}
if (consumed == 0) {
size_t seqLen = utf8CodePointLength(byte0);
if (seqLen > (inputSize - i)) {
seqLen = 1;
}
// Make a mutable copy so we can normalize UTF-8 “smart punctuation” into ASCII first.
std::string s = input;
// Curly single quotes: ‘ ’
replaceAll(s, "\xE2\x80\x98", "'"); // U+2018
replaceAll(s, "\xE2\x80\x99", "'"); // U+2019
// Curly double quotes: “ ”
replaceAll(s, "\xE2\x80\x9C", "\""); // U+201C
replaceAll(s, "\xE2\x80\x9D", "\""); // U+201D
// En dash / Em dash: – —
replaceAll(s, "\xE2\x80\x93", "-"); // U+2013
replaceAll(s, "\xE2\x80\x94", "-"); // U+2014
// Non-breaking space
replaceAll(s, "\xC2\xA0", " "); // U+00A0
// Now do your original sanitize pass over the normalized string.
for (unsigned char uc : s) {
char c = static_cast<char>(uc);
if (std::isalnum(uc) || isAllowedPunctuation(c)) {
output += c;
inReplacement = false;
} else {
if (!inReplacement) {
output.push_back(kReplacementChar);
output += static_cast<char>(0xBF); // ISO-8859-1 for inverted question mark
inReplacement = true;
}
i += seqLen;
continue;
}
const unsigned char normalizedUc = static_cast<unsigned char>(normalized);
if (std::isalnum(normalizedUc) || isAllowedPunctuation(normalized)) {
output.push_back(normalized);
inReplacement = false;
} else if (!inReplacement) {
output.push_back(kReplacementChar);
inReplacement = true;
}
i += consumed;
}
return output;
}
+1 -3
View File
@@ -1,7 +1,6 @@
#pragma once
#include <OLEDDisplay.h>
#include <stdint.h>
#include <string>
namespace graphics
@@ -53,8 +52,7 @@ void drawRoundedHighlight(OLEDDisplay *display, int16_t x, int16_t y, int16_t w,
// Shared battery/time/mail header
void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *titleStr = "", bool force_no_invert = false,
bool show_date = false, bool transparent_background = false, bool use_title_color_override = false,
uint16_t title_color_override = 0);
bool show_date = false);
// Shared battery/time/mail header
void drawCommonFooter(OLEDDisplay *display, int16_t x, int16_t y);
-819
View File
@@ -1,819 +0,0 @@
#include "TFTColorRegions.h"
#include "NodeDB.h"
#include "TFTPalette.h"
#include <string.h>
namespace graphics
{
TFTColorRegion colorRegions[MAX_TFT_COLOR_REGIONS];
namespace
{
struct TFTRoleColorsBe {
uint16_t onColorBe;
uint16_t offColorBe;
};
static uint8_t colorRegionCount = 0;
static constexpr uint32_t kFnv1aOffsetBasis = 2166136261u;
static constexpr uint32_t kFnv1aPrime = 16777619u;
static constexpr uint16_t toBe565(uint16_t color)
{
return static_cast<uint16_t>((color >> 8) | (color << 8));
}
static constexpr bool kRoleIsBody[static_cast<size_t>(TFTColorRole::Count)] = {
false, // HeaderBackground
false, // HeaderTitle
false, // HeaderStatus
true, // SignalBars
true, // ConnectionIcon
true, // UtilizationFill
true, // FavoriteNode
true, // ActionMenuBorder
true, // ActionMenuBody
true, // ActionMenuTitle
true, // FrameMono
false, // BootSplash
true, // FavoriteNodeBGHighlight
false, // NavigationBar
false // NavigationArrow
};
static inline bool isBodyColorRole(TFTColorRole role)
{
return kRoleIsBody[static_cast<size_t>(role)];
}
static inline bool isMonochromeTheme(uint32_t themeId)
{
return themeId == ThemeID::MeshtasticGreen || themeId == ThemeID::ClassicRed || themeId == ThemeID::MonochromeWhite;
}
static inline uint16_t getMonochromeAccent(uint32_t themeId)
{
return (themeId == ThemeID::MeshtasticGreen) ? TFTPalette::MeshtasticGreen
: (themeId == ThemeID::ClassicRed) ? TFTPalette::ClassicRed
: TFTPalette::White;
}
static inline void replaceColor(uint16_t &value, uint16_t from, uint16_t to)
{
if (value == from) {
value = to;
}
}
static inline uint32_t fnv1aAppendByte(uint32_t hash, uint8_t value)
{
return (hash ^ value) * kFnv1aPrime;
}
static inline uint32_t fnv1aAppendU16(uint32_t hash, uint16_t value)
{
hash = fnv1aAppendByte(hash, static_cast<uint8_t>(value & 0xFF));
hash = fnv1aAppendByte(hash, static_cast<uint8_t>((value >> 8) & 0xFF));
return hash;
}
// Compile-time header color overrides (backward-compatible)
#ifdef TFT_HEADER_BG_COLOR_OVERRIDE
static constexpr uint16_t kHeaderBackground = TFT_HEADER_BG_COLOR_OVERRIDE;
#else
static constexpr uint16_t kHeaderBackground = TFTPalette::DarkGray;
#endif
#ifdef TFT_HEADER_TITLE_COLOR_OVERRIDE
static constexpr uint16_t kTitleColor = TFT_HEADER_TITLE_COLOR_OVERRIDE;
#else
static constexpr uint16_t kTitleColor = TFTPalette::White;
#endif
#ifdef TFT_HEADER_STATUS_COLOR_OVERRIDE
static constexpr uint16_t kStatusColor = TFT_HEADER_STATUS_COLOR_OVERRIDE;
#else
static constexpr uint16_t kStatusColor = TFTPalette::White;
#endif
// Theme definitions
// Stored in kThemes[] and looked up by matching uiconfig.screen_rgb_color
// against each entry's .uniqueIdentifier field.
static const TFTThemeDef kThemes[] = {
// Default Dark (ThemeID::DefaultDark = 0)
{
ThemeID::DefaultDark, // id
"Default Dark", // name
0, // uniqueIdentifier
// roles[TFTColorRole::Count]
{
{kHeaderBackground, TFTPalette::Black}, // HeaderBackground
{kHeaderBackground, kTitleColor}, // HeaderTitle
{kHeaderBackground, kStatusColor}, // HeaderStatus
{TFTPalette::Good, TFTPalette::Black}, // SignalBars
{TFTPalette::Blue, TFTPalette::Black}, // ConnectionIcon
{TFTPalette::Good, TFTPalette::Black}, // UtilizationFill
{TFTPalette::Yellow, TFTPalette::Black}, // FavoriteNode
{TFTPalette::DarkGray, TFTPalette::Black}, // ActionMenuBorder
{TFTPalette::White, TFTPalette::Black}, // ActionMenuBody
{TFTPalette::DarkGray, TFTPalette::White}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::Black}, // BootSplash
{TFTPalette::Yellow, TFTPalette::Black}, // FavoriteNodeBGHighlight
{kStatusColor, kHeaderBackground}, // NavigationBar (icon fg, bar bg)
{kTitleColor, TFTPalette::Black}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::Good, // batteryFillGood
TFTPalette::Medium, // batteryFillMedium
TFTPalette::Bad, // batteryFillBad
false, // fullFrameInvert
true, // visible
},
// Default Light (ThemeID::DefaultLight = 1)
{
ThemeID::DefaultLight, // id
"Default Light", // name
1, // uniqueIdentifier
{
{TFTPalette::LightGray, TFTPalette::Black}, // HeaderBackground
{TFTPalette::LightGray, TFTPalette::Black}, // HeaderTitle
{TFTPalette::LightGray, TFTPalette::Black}, // HeaderStatus
{TFTPalette::Good, TFTPalette::White}, // SignalBars
{TFTPalette::Blue, TFTPalette::White}, // ConnectionIcon
{TFTPalette::Good, TFTPalette::White}, // UtilizationFill
{TFTPalette::Black, TFTPalette::Yellow}, // FavoriteNode
{TFTPalette::DarkGray, TFTPalette::White}, // ActionMenuBorder
{TFTPalette::Black, TFTPalette::White}, // ActionMenuBody
{TFTPalette::DarkGray, TFTPalette::Black}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::Black}, // BootSplash
{TFTPalette::Black, TFTPalette::Yellow}, // FavoriteNodeBGHighlight
{TFTPalette::Black, TFTPalette::LightGray}, // NavigationBar (icon fg, bar bg)
{TFTPalette::Black, TFTPalette::White}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::Good, // batteryFillGood
TFTPalette::Medium, // batteryFillMedium
TFTPalette::Bad, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Christmas (ThemeID::Christmas = 2)
{
ThemeID::Christmas, // id
"Christmas", // name
2, // uniqueIdentifier
{
{TFTPalette::ChristmasRed, TFTPalette::Black}, // HeaderBackground
{TFTPalette::ChristmasRed, TFTPalette::Gold}, // HeaderTitle
{TFTPalette::ChristmasRed, TFTPalette::Gold}, // HeaderStatus
{TFTPalette::ChristmasGreen, TFTPalette::Pine}, // SignalBars
{TFTPalette::Gold, TFTPalette::Pine}, // ConnectionIcon
{TFTPalette::ChristmasGreen, TFTPalette::Pine}, // UtilizationFill
{TFTPalette::Gold, TFTPalette::Pine}, // FavoriteNode
{TFTPalette::ChristmasRed, TFTPalette::Pine}, // ActionMenuBorder
{TFTPalette::White, TFTPalette::Pine}, // ActionMenuBody
{TFTPalette::ChristmasRed, TFTPalette::White}, // ActionMenuTitle
{TFTPalette::Pine, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::ChristmasRed}, // BootSplash
{TFTPalette::Gold, TFTPalette::Pine}, // FavoriteNodeBGHighlight
{TFTPalette::Gold, TFTPalette::ChristmasRed}, // NavigationBar (icon fg, bar bg)
{TFTPalette::Gold, TFTPalette::Pine}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::ChristmasGreen, // batteryFillGood
TFTPalette::Gold, // batteryFillMedium
TFTPalette::ChristmasRed, // batteryFillBad
true, // fullFrameInvert
false, // visible
},
// Pink (ThemeID::Pink = 3) light variant
{
ThemeID::Pink, // id
"Pink", // name
3, // uniqueIdentifier
{
{TFTPalette::HotPink, TFTPalette::Black}, // HeaderBackground
{TFTPalette::HotPink, TFTPalette::White}, // HeaderTitle
{TFTPalette::HotPink, TFTPalette::White}, // HeaderStatus
{TFTPalette::DeepPink, TFTPalette::PalePink}, // SignalBars
{TFTPalette::HotPink, TFTPalette::PalePink}, // ConnectionIcon
{TFTPalette::DeepPink, TFTPalette::PalePink}, // UtilizationFill
{TFTPalette::Black, TFTPalette::HotPink}, // FavoriteNode
{TFTPalette::HotPink, TFTPalette::PalePink}, // ActionMenuBorder
{TFTPalette::Black, TFTPalette::PalePink}, // ActionMenuBody
{TFTPalette::HotPink, TFTPalette::White}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::HotPink}, // BootSplash
{TFTPalette::Black, TFTPalette::HotPink}, // FavoriteNodeBGHighlight
{TFTPalette::White, TFTPalette::HotPink}, // NavigationBar (icon fg, bar bg)
{TFTPalette::HotPink, TFTPalette::PalePink}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::DeepPink, // batteryFillGood
TFTPalette::HotPink, // batteryFillMedium
TFTPalette::Bad, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Blue (ThemeID::Blue = 4) dark variant
{
ThemeID::Blue, // id
"Blue", // name
4, // uniqueIdentifier
{
{TFTPalette::DeepBlue, TFTPalette::Black}, // HeaderBackground
{TFTPalette::DeepBlue, TFTPalette::White}, // HeaderTitle
{TFTPalette::DeepBlue, TFTPalette::SkyBlue}, // HeaderStatus
{TFTPalette::SkyBlue, TFTPalette::Navy}, // SignalBars
{TFTPalette::SkyBlue, TFTPalette::Navy}, // ConnectionIcon
{TFTPalette::SkyBlue, TFTPalette::Navy}, // UtilizationFill
{TFTPalette::SkyBlue, TFTPalette::Navy}, // FavoriteNode
{TFTPalette::DeepBlue, TFTPalette::Navy}, // ActionMenuBorder
{TFTPalette::White, TFTPalette::Navy}, // ActionMenuBody
{TFTPalette::DeepBlue, TFTPalette::White}, // ActionMenuTitle
{TFTPalette::Navy, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::DeepBlue}, // BootSplash
{TFTPalette::SkyBlue, TFTPalette::Navy}, // FavoriteNodeBGHighlight
{TFTPalette::SkyBlue, TFTPalette::DeepBlue}, // NavigationBar (icon fg, bar bg)
{TFTPalette::SkyBlue, TFTPalette::Black}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::SkyBlue, // batteryFillGood
TFTPalette::Medium, // batteryFillMedium
TFTPalette::Bad, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Creamsicle (ThemeID::Creamsicle = 5)light variant
{
ThemeID::Creamsicle, // id
"Creamsicle", // name
5, // uniqueIdentifier
{
{TFTPalette::CreamOrange, TFTPalette::Black}, // HeaderBackground
{TFTPalette::CreamOrange, TFTPalette::White}, // HeaderTitle
{TFTPalette::CreamOrange, TFTPalette::White}, // HeaderStatus
{TFTPalette::DeepOrange, TFTPalette::Cream}, // SignalBars
{TFTPalette::CreamOrange, TFTPalette::Cream}, // ConnectionIcon
{TFTPalette::DeepOrange, TFTPalette::Cream}, // UtilizationFill
{TFTPalette::Black, TFTPalette::CreamOrange}, // FavoriteNode
{TFTPalette::CreamOrange, TFTPalette::Cream}, // ActionMenuBorder
{TFTPalette::Black, TFTPalette::Cream}, // ActionMenuBody
{TFTPalette::CreamOrange, TFTPalette::White}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::White}, // FrameMono
{TFTPalette::White, TFTPalette::CreamOrange}, // BootSplash
{TFTPalette::Black, TFTPalette::CreamOrange}, // FavoriteNodeBGHighlight
{TFTPalette::White, TFTPalette::CreamOrange}, // NavigationBar (icon fg, bar bg)
{TFTPalette::CreamOrange, TFTPalette::White}, // NavigationArrow (arrow fg, body bg)
},
TFTPalette::DeepOrange, // batteryFillGood
TFTPalette::Gold, // batteryFillMedium
TFTPalette::Bad, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Meshtastic Green (ThemeID::MeshtasticGreen = 6) classic monochrome
// Pure single-color-on-black look. Every role maps foreground pixels to
// the theme color and background pixels to Black.
{
ThemeID::MeshtasticGreen, // id
"Meshtastic Green", // name
6, // uniqueIdentifier
{
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // HeaderBackground
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // HeaderTitle
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // HeaderStatus
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // SignalBars
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // ConnectionIcon
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // UtilizationFill
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // FavoriteNode
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // ActionMenuBorder
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // ActionMenuBody
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::MeshtasticGreen}, // FrameMono (bodyBg, bodyFg)
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // BootSplash
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // FavoriteNodeBGHighlight
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // NavigationBar
{TFTPalette::MeshtasticGreen, TFTPalette::Black}, // NavigationArrow
},
TFTPalette::Black, // batteryFillGood
TFTPalette::Black, // batteryFillMedium
TFTPalette::Black, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Classic Red (ThemeID::ClassicRed = 7) classic monochrome
{
ThemeID::ClassicRed, // id
"Classic Red", // name
7, // uniqueIdentifier
{
{TFTPalette::ClassicRed, TFTPalette::Black}, // HeaderBackground
{TFTPalette::ClassicRed, TFTPalette::Black}, // HeaderTitle
{TFTPalette::ClassicRed, TFTPalette::Black}, // HeaderStatus
{TFTPalette::ClassicRed, TFTPalette::Black}, // SignalBars
{TFTPalette::ClassicRed, TFTPalette::Black}, // ConnectionIcon
{TFTPalette::ClassicRed, TFTPalette::Black}, // UtilizationFill
{TFTPalette::ClassicRed, TFTPalette::Black}, // FavoriteNode
{TFTPalette::ClassicRed, TFTPalette::Black}, // ActionMenuBorder
{TFTPalette::ClassicRed, TFTPalette::Black}, // ActionMenuBody
{TFTPalette::ClassicRed, TFTPalette::Black}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::ClassicRed}, // FrameMono (bodyBg, bodyFg)
{TFTPalette::ClassicRed, TFTPalette::Black}, // BootSplash
{TFTPalette::ClassicRed, TFTPalette::Black}, // FavoriteNodeBGHighlight
{TFTPalette::ClassicRed, TFTPalette::Black}, // NavigationBar
{TFTPalette::ClassicRed, TFTPalette::Black}, // NavigationArrow
},
TFTPalette::Black, // batteryFillGood
TFTPalette::Black, // batteryFillMedium
TFTPalette::Black, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
// Monochrome White (ThemeID::MonochromeWhite = 8) classic monochrome
{
ThemeID::MonochromeWhite, // id
"Monochrome White", // name
8, // uniqueIdentifier
{
{TFTPalette::White, TFTPalette::Black}, // HeaderBackground
{TFTPalette::White, TFTPalette::Black}, // HeaderTitle
{TFTPalette::White, TFTPalette::Black}, // HeaderStatus
{TFTPalette::White, TFTPalette::Black}, // SignalBars
{TFTPalette::White, TFTPalette::Black}, // ConnectionIcon
{TFTPalette::White, TFTPalette::Black}, // UtilizationFill
{TFTPalette::White, TFTPalette::Black}, // FavoriteNode
{TFTPalette::White, TFTPalette::Black}, // ActionMenuBorder
{TFTPalette::White, TFTPalette::Black}, // ActionMenuBody
{TFTPalette::White, TFTPalette::Black}, // ActionMenuTitle
{TFTPalette::Black, TFTPalette::White}, // FrameMono (bodyBg, bodyFg)
{TFTPalette::White, TFTPalette::Black}, // BootSplash
{TFTPalette::White, TFTPalette::Black}, // FavoriteNodeBGHighlight
{TFTPalette::White, TFTPalette::Black}, // NavigationBar
{TFTPalette::White, TFTPalette::Black}, // NavigationArrow
},
TFTPalette::Black, // batteryFillGood
TFTPalette::Black, // batteryFillMedium
TFTPalette::Black, // batteryFillBad
true, // fullFrameInvert
true, // visible
},
};
static constexpr size_t kInternalThemeCount = sizeof(kThemes) / sizeof(kThemes[0]);
// Resolve the kThemes[] index for the currently persisted theme. Called at
// boot (indirectly via getActiveTheme()) and whenever the active theme is
// queried, so uiconfig.screen_rgb_color remains the single source of truth.
// Matches against .uniqueIdentifier - that's the field whose value is stored
// in the user's config. Falls back to 0 (DefaultDark) if no match is found,
// which gracefully handles removed or retired themes.
static inline size_t resolveThemeIndex()
{
const uint32_t savedIdentifier = uiconfig.screen_rgb_color & 0x1F;
for (size_t i = 0; i < kInternalThemeCount; i++) {
if (kThemes[i].uniqueIdentifier == savedIdentifier)
return i;
}
return 0; // Default Dark fallback
}
static inline bool normalizeRegion(int16_t &x, int16_t &y, int16_t &width, int16_t &height)
{
if (width <= 0 || height <= 0) {
return false;
}
if (x < 0) {
width += x;
x = 0;
}
if (y < 0) {
height += y;
y = 0;
}
return width > 0 && height > 0;
}
static inline void appendColorRegion(int16_t x, int16_t y, int16_t width, int16_t height, uint16_t onColorBe, uint16_t offColorBe)
{
// Keep the last slot permanently disabled as a sentinel for ST7789 scans.
// This leaves MAX_TFT_COLOR_REGIONS - 1 usable entries.
if (colorRegionCount >= MAX_TFT_COLOR_REGIONS - 1) {
memmove(&colorRegions[0], &colorRegions[1], sizeof(TFTColorRegion) * (MAX_TFT_COLOR_REGIONS - 2));
colorRegionCount = MAX_TFT_COLOR_REGIONS - 2;
}
TFTColorRegion &region = colorRegions[colorRegionCount++];
region.x = x;
region.y = y;
region.width = width;
region.height = height;
region.onColorBe = onColorBe;
region.offColorBe = offColorBe;
region.enabled = true;
// Keep one disabled sentinel after the active range for ST7789 countColorRegions().
if (colorRegionCount < MAX_TFT_COLOR_REGIONS) {
colorRegions[colorRegionCount].enabled = false;
}
colorRegions[MAX_TFT_COLOR_REGIONS - 1].enabled = false;
}
// Current working role colors (big-endian). Initialised to Dark defaults;
// call loadThemeDefaults() after boot / theme change to refresh.
static TFTRoleColorsBe roleColors[static_cast<size_t>(TFTColorRole::Count)] = {
{toBe565(kHeaderBackground), toBe565(TFTPalette::Black)}, // HeaderBackground
{toBe565(kHeaderBackground), toBe565(kTitleColor)}, // HeaderTitle
{toBe565(kHeaderBackground), toBe565(kStatusColor)}, // HeaderStatus
{toBe565(TFTPalette::Good), toBe565(TFTPalette::Black)}, // SignalBars
{toBe565(TFTPalette::Blue), toBe565(TFTPalette::Black)}, // ConnectionIcon
{toBe565(TFTPalette::Good), toBe565(TFTPalette::Black)}, // UtilizationFill
{toBe565(TFTPalette::Yellow), toBe565(TFTPalette::Black)}, // FavoriteNode
{toBe565(TFTPalette::DarkGray), toBe565(TFTPalette::Black)}, // ActionMenuBorder
{toBe565(TFTPalette::White), toBe565(TFTPalette::Black)}, // ActionMenuBody
{toBe565(TFTPalette::DarkGray), toBe565(TFTPalette::White)}, // ActionMenuTitle
{toBe565(TFTPalette::Black), toBe565(TFTPalette::White)}, // FrameMono
{toBe565(TFTPalette::White), toBe565(TFTPalette::Black)}, // BootSplash
{toBe565(TFTPalette::Yellow), toBe565(TFTPalette::Black)}, // FavoriteNodeBGHighlight
{toBe565(kStatusColor), toBe565(kHeaderBackground)}, // NavigationBar
{toBe565(kTitleColor), toBe565(TFTPalette::Black)} // NavigationArrow
};
} // namespace
// Theme accessors
const TFTThemeDef &getActiveTheme()
{
return kThemes[resolveThemeIndex()];
}
// Visible-theme accessors
// These iterate only themes flagged .visible = true, preserving kThemes[]
// order. Menu code should use these so hidden themes don't appear in the
// picker while still applying correctly if their ID is persisted.
size_t getVisibleThemeCount()
{
size_t count = 0;
for (size_t i = 0; i < kInternalThemeCount; i++) {
if (kThemes[i].visible)
count++;
}
return count;
}
const TFTThemeDef &getVisibleThemeByIndex(size_t visibleIndex)
{
size_t seen = 0;
for (size_t i = 0; i < kInternalThemeCount; i++) {
if (!kThemes[i].visible)
continue;
if (seen == visibleIndex)
return kThemes[i];
seen++;
}
// Fallback: return first theme (never trust a bad index).
return kThemes[0];
}
size_t getActiveVisibleThemeIndex()
{
const size_t active = resolveThemeIndex();
if (!kThemes[active].visible)
return SIZE_MAX;
size_t visibleIdx = 0;
for (size_t i = 0; i < active; i++) {
if (kThemes[i].visible)
visibleIdx++;
}
return visibleIdx;
}
uint16_t getThemeHeaderBg()
{
#if GRAPHICS_TFT_COLORING_ENABLED
#ifdef TFT_HEADER_BG_COLOR_OVERRIDE
return TFT_HEADER_BG_COLOR_OVERRIDE;
#else
return kThemes[resolveThemeIndex()].roles[static_cast<size_t>(TFTColorRole::HeaderBackground)].onColor;
#endif
#else
return TFTPalette::DarkGray;
#endif
}
uint16_t getThemeHeaderText()
{
#if GRAPHICS_TFT_COLORING_ENABLED
#ifdef TFT_HEADER_TITLE_COLOR_OVERRIDE
return TFT_HEADER_TITLE_COLOR_OVERRIDE;
#else
return kThemes[resolveThemeIndex()].roles[static_cast<size_t>(TFTColorRole::HeaderTitle)].offColor;
#endif
#else
return TFTPalette::White;
#endif
}
uint16_t getThemeHeaderStatus()
{
#if GRAPHICS_TFT_COLORING_ENABLED
#ifdef TFT_HEADER_STATUS_COLOR_OVERRIDE
return TFT_HEADER_STATUS_COLOR_OVERRIDE;
#else
return kThemes[resolveThemeIndex()].roles[static_cast<size_t>(TFTColorRole::HeaderStatus)].offColor;
#endif
#else
return TFTPalette::White;
#endif
}
uint16_t getThemeBodyBg()
{
#if GRAPHICS_TFT_COLORING_ENABLED
return kThemes[resolveThemeIndex()].roles[static_cast<size_t>(TFTColorRole::FrameMono)].onColor;
#else
return TFTPalette::Black;
#endif
}
uint16_t getThemeBodyFg()
{
#if GRAPHICS_TFT_COLORING_ENABLED
return kThemes[resolveThemeIndex()].roles[static_cast<size_t>(TFTColorRole::FrameMono)].offColor;
#else
return TFTPalette::White;
#endif
}
bool isThemeFullFrameInvert()
{
#if GRAPHICS_TFT_COLORING_ENABLED
return kThemes[resolveThemeIndex()].fullFrameInvert;
#else
return false;
#endif
}
uint16_t getThemeBatteryFillColor(int batteryPercent)
{
const TFTThemeDef &theme = kThemes[resolveThemeIndex()];
if (batteryPercent <= 20) {
return theme.batteryFillBad;
}
if (batteryPercent <= 50) {
return theme.batteryFillMedium;
}
return theme.batteryFillGood;
}
void loadThemeDefaults()
{
#if GRAPHICS_TFT_COLORING_ENABLED
const TFTThemeDef &theme = kThemes[resolveThemeIndex()];
for (uint8_t i = 0; i < static_cast<uint8_t>(TFTColorRole::Count); i++) {
roleColors[i].onColorBe = toBe565(theme.roles[i].onColor);
roleColors[i].offColorBe = toBe565(theme.roles[i].offColor);
}
#endif
}
// Role color assignment with theme-aware transforms
void setTFTColorRole(TFTColorRole role, uint16_t onColor, uint16_t offColor)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
return;
#endif
const uint8_t index = static_cast<uint8_t>(role);
if (index >= static_cast<uint8_t>(TFTColorRole::Count)) {
return;
}
const uint32_t themeId = uiconfig.screen_rgb_color & 0x1F;
const bool isHighlightRole = (role == TFTColorRole::FavoriteNode || role == TFTColorRole::FavoriteNodeBGHighlight);
const bool isBodyRole = !isHighlightRole && isBodyColorRole(role);
// Classic monochrome themes collapse all non-black accents into one tone.
if (isMonochromeTheme(themeId)) {
if (onColor != TFTPalette::Black) {
onColor = getMonochromeAccent(themeId);
}
} else {
switch (themeId) {
case ThemeID::DefaultLight:
if (isHighlightRole) {
// High-contrast highlight chips on light UI.
onColor = TFTPalette::Black;
offColor = TFTPalette::Yellow;
} else if (isBodyRole) {
// Invert body colors for readability on white frames.
if (offColor == TFTPalette::Black && role != TFTColorRole::ActionMenuTitle) {
offColor = TFTPalette::White;
}
replaceColor(onColor, TFTPalette::White, TFTPalette::Black);
}
break;
case ThemeID::Christmas:
if (isHighlightRole || isBodyRole) {
replaceColor(onColor, TFTPalette::Yellow, TFTPalette::Gold);
replaceColor(offColor, TFTPalette::Black, TFTPalette::Pine);
}
break;
case ThemeID::Pink:
if (isHighlightRole) {
onColor = TFTPalette::Black;
offColor = TFTPalette::HotPink;
} else if (isBodyRole) {
replaceColor(offColor, TFTPalette::Black, TFTPalette::PalePink);
replaceColor(onColor, TFTPalette::White, TFTPalette::Black);
replaceColor(onColor, TFTPalette::Yellow, TFTPalette::DeepPink);
}
break;
case ThemeID::Creamsicle:
if (isHighlightRole) {
onColor = TFTPalette::Black;
offColor = TFTPalette::CreamOrange;
} else if (isBodyRole) {
replaceColor(offColor, TFTPalette::Black, TFTPalette::Cream);
replaceColor(onColor, TFTPalette::White, TFTPalette::Black);
replaceColor(onColor, TFTPalette::Yellow, TFTPalette::DeepOrange);
}
break;
case ThemeID::Blue:
if (isHighlightRole || isBodyRole) {
replaceColor(onColor, TFTPalette::Yellow, TFTPalette::SkyBlue);
replaceColor(offColor, TFTPalette::Black, TFTPalette::Navy);
}
break;
default:
break;
}
}
roleColors[index].onColorBe = toBe565(onColor);
roleColors[index].offColorBe = toBe565(offColor);
}
// Region registration
void registerTFTColorRegion(TFTColorRole role, int16_t x, int16_t y, int16_t width, int16_t height)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
return;
#endif
const uint8_t roleIndex = static_cast<uint8_t>(role);
if (roleIndex >= static_cast<uint8_t>(TFTColorRole::Count)) {
return;
}
if (!normalizeRegion(x, y, width, height)) {
return;
}
const TFTRoleColorsBe &colors = roleColors[roleIndex];
appendColorRegion(x, y, width, height, colors.onColorBe, colors.offColorBe);
}
void setAndRegisterTFTColorRole(TFTColorRole role, uint16_t onColor, uint16_t offColor, int16_t x, int16_t y, int16_t width,
int16_t height)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
(void)role;
(void)onColor;
(void)offColor;
(void)x;
(void)y;
(void)width;
(void)height;
return;
#else
setTFTColorRole(role, onColor, offColor);
registerTFTColorRegion(role, x, y, width, height);
#endif
}
void registerTFTColorRegionDirect(int16_t x, int16_t y, int16_t width, int16_t height, uint16_t onColor, uint16_t offColor)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
return;
#endif
if (!normalizeRegion(x, y, width, height))
return;
appendColorRegion(x, y, width, height, toBe565(onColor), toBe565(offColor));
}
void registerTFTActionMenuRegions(int16_t boxLeft, int16_t boxTop, int16_t boxWidth, int16_t boxHeight)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
(void)boxLeft;
(void)boxTop;
(void)boxWidth;
(void)boxHeight;
return;
#else
// Use theme-appropriate menu colors.
const TFTThemeDef &theme = kThemes[resolveThemeIndex()];
const TFTThemeRoleColor &menuBody = theme.roles[static_cast<size_t>(TFTColorRole::ActionMenuBody)];
const TFTThemeRoleColor &menuBorder = theme.roles[static_cast<size_t>(TFTColorRole::ActionMenuBorder)];
// Fill role includes a 1px shadow guard so stale frame edges are overwritten uniformly.
setAndRegisterTFTColorRole(TFTColorRole::ActionMenuBody, menuBody.onColor, menuBody.offColor, boxLeft - 1, boxTop - 1,
boxWidth + 2, boxHeight + 2);
registerTFTColorRegion(TFTColorRole::ActionMenuBody, boxLeft, boxTop - 2, boxWidth, 1);
registerTFTColorRegion(TFTColorRole::ActionMenuBody, boxLeft, boxTop + boxHeight + 1, boxWidth, 1);
registerTFTColorRegion(TFTColorRole::ActionMenuBody, boxLeft - 2, boxTop, 1, boxHeight);
registerTFTColorRegion(TFTColorRole::ActionMenuBody, boxLeft + boxWidth + 1, boxTop, 1, boxHeight);
setAndRegisterTFTColorRole(TFTColorRole::ActionMenuBorder, menuBorder.onColor, menuBorder.offColor, boxLeft, boxTop, boxWidth,
1);
registerTFTColorRegion(TFTColorRole::ActionMenuBorder, boxLeft, boxTop + boxHeight - 1, boxWidth, 1);
registerTFTColorRegion(TFTColorRole::ActionMenuBorder, boxLeft, boxTop, 1, boxHeight);
registerTFTColorRegion(TFTColorRole::ActionMenuBorder, boxLeft + boxWidth - 1, boxTop, 1, boxHeight);
#endif
}
// Frame signature & utilities
uint32_t getTFTColorFrameSignature()
{
#if !GRAPHICS_TFT_COLORING_ENABLED
return 0;
#else
uint32_t hash = kFnv1aOffsetBasis;
hash = fnv1aAppendByte(hash, colorRegionCount);
for (uint8_t i = 0; i < colorRegionCount; i++) {
const TFTColorRegion &r = colorRegions[i];
hash = fnv1aAppendU16(hash, static_cast<uint16_t>(r.x));
hash = fnv1aAppendU16(hash, static_cast<uint16_t>(r.y));
hash = fnv1aAppendU16(hash, static_cast<uint16_t>(r.width));
hash = fnv1aAppendU16(hash, static_cast<uint16_t>(r.height));
hash = fnv1aAppendU16(hash, r.onColorBe);
hash = fnv1aAppendU16(hash, r.offColorBe);
}
return hash;
#endif
}
uint8_t getTFTColorRegionCount()
{
#if !GRAPHICS_TFT_COLORING_ENABLED
return 0;
#else
return colorRegionCount;
#endif
}
void clearTFTColorRegions()
{
for (uint8_t i = 0; i < colorRegionCount; i++) {
colorRegions[i].enabled = false;
}
if (colorRegionCount < MAX_TFT_COLOR_REGIONS) {
colorRegions[colorRegionCount].enabled = false;
}
colorRegionCount = 0;
}
uint16_t resolveTFTColorPixel(int16_t x, int16_t y, bool isset, uint16_t defaultOnColor, uint16_t defaultOffColor)
{
for (int i = static_cast<int>(colorRegionCount) - 1; i >= 0; i--) {
const TFTColorRegion &r = colorRegions[i];
if (x >= r.x && x < r.x + r.width && y >= r.y && y < r.y + r.height) {
return isset ? r.onColorBe : r.offColorBe;
}
}
return isset ? defaultOnColor : defaultOffColor;
}
uint16_t resolveTFTOffColorAt(int16_t x, int16_t y, uint16_t defaultOffColor)
{
#if !GRAPHICS_TFT_COLORING_ENABLED
(void)x;
(void)y;
return defaultOffColor;
#else
const uint16_t defaultOffBe = toBe565(defaultOffColor);
const uint16_t sampledBe = resolveTFTColorPixel(x, y, false, defaultOffBe, defaultOffBe);
return static_cast<uint16_t>((sampledBe >> 8) | (sampledBe << 8));
#endif
}
} // namespace graphics
-163
View File
@@ -1,163 +0,0 @@
#pragma once
#include "configuration.h"
#include <stdint.h>
namespace graphics
{
struct TFTColorRegion {
int16_t x;
int16_t y;
int16_t width;
int16_t height;
uint16_t onColorBe;
uint16_t offColorBe;
// Required by ST7789 driver: it scans until the first disabled entry.
bool enabled = false;
};
static constexpr size_t MAX_TFT_COLOR_REGIONS = 48;
extern TFTColorRegion colorRegions[MAX_TFT_COLOR_REGIONS];
enum class TFTColorRole : uint8_t {
HeaderBackground = 0,
HeaderTitle,
HeaderStatus,
SignalBars,
ConnectionIcon,
UtilizationFill,
FavoriteNode,
ActionMenuBorder,
ActionMenuBody,
ActionMenuTitle,
FrameMono,
BootSplash,
FavoriteNodeBGHighlight,
NavigationBar,
NavigationArrow,
Count
};
#if HAS_TFT || defined(ST7701_CS) || defined(ST7735_CS) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || \
defined(ST7789_CS) || defined(HX8357_CS) || defined(USE_ST7789) || defined(ILI9488_CS) || defined(ST7796_CS) || \
defined(USE_ST7796) || defined(HACKADAY_COMMUNICATOR)
#define GRAPHICS_TFT_COLORING_ENABLED 1
#else
#define GRAPHICS_TFT_COLORING_ENABLED 0
#endif
static constexpr bool kTFTColoringEnabled = GRAPHICS_TFT_COLORING_ENABLED != 0;
constexpr bool isTFTColoringEnabled()
{
return kTFTColoringEnabled;
}
void setTFTColorRole(TFTColorRole role, uint16_t onColor, uint16_t offColor);
void registerTFTColorRegion(TFTColorRole role, int16_t x, int16_t y, int16_t width, int16_t height);
// Convenience helper for the common "set role then register one region" flow.
void setAndRegisterTFTColorRole(TFTColorRole role, uint16_t onColor, uint16_t offColor, int16_t x, int16_t y, int16_t width,
int16_t height);
// Register a region using explicit colors (no role lookup). Use when the
// color comes from a theme field rather than a role (e.g. battery fill).
void registerTFTColorRegionDirect(int16_t x, int16_t y, int16_t width, int16_t height, uint16_t onColor, uint16_t offColor);
void registerTFTActionMenuRegions(int16_t boxLeft, int16_t boxTop, int16_t boxWidth, int16_t boxHeight);
uint32_t getTFTColorFrameSignature();
uint8_t getTFTColorRegionCount();
void clearTFTColorRegions();
uint16_t resolveTFTColorPixel(int16_t x, int16_t y, bool isset, uint16_t defaultOnColor, uint16_t defaultOffColor);
// Resolve effective region-mapped OFF color at a coordinate in native-endian RGB565.
uint16_t resolveTFTOffColorAt(int16_t x, int16_t y, uint16_t defaultOffColor);
// -- Theme engine ------------------------------------------------------
// Each theme has four fields that work together:
//
// id - ThemeID:: constant, used for in-code references.
// name - human-readable label shown in the theme picker.
// uniqueIdentifier - the stable numeric value persisted to
// uiconfig.screen_rgb_color and restored at boot.
// This is a CONTRACT with saved configs on disk - once
// assigned, never reuse or renumber, even if the theme is
// deleted or the kThemes[] array is reordered.
// visible - controls whether a theme appears in the picker menu.
// Hidden themes can still be restored and applied if their
// uniqueIdentifier is persisted.
//
// Display order in the menu is controlled by kThemes[] array order among
// themes where visible == true, NOT by any numeric value above.
//
// To add a new theme:
// 1. Add a unique constant in ThemeID below (next unused value).
// 2. Add a kThemes[] entry at the desired menu position, with a unique
// uniqueIdentifier that has never been used by any prior theme.
// 3. Set visible=true if it should appear in the picker.
//
// To retire a theme without breaking saved configs:
// - Preferred: keep the entry and set visible=false so existing saved
// uniqueIdentifier values still resolve to the same theme.
// - If you remove the entry, resolveThemeIndex() falls back to DefaultDark
// when the persisted uniqueIdentifier no longer matches any theme.
// - Do NOT reuse a retired uniqueIdentifier for a future theme.
namespace ThemeID
{
constexpr uint32_t DefaultDark = 0;
constexpr uint32_t DefaultLight = 1;
constexpr uint32_t Christmas = 2;
constexpr uint32_t Pink = 3;
constexpr uint32_t Blue = 4;
constexpr uint32_t Creamsicle = 5;
constexpr uint32_t MeshtasticGreen = 6;
constexpr uint32_t ClassicRed = 7;
constexpr uint32_t MonochromeWhite = 8;
} // namespace ThemeID
// Per-role color pair stored in native (little-endian) RGB565 format.
struct TFTThemeRoleColor {
uint16_t onColor;
uint16_t offColor;
};
// Complete theme definition.
struct TFTThemeDef {
uint32_t id; // ThemeID constant - in-code identifier for this theme.
const char *name; // Human-readable label shown in the theme picker.
uint32_t uniqueIdentifier; // Stable persisted value copied into uiconfig.screen_rgb_color.
// Never reuse or renumber - see file-level notes above.
TFTThemeRoleColor roles[static_cast<size_t>(TFTColorRole::Count)];
uint16_t batteryFillGood;
uint16_t batteryFillMedium;
uint16_t batteryFillBad;
bool fullFrameInvert; // Apply full-frame FrameMono inversion (ST7789 light themes)
bool visible; // Show in the theme picker menu. Hidden themes still apply
// correctly if their uniqueIdentifier is persisted (dev/legacy themes).
};
// Count of themes whose .visible flag is true. Use this when building menus.
size_t getVisibleThemeCount();
// Access the Nth visible theme (0 .. getVisibleThemeCount()-1). Hidden themes
// are skipped, preserving kThemes[] order among the visible entries.
const TFTThemeDef &getVisibleThemeByIndex(size_t visibleIndex);
// Return the theme that matches uiconfig.screen_rgb_color (falls back to Dark).
const TFTThemeDef &getActiveTheme();
// Return the visible-theme index for the currently active theme, or SIZE_MAX
// if the active theme is hidden (so menus can show "no selection").
size_t getActiveVisibleThemeIndex();
// Convenience accessors - safe to call even when coloring is compiled out.
uint16_t getThemeHeaderBg();
uint16_t getThemeHeaderText();
uint16_t getThemeHeaderStatus();
uint16_t getThemeBodyBg();
uint16_t getThemeBodyFg();
bool isThemeFullFrameInvert();
uint16_t getThemeBatteryFillColor(int batteryPercent);
// Reinitialise default roleColors from the active theme. Call after a
// theme change so that any role registered without a prior setTFTColorRole()
// picks up theme-appropriate defaults.
void loadThemeDefaults();
} // namespace graphics
+38 -145
View File
@@ -16,6 +16,12 @@
extern SX1509 gpioExtender;
#endif
#ifdef TFT_MESH_OVERRIDE
uint16_t TFT_MESH = TFT_MESH_OVERRIDE;
#else
uint16_t TFT_MESH = COLOR565(0x67, 0xEA, 0x94);
#endif
#if defined(ST7735S)
#include <LovyanGFX.hpp> // Graphics and font library for ST7735 driver chip
@@ -1145,9 +1151,7 @@ static LGFX *tft = nullptr;
#endif
#include "SPILock.h"
#include "TFTColorRegions.h"
#include "TFTDisplay.h"
#include "TFTPalette.h"
#include <SPI.h>
#ifdef UNPHONE
@@ -1157,25 +1161,6 @@ extern unPhone unphone;
GpioPin *TFTDisplay::backlightEnable = NULL;
namespace
{
static constexpr uint8_t kFullRepaintChunkRows = 8;
static inline uint16_t getThemeDefaultOnColor()
{
return graphics::TFTPalette::White;
}
static inline uint16_t getThemeDefaultOffColor()
{
#if GRAPHICS_TFT_COLORING_ENABLED
return graphics::getThemeBodyBg();
#else
return TFT_BLACK;
#endif
}
} // namespace
TFTDisplay::TFTDisplay(uint8_t address, int sda, int scl, OLEDDISPLAY_GEOMETRY geometry, HW_I2C i2cBus)
{
LOG_DEBUG("TFTDisplay!");
@@ -1215,15 +1200,14 @@ TFTDisplay::~TFTDisplay()
free(linePixelBuffer);
linePixelBuffer = nullptr;
}
if (repaintChunkBuffer != nullptr) {
free(repaintChunkBuffer);
repaintChunkBuffer = nullptr;
}
}
// Write the buffer to the display memory
void TFTDisplay::display(bool fromBlank)
{
if (fromBlank)
tft->fillScreen(TFT_BLACK);
concurrency::LockGuard g(spiLock);
uint32_t x, y;
@@ -1232,70 +1216,12 @@ void TFTDisplay::display(bool fromBlank)
uint32_t x_FirstPixelUpdate;
uint32_t x_LastPixelUpdate;
bool isset, dblbuf_isset;
uint16_t colorTftWhite, colorTftBlack;
uint16_t colorTftMesh, colorTftBlack;
bool somethingChanged = false;
// Theme defaults for non-role pixels.
const uint16_t defaultOnColor = getThemeDefaultOnColor();
const uint16_t defaultOffColor = getThemeDefaultOffColor();
static uint16_t lastDefaultOnColor = 0;
static uint16_t lastDefaultOffColor = 0;
static bool haveLastDefaults = false;
const bool themeDefaultsChanged =
!haveLastDefaults || (defaultOnColor != lastDefaultOnColor) || (defaultOffColor != lastDefaultOffColor);
const bool forceFullRepaint = fromBlank || themeDefaultsChanged;
// If theme defaults changed, reset panel background immediately so stale pixels don't linger.
if (forceFullRepaint) {
tft->fillScreen(defaultOffColor);
}
colorTftWhite = (defaultOnColor >> 8) | ((defaultOnColor & 0xFF) << 8);
colorTftBlack = (defaultOffColor >> 8) | ((defaultOffColor & 0xFF) << 8);
#if GRAPHICS_TFT_COLORING_ENABLED
static uint32_t lastColorFrameSignature = 0;
const bool hasColorRegions = graphics::getTFTColorRegionCount() > 0;
const uint32_t colorFrameSignature = graphics::getTFTColorFrameSignature();
const bool forceFullColorRepaint = forceFullRepaint || (colorFrameSignature != lastColorFrameSignature);
// When region roles/layout changed, color can differ even with identical monochrome glyph bits.
// Repaint full frame only for those frames, then return to diff-based updates.
if (forceFullColorRepaint) {
for (uint32_t yStart = 0; yStart < displayHeight; yStart += kFullRepaintChunkRows) {
const uint32_t rowsThisChunk = min<uint32_t>(kFullRepaintChunkRows, displayHeight - yStart);
for (uint32_t row = 0; row < rowsThisChunk; row++) {
y = yStart + row;
y_byteIndex = (y / 8) * displayWidth;
y_byteMask = (1 << (y & 7));
uint16_t *chunkRow = repaintChunkBuffer + (row * displayWidth);
for (x = 0; x < displayWidth; x++) {
isset = (buffer[x + y_byteIndex] & y_byteMask) != 0;
if (hasColorRegions) {
chunkRow[x] = graphics::resolveTFTColorPixel(static_cast<int16_t>(x), static_cast<int16_t>(y), isset,
colorTftWhite, colorTftBlack);
} else {
chunkRow[x] = isset ? colorTftWhite : colorTftBlack;
}
}
}
#if defined(HACKADAY_COMMUNICATOR)
tft->draw16bitBeRGBBitmap(0, yStart, repaintChunkBuffer, displayWidth, rowsThisChunk);
#else
tft->pushImage(0, yStart, displayWidth, rowsThisChunk, repaintChunkBuffer);
#endif
}
memcpy(buffer_back, buffer, displayBufferSize);
lastColorFrameSignature = colorFrameSignature;
haveLastDefaults = true;
lastDefaultOnColor = defaultOnColor;
lastDefaultOffColor = defaultOffColor;
graphics::clearTFTColorRegions();
return;
}
#endif
// Store colors byte-reversed so that TFT_eSPI doesn't have to swap bytes in a separate step
colorTftMesh = __builtin_bswap16(TFT_MESH);
colorTftBlack = __builtin_bswap16(TFT_BLACK);
y = 0;
while (y < displayHeight) {
@@ -1304,7 +1230,7 @@ void TFTDisplay::display(bool fromBlank)
// Step 1: Do a quick scan of 8 rows together. This allows fast-forwarding over unchanged screen areas.
if (y_byteMask == 1) {
if (!forceFullRepaint) {
if (!fromBlank) {
for (x = 0; x < displayWidth; x++) {
if (buffer[x + y_byteIndex] != buffer_back[x + y_byteIndex])
break;
@@ -1322,14 +1248,13 @@ void TFTDisplay::display(bool fromBlank)
}
}
// Step 2: Scan this row for changed span (first and last changed pixel).
uint32_t x_FirstChanged = 0;
for (x_FirstChanged = 0; x_FirstChanged < displayWidth; x_FirstChanged++) {
isset = buffer[x_FirstChanged + y_byteIndex] & y_byteMask;
// Step 2: Scan each of the 8 rows individually. Find the first pixel in each row that needs updating
for (x_FirstPixelUpdate = 0; x_FirstPixelUpdate < displayWidth; x_FirstPixelUpdate++) {
isset = buffer[x_FirstPixelUpdate + y_byteIndex] & y_byteMask;
if (!forceFullRepaint) {
if (!fromBlank) {
// get src pixel in the page based ordering the OLED lib uses
dblbuf_isset = buffer_back[x_FirstChanged + y_byteIndex] & y_byteMask;
dblbuf_isset = buffer_back[x_FirstPixelUpdate + y_byteIndex] & y_byteMask;
if (isset != dblbuf_isset) {
break;
}
@@ -1339,42 +1264,26 @@ void TFTDisplay::display(bool fromBlank)
}
// Did we find a pixel that needs updating on this row?
if (x_FirstChanged < displayWidth) {
uint32_t x_LastChanged = displayWidth - 1;
while (x_LastChanged > x_FirstChanged) {
isset = buffer[x_LastChanged + y_byteIndex] & y_byteMask;
if (!forceFullRepaint) {
dblbuf_isset = buffer_back[x_LastChanged + y_byteIndex] & y_byteMask;
if (x_FirstPixelUpdate < displayWidth) {
// Quickly write out the first changed pixel (saves another array lookup)
linePixelBuffer[x_FirstPixelUpdate] = isset ? colorTftMesh : colorTftBlack;
x_LastPixelUpdate = x_FirstPixelUpdate;
// Step 3: copy all remaining pixels in this row into the pixel line buffer,
// while also recording the last pixel in the row that needs updating
for (x = x_FirstPixelUpdate + 1; x < displayWidth; x++) {
isset = buffer[x + y_byteIndex] & y_byteMask;
linePixelBuffer[x] = isset ? colorTftMesh : colorTftBlack;
if (!fromBlank) {
dblbuf_isset = buffer_back[x + y_byteIndex] & y_byteMask;
if (isset != dblbuf_isset) {
break;
x_LastPixelUpdate = x;
}
} else if (isset) {
break;
x_LastPixelUpdate = x;
}
x_LastChanged--;
}
// Align the first pixel for update to an even number so the total alignment of
// the data will be at 32-bit boundary, which is required by GDMA SPI transfers.
x_FirstPixelUpdate = x_FirstChanged & ~1U;
x_LastPixelUpdate = x_LastChanged | 1U;
if (x_LastPixelUpdate >= displayWidth) {
x_LastPixelUpdate = displayWidth - 1;
}
// Step 3: Copy only the changed span into the pixel line buffer.
for (x = x_FirstPixelUpdate; x <= x_LastPixelUpdate; x++) {
isset = buffer[x + y_byteIndex] & y_byteMask;
#if GRAPHICS_TFT_COLORING_ENABLED
if (hasColorRegions) {
linePixelBuffer[x] = graphics::resolveTFTColorPixel(static_cast<int16_t>(x), static_cast<int16_t>(y), isset,
colorTftWhite, colorTftBlack);
} else {
linePixelBuffer[x] = isset ? colorTftWhite : colorTftBlack;
}
#else
linePixelBuffer[x] = isset ? colorTftWhite : colorTftBlack;
#endif
}
#if defined(HACKADAY_COMMUNICATOR)
tft->draw16bitBeRGBBitmap(x_FirstPixelUpdate, y, &linePixelBuffer[x_FirstPixelUpdate],
@@ -1382,8 +1291,8 @@ void TFTDisplay::display(bool fromBlank)
#else
// Step 4: Send the changed pixels on this line to the screen as a single block transfer.
// This function accepts pixel data MSB first so it can dump the memory straight out the SPI port.
tft->pushImage(x_FirstPixelUpdate, y, (x_LastPixelUpdate - x_FirstPixelUpdate + 1), 1,
&linePixelBuffer[x_FirstPixelUpdate]);
tft->pushRect(x_FirstPixelUpdate, y, (x_LastPixelUpdate - x_FirstPixelUpdate + 1), 1,
&linePixelBuffer[x_FirstPixelUpdate]);
#endif
somethingChanged = true;
}
@@ -1392,14 +1301,6 @@ void TFTDisplay::display(bool fromBlank)
// Copy the Buffer to the Back Buffer
if (somethingChanged)
memcpy(buffer_back, buffer, displayBufferSize);
#if GRAPHICS_TFT_COLORING_ENABLED
lastColorFrameSignature = colorFrameSignature;
#endif
haveLastDefaults = true;
lastDefaultOnColor = defaultOnColor;
lastDefaultOffColor = defaultOffColor;
graphics::clearTFTColorRegions();
}
void TFTDisplay::sdlLoop()
@@ -1613,7 +1514,7 @@ bool TFTDisplay::connect()
#else
tft->setRotation(3); // Orient horizontal and wide underneath the silkscreen name label
#endif
tft->fillScreen(getThemeDefaultOffColor());
tft->fillScreen(TFT_BLACK);
if (this->linePixelBuffer == NULL) {
this->linePixelBuffer = (uint16_t *)malloc(sizeof(uint16_t) * displayWidth);
@@ -1623,14 +1524,6 @@ bool TFTDisplay::connect()
return false;
}
}
if (this->repaintChunkBuffer == NULL) {
this->repaintChunkBuffer = (uint16_t *)malloc(sizeof(uint16_t) * displayWidth * kFullRepaintChunkRows);
if (!this->repaintChunkBuffer) {
LOG_ERROR("Not enough memory to create TFT repaint chunk buffer\n");
return false;
}
}
return true;
}
+1 -2
View File
@@ -63,5 +63,4 @@ class TFTDisplay : public OLEDDisplay
virtual bool connect() override;
uint16_t *linePixelBuffer = nullptr;
uint16_t *repaintChunkBuffer = nullptr;
};
};
-70
View File
@@ -1,70 +0,0 @@
#pragma once
#include <stdint.h>
namespace graphics
{
namespace TFTPalette
{
constexpr uint16_t rgb565(uint8_t red, uint8_t green, uint8_t blue)
{
return static_cast<uint16_t>(((red & 0xF8) << 8) | ((green & 0xFC) << 3) | ((blue & 0xF8) >> 3));
}
constexpr uint16_t Black = 0x0000;
constexpr uint16_t White = 0xFFFF;
constexpr uint16_t DarkGray = 0x4208;
constexpr uint16_t Gray = 0x8410;
constexpr uint16_t LightGray = 0xC618;
constexpr uint16_t Red = rgb565(255, 0, 0);
constexpr uint16_t Green = rgb565(0, 255, 0);
constexpr uint16_t Blue = rgb565(0, 130, 252);
constexpr uint16_t Yellow = rgb565(255, 255, 0);
constexpr uint16_t Orange = rgb565(255, 165, 0);
constexpr uint16_t Cyan = rgb565(0, 255, 255);
constexpr uint16_t Magenta = rgb565(255, 0, 255);
constexpr uint16_t Good = Green;
constexpr uint16_t Medium = Yellow;
constexpr uint16_t Bad = Red;
// Christmas / seasonal accent colors
constexpr uint16_t ChristmasRed = rgb565(178, 34, 34);
constexpr uint16_t ChristmasGreen = rgb565(0, 128, 0);
constexpr uint16_t Gold = rgb565(255, 215, 0);
constexpr uint16_t Pine = rgb565(15, 35, 10);
// Pink theme colors (light variant)
constexpr uint16_t HotPink = rgb565(255, 105, 180);
constexpr uint16_t PalePink = rgb565(255, 228, 235);
constexpr uint16_t DeepPink = rgb565(200, 50, 120);
// Blue theme colors (dark variant)
constexpr uint16_t SkyBlue = rgb565(100, 180, 255);
constexpr uint16_t Navy = rgb565(15, 15, 50);
constexpr uint16_t DeepBlue = rgb565(30, 60, 120);
// Creamsicle theme colors (light variant)
constexpr uint16_t CreamOrange = rgb565(255, 140, 50);
constexpr uint16_t DeepOrange = rgb565(220, 100, 20);
constexpr uint16_t Cream = rgb565(255, 248, 235);
// Classic monochrome theme accent colors (single-color-on-black themes)
constexpr uint16_t MeshtasticGreen = rgb565(0x67, 0xEA, 0x94);
constexpr uint16_t ClassicRed = rgb565(255, 64, 64);
// Monochrome White reuses TFTPalette::White above.
// Fast contrast picker for monochrome glyph overlays on arbitrary RGB565 backgrounds.
// Uses channel-sum brightness approximation to keep code size small.
constexpr uint16_t pickReadableMonoFg(uint16_t backgroundColor)
{
const uint16_t r = (backgroundColor >> 11) & 0x1F;
const uint16_t g = (backgroundColor >> 5) & 0x3F;
const uint16_t b = backgroundColor & 0x1F;
return ((r + g + b) >= 70) ? DarkGray : White;
}
} // namespace TFTPalette
} // namespace graphics
+3 -7
View File
@@ -145,7 +145,7 @@ void drawDigitalClockFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int1
// === Set Title, Blank for Clock
const char *titleStr = "";
// === Header ===
graphics::drawCommonHeader(display, x, y, titleStr, true, true, true);
graphics::drawCommonHeader(display, x, y, titleStr, true, true);
uint32_t rtc_sec = getValidTime(RTCQuality::RTCQualityDevice, true); // Display local timezone
char timeString[16];
@@ -293,15 +293,11 @@ void drawDigitalClockFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int1
// Draw an analog clock
void drawAnalogClockFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
{
#if GRAPHICS_TFT_COLORING_ENABLED
// Clear previous frame pixels so moving hands don't leave stale artifacts on TFT light theme.
display->clear();
#endif
display->setTextAlignment(TEXT_ALIGN_LEFT);
// === Set Title, Blank for Clock
const char *titleStr = "";
// === Header ===
graphics::drawCommonHeader(display, x, y, titleStr, true, true, true);
graphics::drawCommonHeader(display, x, y, titleStr, true, true);
// clock face center coordinates
int16_t centerX = display->getWidth() / 2;
@@ -482,4 +478,4 @@ void drawAnalogClockFrame(OLEDDisplay *display, OLEDDisplayUiState *state, int16
} // namespace ClockRenderer
} // namespace graphics
#endif
#endif
+82 -70
View File
@@ -1,6 +1,10 @@
#include "configuration.h"
#if HAS_SCREEN
#include "CompassRenderer.h"
#include "NodeDB.h"
#include "UIRenderer.h"
#include "configuration.h"
#include "gps/GeoCoord.h"
#include "graphics/ScreenFonts.h"
#include "graphics/SharedUIDisplay.h"
#include <cmath>
@@ -9,103 +13,111 @@ namespace graphics
{
namespace CompassRenderer
{
// Point helper class for compass calculations
struct Point {
float x, y;
Point(float x, float y) : x(x), y(y) {}
void rotate(float angle)
{
float cos_a = cos(angle);
float sin_a = sin(angle);
float new_x = x * cos_a - y * sin_a;
float new_y = x * sin_a + y * cos_a;
x = new_x;
y = new_y;
}
void scale(float factor)
{
x *= factor;
y *= factor;
}
void translate(float dx, float dy)
{
x += dx;
y += dy;
}
};
void drawCompassNorth(OLEDDisplay *display, int16_t compassX, int16_t compassY, float myHeading, int16_t radius)
{
// Show the compass heading (not implemented in original)
// This could draw a "N" indicator or north arrow
// For now, we'll draw a simple north indicator
// const float radius = 17.0f;
if (currentResolution == ScreenResolution::High) {
radius += 4;
}
const float northAngle = (uiconfig.compass_mode != meshtastic_CompassMode_FIXED_RING) ? -myHeading : 0.0f;
const int16_t nX = compassX + static_cast<int16_t>((radius - 1) * sinf(northAngle));
const int16_t nY = compassY - static_cast<int16_t>((radius - 1) * cosf(northAngle));
Point north(0, -radius);
if (uiconfig.compass_mode != meshtastic_CompassMode_FIXED_RING)
north.rotate(-myHeading);
north.translate(compassX, compassY);
display->setFont(FONT_SMALL);
display->setTextAlignment(TEXT_ALIGN_CENTER);
#if !GRAPHICS_TFT_COLORING_ENABLED
display->setColor(BLACK);
const int16_t nLabelWidth = display->getStringWidth("N");
if (currentResolution == ScreenResolution::High) {
display->fillRect(nX - 8, nY - 1, nLabelWidth + 3, FONT_HEIGHT_SMALL - 6);
display->fillRect(north.x - 8, north.y - 1, display->getStringWidth("N") + 3, FONT_HEIGHT_SMALL - 6);
} else {
display->fillRect(nX - 4, nY - 1, nLabelWidth + 2, FONT_HEIGHT_SMALL - 6);
display->fillRect(north.x - 4, north.y - 1, display->getStringWidth("N") + 2, FONT_HEIGHT_SMALL - 6);
}
#endif
display->setColor(WHITE);
display->drawString(nX, nY - 3, "N");
}
void drawArrowToNode(OLEDDisplay *display, int16_t x, int16_t y, int16_t size, float bearing)
{
const float radians = bearing * DEG_TO_RAD;
const float sinA = sinf(radians);
const float cosA = cosf(radians);
const float tipHalf = size * 0.5f;
const float lx = -(size / 6.0f);
const float ly = size / 4.0f;
const float rx = (size / 6.0f);
const float ry = size / 4.0f;
const float tx = 0.0f;
const float ty = size / 4.5f;
const int16_t tipX = static_cast<int16_t>(x + (tipHalf * sinA));
const int16_t tipY = static_cast<int16_t>(y - (tipHalf * cosA));
const int16_t leftX = static_cast<int16_t>(x + (lx * cosA) - (ly * sinA));
const int16_t leftY = static_cast<int16_t>(y + (lx * sinA) + (ly * cosA));
const int16_t rightX = static_cast<int16_t>(x + (rx * cosA) - (ry * sinA));
const int16_t rightY = static_cast<int16_t>(y + (rx * sinA) + (ry * cosA));
const int16_t tailX = static_cast<int16_t>(x + (tx * cosA) - (ty * sinA));
const int16_t tailY = static_cast<int16_t>(y + (tx * sinA) + (ty * cosA));
display->fillTriangle(tipX, tipY, leftX, leftY, tailX, tailY);
display->fillTriangle(tipX, tipY, rightX, rightY, tailX, tailY);
display->drawString(north.x, north.y - 3, "N");
}
void drawNodeHeading(OLEDDisplay *display, int16_t compassX, int16_t compassY, uint16_t compassDiam, float headingRadian)
{
const int16_t size = static_cast<int16_t>(compassDiam * 0.6f);
drawArrowToNode(display, compassX, compassY, size, headingRadian * RAD_TO_DEG);
}
Point tip(0.0f, -0.5f), tail(0.0f, 0.35f); // pointing up initially
float arrowOffsetX = 0.14f, arrowOffsetY = 0.9f;
Point leftArrow(tip.x - arrowOffsetX, tip.y + arrowOffsetY), rightArrow(tip.x + arrowOffsetX, tip.y + arrowOffsetY);
bool getHeadingRadians(double lat, double lon, float &headingRadian)
{
headingRadian = 0.0f;
Point *arrowPoints[] = {&tip, &tail, &leftArrow, &rightArrow};
if (uiconfig.compass_mode == meshtastic_CompassMode_FREEZE_HEADING)
return true;
if (!screen)
return false;
if (screen->hasHeading()) {
headingRadian = screen->getHeading() * DEG_TO_RAD;
return true;
for (int i = 0; i < 4; i++) {
arrowPoints[i]->rotate(headingRadian);
arrowPoints[i]->scale(compassDiam * 0.6);
arrowPoints[i]->translate(compassX, compassY);
}
const float estimatedHeadingDeg = screen->estimatedHeading(lat, lon);
if (!(estimatedHeadingDeg >= 0.0f))
return false;
headingRadian = estimatedHeadingDeg * DEG_TO_RAD;
return true;
#ifdef USE_EINK
display->drawTriangle(tip.x, tip.y, rightArrow.x, rightArrow.y, tail.x, tail.y);
#else
display->fillTriangle(tip.x, tip.y, rightArrow.x, rightArrow.y, tail.x, tail.y);
#endif
display->drawTriangle(tip.x, tip.y, leftArrow.x, leftArrow.y, tail.x, tail.y);
}
float adjustBearingForCompassMode(float bearingRadian, float headingRadian)
void drawArrowToNode(OLEDDisplay *display, int16_t x, int16_t y, int16_t size, float bearing)
{
if (uiconfig.compass_mode != meshtastic_CompassMode_FIXED_RING)
return bearingRadian - headingRadian;
float radians = bearing * DEG_TO_RAD;
return bearingRadian;
Point tip(0, -size / 2);
Point left(-size / 6, size / 4);
Point right(size / 6, size / 4);
Point tail(0, size / 4.5);
tip.rotate(radians);
left.rotate(radians);
right.rotate(radians);
tail.rotate(radians);
tip.translate(x, y);
left.translate(x, y);
right.translate(x, y);
tail.translate(x, y);
display->fillTriangle(tip.x, tip.y, left.x, left.y, tail.x, tail.y);
display->fillTriangle(tip.x, tip.y, right.x, right.y, tail.x, tail.y);
}
float radiansToDegrees360(float angleRadian)
float estimatedHeading(double lat, double lon)
{
constexpr float fullTurnDeg = 360.0f;
float degrees = angleRadian * RAD_TO_DEG;
if (degrees < 0.0f)
degrees += fullTurnDeg;
else if (degrees >= fullTurnDeg)
degrees -= fullTurnDeg;
return degrees;
// Simple magnetic declination estimation
// This is a very basic implementation - the original might be more sophisticated
return 0.0f; // Return 0 for now, indicating no heading available
}
uint16_t getCompassDiam(uint32_t displayWidth, uint32_t displayHeight)
@@ -125,4 +137,4 @@ uint16_t getCompassDiam(uint32_t displayWidth, uint32_t displayHeight)
} // namespace CompassRenderer
} // namespace graphics
#endif
#endif

Some files were not shown because too many files have changed in this diff Show More