167 Commits

Include renames

Author	SHA1	Message	Date
Tom	de6b23190a	Test suite rebuild (#11322) ... * docs(nodedb): make the native node cap unambiguous The native node cap was stated in four places that disagreed, and the disagreement already caused a wrong diagnosis: a saturated 200-node database looked arithmetically impossible because the cap had been read as 248, computed from a header that does not apply on this platform. The real value is 198. On portduino MAX_NUM_NODES is not a compile-time constant at all - the variant defines it as `portduino_config.MaxNodes`, resolved at runtime, default 200 and settable per host with `General: MaxNodes`. variant.h is reached before mesh-pb-constants.h, so that header's ARCH_PORTDUINO branch never fires and its plausible-looking 250 is dead code. - #error-guard the dead branch rather than leave a wrong number where people grep. The guard found a real defect: seven translation units reach mesh-pb-constants.h without configuration.h (SerialConsole.cpp, StreamAPI.cpp, PacketAPI.cpp, ServerAPI.cpp, PiWebServer.cpp, ServiceEnvelope.cpp, MeshtasticOTA.cpp, and test/TestUtil.cpp), so each was compiling with a different MAX_NUM_NODES - and therefore a different PACKETHISTORY_MAX - than the rest of the build. Each now includes configuration.h first. It cannot be included from mesh-pb-constants.h itself: that reaches SerialConsole.h through DebugConfiguration.h and closes a cycle. - Name the bare 250 in getMaxNodesAllocatedSize() NODEDB_MIGRATION_LOAD_CEILING. It is a decode allowance for files written by larger-cap firmware, not a cap, and it read like one. - Fix docs/node_info_stores.md, which named the wrong source and a "10-250" range that is wrong for native, and the copilot-instructions tunables line that said "portduino 250". * test(harness): give each suite its own scratch HOME and report leftovers Native suites shared one directory. Every suite that constructs a NodeDB loads and saves ~/.portduino/default/prefs/ - nodes.proto, config.proto, channels.proto, module.proto, device.proto, warm.dat, transmit_history.dat - and nothing cleared it, so state leaked suite -> suite within a run and run -> every run after it. A test run could also rewrite a real meshtasticd node database on the same machine. Per-run isolation does not fix this: the leak is generated inside a single run, so the boundary has to be per suite. bin/pio-test-isolate.sh runs each suite in its own scratch $HOME, registered as test_testing_command for env:native and env:coverage so a bare `pio test` and CI get the same boundary, not just bin/run-tests.sh. It runs the binary unchanged and exits with its exit code, so PlatformIO's pass/fail is untouched. Overriding HOME here rather than around `pio` also sidesteps the blocker that a bare HOME= breaks pio's own ~/.platformio/penv/bin/pio lookup. Leftovers are reported as a second axis, PASS/FAIL x CLEAN/DIRTY, because an unintended write has no matching assertion by definition - nobody writes TEST_ASSERT for a save they do not know is happening. The harness asserts it from outside, so it applies to every suite without the author opting in. - Only the *set of changed paths* is asserted, never contents. Hashes answer the boolean "did this change?" and nothing more; content baselines over protobuf bytes would churn on every NodeInfoLite field added, which is how snapshot suites become noise. - Deliberate writes are declared in test/state-manifest.tsv - one central file, suite / flags / mandatory reason. run-tests.sh prints the opt-out count on every run. - Granularity follows the state flag, so the two ship together: per-test by default (TestUtil redefines RUN_TEST to checkpoint after each test, naming the exact test that dirtied things), suite boundary for state=per-suite, where carrying state across test cases is the declared behaviour. - A declared write that does NOT happen is reported as MISSING, not folded into DIRTY. It catches silently broken persistence; a warning for now, since some are conditional. - Graded AMBER, not RED. With isolation in place DIRTY means "undeclared", not "dangerous", and a check that lands red on day one gets switched off. Guard the guard, both halves: state_assert_empty() refuses to run a suite against a sandbox that is not empty (otherwise the after-diff measures against the wrong baseline and reports CLEAN while meaning nothing), and bin/test-state-check.sh drives the real wrapper with fixtures asserting CLEAN / CLEAN / DIRTY / MISSING plus both directions of the empty assertion. A checker that silently matches everything would otherwise pass forever. --write-manifest proposes entries for a human to paste and justify; it never applies them, and neither does CI. * test(harness): stop reporting Unity's exit code as a signal A native suite ends in exit(UNITY_END()), and UNITY_END() returns the failure count. PlatformIO's native runner reads that non-zero exit code as a POSIX signal number, so four failures print "Program received signal SIGILL", five print "SIGTRAP", and the suite is classified [ERRORED] rather than [FAILED]. There is no crash. The signal name tracks the failure count and nothing else - it moved SIGILL -> SIGTRAP when a diagnostic probe added a fifth failure - and it cost hours of hunting a memory bug that did not exist, on an env (native) that carries no sanitizer at all. It also explains the phantom extra test case in the totals: the runner adds a synthetic entry for the signal it thinks it saw. run-tests.sh now says so inline whenever a signal line appears, and the three agent-facing docs say it too. * test(admin): isolate NodeDB and globals per test setUp() did `if (!nodeDB) nodeDB = new NodeDB();` and never deleted it, so 83 of the 85 tests shared one never-reset database and never restored config, owner, devicestate or channelFile. The fixture that does restore them was opt-in and armed by exactly two tests. The setUp comment claiming the rest "set their own config/region state and are unaffected" was not true - the admin handlers under test write all four globals. Route every test through the fixture instead: setUp saves the globals and installs a fresh NodeDB, tearDown restores and deletes it. The two tests that armed it themselves no longer need to. All 85 pass, so nothing was silently relying on the shared state. It costs about 7% of the suite's runtime (a NodeDB construction is a loadFromDisk plus, with a region set, key generation) - worth paying to write the phase 3 tests against a clean fixture rather than 83 tests' residue. Also cap the per-test attribution in the run summary at five entries; the full list stays in the suite's sandbox. * test(fs): cover the bounded file-manifest walk getFiles() runs on every phone sync via STATE_SEND_FILEMANIFEST, and nothing asserted any of its bounding behaviour. It does execute unasserted from test_stream_api's handshakes, but the cap, the depth limit, the wasLimited paths, overlong-path rejection and capacity release were all unguarded. Eight tests, all describing what the code does today: today's code is already correct here, since #10778 landed the by-reference collectFiles(), the 64-entry cap, the strlcpy bounds and the swap-idiom release. They pass on arrival, which is the point - this is the baseline a later change has to leave alone. Two things they do not cover, and cannot: - Moving reserve() outside the __cpp_exceptions guard. Exceptions are on natively, so the #else branch is not compiled. The suite's job there is to prove that change alters nothing observable. - The file.name() null guard. No in-tree backend returns null; the guard is defensive. The manifest-release test pins the swap idiom rather than calling PhoneAPI's releaseFilesManifest(), which is file-local. It asserts capacity() == 0, not just size() == 0 - a size-only check passes on clear(), which is the bug #7924 shipped. Suite count 43 -> 44, recounted against the directories rather than copied. * test(admin): assert node-DB metadata saves skip the radio reload set_favorite_node, set_ignored_node and toggle_muted_node each persist a NodeInfoLite bit and nothing else. MeshService::reloadConfig() gates its region re-derivation and configChanged notification on saveWhat & (SEGMENT_CONFIG | SEGMENT_CHANNELS), so a SEGMENT_NODEDATABASE-only save already skips the live radio reconfigure. Pure characterization - all three pass on develop. Worth pinning because that reconfigure is the path implicated in the WisMesh Tag favourite-node crash, and develop asserts nothing about it: widening the saveWhat mask or reordering the check would currently go unnoticed. Ported from the config-save series along with ConfigChangedCounter (an Observer<void *> counting configChanged notifications, the only externally visible signal that the reload branch was taken) and TEST_NODE_NUM. They join the existing suite, so no suite-count change. * refactor(menu): extract the mute toggle into a named function The node menu's mute action was inline in a banner-callback lambda, and that lambda only ever runs via screen->showOverlayBanner() - which is why nothing in MenuHandler.cpp was reachable from a test. Lift the `selected == Mute` branch into menuHandler::toggleNodeMuted(uint32_t) and call it from the lambda. Behaviour-neutral by construction: same statements, same order, same bare saveToDisk(). The null check moves into the function, so the call site no longer needs its own lookup. Verified by the native build and suite; the byte-identical-image check on a headroom-constrained nRF52 board was not run locally - CI's firmware-size comment covers it. Three tests come with it, all describing today's behaviour: - the bit flips both ways and no configChanged fires (develop never calls reloadConfig on this path); - an unknown node is a no-op rather than a write; - and the segment mask. Flipping one NodeInfoLite bit currently rewrites all five segments via bare saveToDisk(). That is asserted deliberately, with the comment naming it as characterization of a known defect: a pending fix narrows it to SEGMENT_NODEDATABASE, and when it lands this assertion is expected to change, which makes the improvement visible in the diff instead of silent. saveToDisk() is not virtual, so the mask is observed through its effect - remove the five prefs files, toggle, and see which reappear. * docs(test): make every suite count a pointer to the canonical one test/native-suite-count is the registered total and is machine-checked against test/test_* on every full run and by the suite-count-check CI job. Every other statement of the count is a copy that drifts: copilot-instructions said 12, AGENTS.md said 19, and the real number is 44. Replace both literals with a pointer to the file, say explicitly that no document should state the count as a literal, and reframe the two suite listings as descriptions rather than inventories - they carry per-suite information the count does not, so they stay, but nothing should infer completeness from their length. Register the new FS suite in both. * test(harness): randomise suite order, reproducibly Landed last, deliberately. Randomising an order-dependent suite set does not find bugs so much as convert a silent pass into intermittent red, and the first instinct is to revert the randomisation rather than fix the coupling. Phases 1-2 removed the coupling; this keeps it removed. Both runners previously hid order dependence behind a fixed order that happened to differ between them, and neither order was chosen: CI's area rules put admin first, PlatformIO's local discovery is reverse alphabetical and put it last. CI was green by accident. - bin/run-tests.sh --shuffle / --seed <n>. The seed defaults to HEAD's short SHA: one order per commit, so a red is replayable and attributable to the diff instead of flaky, while the project keeps exploring orders. Printed at the start and carried into the RESULT line, so a verdict is replayable from that line alone; the full order is printed on failure, because for an order-dependent failure the order is the diagnostic. - The shuffle is a Fisher-Yates over a MINSTD generator rather than awk's rand(), whose sequence differs between gawk and mawk. A seed that does not reproduce the same order on another machine is not a seed. - Shuffling needs one `pio test -f <suite>` invocation per suite - PlatformIO orders by its own os.walk() over test/ and filters only select - which measures at about 4.7s per suite of extra startup. - CI shuffles its area order, seeded from GITHUB_SHA and printed with the command to replay it locally. Intra-area order stays PlatformIO's; controlling it there would mean per-suite invocations, which is a cost worth deciding separately. Also records the 16 measured entries in test/state-manifest.tsv, each with its reason, taken from a full run's --write-manifest output rather than guessed. * test(default): cover the region-throttle interval overload getConfiguredOrDefaultMsScaled(configured, default, nodes, TrafficType) is the overload every telemetry and position module actually calls, and nothing referenced TrafficType anywhere under test/. All four of its behaviours were unguarded: the no-region guard, the throttle <= 1 short-circuit, the multiply, and the 64-bit overflow clamp. The throttles are real, not hypothetical - EU_866 carries PROFILE_LITE, which sets both positionThrottle and telemetryThrottle to 10, so a change here moves broadcast spacing in that region by an order of magnitude. Each test pins numOnlineNodes at the congestion threshold and uses ROUTER, which never congestion-scales, so the coefficient is 1 and the throttle is the only variable. The overflow case needs a base above INT32_MAX/10, hence three days rather than one. * ci(test): keep pull-request suite order fixed, seed the rest Shuffling the area order on every run - including pull_request - would turn a contributor's PR red for an ordering they did not choose, which is how a randomisation gets reverted instead of the coupling being fixed. That is the exact dynamic the ordering work was sequenced last to avoid, and the previous commit walked straight into it. - pull_request keeps the fixed declared area order. - push and schedule shuffle, seeded from the commit SHA: deterministic per commit, printed, attributable, and never blocking someone else's PR. - A suite_order_seed input on workflow_call and workflow_dispatch overrides both, so a specific failing order can be replayed anywhere, including on a PR. The run log prints which mode it took, the resulting order, and the local command to replay it. * ci(test): satisfy CKV_GHA_7 and yamllint on the seed input The seed is reachable through workflow_call, which callers can pass programmatically. The workflow_dispatch copy tripped checkov's "workflow_dispatch inputs MUST be empty" rule, and suppressing it was not worth it: replaying a specific order is a local operation, and the run log already prints the exact bin/run-tests.sh command to do it. * style(menu): apply the node-ID format convention RadioInterface.cpp documents the rule: 0x%08x in logs, !%08x in user-facing display. MenuHandler held every remaining exception - seven logs printing bare %08X, and two display labels doing the same. Repo-wide there are now no bare %08X node IDs left in log calls. * ci(test): pass workflow inputs through env, not shell interpolation suite_order_seed and github.event_name were spliced into the run: script as ${{ }} text, so a value carrying shell metacharacters would execute as code on the runner rather than being read as data. semgrep (run-shell-injection) and zizmor (template-injection) both flag it. Both now arrive as environment variables and are read as "$VAR". * refactor(test): share the seeded shuffle between the harness and CI bin/run-tests.sh and test_native.yml each carried a byte-identical copy of the MINSTD Fisher-Yates awk. The workflow prints "replay locally: ./bin/run-tests.sh --shuffle --seed $seed" after a shuffled CI run, and that instruction is only true while the two agree - drift would be announced by a replay quietly reproducing a different order than the one that failed. Extract shuffle_suites() to bin/lib/shuffle.sh and source it from both. Permutations verified identical across seeds before and after the move. * fix(test): correct the shared-state MISSING check and summary join Three defects in the new harness: state_classify() matched declarations two different ways - state_path_declared() for "undeclared", a hand-rolled regex for "missing". Interpolating an entry into an ERE also let a metacharacter in a manifest name match a file that is not the declared one. Both directions now go through the one helper. `paste -sd'; '` does not join with "; ": with -s, paste cycles through a multi-character delimiter one character per join, so paths rendered as "a;b c;d e". Replaced with an awk join. test-state-check.sh ran on after a failed cd instead of stopping (SC2164). ./bin/test-state-check.sh: 6/6 fixtures pass, MISSING included. * fix(portduino): bound General.MaxNodes MaxNodes was validated only for <= 0. Any positive value, including a typo'd or pasted-in one, propagates to MAX_NUM_NODES and scales both the node DB and the nodes.proto decode ceiling - failing at boot with no obvious cause. The ceiling is a sanity bound, not a capability limit; raise it if a host genuinely needs more. * docs(nodedb): reconcile the capacity tables The property matrix omitted the ESP32-S3 100-node flash tier that the platform table above it lists, and neither mentioned that the WASM build overrides MaxNodes to 80 in wasm_config_apply(). * fix(nodedb): make mesh-pb-constants.h self-sufficient on portduino The ARCH_PORTDUINO #error assumed it was unreachable in a normal build. It is not: the vendored device-ui sources include this header without configuration.h, which broke both native-tft docker builds. Include configuration.h here instead, ahead of every compile-time default - variant.h overrides MAX_RX_TOPHONE as well as MAX_NUM_NODES, so placing it lower in the file just moves the divergence to a redefinition. The #error stays as a backstop for the case where that include genuinely stops providing the cap. Verified with the native env's own flags: a TU including only this header now compiles, normal-order use of both macros compiles, and NodeDB.cpp compiles. * fix(portduino): raise the MaxNodes ceiling to 16000 Marked artificial: nothing in the node DB fails at 16001. 16000 sits just under the 16384 (128 x 128) population where HopScalingModule saturates its sampling denominator and starts dropping nodes, so a host inside the bound still gets meaningful hop recommendations. * lint(trunk): advise on node IDs logged as bare %08x RadioInterface.cpp documents the convention - 0x%08x in logs, !%08x in display - but nothing enforced it, which is how the MenuHandler cluster drifted. 22 call sites in PacketHistory, NodeInfoModule and PositionModule are still off it. A trunk linter rather than a CI grep job, because trunk checks changed files: new violations get flagged without a 22-site cleanup landing in an unrelated PR. Modelled on the existing too-many-defined definition. Scoped to values it can tell are IDs - an ID-shaped argument (->num, .from, getNodeNum) or message text naming one. A 32-bit hex that is not an ID is out of scope, so the CRC32 logs in ethOTA.cpp are correctly ignored. Emits "note", trunk's only non-blocking level: "warning" and "info" both exit non-zero and would gate CI, which is not what a log-format nit deserves. The pre-existing sites are line-scoped in the allowlist, so a new bad call in those same files is still caught. * lint(trunk): stop exempting the known node-id-format sites The seeded allowlist made the rule green by declaring the backlog acceptable. Empty it instead, so the 22 pre-existing sites are reported and get cleaned up by whoever next edits those files. Costs nothing to do: the rule emits "note", so these are non-blocking either way. The allowlist stays for its real purpose - a value the linter misreads as an ID. * style: log node and packet IDs as 0x%08x Clears the 22 sites the node-id-format linter reports, so the rule starts from zero rather than from a backlog nobody can see - trunk suppresses pre-existing findings by default, so left alone these would not have surfaced on edit the way an empty allowlist implies. Format strings only; no argument or control flow changes. The !%08x user-facing display forms are deliberately untouched - that is the other half of the same convention. * test(harness): build once up front, so suite timings mean something run-tests.sh fused build and run in a single pio invocation, so whichever suite PlatformIO's directory walk reached first absorbed the entire src compile and reported it as its own duration. On a real run that made a 0.03s suite report 13m21s, and hid the build cost from every other number in the summary. Do what .github/workflows/test_native.yml already does: one --without-testing build pass, then run with --without-building. Measured on a full 44-suite run - the build is now a single reported figure and 968 test cases execute in 1.9s, with no suite above 0.084s. Build output goes to its own log rather than $LOG: the outcome regexes match "error:" and "[ERRORED]", so a compiler diagnostic sharing that file would read as a test failure. Both red paths now keep the log they quote from. $LOG and the build log are mktemps the EXIT trap removes, so the three grepped lines were previously all anyone ever saw - and the cause is usually further up than the first [FAILED]. * test(harness): keep the run log on every red path bin/pio-test-isolate.sh already keeps a failing or DIRTY suite's sandbox and log under .pio/test-state/<suite>/. What was missing is the cross-suite view: $LOG is a mktemp the EXIT trap deletes, so run-tests.sh quoted three grepped lines from a file that no longer existed by the time anyone looked. Preserve it as .pio/build/<env>/test-failure.log from both red paths - including "no success summary found", which said "see log" while preserving nothing, and which is exactly the case where the build died before any suite ran and so left no per-suite sandbox either. Cleared at the start of every run, so a green run cannot leave a red one's log lying around looking current. * fix(test): report the real failure count on a shuffled red A shuffled run is one `pio test` invocation per suite, all appending to the same log, so the log carries one PlatformIO "N test cases:" summary per suite. verdict_red() took `tail -1`, which reports whatever the LAST suite did: a failure in suite 3 printed a "0 failed" summary from suite 44 directly under "RED - failures detected:". Sum the summaries instead. A single summary line - every unshuffled run - is passed through verbatim, so the familiar output is byte-identical. The patterns are passed to the awk helper as strings rather than /regex/ literals: awk evaluates a regex literal in argument position as `$0 ~ /re/`, so the callee would receive 0 or 1 and silently sum garbage. * fix(test): do not emit an empty suite name for an empty shuffle `printf '%s\n' "$@"` with no arguments still writes one empty line, and both callers read shuffle_suites through mapfile, so an empty suite list arrived as a single suite named "". Return before the printf when there is nothing to shuffle. * test(harness): state and enforce the Linux host requirement The native harness is a Linux tool: bash 4+ (mapfile), GNU coreutils and GNU find (-printf, md5sum, -executable). Most of that predates this branch - mapfile and both find predicates are already on develop - but none of it was written down, so the requirement was there to be discovered rather than read. Refuse to start on a non-Linux uname instead of degrading. On a BSD userland this would not fail cleanly: it would mis-hash the sandbox and mis-read the suite list, and still print a verdict. A state check that silently measures the wrong thing is worse than one that declines to run. Carrying a per-host fallback was the alternative, and it buys a second code path that nothing in CI exercises. bin/test-native-docker.sh already exists for macOS and non-Linux hosts, and the native-macos PlatformIO env is a build target for meshtasticd, not a test host - the isolation wrapper is registered for env:native and env:coverage only. Documented in the script header, test/README.md, and both agent docs. * fix(test): terminate every suite with exit(UNITY_END()) Two sites across two suites ended on a bare UNITY_END(). That ends the reporting, not the suite: setup() returns, the runtime goes on calling loop(), and the process runs forever. PlatformIO does not notice - it reports a suite from its Unity output, not from process exit - so the suite passes, the run goes green, and the binary stays resident. Thirteen of them had accumulated on one dev box, the oldest 19 hours old. The costs are quiet by construction: - the per-suite sandbox is deleted underneath a live process, so its CLEAN/DIRTY verdict describes what the suite had written when the harness stopped looking, not what it left behind; - .gcda coverage and LeakSanitizer's report both flush from atexit handlers, so a suite that never exits contributes no coverage and gets no leak check; - each survivor pins its own deleted 94 MB binary, which du cannot see. One of the two is the #else of an architecture guard, which is the easiest one to get wrong - it looks like there is nothing to clean up. test_mqtt has a correct exit(UNITY_END()) in its live branch, so a "does this file call exit() anywhere" check passes the file whole. test_serial had two more. develop's serial-config validation rework restructured that suite - the architecture guard is gone and both remaining branches now exit correctly - so this commit no longer has anything to change there; bin/lint-unity-exit.sh, added later on this branch, is what keeps it that way. test/README.md gets a section on it, since the skeleton showing the right shape had not stopped this happening. * test(harness): detect and reap suites that outlive their run A suite that never exits was invisible: PlatformIO reports a suite from its Unity output, so the run stayed green while the binary kept running. Two checks, because they fail differently. Runtime, in bin/pio-test-isolate.sh: the sandbox $HOME is mktemp-unique per suite, so any process still holding it is a survivor of that suite. Matching on the environment rather than a remembered PID identifies one whatever its parentage - a fork, a grandchild, a process already reparented to init - none of which a $! comparison catches. Reaped before the after-fingerprint is taken, so that fingerprint measures a tree nobody is still writing to, and so a run cannot leave processes accumulating on the host. Recorded as a sixth summary column and graded AMBER: the tests did pass, but the CLEAN verdict and the coverage were measured under a false assumption. Author-time, as bin/lint-unity-exit.sh, wired into trunk at "note" like node-id-format: every UNITY_END() must be wrapped in exit(). The rule is per occurrence, and that is the point - a file-level "calls exit() somewhere" check passes test_serial and test_mqtt, which have a correct one in their live branch and a bare one in the #else. Running it over the tree turned up test_mqtt, which the file-level pass had missed. It allows `int rc = UNITY_END(); ...; exit(rc)`, used by test_packet_signing to restore globals between the summary and the exit. That is where the rule gives ground: capturing and never exiting would leak and is not flagged. Flagging a correct idiom would push someone to "fix" working code. bin/test-state-check.sh gains a survivor fixture, asserting the wrapper both reports and reaps - a detector that only reports leaves the host accumulating processes, which is half the harm. 8/8. * fix(lint): make the unity-exit scanner statement-aware The rule judged one physical line at a time, which reports two kinds of correct code as bare: /* a comment that happens to mention UNITY_END() */ <- interior lines were never stripped exit( UNITY_END()); <- exit( and the macro never met On a probe of both, two of three findings were wrong. This is a note-level rule whose whole job is advice, and bin/lint-node-id-format.sh already says why that matters: a false positive costs more than a miss. One that cries wolf gets ignored, and the real finding goes with it. Carry /* ... */ state across lines and accumulate logical statements before testing, with a 12-line cap so one unclosed call cannot swallow the rest of the file - the same structure lint-node-id-format.sh uses, so the two custom linters in bin/ work alike rather than each having its own idea. Verified both directions: the develop-era sources still produce the same four findings, the fixed tree produces none, and a probe covering block-comment interiors, wrapped exit(), line comments, return UNITY_END() and capture-then- exit reports only the genuinely bare calls - including a complete block comment followed by real bare code on the same line, which the state machine has to keep live. Reported by CodeRabbit on #11322. * fix(lint): tokenise instead of pattern-matching, and self-test it Second round of review findings on the same scanner, all confirmed by direct test before changing anything. Six defects, one root cause: layered regexes cannot tokenise C++. False positives (correct code reported): - UNITY_END() inside a string literal read as code False negatives (real leaks missed): - a string containing "/*" opened comment state and swallowed later lines - greedy .* removed everything between two block comments on one line, taking a bare call with it - myexit(UNITY_END()) matched the exit() exemption as a substring - x == UNITY_END() and total += UNITY_END() matched the assignment exemption Replaced with a character-level scan carrying comment state, and token-bounded exemptions: exit must be a whole identifier, and the capture form must be a plain `=`. Raw string literals are still not modelled - there are none under test/, and delimiter tracking for a case that does not occur would be untested code guarding untested code, so it is documented rather than guessed at. Also drops the `return UNITY_END()` exemption. It only terminates from main(), there is no main() under test/, and from a helper it just returns a count. bin/test-lint-unity-exit.sh pins all fifteen cases, every false positive and false negative found in review among them. The rule has been wrong twice in a way that looked fine by inspection; it needed a self-test more than it needed another careful reading. Two further findings in the same review: - bin/run-tests.sh dropped PASSTHRU in shuffled mode, so `--shuffle -vvv` built verbosely and then ran quietly. The shuffled loop now forwards EXTRA_ARGS, which is PASSTHRU minus the -f pair it supplies per suite. - bin/run-tests.sh did not guard `cd "$ROOT_DIR"`. And one that did not reproduce: the survivor fixture's glob does find the pid file (verified with the lookup instrumented - the earlier failure was an artifact of running the script from /tmp, where SCRIPT_DIR cannot resolve). The assertion was still weak, because an empty pid took the "not running" branch and passed vacuously. It now fails if the pid was never recorded, and finds the file by search rather than assuming a directory depth. Reported by CodeRabbit on #11322. * fix(lint): report each UNITY_END occurrence at its own location The self-test only asked "did the linter say anything", so it could not have caught a wrong line, a wrong column, or a missing second finding. Fixtures now assert the exact diagnostics as line:col, and the first run of that assertion found two real problems. The caret pointed at the wrong occurrence. For `exit(UNITY_END()); UNITY_END();` the verdict was right but the column was 17 - the wrapped call - because the scanner stripped terminating forms out of the whole statement and then reported the first occurrence it had seen. Two bare calls on one line reported once. Judged per occurrence now, by looking back through whitespace at what wraps it, so both the count and the caret are right. That also needed a position map from strip_noncode(): removing a comment or collapsing a literal shifts every later column, and counting occurrences in the raw line does not recover it either - TEST_MESSAGE("... UNITY_END() ..."); UNITY_END(); has two occurrences in the raw text and one in the code. Four of the expected columns I wrote by hand were also wrong, off by one. The linter was right in every case; the assertions were not. They are computed from the fixture text now rather than pasted from output, because a baseline accepted from the tool it is testing asserts nothing. 17 fixtures, including the two-on-one-line case from review and its mirror. Reported by CodeRabbit on #11322.	2026-08-06 14:05:07 +00:00
TomandClaude Opus 4.8	e26b6bfc5a	TMM & Warmstore key source naming (#11119) ... * one becomes two * warmstore clarify * Address PR review: key-provenance terminology consistency - Log line now says "not key-proven" (gate is XEdDSA OR manual, not just signer) - Rename markKeySignerProvenForTest -> markKeyXeddsaSignedForTest (sets only the XEdDSA bit) - Docs + test comments: "signer bit" -> "XEdDSA-signed bit" clod helped too * Rename signer-proven -> key-proven for broadened provenance predicate Address PR #11119 review: the copyPublicKey()/copyUser() out-parameter and the cache-path replay gate now report entry->keyProven() (XEdDSA-signed OR manually verified), so the "signerProven" name and "signer-proven" comments were misleading. Rename the public out-param to keyProven, the local cachedKeySignerProven to cachedKeyProven, and update coupled callers, log strings, docs headings, and comments to say "key-proven". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * nitpicks --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-31 10:39:39 +00:00
Tom	2024bb8384	Arrival time fix perhaps (#11274) ... * Add explicit presence for MeshPacket.rx_time (arrival time) rx_time is now proto3 optional with a has_rx_time presence bit, matching the rx_rssi treatment. A node with no GPS and no phone connected yet has no time source at all, so a bare 0 was indistinguishable from a genuine 1970-01-01 reading; downstream consumers (replay packets, JSON serialization) now check has_rx_time instead of the value. * Dedupe rx_time stamping into a shared helper; trim a debug log string Extract the repeated haveTime/rx_time/has_rx_time stamp logic (5 call sites across Router.cpp, MeshBeaconModule.cpp, MeshService.cpp) into Router::computeRxTimeStamp()/stampRxTime(). Also shorten the new RTC.cpp LOG_DEBUG string. Saves 48 bytes of flash on rak4631 (measured), no behavior change. * Fix has_rx_rssi presence carried unconditionally through StoreForward replay preparePayload() set has_rx_rssi = true unconditionally on replay, regardless of whether the packet's rx_rssi at store time was a genuine measurement (e.g. MQTT-relayed packets carry no real RSSI). Store the presence bit alongside rx_rssi in PacketHistoryStruct and restore it on replay instead. Flagged by Copilot on #11271 (same root cause the has_rx_time explicit presence work fixes) but never addressed before that PR merged. * Trim comment blocks to the repo's 1-2 line guideline .github/copilot-instructions.md:338 caps code comments at 1-2 lines; several blocks added across the rx_time explicit-presence work ran well past that. Also consolidates Time.cpp's file-level doc comment into Time.h, where the rest of the Time:: API contract already lives. No behavior change. * Add rx_time explicit-presence test coverage - test_meshpacket_serializer: has_rx_time=false fixture plus tests asserting JsonSerialize/JsonSerializeEncrypted emit 0 rather than leaking the millis() placeholder, alongside the has_rx_time=true baseline. - test_stream_api: two tests driving a real PhoneAPI handshake (want_config_id through STATE_SEND_PACKETS) that simulate a phone time-giving transaction arriving before vs. after a queued packet is drained - covering both the reconciled and the ships-with-placeholder-absent paths of MeshService::reconcilePendingRxTimes(). * Fix three correctness issues flagged in review - Time.h: drop the reserved-identifier include guard (_MT_TIME_H); pragma once already covers it, matching convention elsewhere (e.g. RTC.h). - Time.cpp: rebase getMillis64()'s wrap accumulator when the test seam swaps clock sources, so a real<->injected clock jump isn't miscounted as a genuine 32-bit wrap. - NodeInfoModule: the 12h reply-suppression window is a local dedup duration, not a wall-clock reading - switch it to Time::getMillis64() so RTC-quality jumps and replayed packets' stale rx_time can't perturb it. - StoreForwardModule: has_rx_time was derived from *current* RTC quality at replay time rather than stored at capture time, so a history entry saved while time-blind could be misreported as a valid epoch once the clock later improved. Persist the presence bit in PacketHistoryStruct instead. * tryfix CI * post review fixes * more test fixes	2026-07-29 14:03:25 +00:00
Tom	2c57a17124	Phantom node fix perhaps (#11271) ... * trying to fix phantom nodes * fix comment spam * oops - missed one	2026-07-28 18:32:18 +00:00
3 people	1e982fa78c	Sign plaintext packets in licensed mode (#10969) ... * feat: sign licensed plaintext packets Preserve and publish identity keys in licensed mode so existing XEdDSA signatures can authenticate plaintext traffic. Sign licensed broadcasts and direct messages when they fit, while keeping PKI encryption disabled and normal routing behavior unchanged. * fix(security): persist licensed channel sanitation * fix(security): close licensed migration lifecycle gaps * fix(security): address licensed signing review feedback * test: restore signing globals from Unity teardown * fix(baseui): allow confirmed ham region selection * fix(baseui): guard region picker validation * style: trunk fmt --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Austin <vidplace7@gmail.com>	2026-07-27 02:57:15 +00:00
4 people	45cb8e7500	feat: preserve normal radio profiles across event firmware (#11110) ... * feat: isolate event radio profiles * fix: preserve identity on degraded config boot * fix: guard event profile storage * style: align event profile storage names * fix: discard event profile on normal boot * refactor: simplify event profile cleanup * fix: limit inactive profile migration to event firmware * fix(event): make event-profile capacity check portable across filesystems The USERPREFS_EVENT_MODE storage preflight called FSCom.totalBytes() and FSCom.usedBytes(), which only exist on ESP32's LittleFS wrapper. Every other backend failed to compile once event mode was enabled: error: 'class InternalFileSystem' has no member named 'totalBytes' (nRF52) error: no member named 'totalBytes' in 'fs::FS' (Portduino) This was not caught earlier because the event block is behind #if USERPREFS_EVENT_MODE, and the existing unit tests only cover the pure helpers, which compile identically either way. Add fsTotalBytes()/fsUsedBytes() to FSCommon and implement them per backend: littlefs v1 traversal for nRF52 (Adafruit_LittleFS) and STM32 (STM32_LittleFS), FSInfo for RP2040, statvfs for Portduino, and the native methods for ESP32 and nRF54L15. The nRF52/STM32 path reports "full" if the traversal errors so the capacity check fails safe. Verified with USERPREFS_EVENT_MODE=1: native-macos test_event_profile_storage passes 5/5, and heltec-v3, rak4631, rak11310 and rak3172 all build clean. * fix(event): use std::filesystem for Portduino capacity, bump native-suite-count Two CI fixes: - native-windows has no <sys/statvfs.h>, so the Portduino branch of fsTotalBytes()/fsUsedBytes() broke the Windows build. Switch to std::filesystem::space(), which is already used under ARCH_PORTDUINO in HostMetrics.cpp and works on both POSIX and MinGW. Errors report "full" so the capacity check still fails safe. - This PR adds test/test_event_profile_storage, taking test/ from 40 to 41 suite directories, which trips the native-suite-count reconciliation check. * fix(event): scope boot-write deferral to the radio profile, address review Review feedback: - Copilot: saveProto()'s boot-write deferral applied to every file, so loadFromDisk()'s recovery writes (e.g. restoring owner fields into devicestate) were silently dropped and never retried, because the ctor CRC baselines are computed after loadFromDisk(). Deferral now applies only to the radio-profile files, via isRadioProfileFile(). - jp-bennett: eventConfigFromStandard() wrapped a struct copy plus one field assignment in a header helper with its own unit test. Inlined at its single call site and removed; the behaviour is covered end-to-end by hardware validation instead (RAK4631 normal -> event -> normal preserves NodeNum and public key while swapping LoRa, and restores the original channel byte-identically). - jp-bennett: the active-backup encrypted-storage migration ran for normal builds too, which changed non-event behaviour and contradicted the PR's stated event-only scope. Scoped to USERPREFS_EVENT_MODE; the adjacent block already covers the inactive standard backup in event builds. New tests (all verified to fail under mutation, not vacuous): - test_event_paths_never_collide_with_standard_or_shared_files: the core safety property. A path-table slip would make event firmware overwrite the user's real config, channels or backup, or capture a shared file like devicestate/nodedb. Nothing asserted this before. - test_only_radio_profile_files_defer_boot_writes: guards the deferral fix above so re-widening it fails loudly. - test_event_reservation_fits_smallest_supported_filesystem: the reservation is compile-time but must fit filesystems as small as 14 KiB (STM32WL) and 28 KiB (nRF52840). Protobuf growth pushing it past those would silently stop event profiles persisting - the exact failure mode confirmed by fault injection on a RAK4631. Verified with USERPREFS_EVENT_MODE both on and off: 7/7 tests pass, and rak4631, heltec-v3, rak11310 and rak3172 all build clean. * fix: preserve event profile storage recovery --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Benjamin Faershtein <benjaminfaershtein@Benjamins-MacBook-Pro-2.local>	2026-07-26 22:58:23 +00:00
Thomas Göttgens	f09f2b1366	Gate the PKI-decrypt key on signer-proven, not the opportunistic cache (#11116)	2026-07-22 11:12:06 +02:00
3 people	1872e5b306	TMM extend to ephemeral 3rd tier store (#11050) ... * Harden TMM NodeInfo direct-response: staleness, key hygiene, throttle The NodeInfo direct-response path answered queries on behalf of other nodes from an unauthenticated, never-expiring cache while suppressing the genuine request (STOP). That let a long-gone or forged identity be served as a fresh-looking, authoritative reply indefinitely. Three fixes: - Staleness gate: refuse to spoof a reply for a node not actually heard within ~6h (PSRAM cache via lastObservedMs; NodeDB fallback via sinceLastSeen, which tolerates last_heard==0 when the clock is unset). A stale entry now falls through so the real request propagates instead of being answered for a dead node. - Key hygiene: mirror NodeDB::updateUser() PKI checks when caching overheard NodeInfo - reject a packet advertising our own public key, and pin keys (drop a NodeInfo whose key mismatches an already-known key from NodeDB or from our own cache). Prevents cache poisoning / key substitution via the spoofed-reply path. - Response throttle: at most one spoofed reply per target node per 30s. Direct responses bypass the per-sender rate limiter (they STOP the request first) and the reply target is attacker-controlled, so this bounds airtime exhaustion and reflected floods. Recorded in a new NodeInfoPayloadEntry.lastResponseMs (PSRAM; self-expiring timestamp compare, no sweep needed). Tests: native NodeDB-fallback staleness (stale drops, fresh serves) plus PSRAM-guarded staleness, throttle, and key-mismatch cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Add TODO(T1-T9) markers for code-review findings on NodeInfo hardening Comment-only. Tags each site flagged in the review of the direct-response throttle/staleness/key-hygiene change so the follow-ups are visible in-context: T1 throttle black-holes distinct requestors; T2 per-target key doesn't bound aggregate TX; T3 non-PSRAM fallback unthrottled; T4 millis-wrap defeats the staleness gate; T5 redundant second O(n) scan for the throttle stamp; T6 duplicated 32-byte key compares; T7 sinceLastSeen() called twice; T8 two 6h constants can desync; T9 clockMs()==0 collides with the 0 sentinel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Address code-review cleanups T6, T7, T8 in NodeInfo direct-response T8: derive kNodeInfoMaxServeAgeSecs from kNodeInfoMaxServeAgeMs so the PSRAM and NodeDB-fallback staleness windows cannot desync. T6: collapse the three duplicated 32-byte key compares in cacheNodeInfoPacket (owner-impersonation + NodeDB pin + cache TOFU pin) into one pubKeysEqual() helper, so the compare lives in one place. T7: compute sinceLastSeen(node) once on the fallback staleness path so the tested age and the logged age cannot diverge (it calls getTime() each time). Remaining review items still marked in-code: T1/T2/T3 (throttle design), T4/T9 (millis-wrap + 0-sentinel), T5 (redundant throttle-stamp scan). Native test_traffic_management suite: 48/48 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Throttle the NodeDB fallback direct-response path (T3) The per-target NodeInfo response throttle lived only in the PSRAM NodeInfoPayloadEntry, so non-PSRAM boards - which answer from the NodeDB fallback path - emitted spoofed direct replies with no rate limit at all, leaving the airtime/reflection surface the throttle exists to close. Add a single module-global fallback stamp (nodeInfoFallbackLastResponseMs, 4 bytes, guarded by cacheLock) that throttles the fallback path to one spoofed reply per window across all targets. Coarser than the PSRAM per-target throttle, but the fallback path has no per-node slot to stamp, and a global cap still bounds spoofed transmissions - which is the point. Add a native test exercising the fallback throttle window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Make NodeInfo staleness/throttle wrap- and sentinel-safe (T4, T9) T9: the millis-based fields that use 0 as a "never" sentinel (lastObservedMs, lastResponseMs, nodeInfoFallbackLastResponseMs) could be stamped with a literal 0 during the one-millisecond clockMs()==0 instant at each ~49.7-day wrap, momentarily colliding with the sentinel and disabling the staleness/throttle gate for that entry. Route all such stamps through a new nowStampMs() that maps 0 -> 1; the 1 ms skew is irrelevant to every window these fields feed. T4: the staleness gate's modular age compare is only unambiguous while true age < 2^32 ms (~49.7 days); an entry that lingered unrefreshed that long could wrap to a small age and read as fresh, defeating the gate. Add a NodeInfo eviction pass to the maintenance sweep that drops entries past the serve window, so an entry is removed long before its age can approach the wrap boundary. This also frees slots holding stale identities. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Stamp PSRAM throttle entry by captured index, not a rescan (T5) The post-send throttle stamp did a second full O(n) scan of the PSRAM NodeInfo array under a fresh lock, even though findNodeInfoEntry() had already located the slot at the top of shouldRespondToNodeInfo(). Capture the slot index during that initial lookup and address the entry directly on the stamp path. Because the cache lock is released between the two accesses, the slot could have been evicted or reused, so re-validate node == p->to under the lock before stamping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Document accepted per-target throttle tradeoffs (T1, T2) Convert the T1/T2 TODO markers into a NOTE: the per-target PSRAM throttle keying and its lack of an aggregate bound are deliberate design choices, not pending work. A distinct requestor being throttled for one window is harmless on a redundant mesh, and keeping the PSRAM path per-target preserves throughput for legitimate multi-target responders (the fallback path already bounds aggregate via its global stamp). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Track signed-provenance of cached NodeInfo public keys Add keySignerProven to NodeInfoPayloadEntry, set when an observed NODEINFO frame's XEdDSA signature was verified (mp.xeddsa_signed). This distinguishes a trust-on-first-use key from one proven to belong to a signer. The flag is monotonic - once proven it stays proven, and the existing key-pin checks forbid the underlying key from changing - so a later unsigned frame cannot downgrade it. A signature can only be verified against a key we already held, so a first-contact key is always TOFU until a later signed frame upgrades it. Groundwork for using the TMM cache as a last-resort public-key source, where this flag serves as a trust tier. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Draw public keys from the TMM NodeInfo cache as a last resort Add TrafficManagementModule::copyPublicKey() and consult it from NodeDB::copyPublicKey() after the hot (NodeInfoLite) and warm (WarmNodeStore) tiers miss. This extends the pool of peers the node can encrypt to: a key the NodeInfo direct-response cache overheard for a node that has since aged out of both NodeDB tiers can still be used. The getter reports whether the key is signer-proven or trust-on-first-use. NodeDB serves TOFU keys here too - the same first-contact trust NodeDB already applies in updateUser() - so the pool actually expands to new long-tail nodes rather than only re-confirming known keys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Retain keyed NodeInfo entries as a pubkey pool; trust-tier eviction Now that NodeDB::copyPublicKey() draws keys from the NodeInfo cache, the flat 6 h serve-window eviction would discard useful keys the moment a node stops being servable. Split retention: an entry carrying a 32-byte public key is kept for kNodeInfoKeyRetentionMs (7 d), while a keyless entry still expires at the serve window. Both windows stay well under the ~49.7-day millis wrap, preserving the T4 wrap-safety guarantee. Make LRU victim selection trust-tiered to match: a keyless slot is sacrificed before any keyed one, and a trust-on-first-use key before a signer-proven key; within a tier the oldest loses. Mirrors WarmNodeStore's keyed-first admission so the most valuable keys are the stickiest under memory pressure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Test signed-provenance flag and copyPublicKey key-pool source PSRAM-gated tests (run on the ESP32 build, alongside the existing NodeInfo PSRAM tests): - copyPublicKey serves a TOFU key learned from an unsigned NodeInfo and reports signerProven=false - a later signature-verified NodeInfo upgrades provenance to signer-proven while the pinned key bytes stay unchanged - copyPublicKey reports a miss for an uncached node Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Inherit signer provenance from NodeDB when caching a re-found node When the TrafficManagement NodeInfo cache re-caches a node, mark its key signer-proven if NodeDB already knows the node as a verified signer for that same key - even if this particular (unicast/unsigned) frame carried no signature. Previously the flag only upgraded on a frame we verified ourselves, so a node we had already proven elsewhere looked TOFU here. Add NodeDB::isVerifiedSignerForKey(), which checks both tiers - the hot store's signed bitfield and the warm tier's cached signer bit - and requires the key to match so a rotated/mismatched key never inherits a stale verdict. Add WarmNodeStore::isVerifiedSigner() to expose the warm signer bit (the rebase onto develop added the bit itself; this surfaces it for lookups). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Lock NodeInfoPayloadEntry packing with a size assert keySignerProven cost zero bytes: sourceChannel, the two bools, and decodedBitfield are four 1-byte fields that fill a single 4-byte tail word (struct alignment is 4), so the flag consumed former padding rather than growing the 2000-entry PSRAM array. Add a static_assert pinning the entry to sizeof(meshtastic_User) + 20 so a future 5th trailing byte - which would open a fresh word (~8 KB PSRAM across the array) - fails the build instead of silently costing memory, prompting new flags to be packed into existing bytes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Pack NodeInfo cache booleans into 1-bit fields Fold hasDecodedBitfield and keySignerProven into adjacent uint8_t:1 bitfields so they share a single byte, reserving 6 spare bits for future flags without growing the 2000-entry PSRAM array. sizeof is unchanged (still one packed tail word, sizeof(meshtastic_User) + 20); access is by name exactly as before, so no call sites change. Reorders decodedBitfield ahead of the flags so the two 1-bit members stay adjacent and the compiler packs them together. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Gate NodeInfo replay on signer-proven provenance (compile-time, default on) Add TMM_NODEINFO_REPLAY_REQUIRE_SIGNED (default 1): the direct-response path now only spoofs a reply for a node whose key is signer-proven, in addition to the staleness gate - so replay is based on flagging AND staleness. Replay is a courtesy feature, and vouching for an unverified (trust-on-first-use) identity to other nodes is the risk this closes, so signed-only is the safer default. Define the macro to 0 at build time to also serve fresh TOFU-only nodes. Both paths are gated: the PSRAM path checks the cached keySignerProven flag, the NodeDB fallback checks nodeInfoLiteHasXeddsaSigned(node). The effective gate (TMM_NODEINFO_REPLAY_SIGNED_GATE) is bypassed when PKI is excluded from the build, since nothing can be signed there and the courtesy feature would otherwise be disabled outright. Tests: existing reply-expecting tests establish signer-proven state (a new markKeySignerProvenForTest hook for the PSRAM cache, the NodeDB signed bit for the fallback path); add fallback and PSRAM tests asserting an unsigned/TOFU node is withheld under the default gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Rehydrate re-admitted node names from the TMM NodeInfo cache The warm tier keeps an evicted node's key but not its name (the 40-byte record has no room), so a re-admitted long-tail node is nameless until its next NodeInfo broadcast. The TrafficManagement NodeInfo cache is much larger (2000 PSRAM entries) and commonly still holds the full User, so use it as a name reservoir the warm tier structurally cannot be. On re-admission (getOrCreateMeshNode), after the warm-tier restore, copy the cached identity from the TMM cache via a new copyUser() getter - but only when its cached key matches the key just restored from warm, so a name never attaches to a different identity than the one we encrypt to (key-matched trust). Guarded by HAS_TRAFFIC_MANAGEMENT and a null check; a no-op without the PSRAM cache or when no key is present. CopyUserToNodeInfoLite sets only user-related bits, so the warm-restored signer bit is preserved. Limitations (by design): PSRAM-only, and the TMM cache is RAM-only, so this helps within an uptime session, not across reboot. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Drop non-portable NodeInfoPayloadEntry size assert (fixes pico build) The static_assert pinned sizeof(NodeInfoPayloadEntry) to sizeof(meshtastic_User) + 20, but nanopb packs the generated User struct differently per platform: on pico its size is not a multiple of 4, so alignment padding before the uint32 timestamps makes the overhead 22, not 20, and the assert failed the build (native happened to be +20 and passed, hiding it). The bitfield packing it was guarding still stands; replace the fixed-count assert with a comment, since no portable byte count exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Run the NodeInfo cache paths in native tests (TMM_HAS_NODEINFO_CACHE) The NodeInfo direct-response cache and everything layered on it - key pinning, signer provenance, staleness, throttle - was compiled only for ARCH_ESP32 + BOARD_HAS_PSRAM, so its entire test set was skipped by the native CI suite and ran nowhere but real hardware. Fold the scattered platform guards into one TMM_HAS_NODEINFO_CACHE macro that also enables the cache (plain heap, delete[] path already existed) for ARCH_PORTDUINO unit-test builds. Production portduino and embedded test builds are unchanged. Tests that exercise the NodeDB fallback path drop the cache explicitly via a new dropNodeInfoCacheForTest() hook, so both response paths are covered in one binary. Native suite: 60/60 (was 50 with the 10 cache tests compiled out). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b2dafbbbd6501b42fc27175e599fb38be292b0e3) * Pin NodeInfo cache keys against the warm tier, not just the hot store cacheNodeInfoPacket() mirrored NodeDB::updateUser()'s key pin but checked only getMeshNode() - the hot store. updateUser's own pin effectively covers the warm tier too (getOrCreateMeshNode rehydrates the warm key before the check), so the mirror had a gap: for a node evicted to the warm tier whose cache slot was also gone, an attacker's NodeInfo with a bogus key passed the pin, and the cache's TOFU pin then locked the genuine node's frames out until the poisoned entry aged away. Split the authoritative lookup out of NodeDB::copyPublicKey() as copyPublicKeyAuthoritative() (hot store, then warm tier - never the opportunistic TMM tier, which would compare the cache against itself) and pin against that. Test: warm-tier-only key rejects a mismatching NodeInfo and accepts the matching one. Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 28e880875ef1f8bf16bdf301a4e6036537a84b4b) * Convert NodeInfo cache clocks to uint8 tick counters Replace the entry's uint32 millis stamps (lastObservedMs, lastResponseMs) with three uint8 free-running tick clocks - the same modular scheme the UnifiedCache counters already use: obsTick 3 min/tick (12.8 h period) - replay staleness gate respTick 5 s/tick (21.3 min) - per-target response throttle retTick 1 h/tick (10.67 d) - retention TTL + LRU age Validity is an explicit flag bit (hasObserved / hasResponded), not a 0 sentinel, and the maintenance sweep saturates a stamp (clears its flag) once its window passes, so no stamp can age toward its ~256-tick aliasing horizon. That retires the wrap/sentinel special cases (T4, T9) - nowStampMs() survives only for the fallback path's module-global millis stamp. obsTick is separated from retention by design: only a genuinely heard NODEINFO frame stamps it, so later membership-based retention refresh can never make a silent node look servable. Also drops lastObservedRxTime, which was only echoed in a debug log. Entry shrinks 136 -> 128 B; the 2000-entry array 272 kB -> 256 kB, 8 kB below the original develop footprint while keeping the throttle and retention features. Tick granularity costs at most one tick per window (+-5 s on 30 s, +-3 min on 6 h, +-1 h on 7 d). Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e48ed9cfacfed0131b77ed2f1992e8720720d387) * Keep the NodeInfo cache a superset of NodeDB (seed + keep-alive) The cache learned identities only from overheard NODEINFO frames, so its membership was opportunistic: a NodeDB-tier node whose frame was never heard this boot had no entry (no key pin to protect it, no name to rehydrate), and the retention TTL evicted entries for nodes that still lived in the hot store or warm tier. Add an anti-entropy pass to the maintenance sweep (reconcileNodeInfoMembershipLocked): every hot-store / warm-tier node gets an entry - full identity from the hot store (hasFullUser), key-only from the warm tier - with signer provenance inherited key-matched, and NodeDB's key adopted wholesale on conflict. Member entries (isMember) are re-stamped each sweep, so they never age toward the retention TTL; when a node leaves both tiers the keep-alive stops and its entry ages out from its final member re-stamp. Membership also outranks key trust in LRU victim selection, and the reconcile pass skips seeding rather than churn one member out for another when hot+warm exceeds the cache. Two properties are load-bearing: - seeding/keep-alive never touch obsTick/hasObserved, so a seeded or retained entry is never served as a spoofed reply - only genuinely heard frames make a node servable; - copyUser() now requires hasFullUser, so a key-only warm seed can never stamp HAS_USER onto a nameless node via name-rehydration. Native suite: 64/64. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 4f1b30d838b2f7b11173373b8f1ba97cf91bef6b) * Write-through NodeDB identity commits into the NodeInfo cache updateUser() is the single chokepoint through which every remote identity/key write enters NodeDB (key-verification keys stay in CryptoEngine's pending buffer until a NodeInfo carries them here), so one hook at its tail lets the NodeInfo cache reflect a commit immediately instead of waiting up to a minute for the reconcile sweep - which stays in place as the anti-entropy backstop. onNodeIdentityCommitted() upserts the full User: NodeDB's key is authoritative (a conflicting cached key is stale residue - replaced, provenance dropped), a keyless commit keeps an already-cached TOFU key, and signer provenance transfers only for the committed key. The observation stamp is never touched: knowledge is not observation, so a hook write can never make a never-heard node servable - covered by the new test alongside immediate copyUser/copyPublicKey visibility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e160b899728be8d951d4e7bdf5106c777832b917) * Purge TrafficManagement caches on explicit node removal NodeDB::removeNodeByNum() already forgets the node everywhere NodeDB owns - hot store, satellite stores, warm tier - but the TrafficManagement caches kept the deleted identity: the NodeInfo entry went on feeding the key pool (NodeDB::copyPublicKey) and name rehydration, and the unified slot kept its role/next-hop/dedup state, resurrecting the node on next contact. Add purgeNode(): clears both the unified cache slot and the NodeInfo cache entry, called from removeNodeByNum() alongside the warm-tier removal. Removal means full removal; passive eviction never calls this, and the reconcile sweep will not re-seed a node absent from both NodeDB tiers. Test: an observed identity plus a next-hop hint both vanish after removeNodeByNum(). Left for CI to execute per request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b5564c173337de99e15ef998a5c698550238fd3c) * Document the tmm-fix-2 / tmm-fix-superset reconciliation decisions Two parallel implementations of the same superset design are being combined on this branch. This decision matrix records every divergence and which side each reconciled feature takes, ahead of the code commits that apply them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile retention: no timed eviction, obsTick LRU, hourly seeding Adopt tmm-fix-superset's retention model (reconciliation decisions #3, #4, #9 in .notes/tmm-super-superset-reconciliation.md): - Entries are never evicted on a timer. The 7-day retention TTL and its third tick clock (retTick) are gone; wrap-safety rests entirely on the sweep's presence-bit saturation, and a quiet entry keeps its value (pubkey pool, name rehydration). Slots are reclaimed only by trust/membership-tiered LRU on insert - age scored by obsTick, with never-observed entries counting oldest - or by an explicit purge. - Membership refresh (entry -> NodeDB contains checks) runs every sweep; the heavy seeding pass runs at boot and then hourly, with the write-through hooks carrying immediacy in between. - Tick constants and helpers move into the header beside the existing UnifiedCache tick idiom, with static_asserts tying the tick windows to the fallback path's seconds/ms forms. Kept from tmm-fix-2 (decision #4): the warm tier is still seeded (key-only records via WarmNodeStore::entryAt) so the invariant remains cache superset-of hot AND warm, and the spareMembers guard still stops seeding from churning one member out for another when hot+warm exceeds the cache. Entry shrinks to 128 B (2000 entries = 256 kB). The TTL-based retention test is superseded by a no-timed-eviction test: a quiet keyed entry survives 9 days of sweeps while the serve gate saturates as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile hooks: key-commit write-through, purgeAll, key-matched signer Adopt tmm-fix-superset's hook coverage (reconciliation decisions #5-#8): - onNodeKeyCommitted(): write-through for the two key-write sites that bypass NodeDB::updateUser - the Router's admin-key learn (TOFU-grade) and KeyVerificationModule's manual-verification commit (proven=true, the strongest provenance the cache can carry). Both were coverage gaps in the tmm-fix-2 write-through design. - updateUser's hook call now transfers signer status key-matched (isVerifiedSignerForKey) instead of the node's bare signed flag, and runs on acceptance rather than only on change. - purgeAll(): factoryReset() and resetNodes() clear both TMM tables - removal-is-full-removal applies to bulk resets too, without relying on the usual post-reset reboot. - purgeNode() reuses the existing finders instead of open-coded scans and logs the (user-initiated) purge. - peekNodeInfoFlagsForTest(): flag introspection so saturation and membership tests can assert sweep effects directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Merge the two branches' test suites for the reconciled design Port tmm-fix-superset's unique tests, adapted to run natively under TMM_HAS_NODEINFO_CACHE (reconciliation decision #11): - keyHook_upsertsAndGovernsProvenance: TOFU learn -> manual-verification upgrade -> NodeDB-senior rotation resets provenance. - tickSaturation_sweepClearsObserved: the sweep saturates hasObserved past the serve window, the entry persists (no TTL), and a full 256-tick clock wrap cannot alias a saturated stamp back to fresh. - sweepMembershipMarking: reconciliation seeds a hot identity as member+fullUser+unobserved; the next sweep clears membership once the node leaves NodeDB, while the entry itself persists. tmm-fix-2's tests (warm pin, hot/warm seeding, updateUser write-through, removal purge - now also asserting the entry is gone via the new peek hook - and the no-timed-eviction rewrite) were already on this branch. Suite now counts 70 test functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Adapt cherry-picked and seeding tests to the reconciled semantics Two tests needed updating for interactions the validation run surfaced: - The #11035 test (ignoresUnsignedSignerIdentity) was written for a world without the native NodeInfo cache: it needs the NodeDB fallback path (dropNodeInfoCacheForTest) and the signed replay gate satisfied for its target, like the other fallback tests on this branch. - The hot-seed test's "observed frame makes it servable" step now sends a signature-verified frame: its node is a known signer, and per #11035 an unsigned frame from a signer must not (and does not) drive cache writes - the gate working as designed. Full native suite: 70/70 under ASan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * trunk fml * Doc tidyup * TrafficManagement: key NodeInfo-cache maintenance to its own guard runOnce() nested the entire NodeInfo maintenance block (tick-stamp saturation, membership refresh, boot/hourly reconcile) inside #if TRAFFIC_MANAGEMENT_CACHE_SIZE > 0, so a variant overriding the unified cache to 0 on a PSRAM board would compile the NodeInfo cache without its maintenance: hasObserved would never saturate, obsTick would alias past its 12.8 h wrap, and the 6 h staleness gate - whose wrap-safety explicitly depends on the sweep - would serve spoofed replies for long-gone nodes. Extract maintainNodeInfoCacheLocked(), guarded by TMM_HAS_NODEINFO_CACHE alone, and give runOnce() the same independent-guard structure purgeAll() already has. Also close the runtime cousin of the same mismatch: the write-through hooks (onNodeIdentityCommitted / onNodeKeyCommitted) now no-op while moduleConfig.has_traffic_management is off. Previously they kept filling the cache from NodeDB commits while runOnce() returned INT32_MAX, accumulating entries that were never swept or reconciled. Purges and reads stay ungated: removal must always work, and reads just miss an empty cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: forward throttled NodeInfo requests instead of consuming them The direct-response throttle returned true, which made handleReceived() STOP the request: within the 30 s window a repeat request was neither answered nor forwarded toward the genuine target, and the call site still logged "respond" and bumped nodeinfo_cache_hits for a reply that was never sent. A requester whose first reply was lost on a noisy link got silence for the whole window. Return false instead: the request flows through normal relay handling, so the genuine node (or another cache-holder) can answer, while our own spoofed TX stays bounded. Repeats of the same packet id are already absorbed by the router's duplicate detection, and the stats counter now only counts replies actually sent. Also log purgeNode() only when a slot was actually cleared - it runs for every NodeDB removal, including nodes the caches never tracked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: consolidate NodeInfo-cache #if sprawl into one region Fourteen scattered #if TMM_HAS_NODEINFO_CACHE guards (one per function, each with its own #else stub of (void) casts) collapse into a single guarded region holding every NodeInfo-cache-only function, terminated by one #else block of no-op stub definitions. Because the stubs now exist on every build, the call sites in runOnce(), purgeNode() and purgeAll() drop their guards too; inner guards remain only for orthogonal features (PKI, warm tier) and for real conditional work (constructor allocation, PSRAM paths in shouldRespondToNodeInfo). No behavior change. Compile-checked on both sides of the macro: the native app build (stubs) and the native unit-test build (region). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Router: resolve sender key only for PKI-decrypt candidates perhapsDecode() resolved the sender's public key unconditionally for every encrypted packet, before even checking whether the packet could be PKI-decrypted (channel 0, unicast to us). Since copyPublicKey() grew its TrafficManagement fallback tier, a full hot+warm miss - the common case for channel traffic from senders outside both NodeDB tiers - additionally walked the 2000-entry NodeInfo cache under its lock, per packet, for a key that was then discarded. Move the resolution inside the PKI-candidate branch; remotePublic and haveRemoteKey had no consumers outside it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: refresh NodeInfo membership hourly, not per-sweep The 60 s sweep re-derived isMember with a per-entry NodeDB lookup: O(entries x members) - about 700k node-number comparisons per minute on a full PSRAM cache (2000 entries x 250 hot + 100 warm), with strided PSRAM reads, all while holding cacheLock against the packet path. Move the refresh into the hourly reconcile pass, which is already O(members x entries): after the seeding loops (so the upsert pass still sees last pass's bits for spareMembers protection), clear every isMember bit and re-mark from both NodeDB tiers - including keyless warm records, which seed nothing but are still members. Accepted tradeoff, now documented on the field: membership lags a passive NodeDB eviction by up to an hour (the entry just stays LRU-sticky slightly longer). Additions stay immediate via the write-through hooks, explicit removals via purgeNode(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * NodeDB: centralize bare-key commits in commitRemoteKey() The two key-write sites that bypass updateUser() (admin-channel learn in Router::perhapsDecode, manual verification in KeyVerificationModule) each carried their own #if HAS_TRAFFIC_MANAGEMENT write-through boilerplate - the pattern this PR itself had to retrofit twice, and which any future direct key write would silently miss, leaving the TrafficManagement cache divergent until the next hourly reconcile. Add NodeDB::commitRemoteKey(n, key32, KeyCommitTrust): writes the key to the hot store and routes the TrafficManagement write-through in one place, with provenance explicit at the call site (AdminChannelProven maps to TOFU-grade, ManuallyVerified to proven). The bypass sites keep their reason for existing - bare-key commits with provenance that updateUser's User-payload/TOFU-pin path cannot express - but no longer know about TrafficManagement at all; both stale includes are dropped (Router's was already unused on develop). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: tick wrap-safety analysis, updated cadences, CodeRabbit doc fixes node_info_stores.md gains a "Tick clocks and wrap safety" section recording which mechanism keeps each modular tick clock honest: the unified-cache clocks pair the 60 s sweep with read-time window resets, the NodeInfo obs/resp clocks are sweep-only (hence the compile-time invariant that maintainNodeInfoCacheLocked() is guarded by TMM_HAS_NODEINFO_CACHE alone), and the warm tier is immune by design - absolute unix-seconds, no wrap until 2106. Also brought current with this series: membership refresh moved to the hourly reconcile, throttled direct-response requests forward instead of being consumed, the commitRemoteKey() bare-key funnel, and the module-disabled gate on the write-through hooks. CodeRabbit doc review (PR #11050): present the NodeInfo payload cache as the third *identity* tier with the unified cache beside the chain, and describe NodeInfoLite's flattened fields / satellite copy-out accessors instead of the removed nested members. Two stale docs/tmm_node_stores.md references in the header now point at the real file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: adapt tests to forwarded throttles and hourly membership Throttle tests (PSRAM and NodeDB-fallback paths): a request inside the 30 s window now CONTINUEs into normal relay handling instead of being consumed - assert no spoofed TX, no NAK suppression, and (PSRAM path) that nodeinfo_cache_hits counts only replies actually sent. Membership test (renamed reconcileMembershipMarking): the per-minute sweep no longer refreshes isMember, so the test now pins down both halves of the new contract - the very next sweep after a passive NodeDB eviction still shows the stale member bit (the documented up-to-an-hour lag), and a reconcile interval's worth of sweeps clears it. Disabled-module test additionally proves the write-through hooks share the has_traffic_management gate: a key commit while disabled must not land in the NodeInfo cache. Not covered here: a build permutation with TRAFFIC_MANAGEMENT_CACHE_SIZE overridden to 0 (the configuration the maintenance-guard fix protects) would need its own PlatformIO env plus guards on every unified-cache test - left as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * expand test coverage * nitpicks and docs update * more comment fixes * nitpicks * test: make TMM fixture cleanup abort-safe in tearDown() Several tests reset per-test global state (trafficManagementModule pointing at a stack `module`, owner.public_key, the RTC fake clock) only after their assertions. A TEST_ASSERT_* failure longjmps out and skips that trailing cleanup, leaving the state to dangle into later cases - concretely, the next setUp()'s resetNodes() would call trafficManagementModule->purgeAll() on a destroyed object. Move the resets into tearDown(), which runs unconditionally between tests, and drop the now-redundant per-test cleanups. Addresses a CodeRabbit review comment on PR #11050. clod helped too * docs tidy * Throttle NodeInfo direct responses A direct response is addressed to the requesting packet's from field, which is unauthenticated, and is sent by every neighbour that holds the target in cache. One request therefore makes several nodes transmit at an address the requester chose, and nothing limited how often that could be repeated. Replies are now spaced per requester, which bounds how much any single node can be made to receive, plus a global floor on how much airtime the feature can consume. The check sits where a reply is about to be sent, so requests declined for other reasons do not consume the budget. * test: reconcile TMM direct-response tests with #11104 throttle Makes the suite green for now. #11104's per-requester + global-floor throttle (60 s) is layered over the branch's per-target throttle, so the three existing "served again" checks (psram, fallback, sweep) now use a fresh requester to avoid the per-requester window masking the mechanism each one actually exercises. Adds a dedicated test for the new per-requester + 1 s global-floor behaviour (the reflector-flood gap the per-target throttle leaves open, and the bound that survives on non-PSRAM builds). Deferred: the branch's now-redundant fallback global stamp (nodeInfoFallbackLastResponseMs) is left in place as a follow-up cleanup. Clod wants credit. * TMM: unify direct-response throttles into per-sender + per-target RAM tables Replace the two path-specific NodeInfo-spoof throttles (per-entry respTick on the PSRAM cache path, single global stamp on the fallback path) with three symmetric bounds that behave identically with and without PSRAM: - per requester (60 s): how much any one node can be made to receive; - per target (60 s): how often we vouch for the same identity; - global floor (1 s): total airtime, the backstop once an attacker cycles requester/target past the 8-slot tables. Both axes are fixed 8-slot LRU tables in internal RAM (not the PSRAM NodeInfo cache), compared by wrap-safe uint32 ms subtraction, so there is no tick clock and no sweep to maintain. directResponseAllowed() resolves both slots before stamping either, so a reply one axis throttles never consumes the other's budget, and records the send itself. Retires: nodeInfoFallbackLastResponseMs, kNodeInfoResponseThrottleMs, nowStampMs, the respTick byte + hasResponded bit on every cache entry, currentRespTick/kNodeInfoRespTickMs/kNodeInfoThrottleTicks, and the sweep's respTick clear. Renumbers peekNodeInfoFlagsForTest (drops the responded bit). Tests: the psram/fallback throttle tests now assert the per-target axis at 60 s, isolated via a different requester, proving it holds with and without PSRAM; perRequesterAndGlobalFloor isolates the other two axes; the obsolete respTick wrap-safety test is removed. 83/83 native cases pass. clod helped too * whats up, doc? * trimming the comments * test: bump native-suite-count to 38 after upstream rebase Upstream develop carries 38 test_* suite directories but its native-suite-count file still reads 37 (a suite was added without bumping the count). Rebasing onto develop inherits that stale file, so the runner flags AMBER. Correct the registered total to 38. clod helped too --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>	2026-07-21 12:19:43 +02:00
Thomas Göttgens	40e583c102	Limit node database churn from unauthenticated packets (#11102)	2026-07-21 09:09:53 +02:00
Thomas Göttgens	5ded0ec8c5	Gate identity learning on signature in NodeDB::updateUser (#11084)	2026-07-20 11:56:42 +02:00
Thomas Göttgens	7a25ffd0a2	fix: evaluate pre-hop hop_start bitfield guard post-decode instead of before decryption (#10758)	2026-07-11 12:10:25 -05:00
Ben Meadors	8d20606203	feat(config): make position & telemetry broadcast opt-in (#10929) ... Position sharing was opt-out (the default primary channel shipped position_precision=13) while device telemetry was already opt-in, and a normal firmware upgrade preserves saved config, so existing nodes stayed position-on. This makes both broadcasts opt-in, both on a fresh flash and via a one-time migration for upgrading nodes. - Fresh default: initDefaultChannel now sets position_precision=0. - One-time migration in loadFromDisk, gated on a dedicated POSITION_TELEMETRY_OPTIN_VER (26) watermark kept separate from DEVICESTATE_CUR_VER (bumping that would re-run the NodeDatabase v24 legacy decoder on already-migrated v25 DBs): disables position broadcast on PUBLIC/default-PSK channels and forces every device-telemetry mesh-broadcast flag plus the MQTT map-report location off. - Private-PSK channels are left untouched: a channel with a custom key is a deliberate trusted-group setup, so its configured precision is preserved. - Idempotent: ordinary saves never re-stamp .version, so a user who re-enables sharing is not re-disabled on the next boot. - Added pure channelFileUsesPublicKey() (operates on the raw ChannelFile, since the channels singleton isn't initialized during loadFromDisk) and refactored Channels::usesPublicKey() to delegate to it. - New test/test_optin_migration native suite (14 cases).	2026-07-07 14:58:58 -05:00
Tom	3becaf2d95	emdashes begone (#10847)	2026-07-01 19:01:27 -05:00
Ben Meadors	0488a46a3c	fix: stop unexpected NodeNum regeneration from PKI key loss (#10808) ... * fix: stop unexpected NodeNum regeneration from PKI key loss A node's NodeNum is derived from its PKI public key (my_node_num == crc32(public_key)), so it changes only when the keypair is regenerated -- which happens only when a keygen path runs while config.security.private_key.size != 32. Two paths could trigger that without the user intending a new identity: 1. Boot: loadFromDisk() collapsed every non-success loadProto() result for the config file into installDefaultConfig() (preserveKey=false), which memset()s config and wipes the private key. loadProto() distinguishes DECODE_FAILED (file present but undecodable/undecryptable) from OTHER_FAILURE (absent), so a transient/corrupt read silently minted a new identity. A new configDecodeFailed flag now gates the boot keygen and the boot config-save directly (not via region, so it survives the userprefs/region overrides that run later in loadFromDisk): identity is frozen, the unreadable file is left intact for a later clean boot to recover, and the node boots radio-silent (region UNSET, TX off). A genuinely-absent config still gets defaults + keygen. 2. AdminModule security SET: config.security = c.payload_variant.security wholesale-clobbered the keypair, then regenerated whenever the incoming private_key.size != 32. A client editing an unrelated security field without round-tripping the private key would re-mint identity. The existing keypair is now preserved when the incoming config omits it; first provisioning and key import are unchanged. Intentional reset goes through factory_reset. Native PlatformIO unit suite (Docker): 499/499 test cases pass. * test: cover security keypair preservation; clarify load-failure comment Addresses PR review feedback: - Add native unit tests (test_admin_radio) asserting handleSetConfig preserves the existing identity keypair when a security SET omits the private key, and applies a full supplied keypair (key import). Guards against reintroduced NodeNum/keypair regeneration. AdminModuleTestShim (now a friend) defers saves so the lightweight harness doesn't saveToDisk an uninitialized node database. - Clarify the non-DECODE_FAILED config-load comment: OTHER_FAILURE / NO_FILESYSTEM cover an absent or unopenable file, not just first boot.	2026-06-28 18:48:24 -05:00
TomandClaude Sonnet 4.6	c51c01607d	Traffic Management module: dedup, rate limiting, role-aware policing (#10706) ... Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and next-hop foundations it builds on: - Unified per-node cache (flat array, 8-bit relative ticks) shared by all features; role-aware throttles for tracker / lost-and-found. - Position deduplication: drop unchanged position rebroadcasts within a configurable interval; precision driven off the channel ceiling (clamped to the public-key max on well-known channels). Enabled by default at 11h. - Per-node rate limiting and unknown-packet filtering (config-driven; a non-zero companion field enables each feature -- no bool toggles). - NodeInfo direct response from cache with role-based hop clamps. - Persistent next-hop overflow store: confirmed hops have no TTL, are seeded from NodeInfoLite at boot, and survive hot-store eviction. - Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM cache). Role is cached write-time (seeded on first track, refreshed from NodeInfo), pins its cache entry like a next-hop hint, and is evicted last. - Warm store caches device role + protected category across reboot/eviction. - PositionModule stationary floor for tracker / lost-and-found. - PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded. Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually used; submodule repointed to protobufs develop. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-06-20 20:35:14 -05:00
3 people	22072c5f4b	Pr1.5 tmm nexthop (#10745) ... * TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-06-19 19:52:58 -05:00
4 people	79a7dcc46c	Pr1 nodedb warmstore (#10705) ... * NodeDB: 3-tier node store with persistent warm tier (long-tail identity retention) Introduces a tiered NodeDB so the device retains identity (public key, last_heard) for far more nodes than fit in the full-record hot store, without growing heap or the persisted nodes.proto unboundedly. - Hot store: full NodeInfoLite, MAX_NUM_NODES (120 on nRF52). - Satellite maps: position/telemetry/environment/status capped at MAX_SATELLITE_NODES (40 freshest); eviction via enforceSatelliteCaps / evictSatelliteOverCap. - Warm tier (WarmNodeStore): 40 B {num,last_heard,public_key} records for evicted nodes so DMs to/from long-tail nodes keep encrypting/decrypting. Persisted to /prefs/warm.dat, or on nRF52840 a dedicated 12 KB raw-flash record-ring below LittleFS (3x4 KB pages; see linker scripts + the nrf52_warm_region.py post-link guard). NodeDB::getOrCreateMeshNode now demotes evicted nodes into the warm tier and re-admits them (restoring key/last_heard). Router PKI decrypt/encode resolve the peer key via NodeDB::copyPublicKey (hot store, then warm tier). NodeInfoLite gains snr_q4 (sint32, Q4-encoded dB); the float snr is zeroed on disk. NodeInfoLite grows 105 -> 112 B; backup 2432 -> 2468 B. Note: the snr_q4 .proto change still needs to land in the protobufs submodule (generated header is updated here; submodule pointer left at upstream). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * NodeDB: robust receive + retention for blocked (ignored) nodes Hardens how ignored/favourite nodes are received over admin and retained, closing paths where a block could be lost or accidentally cleared. - Blocking keeps the node's public key (admin set_ignored_node and addFromContact no longer zero it / drop the warm-tier key), so a blocked peer stays a verifiable identity. - set_ignored_node creates the node if absent, so a block by node ID sticks even for a node we've never heard from (e.g. pushed by a remote admin) with no NodeInfo or key. - Eviction protection (favourite/ignored/manually-verified) now also applies to the load-time hot-store migration and is never undone by cleanupMeshDB, which previously purged ignored nodes that lacked user info. - The hot-store migration leaves our own node (index 0) in place and prefers to demote non-protected nodes, like the runtime eviction scan. Caps the protected set (favourite + ignored + verified) at MAX_NUM_NODES-2 via NodeDB::setProtectedFlag(), so at least two evictable slots always remain and getOrCreateMeshNode can always make room — replacing the previous unconditional append that could run off the end of the node vector when every node was protected. A locally-set favourite/ignore that hits the cap reports back to the phone via a ClientNotification. Adds test_nodedb_blocked covering the migration, favourite/ignored eviction protection, ignored-survives-cleanup, and the protected-node cap. The maintenance methods stay private in production; the test reaches them through a PIO_UNIT_TESTING-guarded friend shim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> # Conflicts: # src/mesh/NodeDB.h * fix copilot comments * once again * WarmNodeStore: fix cppcheck warnings (uninitvar, constVariablePointer) Zero-initialise `stranded[]` and `seqs[]/order[]` VLAs so cppcheck can verify there are no unguarded reads of uninitialised memory (the guards exist but are not visible to static analysis). Mark two local pointers `const` where the pointed-to entry is never mutated after assignment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * self-care added to assist 2.7 and 2.8 nodedb migration * Tidy warm-store/self-care: comments, guards, log + flash cleanup Style/cleanup pass over the branch (no behavior change except the noted preprocessor simplifications, which are semantically identical): - Comments: move function descriptions to the headers, cap in-function comments at ~3-4 lines, drop leading-number step markers, label stacked #endif blocks, de-decorate banner comments. - dumpToLog: fully gate decl + definition + AdminModule call site behind MESHTASTIC_NODEDB_MIGRATION_VERBOSE so it compiles out when disabled (~1.2 KB when off). - mesh-pb-constants: drop the dead nRF52832 WARM_NODE_COUNT branch and trim the macro docs. - WarmNodeStore: simplify the redundant `ARCH_NRF52 && NRF52840_XXAA` guards to `NRF52840_XXAA`, add a kNoPage sentinel for the ring page state. - Shorten the always-on LOG_WARN strings (~120 B flash). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * more tidying up, aligning with docs and undoing other-arch regressions * Update protobufs (#19) Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com> * made the migration pathway cleareer * address copilot review * fixed a copilot review on a downstream PR. * Address Copilot review comments for PR #10705 (warmstore/nodedb) - WarmNodeStore.h: default MIGRATION_VERBOSE to 0 (suppress info-level chatter on production builds; opt in with =1) - WarmNodeStore.cpp load(): move memset to top of function so all failure paths (header-read fail, invalid header) leave entries clear - WarmNodeStore.cpp save(): replace manual spiLock lock/unlock around mkdir with LockGuard covering the full SafeFile sequence, matching the lock discipline in load() - Router.cpp: memcpy(&p->public_key.bytes, ...) -> memcpy(p->public_key.bytes, ...) — pass decayed uint8_t* rather than pointer-to-array - AdminModule.cpp: check setProtectedFlag return for PKC auto-favorite; log cap-refusal warning instead of unconditional "auto-favoriting" - nrf52_warm_region.py: error message references both v6.ld and v7.ld Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * NodeDB: formatting cleanup (blank lines after preprocessor blocks) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Lukewarm store --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2026-06-18 08:41:34 +01:00
6 people	8267bb22bd	Packet Signing via XEdDSA (#10478) ... * Test commit for XEdDSA support * Update to Crypto lib in Meshtatic org * Generate a new node identity on key generation (#7628) * Generate a new node identity on key generation * Fixes * Fixes * Fixes * Messed up * Fixes * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Figured it out! * Cleanup * Update src/mesh/NodeDB.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update crypto commit hash * Some fixes for xeddsa pr (#9610) * fix: add null check for getMeshNode() in NodeInfoModule getMeshNode() can return nullptr for unknown nodes. Dereferencing without a check crashes the firmware when receiving NodeInfo from a node not yet in the database. * fix: enforce XEdDSA signature verification and prevent stripping Previously, failed signature verification still allowed the packet through, making signatures purely cosmetic. Now: - Failed verification drops the packet (DECODE_FAILURE) - Successfully verified nodes get HAS_XEDDSA_SIGNED bitfield set - Unsigned packets from previously-signing nodes are rejected - Log levels reduced from WARN/ERROR to DEBUG/WARN as appropriate * fix: include packet metadata in XEdDSA signature The signature now covers [fromNode | packetId | portnum | payload] instead of just the payload bytes. This prevents: - Replay attacks (different packetId fails verification) - Reattribution (different fromNode fails verification) - Portnum redirection (different portnum fails verification) Also adds a key initialization check to xeddsa_sign (returns false if XEdDSA keys are all zeros) and checks the return value in the encode path. * fix: handle existing key pair in AdminModule security config When a user provides both a valid private key and public key via admin config, the crypto engine's DH private key and owner public key were never loaded. DMs and XEdDSA signing would silently break. Add an else branch to load both keys into the crypto engine. * perf: cache Ed25519 public key conversion in xeddsa_verify curve_to_ed_pub() performs field element parsing, inversion, and multiplication on every call. Since packets from the same node tend to arrive in bursts, a single-entry cache avoids repeating this expensive conversion for consecutive packets from one sender. * fix: skip identity cleanup when node number is unchanged createNewIdentity() was called on every generateCryptoKeyPair(), including normal boots where the same key is regenerated. This caused unnecessary NodeDB writes and old-node cleanup logic to run when the node number hadn't actually changed. Also fixes only zeroing byte[0] of the old node's public key instead of clearing the entire array. * fix: replace hardcoded 120 with derived XEDDSA_SIGNATURE_SIZE constant The payload size check for XEdDSA signing used a magic number (120). Replace with a derivation from DATA_PAYLOAD_LEN and XEDDSA_SIGNATURE_SIZE so the limit adjusts automatically if constants change. This also increases the max signable payload from 120 to 169 bytes, which is still safe since the actual encoded size is checked after pb_encode. * fix: add const qualifiers to XEdDSA verify and curve_to_ed_pub inputs pubKey, payload, and signature parameters in xeddsa_verify are input-only and should not be modified. Same for curve_pubkey in curve_to_ed_pub. * chore: remove commented-out old Crypto dependency in portduino.ini * Leave out the admin module change for now --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> * trunk * protobuf re-update * Protobufs * Merge resolution fix * Put XEDDSA on the right bit * NodeDB update to new nodeInfoLite accessors, etc * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Refine unsigned packet rejection logic in Router (#10534) * use hardware random to fill the first 32 signature bytes with entropy prior to signing. * Add XEdDSA packet-signing policy tests and update dependencies for macos * Minor fixes * integrate XEdDSA support and update dependencies across multiple modules --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Wessel <github@weebl.me> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>	2026-06-13 06:45:56 -05:00
3 people	07a87a8254	security: runtime-toggleable MESHTASTIC_LOCKDOWN hardening for nRF52 (#10349) ... * security: add MESHTASTIC_LOCKDOWN hardened build option Meshtastic nodes ship with secrets on flash (channel PSKs, the device private key, admin keys, wifi PSK) and over-the-wire access to admin APIs that can re-key the mesh. Lose the device, at a border crossing, in a raid, off a backpack, and an attacker reads everything in 30s with a USB cable. There's no at-rest encryption, no client auth, the screen leaks contents, and SWD is wide open. This adds an opt-in hardened build for users who care. -DMESHTASTIC_LOCKDOWN=1 on nRF52 (CC310) turns on: DEBUG_MUTE silence USB/serial logs MESHTASTIC_ENCRYPTED_STORAGE AES-128-CTR + HMAC-SHA256 on LocalConfig / channels / NodeDB. Passphrase-gated DEK, TTL/boot unlock token, failed-attempt backoff (within-boot, wall-clock, persisted bootsSinceFail). MESHTASTIC_PHONEAPI_ACCESS_CONTROL per-connection auth gate. Secrets emitted as empty proto structs to unauthenticated clients. MESHTASTIC_ENABLE_APPROTECT one-way UICR APPROTECT, reset applied same boot. Recoverable only via \`nrfjprog --recover\`, which also wipes the DEK. LockdownDisplay screen shows "LOCKED" when locked or idle 30s. OLED only; InkHUD / niche / device-ui not yet wired. Wire format is the LockdownAuth / LockdownStatus pair from meshtastic/protobufs#911 (AdminMessage tag 104, FromRadio tag 18). Access-control state is a file-scope 6-slot table in PhoneAPI.cpp keyed by \`this\`, not class members. Adding *any* per-instance field to PhoneAPI breaks USB-CDC enumeration on the current nRF52 Adafruit framework, one volatile bool was enough. Out-of-line side-steps it. lockdown_auth is handled synchronously in PhoneAPI::handleToRadioPacket rather than routed through the mesh Router into AdminModule. Two reasons: the passphrase never travels through a routed MeshPacket queue, and per-connection authorization runs while \`this\` is still on the call stack. The previous async-via-router design lost connection identity (g_currentContext was null by the time AdminModule processed the auth), so per-connection unlock never actually took effect on the originating client. Non-nRF52: #warning, only DEBUG_MUTE activates. tools/lockdown_provision.py drives provision / unlock / lock-now / watch over USB. Display privacy is a screen-lock latch separate from storage-lock state: shouldRedactDisplay() is true when storage is locked OR the latch is set. Screen::setOn(false) sets the latch when the stock idle timeout powers the display off (reusing config.display.screen_on_secs, no second timer); it is cleared only when a client authenticates with the passphrase. A device idling on the mesh keeps routing but hides its screen until re-auth; button input wakes the backlight to the LOCKED frame, not content. The earlier lockdown-specific 30s idle timer is removed — it duplicated PowerFSM idle detection and showed a misleading LOCKED screen on a merely-idle device. Unlock-token TTL fix: a token carrying both a boot-count and a wall-clock TTL is no longer destroyed when the RTC is invalid at cold boot. The boot count is independently verifiable without a clock, so the token falls back to boot-count enforcement instead of being deleted. A token is only hard-rejected when its wall-clock TTL can be evaluated and is found expired. NodeDB::reloadFromDisk() after unlock is deferred to the main loop via lockdownReloadPending rather than run inline on the transport callback stack — the reload is too heavy for the BLE/serial task stack and was resetting the device immediately after a successful unlock. The screen-lock latch also swallows local input events in InputBroker::handleInputEvent while it (or storage-locked) is set. Without that, a blind operator could drive on-device menus, fire canned messages, or change settings through the joystick/buttons even though the screen content was hidden. PowerFSM is still triggered first so the backlight wakes to the LOCKED frame; the event is dropped before reaching the UI observers. The screen-lock latch is initialised to true at boot, so even a token-auto-unlocked cold boot comes up redacted. Otherwise an attacker holding a screen-locked device could power-cycle it (the RAM latch resets) and recover a content screen. After any boot, the operator must authenticate from a client to reveal screen content. MyNodeInfo.device_id is also redacted for unauthenticated clients — it is a stable hardware identifier useful to an attacker for fingerprinting / correlating the device across observations. The public mesh fields (my_node_num, owner short/long name, public key, hw model) are left as-is because they are already broadcast on-mesh. ModuleConfig.mqtt is also redacted for unauthenticated clients — MQTTConfig carries broker username, password, server address, and root_topic. The empty MQTTConfig is emitted via the same zero-init pattern as the other gated sections. Uptime-based session limit (MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS) caps how long a single auto-unlocked session can hold storage open, measured in firmware millis() since unlock. 0 = unlimited (existing token-only behavior, suitable for tower/infra nodes); non-zero arms a timer on every passphrase unlock and on every token-auto-unlock that inherits the value, since the cap is persisted in the token (token format bumped to v2: adds sessionMaxSeconds, body 56→60 bytes). On expiry the device revokes per-connection auth, re-engages the screen-lock latch, and reboots WITHOUT deleting the token. Next boot auto-unlocks via the boot count (decrementing it) and arms a fresh session window. Hard exposure ceiling: bootsRemaining * sessionMaxSeconds. Explicit user Lock Now still deletes the token (passphrase required to recover); only session expiry preserves it. Why uptime, not wall-clock: getValidTime() is fed by GPS/RTC/client time pushes — all manipulable by an attacker with the device (GPS spoof to roll the clock back, pull the RTC backup cell, Faraday-cage the whole thing). millis() comes off the Cortex-M's internal cycle counter, sealed inside the chip; the only way to reset it is a reboot, which costs a boot from the on-flash token counter. APPROTECT remains the load-bearing defense against forging higher boot counts via SWD. A future LockdownAuth.max_session_seconds proto field will let the client set this per-token; until that lands the build-time MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS macro is the only source. Session expiry now decrements the on-flash boot count in place and re-arms the uptime timer WITHOUT rebooting, while budget remains. Mesh routing keeps running across session boundaries; the device only reboots when bootsRemaining reaches zero (rollback budget exhausted), at which point it hard-locks and forces passphrase re-entry. Each session boundary still: revokes per-connection admin auth so clients must re-authenticate to see content, re-engages the screen lock latch, and emits LockdownStatus{LOCKED, needs_auth, boots=N} so connected clients see the decremented count and know to re-auth. Storage stays unlocked (DEK in RAM) for continuity. The boot count's role as the rollback ledger is unchanged — it decrements monotonically once per session boundary, whether the session ends in a reboot or an in-place roll. Attacker who power- cycles to dodge the session timer still pays a boot via the existing readAndConsumeToken decrement-at-load path. APPROTECT remains the only defense against forging higher counts. Net effect for an unattended/tower node with bootsRemaining=50, sessionSeconds=3600: 50 hours of continuous mesh service, one reboot at the end, vs. the previous design's 50 reboots over the same period. Same exposure ceiling, far better uptime. LockdownAuth.max_session_seconds (proto tag 5) is now consumed: when non-zero the client value wins; 0 falls back to the firmware-side MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS, matching the boots_remaining sentinel convention. Protobufs submodule pin bumped to develop tip which contains meshtastic/protobufs#916 (merged). * security: drop dead is_managed allowlist for set_config(security).private_key The 'isLockdownSecurityCmd' allowlist in handleReceivedProtobuf dates from the pre-LockdownAuth design when the passphrase was smuggled through SecurityConfig.private_key. With lockdown_auth handled synchronously in PhoneAPI::handleToRadioPacket before any admin message reaches the Router, this allowlist now serves no legitimate purpose and lets an unauthenticated local client mutate security settings on a managed device by setting private_key.size>=1 — including potentially disabling is_managed itself. Remove the allowlist. Managed-mode local admin now requires a PhoneAPI connection that has already authenticated via lockdown_auth (or, on the pki_encrypted branch below, a valid PKC admin key). Resolves Copilot review feedback on src/modules/AdminModule.cpp:105. * security: protect lockdown-status drain slot from concurrent writers g_pendingLockdownStatus / g_hasPendingLockdownStatus are written from multiple call sites (PhoneAPI::handleLockdownAuthInline on the BLE/USB transport callback, AdminModule on the Router thread, main loop session expiry) and read in getFromRadio() on whichever transport is draining FromRadio. The struct read/write was unprotected, so a writer could corrupt the slot mid-encode. Same pattern as nodeInfoMutex — wrap both the queue path and the drain in a small lock. Drain re-checks the bool under the lock to handle the case where another reader grabbed the slot first. Resolves Copilot review feedback on src/mesh/PhoneAPI.cpp:1560. * security: derive readAndDecrypt size cap from caller buffer, not a hardcoded 64 KB The MAX_PROTO_FILE_SIZE = 65536 + OVERHEAD ceiling was an absolute constant chosen against a since-outdated assumption that 'meshtastic proto files are well under 64 KB'. On variants where MAX_NUM_NODES pushes the serialised NodeDatabase past 64 KB the legitimate file gets rejected at load and the device treats its own real config as corrupt. The caller already knows the maximum plaintext it expects (outBufSize). Cap the ciphertext at outBufSize + OVERHEAD instead — this is the tightest sound bound (anything larger could not possibly decode into the caller's buffer), still defends against OOM / integer overflow, and scales with the platform's actual NodeDB size rather than an arbitrary constant. Resolves Copilot review feedback on src/security/EncryptedStorage.cpp:1327. * docs: fix stale 'passphrase delivery via AdminModule' references in configuration.h The lockdown overview comment block was written when passphrase delivery ran through AdminModule's handleReceivedProtobuf. With the synchronous refactor that path now lives in PhoneAPI::handleLockdownAuthInline, called before the admin message reaches the Router. Update both the nRF52 feature list and the non-nRF52 degraded-mode rationale to point at the current code path. Resolves Copilot review feedback on src/configuration.h:578 (and :604). * docs: refresh unlock-token format doc to match v2 layout The header comment for the UTOK file still described v1 (version 0x01, no session_max_seconds, 71 bytes) even after the in-flight bump to TOKEN_VERSION=0x02 and TOKEN_TOTAL_SIZE=75. The inline body-size breakdown comment was also wrong (claimed 39 bytes and mismatched the real NONCE_SIZE/AES_KEY_SIZE constants). Rewrite both to match the actual on-flash layout and note how v1 tokens are handled on upgrade (rejected via the version byte; passphrase re-entry mints a v2). Resolves Copilot review feedback on src/security/EncryptedStorage.h:50. * docs: correct session-limit comment re: token-auto-unlock behavior The s_sessionMaxMs comment block claimed 'token-auto-unlocked sessions have no session timer (the session feature is a passphrase-unlock-only knob)'. Stale: readAndConsumeToken() now persists sessionMaxSeconds in the token file and re-calls setSession() from the token-load path, so token-auto-unlocked sessions DO inherit the same cap (and consumeSessionBoot() re-arms in place between sessions on a single boot). Update the comment to match. Resolves Copilot review feedback on src/security/EncryptedStorage.cpp:72. * docs: clarify input-swallow gate re: screen-lock latch vs storage state The previous comment said input is swallowed 'until a client authenticates and unlockScreen() clears the latch (or storage is unlocked)'. The parenthetical was misleading: storage being unlocked is not in itself enough to clear the latch — the latch persists across the storage-unlocked-but-screen-locked steady state, and only an explicit unlockScreen() (called from a successful passphrase auth path) clears it. Reword so the only-passphrase-clears-the-latch invariant is explicit and local input is named as something that does NOT clear it. Resolves Copilot review feedback on src/input/InputBroker.cpp:134. * docs: fix reloadFromDisk() trigger comment in NodeDB.h The header still claimed reloadFromDisk() is called by AdminModule after a successful passphrase op. With the synchronous PhoneAPI refactor the actual trigger is PhoneAPI::handleLockdownAuthInline setting lockdownReloadPending, with main.cpp's loop() dispatching the heavy reload on the main thread (the transport callback stack isn't large enough). Update the comment to point at the real path and explain why the deferral exists. Resolves Copilot review feedback on src/mesh/NodeDB.h:393. * style: clang-format lockdown sources Apply trunk clang-format (16.0.3) to satisfy the format check. * style: black-format lockdown_provision.py Satisfy the trunk black formatter check. * security: drop unused v1 EncryptedStorage formats and migration This storage layer has never shipped, so there are no v1 DEK files, v1 unlock tokens, or v1 backoff records anywhere to stay compatible with. Remove the dead compatibility machinery: - legacy init() (FICR-only KEK, no passphrase) — had no callers - deriveKEKv1() / loadDEKv1() and the v1->v2 DEK migration paths in provisionPassphrase() and unlockWithPassphrase() - the 5-byte v1 backoff file format Also drop the now-pointless version byte from the on-disk MENC, MDEK, and UTOK formats. Each is identified by its 4-byte magic (and, for the keyed formats, its HMAC); with only one version that will ever exist, the version field added nothing. Sizes shrink by one byte each (overhead 54->53, DEK 66->65, token 75->74). Rename the surviving helpers to drop the _v2 suffix (deriveKEK, loadDEK, saveDEK, KEK_DOMAIN). No behavioral change for provisioning, unlock, token consumption, or session handling. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): harden auth-table and lockdown_auth handler (audit) Audit findings addressed: C3 — `~PhoneAPI()` now clears its auth slot unconditionally. The previous slot-clear in `close()` was gated on `state != STATE_SEND_NOTHING`, so a PhoneAPI that never reached config (or that already closed) left `slot.who` pointing at freed memory; a future PhoneAPI heap-allocated at the same address would inherit the prior session's authorization through `findOrAllocSlot`. C4 — All access to `g_authSlots`, `g_authEpoch`, and `g_currentContext` is now serialised through `g_authSlotsMutex`. Previously these were touched without locking from BLE/USB/TCP/Router tasks, so two parallel slot scans could hand out the same slot and mid-update reads could observe authorized=true alongside a stale epoch. Granularity is fine — every critical section is a short linear scan over six entries, and getFromRadio (which calls `getAdminAuthorized()` per redaction check) tolerates the brief blocking. A4 / H1 — `lock_now` now requires the originating connection to be already authorized. Previously any unauthenticated client (BLE/USB/TCP) could submit `lockdown_auth { lock_now=true }` and force a reboot, which was a trivial local-presence DoS — an attacker near the radio could brick-loop it indefinitely. The original "panic button without auth" property is dropped; panic now requires the operator to have passphrase-unlocked the connection. H2 — Empty-passphrase `lockdown_auth` (with `lock_now=false`) used to silently return success. The client received no feedback distinguishing that case from a real success, and an attacker could probe lockdown state for free. Now emits UNLOCK_FAILED with no backoff increment (empty-passphrase is more likely a client bug than an attack, but the honest signal still lets the client correct itself). H14 — `la.boots_remaining > 255` previously truncated silently (256 → 0 → mapped to TOKEN_DEFAULT_BOOTS=50; 257 → 1). Honest clients could not detect the misbehavior. Now rejected explicitly with UNLOCK_FAILED. L1 — The `to == nodeDB->getNodeNum()` allowance in the unauth ToRadio gate now also requires `getNodeNum() != 0`. During the locked-default boot path `getNodeNum()` returns 0, so a packet with `to=0` could otherwise satisfy the equality and bypass the gate. L2 — Comment added on `g_authEpoch` wrap. Practically unreachable (2^32 lockNow events on one boot), but worth recording the behavior. M17 — `findOrAllocSlot_LH` now evicts the first unauthorized stale slot when the table is full of non-nullptr entries, rather than failing closed. Authorized slots are never evicted — they represent live operator sessions. Fail-closed (with LOG_WARN) only when every slot holds a different live authorized PhoneAPI, which would require seven simultaneous authed connections. M18 — `s_screenLocked` is now `std::atomic<bool>` with relaxed ordering. Plain bool happened to work on single-core Cortex-M4 today but breaks silently if lockdown ports to ESP32 / RP2040, or under LTO whole- program elision. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): gate every admin op on per-connection auth + storage unlock Audit findings addressed: H6 — Unauthenticated local clients could previously set_config / set_module_config / set_channel etc. on a lockdown device whenever is_managed was unset. The previous gate inside AdminModule's is_managed branch consulted PhoneAPI::isLocalAdminAuthorized(), which reads a global g_currentContext set during synchronous PhoneAPI dispatch — but AdminModule runs on the Router task, by which time the dispatch task has exited and the global is unrelated to the originating connection. The check was both broken (always false on Router, so even authed clients were rejected) and unsafe (when it did fire, the wrong connection could be authorized). The fix relocates the gate to PhoneAPI::handleToRadioPacket, where dispatch is synchronous and getAdminAuthorized() can be trusted. The admin payload is already decoded there to extract lockdown_auth; extend the same branch so that any non-lockdown_auth admin variant from an unauthorized connection is dropped before ever reaching the Router queue. H7 — Same root cause: get_config_request / get_module_config_request / get_channel_request handlers returned full security/network/mqtt content to unauthorized local clients. With the H6 gate in PhoneAPI, these requests never reach AdminModule, so handleGetConfig / handleGetModuleConfig / handleGetChannel are only callable from authorized connections. H9 — Remote admin (PKC-authorized peers, mesh-relayed admin) bypassed lockdown entirely. If admin_keys were baked in via USERPREFS or set on a prior unlocked boot, a remote attacker could drive factory_reset / set_config against a locked device before the operator ever unlocked it. Added an EncryptedStorage::isUnlocked() early-return at the top of AdminModule::handleReceivedProtobuf. The local lockdown_auth path is unaffected because PhoneAPI handles it synchronously before AdminModule runs. H10 — Removed g_currentContext, the ContextGuard, authorizeLocalAdmin(), and isLocalAdminAuthorized() entirely. The audit's race (Router-thread reads a pointer set by an unrelated parallel dispatch and authorizes the wrong PhoneAPI) and the always-false-on-Router behavior both disappear with the code that produced them. The PKC-admin auto-authorize path is gone — PKC admin and the per-connection lockdown auth are now independent: clients using PKC admin from a local app must also send lockdown_auth to unlock the redacted FromRadio stream. Cleaned up AdminModule's is_managed branch: under lockdown the PhoneAPI-layer gate has already done its job, so no additional check is needed; without lockdown the legacy is_managed-blocks-plain-admin semantics are preserved. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): hold radio silent until storage is unlocked Audit finding H8: while locked, the device beaconed nodeinfo and telemetry on the public LongFast default PSK and routed incoming default- channel packets through the locked router. The locked-default boot path in NodeDB::loadFromDisk installs config via installDefaultConfig, which honours USERPREFS_CONFIG_LORA_REGION (the common shape for managed deployments) and synthesises the default LongFast channel. So a locked device on managed firmware came up TX-enabled on a well-known PSK before any operator interaction. Force config.lora.region = UNSET in the locked-boot block. RadioLibInterface gates both TX (startSend) and RX (readData) on region != UNSET — locked devices no longer initialise the SX12xx for either direction. Also set tx_enabled = false for any code path that checks the flag directly without consulting region. reloadFromDisk() restores the persisted lora config once the operator unlocks. Note: until the audit's M8 (radio re-init after reload, the upcoming commit 5 in this remediation series) lands, an unlocked device may need to reboot before its radio fully comes up under the real config; this is no worse than the pre-fix state, where the radio was already running on the wrong (default) config and any real config change required an explicit reconfigure or reboot anyway. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): per-connection status queue, redaction expansion, log/banner mute (audit) M14 — Replaces the single file-scope LockdownStatus slot with a per- PhoneAPI table keyed by PhoneAPI*, parallel to the auth-slot table and sharing g_authSlotsMutex. Previously a status produced for connection A (UNLOCKED with the active TTL, or UNLOCK_FAILED with a backoff) could be drained by connection B before A read it, leaking A's auth state to B. queueLockdownStatus is now a per-instance method writing to this->slot. A new static broadcastLockdownStatus exists for the main-loop session-expiry callers that have no PhoneAPI* in hand — those want every connected client to learn about the session roll, which is the only legitimate broadcast use case. hasPendingLockdownStatus is a const helper for the FromRadio available()/drain check. M13 — buildStatus_LH (the single point where lock_reason crosses into the on-wire LockdownStatus) collapses any token_* reason to a generic "locked" before emission. The specific reasons (token_hmac_fail, token_wrong_size, token_bad_magic, token_boots_zero, token_expired, token_dek_fail, token_missing) still go to local logs, but no longer tell an unauthenticated client that the firmware noticed their tampering / rollback / corrupt-file attempt. M15 — Extended the STATE_SEND_MY_INFO redaction (previously device_id only) to also wipe pio_env and min_app_version for unauth clients — both are pure build-fingerprint vectors that tell an attacker which known issues to probe. Kept my_node_num (broadcast on the mesh anyway) and nodedb_count (clients need it post-unlock to decide whether to pull the node DB). Added equivalent redaction for STATE_SEND_METADATA: the whole DeviceMetadata struct is wiped for unauth clients (firmware_version, device_state_version, hw_model, hw_model_string, has_bluetooth/has_wifi/has_ethernet, role, position_flags, excluded_modules). Clients re-fetch after authenticating. M16 — LoRa config is now whitelisted for unauth clients to the set that is intrinsically observable on the air anyway: region, modem_preset, use_preset, channel_num, hop_limit. Operator-private knobs (ignore_incoming, override_duty_cycle, override_frequency, sx126x_rx_boosted_gain, tx_power, ignore_mqtt, fem_lna_mode, config_ok_to_mqtt) are zeroed. The whitelist is built as a fresh LoRaConfig stack copy rather than masked in place to avoid touching the persisted struct. M12 — Skip the DEBUG_MUTE "we are muted, FYI" banner under MESHTASTIC_LOCKDOWN. The banner spilled APP_VERSION / APP_ENV / APP_REPO over USB CDC even with all other logging suppressed, which defeats the muting in lockdown builds and gives a USB-attached attacker a free firmware-fingerprint primitive. L9 — Removed the numeric backoff value from the LOG_WARN unlock- failed message. The client receives backoff_seconds via the UNLOCK_FAILED status; printing it again to USB serial under non-DEBUG_MUTE builds (i.e. MESHTASTIC_LOCKDOWN_DEBUG dev builds) was the only place it appeared in logs. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): atomic post-unlock reload with corruption surface (audit) Closes M6, M7, M8, M9 from the lockdown security audit. M6 — handleLockdownAuthInline no longer flips the connection to authorized or emits UNLOCKED on the cold-unlock path (the first successful passphrase verify after a locked boot). The client keeps seeing LOCKED until reloadFromDisk has actually populated config / channelFile / nodeDatabase with the operator's real values. Without this, the window between the auth call and the main-loop reload exposed two race-friendly bugs: (a) the client could read the locked-default placeholders as if they were the real config, and (b) a set_config in the window would silently overwrite a corrupted baseline once the reload swapped values in. A new per-status-slot bool pendingUnlockAfterReload records that the connection is mid-unlock. The re-verify path (storage already unlocked) is unchanged and authorizes immediately — there is nothing to reload. M7 — reloadFromDisk now holds a new file-scope mutex (g_reloadFromDiskMutex) against itself, parks the radio in sleep mode before swapping config / channelFile, and reconfigures the radio with the now-real settings after. Other readers of config.lora / channelFile / nodeDatabase do not take this lock today; closing those races is a wider locking-discipline change outside the audit's M7 scope. The radio standby+reconfigure prevents the SX12xx from sitting in a half-old/half-new register set across the swap, which otherwise required a reboot to recover from. M8 — RadioInterface::reconfigure() is now called at the end of a successful reload, so the SX12xx register set actually reflects the unlocked operator settings (region, modem preset, channels) rather than staying on the locked-default placeholder. Routed through a new Router::getRadioIface() accessor — the radio interface is owned by Router as a unique_ptr and was not exposed. M9 — NodeDB::loadProto now sets a NodeDB::storageCorruptThisLoad flag whenever an encrypted file fails to decrypt or proto-decode. reloadFromDisk consumes the flag and returns false on any failure instead of silently falling back to defaults. main.cpp's reload service then calls EncryptedStorage::lockNow() and PhoneAPI::revokeAllAuth(), and the new PhoneAPI::completePendingUnlocks(false) emits LOCKED(storage_corrupt) to every pending connection — they stay unauthorized so any set_config they send is dropped at the existing unauth gates. The lock_reason string passes through buildStatus_LH's M13 redaction unchanged because it does not start with token_. The success path goes through PhoneAPI::completePendingUnlocks(true) which authorizes each pending connection, emits UNLOCKED with the current TTL, and clears the screen-lock latch once. Snapshots the target PhoneAPI* list outside the auth-table lock to avoid re-entry when setAdminAuthorized takes the same lock. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): UI/pairing fixes for first-pair + content-flash + e-ink (audit) Closes H13, M19, M20, L4 from the lockdown audit. (L3 dropped per explicit decision — battery level is not a meaningful security side channel.) H13 — BLE pairing PIN was suppressed by the lockdown lock screen on locked devices. Screen.cpp updateUiFrame's lockdown short-circuit intercepts before ui->update() runs, so the pairing-PIN overlay banner that NRF52Bluetooth::onPairingPasskey queued never painted. Net effect: a freshly-locked device on first BLE pair could not be unlocked over BLE because the operator could never see the PIN — chicken and egg. Adds a new notificationTypeEnum::pairing_pin value and special-cases it in the short-circuit: paint the LOCKED frame first (so the underlying background remains the redacted view, never dashboard content) then let ui->update() composite the PIN banner overlay on top. The PIN itself is an ephemeral pair-handshake artifact (regenerated per attempt, dies on banner timeout) and is not operator content, so this does not regress the redaction guarantee. NRF52Bluetooth::onPairingPasskey switches from showSimpleBanner to showOverlayBanner with notificationType = pairing_pin so the short-circuit's lookup matches. M19 — Brief content-visible window on Screen::handleSetOn(true) wake. OLED GDDRAM physically retains the last-rendered frame while the panel is powered off; the next ui->update() after displayOn() is async, so an observer (or shoulder-surfer) could see the previous frame's content for 16-50 ms on every wake. Under MESHTASTIC_LOCKDOWN we now paint the LOCKED frame into GDDRAM in handleSetOn(false) before calling displayOff(). On wake the only thing the panel can flash is the redacted view. Gated on lockdown only — non-lockdown builds keep the previous frame as a UX cue. M20 — E-ink panels physically retain the last-rendered image without power. A power-cycled lockdown handheld kept showing operator-identifying content (position, messages, nodeinfo) until the firmware's first natural refresh — which on e-ink can be seconds into boot. Now, under MESHTASTIC_LOCKDOWN && USE_EINK, the panel init path in Screen::setup() paints the LOCKED frame and forces a full refresh (forceDisplay) immediately after ui->init() and before any other rendering. Persistent pixels are wiped to the redacted view before an observer can see them. Build-tested on seeed_wio_tracker_L1_eink; hardware-verified visual confirmation is pending a T-Echo session. L4 — Screen::blink() bypasses the normal ui->update() path that the lockdown short-circuit gates. It draws arbitrary geometry, not node data, so it does not actually leak today; but any future change that puts content into blink would silently leak past redaction. Added an early-return on shouldRedactDisplay() to make the function honor the redaction contract. Verified with nRF52 lockdown builds on both rak4631 (OLED) and seeed_wio_tracker_L1_eink (e-ink). * fix(lockdown): refuse APPROTECT on vulnerable silicon, gate on provision (audit) Closes M22 and M23 from the lockdown audit. M22 — APPROTECT lockout on nRF52840 is publicly known to be bypassable on every silicon revision shipping in current Meshtastic hardware (AAB0..AAF0) via SWD glitching, per LimitedResults' published research on the nRF52 series. Engaging APPROTECT on these revisions has two bad properties: (1) the lockout is irreversible without a destructive nrfjprog --recover, and (2) it gives the operator a false sense of security because the lockout itself can be defeated by anyone with ten minutes and a glitcher. enableAPProtect() now reads FICR.INFO.VARIANT (encoded as a 4-byte ASCII word) and refuses to engage on any known-vulnerable revision, logging the variant so the operator knows their device's specific build code. To override (e.g. for end-to-end testing of the engage path on hardware that's known affected), rebuild with -DMESHTASTIC_APPROTECT_OVERRIDE_VULNERABLE_SILICON=1. The vulnerable list is explicit and easy to update: any future revision shown to be fixed can be removed from the list and APPROTECT will engage on it as before. M23 — APPROTECT engagement moved from very early in setup() to after fsInit() + EncryptedStorage::initLocked(), and gated on EncryptedStorage::isProvisioned(). A misconfigured CI build of a lockdown variant flashed to a dev board would otherwise burn SWD on first boot before the operator had set any passphrase, taking the board out of the development/recovery workflow with zero real security benefit (there is no DEK to protect on an unprovisioned device). Engagement now follows operator intent: SWD locks only once they've committed to lockdown via passphrase provisioning. The SWD-attachable window between boot and APPROTECT engagement widens slightly from this reorder (now ~hundreds of ms while fsInit runs) but APPROTECT remains effective on the only payload it could protect (the in-RAM DEK loaded by initLocked which now runs *after* APPROTECT for already-provisioned devices). Verified with an nRF52 lockdown build (rak4631). * tools: harden lockdown_provision.py (audit) Closes M26-M30 and addresses L7. M26 — passphrase input. --passphrase on argv now requires --insecure-passphrase-on-cmdline as an explicit acknowledgement; without it the tool refuses and points at --passphrase-file or the interactive prompt. --passphrase-file refuses to read anything that isn't mode 0600 (so a passphrase another user can read off the filesystem doesn't silently succeed). With neither, the tool reads the passphrase via getpass.getpass — and on 'provision' double-prompts with a confirm. M27 — provision now requires an explicit 'yes' confirmation unless --yes is passed, after printing the warning that the passphrase cannot be recovered. The double-passphrase prompt is built into gather_passphrase(confirm=True). Reduces the chance of a typo binding a device to an unrecoverable passphrase. M28 — 'lock' subcommand gains a 'lock-now' alias, matching how the audit and wire docs refer to it everywhere. Both forms now require 'yes' confirmation unless --yes is set, so an accidental command doesn't immediately reboot the device into a locked state. M29 — the 4-second sleep is gone. Replaced with a StatusFuture single-shot that the FromRadio interceptor signals when the next LockdownStatus arrives. provision/unlock/lock wait up to --wait seconds (default 8) for the actual reply and exit non-zero with the device's reason on UNLOCK_FAILED, surfacing backoff_seconds in the error line. Exit codes are now meaningful: 0 = UNLOCKED 1 = no status / unexpected 2 = NEEDS_PROVISION (or a precondition fault: missing pkg, bad args) 3 = LOCKED (ambiguous: device reported locked rather than the expected unlocked result) 4 = UNLOCK_FAILED This lets ops scripts decide what to do without parsing stdout. M30 — top-of-file docstring gained an explicit SECURITY MODEL block that names the threat model (USB-only, passphrase cleartext on the cable) and forbids extension to TCP/BLE/UDP without a redesign. A runtime banner reprints the headline on every invocation. --port values starting with tcp:/tcp://, ble:/ble://, udp:/udp://, ws:/wss: are rejected at argument parse before any connection attempt; a copy-paste of an example into a context with a different --port cannot silently leak credentials to the wire. L7 — private meshtastic APIs (_handleFromRadio, _sendToRadio, _generatePacketId) are still in use because the lib does not yet dispatch LockdownStatus on a public pubsub topic and there is no public seam for raw ToRadio. Their use is now wrapped in getattr-with-clear-error so a future lib version that removes them produces an actionable error instead of an obscure traceback. The top-of-file note explains why we're on the private surface. Verified end-to-end on hardware (R1-Neo + Seeed Wio Tracker L1) during the audit-remediation hardware test pass: - provision (interactive, with confirm and double-prompt) - unlock (success returns UNLOCKED + boots TTL) - watch (passive listener emits LockdownStatus events) - lock-now (with --yes) * fix(lockdown): H13 — render pairing PIN steady over LOCKED frame Two bugs in the H13 fix from commit 614b7f001: 1. NotificationRenderer::drawBannercallback's switch had no case for the new notificationTypeEnum::pairing_pin. The function fell through to no-op so the banner never rendered. Added pairing_pin alongside text_banner so it dispatches to drawAlertBannerOverlay (same rendering, distinct type so the lockdown short-circuit in Screen.cpp can recognise it). 2. updateUiFrame's lockdown short-circuit called ui->update() to composite the banner. That redraws the current carousel frame (the dashboard) into the host framebuffer BEFORE the overlay paints, so the panel flashed dashboard content under the banner on every cycle. Replaced with a direct call to drawBannercallback so only the banner box is painted on top of the LOCKED pixels. Also: drawLockdownLockScreen used to commit to the panel (display->display()) at its end. With the banner overlay then painting and committing a second time, the panel visibly flickered between 'just LOCKED' and 'LOCKED + banner' on every render cycle. Split into drawLockdownLockScreenIntoBuffer (no commit) for the lockdown short-circuit, and a thin drawLockdownLockScreen wrapper that calls Buffer + display() for the other call sites that don't composite anything on top. The short-circuit now commits exactly once per frame after both LOCKED + any overlay are in the buffer. Verified end-to-end on hardware (Seeed Wio Tracker L1, OLED): fresh BLE pair against a locked device now shows the pairing PIN steadily on top of the LOCKED frame, no flicker, no dashboard leak, and pair completes normally. * fix(lockdown): backoff MAC + atomic writes + fault wipe + size cap (audit) Closes H3, H4, H12, M10, M11, M25 from the lockdown audit. Non-format- breaking: existing devices keep their .dek and .unlock_token but their old plaintext .backoff file (6 bytes, no MAC) is silently rejected as tampered on first read and reseeded with the MAC'd 38-byte format on the next failed-attempt OR successful unlock. H3 — Pre-increment the failed-attempt counter BEFORE running the HMAC verify in unlockWithPassphrase. The previous order wrote the counter only after a failed verify, so an attacker glitching the chip between verify and write could skip the increment and bypass backoff. The slot is now reserved atomically up front; the success path writes attempts=0 to clear the reservation. Worst case for a legitimate user who power-cycles mid-success is one phantom attempt — backoff recovers next try. H4 — .backoff file is now MAC'd with HMAC-SHA256(ephemeralKEK, "backoff-auth" || body) (32-byte tag), and written atomically via SafeFile (tmp + readback verify + rename). readBackoff treats missing / wrong-size / MAC-fail uniformly as max-attempts (255) so an attacker who deletes or rewrites the file can only INCREASE the wait, never decrease it. clearBackoff() now writes an attempts=0 sentinel instead of removing the file, so 'missing == tamper' is unambiguous post-provision. bumpBootsSinceFailOnBoot() skips on un-provisioned devices to avoid false 'tamper' detection during the legitimate fresh window between fsInit and provisionPassphrase. H12 — saveDEK and writeUnlockToken now write via SafeFile in fullAtomic mode (tmp file + readback verify + atomic rename) instead of remove-then-open-then-write. Power loss during a DEK or token write previously left the device unable to unlock — the encrypted prefs files are unreadable without a valid DEK. The atomic path rolls back to the previous file on partial write. M10 — readAndConsumeToken's 74-byte stack buffer (entire wrapped DEK + HMAC, explicitly called out by the audit as never wiped before return) is now a meshtastic_security::ZeroizingBuffer that the destructor scrubs on every return path. Same treatment for the computedHmac stack array next to it, and for the new backoff state buffers in readBackoff / writeBackoff / computeBackoffHmac. Removes the manual secure_zero calls those buffers had on success paths and fixes the missing wipes on the failure-return paths. M11 — Added EncryptedStorage::secureWipeKeys() public API that zeros dek/kek/ephemeralKek in BSS without touching flash, no logging, no locks (safe from interrupt context). HardFault_Impl now calls it as the very first thing on entry, before the diagnostic print / coredump path runs, so a hard-fault crash dump won't capture the DEK / KEK material that the rest of the module leaves in RAM. M25 — migrateFile now refuses to allocate a buffer for any file larger than 64 KiB. The legitimate ceiling is well under that on every supported variant; anything larger is either corrupt or a DFU-injected OOM attempt. Verified with an nRF52 lockdown build (rak4631). * fix(lockdown): MENC header MAC + token rollback counter (audit) Closes M2 and M4 from the lockdown audit. **FORMAT-BREAKING** — devices provisioned with prior lockdown firmware must factory-erase /prefs and reprovision; the previous tokens and encrypted prefs files will not decrypt under the new HMAC/body layouts. M2 — The HMAC on MENC encrypted proto files now covers the full on-disk header (4-byte magic + 13-byte nonce + 4-byte plaintext_len + ciphertext) instead of just (nonce + ciphertext). Without this, magic and plaintext_len were integrity-protected only by the equality check `plaintextLen == ciphertextLen` — which holds today (no padding / compression / AAD) but would silently produce length-oracle and downgrade vulnerabilities the instant any of those got added. Putting the header inside the MAC closes that pre-condition cleanly. The verify side in readAndDecrypt and the compose side in encryptAndWrite update in lockstep. M4 — UTOK gains a 4-byte monotonic counter field inside its MAC'd body. The highest counter ever issued is persisted to a new /prefs/.tokmono file MAC'd with HMAC-SHA256(ephemeralKEK, "tokmono-auth" || counter). On every readAndConsumeToken, any token whose counter is less than the persisted value is rejected as a rollback attempt and deleted. Defeats the audit's threat: an attacker who once captured a token (e.g. bootsRemaining=255 from before the operator lowered the policy) tries to write it back to disk later. Counter is incremented monotonically across the device's lifetime so any captured snapshot loses to the persisted max-seen. Self-heal: a token whose counter exceeds the persisted value (e.g. the .tokmono write itself failed after the token committed, or the .tokmono got wiped via factory-erase) is accepted AND the counter file is promoted to match. This avoids spuriously rejecting valid tokens after partial-update recovery. Threat model caveat (consistent with C2 acceptance): an attacker who has both flash extraction AND FICR can recompute the .tokmono MAC and restore a matching pair (.unlock_token + .tokmono) from an earlier capture. M4 raises the bar to that combined capability; the flash-write-only attacker is now blocked. Verified with an nRF52 lockdown build (rak4631). MIGRATION: devices already provisioned with the prior lockdown firmware will fail to auto-unlock at boot (token format mismatch), fall back to LOCKED(needs_auth), and every passphrase attempt will fail because the encrypted /prefs files are HMAC'd against the old input. Recovery is: factory-erase via the bootloader UF2 then re-provision via lockdown_provision.py or the Android app. * feat(lockdown): make lockdown a runtime client-toggleable setting Converts MESHTASTIC_LOCKDOWN from a per-variant compile-time flag that forced lockdown ON into an internal capability that is ALWAYS compiled in for nRF52 and gated purely at runtime by whether a passphrase has been provisioned. A device that has never been provisioned (or that the operator disabled) behaves exactly like stock firmware. Build/config: - configuration.h auto-defines MESHTASTIC_LOCKDOWN (+ ACCESS_CONTROL, ENCRYPTED_STORAGE, APPROTECT-capable) for ARCH_NRF52 unconditionally. No variant sets -DMESHTASTIC_LOCKDOWN anymore. Flash-constrained variants can opt out with -DMESHTASTIC_EXCLUDE_LOCKDOWN=1. DEBUG_MUTE is no longer coupled to lockdown (a capable-but-off device must log normally). rak4631 lands at 96.2% flash with lockdown always-in. Runtime predicate: - EncryptedStorage::isLockdownActive() == isProvisioned() (.dek exists) is the single source of truth for active/inactive. - PhoneAPI::getAdminAuthorized() returns true when lockdown is inactive, so every existing redaction gate no-ops on a capable-but-off device with no per-site changes. The locked-boot defaults path (NodeDB), the AdminModule storage-locked gate, the screen-redaction predicate, and the plaintext->encrypted migrate block are all additionally gated on isLockdownActive() so an un-provisioned device loads/serves plaintext normally. - sendConfigComplete emits LockdownStatus{DISABLED} when capable-but-off so the client renders its toggle OFF. Enable (off->on): client provisions a passphrase. provisionPassphrase generates the DEK; the existing reload path encrypts the plaintext config in place (migration runs live with the DEK in RAM) and authorizes the connection -> UNLOCKED. No reboot. Disable (on->off): LockdownAuth{passphrase, disable=true}. PhoneAPI verifies the passphrase (loads DEK), sets lockdownDisablePending; the main loop runs NodeDB::disableLockdownToPlaintext() which decrypts every pref via EncryptedStorage::migrateFileToPlaintext() then removeLockdownArtifacts() deletes the DEK/token/counter/backoff (the .dek delete is the atomic commit), then reboots into normal mode. Power-loss safe and re-runnable without a persistent marker — and the crypto runs live with the operator's passphrase in RAM rather than via a boot-time marker an attacker could plant to trigger an unprompted decrypt. APPROTECT is NOT reversed (sticky; permanent on silicon where it engaged). Generated bindings (admin.pb.h / mesh.pb.h) regenerated against protobufs#927 (LockdownAuth.disable, LockdownStatus.State.DISABLED). Submodule pointer stays at the pinned develop commit; the bindings are ahead until #927 merges and the submodule is bumped, same flow as the max_session_seconds work. Builds clean: rak4631 with no flags now auto-includes lockdown. NOTE: this changes the LockdownStatus the firmware emits and adds the disable path; pairs with protobufs#927 and the upcoming Android client toggle work. * fix(lockdown): re-lock per-connection auth on BLE reconnect A provisioned device reused a single BLE PhoneAPI instance, and the per-connection auth slot (keyed by that instance) was only cleared on the !isConnected() disconnect transition. A fast disconnect/reconnect could begin a new config burst while state was still STATE_SEND_PACKETS, so the reconnected client inherited the prior session's authorization: it received SecurityConfig in the clear and no LockdownStatus, and never re-authenticated. Reset the auth slot in NRF52Bluetooth onConnect(), which fires once per physical link, so every new connection starts locked regardless of whether the previous link's close() raced the new handshake. handleStartConfig keeps its !isConnected() reset (do NOT reset on a same-connection want_config: the post-unlock re-fetch is the client pulling now-unredacted config and must keep the auth it just earned, otherwise config comes back redacted and set_config writes get dropped). * fix(lockdown): persist config on a lockdown-capable but disabled device saveProto always called encryptAndWrite when encrypted storage was compiled, and saveToDiskNoRetry skipped every save when !isUnlocked(). On a disabled (never provisioned) device there is no DEK and isUnlocked() is always false, so both paths fired and NO config ever persisted: a LoRa region set before enabling lockdown lived only in RAM, then provisioning migrated the UNSET default from disk and the region was lost. Gate both on isLockdownActive(): when lockdown is inactive the device writes plaintext exactly like stock firmware; the reloadFromDisk migrate pass then re-saves those plaintext files encrypted once the device is provisioned. Verified on hardware: region set while disabled now survives enable, reboot, and unlock. * fix(lockdown): suppress LoRa region picker under the lock screen A locked-boot lockdown device installs region=UNSET as a deliberate RAM placeholder (the real region is in encrypted storage, restored on unlock). Screen.cpp popped the region picker / onboard message whenever region==UNSET, so it rendered over the lock screen and trapped input with no way out. Skip it while the display is being redacted for lockdown. * fix(lockdown): silence cppcheck void* false positive + ruff docstring lints The nRF52 `check` (cppcheck --fail-on-defect=low) flagged arithOperationsOnVoidPointer on EncryptedStorage.cpp buffers. These are false positives: make_zeroizing_array() returns unique_ptr<uint8_t[], ...> so .get() is uint8_t*, not void* — cppcheck just can't resolve the custom-deleter alias. File-scoped suppression, matching the existing crypto-code convention in suppressions.txt. Trunk flagged 5 ruff docstring issues in lockdown_provision.py: D301 (backslashes need a raw docstring) and D405/D407/D411/D413 (the EXAMPLES heading was being parsed as a numpydoc section). Made the docstring raw and renamed the heading to USAGE to dodge section detection while keeping the ASCII-box formatting. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lockdown): resolve cppcheck const/null-deref defects The nRF52 `check` job (pio check --fail-on-defect=low) flagged seven real cppcheck defects in the lockdown code: - EncryptedStorage.cpp: nonce/encDek are read-only views into the token buffer -> const uint8_t *. - NodeDB.cpp: segments[] lookup table is never mutated -> const. - PhoneAPI.cpp: clearStatusSlot_LH's p is only compared; the auth-check slot and the hasPendingLockdownStatus loop var are read-only -> const. - Screen.cpp: the MESHTASTIC_LOCKDOWN drawLockdownLockScreen() guard introduced a redundant null check (nullPointerRedundantCheck) since dispdev->displayOff() right below derefs it unguarded, as does the rest of the file. Dropped the guard. Verified with cppcheck 2.21 locally against the project suppressions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lockdown): const-qualify clearAuthSlot_LH param (cppcheck cascade) Making clearStatusSlot_LH take const PhoneAPI* let cppcheck propagate the same to clearAuthSlot_LH, whose p is only compared and forwarded. The remaining PhoneAPI* params (findOrAlloc*Slot_LH) store p into the slot table, so they correctly stay non-const. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(lockdown): wire runtime-toggle disable flow into provision tool Addresses Copilot review on tools/lockdown_provision.py — the reference tool advertised the runtime-toggle disable lifecycle but couldn't exercise it: - _STATE_NAMES: map LockdownStatus.DISABLED so a capable-but-off boot prints DISABLED instead of an opaque state=<num>. - build_lockdown_auth(): add a disable param that actually sets la.disable, failing loudly on pre-runtime-toggle bindings instead of silently sending a plain unlock. - cmd_disable() + 'disable' subcommand: send LockdownAuth{disable=true, passphrase=...} and wait for the resulting LockdownStatus. Mirrors the firmware: non-empty passphrase required, DISABLED broadcast precedes the reboot, TTL/session fields ignored. - _exit_code_for_status(): treat DISABLED as a success (exit 0) like UNLOCKED. All DISABLED/disable references are hasattr-guarded so the tool still imports and runs the lock/unlock/provision paths against the currently released meshtastic package (verified: it has LockdownAuth but not yet disable/DISABLED). Verified with ruff 0.15.13 and black. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-11 15:39:49 -05:00
Ben Meadors	eead467ce6	Added NodeDB fixtures and refactored to use std maps for better memory efficiency (#10464) ... * Added NodeDB fixtures and refactored to use std maps for better efficiency * Defer NodeDB save during xmodem transfer to prevent mid-transfer fsFormat	2026-05-12 17:23:29 -05:00
4 people	94bb21ecc7	2.8: NodeDB shrink, decoupling, and restructuring (#10413) ... * 2.8: NodeDB refactor to decouple satellite entries and decrease size * Regen * Refactor node mute handling to use dedicated functions for clarity and consistency * Develop ref * Fix NodeDB review follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Address review validation nits Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Trunk * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Extract legacy NodeDatabase migration * Fix remaining NodeDB review issues Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/c76b9a5a-7244-4fbc-9ef0-98091d8caaea Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fixes * Trunk * Fix latest review compile follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/5198da01-ec4c-4c16-8a09-68b8e6d5d410 Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fix cppcheck style warnings Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e60287ba-4ece-46e0-83d8-a6d89664c0bb Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Change pointer type for mesh node in set_favorite function * Change pointer types for mesh node references to const in multiple applets * Add NodeDB layout v25 documentation and migration guidelines * Remove tests for uninitialized PacketHistory state due to undefined behavior * Fix code block formatting in copilot instructions --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-05-09 15:12:10 -05:00
3 people	53c21eb30d	Deprecate/block packets with a missing/invalid hop_start value (pre-hop firmware) (related to issue #7369) (#9476) ... * Deprecate forwarding for invalid hop_start * Add pre-hop packet drop policy * Log ignored rebroadcasts for pre-hop packets * Respect pre-hop policy ALLOW in routing gates * Exempt local packets from pre-hop drop policy * Format pre-hop log line * Add MODERN_ONLY rebroadcast mode for pre-hop packets * Simplify implementation for drop packet only behaviour * Revert formatting-only changes * Match ReliableRouter EOF formatting * Make pre-hop drop a build-time flag * Rework to compile/build flag MESHTASTIC_PREHOP_DROP * Set MESHTASTIC_PREHOP_DROP off by default * Inline pre-hop hop_start validity check --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Jord <650645+DivineOmega@users.noreply.github.com>	2026-03-16 06:35:33 -05:00
4 people	5ce821c775	Mute specific nodes (#9209) ... * Regen protobufs * Ensure mute state is set when node is ignored * Added mechanism for toggling muted state * Implement the ability to mute specific nodes * Switch boolean value for bitmask * Correctly toggle bitfield position 2 on-change to mute state * Dont push submodule refs * Log correct info * Trunk fmt * Update protobuf ref to master branch of base * Update src/modules/ExternalNotificationModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-sync generated files --------- Co-authored-by: Jason P <applewiz@mac.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>	2026-01-11 22:59:51 -06:00
Jorropo	beb268ff25	Revert "add a .clang-format file (#9154)" (#9172) ... I thought git would be smart enough to understand all the whitespace changes but even with all the flags I know to make it ignore theses it still blows up if there are identical changes on both sides. I have a solution but it require creating a new commit at the merge base for each conflicting PR and merging it into develop. I don't think blowing up all PRs is worth for now, maybe if we can coordinate this for V3 let's say. This reverts commit 0d11331d18.	2026-01-04 05:15:53 -06:00
Jorropo	0d11331d18	add a .clang-format file (#9154)	2026-01-03 14:19:24 -06:00
Eric Severance	1b2dc10e77	Calculate hops correctly even when hop_start==0 (#9120) ... * Calculate hops correctly even when hop_start==0. * Use the same type (int8_t) in the loop, avoiding signed/unsigned mismatches. * Clarify defaultIfUnknown is returned for encrypted packets.	2025-12-30 15:31:35 -06:00
Ben Meadors	661f49ad7a	For our first position send on boot, validate that we have received a fresh position (#9023)	2025-12-20 07:01:00 -06:00
Jason B. Cox	066da492d5	Fix build when MESHTASTIC_EXCLUDE_PKI is defined	2025-11-20 12:26:07 -08:00
Ford Jones	6cad393688	Persist favourites on NodeDB reset (#8292) ... * Conditionally delete favourited nodes on reset * trunk fmt * Fix equality check, use existing macro for role validation * Extend favourite persistence setting to devices of all roles * Refactor: Decoupled role/config check and set role defaults appropriately * Use American-English spelling * Use existing reference * Convert reset to bool, regen protos * Add optional arg to nodedb_reset in favor of additional device setting * Use correct proto commit ID * Regen protos * Log preservation status * Pull latest from master	2025-11-06 16:06:37 -06:00
Ben Meadors	627c0145e7	Centralize getNodeId and fix references to owner.id (#8230)	2025-10-06 07:56:27 -05:00
Ben Meadors	b14e5770d5	Merge pull request #7873 from compumike/compumike/client-base-role ... Add `CLIENT_BASE` role: `ROUTER` for favorites, `CLIENT` otherwise (for attic/roof nodes!)	2025-09-18 20:40:39 -05:00
Jonathan Bennett	27c6b24e3a	Rather than mysteriously rebooting, regenerate the keys and infrom the user. (#7558)	2025-08-05 19:53:25 -05:00
Jonathan Bennett	91049d0db3	Misc cppcheck fixes (#7370)	2025-07-20 06:19:45 -05:00
5 people	a6be2e46ed	2.7 fixes w2 (#7148) ... * Initial work on splitting notification renderer into components for reuse * More progress * Fix notification popup * more fix, less crash * Adjustments for OLED on keeping menus tidy, added Bluetooth Toggle to Home frame. Also widen the frame slightly if you have a scroll bar * Small changes for EInk to not crowd elements * Change System frame menu over to better match actions; added color picker for T114 * Fix build errors and add T190 for testing * Logic gates are hard sometimes * Screen Color Picker changes, defined Yellow as a Color. * Additional colors and tuning * Abandon std::sort in NodeDB, and associated fixes (#7175) * Generate short name for nodes that don't have user yet * Add reboot menu * Sort fixes * noop sort option to avoid infinite loop * Refactor Overlay Banner * Continuing work on Color Picker * Add BaseUI menus to add and remove Favorited Nodes * Create TFT_MESH_OVERRIDE for variants.h and defined colors * Trigger a NodeStatus update at the end of setup() to get fresh data on display at boot. * T114 defaults to White, Yellow is now bright Yellow * Revert "T114 defaults to White, Yellow is now bright Yellow" This reverts commit 8d05e17f11eb48c42460176317893a50abd2eeb2. * Only show OEM text if not OLED * Adjust OEM logo to maximize visible area * Start plumbing in Color Picker changes * Finished plumbing * Fix warning * Revert "Fix warning" This reverts commit 2e8aecd52d6f5b9058e0bde09b72ece43a5f3a48. * Fix display not fully redrawing * T-Deck should get color too * Emote Revamp * Update emotes.cpp * Poo Emote fix * Trunk fix * Add secret test menu and number picker * Missed bits * Save colors between reboots * Save Clock Face election to protobuf * Make reboot first, then settings * Add padding for single line pop-ups * Compass saving and faster menus * Resolve build issue with Excluding GPS * Resolve issue with memory bars on EInk * Add brightness settings for supported screen (#7182) * Add brightness menu. * add loop destination selection. * Bring back color (and sanity) to the menus! * Trunk --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Jason P <applewiz@mac.com> Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com> Co-authored-by: Wilson <m.tools@qq.com>	2025-07-02 20:50:49 -05:00
7 people	29e7a71c97	2.7 Miscellaneous Fixes - Week 1 (#7102) ... * Update Favorite Node Message Options to unify against other screens * Rebuild Horizontal Battery, Resolve overlap concerns * Update positioning on Message frame and fix drawCommonHeader overlay * Beginnings of creating isHighResolution bool * Fixup determineResolution() * Implement isHighResolution in place of SCREEN_WIDTH > 128 checks * Line Spacing bound to isHighResolution * Analog Clock for all * Add AM/PM to Analog Clock if isHighResolution and not TWatch * Simple Menu Queue, and add time menu * Fix prompt string for 12/24 hour picker * More menu banners into functions * Fix Action Menu on Home frame * Correct pop-up calculation size and continue to leverage isHighResolution * Move menu bits to MenuHandler * Plumb in the digital/analog picker * Correct Clock Face Picker title * Clock picker fixes * Migrate the rest of the menus to MenuHandler.* * Add compass menu and needle point option * Minor fix for compass point menu * Correct Home menu into typical format * Fix emoji bounce, overlap, and missing commonHeader * Sanitize long_names and removed unused variables * Slightly better sanitizeString variation * Resolved apostrophe being shown as upside down question mark * Gotta keep height and width in expected order * Remove Second Hand for Analog Clock on EInk displays * Fix Clock menu option decision tree * Improvements to Eink Navigation * Pause Banner for Eink moved to bottom * Updated working for 12-/24-hour menu and Added US/Arizona to timezone picker * Add Adhoc Ping and resolve error with std::string sanitized * Hide quick toggle as option is available within Action Menu, commented out for the moment * Remove old battery icon and option, use drawCommonHeader throughout, re-add battery to Clock frames * fix misc build warnings. NFC * Update Analog Clock on EInk to show more digits * Establish Action Menu on all node list screens, add NodeDB reset (with confirmation) option * Add Toggle Backlight for EInk Displays * Suppress action screen Full refresh for Eink * Adjust drawBluetoothConnectedIcon on TWatch * Maintain clock frame when switching between Clock Faces * Move modules beyond the clock in navigation * addressed the conflicts, and changed target branch to 2.7-MiscFixes-Week1 * cleanup, cheers * Add AM/PM to low resolution clock also * Small adjustments to AM/PM replacement across various devices * Resolve dangling pointer issues with sanitize code * Update comments for Screen.cpp related to module load change * Trunk runs * Update message caching to correct aged timestamp * Menu wording adjustments * Time Format wording * Use all the rows on EInk since with autohide the navigation bar * Finalize Time Format picker word change * Retired drawFunctionOverlay code No longer being used * Actually honor the points-north setting * Trunk * Compressed action list * Update no-op showOverlayBanner function * trunk * Correct T_Watch_S3 specific line * Autosized Action menu per screen * Finalize Autosized Action menu per screen * Unify Message Titles * Reorder Timezones to match expectations * Adjust text location for pop-ups * Revert "Actually honor the points-north setting" This reverts commit 20988aa4fabb0975be644989d556fca7e1176680. * Make NodeDB sort its internal vector when lastheard is updated. Don't sort in NodeListRenderer * Update src/graphics/draw/NodeListRenderer.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Pass by reference -- Thanks Copilot! * Throttle sorting just a touch * Check more carefully for own node * Eliminate some now-unneeded sorting * Move function after include * Putting Modules back to position 0 and some trunk checks found * Add Scrollbar for Action menus * Second attempt to move modules down the navigation bar * Continue effort of moving modules in the navigation * Canned Messages tweak * Replicate Function + Space through the Menu System * Move init button parameters into config struct (#7145) * Remove bundling of web-ui from ESP32 devices (#7143) * Fixed triple click GPS toggle bungle * Move init button parameters into config struct * Reapply "Actually honor the points-north setting" This reverts commit 42c1967e7b3735ec9f5be8acd9582bc9edcbc78a. * Actually do compass pointings correctly * Tweak to node bearings * Menu wording tweaks * Get the compass_north_top logic right * Don't jump frames after setting Compass * Get rid of the extra bearingTo functions * Don't blink Mail on EInk Clock Screens * Actually set lat and long * Calibrate * Convert Radians to Degrees * More degree vs radians fixes * De-duplicate draw arrow function * Don't advertise compass calibration without an accell thread. --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: csrutil <keming.cao@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2025-06-26 22:11:20 -05:00
Jonathan Bennett	e623c70bd0	More clear key warning messages.	2025-06-14 19:35:57 -05:00
Jonathan Bennett	1557219bad	More low-entropy keys, and don't issue a false warning when changing … (#7041) ... * More low-entropy keys, and don't issue a false warning when changing node name * CopyPasta Wasn't Tasty * When the phone sets the publickey size to 0, regenerate right away	2025-06-14 17:09:22 -05:00
Jonathan Bennett	8ff99437cb	Don't include the blank hash	2025-06-12 22:56:40 -05:00
Jonathan Bennett	a1a5503fe9	Another known key	2025-06-12 15:18:26 -05:00
Jonathan Bennett	3b94981e56	Key erase (#7018) ... * Wipe keys if low entropy * Client Notification Payload variant * Don't call service before it's created * Lucky Number 14 * Catch for low-entropy keys even before region is set	2025-06-12 12:13:39 -05:00
Jonathan Bennett	e5f6804421	Add boolean to only warn a user of a duplicated key once per boot	2025-06-10 13:28:36 -05:00
Jonathan BennettandBen Meadors	4bf2dd04ae	Warn users about low entropy keys (#7003) ... Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2025-06-10 06:33:13 -05:00
Jonathan Bennett	41c1b29d70	Add basic handling for is_manually_validated (#6856)	2025-05-20 21:29:29 -05:00
Ben Meadors	0a8bd1e4be	Add contact admin message (for QR code) (#6806)	2025-05-14 05:33:51 -05:00
Ben Meadors	9cc13e628a	Stubbed out backup / restore methods for now and fixed bug	2025-03-17 19:24:47 -05:00
Ben Meadors	dc100e4d3e	Cleanup	2025-03-16 08:19:46 -05:00
GUVWAFandBen Meadors	9893d24c62	Only request all NodeInfo/Position on fresh install (#6184) ... Co-authored-by: Ben Meadors <benmmeadors@gmail.com>	2025-03-01 06:57:44 -06:00
+919 people	99d3e5eb70	2.6 changes (#5806) ... * 2.6 protos * [create-pull-request] automated change (#5789) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Hello world support for UDP broadcasts over the LAN on ESP32 (#5779) * UDP local area network meshing on ESP32 * Logs * Comment * Update UdpMulticastThread.h * Changes * Only use router->send * Make NodeDatabase (and file) independent of DeviceState (#5813) * Make NodeDatabase (and file) independent of DeviceState * 70 * Remove logging statement no longer needed * Explicitly set CAD symbols, improve slot time calculation and adjust CW size accordingly (#5772) * File system persistence fixes * [create-pull-request] automated change (#6000) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Update ref * Back to 80 * [create-pull-request] automated change (#6002) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * 2.6 <- Next hop router (#6005) * Initial version of NextHopRouter * Set original hop limit in header flags * Short-circuit to FloodingRouter for broadcasts * If packet traveled 1 hop, set `relay_node` as `next_hop` for the original transmitter * Set last byte to 0xFF if it ended at 0x00 As per an idea of @S5NC * Also update next-hop based on received DM for us * temp * Add 1 retransmission for intermediate hops when using NextHopRouter * Add next_hop and relayed_by in PacketHistory for setting next-hop and handle flooding fallback * Update protos, store multiple relayers * Remove next-hop update logic from NeighborInfoModule * Fix retransmissions * Improve ACKs for repeated packets and responses * Stop retransmission even if there's not relay node * Revert perhapsRebroadcast() * Remove relayer if we cancel a transmission * Better checking for fallback to flooding * Fix newlines in traceroute print logs * Stop retransmission for original packet * Use relayID * Also when want_ack is set, we should try to retransmit * Fix cppcheck error * Fix 'router' not in scope error * Fix another cppcheck error * Check for hop_limit and also update next hop when `hop_start == hop_limit` on ACK Also check for broadcast in `getNextHop()` * Formatting and correct NUM_RETRANSMISSIONS * Update protos * Start retransmissions in NextHopRouter if ReliableRouter didn't do it * Handle repeated/fallback to flooding packets properly First check if it's not still in the TxQueue * Guard against clients setting `next_hop`/`relay_node` * Don't cancel relay if we were the assigned next-hop * Replies (e.g. tapback emoji) are also a valid confirmation of receipt --------- Co-authored-by: GUVWAF <thijs@havinga.eu> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Tom Fifield <tom@tomfifield.net> Co-authored-by: GUVWAF <78759985+GUVWAF@users.noreply.github.com> * fix "native" compiler errors/warnings NodeDB.h * fancy T-Deck / SenseCAP Indicator / unPhone / PICOmputer-S3 TFT screen (#3259) * lib update: light theme * fix merge issue * lib update: home buttons + button try-fix * lib update: icon color fix * lib update: fix instability/crash on notification * update lib: timezone * timezone label * lib update: fix set owner * fix spiLock in RadioLibInterface * add picomputer tft build * picomputer build * fix compiler error std::find() * fix merge * lib update: theme runtime config * lib update: packet logger + T-Deck Plus * lib update: mesh detector * lib update: fix brightness & trackball crash * try-fix less paranoia * sensecap indicator updates * lib update: indicator fix * lib update: statistic & some fixes * lib-update: other T-Deck touch driver * use custom touch driver for Indicator * lower tft task prio * prepare LVGL ST7789 driver * lib update: try-fix audio * Drop received packets from self * Additional decoded packet ignores * Honor flip & color for Heltec T114 and T190 (#4786) * Honor TFT_MESH color if defined for Heltec T114 or T190 * Temporary: point lib_deps at fork of Heltec's ST7789 library For demo only, until ST7789 is merged * Update lib_deps; tidy preprocessor logic * Download debian files after firmware zip * set title for protobufs bump PR (#4792) * set title for version bump PR (#4791) * Enable Dependabot * chore: trunk fmt * fix dependabot syntax (#4795) * fix dependabot syntax * Update dependabot.yml * Update dependabot.yml * Bump peter-evans/create-pull-request from 6 to 7 in /.github/workflows (#4797) * Bump docker/build-push-action from 5 to 6 in /.github/workflows (#4800) * Actions: Semgrep Images have moved from returntocorp to semgrep (#4774) https://hub.docker.com/r/returntocorp/semgrep notes: "We've moved! Official Docker images for Semgrep now available at semgrep/semgrep." Patch updates our CI workflow for these images. Co-authored-by: Ben Meadors <benmmeadors@gmail.com> * Bump meshtestic from `31ee3d9` to `37245b3` (#4799) Bumps [meshtestic](https://github.com/meshtastic/meshTestic) from `31ee3d9` to `37245b3`. - [Commits](https://github.com/meshtastic/meshTestic/compare/31ee3d90c8bef61e835c3271be2c7cda8c4a5cc2...37245b3d612a9272f546bbb092837bafdad46bc2) --- updated-dependencies: - dependency-name: meshtestic dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [create-pull-request] automated change (#4789) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Bump pnpm/action-setup from 2 to 4 in /.github/workflows (#4798) Bumps [pnpm/action-setup](https://github.com/pnpm/action-setup) from 2 to 4. - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](https://github.com/pnpm/action-setup/compare/v2...v4) --- updated-dependencies: - dependency-name: pnpm/action-setup dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Raspberry Pico2 - needs protos * Re-order doDeepSleep (#4802) Make sure PMU sleep takes place before I2C ends * [create-pull-request] automated change * heltec-wireless-bridge requires Proto PR first * feat: trigger class update when protobufs are changed * meshtastic/ is a test suite; protobufs/ contains protobufs; * Update platform-native to pick up portduino crash fix (#4807) * Hopefully extract and commit to meshtastic.github.io * CI fixes * [Board] DIY "t-energy-s3_e22" (#4782) * New variant "t-energy-s3_e22" - Lilygo T-Energy-S3 - NanoVHF "Mesh-v1.06-TTGO-T18" board - Ebyte E22 Series * add board_level = extra * Update variant.h --------- Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Tom Fifield <tom@tomfifield.net> * Consolidate variant build steps (#4806) * poc: consolidate variant build steps * use build-variant action * only checkout once and clean up after run * Revert "Consolidate variant build steps (#4806)" (#4816) This reverts commit 9f8d86cb25. * Make Ublox code more readable (#4727) * Simplify Ublox code Ublox comes in a myriad of versions and settings. Presently our configuration code does a lot of branching based on versions being or not being present. This patch adds version detection earlier in the piece and branches on the set gnssModel instead to create separate setup methods for Ublox 6, Ublox 7/8/9, and Ublox10. Additionally, adds a macro to make the code much shorter and more readable. * Make trunk happy * Make trunk happy --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> * Consider the LoRa header when checking packet length * Minor fix (#4666) * Minor fixes It turns out setting a map value with the index notation causes an lookup that can be avoided with emplace. Apply this to one line in the StoreForward module. Fix also Cppcheck-determined highly minor performance increase by passing gpiochipname as a const reference :) The amount of cycles used on this laptop while learning about these callouts from cppcheck is unlikely to ever be more than the cycles saved by the fixes ;) * Update PortduinoGlue.cpp * Revert "Update classes on protobufs update" (#4824) * Revert "Update classes on protobufs update" * remove quotes to fix trunk. --------- Co-authored-by: Tom Fifield <tom@tomfifield.net> * Implement optional second I2C bus for NRF52840 Enabled at compile-time if WIRE_INFERFACES_COUNT defined as 2 * Add I2C bus to Heltec T114 header pins SDA: P0.13 SCL: P0.16 Uses bus 1, leaving bus 0 routed to the unpopulated footprint for the RTC (general future-proofing) * Tidier macros * Swap SDA and SCL SDA=P0.16, SCL=P0.13 * Refactor and consolidate time window logic (#4826) * Refactor and consolidate windowing logic * Trunk * Fixes * More * Fix braces and remove unused now variables. There was a brace in src/mesh/RadioLibInterface.cpp that was breaking compile on some architectures. Additionally, there were some brace errors in src/modules/Telemetry/AirQualityTelemetry.cpp src/modules/Telemetry/EnvironmentTelemetry.cpp src/mesh/wifi/WiFiAPClient.cpp Move throttle include in WifiAPClient.cpp to top. Add Default.h to sleep.cpp rest of files just remove unused now variables. * Remove a couple more meows --------- Co-authored-by: Tom Fifield <tom@tomfifield.net> * Rename message length headers and set payload max to 255 (#4827) * Rename message length headers and set payload max to 255 * Add MESHTASTIC_PKC_OVERHEAD * compare to MESHTASTIC_HEADER_LENGTH --------- Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> * Check for null before printing debug (#4835) * fix merge * try-fix crash * lib update: fix neighbors * fix GPIO0 mode after I2S audio * lib update: audio fix * lib update: fixes and improvements * extra * added ILI9342 (from master) * device-ui persistency * review update * fix request, add handled * fix merge issue * fix merge issue * remove newline * remove newlines from debug log * playing with locks; but needs more testing * diy mesh-tab initial files * board definition for mesh-tab (not yet used) * use DISPLAY_SET_RESOLUTION to avoid hw dependency in code * no telemetry for Indicator * 16MB partition for Indicator * 8MB partition for Indicator * stability: add SPI lock before saving via littleFS * dummy for config transfer (#5154) * update indicator (due to compile and linker errors) * remove faulty partition line * fix missing include * update indicator board * update mesh-tab ILI9143 TFT * fix naming * mesh-tab targets * try: disable duplicate locks * fix nodeDB erase loop when free mem returns invalid value (0, -1). * upgrade toolchain for nrf52 to gcc 9.3.1 * try-fix (workaround) T-Deck audio crash * update mesh-tab tft configs * set T-Deck audio to unused 48 (mem mclk) * swap mclk to gpio 21 * update meshtab voltage divider * update mesh-tab ini * Fixed the issue that indicator device uploads via rp2040 serial port in some cases. * Fixed the issue that the touch I2C address definition was not effective. * Fixed the issue that the wifi configuration saved to RAM did not take effect. * rotation fix; added ST7789 3.2" display * dreamcatcher: assign GPIO44 to audio mclk * mesh-tab touch updates * add mesh-tab powersave as default * fix DIO1 wakeup * mesh-tab: enable alert message menu * Streamline board definitions for first tech preview. (#5390) * Streamline board definitions for first tech preview. TBD: Indicator Support * add point-of-checkin * use board/unphone.json --------- Co-authored-by: mverch67 <manuel.verch@gmx.de> * fix native targets * add RadioLib debugging options for (T-Deck) * fix T-Deck build * fix native tft targets for rpi * remove wrong debug defines * t-deck-tft button is handled in device-ui * disable default lightsleep for indicator * Windows Support - Trunk and Platformio (#5397) * Add support for GPG * Add usb device support * Add trunk.io to devcontainer * Trunk things * trunk fmt * formatting * fix trivy/DS002, checkov/CKV_DOCKER_3 * hide docker extension popup * fix trivy/DS026, checkov/CKV_DOCKER_2 * fix radioLib warnings for T-Deck target * wake screen with button only * use custom touch driver * define wake button for unphone * use board definition for mesh-tab * mesh-tab rotation upside-down * update platform native * use MESH_TAB hardware model definition * radioLib update (fix crash/assert) * reference seeed indicator fix commit arduino-esp32 * Remove unneeded file change :) * disable serial module and tcp socket api for standalone devices (#5591) * disable serial module and tcp socket api for standalone devices * just disable webserver, leave wifi available * disable socket api * mesh-tab: lower I2C touch frequency * log error when packet queue is full * add more locking for shared SPI devices (#5595) * add more locking for shared SPI devices * call initSPI before the lock is used * remove old one * don't double lock * Add missing unlock * More missing unlocks * Add locks to SafeFile, remove from `readcb`, introduce some LockGuards * fix lock in setupSDCard() * pull radiolib trunk with SPI-CS fixes * change ContentHandler to Constructor type locks, where applicable --------- Co-authored-by: mverch67 <manuel.verch@gmx.de> Co-authored-by: GUVWAF <thijs@havinga.eu> Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com> * T-Deck: revert back to lovyanGFX touch driver * T-Deck: increase allocated PSRAM by 50% * mesh-tab: streamline target definitions * update RadioLib 7.1.2 * mesh-tab: fix touch rotation 4.0 inch display * Mesh-Tab platformio: 4.0inch: increase SPI frequency to max * mesh-tab: fix rotation for 3.5 IPS capacitive display * mesh-tab: fix rotation for 3.2 IPS capacitive display * restructure device-ui library into sub-directories * preparations for generic DisplayDriverFactory * T-Deck: increase LVGL memory size * update lib * trunk fmt --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: todd-herbert <herbert.todd@gmail.com> Co-authored-by: Jason Murray <15822260+scruplelesswizard@users.noreply.github.com> Co-authored-by: Jason Murray <jason@chaosaffe.io> Co-authored-by: Tom Fifield <tom@tomfifield.net> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> Co-authored-by: Austin <vidplace7@gmail.com> Co-authored-by: virgil <virgil.wang.cj@gmail.com> Co-authored-by: Mark Trevor Birss <markbirss@gmail.com> Co-authored-by: Kalle Lilja <15094562+ThatKalle@users.noreply.github.com> Co-authored-by: GUVWAF <thijs@havinga.eu> * Version this * Update platformio.ini (#6006) * tested higher speed and it works * Un-extra * Add -tft environments to the ci matrix * Exclude unphone tft for now. Something is wonky * fixed Indicator touch issue (causing IO expander issues), added more RAM * update lib * fixed Indicator touch issue (causing IO expander issues), added more RAM (#6013) * increase T-Deck PSRAM to avoid too early out-of-memory when messages fill up the storage * update device-ui lib * Fix T-Deck SD card detection (#6023) * increase T-Deck PSRAM to avoid too early out-of-memory when messages fill up the storage * fix SDCard for T-Deck; allow SPI frequency config * meshtasticd: Add X11 480x480 preset (#6020) * Littlefs per device * 2.6 update * [create-pull-request] automated change (#6037) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * InkHUD UI for E-Ink (#6034) * Decouple ButtonThread from sleep.cpp Reorganize sleep observables. Don't call ButtonThread methods inside doLightSleep. Instead, handle in class with new lightsleep Observables. * InkHUD: initial commit (WIP) Publicly discloses the current work in progress. Not ready for use. * feat: battery icon * chore: implement meshtastic/firmware #5454 Clean up some inline functions * feat: menu & settings for "jump to applet" * Remove the beforeRender pattern It hugely complicates things. If we can achieve acceptable performance without it, so much the better. * Remove previous Map Applet Needs re-implementation to work without the beforeRender pattern * refactor: reimplement map applet Doesn't require own position Doesn't require the beforeRender pattern to precalculate; now all-at-once in render Lays groundwork for fixed-size map with custom background image * feat: autoshow Allow user to select which applets (if any) should be automatically brought to foreground when they have new data to display * refactor: tidy-up applet constructors misc. jobs including: - consistent naming - move initializer-list-only constructors to header - give derived applets unique identifiers for MeshModule and OSThread logging * hotfix: autoshow always uses FAST update In future, it *will* often use FAST, but this will be controlled by a WindowManager component which has not yet been written. Hotfixed, in case anybody is attempting to use this development version on their deployed devices. * refactor: bringToForeground no longer requests FAST update In situations where an applet has moved to foreground because of user input, requestUpdate can be manually called, to upgrade to FAST refresh. More permanent solution for #23e1dfc * refactor: extract string storage from ThreadedMessageApplet Separates the code responsible for storing the limited message history, which was previously part of the ThreadedMessageApplet. We're now also using this code to store the "most recent message". Previously, this was stored in the `InkHUD::settings` struct, which was much less space-efficient. We're also now storing the latest DM, laying the foundation for an applet to display only DMs, which will complement the threaded message applet. * fix: text wrapping Attempts to fix a disparity between `Applet::printWrapped` and `Applet::getWrappedTextHeight`, which would occasionally cause a ThreadedMessageApplet message to render "too short", overlapping other text. * fix: purge old constructor This one slipped through the last commit.. * feat: DM Applet Useful in combination with the ThreadedMessageApplets, which don't show DMs * fix: applets shouldn't handle events while deactivated Only one or two applets were actually doing this, but I'm making a habit of having all applets return early from their event handling methods (as good practice), even if those methods are disabled elsewhere (e.g. not observing observable, return false from wantPacket) * refactor: allow requesting update without requesting autoshow Some applets may want to redraw, if they are displayed, but not feel the information is worth being brought to foreground for. Example: ActiveNodesApplet, when purging old nodes from list. * feat: custom "Recently Active" duration Allows users to tailor how long nodes will appear in the "Recents" applets, to suit the activity level of their mesh. * refactor: rename some applets * fix: autoshow * fix: getWrappedTextHeight Remove the "simulate" option from printWrapped; too hard to keep inline with genuine printing (because of AdafruitGFX Fonts' xAdvance, mabye?). Instead of simulating, we printWrapped as normal, and discard pixel output by setting crop. Both methods are similarly inefficient, apparently. * fix: text wrapping in ThreadedMessageApplet Wrong arguments were passed to Applet::printWrapped * feat: notifications for text messages Only shown if current applet does not already display the same info. Autoshow takes priority over notifications, if both would be used to display the same info. * feat: optimize FAST vs FULL updates New UpdateMediator class counts the number of each update type, and suggets which one to use, if the code doesn't already have an explicit prefence. Also performs "maintenance refreshes" unprovoked if display is not given an opportunity to before a FULL refresh through organic use. * chore: update todo list * fix: rare lock-up of buttons * refactor: backlight Replaces the initial proof-of-concept frontlight code for T-Echo Presses less than 5 seconds momentarily illuminate the display Presses longer than 5 seconds latch the light, requiring another tap to disable If user has previously removed the T-Echo's capacitive touch button (some DIY projects), the light is controlled by the on-screen menu. This fallback is used by all T-Echo devices, until a press of the capacitive touch button is detected. * feat: change tile with aux button Applied to VM-E290. Working as is, but a refactor of WindowManager::render is expected shortly, which will also tidy code from this push. * fix: specify out-of-the-box tile assignments Prevents placeholder applet showing on initial boot, for devices which use a mult-tile layout by default (VM-E290) * fix: verify settings version when loading * fix: wrong settings version * refactor: remove unimplemented argument from requestUpdate Specified whether or not to update "async", however the implementation was slightly broken, Applet::requestUpdate is only handled next time WindowManager::runOnce is called. This didn't allow code to actually await an update, which was misleading. * refactor: renaming Applet::render becomes Applet::onRender. Tile::displayedApplet becomes Tile::assignedApplet. New onRender method name allows us to move some of the pre and post render code from WindowManager into new Applet::render method, which will call onRender for us. * refactor: rendering Bit of a tidy-up. No intended change in behavior. * fix: optimize refresh times Shorter wait between retrying update if display was previously busy. Set anticipated update durations closer to observed values. No signifacant performance increase, but does decrease the amount of polling required. * feat: blocking update for E-Ink Option to wait for display update to complete before proceeding. Important when shutting down the device. * refactor: allow system applets to lock rendering Temporarily prevents other applets from rendering. * feat: boot and shutdown screens * feat: BluetoothStatus Adds a meshtastic::Status object which exposes the state of the Bluetooth connection. Intends to allow decoupling of UI code. * feat: Bluetooth pairing screen * fix: InkHUD defaults not honored * fix: random Bluetooth pin for NicheGraphics UIs * chore: button interrupts tested * fix: emoji reactions show as blank messages * fix: autoshow and notification triggered by outgoing message * feat: save InkHUD data before reboot Implemented with a new Observable. Previously, config and a few recent messages were saved on shutdown. These were lost if the device rebooted, for example when firmware settings were changed by a client. Now, the InkHUD config and recent messages saved on reboot, the same as during an intentional shutdown. * feat: imperial distances Controlled by the config.display.units setting * fix: hide features which are not yet implemented * refactor: faster rendering Previously, only tiles which requested update were re-rendered. Affected tiles had their region blanked before render, pixel by pixel. Benchmarking revealed that it is significantly faster to memset the framebuffer and redraw all tiles. * refactor: tile ownership Tiles and Applets now maintain a reciprocal link, which is enforced by asserts. Less confusing than the old situation, where an applet and a tile may disagree on their relationship. Empty tiles are now identified by a nullptr *Applet, instead of by having the placeholderApplet assigned. * fix: notifications and battery when menu open Do render notifications in front of menu; don't render battery icon in front of menu. * fix: simpler defaults Don't expose new users to multiplexed applets straight away: make them enable the feature for themselves. * fix: Inputs::TwoButton interrupts, when only one button in use * fix: ensure display update is complete when ESP32 enters light sleep Many panels power down automatically, but some require active intervention from us. If light sleep (ESP32) occurs during a display update, these panels could potentially remain powered on, applying voltage the pixels for an extended period of time, and potentially damaging the display. * fix: honor per-variant user tile limit Set as the default value for InkHUD::settings.userTiles.maxCount in nicheGraphics.h * feat: initial InkHUD support for Wireless Paper v1.1 and VM-E213 * refactor: Heard and Recents Applets Tidier code, significant speed boost. Possibly no noticable change in responsiveness, but rendering now spends much less time blocking execution, which is important for correction functioning of the other firmware components. * refactor: use a common pio base config Easier to make any future PlatformIO config changes * feat: tips Show information that we think the user might find helpful. Some info shown first boot only. Other info shown when / if relevant. * fix: text wrapping for '\n' Previously, the newline was honored, but the adojining word was not printed. * Decouple ButtonThread from sleep.cpp Reorganize sleep observables. Don't call ButtonThread methods inside doLightSleep. Instead, handle in class with new lightsleep Observables. * feat: BluetoothStatus Adds a meshtastic::Status object which exposes the state of the Bluetooth connection. Intends to allow decoupling of UI code. * feat: observable for reboot * refactor: Heltec VM-E290 installDefaultConfig * fix: random Bluetooth pin for NicheGraphics UIs * update device-ui: fix touch/crash issue while light sleep * Collect inkhud * fix: InkHUD shouldn't nag about timezone (#6040) * Guard eink drivers w/ MESHTASTIC_INCLUDE_NICHE_GRAPHICS * Case sensitive perhaps? * More case-sensitivity instances * Moar * RTC * Yet another case issue! * Sigh... * MUI: BT programming mode (#6046) * allow BT connection with disabled MUI * Update device-ui --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> * MUI: fix nag timeout, disable BT programming mode for native (#6052) * allow BT connection with disabled MUI * Update device-ui * MUI: fix nag timeout default and remove programming mode for native --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> * remove debuglog leftover * Wireless Paper: remove stray board_level = extra (#6060) Makes sure the InkHUD version gets build into the release zip * Fixed persistence stragglers from NodeDB / Device State divorce (#6059) * Increase `MAX_THREADS` for InkHUD variants with WiFi (#6064) * Licensed usage compliance (#6047) * Prevent psk and legacy admin channel on licensed mode * Move it * Consolidate warning strings * More holes * Device UI submodule bump * Prevent licensed users from rebroadcasting unlicensed traffic (#6068) * Prevent licensed users from rebroadcasting unlicensed traffic * Added method and enum to make user license status more clear * MUI: move UI initialization out of main.cpp and adding lightsleep observer + mutex (#6078) * added device-ui to lightSleep observers for handling graceful sleep; refactoring main.cpp * bump lib version * Update device-ui * unPhone TFT: include into build, enable SD card, increase PSRAM (#6082) * unPhone-tft: include into build, enable SD card, increase assigned PSRAM * lib update * Backup / migrate pub private keys when upgrading to new files in 2.6 (#6096) * Save a backup of pub/private keys before factory reset * Fix licensed mode warning * Unlock spi on else file doesn't exist * Update device-ui * Update protos and device-ui * [create-pull-request] automated change (#6129) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Proto * [create-pull-request] automated change (#6131) * Proto update for backup * [create-pull-request] automated change (#6133) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Update protobufs * Space * [create-pull-request] automated change (#6144) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Protos * [create-pull-request] automated change (#6152) Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Updeet * device-ui lib update * fix channel OK button * device-lib update: fix settings panel -> no scrolling * device-ui lib: last minute update * defined(SENSECAP_INDICATOR) * MUI hot-fix pub/priv keys * MUI hot-fix username dialog * MUI: BT programming mode button * Update protobufs --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: GUVWAF <78759985+GUVWAF@users.noreply.github.com> Co-authored-by: GUVWAF <thijs@havinga.eu> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Tom Fifield <tom@tomfifield.net> Co-authored-by: mverch67 <manuel.verch@gmx.de> Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com> Co-authored-by: todd-herbert <herbert.todd@gmail.com> Co-authored-by: Jason Murray <15822260+scruplelesswizard@users.noreply.github.com> Co-authored-by: Jason Murray <jason@chaosaffe.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> Co-authored-by: Austin <vidplace7@gmail.com> Co-authored-by: virgil <virgil.wang.cj@gmail.com> Co-authored-by: Mark Trevor Birss <markbirss@gmail.com> Co-authored-by: Kalle Lilja <15094562+ThatKalle@users.noreply.github.com> Co-authored-by: rcarteraz <robert.l.carter2@gmail.com>	2025-03-01 06:18:33 -06:00
Eric Severance	27fbfd03d6	Add unit tests for MQTT (#5724) ... * Add unit tests for MQTT * Test received fields	2025-01-07 07:10:42 -06:00
Tom Fifieldandmverch67	cd198fcf3f	cherry-pick: device-ui persistency (#5570) ... * device-ui persistency * review update --------- Co-authored-by: mverch67 <manuel.verch@gmx.de>	2024-12-26 17:46:21 -06:00