Compare commits

..
549 changed files with 11957 additions and 33613 deletions
+1 -7
View File
@@ -49,17 +49,11 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
8. **Suggest next actions only for specific, recognisable failure modes**:
7. **Suggest next actions only for specific, recognisable failure modes**:
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
## What NOT to do
-103
View File
@@ -1,103 +0,0 @@
---
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
argument-hint: [window=1h] [field=free_heap] [variant=local]
---
<!-- markdownlint-disable MD029 -->
# `/leakhunt` — read the recorder, classify a memory leak
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
## Two signal paths — pick the right one
| Path | Build flag | Cadence | Per-thread attribution | Cost |
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
| LocalStats packet | (default) | ~60 s | No | Free — always on |
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
## What to do
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
- `running == True`
- `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
- `samples` — how many raw points contributed
- `min`, `max` — total swing
- `slope_per_min` — units per minute (linear regression over the whole window)
5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
- **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
8. **Report**:
```text
/leakhunt window=6h field=free_heap variant=local
────────────────────────────────────────────────────
recorder : running, telem last_ts 8s ago
build : DEBUG_HEAP=ON (per-line prefix detected)
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
free_heap : min 92,344 / max 124,008 / range 31,664
slope : -82 bytes/min (negative — heap declining)
reboots : none in window
OOM events : none
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
Router -1,408 B over 4 events
others -240 B
verdict : SLOW LEAK — primary suspect MeshPacket thread
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
evidence : (3 log line citations with uptimes)
```
Then: **what to do next.**
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
```text
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
```
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
- STEADY → say so; stop. Don't invent problems.
## What NOT to do
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
## Companion: `mark_event` for stress runs
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
```text
mark_event("burst-start")
… run the workload …
mark_event("burst-end")
/leakhunt window=15m
```
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
-4
View File
@@ -3,8 +3,6 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
argument-hint: <test-node-id> [count=5]
---
<!-- markdownlint-disable MD029 -->
# `/repro` — flakiness triage for one test
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -42,8 +40,6 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
6. **Classify the flake** into one of:
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
+3 -7
View File
@@ -8,21 +8,17 @@
"features": {
"ghcr.io/devcontainers/features/python:1": {
"installTools": true,
"version": "3.13"
"version": "3.14"
}
},
"customizations": {
"vscode": {
"extensions": [
"ms-vscode.cpptools",
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide",
"platformio.platformio-ide",
"Trunk.io"
],
"unwantedRecommendations": [
"ms-azuretools.vscode-docker",
"platformio.platformio-ide"
],
"unwantedRecommendations": ["ms-azuretools.vscode-docker"],
"settings": {
"extensions.ignoreRecommendations": true
}
+29
View File
@@ -0,0 +1,29 @@
# firmware Development Guidelines
Auto-generated from all feature plans. Last updated: 2026-03-25
## Active Technologies
- Python 3.x for workflow tooling, Markdown for generated artifacts, existing C/C++/PlatformIO repository conventions for downstream scaffold targets + Python standard library for inventory/intake tooling, existing Spec Kit artifacts, repository-local Copilot prompt/agent files, PlatformIO environment metadata conventions already in `variants/**/platformio.ini` (129-hardware-support-agent)
## Project Structure
```text
src/
tests/
```
## Commands
cd src [ONLY COMMANDS FOR ACTIVE TECHNOLOGIES][ONLY COMMANDS FOR ACTIVE TECHNOLOGIES] pytest [ONLY COMMANDS FOR ACTIVE TECHNOLOGIES][ONLY COMMANDS FOR ACTIVE TECHNOLOGIES] ruff check .
## Code Style
Python 3.x for workflow tooling, Markdown for generated artifacts, existing C/C++/PlatformIO repository conventions for downstream scaffold targets: Follow standard conventions
## Recent Changes
- 129-hardware-support-agent: Added Python 3.x for workflow tooling, Markdown for generated artifacts, existing C/C++/PlatformIO repository conventions for downstream scaffold targets + Python standard library for inventory/intake tooling, existing Spec Kit artifacts, repository-local Copilot prompt/agent files, PlatformIO environment metadata conventions already in `variants/**/platformio.ini`
<!-- MANUAL ADDITIONS START -->
<!-- MANUAL ADDITIONS END -->
+124
View File
@@ -0,0 +1,124 @@
---
description: Guide maintainers through Meshtastic hardware support context generation, board intake assessment, and optional scaffold generation.
---
# Hardware Support Workflow
Use this workflow when a maintainer wants to add support for a new board variant in the Meshtastic firmware repository.
## Goals
- Reuse repository-backed hardware patterns before drafting new board files.
- Keep all generated output scoped to the requested architecture and board.
- Stop and surface evidence gaps instead of inventing pin mappings or metadata.
## Required Inputs
Collect or confirm these fields before scaffold generation:
- PlatformIO environment name
- hardware model identifier
- display name
- architecture
Recommended additional inputs:
- hardware model slug
- actively supported flag
- support level
- source materials such as schematic, pinout, or datasheet links
- board notes covering revision scope and known uncertainty
## Workflow
### 1. Refresh Repository Context
Run from the repository root:
```bash
python3 bin/generate_hardware_support_context.py
```
Review [docs/hardware-support-context.md](../../docs/hardware-support-context.md) for architecture-specific examples, metadata keys, and inherited-default notes.
### 2. Capture The Intake Request
Create or update a JSON file matching the contract in [specs/129-hardware-support-agent/contracts/board-intake-contract.md](../../specs/129-hardware-support-agent/contracts/board-intake-contract.md).
### 3. Assess Intake Readiness
Run:
```bash
python3 bin/board_intake.py path/to/intake.json
```
What to look for:
- expected artifacts
- required metadata
- matched repository patterns
- evidence gaps
- risk flags
- next actions
- scaffold readiness decision
For CI-style gating, use:
```bash
python3 bin/board_intake.py path/to/intake.json --validate
```
If the assessment is not scaffold-ready, stop and resolve the blocking gaps before continuing.
### 4. Generate Scaffold Output When Ready
Only run this when the intake assessment reports `Scaffold ready: Yes`.
```bash
python3 bin/board_scaffold.py path/to/intake.json --output-dir generated/hardware-support
```
Expected outputs:
- draft `variant.h`
- draft `platformio.ini`
- optional `variant.cpp` for ESP32-family targets
Review all `// TODO: verify — ...` annotations before treating the scaffold as merge-ready.
### 5. Compile-Gate The Target Environment (Required)
The end stage must always validate that the target environment is at least compilable.
Run:
```bash
pio run -e <environment_name>
```
Expected behavior:
- If compile succeeds, include a "compile check passed" note in the review summary.
- If compile fails, treat it as a blocking issue and report the exact failing error.
- Do not mark the workflow complete while compile is failing.
Common first-pass blocker for new scaffolds:
- Missing or placeholder `board = ...` in `platformio.ini` causes `BoardConfig: Board is not defined`.
## Guardrails
- Do not change live firmware runtime code under `src/` as part of this workflow.
- Do not modify existing board definitions under `variants/` automatically.
- Do not guess unresolved radio, display, GPS, power, or input pin mappings.
- Treat multi-revision or multi-option board notes as blocking until the revision scope is explicit.
- Check inherited BSP defaults for `nrf52840`, `rp2040`, `stm32`, and `native` targets before declaring a missing define.
## Validation Expectations
- Run `trunk fmt --force` on touched Python workflow files.
- Re-run `python3 bin/generate_hardware_support_context.py` after changes affecting context output.
- Use fixture-driven smoke tests in `bin/fixtures/` for intake and scaffold workflows.
- Run `pio run -e <environment_name>` as a required final compile gate for the generated board environment.
- Report any skipped validation or remaining TODO annotations in the review notes.
+184
View File
@@ -0,0 +1,184 @@
---
description: Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation.
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Goal
Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/speckit.tasks` has successfully produced a complete `tasks.md`.
## Operating Constraints
**STRICTLY READ-ONLY**: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually).
**Constitution Authority**: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/speckit.analyze`.
## Execution Steps
### 1. Initialize Analysis Context
Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths:
- SPEC = FEATURE_DIR/spec.md
- PLAN = FEATURE_DIR/plan.md
- TASKS = FEATURE_DIR/tasks.md
Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command).
For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
### 2. Load Artifacts (Progressive Disclosure)
Load only the minimal necessary context from each artifact:
**From spec.md:**
- Overview/Context
- Functional Requirements
- Success Criteria (measurable outcomes — e.g., performance, security, availability, user success, business impact)
- User Stories
- Edge Cases (if present)
**From plan.md:**
- Architecture/stack choices
- Data Model references
- Phases
- Technical constraints
**From tasks.md:**
- Task IDs
- Descriptions
- Phase grouping
- Parallel markers [P]
- Referenced file paths
**From constitution:**
- Load `.specify/memory/constitution.md` for principle validation
### 3. Build Semantic Models
Create internal representations (do not include raw artifacts in output):
- **Requirements inventory**: For each Functional Requirement (FR-###) and Success Criterion (SC-###), record a stable key. Use the explicit FR-/SC- identifier as the primary key when present, and optionally also derive an imperative-phrase slug for readability (e.g., "User can upload file" → `user-can-upload-file`). Include only Success Criteria items that require buildable work (e.g., load-testing infrastructure, security audit tooling), and exclude post-launch outcome metrics and business KPIs (e.g., "Reduce support tickets by 50%").
- **User story/action inventory**: Discrete user actions with acceptance criteria
- **Task coverage mapping**: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases)
- **Constitution rule set**: Extract principle names and MUST/SHOULD normative statements
### 4. Detection Passes (Token-Efficient Analysis)
Focus on high-signal findings. Limit to 50 findings total; aggregate remainder in overflow summary.
#### A. Duplication Detection
- Identify near-duplicate requirements
- Mark lower-quality phrasing for consolidation
#### B. Ambiguity Detection
- Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria
- Flag unresolved placeholders (TODO, TKTK, ???, `<placeholder>`, etc.)
#### C. Underspecification
- Requirements with verbs but missing object or measurable outcome
- User stories missing acceptance criteria alignment
- Tasks referencing files or components not defined in spec/plan
#### D. Constitution Alignment
- Any requirement or plan element conflicting with a MUST principle
- Missing mandated sections or quality gates from constitution
#### E. Coverage Gaps
- Requirements with zero associated tasks
- Tasks with no mapped requirement/story
- Success Criteria requiring buildable work (performance, security, availability) not reflected in tasks
#### F. Inconsistency
- Terminology drift (same concept named differently across files)
- Data entities referenced in plan but absent in spec (or vice versa)
- Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note)
- Conflicting requirements (e.g., one requires Next.js while other specifies Vue)
### 5. Severity Assignment
Use this heuristic to prioritize findings:
- **CRITICAL**: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality
- **HIGH**: Duplicate or conflicting requirement, ambiguous security/performance attribute, untestable acceptance criterion
- **MEDIUM**: Terminology drift, missing non-functional task coverage, underspecified edge case
- **LOW**: Style/wording improvements, minor redundancy not affecting execution order
### 6. Produce Compact Analysis Report
Output a Markdown report (no file writes) with the following structure:
## Specification Analysis Report
| ID | Category | Severity | Location(s) | Summary | Recommendation |
| --- | ----------- | -------- | ---------------- | ---------------------------- | ------------------------------------ |
| A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version |
(Add one row per finding; generate stable IDs prefixed by category initial.)
**Coverage Summary Table:**
| Requirement Key | Has Task? | Task IDs | Notes |
| --------------- | --------- | -------- | ----- |
**Constitution Alignment Issues:** (if any)
**Unmapped Tasks:** (if any)
**Metrics:**
- Total Requirements
- Total Tasks
- Coverage % (requirements with >=1 task)
- Ambiguity Count
- Duplication Count
- Critical Issues Count
### 7. Provide Next Actions
At end of report, output a concise Next Actions block:
- If CRITICAL issues exist: Recommend resolving before `/speckit.implement`
- If only LOW/MEDIUM: User may proceed, but provide improvement suggestions
- Provide explicit command suggestions: e.g., "Run /speckit.specify with refinement", "Run /speckit.plan to adjust architecture", "Manually edit tasks.md to add coverage for 'performance-metrics'"
### 8. Offer Remediation
Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.)
## Operating Principles
### Context Efficiency
- **Minimal high-signal tokens**: Focus on actionable findings, not exhaustive documentation
- **Progressive disclosure**: Load artifacts incrementally; don't dump all content into analysis
- **Token-efficient output**: Limit findings table to 50 rows; summarize overflow
- **Deterministic results**: Rerunning without changes should produce consistent IDs and counts
### Analysis Guidelines
- **NEVER modify files** (this is read-only analysis)
- **NEVER hallucinate missing sections** (if absent, report them accurately)
- **Prioritize constitution violations** (these are always CRITICAL)
- **Use examples over exhaustive rules** (cite specific instances, not generic patterns)
- **Report zero issues gracefully** (emit success report with coverage statistics)
## Context
$ARGUMENTS
+295
View File
@@ -0,0 +1,295 @@
---
description: Generate a custom checklist for the current feature based on user requirements.
---
## Checklist Purpose: "Unit Tests for English"
**CRITICAL CONCEPT**: Checklists are **UNIT TESTS FOR REQUIREMENTS WRITING** - they validate the quality, clarity, and completeness of requirements in a given domain.
**NOT for verification/testing**:
- ❌ NOT "Verify the button clicks correctly"
- ❌ NOT "Test error handling works"
- ❌ NOT "Confirm the API returns 200"
- ❌ NOT checking if code/implementation matches the spec
**FOR requirements quality validation**:
- ✅ "Are visual hierarchy requirements defined for all card types?" (completeness)
- ✅ "Is 'prominent display' quantified with specific sizing/positioning?" (clarity)
- ✅ "Are hover state requirements consistent across all interactive elements?" (consistency)
- ✅ "Are accessibility requirements defined for keyboard navigation?" (coverage)
- ✅ "Does the spec define what happens when logo image fails to load?" (edge cases)
**Metaphor**: If your spec is code written in English, the checklist is its unit test suite. You're testing whether the requirements are well-written, complete, unambiguous, and ready for implementation - NOT whether the implementation works.
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Execution Steps
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list.
- All file paths must be absolute.
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
2. **Clarify intent (dynamic)**: Derive up to THREE initial contextual clarifying questions (no pre-baked catalog). They MUST:
- Be generated from the user's phrasing + extracted signals from spec/plan/tasks
- Only ask about information that materially changes checklist content
- Be skipped individually if already unambiguous in `$ARGUMENTS`
- Prefer precision over breadth
Generation algorithm:
1. Extract signals: feature domain keywords (e.g., auth, latency, UX, API), risk indicators ("critical", "must", "compliance"), stakeholder hints ("QA", "review", "security team"), and explicit deliverables ("a11y", "rollback", "contracts").
2. Cluster signals into candidate focus areas (max 4) ranked by relevance.
3. Identify probable audience & timing (author, reviewer, QA, release) if not explicit.
4. Detect missing dimensions: scope breadth, depth/rigor, risk emphasis, exclusion boundaries, measurable acceptance criteria.
5. Formulate questions chosen from these archetypes:
- Scope refinement (e.g., "Should this include integration touchpoints with X and Y or stay limited to local module correctness?")
- Risk prioritization (e.g., "Which of these potential risk areas should receive mandatory gating checks?")
- Depth calibration (e.g., "Is this a lightweight pre-commit sanity list or a formal release gate?")
- Audience framing (e.g., "Will this be used by the author only or peers during PR review?")
- Boundary exclusion (e.g., "Should we explicitly exclude performance tuning items this round?")
- Scenario class gap (e.g., "No recovery flows detected—are rollback / partial failure paths in scope?")
Question formatting rules:
- If presenting options, generate a compact table with columns: Option | Candidate | Why It Matters
- Limit to A–E options maximum; omit table if a free-form answer is clearer
- Never ask the user to restate what they already said
- Avoid speculative categories (no hallucination). If uncertain, ask explicitly: "Confirm whether X belongs in scope."
Defaults when interaction impossible:
- Depth: Standard
- Audience: Reviewer (PR) if code-related; Author otherwise
- Focus: Top 2 relevance clusters
Output the questions (label Q1/Q2/Q3). After answers: if ≥2 scenario classes (Alternate / Exception / Recovery / Non-Functional domain) remain unclear, you MAY ask up to TWO more targeted follow‑ups (Q4/Q5) with a one-line justification each (e.g., "Unresolved recovery path risk"). Do not exceed five total questions. Skip escalation if user explicitly declines more.
3. **Understand user request**: Combine `$ARGUMENTS` + clarifying answers:
- Derive checklist theme (e.g., security, review, deploy, ux)
- Consolidate explicit must-have items mentioned by user
- Map focus selections to category scaffolding
- Infer any missing context from spec/plan/tasks (do NOT hallucinate)
4. **Load feature context**: Read from FEATURE_DIR:
- spec.md: Feature requirements and scope
- plan.md (if exists): Technical details, dependencies
- tasks.md (if exists): Implementation tasks
**Context Loading Strategy**:
- Load only necessary portions relevant to active focus areas (avoid full-file dumping)
- Prefer summarizing long sections into concise scenario/requirement bullets
- Use progressive disclosure: add follow-on retrieval only if gaps detected
- If source docs are large, generate interim summary items instead of embedding raw text
5. **Generate checklist** - Create "Unit Tests for Requirements":
- Create `FEATURE_DIR/checklists/` directory if it doesn't exist
- Generate unique checklist filename:
- Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`)
- Format: `[domain].md`
- File handling behavior:
- If file does NOT exist: Create new file and number items starting from CHK001
- If file exists: Append new items to existing file, continuing from the last CHK ID (e.g., if last item is CHK015, start new items at CHK016)
- Never delete or replace existing checklist content - always preserve and append
**CORE PRINCIPLE - Test the Requirements, Not the Implementation**:
Every checklist item MUST evaluate the REQUIREMENTS THEMSELVES for:
- **Completeness**: Are all necessary requirements present?
- **Clarity**: Are requirements unambiguous and specific?
- **Consistency**: Do requirements align with each other?
- **Measurability**: Can requirements be objectively verified?
- **Coverage**: Are all scenarios/edge cases addressed?
**Category Structure** - Group items by requirement quality dimensions:
- **Requirement Completeness** (Are all necessary requirements documented?)
- **Requirement Clarity** (Are requirements specific and unambiguous?)
- **Requirement Consistency** (Do requirements align without conflicts?)
- **Acceptance Criteria Quality** (Are success criteria measurable?)
- **Scenario Coverage** (Are all flows/cases addressed?)
- **Edge Case Coverage** (Are boundary conditions defined?)
- **Non-Functional Requirements** (Performance, Security, Accessibility, etc. - are they specified?)
- **Dependencies & Assumptions** (Are they documented and validated?)
- **Ambiguities & Conflicts** (What needs clarification?)
**HOW TO WRITE CHECKLIST ITEMS - "Unit Tests for English"**:
❌ **WRONG** (Testing implementation):
- "Verify landing page displays 3 episode cards"
- "Test hover states work on desktop"
- "Confirm logo click navigates home"
✅ **CORRECT** (Testing requirements quality):
- "Are the exact number and layout of featured episodes specified?" [Completeness]
- "Is 'prominent display' quantified with specific sizing/positioning?" [Clarity]
- "Are hover state requirements consistent across all interactive elements?" [Consistency]
- "Are keyboard navigation requirements defined for all interactive UI?" [Coverage]
- "Is the fallback behavior specified when logo image fails to load?" [Edge Cases]
- "Are loading states defined for asynchronous episode data?" [Completeness]
- "Does the spec define visual hierarchy for competing UI elements?" [Clarity]
**ITEM STRUCTURE**:
Each item should follow this pattern:
- Question format asking about requirement quality
- Focus on what's WRITTEN (or not written) in the spec/plan
- Include quality dimension in brackets [Completeness/Clarity/Consistency/etc.]
- Reference spec section `[Spec §X.Y]` when checking existing requirements
- Use `[Gap]` marker when checking for missing requirements
**EXAMPLES BY QUALITY DIMENSION**:
Completeness:
- "Are error handling requirements defined for all API failure modes? [Gap]"
- "Are accessibility requirements specified for all interactive elements? [Completeness]"
- "Are mobile breakpoint requirements defined for responsive layouts? [Gap]"
Clarity:
- "Is 'fast loading' quantified with specific timing thresholds? [Clarity, Spec §NFR-2]"
- "Are 'related episodes' selection criteria explicitly defined? [Clarity, Spec §FR-5]"
- "Is 'prominent' defined with measurable visual properties? [Ambiguity, Spec §FR-4]"
Consistency:
- "Do navigation requirements align across all pages? [Consistency, Spec §FR-10]"
- "Are card component requirements consistent between landing and detail pages? [Consistency]"
Coverage:
- "Are requirements defined for zero-state scenarios (no episodes)? [Coverage, Edge Case]"
- "Are concurrent user interaction scenarios addressed? [Coverage, Gap]"
- "Are requirements specified for partial data loading failures? [Coverage, Exception Flow]"
Measurability:
- "Are visual hierarchy requirements measurable/testable? [Acceptance Criteria, Spec §FR-1]"
- "Can 'balanced visual weight' be objectively verified? [Measurability, Spec §FR-2]"
**Scenario Classification & Coverage** (Requirements Quality Focus):
- Check if requirements exist for: Primary, Alternate, Exception/Error, Recovery, Non-Functional scenarios
- For each scenario class, ask: "Are [scenario type] requirements complete, clear, and consistent?"
- If scenario class missing: "Are [scenario type] requirements intentionally excluded or missing? [Gap]"
- Include resilience/rollback when state mutation occurs: "Are rollback requirements defined for migration failures? [Gap]"
**Traceability Requirements**:
- MINIMUM: ≥80% of items MUST include at least one traceability reference
- Each item should reference: spec section `[Spec §X.Y]`, or use markers: `[Gap]`, `[Ambiguity]`, `[Conflict]`, `[Assumption]`
- If no ID system exists: "Is a requirement & acceptance criteria ID scheme established? [Traceability]"
**Surface & Resolve Issues** (Requirements Quality Problems):
Ask questions about the requirements themselves:
- Ambiguities: "Is the term 'fast' quantified with specific metrics? [Ambiguity, Spec §NFR-1]"
- Conflicts: "Do navigation requirements conflict between §FR-10 and §FR-10a? [Conflict]"
- Assumptions: "Is the assumption of 'always available podcast API' validated? [Assumption]"
- Dependencies: "Are external podcast API requirements documented? [Dependency, Gap]"
- Missing definitions: "Is 'visual hierarchy' defined with measurable criteria? [Gap]"
**Content Consolidation**:
- Soft cap: If raw candidate items > 40, prioritize by risk/impact
- Merge near-duplicates checking the same requirement aspect
- If >5 low-impact edge cases, create one item: "Are edge cases X, Y, Z addressed in requirements? [Coverage]"
**🚫 ABSOLUTELY PROHIBITED** - These make it an implementation test, not a requirements test:
- ❌ Any item starting with "Verify", "Test", "Confirm", "Check" + implementation behavior
- ❌ References to code execution, user actions, system behavior
- ❌ "Displays correctly", "works properly", "functions as expected"
- ❌ "Click", "navigate", "render", "load", "execute"
- ❌ Test cases, test plans, QA procedures
- ❌ Implementation details (frameworks, APIs, algorithms)
**✅ REQUIRED PATTERNS** - These test requirements quality:
- ✅ "Are [requirement type] defined/specified/documented for [scenario]?"
- ✅ "Is [vague term] quantified/clarified with specific criteria?"
- ✅ "Are requirements consistent between [section A] and [section B]?"
- ✅ "Can [requirement] be objectively measured/verified?"
- ✅ "Are [edge cases/scenarios] addressed in requirements?"
- ✅ "Does the spec define [missing aspect]?"
6. **Structure Reference**: Generate the checklist following the canonical template in `.specify/templates/checklist-template.md` for title, meta section, category headings, and ID formatting. If template is unavailable, use: H1 title, purpose/created meta lines, `##` category sections containing `- [ ] CHK### <requirement item>` lines with globally incrementing IDs starting at CHK001.
7. **Report**: Output full path to checklist file, item count, and summarize whether the run created a new file or appended to an existing one. Summarize:
- Focus areas selected
- Depth level
- Actor/timing
- Any explicit user-specified must-have items incorporated
**Important**: Each `/speckit.checklist` command invocation uses a short, descriptive checklist filename and either creates a new file or appends to an existing one. This allows:
- Multiple checklists of different types (e.g., `ux.md`, `test.md`, `security.md`)
- Simple, memorable filenames that indicate checklist purpose
- Easy identification and navigation in the `checklists/` folder
To avoid clutter, use descriptive types and clean up obsolete checklists when done.
## Example Checklist Types & Sample Items
**UX Requirements Quality:** `ux.md`
Sample items (testing the requirements, NOT the implementation):
- "Are visual hierarchy requirements defined with measurable criteria? [Clarity, Spec §FR-1]"
- "Is the number and positioning of UI elements explicitly specified? [Completeness, Spec §FR-1]"
- "Are interaction state requirements (hover, focus, active) consistently defined? [Consistency]"
- "Are accessibility requirements specified for all interactive elements? [Coverage, Gap]"
- "Is fallback behavior defined when images fail to load? [Edge Case, Gap]"
- "Can 'prominent display' be objectively measured? [Measurability, Spec §FR-4]"
**API Requirements Quality:** `api.md`
Sample items:
- "Are error response formats specified for all failure scenarios? [Completeness]"
- "Are rate limiting requirements quantified with specific thresholds? [Clarity]"
- "Are authentication requirements consistent across all endpoints? [Consistency]"
- "Are retry/timeout requirements defined for external dependencies? [Coverage, Gap]"
- "Is versioning strategy documented in requirements? [Gap]"
**Performance Requirements Quality:** `performance.md`
Sample items:
- "Are performance requirements quantified with specific metrics? [Clarity]"
- "Are performance targets defined for all critical user journeys? [Coverage]"
- "Are performance requirements under different load conditions specified? [Completeness]"
- "Can performance requirements be objectively measured? [Measurability]"
- "Are degradation requirements defined for high-load scenarios? [Edge Case, Gap]"
**Security Requirements Quality:** `security.md`
Sample items:
- "Are authentication requirements specified for all protected resources? [Coverage]"
- "Are data protection requirements defined for sensitive information? [Completeness]"
- "Is the threat model documented and requirements aligned to it? [Traceability]"
- "Are security requirements consistent with compliance obligations? [Consistency]"
- "Are security failure/breach response requirements defined? [Gap, Exception Flow]"
## Anti-Examples: What NOT To Do
**❌ WRONG - These test implementation, not requirements:**
```markdown
- [ ] CHK001 - Verify landing page displays 3 episode cards [Spec §FR-001]
- [ ] CHK002 - Test hover states work correctly on desktop [Spec §FR-003]
- [ ] CHK003 - Confirm logo click navigates to home page [Spec §FR-010]
- [ ] CHK004 - Check that related episodes section shows 3-5 items [Spec §FR-005]
```
**✅ CORRECT - These test requirements quality:**
```markdown
- [ ] CHK001 - Are the number and layout of featured episodes explicitly specified? [Completeness, Spec §FR-001]
- [ ] CHK002 - Are hover state requirements consistently defined for all interactive elements? [Consistency, Spec §FR-003]
- [ ] CHK003 - Are navigation requirements clear for all clickable brand elements? [Clarity, Spec §FR-010]
- [ ] CHK004 - Is the selection criteria for related episodes documented? [Gap, Spec §FR-005]
- [ ] CHK005 - Are loading state requirements defined for asynchronous episode data? [Gap]
- [ ] CHK006 - Can "visual hierarchy" requirements be objectively measured? [Measurability, Spec §FR-001]
```
**Key Differences:**
- Wrong: Tests if the system works correctly
- Correct: Tests if the requirements are written correctly
- Wrong: Verification of behavior
- Correct: Validation of requirement quality
- Wrong: "Does it do X?"
- Correct: "Is X clearly specified?"
+181
View File
@@ -0,0 +1,181 @@
---
description: Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec.
handoffs:
- label: Build Technical Plan
agent: speckit.plan
prompt: Create a plan for the spec. I am building with...
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Outline
Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file.
Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/speckit.plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases.
Execution steps:
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --paths-only` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields:
- `FEATURE_DIR`
- `FEATURE_SPEC`
- (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.)
- If JSON parsing fails, abort and instruct user to re-run `/speckit.specify` or verify feature branch environment.
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked).
Functional Scope & Behavior:
- Core user goals & success criteria
- Explicit out-of-scope declarations
- User roles / personas differentiation
Domain & Data Model:
- Entities, attributes, relationships
- Identity & uniqueness rules
- Lifecycle/state transitions
- Data volume / scale assumptions
Interaction & UX Flow:
- Critical user journeys / sequences
- Error/empty/loading states
- Accessibility or localization notes
Non-Functional Quality Attributes:
- Performance (latency, throughput targets)
- Scalability (horizontal/vertical, limits)
- Reliability & availability (uptime, recovery expectations)
- Observability (logging, metrics, tracing signals)
- Security & privacy (authN/Z, data protection, threat assumptions)
- Compliance / regulatory constraints (if any)
Integration & External Dependencies:
- External services/APIs and failure modes
- Data import/export formats
- Protocol/versioning assumptions
Edge Cases & Failure Handling:
- Negative scenarios
- Rate limiting / throttling
- Conflict resolution (e.g., concurrent edits)
Constraints & Tradeoffs:
- Technical constraints (language, storage, hosting)
- Explicit tradeoffs or rejected alternatives
Terminology & Consistency:
- Canonical glossary terms
- Avoided synonyms / deprecated terms
Completion Signals:
- Acceptance criteria testability
- Measurable Definition of Done style indicators
Misc / Placeholders:
- TODO markers / unresolved decisions
- Ambiguous adjectives ("robust", "intuitive") lacking quantification
For each category with Partial or Missing status, add a candidate question opportunity unless:
- Clarification would not materially change implementation or validation strategy
- Information is better deferred to planning phase (note internally)
3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints:
- Maximum of 5 total questions across the whole session.
- Each question must be answerable with EITHER:
- A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR
- A one-word / short‑phrase answer (explicitly constrain: "Answer in <=5 words").
- Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation.
- Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved.
- Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness).
- Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests.
- If more than 5 categories remain unresolved, select the top 5 by (Impact \* Uncertainty) heuristic.
4. Sequential questioning loop (interactive):
- Present EXACTLY ONE question at a time.
- For multiple‑choice questions:
- **Analyze all options** and determine the **most suitable option** based on:
- Best practices for the project type
- Common patterns in similar implementations
- Risk reduction (security, performance, maintainability)
- Alignment with any explicit project goals or constraints visible in the spec
- Present your **recommended option prominently** at the top with clear reasoning (1-2 sentences explaining why this is the best choice).
- Format as: `**Recommended:** Option [X] - <reasoning>`
- Then render all options as a Markdown table:
| Option | Description |
| ------ | --------------------------------------------------------------------------------------------------- |
| A | <Option A description> |
| B | <Option B description> |
| C | <Option C description> (add D/E as needed up to 5) |
| Short | Provide a different short answer (<=5 words) (Include only if free-form alternative is appropriate) |
- After the table, add: `You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.`
- For short‑answer style (no meaningful discrete options):
- Provide your **suggested answer** based on best practices and context.
- Format as: `**Suggested:** <your proposed answer> - <brief reasoning>`
- Then output: `Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.`
- After the user answers:
- If the user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer.
- Otherwise, validate the answer maps to one option or fits the <=5 word constraint.
- If ambiguous, ask for a quick disambiguation (count still belongs to same question; do not advance).
- Once satisfactory, record it in working memory (do not yet write to disk) and move to the next queued question.
- Stop asking further questions when:
- All critical ambiguities resolved early (remaining queued items become unnecessary), OR
- User signals completion ("done", "good", "no more"), OR
- You reach 5 asked questions.
- Never reveal future queued questions in advance.
- If no valid questions exist at start, immediately report no critical ambiguities.
5. Integration after EACH accepted answer (incremental update approach):
- Maintain in-memory representation of the spec (loaded once at start) plus the raw file contents.
- For the first integrated answer in this session:
- Ensure a `## Clarifications` section exists (create it just after the highest-level contextual/overview section per the spec template if missing).
- Under it, create (if not present) a `### Session YYYY-MM-DD` subheading for today.
- Append a bullet line immediately after acceptance: `- Q: <question> → A: <final answer>`.
- Then immediately apply the clarification to the most appropriate section(s):
- Functional ambiguity → Update or add a bullet in Functional Requirements.
- User interaction / actor distinction → Update User Stories or Actors subsection (if present) with clarified role, constraint, or scenario.
- Data shape / entities → Update Data Model (add fields, types, relationships) preserving ordering; note added constraints succinctly.
- Non-functional constraint → Add/modify measurable criteria in Success Criteria > Measurable Outcomes (convert vague adjective to metric or explicit target).
- Edge case / negative flow → Add a new bullet under Edge Cases / Error Handling (or create such subsection if template provides placeholder for it).
- Terminology conflict → Normalize term across spec; retain original only if necessary by adding `(formerly referred to as "X")` once.
- If the clarification invalidates an earlier ambiguous statement, replace that statement instead of duplicating; leave no obsolete contradictory text.
- Save the spec file AFTER each integration to minimize risk of context loss (atomic overwrite).
- Preserve formatting: do not reorder unrelated sections; keep heading hierarchy intact.
- Keep each inserted clarification minimal and testable (avoid narrative drift).
6. Validation (performed after EACH write plus final pass):
- Clarifications session contains exactly one bullet per accepted answer (no duplicates).
- Total asked (accepted) questions ≤ 5.
- Updated sections contain no lingering vague placeholders the new answer was meant to resolve.
- No contradictory earlier statement remains (scan for now-invalid alternative choices removed).
- Markdown structure valid; only allowed new headings: `## Clarifications`, `### Session YYYY-MM-DD`.
- Terminology consistency: same canonical term used across all updated sections.
7. Write the updated spec back to `FEATURE_SPEC`.
8. Report completion (after questioning loop ends or early termination):
- Number of questions asked & answered.
- Path to updated spec.
- Sections touched (list names).
- Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
- If any Outstanding or Deferred remain, recommend whether to proceed to `/speckit.plan` or run `/speckit.clarify` again later post-plan.
- Suggested next command.
Behavior rules:
- If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
- If spec file missing, instruct user to run `/speckit.specify` first (do not create a new spec here).
- Never exceed 5 total asked questions (clarification retries for a single question do not count as new questions).
- Avoid speculative tech stack questions unless the absence blocks functional clarity.
- Respect user early termination signals ("stop", "done", "proceed").
- If no questions asked due to full coverage, output a compact coverage summary (all categories Clear) then suggest advancing.
- If quota reached with unresolved high-impact categories remaining, explicitly flag them under Deferred with rationale.
Context for prioritization: $ARGUMENTS
@@ -0,0 +1,84 @@
---
description: Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync.
handoffs:
- label: Build Specification
agent: speckit.specify
prompt: Implement the feature specification based on the updated constitution. I want to build...
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Outline
You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values, (b) fill the template precisely, and (c) propagate any amendments across dependent artifacts.
**Note**: If `.specify/memory/constitution.md` does not exist yet, it should have been initialized from `.specify/templates/constitution-template.md` during project setup. If it's missing, copy the template first.
Follow this execution flow:
1. Load the existing constitution at `.specify/memory/constitution.md`.
- Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`.
**IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly.
2. Collect/derive values for placeholders:
- If user input (conversation) supplies a value, use it.
- Otherwise infer from existing repo context (README, docs, prior constitution versions if embedded).
- For governance dates: `RATIFICATION_DATE` is the original adoption date (if unknown ask or mark TODO), `LAST_AMENDED_DATE` is today if changes are made, otherwise keep previous.
- `CONSTITUTION_VERSION` must increment according to semantic versioning rules:
- MAJOR: Backward incompatible governance/principle removals or redefinitions.
- MINOR: New principle/section added or materially expanded guidance.
- PATCH: Clarifications, wording, typo fixes, non-semantic refinements.
- If version bump type ambiguous, propose reasoning before finalizing.
3. Draft the updated constitution content:
- Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left).
- Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance.
- Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious.
- Ensure Governance section lists amendment procedure, versioning policy, and compliance review expectations.
4. Consistency propagation checklist (convert prior checklist into active validations):
- Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles.
- Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints.
- Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline).
- Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required.
- Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed.
5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update):
- Version change: old → new
- List of modified principles (old title → new title if renamed)
- Added sections
- Removed sections
- Templates requiring updates (✅ updated / ⚠ pending) with file paths
- Follow-up TODOs if any placeholders intentionally deferred.
6. Validation before final output:
- No remaining unexplained bracket tokens.
- Version line matches report.
- Dates ISO format YYYY-MM-DD.
- Principles are declarative, testable, and free of vague language ("should" → replace with MUST/SHOULD rationale where appropriate).
7. Write the completed constitution back to `.specify/memory/constitution.md` (overwrite).
8. Output a final summary to the user with:
- New version and bump rationale.
- Any files flagged for manual follow-up.
- Suggested commit message (e.g., `docs: amend constitution to vX.Y.Z (principle additions + governance update)`).
Formatting & Style Requirements:
- Use Markdown headings exactly as in the template (do not demote/promote levels).
- Wrap long rationale lines to keep readability (<100 chars ideally) but do not hard enforce with awkward breaks.
- Keep a single blank line between sections.
- Avoid trailing whitespace.
If the user supplies partial updates (e.g., only one principle revision), still perform validation and version decision steps.
If critical info missing (e.g., ratification date truly unknown), insert `TODO(<FIELD_NAME>): explanation` and include in the Sync Impact Report under deferred items.
Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file.
+207
View File
@@ -0,0 +1,207 @@
---
description: Execute the implementation plan by processing and executing all tasks defined in tasks.md
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Pre-Execution Checks
**Check for extension hooks (before implementation)**:
- Check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.before_implement` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Pre-Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Pre-Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
Wait for the result of the hook command before proceeding to the Outline.
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
## Outline
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
2. **Check checklists status** (if FEATURE_DIR/checklists/ exists):
- Scan all checklist files in the checklists/ directory
- For each checklist, count:
- Total items: All lines matching `- [ ]` or `- [X]` or `- [x]`
- Completed items: Lines matching `- [X]` or `- [x]`
- Incomplete items: Lines matching `- [ ]`
- Create a status table:
```text
| Checklist | Total | Completed | Incomplete | Status |
|-----------|-------|-----------|------------|--------|
| ux.md | 12 | 12 | 0 | ✓ PASS |
| test.md | 8 | 5 | 3 | ✗ FAIL |
| security.md | 6 | 6 | 0 | ✓ PASS |
```
- Calculate overall status:
- **PASS**: All checklists have 0 incomplete items
- **FAIL**: One or more checklists have incomplete items
- **If any checklist is incomplete**:
- Display the table with incomplete item counts
- **STOP** and ask: "Some checklists are incomplete. Do you want to proceed with implementation anyway? (yes/no)"
- Wait for user response before continuing
- If user says "no" or "wait" or "stop", halt execution
- If user says "yes" or "proceed" or "continue", proceed to step 3
- **If all checklists are complete**:
- Display the table showing all checklists passed
- Automatically proceed to step 3
3. Load and analyze the implementation context:
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
- **IF EXISTS**: Read data-model.md for entities and relationships
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
- **IF EXISTS**: Read research.md for technical decisions and constraints
- **IF EXISTS**: Read quickstart.md for integration scenarios
4. **Project Setup Verification**:
- **REQUIRED**: Create/verify ignore files based on actual project setup:
**Detection & Creation Logic**:
- Check if the following command succeeds to determine if the repository is a git repo (create/verify .gitignore if so):
```sh
git rev-parse --git-dir 2>/dev/null
```
- Check if Dockerfile\* exists or Docker in plan.md → create/verify .dockerignore
- Check if .eslintrc\* exists → create/verify .eslintignore
- Check if eslint.config.\* exists → ensure the config's `ignores` entries cover required patterns
- Check if .prettierrc\* exists → create/verify .prettierignore
- Check if .npmrc or package.json exists → create/verify .npmignore (if publishing)
- Check if terraform files (\*.tf) exist → create/verify .terraformignore
- Check if .helmignore needed (helm charts present) → create/verify .helmignore
**If ignore file already exists**: Verify it contains essential patterns, append missing critical patterns only
**If ignore file missing**: Create with full pattern set for detected technology
**Common Patterns by Technology** (from plan.md tech stack):
- **Node.js/JavaScript/TypeScript**: `node_modules/`, `dist/`, `build/`, `*.log`, `.env*`
- **Python**: `__pycache__/`, `*.pyc`, `.venv/`, `venv/`, `dist/`, `*.egg-info/`
- **Java**: `target/`, `*.class`, `*.jar`, `.gradle/`, `build/`
- **C#/.NET**: `bin/`, `obj/`, `*.user`, `*.suo`, `packages/`
- **Go**: `*.exe`, `*.test`, `vendor/`, `*.out`
- **Ruby**: `.bundle/`, `log/`, `tmp/`, `*.gem`, `vendor/bundle/`
- **PHP**: `vendor/`, `*.log`, `*.cache`, `*.env`
- **Rust**: `target/`, `debug/`, `release/`, `*.rs.bk`, `*.rlib`, `*.prof*`, `.idea/`, `*.log`, `.env*`
- **Kotlin**: `build/`, `out/`, `.gradle/`, `.idea/`, `*.class`, `*.jar`, `*.iml`, `*.log`, `.env*`
- **C++**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.so`, `*.a`, `*.exe`, `*.dll`, `.idea/`, `*.log`, `.env*`
- **C**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.a`, `*.so`, `*.exe`, `*.dll`, `autom4te.cache/`, `config.status`, `config.log`, `.idea/`, `*.log`, `.env*`
- **Swift**: `.build/`, `DerivedData/`, `*.swiftpm/`, `Packages/`
- **R**: `.Rproj.user/`, `.Rhistory`, `.RData`, `.Ruserdata`, `*.Rproj`, `packrat/`, `renv/`
- **Universal**: `.DS_Store`, `Thumbs.db`, `*.tmp`, `*.swp`, `.vscode/`, `.idea/`
**Tool-Specific Patterns**:
- **Docker**: `node_modules/`, `.git/`, `Dockerfile*`, `.dockerignore`, `*.log*`, `.env*`, `coverage/`
- **ESLint**: `node_modules/`, `dist/`, `build/`, `coverage/`, `*.min.js`
- **Prettier**: `node_modules/`, `dist/`, `build/`, `coverage/`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
- **Terraform**: `.terraform/`, `*.tfstate*`, `*.tfvars`, `.terraform.lock.hcl`
- **Kubernetes/k8s**: `*.secret.yaml`, `secrets/`, `.kube/`, `kubeconfig*`, `*.key`, `*.crt`
5. Parse tasks.md structure and extract:
- **Task phases**: Setup, Tests, Core, Integration, Polish
- **Task dependencies**: Sequential vs parallel execution rules
- **Task details**: ID, description, file paths, parallel markers [P]
- **Execution flow**: Order and dependency requirements
6. Execute implementation following the task plan:
- **Phase-by-phase execution**: Complete each phase before moving to the next
- **Respect dependencies**: Run sequential tasks in order, parallel tasks [P] can run together
- **Follow TDD approach**: Execute test tasks before their corresponding implementation tasks
- **File-based coordination**: Tasks affecting the same files must run sequentially
- **Validation checkpoints**: Verify each phase completion before proceeding
7. Implementation execution rules:
- **Setup first**: Initialize project structure, dependencies, configuration
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
- **Core development**: Implement models, services, CLI commands, endpoints
- **Integration work**: Database connections, middleware, logging, external services
- **Polish and validation**: Unit tests, performance optimization, documentation
8. Progress tracking and error handling:
- Report progress after each completed task
- Halt execution if any non-parallel task fails
- For parallel tasks [P], continue with successful tasks, report failed ones
- Provide clear error messages with context for debugging
- Suggest next steps if implementation cannot proceed
- **IMPORTANT** For completed tasks, make sure to mark the task off as [X] in the tasks file.
9. Completion validation:
- Verify all required tasks are completed
- Check that implemented features match the original specification
- Validate that tests pass and coverage meets requirements
- Confirm the implementation follows the technical plan
- Report final status with summary of completed work
Note: This command assumes a complete task breakdown exists in tasks.md. If tasks are incomplete or missing, suggest running `/speckit.tasks` first to regenerate the task list.
10. **Check for extension hooks**: After completion validation, check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.after_implement` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
+162
View File
@@ -0,0 +1,162 @@
---
description: Execute the implementation planning workflow using the plan template to generate design artifacts.
handoffs:
- label: Create Tasks
agent: speckit.tasks
prompt: Break the plan into tasks
send: true
- label: Create Checklist
agent: speckit.checklist
prompt: Create a checklist for the following domain...
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Pre-Execution Checks
**Check for extension hooks (before planning)**:
- Check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.before_plan` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Pre-Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Pre-Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
Wait for the result of the hook command before proceeding to the Outline.
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
## Outline
1. **Setup**: Run `.specify/scripts/bash/setup-plan.sh --json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
- Fill Constitution Check section from constitution
- Evaluate gates (ERROR if violations unjustified)
- Phase 0: Generate research.md (resolve all NEEDS CLARIFICATION)
- Phase 1: Generate data-model.md, contracts/, quickstart.md
- Phase 1: Update agent context by running the agent script
- Re-evaluate Constitution Check post-design
4. **Stop and report**: Command ends after Phase 2 planning. Report branch, IMPL_PLAN path, and generated artifacts.
5. **Check for extension hooks**: After reporting, check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.after_plan` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
## Phases
### Phase 0: Outline & Research
1. **Extract unknowns from Technical Context** above:
- For each NEEDS CLARIFICATION → research task
- For each dependency → best practices task
- For each integration → patterns task
2. **Generate and dispatch research agents**:
```text
For each unknown in Technical Context:
Task: "Research {unknown} for {feature context}"
For each technology choice:
Task: "Find best practices for {tech} in {domain}"
```
3. **Consolidate findings** in `research.md` using format:
- Decision: [what was chosen]
- Rationale: [why chosen]
- Alternatives considered: [what else evaluated]
**Output**: research.md with all NEEDS CLARIFICATION resolved
### Phase 1: Design & Contracts
**Prerequisites:** `research.md` complete
1. **Extract entities from feature spec** → `data-model.md`:
- Entity name, fields, relationships
- Validation rules from requirements
- State transitions if applicable
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
- Identify what interfaces the project exposes to users or other systems
- Document the contract format appropriate for the project type
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
- Skip if project is purely internal (build scripts, one-off tools, etc.)
3. **Agent context update**:
- Run `.specify/scripts/bash/update-agent-context.sh copilot`
- These scripts detect which AI agent is in use
- Update the appropriate agent-specific context file
- Add only new technology from current plan
- Preserve manual additions between markers
**Output**: data-model.md, /contracts/\*, quickstart.md, agent-specific file
## Key rules
- Use absolute paths
- ERROR on gate failures or unresolved clarifications
+313
View File
@@ -0,0 +1,313 @@
---
description: Create or update the feature specification from a natural language feature description.
handoffs:
- label: Build Technical Plan
agent: speckit.plan
prompt: Create a plan for the spec. I am building with...
- label: Clarify Spec Requirements
agent: speckit.clarify
prompt: Clarify specification requirements
send: true
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Pre-Execution Checks
**Check for extension hooks (before specification)**:
- Check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.before_specify` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Pre-Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Pre-Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
Wait for the result of the hook command before proceeding to the Outline.
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
## Outline
The text the user typed after `/speckit.specify` in the triggering message **is** the feature description. Assume you always have it available in this conversation even if `$ARGUMENTS` appears literally below. Do not ask the user to repeat it unless they provided an empty command.
Given that feature description, do this:
1. **Generate a concise short name** (2-4 words) for the branch:
- Analyze the feature description and extract the most meaningful keywords
- Create a 2-4 word short name that captures the essence of the feature
- Use action-noun format when possible (e.g., "add-user-auth", "fix-payment-bug")
- Preserve technical terms and acronyms (OAuth2, API, JWT, etc.)
- Keep it concise but descriptive enough to understand the feature at a glance
- Examples:
- "I want to add user authentication" → "user-auth"
- "Implement OAuth2 integration for the API" → "oauth2-api-integration"
- "Create a dashboard for analytics" → "analytics-dashboard"
- "Fix payment processing timeout bug" → "fix-payment-timeout"
2. **Create the feature branch** by running the script with `--short-name` (and `--json`). In sequential mode, do NOT pass `--number` — the script auto-detects the next available number. In timestamp mode, the script generates a `YYYYMMDD-HHMMSS` prefix automatically:
**Branch numbering mode**: Before running the script, check if `.specify/init-options.json` exists and read the `branch_numbering` value.
- If `"timestamp"`, add `--timestamp` (Bash) or `-Timestamp` (PowerShell) to the script invocation
- If `"sequential"` or absent, do not add any extra flag (default behavior)
- Bash example: `.specify/scripts/bash/create-new-feature.sh "$ARGUMENTS" --json --short-name "user-auth" "Add user authentication"`
- Bash (timestamp): `.specify/scripts/bash/create-new-feature.sh "$ARGUMENTS" --json --timestamp --short-name "user-auth" "Add user authentication"`
- PowerShell example: `.specify/scripts/bash/create-new-feature.sh "$ARGUMENTS" -Json -ShortName "user-auth" "Add user authentication"`
- PowerShell (timestamp): `.specify/scripts/bash/create-new-feature.sh "$ARGUMENTS" -Json -Timestamp -ShortName "user-auth" "Add user authentication"`
**IMPORTANT**:
- Do NOT pass `--number` — the script determines the correct next number automatically
- Always include the JSON flag (`--json` for Bash, `-Json` for PowerShell) so the output can be parsed reliably
- You must only ever run this script once per feature
- The JSON is provided in the terminal as output - always refer to it to get the actual content you're looking for
- The JSON output will contain BRANCH_NAME and SPEC_FILE paths
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot")
3. Load `.specify/templates/spec-template.md` to understand required sections.
4. Follow this execution flow:
1. Parse user description from Input
If empty: ERROR "No feature description provided"
2. Extract key concepts from description
Identify: actors, actions, data, constraints
3. For unclear aspects:
- Make informed guesses based on context and industry standards
- Only mark with [NEEDS CLARIFICATION: specific question] if:
- The choice significantly impacts feature scope or user experience
- Multiple reasonable interpretations exist with different implications
- No reasonable default exists
- **LIMIT: Maximum 3 [NEEDS CLARIFICATION] markers total**
- Prioritize clarifications by impact: scope > security/privacy > user experience > technical details
4. Fill User Scenarios & Testing section
If no clear user flow: ERROR "Cannot determine user scenarios"
5. Generate Functional Requirements
Each requirement must be testable
Use reasonable defaults for unspecified details (document assumptions in Assumptions section)
6. Define Success Criteria
Create measurable, technology-agnostic outcomes
Include both quantitative metrics (time, performance, volume) and qualitative measures (user satisfaction, task completion)
Each criterion must be verifiable without implementation details
7. Identify Key Entities (if data involved)
8. Return: SUCCESS (spec ready for planning)
5. Write the specification to SPEC_FILE using the template structure, replacing placeholders with concrete details derived from the feature description (arguments) while preserving section order and headings.
6. **Specification Quality Validation**: After writing the initial spec, validate it against quality criteria:
a. **Create Spec Quality Checklist**: Generate a checklist file at `FEATURE_DIR/checklists/requirements.md` using the checklist template structure with these validation items:
```markdown
# Specification Quality Checklist: [FEATURE NAME]
**Purpose**: Validate specification completeness and quality before proceeding to planning
**Created**: [DATE]
**Feature**: [Link to spec.md]
## Content Quality
- [ ] No implementation details (languages, frameworks, APIs)
- [ ] Focused on user value and business needs
- [ ] Written for non-technical stakeholders
- [ ] All mandatory sections completed
## Requirement Completeness
- [ ] No [NEEDS CLARIFICATION] markers remain
- [ ] Requirements are testable and unambiguous
- [ ] Success criteria are measurable
- [ ] Success criteria are technology-agnostic (no implementation details)
- [ ] All acceptance scenarios are defined
- [ ] Edge cases are identified
- [ ] Scope is clearly bounded
- [ ] Dependencies and assumptions identified
## Feature Readiness
- [ ] All functional requirements have clear acceptance criteria
- [ ] User scenarios cover primary flows
- [ ] Feature meets measurable outcomes defined in Success Criteria
- [ ] No implementation details leak into specification
## Notes
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
```
b. **Run Validation Check**: Review the spec against each checklist item:
- For each item, determine if it passes or fails
- Document specific issues found (quote relevant spec sections)
c. **Handle Validation Results**:
- **If all items pass**: Mark checklist complete and proceed to step 7
- **If items fail (excluding [NEEDS CLARIFICATION])**:
1. List the failing items and specific issues
2. Update the spec to address each issue
3. Re-run validation until all items pass (max 3 iterations)
4. If still failing after 3 iterations, document remaining issues in checklist notes and warn user
- **If [NEEDS CLARIFICATION] markers remain**:
1. Extract all [NEEDS CLARIFICATION: ...] markers from the spec
2. **LIMIT CHECK**: If more than 3 markers exist, keep only the 3 most critical (by scope/security/UX impact) and make informed guesses for the rest
3. For each clarification needed (max 3), present options to user in this format:
```markdown
## Question [N]: [Topic]
**Context**: [Quote relevant spec section]
**What we need to know**: [Specific question from NEEDS CLARIFICATION marker]
**Suggested Answers**:
| Option | Answer | Implications |
| ------ | ------------------------- | ------------------------------------- |
| A | [First suggested answer] | [What this means for the feature] |
| B | [Second suggested answer] | [What this means for the feature] |
| C | [Third suggested answer] | [What this means for the feature] |
| Custom | Provide your own answer | [Explain how to provide custom input] |
**Your choice**: _[Wait for user response]_
```
4. **CRITICAL - Table Formatting**: Ensure markdown tables are properly formatted:
- Use consistent spacing with pipes aligned
- Each cell should have spaces around content: `| Content |` not `|Content|`
- Header separator must have at least 3 dashes: `|--------|`
- Test that the table renders correctly in markdown preview
5. Number questions sequentially (Q1, Q2, Q3 - max 3 total)
6. Present all questions together before waiting for responses
7. Wait for user to respond with their choices for all questions (e.g., "Q1: A, Q2: Custom - [details], Q3: B")
8. Update the spec by replacing each [NEEDS CLARIFICATION] marker with the user's selected or provided answer
9. Re-run validation after all clarifications are resolved
d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
7. Report completion with branch name, spec file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
8. **Check for extension hooks**: After reporting completion, check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.after_specify` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
**NOTE:** The script creates and checks out the new branch and initializes the spec file before writing.
## Quick Guidelines
- Focus on **WHAT** users need and **WHY**.
- Avoid HOW to implement (no tech stack, APIs, code structure).
- Written for business stakeholders, not developers.
- DO NOT create any checklists that are embedded in the spec. That will be a separate command.
### Section Requirements
- **Mandatory sections**: Must be completed for every feature
- **Optional sections**: Include only when relevant to the feature
- When a section doesn't apply, remove it entirely (don't leave as "N/A")
### For AI Generation
When creating this spec from a user prompt:
1. **Make informed guesses**: Use context, industry standards, and common patterns to fill gaps
2. **Document assumptions**: Record reasonable defaults in the Assumptions section
3. **Limit clarifications**: Maximum 3 [NEEDS CLARIFICATION] markers - use only for critical decisions that:
- Significantly impact feature scope or user experience
- Have multiple reasonable interpretations with different implications
- Lack any reasonable default
4. **Prioritize clarifications**: scope > security/privacy > user experience > technical details
5. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item
6. **Common areas needing clarification** (only if no reasonable default exists):
- Feature scope and boundaries (include/exclude specific use cases)
- User types and permissions (if multiple conflicting interpretations possible)
- Security/compliance requirements (when legally/financially significant)
**Examples of reasonable defaults** (don't ask about these):
- Data retention: Industry-standard practices for the domain
- Performance targets: Standard web/mobile app expectations unless specified
- Error handling: User-friendly messages with appropriate fallbacks
- Authentication method: Standard session-based or OAuth2 for web apps
- Integration patterns: Use project-appropriate patterns (REST/GraphQL for web services, function calls for libraries, CLI args for tools, etc.)
### Success Criteria Guidelines
Success criteria must be:
1. **Measurable**: Include specific metrics (time, percentage, count, rate)
2. **Technology-agnostic**: No mention of frameworks, languages, databases, or tools
3. **User-focused**: Describe outcomes from user/business perspective, not system internals
4. **Verifiable**: Can be tested/validated without knowing implementation details
**Good examples**:
- "Users can complete checkout in under 3 minutes"
- "System supports 10,000 concurrent users"
- "95% of searches return results in under 1 second"
- "Task completion rate improves by 40%"
**Bad examples** (implementation-focused):
- "API response time is under 200ms" (too technical, use "Users see results instantly")
- "Database can handle 1000 TPS" (implementation detail, use user-facing metric)
- "React components render efficiently" (framework-specific)
- "Redis cache hit rate above 80%" (technology-specific)
+209
View File
@@ -0,0 +1,209 @@
---
description: Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts.
handoffs:
- label: Analyze For Consistency
agent: speckit.analyze
prompt: Run a project analysis for consistency
send: true
- label: Implement Project
agent: speckit.implement
prompt: Start the implementation in phases
send: true
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Pre-Execution Checks
**Check for extension hooks (before tasks generation)**:
- Check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.before_tasks` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Pre-Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Pre-Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
Wait for the result of the hook command before proceeding to the Outline.
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
## Outline
1. **Setup**: Run `.specify/scripts/bash/check-prerequisites.sh --json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
2. **Load design documents**: Read from FEATURE_DIR:
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
- Note: Not all projects have all documents. Generate tasks based on what's available.
3. **Execute task generation workflow**:
- Load plan.md and extract tech stack, libraries, project structure
- Load spec.md and extract user stories with their priorities (P1, P2, P3, etc.)
- If data-model.md exists: Extract entities and map to user stories
- If contracts/ exists: Map interface contracts to user stories
- If research.md exists: Extract decisions for setup tasks
- Generate tasks organized by user story (see Task Generation Rules below)
- Generate dependency graph showing user story completion order
- Create parallel execution examples per user story
- Validate task completeness (each user story has all needed tasks, independently testable)
4. **Generate tasks.md**: Use `.specify/templates/tasks-template.md` as structure, fill with:
- Correct feature name from plan.md
- Phase 1: Setup tasks (project initialization)
- Phase 2: Foundational tasks (blocking prerequisites for all user stories)
- Phase 3+: One phase per user story (in priority order from spec.md)
- Each phase includes: story goal, independent test criteria, tests (if requested), implementation tasks
- Final Phase: Polish & cross-cutting concerns
- All tasks must follow the strict checklist format (see Task Generation Rules below)
- Clear file paths for each task
- Dependencies section showing story completion order
- Parallel execution examples per story
- Implementation strategy section (MVP first, incremental delivery)
5. **Report**: Output path to generated tasks.md and summary:
- Total task count
- Task count per user story
- Parallel opportunities identified
- Independent test criteria for each story
- Suggested MVP scope (typically just User Story 1)
- Format validation: Confirm ALL tasks follow the checklist format (checkbox, ID, labels, file paths)
6. **Check for extension hooks**: After tasks.md is generated, check if `.specify/extensions.yml` exists in the project root.
- If it exists, read it and look for entries under the `hooks.after_tasks` key
- If the YAML cannot be parsed or is invalid, skip hook checking silently and continue normally
- Filter out hooks where `enabled` is explicitly `false`. Treat hooks without an `enabled` field as enabled by default.
- For each remaining hook, do **not** attempt to interpret or evaluate hook `condition` expressions:
- If the hook has no `condition` field, or it is null/empty, treat the hook as executable
- If the hook defines a non-empty `condition`, skip the hook and leave condition evaluation to the HookExecutor implementation
- For each executable hook, output the following based on its `optional` flag:
- **Optional hook** (`optional: true`):
```
## Extension Hooks
**Optional Hook**: {extension}
Command: `/{command}`
Description: {description}
Prompt: {prompt}
To execute: `/{command}`
```
- **Mandatory hook** (`optional: false`):
```
## Extension Hooks
**Automatic Hook**: {extension}
Executing: `/{command}`
EXECUTE_COMMAND: {command}
```
- If no hooks are registered or `.specify/extensions.yml` does not exist, skip silently
Context for task generation: $ARGUMENTS
The tasks.md should be immediately executable - each task must be specific enough that an LLM can complete it without additional context.
## Task Generation Rules
**CRITICAL**: Tasks MUST be organized by user story to enable independent implementation and testing.
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
### Checklist Format (REQUIRED)
Every task MUST strictly follow this format:
```text
- [ ] [TaskID] [P?] [Story?] Description with file path
```
**Format Components**:
1. **Checkbox**: ALWAYS start with `- [ ]` (markdown checkbox)
2. **Task ID**: Sequential number (T001, T002, T003...) in execution order
3. **[P] marker**: Include ONLY if task is parallelizable (different files, no dependencies on incomplete tasks)
4. **[Story] label**: REQUIRED for user story phase tasks only
- Format: [US1], [US2], [US3], etc. (maps to user stories from spec.md)
- Setup phase: NO story label
- Foundational phase: NO story label
- User Story phases: MUST have story label
- Polish phase: NO story label
5. **Description**: Clear action with exact file path
**Examples**:
- ✅ CORRECT: `- [ ] T001 Create project structure per implementation plan`
- ✅ CORRECT: `- [ ] T005 [P] Implement authentication middleware in src/middleware/auth.py`
- ✅ CORRECT: `- [ ] T012 [P] [US1] Create User model in src/models/user.py`
- ✅ CORRECT: `- [ ] T014 [US1] Implement UserService in src/services/user_service.py`
- ❌ WRONG: `- [ ] Create User model` (missing ID and Story label)
- ❌ WRONG: `T001 [US1] Create model` (missing checkbox)
- ❌ WRONG: `- [ ] [US1] Create User model` (missing Task ID)
- ❌ WRONG: `- [ ] T001 [US1] Create model` (missing file path)
### Task Organization
1. **From User Stories (spec.md)** - PRIMARY ORGANIZATION:
- Each user story (P1, P2, P3...) gets its own phase
- Map all related components to their story:
- Models needed for that story
- Services needed for that story
- Interfaces/UI needed for that story
- If tests requested: Tests specific to that story
- Mark story dependencies (most stories should be independent)
2. **From Contracts**:
- Map each interface contract → to the user story it serves
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
3. **From Data Model**:
- Map each entity to the user story(ies) that need it
- If entity serves multiple stories: Put in earliest story or Setup phase
- Relationships → service layer tasks in appropriate story phase
4. **From Setup/Infrastructure**:
- Shared infrastructure → Setup phase (Phase 1)
- Foundational/blocking tasks → Foundational phase (Phase 2)
- Story-specific setup → within that story's phase
### Phase Structure
- **Phase 1**: Setup (project initialization)
- **Phase 2**: Foundational (blocking prerequisites - MUST complete before user stories)
- **Phase 3+**: User Stories in priority order (P1, P2, P3...)
- Within each story: Tests (if requested) → Models → Services → Endpoints → Integration
- Each phase should be a complete, independently testable increment
- **Final Phase**: Polish & Cross-Cutting Concerns
@@ -0,0 +1,30 @@
---
description: Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts.
tools: ["github/github-mcp-server/issue_write"]
---
## User Input
```text
$ARGUMENTS
```
You **MUST** consider the user input before proceeding (if not empty).
## Outline
1. Run `.specify/scripts/bash/check-prerequisites.sh --json --require-tasks --include-tasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
1. From the executed script, extract the path to **tasks**.
1. Get the Git remote by running:
```bash
git config --get remote.origin.url
```
> [!CAUTION]
> ONLY PROCEED TO NEXT STEPS IF THE REMOTE IS A GITHUB URL
1. For each task in the list, use the GitHub MCP server to create a new issue in the repository that is representative of the Git remote.
> [!CAUTION]
> UNDER NO CIRCUMSTANCES EVER CREATE ISSUES IN REPOSITORIES THAT DO NOT MATCH THE REMOTE URL
-95
View File
@@ -135,99 +135,6 @@ On top of authorization, any remote admin message that **mutates** state (not a
- **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
- **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
## NodeDB Layout (v25)
`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
### Slim `NodeInfoLite`
`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
```cpp
nodeInfoLiteHasUser(n) // bit 5 — user fields populated
nodeInfoLiteIsFavorite(n) // bit 3
nodeInfoLiteIsIgnored(n) // bit 4
nodeInfoLiteIsMuted(n) // bit 1
nodeInfoLiteIsLicensed(n) // bit 6 — Ham mode peer
nodeInfoLiteIsKeyManuallyVerified(n) // bit 0
nodeInfoLiteHasIsUnmessagable(n) // bit 8 — "is_unmessagable was sent"
nodeInfoLiteIsUnmessagable(n) // bit 7
// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true); // setter
```
### Satellite stores
Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its own build flag:
| Map | Value type | Build flag |
| ----------------- | ------------------------------- | ---------------------------------- |
| `nodePositions` | `meshtastic_PositionLite` | `MESHTASTIC_EXCLUDE_POSITIONDB` |
| `nodeTelemetry` | `meshtastic_DeviceMetrics` | `MESHTASTIC_EXCLUDE_TELEMETRYDB` |
| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
| `nodeStatus` | `meshtastic_StatusMessage` | `MESHTASTIC_EXCLUDE_STATUSDB` |
Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
### Accessor convention
**Never hand out pointers into the maps.** Use the copy-out accessors on `NodeDB`:
```cpp
bool copyNodePosition(NodeNum, meshtastic_PositionLite &out) const;
bool copyNodeTelemetry(NodeNum, meshtastic_DeviceMetrics &out) const;
bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out) const;
```
Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
### Eviction
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
1. Config / module-config / channel / metadata segments (same as before).
2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
Special nonces that still mean something:
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
### v24 → v25 migration
The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not in `NodeDB.cpp`. It owns the `meshtastic_NodeDatabase_Legacy` callback and `NodeDB::migrateLegacyNodeDatabase()`. The legacy proto descriptor is `protobufs/meshtastic/deviceonly_legacy.proto` (only included by the migration TU). The boot path peeks the file's leading version tag, runs the migration if `version < 25`, then re-saves in v25 layout. The legacy descriptor is scheduled for removal once `DEVICESTATE_MIN_VER` is bumped.
### Read-site rules of thumb
- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
- Never `node->user.long_name` — `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
## Project Structure
```
@@ -289,8 +196,6 @@ firmware/
- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
- Use `assert()` for invariants that should never fail
- C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
### Naming Conventions
@@ -0,0 +1,3 @@
---
agent: hardware-support
---
@@ -0,0 +1,3 @@
---
agent: speckit.analyze
---
@@ -0,0 +1,3 @@
---
agent: speckit.checklist
---
@@ -0,0 +1,3 @@
---
agent: speckit.clarify
---
@@ -0,0 +1,3 @@
---
agent: speckit.constitution
---
@@ -0,0 +1,3 @@
---
agent: speckit.implement
---
+3
View File
@@ -0,0 +1,3 @@
---
agent: speckit.plan
---
@@ -0,0 +1,3 @@
---
agent: speckit.specify
---
+3
View File
@@ -0,0 +1,3 @@
---
agent: speckit.tasks
---
@@ -0,0 +1,3 @@
---
agent: speckit.taskstoissues
---
+25 -39
View File
@@ -4,14 +4,9 @@ on:
workflow_dispatch:
inputs:
# trunk-ignore(checkov/CKV_GHA_7)
target:
type: string
required: false
description: Choose the target board, e.g. nrf52_promicro_diy_tcxo. If blank, will find available targets.
arch:
type: choice
options:
- all
- esp32
- esp32s3
- esp32c3
@@ -20,18 +15,32 @@ on:
- rp2040
- rp2350
- stm32
description: Choose an arch to limit the search, or 'all' to search all architectures.
default: all
target:
type: string
required: false
description: Choose the target board, e.g. nrf52_promicro_diy_tcxo. If blank, will find available targets.
# find-target:
# type: boolean
# default: true
# description: 'Find the available targets'
permissions: read-all
jobs:
find-targets:
if: ${{ inputs.target == '' }}
strategy:
fail-fast: false
matrix:
arch:
- all
- esp32
- esp32s3
- esp32c3
- esp32c6
- nrf52840
- rp2040
- rp2350
- stm32
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
@@ -42,37 +51,14 @@ jobs:
- run: pip install -U platformio
- name: Generate matrix
id: jsonStep
env:
BUILDTARGET: ${{ inputs.target }}
MATRIXARCH: ${{ inputs.arch }}
run: |
TARGETS=$(./bin/generate_ci_matrix.py ${{matrix.arch}} --level extra)
if [ "$BUILDTARGET" = "" ]; then
echo "Name: $GITHUB_REF_NAME" >> $GITHUB_STEP_SUMMARY
echo "Base: $GITHUB_BASE_REF" >> $GITHUB_STEP_SUMMARY
echo "Arch: $MATRIXARCH" >> $GITHUB_STEP_SUMMARY
echo "Ref: $GITHUB_REF" >> $GITHUB_STEP_SUMMARY
echo "## 🎯 The following target boards are available to build:" >> $GITHUB_STEP_SUMMARY
echo "| Platform | Board |" >> $GITHUB_STEP_SUMMARY
echo "| -------- | ----- |" >> $GITHUB_STEP_SUMMARY
echo $TARGETS | jq -r 'sort_by(.board) | sort_by(.platform) |.[] | "| " + .platform + " | " + .board + " |" ' >> $GITHUB_STEP_SUMMARY
else
echo "We build this one:" >> $GITHUB_STEP_SUMMARY
ARCH=$(echo "$TARGETS" | jq --arg BUILDTARGET "$BUILDTARGET" -r '.[] | select(.board==$BUILDTARGET) | .platform')
echo "| Platform | Board |" >> $GITHUB_STEP_SUMMARY
echo "| -------- | ----- |" >> $GITHUB_STEP_SUMMARY
echo "| $ARCH | "$BUILDTARGET" |" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [[ "$ARCH" == "" ]]; then
echo "## ❌ Error: Target "$BUILDTARGET" not found!" >> $GITHUB_STEP_SUMMARY
else
echo "## ✅ Target "$BUILDTARGET" found, proceeding to build." >> $GITHUB_STEP_SUMMARY
fi
echo "You may need to refresh this page to make the built firmware appear below." >> $GITHUB_STEP_SUMMARY
echo "arch=$ARCH" >> $GITHUB_OUTPUT
fi
outputs:
arch: ${{ steps.jsonStep.outputs.arch }}
echo "Name: $GITHUB_REF_NAME" >> $GITHUB_STEP_SUMMARY
echo "Base: $GITHUB_BASE_REF" >> $GITHUB_STEP_SUMMARY
echo "Arch: ${{matrix.arch}}" >> $GITHUB_STEP_SUMMARY
echo "Ref: $GITHUB_REF" >> $GITHUB_STEP_SUMMARY
echo "Targets:" >> $GITHUB_STEP_SUMMARY
echo $TARGETS | jq -r 'sort_by(.board) |.[] | "- " + .board' >> $GITHUB_STEP_SUMMARY
version:
if: ${{ inputs.target != '' }}
@@ -92,12 +78,12 @@ jobs:
build:
if: ${{ inputs.target != '' && inputs.arch != 'native' }}
needs: [version, find-targets]
needs: [version]
uses: ./.github/workflows/build_firmware.yml
with:
version: ${{ needs.version.outputs.long }}
pio_env: ${{ inputs.target }}
platform: ${{ needs.find-targets.outputs.arch }}
platform: ${{ inputs.arch }}
gather-artifacts:
permissions:
+1 -7
View File
@@ -86,13 +86,7 @@ jobs:
run: sed -i 's/-DBUILD_EPOCH=$UNIX_TIME/#-DBUILD_EPOCH=$UNIX_TIME/' platformio.ini
- name: PlatformIO Tests
run: |
set -o pipefail
# Filter out SKIPPED summary rows for hardware variants that can't run on the
# native host. They flood the log and make it harder to spot real failures.
# The JUnit XML is written directly to testreport.xml before the pipe, so
# the test artifact is unaffected.
platformio test -e coverage -v --junit-output-path testreport.xml 2>&1 | grep -v "[[:space:]]SKIPPED$"
run: platformio test -e coverage -v --junit-output-path testreport.xml
- name: Save test results
if: always() # run this step even if previous step failed
-10
View File
@@ -47,10 +47,6 @@ data/boot/logo.*
managed_components/*
arduino-lib-builder*
dependencies.lock
# JLink / RTT debug artifacts (nRF SoCs)
flash.jlink
rtt_*.txt
idf_component.yml
CMakeLists.txt
/sdkconfig.*
@@ -60,9 +56,3 @@ CMakeLists.txt
.python3
.claude/scheduled_tasks.lock
userPrefs.jsonc.mcp-session-bak
# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
# compiled .proto outputs are ephemeral build artifacts.
build/fixtures/
bin/_generated/
+11
View File
@@ -0,0 +1,11 @@
{
"ai": "copilot",
"ai_commands_dir": null,
"ai_skills": false,
"branch_numbering": "sequential",
"here": true,
"offline": false,
"preset": null,
"script": "sh",
"speckit_version": "0.4.2"
}
+136
View File
@@ -0,0 +1,136 @@
<!--
Sync Impact Report
Version change: template -> 1.0.0
Modified principles:
- [PRINCIPLE_1_NAME] -> I. Safety-Critical Mesh Behavior
- [PRINCIPLE_2_NAME] -> II. Variant-Scoped Hardware Truth
- [PRINCIPLE_3_NAME] -> III. Verification by Targeted Evidence
- [PRINCIPLE_4_NAME] -> IV. Resource and Power Discipline
- [PRINCIPLE_5_NAME] -> V. Minimal, Reviewable Change Sets
Added sections:
- Engineering Constraints
- Delivery Workflow
Removed sections:
- None
Templates requiring updates:
- ✅ .specify/templates/plan-template.md
- ✅ .specify/templates/spec-template.md
- ✅ .specify/templates/tasks-template.md
- ⚠ pending .specify/templates/checklist-template.md (not required for current constitution alignment)
- ⚠ pending .specify/templates/agent-file-template.md (generic scaffold, no constitution-specific drift found)
Follow-up TODOs:
- None
-->
# Meshtastic Firmware Constitution
## Core Principles
### I. Safety-Critical Mesh Behavior
All feature and bug-fix work MUST preserve safe, predictable behavior on live mesh networks.
Changes that affect routing, airtime, broadcast intervals, MQTT bridging, channel handling,
or packet processing MUST document the operational impact on shared bandwidth, public-channel
abuse protections, and interoperability. Any relaxation of rate limits, encryption behavior,
or default-channel safeguards MUST be treated as a breaking governance change unless explicitly
approved and justified.
Rationale: Meshtastic devices operate in constrained radio environments where seemingly small
behavior changes can degrade network reliability, privacy, and fairness for other nodes.
### II. Variant-Scoped Hardware Truth
Board definitions, platform conditionals, and peripheral flags MUST reflect verified hardware
truth and MUST remain scoped to the exact supported target. New capabilities in `variant.h`,
`platformio.ini`, `pins_arduino.h`, or related board files MUST be backed by pin mappings,
chip selection, and power assumptions that are consistent with the board design. Cross-board
copying is prohibited unless every reused define is revalidated for the destination variant.
Rationale: This firmware spans many architectures and board revisions; incorrect hardware
declarations create silent regressions that are hard to detect until devices are flashed.
### III. Verification by Targeted Evidence
Every change MUST be validated by the smallest credible evidence that matches its risk.
At minimum, contributors MUST run formatting or static validation for touched files and MUST
run a targeted build, test, or simulation path for the affected platform when feasible.
Changes to shared core logic, protobufs, routing, or configuration defaults SHOULD include a
native test, simulator run, or equivalent cross-target evidence. If validation cannot be run,
the gap MUST be stated explicitly in the plan, tasks, and final review.
Rationale: The repository supports many targets, so quality depends on explicit validation
rather than assumptions that one successful build implies system-wide safety.
### IV. Resource and Power Discipline
Implementations MUST respect embedded constraints for memory, flash, CPU, battery, and radio
duty cycle. New dependencies, background tasks, logging, polling, display work, and peripheral
power use MUST be justified against the target hardware footprint. Defaults MUST prefer safe
operation on constrained devices, and network-facing behavior MUST account for scaling with
node count where existing project patterns provide that mechanism.
Rationale: Meshtastic firmware runs on low-power devices where unnecessary work directly harms
battery life, responsiveness, thermal behavior, and mesh capacity.
### V. Minimal, Reviewable Change Sets
Changes MUST solve the root problem with the smallest coherent diff that fits the existing
architecture and coding patterns. Unrelated refactors, opportunistic renames, and speculative
abstractions are prohibited in the same change unless they are required to make the fix safe.
Public behavior, configuration semantics, and generated artifacts MUST remain stable unless the
specification and plan explicitly call out the intended change.
Rationale: Small, scoped changes are easier to review across board variants and reduce the
risk of hidden regressions in a large multi-platform firmware repository.
## Engineering Constraints
The authoritative implementation context for this repository is `.github/copilot-instructions.md`.
Plans and tasks MUST align with the existing PlatformIO-based build system, generated protobuf
workflow, architecture-specific source layout, and hardware-variant structure already used in
the repository.
Feature work MUST honor these constraints:
- Code MUST follow existing logging, naming, threading, and configuration patterns.
- Default values and user-facing configuration changes MUST use existing `Default` helpers and
public-channel safeguards where applicable.
- Protobuf changes MUST include regeneration steps and identify downstream effects in generated
code and dependent modules.
- Build and validation steps MUST prefer repository-standard commands such as targeted `pio run`,
`pio test -e native`, simulator tooling, and `trunk fmt` where applicable.
- Platform-specific logic MUST be isolated to the narrowest valid architecture or variant scope.
## Delivery Workflow
Spec-driven work in this repository MUST produce artifacts that make operational risk visible
before code is written.
- Specifications MUST describe affected user or device behavior, impacted platforms, and edge
cases for unavailable peripherals, misconfigured variants, and constrained-network scenarios.
- Plans MUST include a Constitution Check that names the exact validation evidence, target
environments, and any justified deviations from the constitution.
- Tasks MUST be organized so that foundational hardware, protocol, or configuration work is
completed before feature-specific behavior that depends on it.
- Review and implementation notes MUST call out any skipped validation, generated-file updates,
migration considerations, or behavior changes requiring maintainer scrutiny.
## Governance
This constitution supersedes ad hoc workflow preferences for spec-driven work in this repository.
All plans, tasks, reviews, and implementation summaries MUST verify compliance with these
principles.
- Amendments MUST be documented in this file and reflected in dependent templates before new
work proceeds under the changed rule set.
- Versioning policy for this constitution follows semantic versioning.
MAJOR versions indicate removed or materially redefined principles.
MINOR versions indicate new principles, sections, or materially expanded guidance.
PATCH versions indicate clarifications, wording improvements, or non-semantic refinements.
- Compliance review is mandatory for every feature plan and code review. Any constitutional
violation MUST be listed in the plan's Complexity Tracking section or equivalent justification.
- Repository guidance files remain authoritative for implementation detail. Where conflict is
discovered, this constitution governs process and quality gates, while `.github/copilot-instructions.md`
governs repository-specific coding practice until the conflict is resolved by amendment.
**Version**: 1.0.0 | **Ratified**: 2026-03-25 | **Last Amended**: 2026-03-25
+193
View File
@@ -0,0 +1,193 @@
#!/usr/bin/env bash
# Consolidated prerequisite checking script
#
# This script provides unified prerequisite checking for Spec-Driven Development workflow.
# It replaces the functionality previously spread across multiple scripts.
#
# Usage: ./check-prerequisites.sh [OPTIONS]
#
# OPTIONS:
# --json Output in JSON format
# --require-tasks Require tasks.md to exist (for implementation phase)
# --include-tasks Include tasks.md in AVAILABLE_DOCS list
# --paths-only Only output path variables (no validation)
# --help, -h Show help message
#
# OUTPUTS:
# JSON mode: {"FEATURE_DIR":"...", "AVAILABLE_DOCS":["..."]}
# Text mode: FEATURE_DIR:... \n AVAILABLE_DOCS: \n ✓/✗ file.md
# Paths only: REPO_ROOT: ... \n BRANCH: ... \n FEATURE_DIR: ... etc.
set -e
# Parse command line arguments
JSON_MODE=false
REQUIRE_TASKS=false
INCLUDE_TASKS=false
PATHS_ONLY=false
for arg in "$@"; do
case "$arg" in
--json)
JSON_MODE=true
;;
--require-tasks)
REQUIRE_TASKS=true
;;
--include-tasks)
INCLUDE_TASKS=true
;;
--paths-only)
PATHS_ONLY=true
;;
--help | -h)
cat <<'EOF'
Usage: check-prerequisites.sh [OPTIONS]
Consolidated prerequisite checking for Spec-Driven Development workflow.
OPTIONS:
--json Output in JSON format
--require-tasks Require tasks.md to exist (for implementation phase)
--include-tasks Include tasks.md in AVAILABLE_DOCS list
--paths-only Only output path variables (no prerequisite validation)
--help, -h Show this help message
EXAMPLES:
# Check task prerequisites (plan.md required)
./check-prerequisites.sh --json
# Check implementation prerequisites (plan.md + tasks.md required)
./check-prerequisites.sh --json --require-tasks --include-tasks
# Get feature paths only (no validation)
./check-prerequisites.sh --paths-only
EOF
exit 0
;;
*)
echo "ERROR: Unknown option '$arg'. Use --help for usage information." >&2
exit 1
;;
esac
done
# Source common functions
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/common.sh"
# Get feature paths and validate branch
_paths_output=$(get_feature_paths) || {
echo "ERROR: Failed to resolve feature paths" >&2
exit 1
}
eval "$_paths_output"
unset _paths_output
check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
# If paths-only mode, output paths and exit (support JSON + paths-only combined)
if $PATHS_ONLY; then
if $JSON_MODE; then
# Minimal JSON paths payload (no validation performed)
if has_jq; then
jq -cn \
--arg repo_root "$REPO_ROOT" \
--arg branch "$CURRENT_BRANCH" \
--arg feature_dir "$FEATURE_DIR" \
--arg feature_spec "$FEATURE_SPEC" \
--arg impl_plan "$IMPL_PLAN" \
--arg tasks "$TASKS" \
'{REPO_ROOT:$repo_root,BRANCH:$branch,FEATURE_DIR:$feature_dir,FEATURE_SPEC:$feature_spec,IMPL_PLAN:$impl_plan,TASKS:$tasks}'
else
printf '{"REPO_ROOT":"%s","BRANCH":"%s","FEATURE_DIR":"%s","FEATURE_SPEC":"%s","IMPL_PLAN":"%s","TASKS":"%s"}\n' \
"$(json_escape "$REPO_ROOT")" "$(json_escape "$CURRENT_BRANCH")" "$(json_escape "$FEATURE_DIR")" "$(json_escape "$FEATURE_SPEC")" "$(json_escape "$IMPL_PLAN")" "$(json_escape "$TASKS")"
fi
else
echo "REPO_ROOT: $REPO_ROOT"
echo "BRANCH: $CURRENT_BRANCH"
echo "FEATURE_DIR: $FEATURE_DIR"
echo "FEATURE_SPEC: $FEATURE_SPEC"
echo "IMPL_PLAN: $IMPL_PLAN"
echo "TASKS: $TASKS"
fi
exit 0
fi
# Validate required directories and files
if [[ ! -d $FEATURE_DIR ]]; then
echo "ERROR: Feature directory not found: $FEATURE_DIR" >&2
echo "Run /speckit.specify first to create the feature structure." >&2
exit 1
fi
if [[ ! -f $IMPL_PLAN ]]; then
echo "ERROR: plan.md not found in $FEATURE_DIR" >&2
echo "Run /speckit.plan first to create the implementation plan." >&2
exit 1
fi
# Check for tasks.md if required
if $REQUIRE_TASKS && [[ ! -f $TASKS ]]; then
echo "ERROR: tasks.md not found in $FEATURE_DIR" >&2
echo "Run /speckit.tasks first to create the task list." >&2
exit 1
fi
# Build list of available documents
docs=()
# Always check these optional docs
[[ -f $RESEARCH ]] && docs+=("research.md")
[[ -f $DATA_MODEL ]] && docs+=("data-model.md")
# Check contracts directory (only if it exists and has files)
if [[ -d $CONTRACTS_DIR ]] && [[ -n "$(ls -A "$CONTRACTS_DIR" 2>/dev/null)" ]]; then
docs+=("contracts/")
fi
[[ -f $QUICKSTART ]] && docs+=("quickstart.md")
# Include tasks.md if requested and it exists
if $INCLUDE_TASKS && [[ -f $TASKS ]]; then
docs+=("tasks.md")
fi
# Output results
if $JSON_MODE; then
# Build JSON array of documents
if has_jq; then
if [[ ${#docs[@]} -eq 0 ]]; then
json_docs="[]"
else
json_docs=$(printf '%s\n' "${docs[@]}" | jq -R . | jq -s .)
fi
jq -cn \
--arg feature_dir "$FEATURE_DIR" \
--argjson docs "$json_docs" \
'{FEATURE_DIR:$feature_dir,AVAILABLE_DOCS:$docs}'
else
if [[ ${#docs[@]} -eq 0 ]]; then
json_docs="[]"
else
json_docs=$(for d in "${docs[@]}"; do printf '"%s",' "$(json_escape "$d")"; done)
json_docs="[${json_docs%,}]"
fi
printf '{"FEATURE_DIR":"%s","AVAILABLE_DOCS":%s}\n' "$(json_escape "$FEATURE_DIR")" "$json_docs"
fi
else
# Text output
echo "FEATURE_DIR:$FEATURE_DIR"
echo "AVAILABLE_DOCS:"
# Show status of each potential document
check_file "$RESEARCH" "research.md"
check_file "$DATA_MODEL" "data-model.md"
check_dir "$CONTRACTS_DIR" "contracts/"
check_file "$QUICKSTART" "quickstart.md"
if $INCLUDE_TASKS; then
check_file "$TASKS" "tasks.md"
fi
fi
+329
View File
@@ -0,0 +1,329 @@
#!/usr/bin/env bash
# Common functions and variables for all scripts
# Find repository root by searching upward for .specify directory
# This is the primary marker for spec-kit projects
find_specify_root() {
local dir="${1:-$(pwd)}"
# Normalize to absolute path to prevent infinite loop with relative paths
# Use -- to handle paths starting with - (e.g., -P, -L)
dir="$(cd -- "$dir" 2>/dev/null && pwd)" || return 1
local prev_dir=""
while true; do
if [ -d "$dir/.specify" ]; then
echo "$dir"
return 0
fi
# Stop if we've reached filesystem root or dirname stops changing
if [ "$dir" = "/" ] || [ "$dir" = "$prev_dir" ]; then
break
fi
prev_dir="$dir"
dir="$(dirname "$dir")"
done
return 1
}
# Get repository root, prioritizing .specify directory over git
# This prevents using a parent git repo when spec-kit is initialized in a subdirectory
get_repo_root() {
# First, look for .specify directory (spec-kit's own marker)
local specify_root
if specify_root=$(find_specify_root); then
echo "$specify_root"
return
fi
# Fallback to git if no .specify found
if git rev-parse --show-toplevel >/dev/null 2>&1; then
git rev-parse --show-toplevel
return
fi
# Final fallback to script location for non-git repos
local script_dir="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
(cd "$script_dir/../../.." && pwd)
}
# Get current branch, with fallback for non-git repositories
get_current_branch() {
# First check if SPECIFY_FEATURE environment variable is set
if [[ -n ${SPECIFY_FEATURE-} ]]; then
echo "$SPECIFY_FEATURE"
return
fi
# Then check git if available at the spec-kit root (not parent)
local repo_root=$(get_repo_root)
if has_git; then
git -C "$repo_root" rev-parse --abbrev-ref HEAD
return
fi
# For non-git repos, try to find the latest feature directory
local specs_dir="$repo_root/specs"
if [[ -d $specs_dir ]]; then
local latest_feature=""
local highest=0
local latest_timestamp=""
for dir in "$specs_dir"/*; do
if [[ -d $dir ]]; then
local dirname=$(basename "$dir")
if [[ $dirname =~ ^([0-9]{8}-[0-9]{6})- ]]; then
# Timestamp-based branch: compare lexicographically
local ts="${BASH_REMATCH[1]}"
if [[ $ts > $latest_timestamp ]]; then
latest_timestamp="$ts"
latest_feature=$dirname
fi
elif [[ $dirname =~ ^([0-9]{3})- ]]; then
local number=${BASH_REMATCH[1]}
number=$((10#$number))
if [[ $number -gt $highest ]]; then
highest=$number
# Only update if no timestamp branch found yet
if [[ -z $latest_timestamp ]]; then
latest_feature=$dirname
fi
fi
fi
fi
done
if [[ -n $latest_feature ]]; then
echo "$latest_feature"
return
fi
fi
echo "main" # Final fallback
}
# Check if we have git available at the spec-kit root level
# Returns true only if git is installed and the repo root is inside a git work tree
# Handles both regular repos (.git directory) and worktrees/submodules (.git file)
has_git() {
# First check if git command is available (before calling get_repo_root which may use git)
command -v git >/dev/null 2>&1 || return 1
local repo_root=$(get_repo_root)
# Check if .git exists (directory or file for worktrees/submodules)
[ -e "$repo_root/.git" ] || return 1
# Verify it's actually a valid git work tree
git -C "$repo_root" rev-parse --is-inside-work-tree >/dev/null 2>&1
}
check_feature_branch() {
local branch="$1"
local has_git_repo="$2"
# For non-git repos, we can't enforce branch naming but still provide output
if [[ $has_git_repo != "true" ]]; then
echo "[specify] Warning: Git repository not detected; skipped branch validation" >&2
return 0
fi
if [[ ! $branch =~ ^[0-9]{3}- ]] && [[ ! $branch =~ ^[0-9]{8}-[0-9]{6}- ]]; then
echo "ERROR: Not on a feature branch. Current branch: $branch" >&2
echo "Feature branches should be named like: 001-feature-name or 20260319-143022-feature-name" >&2
return 1
fi
return 0
}
get_feature_dir() { echo "$1/specs/$2"; }
# Find feature directory by numeric prefix instead of exact branch match
# This allows multiple branches to work on the same spec (e.g., 004-fix-bug, 004-add-feature)
find_feature_dir_by_prefix() {
local repo_root="$1"
local branch_name="$2"
local specs_dir="$repo_root/specs"
# Extract prefix from branch (e.g., "004" from "004-whatever" or "20260319-143022" from timestamp branches)
local prefix=""
if [[ $branch_name =~ ^([0-9]{8}-[0-9]{6})- ]]; then
prefix="${BASH_REMATCH[1]}"
elif [[ $branch_name =~ ^([0-9]{3})- ]]; then
prefix="${BASH_REMATCH[1]}"
else
# If branch doesn't have a recognized prefix, fall back to exact match
echo "$specs_dir/$branch_name"
return
fi
# Search for directories in specs/ that start with this prefix
local matches=()
if [[ -d $specs_dir ]]; then
for dir in "$specs_dir"/"$prefix"-*; do
if [[ -d $dir ]]; then
matches+=("$(basename "$dir")")
fi
done
fi
# Handle results
if [[ ${#matches[@]} -eq 0 ]]; then
# No match found - return the branch name path (will fail later with clear error)
echo "$specs_dir/$branch_name"
elif [[ ${#matches[@]} -eq 1 ]]; then
# Exactly one match - perfect!
echo "$specs_dir/${matches[0]}"
else
# Multiple matches - this shouldn't happen with proper naming convention
echo "ERROR: Multiple spec directories found with prefix '$prefix': ${matches[*]}" >&2
echo "Please ensure only one spec directory exists per prefix." >&2
return 1
fi
}
get_feature_paths() {
local repo_root=$(get_repo_root)
local current_branch=$(get_current_branch)
local has_git_repo="false"
if has_git; then
has_git_repo="true"
fi
# Use prefix-based lookup to support multiple branches per spec
local feature_dir
if ! feature_dir=$(find_feature_dir_by_prefix "$repo_root" "$current_branch"); then
echo "ERROR: Failed to resolve feature directory" >&2
return 1
fi
# Use printf '%q' to safely quote values, preventing shell injection
# via crafted branch names or paths containing special characters
printf 'REPO_ROOT=%q\n' "$repo_root"
printf 'CURRENT_BRANCH=%q\n' "$current_branch"
printf 'HAS_GIT=%q\n' "$has_git_repo"
printf 'FEATURE_DIR=%q\n' "$feature_dir"
printf 'FEATURE_SPEC=%q\n' "$feature_dir/spec.md"
printf 'IMPL_PLAN=%q\n' "$feature_dir/plan.md"
printf 'TASKS=%q\n' "$feature_dir/tasks.md"
printf 'RESEARCH=%q\n' "$feature_dir/research.md"
printf 'DATA_MODEL=%q\n' "$feature_dir/data-model.md"
printf 'QUICKSTART=%q\n' "$feature_dir/quickstart.md"
printf 'CONTRACTS_DIR=%q\n' "$feature_dir/contracts"
}
# Check if jq is available for safe JSON construction
has_jq() {
command -v jq >/dev/null 2>&1
}
# Escape a string for safe embedding in a JSON value (fallback when jq is unavailable).
# Handles backslash, double-quote, and JSON-required control character escapes (RFC 8259).
json_escape() {
local s="$1"
s="${s//\\/\\\\}"
s="${s//\"/\\\"}"
s="${s//$'\n'/\\n}"
s="${s//$'\t'/\\t}"
s="${s//$'\r'/\\r}"
s="${s//$'\b'/\\b}"
s="${s//$'\f'/\\f}"
# Escape any remaining U+0001-U+001F control characters as \uXXXX.
# (U+0000/NUL cannot appear in bash strings and is excluded.)
# LC_ALL=C ensures ${#s} counts bytes and ${s:$i:1} yields single bytes,
# so multi-byte UTF-8 sequences (first byte >= 0xC0) pass through intact.
local LC_ALL=C
local i char code
for ((i = 0; i < ${#s}; i++)); do
char="${s:i:1}"
printf -v code '%d' "'$char" 2>/dev/null || code=256
if ((code >= 1 && code <= 31)); then
printf '\\u%04x' "$code"
else
printf '%s' "$char"
fi
done
}
check_file() { [[ -f $1 ]] && echo " ✓ $2" || echo " ✗ $2"; }
check_dir() { [[ -d $1 && -n $(ls -A "$1" 2>/dev/null) ]] && echo " ✓ $2" || echo " ✗ $2"; }
# Resolve a template name to a file path using the priority stack:
# 1. .specify/templates/overrides/
# 2. .specify/presets/<preset-id>/templates/ (sorted by priority from .registry)
# 3. .specify/extensions/<ext-id>/templates/
# 4. .specify/templates/ (core)
resolve_template() {
local template_name="$1"
local repo_root="$2"
local base="$repo_root/.specify/templates"
# Priority 1: Project overrides
local override="$base/overrides/${template_name}.md"
[ -f "$override" ] && echo "$override" && return 0
# Priority 2: Installed presets (sorted by priority from .registry)
local presets_dir="$repo_root/.specify/presets"
if [ -d "$presets_dir" ]; then
local registry_file="$presets_dir/.registry"
if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then
# Read preset IDs sorted by priority (lower number = higher precedence).
# The python3 call is wrapped in an if-condition so that set -e does not
# abort the function when python3 exits non-zero (e.g. invalid JSON).
local sorted_presets=""
if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c "
import json, sys, os
try:
with open(os.environ['SPECKIT_REGISTRY']) as f:
data = json.load(f)
presets = data.get('presets', {})
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10)):
print(pid)
except Exception:
sys.exit(1)
" 2>/dev/null); then
if [ -n "$sorted_presets" ]; then
# python3 succeeded and returned preset IDs — search in priority order
while IFS= read -r preset_id; do
local candidate="$presets_dir/$preset_id/templates/${template_name}.md"
[ -f "$candidate" ] && echo "$candidate" && return 0
done <<<"$sorted_presets"
fi
# python3 succeeded but registry has no presets — nothing to search
else
# python3 failed (missing, or registry parse error) — fall back to unordered directory scan
for preset in "$presets_dir"/*/; do
[ -d "$preset" ] || continue
local candidate="$preset/templates/${template_name}.md"
[ -f "$candidate" ] && echo "$candidate" && return 0
done
fi
else
# Fallback: alphabetical directory order (no python3 available)
for preset in "$presets_dir"/*/; do
[ -d "$preset" ] || continue
local candidate="$preset/templates/${template_name}.md"
[ -f "$candidate" ] && echo "$candidate" && return 0
done
fi
fi
# Priority 3: Extension-provided templates
local ext_dir="$repo_root/.specify/extensions"
if [ -d "$ext_dir" ]; then
for ext in "$ext_dir"/*/; do
[ -d "$ext" ] || continue
# Skip hidden directories (e.g. .backup, .cache)
case "$(basename "$ext")" in .*) continue ;; esac
local candidate="$ext/templates/${template_name}.md"
[ -f "$candidate" ] && echo "$candidate" && return 0
done
fi
# Priority 4: Core templates
local core="$base/${template_name}.md"
[ -f "$core" ] && echo "$core" && return 0
# Template not found in any location.
# Return 1 so callers can distinguish "not found" from "found".
# Callers running under set -e should use: TEMPLATE=$(resolve_template ...) || true
return 1
}
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env bash
set -e
JSON_MODE=false
SHORT_NAME=""
BRANCH_NUMBER=""
USE_TIMESTAMP=false
ARGS=()
i=1
while [ $i -le $# ]; do
arg="${!i}"
case "$arg" in
--json)
JSON_MODE=true
;;
--short-name)
if [ $((i + 1)) -gt $# ]; then
echo 'Error: --short-name requires a value' >&2
exit 1
fi
i=$((i + 1))
next_arg="${!i}"
# Check if the next argument is another option (starts with --)
if [[ $next_arg == --* ]]; then
echo 'Error: --short-name requires a value' >&2
exit 1
fi
SHORT_NAME="$next_arg"
;;
--number)
if [ $((i + 1)) -gt $# ]; then
echo 'Error: --number requires a value' >&2
exit 1
fi
i=$((i + 1))
next_arg="${!i}"
if [[ $next_arg == --* ]]; then
echo 'Error: --number requires a value' >&2
exit 1
fi
BRANCH_NUMBER="$next_arg"
;;
--timestamp)
USE_TIMESTAMP=true
;;
--help | -h)
echo "Usage: $0 [--json] [--short-name <name>] [--number N] [--timestamp] <feature_description>"
echo ""
echo "Options:"
echo " --json Output in JSON format"
echo " --short-name <name> Provide a custom short name (2-4 words) for the branch"
echo " --number N Specify branch number manually (overrides auto-detection)"
echo " --timestamp Use timestamp prefix (YYYYMMDD-HHMMSS) instead of sequential numbering"
echo " --help, -h Show this help message"
echo ""
echo "Examples:"
echo " $0 'Add user authentication system' --short-name 'user-auth'"
echo " $0 'Implement OAuth2 integration for API' --number 5"
echo " $0 --timestamp --short-name 'user-auth' 'Add user authentication'"
exit 0
;;
*)
ARGS+=("$arg")
;;
esac
i=$((i + 1))
done
FEATURE_DESCRIPTION="${ARGS[*]}"
if [ -z "$FEATURE_DESCRIPTION" ]; then
echo "Usage: $0 [--json] [--short-name <name>] [--number N] [--timestamp] <feature_description>" >&2
exit 1
fi
# Trim whitespace and validate description is not empty (e.g., user passed only whitespace)
FEATURE_DESCRIPTION=$(echo "$FEATURE_DESCRIPTION" | xargs)
if [ -z "$FEATURE_DESCRIPTION" ]; then
echo "Error: Feature description cannot be empty or contain only whitespace" >&2
exit 1
fi
# Function to get highest number from specs directory
get_highest_from_specs() {
local specs_dir="$1"
local highest=0
if [ -d "$specs_dir" ]; then
for dir in "$specs_dir"/*; do
[ -d "$dir" ] || continue
dirname=$(basename "$dir")
# Only match sequential prefixes (###-*), skip timestamp dirs
if echo "$dirname" | grep -q '^[0-9]\{3\}-'; then
number=$(echo "$dirname" | grep -o '^[0-9]\{3\}')
number=$((10#$number))
if [ "$number" -gt "$highest" ]; then
highest=$number
fi
fi
done
fi
echo "$highest"
}
# Function to get highest number from git branches
get_highest_from_branches() {
local highest=0
# Get all branches (local and remote)
branches=$(git branch -a 2>/dev/null || echo "")
if [ -n "$branches" ]; then
while IFS= read -r branch; do
# Clean branch name: remove leading markers and remote prefixes
clean_branch=$(echo "$branch" | sed 's/^[* ]*//; s|^remotes/[^/]*/||')
# Extract feature number if branch matches pattern ###-*
if echo "$clean_branch" | grep -q '^[0-9]\{3\}-'; then
number=$(echo "$clean_branch" | grep -o '^[0-9]\{3\}' || echo "0")
number=$((10#$number))
if [ "$number" -gt "$highest" ]; then
highest=$number
fi
fi
done <<<"$branches"
fi
echo "$highest"
}
# Function to check existing branches (local and remote) and return next available number
check_existing_branches() {
local specs_dir="$1"
# Fetch all remotes to get latest branch info (suppress errors if no remotes)
git fetch --all --prune >/dev/null 2>&1 || true
# Get highest number from ALL branches (not just matching short name)
local highest_branch=$(get_highest_from_branches)
# Get highest number from ALL specs (not just matching short name)
local highest_spec=$(get_highest_from_specs "$specs_dir")
# Take the maximum of both
local max_num=$highest_branch
if [ "$highest_spec" -gt "$max_num" ]; then
max_num=$highest_spec
fi
# Return next number
echo $((max_num + 1))
}
# Function to clean and format a branch name
clean_branch_name() {
local name="$1"
echo "$name" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/-\+/-/g' | sed 's/^-//' | sed 's/-$//'
}
# Resolve repository root using common.sh functions which prioritize .specify over git
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/common.sh"
REPO_ROOT=$(get_repo_root)
# Check if git is available at this repo root (not a parent)
if has_git; then
HAS_GIT=true
else
HAS_GIT=false
fi
cd "$REPO_ROOT"
SPECS_DIR="$REPO_ROOT/specs"
mkdir -p "$SPECS_DIR"
# Function to generate branch name with stop word filtering and length filtering
generate_branch_name() {
local description="$1"
# Common stop words to filter out
local stop_words="^(i|a|an|the|to|for|of|in|on|at|by|with|from|is|are|was|were|be|been|being|have|has|had|do|does|did|will|would|should|could|can|may|might|must|shall|this|that|these|those|my|your|our|their|want|need|add|get|set)$"
# Convert to lowercase and split into words
local clean_name=$(echo "$description" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/ /g')
# Filter words: remove stop words and words shorter than 3 chars (unless they're uppercase acronyms in original)
local meaningful_words=()
for word in $clean_name; do
# Skip empty words
[ -z "$word" ] && continue
# Keep words that are NOT stop words AND (length >= 3 OR are potential acronyms)
if ! echo "$word" | grep -qiE "$stop_words"; then
if [ ${#word} -ge 3 ]; then
meaningful_words+=("$word")
elif echo "$description" | grep -q "\b${word^^}\b"; then
# Keep short words if they appear as uppercase in original (likely acronyms)
meaningful_words+=("$word")
fi
fi
done
# If we have meaningful words, use first 3-4 of them
if [ ${#meaningful_words[@]} -gt 0 ]; then
local max_words=3
if [ ${#meaningful_words[@]} -eq 4 ]; then max_words=4; fi
local result=""
local count=0
for word in "${meaningful_words[@]}"; do
if [ $count -ge $max_words ]; then break; fi
if [ -n "$result" ]; then result="$result-"; fi
result="$result$word"
count=$((count + 1))
done
echo "$result"
else
# Fallback to original logic if no meaningful words found
local cleaned=$(clean_branch_name "$description")
echo "$cleaned" | tr '-' '\n' | grep -v '^$' | head -3 | tr '\n' '-' | sed 's/-$//'
fi
}
# Generate branch name
if [ -n "$SHORT_NAME" ]; then
# Use provided short name, just clean it up
BRANCH_SUFFIX=$(clean_branch_name "$SHORT_NAME")
else
# Generate from description with smart filtering
BRANCH_SUFFIX=$(generate_branch_name "$FEATURE_DESCRIPTION")
fi
# Warn if --number and --timestamp are both specified
if [ "$USE_TIMESTAMP" = true ] && [ -n "$BRANCH_NUMBER" ]; then
echo >&2 "[specify] Warning: --number is ignored when --timestamp is used"
BRANCH_NUMBER=""
fi
# Determine branch prefix
if [ "$USE_TIMESTAMP" = true ]; then
FEATURE_NUM=$(date +%Y%m%d-%H%M%S)
BRANCH_NAME="${FEATURE_NUM}-${BRANCH_SUFFIX}"
else
# Determine branch number
if [ -z "$BRANCH_NUMBER" ]; then
if [ "$HAS_GIT" = true ]; then
# Check existing branches on remotes
BRANCH_NUMBER=$(check_existing_branches "$SPECS_DIR")
else
# Fall back to local directory check
HIGHEST=$(get_highest_from_specs "$SPECS_DIR")
BRANCH_NUMBER=$((HIGHEST + 1))
fi
fi
# Force base-10 interpretation to prevent octal conversion (e.g., 010 → 8 in octal, but should be 10 in decimal)
FEATURE_NUM=$(printf "%03d" "$((10#$BRANCH_NUMBER))")
BRANCH_NAME="${FEATURE_NUM}-${BRANCH_SUFFIX}"
fi
# GitHub enforces a 244-byte limit on branch names
# Validate and truncate if necessary
MAX_BRANCH_LENGTH=244
if [ ${#BRANCH_NAME} -gt $MAX_BRANCH_LENGTH ]; then
# Calculate how much we need to trim from suffix
# Account for prefix length: timestamp (15) + hyphen (1) = 16, or sequential (3) + hyphen (1) = 4
PREFIX_LENGTH=$((${#FEATURE_NUM} + 1))
MAX_SUFFIX_LENGTH=$((MAX_BRANCH_LENGTH - PREFIX_LENGTH))
# Truncate suffix at word boundary if possible
TRUNCATED_SUFFIX=$(echo "$BRANCH_SUFFIX" | cut -c1-$MAX_SUFFIX_LENGTH)
# Remove trailing hyphen if truncation created one
TRUNCATED_SUFFIX=$(echo "$TRUNCATED_SUFFIX" | sed 's/-$//')
ORIGINAL_BRANCH_NAME="$BRANCH_NAME"
BRANCH_NAME="${FEATURE_NUM}-${TRUNCATED_SUFFIX}"
echo >&2 "[specify] Warning: Branch name exceeded GitHub's 244-byte limit"
echo >&2 "[specify] Original: $ORIGINAL_BRANCH_NAME (${#ORIGINAL_BRANCH_NAME} bytes)"
echo >&2 "[specify] Truncated to: $BRANCH_NAME (${#BRANCH_NAME} bytes)"
fi
if [ "$HAS_GIT" = true ]; then
if ! git checkout -b "$BRANCH_NAME" 2>/dev/null; then
# Check if branch already exists
if git branch --list "$BRANCH_NAME" | grep -q .; then
if [ "$USE_TIMESTAMP" = true ]; then
echo >&2 "Error: Branch '$BRANCH_NAME' already exists. Rerun to get a new timestamp or use a different --short-name."
else
echo >&2 "Error: Branch '$BRANCH_NAME' already exists. Please use a different feature name or specify a different number with --number."
fi
exit 1
else
echo >&2 "Error: Failed to create git branch '$BRANCH_NAME'. Please check your git configuration and try again."
exit 1
fi
fi
else
echo >&2 "[specify] Warning: Git repository not detected; skipped branch creation for $BRANCH_NAME"
fi
FEATURE_DIR="$SPECS_DIR/$BRANCH_NAME"
mkdir -p "$FEATURE_DIR"
TEMPLATE=$(resolve_template "spec-template" "$REPO_ROOT") || true
SPEC_FILE="$FEATURE_DIR/spec.md"
if [ -n "$TEMPLATE" ] && [ -f "$TEMPLATE" ]; then
cp "$TEMPLATE" "$SPEC_FILE"
else
echo "Warning: Spec template not found; created empty spec file" >&2
touch "$SPEC_FILE"
fi
# Inform the user how to persist the feature variable in their own shell
printf '# To persist: export SPECIFY_FEATURE=%q\n' "$BRANCH_NAME" >&2
if $JSON_MODE; then
if command -v jq >/dev/null 2>&1; then
jq -cn \
--arg branch_name "$BRANCH_NAME" \
--arg spec_file "$SPEC_FILE" \
--arg feature_num "$FEATURE_NUM" \
'{BRANCH_NAME:$branch_name,SPEC_FILE:$spec_file,FEATURE_NUM:$feature_num}'
else
printf '{"BRANCH_NAME":"%s","SPEC_FILE":"%s","FEATURE_NUM":"%s"}\n' "$(json_escape "$BRANCH_NAME")" "$(json_escape "$SPEC_FILE")" "$(json_escape "$FEATURE_NUM")"
fi
else
echo "BRANCH_NAME: $BRANCH_NAME"
echo "SPEC_FILE: $SPEC_FILE"
echo "FEATURE_NUM: $FEATURE_NUM"
printf '# To persist in your shell: export SPECIFY_FEATURE=%q\n' "$BRANCH_NAME"
fi
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
set -e
# Parse command line arguments
JSON_MODE=false
ARGS=()
for arg in "$@"; do
case "$arg" in
--json)
JSON_MODE=true
;;
--help | -h)
echo "Usage: $0 [--json]"
echo " --json Output results in JSON format"
echo " --help Show this help message"
exit 0
;;
*)
ARGS+=("$arg")
;;
esac
done
# Get script directory and load common functions
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/common.sh"
# Get all paths and variables from common functions
_paths_output=$(get_feature_paths) || {
echo "ERROR: Failed to resolve feature paths" >&2
exit 1
}
eval "$_paths_output"
unset _paths_output
# Check if we're on a proper feature branch (only for git repos)
check_feature_branch "$CURRENT_BRANCH" "$HAS_GIT" || exit 1
# Ensure the feature directory exists
mkdir -p "$FEATURE_DIR"
# Copy plan template if it exists
TEMPLATE=$(resolve_template "plan-template" "$REPO_ROOT") || true
if [[ -n $TEMPLATE ]] && [[ -f $TEMPLATE ]]; then
cp "$TEMPLATE" "$IMPL_PLAN"
echo "Copied plan template to $IMPL_PLAN"
else
echo "Warning: Plan template not found"
# Create a basic plan file if template doesn't exist
touch "$IMPL_PLAN"
fi
# Output results
if $JSON_MODE; then
if has_jq; then
jq -cn \
--arg feature_spec "$FEATURE_SPEC" \
--arg impl_plan "$IMPL_PLAN" \
--arg specs_dir "$FEATURE_DIR" \
--arg branch "$CURRENT_BRANCH" \
--arg has_git "$HAS_GIT" \
'{FEATURE_SPEC:$feature_spec,IMPL_PLAN:$impl_plan,SPECS_DIR:$specs_dir,BRANCH:$branch,HAS_GIT:$has_git}'
else
printf '{"FEATURE_SPEC":"%s","IMPL_PLAN":"%s","SPECS_DIR":"%s","BRANCH":"%s","HAS_GIT":"%s"}\n' \
"$(json_escape "$FEATURE_SPEC")" "$(json_escape "$IMPL_PLAN")" "$(json_escape "$FEATURE_DIR")" "$(json_escape "$CURRENT_BRANCH")" "$(json_escape "$HAS_GIT")"
fi
else
echo "FEATURE_SPEC: $FEATURE_SPEC"
echo "IMPL_PLAN: $IMPL_PLAN"
echo "SPECS_DIR: $FEATURE_DIR"
echo "BRANCH: $CURRENT_BRANCH"
echo "HAS_GIT: $HAS_GIT"
fi
+840
View File
@@ -0,0 +1,840 @@
#!/usr/bin/env bash
# Update agent context files with information from plan.md
#
# This script maintains AI agent context files by parsing feature specifications
# and updating agent-specific configuration files with project information.
#
# MAIN FUNCTIONS:
# 1. Environment Validation
# - Verifies git repository structure and branch information
# - Checks for required plan.md files and templates
# - Validates file permissions and accessibility
#
# 2. Plan Data Extraction
# - Parses plan.md files to extract project metadata
# - Identifies language/version, frameworks, databases, and project types
# - Handles missing or incomplete specification data gracefully
#
# 3. Agent File Management
# - Creates new agent context files from templates when needed
# - Updates existing agent files with new project information
# - Preserves manual additions and custom configurations
# - Supports multiple AI agent formats and directory structures
#
# 4. Content Generation
# - Generates language-specific build/test commands
# - Creates appropriate project directory structures
# - Updates technology stacks and recent changes sections
# - Maintains consistent formatting and timestamps
#
# 5. Multi-Agent Support
# - Handles agent-specific file paths and naming conventions
# - Supports: Claude, Gemini, Copilot, Cursor, Qwen, opencode, Codex, Windsurf, Junie, Kilo Code, Auggie CLI, Roo Code, CodeBuddy CLI, Qoder CLI, Amp, SHAI, Tabnine CLI, Kiro CLI, Mistral Vibe, Kimi Code, Pi Coding Agent, iFlow CLI, Antigravity or Generic
# - Can update single agents or all existing agent files
# - Creates default Claude file if no agent files exist
#
# Usage: ./update-agent-context.sh [agent_type]
# Agent types: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|junie|kilocode|auggie|roo|codebuddy|amp|shai|tabnine|kiro-cli|agy|bob|vibe|qodercli|kimi|trae|pi|iflow|generic
# Leave empty to update all existing agent files
set -e
# Enable strict error handling
set -u
set -o pipefail
#==============================================================================
# Configuration and Global Variables
#==============================================================================
# Get script directory and load common functions
SCRIPT_DIR="$(CDPATH="" cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "$SCRIPT_DIR/common.sh"
# Get all paths and variables from common functions
_paths_output=$(get_feature_paths) || {
echo "ERROR: Failed to resolve feature paths" >&2
exit 1
}
eval "$_paths_output"
unset _paths_output
NEW_PLAN="$IMPL_PLAN" # Alias for compatibility with existing code
AGENT_TYPE="${1-}"
# Agent-specific file paths
CLAUDE_FILE="$REPO_ROOT/CLAUDE.md"
GEMINI_FILE="$REPO_ROOT/GEMINI.md"
COPILOT_FILE="$REPO_ROOT/.github/agents/copilot-instructions.md"
CURSOR_FILE="$REPO_ROOT/.cursor/rules/specify-rules.mdc"
QWEN_FILE="$REPO_ROOT/QWEN.md"
AGENTS_FILE="$REPO_ROOT/AGENTS.md"
WINDSURF_FILE="$REPO_ROOT/.windsurf/rules/specify-rules.md"
JUNIE_FILE="$REPO_ROOT/.junie/AGENTS.md"
KILOCODE_FILE="$REPO_ROOT/.kilocode/rules/specify-rules.md"
AUGGIE_FILE="$REPO_ROOT/.augment/rules/specify-rules.md"
ROO_FILE="$REPO_ROOT/.roo/rules/specify-rules.md"
CODEBUDDY_FILE="$REPO_ROOT/CODEBUDDY.md"
QODER_FILE="$REPO_ROOT/QODER.md"
# Amp, Kiro CLI, IBM Bob, and Pi all share AGENTS.md — use AGENTS_FILE to avoid
# updating the same file multiple times.
AMP_FILE="$AGENTS_FILE"
SHAI_FILE="$REPO_ROOT/SHAI.md"
TABNINE_FILE="$REPO_ROOT/TABNINE.md"
KIRO_FILE="$AGENTS_FILE"
AGY_FILE="$REPO_ROOT/.agent/rules/specify-rules.md"
BOB_FILE="$AGENTS_FILE"
VIBE_FILE="$REPO_ROOT/.vibe/agents/specify-agents.md"
KIMI_FILE="$REPO_ROOT/KIMI.md"
TRAE_FILE="$REPO_ROOT/.trae/rules/AGENTS.md"
IFLOW_FILE="$REPO_ROOT/IFLOW.md"
# Template file
TEMPLATE_FILE="$REPO_ROOT/.specify/templates/agent-file-template.md"
# Global variables for parsed plan data
NEW_LANG=""
NEW_FRAMEWORK=""
NEW_DB=""
NEW_PROJECT_TYPE=""
#==============================================================================
# Utility Functions
#==============================================================================
log_info() {
echo "INFO: $1"
}
log_success() {
echo "✓ $1"
}
log_error() {
echo "ERROR: $1" >&2
}
log_warning() {
echo "WARNING: $1" >&2
}
# Cleanup function for temporary files
cleanup() {
local exit_code=$?
# Disarm traps to prevent re-entrant loop
trap - EXIT INT TERM
rm -f /tmp/agent_update_*_$$
rm -f /tmp/manual_additions_$$
exit $exit_code
}
# Set up cleanup trap
trap cleanup EXIT INT TERM
#==============================================================================
# Validation Functions
#==============================================================================
validate_environment() {
# Check if we have a current branch/feature (git or non-git)
if [[ -z $CURRENT_BRANCH ]]; then
log_error "Unable to determine current feature"
if [[ $HAS_GIT == "true" ]]; then
log_info "Make sure you're on a feature branch"
else
log_info "Set SPECIFY_FEATURE environment variable or create a feature first"
fi
exit 1
fi
# Check if plan.md exists
if [[ ! -f $NEW_PLAN ]]; then
log_error "No plan.md found at $NEW_PLAN"
log_info "Make sure you're working on a feature with a corresponding spec directory"
if [[ $HAS_GIT != "true" ]]; then
log_info "Use: export SPECIFY_FEATURE=your-feature-name or create a new feature first"
fi
exit 1
fi
# Check if template exists (needed for new files)
if [[ ! -f $TEMPLATE_FILE ]]; then
log_warning "Template file not found at $TEMPLATE_FILE"
log_warning "Creating new agent files will fail"
fi
}
#==============================================================================
# Plan Parsing Functions
#==============================================================================
extract_plan_field() {
local field_pattern="$1"
local plan_file="$2"
grep "^\*\*${field_pattern}\*\*: " "$plan_file" 2>/dev/null |
head -1 |
sed "s|^\*\*${field_pattern}\*\*: ||" |
sed 's/^[ \t]*//;s/[ \t]*$//' |
grep -v "NEEDS CLARIFICATION" |
grep -v "^N/A$" || echo ""
}
parse_plan_data() {
local plan_file="$1"
if [[ ! -f $plan_file ]]; then
log_error "Plan file not found: $plan_file"
return 1
fi
if [[ ! -r $plan_file ]]; then
log_error "Plan file is not readable: $plan_file"
return 1
fi
log_info "Parsing plan data from $plan_file"
NEW_LANG=$(extract_plan_field "Language/Version" "$plan_file")
NEW_FRAMEWORK=$(extract_plan_field "Primary Dependencies" "$plan_file")
NEW_DB=$(extract_plan_field "Storage" "$plan_file")
NEW_PROJECT_TYPE=$(extract_plan_field "Project Type" "$plan_file")
# Log what we found
if [[ -n $NEW_LANG ]]; then
log_info "Found language: $NEW_LANG"
else
log_warning "No language information found in plan"
fi
if [[ -n $NEW_FRAMEWORK ]]; then
log_info "Found framework: $NEW_FRAMEWORK"
fi
if [[ -n $NEW_DB ]] && [[ $NEW_DB != "N/A" ]]; then
log_info "Found database: $NEW_DB"
fi
if [[ -n $NEW_PROJECT_TYPE ]]; then
log_info "Found project type: $NEW_PROJECT_TYPE"
fi
}
format_technology_stack() {
local lang="$1"
local framework="$2"
local parts=()
# Add non-empty parts
[[ -n $lang && $lang != "NEEDS CLARIFICATION" ]] && parts+=("$lang")
[[ -n $framework && $framework != "NEEDS CLARIFICATION" && $framework != "N/A" ]] && parts+=("$framework")
# Join with proper formatting
if [[ ${#parts[@]} -eq 0 ]]; then
echo ""
elif [[ ${#parts[@]} -eq 1 ]]; then
echo "${parts[0]}"
else
# Join multiple parts with " + "
local result="${parts[0]}"
for ((i = 1; i < ${#parts[@]}; i++)); do
result="$result + ${parts[i]}"
done
echo "$result"
fi
}
#==============================================================================
# Template and Content Generation Functions
#==============================================================================
get_project_structure() {
local project_type="$1"
if [[ $project_type == *"web"* ]]; then
echo 'backend/\nfrontend/\ntests/'
else
echo 'src/\ntests/'
fi
}
get_commands_for_language() {
local lang="$1"
case "$lang" in
*"Python"*)
echo "cd src && pytest && ruff check ."
;;
*"Rust"*)
echo "cargo test && cargo clippy"
;;
*"JavaScript"* | *"TypeScript"*)
echo 'npm test \&\& npm run lint'
;;
*)
echo "# Add commands for $lang"
;;
esac
}
get_language_conventions() {
local lang="$1"
echo "$lang: Follow standard conventions"
}
create_new_agent_file() {
local target_file="$1"
local temp_file="$2"
local project_name="$3"
local current_date="$4"
if [[ ! -f $TEMPLATE_FILE ]]; then
log_error "Template not found at $TEMPLATE_FILE"
return 1
fi
if [[ ! -r $TEMPLATE_FILE ]]; then
log_error "Template file is not readable: $TEMPLATE_FILE"
return 1
fi
log_info "Creating new agent context file from template..."
if ! cp "$TEMPLATE_FILE" "$temp_file"; then
log_error "Failed to copy template file"
return 1
fi
# Replace template placeholders
local project_structure
project_structure=$(get_project_structure "$NEW_PROJECT_TYPE")
local commands
commands=$(get_commands_for_language "$NEW_LANG")
local language_conventions
language_conventions=$(get_language_conventions "$NEW_LANG")
# Perform substitutions with error checking using safer approach
# Escape special characters for sed by using a different delimiter or escaping
local escaped_lang=$(printf '%s\n' "$NEW_LANG" | sed 's/[\[\.*^$()+{}|]/\\&/g')
local escaped_framework=$(printf '%s\n' "$NEW_FRAMEWORK" | sed 's/[\[\.*^$()+{}|]/\\&/g')
local escaped_branch=$(printf '%s\n' "$CURRENT_BRANCH" | sed 's/[\[\.*^$()+{}|]/\\&/g')
# Build technology stack and recent change strings conditionally
local tech_stack
if [[ -n $escaped_lang && -n $escaped_framework ]]; then
tech_stack="- $escaped_lang + $escaped_framework ($escaped_branch)"
elif [[ -n $escaped_lang ]]; then
tech_stack="- $escaped_lang ($escaped_branch)"
elif [[ -n $escaped_framework ]]; then
tech_stack="- $escaped_framework ($escaped_branch)"
else
tech_stack="- ($escaped_branch)"
fi
local recent_change
if [[ -n $escaped_lang && -n $escaped_framework ]]; then
recent_change="- $escaped_branch: Added $escaped_lang + $escaped_framework"
elif [[ -n $escaped_lang ]]; then
recent_change="- $escaped_branch: Added $escaped_lang"
elif [[ -n $escaped_framework ]]; then
recent_change="- $escaped_branch: Added $escaped_framework"
else
recent_change="- $escaped_branch: Added"
fi
local substitutions=(
"s|\[PROJECT NAME\]|$project_name|"
"s|\[DATE\]|$current_date|"
"s|\[EXTRACTED FROM ALL PLAN.MD FILES\]|$tech_stack|"
"s|\[ACTUAL STRUCTURE FROM PLANS\]|$project_structure|g"
"s|\[ONLY COMMANDS FOR ACTIVE TECHNOLOGIES\]|$commands|"
"s|\[LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE\]|$language_conventions|"
"s|\[LAST 3 FEATURES AND WHAT THEY ADDED\]|$recent_change|"
)
for substitution in "${substitutions[@]}"; do
if ! sed -i.bak -e "$substitution" "$temp_file"; then
log_error "Failed to perform substitution: $substitution"
rm -f "$temp_file" "$temp_file.bak"
return 1
fi
done
# Convert \n sequences to actual newlines
newline=$(printf '\n')
sed -i.bak2 "s/\\\\n/${newline}/g" "$temp_file"
# Clean up backup files
rm -f "$temp_file.bak" "$temp_file.bak2"
# Prepend Cursor frontmatter for .mdc files so rules are auto-included
if [[ $target_file == *.mdc ]]; then
local frontmatter_file
frontmatter_file=$(mktemp) || return 1
printf '%s\n' "---" "description: Project Development Guidelines" 'globs: ["**/*"]' "alwaysApply: true" "---" "" >"$frontmatter_file"
cat "$temp_file" >>"$frontmatter_file"
mv "$frontmatter_file" "$temp_file"
fi
return 0
}
update_existing_agent_file() {
local target_file="$1"
local current_date="$2"
log_info "Updating existing agent context file..."
# Use a single temporary file for atomic update
local temp_file
temp_file=$(mktemp) || {
log_error "Failed to create temporary file"
return 1
}
# Process the file in one pass
local tech_stack=$(format_technology_stack "$NEW_LANG" "$NEW_FRAMEWORK")
local new_tech_entries=()
local new_change_entry=""
# Prepare new technology entries
if [[ -n $tech_stack ]] && ! grep -q "$tech_stack" "$target_file"; then
new_tech_entries+=("- $tech_stack ($CURRENT_BRANCH)")
fi
if [[ -n $NEW_DB ]] && [[ $NEW_DB != "N/A" ]] && [[ $NEW_DB != "NEEDS CLARIFICATION" ]] && ! grep -q "$NEW_DB" "$target_file"; then
new_tech_entries+=("- $NEW_DB ($CURRENT_BRANCH)")
fi
# Prepare new change entry
if [[ -n $tech_stack ]]; then
new_change_entry="- $CURRENT_BRANCH: Added $tech_stack"
elif [[ -n $NEW_DB ]] && [[ $NEW_DB != "N/A" ]] && [[ $NEW_DB != "NEEDS CLARIFICATION" ]]; then
new_change_entry="- $CURRENT_BRANCH: Added $NEW_DB"
fi
# Check if sections exist in the file
local has_active_technologies=0
local has_recent_changes=0
if grep -q "^## Active Technologies" "$target_file" 2>/dev/null; then
has_active_technologies=1
fi
if grep -q "^## Recent Changes" "$target_file" 2>/dev/null; then
has_recent_changes=1
fi
# Process file line by line
local in_tech_section=false
local in_changes_section=false
local tech_entries_added=false
local changes_entries_added=false
local existing_changes_count=0
local file_ended=false
while IFS= read -r line || [[ -n $line ]]; do
# Handle Active Technologies section
if [[ $line == "## Active Technologies" ]]; then
echo "$line" >>"$temp_file"
in_tech_section=true
continue
elif [[ $in_tech_section == true ]] && [[ $line =~ ^##[[:space:]] ]]; then
# Add new tech entries before closing the section
if [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
printf '%s\n' "${new_tech_entries[@]}" >>"$temp_file"
tech_entries_added=true
fi
echo "$line" >>"$temp_file"
in_tech_section=false
continue
elif [[ $in_tech_section == true ]] && [[ -z $line ]]; then
# Add new tech entries before empty line in tech section
if [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
printf '%s\n' "${new_tech_entries[@]}" >>"$temp_file"
tech_entries_added=true
fi
echo "$line" >>"$temp_file"
continue
fi
# Handle Recent Changes section
if [[ $line == "## Recent Changes" ]]; then
echo "$line" >>"$temp_file"
# Add new change entry right after the heading
if [[ -n $new_change_entry ]]; then
echo "$new_change_entry" >>"$temp_file"
fi
in_changes_section=true
changes_entries_added=true
continue
elif [[ $in_changes_section == true ]] && [[ $line =~ ^##[[:space:]] ]]; then
echo "$line" >>"$temp_file"
in_changes_section=false
continue
elif [[ $in_changes_section == true ]] && [[ $line == "- "* ]]; then
# Keep only first 2 existing changes
if [[ $existing_changes_count -lt 2 ]]; then
echo "$line" >>"$temp_file"
((existing_changes_count++))
fi
continue
fi
# Update timestamp
if [[ $line =~ (\*\*)?Last\ updated(\*\*)?:.*[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] ]]; then
echo "$line" | sed "s/[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]/$current_date/" >>"$temp_file"
else
echo "$line" >>"$temp_file"
fi
done <"$target_file"
# Post-loop check: if we're still in the Active Technologies section and haven't added new entries
if [[ $in_tech_section == true ]] && [[ $tech_entries_added == false ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
printf '%s\n' "${new_tech_entries[@]}" >>"$temp_file"
tech_entries_added=true
fi
# If sections don't exist, add them at the end of the file
if [[ $has_active_technologies -eq 0 ]] && [[ ${#new_tech_entries[@]} -gt 0 ]]; then
echo "" >>"$temp_file"
echo "## Active Technologies" >>"$temp_file"
printf '%s\n' "${new_tech_entries[@]}" >>"$temp_file"
tech_entries_added=true
fi
if [[ $has_recent_changes -eq 0 ]] && [[ -n $new_change_entry ]]; then
echo "" >>"$temp_file"
echo "## Recent Changes" >>"$temp_file"
echo "$new_change_entry" >>"$temp_file"
changes_entries_added=true
fi
# Ensure Cursor .mdc files have YAML frontmatter for auto-inclusion
if [[ $target_file == *.mdc ]]; then
if ! head -1 "$temp_file" | grep -q '^---'; then
local frontmatter_file
frontmatter_file=$(mktemp) || {
rm -f "$temp_file"
return 1
}
printf '%s\n' "---" "description: Project Development Guidelines" 'globs: ["**/*"]' "alwaysApply: true" "---" "" >"$frontmatter_file"
cat "$temp_file" >>"$frontmatter_file"
mv "$frontmatter_file" "$temp_file"
fi
fi
# Move temp file to target atomically
if ! mv "$temp_file" "$target_file"; then
log_error "Failed to update target file"
rm -f "$temp_file"
return 1
fi
return 0
}
#==============================================================================
# Main Agent File Update Function
#==============================================================================
update_agent_file() {
local target_file="$1"
local agent_name="$2"
if [[ -z $target_file ]] || [[ -z $agent_name ]]; then
log_error "update_agent_file requires target_file and agent_name parameters"
return 1
fi
log_info "Updating $agent_name context file: $target_file"
local project_name
project_name=$(basename "$REPO_ROOT")
local current_date
current_date=$(date +%Y-%m-%d)
# Create directory if it doesn't exist
local target_dir
target_dir=$(dirname "$target_file")
if [[ ! -d $target_dir ]]; then
if ! mkdir -p "$target_dir"; then
log_error "Failed to create directory: $target_dir"
return 1
fi
fi
if [[ ! -f $target_file ]]; then
# Create new file from template
local temp_file
temp_file=$(mktemp) || {
log_error "Failed to create temporary file"
return 1
}
if create_new_agent_file "$target_file" "$temp_file" "$project_name" "$current_date"; then
if mv "$temp_file" "$target_file"; then
log_success "Created new $agent_name context file"
else
log_error "Failed to move temporary file to $target_file"
rm -f "$temp_file"
return 1
fi
else
log_error "Failed to create new agent file"
rm -f "$temp_file"
return 1
fi
else
# Update existing file
if [[ ! -r $target_file ]]; then
log_error "Cannot read existing file: $target_file"
return 1
fi
if [[ ! -w $target_file ]]; then
log_error "Cannot write to existing file: $target_file"
return 1
fi
if update_existing_agent_file "$target_file" "$current_date"; then
log_success "Updated existing $agent_name context file"
else
log_error "Failed to update existing agent file"
return 1
fi
fi
return 0
}
#==============================================================================
# Agent Selection and Processing
#==============================================================================
update_specific_agent() {
local agent_type="$1"
case "$agent_type" in
claude)
update_agent_file "$CLAUDE_FILE" "Claude Code" || return 1
;;
gemini)
update_agent_file "$GEMINI_FILE" "Gemini CLI" || return 1
;;
copilot)
update_agent_file "$COPILOT_FILE" "GitHub Copilot" || return 1
;;
cursor-agent)
update_agent_file "$CURSOR_FILE" "Cursor IDE" || return 1
;;
qwen)
update_agent_file "$QWEN_FILE" "Qwen Code" || return 1
;;
opencode)
update_agent_file "$AGENTS_FILE" "opencode" || return 1
;;
codex)
update_agent_file "$AGENTS_FILE" "Codex CLI" || return 1
;;
windsurf)
update_agent_file "$WINDSURF_FILE" "Windsurf" || return 1
;;
junie)
update_agent_file "$JUNIE_FILE" "Junie" || return 1
;;
kilocode)
update_agent_file "$KILOCODE_FILE" "Kilo Code" || return 1
;;
auggie)
update_agent_file "$AUGGIE_FILE" "Auggie CLI" || return 1
;;
roo)
update_agent_file "$ROO_FILE" "Roo Code" || return 1
;;
codebuddy)
update_agent_file "$CODEBUDDY_FILE" "CodeBuddy CLI" || return 1
;;
qodercli)
update_agent_file "$QODER_FILE" "Qoder CLI" || return 1
;;
amp)
update_agent_file "$AMP_FILE" "Amp" || return 1
;;
shai)
update_agent_file "$SHAI_FILE" "SHAI" || return 1
;;
tabnine)
update_agent_file "$TABNINE_FILE" "Tabnine CLI" || return 1
;;
kiro-cli)
update_agent_file "$KIRO_FILE" "Kiro CLI" || return 1
;;
agy)
update_agent_file "$AGY_FILE" "Antigravity" || return 1
;;
bob)
update_agent_file "$BOB_FILE" "IBM Bob" || return 1
;;
vibe)
update_agent_file "$VIBE_FILE" "Mistral Vibe" || return 1
;;
kimi)
update_agent_file "$KIMI_FILE" "Kimi Code" || return 1
;;
trae)
update_agent_file "$TRAE_FILE" "Trae" || return 1
;;
pi)
update_agent_file "$AGENTS_FILE" "Pi Coding Agent" || return 1
;;
iflow)
update_agent_file "$IFLOW_FILE" "iFlow CLI" || return 1
;;
generic)
log_info "Generic agent: no predefined context file. Use the agent-specific update script for your agent."
;;
*)
log_error "Unknown agent type '$agent_type'"
log_error "Expected: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|junie|kilocode|auggie|roo|codebuddy|amp|shai|tabnine|kiro-cli|agy|bob|vibe|qodercli|kimi|trae|pi|iflow|generic"
exit 1
;;
esac
}
# Helper: skip non-existent files and files already updated (dedup by
# realpath so that variables pointing to the same file — e.g. AMP_FILE,
# KIRO_FILE, BOB_FILE all resolving to AGENTS_FILE — are only written once).
# Uses a linear array instead of associative array for bash 3.2 compatibility.
# Note: defined at top level because bash 3.2 does not support true
# nested/local functions. _updated_paths, _found_agent, and _all_ok are
# initialised exclusively inside update_all_existing_agents so that
# sourcing this script has no side effects on the caller's environment.
_update_if_new() {
local file="$1" name="$2"
[[ -f $file ]] || return 0
local real_path
real_path=$(realpath "$file" 2>/dev/null || echo "$file")
local p
if [[ ${#_updated_paths[@]} -gt 0 ]]; then
for p in "${_updated_paths[@]}"; do
[[ $p == "$real_path" ]] && return 0
done
fi
# Record the file as seen before attempting the update so that:
# (a) aliases pointing to the same path are not retried on failure
# (b) _found_agent reflects file existence, not update success
_updated_paths+=("$real_path")
_found_agent=true
update_agent_file "$file" "$name"
}
update_all_existing_agents() {
_found_agent=false
_updated_paths=()
local _all_ok=true
_update_if_new "$CLAUDE_FILE" "Claude Code" || _all_ok=false
_update_if_new "$GEMINI_FILE" "Gemini CLI" || _all_ok=false
_update_if_new "$COPILOT_FILE" "GitHub Copilot" || _all_ok=false
_update_if_new "$CURSOR_FILE" "Cursor IDE" || _all_ok=false
_update_if_new "$QWEN_FILE" "Qwen Code" || _all_ok=false
_update_if_new "$AGENTS_FILE" "Codex/opencode" || _all_ok=false
_update_if_new "$AMP_FILE" "Amp" || _all_ok=false
_update_if_new "$KIRO_FILE" "Kiro CLI" || _all_ok=false
_update_if_new "$BOB_FILE" "IBM Bob" || _all_ok=false
_update_if_new "$WINDSURF_FILE" "Windsurf" || _all_ok=false
_update_if_new "$JUNIE_FILE" "Junie" || _all_ok=false
_update_if_new "$KILOCODE_FILE" "Kilo Code" || _all_ok=false
_update_if_new "$AUGGIE_FILE" "Auggie CLI" || _all_ok=false
_update_if_new "$ROO_FILE" "Roo Code" || _all_ok=false
_update_if_new "$CODEBUDDY_FILE" "CodeBuddy CLI" || _all_ok=false
_update_if_new "$SHAI_FILE" "SHAI" || _all_ok=false
_update_if_new "$TABNINE_FILE" "Tabnine CLI" || _all_ok=false
_update_if_new "$QODER_FILE" "Qoder CLI" || _all_ok=false
_update_if_new "$AGY_FILE" "Antigravity" || _all_ok=false
_update_if_new "$VIBE_FILE" "Mistral Vibe" || _all_ok=false
_update_if_new "$KIMI_FILE" "Kimi Code" || _all_ok=false
_update_if_new "$TRAE_FILE" "Trae" || _all_ok=false
_update_if_new "$IFLOW_FILE" "iFlow CLI" || _all_ok=false
# If no agent files exist, create a default Claude file
if [[ $_found_agent == false ]]; then
log_info "No existing agent files found, creating default Claude file..."
update_agent_file "$CLAUDE_FILE" "Claude Code" || return 1
fi
[[ $_all_ok == true ]]
}
print_summary() {
echo
log_info "Summary of changes:"
if [[ -n $NEW_LANG ]]; then
echo " - Added language: $NEW_LANG"
fi
if [[ -n $NEW_FRAMEWORK ]]; then
echo " - Added framework: $NEW_FRAMEWORK"
fi
if [[ -n $NEW_DB ]] && [[ $NEW_DB != "N/A" ]]; then
echo " - Added database: $NEW_DB"
fi
echo
log_info "Usage: $0 [claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|junie|kilocode|auggie|roo|codebuddy|amp|shai|tabnine|kiro-cli|agy|bob|vibe|qodercli|kimi|trae|pi|iflow|generic]"
}
#==============================================================================
# Main Execution
#==============================================================================
main() {
# Validate environment before proceeding
validate_environment
log_info "=== Updating agent context files for feature $CURRENT_BRANCH ==="
# Parse the plan file to extract project information
if ! parse_plan_data "$NEW_PLAN"; then
log_error "Failed to parse plan data"
exit 1
fi
# Process based on agent type argument
local success=true
if [[ -z $AGENT_TYPE ]]; then
# No specific agent provided - update all existing agent files
log_info "No agent specified, updating all existing agent files..."
if ! update_all_existing_agents; then
success=false
fi
else
# Specific agent provided - update only that agent
log_info "Updating specific agent: $AGENT_TYPE"
if ! update_specific_agent "$AGENT_TYPE"; then
success=false
fi
fi
# Print summary
print_summary
if [[ $success == true ]]; then
log_success "Agent context update completed successfully"
exit 0
else
log_error "Agent context update completed with errors"
exit 1
fi
}
# Execute main function if script is run directly
if [[ ${BASH_SOURCE[0]} == "${0}" ]]; then
main "$@"
fi
+28
View File
@@ -0,0 +1,28 @@
# [PROJECT NAME] Development Guidelines
Auto-generated from all feature plans. Last updated: [DATE]
## Active Technologies
[EXTRACTED FROM ALL PLAN.MD FILES]
## Project Structure
```text
[ACTUAL STRUCTURE FROM PLANS]
```
## Commands
[ONLY COMMANDS FOR ACTIVE TECHNOLOGIES]
## Code Style
[LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE]
## Recent Changes
[LAST 3 FEATURES AND WHAT THEY ADDED]
<!-- MANUAL ADDITIONS START -->
<!-- MANUAL ADDITIONS END -->
+40
View File
@@ -0,0 +1,40 @@
# [CHECKLIST TYPE] Checklist: [FEATURE NAME]
**Purpose**: [Brief description of what this checklist covers]
**Created**: [DATE]
**Feature**: [Link to spec.md or relevant documentation]
**Note**: This checklist is generated by the `/speckit.checklist` command based on feature context and requirements.
<!--
============================================================================
IMPORTANT: The checklist items below are SAMPLE ITEMS for illustration only.
The /speckit.checklist command MUST replace these with actual items based on:
- User's specific checklist request
- Feature requirements from spec.md
- Technical context from plan.md
- Implementation details from tasks.md
DO NOT keep these sample items in the generated checklist file.
============================================================================
-->
## [Category 1]
- [ ] CHK001 First checklist item with clear action
- [ ] CHK002 Second checklist item
- [ ] CHK003 Third checklist item
## [Category 2]
- [ ] CHK004 Another category item
- [ ] CHK005 Item with specific criteria
- [ ] CHK006 Final item in this category
## Notes
- Check items off as completed: `[x]`
- Add comments or findings inline
- Link to relevant resources or documentation
- Items are numbered sequentially for easy reference
@@ -0,0 +1,73 @@
# [PROJECT_NAME] Constitution
<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
## Core Principles
### [PRINCIPLE_1_NAME]
<!-- Example: I. Library-First -->
[PRINCIPLE_1_DESCRIPTION]
<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
### [PRINCIPLE_2_NAME]
<!-- Example: II. CLI Interface -->
[PRINCIPLE_2_DESCRIPTION]
<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
### [PRINCIPLE_3_NAME]
<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
[PRINCIPLE_3_DESCRIPTION]
<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
### [PRINCIPLE_4_NAME]
<!-- Example: IV. Integration Testing -->
[PRINCIPLE_4_DESCRIPTION]
<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
### [PRINCIPLE_5_NAME]
<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
[PRINCIPLE_5_DESCRIPTION]
<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
## [SECTION_2_NAME]
<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
[SECTION_2_CONTENT]
<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
## [SECTION_3_NAME]
<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
[SECTION_3_CONTENT]
<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
## Governance
<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
[GOVERNANCE_RULES]
<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
+109
View File
@@ -0,0 +1,109 @@
# Implementation Plan: [FEATURE]
**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
**Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/plan-template.md` for the execution workflow.
## Summary
[Extract from feature spec: primary requirement + technical approach from research]
## Technical Context
<!--
ACTION REQUIRED: Replace the content in this section with the technical details
for the project. The structure here is presented in advisory capacity to guide
the iteration process.
-->
**Language/Version**: [e.g., Python 3.11, Swift 5.9, Rust 1.75 or NEEDS CLARIFICATION]
**Primary Dependencies**: [e.g., FastAPI, UIKit, LLVM or NEEDS CLARIFICATION]
**Storage**: [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
**Testing**: [e.g., pytest, XCTest, cargo test or NEEDS CLARIFICATION]
**Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
**Project Type**: [e.g., library/cli/web-service/mobile-app/compiler/desktop-app or NEEDS CLARIFICATION]
**Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
**Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
**Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]
## Constitution Check
_GATE: Must pass before Phase 0 research. Re-check after Phase 1 design._
- Safety-critical mesh impact is identified for any routing, airtime, MQTT, channel, or packet-path change.
- Variant and platform scope are explicit, and all hardware flags or pin mappings are verified against the target board.
- Validation evidence is defined, including the exact `pio`, native test, simulator, formatting, or static checks to run.
- Resource, power, memory, and dependency impact are assessed for the affected targets.
- Any constitutional violation or validation gap is documented with justification in Complexity Tracking.
## Project Structure
### Documentation (this feature)
```text
specs/[###-feature]/
├── plan.md # This file (/speckit.plan command output)
├── research.md # Phase 0 output (/speckit.plan command)
├── data-model.md # Phase 1 output (/speckit.plan command)
├── quickstart.md # Phase 1 output (/speckit.plan command)
├── contracts/ # Phase 1 output (/speckit.plan command)
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
```
### Source Code (repository root)
<!--
ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
for this feature. Delete unused options and expand the chosen structure with
real paths (e.g., apps/admin, packages/something). The delivered plan must
not include Option labels.
-->
```text
# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
src/
├── models/
├── services/
├── cli/
└── lib/
tests/
├── contract/
├── integration/
└── unit/
# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
backend/
├── src/
│ ├── models/
│ ├── services/
│ └── api/
└── tests/
frontend/
├── src/
│ ├── components/
│ ├── pages/
│ └── services/
└── tests/
# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
api/
└── [same as backend above]
ios/ or android/
└── [platform-specific structure: feature modules, UI flows, platform tests]
```
**Structure Decision**: [Document the selected structure and reference the real
directories captured above]
## Complexity Tracking
> **Fill ONLY if Constitution Check has violations that must be justified**
| Violation | Why Needed | Simpler Alternative Rejected Because |
| -------------------------- | ------------------ | ------------------------------------ |
| [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
+140
View File
@@ -0,0 +1,140 @@
# Feature Specification: [FEATURE NAME]
**Feature Branch**: `[###-feature-name]`
**Created**: [DATE]
**Status**: Draft
**Input**: User description: "$ARGUMENTS"
## User Scenarios & Testing _(mandatory)_
<!--
IMPORTANT: User stories should be PRIORITIZED as user journeys ordered by importance.
Each user story/journey must be INDEPENDENTLY TESTABLE - meaning if you implement just ONE of them,
you should still have a viable MVP (Minimum Viable Product) that delivers value.
Assign priorities (P1, P2, P3, etc.) to each story, where P1 is the most critical.
Think of each story as a standalone slice of functionality that can be:
- Developed independently
- Tested independently
- Deployed independently
- Demonstrated to users independently
-->
### User Story 1 - [Brief Title] (Priority: P1)
[Describe this user journey in plain language]
**Why this priority**: [Explain the value and why it has this priority level]
**Independent Test**: [Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"]
**Acceptance Scenarios**:
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
2. **Given** [initial state], **When** [action], **Then** [expected outcome]
---
### User Story 2 - [Brief Title] (Priority: P2)
[Describe this user journey in plain language]
**Why this priority**: [Explain the value and why it has this priority level]
**Independent Test**: [Describe how this can be tested independently]
**Acceptance Scenarios**:
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
---
### User Story 3 - [Brief Title] (Priority: P3)
[Describe this user journey in plain language]
**Why this priority**: [Explain the value and why it has this priority level]
**Independent Test**: [Describe how this can be tested independently]
**Acceptance Scenarios**:
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
---
[Add more user stories as needed, each with an assigned priority]
For firmware and hardware-facing work, each story MUST identify affected platforms or variants
and describe how the behavior is validated on its own.
### Edge Cases
<!--
ACTION REQUIRED: The content in this section represents placeholders.
Fill them out with the right edge cases.
-->
- What happens when [boundary condition]?
- How does system handle [error scenario]?
- What happens when the target hardware capability is absent, misdeclared, or only present on some variants?
- How does the system behave when radio, power, timing, or memory constraints are tighter than expected?
## Requirements _(mandatory)_
<!--
ACTION REQUIRED: The content in this section represents placeholders.
Fill them out with the right functional requirements.
-->
### Functional Requirements
- **FR-001**: System MUST [specific capability, e.g., "allow users to create accounts"]
- **FR-002**: System MUST [specific capability, e.g., "validate email addresses"]
- **FR-003**: Users MUST be able to [key interaction, e.g., "reset their password"]
- **FR-004**: System MUST [data requirement, e.g., "persist user preferences"]
- **FR-005**: System MUST [behavior, e.g., "log all security events"]
_Example of marking unclear requirements:_
- **FR-006**: System MUST authenticate users via [NEEDS CLARIFICATION: auth method not specified - email/password, SSO, OAuth?]
- **FR-007**: System MUST retain user data for [NEEDS CLARIFICATION: retention period not specified]
Where relevant, requirements MUST also state:
- affected architectures, boards, or modules
- whether behavior changes public defaults, protocol compatibility, or generated artifacts
- any required validation evidence for high-risk mesh, hardware, or power behavior
### Key Entities _(include if feature involves data)_
- **[Entity 1]**: [What it represents, key attributes without implementation]
- **[Entity 2]**: [What it represents, relationships to other entities]
## Success Criteria _(mandatory)_
<!--
ACTION REQUIRED: Define measurable success criteria.
These must be technology-agnostic and measurable.
-->
### Measurable Outcomes
- **SC-001**: [Measurable metric, e.g., "Users can complete account creation in under 2 minutes"]
- **SC-002**: [Measurable metric, e.g., "System handles 1000 concurrent users without degradation"]
- **SC-003**: [User satisfaction metric, e.g., "90% of users successfully complete primary task on first attempt"]
- **SC-004**: [Business metric, e.g., "Reduce support tickets related to [X] by 50%"]
## Assumptions
<!--
ACTION REQUIRED: The content in this section represents placeholders.
Fill them out with the right assumptions based on reasonable defaults
chosen when the feature description did not specify certain details.
-->
- [Assumption about target users, e.g., "Users have stable internet connectivity"]
- [Assumption about scope boundaries, e.g., "Mobile support is out of scope for v1"]
- [Assumption about data/environment, e.g., "Existing authentication system will be reused"]
- [Dependency on existing system/service, e.g., "Requires access to the existing user profile API"]
- [Assumption about available hardware capabilities, board revisions, or build targets]
+259
View File
@@ -0,0 +1,259 @@
---
description: "Task list template for feature implementation"
---
# Tasks: [FEATURE NAME]
**Input**: Design documents from `/specs/[###-feature-name]/`
**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/
**Tests**: Validation tasks are REQUIRED whenever the constitution or feature risk profile demands evidence. Include targeted builds, native tests, simulator runs, protobuf regeneration checks, formatting, or other repository-standard validation appropriate to the change.
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
## Format: `[ID] [P?] [Story] Description`
- **[P]**: Can run in parallel (different files, no dependencies)
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
- Include exact file paths in descriptions
## Path Conventions
- **Single project**: `src/`, `tests/` at repository root
- **Web app**: `backend/src/`, `frontend/src/`
- **Mobile**: `api/src/`, `ios/src/` or `android/src/`
- Paths shown below assume single project - adjust based on plan.md structure
<!--
============================================================================
IMPORTANT: The tasks below are SAMPLE TASKS for illustration purposes only.
The /speckit.tasks command MUST replace these with actual tasks based on:
- User stories from spec.md (with their priorities P1, P2, P3...)
- Feature requirements from plan.md
- Entities from data-model.md
- Endpoints from contracts/
Tasks MUST be organized by user story so each story can be:
- Implemented independently
- Tested independently
- Delivered as an MVP increment
DO NOT keep these sample tasks in the generated tasks.md file.
============================================================================
-->
## Phase 1: Setup (Shared Infrastructure)
**Purpose**: Project initialization and basic structure
- [ ] T001 Create project structure per implementation plan
- [ ] T002 Initialize [language] project with [framework] dependencies
- [ ] T003 [P] Configure linting and formatting tools
- [ ] T00X Identify affected targets, variants, and validation commands
---
## Phase 2: Foundational (Blocking Prerequisites)
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
Examples of foundational tasks (adjust based on your project):
- [ ] T004 Setup database schema and migrations framework
- [ ] T005 [P] Implement authentication/authorization framework
- [ ] T006 [P] Setup API routing and middleware structure
- [ ] T007 Create base models/entities that all stories depend on
- [ ] T008 Configure error handling and logging infrastructure
- [ ] T009 Setup environment configuration management
- [ ] T00X Verify board-specific flags, pins, generated assets, or protocol prerequisites needed by all stories
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
---
## Phase 3: User Story 1 - [Title] (Priority: P1) 🎯 MVP
**Goal**: [Brief description of what this story delivers]
**Independent Test**: [How to verify this story works on its own]
### Validation for User Story 1 ⚠️
> **NOTE: Add the smallest credible validation for the story's risk level before declaring it complete**
- [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test\_[name].py
- [ ] T01X [P] [US1] Run targeted build or simulation for affected platform(s)
### Implementation for User Story 1
- [ ] T012 [P] [US1] Create [Entity1] model in src/models/[entity1].py
- [ ] T013 [P] [US1] Create [Entity2] model in src/models/[entity2].py
- [ ] T014 [US1] Implement [Service] in src/services/[service].py (depends on T012, T013)
- [ ] T015 [US1] Implement [endpoint/feature] in src/[location]/[file].py
- [ ] T016 [US1] Add validation and error handling
- [ ] T017 [US1] Add logging for user story 1 operations
- [ ] T01Y [US1] Confirm variant-scoped behavior and configuration defaults remain correct
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
---
## Phase 4: User Story 2 - [Title] (Priority: P2)
**Goal**: [Brief description of what this story delivers]
**Independent Test**: [How to verify this story works on its own]
### Validation for User Story 2 ⚠️
- [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test\_[name].py
- [ ] T02X [P] [US2] Run targeted build or simulation for affected platform(s)
### Implementation for User Story 2
- [ ] T020 [P] [US2] Create [Entity] model in src/models/[entity].py
- [ ] T021 [US2] Implement [Service] in src/services/[service].py
- [ ] T022 [US2] Implement [endpoint/feature] in src/[location]/[file].py
- [ ] T023 [US2] Integrate with User Story 1 components (if needed)
- [ ] T02Y [US2] Confirm resource, power, and compatibility impacts stay within plan constraints
**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently
---
## Phase 5: User Story 3 - [Title] (Priority: P3)
**Goal**: [Brief description of what this story delivers]
**Independent Test**: [How to verify this story works on its own]
### Validation for User Story 3 ⚠️
- [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test\_[name].py
- [ ] T03X [P] [US3] Run targeted build or simulation for affected platform(s)
### Implementation for User Story 3
- [ ] T026 [P] [US3] Create [Entity] model in src/models/[entity].py
- [ ] T027 [US3] Implement [Service] in src/services/[service].py
- [ ] T028 [US3] Implement [endpoint/feature] in src/[location]/[file].py
- [ ] T03Y [US3] Confirm backward compatibility or explicitly document intentional behavior changes
**Checkpoint**: All user stories should now be independently functional
---
[Add more user story phases as needed, following the same pattern]
---
## Phase N: Polish & Cross-Cutting Concerns
**Purpose**: Improvements that affect multiple user stories
- [ ] TXXX [P] Documentation updates in docs/
- [ ] TXXX Code cleanup and refactoring
- [ ] TXXX Performance optimization across all stories
- [ ] TXXX [P] Additional unit tests (if requested) in tests/unit/
- [ ] TXXX Security hardening
- [ ] TXXX Run quickstart.md validation
- [ ] TXXX Summarize skipped validations, migration notes, and generated-file updates for review
---
## Dependencies & Execution Order
### Phase Dependencies
- **Setup (Phase 1)**: No dependencies - can start immediately
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
- User stories can then proceed in parallel (if staffed)
- Or sequentially in priority order (P1 → P2 → P3)
- **Polish (Final Phase)**: Depends on all desired user stories being complete
### User Story Dependencies
- **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
- **User Story 2 (P2)**: Can start after Foundational (Phase 2) - May integrate with US1 but should be independently testable
- **User Story 3 (P3)**: Can start after Foundational (Phase 2) - May integrate with US1/US2 but should be independently testable
### Within Each User Story
- Tests (if included) MUST be written and FAIL before implementation
- Models before services
- Services before endpoints
- Core implementation before integration
- Story complete before moving to next priority
### Parallel Opportunities
- All Setup tasks marked [P] can run in parallel
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
- Once Foundational phase completes, all user stories can start in parallel (if team capacity allows)
- All tests for a user story marked [P] can run in parallel
- Models within a story marked [P] can run in parallel
- Different user stories can be worked on in parallel by different team members
---
## Parallel Example: User Story 1
```bash
# Launch all tests for User Story 1 together (if tests requested):
Task: "Contract test for [endpoint] in tests/contract/test_[name].py"
Task: "Integration test for [user journey] in tests/integration/test_[name].py"
# Launch all models for User Story 1 together:
Task: "Create [Entity1] model in src/models/[entity1].py"
Task: "Create [Entity2] model in src/models/[entity2].py"
```
---
## Implementation Strategy
### MVP First (User Story 1 Only)
1. Complete Phase 1: Setup
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
3. Complete Phase 3: User Story 1
4. **STOP and VALIDATE**: Test User Story 1 independently
5. Deploy/demo if ready
### Incremental Delivery
1. Complete Setup + Foundational → Foundation ready
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
3. Add User Story 2 → Test independently → Deploy/Demo
4. Add User Story 3 → Test independently → Deploy/Demo
5. Each story adds value without breaking previous stories
### Parallel Team Strategy
With multiple developers:
1. Team completes Setup + Foundational together
2. Once Foundational is done:
- Developer A: User Story 1
- Developer B: User Story 2
- Developer C: User Story 3
3. Stories complete and integrate independently
---
## Notes
- [P] tasks = different files, no dependencies
- [Story] label maps task to specific user story for traceability
- Each user story should be independently completable and testable
- Verify tests fail before implementing
- Commit after each task or logical group
- Stop at any checkpoint to validate story independently
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence
+5 -12
View File
@@ -4,19 +4,19 @@ cli:
plugins:
sources:
- id: trunk
ref: v1.10.0
ref: v1.8.0
uri: https://github.com/trunk-io/plugins
lint:
enabled:
- checkov@3.2.529
- checkov@3.2.526
- renovate@43.150.0
- prettier@3.8.3
- trufflehog@3.95.3
- trufflehog@3.95.2
- yamllint@1.38.0
- bandit@1.9.4
- trivy@0.70.0
- taplo@0.10.0
- ruff@0.15.13
- ruff@0.15.12
- isort@8.0.1
- markdownlint@0.48.0
- oxipng@10.1.1
@@ -26,7 +26,7 @@ lint:
- hadolint@2.14.0
- shfmt@3.6.0
- shellcheck@0.11.0
- black@26.5.1
- black@26.3.1
- git-diff-check
- gitleaks@8.30.1
- clang-format@16.0.3
@@ -34,13 +34,6 @@ lint:
- linters: [ALL]
paths:
- bin/**
# Fake-NodeDB fixture JSONL files contain deterministic synthetic
# public_key_hex (64-char hex) values that gitleaks misidentifies as
# generic-api-key. These are not secrets — they're test fixtures
# produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
- linters: [gitleaks]
paths:
- test/fixtures/nodedb/seed_v25_*.jsonl
runtimes:
enabled:
- python@3.14.4
+4 -4
View File
@@ -1,10 +1,10 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide"
"platformio.platformio-ide"
],
"unwantedRecommendations": [
"ms-vscode.cpptools-extension-pack",
"platformio.platformio-ide"
"ms-vscode.cpptools-extension-pack"
]
}
+11
View File
@@ -10,5 +10,16 @@
},
"[powershell]": {
"editor.defaultFormatter": "ms-vscode.powershell"
},
"chat.promptFilesRecommendations": {
"speckit.constitution": true,
"speckit.specify": true,
"speckit.plan": true,
"speckit.tasks": true,
"speckit.implement": true
},
"chat.tools.terminal.autoApprove": {
".specify/scripts/bash/": true,
".specify/scripts/powershell/": true
}
}
-2
View File
@@ -66,8 +66,6 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
- **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
- **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
## Typical agent workflows
-64
View File
@@ -1,64 +0,0 @@
#!/usr/bin/env python3
"""Post-process protoc-generated Python files to live under a local namespace.
Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
target directory and rewrites every `meshtastic` reference (imports, dotted
attribute access) to use the new namespace (e.g., `meshtastic_v25`).
Why: the .proto files declare `package meshtastic;`, so protoc emits
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
`meshtastic` package which other parts of the mcp-server depend on. Renaming
to a local namespace keeps both available.
Usage:
_rewrite_proto_namespace.py <generated_dir> <new_namespace>
"""
from __future__ import annotations
import pathlib
import re
import sys
def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
# Standard protoc import forms:
# from meshtastic.X_pb2 import ... (rare, for direct symbol pulls)
# from meshtastic import X_pb2 as ... (common, the cross-file ref)
# import meshtastic.X_pb2 (also possible)
pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
count = 0
for p in dir_path.glob("*.py"):
text = p.read_text(encoding="utf-8")
new = pattern_dotted_from.sub(f"from {new_ns}.", text)
new = pattern_bare_from.sub(f"from {new_ns} import ", new)
new = pattern_dotted_import.sub(f"import {new_ns}.", new)
# NOTE: we deliberately leave `meshtastic/X.proto` source-filename
# references inside descriptor strings alone. The descriptor pool is
# keyed by source filename (independent of Python package layout), so
# those don't collide with the PyPI package's descriptors.
if new != text:
p.write_text(new, encoding="utf-8")
count += 1
return count
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
return 2
dir_path = pathlib.Path(argv[0])
new_ns = argv[1]
if not dir_path.is_dir():
print(f"directory not found: {dir_path}", file=sys.stderr)
return 2
n = rewrite(dir_path, new_ns)
print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+628
View File
@@ -0,0 +1,628 @@
#!/usr/bin/env python3
"""Board intake assessment for new Meshtastic hardware support.
Validates a board intake request against the repository hardware context,
identifies evidence gaps, and produces a structured readiness report.
Usage:
python3 bin/board_intake.py <intake.json>
python3 bin/board_intake.py <intake.json> --output report.md
python3 bin/board_intake.py <intake.json> --validate # gaps check only, exit 1 if not scaffold_ready
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from dataclasses import dataclass, field
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_CONTEXT_PATH = ROOT / "docs" / "hardware-support-context.md"
# Metadata keys every new PlatformIO environment should declare.
REQUIRED_METADATA_KEYS = [
"custom_meshtastic_hw_model",
"custom_meshtastic_hw_model_slug",
"custom_meshtastic_architecture",
"custom_meshtastic_actively_supported",
"custom_meshtastic_support_level",
"custom_meshtastic_display_name",
]
RECOMMENDED_METADATA_KEYS = [
"custom_meshtastic_images",
"custom_meshtastic_tags",
"custom_meshtastic_requires_dfu",
"custom_meshtastic_partition_scheme",
]
# Pin groups that should be backed by evidence before scaffolding.
EVIDENCE_CATEGORIES = ["radio", "display", "input", "GPS", "power"]
# Architecture families that rely on BSP defaults for many pin defines.
BSP_DEFAULT_FAMILIES = {"nrf52840", "rp2040", "stm32", "native"}
# Known valid architectures from the repository.
KNOWN_ARCHITECTURES = {
"esp32",
"esp32-s3",
"esp32-c3",
"esp32-c6",
"esp32s2",
"nrf52840",
"rp2040",
"rp2350",
"stm32",
"native",
}
# ---------------------------------------------------------------------------
# T004 – BoardIntakeRequest dataclass
# ---------------------------------------------------------------------------
@dataclass
class BoardIntakeRequest:
"""Maintainer-supplied description of a proposed new board."""
# Required
environment_name: str
hardware_model: str
display_name: str
architecture: str
# Recommended
hardware_model_slug: str = ""
actively_supported: bool | None = None
support_level: str = ""
source_materials: list[str] = field(default_factory=list)
board_notes: str = ""
@classmethod
def from_dict(cls, data: dict) -> "BoardIntakeRequest":
return cls(
environment_name=data.get("environment_name", ""),
hardware_model=str(data.get("hardware_model", "")),
display_name=data.get("display_name", ""),
architecture=data.get("architecture", ""),
hardware_model_slug=data.get("hardware_model_slug", ""),
actively_supported=data.get("actively_supported"),
support_level=str(data.get("support_level", "")),
source_materials=list(data.get("source_materials", [])),
board_notes=data.get("board_notes", ""),
)
@classmethod
def from_json(cls, path: Path) -> "BoardIntakeRequest":
data = json.loads(path.read_text(encoding="utf-8"))
return cls.from_dict(data)
# ---------------------------------------------------------------------------
# T005 – EvidenceGap dataclass
# ---------------------------------------------------------------------------
@dataclass
class EvidenceGap:
"""A specific missing, conflicting, or ambiguous hardware fact."""
category: str # metadata | radio | display | input | GPS | power | storage | connectivity | revision-scope
description: str
affected_artifact: str
required_evidence: str
blocking: bool
# ---------------------------------------------------------------------------
# T006 – IntakeAssessment dataclass
# ---------------------------------------------------------------------------
@dataclass
class IntakeAssessment:
"""Structured result of evaluating a BoardIntakeRequest."""
request: BoardIntakeRequest
expected_artifacts: list[str]
required_metadata: list[str]
matched_patterns: list[dict]
evidence_gaps: list[EvidenceGap]
risk_flags: list[str]
next_actions: list[str]
scaffold_ready: bool
# ---------------------------------------------------------------------------
# T007 – load_hardware_context
# ---------------------------------------------------------------------------
def load_hardware_context(path: Path = DEFAULT_CONTEXT_PATH) -> dict:
"""Parse the generated hardware-support-context.md into a usable dict.
Returns:
{
"architecture_names": list[str], # families present in the inventory
"metadata_keys": list[str], # custom_meshtastic_* keys observed
"environments": dict[str, dict], # env_name -> {display, hw_model, hw_slug, variant_dir, arch}
}
"""
if not path.exists():
raise FileNotFoundError(
f"Hardware context not found at {path}. "
"Run: python3 bin/generate_hardware_support_context.py"
)
text = path.read_text(encoding="utf-8")
# Extract metadata keys from the "## Repository Metadata Inputs" section.
metadata_keys: list[str] = re.findall(r"`(custom_meshtastic_[^`]+)`", text)
metadata_keys = list(dict.fromkeys(metadata_keys)) # deduplicate, preserve order
# Extract architecture families from the "## Architecture and Environment Inventory" section.
arch_names: list[str] = re.findall(r"^### ([a-z0-9\-]+)\s*$", text, re.MULTILINE)
# Filter out sub-headings that are architecture names (exclude e.g. "nrf52840" inside examples)
# The inventory section has short arch names; filter to known set plus any that look like archs.
arch_names = [a for a in dict.fromkeys(arch_names) if not a[0].isupper()]
# Extract environments from inventory table rows.
environments: dict[str, dict] = {}
current_arch = ""
for line in text.splitlines():
arch_match = re.match(r"^### ([a-z0-9\-]+)\s*$", line)
if arch_match:
current_arch = arch_match.group(1)
continue
# Table row: | env | display | hw_model | hw_slug | variant_dir | categories |
row = re.match(
r"^\|\s*([^|]+?)\s*\|\s*([^|]*?)\s*\|\s*([^|]*?)\s*\|\s*([^|]*?)\s*\|\s*([^|]*?)\s*\|",
line,
)
if (
row
and not row.group(1).startswith("Environment")
and not row.group(1).startswith("---")
):
env_name = row.group(1).strip()
if env_name:
environments[env_name] = {
"display_name": row.group(2).strip(),
"hw_model": row.group(3).strip(),
"hw_slug": row.group(4).strip(),
"variant_dir": row.group(5).strip(),
"architecture": current_arch,
}
return {
"architecture_names": arch_names,
"metadata_keys": metadata_keys,
"environments": environments,
}
# ---------------------------------------------------------------------------
# T018 – validate_intake
# ---------------------------------------------------------------------------
def validate_intake(request: BoardIntakeRequest, context: dict) -> list[str]:
"""Check required fields and detect conflicts with existing environments/models.
Returns a list of validation error strings (empty = valid).
"""
errors: list[str] = []
if not request.environment_name:
errors.append("environment_name is required.")
if not request.hardware_model:
errors.append("hardware_model is required.")
if not request.display_name:
errors.append("display_name is required.")
if not request.architecture:
errors.append("architecture is required.")
if request.architecture and request.architecture not in KNOWN_ARCHITECTURES:
errors.append(
f"architecture '{request.architecture}' is not a known repository architecture. "
f"Known: {', '.join(sorted(KNOWN_ARCHITECTURES))}"
)
# Conflict: environment name already exists
if request.environment_name and request.environment_name in context.get(
"environments", {}
):
errors.append(
f"environment_name '{request.environment_name}' already exists in the repository. "
"Choose a unique name or confirm this is an intentional update."
)
# Conflict: hardware model already assigned to a different environment
if request.hardware_model:
for env_name, env_data in context.get("environments", {}).items():
if (
env_data.get("hw_model") == request.hardware_model
and env_name != request.environment_name
):
errors.append(
f"hardware_model '{request.hardware_model}' is already assigned to "
f"environment '{env_name}' ({env_data.get('display_name', '')!r}). "
"Verify this is a new model number or confirm the shared-model intent."
)
break # report once
return errors
# ---------------------------------------------------------------------------
# T019 – find_matched_patterns
# ---------------------------------------------------------------------------
def find_matched_patterns(request: BoardIntakeRequest, context: dict) -> list[dict]:
"""Return up to 3 existing environments closest to the request by architecture."""
envs = context.get("environments", {})
arch = request.architecture
# Prefer exact architecture match, then partial (e.g., "esp32" matches "esp32-s3").
exact: list[dict] = []
partial: list[dict] = []
for env_name, env_data in envs.items():
entry = {**env_data, "environment": env_name}
env_arch = env_data.get("architecture", "")
if env_arch == arch:
exact.append(entry)
elif arch and (env_arch.startswith(arch) or arch.startswith(env_arch)):
partial.append(entry)
candidates = exact + partial
# Prefer boards that have a display_name and hw_model (more complete entries).
candidates.sort(key=lambda e: (not e.get("display_name"), not e.get("hw_model")))
return candidates[:3]
# ---------------------------------------------------------------------------
# T020 – build_evidence_gaps
# ---------------------------------------------------------------------------
def build_evidence_gaps(request: BoardIntakeRequest) -> list[EvidenceGap]:
"""Identify missing pin-group evidence and metadata gaps."""
gaps: list[EvidenceGap] = []
has_sources = bool(request.source_materials)
if not has_sources:
# Every pin category is unresolvable without sources.
for cat in EVIDENCE_CATEGORIES:
gaps.append(
EvidenceGap(
category=cat,
description=(
f"No source materials supplied. {cat.capitalize()} pin mappings cannot be "
"verified without a schematic, pinout diagram, or vendor datasheet."
),
affected_artifact="variant.h",
required_evidence="Schematic, pinout image, or vendor board page",
blocking=True,
)
)
else:
# Sources exist but may still be incomplete; flag as non-blocking advisory.
for cat in EVIDENCE_CATEGORIES:
gaps.append(
EvidenceGap(
category=cat,
description=(
f"Source materials are present but {cat} pin assignments have not been "
"extracted and cross-checked against repository macro conventions."
),
affected_artifact="variant.h",
required_evidence=f"Explicit {cat} pin listing matched to repository #define names",
blocking=False,
)
)
# Metadata gaps
if not request.hardware_model_slug:
gaps.append(
EvidenceGap(
category="metadata",
description="hardware_model_slug is not set. The repository requires an UPPER_SNAKE_CASE slug for PlatformIO metadata.",
affected_artifact="platformio.ini (custom_meshtastic_hw_model_slug)",
required_evidence="Agreed slug from the project maintainers",
blocking=True,
)
)
if request.actively_supported is None:
gaps.append(
EvidenceGap(
category="metadata",
description="actively_supported is not specified. This controls CI matrix inclusion.",
affected_artifact="platformio.ini (custom_meshtastic_actively_supported)",
required_evidence="Maintainer decision on support status",
blocking=False,
)
)
if not request.support_level:
gaps.append(
EvidenceGap(
category="metadata",
description="support_level is not specified (expected: 1 = active, 2 = supported, 3 = extra).",
affected_artifact="platformio.ini (custom_meshtastic_support_level)",
required_evidence="Maintainer decision on support tier",
blocking=False,
)
)
# Revision-scope: multiple display/radio options without disambiguation.
# Match whole-word choice language rather than raw substrings so normal text
# like "radio" does not trigger the ambiguity gate.
note_text = request.board_notes.lower()
revision_scope_patterns = (
r"\brevision\b",
r"\bvariant\b",
r"\bvariants\b",
r"\boption\b",
r"\boptions\b",
r"\balternative\b",
r"\balternatives\b",
r"\bmulti\b",
r"\btwo\b",
r"\beither\b",
r"\bor\b",
)
if note_text and any(re.search(pattern, note_text) for pattern in revision_scope_patterns):
gaps.append(
EvidenceGap(
category="revision-scope",
description=(
"Board notes mention multiple variants, revisions, or options. "
"The intake must be scoped to a single hardware revision before scaffolding can proceed."
),
affected_artifact="variant.h, platformio.ini",
required_evidence="Explicit decision on which revision this intake covers",
blocking=True,
)
)
return gaps
# ---------------------------------------------------------------------------
# T021 – assess_intake
# ---------------------------------------------------------------------------
def assess_intake(request: BoardIntakeRequest, context: dict) -> IntakeAssessment:
"""Produce a full structured assessment from intake request and context."""
validation_errors = validate_intake(request, context)
matched = find_matched_patterns(request, context)
gaps = build_evidence_gaps(request)
expected_artifacts = [
f"variants/{request.architecture or '<architecture>'}/<variant-dir>/variant.h",
f"variants/{request.architecture or '<architecture>'}/<variant-dir>/platformio.ini (env:{request.environment_name or '<env>'})",
]
if request.architecture in {"esp32", "esp32-s3", "esp32-c3", "esp32-c6"}:
expected_artifacts.append(
"(optional) variants/.../variant.cpp — only if board requires custom init hooks"
)
expected_artifacts += [
"PlatformIO metadata: all required custom_meshtastic_* keys (see required_metadata below)",
"(optional) board image under branding/ or images/ if custom_meshtastic_images is set",
]
required_metadata = list(REQUIRED_METADATA_KEYS)
if request.architecture in BSP_DEFAULT_FAMILIES:
required_metadata.append(
f"(BSP note) {request.architecture} boards may inherit some pin defines from BSP headers — "
"check the Inherited Defaults section of docs/hardware-support-context.md before assuming a missing define is an error."
)
risk_flags: list[str] = []
for err in validation_errors:
risk_flags.append(f"Validation error: {err}")
if request.architecture in BSP_DEFAULT_FAMILIES:
risk_flags.append(
f"Architecture '{request.architecture}' uses BSP defaults for some pin defines. "
"Verify which macros are inherited before declaring them explicitly in variant.h."
)
if not matched:
risk_flags.append(
"No closely matched existing board found for this architecture. "
"Manual review of variant structure is required."
)
next_actions: list[str] = []
if validation_errors:
next_actions.append("Resolve validation errors before proceeding.")
blocking_gaps = [g for g in gaps if g.blocking]
non_blocking_gaps = [g for g in gaps if not g.blocking]
for gap in blocking_gaps:
next_actions.append(
f"Provide {gap.required_evidence} for {gap.category} ({gap.affected_artifact})."
)
if non_blocking_gaps:
next_actions.append(
f"Review {len(non_blocking_gaps)} non-blocking gap(s) before merging scaffold output."
)
if not next_actions:
next_actions.append(
"All required evidence is present. Proceed to scaffold generation."
)
scaffold_ready = len(validation_errors) == 0 and all(not g.blocking for g in gaps)
return IntakeAssessment(
request=request,
expected_artifacts=expected_artifacts,
required_metadata=required_metadata,
matched_patterns=matched,
evidence_gaps=gaps,
risk_flags=risk_flags,
next_actions=next_actions,
scaffold_ready=scaffold_ready,
)
# ---------------------------------------------------------------------------
# T022 – render_assessment_markdown
# ---------------------------------------------------------------------------
def render_assessment_markdown(assessment: IntakeAssessment) -> str:
req = assessment.request
lines: list[str] = []
lines.append(f"# Board Intake Assessment: `{req.environment_name or '(unnamed)'}`")
lines.append("")
lines.append(
f"**Scaffold ready**: {'✅ Yes' if assessment.scaffold_ready else '❌ No — see blocking gaps below'}"
)
lines.append("")
lines.append("## Request Summary")
lines.append("")
lines.append(f"- **Environment name**: `{req.environment_name}`")
lines.append(f"- **Hardware model**: `{req.hardware_model}`")
lines.append(
f"- **Hardware model slug**: `{req.hardware_model_slug or '(not set)'}`"
)
lines.append(f"- **Display name**: {req.display_name}")
lines.append(f"- **Architecture**: `{req.architecture}`")
if req.actively_supported is not None:
lines.append(f"- **Actively supported**: {req.actively_supported}")
if req.support_level:
lines.append(f"- **Support level**: {req.support_level}")
if req.source_materials:
lines.append("- **Source materials**:")
for src in req.source_materials:
lines.append(f" - {src}")
if req.board_notes:
lines.append(f"- **Board notes**: {req.board_notes}")
lines.append("")
lines.append("## Expected Artifacts")
lines.append("")
for artifact in assessment.expected_artifacts:
lines.append(f"- {artifact}")
lines.append("")
lines.append("## Required Metadata")
lines.append("")
for key in assessment.required_metadata:
lines.append(
f"- `{key}`" if key.startswith("custom_meshtastic") else f"- {key}"
)
lines.append("")
lines.append("## Matched Repository Patterns")
lines.append("")
if assessment.matched_patterns:
for pattern in assessment.matched_patterns:
name = pattern.get("display_name") or pattern.get("environment", "")
env = pattern.get("environment", "")
arch = pattern.get("architecture", "")
vdir = pattern.get("variant_dir", "")
lines.append(f"- **{name}** (`{env}`, {arch}) — `{vdir}`")
else:
lines.append("- No closely matched patterns found for this architecture.")
lines.append("")
blocking = [g for g in assessment.evidence_gaps if g.blocking]
non_blocking = [g for g in assessment.evidence_gaps if not g.blocking]
lines.append("## Evidence Gaps")
lines.append("")
if blocking:
lines.append("### Blocking")
lines.append("")
for gap in blocking:
lines.append(f"- **[{gap.category}]** {gap.description}")
lines.append(f" - Affected: `{gap.affected_artifact}`")
lines.append(f" - Required evidence: {gap.required_evidence}")
else:
lines.append("*No blocking evidence gaps.*")
lines.append("")
if non_blocking:
lines.append("### Non-blocking (review before merge)")
lines.append("")
for gap in non_blocking:
lines.append(f"- **[{gap.category}]** {gap.description}")
lines.append(f" - Affected: `{gap.affected_artifact}`")
lines.append(f" - Required evidence: {gap.required_evidence}")
lines.append("")
if assessment.risk_flags:
lines.append("## Risk Flags")
lines.append("")
for flag in assessment.risk_flags:
lines.append(f"- {flag}")
lines.append("")
lines.append("## Next Actions")
lines.append("")
for i, action in enumerate(assessment.next_actions, 1):
lines.append(f"{i}. {action}")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# T023 – CLI entry point
# ---------------------------------------------------------------------------
def main() -> None:
parser = argparse.ArgumentParser(
description="Evaluate a board intake request against the repository hardware context."
)
parser.add_argument("intake", help="Path to the intake JSON file")
parser.add_argument(
"--output",
default="-",
help="Output path for the assessment markdown (default: stdout)",
)
parser.add_argument(
"--validate",
action="store_true",
help="Exit with code 1 if scaffold_ready is false (useful for CI gate checks)",
)
parser.add_argument(
"--context",
default=str(DEFAULT_CONTEXT_PATH),
help="Path to docs/hardware-support-context.md (default: auto-detected)",
)
args = parser.parse_args()
intake_path = Path(args.intake)
if not intake_path.exists():
print(f"Error: intake file not found: {intake_path}", file=sys.stderr)
sys.exit(2)
request = BoardIntakeRequest.from_json(intake_path)
context = load_hardware_context(Path(args.context))
assessment = assess_intake(request, context)
report = render_assessment_markdown(assessment)
if args.output == "-":
print(report)
else:
output_path = Path(args.output)
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(report, encoding="utf-8")
print(f"Assessment written to {output_path}")
if args.validate and not assessment.scaffold_ready:
sys.exit(1)
if __name__ == "__main__":
main()
+392
View File
@@ -0,0 +1,392 @@
#!/usr/bin/env python3
"""Scaffold board-support files from a validated intake assessment."""
from __future__ import annotations
import argparse
import re
import sys
from pathlib import Path
from board_intake import (
BoardIntakeRequest,
EvidenceGap,
IntakeAssessment,
assess_intake,
load_hardware_context,
render_assessment_markdown,
)
ROOT = Path(__file__).resolve().parents[1]
DEFAULT_OUTPUT_ROOT = ROOT / "generated" / "hardware-support"
ARCH_BASE_ENV = {
"esp32": "esp32_base",
"esp32-s3": "esp32s3_base",
"esp32-c3": "esp32c3_base",
"esp32-c6": "esp32c6_base",
"esp32s2": "esp32s2_base",
"nrf52840": "nrf52_base",
"rp2040": "rp2040_base",
"rp2350": "rp2350_base",
"stm32": "stm32_base",
"native": "native_base",
}
ARCH_VARIANT_ROOT = {
"esp32": "esp32",
"esp32-s3": "esp32s3",
"esp32-c3": "esp32c3",
"esp32-c6": "esp32c6",
"esp32s2": "esp32s2",
"nrf52840": "nrf52840",
"rp2040": "rp2040",
"rp2350": "rp2350",
"stm32": "stm32",
"native": "native",
}
PLACEHOLDER_DEFINES = {
"status": [
"#define LED_PIN // TODO: verify status LED pin if present",
],
"input": [
"#define BUTTON_PIN // TODO: verify user button pin",
],
"display": [
"#define HAS_SCREEN 1",
"#define USE_SSD1306 // TODO: verify display controller",
"#define I2C_SCL // TODO: verify display I2C clock pin",
"#define I2C_SDA // TODO: verify display I2C data pin",
],
"GPS": [
"#define GPS_RX_PIN // TODO: verify GPS RX pin or remove if GPS absent",
"#define GPS_TX_PIN // TODO: verify GPS TX pin or remove if GPS absent",
],
"power": [
"#define BATTERY_PIN // TODO: verify battery sense pin",
"#define ADC_MULTIPLIER // TODO: verify voltage divider ratio",
],
"radio": [
"#define USE_SX1262 // TODO: verify radio chip selection from schematic",
"#define SX126X_CS // TODO: verify radio chip-select pin",
"#define SX126X_BUSY // TODO: verify radio busy pin",
"#define SX126X_DIO1 // TODO: verify radio IRQ pin",
"#define SX126X_RESET // TODO: verify radio reset pin",
"#define LORA_SCK // TODO: verify radio SPI clock pin",
"#define LORA_MISO // TODO: verify radio SPI MISO pin",
"#define LORA_MOSI // TODO: verify radio SPI MOSI pin",
],
}
DEFINE_PATTERN = re.compile(r"^#define\s+([A-Za-z0-9_]+)(?:\s+(.*?))?\s*(?://.*)?$")
CATEGORY_MACROS = {
"status": ["LED_POWER", "LED_PIN", "LED_STATE_ON", "VEXT_ENABLE"],
"input": ["BUTTON_PIN", "BUTTON_NEED_PULLUP", "ROTARY_A", "ROTARY_B"],
"display": [
"HAS_SCREEN",
"USE_SSD1306",
"USE_SH1106",
"USE_TFTDISPLAY",
"USE_ST7789",
"I2C_SCL",
"I2C_SDA",
"I2C_SCL1",
"I2C_SDA1",
"TFT_HEIGHT",
"TFT_WIDTH",
"TFT_OFFSET_X",
"TFT_OFFSET_Y",
"SCREEN_ROTATE",
"SCREEN_TRANSITION_FRAMERATE",
],
"GPS": [
"HAS_GPS",
"GPS_RX_PIN",
"GPS_TX_PIN",
"GPS_BAUDRATE",
"PIN_GPS_PPS",
"GPS_THREAD_INTERVAL",
],
"power": [
"ADC_CTRL",
"ADC_CTRL_ENABLED",
"BATTERY_PIN",
"ADC_CHANNEL",
"ADC_ATTENUATION",
"ADC_MULTIPLIER",
"USE_POWERSAVE",
"SLEEP_TIME",
],
"radio": [
"USE_SX1262",
"USE_SX1268",
"USE_LR1121",
"USE_SX1280",
"LORA_DIO0",
"LORA_RESET",
"LORA_DIO1",
"LORA_DIO2",
"LORA_SCK",
"LORA_MISO",
"LORA_MOSI",
"LORA_CS",
"SX126X_CS",
"SX126X_DIO1",
"SX126X_BUSY",
"SX126X_RESET",
"SX126X_DIO2_AS_RF_SWITCH",
"SX126X_DIO3_TCXO_VOLTAGE",
"LR1121_IRQ_PIN",
"LR1121_NRESET_PIN",
"LR1121_BUSY_PIN",
"LR1121_SPI_NSS_PIN",
"LR1121_SPI_SCK_PIN",
"LR1121_SPI_MOSI_PIN",
"LR1121_SPI_MISO_PIN",
"LR11X0_DIO3_TCXO_VOLTAGE",
"LR11X0_DIO_AS_RF_SWITCH",
],
}
def slugify_variant_dir(req: BoardIntakeRequest) -> str:
return req.environment_name.replace("_", "-").lower()
def target_variant_dir(req: BoardIntakeRequest) -> Path:
arch_root = ARCH_VARIANT_ROOT.get(req.architecture, req.architecture)
return Path("variants") / arch_root / slugify_variant_dir(req)
def pick_pattern(assessment: IntakeAssessment) -> dict | None:
return assessment.matched_patterns[0] if assessment.matched_patterns else None
def source_basis_lines(assessment: IntakeAssessment, context: dict) -> list[str]:
lines = [
f"// Intake environment: {assessment.request.environment_name}",
f"// Intake architecture: {assessment.request.architecture}",
]
pattern = pick_pattern(assessment)
if pattern:
lines.append(
"// Repository pattern basis: "
f"{pattern.get('environment', '')} -> {pattern.get('variant_dir', '')}"
)
lines.append(
f"// Context source: {context.get('context_path', 'docs/hardware-support-context.md')}"
)
return lines
def parse_variant_defines(variant_path: Path) -> dict[str, str]:
defines: dict[str, str] = {}
if not variant_path.exists():
return defines
for raw_line in variant_path.read_text(encoding="utf-8").splitlines():
match = DEFINE_PATTERN.match(raw_line.strip())
if not match:
continue
name = match.group(1)
value = (match.group(2) or "1").strip()
defines[name] = value
return defines
def infer_category_lines(pattern_variant_path: Path, category: str) -> list[str]:
defines = parse_variant_defines(pattern_variant_path)
lines: list[str] = []
for macro_name in CATEGORY_MACROS[category]:
value = defines.get(macro_name)
if value is None:
continue
if value == "1":
lines.append(f"#define {macro_name}")
else:
lines.append(f"#define {macro_name} {value}")
if lines:
return lines
return PLACEHOLDER_DEFINES[category]
# T028
def generate_variant_h(assessment: IntakeAssessment, context: dict) -> str:
req = assessment.request
lines: list[str] = []
lines.extend(source_basis_lines(assessment, context))
lines.append("")
lines.append("#pragma once")
lines.append("")
pattern = pick_pattern(assessment)
pattern_variant_path = None
if pattern and pattern.get("variant_dir"):
pattern_variant_path = (ROOT / str(pattern["variant_dir"]) / "variant.h").resolve()
for category in ("status", "input", "display", "GPS", "power", "radio"):
lines.append(f"// {category}")
if pattern_variant_path is not None:
lines.extend(infer_category_lines(pattern_variant_path, category))
else:
lines.extend(PLACEHOLDER_DEFINES[category])
lines.append("")
lines.append("// Board identity")
macro_name = req.hardware_model_slug or req.environment_name.upper().replace(
"-", "_"
)
lines.append(f"#define {macro_name} 1")
lines.append("")
return "\n".join(lines).rstrip() + "\n"
# T029
def generate_platformio_env(assessment: IntakeAssessment) -> str:
req = assessment.request
extends = ARCH_BASE_ENV.get(req.architecture, f"{req.architecture}_base")
variant_dir = target_variant_dir(req)
build_define = (
req.hardware_model_slug or req.environment_name.upper().replace("-", "_")
).replace(" ", "_")
lines = [
f"[env:{req.environment_name}]",
f"custom_meshtastic_hw_model = {req.hardware_model}",
f"custom_meshtastic_hw_model_slug = {req.hardware_model_slug or 'TODO_SET_HW_MODEL_SLUG'}",
f"custom_meshtastic_architecture = {req.architecture}",
f"custom_meshtastic_actively_supported = {str(req.actively_supported).lower() if req.actively_supported is not None else 'TODO_SET_ACTIVELY_SUPPORTED'}",
f"custom_meshtastic_support_level = {req.support_level or 'TODO_SET_SUPPORT_LEVEL'}",
f"custom_meshtastic_display_name = {req.display_name}",
"custom_meshtastic_images = TODO_SET_IMAGES",
"custom_meshtastic_tags = TODO_SET_TAGS",
"custom_meshtastic_requires_dfu = TODO_SET_REQUIRES_DFU",
"custom_meshtastic_partition_scheme = TODO_SET_PARTITION_SCHEME",
"",
"board = TODO_SET_PLATFORMIO_BOARD",
f"extends = {extends}",
"build_flags =",
f" ${{{extends}.build_flags}}",
f" -D {build_define}",
f" -I {variant_dir.as_posix()}",
]
return "\n".join(lines).rstrip() + "\n"
# T030
def annotate_unresolved(content: str, gaps: list[EvidenceGap], is_ini: bool = False) -> str:
annotations = [
gap
for gap in gaps
if not gap.blocking
and gap.category in {"radio", "display", "input", "GPS", "power", "metadata"}
]
if not annotations:
return content
lines = content.splitlines()
todo_lines = [f"; TODO: verify — {gap.description}" for gap in annotations]
if is_ini:
# For INI files, insert TODOs after the first section header
if lines and lines[0].startswith("["):
return "\n".join(lines[:1] + todo_lines + [""] + lines[1:]) + "\n"
else:
# For .h files, prepend TODOs with C++ comment syntax
todo_lines = [f"// TODO: verify — {gap.description}" for gap in annotations]
return "\n".join(todo_lines + [""] + lines) + "\n"
return content
# T031
def scaffold_board(
assessment: IntakeAssessment, context: dict, output_dir: Path
) -> dict[str, Path]:
req = assessment.request
variant_dir = output_dir / target_variant_dir(req)
variant_dir.mkdir(parents=True, exist_ok=True)
variant_h_path = variant_dir / "variant.h"
platformio_path = variant_dir / "platformio.ini"
variant_h_content = annotate_unresolved(
generate_variant_h(assessment, context), assessment.evidence_gaps, is_ini=False
)
platformio_content = annotate_unresolved(
generate_platformio_env(assessment), assessment.evidence_gaps, is_ini=True
)
variant_h_path.write_text(variant_h_content, encoding="utf-8")
platformio_path.write_text(platformio_content, encoding="utf-8")
generated = {
"variant_h": variant_h_path,
"platformio": platformio_path,
}
if req.architecture in {"esp32", "esp32-s3", "esp32-c3", "esp32-c6"}:
variant_cpp_path = variant_dir / "variant.cpp"
variant_cpp_content = annotate_unresolved(
"\n".join(
[
'#include "variant.h"',
"",
"// Optional board-specific initialization hooks go here.",
"// Leave this file out if the board does not need custom startup behavior.",
]
)
+ "\n",
assessment.evidence_gaps,
)
variant_cpp_path.write_text(variant_cpp_content, encoding="utf-8")
generated["variant_cpp"] = variant_cpp_path
return generated
# T032 + T033
def main() -> None:
parser = argparse.ArgumentParser(
description="Generate board-support scaffold files from an intake JSON file."
)
parser.add_argument("intake", help="Path to intake JSON")
parser.add_argument(
"--output-dir",
default=str(DEFAULT_OUTPUT_ROOT),
help="Directory where scaffold files will be written",
)
args = parser.parse_args()
intake_path = Path(args.intake)
request = BoardIntakeRequest.from_json(intake_path)
context = load_hardware_context()
context["context_path"] = "docs/hardware-support-context.md"
assessment = assess_intake(request, context)
if not assessment.scaffold_ready:
print(render_assessment_markdown(assessment))
sys.exit(1)
generated = scaffold_board(assessment, context, Path(args.output_dir))
print("Generated scaffold files:")
for name, path in generated.items():
print(f"- {name}: {path}")
if __name__ == "__main__":
main()
+1 -1
View File
@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
cp bin/device-update.* $OUTDIR/
echo "Copying manifest"
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true
@@ -1,30 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins: [DIO5, DIO6, DIO7]
MODE_STBY: [LOW, LOW, LOW]
MODE_RX: [LOW, HIGH, LOW]
MODE_TX: [HIGH, HIGH, LOW]
MODE_TX_HP: [HIGH, LOW, LOW]
MODE_TX_HF: [LOW, LOW, LOW]
MODE_GNSS: [LOW, LOW, HIGH]
MODE_WIFI: [LOW, LOW, LOW]
General:
MACAddressSource: eth0
@@ -1,46 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins:
- DIO5
- DIO6
MODE_STBY:
- LOW
- LOW
MODE_RX:
- HIGH
- LOW
MODE_TX:
- HIGH
- HIGH
MODE_TX_HP:
- LOW
- HIGH
MODE_TX_HF:
- LOW
- LOW
MODE_GNSS:
- LOW
- LOW
MODE_WIFI:
- LOW
- LOW
General:
MACAddressSource: eth0
@@ -1,30 +0,0 @@
---
Lora:
## Ebyte E80-900M22S
## This is a bit experimental
##
##
Module: lr1121
gpiochip: 1 # subtract 32 from the gpio numbers
DIO3_TCXO_VOLTAGE: 1.8
CS: 16 #pin6 / GPIO48 1C0
IRQ: 23 #pin17 / GPIO55 1C7
Busy: 22 #pin16 / GPIO54 1C6
Reset: 25 #pin13 / GPIO57 1D1
spidev: spidev0.0 #pins are (CS=16, CLK=17, MOSI=18, MISO=19)
spiSpeed: 2000000
rfswitch_table:
pins: [DIO5, DIO6, DIO7]
MODE_STBY: [LOW, LOW, LOW]
MODE_RX: [LOW, LOW, LOW]
MODE_TX: [LOW, HIGH, LOW]
MODE_TX_HP: [HIGH, LOW, LOW]
# MODE_TX_HF: []
# MODE_GNSS: []
MODE_WIFI: [LOW, LOW, LOW]
General:
MACAddressSource: eth0
-18
View File
@@ -1,18 +0,0 @@
# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
# Verify spidev / I2C device paths for your OS — they may differ.
Meta:
name: Station G3
support: community
compatible:
- raspberry-pi
Lora:
Module: sx1262
IRQ: 22 # BCM pin — wiki spec
Reset: 16 # BCM pin — wiki spec
Busy: 24 # BCM pin — wiki spec
# CS: 8 # BCM 8 = SPI0 CE0 (default); uncomment only to override
DIO2_AS_RF_SWITCH: true
DIO3_TCXO_VOLTAGE: true
spidev: spidev0.0
# SX126X_MAX_POWER: 19 # matches Station G2 firmware cap; raise carefully per PA jumper mode
+14
View File
@@ -0,0 +1,14 @@
{
"environment_name": "new_esp32s3_board",
"hardware_model": "999",
"hardware_model_slug": "NEW_ESP32S3_BOARD",
"display_name": "Acme ESP32-S3 Dev Board v1",
"architecture": "esp32-s3",
"actively_supported": true,
"support_level": "1",
"source_materials": [
"https://example.com/acme-esp32s3-schematic.pdf",
"https://example.com/acme-esp32s3-pinout.png"
],
"board_notes": "Initial bring-up for v1 PCB only. SX1262 radio on SPI2, OLED on I2C bus."
}
+6
View File
@@ -0,0 +1,6 @@
{
"environment_name": "hypothetical_esp32s3_board",
"hardware_model": "9981",
"display_name": "Hypothetical ESP32-S3 Board",
"architecture": "esp32-s3"
}
+13
View File
@@ -0,0 +1,13 @@
{
"environment_name": "new_nrf52_tracker",
"hardware_model": "998",
"hardware_model_slug": "NEW_NRF52_TRACKER",
"display_name": "Acme nRF52840 Tracker v2",
"architecture": "nrf52840",
"actively_supported": null,
"support_level": "2",
"source_materials": [
"https://example.com/acme-tracker-pinout.pdf"
],
"board_notes": "Two revisions exist: v2 (SSD1306 OLED) and v2a (no display). This intake covers v2 only."
}
-439
View File
@@ -1,439 +0,0 @@
#!/usr/bin/env python3
"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
The output is hand-editable and committed; a sibling compile step
(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
v25 protobuf with fresh "now-relative" timestamps.
Determinism contract:
Same --seed -> byte-identical JSONL output, regardless of wall clock.
All timestamps are stored as `*_offset_sec` (seconds before "now"); the
compile step resolves them to absolute epochs at compile time.
Structural fields covered:
* NodeInfoLite header: num, long_name, short_name, hw_model, role,
public_key, snr, channel, hops_away, next_hop, bitfield flags
* PositionLite: lat/long Gaussian around --centroid, altitude, source
* DeviceMetrics: battery/voltage/util/uptime
* EnvironmentMetrics: temp/humidity/pressure/iaq
* StatusMessage: error_code (usually zero)
Active-board allow-list:
hw_model values are restricted to the intersection of
(a) variants with `custom_meshtastic_support_level = 1` in
variants/*/*/platformio.ini, AND
(b) values present in the `HardwareModel` enum in mesh.proto.
See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
(PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
Active-role allow-list:
Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
"""
from __future__ import annotations
import argparse
import datetime as _dt
import json
import math
import pathlib
import random
import sys
# --------------------------------------------------------------------------
# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
# Refresh by running:
# for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
# grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
# done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
# --------------------------------------------------------------------------
HW_MODEL_WEIGHTS: dict[str, float] = {
"HELTEC_V3": 14.0,
"T_DECK": 9.0,
"HELTEC_V4": 8.0,
"RAK4631": 8.0,
"HELTEC_MESH_POCKET": 6.0,
"TRACKER_T1000_E": 5.0,
"HELTEC_MESH_NODE_T114": 5.0,
"T_DECK_PRO": 5.0,
"LILYGO_TBEAM_S3_CORE": 4.0,
"HELTEC_WIRELESS_PAPER": 4.0,
"HELTEC_WSL_V3": 3.0,
"T_ECHO": 3.0,
"HELTEC_WIRELESS_TRACKER": 3.0,
"HELTEC_WIRELESS_TRACKER_V2": 2.0,
"HELTEC_VISION_MASTER_E290": 2.0,
"HELTEC_MESH_SOLAR": 2.0,
"SEEED_WIO_TRACKER_L1": 2.0,
"T_LORA_PAGER": 1.5,
"HELTEC_VISION_MASTER_E213": 1.5,
"T_ECHO_PLUS": 1.0,
"MUZI_BASE": 1.0,
"WISMESH_TAP_V2": 1.0,
"THINKNODE_M2": 1.0,
"THINKNODE_M5": 1.0,
"TLORA_T3_S3": 1.0,
# Long tail (uniform low weight across remaining tier-1 boards):
"HELTEC_V4_R8": 0.3,
"HELTEC_VISION_MASTER_T190": 0.3,
"HELTEC_HT62": 0.3,
"HELTEC_MESH_NODE_T096": 0.3,
"M5STACK_C6L": 0.3,
"MINI_EPAPER_S3": 0.3,
"MUZI_R1_NEO": 0.3,
"NOMADSTAR_METEOR_PRO": 0.3,
"RAK3312": 0.3,
"RAK3401": 0.3,
"SEEED_SOLAR_NODE": 0.3,
"SEEED_WIO_TRACKER_L1_EINK": 0.3,
"SENSECAP_INDICATOR": 0.3,
"TBEAM_1_WATT": 0.3,
"THINKNODE_M1": 0.3,
"THINKNODE_M3": 0.3,
"THINKNODE_M6": 0.3,
"T_ECHO_LITE": 0.3,
"WISMESH_TAG": 0.3,
"WISMESH_TAP": 0.3,
"XIAO_NRF52_KIT": 0.3,
"CROWPANEL": 0.3,
}
# Non-deprecated roles only.
ROLE_WEIGHTS: dict[str, float] = {
"CLIENT": 75.0,
"CLIENT_MUTE": 5.0,
"ROUTER": 7.0,
"TRACKER": 3.0,
"SENSOR": 2.0,
"CLIENT_HIDDEN": 2.0,
"ROUTER_LATE": 2.0,
"CLIENT_BASE": 2.0,
"TAK": 1.0,
"TAK_TRACKER": 0.5,
"LOST_AND_FOUND": 0.5,
}
# Name pools — 60 firsts × 60 lasts = 3600 combinations.
FIRSTS = [
"Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
"Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
"Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
"Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
"Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
"Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
"Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
"Burning", "Frozen", "Whispering", "Howling",
]
LASTS = [
"Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
"Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
"Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
"Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
"Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
"Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
"Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
"Bluff", "Mesa", "Arroyo", "Ridge",
]
# Brief callsign pool for licensed-looking suffixes.
CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
# 6 bytes due to U+FE0F variation selector — explicitly excluded.
EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
"🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
"🦇", "🦋"]
# --------------------------------------------------------------------------
# Helpers
# --------------------------------------------------------------------------
NUM_RESERVED = 4 # firmware reserves 0..3 (per NodeDB constants)
NUM_MAX_EXCLUSIVE = 0x80000000 # restrict to positive int32 range for readability
def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
"""Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
keys = sorted(weights.keys())
totals = [weights[k] for k in keys]
return rng.choices(keys, weights=totals, k=1)[0]
def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
if is_licensed:
prefix = rng.choice(CALLSIGN_PREFIXES)
# Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
# nanopb max_size:25 means C string fits 24 bytes + NUL.
if len(base) + len(suffix) <= 24:
base = base + suffix
# Hard cap to 24 chars (nanopb max_size:25 minus NUL).
return base[:24]
def _gen_short_name(rng: random.Random, long_name: str) -> str:
# 10% emoji-only short_name
if rng.random() < 0.10:
return rng.choice(EMOJI_SHORTNAMES)
first_char = long_name[0].upper() if long_name else "X"
alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
return first_char + "".join(rng.choices(alphanums, k=3))
def _gen_hops_away(rng: random.Random) -> int:
# Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
r = rng.random()
if r < 0.55:
return 0
if r < 0.80:
return 1
if r < 0.92:
return 2
if r < 0.97:
return 3
if r < 0.99:
return 4
return rng.randint(5, 7)
def _gen_position(
rng: random.Random,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
last_heard_offset_sec: int,
) -> dict:
# 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
altitude = max(0, round(rng.gauss(1376.0, 250.0))) # T or C valley floor + relief
# Position was reported up to 300s before last_heard.
time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
return {
"latitude": round(lat, 6),
"longitude": round(lon, 6),
"altitude": altitude,
"time_offset_sec": time_offset_sec,
"location_source": "LOC_INTERNAL",
}
def _gen_telemetry(rng: random.Random) -> dict:
# 5% plugged-in (battery_level == 101); rest uniform [10..100].
if rng.random() < 0.05:
battery_level = 101
voltage = 4.20
else:
battery_level = rng.randint(10, 100)
voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
# Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
def _beta(a: float, b: float) -> float:
x = rng.gammavariate(a, 1.0)
y = rng.gammavariate(b, 1.0)
return x / (x + y)
channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
return {
"battery_level": battery_level,
"voltage": voltage,
"channel_utilization": channel_utilization,
"air_util_tx": air_util_tx,
"uptime_seconds": uptime_seconds,
}
def _gen_environment(rng: random.Random) -> dict:
return {
"temperature": round(rng.gauss(22.0, 8.0), 2),
"relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
"barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
"iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
}
def _gen_status(rng: random.Random) -> dict:
# `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
# Most peers report a healthy short status; occasional alert string.
healthy = ["OK", "online", "active", "running", "ready", "nominal"]
alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
if rng.random() < 0.92:
return {"status": rng.choice(healthy)}
return {"status": rng.choice(alert)}
def _gen_node(
rng: random.Random,
num: int,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
coverage: dict[str, float],
last_heard_mean_sec: int,
last_heard_max_sec: int,
) -> dict:
is_licensed = rng.random() < 0.05
long_name = _gen_long_name(rng, is_licensed)
short_name = _gen_short_name(rng, long_name)
hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
role = _weighted_choice(rng, ROLE_WEIGHTS)
has_public_key = rng.random() < 0.92
public_key_hex = (
"".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
)
snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
hops_away = _gen_hops_away(rng)
next_hop = rng.randint(0, 255) if hops_away > 0 else 0
last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
bitfield = {
"has_user": True,
"is_favorite": rng.random() < 0.08,
"is_muted": rng.random() < 0.03,
"via_mqtt": rng.random() < 0.12,
"is_ignored": rng.random() < 0.01,
"is_licensed": is_licensed,
"has_is_unmessagable": True,
"is_unmessagable": rng.random() < 0.02,
"is_key_manually_verified": rng.random() < 0.04,
}
node: dict = {
"num": f"0x{num:08x}",
"long_name": long_name,
"short_name": short_name,
"hw_model": hw_model,
"role": role,
"public_key_hex": public_key_hex,
"snr": snr,
"channel": channel,
"hops_away": hops_away,
"next_hop": next_hop,
"last_heard_offset_sec": last_heard_offset_sec,
"bitfield": bitfield,
"position": (
_gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
if rng.random() < coverage["position"]
else None
),
"telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
"environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
"status": _gen_status(rng) if rng.random() < coverage["status"] else None,
}
return node
def _parse_my_node_num(s: str | None) -> int | None:
if s is None:
return None
s = s.strip()
if s.startswith("0x") or s.startswith("0X"):
return int(s, 16)
return int(s)
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Deterministic JSONL seed for the fake NodeDB fixture.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
p.add_argument("--out", required=True, help="Output JSONL path.")
p.add_argument(
"--centroid",
default="33.1284,-107.2528",
help="LAT,LON centroid (default: Truth or Consequences, NM).",
)
p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
p.add_argument("--position-coverage", type=float, default=0.85)
p.add_argument("--telemetry-coverage", type=float, default=0.70)
p.add_argument("--environment-coverage", type=float, default=0.25)
p.add_argument("--status-coverage", type=float, default=0.40)
p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
p.add_argument("--last-heard-mean-sec", type=int, default=3600)
p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
args = p.parse_args(argv)
if args.count <= 0:
print("--count must be positive", file=sys.stderr)
return 2
try:
centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
except ValueError:
print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
return 2
my_node_num = _parse_my_node_num(args.my_node_num)
rng = random.Random(args.seed)
# 1) Generate a unique deterministic set of NodeNums.
nums: set[int] = set()
while len(nums) < args.count:
n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
if my_node_num is not None and n == my_node_num:
continue
nums.add(n)
ordered_nums = sorted(nums) # sort to fix output order independent of set hash
# 2) Per-node generation (in num order, single RNG continues).
coverage = {
"position": args.position_coverage,
"telemetry": args.telemetry_coverage,
"environment": args.environment_coverage,
"status": args.status_coverage,
}
nodes = [
_gen_node(
rng,
n,
centroid_lat,
centroid_lon,
args.spread_km,
coverage,
args.last_heard_mean_sec,
args.last_heard_max_sec,
)
for n in ordered_nums
]
# 3) Write JSONL.
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
# `generated_at_iso` is informational; it does NOT affect determinism because
# we derive it from the seed, not from wall clock. (Same seed -> same string.)
generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
meta = {
"_meta": {
"version": 25,
"seed": args.seed,
"count": args.count,
"centroid": [centroid_lat, centroid_lon],
"spread_km": args.spread_km,
"generated_at_iso": generated_at,
"my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
"coverage": coverage,
"last_heard_mean_sec": args.last_heard_mean_sec,
"last_heard_max_sec": args.last_heard_max_sec,
}
}
with out_path.open("w", encoding="utf-8") as f:
# `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
# determinism (insertion order varies by Python version otherwise).
f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
for node in nodes:
f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+592
View File
@@ -0,0 +1,592 @@
#!/usr/bin/env python3
"""Generate a markdown inventory of board-support metadata and common target-definition macros."""
from __future__ import annotations
import argparse
import re
from collections import Counter, defaultdict
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
VARIANTS_DIR = ROOT / "variants"
OUTPUT_PATH = ROOT / "docs" / "hardware-support-context.md"
PLATFORMIO_METADATA_KEYS = [
"custom_meshtastic_hw_model",
"custom_meshtastic_hw_model_slug",
"custom_meshtastic_architecture",
"custom_meshtastic_actively_supported",
"custom_meshtastic_support_level",
"custom_meshtastic_display_name",
"custom_meshtastic_images",
"custom_meshtastic_tags",
"custom_meshtastic_requires_dfu",
"custom_meshtastic_partition_scheme",
]
MACRO_PATTERNS = {
"Input": [
"BUTTON_PIN",
"BUTTON_PIN_TOUCH",
"ALT_BUTTON_PIN",
"CANCEL_BUTTON_PIN",
"ROTARY_",
"KB_",
"INPUTDRIVER_",
],
"Radio": [
"USE_RF95",
"USE_SX126",
"USE_SX128",
"USE_LLCC68",
"USE_LR11",
"LORA_",
"SX126X_",
"SX128X_",
"LR1121_",
"RF95_",
],
"GPS": [
"HAS_GPS",
"GPS_",
"PIN_GPS_",
"GPS_DEFAULT_NOT_PRESENT",
],
"Display": [
"HAS_SCREEN",
"USE_TFTDISPLAY",
"USE_TFT",
"USE_SSD",
"USE_SH",
"USE_ST",
"TFT_",
"OLED_",
"SCREEN_",
"PIN_EINK_",
"EINK_",
"DISPLAY_",
"LGFX_",
"HAS_TFT",
],
"I2C/SPI": [
"I2C_",
"SPI_",
"PIN_SPI",
"WIRE_",
],
"Power": [
"BATTERY_",
"ADC_",
"PIN_POWER",
"USE_POWERSAVE",
"SLEEP_TIME",
"XPOWERS_",
"HAS_AXP",
"HAS_PPM",
"HAS_BQ",
"EXT_NOTIFY_OUT",
"FAN_CTRL_PIN",
"RF95_FAN_EN",
],
"Connectivity/Other": [
"HAS_WIFI",
"HAS_BLUETOOTH",
"HAS_ETHERNET",
"HAS_NEOPIXEL",
"HAS_I2S",
"HAS_TOUCH",
"HAS_TOUCHSCREEN",
"HAS_NFC",
"NFC_",
"USE_XL9555",
"EXPANDS_",
"PCF",
"RTC",
],
}
def classify_macro(name: str) -> str:
for category, prefixes in MACRO_PATTERNS.items():
for prefix in prefixes:
if name.startswith(prefix):
return category
return "Other"
def parse_platformio_envs(path: Path) -> list[dict[str, object]]:
envs: list[dict[str, object]] = []
current: dict[str, object] | None = None
for raw_line in path.read_text(encoding="utf-8").splitlines():
line = raw_line.strip()
if not line or line.startswith(";"):
continue
if line.startswith("[") and line.endswith("]"):
if current:
envs.append(current)
current = {"section": line[1:-1], "metadata": {}}
continue
if current is None or "=" not in line:
continue
key, value = [part.strip() for part in line.split("=", 1)]
if key in PLATFORMIO_METADATA_KEYS:
current["metadata"][key] = value
if current:
envs.append(current)
return envs
def parse_variant_macros(path: Path) -> tuple[dict[str, str], Counter[str]]:
defines: dict[str, str] = {}
category_counts: Counter[str] = Counter()
pattern = re.compile(r"^#define\s+([A-Za-z0-9_]+)(?:\s+(.*?))?\s*(?://.*)?$")
for raw_line in path.read_text(encoding="utf-8").splitlines():
match = pattern.match(raw_line.strip())
if not match:
continue
name = match.group(1)
value = (match.group(2) or "1").strip()
defines[name] = value
category_counts[classify_macro(name)] += 1
return defines, category_counts
MACRO_DESCRIPTIONS: dict[str, str] = {
# ---------- Input ----------
"BUTTON_PIN": "Primary user button GPIO pin number.",
"ALT_BUTTON_PIN": "Secondary / alternate button pin, used on boards with two buttons.",
"CANCEL_BUTTON_PIN": "Button wired to cancel/back actions (e.g., T-Deck cancel key).",
"BUTTON_NEED_PULLUP": "Set to 1 when the button GPIO requires the internal pull-up to be enabled.",
"BUTTON_PIN_ALT": "Alternative spelling used by some older board families for a second button.",
"KB_BL_PIN": "Keyboard backlight control pin (e.g., T-LoRa Pager keyboard).",
"KB_INT": "Keyboard interrupt input pin; signals a keypress to the MCU.",
"KB_POWERON": "Pin used to power-on or enable the keyboard peripheral.",
"KB_SLAVE_ADDRESS": "I2C address of the keyboard controller IC.",
"INPUTDRIVER_ENCODER_TYPE": "Selects the rotary encoder driver variant (0 = none, 1+ = specific type).",
"INPUTDRIVER_TWO_WAY_ROCKER": "Enables the two-way rocker input driver.",
"INPUTDRIVER_TWO_WAY_ROCKER_RIGHT": "GPIO pin for the rightward rocker direction.",
"INPUTDRIVER_TWO_WAY_ROCKER_LEFT": "GPIO pin for the leftward rocker direction.",
"INPUTDRIVER_TWO_WAY_ROCKER_BTN": "GPIO pin for the rocker click/press direction.",
"ROTARY_A": "First encoder channel pin (clock / A signal).",
"ROTARY_B": "Second encoder channel pin (data / B signal).",
# ---------- Radio ----------
"USE_SX1262": "Select the SX1262 sub-GHz LoRa chip driver.",
"USE_SX1268": "Select the SX1268 sub-GHz LoRa chip driver (higher power variant of SX1262).",
"USE_SX1280": "Select the SX1280 2.4 GHz LoRa chip driver.",
"USE_RF95": "Select the RFM95/SX1276 legacy LoRa chip driver.",
"USE_LLCC68": "Select the LLCC68 low-cost LoRa chip driver.",
"USE_LR1110": "Select the LR1110 wideband radio driver.",
"USE_LR1120": "Select the LR1120 wideband radio driver.",
"USE_LR1121": "Select the LR1121 wideband radio driver.",
"LORA_CS": "LoRa radio SPI chip-select GPIO pin.",
"LORA_RESET": "LoRa radio hardware-reset GPIO pin (active low).",
"LORA_DIO0": "LoRa radio DIO0 interrupt pin (RF95/SX1276 done/RxDone).",
"LORA_DIO1": "LoRa radio DIO1 interrupt pin; on SX126x this is the primary IRQ line.",
"LORA_DIO2": "LoRa radio DIO2 pin; on RF95 used for FSK interrupt; on SX126x tx/rx control.",
"LORA_SCK": "LoRa radio SPI clock pin.",
"LORA_MISO": "LoRa radio SPI MISO pin.",
"LORA_MOSI": "LoRa radio SPI MOSI pin.",
"SX126X_CS": "SX126x chip-select pin (often same as LORA_CS).",
"SX126X_RESET": "SX126x reset pin (often same as LORA_RESET).",
"SX126X_BUSY": "SX126x BUSY pin; must be polled low before issuing SPI commands.",
"SX126X_DIO1": "SX126x DIO1 interrupt output used for all IRQ events.",
"SX126X_DIO3_TCXO_VOLTAGE": "Drives the TCXO regulator via DIO3; set to the supply voltage (e.g., 1.8).",
"SX126X_DIO2_AS_RF_SWITCH": "Set to 1 so the driver controls the TX/RX RF switch via DIO2.",
"LR1121_IRQ_PIN": "LR1121 interrupt request pin.",
"RF95_FAN_EN": "Enables a cooling fan for high-power RF95 installations.",
# ---------- GPS ----------
"HAS_GPS": "Set to 1 if the board has an on-board GPS receiver; 0 to disable GPS entirely.",
"GPS_RX_PIN": "UART RX pin connected to the GPS module TX output.",
"GPS_TX_PIN": "UART TX pin connected to the GPS module RX input.",
"PIN_GPS_EN": "GPIO to power-enable or power-gate the GPS module.",
"PIN_GPS_PPS": "GPS pulse-per-second input pin for timing synchronisation.",
"PIN_GPS_STANDBY": "Places the GPS into standby/low-power mode when driven.",
"PIN_GPS_RESET": "Hardware-reset line to the GPS module.",
"PIN_GPS_REINIT": "Pin used to trigger GPS re-initialisation sequences.",
"GPS_THREAD_INTERVAL": "Millisecond poll interval for the GPS background thread.",
"GPS_BAUDRATE": "UART baud rate for GPS serial communication.",
"GPS_L76K": "Selects the Quectel L76K GPS driver and protocol.",
"GPS_UBLOX": "Selects the u-blox GPS driver and UBX protocol.",
"GPS_EN_ACTIVE": "Logic level (HIGH or LOW) that enables the GPS power pin.",
"GPS_RESET_MODE": "Defines the reset signal polarity or protocol for the GPS chip.",
"GPS_DEFAULT_NOT_PRESENT": "Compiled-in default assuming no GPS; overridden at runtime if detected.",
# ---------- Display ----------
"HAS_SCREEN": "Set to 1 if the board has any display hardware.",
"USE_TFTDISPLAY": "Selects the TFT LCD driver path.",
"USE_SSD1306": "Selects the SSD1306 128×64 OLED driver over I2C.",
"USE_SH1106": "Selects the SH1106 128×64 OLED driver.",
"USE_ST7735": "Selects the ST7735 TFT SPI driver.",
"USE_ST7789": "Selects the ST7789 TFT SPI driver.",
"TFT_WIDTH": "Horizontal pixel resolution of the TFT display.",
"TFT_HEIGHT": "Vertical pixel resolution of the TFT display.",
"TFT_OFFSET_X": "Horizontal pixel offset for display alignment correction.",
"TFT_OFFSET_Y": "Vertical pixel offset for display alignment correction.",
"TFT_BL": "Backlight control PWM or GPIO pin for the TFT panel.",
"SCREEN_ROTATE": "Non-zero value rotates the display 180° for mounted-upside-down screens.",
"SCREEN_TRANSITION_FRAMERATE": "Target framerate for UI animations and transitions.",
"PIN_EINK_CS": "E-ink display SPI chip-select pin.",
"PIN_EINK_BUSY": "E-ink display BUSY output; high when a page update is in progress.",
"PIN_EINK_DC": "E-ink display data/command select pin.",
"PIN_EINK_RES": "E-ink display hardware-reset pin.",
"PIN_EINK_SCLK": "E-ink display SPI clock pin.",
"PIN_EINK_MOSI": "E-ink display SPI MOSI pin.",
# ---------- I2C/SPI ----------
"I2C_SDA": "Primary I2C data line GPIO pin.",
"I2C_SCL": "Primary I2C clock line GPIO pin.",
"PIN_SPI_MISO": "Primary SPI MISO (data from peripheral) GPIO pin.",
"PIN_SPI_MOSI": "Primary SPI MOSI (data to peripheral) GPIO pin.",
"PIN_SPI_SCK": "Primary SPI clock GPIO pin.",
"SPI_INTERFACES_COUNT": "Number of hardware SPI buses available on this board.",
"WIRE_INTERFACES_COUNT": "Number of hardware I2C buses available on this board.",
"PIN_SPI1_MISO": "Secondary SPI bus MISO pin.",
"PIN_SPI1_MOSI": "Secondary SPI bus MOSI pin.",
"PIN_SPI1_SCK": "Secondary SPI bus clock pin.",
"SPI_FREQUENCY": "Default SPI clock frequency in Hz for this board.",
"SPI_READ_FREQUENCY": "Reduced SPI clock rate used for read transactions.",
"SPI_SCK": "SPI clock pin alias used in older board files.",
"SPI_MOSI": "SPI MOSI pin alias used in older board files.",
"SPI_MISO": "SPI MISO pin alias used in older board files.",
# ---------- Power ----------
"BATTERY_PIN": "ADC input GPIO connected to the battery voltage divider.",
"ADC_MULTIPLIER": "Floating-point scale factor to convert raw ADC reading to battery voltage.",
"ADC_CHANNEL": "ADC channel enum or number for the battery sense input.",
"BATTERY_SENSE_RESOLUTION_BITS": "ADC resolution in bits used for battery voltage sampling.",
"ADC_RESOLUTION": "Board-level ADC resolution definition, referenced by other power macros.",
"BATTERY_SENSE_RESOLUTION": "Alias for the effective ADC resolution for battery sense.",
"ADC_CTRL": "GPIO that enables or gates the ADC voltage-divider circuit.",
"ADC_CTRL_ENABLED": "Logic level (HIGH or LOW) that turns on the ADC control switch.",
"ADC_ATTENUATION": "ESP32 ADC input attenuation setting; controls measurable voltage range.",
"BATTERY_SENSE_SAMPLES": "Number of ADC samples to average for a stable battery reading.",
"EXT_NOTIFY_OUT": "GPIO output used to signal an external LED or buzzer for notifications.",
"USE_POWERSAVE": "Enables aggressive power-save mode (deep sleep, reduced poll intervals).",
"SLEEP_TIME": "Default light-sleep duration in milliseconds between wakeups.",
"PIN_POWER_EN": "GPIO to assert to enable a board power rail or load switch.",
"HAS_PPM": "Set to 1 if the board has an IP5306 or similar PPM power path IC.",
# ---------- Connectivity/Other ----------
"HAS_TOUCHSCREEN": "Set to 1 for boards with a capacitive or resistive touch panel.",
"HAS_NEOPIXEL": "Set to 1 if the board has addressable RGB LEDs (WS2812 / NeoPixel).",
"PCF8563_RTC": "I2C address of the PCF8563 real-time clock IC.",
"PCF85063_RTC": "I2C address of the PCF85063 real-time clock IC.",
"HAS_ETHERNET": "Set to 1 for boards with a wired Ethernet interface.",
"HAS_I2S": "Set to 1 if I2S audio output is present.",
"NFC_INT": "Interrupt pin from the NFC controller IC.",
"NFC_CS": "SPI chip-select for the NFC controller.",
"USE_XL9555": "Enables the XL9555 16-bit I2C GPIO expander driver.",
"EXPANDS_DRV_EN": "GPIO expander pin used to enable the haptic driver.",
"EXPANDS_AMP_EN": "GPIO expander pin used to power-on the audio amplifier.",
"EXPANDS_KB_RST": "GPIO expander pin used to reset the keyboard controller.",
"EXPANDS_LORA_EN": "GPIO expander pin used to power-gate the LoRa radio.",
"EXPANDS_GPS_EN": "GPIO expander pin used to power-gate the GPS module.",
"EXPANDS_NFC_EN": "GPIO expander pin used to power-gate the NFC controller.",
# ---------- Other ----------
"LED_POWER": "GPIO for the status LED, defines the LED pin number.",
"LED_STATE_ON": "Logic level (HIGH or LOW) that turns the status LED on.",
"PIN_SERIAL1_RX": "Secondary UART RX pin (used for accessories, GPS on some boards).",
"PIN_SERIAL1_TX": "Secondary UART TX pin.",
"PIN_WIRE_SDA": "Arduino-framework I2C SDA pin alias (nRF52 / RP2040 style).",
"PIN_WIRE_SCL": "Arduino-framework I2C SCL pin alias.",
"PIN_LED1": "First LED GPIO pin in the nRF52 Arduino BSP pin table.",
"VARIANT_MCK": "Crystal oscillator frequency in Hz for nRF52 variant clock configuration.",
"PINS_COUNT": "Total number of GPIO pins defined in the Arduino BSP variant table.",
"NUM_DIGITAL_PINS": "Count of digital-capable pins in the BSP variant table.",
"NUM_ANALOG_INPUTS": "Count of analog-input pins in the BSP variant table.",
"NUM_ANALOG_OUTPUTS": "Count of analog-output (DAC) pins in the BSP variant table.",
"LED_BLUE": "GPIO number of the blue status LED (typical on nRF52 and RP2040 boards).",
"USE_LFXO": "Instructs the nRF52 BSP to use the low-frequency crystal oscillator.",
"BUTTON_NEED_PULLUP": "Enables internal pull-up on the button GPIO (duplicate entry for clarity).",
}
def shorten(value: str, limit: int = 60) -> str:
compact = " ".join(value.split())
return compact if len(compact) <= limit else compact[: limit - 3] + "..."
# Architecture families known to rely heavily on BSP or base-environment defaults rather
# than declaring every field locally. Used in the Inherited Defaults Note section.
_BSP_DEFAULT_FAMILIES: dict[str, list[str]] = {
"nrf52840": [
"VARIANT_MCK — nRF52 BSP clock constant (e.g., 64000000ul); always inherited from BSP unless overridden.",
"USE_LFXO — low-frequency crystal oscillator selection; declared locally only when the board uses LFXO rather than the RC oscillator.",
"PIN_SPI_* / PIN_SPI1_* — SPI bus pin numbers come from the BSP variant table; boards override only when the LoRa radio or display uses non-default SPI routing.",
"WIRE_INTERFACES_COUNT / SPI_INTERFACES_COUNT — bus count comes from BSP; explicitly set only when the board deviates.",
"LED_BLUE / PIN_LED1 / PINS_COUNT / NUM_DIGITAL_PINS — standard BSP pin-table entries inherited from the nRF52 Arduino core.",
],
"rp2040": [
"PIN_SPI_* — primary SPI pins come from the RP2040 Arduino BSP; most boards declare them explicitly, but the defaults align with the Pico pin assignments.",
"NUM_DIGITAL_PINS / NUM_ANALOG_INPUTS — Arduino BSP counts; rarely overridden locally.",
],
"stm32": [
"Radio and pin assignments for STM32WL targets are largely internal to the WL SoC and declared via STM32 HAL/BSP headers; variant.h files are minimal.",
"USE_STM32WLx is typically the only explicit define; all other radio config comes from the BSP.",
],
"native": [
"The native/Portduino target uses runtime configuration rather than compile-time pin defines; variant.h only sets display and GPS stubs.",
],
}
def collect_inventory() -> dict[str, object]:
architectures: defaultdict[str, list[dict[str, object]]] = defaultdict(list)
category_frequency: defaultdict[str, Counter[str]] = defaultdict(Counter)
total_variant_dirs = 0
total_envs = 0
no_variant_h: defaultdict[str, int] = defaultdict(int) # arch -> count of dirs with no variant.h
has_metadata: int = 0 # env count that has at least one custom_meshtastic_* key
for platformio_path in sorted(VARIANTS_DIR.glob("**/platformio.ini")):
variant_dir = platformio_path.parent
total_variant_dirs += 1
variant_path = variant_dir / "variant.h"
board_level_arch_raw = variant_dir.parts[-2] if len(variant_dir.parts) >= 2 else "unknown"
if not variant_path.exists():
no_variant_h[board_level_arch_raw] += 1
continue
envs = parse_platformio_envs(platformio_path)
defines, category_counts = parse_variant_macros(variant_path)
total_envs += len(envs)
board_level_arch = board_level_arch_raw
for env in envs:
section = str(env["section"])
if not section.startswith("env:"):
continue
metadata = dict(env["metadata"])
if metadata:
has_metadata += 1
architecture = str(metadata.get("custom_meshtastic_architecture") or board_level_arch)
if architecture == "esp32s3":
architecture = "esp32-s3"
if architecture == "esp32c3":
architecture = "esp32-c3"
if architecture == "esp32c6":
architecture = "esp32-c6"
entry = {
"environment": section.split(":", 1)[1],
"variant_dir": str(variant_dir.relative_to(ROOT)),
"metadata": metadata,
"defines": defines,
"category_counts": category_counts,
}
architectures[architecture].append(entry)
for name in defines:
category = classify_macro(name)
category_frequency[category][name] += 1
return {
"architectures": architectures,
"category_frequency": category_frequency,
"total_variant_dirs": total_variant_dirs,
"total_envs": total_envs,
"no_variant_h": dict(no_variant_h),
"has_metadata": has_metadata,
}
def render_markdown(inventory: dict[str, object]) -> str:
architectures = inventory["architectures"]
category_frequency = inventory["category_frequency"]
lines: list[str] = []
lines.append("# Hardware Support Context")
lines.append("")
lines.append("This document inventories the board-support inputs and common target-definition fields")
lines.append("currently used in the Meshtastic firmware repository. It is intended as reusable context")
lines.append("for adding new hardware support without re-discovering naming patterns, metadata keys, and")
lines.append("frequently used pin or capability macros from scratch.")
lines.append("")
lines.append("## Scope")
lines.append("")
lines.append(f"- Variant directories scanned: {inventory['total_variant_dirs']}")
lines.append(f"- PlatformIO environments summarized: {inventory['total_envs']}")
lines.append("- Sources: `variants/**/platformio.ini` and `variants/**/variant.h`")
lines.append("- Notes: This inventory reflects explicit per-variant declarations. Some boards also inherit")
lines.append(" defaults from architecture headers or shared base environments, which must still be checked")
lines.append(" before creating new hardware support.")
lines.append("")
lines.append("## Repository Metadata Inputs")
lines.append("")
lines.append("The following `custom_meshtastic_*` metadata keys are already used across board environments:")
lines.append("")
for key in PLATFORMIO_METADATA_KEYS:
lines.append(f"- `{key}`")
lines.append("")
lines.append("## Common Target-Definition Categories")
lines.append("")
for category in ["Input", "Radio", "GPS", "Display", "I2C/SPI", "Power", "Connectivity/Other", "Other"]:
counter = category_frequency.get(category, Counter())
if not counter:
continue
lines.append(f"### {category}")
lines.append("")
lines.append("| Macro | Used in | Description |")
lines.append("| --- | --- | --- |")
for name, count in counter.most_common(15):
description = MACRO_DESCRIPTIONS.get(name, "")
lines.append(f"| `{name}` | {count} variants | {description} |")
lines.append("")
lines.append("## Architecture and Environment Inventory")
lines.append("")
for architecture in sorted(architectures):
entries = sorted(architectures[architecture], key=lambda item: item["environment"])
lines.append(f"### {architecture}")
lines.append("")
lines.append("| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |")
lines.append("| --- | --- | --- | --- | --- | --- |")
for entry in entries:
metadata = entry["metadata"]
category_counts = entry["category_counts"]
categories = []
for category in ["Display", "Radio", "Input", "GPS", "Power", "Connectivity/Other"]:
count = category_counts.get(category, 0)
if count:
categories.append(f"{category}:{count}")
category_summary = ", ".join(categories) if categories else "none"
lines.append(
"| {environment} | {display_name} | {hw_model} | {hw_slug} | {variant_dir} | {categories} |".format(
environment=entry["environment"],
display_name=metadata.get("custom_meshtastic_display_name", ""),
hw_model=metadata.get("custom_meshtastic_hw_model", ""),
hw_slug=metadata.get("custom_meshtastic_hw_model_slug", ""),
variant_dir=entry["variant_dir"],
categories=category_summary,
)
)
lines.append("")
lines.append("## Representative Board Examples")
lines.append("")
for architecture in sorted(architectures):
entries = sorted(architectures[architecture], key=lambda item: item["environment"])
if not entries:
continue
sample = entries[0]
lines.append(f"### {architecture}: `{sample['environment']}`")
lines.append("")
lines.append(f"- Variant directory: `{sample['variant_dir']}`")
metadata = sample["metadata"]
for key in PLATFORMIO_METADATA_KEYS:
value = metadata.get(key)
if value:
lines.append(f"- `{key}`: `{value}`")
defines = sample["defines"]
for category in ["Input", "Radio", "GPS", "Display", "I2C/SPI", "Power", "Connectivity/Other"]:
selected = [name for name in defines if classify_macro(name) == category][:8]
if not selected:
continue
lines.append(f"- {category} examples:")
for name in selected:
lines.append(f" - `{name}` = `{shorten(defines[name])}`")
lines.append("")
lines.append("## Intake Guidance For New Hardware")
lines.append("")
lines.append("When using this context to add a new board, collect these inputs before generating files:")
lines.append("")
lines.append("- PlatformIO environment name")
lines.append("- `custom_meshtastic_hw_model` and `custom_meshtastic_hw_model_slug`")
lines.append("- Display name and architecture")
lines.append("- Whether the board is actively supported and its support level")
lines.append("- Partition scheme, DFU requirement, and image/tag metadata if applicable")
lines.append("- Radio chip family and complete radio pin group")
lines.append("- Input/button/rotary/keyboard pins")
lines.append("- Display interface pins and driver-related macros")
lines.append("- GPS, power-management, I2C, SPI, storage, and auxiliary peripheral definitions")
lines.append("- Any board-specific initialization that requires `variant.cpp` or extra variant hooks")
lines.append("")
lines.append("## Inherited Defaults Note")
lines.append("")
lines.append(
"Some architecture families rely on BSP (Board Support Package) or base-environment defaults "
"rather than declaring every pin or capability macro explicitly in `variant.h`. "
"When adding a new board for one of these families, check the relevant BSP headers before "
"assuming a missing define means a feature is absent."
)
lines.append("")
no_variant_h = inventory.get("no_variant_h", {})
if no_variant_h:
lines.append(
f"Directories scanned that had no `variant.h` "
f"(relying entirely on BSP/base-environment): "
+ ", ".join(f"{arch}: {count}" for arch, count in sorted(no_variant_h.items()))
)
lines.append("")
for family, notes in _BSP_DEFAULT_FAMILIES.items():
lines.append(f"### {family}")
lines.append("")
for note in notes:
lines.append(f"- {note}")
lines.append("")
lines.append("## Cautions")
lines.append("")
lines.append("- Some boards rely on architecture defaults rather than declaring every field locally.")
lines.append("- Some board families expose multiple environments or display variants that share one hardware model.")
lines.append("- Source materials such as schematics still need human verification before new pin mappings are trusted.")
lines.append("- This document is a starting context artifact, not proof that a new board definition is safe to merge.")
lines.append("")
return "\n".join(lines)
def print_validate_summary(inventory: dict[str, object]) -> None:
"""Print a validation summary to stdout without writing the output file."""
total_dirs = inventory["total_variant_dirs"]
total_envs = inventory["total_envs"]
has_metadata = inventory["has_metadata"]
no_variant_h = inventory.get("no_variant_h", {})
no_variant_h_total = sum(no_variant_h.values())
architectures = inventory["architectures"]
arch_count = len(architectures)
print("Hardware Support Context — Validation Summary")
print("=" * 48)
print(f"Variant directories scanned : {total_dirs}")
print(f"Directories with no variant.h (BSP-only) : {no_variant_h_total}")
if no_variant_h:
for arch, count in sorted(no_variant_h.items()):
print(f" {arch}: {count}")
print(f"PlatformIO environments found : {total_envs}")
print(f"Environments with custom_meshtastic_* metadata : {has_metadata}")
print(f"Architecture families represented : {arch_count}")
print("")
print("Architecture families:")
for arch in sorted(architectures):
entries = architectures[arch]
print(f" {arch}: {len(entries)} environment(s)")
print("")
print("BSP-default families with special notes:")
for family in _BSP_DEFAULT_FAMILIES:
marker = "YES" if family in {a.split("-")[0] for a in architectures} else "not in scan"
print(f" {family}: {marker}")
def main() -> None:
parser = argparse.ArgumentParser(description="Generate hardware support context markdown")
parser.add_argument("--output", default=str(OUTPUT_PATH), help="Output markdown path")
parser.add_argument(
"--validate",
action="store_true",
help="Print a validation summary (variant counts, BSP-only dirs, metadata coverage) without writing output",
)
args = parser.parse_args()
inventory = collect_inventory()
if args.validate:
print_validate_summary(inventory)
return
markdown = render_markdown(inventory)
output_path = Path(args.output)
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(markdown + "\n", encoding="utf-8")
if __name__ == "__main__":
main()
@@ -87,9 +87,6 @@
</screenshots>
<releases>
<release version="2.7.24" date="2026-05-08">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.24</url>
</release>
<release version="2.7.23" date="2026-04-14">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.23</url>
</release>
+4 -8
View File
@@ -293,12 +293,9 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
board_arch = infer_architecture(env.BoardConfig())
should_skip_manifest = board_arch is None
# Most platforms can generate the manifest as part of the default 'buildprog' target.
# Typically this passes success/failure properly.
mtjson_deps = ["buildprog"]
if platform.name == "espressif32":
# On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
mtjson_deps = [] if should_skip_manifest else ["buildprog"]
if not should_skip_manifest and platform.name == "espressif32":
# Build littlefs image as part of mtjson target
# Equivalent to `pio run -t buildfs`
target_lfs = env.DataToBin(
@@ -312,8 +309,7 @@ if should_skip_manifest:
env.AddCustomTarget(
name="mtjson",
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
dependencies=[],
dependencies=mtjson_deps,
actions=[skip_manifest],
title="Meshtastic Manifest (skipped)",
description="mtjson generation is skipped for native environments",
-73
View File
@@ -1,73 +0,0 @@
#!/usr/bin/env bash
# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
# JSONL seed files + their compiled v25 protobufs.
#
# Layout:
# test/fixtures/nodedb/seed_v25_<N>.jsonl — COMMITTED, hand-editable.
# build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
# Drop into /prefs/nodes.proto.
#
# Daily use: ./bin/regen-fake-nodedbs.sh
# - Recompiles protos from committed seeds (fresh wall-clock timestamps).
# Intentional seed bump: REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
# - Overwrites the committed JSONL files with freshly-seeded data.
set -euo pipefail
cd "$(dirname "$0")/.."
# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
if [[ ! -d bin/_generated/meshtastic ]]; then
echo "regenerating Python protobuf bindings (one-time)..."
./bin/regen-py-protos.sh
fi
# 2) Pick a Python interpreter that has the meshtastic deps installed.
# Prefer the mcp-server venv (most likely to be set up by the operator).
PY="python3"
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
if [[ -x "$cand" ]]; then
PY="$cand"
break
fi
done
# 3) Pinned seeds per size — bump only when you intentionally want different
# structural data committed. Parallel arrays so the script works on
# macOS bash 3.2 (no `declare -A`).
SIZES=(250 500 1000 2000)
SEEDS=(20260511 20260512 20260513 20260514)
REGEN_SEEDS="${REGEN_SEEDS:-no}"
mkdir -p build/fixtures/nodedb test/fixtures/nodedb
for i in 0 1 2 3; do
n="${SIZES[$i]}"
seed="${SEEDS[$i]}"
jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
$PY bin/gen-fake-nodedb-seed.py \
--count "$n" \
--seed "$seed" \
--out "$jsonl" \
--centroid 33.1284,-107.2528 \
--spread-km 60 \
--position-coverage 0.85 \
--telemetry-coverage 0.70 \
--environment-coverage 0.25 \
--status-coverage 0.40
echo " seed: $jsonl ($(wc -c < "$jsonl") bytes)"
fi
$PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
echo " proto: $proto ($(wc -c < "$proto") bytes)"
done
echo ""
echo "Done. To load on Portduino native:"
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
echo ""
echo "To push to a hardware device:"
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
-51
View File
@@ -1,51 +0,0 @@
#!/usr/bin/env bash
# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
# a standalone refresh after any change to a .proto file.
#
# Output is .gitignored — bindings are a build artifact.
#
# Namespace rewrite:
# The .proto files declare `package meshtastic;`, which makes protoc emit
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
# PyPI `meshtastic` package (which the mcp-server relies on for its
# SerialInterface/BLEInterface transport). We post-process the generated
# files to live under `meshtastic_v25` instead — both the directory layout
# and all internal imports — so they coexist cleanly with the PyPI package.
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v protoc >/dev/null 2>&1; then
echo "ERROR: protoc not found in PATH." >&2
echo " macOS: brew install protobuf" >&2
echo " Ubuntu/Debian: apt install protobuf-compiler" >&2
exit 1
fi
OUT=bin/_generated
LOCAL_NS=meshtastic_v25
rm -rf "$OUT"
mkdir -p "$OUT"
# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
# is available for the [(nanopb).*] options on other messages.
protoc \
--proto_path=protobufs \
--python_out="$OUT" \
protobufs/nanopb.proto \
protobufs/meshtastic/*.proto
# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
# `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
# 4) Make the package importable.
touch "$OUT/__init__.py"
touch "$OUT/$LOCAL_NS/__init__.py"
echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2
-342
View File
@@ -1,342 +0,0 @@
#!/usr/bin/env python3
"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
are stored as `*_offset_sec` (seconds before "now"); this script resolves them
to absolute epochs using `--now-epoch` (default: current wall clock).
Output is a raw `pb_encode`-compatible binary that can be dropped at
`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
XModem) and loaded by `NodeDB::loadFromDisk` at boot.
Wire format reference:
protobufs/meshtastic/deviceonly.proto (NodeDatabase, NodeInfoLite, sat entries)
src/mesh/NodeDB.h:467-484 (bitfield bit positions)
src/mesh/NodeDB.cpp:1523-1524 (pb_decode entry point)
"""
from __future__ import annotations
import argparse
import json
import pathlib
import sys
import time
from typing import Any
# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
# ships. Run `bin/regen-py-protos.sh` to (re)generate.
#
# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
# post-processes the protoc output to rename the package.
_HERE = pathlib.Path(__file__).resolve().parent
_LOCAL_PROTO_DIR = _HERE / "_generated"
if _LOCAL_PROTO_DIR.is_dir():
sys.path.insert(0, str(_LOCAL_PROTO_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import ( # type: ignore[import-not-found]
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage # type: ignore[import-not-found]
from meshtastic_v25.config_pb2 import Config # type: ignore[import-not-found]
from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics # type: ignore[import-not-found]
except ImportError as local_err:
# Fall back to the PyPI package if in-tree bindings haven't been generated.
# Will fail the v25 assertion below if the PyPI package predates the
# satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
# error message instead of an opaque ImportError.
try:
from meshtastic.protobuf.deviceonly_pb2 import (
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
from meshtastic.protobuf.config_pb2 import Config
from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
except ImportError as pypi_err:
print(
"ERROR: could not import meshtastic protobuf bindings.\n"
" In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
" PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
f" local error (meshtastic_v25): {local_err}\n"
f" pypi error (meshtastic.protobuf): {pypi_err}",
file=sys.stderr,
)
sys.exit(1)
# Fail loudly if bindings predate v25 (no satellite arrays).
assert (
hasattr(NodeDatabase, "DESCRIPTOR")
and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
), (
"Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
"Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
)
# ---------------------------------------------------------------------------
# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
# ---------------------------------------------------------------------------
BIT_IS_KEY_MANUALLY_VERIFIED = 0
BIT_IS_MUTED = 1
BIT_VIA_MQTT = 2
BIT_IS_FAVORITE = 3
BIT_IS_IGNORED = 4
BIT_HAS_USER = 5
BIT_IS_LICENSED = 6
BIT_IS_UNMESSAGABLE = 7
BIT_HAS_IS_UNMESSAGABLE = 8
BITFIELD_LAYOUT = (
# JSON key bit position
("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
("is_muted", BIT_IS_MUTED),
("via_mqtt", BIT_VIA_MQTT),
("is_favorite", BIT_IS_FAVORITE),
("is_ignored", BIT_IS_IGNORED),
("has_user", BIT_HAS_USER),
("is_licensed", BIT_IS_LICENSED),
("is_unmessagable", BIT_IS_UNMESSAGABLE),
("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
)
def _pack_bitfield(bf: dict[str, bool]) -> int:
out = 0
for key, shift in BITFIELD_LAYOUT:
if bf.get(key, False):
out |= (1 << shift)
return out
def _validate_node(node: dict[str, Any]) -> None:
"""Friendly errors so hand-editors get clear feedback."""
if "num" not in node or not isinstance(node["num"], str):
raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
if "long_name" not in node:
raise ValueError(f"node {node['num']}: missing 'long_name'")
if len(node["long_name"]) > 24:
raise ValueError(
f"node {node['num']}: long_name {node['long_name']!r} is "
f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
)
if "short_name" in node:
# short_name max_size:5 (incl. NUL) → 4 bytes of content.
# Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
# would slip past a `len(str) > 4` check. Always measure bytes.
b = node["short_name"].encode("utf-8")
if len(b) > 4:
raise ValueError(
f"node {node['num']}: short_name {node['short_name']!r} is "
f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
)
pk = node.get("public_key_hex", "")
if pk and len(pk) != 64:
raise ValueError(
f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
f"got {len(pk)} chars"
)
if pk:
try:
bytes.fromhex(pk)
except ValueError as e:
raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
def _resolve_time(
node: dict[str, Any],
field_absolute: str,
field_offset: str,
now_epoch: int,
) -> int:
"""If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
if field_absolute in node and node[field_absolute] is not None:
return int(node[field_absolute])
offset = node.get(field_offset, 0)
return max(0, int(now_epoch) - int(offset))
def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
_validate_node(node)
info = NodeInfoLite()
info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
info.long_name = node.get("long_name", "")
info.short_name = node.get("short_name", "")
# Enum lookups will raise ValueError on unknown names — that's exactly what we want.
info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
pk_hex = node.get("public_key_hex", "")
if pk_hex:
info.public_key = bytes.fromhex(pk_hex)
info.snr = float(node.get("snr", 0.0))
info.channel = int(node.get("channel", 0))
if "hops_away" in node:
# `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
# field implicitly sets HasField("hops_away") to True. No has_hops_away
# setter exists (unlike the C++ nanopb-generated header).
info.hops_away = int(node["hops_away"])
info.next_hop = int(node.get("next_hop", 0))
info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
info.bitfield = _pack_bitfield(node.get("bitfield", {}))
return info
def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
entry = NodePositionEntry()
entry.num = num
pl = PositionLite()
# Firmware stores lat/long as int32 in 1e-7 degrees.
pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
pl.altitude = int(pos.get("altitude", 0))
pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
entry.position.CopyFrom(pl)
return entry
def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
entry = NodeTelemetryEntry()
entry.num = num
dm = DeviceMetrics()
if "battery_level" in tel:
dm.battery_level = int(tel["battery_level"])
if "voltage" in tel:
dm.voltage = float(tel["voltage"])
if "channel_utilization" in tel:
dm.channel_utilization = float(tel["channel_utilization"])
if "air_util_tx" in tel:
dm.air_util_tx = float(tel["air_util_tx"])
if "uptime_seconds" in tel:
dm.uptime_seconds = int(tel["uptime_seconds"])
entry.device_metrics.CopyFrom(dm)
return entry
def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
entry = NodeEnvironmentEntry()
entry.num = num
em = EnvironmentMetrics()
if "temperature" in env:
em.temperature = float(env["temperature"])
if "relative_humidity" in env:
em.relative_humidity = float(env["relative_humidity"])
if "barometric_pressure" in env:
em.barometric_pressure = float(env["barometric_pressure"])
if "iaq" in env:
em.iaq = int(env["iaq"])
entry.environment_metrics.CopyFrom(em)
return entry
def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
# `StatusMessage` (mesh.proto:1445) has a single `string status` field.
entry = NodeStatusEntry()
entry.num = num
sm = StatusMessage()
if "status" in status:
sm.status = str(status["status"])
entry.status.CopyFrom(sm)
return entry
def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
"""Read a seed JSONL and return the encoded NodeDatabase bytes."""
lines = jsonl_path.read_text(encoding="utf-8").splitlines()
if not lines:
raise ValueError(f"{jsonl_path} is empty")
meta_line = lines[0]
meta_obj = json.loads(meta_line)
meta = meta_obj.get("_meta", {})
version = meta.get("version")
if version != 25:
raise ValueError(
f"{jsonl_path}: meta version is {version!r}; this compiler "
f"requires version=25. Regenerate the seed with the matching tooling."
)
db = NodeDatabase()
db.version = 25
for ln, raw in enumerate(lines[1:], start=2):
raw = raw.strip()
if not raw:
continue
try:
node = json.loads(raw)
except json.JSONDecodeError as e:
raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
# Header
info = _build_node_info_lite(node, now_epoch)
db.nodes.append(info)
# Satellites (nullable)
if node.get("position"):
db.positions.append(_build_position_entry(num, node["position"], now_epoch))
if node.get("telemetry"):
db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
if node.get("environment"):
db.environment.append(_build_environment_entry(num, node["environment"]))
if node.get("status"):
db.status.append(_build_status_entry(num, node["status"]))
return db.SerializeToString()
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
p.add_argument("--out", required=True, help="Output binary .proto path.")
p.add_argument(
"--now-epoch",
type=int,
default=None,
help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
)
args = p.parse_args(argv)
in_path = pathlib.Path(args.in_path)
if not in_path.is_file():
print(f"input not found: {in_path}", file=sys.stderr)
return 2
now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
try:
encoded = compile_jsonl_to_proto(in_path, now_epoch)
except ValueError as e:
print(f"ERROR: {e}", file=sys.stderr)
return 3
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(encoded)
print(
f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-D CDEBYTE_EORA_S3",
"-D ARDUINO_USB_CDC_ON_BOOT=1",
"-D ARDUINO_USB_MODE=1",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=1",
"-D BOARD_HAS_PSRAM"
-42
View File
@@ -1,42 +0,0 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-D BOARD_HAS_PSRAM",
"-D ARDUINO_USB_CDC_ON_BOOT=0",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"psram_type": "qio_opi",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "ELECROW-ThinkNode-M7"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "ELECROW ThinkNode M7",
"upload": {
"flash_size": "8MB",
"maximum_ram_size": 524288,
"maximum_size": 8388608,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "https://www.elecrow.com",
"vendor": "ELECROW"
}
+1 -1
View File
@@ -6,7 +6,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DHELTEC_WIRELESS_TRACKER",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
-26
View File
@@ -1,26 +0,0 @@
{
"build": {
"cpu": "cortex-m33",
"f_cpu": "128000000L",
"mcu": "nrf54l15",
"zephyr": {
"variant": "nrf54l15dk/nrf54l15/cpuapp"
}
},
"connectivity": ["bluetooth"],
"debug": {
"default_tools": ["jlink"],
"jlink_device": "nRF54L15_M33",
"svd_path": "nrf54l15.svd"
},
"frameworks": ["zephyr"],
"name": "Nordic nRF54L15-DK (PCA10156)",
"upload": {
"maximum_ram_size": 262144,
"maximum_size": 1572864,
"protocol": "jlink",
"protocols": ["jlink"]
},
"url": "https://www.nordicsemi.com/Products/nRF54L15",
"vendor": "Nordic Semiconductor"
}
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
-41
View File
@@ -1,41 +0,0 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "station-g3"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "BQ Station G3",
"upload": {
"flash_size": "16MB",
"maximum_ram_size": 327680,
"maximum_size": 16777216,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "",
"vendor": "BQ Consulting"
}
+1 -1
View File
@@ -9,7 +9,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_1W",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_S3_CORE",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-DLILYGO_T3S3_V1",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -10,7 +10,7 @@
"-DBOARD_HAS_PSRAM",
"-DUNPHONE_SPIN=9",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
-6
View File
@@ -1,9 +1,3 @@
meshtasticd (2.7.24.0) unstable; urgency=medium
* Version 2.7.24
-- GitHub Actions <github-actions[bot]@users.noreply.github.com> Fri, 08 May 2026 10:44:12 +0000
meshtasticd (2.7.23.0) unstable; urgency=medium
* Version 2.7.23
-7
View File
@@ -1,7 +0,0 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x640000,
app1, app, ota_1, 0x650000,0x640000,
spiffs, data, spiffs, 0xc90000,0x360000,
coredump, data, coredump,0xFF0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x640000
5 app1 app ota_1 0x650000 0x640000
6 spiffs data spiffs 0xc90000 0x360000
7 coredump data coredump 0xFF0000 0x10000
-7
View File
@@ -1,7 +0,0 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x330000,
app1, app, ota_1, 0x340000,0x330000,
spiffs, data, spiffs, 0x670000,0x180000,
coredump, data, coredump,0x7F0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x330000
5 app1 app ota_1 0x340000 0x330000
6 spiffs data spiffs 0x670000 0x180000
7 coredump data coredump 0x7F0000 0x10000
+742
View File
@@ -0,0 +1,742 @@
# Hardware Support Context
This document inventories the board-support inputs and common target-definition fields
currently used in the Meshtastic firmware repository. It is intended as reusable context
for adding new hardware support without re-discovering naming patterns, metadata keys, and
frequently used pin or capability macros from scratch.
## Scope
- Variant directories scanned: 166
- PlatformIO environments summarized: 212
- Sources: `variants/**/platformio.ini` and `variants/**/variant.h`
- Notes: This inventory reflects explicit per-variant declarations. Some boards also inherit
defaults from architecture headers or shared base environments, which must still be checked
before creating new hardware support.
## Repository Metadata Inputs
The following `custom_meshtastic_*` metadata keys are already used across board environments:
- `custom_meshtastic_hw_model`
- `custom_meshtastic_hw_model_slug`
- `custom_meshtastic_architecture`
- `custom_meshtastic_actively_supported`
- `custom_meshtastic_support_level`
- `custom_meshtastic_display_name`
- `custom_meshtastic_images`
- `custom_meshtastic_tags`
- `custom_meshtastic_requires_dfu`
- `custom_meshtastic_partition_scheme`
## Common Target-Definition Categories
### Input
| Macro | Used in | Description |
| ---------------------------------- | ------------ | --------------------------------------------------------------------------- |
| `BUTTON_PIN` | 127 variants | Primary user button GPIO pin number. |
| `ALT_BUTTON_PIN` | 12 variants | Secondary / alternate button pin, used on boards with two buttons. |
| `CANCEL_BUTTON_PIN` | 8 variants | Button wired to cancel/back actions (e.g., T-Deck cancel key). |
| `KB_BL_PIN` | 5 variants | Keyboard backlight control pin (e.g., T-LoRa Pager keyboard). |
| `KB_INT` | 4 variants | Keyboard interrupt input pin; signals a keypress to the MCU. |
| `INPUTDRIVER_ENCODER_TYPE` | 3 variants | Selects the rotary encoder driver variant (0 = none, 1+ = specific type). |
| `BUTTON_PIN_ALT` | 2 variants | Alternative spelling used by some older board families for a second button. |
| `INPUTDRIVER_TWO_WAY_ROCKER` | 2 variants | Enables the two-way rocker input driver. |
| `INPUTDRIVER_TWO_WAY_ROCKER_RIGHT` | 2 variants | GPIO pin for the rightward rocker direction. |
| `INPUTDRIVER_TWO_WAY_ROCKER_LEFT` | 2 variants | GPIO pin for the leftward rocker direction. |
| `INPUTDRIVER_TWO_WAY_ROCKER_BTN` | 2 variants | GPIO pin for the rocker click/press direction. |
| `KB_POWERON` | 2 variants | Pin used to power-on or enable the keyboard peripheral. |
| `KB_SLAVE_ADDRESS` | 2 variants | I2C address of the keyboard controller IC. |
| `ROTARY_A` | 2 variants | First encoder channel pin (clock / A signal). |
| `ROTARY_B` | 2 variants | Second encoder channel pin (data / B signal). |
### Radio
| Macro | Used in | Description |
| -------------------------- | ------------ | ----------------------------------------------------------------------------- |
| `SX126X_CS` | 150 variants | SX126x chip-select pin (often same as LORA_CS). |
| `SX126X_RESET` | 150 variants | SX126x reset pin (often same as LORA_RESET). |
| `SX126X_BUSY` | 150 variants | SX126x BUSY pin; must be polled low before issuing SPI commands. |
| `SX126X_DIO1` | 150 variants | SX126x DIO1 interrupt output used for all IRQ events. |
| `USE_SX1262` | 145 variants | Select the SX1262 sub-GHz LoRa chip driver. |
| `SX126X_DIO3_TCXO_VOLTAGE` | 140 variants | Drives the TCXO regulator via DIO3; set to the supply voltage (e.g., 1.8). |
| `LORA_CS` | 137 variants | LoRa radio SPI chip-select GPIO pin. |
| `LORA_RESET` | 135 variants | LoRa radio hardware-reset GPIO pin (active low). |
| `SX126X_DIO2_AS_RF_SWITCH` | 135 variants | Set to 1 so the driver controls the TX/RX RF switch via DIO2. |
| `LORA_DIO1` | 130 variants | LoRa radio DIO1 interrupt pin; on SX126x this is the primary IRQ line. |
| `LORA_SCK` | 125 variants | LoRa radio SPI clock pin. |
| `LORA_MISO` | 125 variants | LoRa radio SPI MISO pin. |
| `LORA_MOSI` | 125 variants | LoRa radio SPI MOSI pin. |
| `LORA_DIO2` | 110 variants | LoRa radio DIO2 pin; on RF95 used for FSK interrupt; on SX126x tx/rx control. |
| `LORA_DIO0` | 98 variants | LoRa radio DIO0 interrupt pin (RF95/SX1276 done/RxDone). |
### GPS
| Macro | Used in | Description |
| ------------------------- | ------------ | ------------------------------------------------------------------------------ |
| `GPS_RX_PIN` | 114 variants | UART RX pin connected to the GPS module TX output. |
| `GPS_TX_PIN` | 111 variants | UART TX pin connected to the GPS module RX input. |
| `HAS_GPS` | 69 variants | Set to 1 if the board has an on-board GPS receiver; 0 to disable GPS entirely. |
| `PIN_GPS_PPS` | 43 variants | GPS pulse-per-second input pin for timing synchronisation. |
| `PIN_GPS_EN` | 30 variants | GPIO to power-enable or power-gate the GPS module. |
| `PIN_GPS_STANDBY` | 29 variants | Places the GPS into standby/low-power mode when driven. |
| `GPS_THREAD_INTERVAL` | 28 variants | Millisecond poll interval for the GPS background thread. |
| `GPS_L76K` | 27 variants | Selects the Quectel L76K GPS driver and protocol. |
| `GPS_BAUDRATE` | 27 variants | UART baud rate for GPS serial communication. |
| `GPS_EN_ACTIVE` | 17 variants | Logic level (HIGH or LOW) that enables the GPS power pin. |
| `PIN_GPS_RESET` | 16 variants | Hardware-reset line to the GPS module. |
| `GPS_DEFAULT_NOT_PRESENT` | 16 variants | Compiled-in default assuming no GPS; overridden at runtime if detected. |
| `GPS_RESET_MODE` | 13 variants | Defines the reset signal polarity or protocol for the GPS chip. |
| `GPS_UBLOX` | 8 variants | Selects the u-blox GPS driver and UBX protocol. |
| `PIN_GPS_REINIT` | 7 variants | Pin used to trigger GPS re-initialisation sequences. |
### Display
| Macro | Used in | Description |
| ----------------------------- | ----------- | ------------------------------------------------------------------------ |
| `PIN_EINK_CS` | 51 variants | E-ink display SPI chip-select pin. |
| `PIN_EINK_BUSY` | 51 variants | E-ink display BUSY output; high when a page update is in progress. |
| `PIN_EINK_DC` | 51 variants | E-ink display data/command select pin. |
| `PIN_EINK_RES` | 51 variants | E-ink display hardware-reset pin. |
| `PIN_EINK_SCLK` | 49 variants | E-ink display SPI clock pin. |
| `PIN_EINK_MOSI` | 49 variants | E-ink display SPI MOSI pin. |
| `HAS_SCREEN` | 34 variants | Set to 1 if the board has any display hardware. |
| `USE_TFTDISPLAY` | 27 variants | Selects the TFT LCD driver path. |
| `TFT_HEIGHT` | 25 variants | Vertical pixel resolution of the TFT display. |
| `TFT_WIDTH` | 25 variants | Horizontal pixel resolution of the TFT display. |
| `SCREEN_TRANSITION_FRAMERATE` | 23 variants | Target framerate for UI animations and transitions. |
| `TFT_OFFSET_X` | 22 variants | Horizontal pixel offset for display alignment correction. |
| `TFT_OFFSET_Y` | 22 variants | Vertical pixel offset for display alignment correction. |
| `SCREEN_ROTATE` | 18 variants | Non-zero value rotates the display 180° for mounted-upside-down screens. |
| `TFT_BL` | 15 variants | Backlight control PWM or GPIO pin for the TFT panel. |
### I2C/SPI
| Macro | Used in | Description |
| ----------------------- | ------------ | ----------------------------------------------------- |
| `I2C_SDA` | 103 variants | Primary I2C data line GPIO pin. |
| `I2C_SCL` | 103 variants | Primary I2C clock line GPIO pin. |
| `PIN_SPI_MISO` | 72 variants | Primary SPI MISO (data from peripheral) GPIO pin. |
| `PIN_SPI_MOSI` | 72 variants | Primary SPI MOSI (data to peripheral) GPIO pin. |
| `PIN_SPI_SCK` | 72 variants | Primary SPI clock GPIO pin. |
| `SPI_INTERFACES_COUNT` | 66 variants | Number of hardware SPI buses available on this board. |
| `WIRE_INTERFACES_COUNT` | 64 variants | Number of hardware I2C buses available on this board. |
| `PIN_SPI1_MISO` | 35 variants | Secondary SPI bus MISO pin. |
| `PIN_SPI1_MOSI` | 35 variants | Secondary SPI bus MOSI pin. |
| `PIN_SPI1_SCK` | 35 variants | Secondary SPI bus clock pin. |
| `SPI_FREQUENCY` | 24 variants | Default SPI clock frequency in Hz for this board. |
| `SPI_READ_FREQUENCY` | 21 variants | Reduced SPI clock rate used for read transactions. |
| `SPI_SCK` | 19 variants | SPI clock pin alias used in older board files. |
| `SPI_MOSI` | 19 variants | SPI MOSI pin alias used in older board files. |
| `SPI_MISO` | 19 variants | SPI MISO pin alias used in older board files. |
### Power
| Macro | Used in | Description |
| ------------------------------- | ------------ | -------------------------------------------------------------------------- |
| `BATTERY_PIN` | 128 variants | ADC input GPIO connected to the battery voltage divider. |
| `ADC_MULTIPLIER` | 122 variants | Floating-point scale factor to convert raw ADC reading to battery voltage. |
| `ADC_CHANNEL` | 71 variants | ADC channel enum or number for the battery sense input. |
| `BATTERY_SENSE_RESOLUTION_BITS` | 64 variants | ADC resolution in bits used for battery voltage sampling. |
| `ADC_RESOLUTION` | 49 variants | Board-level ADC resolution definition, referenced by other power macros. |
| `BATTERY_SENSE_RESOLUTION` | 44 variants | Alias for the effective ADC resolution for battery sense. |
| `ADC_CTRL` | 29 variants | GPIO that enables or gates the ADC voltage-divider circuit. |
| `ADC_CTRL_ENABLED` | 27 variants | Logic level (HIGH or LOW) that turns on the ADC control switch. |
| `EXT_NOTIFY_OUT` | 23 variants | GPIO output used to signal an external LED or buzzer for notifications. |
| `USE_POWERSAVE` | 21 variants | Enables aggressive power-save mode (deep sleep, reduced poll intervals). |
| `SLEEP_TIME` | 21 variants | Default light-sleep duration in milliseconds between wakeups. |
| `ADC_ATTENUATION` | 20 variants | ESP32 ADC input attenuation setting; controls measurable voltage range. |
| `PIN_POWER_EN` | 18 variants | GPIO to assert to enable a board power rail or load switch. |
| `BATTERY_SENSE_SAMPLES` | 11 variants | Number of ADC samples to average for a stable battery reading. |
| `HAS_PPM` | 4 variants | Set to 1 if the board has an IP5306 or similar PPM power path IC. |
### Connectivity/Other
| Macro | Used in | Description |
| ----------------- | ----------- | ------------------------------------------------------------------- |
| `HAS_TOUCHSCREEN` | 19 variants | Set to 1 for boards with a capacitive or resistive touch panel. |
| `HAS_NEOPIXEL` | 14 variants | Set to 1 if the board has addressable RGB LEDs (WS2812 / NeoPixel). |
| `PCF8563_RTC` | 11 variants | I2C address of the PCF8563 real-time clock IC. |
| `HAS_ETHERNET` | 9 variants | Set to 1 for boards with a wired Ethernet interface. |
| `HAS_I2S` | 7 variants | Set to 1 if I2S audio output is present. |
| `PCF85063_RTC` | 3 variants | I2C address of the PCF85063 real-time clock IC. |
| `NFC_INT` | 2 variants | Interrupt pin from the NFC controller IC. |
| `NFC_CS` | 2 variants | SPI chip-select for the NFC controller. |
| `USE_XL9555` | 2 variants | Enables the XL9555 16-bit I2C GPIO expander driver. |
| `EXPANDS_DRV_EN` | 2 variants | GPIO expander pin used to enable the haptic driver. |
| `EXPANDS_AMP_EN` | 2 variants | GPIO expander pin used to power-on the audio amplifier. |
| `EXPANDS_KB_RST` | 2 variants | GPIO expander pin used to reset the keyboard controller. |
| `EXPANDS_LORA_EN` | 2 variants | GPIO expander pin used to power-gate the LoRa radio. |
| `EXPANDS_GPS_EN` | 2 variants | GPIO expander pin used to power-gate the GPS module. |
| `EXPANDS_NFC_EN` | 2 variants | GPIO expander pin used to power-gate the NFC controller. |
### Other
| Macro | Used in | Description |
| -------------------- | ----------- | -------------------------------------------------------------------------- |
| `LED_POWER` | 94 variants | GPIO for the status LED, defines the LED pin number. |
| `LED_STATE_ON` | 88 variants | Logic level (HIGH or LOW) that turns the status LED on. |
| `PIN_SERIAL1_RX` | 67 variants | Secondary UART RX pin (used for accessories, GPS on some boards). |
| `PIN_SERIAL1_TX` | 67 variants | Secondary UART TX pin. |
| `PIN_WIRE_SDA` | 64 variants | Arduino-framework I2C SDA pin alias (nRF52 / RP2040 style). |
| `PIN_WIRE_SCL` | 64 variants | Arduino-framework I2C SCL pin alias. |
| `PIN_LED1` | 63 variants | First LED GPIO pin in the nRF52 Arduino BSP pin table. |
| `VARIANT_MCK` | 63 variants | Crystal oscillator frequency in Hz for nRF52 variant clock configuration. |
| `PINS_COUNT` | 63 variants | Total number of GPIO pins defined in the Arduino BSP variant table. |
| `NUM_DIGITAL_PINS` | 63 variants | Count of digital-capable pins in the BSP variant table. |
| `NUM_ANALOG_INPUTS` | 63 variants | Count of analog-input pins in the BSP variant table. |
| `NUM_ANALOG_OUTPUTS` | 63 variants | Count of analog-output (DAC) pins in the BSP variant table. |
| `LED_BLUE` | 62 variants | GPIO number of the blue status LED (typical on nRF52 and RP2040 boards). |
| `USE_LFXO` | 58 variants | Instructs the nRF52 BSP to use the low-frequency crystal oscillator. |
| `BUTTON_NEED_PULLUP` | 54 variants | Enables internal pull-up on the button GPIO (duplicate entry for clarity). |
## Architecture and Environment Inventory
### diy
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| ------------------------- | ------------ | -------- | ------- | --------------------------------------------- | --------------------------------------------------------- |
| 9m2ibr_aprs_lora_tracker | | | | variants/esp32/diy/9m2ibr_aprs_lora_tracker | Display:1, Radio:24, Input:1, GPS:2, Power:4 |
| esp32c3_super_mini | | | | variants/esp32c3/diy/esp32c3_super_mini | Display:1, Radio:18, Input:1, GPS:2 |
| meshtastic-diy-v1_1 | | | | variants/esp32/diy/v1_1 | Radio:24, Input:1, GPS:1, Power:1 |
| my-esp32s3-diy-eink | | | | variants/esp32s3/diy/my_esp32s3_diy_eink | Display:6, Radio:17, Input:1, GPS:1, Connectivity/Other:1 |
| my-esp32s3-diy-oled | | | | variants/esp32s3/diy/my_esp32s3_diy_oled | Display:1, Radio:17, Input:1, GPS:1, Connectivity/Other:1 |
| nrf52_promicro_diy-inkhud | | | | variants/nrf52840/diy/nrf52_promicro_diy_tcxo | Display:4, Radio:29, Input:1, GPS:4, Power:6 |
| t-energy-s3_e22 | | | | variants/esp32s3/diy/t-energy-s3_e22 | Display:1, Radio:18, Input:1, GPS:3, Power:3 |
### esp32
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| --------------------------- | --------------------------- | -------- | --------------------------- | ------------------------------------------ | ------------------------------------------------------------------ |
| betafpv_2400_tx_micro | | | | variants/esp32/betafpv_2400_tx_micro | Radio:14, Input:1, Connectivity/Other:1 |
| betafpv_900_tx_nano | | | | variants/esp32/betafpv_900_tx_nano | Display:1, Radio:10, Input:1 |
| chatter2 | | | | variants/esp32/chatter2 | Display:12, Radio:16, Input:3, GPS:3, Power:4 |
| hackerboxes-esp32-io | | | | variants/esp32/hackerboxes_esp32_io | Display:1, Radio:16, Input:1, GPS:1 |
| heltec-v1 | Heltec V1 | 11 | HELTEC_V1 | variants/esp32/heltec_v1 | Radio:5, Input:1, GPS:2, Power:3 |
| heltec-v2_0 | Heltec V2.0 | 5 | HELTEC_V2_0 | variants/esp32/heltec_v2 | Radio:5, Input:1, GPS:2, Power:3 |
| heltec-v2_1 | Heltec V2.1 | 10 | HELTEC_V2_1 | variants/esp32/heltec_v2.1 | Radio:5, Input:1, GPS:3, Power:4 |
| heltec-wireless-bridge | | | | variants/esp32/heltec_wireless_bridge | Radio:9, GPS:1 |
| heltec-wsl-v2_1 | | | | variants/esp32/heltec_wsl_v2.1 | Radio:9, Input:1, Power:5 |
| hydra | Hydra | 39 | HYDRA | variants/esp32/diy/hydra | Radio:22, Input:1, GPS:3, Power:4 |
| m5stack-core | M5 Stack | 42 | M5STACK | variants/esp32/m5stack_core | Display:7, Radio:9, Input:1, GPS:2 |
| m5stack-coreink | | | | variants/esp32/m5stack_coreink | Display:7, Radio:34, Input:1, GPS:2, Power:3, Connectivity/Other:1 |
| meshtastic-diy-v1 | DIY V1 | 39 | DIY_V1 | variants/esp32/diy/v1 | Radio:23, Input:1, GPS:3, Power:4 |
| meshtastic-dr-dev | DR-DEV | 41 | DR_DEV | variants/esp32/diy/dr-dev | Display:1, Radio:26, Input:1, GPS:2, Power:3 |
| nano-g1 | Nano G1 | 14 | NANO_G1 | variants/esp32/nano-g1 | Display:1, Radio:13, Input:1, GPS:2, Power:1 |
| nano-g1-explorer | Nano G1 Explorer | 17 | NANO_G1_EXPLORER | variants/esp32/nano-g1-explorer | Display:1, Radio:13, Input:1, GPS:2, Power:5 |
| radiomaster_900_bandit | | | | variants/esp32/radiomaster_900_bandit | Radio:19, Input:1, GPS:1, Connectivity/Other:1 |
| radiomaster_900_bandit_nano | RadioMaster 900 Bandit Nano | 64 | RADIOMASTER_900_BANDIT_NANO | variants/esp32/radiomaster_900_bandit_nano | Display:1, Radio:19 |
| rak11200 | RAK WisBlock 11200 | 13 | RAK11200 | variants/esp32/rak11200 | Radio:18, GPS:2, Power:2 |
| station-g1 | Station G1 | 25 | STATION_G1 | variants/esp32/station-g1 | Display:1, Radio:13, Input:1, GPS:2, Power:5 |
| sugarcube | | | | variants/esp32/tlora_v2_1_16 | Radio:6, Power:4 |
| tbeam | LILYGO T-Beam | 4 | TBEAM | variants/esp32/tbeam | Display:4, Radio:14, Input:1, GPS:3, Power:2, Connectivity/Other:1 |
| tbeam-displayshield | | | | variants/esp32/tbeam | Display:4, Radio:14, Input:1, GPS:3, Power:2, Connectivity/Other:1 |
| tbeam0_7 | LILYGO T-Beam V0.7 | 6 | TBEAM_V0P7 | variants/esp32/tbeam_v07 | Radio:5, Input:1, GPS:3, Power:3 |
| tlora-v1 | LILYGO T-LoRa V1 | 2 | TLORA_V1 | variants/esp32/tlora_v1 | Radio:5, Input:1, Power:1 |
| tlora-v2 | LILYGO T-LoRa V2 | 1 | TLORA_V2 | variants/esp32/tlora_v2 | Radio:5, Input:1, Power:2 |
| tlora-v2-1-1_6 | LILYGO T-LoRa V2.1-1.6 | 3 | TLORA_V2_1_1P6 | variants/esp32/tlora_v2_1_16 | Radio:6, Power:4 |
| tlora-v2-1-1_8 | LILYGO T-LoRa V2.1-1.8 | 15 | TLORA_V2_1_1P8 | variants/esp32/tlora_v2_1_18 | Radio:6, Input:1, Power:3 |
| tlora_v1_3 | | | | variants/esp32/tlora_v1_3 | Radio:5, Input:1, Power:2 |
| trackerd | | | | variants/esp32/trackerd | Display:1, Radio:5, Input:1, GPS:9, Power:4 |
| wiphone | | | | variants/esp32/wiphone | Display:9, Radio:9, GPS:1 |
### esp32-c3
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| -------------------------- | ------------ | -------- | ----------- | ----------------------------------------- | ------------------------------------------------------------------ |
| ai-c3 | | | | variants/esp32c3/ai-c3 | Radio:15, Input:1, GPS:1 |
| hackerboxes-esp32c3-oled | | | | variants/esp32c3/hackerboxes_esp32c3_oled | Display:1, Radio:16, Input:1, GPS:1 |
| heltec-hru-3601 | | | | variants/esp32c3/heltec_hru_3601 | Display:1, Radio:16, Input:1, GPS:1, Power:1, Connectivity/Other:1 |
| heltec-ht62-esp32c3-sx1262 | Heltec HT62 | 53 | HELTEC_HT62 | variants/esp32c3/heltec_esp32c3 | Display:1, Radio:16, Input:1, GPS:1 |
| m5stack-stamp-c3 | | | | variants/esp32c3/m5stack-stamp-c3 | Radio:9, Input:1, GPS:1 |
### esp32-c6
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| --------------- | ---------------- | -------- | ----------- | -------------------------------- | ------------------------------------------------ |
| m5stack-unitc6l | M5Stack Unit C6L | 111 | M5STACK_C6L | variants/esp32c6/m5stack_unitc6l | Display:1, Radio:14, GPS:3, Connectivity/Other:1 |
| tlora-c6 | | | | variants/esp32c6/tlora_c6 | Radio:15 |
### esp32-s3
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| -------------------------------- | ----------------------------- | -------- | ---------------------------- | --------------------------------------------- | -------------------------------------------------------------------- |
| CDEBYTE_EoRa-Hub | | | | variants/esp32s3/CDEBYTE_EoRa-Hub | Display:2, Radio:14, Input:1, Power:6 |
| CDEBYTE_EoRa-S3 | EBYTE EoRa-S3 | 61 | CDEBYTE_EORA_S3 | variants/esp32s3/CDEBYTE_EoRa-S3 | Display:2, Radio:13, Input:1, Power:3 |
| EBYTE_ESP32-S3 | | | | variants/esp32s3/EBYTE_ESP32-S3 | Display:1, Radio:26, Input:1, GPS:5, Power:1 |
| ESP32-S3-Pico | | | | variants/esp32s3/esp32-s3-pico | Display:6, Radio:18, Input:2, GPS:1, Power:4, Connectivity/Other:1 |
| bpi_picow_esp32_s3 | | | | variants/esp32s3/bpi_picow_esp32_s3 | Display:1, Radio:25, Input:1, GPS:1 |
| crowpanel-esp32s3-2-epaper | | | | variants/esp32s3/crowpanel-esp32s3-5-epaper | Display:6, Radio:21, Input:1, GPS:1, Power:1 |
| crowpanel-esp32s3-4-epaper | | | | variants/esp32s3/crowpanel-esp32s3-5-epaper | Display:6, Radio:21, Input:1, GPS:1, Power:1 |
| crowpanel-esp32s3-5-epaper | | | | variants/esp32s3/crowpanel-esp32s3-5-epaper | Display:6, Radio:21, Input:1, GPS:1, Power:1 |
| dreamcatcher-2206 | | | | variants/esp32s3/dreamcatcher | Radio:16, Input:1, Power:1, Connectivity/Other:1 |
| elecrow-adv-24-28-tft | Crowpanel Adv 2.4/2.8 TFT | 97 | CROWPANEL | variants/esp32s3/elecrow_panel | Display:1, Radio:21, GPS:6, Power:2 |
| elecrow-adv-35-tft | Crowpanel Adv 3.5 TFT | 97 | CROWPANEL | variants/esp32s3/elecrow_panel | Display:1, Radio:21, GPS:6, Power:2 |
| elecrow-adv1-43-50-70-tft | Crowpanel Adv 4.3/5.0/7.0 TFT | 97 | CROWPANEL | variants/esp32s3/elecrow_panel | Display:1, Radio:21, GPS:6, Power:2 |
| hackaday-communicator | | | | variants/esp32s3/hackaday-communicator | Display:12, Radio:14, Input:1, GPS:1, Power:2 |
| heltec-v3 | Heltec V3 | 43 | HELTEC_V3 | variants/esp32s3/heltec_v3 | Display:1, Radio:16, Input:1, Power:6 |
| heltec-v4 | Heltec V4 | 110 | HELTEC_V4 | variants/esp32s3/heltec_v4 | Display:1, Radio:21, Input:1, GPS:10, Power:6 |
| heltec-v4-tft | Heltec V4 TFT | 110 | HELTEC_V4 | variants/esp32s3/heltec_v4 | Display:1, Radio:21, Input:1, GPS:10, Power:6 |
| heltec-vision-master-e213 | Heltec Vision Master E213 | 67 | HELTEC_VISION_MASTER_E213 | variants/esp32s3/heltec_vision_master_e213 | Display:6, Radio:15, Input:2, Power:6 |
| heltec-vision-master-e213-inkhud | | | | variants/esp32s3/heltec_vision_master_e213 | Display:6, Radio:15, Input:2, Power:6 |
| heltec-vision-master-e290 | Heltec Vision Master E290 | 68 | HELTEC_VISION_MASTER_E290 | variants/esp32s3/heltec_vision_master_e290 | Display:6, Radio:15, Input:2, Power:6 |
| heltec-vision-master-e290-inkhud | | | | variants/esp32s3/heltec_vision_master_e290 | Display:6, Radio:15, Input:2, Power:6 |
| heltec-vision-master-t190 | Heltec Vision Master T190 | 66 | HELTEC_VISION_MASTER_T190 | variants/esp32s3/heltec_vision_master_t190 | Display:6, Radio:16, Input:2, Power:6 |
| heltec-wireless-paper | Heltec Wireless Paper | 49 | HELTEC_WIRELESS_PAPER | variants/esp32s3/heltec_wireless_paper | Display:6, Radio:15, Input:1, Power:6 |
| heltec-wireless-paper-inkhud | | | | variants/esp32s3/heltec_wireless_paper | Display:6, Radio:15, Input:1, Power:6 |
| heltec-wireless-paper-v1_0 | Heltec Wireless Paper V1.0 | 57 | HELTEC_WIRELESS_PAPER_V1_0 | variants/esp32s3/heltec_wireless_paper_v1 | Display:6, Radio:15, Input:1, Power:6 |
| heltec-wireless-tracker | Heltec Wireless Tracker V1.1 | 48 | HELTEC_WIRELESS_TRACKER | variants/esp32s3/heltec_wireless_tracker | Display:9, Radio:16, Input:1, GPS:7, Power:6 |
| heltec-wireless-tracker-V1-0 | Heltec Wireless Tracker V1.0 | 58 | HELTEC_WIRELESS_TRACKER_V1_0 | variants/esp32s3/heltec_wireless_tracker_V1_0 | Display:9, Radio:16, Input:1, GPS:9, Power:4 |
| heltec-wireless-tracker-v2 | Heltec Wireless Tracker V2 | 113 | HELTEC_WIRELESS_TRACKER_V2 | variants/esp32s3/heltec_wireless_tracker_v2 | Display:10, Radio:19, Input:1, GPS:7, Power:6 |
| heltec-wsl-v3 | Heltec Wireless Stick Lite V3 | 44 | HELTEC_WSL_V3 | variants/esp32s3/heltec_wsl_v3 | Radio:16, Input:1, Power:6 |
| heltec_capsule_sensor_v3 | | | | variants/esp32s3/heltec_capsule_sensor_v3 | Display:1, Radio:16, Input:1, GPS:7, Power:6 |
| heltec_sensor_hub | | | | variants/esp32s3/heltec_sensor_hub | Display:1, Radio:15, Input:1, Power:6, Connectivity/Other:1 |
| icarus | | | | variants/esp32s3/icarus | Display:1, Radio:12, Input:1 |
| link32-s3-v1 | | | | variants/esp32s3/link32_s3_v1 | Display:1, Radio:16, Input:2, Power:4, Connectivity/Other:1 |
| m5stack-cardputer-adv | | | | variants/esp32s3/m5stack_cardputer_adv | Display:5, Radio:17, Input:2, GPS:4, Power:3, Connectivity/Other:2 |
| m5stack-cores3 | | | | variants/esp32s3/m5stack_cores3 | Radio:11, Power:1 |
| mesh-tab-3-2-IPS-capacitive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-3-2-IPS-resistive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-3-2-TN-resistive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-3-5-IPS-capacitive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-3-5-IPS-resistive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-3-5-TN-resistive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mesh-tab-4-0-IPS-capacitive | | | | variants/esp32s3/mesh-tab | Radio:18, Input:1, GPS:2, Power:5, Connectivity/Other:1 |
| mini-epaper-s3 | LILYGO Mini ePaper S3 E-Ink | | MINI_EPAPER_S3 | variants/esp32s3/mini-epaper-s3 | Display:8, Radio:12, Input:5, GPS:1, Power:3, Connectivity/Other:1 |
| mini-epaper-s3-inkhud | | | | variants/esp32s3/mini-epaper-s3 | Display:8, Radio:12, Input:5, GPS:1, Power:3, Connectivity/Other:1 |
| nibble-esp32 | | | | variants/esp32s3/nibble_esp32 | Radio:9, Input:1 |
| nugget-s3-lora | | | | variants/esp32s3/nugget_s3_lora | Display:2, Radio:9, Input:1, Connectivity/Other:1 |
| picomputer-s3 | Pi Computer S3 | 52 | PICOMPUTER_S3 | variants/esp32s3/picomputer-s3 | Display:10, Radio:9, Input:1, Power:3 |
| picomputer-s3-tft | | | | variants/esp32s3/picomputer-s3 | Display:10, Radio:9, Input:1, Power:3 |
| rak3112 | | | | variants/esp32s3/rak3312 | Radio:13, GPS:3, Power:5 |
| rak3312 | RAK3312 | 106 | RAK3312 | variants/esp32s3/rak3312 | Radio:13, GPS:3, Power:5 |
| rak_wismesh_tap_v2 | RAK WisMesh Tap V2 | 116 | WISMESH_TAP_V2 | variants/esp32s3/rak_wismesh_tap_v2 | Radio:13, Input:1, GPS:3, Power:4 |
| rak_wismesh_tap_v2-tft | | | | variants/esp32s3/rak_wismesh_tap_v2 | Radio:13, Input:1, GPS:3, Power:4 |
| seeed-sensecap-indicator | Seeed SenseCAP Indicator | 70 | SENSECAP_INDICATOR | variants/esp32s3/seeed-sensecap-indicator | Display:12, Radio:17, Input:1, GPS:4, Connectivity/Other:1 |
| seeed-sensecap-indicator-tft | | | | variants/esp32s3/seeed-sensecap-indicator | Display:12, Radio:17, Input:1, GPS:4, Connectivity/Other:1 |
| seeed-xiao-s3 | Seeed Xiao ESP32-S3 | 81 | SEEED_XIAO_S3 | variants/esp32s3/seeed_xiao_s3 | Radio:16, Input:1, GPS:6, Power:3 |
| station-g2 | Station G2 | 31 | STATION_G2 | variants/esp32s3/station-g2 | Display:1, Radio:14, Input:1, GPS:2, Power:4 |
| t-beam-1w | LILYGO T-Beam 1W | 122 | TBEAM_1_WATT | variants/esp32s3/t-beam-1w | Display:3, Radio:19, Input:2, GPS:6, Power:5 |
| t-deck | LILYGO T-Deck | 50 | T_DECK | variants/esp32s3/t-deck | Display:11, Radio:17, Input:3, GPS:3, Power:5, Connectivity/Other:2 |
| t-deck-pro | LILYGO T-Deck Pro | 102 | T_DECK_PRO | variants/esp32s3/t-deck-pro | Display:6, Radio:18, Input:2, GPS:7, Power:5, Connectivity/Other:1 |
| t-deck-tft | | | | variants/esp32s3/t-deck | Display:11, Radio:17, Input:3, GPS:3, Power:5, Connectivity/Other:2 |
| t-eth-elite | | | | variants/esp32s3/t-eth-elite | Display:1, Radio:34, Input:1, GPS:5, Connectivity/Other:1 |
| t-watch-s3 | LILYGO T-Watch S3 | 51 | T_WATCH_S3 | variants/esp32s3/t-watch-s3 | Display:12, Radio:17, Input:1, GPS:5, Power:3, Connectivity/Other:3 |
| t5s3_epaper_inkhud | | | | variants/esp32s3/t5s3_epaper | Radio:21, Input:3, GPS:2, Power:5, Connectivity/Other:3 |
| tbeam-s3-core | LILYGO T-Beam Supreme | 12 | LILYGO_TBEAM_S3_CORE | variants/esp32s3/tbeam-s3-core | Display:1, Radio:25, Input:1, GPS:4, Power:1, Connectivity/Other:2 |
| thinknode_m2 | ThinkNode M2 | 90 | THINKNODE_M2 | variants/esp32s3/ELECROW-ThinkNode-M2 | Display:2, Radio:14, Input:2, GPS:1, Power:6 |
| thinknode_m5 | ThinkNode M5 | 107 | THINKNODE_M5 | variants/esp32s3/ELECROW-ThinkNode-M5 | Display:6, Radio:14, Input:2, GPS:8, Power:4, Connectivity/Other:1 |
| tlora-pager | LILYGO T-LoRa Pager | 103 | T_LORA_PAGER | variants/esp32s3/tlora-pager | Display:10, Radio:30, Input:6, GPS:5, Power:5, Connectivity/Other:17 |
| tlora-pager-tft | | | | variants/esp32s3/tlora-pager | Display:10, Radio:30, Input:6, GPS:5, Power:5, Connectivity/Other:17 |
| tlora-t3s3-epaper | LILYGO T-LoRa T3-S3 E-Ink | 16 | TLORA_T3_S3 | variants/esp32s3/tlora_t3s3_epaper | Display:6, Radio:28, Input:1, GPS:3, Power:3 |
| tlora-t3s3-epaper-inkhud | | | | variants/esp32s3/tlora_t3s3_epaper | Display:6, Radio:28, Input:1, GPS:3, Power:3 |
| tlora-t3s3-v1 | LILYGO T-LoRa T3-S3 | 16 | TLORA_T3_S3 | variants/esp32s3/tlora_t3s3_v1 | Display:1, Radio:36, Input:1, Power:3 |
| unphone | unPhone | 59 | UNPHONE | variants/esp32s3/unphone | Display:9, Radio:9, Input:3, GPS:1, Power:2, Connectivity/Other:1 |
| unphone-tft | | | | variants/esp32s3/unphone | Display:9, Radio:9, Input:3, GPS:1, Power:2, Connectivity/Other:1 |
### esp32s2
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| -------------- | ------------ | -------- | ------- | ------------------------------- | -------------------------------------- |
| nugget-s2-lora | | | | variants/esp32s2/nugget_s2_lora | Radio:9, Input:1, Connectivity/Other:1 |
### native
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| ---------------- | ------------ | -------- | ------- | ----------------------------------- | ----------------- |
| buildroot | | | | variants/native/portduino-buildroot | Display:2, GPS:1 |
| coverage | | | | variants/native/portduino | Display:2, GPS:1 |
| native | | | | variants/native/portduino | Display:2, GPS:1 |
| native-fb | | | | variants/native/portduino | Display:2, GPS:1 |
| native-tft | | | | variants/native/portduino | Display:2, GPS:1 |
| native-tft-debug | | | | variants/native/portduino | Display:2, GPS:1 |
### nrf52840
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| -------------------------------- | -------------------------- | -------- | ------------------------- | ---------------------------------------------- | --------------------------------------------------------- |
| ME25LS01-4Y10TD | | | | variants/nrf52840/ME25LS01-4Y10TD | Radio:8, Input:1, GPS:8, Power:4 |
| ME25LS01-4Y10TD_e-ink | | | | variants/nrf52840/ME25LS01-4Y10TD_e-ink | Display:6, Radio:8, Input:1, GPS:8, Power:4 |
| TWC_mesh_v4 | | | | variants/nrf52840/TWC_mesh_v4 | Display:1, Radio:7, GPS:4, Power:5 |
| canaryone | Canary One | 29 | CANARYONE | variants/nrf52840/canaryone | Radio:7, GPS:8, Power:5 |
| feather_diy | | | | variants/nrf52840/feather_diy | Radio:17, Input:1 |
| gat562_mesh_trial_tracker | | | | variants/nrf52840/gat562_mesh_trial_tracker | Display:2, Radio:8, GPS:4, Power:5 |
| heltec-mesh-node-t096 | Heltec Mesh Node 096 | 127 | HELTEC_MESH_NODE_T096 | variants/nrf52840/heltec_mesh_node_t096 | Display:8, Radio:11, GPS:10, Power:9 |
| heltec-mesh-node-t114 | Heltec Mesh Node T114 | 69 | HELTEC_MESH_NODE_T114 | variants/nrf52840/heltec_mesh_node_t114 | Display:7, Radio:8, GPS:7, Power:8, Connectivity/Other:1 |
| heltec-mesh-node-t114-inkhud | | | | variants/nrf52840/heltec_mesh_node_t114-inkhud | Display:6, Radio:8, GPS:7, Power:7, Connectivity/Other:1 |
| heltec-mesh-pocket-10000 | Heltec Mesh Pocket | 94 | HELTEC_MESH_POCKET | variants/nrf52840/heltec_mesh_pocket | Display:6, Radio:8, GPS:1, Power:7 |
| heltec-mesh-pocket-10000-inkhud | Heltec Mesh Pocket | 94 | HELTEC_MESH_POCKET | variants/nrf52840/heltec_mesh_pocket | Display:6, Radio:8, GPS:1, Power:7 |
| heltec-mesh-pocket-5000 | Heltec Mesh Pocket | 94 | HELTEC_MESH_POCKET | variants/nrf52840/heltec_mesh_pocket | Display:6, Radio:8, GPS:1, Power:7 |
| heltec-mesh-pocket-5000-inkhud | Heltec Mesh Pocket | 94 | HELTEC_MESH_POCKET | variants/nrf52840/heltec_mesh_pocket | Display:6, Radio:8, GPS:1, Power:7 |
| heltec-mesh-solar | Heltec MeshSolar | 108 | HELTEC_MESH_SOLAR | variants/nrf52840/heltec_mesh_solar | Radio:8, GPS:6 |
| heltec-mesh-solar-eink | | | | variants/nrf52840/heltec_mesh_solar | Radio:8, GPS:6 |
| heltec-mesh-solar-inkhud | | | | variants/nrf52840/heltec_mesh_solar | Radio:8, GPS:6 |
| heltec-mesh-solar-oled | | | | variants/nrf52840/heltec_mesh_solar | Radio:8, GPS:6 |
| heltec-mesh-solar-tft | | | | variants/nrf52840/heltec_mesh_solar | Radio:8, GPS:6 |
| makerpython_nrf52840_sx1280_eink | | | | variants/nrf52840/MakePython_nRF52840_eink | Display:6, Radio:6, GPS:4, Power:5 |
| makerpython_nrf52840_sx1280_oled | | | | variants/nrf52840/MakePython_nRF52840_oled | Radio:6, GPS:4, Power:5 |
| meshlink | | | | variants/nrf52840/meshlink | Display:6, Radio:7, Input:1, GPS:5, Power:5 |
| meshlink_eink | | | | variants/nrf52840/meshlink | Display:6, Radio:7, Input:1, GPS:5, Power:5 |
| meshtiny | | | | variants/nrf52840/meshtiny | Display:2, Radio:8, Input:5, Power:5 |
| minimesh_lite | | | | variants/nrf52840/dls_Minimesh_Lite | Radio:15, Input:1, GPS:4, Power:6 |
| monteops_hw1 | | | | variants/nrf52840/monteops_hw1 | Radio:8, GPS:3, Power:5, Connectivity/Other:1 |
| ms24sf1 | | | | variants/nrf52840/MS24SF1 | Radio:8, Input:1, GPS:8, Power:4 |
| muzi-base | muzi BASE | 93 | MUZI_BASE | variants/nrf52840/muzi_base | Display:3, Radio:19, Input:1, GPS:4, Power:5 |
| nano-g2-ultra | Nano G2 Ultra | 18 | NANO_G2_ULTRA | variants/nrf52840/nano-g2-ultra | Display:1, Radio:7, GPS:4, Power:6, Connectivity/Other:1 |
| nrf52_promicro_diy_tcxo | NRF52 Pro-micro DIY | 63 | NRF52_PROMICRO_DIY | variants/nrf52840/diy/nrf52_promicro_diy_tcxo | Display:4, Radio:29, Input:1, GPS:4, Power:6 |
| pca10059_diy_eink | | | | variants/nrf52840/Dongle_nRF52840-pca10059-v1 | Display:7, Radio:8, GPS:4, Power:5 |
| r1-neo | muzi R1 Neo | 101 | MUZI_R1_NEO | variants/nrf52840/r1-neo | Display:1, Radio:8, GPS:4, Power:5 |
| rak2560 | RAK WisMesh Repeater | 22 | WISMESH_HUB | variants/nrf52840/rak2560 | Display:6, Radio:8, GPS:2, Power:5 |
| rak3401-1watt | RAK3401 1W | 117 | RAK3401 | variants/nrf52840/rak3401_1watt | Display:6, Radio:12, GPS:3, Power:5 |
| rak4631 | RAK WisBlock 4631 | 9 | RAK4631 | variants/nrf52840/rak4631 | Display:6, Radio:8, GPS:3, Power:7, Connectivity/Other:1 |
| rak4631_dbg | | | | variants/nrf52840/rak4631 | Display:6, Radio:8, GPS:3, Power:7, Connectivity/Other:1 |
| rak4631_eink | | | | variants/nrf52840/rak4631_epaper | Display:6, Radio:8, GPS:4, Power:5 |
| rak4631_eink_onrxtx | | | | variants/nrf52840/rak4631_epaper_onrxtx | Display:6, Radio:8, Power:1 |
| rak4631_eth_gw | | | | variants/nrf52840/rak4631_eth_gw | Display:6, Radio:8, GPS:3, Power:5, Connectivity/Other:1 |
| rak4631_eth_gw_dbg | | | | variants/nrf52840/rak4631_eth_gw | Display:6, Radio:8, GPS:3, Power:5, Connectivity/Other:1 |
| rak4631_nomadstar_meteor_pro | NomadStar Meteor Pro | 96 | NOMADSTAR_METEOR_PRO | variants/nrf52840/rak4631_nomadstar_meteor_pro | Display:6, Radio:8, GPS:3, Power:5, Connectivity/Other:1 |
| rak4631_nomadstar_meteor_pro_dbg | | | | variants/nrf52840/rak4631_nomadstar_meteor_pro | Display:6, Radio:8, GPS:3, Power:5, Connectivity/Other:1 |
| rak_wismeshtag | RAK WisMesh Tag | 105 | WISMESH_TAG | variants/nrf52840/rak_wismeshtag | Display:7, Radio:8, GPS:4, Power:5 |
| rak_wismeshtap | RAK WisMesh Tap | 84 | WISMESH_TAP | variants/nrf52840/rak_wismeshtap | Display:21, Radio:8, GPS:3, Power:7, Connectivity/Other:1 |
| seeed_solar_node | Seeed SenseCAP Solar Node | 95 | SEEED_SOLAR_NODE | variants/nrf52840/seeed_solar_node | Radio:9, Input:2, GPS:8, Power:3 |
| seeed_wio_tracker_L1 | Seeed Wio Tracker L1 | 99 | SEEED_WIO_TRACKER_L1 | variants/nrf52840/seeed_wio_tracker_L1 | Display:2, Radio:9, Input:1, GPS:7, Power:3 |
| seeed_wio_tracker_L1_eink | Seeed Wio Tracker L1 E-Ink | 100 | SEEED_WIO_TRACKER_L1_EINK | variants/nrf52840/seeed_wio_tracker_L1_eink | Display:7, Radio:9, Input:1, GPS:7, Power:3 |
| seeed_wio_tracker_L1_eink-inkhud | | | | variants/nrf52840/seeed_wio_tracker_L1_eink | Display:7, Radio:9, Input:1, GPS:7, Power:3 |
| seeed_xiao_nrf52840_kit | Seeed Xiao NRF52840 Kit | 88 | XIAO_NRF52_KIT | variants/nrf52840/seeed_xiao_nrf52840_kit | Radio:19, Input:2, GPS:9, Power:6 |
| seeed_xiao_nrf52840_kit_i2c | | | | variants/nrf52840/seeed_xiao_nrf52840_kit | Radio:19, Input:2, GPS:9, Power:6 |
| t-echo | LILYGO T-Echo | 7 | T_ECHO | variants/nrf52840/t-echo | Display:7, Radio:8, GPS:7, Power:6, Connectivity/Other:1 |
| t-echo-inkhud | | | | variants/nrf52840/t-echo | Display:7, Radio:8, GPS:7, Power:6, Connectivity/Other:1 |
| t-echo-lite | LILYGO T-Echo Lite | 109 | T_ECHO_LITE | variants/nrf52840/t-echo-lite | Display:6, Radio:9, GPS:10, Power:8 |
| t-echo-plus | | | | variants/nrf52840/t-echo-plus | Display:8, Radio:8, GPS:7, Power:6 |
| thinknode_m1 | ThinkNode M1 | 89 | THINKNODE_M1 | variants/nrf52840/ELECROW-ThinkNode-M1 | Display:7, Radio:8, Input:1, GPS:8, Power:8 |
| thinknode_m1-inkhud | | | | variants/nrf52840/ELECROW-ThinkNode-M1 | Display:7, Radio:8, Input:1, GPS:8, Power:8 |
| thinknode_m3 | Elecrow ThinkNode M3 | 115 | THINKNODE_M3 | variants/nrf52840/ELECROW-ThinkNode-M3 | Radio:1, Input:2, GPS:8, Power:7, Connectivity/Other:1 |
| thinknode_m4 | | | | variants/nrf52840/ELECROW-ThinkNode-M4 | Radio:8, GPS:12, Power:7 |
| thinknode_m6 | ThinkNode M6 | 120 | THINKNODE_M6 | variants/nrf52840/ELECROW-ThinkNode-M6 | Radio:7, GPS:9, Power:9, Connectivity/Other:1 |
| tracker-t1000-e | Seeed SenseCAP T1000-E | 71 | TRACKER_T1000_E | variants/nrf52840/tracker-t1000-e | Display:1, Radio:8, Input:1, GPS:13, Power:5 |
| wio-sdk-wm1110 | | | | variants/nrf52840/wio-sdk-wm1110 | Radio:8, Input:1 |
| wio-t1000-s | | | | variants/nrf52840/wio-t1000-s | Radio:8, Input:1, GPS:12, Power:4 |
| wio-tracker-wm1110 | Seeed Wio WM1110 Tracker | 21 | WIO_WM1110 | variants/nrf52840/wio-tracker-wm1110 | Radio:8, Input:1, GPS:3 |
### rp2040
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| -------------------- | ------------------- | -------- | ----------- | ------------------------------------ | ------------------------------------------------ |
| catsniffer | | | | variants/rp2040/ec_catsniffer | Display:1, Radio:17, GPS:1 |
| challenger_2040_lora | | | | variants/rp2040/challenger_2040_lora | Radio:17, Input:1, Power:1 |
| feather_rp2040_rfm95 | | | | variants/rp2040/feather_rp2040_rfm95 | Radio:17, Input:1, Power:1 |
| nibble-rp2040 | | | | variants/rp2040/nibble_rp2040 | Radio:9, Input:1 |
| pico | Raspberry Pi Pico | 47 | RPI_PICO | variants/rp2040/rpipico | Radio:16, Input:1, Power:4 |
| pico_slowclock | | | | variants/rp2040/rpipico-slowclock | Display:2, Radio:16, Input:1, GPS:5, Power:4 |
| picow | Raspberry Pi Pico W | 47 | RPI_PICO | variants/rp2040/rpipicow | Radio:16, Input:1, Power:4, Connectivity/Other:1 |
| rak11310 | RAK WisBlock 11310 | 26 | RAK11310 | variants/rp2040/rak11310 | Radio:17, Input:1, Power:3, Connectivity/Other:1 |
| rp2040-lora | RP2040 LoRa | 30 | RP2040_LORA | variants/rp2040/rp2040-lora | Radio:18, Input:1, Power:1 |
| senselora_rp2040 | | | | variants/rp2040/senselora_rp2040 | Display:1, Radio:9, Input:1, Power:1 |
### rp2350
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| ----------- | ------------ | -------- | ------- | ------------------------- | ------------------------------------------------ |
| pico2 | | | | variants/rp2350/rpipico2 | Radio:16, Input:1, Power:4 |
| pico2w | | | | variants/rp2350/rpipico2w | Radio:16, Input:1, Power:4, Connectivity/Other:1 |
### stm32
| Environment | Display Name | HW Model | HW Slug | Variant Dir | Common Categories |
| --------------- | ------------ | -------- | ------- | ------------------------------ | ---------------------------------- |
| CDEBYTE_E77-MBL | | | | variants/stm32/CDEBYTE_E77-MBL | Display:1 |
| milesight_gs301 | | | | variants/stm32/milesight_gs301 | Display:1, Radio:1, Input:1 |
| rak3172 | | | | variants/stm32/rak3172 | Display:1 |
| russell | | | | variants/stm32/russell | Display:1, Radio:1, Input:1, GPS:4 |
| wio-e5 | | | | variants/stm32/wio-e5 | Display:1 |
## Representative Board Examples
### diy: `9m2ibr_aprs_lora_tracker`
- Variant directory: `variants/esp32/diy/9m2ibr_aprs_lora_tracker`
- Input examples:
- `BUTTON_PIN` = `15`
- Radio examples:
- `LORA_SCK` = `18`
- `LORA_MISO` = `19`
- `LORA_MOSI` = `23`
- `LORA_CS` = `5`
- `LORA_DIO0` = `26`
- `LORA_RESET` = `27`
- `LORA_DIO1` = `12`
- `LORA_DIO2` = `RADIOLIB_NC`
- GPS examples:
- `GPS_RX_PIN` = `16`
- `GPS_TX_PIN` = `17`
- Display examples:
- `HAS_SCREEN` = `1`
- I2C/SPI examples:
- `I2C_SDA` = `21`
- `I2C_SCL` = `22`
- Power examples:
- `BATTERY_PIN` = `35`
- `ADC_MULTIPLIER` = `2.01`
- `ADC_CHANNEL` = `ADC1_GPIO35_CHANNEL`
- `BATTERY_SENSE_RESOLUTION_BITS` = `ADC_RESOLUTION`
### esp32: `betafpv_2400_tx_micro`
- Variant directory: `variants/esp32/betafpv_2400_tx_micro`
- Input examples:
- `BUTTON_PIN` = `25`
- Radio examples:
- `LORA_SCK` = `18`
- `LORA_MISO` = `19`
- `LORA_MOSI` = `23`
- `LORA_CS` = `5`
- `RF95_FAN_EN` = `17`
- `USE_SX1280` = `1`
- `LORA_RESET` = `14`
- `SX128X_CS` = `5`
- I2C/SPI examples:
- `I2C_SDA` = `22`
- `I2C_SCL` = `32`
- Connectivity/Other examples:
- `HAS_NEOPIXEL` = `1`
### esp32-c3: `ai-c3`
- Variant directory: `variants/esp32c3/ai-c3`
- Input examples:
- `BUTTON_PIN` = `9`
- Radio examples:
- `USE_RF95` = `1`
- `LORA_SCK` = `4`
- `LORA_MISO` = `5`
- `LORA_MOSI` = `6`
- `LORA_CS` = `7`
- `LORA_DIO0` = `10`
- `LORA_DIO1` = `3`
- `LORA_RESET` = `2`
- GPS examples:
- `HAS_GPS` = `0`
- I2C/SPI examples:
- `I2C_SDA` = `SDA`
- `I2C_SCL` = `SCL`
### esp32-c6: `m5stack-unitc6l`
- Variant directory: `variants/esp32c6/m5stack_unitc6l`
- `custom_meshtastic_hw_model`: `111`
- `custom_meshtastic_hw_model_slug`: `M5STACK_C6L`
- `custom_meshtastic_architecture`: `esp32-c6`
- `custom_meshtastic_actively_supported`: `true`
- `custom_meshtastic_support_level`: `1`
- `custom_meshtastic_display_name`: `M5Stack Unit C6L`
- `custom_meshtastic_images`: `m5_c6l.svg`
- `custom_meshtastic_tags`: `M5Stack`
- Radio examples:
- `USE_SX1262` = `1`
- `LORA_MISO` = `22`
- `LORA_SCK` = `20`
- `LORA_MOSI` = `21`
- `LORA_CS` = `23`
- `LORA_RESET` = `RADIOLIB_NC`
- `LORA_DIO1` = `7`
- `LORA_BUSY` = `19`
- GPS examples:
- `HAS_GPS` = `1`
- `GPS_RX_PIN` = `4`
- `GPS_TX_PIN` = `5`
- Display examples:
- `SCREEN_TRANSITION_FRAMERATE` = `10`
- I2C/SPI examples:
- `I2C_SDA` = `10`
- `I2C_SCL` = `8`
- Connectivity/Other examples:
- `HAS_NEOPIXEL` = `1`
### esp32-s3: `CDEBYTE_EoRa-Hub`
- Variant directory: `variants/esp32s3/CDEBYTE_EoRa-Hub`
- Input examples:
- `BUTTON_PIN` = `0`
- Radio examples:
- `USE_LR1121` = `1`
- `LORA_SCK` = `9`
- `LORA_MOSI` = `10`
- `LORA_MISO` = `11`
- `LORA_RESET` = `12`
- `LORA_CS` = `8`
- `LORA_DIO9` = `13`
- `LR1121_IRQ_PIN` = `14`
- Display examples:
- `HAS_SCREEN` = `1`
- `USE_SSD1306` = `1`
- I2C/SPI examples:
- `I2C_SCL` = `17`
- `I2C_SDA` = `18`
- `I2C_SCL1` = `21`
- `I2C_SDA1` = `10`
- Power examples:
- `BATTERY_PIN` = `1`
- `ADC_CHANNEL` = `ADC1_GPIO1_CHANNEL`
- `ADC_MULTIPLIER` = `103.0`
- `ADC_ATTENUATION` = `ADC_ATTEN_DB_0`
- `ADC_CTRL` = `37`
- `ADC_CTRL_ENABLED` = `LOW`
### esp32s2: `nugget-s2-lora`
- Variant directory: `variants/esp32s2/nugget_s2_lora`
- Input examples:
- `BUTTON_PIN` = `0`
- Radio examples:
- `USE_RF95` = `1`
- `LORA_SCK` = `6`
- `LORA_MISO` = `8`
- `LORA_MOSI` = `10`
- `LORA_CS` = `13`
- `LORA_DIO0` = `16`
- `LORA_RESET` = `5`
- `LORA_DIO1` = `RADIOLIB_NC`
- I2C/SPI examples:
- `I2C_SDA` = `34`
- `I2C_SCL` = `36`
- Connectivity/Other examples:
- `HAS_NEOPIXEL` = `1`
### native: `buildroot`
- Variant directory: `variants/native/portduino-buildroot`
- GPS examples:
- `HAS_GPS` = `1`
- Display examples:
- `HAS_SCREEN` = `1`
- `USE_TFTDISPLAY` = `1`
### nrf52840: `ME25LS01-4Y10TD`
- Variant directory: `variants/nrf52840/ME25LS01-4Y10TD`
- Input examples:
- `BUTTON_PIN` = `(0 + 27)`
- Radio examples:
- `LORA_RESET` = `(32 + 11)`
- `LORA_DIO1` = `(32 + 12)`
- `LORA_DIO2` = `(32 + 10)`
- `LORA_SCK` = `PIN_SPI_SCK`
- `LORA_MISO` = `PIN_SPI_MISO`
- `LORA_MOSI` = `PIN_SPI_MOSI`
- `LORA_CS` = `PIN_SPI_NSS`
- `USE_LR1110` = `1`
- GPS examples:
- `HAS_GPS` = `0`
- `PIN_GPS_EN` = `-1`
- `GPS_EN_ACTIVE` = `HIGH`
- `PIN_GPS_RESET` = `-1`
- `GPS_VRTC_EN` = `-1`
- `GPS_SLEEP_INT` = `-1`
- `GPS_RTC_INT` = `-1`
- `GPS_RESETB_OUT` = `-1`
- I2C/SPI examples:
- `WIRE_INTERFACES_COUNT` = `1`
- `SPI_INTERFACES_COUNT` = `1`
- `PIN_SPI_MISO` = `(0 + 29)`
- `PIN_SPI_MOSI` = `(0 + 2)`
- `PIN_SPI_SCK` = `(32 + 15)`
- `PIN_SPI_NSS` = `(32 + 13)`
- Power examples:
- `BATTERY_PIN` = `-1`
- `ADC_MULTIPLIER` = `(2.0F)`
- `ADC_RESOLUTION` = `14`
- `BATTERY_SENSE_RESOLUTION_BITS` = `12`
### rp2040: `catsniffer`
- Variant directory: `variants/rp2040/ec_catsniffer`
- Radio examples:
- `USE_SX1262` = `1`
- `LORA_SCK` = `18`
- `LORA_MISO` = `16`
- `LORA_MOSI` = `19`
- `LORA_CS` = `17`
- `LORA_DIO0` = `5`
- `LORA_RESET` = `24`
- `LORA_DIO1` = `4`
- GPS examples:
- `HAS_GPS` = `0`
- Display examples:
- `HAS_SCREEN` = `0`
### rp2350: `pico2`
- Variant directory: `variants/rp2350/rpipico2`
- Input examples:
- `BUTTON_PIN` = `17`
- Radio examples:
- `USE_SX1262` = `1`
- `LORA_SCK` = `10`
- `LORA_MISO` = `12`
- `LORA_MOSI` = `11`
- `LORA_CS` = `3`
- `LORA_DIO0` = `RADIOLIB_NC`
- `LORA_RESET` = `15`
- `LORA_DIO1` = `20`
- Power examples:
- `EXT_NOTIFY_OUT` = `22`
- `BATTERY_PIN` = `26`
- `ADC_MULTIPLIER` = `3.1`
- `BATTERY_SENSE_RESOLUTION_BITS` = `ADC_RESOLUTION`
### stm32: `CDEBYTE_E77-MBL`
- Variant directory: `variants/stm32/CDEBYTE_E77-MBL`
- Display examples:
- `USE_STM32WLx` = `1`
## Intake Guidance For New Hardware
When using this context to add a new board, collect these inputs before generating files:
- PlatformIO environment name
- `custom_meshtastic_hw_model` and `custom_meshtastic_hw_model_slug`
- Display name and architecture
- Whether the board is actively supported and its support level
- Partition scheme, DFU requirement, and image/tag metadata if applicable
- Radio chip family and complete radio pin group
- Input/button/rotary/keyboard pins
- Display interface pins and driver-related macros
- GPS, power-management, I2C, SPI, storage, and auxiliary peripheral definitions
- Any board-specific initialization that requires `variant.cpp` or extra variant hooks
## Inherited Defaults Note
Some architecture families rely on BSP (Board Support Package) or base-environment defaults rather than declaring every pin or capability macro explicitly in `variant.h`. When adding a new board for one of these families, check the relevant BSP headers before assuming a missing define means a feature is absent.
Directories scanned that had no `variant.h` (relying entirely on BSP/base-environment): diy: 4, esp32: 3, esp32s3: 1
### nrf52840
- VARIANT_MCK — nRF52 BSP clock constant (e.g., 64000000ul); always inherited from BSP unless overridden.
- USE_LFXO — low-frequency crystal oscillator selection; declared locally only when the board uses LFXO rather than the RC oscillator.
- PIN*SPI*_ / PIN*SPI1*_ — SPI bus pin numbers come from the BSP variant table; boards override only when the LoRa radio or display uses non-default SPI routing.
- WIRE_INTERFACES_COUNT / SPI_INTERFACES_COUNT — bus count comes from BSP; explicitly set only when the board deviates.
- LED_BLUE / PIN_LED1 / PINS_COUNT / NUM_DIGITAL_PINS — standard BSP pin-table entries inherited from the nRF52 Arduino core.
### rp2040
- PIN*SPI*\* — primary SPI pins come from the RP2040 Arduino BSP; most boards declare them explicitly, but the defaults align with the Pico pin assignments.
- NUM_DIGITAL_PINS / NUM_ANALOG_INPUTS — Arduino BSP counts; rarely overridden locally.
### stm32
- Radio and pin assignments for STM32WL targets are largely internal to the WL SoC and declared via STM32 HAL/BSP headers; variant.h files are minimal.
- USE_STM32WLx is typically the only explicit define; all other radio config comes from the BSP.
### native
- The native/Portduino target uses runtime configuration rather than compile-time pin defines; variant.h only sets display and GPS stubs.
## Cautions
- Some boards rely on architecture defaults rather than declaring every field locally.
- Some board families expose multiple environments or display variants that share one hardware model.
- Source materials such as schematics still need human verification before new pin mappings are trusted.
- This document is a starting context artifact, not proof that a new board definition is safe to merge.
+14 -3
View File
@@ -70,6 +70,17 @@ def esp32_create_combined_bin(source, target, env):
env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
# Enable Newlib Nano formatting to save space
# ...but allow printf float support (compromise)
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
esp32_kind = env.GetProjectOption("custom_esp32_kind")
if esp32_kind == "esp32":
# Free up some IRAM by removing auxiliary SPI flash chip drivers.
# Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
env.Append(
LINKFLAGS=[
"-Wl,--wrap=esp_flash_chip_gd",
"-Wl,--wrap=esp_flash_chip_issi",
"-Wl,--wrap=esp_flash_chip_winbond",
]
)
else:
# For newer ESP32 targets, using newlib nano works better.
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
# force linker response file instead of command line arguments
Import("env")
def wrap_with_tempfile(command_key):
command = env.get(command_key)
if not command or not isinstance(command, str):
return
if "TEMPFILE(" in command:
return
env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
# Force SCons to spill long commands into response files on this target.
env.Replace(MAXLINELENGTH=8192)
for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
wrap_with_tempfile(key)
-140
View File
@@ -1,140 +0,0 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
#
# post:extra_scripts/nrf54l15_linker.py
#
# Fix for Zephyr two-pass link on nRF54L15:
# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
# the SCons dependency chain is broken (final_ld_script Command never runs).
# This script adds a PreAction on the final firmware binary that runs the gcc
# preprocessing command directly (extracted from build.ninja) to generate
# zephyr/linker.cmd before the link step.
#
# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
import os
import re
import subprocess
Import("env")
if env.get("PIOENV") != "nrf54l15dk":
pass # Only for the nrf54l15dk environment
else:
def _extract_gcc_command(ninja_build):
"""Parse build.ninja to find the gcc -E command that generates linker.cmd.
The rule format depends on the host:
Windows (CMake's RunCMake wraps every command):
COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
POSIX (Linux/macOS — no wrapper):
COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
"""
in_rule = False
with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
for line in f:
# Detect start of the linker.cmd custom command rule
if not in_rule:
if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
in_rule = True
continue
stripped = line.strip()
if not stripped.startswith("COMMAND = "):
continue
command_val = stripped[len("COMMAND = ") :]
# On Windows the value is wrapped in `cmd.exe /C "..."` — strip
# the wrapper. On POSIX hosts the inner sequence is the value
# itself (no quoting layer).
m = re.search(r'/C\s+"(.*)"\s*$', command_val)
inner = m.group(1) if m else command_val
parts = inner.split(" && ")
cwd = None
gcc_cmd = None
for part in parts:
part = part.strip()
if part.startswith("cd /D "): # Windows form
cwd = part[len("cd /D ") :]
elif part.startswith("cd "): # POSIX form
cwd = part[len("cd ") :]
elif "arm-none-eabi-gcc" in part:
gcc_cmd = part
if not gcc_cmd:
raise RuntimeError(
"nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
% inner[:400]
)
return gcc_cmd, cwd
raise RuntimeError(
"nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
)
def _generate_linker_cmd(target, source, env):
"""Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
build_dir = env.subst("$BUILD_DIR")
zephyr_dir = os.path.join(build_dir, "zephyr")
linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
if os.path.exists(linker_cmd):
return # Already present — nothing to do
ninja_build = os.path.join(build_dir, "build.ninja")
if not os.path.exists(ninja_build):
raise RuntimeError(
"nRF54L15 linker fix: build.ninja not found at %s\n"
"Run a full build first so CMake generates the Ninja files."
% ninja_build
)
gcc_cmd, cwd = _extract_gcc_command(ninja_build)
run_cwd = cwd if cwd else zephyr_dir
print(
"==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
)
# gcc_cmd comes verbatim from our own build.ninja (never user input) and
# contains Windows-style paths with spaces that cannot be safely argv-split
# with shlex, so we run it via the platform shell. nosec/nosemgrep below
# acknowledge this deliberate, scoped use of shell=True.
result = subprocess.run( # nosec B602
gcc_cmd,
shell=True, # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
cwd=run_cwd,
capture_output=True,
text=True,
)
if result.returncode != 0:
print("GCC stdout:", result.stdout[:2000])
print("GCC stderr:", result.stderr[:2000])
raise RuntimeError(
"nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
% result.returncode
)
if not os.path.exists(linker_cmd):
raise RuntimeError(
"nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
% linker_cmd
)
print("==> linker.cmd generated successfully")
# Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
prog = env.get("PIOMAINPROG")
if prog:
env.AddPreAction(prog, _generate_linker_cmd)
else:
print(
"[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
)
env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)
-6
View File
@@ -7,12 +7,6 @@ __pycache__/
dist/
build/
# Persistent device-log capture (recorder + Datadog cursor).
# Cross-session JSONL streams written by the autouse Recorder singleton
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
# fixture truncate doesn't touch it.
.mtlog/
# Test harness artifacts
tests/report.html
tests/junit.xml
-217
View File
@@ -1,217 +0,0 @@
{
"title": "Meshtastic Firmware — Recorder Stream",
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
"widgets": [
{
"definition": {
"title": "Free heap (bytes)",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"queries": [
{
"name": "free_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "label": "bytes" }
}
},
{
"definition": {
"title": "Heap slope (bytes/min) — last 1h",
"type": "query_value",
"precision": 0,
"requests": [
{
"queries": [
{
"name": "slope",
"data_source": "metrics",
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
"aggregator": "avg"
}
],
"response_format": "scalar"
}
],
"conditional_formats": [
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
]
}
},
{
"definition": {
"title": "Total heap (bytes)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "total_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Battery level (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "battery",
"data_source": "metrics",
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "min": "0", "max": "105" }
}
},
{
"definition": {
"title": "Air utilization (TX %)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "airutil",
"data_source": "metrics",
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Channel utilization (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "chutil",
"data_source": "metrics",
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Log volume by level",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"response_format": "timeseries",
"display_type": "bars",
"queries": [
{
"name": "log_count",
"data_source": "logs",
"indexes": ["*"],
"compute": { "aggregation": "count" },
"search": { "query": "service:meshtastic-firmware" },
"group_by": [
{
"facet": "@level",
"limit": 10,
"sort": { "order": "desc", "aggregation": "count" }
}
]
}
]
}
]
}
},
{
"definition": {
"title": "Recent ERROR / CRIT firmware logs",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "@port", "width": "auto" },
{ "field": "@level", "width": "auto" },
{ "field": "@thread", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
},
{
"definition": {
"title": "Recorder marker events",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware @level:MARK",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
}
],
"template_variables": [
{
"name": "port",
"prefix": "port",
"available_values": [],
"default": "*"
},
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
],
"layout_type": "ordered",
"notify_list": [],
"reflow_type": "auto"
}
-389
View File
@@ -1,389 +0,0 @@
#!/usr/bin/env python3
"""Forward selected recorder JSONL streams to Datadog.
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
Logs Intake API and telemetry numerics to the Metrics v2 series API.
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
duplicate rows already shipped from the current live files.
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
If the recorder rotates before this process drains the live file, or the
forwarder is down across a rotation, those older rows are skipped.
Usage:
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
./scripts/mtlog_to_datadog.py --once # catch up + exit
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
The forwarder is a separate process by design — a Datadog outage or
auth failure must not backpressure the recorder. We exit non-zero on
fatal config errors (missing API key) and keep retrying on transient
network/HTTP errors.
"""
from __future__ import annotations
import argparse
import json
import os
import socket
import sys
import time
from pathlib import Path
from typing import Any, Iterator
try:
import requests
except ImportError:
print(
"requests is required. Install it in the mcp-server venv: "
"uv pip install requests",
file=sys.stderr,
)
sys.exit(2)
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
_METRICS_TPL = "https://api.{site}/api/v2/series"
_LOG_BATCH = 50
_METRICS_BATCH = 100
_MAX_RETRIES = 5
_RETRY_BASE_S = 1.5
# --- streaming JSONL with byte-position cursor -------------------------
class _StreamReader:
"""Reads a single rotating JSONL with cursor-based resume.
This tails only the live `.jsonl` file. The recorder rotates files
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
the live file shrinks abruptly. We detect that via inode change OR live
size < cursor position, and reset the live-file cursor to 0.
"""
def __init__(self, path: Path, cursor: dict[str, Any]):
self.path = path
self.cursor = cursor
def _state(self) -> tuple[int, int]:
"""Return (inode, size) for the live file. (0, 0) if missing."""
try:
st = self.path.stat()
return (st.st_ino, st.st_size)
except FileNotFoundError:
return (0, 0)
def iter_new_records(self) -> Iterator[dict[str, Any]]:
ino, size = self._state()
last_ino = self.cursor.get("ino")
last_pos = int(self.cursor.get("pos") or 0)
if ino == 0:
return
if last_ino is not None and last_ino != ino:
# Rotation happened. Start over.
last_pos = 0
if last_pos > size:
# Live file truncated/shrunk under us — recorder rotated.
last_pos = 0
try:
with self.path.open("r", encoding="utf-8") as fh:
fh.seek(last_pos)
for line in fh:
line = line.rstrip("\n")
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
last_pos = fh.tell()
except FileNotFoundError:
return
self.cursor["ino"] = ino
self.cursor["pos"] = last_pos
def _load_cursor(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except (OSError, json.JSONDecodeError):
return {}
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(data, separators=(",", ":")))
tmp.replace(path)
# --- Datadog clients ---------------------------------------------------
class _DDSession:
"""Pool one HTTPS session, share retry logic."""
def __init__(self, api_key: str, site: str, hostname: str) -> None:
self.api_key = api_key
self.site = site
self.hostname = hostname
self.session = requests.Session()
self.session.headers.update(
{
"DD-API-KEY": api_key,
"Content-Type": "application/json",
}
)
def _post(self, url: str, payload: Any) -> bool:
for attempt in range(_MAX_RETRIES):
try:
resp = self.session.post(url, json=payload, timeout=30)
except requests.RequestException as e:
_wait_retry(attempt, f"network error: {e}")
continue
if 200 <= resp.status_code < 300:
return True
if resp.status_code in (408, 429, 500, 502, 503, 504):
_wait_retry(
attempt,
f"HTTP {resp.status_code} retrying",
)
continue
print(
f"datadog refused: {resp.status_code} {resp.text[:200]}",
file=sys.stderr,
)
return False
return False
def send_logs(self, records: list[dict[str, Any]]) -> int:
if not records:
return 0
url = _LOG_INTAKE_TPL.format(site=self.site)
sent = 0
for i in range(0, len(records), _LOG_BATCH):
batch = records[i : i + _LOG_BATCH]
if self._post(url, batch):
sent += len(batch)
return sent
def send_metrics(self, series: list[dict[str, Any]]) -> int:
if not series:
return 0
url = _METRICS_TPL.format(site=self.site)
sent = 0
for i in range(0, len(series), _METRICS_BATCH):
batch = series[i : i + _METRICS_BATCH]
if self._post(url, {"series": batch}):
sent += len(batch)
return sent
def _wait_retry(attempt: int, reason: str) -> None:
wait = _RETRY_BASE_S * (2**attempt)
print(
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
file=sys.stderr,
)
time.sleep(wait)
# --- record → datadog payload ------------------------------------------
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
line = rec.get("line") or ""
tags = [
f"role:{rec.get('role')}",
f"port:{rec.get('port')}",
]
level = rec.get("level")
if level:
tags.append(f"level:{level}")
tag = rec.get("tag")
if tag:
tags.append(f"thread:{tag}")
return {
"ddsource": "meshtastic-firmware",
"service": "meshtastic-firmware",
"hostname": host,
"message": line,
"ddtags": ",".join(t for t in tags if t and "None" not in t),
"timestamp": int((rec.get("ts") or time.time()) * 1000),
"level": level,
}
def _telemetry_record_to_metrics(
rec: dict[str, Any], host: str
) -> list[dict[str, Any]]:
fields = rec.get("fields") or {}
if not isinstance(fields, dict):
return []
variant = rec.get("variant") or "unknown"
ts = int(rec.get("ts") or time.time())
out: list[dict[str, Any]] = []
tags = []
if rec.get("port"):
tags.append(f"port:{rec['port']}")
if rec.get("role"):
tags.append(f"role:{rec['role']}")
if rec.get("from_node"):
tags.append(f"from_node:{rec['from_node']}")
tags.append(f"variant:{variant}")
for field, value in fields.items():
if not isinstance(value, (int, float)) or isinstance(value, bool):
continue
metric = f"mesh.{variant}.{_metric_safe(field)}"
out.append(
{
"metric": metric,
"type": 3, # GAUGE
"points": [{"timestamp": ts, "value": float(value)}],
"tags": tags,
"resources": [{"type": "host", "name": host}],
}
)
return out
def _metric_safe(name: str) -> str:
# Lowercase, replace non-alnum with underscore for safe metric names.
return "".join(c.lower() if c.isalnum() else "_" for c in name)
# --- main loop ---------------------------------------------------------
def run(
log_dir: Path,
*,
once: bool,
since_seconds: float | None,
poll_interval: float,
dd: _DDSession,
) -> int:
cursor_path = log_dir / ".dd-cursor.json"
cursors = _load_cursor(cursor_path)
# `--since` overrides cursor: rewind to (now-since) timestamp.
# We can't seek by timestamp directly (cursor is byte position), so
# we just reset cursors to 0 and let the time filter in iter_new
# drop older records.
cutoff_ts: float | None = None
if since_seconds is not None:
cursors = {}
cutoff_ts = time.time() - since_seconds
sent_total = {"logs": 0, "telemetry": 0}
while True:
# logs.jsonl → DD logs
log_cursor = cursors.setdefault("logs", {})
log_batch: list[dict[str, Any]] = []
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
log_batch.append(_log_record_to_dd(rec, dd.hostname))
if log_batch:
n = dd.send_logs(log_batch)
sent_total["logs"] += n
print(f"logs: sent {n}/{len(log_batch)}")
# telemetry.jsonl → DD metrics
telem_cursor = cursors.setdefault("telemetry", {})
metric_series: list[dict[str, Any]] = []
for rec in _StreamReader(
log_dir / "telemetry.jsonl", telem_cursor
).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
if metric_series:
n = dd.send_metrics(metric_series)
sent_total["telemetry"] += n
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
_save_cursor(cursor_path, cursors)
if once:
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
return 0
time.sleep(poll_interval)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--log-dir",
default=str(_DEFAULT_LOG_DIR),
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--once", action="store_true", help="Catch up then exit")
mode.add_argument(
"--tail",
action="store_true",
help="Daemon: poll forever (default)",
)
parser.add_argument(
"--since",
type=float,
default=None,
help="Backfill last N seconds. Resets cursor.",
)
parser.add_argument(
"--poll-interval",
type=float,
default=5.0,
help="Seconds between tail polls (default 5)",
)
parser.add_argument(
"--site",
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
help=(
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
"Override via DD_SITE env var or this flag."
),
)
parser.add_argument(
"--host",
default=socket.gethostname(),
help="Hostname tag (default: socket.gethostname())",
)
args = parser.parse_args(argv)
api_key = os.environ.get("DD_API_KEY")
if not api_key:
print("DD_API_KEY env var required.", file=sys.stderr)
return 2
log_dir = Path(args.log_dir)
if not log_dir.exists():
print(
f"log dir {log_dir} does not exist — start the mcp-server first.",
file=sys.stderr,
)
return 2
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
once = args.once and not args.tail
return run(
log_dir,
once=once,
since_seconds=args.since,
poll_interval=args.poll_interval,
dd=dd,
)
if __name__ == "__main__":
sys.exit(main())
-382
View File
@@ -1,382 +0,0 @@
"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
The fixture pipeline is two-stage:
1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
protobuf with fresh wall-clock timestamps.
This module exposes `push_fake_nodedb(...)`, the MCP tool that:
- target="portduino": compiles the JSONL into the device's prefs dir on
the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
- target="hardware": compiles to a temp file, then streams it over the
XModem protocol (via the meshtastic SerialInterface/BLEInterface +
`meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
device. Triggers a reboot so the firmware loads the new state on next
boot.
XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
* 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
* CRC16-CCITT (poly 0x1021, init 0x0000).
* SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
`FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
* SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
* EOT after the last chunk flushes + closes the file on-device.
Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
in the .github/copilot-instructions.md "never do these without asking" list).
"""
from __future__ import annotations
import dataclasses
import hashlib
import pathlib
import queue
import shutil
import subprocess
import sys
import tempfile
import time
from typing import Any, Literal
from .connection import connect, is_tcp_port
# Resolve repo root so the tool works regardless of mcp-server cwd.
_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
_XMODEM_CHUNK = 128
_XMODEM_SUB = 0x1A
_ACK_TIMEOUT_INIT_S = 5.0
_ACK_TIMEOUT_CHUNK_S = 2.0
_MAX_CHUNK_RETRIES = 5
_VALID_SIZES = (250, 500, 1000, 2000)
class FixtureError(RuntimeError):
"""Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
# ---------------------------------------------------------------------------
# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
# Hand-rolled to avoid the optional `crcmod` dep.
# ---------------------------------------------------------------------------
def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
crc = init
for b in data:
crc ^= b << 8
for _ in range(8):
if crc & 0x8000:
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
else:
crc = (crc << 1) & 0xFFFF
return crc
# ---------------------------------------------------------------------------
# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
# doesn't have to duplicate the proto-encoding logic.
# ---------------------------------------------------------------------------
def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
if not _COMPILE_SCRIPT.is_file():
raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
cmd = [
sys.executable,
str(_COMPILE_SCRIPT),
"--in",
str(jsonl_path),
"--out",
str(out_path),
]
try:
subprocess.run(cmd, check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as exc:
raise FixtureError(
f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
f" stdout: {exc.stdout}\n stderr: {exc.stderr}"
) from exc
def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
if custom is not None:
p = pathlib.Path(custom).expanduser().resolve()
if not p.is_file():
raise FixtureError(f"custom_seed_jsonl not found: {p}")
return p
p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
if not p.is_file():
raise FixtureError(
f"missing committed seed at {p}. "
f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
)
return p
# ---------------------------------------------------------------------------
# Portduino push — file copy into ~/.portduino/<config>/prefs/
# ---------------------------------------------------------------------------
def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
home = pathlib.Path.home()
return home / ".portduino" / config_name / "prefs"
def _push_portduino(
size: int,
jsonl: pathlib.Path,
portduino_config: str,
backup_existing: bool,
) -> dict[str, Any]:
prefs = _portduino_prefs_dir(portduino_config)
prefs.mkdir(parents=True, exist_ok=True)
target = prefs / "nodes.proto"
backed_up_to: str | None = None
if backup_existing and target.is_file():
ts = int(time.time())
backup = prefs / f"nodes.proto.bak.{ts}"
shutil.move(str(target), str(backup))
backed_up_to = str(backup)
_compile_proto(jsonl, target)
raw = target.read_bytes()
return {
"transport": "portduino",
"path": str(target),
"bytes": len(raw),
"sha256": hashlib.sha256(raw).hexdigest(),
"jsonl_source": str(jsonl),
"backed_up_to": backed_up_to,
}
# ---------------------------------------------------------------------------
# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
# ---------------------------------------------------------------------------
@dataclasses.dataclass
class _AckEvent:
control: int
seq: int
def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
try:
return q.get(timeout=timeout_s)
except queue.Empty as exc:
raise FixtureError(
f"XModem response timeout after {timeout_s:.1f}s — device not responding"
) from exc
def _push_hardware(
size: int,
jsonl: pathlib.Path,
port: str | None,
reboot_after: bool,
) -> dict[str, Any]:
# Lazy imports so the module loads even when the meshtastic deps aren't
# available (e.g. CI in a Python env without the package installed).
try:
from meshtastic.protobuf import mesh_pb2, xmodem_pb2
from pubsub import pub
except ImportError as exc: # pragma: no cover — dep missing
raise FixtureError(
f"hardware push requires the meshtastic + pypubsub packages: {exc}"
) from exc
if is_tcp_port(port):
raise FixtureError(
"hardware push over TCP/portduino is not supported — use "
"target='portduino' to drop the fixture directly into the prefs dir."
)
# Compile the fixture to a temp file with fresh timestamps.
with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
proto_path = pathlib.Path(tf.name)
try:
_compile_proto(jsonl, proto_path)
payload = proto_path.read_bytes()
finally:
proto_path.unlink(missing_ok=True)
sha256 = hashlib.sha256(payload).hexdigest()
total_bytes = len(payload)
# Subscribe to XModem responses BEFORE we open the interface, so we don't
# race the first ACK that arrives during the SOH/seq=0 handshake.
#
# NB: the signature MUST declare every kwarg pypubsub will see for this
# topic, or pubsub locks the topic spec to a smaller set (whichever
# subscribe arrives first) and then *rejects* the meshtastic library's
# publish call with `SenderUnknownMsgDataError: unknown ... interface`.
# The meshtastic lib publishes both `packet=` and `interface=`
# (mesh_interface.py:1389-1395), so both must appear here.
response_q: "queue.Queue[_AckEvent]" = queue.Queue()
def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
if packet is None:
return
response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
chunks_sent = 0
retried = 0
rebooted = False
XMC = xmodem_pb2.XModem.Control
try:
with connect(port=port) as iface:
# 1) Send the filename (SOH, seq=0).
init_pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=0,
buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(
f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
f"(got control={ack.control}, expected ACK). "
f"Filesystem full or permissions issue?"
)
# 2) Stream the payload in 128 B chunks.
for offset in range(0, total_bytes, _XMODEM_CHUNK):
chunk = payload[offset : offset + _XMODEM_CHUNK]
if len(chunk) < _XMODEM_CHUNK:
# Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
# become part of the file on-device, but nanopb ignores
# bytes past the end of the top-level message.
chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
seq = ((offset // _XMODEM_CHUNK) + 1) % 256
# Retry loop on NAK / timeout.
attempts = 0
while True:
pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=seq,
buffer=chunk,
crc16=_crc16_ccitt(chunk),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control == XMC.Value("ACK"):
chunks_sent += 1
break
if ack.control == XMC.Value("NAK"):
attempts += 1
retried += 1
if attempts >= _MAX_CHUNK_RETRIES:
# Abort: send CAN so the firmware removes the half-
# written file via FSCom.remove(filename).
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(
control=XMC.Value("CAN")
)
)
)
raise FixtureError(
f"chunk seq={seq} NAK'd {attempts} times; "
f"aborted transfer (file removed on-device)."
)
continue # retry the same chunk
raise FixtureError(
f"unexpected XModem control={ack.control} on seq={seq}"
)
# 3) Tell the device we're done.
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
)
)
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(f"EOT not ACKed (got control={ack.control})")
# 4) Reboot so loadFromDisk picks up the new file.
if reboot_after:
iface.localNode.reboot(secs=1)
rebooted = True
finally:
try:
pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
except Exception:
pass
return {
"transport": "hardware",
"port": port,
"filename_on_device": _DEFAULT_NODES_FILENAME,
"bytes": total_bytes,
"chunks_sent": chunks_sent,
"retried": retried,
"sha256": sha256,
"jsonl_source": str(jsonl),
"rebooted": rebooted,
}
# ---------------------------------------------------------------------------
# Public entry point — registered as an MCP tool in server.py.
# ---------------------------------------------------------------------------
def push_fake_nodedb(
size: int,
target: Literal["portduino", "hardware"] = "portduino",
*,
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
Args:
size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
"hardware" (XModem upload to /prefs/nodes.proto + reboot).
port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
or BLE identifier. TCP endpoints are rejected — use target="portduino"
instead.
portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
if present, so you can roll back.
confirm: required True for target="hardware" (writes flash + reboots).
reboot_after: hardware only. If True, send a 1-second reboot after the
final ACK so loadFromDisk picks up the new file at next boot.
custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
test scenario.
Returns:
dict with transport, bytes, sha256, etc. — depends on target.
"""
if size not in _VALID_SIZES:
raise FixtureError(
f"size must be one of {_VALID_SIZES}; got {size!r}. "
f"Add a new committed seed if you need a different cardinality."
)
jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
if target == "portduino":
return _push_portduino(size, jsonl, portduino_config, backup_existing)
if target == "hardware":
if not confirm:
raise FixtureError(
"hardware push writes flash and triggers a reboot — pass confirm=True."
)
if not port:
raise FixtureError(
"target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
)
return _push_hardware(size, jsonl, port, reboot_after)
raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")
+1 -44
View File
@@ -108,33 +108,18 @@ def build(
env: str,
with_manifest: bool = True,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Run `pio run -e <env>` and return artifact paths.
`userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
into userPrefs.jsonc for this build only. File is restored byte-for-byte
on exit. Use `userprefs_set()` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
`{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
prefix on every log line. Combines with the recorder so heap shows
up at log cadence (much higher resolution than the ~60 s LocalStats
packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
expand to bare `-D<NAME>` (presence-only flags).
"""
args = ["run", "-e", env]
if with_manifest:
args.extend(["-t", "mtjson"])
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
args,
timeout=pio.TIMEOUT_BUILD,
check=False,
extra_env=extra_env,
)
result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
return {
"exit_code": result.returncode,
"artifacts": [str(p) for p in _artifacts_for(env)],
@@ -142,27 +127,9 @@ def build(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
"""Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
"-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
the flag entirely. Other types stringify."""
parts: list[str] = []
for key, value in build_flags.items():
if value is False or value is None:
continue
if value is True:
parts.append(f"-D{key}")
else:
parts.append(f"-D{key}={value}")
if not parts:
return {}
return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
def clean(env: str) -> dict[str, Any]:
"""Run `pio run -e <env> -t clean`."""
result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -179,29 +146,20 @@ def flash(
port: str,
confirm: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""`pio run -e <env> -t upload --upload-port <port>`. All architectures.
`userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
that pio performs will pick up the injected values.
`build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
is exported for the rebuild-before-upload, so the uploaded firmware
actually carries the flags. Without this propagation, `pio run -t upload`
would relink without the env var and silently drop them. Common use:
`build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
"""
_require_confirm(confirm, "flash")
_reject_native_env(env, "flash")
connection.reject_if_tcp(port, "flash")
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
["run", "-e", env, "-t", "upload", "--upload-port", port],
timeout=pio.TIMEOUT_UPLOAD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
@@ -209,7 +167,6 @@ def flash(
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
-410
View File
@@ -1,410 +0,0 @@
"""Read-side queries over the recorder's JSONL streams.
Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
loads a whole file. Time-bound queries short-circuit as soon as `ts`
exceeds the requested end. The recorder writes monotonically, so a
forward scan is cheap; we don't need an index.
All time arguments accept:
- epoch seconds (int/float)
- relative strings: "-15m", "-2h", "-3d", "now"
- ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
treated as UTC)
Tools that return data ALWAYS cap their output (max_lines / max_points
/ max), and report whether more matched than was returned.
"""
from __future__ import annotations
import gzip
import json
import re
import statistics
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterator
from .recorder.recorder import get_recorder
_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
_REGEX_PREVIEW_MAX = 100
_REGEX_PREVIEW_TRUNCATE = 97
def _parse_time(value: Any, *, now: float | None = None) -> float:
"""Coerce to epoch seconds. Defaults `now` to `time.time()`."""
if value is None:
return time.time()
if isinstance(value, (int, float)):
return float(value)
if not isinstance(value, str):
raise ValueError(f"invalid time: {value!r}")
s = value.strip().lower()
if s in ("", "now"):
return time.time() if now is None else now
m = _REL_RE.match(s)
if m:
n = float(m.group(1))
unit = m.group(2)
secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
base = time.time() if now is None else now
return base - secs
# Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
try:
if s.endswith("z"):
s = s[:-1] + "+00:00"
dt = datetime.fromisoformat(s)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.timestamp()
except ValueError as e:
raise ValueError(f"unparseable time: {value!r}") from e
def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
"""Stream records in chronological order: rotated archives first
(oldest → newest by lex sort, which is chronological for our
`YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
last. The "keep last N" pop-front logic in the window queries
relies on records arriving in time order across files.
"""
files: list[Path] = []
# Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
files.append(archive)
if path.exists():
files.append(path)
for f in files:
opener = gzip.open if f.suffix == ".gz" else open
try:
with opener(f, "rt", encoding="utf-8") as fh: # type: ignore[arg-type]
for line in fh:
line = line.strip()
if not line:
continue
try:
rec = json.loads(line)
except json.JSONDecodeError:
continue
ts = rec.get("ts")
if not isinstance(ts, (int, float)):
continue
if ts < since:
continue
if ts > until:
# Records are append-monotonic within a file, so
# the rest of this file is also past `until`.
# Archives can still overlap each other, so only
# short-circuit this file, not the whole scan.
break
yield rec
except (FileNotFoundError, OSError):
continue
# -- queries ------------------------------------------------------------
def logs_window(
start: Any = "-15m",
end: Any = "now",
*,
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines, filtered.
`level` accepts a single level name or pipe-separated set
("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
`line` field. Returns the last `max_lines` matches.
"""
s = _parse_time(start)
e = _parse_time(end)
levels = _split_set(level)
if grep:
try:
grep_re = re.compile(grep)
except re.error as exc:
preview = (
grep
if len(grep) <= _REGEX_PREVIEW_MAX
else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
)
raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
else:
grep_re = None
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
if levels and rec.get("level") not in levels:
continue
if tag and rec.get("tag") != tag:
continue
if port and rec.get("port") != port:
continue
if grep_re and not grep_re.search(rec.get("line") or ""):
continue
matched += 1
out.append(rec)
if len(out) > max_lines:
out.pop(0) # keep the most recent N
return {
"lines": out,
"total_matched": matched,
"dropped": max(0, matched - max_lines),
"window": {"start": s, "end": e},
}
def telemetry_timeline(
window: Any = "1h",
*,
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Timeseries of one telemetry field, downsampled.
`field` matches both the protobuf snake_case name (`free_heap`,
`heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
Server-side bucket-mean downsamples to ≤ `max_points`. Returns
`slope_per_min` (linear regression slope, units/min) so a leak
detector can read one number.
"""
end = time.time()
if isinstance(window, (int, float)):
# Numeric `window` is a duration in seconds — "last N seconds".
# Without this branch, `_parse_time(-N)` would treat -N as an
# absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
# producing a wildly negative `start` and matching nothing.
start = end - float(window)
elif isinstance(window, str) and not window.startswith("-"):
# Bare string like "1h" is sugar for "-1h".
start = _parse_time(f"-{window}", now=end)
else:
start = _parse_time(window, now=end)
base = get_recorder().base_dir
raw: list[tuple[float, float]] = []
field_aliases = _field_aliases(field)
for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
if rec.get("variant") != variant:
continue
if port and rec.get("port") != port:
continue
fields = rec.get("fields") or {}
value: Any = None
for alias in field_aliases:
if alias in fields:
value = fields[alias]
break
if not isinstance(value, (int, float)):
continue
raw.append((float(rec["ts"]), float(value)))
if not raw:
return {
"points": [],
"samples": 0,
"min": None,
"max": None,
"slope_per_min": None,
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
points = _downsample(raw, max_points=max_points)
values = [v for _, v in raw]
return {
"points": [{"ts": ts, "value": v} for ts, v in points],
"samples": len(raw),
"min": min(values),
"max": max(values),
"slope_per_min": _slope_per_min(raw),
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
def packets_window(
start: Any = "-5m",
end: Any = "now",
*,
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
portnums = _split_set(portnum)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
if portnums and rec.get("portnum") not in portnums:
continue
if from_node and str(rec.get("from_node")) != str(from_node):
continue
if to_node and str(rec.get("to_node")) != str(to_node):
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"packets": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def events_window(
start: Any = "-1h",
end: Any = "now",
*,
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
kinds = _split_set(kind)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
if kinds and rec.get("kind") not in kinds:
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"events": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def export(
start: Any,
end: Any,
dest_dir: str,
*,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of each requested stream into `dest_dir`.
For a notebook, a bug report, or a Datadog backfill. Output files
are uncompressed JSONL (callers gzip themselves if they want to).
"""
s = _parse_time(start)
e = _parse_time(end)
selected = streams or ["logs", "telemetry", "packets", "events"]
dest = Path(dest_dir)
dest.mkdir(parents=True, exist_ok=True)
base = get_recorder().base_dir
paths: dict[str, str] = {}
for stream in selected:
src = base / f"{stream}.jsonl"
if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
continue
out_path = dest / f"{stream}.jsonl"
n = 0
with out_path.open("w", encoding="utf-8") as fh:
for rec in _iter_jsonl(src, since=s, until=e):
fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
n += 1
paths[stream] = str(out_path)
paths[f"{stream}_count"] = str(n)
return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
# -- helpers ------------------------------------------------------------
def _split_set(value: str | None) -> set[str] | None:
if not value:
return None
return {v.strip() for v in value.split("|") if v.strip()}
def _field_aliases(field: str) -> list[str]:
"""Accept snake_case OR camelCase, plus a few legacy aliases."""
snake = field
camel = _snake_to_camel(field)
aliases = {snake, camel}
# Old protobuf fields (pre-LocalStats) used different names
legacy = {
"free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
"heap_free_bytes": [
"heap_free_bytes",
"heapFreeBytes",
"free_heap",
"freeHeap",
],
"total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
"heap_total_bytes": [
"heap_total_bytes",
"heapTotalBytes",
"total_heap",
"totalHeap",
],
}
if field in legacy:
aliases.update(legacy[field])
return list(aliases)
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _downsample(
points: list[tuple[float, float]], *, max_points: int
) -> list[tuple[float, float]]:
if len(points) <= max_points:
return points
# Even-bucket mean. Preserves shape better than nth-sample picking.
n = len(points)
bucket = n / max_points
out: list[tuple[float, float]] = []
i = 0
for k in range(max_points):
end = int((k + 1) * bucket)
end = min(end, n)
if end <= i:
continue
chunk = points[i:end]
ts = chunk[len(chunk) // 2][0]
val = statistics.fmean(v for _, v in chunk)
out.append((ts, val))
i = end
return out
def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
"""Least-squares slope (units per minute). None if too few points."""
if len(points) < 2:
return None
xs = [t for t, _ in points]
ys = [v for _, v in points]
n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
den = sum((x - mean_x) ** 2 for x in xs)
if den == 0:
return None
slope_per_sec = num / den
return slope_per_sec * 60.0
-15
View File
@@ -92,7 +92,6 @@ def _run_capturing(
cwd: Path | None = None,
timeout: float | None = None,
tee_header: str | None = None,
extra_env: dict[str, str] | None = None,
) -> tuple[int, str, str, float]:
"""Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
@@ -100,9 +99,6 @@ def _run_capturing(
`subprocess.TimeoutExpired` on timeout (callers map this to their own
domain-specific error).
`extra_env` merges into the subprocess environment (parent env stays
intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
Fast path: `subprocess.run(capture_output=True)` when no flash log is
configured (unchanged behavior).
@@ -114,9 +110,6 @@ def _run_capturing(
"""
log_path = _flash_log_path()
t0 = time.monotonic()
env = None
if extra_env:
env = {**os.environ, **extra_env}
if log_path is None:
# Fast path — unchanged.
@@ -126,7 +119,6 @@ def _run_capturing(
capture_output=True,
text=True,
timeout=timeout,
env=env,
)
return (
proc.returncode,
@@ -153,7 +145,6 @@ def _run_capturing(
stderr=subprocess.PIPE,
text=True,
bufsize=1, # line-buffered
env=env,
)
stdout_chunks: list[str] = []
stderr_chunks: list[str] = []
@@ -241,17 +232,12 @@ def run(
cwd: Path | None = None,
timeout: float | None = TIMEOUT_DEFAULT,
check: bool = True,
extra_env: dict[str, str] | None = None,
) -> PioResult:
"""Invoke `pio <args>` and return captured output.
`cwd` defaults to the firmware root. `check=True` raises `PioError` on
non-zero exit; set `check=False` to inspect `returncode` manually.
`extra_env` merges into the subprocess environment — used for
`PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
toggles that can't be expressed as command-line args.
If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
line-by-line as it arrives (for live flash progress in the TUI).
"""
@@ -264,7 +250,6 @@ def run(
cwd=work_dir,
timeout=timeout,
tee_header=f"pio {' '.join(args)}",
extra_env=extra_env,
)
except subprocess.TimeoutExpired as exc:
raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc
@@ -1,19 +0,0 @@
"""Persistent device-log capture.
Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
process-global so a single subscription captures every active interface
(serial / TCP / BLE) without any per-connection bookkeeping.
The recorder is opt-in-by-import: importing this package is a no-op; call
`get_recorder().start()` (which `server.py` does at FastMCP app init) to
begin writing. `pause()` / `resume()` exist for the rare case the user
wants a clean stretch of file (e.g. capturing a known-good baseline).
"""
from __future__ import annotations
from .recorder import Recorder, get_recorder
__all__ = ["Recorder", "get_recorder"]
@@ -1,309 +0,0 @@
"""Best-effort parsers for log lines and telemetry packets.
Two flavors of log line cross our pubsub subscription:
1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
accumulates bytes between protobuf frames and emits the full
firmware-formatted line, e.g.
"INFO | 12:34:56 12345 [Main] Booting"
— level, HH:MM:SS, uptime seconds, thread bracket, then message.
2. LogRecord protobuf path (debug_log_api enabled): the lib calls
`_handleLogLine(record.message)` with ONLY the message body. The
level/source/time fields on the LogRecord are dropped before
pubsub fan-out. We get e.g. just "Booting".
Both arrive on `meshtastic.log.line`. The parser tries to recover a
level + thread when the prefix is present and falls back to level=None
otherwise. Consumers who want level filtering on protobuf-mode hosts
should grep the raw `line` field instead.
Telemetry: `meshtastic.receive.telemetry` packets carry one of several
metric variants in `packet["decoded"]["telemetry"]`. We flatten the
chosen variant into a {field: value} dict so callers don't have to
know the protobuf shape.
"""
from __future__ import annotations
import re
from typing import Any
# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
# macros must stay in sync with these strings.
_LINE_RE = re.compile(
r"""
^
(?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
\s*\|\s*
(?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
\s+
(?P<uptime>\d+)
\s+
(?:\[(?P<thread>[^\]]+)\]\s+)?
(?P<msg>.*)
$
""",
re.VERBOSE,
)
# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
# thread bracket. See src/RedirectablePrint.cpp:175.
_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
# "++++++ Thread NAME freed heap A -> B (delta) ++++++"
_THREAD_HEAP_RE = re.compile(
r"""
^[\-+]+\s*
Thread\s+(?P<thread>\S+)\s+
(?P<kind>leaked|freed)\s+heap\s+
(?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
\((?P<delta>-?\d+)\)
""",
re.VERBOSE,
)
# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
_HEAP_STATUS_RE = re.compile(
r"""
Heap\s+status:\s+
(?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
(?:\s+\((?P<delta>-?\d+)\))?
""",
re.VERBOSE,
)
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
def parse_log_line(line: str) -> dict[str, Any]:
"""Best-effort decompose a raw firmware log line.
Returns a dict with at least `line` (the original, unmodified — ANSI
codes preserved for fidelity). Adds `level`, `tag`, `clock`,
`uptime_s`, and `msg` when the full prefix is present.
Handles two firmware quirks:
- LogRecord.message can carry ANSI color escapes from RedirectablePrint
(the BLE/StreamAPI path inherited the colored body in some builds).
We strip ANSI before regex matching so the prefix survives.
- DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
thread name is set, the heap takes the thread bracket position —
looks like `[heap 12345] msg`. We detect that shape and move it
out of `tag` and into `heap_free`.
DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
(bytes), and when a `Thread X leaked|freed heap` line is recognized,
`heap_event` = {kind, thread, before, after, delta}.
Never raises.
"""
out: dict[str, Any] = {"line": line}
if not line:
return out
# Strip ANSI escapes BEFORE any regex matching. The original `line`
# stays in `out["line"]` for fidelity / future grep.
clean = _ANSI_RE.sub("", line)
m = _LINE_RE.match(clean)
msg: str | None = None
if m:
level = m.group("level").rstrip()
out["level"] = level
out["clock"] = m.group("clock")
try:
out["uptime_s"] = int(m.group("uptime"))
except (TypeError, ValueError):
out["uptime_s"] = None
thread = m.group("thread")
if thread:
# If "thread" is actually the heap prefix taking the bracket
# position (DEBUG_HEAP build, no thread set), capture heap
# and leave tag unset.
hb = _HEAP_BRACKET_RE.match(thread.strip())
if hb:
try:
out["heap_free"] = int(hb.group("heap"))
except (TypeError, ValueError):
pass
else:
out["tag"] = thread
msg = m.group("msg")
out["msg"] = msg
else:
# No prefix — bare LogRecord.message body. Inspect the whole
# line for DEBUG_HEAP-style content; the heap-prefix and
# thread-leak patterns can survive on either path.
msg = clean
# DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
# Sits AFTER the thread bracket and BEFORE the message body, but
# for bare LogRecord lines it's at the start. Match it at the
# head of `msg`.
if msg:
hp = _HEAP_PREFIX_RE.match(msg)
if hp:
try:
out["heap_free"] = int(hp.group("heap"))
except (TypeError, ValueError):
pass
else:
# Strip the prefix from `msg` so a grep on the message
# body doesn't have to know about it.
out["msg"] = hp.group("rest")
msg = hp.group("rest")
# Thread-level leak/free detection.
thr = _THREAD_HEAP_RE.search(msg)
if thr:
try:
out["heap_event"] = {
"kind": thr.group("kind"),
"thread": thr.group("thread"),
"before": int(thr.group("before")),
"after": int(thr.group("after")),
"delta": int(thr.group("delta")),
}
except (TypeError, ValueError):
pass
# Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
hs = _HEAP_STATUS_RE.search(msg)
if hs:
try:
out["heap_free"] = int(hs.group("free"))
out["heap_total"] = int(hs.group("total"))
if hs.group("delta") is not None:
out["heap_delta"] = int(hs.group("delta"))
except (TypeError, ValueError):
pass
return out
# -- Telemetry ----------------------------------------------------------
# Order matters: meshtastic-python decoded packets use the protobuf
# `oneof variant` field name (snake_case) as the dict key.
_TELEMETRY_VARIANTS = (
("device_metrics", "device"),
("local_stats", "local"),
("environment_metrics", "environment"),
("power_metrics", "power"),
("air_quality_metrics", "airQuality"),
("health_metrics", "health"),
("host_metrics", "host"),
)
def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
"""Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
packet. Returns None when the shape isn't what we expect — so the
caller can fall back to a generic packets.jsonl row.
"""
if not isinstance(packet, dict):
return None
decoded = packet.get("decoded")
if not isinstance(decoded, dict):
return None
telem = decoded.get("telemetry")
if not isinstance(telem, dict):
return None
# The Python lib produces dict-of-camelCase keys via MessageToDict.
# Try both camelCase and snake_case to be robust to lib version drift.
for snake, label in _TELEMETRY_VARIANTS:
camel = _snake_to_camel(snake)
for key in (snake, camel):
value = telem.get(key)
if isinstance(value, dict):
return {
"variant": label,
"fields": {k: _scalarize(v) for k, v in value.items()},
"time": telem.get("time"),
}
return None
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _scalarize(value: Any) -> Any:
"""Keep telemetry fields JSON-friendly. Lists/dicts pass through
untouched; bytes -> hex string; protobuf enums occasionally arrive
as ints (fine) or strings (also fine)."""
if isinstance(value, (bytes, bytearray, memoryview)):
return bytes(value).hex()
return value
# -- Generic packet summary ---------------------------------------------
def summarize_packet(
packet: dict[str, Any], *, payload_hex_len: int = 64
) -> dict[str, Any]:
"""Reduce a packet dict to a stable, queryable summary. Drops the
full payload bytes — the recorder records summaries, not pcaps.
"""
if not isinstance(packet, dict):
return {"raw_type": type(packet).__name__}
decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
payload = decoded.get("payload") if isinstance(decoded, dict) else None
payload_hex = None
payload_size = None
if isinstance(payload, (bytes, bytearray, memoryview)):
b = bytes(payload)
payload_size = len(b)
payload_hex = b[:payload_hex_len].hex() if b else ""
elif isinstance(payload, str):
# Some decoded payloads (text messages) come as decoded strings.
payload_size = len(payload)
payload_hex = None # not bytes
return {
"from_node": packet.get("fromId") or packet.get("from"),
"to_node": packet.get("toId") or packet.get("to"),
"portnum": portnum,
"hop_limit": packet.get("hopLimit"),
"want_ack": packet.get("wantAck"),
"rx_rssi": packet.get("rxRssi"),
"rx_snr": packet.get("rxSnr"),
"channel": packet.get("channel"),
"id": packet.get("id"),
"payload_size": payload_size,
"payload_hex_prefix": payload_hex,
}
# -- Interface identification ------------------------------------------
def interface_label(interface: Any) -> dict[str, Any]:
"""Stable identifier for the meshtastic interface that emitted an event.
Used as the `port`/`role` tag on every recorded row. SerialInterface
has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
has `address`. Falls back to the class name when none of those exist.
"""
if interface is None:
return {"port": None, "role": None}
dev_path = getattr(interface, "devPath", None)
if dev_path:
return {"port": str(dev_path), "role": "serial"}
hostname = getattr(interface, "hostname", None)
if hostname:
port_num = getattr(interface, "portNumber", None)
endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
return {"port": endpoint, "role": "tcp"}
address = getattr(interface, "address", None)
if address:
return {"port": str(address), "role": "ble"}
return {"port": type(interface).__name__, "role": None}
@@ -1,467 +0,0 @@
"""Process-global recorder singleton.
Subscribes once to the meshtastic pubsub fan-out and writes four append-only
JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
process-global — a single subscription captures every active interface
without per-connection bookkeeping.
Files:
logs.jsonl — every `meshtastic.log.line` event (best-effort prefix
parsed for level/tag/uptime; raw `line` always preserved)
telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
variant (device / local / environment / power / etc.)
packets.jsonl — every other `meshtastic.receive.*` packet, summarized
(portnum, hops, RSSI/SNR, payload size + 64-byte hex)
events.jsonl — connection lifecycle, node-DB updates, and manual
`mark_event` rows. Lower volume; useful for aligning
timelines.
Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
write methods short-circuit when paused, so we don't lose ordering when
resumed (we just have a gap). No queueing.
"""
from __future__ import annotations
import logging
import os
import threading
import time
from pathlib import Path
from typing import Any
from . import parsers
from .rotating import _RotatingJsonl
_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
log = logging.getLogger(__name__)
class Recorder:
"""Singleton write-side of the persistent log capture system."""
def __init__(self, base_dir: Path | None = None) -> None:
self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
self._lock = threading.RLock()
self._started = False
self._paused = False
self._pause_reason: str | None = None
self._started_at: float | None = None
self._handlers: list[tuple[str, Any]] = []
self._files: dict[str, _RotatingJsonl] = {}
# -- lifecycle ----------------------------------------------------
def start(self) -> None:
"""Idempotent. Safe to call from FastMCP app startup."""
with self._lock:
if self._started:
return
self.base_dir.mkdir(parents=True, exist_ok=True)
self._files = {
"logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
"telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
"packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
"events": _RotatingJsonl(self.base_dir / "events.jsonl"),
}
self._wire_pubsub()
self._started = True
self._started_at = time.time()
# Write the recorder_start marker after the initialization block.
# `_write_event()` re-checks recorder state via `_files_snapshot()`,
# so keeping this out of the setup block avoids nested lifecycle work.
self._write_event(kind="recorder_start", label="recorder_started")
def stop(self) -> None:
with self._lock:
if not self._started:
return
self._unwire_pubsub()
for f in self._files.values():
f.close()
self._files = {}
self._started = False
def pause(self, reason: str | None = None) -> None:
# Write the pause marker BEFORE flipping the flag — `_write_event`
# short-circuits when paused, so the order matters for this event
# to actually land in events.jsonl.
self._write_event(
kind="recorder_pause",
label="paused",
note=reason,
)
with self._lock:
self._paused = True
self._pause_reason = reason
def resume(self) -> None:
# Mirror of `pause()`: clear the flag first, then write the marker
# so it isn't suppressed by the still-paused short-circuit.
with self._lock:
self._paused = False
self._pause_reason = None
self._write_event(kind="recorder_resume", label="resumed")
# -- pubsub wiring ------------------------------------------------
def _wire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
# Subscribers — one per topic. Each pubsub publisher sends
# keyword args matching its handler's signature; pubsub
# introspects the function signature to route args.
bindings = [
("meshtastic.log.line", self._on_log_line),
("meshtastic.serial.line", self._on_serial_line),
("meshtastic.receive", self._on_receive),
("meshtastic.receive.telemetry", self._on_telemetry),
("meshtastic.connection.established", self._on_connection_established),
("meshtastic.connection.lost", self._on_connection_lost),
("meshtastic.node.updated", self._on_node_updated),
]
for topic, handler in bindings:
try:
pub.subscribe(handler, topic)
self._handlers.append((topic, handler))
except Exception as exc:
# If pubsub refuses one binding (signature mismatch on
# an old lib version), log it and keep the rest.
log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
def _unwire_pubsub(self) -> None:
from pubsub import pub # type: ignore[import-untyped]
for topic, handler in self._handlers:
try:
pub.unsubscribe(handler, topic)
except Exception:
pass
self._handlers.clear()
# -- handlers -----------------------------------------------------
#
# Pubsub callbacks must never raise. Every handler is wrapped in a
# try/except that swallows so a bug here can't take down the
# SerialInterface receive thread.
#
# Threading: handlers fire on whatever thread the meshtastic library
# dispatches from (varies by interface), while `stop()` clears
# `self._files` under `self._lock`. We snapshot `_files` under the
# lock at the top of each handler so a concurrent stop can't
# KeyError us mid-write. The actual file write goes through
# `_RotatingJsonl` which has its own lock.
def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
"""Atomic-ish view of `self._files`. Returns None when the recorder
is paused or stopped, so handlers can early-exit cleanly without
racing `stop()`'s clear."""
with self._lock:
if not self._started or self._paused:
return None
return dict(self._files)
def _on_log_line(self, line: str, interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
# DEBUG_HEAP enrichments (only present when the firmware
# was built with -DDEBUG_HEAP=1). Surface as first-class
# fields so logs_window can grep/filter on them and so
# heap_free synthesizes a telemetry point below.
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# If the line carried a heap snapshot, also write it as a
# synthesized LocalStats-shaped row so telemetry_timeline
# picks it up at log cadence (much higher resolution than
# the ~60 s LocalStats packet). Tagged source=debug_heap so
# consumers can filter if mixing scales is unwanted.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": tags["port"],
"role": tags["role"],
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap",
}
)
except Exception:
pass
def _on_serial_line(self, line: str, port: str | None = None) -> None:
"""Text-mode passive tap. Fired from `serial_session._drain` when a
`pio device monitor` subprocess is running.
Same parse + heap-synthesis path as `_on_log_line`, but receives
the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
body). On DEBUG_HEAP builds in text mode this gives us per-log-line
heap data — far higher cadence than LocalStats, and works without
protobuf API mode (no SerialInterface required).
"""
files = self._files_snapshot()
if files is None:
return
try:
parsed = parsers.parse_log_line(str(line))
ts = time.time()
record: dict[str, Any] = {
"ts": ts,
"port": port,
"role": "serial_session",
"level": parsed.get("level"),
"tag": parsed.get("tag"),
"uptime_s": parsed.get("uptime_s"),
"line": parsed["line"],
}
if "heap_free" in parsed:
record["heap_free"] = parsed["heap_free"]
if "heap_total" in parsed:
record["heap_total"] = parsed["heap_total"]
if "heap_delta" in parsed:
record["heap_delta"] = parsed["heap_delta"]
heap_event = parsed.get("heap_event")
if heap_event:
record["heap_event"] = heap_event
files["logs"].write(record)
# Synthesize a heap_free telemetry sample whenever the line
# carries one — same logic as _on_log_line, tagged source so
# consumers can distinguish text-mode tap from protobuf path.
heap_free = parsed.get("heap_free")
if isinstance(heap_free, int):
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
heap_total = parsed.get("heap_total")
if isinstance(heap_total, int):
fields["heap_total_bytes"] = heap_total
files["telemetry"].write(
{
"ts": ts,
"port": port,
"role": "serial_session",
"from_node": None,
"variant": "local",
"fields": fields,
"source": "debug_heap_serial",
}
)
except Exception:
pass
def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
extracted = parsers.extract_telemetry(packet)
if extracted is None:
# Couldn't extract a known variant — fall through to the
# generic `_on_receive` path, which will still fire for
# this packet via the parent topic.
return
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
"from_node": packet.get("fromId") or packet.get("from"),
"variant": extracted["variant"],
"fields": extracted["fields"],
"device_time": extracted.get("time"),
}
files["telemetry"].write(record)
except Exception:
pass
def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
# Generic-receive fires for EVERY packet. Telemetry packets get
# recorded twice (here and in _on_telemetry) — that's intentional:
# packets.jsonl is the universal record, telemetry.jsonl is the
# structured timeseries view.
files = self._files_snapshot()
if files is None:
return
try:
tags = parsers.interface_label(interface)
summary = parsers.summarize_packet(packet)
record = {
"ts": time.time(),
"port": tags["port"],
"role": tags["role"],
**summary,
}
files["packets"].write(record)
except Exception:
pass
def _on_connection_established(self, interface: Any = None) -> None:
self._write_event(
kind="connection_established",
interface=interface,
)
def _on_connection_lost(self, interface: Any = None) -> None:
self._write_event(
kind="connection_lost",
interface=interface,
)
def _on_node_updated(
self, node: dict[str, Any] | None = None, interface: Any = None
) -> None:
# Lower-volume than packets but informative — node ID, hops away,
# last heard. Skip the user dict if absent.
try:
user = (node or {}).get("user") if isinstance(node, dict) else None
self._write_event(
kind="node_updated",
interface=interface,
data={
"num": (node or {}).get("num"),
"id": (user or {}).get("id"),
"short": (user or {}).get("shortName"),
"long": (user or {}).get("longName"),
"hops_away": (node or {}).get("hopsAway"),
"snr": (node or {}).get("snr"),
"last_heard": (node or {}).get("lastHeard"),
},
)
except Exception:
pass
# -- public write helpers -----------------------------------------
def mark_event(
self,
label: str,
note: str | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""User-facing marker. Writes to events.jsonl AND emits a
synthetic logs.jsonl row tagged level=MARK so timelines align.
"""
ts = self._write_event(kind="mark", label=label, note=note, data=data)
# Mirror into logs so a single logs_window grep finds it.
files = self._files_snapshot()
if files is not None:
try:
files["logs"].write(
{
"ts": ts,
"port": None,
"role": "marker",
"level": "MARK",
"tag": "mark_event",
"line": f"[mark] {label}" + (f" — {note}" if note else ""),
}
)
except Exception:
pass
return {"ts": ts, "label": label}
def _write_event(
self,
*,
kind: str,
label: str | None = None,
note: str | None = None,
interface: Any = None,
data: dict[str, Any] | None = None,
) -> float:
ts = time.time()
# Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
# arrive at choreographed moments — `pause()` writes BEFORE flipping
# the flag and `resume()` writes AFTER clearing it, so those calls
# see _paused=False here. Other event kinds short-circuit when
# paused via the snapshot guard below.
files = self._files_snapshot()
if files is None:
return ts
try:
tags = parsers.interface_label(interface)
files["events"].write(
{
"ts": ts,
"kind": kind,
"label": label,
"note": note,
"port": tags["port"],
"role": tags["role"],
"data": data,
}
)
except Exception:
pass
return ts
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
return {
"running": self._started,
"paused": self._paused,
"pause_reason": self._pause_reason,
"started_at": self._started_at,
"base_dir": str(self.base_dir),
"files": {name: f.status() for name, f in self._files.items()},
}
def force_rotate_all(self) -> dict[str, Any]:
"""Test/admin hook: rotate every stream right now."""
with self._lock:
files = list(self._files.values())
for f in files:
f.force_rotate()
# `status()` re-acquires `self._lock`; release before calling it.
return self.status()
# -- module-level singleton accessor ------------------------------------
_INSTANCE_LOCK = threading.Lock()
_INSTANCE: Recorder | None = None
def get_recorder() -> Recorder:
"""Return the process-global Recorder. Created on first call.
Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
(used by tests to redirect to a tmpdir).
"""
global _INSTANCE
with _INSTANCE_LOCK:
if _INSTANCE is None:
override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
base = Path(override) if override else None
_INSTANCE = Recorder(base_dir=base)
return _INSTANCE
@@ -1,163 +0,0 @@
"""Append-only JSONL writer with size-capped rotation.
A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
JSON objects (one row per call). When the live file exceeds `max_bytes`,
it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
and the live file resets to empty. Old archives past `keep_archives` are
unlinked oldest-first.
Size check is amortized — `os.fstat` runs every `check_every` writes,
not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
Threading: every public method acquires `self._lock`. The recorder runs
several pubsub handlers on whatever thread the meshtastic library
dispatches from (varies by interface), and queries from MCP tool calls
arrive on the FastMCP request thread, so this lock is not optional.
"""
from __future__ import annotations
import gzip
import json
import os
import shutil
import threading
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
class _RotatingJsonl:
"""Append-only JSONL with size rotation. Thread-safe."""
def __init__(
self,
path: Path,
*,
max_bytes: int = 100 * 1024 * 1024,
keep_archives: int = 5,
check_every: int = 1000,
) -> None:
self.path = path
self.max_bytes = max_bytes
self.keep_archives = keep_archives
self.check_every = check_every
self._lock = threading.Lock()
self._fh: Any = None
self._writes_since_check = 0
self._rotations = 0
self._lines_written = 0
self._last_ts: float | None = None
self._open()
# -- lifecycle ----------------------------------------------------
def _open(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
self._fh = self.path.open("a", encoding="utf-8")
def close(self) -> None:
with self._lock:
if self._fh is not None:
try:
self._fh.close()
finally:
self._fh = None
# -- write --------------------------------------------------------
def write(self, record: dict[str, Any]) -> None:
"""Append one JSON object as a line. Triggers rotation if oversized."""
line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
with self._lock:
if self._fh is None:
return
try:
self._fh.write(line)
self._fh.flush()
except Exception:
# Best-effort: a failed write must not crash the pubsub
# handler. Caller has no way to react anyway.
return
self._lines_written += 1
ts = record.get("ts")
if isinstance(ts, (int, float)):
self._last_ts = float(ts)
self._writes_since_check += 1
if self._writes_since_check >= self.check_every:
self._writes_since_check = 0
self._maybe_rotate()
# -- rotation -----------------------------------------------------
def _maybe_rotate(self) -> None:
# Caller holds self._lock.
try:
size = os.fstat(self._fh.fileno()).st_size
except OSError:
return
if size < self.max_bytes:
return
self._rotate_locked()
def _rotate_locked(self) -> None:
# Close, gzip-rename, reopen empty, prune oldest archives.
try:
self._fh.close()
except Exception:
pass
self._fh = None
# Microsecond-resolution timestamp + per-instance counter so back-
# to-back rotations (small max_bytes, repeated `force_rotate()`,
# or chatty test loops) get unique archive filenames. The lex
# sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
# which `_prune_archives()` and `log_query._iter_jsonl()` both
# rely on.
stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
try:
with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
shutil.copyfileobj(src, dst, length=1024 * 1024)
self.path.unlink()
except Exception:
# Rotation is best-effort. If gzip fails, leave the file
# in place and re-open it; we'll try again next check.
pass
self._open()
self._rotations += 1
self._prune_archives()
def _prune_archives(self) -> None:
# Match siblings of self.path.name with `.jsonl.gz` suffix.
prefix = self.path.stem # "logs" for "logs.jsonl"
# Archive filenames are already lexicographically chronological.
# Prune by name, not mtime, so copied/restored files don't reorder.
archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
excess = len(archives) - self.keep_archives
for old in archives[: max(0, excess)]:
try:
old.unlink()
except OSError:
pass
def force_rotate(self) -> None:
"""Test/admin hook: rotate immediately regardless of size."""
with self._lock:
if self._fh is not None:
self._rotate_locked()
# -- introspection ------------------------------------------------
def status(self) -> dict[str, Any]:
with self._lock:
try:
size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
except OSError:
size = 0
return {
"path": str(self.path),
"size": size,
"lines": self._lines_written,
"last_ts": self._last_ts,
"rotations": self._rotations,
}
@@ -46,23 +46,7 @@ class SerialSession:
def _drain(session: SerialSession) -> None:
"""Reader thread: line-by-line pull stdout into buffer.
Each line is also published to the `meshtastic.serial.line` pubsub
topic so the persistent recorder can capture it without holding its
own port. This is the text-mode tap path: when no SerialInterface is
open, the firmware emits full formatted lines (level + clock + uptime
+ thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
fan them out to whoever is listening. Pubsub is best-effort —
publish failures must never block the reader.
"""
# Lazy import: pubsub isn't required just to import this module
# (e.g., during static analysis), and we want a clean test surface.
try:
from pubsub import pub # type: ignore[import-untyped]
except Exception: # pragma: no cover - defensive
pub = None
"""Reader thread: line-by-line pull stdout into buffer."""
assert session.proc.stdout is not None
try:
for line in session.proc.stdout:
@@ -70,16 +54,6 @@ def _drain(session: SerialSession) -> None:
with session.lock:
session.buffer.append(line_stripped)
session.total_lines += 1
if pub is not None:
try:
pub.sendMessage(
"meshtastic.serial.line",
line=line_stripped,
port=session.port,
)
except Exception:
# A subscriber raising must not break the reader.
pass
except Exception: # pragma: no cover - defensive
pass
finally:

Some files were not shown because too many files have changed in this diff Show More