dmshell reliability tweaks

Co-authored-by: jp-bennett <5630967+jp-bennett@users.noreply.github.com>

.github/copilot-instructions.md

@@ -575,8 +575,6 @@ Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-v
 
 `confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve — it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
 
-**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step — use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
-
 ### Hardware test suite (`mcp-server/run-tests.sh`)
 
 The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.

.github/workflows/build_debian_src.yml

@@ -32,15 +32,10 @@ jobs:
         shell: bash
         working-directory: meshtasticd
         run: |
-          # Build-tools (notably platformio) come from the Meshtastic project
-          # on the OpenSUSE Build Service:
-          # https://build.opensuse.org/project/show/network:Meshtastic:build-tools
-          echo 'deb http://download.opensuse.org/repositories/network:/Meshtastic:/build-tools/xUbuntu_24.04/ /' \
-            | sudo tee /etc/apt/sources.list.d/network:Meshtastic:build-tools.list
-          curl -fsSL https://download.opensuse.org/repositories/network:Meshtastic:build-tools/xUbuntu_24.04/Release.key \
-            | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/network_Meshtastic_build-tools.gpg >/dev/null
           sudo apt-get update -y --fix-missing
-          sudo apt-get install -y build-essential devscripts equivs
+          sudo apt-get install -y software-properties-common build-essential devscripts equivs
+          sudo add-apt-repository ppa:meshtastic/build-tools -y
+          sudo apt-get update -y --fix-missing
           sudo mk-build-deps --install --remove --tool='apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes' debian/control
 
       - name: Import GPG key

.github/workflows/build_macos_bin.yml

@@ -24,7 +24,7 @@ jobs:
         shell: bash
         run: |
           brew update
-          brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config ulfius
+          brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config
 
       - name: Get release version string
         run: |

.trunk/trunk.yaml

@@ -8,7 +8,7 @@ plugins:
       uri: https://github.com/trunk-io/plugins
 lint:
   enabled:
-    - checkov@3.2.526
+    - checkov@3.2.525
     - renovate@43.150.0
     - prettier@3.8.3
     - trufflehog@3.95.2

AGENTS.md

@@ -126,17 +126,16 @@ Sequence these; don't parallelize on the same port.
 
 ## Environment variables (test harness)
 
-| Var                                  | Purpose                                                                                                                                                                                                    |
-| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `MESHTASTIC_MCP_ENV_<ROLE>`          | Override PlatformIO env for a role (e.g. `MESHTASTIC_MCP_ENV_NRF52=rak4631-dap`). Default map: `nrf52→rak4631`, `esp32s3→heltec-v3`.                                                                       |
-| `MESHTASTIC_MCP_SEED`                | PSK seed for the session test profile. Defaults to `mcp-<user>-<host>`.                                                                                                                                    |
-| `MESHTASTIC_MCP_FLASH_LOG`           | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress.                                                             |
-| `MESHTASTIC_MCP_TCP_HOST`            | `host` or `host:port` of a `meshtasticd` daemon (e.g. the `native-macos` build). Surfaces it in `list_devices` as `tcp://host:port` so `connect()`-based tools target it transparently. Default port 4403. |
-| `MESHTASTIC_UHUBCTL_BIN`             | Absolute path to `uhubctl` binary. Default: PATH lookup.                                                                                                                                                   |
-| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID.                                                                        |
-| `MESHTASTIC_UHUBCTL_PORT_<ROLE>`     | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`.                                                                                                                            |
-| `MESHTASTIC_UI_CAMERA_BACKEND`       | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default).                                                                                                       |
-| `MESHTASTIC_UI_CAMERA_DEVICE`        | Generic camera device (index or path). Used by the UI tier when no per-role var is set.                                                                                                                    |
-| `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` | Per-role camera pinning (e.g. `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3).                                                                                                     |
-| `MESHTASTIC_UI_OCR_BACKEND`          | OCR engine selection: `easyocr` / `pytesseract` / `null` / `auto` (default).                                                                                                                               |
-| `MESHTASTIC_UI_TUI_CAMERA`           | Set to `1` to mount the live camera-feed panel in `meshtastic-mcp-test-tui`.                                                                                                                               |
+| Var                                  | Purpose                                                                                                                                        |
+| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| `MESHTASTIC_MCP_ENV_<ROLE>`          | Override PlatformIO env for a role (e.g. `MESHTASTIC_MCP_ENV_NRF52=rak4631-dap`). Default map: `nrf52→rak4631`, `esp32s3→heltec-v3`.           |
+| `MESHTASTIC_MCP_SEED`                | PSK seed for the session test profile. Defaults to `mcp-<user>-<host>`.                                                                        |
+| `MESHTASTIC_MCP_FLASH_LOG`           | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress. |
+| `MESHTASTIC_UHUBCTL_BIN`             | Absolute path to `uhubctl` binary. Default: PATH lookup.                                                                                       |
+| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID.            |
+| `MESHTASTIC_UHUBCTL_PORT_<ROLE>`     | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`.                                                                |
+| `MESHTASTIC_UI_CAMERA_BACKEND`       | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default).                                           |
+| `MESHTASTIC_UI_CAMERA_DEVICE`        | Generic camera device (index or path). Used by the UI tier when no per-role var is set.                                                        |
+| `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` | Per-role camera pinning (e.g. `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3).                                         |
+| `MESHTASTIC_UI_OCR_BACKEND`          | OCR engine selection: `easyocr` / `pytesseract` / `null` / `auto` (default).                                                                   |
+| `MESHTASTIC_UI_TUI_CAMERA`           | Set to `1` to mount the live camera-feed panel in `meshtastic-mcp-test-tui`.                                                                   |

Dockerfile

@@ -3,17 +3,15 @@
 # trunk-ignore-all(hadolint/DL3008): Do not pin apt package versions
 # trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
 
-FROM debian:trixie AS builder
+FROM python:3.14-slim-trixie AS builder
 ARG PIO_ENV=native
 ENV DEBIAN_FRONTEND=noninteractive
 ENV TZ=Etc/UTC
 
 # Install Dependencies
 ENV PIP_ROOT_USER_ACTION=ignore
-ENV PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apt-get update && apt-get install --no-install-recommends -y \
         curl wget g++ zip git ca-certificates pkg-config \
-        python3-pip python3-grpc-tools \
         libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
         libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
         libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev libsdl2-dev \

alpine.Dockerfile

@@ -4,18 +4,12 @@
 # trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
 
 # Ensure the Alpine version is updated in both stages of the container!
-FROM alpine:3.23 AS builder
+FROM python:3.14-alpine3.23 AS builder
 ARG PIO_ENV=native
-
-# Enable Alpine community repository (for 'py3-grpcio-tools')
-RUN echo "https://dl-cdn.alpinelinux.org/alpine/v$(cut -d. -f1,2 /etc/alpine-release)/community" >> /etc/apk/repositories
-
-# Install Dependencies
 ENV PIP_ROOT_USER_ACTION=ignore
-ENV PIP_BREAK_SYSTEM_PACKAGES=1
+
 RUN apk --no-cache add \
         bash g++ libstdc++-dev linux-headers zip git ca-certificates libbsd-dev \
-        py3-pip py3-grpcio-tools \
         libgpiod-dev yaml-cpp-dev bluez-dev \
         libusb-dev i2c-tools-dev libuv-dev openssl-dev pkgconf argp-standalone \
         libx11-dev libinput-dev libxkbcommon-dev sqlite-dev sdl2-dev \

bin/dmshell_client.py

@@ -0,0 +1,941 @@
+#!/usr/bin/env python3
+
+import argparse
+import os
+import queue
+import random
+import shutil
+import socket
+import select
+import subprocess
+import sys
+import tempfile
+import termios
+import threading
+import time
+import tty
+from collections import deque
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Optional, TextIO
+
+
+START1 = 0x94
+START2 = 0xC3
+HEADER_LEN = 4
+DEFAULT_API_PORT = 4403
+DEFAULT_HOP_LIMIT = 0
+LOCAL_ESCAPE_BYTE = b"\x1d"  # Ctrl+]
+MISSING_SEQ_RETRY_INTERVAL_SEC = 1.0
+INPUT_BATCH_WINDOW_SEC = .5
+INPUT_BATCH_MAX_BYTES = 64
+HEARTBEAT_IDLE_DELAY_SEC = 5.0
+HEARTBEAT_REPEAT_SEC = 15.0
+HEARTBEAT_POLL_INTERVAL_SEC = 0.25
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description="Tiny DMShell client for Meshtastic native TCP API",
+        epilog=(
+            "Examples:\n"
+            "  bin/dmshell_client.py --to !170896f7\n"
+            "  bin/dmshell_client.py --to 0x170896f7 --command 'uname -a' --command 'id'"
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument("--host", default="127.0.0.1", help="meshtasticd API host")
+    parser.add_argument("--port", type=int, default=DEFAULT_API_PORT, help="meshtasticd API port")
+    parser.add_argument(
+        "--serial",
+        nargs="?",
+        const="auto",
+        default=None,
+        help="use USB serial transport (optionally provide device path, default: auto-detect)",
+    )
+    parser.add_argument("--baud", type=int, default=115200, help="serial baud rate when using --serial")
+    parser.add_argument("--to", required=True, help="destination node number, e.g. !170896f7 or 0x170896f7")
+    parser.add_argument("--channel", type=int, default=0, help="channel index to use")
+    parser.add_argument("--cols", type=int, default=None, help="initial terminal columns (default: detect local terminal)")
+    parser.add_argument("--rows", type=int, default=None, help="initial terminal rows (default: detect local terminal)")
+    parser.add_argument("--command", action="append", default=[], help="send a command line after opening")
+    parser.add_argument("--close-after", type=float, default=2.0, help="seconds to wait before closing in command mode")
+    parser.add_argument("--timeout", type=float, default=10.0, help="seconds to wait for API/session events")
+    parser.add_argument("--verbose", action="store_true", help="print extra protocol events")
+    return parser.parse_args()
+
+
+def repo_root() -> Path:
+    return Path(__file__).resolve().parent.parent
+
+
+def load_proto_modules() -> object:
+    try:
+        import google.protobuf  # noqa: F401
+    except ImportError as exc:
+        raise SystemExit("python package 'protobuf' is required to run this client") from exc
+
+    protoc = shutil.which("protoc")
+    if not protoc:
+        raise SystemExit("'protoc' is required to generate temporary Python protobuf bindings")
+
+    out_dir = Path(tempfile.mkdtemp(prefix="meshtastic_dmshell_proto_"))
+    proto_dir = repo_root() / "protobufs"
+    
+    # Compile all required protos for DMShell client (mesh and dependencies)
+    # Excludes nanopb.proto and other complex build artifacts
+    required_protos = [
+        "mesh.proto",
+        "channel.proto", 
+        "config.proto",
+        "device_ui.proto",
+        "module_config.proto",
+        "atak.proto",
+        "portnums.proto",
+        "telemetry.proto",
+        "xmodem.proto",
+    ]
+    proto_files = [proto_dir / "meshtastic" / name for name in required_protos]
+    for pf in proto_files:
+        if not pf.exists():
+            raise SystemExit(f"could not find required proto file: {pf}")
+
+    # Create __init__.py to make meshtastic a package
+    (out_dir / "meshtastic").mkdir(exist_ok=True)
+    (out_dir / "meshtastic" / "__init__.py").touch()
+
+    # Build protoc command with just the meshtastic proto directory as include path
+    # protoc will use its built-in includes for standard google protobuf types
+    cmd = [protoc, f"-I{proto_dir}", f"--python_out={out_dir}", *[str(path) for path in proto_files]]
+    result = subprocess.run(cmd, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"protoc stderr: {result.stderr}", file=sys.stderr)
+        print(f"protoc stdout: {result.stdout}", file=sys.stderr)
+        print(f"protoc command: {' '.join(cmd)}", file=sys.stderr)
+        raise SystemExit(f"protoc failed with return code {result.returncode}")
+
+    # Create _pb2_grpc module stub if not present (protoc 3.20+)
+    mesh_pb2_file = out_dir / "meshtastic" / "mesh_pb2.py"
+    if not mesh_pb2_file.exists():
+        raise SystemExit(f"protoc did not generate mesh_pb2.py in {out_dir / 'meshtastic'}")
+
+    sys.path.insert(0, str(out_dir))
+    try:
+        from meshtastic import mesh_pb2, portnums_pb2  # type: ignore
+    except ImportError as exc:
+        print(f"Failed to import protobuf modules. Output dir contents:", file=sys.stderr)
+        for item in (out_dir / "meshtastic").iterdir():
+            print(f"  {item.name}", file=sys.stderr)
+        raise SystemExit(f"could not import meshtastic proto modules: {exc}") from exc
+
+    # Return an object that has both modules accessible
+    class ProtoModules:
+        pass
+    
+    pb2 = ProtoModules()
+    pb2.mesh = mesh_pb2
+    pb2.portnums = portnums_pb2
+    return pb2
+
+
+def parse_node_num(raw: str) -> int:
+    value = raw.strip()
+    if value.startswith("!"):
+        value = value[1:]
+    if value.lower().startswith("0x"):
+        return int(value, 16)
+    if any(ch in "abcdefABCDEF" for ch in value):
+        return int(value, 16)
+    return int(value, 10)
+
+
+class SerialTransport:
+    def __init__(self, serial_obj):
+        self._serial = serial_obj
+
+    def recv(self, length: int) -> bytes:
+        return self._serial.read(length)
+
+    def sendall(self, data: bytes) -> None:
+        self._serial.write(data)
+        self._serial.flush()
+
+    def close(self) -> None:
+        self._serial.close()
+
+
+def detect_meshtastic_serial_port() -> str:
+    try:
+        from serial.tools import list_ports
+    except ImportError as exc:
+        raise SystemExit("python package 'pyserial' is required for --serial mode") from exc
+
+    ports = list(list_ports.comports())
+    if not ports:
+        raise SystemExit("no serial ports found for --serial mode")
+
+    scored: list[tuple[int, str]] = []
+    for port in ports:
+        text = " ".join(
+            filter(
+                None,
+                [port.device, port.description, port.manufacturer, port.product, port.hwid],
+            )
+        ).lower()
+        score = 0
+        if "meshtastic" in text:
+            score += 100
+        if "lora" in text or "mesh" in text:
+            score += 10
+        if "ttyacm" in (port.device or "").lower() or "ttyusb" in (port.device or "").lower():
+            score += 1
+        scored.append((score, port.device))
+
+    scored.sort(reverse=True)
+    best_score, best_device = scored[0]
+    if best_score <= 0 and len(scored) > 1:
+        raise SystemExit(
+            "could not confidently auto-detect a Meshtastic serial port; pass --serial /dev/ttyXXX explicitly"
+        )
+    return best_device
+
+
+def open_transport(args: argparse.Namespace):
+    if args.serial is None:
+        sock = socket.create_connection((args.host, args.port), timeout=args.timeout)
+        sock.settimeout(None)
+        return sock
+
+    serial_path = args.serial
+    if serial_path == "auto":
+        serial_path = detect_meshtastic_serial_port()
+        print(f"[dmshell] using serial port {serial_path}", file=sys.stderr)
+
+    try:
+        import serial
+    except ImportError as exc:
+        raise SystemExit("python package 'pyserial' is required for --serial mode") from exc
+
+    try:
+        serial_obj = serial.Serial(serial_path, baudrate=args.baud, timeout=None, write_timeout=2)
+    except Exception as exc:
+        raise SystemExit(f"failed to open serial device {serial_path}: {exc}") from exc
+
+    return SerialTransport(serial_obj)
+
+
+def recv_exact(transport, length: int) -> bytes:
+    chunks = bytearray()
+    while len(chunks) < length:
+        piece = transport.recv(length - len(chunks))
+        if not piece:
+            raise ConnectionError("connection closed by transport")
+        chunks.extend(piece)
+    return bytes(chunks)
+
+
+def detect_local_terminal_size() -> tuple[int, int]:
+    size = shutil.get_terminal_size(fallback=(100, 40))
+    cols = max(1, int(size.columns))
+    rows = max(1, int(size.lines))
+    return cols, rows
+
+
+def resolve_initial_terminal_size(cols_override: Optional[int], rows_override: Optional[int]) -> tuple[int, int]:
+    detected_cols, detected_rows = detect_local_terminal_size()
+    cols = detected_cols if cols_override is None else max(1, cols_override)
+    rows = detected_rows if rows_override is None else max(1, rows_override)
+    return cols, rows
+
+
+def recv_stream_frame(transport) -> bytes:
+    while True:
+        start = recv_exact(transport, 1)[0]
+        if start != START1:
+            continue
+        if recv_exact(transport, 1)[0] != START2:
+            continue
+        header = recv_exact(transport, 2)
+        length = (header[0] << 8) | header[1]
+        return recv_exact(transport, length)
+
+
+def send_stream_frame(transport, payload: bytes) -> None:
+    if len(payload) > 0xFFFF:
+        raise ValueError("payload too large for stream API")
+    header = bytes((START1, START2, (len(payload) >> 8) & 0xFF, len(payload) & 0xFF))
+    transport.sendall(header + payload)
+
+
+@dataclass
+class SentShellFrame:
+    op: int
+    session_id: int
+    seq: int
+    ack_seq: int
+    payload: bytes = b""
+    cols: int = 0
+    rows: int = 0
+    flags: int = 0
+    last_tx_seq: int = 0
+    last_rx_seq: int = 0
+
+
+@dataclass
+class SessionState:
+    pb2: object  # ProtoModules with mesh and portnums attributes
+    target: int
+    channel: int
+    verbose: bool
+    session_id: int = field(default_factory=lambda: random.randint(1, 0x7FFFFFFF))
+    next_seq: int = 1
+    last_rx_seq: int = 0
+    next_expected_rx_seq: int = 1
+    highest_seen_rx_seq: int = 0
+    active: bool = False
+    stopped: bool = False
+    opened_event: threading.Event = field(default_factory=threading.Event)
+    closed_event: threading.Event = field(default_factory=threading.Event)
+    event_queue: "queue.Queue[str]" = field(default_factory=queue.Queue)
+    tx_lock: threading.Lock = field(default_factory=threading.Lock)
+    socket_lock: threading.Lock = field(default_factory=threading.Lock)
+    tx_history: deque[SentShellFrame] = field(default_factory=lambda: deque(maxlen=50))
+    pending_rx_frames: dict[int, object] = field(default_factory=dict)
+    last_requested_missing_seq: int = 0
+    last_missing_request_time: float = 0.0
+    requested_missing_seqs: set[int] = field(default_factory=set)
+    replay_log_lock: threading.Lock = field(default_factory=threading.Lock)
+    replay_log_file: Optional[TextIO] = None
+    replay_log_path: Optional[Path] = None
+    last_transport_activity_time: float = field(default_factory=time.monotonic)
+    last_heartbeat_sent_time: float = 0.0
+
+    def alloc_seq(self) -> int:
+        with self.tx_lock:
+            value = self.next_seq
+            self.next_seq += 1
+            return value
+
+    def current_ack_seq(self) -> int:
+        with self.tx_lock:
+            return self.last_rx_seq
+
+    def highest_sent_seq(self) -> int:
+        with self.tx_lock:
+            return max(0, self.next_seq - 1)
+
+    def note_outbound_packet(self, heartbeat: bool = False) -> None:
+        with self.tx_lock:
+            now = time.monotonic()
+            if heartbeat:
+                self.last_heartbeat_sent_time = now
+            else:
+                self.last_transport_activity_time = now
+
+    def note_inbound_packet(self) -> None:
+        with self.tx_lock:
+            self.last_transport_activity_time = time.monotonic()
+
+    def heartbeat_due(self) -> bool:
+        with self.tx_lock:
+            now = time.monotonic()
+            if (now - self.last_transport_activity_time) < HEARTBEAT_IDLE_DELAY_SEC:
+                return False
+            if self.last_heartbeat_sent_time <= self.last_transport_activity_time:
+                return True
+            return (now - self.last_heartbeat_sent_time) >= HEARTBEAT_REPEAT_SEC
+
+    def note_peer_reported_tx_seq(self, seq: int) -> None:
+        with self.tx_lock:
+            if seq > self.highest_seen_rx_seq:
+                self.highest_seen_rx_seq = seq
+
+    def note_received_seq(self, seq: int) -> tuple[str, Optional[int]]:
+        with self.tx_lock:
+            if seq == 0:
+                return ("process", None)
+            if seq < self.next_expected_rx_seq:
+                if self.highest_seen_rx_seq >= self.next_expected_rx_seq:
+                    return ("gap", self.next_expected_rx_seq)
+                return ("duplicate", None)
+            if seq > self.next_expected_rx_seq:
+                if seq > self.highest_seen_rx_seq:
+                    self.highest_seen_rx_seq = seq
+                return ("gap", self.next_expected_rx_seq)
+            self.last_rx_seq = seq
+            self.next_expected_rx_seq = seq + 1
+            if self.last_requested_missing_seq != 0 and self.next_expected_rx_seq > self.last_requested_missing_seq:
+                self.last_requested_missing_seq = 0
+            if seq > self.highest_seen_rx_seq:
+                self.highest_seen_rx_seq = seq
+            if self.highest_seen_rx_seq < self.next_expected_rx_seq:
+                self.highest_seen_rx_seq = 0
+            return ("process", None)
+
+    def remember_out_of_order_frame(self, shell) -> None:
+        with self.tx_lock:
+            if shell.seq <= self.next_expected_rx_seq:
+                return
+            if shell.seq not in self.pending_rx_frames:
+                self.pending_rx_frames[shell.seq] = shell
+            if shell.seq > self.highest_seen_rx_seq:
+                self.highest_seen_rx_seq = shell.seq
+
+    def pop_next_buffered_frame(self):
+        with self.tx_lock:
+            return self.pending_rx_frames.pop(self.next_expected_rx_seq, None)
+
+    def pending_missing_seq(self) -> Optional[int]:
+        with self.tx_lock:
+            if self.highest_seen_rx_seq >= self.next_expected_rx_seq:
+                return self.next_expected_rx_seq
+            return None
+
+    def request_missing_seq_once(self) -> Optional[int]:
+        with self.tx_lock:
+            if self.highest_seen_rx_seq < self.next_expected_rx_seq:
+                return None
+            now = time.monotonic()
+            if (
+                self.last_requested_missing_seq == self.next_expected_rx_seq
+                and (now - self.last_missing_request_time) < MISSING_SEQ_RETRY_INTERVAL_SEC
+            ):
+                return None
+            self.last_requested_missing_seq = self.next_expected_rx_seq
+            self.last_missing_request_time = now
+            return self.last_requested_missing_seq
+
+    def set_receive_cursor(self, seq: int) -> None:
+        with self.tx_lock:
+            self.last_rx_seq = seq
+            self.next_expected_rx_seq = seq + 1
+            self.highest_seen_rx_seq = seq
+
+    def open_replay_log(self, session_id: int) -> None:
+        with self.replay_log_lock:
+            if self.replay_log_file is not None:
+                return
+            path = Path.cwd() / f"{session_id:08x}.log"
+            self.replay_log_file = path.open("a", encoding="utf-8")
+            self.replay_log_path = path
+            self.replay_log_file.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} session_open session=0x{session_id:08x}\n")
+            self.replay_log_file.flush()
+
+    def log_replay_event(self, event: str, seq: int, detail: str = "") -> None:
+        with self.replay_log_lock:
+            if self.replay_log_file is None:
+                return
+            extra = f" {detail}" if detail else ""
+            self.replay_log_file.write(
+                f"{time.strftime('%Y-%m-%d %H:%M:%S')} {event} seq={seq}{extra}\n"
+            )
+            self.replay_log_file.flush()
+
+    def note_missing_seq_requested(self, seq: int, reason: str) -> None:
+        with self.tx_lock:
+            self.requested_missing_seqs.add(seq)
+        self.log_replay_event("missing_requested", seq, f"reason={reason}")
+
+    def note_replayed_seq_received(self, seq: int) -> None:
+        with self.tx_lock:
+            was_requested = seq in self.requested_missing_seqs
+            if was_requested:
+                self.requested_missing_seqs.remove(seq)
+        if was_requested:
+            self.log_replay_event("replay_received", seq)
+
+    def close_replay_log(self) -> None:
+        with self.replay_log_lock:
+            if self.replay_log_file is None:
+                return
+            self.replay_log_file.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} session_close\n")
+            self.replay_log_file.flush()
+            self.replay_log_file.close()
+            self.replay_log_file = None
+
+    def remember_sent_frame(self, frame: SentShellFrame) -> None:
+        if frame.seq == 0 or frame.op == self.pb2.mesh.RemoteShell.ACK:
+            return
+        with self.tx_lock:
+            self.tx_history.append(frame)
+
+    def prune_sent_frames(self, ack_seq: int) -> None:
+        if ack_seq <= 0:
+            return
+        with self.tx_lock:
+            self.tx_history = deque((frame for frame in self.tx_history if frame.seq > ack_seq), maxlen=50)
+
+    def replay_frames_from(self, start_seq: int) -> list[SentShellFrame]:
+        with self.tx_lock:
+            return [frame for frame in self.tx_history if frame.seq >= start_seq]
+
+
+def send_toradio(transport, toradio) -> None:
+    send_stream_frame(transport, toradio.SerializeToString())
+
+
+def make_toradio_packet(pb2, state: SessionState, shell_msg) -> object:
+    packet = pb2.mesh.MeshPacket()
+    packet.id = random.randint(1, 0x7FFFFFFF)
+    packet.to = state.target
+    # The 'from' field is a reserved keyword in Python, so use setattr
+    setattr(packet, "from", 0)
+    packet.channel = state.channel
+    packet.hop_limit = DEFAULT_HOP_LIMIT
+    packet.want_ack = False
+    packet.decoded.portnum = pb2.portnums.REMOTE_SHELL_APP
+    packet.decoded.payload = shell_msg.SerializeToString()
+    packet.decoded.want_response = False
+    packet.decoded.dest = state.target
+    packet.decoded.source = 0
+
+    toradio = pb2.mesh.ToRadio()
+    toradio.packet.CopyFrom(packet)
+    return toradio
+
+
+def send_shell_frame(
+    transport,
+    state: SessionState,
+    op: int,
+    payload: bytes = b"",
+    cols: int = 0,
+    rows: int = 0,
+    session_id: Optional[int] = None,
+    ack_seq: Optional[int] = None,
+    seq: Optional[int] = None,
+    flags: int = 0,
+    last_tx_seq: int = 0,
+    last_rx_seq: int = 0,
+    remember: bool = True,
+    heartbeat: bool = False,
+) -> int:
+    if seq is None:
+        seq = 0 if op == state.pb2.mesh.RemoteShell.ACK else state.alloc_seq()
+    if ack_seq is None:
+        ack_seq = state.current_ack_seq()
+    if session_id is None:
+        session_id = state.session_id
+
+    shell = state.pb2.mesh.RemoteShell()
+    shell.op = op
+    shell.session_id = session_id
+    shell.seq = seq
+    shell.ack_seq = ack_seq
+    shell.cols = cols
+    shell.rows = rows
+    shell.flags = flags
+    shell.last_tx_seq = last_tx_seq
+    shell.last_rx_seq = last_rx_seq
+    if payload:
+        shell.payload = payload
+    with state.socket_lock:
+        send_toradio(transport, make_toradio_packet(state.pb2, state, shell))
+    if remember:
+        state.remember_sent_frame(
+            SentShellFrame(
+                op=op,
+                session_id=session_id,
+                seq=seq,
+                ack_seq=ack_seq,
+                payload=payload,
+                cols=cols,
+                rows=rows,
+                flags=flags,
+                last_tx_seq=last_tx_seq,
+                last_rx_seq=last_rx_seq,
+            )
+        )
+    state.note_outbound_packet(heartbeat=heartbeat)
+    return seq
+
+
+def send_ack_frame(transport, state: SessionState, replay_from: Optional[int] = None) -> None:
+    send_shell_frame(
+        transport,
+        state,
+        state.pb2.mesh.RemoteShell.ACK,
+        seq=0,
+        last_rx_seq=0 if replay_from is None else replay_from - 1,
+        remember=False,
+    )
+
+
+def replay_frames_from(transport, state: SessionState, start_seq: int) -> None:
+    frame = next((f for f in state.replay_frames_from(start_seq) if f.seq == start_seq), None)
+    if frame is None:
+        #state.event_queue.put(f"replay unavailable from seq={start_seq}")
+        state.log_replay_event("replay_unavailable", start_seq)
+        return
+    state.log_replay_event("replay_sent", start_seq)
+    #state.event_queue.put(f"replay frame seq={start_seq}")
+    send_shell_frame(
+        transport,
+        state,
+        frame.op,
+        payload=frame.payload,
+        cols=frame.cols,
+        rows=frame.rows,
+        session_id=frame.session_id,
+        ack_seq=frame.ack_seq,
+        seq=frame.seq,
+        flags=frame.flags,
+        last_tx_seq=frame.last_tx_seq,
+        last_rx_seq=frame.last_rx_seq,
+        remember=False,
+    )
+
+
+def wait_for_config_complete(transport, pb2, timeout: float, verbose: bool) -> None:
+    nonce = random.randint(1, 0x7FFFFFFF)
+    toradio = pb2.mesh.ToRadio()
+    toradio.want_config_id = nonce
+    send_toradio(transport, toradio)
+
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        fromradio = pb2.mesh.FromRadio()
+        fromradio.ParseFromString(recv_stream_frame(transport))
+        variant = fromradio.WhichOneof("payload_variant")
+        if verbose and variant:
+            print(f"[api] fromradio {variant}", file=sys.stderr)
+        if variant == "config_complete_id" and fromradio.config_complete_id == nonce:
+            return
+    raise TimeoutError("timed out waiting for config handshake to complete")
+
+
+def decode_shell_packet(state: SessionState, packet) -> Optional[object]:
+    if packet.WhichOneof("payload_variant") != "decoded":
+        return None
+    if packet.decoded.portnum != state.pb2.portnums.REMOTE_SHELL_APP:
+        return None
+    shell = state.pb2.mesh.RemoteShell()
+    shell.ParseFromString(packet.decoded.payload)
+    return shell
+
+
+def reader_loop(transport, state: SessionState) -> None:
+    def handle_in_order_shell(shell) -> bool:
+        state.note_replayed_seq_received(shell.seq)
+        if shell.op == state.pb2.mesh.RemoteShell.OPEN_OK:
+            state.session_id = shell.session_id
+            state.open_replay_log(state.session_id)
+            state.set_receive_cursor(shell.seq)
+            state.active = True
+            state.opened_event.set()
+            state.event_queue.put(
+                f"opened session=0x{shell.session_id:08x} cols={shell.cols} rows={shell.rows}"
+            )
+            if state.replay_log_path is not None:
+                state.event_queue.put(f"replay log: {state.replay_log_path}")
+        elif shell.op == state.pb2.mesh.RemoteShell.OUTPUT:
+            if shell.payload:
+                sys.stdout.buffer.write(shell.payload)
+                sys.stdout.buffer.flush()
+        elif shell.op == state.pb2.mesh.RemoteShell.ERROR:
+            message = shell.payload.decode("utf-8", errors="replace")
+            if state.replay_log_file is None:
+                state.open_replay_log(shell.session_id or state.session_id)
+            sanitized = message.replace("\n", "\\n")
+            state.log_replay_event("error_received", shell.seq, f"message={sanitized}")
+            state.event_queue.put(f"remote error: {message}")
+        elif shell.op == state.pb2.mesh.RemoteShell.CLOSED:
+            message = shell.payload.decode("utf-8", errors="replace")
+            state.event_queue.put(f"session closed: {message}")
+            state.closed_event.set()
+            state.active = False
+            return True
+        elif shell.op == state.pb2.mesh.RemoteShell.PONG:
+            remote_last_tx_seq = shell.last_tx_seq
+            remote_last_rx_seq = shell.last_rx_seq
+            local_latest_tx_seq = state.highest_sent_seq()
+            if remote_last_rx_seq != 0 and remote_last_rx_seq < local_latest_tx_seq:
+                replay_frames_from(transport, state, remote_last_rx_seq + 1)
+            if remote_last_tx_seq > state.current_ack_seq():
+                state.note_peer_reported_tx_seq(remote_last_tx_seq)
+                req = state.request_missing_seq_once()
+                if req is not None:
+                    state.note_missing_seq_requested(req, "heartbeat_status")
+                    send_ack_frame(transport, state, replay_from=req)
+            #state.event_queue.put("pong")
+        return False
+
+    while not state.stopped:
+        try:
+            fromradio = state.pb2.mesh.FromRadio()
+            fromradio.ParseFromString(recv_stream_frame(transport))
+        except Exception as exc:
+            if not state.stopped:
+                state.event_queue.put(f"connection error: {exc}")
+                state.closed_event.set()
+            return
+
+        variant = fromradio.WhichOneof("payload_variant")
+        if variant == "packet":
+            shell = decode_shell_packet(state, fromradio.packet)
+            if not shell:
+                continue
+            state.note_inbound_packet()
+            #state.prune_sent_frames(shell.ack_seq)
+            if shell.op == state.pb2.mesh.RemoteShell.ACK:
+                #state.event_queue.put("peer requested replay")
+                replay_from = shell.last_rx_seq + 1 if shell.last_rx_seq > 0 else None
+                if replay_from is not None:
+                    #state.event_queue.put(f"peer requested replay from seq={replay_from}")
+                    replay_frames_from(transport, state, replay_from)
+                continue
+
+            action, missing_from = state.note_received_seq(shell.seq)
+            if action == "duplicate":
+                req = state.request_missing_seq_once()
+                if req is not None:
+                    state.note_missing_seq_requested(req, "duplicate")
+                    send_ack_frame(transport, state, replay_from=req)
+                continue
+            if action == "gap":
+                state.remember_out_of_order_frame(shell)
+                req = state.request_missing_seq_once()
+                if req is not None:
+                    state.note_missing_seq_requested(req, "gap")
+                    send_ack_frame(transport, state, replay_from=req)
+                continue
+
+            if handle_in_order_shell(shell):
+                return
+
+            while True:
+                buffered_shell = state.pop_next_buffered_frame()
+                if buffered_shell is None:
+                    break
+                buffered_action, _ = state.note_received_seq(buffered_shell.seq)
+                if buffered_action != "process":
+                    state.remember_out_of_order_frame(buffered_shell)
+                    break
+                if handle_in_order_shell(buffered_shell):
+                    return
+
+            req = state.request_missing_seq_once()
+            if req is not None:
+                state.note_missing_seq_requested(req, "post_process_gap")
+                send_ack_frame(transport, state, replay_from=req)
+        elif state.verbose and variant:
+            state.event_queue.put(f"fromradio {variant}")
+
+
+def drain_events(state: SessionState) -> None:
+    while True:
+        try:
+            event = state.event_queue.get_nowait()
+        except queue.Empty:
+            return
+        print(f"[dmshell] {event}", file=sys.stderr)
+
+
+def heartbeat_loop(transport, state: SessionState) -> None:
+    while not state.stopped and not state.closed_event.is_set():
+        if not state.active:
+            time.sleep(HEARTBEAT_POLL_INTERVAL_SEC)
+            continue
+        if state.heartbeat_due():
+            try:
+                send_shell_frame(
+                    transport,
+                    state,
+                    state.pb2.mesh.RemoteShell.PING,
+                    last_tx_seq=state.highest_sent_seq(),
+                    last_rx_seq=state.current_ack_seq(),
+                    remember=True,
+                    heartbeat=True,
+                )
+            except Exception as exc:
+                if not state.stopped:
+                    state.event_queue.put(f"heartbeat error: {exc}")
+                    state.closed_event.set()
+                    return
+        time.sleep(HEARTBEAT_POLL_INTERVAL_SEC)
+
+
+def run_command_mode(transport, state: SessionState, commands: list[str], close_after: float) -> None:
+    for command in commands:
+        send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.INPUT, (command + "\n").encode("utf-8"))
+    time.sleep(close_after)
+    send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.CLOSE)
+    state.closed_event.wait(timeout=close_after + 5.0)
+
+
+def run_interactive_mode(transport, state: SessionState) -> None:
+    def read_local_command() -> str:
+        prompt = "\r\n[dmshell] local command (resume|close|ping|resize C R): "
+        sys.stderr.write(prompt)
+        sys.stderr.flush()
+        buf = bytearray()
+
+        while True:
+            ch = os.read(sys.stdin.fileno(), 1)
+            if not ch:
+                sys.stderr.write("\r\n")
+                sys.stderr.flush()
+                return "close"
+
+            b = ch[0]
+            if b in (10, 13):
+                sys.stderr.write("\r\n")
+                sys.stderr.flush()
+                return buf.decode("utf-8", errors="replace").strip()
+
+            if b in (8, 127):
+                if buf:
+                    buf.pop()
+                    sys.stderr.write("\b \b")
+                    sys.stderr.flush()
+                continue
+
+            if b < 32:
+                continue
+
+            buf.append(b)
+            sys.stderr.write(chr(b))
+            sys.stderr.flush()
+
+    def handle_local_command(cmd: str) -> bool:
+        if cmd in ("", "resume"):
+            return True
+        if cmd == "close":
+            send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.CLOSE)
+            return False
+        if cmd == "ping":
+            send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.PING)
+            return True
+        if cmd.startswith("resize "):
+            parts = cmd.split()
+            if len(parts) != 3:
+                state.event_queue.put("usage: resize COLS ROWS")
+                return True
+            try:
+                cols = int(parts[1])
+                rows = int(parts[2])
+            except ValueError:
+                state.event_queue.put("usage: resize COLS ROWS")
+                return True
+            send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.RESIZE, cols=cols, rows=rows)
+            return True
+
+        state.event_queue.put(f"unknown local command: {cmd}")
+        return True
+
+    print(
+        "Raw input mode active. All keys (including Ctrl+C/Ctrl+X) are sent to remote. Ctrl+] for local commands.",
+        file=sys.stderr,
+    )
+
+    if not sys.stdin.isatty():
+        # Fallback for non-TTY stdin: still send input as it arrives.
+        while not state.closed_event.is_set():
+            drain_events(state)
+            data = sys.stdin.buffer.read(INPUT_BATCH_MAX_BYTES)
+            if not data:
+                send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.CLOSE)
+                break
+            send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.INPUT, data)
+        return
+
+    fd = sys.stdin.fileno()
+    old_attrs = termios.tcgetattr(fd)
+    try:
+        tty.setraw(fd)
+        while not state.closed_event.is_set():
+            drain_events(state)
+            ready, _, _ = select.select([sys.stdin], [], [], 0.05)
+            if not ready:
+                continue
+
+            data = os.read(fd, 1)
+            if not data:
+                send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.CLOSE)
+                break
+
+            if data == LOCAL_ESCAPE_BYTE:
+                keep_running = handle_local_command(read_local_command())
+                if not keep_running:
+                    break
+                continue
+
+            # Coalesce a short burst of bytes to reduce packet overhead for fast typing.
+            batched = bytearray(data)
+            enter_local_command = False
+            deadline = time.monotonic() + INPUT_BATCH_WINDOW_SEC
+            while len(batched) < INPUT_BATCH_MAX_BYTES:
+                remaining = deadline - time.monotonic()
+                if remaining <= 0:
+                    break
+                more_ready, _, _ = select.select([sys.stdin], [], [], remaining)
+                if not more_ready:
+                    break
+                next_byte = os.read(fd, 1)
+                if not next_byte:
+                    break
+                if next_byte == LOCAL_ESCAPE_BYTE:
+                    enter_local_command = True
+                    break
+                batched.extend(next_byte)
+                if next_byte == b'\r' or next_byte == b'\t':
+                    break
+                deadline = time.monotonic() + INPUT_BATCH_WINDOW_SEC
+
+            if batched:
+                send_shell_frame(transport, state, state.pb2.mesh.RemoteShell.INPUT, bytes(batched))
+
+            if enter_local_command:
+                keep_running = handle_local_command(read_local_command())
+                if not keep_running:
+                    break
+    finally:
+        termios.tcsetattr(fd, termios.TCSADRAIN, old_attrs)
+
+
+def main() -> int:
+    args = parse_args()
+    pb2 = load_proto_modules()
+
+    state = SessionState(
+            pb2=pb2,
+        target=parse_node_num(args.to),
+        channel=args.channel,
+        verbose=args.verbose,
+    )
+
+    cols, rows = resolve_initial_terminal_size(args.cols, args.rows)
+
+    transport = open_transport(args)
+    try:
+        wait_for_config_complete(transport, pb2, args.timeout, args.verbose)
+
+        reader = threading.Thread(target=reader_loop, args=(transport, state), daemon=True)
+        reader.start()
+
+        send_shell_frame(transport, state, pb2.mesh.RemoteShell.OPEN, cols=cols, rows=rows)
+        if not state.opened_event.wait(timeout=args.timeout):
+            raise SystemExit("timed out waiting for OPEN_OK from remote DMShell")
+
+        heartbeat = threading.Thread(target=heartbeat_loop, args=(transport, state), daemon=True)
+        heartbeat.start()
+
+        drain_events(state)
+        if args.command:
+            run_command_mode(transport, state, args.command, args.close_after)
+        else:
+            run_interactive_mode(transport, state)
+
+        state.stopped = True
+        drain_events(state)
+        reader.join(timeout=1.0)
+        heartbeat.join(timeout=1.0)
+        state.close_replay_log()
+    finally:
+        transport.close()
+
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

mcp-server/README.md

@@ -166,73 +166,15 @@ rather than auto-`sudo`'ing mid-run.
 
 ## Environment variables
 
-| Var                        | Default                                                     | Purpose                                                                                                          |
-| -------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
-| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini`                      | Pin the firmware repo                                                                                            |
-| `MESHTASTIC_PIO_BIN`       | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location                                                                                          |
-| `MESHTASTIC_ESPTOOL_BIN`   | `<firmware>/.venv/bin/esptool` → `$PATH`                    | Override esptool                                                                                                 |
-| `MESHTASTIC_NRFUTIL_BIN`   | `$PATH`                                                     | Override nrfutil                                                                                                 |
-| `MESHTASTIC_PICOTOOL_BIN`  | `$PATH`                                                     | Override picotool                                                                                                |
-| `MESHTASTIC_MCP_SEED`      | `mcp-<user>-<host>`                                         | PSK seed for test-harness session (CI override)                                                                  |
-| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log`                              | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it)                                              |
-| `MESHTASTIC_MCP_TCP_HOST`  | unset                                                       | `host` or `host:port` of a `meshtasticd` daemon to surface as a TCP device (see "TCP / native-host nodes" below) |
-
-## TCP / native-host nodes
-
-The `native-macos` and `native` PlatformIO envs build a headless `meshtasticd`
-binary that runs on the host (Apple Silicon / Intel macOS, or Linux Portduino).
-The daemon exposes the meshtastic TCP API on port `4403` rather than a USB
-serial endpoint — point the MCP server at it via `MESHTASTIC_MCP_TCP_HOST`:
-
-```bash
-# 1. Build + run a daemon on this host (see variants/native/portduino/platformio.ini
-#    for full Homebrew prereqs and CH341 LoRa-adapter setup).
-pio run -e native-macos
-~/.meshtasticd/meshtasticd
-
-# 2. Point the MCP server at it.
-export MESHTASTIC_MCP_TCP_HOST=localhost     # or host:port, default port 4403
-```
-
-**First-run gotcha — MAC address.** `meshtasticd` derives its MAC from the
-USB adapter's serial-number / product strings. Many cheap CH341 dongles
-(MeshStick included — VID 0x1A86 / PID 0x5512) ship with `iSerialNumber=0`
-and `iProduct=0`, so the daemon aborts on boot with `*** Blank MAC Address
-not allowed!`. Set the MAC explicitly in `config.yaml`:
-
-```yaml
-# Under General:
-MACAddress: 02:CA:FE:BA:BE:01
-```
-
-Use a locally-administered address (first byte's second-LSB set, e.g.
-`02:*` / `06:*` / `0A:*` / `0E:*`) to avoid colliding with a real OUI.
-
-There is also a `--hwid AA:BB:CC:DD:EE:FF` CLI flag visible in
-`meshtasticd --help`, but it is **currently broken** in
-`MAC_from_string()` (`src/platform/portduino/PortduinoGlue.cpp`): the
-function strips colons from its parameter but then reads bytes from the
-global `portduino_config.mac_address`, so `--hwid` is silently overridden
-when `MACAddress:` is also set, and crashes the daemon (uncaught
-`std::invalid_argument: stoi: no conversion`) when it isn't. Use the YAML
-form until that's fixed upstream.
-
-`list_devices` will surface the daemon as `tcp://localhost:4403` with
-`likely_meshtastic=True`, so `device_info`, `list_nodes`, `get_config`,
-`set_config`, `set_owner`, `send_text`, `userprefs_*`, and the admin RPCs
-auto-select it when no `port` is passed. Pass `port="tcp://other-host:9999"`
-explicitly to target a different daemon.
-
-**Tools that don't apply to a TCP/native node** (no USB hardware to operate
-on) raise a clear `ConnectionError` rather than failing mysteriously:
-`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`,
-`serial_open` (use info/admin tools directly), and the vendor escape hatches
-`esptool_*`, `nrfutil_*`, `picotool_*`. `pio_flash` against a `native*` env
-similarly raises — there's no upload step; use `build` and run the binary
-directly.
-
-The pytest harness in `tests/` still assumes USB-attached devices per role —
-TCP-aware fixtures are not part of this surface yet.
+| Var                        | Default                                                     | Purpose                                                             |
+| -------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------- |
+| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini`                      | Pin the firmware repo                                               |
+| `MESHTASTIC_PIO_BIN`       | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location                                             |
+| `MESHTASTIC_ESPTOOL_BIN`   | `<firmware>/.venv/bin/esptool` → `$PATH`                    | Override esptool                                                    |
+| `MESHTASTIC_NRFUTIL_BIN`   | `$PATH`                                                     | Override nrfutil                                                    |
+| `MESHTASTIC_PICOTOOL_BIN`  | `$PATH`                                                     | Override picotool                                                   |
+| `MESHTASTIC_MCP_SEED`      | `mcp-<user>-<host>`                                         | PSK seed for test-harness session (CI override)                     |
+| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log`                              | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it) |
 
 ## Hardware Test Suite
 

mcp-server/src/meshtastic_mcp/connection.py

@@ -1,4 +1,4 @@
-"""Context manager for meshtastic interface connections (serial + TCP).
+"""Context manager for meshtastic.SerialInterface connections.
 
 Every info/admin tool goes through `connect(port)` so we have a single place
 that:
@@ -6,16 +6,8 @@ that:
   - fails fast if a serial_session is already holding the port,
   - guarantees `.close()` is called, even on exception.
 
-Two transports:
-  - Serial: USB-attached firmware on `/dev/cu.*` / `/dev/ttyUSB*` / `COM*`.
-  - TCP: a `meshtasticd` daemon (e.g. the native macOS / Linux Portduino
-    headless build) addressed as `tcp://host[:port]` (default port 4403).
-    Surfaced by `devices.list_devices()` when `MESHTASTIC_MCP_TCP_HOST` is
-    set, so `resolve_port(None)` auto-selects it like a USB candidate.
-
-Both `SerialInterface` and `TCPInterface` block on construction waiting for
-the node database; that's fine for v1 since every tool is a short-lived
-request.
+The `SerialInterface` blocks on construction waiting for the node database;
+that's fine for v1 since every tool is a short-lived request.
 """
 
 from __future__ import annotations
@@ -25,107 +17,20 @@ from typing import Iterator
 
 from . import devices, registry
 
-DEFAULT_TCP_PORT = 4403
-TCP_SCHEME = "tcp://"
-TCP_HOST_ENV = "MESHTASTIC_MCP_TCP_HOST"
-
 
 class ConnectionError(RuntimeError):
     pass
 
 
-def is_tcp_port(port: str | None) -> bool:
-    return bool(port) and port.startswith(TCP_SCHEME)
-
-
-def parse_tcp_port(port: str) -> tuple[str, int]:
-    """Parse `tcp://host[:port]` → (host, port). Defaults to 4403.
-
-    Validates host shape (non-empty, no path separators) and port range
-    (1..65535). Raises `ConnectionError` on malformed input — never lets
-    a raw `ValueError` bubble up to a tool surface.
-    """
-    if not port.startswith(TCP_SCHEME):
-        raise ConnectionError(
-            f"Invalid TCP endpoint {port!r}: expected '{TCP_SCHEME}host[:port]'."
-        )
-    rest = port[len(TCP_SCHEME) :]
-    if ":" in rest:
-        host, port_str = rest.rsplit(":", 1)
-        try:
-            tcp_port = int(port_str)
-        except ValueError as e:
-            raise ConnectionError(
-                f"Invalid TCP endpoint {port!r}: port {port_str!r} is not an integer."
-            ) from e
-    else:
-        host, tcp_port = rest, DEFAULT_TCP_PORT
-    if not host:
-        raise ConnectionError(f"Invalid TCP endpoint {port!r}: empty host.")
-    if any(c in host for c in ("/", "\\")):
-        raise ConnectionError(
-            f"Invalid TCP endpoint {port!r}: host {host!r} contains a path "
-            "separator. TCP hostnames cannot contain '/' or '\\' — did you "
-            "pass a serial port path or a Windows drive path by mistake?"
-        )
-    if not (1 <= tcp_port <= 65535):
-        raise ConnectionError(
-            f"Invalid TCP endpoint {port!r}: port {tcp_port} out of range "
-            "(must be 1..65535)."
-        )
-    return host, tcp_port
-
-
-def normalize_tcp_endpoint(endpoint: str) -> str:
-    r"""Normalize `host`, `host:port`, or `tcp://host[:port]` → canonical
-    `tcp://host:port` form. One place that owns the lock-key shape.
-
-    Defers all validation to `parse_tcp_port`, so path-like inputs
-    (`/dev/cu.foo`, `C:\Windows\…`), empty hosts, non-integer ports,
-    and out-of-range ports raise `ConnectionError` here too.
-    """
-    if endpoint.startswith(TCP_SCHEME):
-        canonical = endpoint
-    elif ":" in endpoint:
-        canonical = f"{TCP_SCHEME}{endpoint}"
-    else:
-        canonical = f"{TCP_SCHEME}{endpoint}:{DEFAULT_TCP_PORT}"
-    host, port = parse_tcp_port(canonical)
-    return f"{TCP_SCHEME}{host}:{port}"
-
-
-def reject_if_tcp(port: str | None, tool_name: str) -> None:
-    """Raise if `port` is a TCP endpoint — for tools that need real USB
-    hardware (flash, bootloader, vendor escape hatches, serial monitor).
-
-    Only checks the explicit arg; auto-selection via env var is the caller's
-    responsibility to handle if it matters.
-    """
-    if is_tcp_port(port):
-        raise ConnectionError(
-            f"{tool_name} is not applicable to TCP/native nodes ({port}). "
-            "This tool requires USB-attached hardware."
-        )
-
-
 def resolve_port(port: str | None) -> str:
-    """Pick a port: explicit > sole likely_meshtastic candidate > error.
-
-    A `tcp://` string passes through (after canonicalization). When `port`
-    is None and no USB candidates are present, `MESHTASTIC_MCP_TCP_HOST`
-    is consulted via `devices.list_devices()`.
-    """
+    """Pick a port: explicit > sole likely_meshtastic candidate > error."""
     if port:
-        if is_tcp_port(port):
-            return normalize_tcp_endpoint(port)
         return port
     candidates = [d for d in devices.list_devices() if d["likely_meshtastic"]]
     if not candidates:
         raise ConnectionError(
-            "No Meshtastic devices detected. Plug one in, set "
-            f"{TCP_HOST_ENV}=<host[:port]> for a meshtasticd daemon, "
-            "or pass `port` explicitly. Run `list_devices` with "
-            "include_unknown=True to see all serial ports."
+            "No Meshtastic devices detected. Plug one in or pass `port` explicitly. "
+            "Run `list_devices` with include_unknown=True to see all serial ports."
         )
     if len(candidates) > 1:
         ports = ", ".join(c["port"] for c in candidates)
@@ -138,62 +43,17 @@ def resolve_port(port: str | None) -> str:
 
 @contextmanager
 def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
-    """Open a meshtastic interface (serial or TCP) and always close it.
+    """Open a `meshtastic.SerialInterface` and always close it.
 
-    For serial: raises `ConnectionError` immediately if another serial
-    session holds the port (a `pio device monitor` in `serial_sessions/`).
-    For TCP: no exclusive-access requirement, so the serial-session check
-    is skipped — but the `port_lock` still serializes parallel `connect()`
-    calls to the same daemon endpoint.
-
-    `timeout_s` is plumbed through to both `SerialInterface(timeout=...)`
-    and `TCPInterface(timeout=...)`. The meshtastic library uses the value
-    as the reply-wait deadline for `localNode.waitForConfig()` during
-    construction and for any subsequent admin RPC. `int()`-converted at
-    the boundary because the upstream API expects whole seconds.
+    Raises `ConnectionError` immediately if another serial session holds the
+    port (a `pio device monitor` in `serial_sessions/`, for instance).
     """
-    resolved = resolve_port(port)
-    timeout = int(timeout_s)
-
-    if is_tcp_port(resolved):
-        from meshtastic.tcp_interface import (
-            TCPInterface,  # type: ignore[import-untyped]
-        )
-
-        host, tcp_port = parse_tcp_port(resolved)
-        lock = registry.port_lock(resolved)
-        if not lock.acquire(blocking=False):
-            raise ConnectionError(
-                f"TCP endpoint {resolved} is busy — another device operation "
-                "is in flight. Retry shortly."
-            )
-
-        iface = None
-        try:
-            iface = TCPInterface(
-                hostname=host,
-                portNumber=tcp_port,
-                connectNow=True,
-                noProto=False,
-                timeout=timeout,
-            )
-            yield iface
-        finally:
-            if iface is not None:
-                try:
-                    iface.close()
-                except Exception:
-                    pass
-            try:
-                lock.release()
-            except RuntimeError:
-                pass
-        return
-
     from meshtastic.serial_interface import (
         SerialInterface,  # type: ignore[import-untyped]
     )
 
+    resolved = resolve_port(port)
+
     active = registry.active_session_for_port(resolved)
     if active is not None:
         raise ConnectionError(
@@ -210,12 +70,7 @@ def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
 
     iface = None
     try:
-        iface = SerialInterface(
-            devPath=resolved,
-            connectNow=True,
-            noProto=False,
-            timeout=timeout,
-        )
+        iface = SerialInterface(devPath=resolved, connectNow=True, noProto=False)
         yield iface
     finally:
         if iface is not None:

mcp-server/src/meshtastic_mcp/devices.py

@@ -1,18 +1,13 @@
-"""USB/serial + TCP device discovery.
+"""USB/serial device discovery.
 
 Combines the canonical `meshtastic.util.findPorts()` allowlist/blocklist with
 the richer metadata (`serial.tools.list_ports.comports()`) so callers see
 VID/PID, descriptions, and manufacturer strings alongside the "is this likely
 a Meshtastic device" signal.
-
-If `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` is set, a synthetic entry for the
-`meshtasticd` daemon at that endpoint is prepended to the result, so
-`resolve_port(None)` auto-selects it like a USB candidate.
 """
 
 from __future__ import annotations
 
-import os
 from typing import Any
 
 from serial.tools import list_ports
@@ -24,45 +19,6 @@ def _to_hex(value: int | None) -> str | None:
     return f"0x{value:04x}"
 
 
-def _tcp_endpoint_from_env() -> dict[str, Any] | None:
-    """Synthesize a TCP device entry from MESHTASTIC_MCP_TCP_HOST, if set.
-
-    If the env var is malformed (non-integer port, path-like host, etc.),
-    return an entry with `likely_meshtastic=False` and the parser error in
-    the description, rather than raising — `list_devices` is the diagnostic
-    tool a user reaches for when their env var isn't working, so it must
-    not crash on misconfiguration.
-    """
-    host = os.environ.get("MESHTASTIC_MCP_TCP_HOST")
-    if not host:
-        return None
-    # Lazy import to avoid a circular dependency (connection imports devices).
-    from . import connection
-
-    try:
-        port = connection.normalize_tcp_endpoint(host)
-        description = "meshtasticd (TCP)"
-        likely = True
-    except connection.ConnectionError as e:
-        # Surface the raw env-var value plus the parser's reason so the
-        # user can see exactly what they set and why it was rejected.
-        # Don't double the scheme if the user already prefixed `tcp://`.
-        port = host if host.startswith(connection.TCP_SCHEME) else f"tcp://{host}"
-        description = f"meshtasticd (TCP) — invalid MESHTASTIC_MCP_TCP_HOST: {e}"
-        likely = False
-    return {
-        "port": port,
-        "vid": None,
-        "pid": None,
-        "description": description,
-        "manufacturer": None,
-        "product": None,
-        "serial_number": None,
-        "likely_meshtastic": likely,
-        "blacklisted": False,
-    }
-
-
 def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
     """Return enriched info for serial ports, flagging Meshtastic candidates.
 
@@ -114,22 +70,6 @@ def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
             }
         )
 
-    # Append the TCP endpoint (if env var set) and sort everything together.
-    tcp_entry = _tcp_endpoint_from_env()
-    if tcp_entry is not None:
-        results.append(tcp_entry)
-
-    # Stable ordering: likely_meshtastic first; within rank, TCP wins over
-    # USB (explicit env-var configuration takes precedence over USB
-    # enumeration); then by port path. A misconfigured TCP entry has
-    # likely_meshtastic=False and lands among the other ignored entries —
-    # it does NOT pre-empt real USB devices at the top of the list.
-    results.sort(
-        key=lambda r: (
-            not r["likely_meshtastic"],
-            not r["port"].startswith("tcp://"),
-            r["port"],
-        )
-    )
-
+    # Stable ordering: likely_meshtastic first, then by port path
+    results.sort(key=lambda r: (not r["likely_meshtastic"], r["port"]))
     return results

mcp-server/src/meshtastic_mcp/flash.py

@@ -17,7 +17,7 @@ from typing import Any
 
 import serial
 
-from . import boards, config, connection, devices, pio, userprefs
+from . import boards, config, devices, pio, userprefs
 
 # Meshtastic variants use both `esp32s3` and `esp32-s3` style names across
 # variants/*/platformio.ini (no consistency enforced). Accept both spellings.
@@ -46,18 +46,6 @@ def _require_confirm(confirm: bool, operation: str) -> None:
         )
 
 
-def _reject_native_env(env: str, operation: str) -> None:
-    """`native*` envs build a host executable, not firmware — there's no
-    upload step. The user wants `build` (or just runs the binary directly).
-    """
-    if env.startswith("native"):
-        raise FlashError(
-            f"{operation} is not applicable for env {env!r}: native envs "
-            "produce a host executable, not flashable firmware. Use `build` "
-            "instead, then run the resulting binary directly."
-        )
-
-
 def _artifacts_for(env: str) -> list[Path]:
     build_dir = config.firmware_root() / ".pio" / "build" / env
     if not build_dir.is_dir():
@@ -153,8 +141,6 @@ def flash(
     that pio performs will pick up the injected values.
     """
     _require_confirm(confirm, "flash")
-    _reject_native_env(env, "flash")
-    connection.reject_if_tcp(port, "flash")
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
         result = pio.run(
             ["run", "-e", env, "-t", "upload", "--upload-port", port],
@@ -214,7 +200,6 @@ def erase_and_flash(
     in that case) since a cached factory.bin would not reflect the new prefs.
     """
     _require_confirm(confirm, "erase_and_flash")
-    connection.reject_if_tcp(port, "erase_and_flash")
     _check_esp32_env(env)
 
     if userprefs_overrides and skip_build:
@@ -272,7 +257,6 @@ def update_flash(
     overrides are provided we always force a rebuild.
     """
     _require_confirm(confirm, "update_flash")
-    connection.reject_if_tcp(port, "update_flash")
     _check_esp32_env(env)
 
     if userprefs_overrides and skip_build:
@@ -407,7 +391,6 @@ def touch_1200bps(
 
     Returns `{ok, former_port, new_port, new_port_vid_pid, attempts}`.
     """
-    connection.reject_if_tcp(port, "touch_1200bps")
     before_list = devices.list_devices(include_unknown=True)
     before_ports = {d["port"] for d in before_list}
 

mcp-server/src/meshtastic_mcp/hw_tools.py

@@ -16,7 +16,7 @@ import subprocess
 from pathlib import Path
 from typing import Any, Sequence
 
-from . import config, connection, pio
+from . import config, pio
 
 _TIMEOUT_SHORT = 30
 _TIMEOUT_LONG = 600
@@ -102,7 +102,6 @@ def _parse_esptool_chip_info(stdout: str) -> dict[str, Any]:
 
 
 def esptool_chip_info(port: str) -> dict[str, Any]:
-    connection.reject_if_tcp(port, "esptool_chip_info")
     binary = config.esptool_bin()
     # `chip_id` prints chip + mac + crystal + features. `flash_id` adds flash.
     combined = _run(binary, ["--port", port, "flash_id"], timeout=_TIMEOUT_SHORT)
@@ -117,7 +116,6 @@ def esptool_chip_info(port: str) -> dict[str, Any]:
 def esptool_erase_flash(port: str, confirm: bool = False) -> dict[str, Any]:
     """Full-chip erase. Leaves the device unbootable until reflashed."""
     _require_confirm(confirm, "esptool_erase_flash")
-    connection.reject_if_tcp(port, "esptool_erase_flash")
     binary = config.esptool_bin()
     # esptool v5 uses `erase-flash`, older uses `erase_flash`. Try the new name
     # first; if it fails with unknown command, retry old.
@@ -136,7 +134,6 @@ def esptool_raw(
     """Raw esptool passthrough. Destructive subcommands require confirm=True."""
     if not args:
         raise ToolError("args must not be empty")
-    connection.reject_if_tcp(port, "esptool_raw")
     # Find the first non-flag arg (the subcommand).
     subcommand = next((a for a in args if not a.startswith("-")), None)
     if subcommand and subcommand.replace("-", "_") in {
@@ -159,7 +156,6 @@ NRFUTIL_DESTRUCTIVE = {"dfu", "settings"}
 
 def nrfutil_dfu(port: str, package_path: str, confirm: bool = False) -> dict[str, Any]:
     _require_confirm(confirm, "nrfutil_dfu")
-    connection.reject_if_tcp(port, "nrfutil_dfu")
     pkg = Path(package_path).expanduser()
     if not pkg.is_file():
         raise ToolError(f"Package not found: {pkg}")
@@ -217,7 +213,6 @@ def _parse_picotool_info(stdout: str) -> dict[str, Any]:
 def picotool_info(port: str | None = None) -> dict[str, Any]:
     """Read device info from a Pico in BOOTSEL mode. `port` is informational
     only — picotool auto-detects."""
-    connection.reject_if_tcp(port, "picotool_info")
     binary = config.picotool_bin()
     res = _run(binary, ["info", "-a"], timeout=_TIMEOUT_SHORT)
     if res["exit_code"] != 0:

mcp-server/src/meshtastic_mcp/serial_session.py

@@ -71,10 +71,6 @@ def open_session(
     If `env` is supplied, pio resolves baud and filters from platformio.ini.
     Otherwise uses the supplied `baud` and `filters` (default `['direct']`).
     """
-    # Lazy import to avoid circular: registry imports serial_session.
-    from . import connection
-
-    connection.reject_if_tcp(port, "serial_open")
     args = ["device", "monitor", "--port", port, "--no-reconnect"]
     effective_filters: list[str]
     effective_baud: int = baud

mcp-server/tests/unit/test_connection_tcp.py

@@ -1,383 +0,0 @@
-"""TCP transport plumbing in connection.py + devices.py.
-
-Pure-Python tests — no real device or daemon required. Mocks `TCPInterface`
-when exercising `connect()`.
-"""
-
-from __future__ import annotations
-
-from unittest.mock import patch
-
-import pytest
-from meshtastic_mcp import connection, devices
-
-# ---------- helpers --------------------------------------------------------
-
-
-class TestIsTcpPort:
-    def test_tcp_scheme(self) -> None:
-        assert connection.is_tcp_port("tcp://localhost") is True
-        assert connection.is_tcp_port("tcp://localhost:4403") is True
-        assert connection.is_tcp_port("tcp://192.168.1.50:9999") is True
-
-    def test_serial_paths(self) -> None:
-        assert connection.is_tcp_port("/dev/cu.usbmodem1234") is False
-        assert connection.is_tcp_port("/dev/ttyUSB0") is False
-        assert connection.is_tcp_port("COM3") is False
-
-    def test_empty_or_none(self) -> None:
-        assert connection.is_tcp_port(None) is False
-        assert connection.is_tcp_port("") is False
-
-
-class TestParseTcpPort:
-    def test_default_port(self) -> None:
-        assert connection.parse_tcp_port("tcp://localhost") == ("localhost", 4403)
-
-    def test_explicit_port(self) -> None:
-        assert connection.parse_tcp_port("tcp://localhost:9999") == (
-            "localhost",
-            9999,
-        )
-
-    def test_ip_with_port(self) -> None:
-        assert connection.parse_tcp_port("tcp://192.168.1.50:4403") == (
-            "192.168.1.50",
-            4403,
-        )
-
-
-class TestNormalizeTcpEndpoint:
-    def test_bare_host(self) -> None:
-        assert connection.normalize_tcp_endpoint("localhost") == "tcp://localhost:4403"
-
-    def test_host_port(self) -> None:
-        assert (
-            connection.normalize_tcp_endpoint("localhost:5000")
-            == "tcp://localhost:5000"
-        )
-
-    def test_full_url(self) -> None:
-        assert (
-            connection.normalize_tcp_endpoint("tcp://1.2.3.4") == "tcp://1.2.3.4:4403"
-        )
-        assert (
-            connection.normalize_tcp_endpoint("tcp://1.2.3.4:9999")
-            == "tcp://1.2.3.4:9999"
-        )
-
-    def test_idempotent(self) -> None:
-        once = connection.normalize_tcp_endpoint("localhost:4403")
-        twice = connection.normalize_tcp_endpoint(once)
-        assert once == twice == "tcp://localhost:4403"
-
-    def test_path_like_endpoint_rejected(self) -> None:
-        # Serial port paths and Windows drive paths are common config typos
-        # (someone passes a serial path to MESHTASTIC_MCP_TCP_HOST). Reject
-        # rather than producing a nonsense `tcp:///dev/cu.foo:4403` URL.
-        with pytest.raises(connection.ConnectionError, match="path separator"):
-            connection.normalize_tcp_endpoint("/dev/cu.foo")
-        with pytest.raises(connection.ConnectionError):
-            connection.normalize_tcp_endpoint("tcp:///dev/cu.foo:4403")
-        with pytest.raises(connection.ConnectionError):
-            connection.normalize_tcp_endpoint(r"C:\Windows\System32")
-
-    def test_non_integer_port_rejected(self) -> None:
-        with pytest.raises(connection.ConnectionError, match="not an integer"):
-            connection.normalize_tcp_endpoint("tcp://host:notaport")
-        with pytest.raises(connection.ConnectionError, match="not an integer"):
-            connection.normalize_tcp_endpoint("host:notaport")
-
-    def test_empty_host_rejected(self) -> None:
-        with pytest.raises(connection.ConnectionError, match="empty host"):
-            connection.normalize_tcp_endpoint("tcp://:4403")
-
-    def test_port_out_of_range_rejected(self) -> None:
-        with pytest.raises(connection.ConnectionError, match="out of range"):
-            connection.normalize_tcp_endpoint("tcp://host:0")
-        with pytest.raises(connection.ConnectionError, match="out of range"):
-            connection.normalize_tcp_endpoint("tcp://host:65536")
-        with pytest.raises(connection.ConnectionError, match="out of range"):
-            connection.normalize_tcp_endpoint("host:99999")
-
-
-class TestParseTcpPortValidation:
-    def test_missing_scheme_rejected(self) -> None:
-        # parse_tcp_port is a low-level helper that requires the scheme.
-        # Misuse should fail loudly rather than silently mis-parsing.
-        with pytest.raises(connection.ConnectionError, match="expected"):
-            connection.parse_tcp_port("localhost:4403")
-
-    def test_negative_port_rejected(self) -> None:
-        with pytest.raises(connection.ConnectionError, match="out of range"):
-            connection.parse_tcp_port("tcp://host:-1")
-
-
-# ---------- reject_if_tcp --------------------------------------------------
-
-
-class TestRejectIfTcp:
-    def test_rejects_tcp(self) -> None:
-        with pytest.raises(connection.ConnectionError, match="not applicable"):
-            connection.reject_if_tcp("tcp://localhost", "esptool_chip_info")
-
-    def test_passes_through_serial(self) -> None:
-        connection.reject_if_tcp("/dev/cu.usbmodem1", "esptool_chip_info")  # no raise
-
-    def test_passes_through_none(self) -> None:
-        # None means "auto-detect"; not the explicit-arg case we guard.
-        connection.reject_if_tcp(None, "esptool_chip_info")  # no raise
-
-
-# ---------- resolve_port ---------------------------------------------------
-
-
-class TestResolvePort:
-    def test_explicit_serial_passthrough(self) -> None:
-        assert connection.resolve_port("/dev/cu.usbmodem999") == "/dev/cu.usbmodem999"
-
-    def test_explicit_tcp_normalized(self) -> None:
-        assert connection.resolve_port("tcp://localhost") == "tcp://localhost:4403"
-
-    def test_no_port_no_devices_errors(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-        with patch.object(devices, "list_devices", return_value=[]):
-            with pytest.raises(
-                connection.ConnectionError, match="No Meshtastic devices"
-            ):
-                connection.resolve_port(None)
-
-    def test_no_port_one_candidate_selected(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-        fake = [{"port": "/dev/cu.usbmodem1", "likely_meshtastic": True}]
-        with patch.object(devices, "list_devices", return_value=fake):
-            assert connection.resolve_port(None) == "/dev/cu.usbmodem1"
-
-    def test_no_port_multiple_candidates_errors(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-        fake = [
-            {"port": "/dev/cu.usbmodem1", "likely_meshtastic": True},
-            {"port": "/dev/cu.usbmodem2", "likely_meshtastic": True},
-        ]
-        with patch.object(devices, "list_devices", return_value=fake):
-            with pytest.raises(connection.ConnectionError, match="Multiple"):
-                connection.resolve_port(None)
-
-    def test_env_var_surfaces_tcp_via_devices(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost")
-        # Don't patch list_devices — let the real env-var path run, but stub
-        # the USB enumeration to keep the test hermetic.
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            assert connection.resolve_port(None) == "tcp://localhost:4403"
-
-
-# ---------- devices.list_devices TCP entry --------------------------------
-
-
-class TestDevicesTcpEntry:
-    def test_no_env_var_no_tcp_entry(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            ds = devices.list_devices()
-        assert all(not d["port"].startswith("tcp://") for d in ds)
-
-    def test_env_var_adds_tcp_entry(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "myhost:9999")
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            ds = devices.list_devices()
-        tcp = [d for d in ds if d["port"].startswith("tcp://")]
-        assert len(tcp) == 1
-        assert tcp[0]["port"] == "tcp://myhost:9999"
-        assert tcp[0]["likely_meshtastic"] is True
-        assert tcp[0]["description"] == "meshtasticd (TCP)"
-
-    def test_tcp_entry_first_in_results(self, monkeypatch: pytest.MonkeyPatch) -> None:
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost")
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            ds = devices.list_devices()
-        assert ds, "expected at least the TCP entry"
-        assert ds[0]["port"].startswith("tcp://")
-
-    def test_invalid_env_var_does_not_break_list_devices(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        # `list_devices` is the diagnostic tool reached for when an env var
-        # isn't working — it must not throw on misconfiguration.
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            ds = devices.list_devices(include_unknown=True)
-        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
-        assert len(tcp) == 1
-        assert tcp[0]["likely_meshtastic"] is False
-        assert "invalid MESHTASTIC_MCP_TCP_HOST" in tcp[0]["description"]
-        assert "not an integer" in tcp[0]["description"]
-
-    def test_invalid_env_var_excluded_from_resolve_port_autodetect(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        # `likely_meshtastic=False` keeps the bad TCP entry out of the
-        # auto-select path — `resolve_port(None)` should still report
-        # "no Meshtastic devices" rather than picking a broken endpoint.
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            with pytest.raises(connection.ConnectionError, match="No Meshtastic"):
-                connection.resolve_port(None)
-
-    def test_invalid_env_var_does_not_double_tcp_scheme(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        # If a user mistakenly sets `MESHTASTIC_MCP_TCP_HOST=tcp://host:bad`,
-        # the diagnostic entry must surface the raw value as-is rather than
-        # producing `tcp://tcp://host:bad`.
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "tcp://host:notaport")
-        with patch("meshtastic_mcp.devices.list_ports.comports", return_value=[]):
-            ds = devices.list_devices(include_unknown=True)
-        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
-        assert len(tcp) == 1
-        assert tcp[0]["port"] == "tcp://host:notaport"
-        assert "tcp://tcp://" not in tcp[0]["port"]
-
-    def test_invalid_env_var_does_not_pre_empt_real_usb_devices(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        # Sort ordering: a misconfigured TCP env var must NOT take position 0
-        # ahead of real USB candidates. Position 0 is reserved for the highest
-        # rank (likely_meshtastic=True), with TCP-before-USB as a tiebreaker
-        # within rank.
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "host:notaport")
-
-        # Stub a USB Meshtastic candidate (Espressif VID, port present in
-        # findPorts).
-        class FakeInfo:
-            def __init__(self, device: str, vid: int, pid: int) -> None:
-                self.device = device
-                self.vid = vid
-                self.pid = pid
-                self.description = "Heltec V3"
-                self.manufacturer = "Espressif"
-                self.product = "USB JTAG/serial"
-                self.serial_number = "abc"
-
-        fake_port = FakeInfo("/dev/cu.usbmodem4201", 0x303A, 0x1001)
-        with patch(
-            "meshtastic_mcp.devices.list_ports.comports", return_value=[fake_port]
-        ), patch(
-            "meshtastic.util.findPorts",
-            return_value=["/dev/cu.usbmodem4201"],
-        ):
-            ds = devices.list_devices(include_unknown=True)
-
-        assert ds, "expected at least the USB + TCP entries"
-        # Real USB candidate must be at position 0 — it's likely_meshtastic.
-        assert ds[0]["port"] == "/dev/cu.usbmodem4201"
-        assert ds[0]["likely_meshtastic"] is True
-        # The malformed TCP entry exists but lands among the unlikely entries.
-        tcp = [d for d in ds if "TCP" in (d["description"] or "")]
-        assert len(tcp) == 1
-        assert tcp[0]["likely_meshtastic"] is False
-        assert ds.index(tcp[0]) > 0
-
-    def test_likely_tcp_entry_wins_tiebreak_over_usb(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        # Conversely, a *valid* TCP env var should sort ahead of USB
-        # candidates of equal likely_meshtastic rank — explicit env-var
-        # configuration is a precedence signal.
-        monkeypatch.setenv("MESHTASTIC_MCP_TCP_HOST", "localhost:4403")
-
-        class FakeInfo:
-            def __init__(self, device: str, vid: int, pid: int) -> None:
-                self.device = device
-                self.vid = vid
-                self.pid = pid
-                self.description = "Heltec V3"
-                self.manufacturer = "Espressif"
-                self.product = "USB JTAG/serial"
-                self.serial_number = "abc"
-
-        fake_port = FakeInfo("/dev/cu.usbmodem4201", 0x303A, 0x1001)
-        with patch(
-            "meshtastic_mcp.devices.list_ports.comports", return_value=[fake_port]
-        ), patch(
-            "meshtastic.util.findPorts",
-            return_value=["/dev/cu.usbmodem4201"],
-        ):
-            ds = devices.list_devices()
-
-        assert ds[0]["port"] == "tcp://localhost:4403"
-        assert ds[0]["likely_meshtastic"] is True
-
-
-# ---------- connect() routing ---------------------------------------------
-
-
-class TestConnectRoutesTcp:
-    def test_connect_uses_tcp_interface_for_tcp_port(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        """Verify the TCP branch instantiates `TCPInterface(hostname, portNumber)`
-        and never touches `SerialInterface`."""
-        # Make sure the env var doesn't leak in and confuse resolve_port.
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-
-        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp, patch(
-            "meshtastic.serial_interface.SerialInterface"
-        ) as mock_serial:
-            mock_tcp.return_value.close.return_value = None
-            with connection.connect(port="tcp://example.com:1234", timeout_s=12.0):
-                pass
-
-        mock_tcp.assert_called_once_with(
-            hostname="example.com",
-            portNumber=1234,
-            connectNow=True,
-            noProto=False,
-            timeout=12,
-        )
-        mock_serial.assert_not_called()
-
-    def test_connect_plumbs_timeout_to_serial_interface(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        """Verify the serial branch also propagates `timeout_s` so callers
-        passing a custom timeout to `device_info` / `list_nodes` / etc. don't
-        silently get the library default."""
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-
-        with patch("meshtastic.serial_interface.SerialInterface") as mock_serial, patch(
-            "meshtastic.tcp_interface.TCPInterface"
-        ) as mock_tcp:
-            mock_serial.return_value.close.return_value = None
-            with connection.connect(port="/dev/cu.fake", timeout_s=20.0):
-                pass
-
-        mock_serial.assert_called_once_with(
-            devPath="/dev/cu.fake",
-            connectNow=True,
-            noProto=False,
-            timeout=20,
-        )
-        mock_tcp.assert_not_called()
-
-    def test_connect_releases_lock_on_tcp_failure(
-        self, monkeypatch: pytest.MonkeyPatch
-    ) -> None:
-        monkeypatch.delenv("MESHTASTIC_MCP_TCP_HOST", raising=False)
-        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp:
-            mock_tcp.side_effect = RuntimeError("boom")
-            with pytest.raises(RuntimeError, match="boom"):
-                with connection.connect(port="tcp://locktest:4403"):
-                    pass
-
-        # Lock should be released — a second connect attempt must not fail
-        # with "busy".
-        with patch("meshtastic.tcp_interface.TCPInterface") as mock_tcp:
-            mock_tcp.return_value.close.return_value = None
-            with connection.connect(port="tcp://locktest:4403"):
-                pass

platformio.ini

@@ -120,7 +120,7 @@ lib_deps =
 [radiolib_base]
 lib_deps =
 	# renovate: datasource=github-tags depName=RadioLib packageName=jgromes/RadioLib
-	https://github.com/jgromes/RadioLib/archive/afe72ae46a343e15e3cac7f26ac585c7f98bffe5.zip
+	https://github.com/jgromes/RadioLib/archive/refs/tags/7.6.0.zip
 
 [device-ui_base]
 lib_deps =

src/FSCommon.cpp

@@ -79,46 +79,28 @@ bool copyFile(const char *from, const char *to)
 bool renameFile(const char *pathFrom, const char *pathTo)
 {
 #ifdef FSCom
+
+#ifdef ARCH_ESP32
+    // take SPI Lock
     spiLock->lock();
+    // rename was fixed for ESP32 IDF LittleFS in April
     bool result = FSCom.rename(pathFrom, pathTo);
     spiLock->unlock();
     return result;
 #else
-    return false;
+    // copyFile does its own locking.
+    if (copyFile(pathFrom, pathTo) && FSCom.remove(pathFrom)) {
+        return true;
+    } else {
+        return false;
+    }
+#endif
+
 #endif
 }
 
 #include <vector>
 
-/**
- * @brief Platform-agnostic filesystem format / wipe.
- *
- * On embedded targets (ESP32, NRF52, STM32WL, RP2040) this calls the
- * native FSCom.format() which erases and reinitialises the LittleFS
- * partition.
- *
- * On Portduino the fs::FS backend has no format() method. We instead
- * delete /prefs (the only meshtastic data directory written at runtime)
- * and return. rmDir("/prefs") is already called unconditionally by
- * factoryReset() so this is a proven primitive on Portduino.
- * FSBegin() is a no-op (#define FSBegin() true) on Portduino.
- *
- * @return true on success, false on failure or if no filesystem is configured.
- */
-bool fsFormat()
-{
-#ifdef FSCom
-#if defined(ARCH_PORTDUINO)
-    rmDir("/prefs");
-    return FSBegin();
-#else
-    return FSCom.format();
-#endif
-#else
-    return false;
-#endif
-}
-
 /**
  * @brief Get the list of files in a directory.
  *
@@ -141,21 +123,23 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
 
     File file = root.openNextFile();
     while (file) {
-#ifdef ARCH_ESP32
-        const char *filepath = file.path();
-#else
-        const char *filepath = file.name();
-#endif
         if (file.isDirectory() && !String(file.name()).endsWith(".")) {
             if (levels) {
-                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(filepath, levels - 1);
+#ifdef ARCH_ESP32
+                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.path(), levels - 1);
+#else
+                std::vector<meshtastic_FileInfo> subDirFilenames = getFiles(file.name(), levels - 1);
+#endif
                 filenames.insert(filenames.end(), subDirFilenames.begin(), subDirFilenames.end());
                 file.close();
             }
         } else {
             meshtastic_FileInfo fileInfo = {"", static_cast<uint32_t>(file.size())};
-            strncpy(fileInfo.file_name, filepath, sizeof(fileInfo.file_name) - 1);
-            fileInfo.file_name[sizeof(fileInfo.file_name) - 1] = '\0';
+#ifdef ARCH_ESP32
+            strcpy(fileInfo.file_name, file.path());
+#else
+            strcpy(fileInfo.file_name, file.name());
+#endif
             if (!String(fileInfo.file_name).endsWith(".")) {
                 filenames.push_back(fileInfo);
             }
@@ -179,59 +163,98 @@ std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels)
 void listDir(const char *dirname, uint8_t levels, bool del)
 {
 #ifdef FSCom
+#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
     char buffer[255];
+#endif
     File root = FSCom.open(dirname, FILE_O_READ);
-    if (!root || !root.isDirectory())
+    if (!root) {
         return;
+    }
+    if (!root.isDirectory()) {
+        return;
+    }
 
     File file = root.openNextFile();
-    while (file && file.name()[0]) { // file.name()[0] check: workaround for Adafruit LittleFS nRF52 bug #4395
-#ifdef ARCH_ESP32
-        const char *filepath = file.path();
-#else
-        const char *filepath = file.name();
-#endif
+    while (
+        file &&
+        file.name()[0]) { // This file.name() check is a workaround for a bug in the Adafruit LittleFS nrf52 glue (see issue 4395)
         if (file.isDirectory() && !String(file.name()).endsWith(".")) {
             if (levels) {
-                listDir(filepath, levels - 1, del);
+#ifdef ARCH_ESP32
+                listDir(file.path(), levels - 1, del);
                 if (del) {
-                    LOG_DEBUG("Remove %s", filepath);
-                    strncpy(buffer, filepath, sizeof(buffer) - 1);
-                    buffer[sizeof(buffer) - 1] = '\0';
+                    LOG_DEBUG("Remove %s", file.path());
+                    strncpy(buffer, file.path(), sizeof(buffer));
                     file.close();
                     FSCom.rmdir(buffer);
                 } else {
                     file.close();
                 }
+#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
+                listDir(file.name(), levels - 1, del);
+                if (del) {
+                    LOG_DEBUG("Remove %s", file.name());
+                    strncpy(buffer, file.name(), sizeof(buffer));
+                    file.close();
+                    FSCom.rmdir(buffer);
+                } else {
+                    file.close();
+                }
+#else
+                LOG_DEBUG(" %s (directory)", file.name());
+                listDir(file.name(), levels - 1, del);
+                file.close();
+#endif
             }
         } else {
+#ifdef ARCH_ESP32
             if (del) {
-                LOG_DEBUG("Delete %s", filepath);
-                strncpy(buffer, filepath, sizeof(buffer) - 1);
-                buffer[sizeof(buffer) - 1] = '\0';
+                LOG_DEBUG("Delete %s", file.path());
+                strncpy(buffer, file.path(), sizeof(buffer));
                 file.close();
                 FSCom.remove(buffer);
             } else {
-                LOG_DEBUG(" %s (%i Bytes)", filepath, file.size());
+                LOG_DEBUG(" %s (%i Bytes)", file.path(), file.size());
                 file.close();
             }
+#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
+            if (del) {
+                LOG_DEBUG("Delete %s", file.name());
+                strncpy(buffer, file.name(), sizeof(buffer));
+                file.close();
+                FSCom.remove(buffer);
+            } else {
+                LOG_DEBUG(" %s (%i Bytes)", file.name(), file.size());
+                file.close();
+            }
+#else
+            LOG_DEBUG("   %s (%i Bytes)", file.name(), file.size());
+            file.close();
+#endif
         }
         file = root.openNextFile();
     }
 #ifdef ARCH_ESP32
-    const char *rootpath = root.path();
-#else
-    const char *rootpath = root.name();
-#endif
     if (del) {
-        LOG_DEBUG("Remove %s", rootpath);
-        strncpy(buffer, rootpath, sizeof(buffer) - 1);
-        buffer[sizeof(buffer) - 1] = '\0';
+        LOG_DEBUG("Remove %s", root.path());
+        strncpy(buffer, root.path(), sizeof(buffer));
         root.close();
         FSCom.rmdir(buffer);
     } else {
         root.close();
     }
+#elif (defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
+    if (del) {
+        LOG_DEBUG("Remove %s", root.name());
+        strncpy(buffer, root.name(), sizeof(buffer));
+        root.close();
+        FSCom.rmdir(buffer);
+    } else {
+        root.close();
+    }
+#else
+    root.close();
+#endif
 #endif
 }
 
@@ -245,7 +268,14 @@ void listDir(const char *dirname, uint8_t levels, bool del)
 void rmDir(const char *dirname)
 {
 #ifdef FSCom
+
+#if (defined(ARCH_ESP32) || defined(ARCH_RP2040) || defined(ARCH_PORTDUINO))
     listDir(dirname, 10, true);
+#elif defined(ARCH_NRF52)
+    // nRF52 implementation of LittleFS has a recursive delete function
+    FSCom.rmdir_r(dirname);
+#endif
+
 #endif
 }
 

src/FSCommon.h

@@ -52,7 +52,6 @@ void fsInit();
 void fsListFiles();
 bool copyFile(const char *from, const char *to);
 bool renameFile(const char *pathFrom, const char *pathTo);
-bool fsFormat();
 std::vector<meshtastic_FileInfo> getFiles(const char *dirname, uint8_t levels);
 void listDir(const char *dirname, uint8_t levels, bool del = false);
 void rmDir(const char *dirname);

src/RedirectablePrint.cpp

@@ -7,7 +7,6 @@
 #include "memGet.h"
 #include "mesh/generated/meshtastic/mesh.pb.h"
 #include <assert.h>
-#include <atomic>
 #include <cstring>
 #include <memory>
 #include <stdexcept>
@@ -21,22 +20,6 @@
 #if HAS_NETWORKING
 extern meshtastic::Syslog syslog;
 #endif
-
-namespace
-{
-std::atomic<bool> serialHalLogSuppressed{false};
-}
-
-void RedirectablePrint::setSerialHalLogSuppressed(bool suppressed)
-{
-    serialHalLogSuppressed.store(suppressed);
-}
-
-bool RedirectablePrint::isSerialHalLogSuppressed()
-{
-    return serialHalLogSuppressed.load();
-}
-
 void RedirectablePrint::rpInit()
 {
 #ifdef HAS_FREE_RTOS
@@ -294,10 +277,6 @@ meshtastic_LogRecord_Level RedirectablePrint::getLogLevel(const char *logLevel)
 void RedirectablePrint::log(const char *logLevel, const char *format, ...)
 {
 
-    if (isSerialHalLogSuppressed()) {
-        return;
-    }
-
     // append \n to format
     size_t len = strlen(format);
     auto newFormat = std::unique_ptr<char[]>(new char[len + 2]);

src/RedirectablePrint.h

@@ -24,11 +24,6 @@ class RedirectablePrint : public Print
   public:
     explicit RedirectablePrint(Print *_dest) : dest(_dest) {}
 
-    /// Suppress all log output while a SerialHal transaction is in progress.
-    // Unclear if this is necessary, but it seems to help with response speeds.
-    static void setSerialHalLogSuppressed(bool suppressed);
-    static bool isSerialHalLogSuppressed();
-
     /**
      * Set a new destination
      */

src/SafeFile.cpp

@@ -7,6 +7,10 @@ static File openFile(const char *filename, bool fullAtomic)
 {
     concurrency::LockGuard g(spiLock);
     LOG_DEBUG("Opening %s, fullAtomic=%d", filename, fullAtomic);
+#ifdef ARCH_NRF52
+    FSCom.remove(filename);
+    return FSCom.open(filename, FILE_O_WRITE);
+#endif
     if (!fullAtomic) {
         FSCom.remove(filename); // Nuke the old file to make space (ignore if it !exists)
     }
@@ -63,6 +67,9 @@ bool SafeFile::close()
     f.close();
     spiLock->unlock();
 
+#ifdef ARCH_NRF52
+    return true;
+#endif
     if (!testReadback())
         return false;
 

src/configuration.h

@@ -80,7 +80,7 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 // Pre-hop drop handling (compile-time flag).
 #ifndef MESHTASTIC_PREHOP_DROP
-#define MESHTASTIC_PREHOP_DROP 1
+#define MESHTASTIC_PREHOP_DROP 0
 #endif
 
 /// Convert a preprocessor name into a quoted string

src/main.cpp

@@ -951,12 +951,8 @@ void setup()
 #endif
 #endif
 
-    std::unique_ptr<RadioInterface> rIf;
-    if (!config.lora.serial_hal_only) {
-        rIf = initLoRa();
-    } else {
-        LOG_INFO("skipping LoRa radio init, for serialHal");
-    }
+    auto rIf = initLoRa();
+
     lateInitVariant(); // Do board specific init (see extra_variants/README.md for documentation)
 
 #if !MESHTASTIC_EXCLUDE_MQTT
@@ -1000,9 +996,10 @@ void setup()
 
     // Start airtime logger thread.
     airTime = new AirTime();
-    if (!rIf && !config.lora.serial_hal_only)
+
+    if (!rIf)
         RECORD_CRITICALERROR(meshtastic_CriticalErrorCode_NO_RADIO);
-    else if (rIf) {
+    else {
         // Log bit rate to debug output
         LOG_DEBUG("LoRA bitrate = %f bytes / sec", (float(meshtastic_Constants_DATA_PAYLOAD_LEN) /
                                                     (float(rIf->getPacketTime(meshtastic_Constants_DATA_PAYLOAD_LEN)))) *

src/mesh/NodeDB.cpp

@@ -1611,10 +1611,12 @@ bool NodeDB::saveToDisk(int saveWhat)
 
     if (!success) {
         LOG_ERROR("Failed to save to disk, retrying");
+#ifdef ARCH_NRF52 // @geeksville is not ready yet to say we should do this on other platforms.  See bug #4184 discussion
         spiLock->lock();
-        fsFormat();
+        FSCom.format();
         spiLock->unlock();
 
+#endif
         success = saveToDiskNoRetry(saveWhat);
 
         RECORD_CRITICALERROR(success ? meshtastic_CriticalErrorCode_FLASH_CORRUPTION_RECOVERABLE

src/mesh/PhoneAPI.cpp

@@ -522,6 +522,11 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
             // LOG_INFO("nodeinfo: num=0x%x, lastseen=%u, id=%s, name=%s", nodeInfoForPhone.num, nodeInfoForPhone.last_heard,
             // nodeInfoForPhone.user.id, nodeInfoForPhone.user.long_name);
 
+            // Occasional progress logging. (readIndex==2 will be true for the first non-us node)
+            if (readIndex == 2 || readIndex % 20 == 0) {
+                LOG_DEBUG("nodeinfo: %d/%d", readIndex, nodeDB->getNumMeshNodes());
+            }
+
             fromRadioScratch.which_payload_variant = meshtastic_FromRadio_node_info_tag;
             fromRadioScratch.node_info = infoToSend;
             prefetchNodeInfos();
@@ -652,11 +657,9 @@ void PhoneAPI::releaseQueueStatusPhonePacket()
 void PhoneAPI::prefetchNodeInfos()
 {
     bool added = false;
-    bool wasEmpty = false;
     // Keep the queue topped up so BLE reads stay responsive even if DB fetches take a moment.
     {
         concurrency::LockGuard guard(&nodeInfoMutex);
-        wasEmpty = nodeInfoQueue.empty();
         while (nodeInfoQueue.size() < kNodePrefetchDepth) {
             auto nextNode = nodeDB->readNextMeshNode(readIndex);
             if (!nextNode)
@@ -670,15 +673,11 @@ void PhoneAPI::prefetchNodeInfos()
             info.via_mqtt = isUs ? false : info.via_mqtt;
             info.is_favorite = info.is_favorite || isUs;
             nodeInfoQueue.push_back(info);
-            // Log progress here (at fetch time) so readIndex is accurate and each value logs only once.
-            if (readIndex == 2 || readIndex % 20 == 0) {
-                LOG_DEBUG("nodeinfo: %d/%d", readIndex, nodeDB->getNumMeshNodes());
-            }
             added = true;
         }
     }
 
-    if (added && wasEmpty)
+    if (added)
         onNowHasData(0);
 }
 

src/mesh/RadioInterface.cpp

@@ -25,7 +25,6 @@
 
 #ifdef ARCH_PORTDUINO
 #include "platform/portduino/PortduinoGlue.h"
-#include "platform/portduino/SerialHal.h"
 #include "platform/portduino/SimRadio.h"
 #include "platform/portduino/USBHal.h"
 #endif
@@ -293,9 +292,6 @@ std::unique_ptr<RadioInterface> initLoRa()
               portduino_config.lora_spi_dev.c_str());
     if (portduino_config.lora_spi_dev == "ch341") {
         RadioLibHAL = ch341Hal;
-    } else if (portduino_config.lora_spi_dev == "serial") {
-        RadioLibHAL = new SerialHal(portduino_config.lora_serial_device, portduino_config.lora_serial_baud,
-                                    (uint32_t)portduino_config.lora_serial_timeout_ms);
     } else {
         if (RadioLibHAL != nullptr) {
             delete RadioLibHAL;

src/mesh/RadioLibInterface.h

@@ -179,6 +179,8 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
     /** Attempt to find a packet in the TxQueue. Returns true if the packet was found. */
     virtual bool findInTxQueue(NodeNum from, PacketId id) override;
 
+    uint8_t packetsInTxQueue() { return txQueue.getMaxLen() - txQueue.getFree(); }
+
     /**
      * Request randomness sourced from the LoRa modem, if supported by the active RadioLib interface.
      * @return true if len bytes were produced, false otherwise.

src/mesh/SerialHalDevice.cpp

@@ -1,385 +0,0 @@
-#include "mesh/SerialHalDevice.h"
-#include "NodeDB.h"
-#include "SPILock.h"
-#include "concurrency/Periodic.h"
-#include "configuration.h"
-#include "mesh/StreamAPI.h"
-#include "mesh/generated/meshtastic/config.pb.h"
-#include <Arduino.h>
-#include <SPI.h>
-#include <cstring>
-#include <stdint.h>
-
-#if defined(ARCH_ESP32)
-#if defined(HW_SPI1_DEVICE)
-extern SPIClass SPI1;
-#endif
-#endif
-
-namespace
-{
-constexpr uint32_t SERIAL_PI_RISING = 1;
-constexpr uint32_t SERIAL_PI_FALLING = 2;
-constexpr uint32_t SERIAL_PI_INPUT = 0;
-constexpr uint32_t SERIAL_PI_OUTPUT = 1;
-constexpr size_t MAX_INTERRUPT_SLOTS = 8;
-constexpr int32_t INTERRUPT_POLL_MS = 5;
-
-struct InterruptSlot {
-    bool used = false;
-    uint32_t pin = 0;
-    uint32_t mode = 0;
-    volatile bool pending = false;
-};
-
-concurrency::Lock interruptMutex;
-InterruptSlot interruptSlots[MAX_INTERRUPT_SLOTS];
-StreamAPI *interruptStreamApi = nullptr;
-concurrency::Periodic *interruptEmitter = nullptr;
-
-int findSlotByPinLocked(uint32_t pin)
-{
-    for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-        if (interruptSlots[i].used && interruptSlots[i].pin == pin) {
-            return (int)i;
-        }
-    }
-    return -1;
-}
-
-int allocateSlotLocked()
-{
-    for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-        if (!interruptSlots[i].used) {
-            return (int)i;
-        }
-    }
-    return -1;
-}
-
-#if defined(ARCH_PORTDUINO) || defined(ARCH_RP2040)
-PinStatus toInterruptMode(uint32_t serialMode)
-{
-    if (serialMode == SERIAL_PI_RISING) {
-        return PinStatus::RISING;
-    }
-    if (serialMode == SERIAL_PI_FALLING) {
-        return PinStatus::FALLING;
-    }
-    return PinStatus::CHANGE;
-}
-#else
-int toInterruptMode(uint32_t serialMode)
-{
-    if (serialMode == SERIAL_PI_RISING) {
-        return RISING;
-    }
-    if (serialMode == SERIAL_PI_FALLING) {
-        return FALLING;
-    }
-    return CHANGE;
-}
-#endif
-
-int32_t pumpInterruptEvents();
-
-void ensureInterruptEmitter()
-{
-    if (!interruptEmitter) {
-        interruptEmitter = new concurrency::Periodic("SerialHalIrqEmitter", pumpInterruptEvents);
-    }
-}
-
-void emitInterruptEvent(uint32_t pin, StreamAPI *streamApi)
-{
-    if (streamApi == nullptr) {
-        return;
-    }
-
-    meshtastic_SerialHalResponse event = meshtastic_SerialHalResponse_init_zero;
-    event.transaction_id = 0; // asynchronous interrupt notification
-    event.result = meshtastic_SerialHalResponse_Result_OK;
-    event.value = pin; // host-side SerialHal treats value as interrupt pin
-    SerialHalDevice::emitResponse(event, streamApi);
-}
-
-void markPendingBySlot(uint8_t slot)
-{
-    if (slot < MAX_INTERRUPT_SLOTS && interruptSlots[slot].used) {
-        interruptSlots[slot].pending = true;
-    }
-}
-
-void isr0()
-{
-    markPendingBySlot(0);
-}
-void isr1()
-{
-    markPendingBySlot(1);
-}
-void isr2()
-{
-    markPendingBySlot(2);
-}
-void isr3()
-{
-    markPendingBySlot(3);
-}
-void isr4()
-{
-    markPendingBySlot(4);
-}
-void isr5()
-{
-    markPendingBySlot(5);
-}
-void isr6()
-{
-    markPendingBySlot(6);
-}
-void isr7()
-{
-    markPendingBySlot(7);
-}
-
-void (*const isrTable[MAX_INTERRUPT_SLOTS])() = {isr0, isr1, isr2, isr3, isr4, isr5, isr6, isr7};
-
-int32_t pumpInterruptEvents()
-{
-    uint32_t toEmit[MAX_INTERRUPT_SLOTS] = {0};
-    size_t emitCount = 0;
-    StreamAPI *streamApi = nullptr;
-
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        streamApi = interruptStreamApi;
-        for (size_t i = 0; i < MAX_INTERRUPT_SLOTS; ++i) {
-            if (interruptSlots[i].used && interruptSlots[i].pending) {
-                interruptSlots[i].pending = false;
-                toEmit[emitCount++] = interruptSlots[i].pin;
-            }
-        }
-    }
-
-    for (size_t i = 0; i < emitCount; ++i) {
-        emitInterruptEvent(toEmit[i], streamApi);
-    }
-
-    return INTERRUPT_POLL_MS;
-}
-} // namespace
-
-// Helper to safely set response result
-static inline void setResponseError(meshtastic_SerialHalResponse &response, meshtastic_SerialHalResponse_Result result,
-                                    const char *error = nullptr)
-{
-    response.result = result;
-    if (error != nullptr) {
-        snprintf(response.error, sizeof(response.error), "%s", error);
-    }
-}
-
-void SerialHalDevice::handleCommand(const uint8_t *buf, size_t len, StreamAPI *streamApi)
-{
-    if (buf == nullptr || streamApi == nullptr) {
-        return;
-    }
-
-    // Validate role - SerialHal commands only handled when config.lora.serial_hal_only
-    if (!config.lora.serial_hal_only) {
-        meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-        response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-        snprintf(response.error, sizeof(response.error), "SerialHal not enabled for this role");
-        emitResponse(response, streamApi);
-        return;
-    }
-
-    // Decode the command
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    if (!pb_decode_from_bytes(buf, len, &meshtastic_SerialHalCommand_msg, &cmd)) {
-        meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-        response.result = meshtastic_SerialHalResponse_Result_BAD_REQUEST;
-        snprintf(response.error, sizeof(response.error), "Failed to decode SerialHalCommand");
-        emitResponse(response, streamApi);
-        return;
-    }
-
-    // Initialize response with matching transaction_id
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    response.transaction_id = cmd.transaction_id;
-    response.result = meshtastic_SerialHalResponse_Result_OK;
-
-    // Dispatch to operation handler
-    switch (cmd.type) {
-    case meshtastic_SerialHalCommand_Type_PIN_MODE:
-        handlePinMode(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DIGITAL_WRITE:
-        handleDigitalWrite(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DIGITAL_READ:
-        handleDigitalRead(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_ATTACH_INTERRUPT:
-        handleAttachInterrupt(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_DETACH_INTERRUPT:
-        handleDetachInterrupt(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_SPI_TRANSFER:
-        handleSpiTransfer(cmd, response);
-        break;
-    case meshtastic_SerialHalCommand_Type_NOOP:
-        // NOOP: just return OK
-        break;
-    default:
-        response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-        snprintf(response.error, sizeof(response.error), "Unknown SerialHal operation type");
-        break;
-    }
-
-    emitResponse(response, streamApi);
-}
-
-void SerialHalDevice::handlePinMode(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: pinMode pin=%u mode=%u", cmd.pin, cmd.mode);
-    if (cmd.mode == SERIAL_PI_INPUT) {
-        pinMode((int)cmd.pin, INPUT);
-    } else if (cmd.mode == SERIAL_PI_OUTPUT) {
-        pinMode((int)cmd.pin, OUTPUT);
-    } else {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_BAD_REQUEST, "Unsupported pin mode");
-    }
-}
-
-void SerialHalDevice::handleDigitalWrite(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: digitalWrite pin=%u value=%u", cmd.pin, cmd.value);
-    digitalWrite((int)cmd.pin, cmd.value ? HIGH : LOW);
-}
-
-void SerialHalDevice::handleDigitalRead(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: digitalRead pin=%u", cmd.pin);
-    response.value = (uint32_t)digitalRead((int)cmd.pin);
-}
-
-void SerialHalDevice::handleAttachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: attachInterrupt pin=%u mode=%u", cmd.pin, cmd.mode);
-
-    ensureInterruptEmitter();
-
-    int slot = -1;
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        slot = findSlotByPinLocked(cmd.pin);
-        if (slot < 0) {
-            slot = allocateSlotLocked();
-        }
-
-        if (slot >= 0) {
-            interruptSlots[slot].used = true;
-            interruptSlots[slot].pin = cmd.pin;
-            interruptSlots[slot].mode = cmd.mode;
-            interruptSlots[slot].pending = false;
-        }
-    }
-
-    if (slot < 0) {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_ERROR, "No interrupt slots available");
-        return;
-    }
-
-    ::attachInterrupt((int)cmd.pin, isrTable[slot], toInterruptMode(cmd.mode));
-}
-
-void SerialHalDevice::handleDetachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    // LOG_DEBUG("SerialHalDevice: detachInterrupt pin=%u", cmd.pin);
-
-    ::detachInterrupt((int)cmd.pin);
-
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        const int slot = findSlotByPinLocked(cmd.pin);
-        if (slot >= 0) {
-            interruptSlots[slot] = InterruptSlot{};
-        }
-    }
-}
-
-void SerialHalDevice::handleSpiTransfer(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response)
-{
-    if (cmd.data.size == 0) {
-        return;
-    }
-
-#if !ARCH_PORTDUINO
-    if (spiLock == nullptr) {
-        setResponseError(response, meshtastic_SerialHalResponse_Result_ERROR, "SPI lock not initialized");
-        return;
-    }
-
-#if defined(HW_SPI1_DEVICE)
-    SPIClass &spiBus = SPI1;
-#else
-    SPIClass &spiBus = SPI;
-#endif
-
-    response.data.size = cmd.data.size;
-
-    {
-        concurrency::LockGuard guard(spiLock);
-        spiBus.beginTransaction(SPISettings(4000000, MSBFIRST, SPI_MODE0));
-#ifdef ARCH_ESP32
-        spiBus.transferBytes(cmd.data.bytes, response.data.bytes, cmd.data.size);
-#else
-        spiBus.transfer(cmd.data.bytes, response.data.bytes, cmd.data.size);
-#endif
-        spiBus.endTransaction();
-    }
-#else
-    // SPI wiring is board/radio-specific; keep this explicit for now.
-    response.result = meshtastic_SerialHalResponse_Result_UNSUPPORTED;
-    snprintf(response.error, sizeof(response.error), "SPI not supported on this platform");
-#endif
-}
-
-void SerialHalDevice::emitResponse(const meshtastic_SerialHalResponse &response, StreamAPI *streamApi)
-{
-    if (streamApi == nullptr) {
-        return;
-    }
-
-    // Encode the response
-    uint8_t encoded[meshtastic_SerialHalResponse_size] = {0};
-    const size_t responseLen =
-        pb_encode_to_bytes(encoded, sizeof(encoded), &meshtastic_SerialHalResponse_msg, static_cast<const void *>(&response));
-
-    if (responseLen == 0 || responseLen > 0xFFFF) {
-        LOG_ERROR("SerialHalDevice: Failed to encode response (len=%zu)", responseLen);
-        return;
-    }
-
-    // Build frame with StreamAPI framing: START1 SERIALHAL_MAGIC LEN_H LEN_L [payload]
-    constexpr uint8_t START1 = 0x94;
-    constexpr uint8_t SERIALHAL_MAGIC = 0xA5;
-
-    uint8_t hdr[4];
-    hdr[0] = START1;
-    hdr[1] = SERIALHAL_MAGIC;
-    hdr[2] = (uint8_t)((responseLen >> 8) & 0xFF); // LEN_H
-    hdr[3] = (uint8_t)(responseLen & 0xFF);        // LEN_L
-
-    // Emit via StreamAPI (this uses the internal txBuf + framing)
-    streamApi->emitSerialHalResponse(hdr, sizeof(hdr), encoded, responseLen);
-
-    // Keep a recent stream instance so async interrupt events can be emitted.
-    {
-        concurrency::LockGuard lock(&interruptMutex);
-        interruptStreamApi = streamApi;
-    }
-}

src/mesh/SerialHalDevice.h

@@ -1,90 +0,0 @@
-#pragma once
-
-#include "mesh/generated/meshtastic/serial_hal.pb.h"
-#include <cstdint>
-
-/**
- * @brief Device-side handler for SerialHal GPIO/SPI operations over StreamAPI framing.
- *
- * This module decodes SerialHalCommand protobufs received from a host and executes
- * the requested GPIO (pinMode, digitalWrite, digitalRead, attach/detachInterrupt) or
- * SPI operations, then returns results via SerialHalResponse.
- *
- * Usage:
- *  1. Override StreamAPI::handleSerialHalCommand() in a subclass
- *  2. Call SerialHalDevice::handleCommand(buf, len, streamApi)
- *  3. SerialHalDevice will decode, execute, and emit the response
- *
- * The handler is only active when config.lora.serial_hal_only is true.
- */
-
-class StreamAPI; // forward declaration
-
-class SerialHalDevice
-{
-  public:
-    /**
-     * @brief Process a SerialHalCommand and emit a response.
-     *
-     * Decodes the protobuf, validates the operation, executes it on the device,
-     * and writes the response back via the StreamAPI instance.
-     *
-     * @param buf     Pointer to the encoded SerialHalCommand protobuf payload (not including framing)
-     * @param len     Length of the encoded payload
-     * @param streamApi Pointer to the StreamAPI instance (used for emitting responses)
-     */
-    static void handleCommand(const uint8_t *buf, size_t len, StreamAPI *streamApi);
-
-    /**
-     * @brief Emit a SerialHalResponse back to the host via StreamAPI framing.
-     *
-     * Encodes the response protobuf and sends it with proper framing (START1 SERIALHAL_MAGIC LEN_H LEN_L payload).
-     *
-     * @param response The response to send
-     * @param streamApi Pointer to the StreamAPI instance
-     */
-    static void emitResponse(const meshtastic_SerialHalResponse &response, StreamAPI *streamApi);
-
-  private:
-    /**
-     * @brief Execute a GPIO pinMode operation.
-     * @param cmd Decoded SerialHalCommand with PIN_MODE type
-     * @param response Response object to fill with result
-     */
-    static void handlePinMode(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a GPIO digitalWrite operation.
-     * @param cmd Decoded SerialHalCommand with DIGITAL_WRITE type
-     * @param response Response object to fill with result
-     */
-    static void handleDigitalWrite(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a GPIO digitalRead operation.
-     * @param cmd Decoded SerialHalCommand with DIGITAL_READ type
-     * @param response Response object to fill with result (value field contains read result)
-     */
-    static void handleDigitalRead(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute an attachInterrupt operation.
-     * @param cmd Decoded SerialHalCommand with ATTACH_INTERRUPT type
-     * @param response Response object to fill with result
-     */
-    static void handleAttachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute a detachInterrupt operation.
-     * @param cmd Decoded SerialHalCommand with DETACH_INTERRUPT type
-     * @param response Response object to fill with result
-     */
-    static void handleDetachInterrupt(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-
-    /**
-     * @brief Execute an SPI transfer operation.
-     * @param cmd Decoded SerialHalCommand with SPI_TRANSFER type and data to send
-     * @param response Response object to fill with result (data field contains received bytes)
-     */
-    static void handleSpiTransfer(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse &response);
-};

src/mesh/StreamAPI.cpp

@@ -1,15 +1,12 @@
 #include "StreamAPI.h"
 #include "PowerFSM.h"
 #include "RTC.h"
-#include "RedirectablePrint.h"
-#include "SerialHalDevice.h"
 #include "Throttle.h"
 #include "concurrency/LockGuard.h"
 #include "configuration.h"
 
 #define START1 0x94
 #define START2 0xc3
-#define SERIALHAL_MAGIC 0xa5 // second framing byte for SerialHal frames (START1 SH_MAGIC LEN_H LEN_L PAYLOAD)
 #define HEADER_LEN 4
 
 int32_t StreamAPI::runOncePart()
@@ -82,29 +79,18 @@ int32_t StreamAPI::handleRecStream(const char *buf, uint16_t bufLen)
         if (ptr == 0) { // looking for START1
             if (c != START1)
                 rxPtr = 0;     // failed to find framing
-        } else if (ptr == 1) { // discriminate frame type on second byte
-            if (c == START2) {
-                rxIsSerialHal = false; // standard ToRadio frame
-                serialHalRxActive.store(false);
-                RedirectablePrint::setSerialHalLogSuppressed(false);
-            } else if (c == SERIALHAL_MAGIC) {
-                rxIsSerialHal = true; // SerialHal command frame
-                serialHalRxActive.store(true);
-                RedirectablePrint::setSerialHalLogSuppressed(true);
-            } else {
-                rxPtr = 0; // unrecognised second byte — not our frame
-                serialHalRxActive.store(false);
-                RedirectablePrint::setSerialHalLogSuppressed(false);
-            }
+        } else if (ptr == 1) { // looking for START2
+            if (c != START2)
+                rxPtr = 0;                             // failed to find framing
         } else if (ptr >= HEADER_LEN - 1) {            // we have at least read our 4 byte framing
             uint32_t len = (rxBuf[2] << 8) + rxBuf[3]; // big endian 16 bit length follows framing
 
             // console->printf("len %d\n", len);
 
             if (ptr == HEADER_LEN - 1) {
-                // we _just_ finished our 4 byte header, validate length now
-                uint32_t maxLen = rxIsSerialHal ? (uint32_t)meshtastic_SerialHalCommand_size : MAX_TO_FROM_RADIO_SIZE;
-                if (len > maxLen)
+                // we _just_ finished our 4 byte header, validate length now (note: a length of zero is a valid
+                // protobuf also)
+                if (len > MAX_TO_FROM_RADIO_SIZE)
                     rxPtr = 0; // length is bogus, restart search for framing
             }
 
@@ -112,16 +98,8 @@ int32_t StreamAPI::handleRecStream(const char *buf, uint16_t bufLen)
                 if (ptr + 1 >= len + HEADER_LEN) { // have we received all of the payload?
                     rxPtr = 0;                     // start over again on the next packet
 
-                    // Dispatch based on which frame type we identified at byte 1
-                    if (rxIsSerialHal)
-                        handleSerialHalCommand(rxBuf + HEADER_LEN, len);
-                    else
-                        handleToRadio(rxBuf + HEADER_LEN, len);
-
-                    if (rxIsSerialHal)
-                        serialHalRxActive.store(false);
-                    if (rxIsSerialHal)
-                        RedirectablePrint::setSerialHalLogSuppressed(false);
+                    // If we didn't just fail the packet and we now have the right # of bytes, parse it
+                    handleToRadio(rxBuf + HEADER_LEN, len);
                 }
         }
     }
@@ -136,12 +114,7 @@ int32_t StreamAPI::readStream()
     if (!stream->available()) {
         // Nothing available this time, if the computer has talked to us recently, poll often, otherwise let CPU sleep a long time
         bool recentRx = Throttle::isWithinTimespanMs(lastRxMsec, 2000);
-        if (!recentRx)
-            return 250; // Sleep a long time if we haven't heard from the computer in a while
-        if (serialHalRxActive.load())
-            return 0; // If we are in the middle of a SerialHal transaction, don't sleep at all because we want to be as
-                      // responsive as possible to incoming SerialHal bytes
-        return 5;     // Otherwise, poll frequently for new data
+        return recentRx ? 5 : 250;
     } else {
         while (stream->available()) { // Currently we never want to block
             int cInt = stream->read();
@@ -162,30 +135,18 @@ int32_t StreamAPI::readStream()
             if (ptr == 0) { // looking for START1
                 if (c != START1)
                     rxPtr = 0;     // failed to find framing
-            } else if (ptr == 1) { // discriminate frame type on second byte
-                if (c == START2) {
-                    rxIsSerialHal = false; // standard ToRadio frame
-                    serialHalRxActive.store(false);
-                    RedirectablePrint::setSerialHalLogSuppressed(false);
-                } else if (c == SERIALHAL_MAGIC) {
-                    rxIsSerialHal = true; // SerialHal command frame
-                    serialHalRxActive.store(true);
-                    RedirectablePrint::setSerialHalLogSuppressed(true);
-                    LOG_WARN("StreamAPI: Detected SerialHal command frame");
-                } else {
-                    rxPtr = 0; // unrecognised second byte — not our frame
-                    serialHalRxActive.store(false);
-                    RedirectablePrint::setSerialHalLogSuppressed(false);
-                }
+            } else if (ptr == 1) { // looking for START2
+                if (c != START2)
+                    rxPtr = 0;                             // failed to find framing
             } else if (ptr >= HEADER_LEN - 1) {            // we have at least read our 4 byte framing
                 uint32_t len = (rxBuf[2] << 8) + rxBuf[3]; // big endian 16 bit length follows framing
 
                 // console->printf("len %d\n", len);
 
                 if (ptr == HEADER_LEN - 1) {
-                    // we _just_ finished our 4 byte header, validate length now
-                    uint32_t maxLen = rxIsSerialHal ? (uint32_t)meshtastic_SerialHalCommand_size : MAX_TO_FROM_RADIO_SIZE;
-                    if (len > maxLen)
+                    // we _just_ finished our 4 byte header, validate length now (note: a length of zero is a valid
+                    // protobuf also)
+                    if (len > MAX_TO_FROM_RADIO_SIZE)
                         rxPtr = 0; // length is bogus, restart search for framing
                 }
 
@@ -193,16 +154,8 @@ int32_t StreamAPI::readStream()
                     if (ptr + 1 >= len + HEADER_LEN) { // have we received all of the payload?
                         rxPtr = 0;                     // start over again on the next packet
 
-                        // Dispatch based on which frame type we identified at byte 1
-                        if (rxIsSerialHal)
-                            handleSerialHalCommand(rxBuf + HEADER_LEN, len);
-                        else
-                            handleToRadio(rxBuf + HEADER_LEN, len);
-
-                        if (rxIsSerialHal)
-                            serialHalRxActive.store(false);
-                        if (rxIsSerialHal)
-                            RedirectablePrint::setSerialHalLogSuppressed(false);
+                        // If we didn't just fail the packet and we now have the right # of bytes, parse it
+                        handleToRadio(rxBuf + HEADER_LEN, len);
                     }
             }
         }
@@ -246,10 +199,6 @@ void StreamAPI::emitRebooted()
 
 void StreamAPI::emitLogRecord(meshtastic_LogRecord_Level level, const char *src, const char *format, va_list arg)
 {
-    if (serialHalRxActive.load()) {
-        return;
-    }
-
     // IMPORTANT: do NOT touch `fromRadioScratch` or `txBuf` here — those
     // belong to the main packet-emission path and a LOG_ firing during
     // `writeStream()` would corrupt an in-flight encode. We keep a
@@ -300,31 +249,4 @@ void StreamAPI::onConnectionChanged(bool connected)
         // received a packet in a while
         powerFSM.trigger(EVENT_SERIAL_DISCONNECTED);
     }
-}
-
-void StreamAPI::handleSerialHalCommand(const uint8_t *buf, size_t len)
-{
-    // Default implementation: dispatch to SerialHalDevice for GPIO/SPI handling
-    SerialHalDevice::handleCommand(buf, len, this);
-}
-
-void StreamAPI::emitSerialHalResponse(const uint8_t *hdr, size_t hdrLen, const uint8_t *payload, size_t payloadLen)
-{
-    if (hdr == nullptr || hdrLen != 4 || payload == nullptr || payloadLen > meshtastic_SerialHalResponse_size) {
-        LOG_ERROR("StreamAPI: Invalid SerialHal response parameters");
-        return;
-    }
-
-    // Build complete frame in a temporary buffer
-    uint8_t frame[4 + meshtastic_SerialHalResponse_size];
-    memcpy(frame, hdr, hdrLen);
-    memcpy(frame + hdrLen, payload, payloadLen);
-
-    size_t totalLen = hdrLen + payloadLen;
-
-    // Serialize stream writes against other emit operations via streamLock
-    concurrency::LockGuard guard(&streamLock);
-    stream->write(frame, totalLen);
-    stream->flush();
-    LOG_WARN("StreamAPI: Emitted SerialHal response frame (len=%zu)", totalLen);
 }

src/mesh/StreamAPI.h

@@ -4,15 +4,10 @@
 #include "Stream.h"
 #include "concurrency/Lock.h"
 #include "concurrency/OSThread.h"
-#include "generated/meshtastic/serial_hal.pb.h"
-#include <atomic>
 #include <cstdarg>
 
-// Buffer sized for the larger of a full ToRadio/FromRadio payload or a full SerialHalCommand payload, plus header.
-#define MAX_STREAM_PAYLOAD_SIZE                                                                                                  \
-    (MAX_TO_FROM_RADIO_SIZE > (int)meshtastic_SerialHalCommand_size ? MAX_TO_FROM_RADIO_SIZE                                     \
-                                                                    : (int)meshtastic_SerialHalCommand_size)
-#define MAX_STREAM_BUF_SIZE (MAX_STREAM_PAYLOAD_SIZE + (int)sizeof(uint32_t))
+// A To/FromRadio packet + our 32 bit header
+#define MAX_STREAM_BUF_SIZE (MAX_TO_FROM_RADIO_SIZE + sizeof(uint32_t))
 
 /**
  * A version of our 'phone' API that talks over a Stream.  So therefore well suited to use with serial links
@@ -44,8 +39,6 @@ class StreamAPI : public PhoneAPI
 
     uint8_t rxBuf[MAX_STREAM_BUF_SIZE] = {0};
     size_t rxPtr = 0;
-    bool rxIsSerialHal = false; ///< true when the current in-progress frame is a SerialHal frame (START1 SH_MAGIC ...)
-    std::atomic<bool> serialHalRxActive{false};
 
     /// time of last rx, used, to slow down our polling if we haven't heard from anyone
     uint32_t lastRxMsec = 0;
@@ -63,17 +56,6 @@ class StreamAPI : public PhoneAPI
     /// Check the current underlying physical link to see if the client is currently connected
     virtual bool checkIsConnected() override = 0;
 
-    /**
-     * Emit a SerialHal response frame with proper framing (START1 SERIALHAL_MAGIC LEN_H LEN_L payload).
-     * Called by SerialHalDevice to send responses back to the host.
-     *
-     * @param hdr     4-byte header (START1 SERIALHAL_MAGIC LEN_H LEN_L)
-     * @param hdrLen  Length of header (should be 4)
-     * @param payload Encoded SerialHalResponse protobuf payload
-     * @param payloadLen Length of payload
-     */
-    void emitSerialHalResponse(const uint8_t *hdr, size_t hdrLen, const uint8_t *payload, size_t payloadLen);
-
   private:
     /**
      * Read any rx chars from the link and call handleToRadio
@@ -93,12 +75,6 @@ class StreamAPI : public PhoneAPI
      */
     void emitRebooted();
 
-    /**
-     * Called when a complete SerialHal-framed packet has been received.
-     * Default implementation dispatches to SerialHalDevice for GPIO/SPI handling.
-     */
-    virtual void handleSerialHalCommand(const uint8_t *buf, size_t len);
-
     virtual void onConnectionChanged(bool connected) override;
 
     /**

src/mesh/raspihttp/PiWebServer.cpp

@@ -1,34 +1,22 @@
 /*
-Adds a WebServer and WebService callbacks to meshtastic via the Portduino/native target (Linux and
-macOS). The WebServer & Webservices run in a real host thread beside the Portduino threading
-emulation. It replaces the complete ESP32 Webserver libs including generation of SSL certificates,
-because those libs use ESP-specific details that can't be emulated.
+Adds a WebServer and WebService callbacks to meshtastic as Linux Version. The WebServer & Webservices
+runs in a real linux thread beside the portdunio threading emulation. It replaces the complete ESP32
+Webserver libs including generation of SSL certifcicates,  because the use ESP specific details in
+the lib that can't be emulated.
 
 The WebServices adapt to the two major phoneapi functions "handleAPIv1FromRadio,handleAPIv1ToRadio"
-The WebServer just adds basic support to deliver WebContent, so it can be used to
-deliver the WebGui defined by the WebClient Project.
+The WebServer just adds basaic support to deliver WebContent, so it can be used to
+deliver the WebGui definded by the WebClient Project.
 
 Steps to get it running:
-
-Linux (apt):
-1.) Add these libs to the compile and target machine:
+1.) Add these Linux Libs to the compile and target machine:
 
     sudo apt update && \
-        apt -y install openssl libssl-dev libsdl2-dev \
+        apt -y install openssl libssl-dev libopenssl libsdl2-dev \
                      libulfius-dev liborcania-dev
 
-macOS (Homebrew):
-1.) Install prerequisites via Homebrew:
-
-    brew install ulfius openssl@3
-
-    The PlatformIO env (native-macos) picks up compiler/linker flags via
-    `pkg-config`. In particular, OpenSSL needs `pkg-config --cflags --libs openssl@3`
-    so both the Homebrew include path and linker flags are provided; ulfius and its
-    dependencies (liborcania, libyder) are also resolved via `pkg-config`.
-
 2.) Configure the root directory of the web Content in the config.yaml file.
-    The following tags should be included and set at your needs
+    The followinng tags should be included and set at your needs
 
     Example entry in the config.yaml
     Webserver:
@@ -46,10 +34,7 @@ Author: Marc Philipp Hammermann
 mail:   marchammermann@googlemail.com
 
 */
-// Mirrors the guard in PiWebServer.h — see comment there. macOS Homebrew
-// provides ulfius + deps; Linux pulls them via apt. Either way, this
-// translation unit only compiles when the headers are present.
-#ifdef ARCH_PORTDUINO
+#ifdef PORTDUINO_LINUX_HARDWARE
 #if __has_include(<ulfius.h>)
 #include "PiWebServer.h"
 #include "NodeDB.h"

src/mesh/raspihttp/PiWebServer.h

@@ -1,9 +1,5 @@
 #pragma once
-// Portduino webserver is built whenever the ulfius headers are reachable,
-// not only on Linux. macOS users can `brew install ulfius` to enable it;
-// without ulfius the entire body is skipped and main.cpp's matching
-// __has_include guard avoids referencing the type.
-#ifdef ARCH_PORTDUINO
+#ifdef PORTDUINO_LINUX_HARDWARE
 #if __has_include(<ulfius.h>)
 #include "PhoneAPI.h"
 #include "ulfius-cfg.h"

src/modules/AdminModule.cpp

@@ -1412,9 +1412,7 @@ void AdminModule::saveChanges(int saveWhat, bool shouldReboot)
 
 void AdminModule::handleStoreDeviceUIConfig(const meshtastic_DeviceUIConfig &uicfg)
 {
-#if HAS_SCREEN
     nodeDB->saveProto("/prefs/uiconfig.proto", meshtastic_DeviceUIConfig_size, &meshtastic_DeviceUIConfig_msg, &uicfg);
-#endif
 }
 
 void AdminModule::handleSetHamMode(const meshtastic_HamParameters &p)

src/modules/DMShell.cpp

@@ -0,0 +1,604 @@
+#include "DMShell.h"
+
+#if defined(ARCH_PORTDUINO)
+
+#include "Channels.h"
+#include "MeshService.h"
+#include "NodeDB.h"
+#include "Throttle.h"
+#include "configuration.h"
+#include "mesh/generated/meshtastic/mesh.pb.h"
+#include "mesh/mesh-pb-constants.h"
+#include "pb_decode.h"
+#include "pb_encode.h"
+#include <errno.h>
+#include <fcntl.h>
+#include <pty.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+DMShellModule *dmShellModule;
+
+namespace
+{
+constexpr uint16_t PTY_COLS_DEFAULT = 120;
+constexpr uint16_t PTY_ROWS_DEFAULT = 40;
+constexpr size_t MAX_MESSAGE_SIZE = 200;
+} // namespace
+
+DMShellModule::DMShellModule()
+    : SinglePortModule("DMShellModule", meshtastic_PortNum_REMOTE_SHELL_APP), concurrency::OSThread("DMShell", 100)
+{
+    LOG_WARN("DMShell enabled on Portduino: remote shell access is dangerous and intended for trusted debugging only");
+}
+
+ProcessMessage DMShellModule::handleReceived(const meshtastic_MeshPacket &mp)
+{
+    meshtastic_RemoteShell frame = meshtastic_RemoteShell_init_zero;
+    if (!mp.pki_encrypted) {
+        LOG_WARN("DMShell: ignoring packet without PKI from 0x%x", mp.from);
+        return ProcessMessage::STOP;
+    }
+
+    if (!parseFrame(mp, frame)) {
+        LOG_WARN("DMShell: ignoring malformed frame");
+        return ProcessMessage::STOP;
+    }
+
+    if (frame.op == meshtastic_RemoteShell_OpCode_ACK) {
+        if (session.active && frame.session_id == session.sessionId && getFrom(&mp) == session.peer && frame.last_rx_seq > 0) {
+            resendFramesFrom(frame.last_rx_seq + 1);
+        }
+        return ProcessMessage::CONTINUE;
+    }
+
+    if (frame.op >= 64) {
+        LOG_WARN("DMShell: ignoring frame with op code %d, seq %d", frame.op, frame.seq);
+        return ProcessMessage::CONTINUE;
+    }
+
+    if (!isAuthorizedPacket(mp)) {
+        LOG_WARN("DMShell: unauthorized sender 0x%x, %u", mp.from, frame.op);
+        myReply = allocErrorResponse(meshtastic_Routing_Error_NOT_AUTHORIZED, &mp);
+        return ProcessMessage::STOP;
+    }
+
+    if (frame.op == meshtastic_RemoteShell_OpCode_OPEN) {
+        LOG_WARN("DMShell: received OPEN from 0x%x sessionId=0x%x", mp.from, frame.session_id);
+        if (!openSession(mp, frame)) {
+            sendError("open_failed", getFrom(&mp));
+        }
+        return ProcessMessage::STOP;
+    }
+
+    if (!session.active || frame.session_id != session.sessionId || getFrom(&mp) != session.peer) {
+        if (!session.active) {
+            LOG_WARN("DMShell: no active session, rejecting op %d from 0x%x", frame.op, mp.from);
+        } else {
+            LOG_WARN("DMShell: session ID mismatch (got 0x%x expected 0x%x) or peer mismatch (got 0x%x expected 0x%x), rejecting "
+                     "op %d",
+                     frame.session_id, session.sessionId, mp.from, session.peer, frame.op);
+        }
+        sendError("invalid_session", getFrom(&mp));
+        return ProcessMessage::STOP;
+    }
+
+    if (!shouldProcessIncomingFrame(frame)) {
+        return ProcessMessage::STOP;
+    }
+
+    session.lastActivityMs = millis();
+
+    switch (frame.op) {
+    case meshtastic_RemoteShell_OpCode_INPUT:
+        if (!writeSessionInput(frame)) {
+            sendError("input_write_failed");
+        } else {
+            uint8_t outBuf[MAX_MESSAGE_SIZE];
+            const ssize_t bytesRead = read(session.masterFd, outBuf, sizeof(outBuf));
+            if (bytesRead > 0) {
+                LOG_WARN("DMShell: read %zd bytes from PTY", bytesRead);
+                meshtastic_RemoteShell frame = {
+                    .op = meshtastic_RemoteShell_OpCode_OUTPUT,
+                    .session_id = session.sessionId,
+                    .seq = session.nextTxSeq++,
+                    .ack_seq = session.lastAckedRxSeq,
+                    .cols = 0,
+                    .rows = 0,
+                    .flags = 0,
+                };
+                assert(bytesRead <= sizeof(frame.payload.bytes));
+                memcpy(frame.payload.bytes, outBuf, bytesRead);
+                frame.payload.size = bytesRead;
+                sendFrameToPeer(session.peer, frame, true);
+                session.lastActivityMs = millis();
+            }
+        }
+        break;
+    case meshtastic_RemoteShell_OpCode_RESIZE:
+        if (frame.rows > 0 && frame.cols > 0) {
+            struct winsize ws = {};
+            ws.ws_row = frame.rows;
+            ws.ws_col = frame.cols;
+            if (session.masterFd >= 0) {
+                ioctl(session.masterFd, TIOCSWINSZ, &ws);
+            }
+        }
+        break;
+    case meshtastic_RemoteShell_OpCode_PING: {
+        uint32_t peerLastRxSeq = frame.ack_seq;
+        if (frame.last_rx_seq > 0) {
+            peerLastRxSeq = frame.last_rx_seq;
+        }
+
+        const uint32_t nextMissingForPeer = peerLastRxSeq + 1;
+        if (nextMissingForPeer > 0 && nextMissingForPeer < session.nextTxSeq) {
+            resendFramesFrom(nextMissingForPeer);
+        }
+
+        meshtastic_RemoteShell frame = {
+            .op = meshtastic_RemoteShell_OpCode_PONG,
+            .session_id = session.sessionId,
+            .seq = session.nextTxSeq++,
+            .ack_seq = session.lastAckedRxSeq,
+            .cols = 0,
+            .rows = 0,
+            .flags = 0,
+            .last_tx_seq = session.nextTxSeq > 0 ? session.nextTxSeq - 1 : 0,
+            .last_rx_seq = session.lastAckedRxSeq,
+        };
+        frame.payload.size = 0;
+        sendFrameToPeer(session.peer, frame, true);
+        break;
+    }
+    case meshtastic_RemoteShell_OpCode_CLOSE:
+        closeSession("peer_close", true);
+        break;
+    default:
+        sendError("unsupported_op");
+        break;
+    }
+
+    return ProcessMessage::STOP;
+}
+
+int32_t DMShellModule::runOnce()
+{
+    processPendingChildReap();
+
+    if (!session.active) {
+        return 100;
+    }
+
+    reapChildIfExited();
+    if (!session.active) {
+        return 100;
+    }
+
+    if (Throttle::isWithinTimespanMs(session.lastActivityMs, SESSION_IDLE_TIMEOUT_MS) == false) {
+        closeSession("idle_timeout", true);
+        return 100;
+    }
+
+    if (RadioLibInterface::instance->packetsInTxQueue() > 1) {
+        return 50;
+    }
+
+    uint8_t outBuf[MAX_MESSAGE_SIZE];
+    while (session.masterFd >= 0) {
+        const ssize_t bytesRead = read(session.masterFd, outBuf, sizeof(outBuf));
+        if (bytesRead > 0) {
+            LOG_WARN("DMShell: read %zd bytes from PTY", bytesRead);
+
+            meshtastic_RemoteShell frame = {
+                .op = meshtastic_RemoteShell_OpCode_OUTPUT,
+                .session_id = session.sessionId,
+                .seq = session.nextTxSeq++,
+                .ack_seq = session.lastAckedRxSeq,
+                .cols = 0,
+                .rows = 0,
+                .flags = 0,
+            };
+            assert(bytesRead <= sizeof(frame.payload.bytes));
+            memcpy(frame.payload.bytes, outBuf, bytesRead);
+            frame.payload.size = bytesRead;
+            sendFrameToPeer(session.peer, frame, true);
+
+            session.lastActivityMs = millis();
+            // continue;
+            // do we want to ack every data message, and only send the next on ack?
+            // would require some retry logic. Maybe re-use the wantAck bit
+            return 50;
+        }
+
+        if (bytesRead == 0) {
+            closeSession("pty_eof", true);
+            break;
+        }
+
+        if (errno == EAGAIN || errno == EWOULDBLOCK) {
+            break;
+        }
+
+        LOG_WARN("DMShell: PTY read error errno=%d", errno);
+        closeSession("pty_read_error", true);
+        break;
+    }
+
+    return 100;
+}
+
+bool DMShellModule::parseFrame(const meshtastic_MeshPacket &mp, meshtastic_RemoteShell &outFrame)
+{
+    if (mp.which_payload_variant != meshtastic_MeshPacket_decoded_tag) {
+        return false;
+    }
+
+    if (pb_decode_from_bytes(mp.decoded.payload.bytes, mp.decoded.payload.size, meshtastic_RemoteShell_fields, &outFrame)) {
+        LOG_INFO("Received a DMShell message");
+    } else {
+        LOG_ERROR("Error decoding DMShell message!");
+        return false;
+    }
+
+    return true;
+}
+
+bool DMShellModule::isAuthorizedPacket(const meshtastic_MeshPacket &mp) const
+{
+    if (mp.from == 0) {
+        return !config.security.is_managed;
+    }
+
+    const meshtastic_Channel *ch = &channels.getByIndex(mp.channel);
+    if (strcasecmp(ch->settings.name, Channels::adminChannel) == 0) {
+        return config.security.admin_channel_enabled;
+    }
+
+    if (mp.pki_encrypted) {
+        for (uint8_t i = 0; i < 3; ++i) {
+            if (config.security.admin_key[i].size == 32 &&
+                memcmp(mp.public_key.bytes, config.security.admin_key[i].bytes, 32) == 0) {
+                return true;
+            }
+        }
+    }
+
+    return false;
+}
+
+bool DMShellModule::openSession(const meshtastic_MeshPacket &mp, const meshtastic_RemoteShell &frame)
+{
+    if (session.active) {
+        closeSession("preempted", false);
+    }
+
+    int masterFd = -1;
+    struct winsize ws = {};
+    if (frame.rows > 0) {
+        ws.ws_row = frame.rows;
+    } else {
+        ws.ws_row = PTY_ROWS_DEFAULT;
+    }
+    if (frame.cols > 0) {
+        ws.ws_col = frame.cols;
+    } else {
+        ws.ws_col = PTY_COLS_DEFAULT;
+    }
+    const pid_t childPid = forkpty(&masterFd, nullptr, nullptr, &ws);
+    if (childPid < 0) {
+        LOG_ERROR("DMShell: forkpty failed errno=%d", errno);
+        return false;
+    }
+
+    if (childPid == 0) {
+        const char *shell = getenv("SHELL");
+        if (!shell || !*shell) {
+            shell = "/bin/sh";
+        }
+        execl(shell, shell, "-i", static_cast<char *>(nullptr));
+        _exit(127);
+    }
+
+    const int flags = fcntl(masterFd, F_GETFL, 0);
+    if (flags >= 0) {
+        fcntl(masterFd, F_SETFL, flags | O_NONBLOCK);
+    }
+
+    session.active = true;
+    session.sessionId = (frame.session_id != 0) ? frame.session_id : static_cast<uint32_t>(random(1, 0x7fffffff));
+    session.peer = getFrom(&mp);
+    session.channel = mp.channel;
+    session.masterFd = masterFd;
+    session.childPid = childPid;
+    session.nextTxSeq = 1;
+    session.lastAckedRxSeq = frame.seq;
+    session.nextExpectedRxSeq = frame.seq + 1;
+    session.highestSeenRxSeq = frame.seq;
+    session.lastActivityMs = millis();
+
+    meshtastic_RemoteShell newFrame = {
+        .op = meshtastic_RemoteShell_OpCode_OPEN_OK,
+        .session_id = session.sessionId,
+        .seq = session.nextTxSeq++,
+        .ack_seq = frame.seq,
+        .cols = ws.ws_col,
+        .rows = ws.ws_row,
+        .flags = 0,
+    };
+    newFrame.payload.size = 0;
+    sendFrameToPeer(session.peer, newFrame, true);
+
+    LOG_INFO("DMShell: opened session=0x%x peer=0x%x pid=%d", session.sessionId, session.peer, session.childPid);
+    return true;
+}
+
+bool DMShellModule::writeSessionInput(const meshtastic_RemoteShell &frame)
+{
+    if (session.masterFd < 0) {
+        return false;
+    }
+    if (frame.payload.size == 0) {
+        return true;
+    }
+
+    const ssize_t bytesWritten = write(session.masterFd, frame.payload.bytes, frame.payload.size);
+    return bytesWritten >= 0;
+}
+
+void DMShellModule::closeSession(const char *reason, bool notifyPeer)
+{
+    if (!session.active) {
+        return;
+    }
+
+    if (notifyPeer) {
+        const size_t reasonLen = strnlen(reason, 256);
+        meshtastic_RemoteShell frame = {
+            .op = meshtastic_RemoteShell_OpCode_CLOSED,
+            .session_id = session.sessionId,
+            .seq = session.nextTxSeq++,
+            .ack_seq = session.lastAckedRxSeq,
+            .cols = 0,
+            .rows = 0,
+            .flags = 0,
+        };
+        assert(reasonLen <= sizeof(frame.payload.bytes));
+        memcpy(frame.payload.bytes, reason, reasonLen);
+        frame.payload.size = reasonLen;
+        sendFrameToPeer(session.peer, frame, true);
+    }
+
+    if (session.masterFd >= 0) {
+        close(session.masterFd);
+        session.masterFd = -1;
+    }
+
+    if (session.childPid > 0) {
+        // Run this to avoid forgetting a child
+        processPendingChildReap();
+
+        if (kill(session.childPid, SIGTERM) < 0 && errno != ESRCH) {
+            LOG_WARN("DMShell: failed to send SIGTERM to pid=%d errno=%d", session.childPid, errno);
+        }
+
+        pendingChildPid = session.childPid;
+        session.childPid = -1;
+    }
+
+    LOG_INFO("DMShell: closed session=0x%x reason=%s", session.sessionId, reason);
+    session = DMShellSession{};
+}
+
+void DMShellModule::reapChildIfExited()
+{
+    if (!session.active || session.childPid <= 0) {
+        return;
+    }
+
+    int status = 0;
+    const pid_t result = waitpid(session.childPid, &status, WNOHANG);
+    if (result == session.childPid) {
+        closeSession("shell_exited", true);
+    }
+}
+
+void DMShellModule::processPendingChildReap()
+{
+    if (pendingChildPid <= 0) {
+        return;
+    }
+
+    int status = 0;
+    const pid_t result = waitpid(pendingChildPid, &status, WNOHANG);
+
+    if (result == pendingChildPid || (result < 0 && errno == ECHILD)) {
+        pendingChildPid = -1;
+        return;
+    }
+
+    if (result < 0) {
+        LOG_WARN("DMShell: waitpid failed for pid=%d errno=%d", pendingChildPid, errno);
+        pendingChildPid = -1;
+        return;
+    }
+
+    if (pendingChildPid > 0) {
+        if (kill(pendingChildPid, SIGKILL) < 0 && errno != ESRCH) {
+            LOG_WARN("DMShell: failed to send SIGKILL to pid=%d errno=%d", pendingChildPid, errno);
+        }
+        pendingChildPid = -1;
+    }
+}
+
+void DMShellModule::rememberSentFrame(meshtastic_RemoteShell frame)
+{
+    if (frame.seq == 0 || frame.op == meshtastic_RemoteShell_OpCode_ACK) {
+        return;
+    }
+
+    auto &entry = session.txHistory[session.txHistoryNext];
+    entry.valid = true;
+    entry.op = frame.op;
+    entry.sessionId = frame.session_id;
+    entry.seq = frame.seq;
+    entry.ackSeq = frame.ack_seq;
+    entry.cols = frame.cols;
+    entry.rows = frame.rows;
+    entry.flags = frame.flags;
+    entry.payloadLen = frame.payload.size;
+    if (frame.payload.size > 0) {
+        memcpy(entry.payload, frame.payload.bytes, frame.payload.size);
+    }
+
+    session.txHistoryNext = (session.txHistoryNext + 1) % session.txHistory.size();
+}
+
+void DMShellModule::resendFramesFrom(uint32_t startSeq)
+{
+    if (startSeq == 0) {
+        return;
+    }
+
+    DMShellSession::SentFrame *match = nullptr;
+    for (auto &entry : session.txHistory) {
+        if (!entry.valid || entry.seq != startSeq) {
+            continue;
+        }
+        match = &entry;
+        break;
+    }
+
+    if (!match) {
+        LOG_WARN("DMShell: replay request for seq=%u not found in history", startSeq);
+        return;
+    }
+
+    LOG_INFO("DMShell: replaying frame seq=%u op=%d", match->seq, match->op);
+    meshtastic_RemoteShell frame = {
+        .op = match->op,
+        .session_id = match->sessionId,
+        .seq = match->seq,
+        .ack_seq = match->ackSeq,
+        .cols = match->cols,
+        .rows = match->rows,
+        .flags = match->flags,
+    };
+    assert(match->payloadLen <= sizeof(frame.payload.bytes));
+    memcpy(frame.payload.bytes, match->payload, match->payloadLen);
+    frame.payload.size = match->payloadLen;
+    sendFrameToPeer(session.peer, frame, false);
+}
+
+void DMShellModule::sendAck(uint32_t replayFromSeq)
+{
+    if (replayFromSeq > 0) {
+        LOG_WARN("DMShell: requesting replay from seq=%u", replayFromSeq);
+    }
+    meshtastic_RemoteShell frame = {
+        .op = meshtastic_RemoteShell_OpCode_ACK,
+        .session_id = session.sessionId,
+        .seq = 0,
+        .ack_seq = session.lastAckedRxSeq,
+        .cols = 0,
+        .rows = 0,
+        .flags = 0,
+        .last_rx_seq = replayFromSeq - 1,
+    };
+    frame.payload.size = 0;
+    sendFrameToPeer(session.peer, frame, false);
+}
+
+bool DMShellModule::shouldProcessIncomingFrame(const meshtastic_RemoteShell &frame)
+{
+    if (frame.seq == 0) {
+        return true;
+    }
+
+    if (frame.seq < session.nextExpectedRxSeq) {
+        if (session.highestSeenRxSeq >= session.nextExpectedRxSeq) {
+            sendAck(session.nextExpectedRxSeq);
+        } else {
+            sendAck();
+        }
+        return false;
+    }
+
+    if (frame.seq > session.nextExpectedRxSeq) {
+        if (frame.seq > session.highestSeenRxSeq) {
+            session.highestSeenRxSeq = frame.seq;
+        }
+        sendAck(session.nextExpectedRxSeq);
+        return false;
+    }
+
+    session.lastAckedRxSeq = frame.seq;
+    session.nextExpectedRxSeq = frame.seq + 1;
+    if (frame.seq > session.highestSeenRxSeq) {
+        session.highestSeenRxSeq = frame.seq;
+    }
+    if (session.highestSeenRxSeq >= session.nextExpectedRxSeq) {
+        sendAck(session.nextExpectedRxSeq);
+    } else {
+        session.highestSeenRxSeq = 0;
+    }
+    return true;
+}
+
+void DMShellModule::sendFrameToPeer(NodeNum peer, meshtastic_RemoteShell frame, bool remember)
+{
+    meshtastic_MeshPacket *packet = allocDataPacket();
+    if (!packet) {
+        return;
+    }
+    LOG_WARN("DMShell: building packet op=%u session=0x%x seq=%u payloadLen=%zu", frame.op, frame.session_id, frame.seq,
+             frame.payload.size);
+    const size_t encoded = pb_encode_to_bytes(packet->decoded.payload.bytes, sizeof(packet->decoded.payload.bytes),
+                                              meshtastic_RemoteShell_fields, &frame);
+    if (encoded == 0) {
+        return;
+    }
+    packet->decoded.payload.size = encoded;
+
+    if (remember) {
+        rememberSentFrame(frame);
+    }
+
+    packet->to = peer;
+    packet->hop_limit = 0;
+    packet->hop_start = 0;
+    packet->channel = 0;
+    packet->want_ack = false;
+    packet->pki_encrypted = true;
+    packet->priority = meshtastic_MeshPacket_Priority_RELIABLE;
+    service->sendToMesh(packet);
+}
+
+void DMShellModule::sendError(const char *message, NodeNum peer)
+{
+    const size_t len = strnlen(message, MAX_MESSAGE_SIZE);
+    meshtastic_RemoteShell frame = {
+        .op = meshtastic_RemoteShell_OpCode_ERROR,
+        .session_id = session.sessionId,
+        .seq = session.nextTxSeq++,
+        .ack_seq = session.lastAckedRxSeq,
+        .cols = 0,
+        .rows = 0,
+        .flags = 0,
+    };
+    if (message && len > 0) {
+        assert(len <= sizeof(frame.payload.bytes));
+        memcpy(frame.payload.bytes, message, len);
+        frame.payload.size = len;
+    }
+    if (peer == 0) {
+        peer = session.peer;
+    }
+    sendFrameToPeer(peer, frame, true);
+}
+#endif

src/modules/DMShell.h

@@ -0,0 +1,77 @@
+#pragma once
+
+#include "MeshModule.h"
+#include "Router.h"
+#include "SinglePortModule.h"
+#include "concurrency/OSThread.h"
+#include "configuration.h"
+#include "mesh/generated/meshtastic/mesh.pb.h"
+#include <Arduino.h>
+#include <array>
+#include <functional>
+
+#if defined(ARCH_PORTDUINO)
+
+struct DMShellSession {
+    bool active = false;
+    uint32_t sessionId = 0;
+    NodeNum peer = 0;
+    uint8_t channel = 0;
+    int masterFd = -1;
+    int childPid = -1;
+    uint32_t nextTxSeq = 1;
+    uint32_t lastAckedRxSeq = 0;
+    uint32_t nextExpectedRxSeq = 1;
+    uint32_t highestSeenRxSeq = 0;
+    uint32_t lastActivityMs = 0;
+    struct SentFrame {
+        bool valid = false;
+        meshtastic_RemoteShell_OpCode op = meshtastic_RemoteShell_OpCode_ERROR;
+        uint32_t sessionId = 0;
+        uint32_t seq = 0;
+        uint32_t ackSeq = 0;
+        uint32_t cols = 0;
+        uint32_t rows = 0;
+        uint32_t flags = 0;
+        uint8_t payload[meshtastic_Constants_DATA_PAYLOAD_LEN] = {0};
+        size_t payloadLen = 0;
+    };
+    std::array<SentFrame, 50> txHistory = {};
+    size_t txHistoryNext = 0;
+};
+
+class DMShellModule : private concurrency::OSThread, public SinglePortModule
+{
+
+  public:
+    DMShellModule();
+
+  protected:
+    virtual ProcessMessage handleReceived(const meshtastic_MeshPacket &mp) override;
+    virtual int32_t runOnce() override;
+
+  private:
+    static constexpr uint32_t SESSION_IDLE_TIMEOUT_MS = 5 * 60 * 1000;
+
+    DMShellSession session;
+    pid_t pendingChildPid = -1;
+
+    bool parseFrame(const meshtastic_MeshPacket &mp, meshtastic_RemoteShell &outFrame);
+    bool isAuthorizedPacket(const meshtastic_MeshPacket &mp) const;
+    bool openSession(const meshtastic_MeshPacket &mp, const meshtastic_RemoteShell &frame);
+    bool shouldProcessIncomingFrame(const meshtastic_RemoteShell &frame);
+    bool writeSessionInput(const meshtastic_RemoteShell &frame);
+    void closeSession(const char *reason, bool notifyPeer);
+    void reapChildIfExited();
+    void processPendingChildReap();
+
+    void rememberSentFrame(meshtastic_RemoteShell frame);
+    void resendFramesFrom(uint32_t startSeq);
+    void sendAck(uint32_t replayFromSeq = 0);
+    void sendFrameToPeer(NodeNum peer, meshtastic_RemoteShell frame, bool remember = true);
+    void sendError(const char *message, NodeNum peer = 0);
+};
+
+extern DMShellModule *dmShellModule;
+
+#endif

src/modules/Modules.cpp

@@ -49,6 +49,7 @@
 #include "modules/WaypointModule.h"
 #endif
 #if ARCH_PORTDUINO
+#include "modules/DMShell.h"
 #include "modules/Telemetry/HostMetrics.h"
 #if !MESHTASTIC_EXCLUDE_STOREFORWARD
 #include "modules/StoreForwardModule.h"
@@ -195,6 +196,7 @@ void setupModules()
 #endif
 #if ARCH_PORTDUINO
     new HostMetricsModule();
+    dmShellModule = new DMShellModule();
 #endif
 #if HAS_TELEMETRY
     new DeviceTelemetryModule();

src/platform/portduino/PortduinoGlue.cpp

@@ -13,7 +13,6 @@
 #include <ErriezCRC32.h>
 #include <Utility.h>
 #include <assert.h>
-#include <cctype>
 #include <filesystem>
 #include <fstream>
 #include <iostream>
@@ -33,16 +32,6 @@
 #include <cxxabi.h>
 #endif
 
-#ifdef __APPLE__
-// Used by getMacAddr()'s macOS fallback to read the en0 link-layer address.
-// `getifaddrs()` is the BSD-portable way; `<net/if_dl.h>` provides the
-// `sockaddr_dl` cast and the `LLADDR()` macro that points at the 6-byte MAC.
-#include <cstring> // strcmp, memcpy
-#include <ifaddrs.h>
-#include <net/if.h>
-#include <net/if_dl.h>
-#endif
-
 #include "platform/portduino/USBHal.h"
 
 portduino_config_struct portduino_config;
@@ -166,35 +155,9 @@ void getMacAddr(uint8_t *dmac)
         dmac[3] = di.bdaddr.b[2];
         dmac[4] = di.bdaddr.b[1];
         dmac[5] = di.bdaddr.b[0];
-#elif defined(__APPLE__)
-        // No BlueZ on macOS, but we can fall back to the host's primary
-        // network interface MAC. `en0` is Wi-Fi on every shipping Mac
-        // (Ethernet, when present, is en1 or higher), which gives the user
-        // the same kind of stable, host-derived identifier that the BlueZ
-        // path provides on Linux. If en0 isn't found or has no MAC, dmac is
-        // left untouched and the caller's "Blank MAC Address not allowed!"
-        // check will still fire — preserving existing behavior for users
-        // who deliberately rely on --hwid or YAML override.
-        struct ifaddrs *ifap = nullptr;
-        if (getifaddrs(&ifap) == 0) {
-            for (struct ifaddrs *p = ifap; p != nullptr; p = p->ifa_next) {
-                if (p->ifa_addr == nullptr || p->ifa_addr->sa_family != AF_LINK) {
-                    continue;
-                }
-                if (strcmp(p->ifa_name, "en0") != 0) {
-                    continue;
-                }
-                auto *sdl = reinterpret_cast<struct sockaddr_dl *>(p->ifa_addr);
-                if (sdl->sdl_alen == 6) {
-                    memcpy(dmac, LLADDR(sdl), 6);
-                    break;
-                }
-            }
-            freeifaddrs(ifap);
-        }
 #else
-        // No platform-specific MAC source; leave dmac at its default. Caller
-        // can override via the --hwid CLI flag or the YAML config.
+        // No BlueZ on non-Linux hosts (e.g. macOS). Leave dmac at its default;
+        // the caller can override via the --hwid CLI flag or the YAML config.
         (void)dmac;
 #endif
     }
@@ -598,8 +561,7 @@ void portduinoSetup()
     for (const auto *i : portduino_config.all_pins) {
         // In the case of a ch341 Lora device, we don't want to touch the system GPIO lines for Lora
         // Those GPIO are handled in our usermode driver instead.
-        if (i->config_section == "Lora" &&
-            (portduino_config.lora_spi_dev == "ch341" || portduino_config.lora_spi_dev == "serial")) {
+        if (i->config_section == "Lora" && portduino_config.lora_spi_dev == "ch341") {
             continue;
         }
         if (i->enabled) {
@@ -618,8 +580,7 @@ void portduinoSetup()
     for (auto i : portduino_config.extra_pins) {
         // In the case of a ch341 Lora device, we don't want to touch the system GPIO lines for Lora
         // Those GPIO are handled in our usermode driver instead.
-        if (i.config_section == "Lora" &&
-            (portduino_config.lora_spi_dev == "ch341" || portduino_config.lora_spi_dev == "serial")) {
+        if (i.config_section == "Lora" && portduino_config.lora_spi_dev == "ch341") {
             continue;
         }
         if (i.enabled) {
@@ -666,8 +627,7 @@ void portduinoSetup()
     for (auto i : portduino_config.extra_pins) {
         // In the case of a ch341 Lora device, we don't want to touch the system GPIO lines for Lora
         // Those GPIO are handled in our usermode driver instead.
-        if (i.config_section == "Lora" &&
-            (portduino_config.lora_spi_dev == "ch341" || portduino_config.lora_spi_dev == "serial")) {
+        if (i.config_section == "Lora" && portduino_config.lora_spi_dev == "ch341") {
             continue;
         }
         if (i.enabled && i.default_high) {
@@ -677,8 +637,7 @@ void portduinoSetup()
     }
 
     // Only initialize the radio pins when dealing with real, kernel controlled SPI hardware
-    if (portduino_config.lora_spi_dev != "" && portduino_config.lora_spi_dev != "ch341" &&
-        portduino_config.lora_spi_dev != "serial") {
+    if (portduino_config.lora_spi_dev != "" && portduino_config.lora_spi_dev != "ch341") {
         SPI.begin(portduino_config.lora_spi_dev.c_str());
     }
 
@@ -712,7 +671,6 @@ void portduinoSetup()
     }
     if (portduino_config.lora_spi_dev != "") {
         portduinoSetOptions({.realHardware = true});
-        LOG_DEBUG("Running with real hardware SPI device %s", portduino_config.lora_spi_dev.c_str());
     }
     return;
 }
@@ -849,15 +807,12 @@ bool loadConfig(const char *configPath)
             }
 
             portduino_config.spiSpeed = yamlConfig["Lora"]["spiSpeed"].as<int>(2000000);
-            portduino_config.lora_serial_device = yamlConfig["Lora"]["SerialDevice"].as<std::string>("");
-            portduino_config.lora_serial_baud = yamlConfig["Lora"]["SerialBaud"].as<int>(115200);
-            portduino_config.lora_serial_timeout_ms = yamlConfig["Lora"]["SerialTimeoutMs"].as<int>(500);
             portduino_config.lora_usb_serial_num = yamlConfig["Lora"]["USB_Serialnum"].as<std::string>("");
             portduino_config.lora_usb_pid = yamlConfig["Lora"]["USB_PID"].as<int>(0x5512);
             portduino_config.lora_usb_vid = yamlConfig["Lora"]["USB_VID"].as<int>(0x1A86);
 
             portduino_config.lora_spi_dev = yamlConfig["Lora"]["spidev"].as<std::string>("spidev0.0");
-            if (portduino_config.lora_spi_dev != "ch341" && portduino_config.lora_spi_dev != "serial") {
+            if (portduino_config.lora_spi_dev != "ch341") {
                 portduino_config.lora_spi_dev = "/dev/" + portduino_config.lora_spi_dev;
                 if (portduino_config.lora_spi_dev.length() == 14) {
                     int x = portduino_config.lora_spi_dev.at(11) - '0';
@@ -1101,31 +1056,17 @@ static bool ends_with(std::string_view str, std::string_view suffix)
 bool MAC_from_string(std::string mac_str, uint8_t *dmac)
 {
     mac_str.erase(std::remove(mac_str.begin(), mac_str.end(), ':'), mac_str.end());
-    if (mac_str.length() != 12) {
+    if (mac_str.length() == 12) {
+        dmac[0] = std::stoi(portduino_config.mac_address.substr(0, 2), nullptr, 16);
+        dmac[1] = std::stoi(portduino_config.mac_address.substr(2, 2), nullptr, 16);
+        dmac[2] = std::stoi(portduino_config.mac_address.substr(4, 2), nullptr, 16);
+        dmac[3] = std::stoi(portduino_config.mac_address.substr(6, 2), nullptr, 16);
+        dmac[4] = std::stoi(portduino_config.mac_address.substr(8, 2), nullptr, 16);
+        dmac[5] = std::stoi(portduino_config.mac_address.substr(10, 2), nullptr, 16);
+        return true;
+    } else {
         return false;
     }
-    // Validate every character is a hex digit before parsing. std::stoi
-    // would otherwise skip leading whitespace and silently truncate at the
-    // first non-digit, which is too lenient for a MAC address.
-    for (char c : mac_str) {
-        if (!isxdigit(static_cast<unsigned char>(c))) {
-            return false;
-        }
-    }
-    // Parse into a temporary so dmac is not partially modified if a later
-    // byte fails. At least one caller in getMacAddr() ignores the bool
-    // return, so leaving stale bytes in dmac on failure would silently
-    // produce a wrong MAC.
-    uint8_t tmp[6];
-    try {
-        for (int i = 0; i < 6; i++) {
-            tmp[i] = static_cast<uint8_t>(std::stoi(mac_str.substr(i * 2, 2), nullptr, 16));
-        }
-    } catch (const std::exception &) {
-        return false;
-    }
-    memcpy(dmac, tmp, 6);
-    return true;
 }
 
 std::string exec(const char *cmd)

src/platform/portduino/PortduinoGlue.h

@@ -86,9 +86,6 @@ extern struct portduino_config_struct {
     bool has_device_id = false;
     uint8_t device_id[16] = {0};
     std::string lora_spi_dev = "";
-    std::string lora_serial_device = "";
-    int lora_serial_baud = 115200;
-    int lora_serial_timeout_ms = 500;
     std::string lora_usb_serial_num = "";
     int lora_spi_dev_int = 0;
     int lora_default_gpiochip = 0;
@@ -279,12 +276,6 @@ extern struct portduino_config_struct {
         }
         if (lora_usb_serial_num != "")
             out << YAML::Key << "USB_Serialnum" << YAML::Value << lora_usb_serial_num;
-        if (lora_serial_device != "")
-            out << YAML::Key << "SerialDevice" << YAML::Value << lora_serial_device;
-        if (lora_serial_baud != 115200)
-            out << YAML::Key << "SerialBaud" << YAML::Value << lora_serial_baud;
-        if (lora_serial_timeout_ms != 500)
-            out << YAML::Key << "SerialTimeoutMs" << YAML::Value << lora_serial_timeout_ms;
         if (spiSpeed != 2000000)
             out << YAML::Key << "spiSpeed" << YAML::Value << spiSpeed;
         if (rfswitch_dio_pins[0] != RADIOLIB_NC) {

src/platform/portduino/SerialHal.cpp

@@ -1,595 +0,0 @@
-#include "platform/portduino/SerialHal.h"
-
-#include "mesh/mesh-pb-constants.h"
-#include "platform/portduino/PortduinoGlue.h"
-#include <cerrno>
-#include <chrono>
-#include <cstring>
-#include <fcntl.h>
-#include <poll.h>
-#include <sched.h>
-#include <sys/time.h>
-#include <termios.h>
-#include <unistd.h>
-#include <utility>
-
-namespace
-{
-constexpr uint8_t START1 = 0x94;
-constexpr uint8_t SERIALHAL_MAGIC = 0xA5;
-constexpr size_t HEADER_SIZE = 4; // START1 + SERIALHAL_MAGIC + LEN_H + LEN_L
-constexpr uint8_t START2 = 0xC3;  // second byte of a normal FromRadio frame
-
-speed_t toTermiosBaud(uint32_t baud)
-{
-    switch (baud) {
-    case 9600:
-        return B9600;
-    case 19200:
-        return B19200;
-    case 38400:
-        return B38400;
-    case 57600:
-        return B57600;
-    case 115200:
-        return B115200;
-    case 230400:
-        return B230400;
-    case 460800:
-        return B460800;
-    case 921600:
-        return B921600;
-    default:
-        return B115200;
-    }
-}
-} // namespace
-
-SerialHal::SerialHal(const std::string &devicePath, uint32_t baudRate, uint32_t opTimeoutMs)
-    : RadioLibHal(SERIAL_PI_INPUT, SERIAL_PI_OUTPUT, SERIAL_PI_LOW, SERIAL_PI_HIGH, SERIAL_PI_RISING, SERIAL_PI_FALLING),
-      device(devicePath), baud(baudRate), timeoutMs(opTimeoutMs)
-{
-    if (!openPort()) {
-        setTransportError("unable to open serial device");
-    }
-}
-
-SerialHal::~SerialHal()
-{
-    closePort();
-}
-
-bool SerialHal::openPort()
-{
-    closePort();
-    fd = ::open(device.c_str(), O_RDWR | O_NOCTTY | O_SYNC);
-    if (fd < 0) {
-        return false;
-    }
-
-    termios tty = {};
-    if (tcgetattr(fd, &tty) != 0) {
-        closePort();
-        return false;
-    }
-
-    cfsetospeed(&tty, toTermiosBaud(baud));
-    cfsetispeed(&tty, toTermiosBaud(baud));
-
-    tty.c_cflag = (tty.c_cflag & ~CSIZE) | CS8;
-    tty.c_iflag &= ~(IGNBRK | IXON | IXOFF | IXANY);
-    tty.c_lflag = 0;
-    tty.c_oflag = 0;
-    tty.c_cc[VMIN] = 0;
-    tty.c_cc[VTIME] = 0;
-    tty.c_cflag |= (CLOCAL | CREAD);
-    tty.c_cflag &= ~(PARENB | PARODD);
-    tty.c_cflag &= ~CSTOPB;
-    tty.c_cflag &= ~CRTSCTS;
-
-    if (tcsetattr(fd, TCSANOW, &tty) != 0) {
-        closePort();
-        return false;
-    }
-
-    tcflush(fd, TCIOFLUSH);
-    inError = false;
-    startReaderThread();
-    return true;
-}
-
-void SerialHal::closePort()
-{
-    stopReaderThread();
-    if (fd >= 0) {
-        ::close(fd);
-        fd = -1;
-    }
-}
-
-void SerialHal::setTransportError(const char *msg)
-{
-    if (!inError.load() || !hasWarned) {
-        LOG_ERROR("SerialHal: %s (%s)", msg, device.c_str());
-    }
-    inError = true;
-    hasWarned = true;
-    portduino_status.LoRa_in_error = true;
-}
-
-bool SerialHal::waitForReadable(int timeout)
-{
-    if (fd < 0) {
-        return false;
-    }
-    pollfd pfd = {};
-    pfd.fd = fd;
-    pfd.events = POLLIN;
-    int ret = poll(&pfd, 1, timeout);
-    return ret > 0 && (pfd.revents & POLLIN);
-}
-
-bool SerialHal::writeAll(const uint8_t *data, size_t len)
-{
-    size_t off = 0;
-    while (off < len) {
-        ssize_t rc = ::write(fd, data + off, len - off);
-        if (rc < 0) {
-            if (errno == EINTR) {
-                continue;
-            }
-            return false;
-        }
-        off += (size_t)rc;
-    }
-    return true;
-}
-
-bool SerialHal::readExact(uint8_t *data, size_t len)
-{
-    size_t off = 0;
-    auto start = std::chrono::steady_clock::now();
-    while (off < len) {
-        auto now = std::chrono::steady_clock::now();
-        int elapsed = (int)std::chrono::duration_cast<std::chrono::milliseconds>(now - start).count();
-        int remaining = (int)timeoutMs - elapsed;
-        if (remaining <= 0 || !waitForReadable(remaining)) {
-            return false;
-        }
-        ssize_t rc = ::read(fd, data + off, len - off);
-        if (rc < 0) {
-            if (errno == EINTR) {
-                continue;
-            }
-            return false;
-        }
-        if (rc == 0) {
-            return false;
-        }
-        off += (size_t)rc;
-    }
-    return true;
-}
-
-uint16_t SerialHal::crc16(const uint8_t *data, size_t len) const
-{
-    uint16_t crc = 0xFFFF;
-    for (size_t i = 0; i < len; ++i) {
-        crc ^= ((uint16_t)data[i] << 8);
-        for (int bit = 0; bit < 8; ++bit) {
-            if (crc & 0x8000) {
-                crc = (uint16_t)((crc << 1) ^ 0x1021);
-            } else {
-                crc = (uint16_t)(crc << 1);
-            }
-        }
-    }
-    return crc;
-}
-
-bool SerialHal::sendRequest(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse *response)
-{
-    if (fd < 0 && !openPort()) {
-        setTransportError("serial open failed");
-        return false;
-    }
-
-    uint8_t encoded[meshtastic_SerialHalCommand_size] = {0};
-    const size_t payloadLen =
-        pb_encode_to_bytes(encoded, sizeof(encoded), &meshtastic_SerialHalCommand_msg, static_cast<const void *>(&cmd));
-    if (payloadLen == 0 || payloadLen > 0xFFFF) {
-        setTransportError("serial command encode failed");
-        return false;
-    }
-
-    // Build frame with StreamAPI canonical framing: START1 SERIALHAL_MAGIC LEN_H LEN_L [payload]
-    std::vector<uint8_t> frame;
-    frame.resize(HEADER_SIZE + payloadLen);
-
-    frame[0] = START1;
-    frame[1] = SERIALHAL_MAGIC;
-    frame[2] = (uint8_t)((payloadLen >> 8) & 0xFF); // LEN_H (big-endian)
-    frame[3] = (uint8_t)(payloadLen & 0xFF);        // LEN_L
-    memcpy(frame.data() + HEADER_SIZE, encoded, payloadLen);
-
-    {
-        std::lock_guard<std::mutex> writeGuard(writeMutex);
-        if (!writeAll(frame.data(), frame.size())) {
-            setTransportError("serial write failed");
-            return false;
-        }
-    }
-
-    meshtastic_SerialHalResponse got = meshtastic_SerialHalResponse_init_zero;
-    {
-        std::unique_lock<std::mutex> lock(stateMutex);
-        const auto timeout = std::chrono::milliseconds(timeoutMs);
-        const bool arrived = responseCv.wait_for(lock, timeout, [&]() { return pendingResponses.count(cmd.transaction_id) > 0; });
-        if (!arrived) {
-            setTransportError("serial response timeout");
-            LOG_WARN("SerialHal: response timeout for transaction_id %u, cmd type %u", cmd.transaction_id, cmd.type);
-            return false;
-        }
-
-        got = pendingResponses[cmd.transaction_id];
-        pendingResponses.erase(cmd.transaction_id);
-    }
-
-    if (got.result != meshtastic_SerialHalResponse_Result_OK) {
-        setTransportError("serial response reported error");
-        LOG_WARN("SerialHal: response error: %s, %u, %u", got.error, cmd.type, cmd.data.size);
-
-        return false;
-    }
-
-    if (response != nullptr) {
-        *response = got;
-    }
-
-    inError = false;
-    hasWarned = false;
-    return true;
-}
-
-void SerialHal::pinMode(uint32_t pin, uint32_t mode)
-{
-    if (checkError() || pin == RADIOLIB_NC) {
-        return;
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_PIN_MODE;
-    cmd.pin = pin;
-    cmd.mode = mode;
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    sendRequest(cmd, &response);
-}
-
-void SerialHal::digitalWrite(uint32_t pin, uint32_t value)
-{
-    if (checkError() || pin == RADIOLIB_NC) {
-        return;
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_DIGITAL_WRITE;
-    cmd.pin = pin;
-    cmd.value = value;
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    sendRequest(cmd, &response);
-}
-
-uint32_t SerialHal::digitalRead(uint32_t pin)
-{
-    if (checkError() || pin == RADIOLIB_NC) {
-        return 0;
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_DIGITAL_READ;
-    cmd.pin = pin;
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    if (!sendRequest(cmd, &response)) {
-        return 0;
-    }
-    return response.value;
-}
-
-void SerialHal::attachInterrupt(uint32_t interruptNum, void (*interruptCb)(void), uint32_t mode)
-{
-    if (checkError() || interruptNum == RADIOLIB_NC) {
-        return;
-    }
-
-    {
-        std::lock_guard<std::mutex> lock(stateMutex);
-        interruptCallbacks[interruptNum] = interruptCb;
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_ATTACH_INTERRUPT;
-    cmd.pin = interruptNum;
-    cmd.mode = mode;
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    sendRequest(cmd, &response);
-}
-
-void SerialHal::detachInterrupt(uint32_t interruptNum)
-{
-    if (checkError() || interruptNum == RADIOLIB_NC) {
-        return;
-    }
-
-    {
-        std::lock_guard<std::mutex> lock(stateMutex);
-        interruptCallbacks.erase(interruptNum);
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_DETACH_INTERRUPT;
-    cmd.pin = interruptNum;
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    sendRequest(cmd, &response);
-}
-
-void SerialHal::delay(unsigned long ms)
-{
-    delayMicroseconds(ms * 1000);
-}
-
-void SerialHal::delayMicroseconds(unsigned long us)
-{
-    if (us == 0) {
-        sched_yield();
-        return;
-    }
-    usleep(us);
-}
-
-void SerialHal::yield()
-{
-    sched_yield();
-}
-
-unsigned long SerialHal::millis()
-{
-    struct timeval tv;
-    gettimeofday(&tv, nullptr);
-    return (tv.tv_sec * 1000ULL) + (tv.tv_usec / 1000ULL);
-}
-
-unsigned long SerialHal::micros()
-{
-    struct timeval tv;
-    gettimeofday(&tv, nullptr);
-    return (tv.tv_sec * 1000000ULL) + tv.tv_usec;
-}
-
-long SerialHal::pulseIn(uint32_t pin, uint32_t state, unsigned long timeout)
-{
-    (void)pin;
-    (void)state;
-    (void)timeout;
-    LOG_WARN("SerialHal pulseIn is not supported");
-    return 0;
-}
-
-void SerialHal::spiTransfer(uint8_t *out, size_t len, uint8_t *in)
-{
-    if (checkError()) {
-        return;
-    }
-
-    if (len == 0) {
-        return;
-    }
-
-    meshtastic_SerialHalCommand cmd = meshtastic_SerialHalCommand_init_zero;
-    cmd.transaction_id = txId.fetch_add(1);
-    cmd.type = meshtastic_SerialHalCommand_Type_SPI_TRANSFER;
-
-    const size_t maxTx = sizeof(cmd.data.bytes);
-    const size_t txLen = len < maxTx ? len : maxTx;
-    cmd.data.size = txLen;
-    if (out != nullptr) {
-        memcpy(cmd.data.bytes, out, txLen);
-    } else {
-        memset(cmd.data.bytes, 0, txLen);
-    }
-
-    meshtastic_SerialHalResponse response = meshtastic_SerialHalResponse_init_zero;
-    if (!sendRequest(cmd, &response)) {
-        return;
-    }
-
-    if (in != nullptr) {
-        size_t copyLen = response.data.size < len ? response.data.size : len;
-        memcpy(in, response.data.bytes, copyLen);
-        if (copyLen < len) {
-            memset(in + copyLen, 0, len - copyLen);
-        }
-    }
-}
-
-bool SerialHal::checkError()
-{
-    if (inError.load()) {
-        if (!hasWarned) {
-            LOG_ERROR("SerialHal in_error detected");
-            hasWarned = true;
-        }
-        portduino_status.LoRa_in_error = true;
-        return true;
-    }
-    hasWarned = false;
-    return false;
-}
-
-bool SerialHal::readFrame(std::vector<uint8_t> &payload, int firstByteTimeoutMs)
-{
-    payload.clear();
-
-    // Loop so that normal FromRadio frames (START1 START2 ...) emitted by the
-    // device on the same serial port are drained and discarded rather than
-    // causing the byte stream to desync.
-    for (;;) {
-        uint8_t hdr[HEADER_SIZE] = {0};
-        for (;;) {
-
-            ssize_t rc = ::read(fd, &hdr[0], 1);
-            if (rc < 0) {
-                if (errno == EINTR) {
-                    continue;
-                }
-                return false;
-            }
-            if (rc == 0) {
-                return false;
-            }
-            if (hdr[0] == START1) {
-                break;
-            }
-        }
-
-        if (!readExact(hdr + 1, HEADER_SIZE - 1)) {
-            return false;
-        }
-
-        const uint16_t len = ((uint16_t)hdr[2] << 8) | (uint16_t)hdr[3];
-
-        if (hdr[1] == SERIALHAL_MAGIC) {
-            // SerialHal response frame — this is what we want.
-            if (len > meshtastic_SerialHalResponse_size) {
-                return false;
-            }
-            payload.resize(len);
-            if (len > 0 && !readExact(payload.data(), len)) {
-                payload.clear();
-                return false;
-            }
-            return true;
-        } else if (hdr[1] == START2) {
-            // Normal FromRadio frame emitted by the device — drain and discard
-            // its payload so we stay in sync, then loop to find a SerialHal frame.
-            if (len > 0) {
-                std::vector<uint8_t> discard(len);
-                if (!readExact(discard.data(), len)) {
-                    return false;
-                }
-            }
-            // continue looping, look for next frame
-        } else {
-            // Unknown second byte after START1 — restart search for framing.
-            continue;
-        }
-    }
-}
-
-void SerialHal::readerLoop()
-{
-    readerRunning = true;
-    while (!readerStopRequested.load()) {
-        if (fd < 0) {
-            break;
-        }
-
-        if (!waitForReadable(100)) {
-            continue;
-        }
-
-        std::vector<uint8_t> payload;
-        if (!readFrame(payload, 40)) {
-            continue;
-        }
-
-        meshtastic_SerialHalResponse resp = meshtastic_SerialHalResponse_init_zero;
-        if (payload.empty() || !pb_decode_from_bytes(payload.data(), payload.size(), &meshtastic_SerialHalResponse_msg, &resp)) {
-            continue;
-        }
-
-        if (resp.transaction_id == 0) {
-            LOG_WARN("SerialHal: received unsolicited interrupt event: pin=%u", resp.value);
-            // transaction_id 0 is reserved for unsolicited interrupt events.
-            // The device reports the triggered pin in resp.value instead of
-            // matching one of the synchronous request/response transactions.
-            {
-                std::lock_guard<std::mutex> lock(stateMutex);
-                if (interruptCallbacks.count(resp.value) > 0) {
-                    pendingInterruptPins.push_back(resp.value);
-                }
-            }
-            interruptCv.notify_one();
-            continue;
-        }
-
-        {
-            std::lock_guard<std::mutex> lock(stateMutex);
-            pendingResponses[resp.transaction_id] = resp;
-        }
-        responseCv.notify_all();
-    }
-    readerRunning = false;
-}
-
-void SerialHal::interruptDispatchLoop()
-{
-    interruptDispatcherRunning = true;
-    while (!readerStopRequested.load()) {
-        uint32_t pin = 0;
-        void (*cb)(void) = nullptr;
-
-        {
-            std::unique_lock<std::mutex> lock(stateMutex);
-            interruptCv.wait(lock, [&]() { return readerStopRequested.load() || !pendingInterruptPins.empty(); });
-            if (readerStopRequested.load()) {
-                break;
-            }
-
-            pin = pendingInterruptPins.front();
-            pendingInterruptPins.pop_front();
-
-            auto it = interruptCallbacks.find(pin);
-            if (it != interruptCallbacks.end()) {
-                cb = it->second;
-            }
-        }
-
-        if (cb != nullptr) {
-            cb();
-        }
-    }
-    interruptDispatcherRunning = false;
-}
-
-void SerialHal::startReaderThread()
-{
-    stopReaderThread();
-    readerStopRequested = false;
-    readerThread = std::thread(&SerialHal::readerLoop, this);
-    interruptThread = std::thread(&SerialHal::interruptDispatchLoop, this);
-}
-
-void SerialHal::stopReaderThread()
-{
-    readerStopRequested = true;
-    interruptCv.notify_all();
-    if (readerThread.joinable()) {
-        readerThread.join();
-    }
-    if (interruptThread.joinable()) {
-        interruptThread.join();
-    }
-
-    std::lock_guard<std::mutex> lock(stateMutex);
-    pendingInterruptPins.clear();
-}

src/platform/portduino/SerialHal.h

@@ -1,99 +0,0 @@
-#ifndef PI_HAL_SERIAL_H
-#define PI_HAL_SERIAL_H
-
-#include <RadioLib.h>
-#include <atomic>
-#include <condition_variable>
-#include <cstdint>
-#include <deque>
-#include <functional>
-#include <mesh/generated/meshtastic/serial_hal.pb.h>
-#include <mutex>
-#include <string>
-#include <thread>
-#include <unordered_map>
-#include <vector>
-
-#define SERIAL_PI_INPUT (0)
-#define SERIAL_PI_OUTPUT (1)
-#define SERIAL_PI_LOW (0)
-#define SERIAL_PI_HIGH (1)
-#define SERIAL_PI_RISING (1)
-#define SERIAL_PI_FALLING (2)
-
-class SerialHal : public RadioLibHal
-{
-  public:
-    explicit SerialHal(const std::string &device, uint32_t baud = 115200, uint32_t timeoutMs = 500);
-    ~SerialHal() override;
-
-    void init() override {}
-    void term() override {}
-
-    void pinMode(uint32_t pin, uint32_t mode) override;
-    void digitalWrite(uint32_t pin, uint32_t value) override;
-    uint32_t digitalRead(uint32_t pin) override;
-
-    void attachInterrupt(uint32_t interruptNum, void (*interruptCb)(void), uint32_t mode) override;
-    void detachInterrupt(uint32_t interruptNum) override;
-
-    void delay(unsigned long ms) override;
-    void delayMicroseconds(unsigned long us) override;
-    void yield() override;
-
-    unsigned long millis() override;
-    unsigned long micros() override;
-
-    long pulseIn(uint32_t pin, uint32_t state, unsigned long timeout) override;
-
-    void spiBegin() override {}
-    void spiBeginTransaction() override {}
-    void spiTransfer(uint8_t *out, size_t len, uint8_t *in) override;
-    void spiEndTransaction() override {}
-    void spiEnd() override {}
-
-    bool checkError();
-
-  private:
-    bool openPort();
-    void closePort();
-    bool sendRequest(const meshtastic_SerialHalCommand &cmd, meshtastic_SerialHalResponse *response);
-    bool writeAll(const uint8_t *data, size_t len);
-    bool readExact(uint8_t *data, size_t len);
-    bool waitForReadable(int timeoutMs);
-    bool readFrame(std::vector<uint8_t> &payload, int firstByteTimeoutMs);
-    void readerLoop();
-    void interruptDispatchLoop();
-    void startReaderThread();
-    void stopReaderThread();
-
-    uint16_t crc16(const uint8_t *data, size_t len) const;
-    void setTransportError(const char *msg);
-
-    std::string device;
-    uint32_t baud;
-    uint32_t timeoutMs;
-    int fd = -1;
-    bool hasWarned = false;
-    std::atomic<bool> inError{false};
-    std::atomic<uint16_t> txId{1};
-
-    std::mutex fdMutex;
-    std::mutex writeMutex;
-    std::mutex stateMutex;
-    std::condition_variable responseCv;
-
-    std::thread readerThread;
-    std::thread interruptThread;
-    std::atomic<bool> readerStopRequested{false};
-    std::atomic<bool> readerRunning{false};
-    std::atomic<bool> interruptDispatcherRunning{false};
-
-    std::condition_variable interruptCv;
-    std::deque<uint32_t> pendingInterruptPins;
-
-    std::unordered_map<uint32_t, void (*)(void)> interruptCallbacks;
-    std::unordered_map<uint16_t, meshtastic_SerialHalResponse> pendingResponses;
-};
-
-#endif

src/platform/stm32wl/LittleFS.cpp

@@ -25,27 +25,23 @@
 #include "LittleFS.h"
 #include "stm32wlxx_hal_flash.h"
 
-/** Suppress unused-parameter warnings. */
+/**********************************************************************************************************************
+ * Macro definitions
+ **********************************************************************************************************************/
+/** This macro is used to suppress compiler messages about a parameter not being used in a function. */
 #define LFS_UNUSED(p) (void)((p))
 
-#define STM32WL_PAGE_SIZE (FLASH_PAGE_SIZE) // physical flash erase granularity: 2048 B
+#define STM32WL_PAGE_SIZE (FLASH_PAGE_SIZE)
 #define STM32WL_PAGE_COUNT (FLASH_PAGE_NB)
 #define STM32WL_FLASH_BASE (FLASH_BASE)
 
 /*
- * LFS tunables — all of these are stored in the LFS superblock.
- * Changing ANY of them is incompatible with the existing on-disk format;
- * the filesystem will be detected as corrupted and reformatted on first boot.
- *
- * LFS_FLASH_TOTAL_SIZE and LFS_BLOCK_SIZE are the only values to edit here.
- * All other parameters are derived.
- *
- * FLASH_END_ADDR is computed from FLASH_SIZE (read from the chip at link time).
+ * FLASH_SIZE from stm32wle5xx.h will read the actual FLASH size from the chip.
+ * FLASH_END_ADDR is calculated from FLASH_SIZE.
+ * Use the last 28 KiB of the FLASH
  */
-#define LFS_FLASH_TOTAL_SIZE                                                                                                     \
-    (7 * STM32WL_PAGE_SIZE)  /* 14 KiB — last 7 physical pages (FORMAT BREAK: reduced from 10 pages / 20 KiB) */
-#define LFS_BLOCK_SIZE (256) /* virtual block size (FORMAT BREAK if changed) */
-
+#define LFS_FLASH_TOTAL_SIZE (14 * 2048) /* needs to be a multiple of LFS_BLOCK_SIZE */
+#define LFS_BLOCK_SIZE (2048)
 #define LFS_FLASH_ADDR_END (FLASH_END_ADDR)
 #define LFS_FLASH_ADDR_BASE (LFS_FLASH_ADDR_END - LFS_FLASH_TOTAL_SIZE + 1)
 
@@ -55,80 +51,6 @@
 #define _LFS_DBG(fmt, ...) printf("%s:%d (%s): " fmt "\n", __FILE__, __LINE__, __func__, __VA_ARGS__)
 #endif
 
-//--------------------------------------------------------------------+
-// Write-behind page cache
-//
-// LFS requires block_size == erase granularity, but the STM32WL flash
-// erases in 2048-byte pages.  To use smaller virtual LFS blocks we
-// maintain a single-page RAM cache: the LFS erase/prog callbacks only
-// update this buffer; the physical erase+reprogram is deferred to
-// _internal_flash_sync() (or triggered automatically when a different
-// physical page is addressed).
-//
-// This mirrors the approach used by the NRF52 Adafruit driver
-// (flash_cache.c / flash_nrf5x.c) but adapted for the 2048-byte STM32WL
-// page size and HAL doubleword-program requirement.
-//--------------------------------------------------------------------+
-
-alignas(8) static uint8_t _page_cache[STM32WL_PAGE_SIZE];
-static uint32_t _page_cache_addr = UINT32_MAX; // UINT32_MAX = no page cached
-static bool _page_cache_dirty = false;
-
-/** Flush the cached page to flash (physical erase + doubleword program). */
-static int _flash_cache_flush(void)
-{
-    if (!_page_cache_dirty)
-        return LFS_ERR_OK;
-
-    FLASH_EraseInitTypeDef erase = {
-        .TypeErase = FLASH_TYPEERASE_PAGES,
-        .Page = (_page_cache_addr - STM32WL_FLASH_BASE) / STM32WL_PAGE_SIZE,
-        .NbPages = 1,
-    };
-    uint32_t page_error = 0;
-
-    if (HAL_FLASH_Unlock() != HAL_OK)
-        return LFS_ERR_IO;
-    __HAL_FLASH_CLEAR_FLAG(FLASH_FLAG_ALL_ERRORS);
-
-    HAL_StatusTypeDef rc = HAL_FLASHEx_Erase(&erase, &page_error);
-    if (rc == HAL_OK) {
-        const uint64_t *p = (const uint64_t *)_page_cache;
-        uint32_t addr = _page_cache_addr;
-        for (size_t i = 0; i < STM32WL_PAGE_SIZE / 8; i++) {
-            rc = HAL_FLASH_Program(FLASH_TYPEPROGRAM_DOUBLEWORD, addr, *p++);
-            if (rc != HAL_OK)
-                break;
-            addr += 8;
-        }
-    }
-    HAL_StatusTypeDef lock_rc = HAL_FLASH_Lock();
-    if (rc == HAL_OK)
-        rc = lock_rc;
-
-    if (rc == HAL_OK)
-        _page_cache_dirty = false;
-    return rc == HAL_OK ? LFS_ERR_OK : LFS_ERR_IO;
-}
-
-/**
- * Ensure the physical page containing `page_addr` is loaded into the cache.
- * If a different dirty page is already cached it is flushed first.
- */
-static int _flash_cache_load(uint32_t page_addr)
-{
-    if (_page_cache_addr == page_addr)
-        return LFS_ERR_OK; // already cached
-
-    int rc = _flash_cache_flush();
-    if (rc != LFS_ERR_OK)
-        return rc;
-
-    memcpy(_page_cache, (const void *)page_addr, STM32WL_PAGE_SIZE);
-    _page_cache_addr = page_addr;
-    return LFS_ERR_OK;
-}
-
 //--------------------------------------------------------------------+
 // LFS Disk IO
 //--------------------------------------------------------------------+
@@ -137,82 +59,111 @@ static int _internal_flash_read(const struct lfs_config *c, lfs_block_t block, l
 {
     LFS_UNUSED(c);
 
-    uint32_t addr = LFS_FLASH_ADDR_BASE + block * LFS_BLOCK_SIZE + off;
-    uint32_t page_addr = addr & ~(uint32_t)(STM32WL_PAGE_SIZE - 1);
-    uint32_t page_off = addr & (uint32_t)(STM32WL_PAGE_SIZE - 1);
+    if (!buffer || !size) {
+        _LFS_DBG("%s Invalid parameter!\r\n", __func__);
+        return LFS_ERR_INVAL;
+    }
 
-    if (_page_cache_addr == page_addr)
-        memcpy(buffer, _page_cache + page_off, size);
-    else
-        memcpy(buffer, (const void *)addr, size);
+    lfs_block_t address = LFS_FLASH_ADDR_BASE + (block * STM32WL_PAGE_SIZE + off);
+
+    memcpy(buffer, (void *)address, size);
 
     return LFS_ERR_OK;
 }
 
-// Program a region in a block. The block must have previously been erased.
-// Writes are accumulated in the page cache and flushed on sync or page eviction.
+// Program a region in a block. The block must have previously
+// been erased. Negative error codes are propogated to the user.
+// May return LFS_ERR_CORRUPT if the block should be considered bad.
 static int _internal_flash_prog(const struct lfs_config *c, lfs_block_t block, lfs_off_t off, const void *buffer, lfs_size_t size)
 {
+    lfs_block_t address = LFS_FLASH_ADDR_BASE + (block * STM32WL_PAGE_SIZE + off);
+    HAL_StatusTypeDef hal_rc = HAL_OK;
+    uint32_t dw_count = size / 8;
+    uint64_t *bufp = (uint64_t *)buffer;
+
     LFS_UNUSED(c);
 
-    uint32_t addr = LFS_FLASH_ADDR_BASE + block * LFS_BLOCK_SIZE + off;
-    uint32_t page_addr = addr & ~(uint32_t)(STM32WL_PAGE_SIZE - 1);
-    uint32_t page_off = addr & (uint32_t)(STM32WL_PAGE_SIZE - 1);
+    _LFS_DBG("Programming %d bytes/%d doublewords at address 0x%08x/block %d, offset %d.", size, dw_count, address, block, off);
+    if (HAL_FLASH_Unlock() != HAL_OK) {
+        return LFS_ERR_IO;
+    }
+    for (uint32_t i = 0; i < dw_count; i++) {
+        if ((address < LFS_FLASH_ADDR_BASE) || (address > LFS_FLASH_ADDR_END)) {
+            _LFS_DBG("Wanted to program out of bound of FLASH: 0x%08x.\n", address);
+            HAL_FLASH_Lock();
+            return LFS_ERR_INVAL;
+        }
+        hal_rc = HAL_FLASH_Program(FLASH_TYPEPROGRAM_DOUBLEWORD, address, *bufp);
+        if (hal_rc != HAL_OK) {
+            /* Error occurred while writing data in Flash memory.
+             * User can add here some code to deal with this error.
+             */
+            _LFS_DBG("Program error at (0x%08x), 0x%X, error: 0x%08x\n", address, hal_rc, HAL_FLASH_GetError());
+        }
+        address += 8;
+        bufp += 1;
+    }
+    if (HAL_FLASH_Lock() != HAL_OK) {
+        return LFS_ERR_IO;
+    }
 
-    int rc = _flash_cache_load(page_addr);
-    if (rc != LFS_ERR_OK)
-        return rc;
-
-    memcpy(_page_cache + page_off, buffer, size);
-    _page_cache_dirty = true;
-    return LFS_ERR_OK;
+    return hal_rc == HAL_OK ? LFS_ERR_OK : LFS_ERR_IO; // If HAL_OK, return LFS_ERR_OK, else return LFS_ERR_IO
 }
 
-// Erase a virtual block. Marks the corresponding region in the page cache as 0xFF.
-// Physical erase of the containing page is deferred until sync or page eviction.
+// Erase a block. A block must be erased before being programmed.
+// The state of an erased block is undefined. Negative error codes
+// are propogated to the user.
+// May return LFS_ERR_CORRUPT if the block should be considered bad.
 static int _internal_flash_erase(const struct lfs_config *c, lfs_block_t block)
 {
+    lfs_block_t address = LFS_FLASH_ADDR_BASE + (block * STM32WL_PAGE_SIZE);
+    HAL_StatusTypeDef hal_rc;
+    FLASH_EraseInitTypeDef EraseInitStruct = {.TypeErase = FLASH_TYPEERASE_PAGES, .Page = 0, .NbPages = 1};
+    uint32_t PAGEError = 0;
+
     LFS_UNUSED(c);
 
-    uint32_t addr = LFS_FLASH_ADDR_BASE + block * LFS_BLOCK_SIZE;
-    uint32_t page_addr = addr & ~(uint32_t)(STM32WL_PAGE_SIZE - 1);
-    uint32_t page_off = addr & (uint32_t)(STM32WL_PAGE_SIZE - 1);
+    if ((address < LFS_FLASH_ADDR_BASE) || (address > LFS_FLASH_ADDR_END)) {
+        _LFS_DBG("Wanted to erase out of bound of FLASH: 0x%08x.\n", address);
+        return LFS_ERR_INVAL;
+    }
+    /* calculate the absolute page, i.e. what the ST wants */
+    EraseInitStruct.Page = (address - STM32WL_FLASH_BASE) / STM32WL_PAGE_SIZE;
+    _LFS_DBG("Erasing block %d at 0x%08x... ", block, address);
+    HAL_FLASH_Unlock();
+    hal_rc = HAL_FLASHEx_Erase(&EraseInitStruct, &PAGEError);
+    HAL_FLASH_Lock();
 
-    int rc = _flash_cache_load(page_addr);
-    if (rc != LFS_ERR_OK)
-        return rc;
-
-    memset(_page_cache + page_off, 0xFF, LFS_BLOCK_SIZE);
-    _page_cache_dirty = true;
-    return LFS_ERR_OK;
+    return hal_rc == HAL_OK ? LFS_ERR_OK : LFS_ERR_IO; // If HAL_OK, return LFS_ERR_OK, else return LFS_ERR_IO
 }
 
-// Flush the write-behind cache to flash.
+// Sync the state of the underlying block device. Negative error codes
+// are propogated to the user.
 static int _internal_flash_sync(const struct lfs_config *c)
 {
     LFS_UNUSED(c);
-    return _flash_cache_flush();
+    // write function performs no caching.  No need for sync.
+
+    return LFS_ERR_OK;
 }
 
-static struct lfs_config _InternalFSConfig = {
-    .context = NULL,
+static struct lfs_config _InternalFSConfig = {.context = NULL,
 
-    .read = _internal_flash_read,
-    .prog = _internal_flash_prog,
-    .erase = _internal_flash_erase,
-    .sync = _internal_flash_sync,
+                                              .read = _internal_flash_read,
+                                              .prog = _internal_flash_prog,
+                                              .erase = _internal_flash_erase,
+                                              .sync = _internal_flash_sync,
 
-    .read_size = LFS_BLOCK_SIZE,
-    .prog_size = LFS_BLOCK_SIZE,
-    .block_size = LFS_BLOCK_SIZE,
-    .block_count = LFS_FLASH_TOTAL_SIZE / LFS_BLOCK_SIZE,
-    .lookahead = 128,
+                                              .read_size = LFS_BLOCK_SIZE,
+                                              .prog_size = LFS_BLOCK_SIZE,
+                                              .block_size = LFS_BLOCK_SIZE,
+                                              .block_count = LFS_FLASH_TOTAL_SIZE / LFS_BLOCK_SIZE,
+                                              .lookahead = 128,
 
-    .read_buffer = NULL,
-    .prog_buffer = NULL,
-    .lookahead_buffer = NULL,
-    .file_buffer = NULL,
-};
+                                              .read_buffer = NULL,
+                                              .prog_buffer = NULL,
+                                              .lookahead_buffer = NULL,
+                                              .file_buffer = NULL};
 
 LittleFS InternalFS;
 
@@ -228,17 +179,17 @@ bool LittleFS::begin(void)
         /* There is not enough space on this device for a filesystem. */
         return false;
     }
-    // failed to mount, erase all virtual blocks then format and mount again
+    // failed to mount, erase all pages then format and mount again
     if (!STM32_LittleFS::begin()) {
-        for (lfs_block_t block = 0; block < _InternalFSConfig.block_count; block++) {
-            _internal_flash_erase(&_InternalFSConfig, block);
+        // Erase all pages of internal flash region for Filesystem.
+        for (uint32_t addr = LFS_FLASH_ADDR_BASE; addr < (LFS_FLASH_ADDR_END + 1); addr += STM32WL_PAGE_SIZE) {
+            _internal_flash_erase(&_InternalFSConfig, (addr - LFS_FLASH_ADDR_BASE) / STM32WL_PAGE_SIZE);
         }
-        _flash_cache_flush(); // flush the last cached page
 
         // lfs format
         this->format();
 
-        // mount again; if still failed, give up
+        // mount again if still failed, give up
         if (!STM32_LittleFS::begin())
             return false;
     }

src/platform/stm32wl/STM32_LittleFS_File.cpp

@@ -22,7 +22,6 @@
  * THE SOFTWARE.
  */
 
-#include "LittleFS.h"
 #include "STM32_LittleFS.h"
 #include <Arduino.h>
 
@@ -392,8 +391,3 @@ void File::rewindDirectory(void)
     }
     _fs->_unlockFS();
 }
-
-// Default constructor — binds to the global InternalFS instance.
-// Allows File to be declared without an explicit filesystem argument,
-// matching the API of ESP32/RP2040/Portduino File objects.
-File::File() : File(InternalFS) {}

src/platform/stm32wl/STM32_LittleFS_File.h

@@ -44,7 +44,6 @@ class File : public Stream
   public:
     explicit File(STM32_LittleFS &fs);
     File(char const *filename, uint8_t mode, STM32_LittleFS &fs);
-    File(); // default-constructs against InternalFS; defined in STM32_LittleFS_File.cpp
 
   public:
     bool open(char const *filename, uint8_t mode);

src/xmodem.h

@@ -61,7 +61,11 @@ class XModemAdapter
 
     uint16_t packetno = 0;
 
+#if defined(ARCH_NRF52) || defined(ARCH_STM32WL)
+    File file = File(FSCom);
+#else
     File file;
+#endif
 
     char filename[sizeof(meshtastic_XModem_buffer_t::bytes)] = {0};
 

test/test_mac_from_string/test_main.cpp

@@ -1,195 +0,0 @@
-// Unit tests for MAC_from_string in src/platform/portduino/PortduinoGlue.cpp.
-//
-// Regression coverage for when the function stripped colons from
-// its mac_str parameter but then read bytes from the global
-// portduino_config.mac_address. Symptoms: --hwid silently ignored when
-// MACAddress: was also set, and SIGABRT (stoi: no conversion) when --hwid
-// was used without MACAddress: in config.yaml.
-#include "Arduino.h"
-#include "TestUtil.h"
-#include <cstdint>
-#include <cstring>
-#include <string>
-#include <unity.h>
-
-// Forward-declare instead of including PortduinoGlue.h to avoid pulling in
-// LR11x0Interface, USBHal, mesh.pb.h, yaml-cpp, and the full portduino_config
-// struct just to test a self-contained string parser. The symbol is defined
-// in PortduinoGlue.cpp and resolved at link time.
-bool MAC_from_string(std::string mac_str, uint8_t *dmac);
-
-void setUp(void) {}
-void tearDown(void) {}
-
-// --- Happy-path parsing ---
-
-void test_colon_separated_uppercase()
-{
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("AA:BB:CC:DD:EE:FF", dmac));
-    TEST_ASSERT_EQUAL_HEX8(0xAA, dmac[0]);
-    TEST_ASSERT_EQUAL_HEX8(0xBB, dmac[1]);
-    TEST_ASSERT_EQUAL_HEX8(0xCC, dmac[2]);
-    TEST_ASSERT_EQUAL_HEX8(0xDD, dmac[3]);
-    TEST_ASSERT_EQUAL_HEX8(0xEE, dmac[4]);
-    TEST_ASSERT_EQUAL_HEX8(0xFF, dmac[5]);
-}
-
-void test_colon_separated_lowercase()
-{
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("02:ca:fe:ba:be:01", dmac));
-    TEST_ASSERT_EQUAL_HEX8(0x02, dmac[0]);
-    TEST_ASSERT_EQUAL_HEX8(0xCA, dmac[1]);
-    TEST_ASSERT_EQUAL_HEX8(0xFE, dmac[2]);
-    TEST_ASSERT_EQUAL_HEX8(0xBA, dmac[3]);
-    TEST_ASSERT_EQUAL_HEX8(0xBE, dmac[4]);
-    TEST_ASSERT_EQUAL_HEX8(0x01, dmac[5]);
-}
-
-void test_no_colons_packed_hex()
-{
-    // The CLI form produced by some tools — 12 hex chars, no separators.
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("AABBCCDDEEFF", dmac));
-    TEST_ASSERT_EQUAL_HEX8(0xAA, dmac[0]);
-    TEST_ASSERT_EQUAL_HEX8(0xFF, dmac[5]);
-}
-
-void test_two_distinct_inputs_yield_distinct_outputs()
-{
-    // Direct regression for the original bug: parsing two different MAC
-    // strings in succession must produce two different byte sequences.
-    // Pre-fix, both calls would have produced identical bytes derived from
-    // the (untouched) global portduino_config.mac_address.
-    uint8_t a[6] = {0};
-    uint8_t b[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("AA:BB:CC:DD:EE:FF", a));
-    TEST_ASSERT_TRUE(MAC_from_string("02:CA:FE:BA:BE:01", b));
-    TEST_ASSERT_NOT_EQUAL(0, std::memcmp(a, b, 6));
-    TEST_ASSERT_EQUAL_HEX8(0xAA, a[0]);
-    TEST_ASSERT_EQUAL_HEX8(0x02, b[0]);
-}
-
-void test_does_not_read_external_state()
-{
-    // The function must derive every byte from its parameter, not from any
-    // global. Provide a unique MAC and verify all six bytes match the input
-    // exactly — leaves no room for the function to be smuggling bytes from
-    // elsewhere.
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("12:34:56:78:9A:BC", dmac));
-    const uint8_t expected[6] = {0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC};
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(expected, dmac, 6);
-}
-
-// --- Rejected inputs ---
-// Pre-fix, the empty/short cases either crashed (stoi exception on substr("")
-// of the empty global) or silently filled dmac with stale bytes. Post-fix,
-// the length guard rejects them cleanly with `false` and dmac is unchanged.
-
-void test_empty_string_returns_false()
-{
-    uint8_t dmac[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11};
-    uint8_t before[6];
-    std::memcpy(before, dmac, 6);
-    TEST_ASSERT_FALSE(MAC_from_string("", dmac));
-    // dmac must be untouched on failure.
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(before, dmac, 6);
-}
-
-void test_too_short_returns_false()
-{
-    uint8_t dmac[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11};
-    uint8_t before[6];
-    std::memcpy(before, dmac, 6);
-    TEST_ASSERT_FALSE(MAC_from_string("AA:BB:CC", dmac));
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(before, dmac, 6);
-}
-
-void test_too_long_returns_false()
-{
-    uint8_t dmac[6] = {0};
-    // 14 hex chars after colon-strip > 12.
-    TEST_ASSERT_FALSE(MAC_from_string("AA:BB:CC:DD:EE:FF:00", dmac));
-}
-
-void test_only_colons_returns_false()
-{
-    uint8_t dmac[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11};
-    uint8_t before[6];
-    std::memcpy(before, dmac, 6);
-    TEST_ASSERT_FALSE(MAC_from_string(":::::", dmac));
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(before, dmac, 6);
-}
-
-void test_extra_colons_still_parses()
-{
-    // Colon stripping happens before length check, so an unconventional
-    // grouping that totals 12 hex chars after stripping is still accepted.
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_TRUE(MAC_from_string("AABB:CCDD:EEFF", dmac));
-    TEST_ASSERT_EQUAL_HEX8(0xAA, dmac[0]);
-    TEST_ASSERT_EQUAL_HEX8(0xFF, dmac[5]);
-}
-
-void test_non_hex_input_returns_false()
-{
-    // 12 chars of non-hex would have made std::stoi throw before the
-    // try/catch wrapper was added, killing the daemon. Now must return false
-    // and leave dmac untouched.
-    uint8_t dmac[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11};
-    uint8_t before[6];
-    std::memcpy(before, dmac, 6);
-    TEST_ASSERT_FALSE(MAC_from_string("ZZ:ZZ:ZZ:ZZ:ZZ:ZZ", dmac));
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(before, dmac, 6);
-}
-
-void test_partial_hex_failure_preserves_dmac()
-{
-    // First five bytes are valid hex; the sixth ("ZZ") is not. Without the
-    // temp-buffer staging, dmac would be partially overwritten with the five
-    // good bytes plus stale data in slot 5 — silently producing a wrong MAC
-    // since the only caller that uses this in getMacAddr() ignores the bool
-    // return value.
-    uint8_t dmac[6] = {0xDE, 0xAD, 0xBE, 0xEF, 0x00, 0x11};
-    uint8_t before[6];
-    std::memcpy(before, dmac, 6);
-    TEST_ASSERT_FALSE(MAC_from_string("AA:BB:CC:DD:EE:ZZ", dmac));
-    TEST_ASSERT_EQUAL_HEX8_ARRAY(before, dmac, 6);
-}
-
-void test_embedded_non_hex_returns_false()
-{
-    // std::stoi tolerates leading whitespace and a "0x" prefix, so a stray
-    // space inside a 2-char window like " F" would silently parse as 0xF.
-    // The per-character isxdigit() pre-check rejects these. The 14-char
-    // "0xAABBCCDDEEFF" is also rejected by the length check.
-    uint8_t dmac[6] = {0};
-    TEST_ASSERT_FALSE(MAC_from_string("AA:BB:CC:DD:EE: F", dmac));
-    TEST_ASSERT_FALSE(MAC_from_string("0xAABBCCDDEEFF", dmac));
-}
-
-// --- Unity lifecycle ---
-
-void setup()
-{
-    initializeTestEnvironment();
-    UNITY_BEGIN();
-    RUN_TEST(test_colon_separated_uppercase);
-    RUN_TEST(test_colon_separated_lowercase);
-    RUN_TEST(test_no_colons_packed_hex);
-    RUN_TEST(test_two_distinct_inputs_yield_distinct_outputs);
-    RUN_TEST(test_does_not_read_external_state);
-    RUN_TEST(test_empty_string_returns_false);
-    RUN_TEST(test_too_short_returns_false);
-    RUN_TEST(test_too_long_returns_false);
-    RUN_TEST(test_only_colons_returns_false);
-    RUN_TEST(test_extra_colons_still_parses);
-    RUN_TEST(test_non_hex_input_returns_false);
-    RUN_TEST(test_partial_hex_failure_preserves_dmac);
-    RUN_TEST(test_embedded_non_hex_returns_false);
-    exit(UNITY_END());
-}
-
-void loop() {}

test/test_meshpacket_serializer/ports/test_dmshell.cpp

@@ -0,0 +1,97 @@
+#include "../test_helpers.h"
+#include "mesh/mesh-pb-constants.h"
+
+namespace
+{
+struct BytesDecodeState {
+    uint8_t *buffer;
+    size_t capacity;
+    size_t length;
+};
+
+struct BytesEncodeState {
+    const uint8_t *buffer;
+    size_t length;
+};
+
+bool decodeBytesField(pb_istream_t *stream, const pb_field_iter_t *field, void **arg)
+{
+    (void)field;
+    auto *state = static_cast<BytesDecodeState *>(*arg);
+    if (!state) {
+        return false;
+    }
+
+    const size_t fieldLen = stream->bytes_left;
+    if (fieldLen > state->capacity) {
+        return false;
+    }
+
+    if (!pb_read(stream, state->buffer, fieldLen)) {
+        return false;
+    }
+
+    state->length = fieldLen;
+    return true;
+}
+
+bool encodeBytesField(pb_ostream_t *stream, const pb_field_iter_t *field, void *const *arg)
+{
+    auto *state = static_cast<const BytesEncodeState *>(*arg);
+    if (!state || !state->buffer || state->length == 0) {
+        return true;
+    }
+
+    if (!pb_encode_tag_for_field(stream, field)) {
+        return false;
+    }
+
+    return pb_encode_string(stream, state->buffer, state->length);
+}
+
+void assert_dmshell_roundtrip(meshtastic_RemoteShell_OpCode op, uint32_t sessionId, uint32_t seq, const uint8_t *payload,
+                              size_t payloadLen, uint32_t cols = 0, uint32_t rows = 0)
+{
+    meshtastic_RemoteShell tx = meshtastic_RemoteShell_init_zero;
+    tx.op = op;
+    tx.session_id = sessionId;
+    tx.seq = seq;
+    tx.cols = cols;
+    tx.rows = rows;
+
+    uint8_t encoded[meshtastic_Constants_DATA_PAYLOAD_LEN] = {0};
+    size_t encodedLen = pb_encode_to_bytes(encoded, sizeof(encoded), meshtastic_RemoteShell_fields, &tx);
+    TEST_ASSERT_GREATER_THAN_UINT32(0, encodedLen);
+
+    meshtastic_RemoteShell rx = meshtastic_RemoteShell_init_zero;
+
+    TEST_ASSERT_TRUE(pb_decode_from_bytes(encoded, encodedLen, meshtastic_RemoteShell_fields, &rx));
+    TEST_ASSERT_EQUAL(op, rx.op);
+    TEST_ASSERT_EQUAL_UINT32(sessionId, rx.session_id);
+    TEST_ASSERT_EQUAL_UINT32(seq, rx.seq);
+    TEST_ASSERT_EQUAL_UINT32(cols, rx.cols);
+    TEST_ASSERT_EQUAL_UINT32(rows, rx.rows);
+}
+} // namespace
+
+void test_dmshell_open_roundtrip()
+{
+    assert_dmshell_roundtrip(meshtastic_RemoteShell_OpCode_OPEN, 0x101, 1, nullptr, 0, 120, 40);
+}
+
+void test_dmshell_input_roundtrip()
+{
+    const uint8_t payload[] = {'l', 's', '\n'};
+    assert_dmshell_roundtrip(meshtastic_RemoteShell_OpCode_INPUT, 0x202, 2, payload, sizeof(payload));
+}
+
+void test_dmshell_resize_roundtrip()
+{
+    assert_dmshell_roundtrip(meshtastic_RemoteShell_OpCode_RESIZE, 0x303, 3, nullptr, 0, 180, 55);
+}
+
+void test_dmshell_close_roundtrip()
+{
+    const uint8_t reason[] = {'b', 'y', 'e'};
+    assert_dmshell_roundtrip(meshtastic_RemoteShell_OpCode_CLOSE, 0x404, 4, reason, sizeof(reason));
+}

test/test_meshpacket_serializer/test_serializer.cpp

@@ -19,6 +19,10 @@ void test_telemetry_environment_metrics_complete_coverage();
 void test_telemetry_environment_metrics_unset_fields();
 void test_encrypted_packet_serialization();
 void test_empty_encrypted_packet();
+void test_dmshell_open_roundtrip();
+void test_dmshell_input_roundtrip();
+void test_dmshell_resize_roundtrip();
+void test_dmshell_close_roundtrip();
 
 void setup()
 {
@@ -52,6 +56,12 @@ void setup()
     RUN_TEST(test_encrypted_packet_serialization);
     RUN_TEST(test_empty_encrypted_packet);
 
+    // DMShell protobuf transport tests
+    RUN_TEST(test_dmshell_open_roundtrip);
+    RUN_TEST(test_dmshell_input_roundtrip);
+    RUN_TEST(test_dmshell_resize_roundtrip);
+    RUN_TEST(test_dmshell_close_roundtrip);
+
     UNITY_END();
 }
 

variants/esp32s3/m5stack_cardputer_adv/variant.h

@@ -9,7 +9,6 @@
 #define ST7789_BUSY -1
 // #define VTFT_CTRL 38
 #define VTFT_LEDA 38
-#define TFT_BACKLIGHT_ON HIGH
 // #define ST7789_BL (32+6)
 #define ST7789_SPI_HOST SPI2_HOST
 // #define TFT_BL (32+6)

variants/native/portduino.ini

@@ -2,7 +2,7 @@
 [portduino_base]
 platform =
   # renovate: datasource=git-refs depName=platform-native packageName=https://github.com/meshtastic/platform-native gitBranch=develop
-  https://github.com/meshtastic/platform-native/archive/cab4b21d902973e43c938dab3cf4844ba02547ec.zip
+  https://github.com/meshtastic/platform-native/archive/4ea5e09ac7d51a593e12ec7c1ebb6cd06745ce53.zip
 framework = arduino
 
 build_src_filter = 

variants/native/portduino/platformio.ini

@@ -125,8 +125,6 @@ test_testing_command =
 ;
 ; Prerequisites (Homebrew):
 ;   brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config
-;   # Optional: enable the HTTP API (PiWebServer) on macOS:
-;   brew install ulfius
 ;
 ; The macOS-side patches now live upstream:
 ;   * meshtastic/platform-native — `String.h`-shadow shim, `-Wno-enum-constexpr-conversion`,
@@ -193,16 +191,7 @@ build_flags = ${portduino_base.build_flags_common}
   ; style screen-driver hooks scattered through sensor sources.
   -DHAS_SCREEN=0
   -DMESHTASTIC_EXCLUDE_SCREEN=1
-  ; openssl@3 is the keg-only Homebrew formula; --cflags is required so the
-  ; compiler finds <openssl/*.h> in the Homebrew prefix (not just the linker).
-  !pkg-config --cflags --libs openssl --silence-errors || :
-  ; PiWebServer (src/mesh/raspihttp/PiWebServer.cpp) auto-engages when ulfius
-  ; headers are reachable via `#if __has_include(<ulfius.h>)`. The `|| :`
-  ; tail keeps the build green when the user hasn't run `brew install ulfius`
-  ; — they just don't get the HTTP API in that case.
-  !pkg-config --cflags --libs liborcania --silence-errors || :
-  !pkg-config --cflags --libs libyder --silence-errors || :
-  !pkg-config --cflags --libs libulfius --silence-errors || :
+  !pkg-config --libs openssl --silence-errors || :
 ; src/input/Linux*.{cpp,h} drive evdev (`<linux/input.h>`) which doesn't exist
 ; on macOS. graphics/Panel_sdl.* and graphics/TFTDisplay.cpp pull LovyanGFX
 ; (which we lib_ignore on macOS for the <malloc.h> issue). Neither is needed
@@ -217,28 +206,3 @@ build_src_filter = ${native_base.build_src_filter}
 lib_ignore =
   ${portduino_base.lib_ignore}
   LovyanGFX
-
-; ---------------------------------------------------------------------------
-; Same as [env:native-macos] but built with AddressSanitizer for catching
-; use-after-free, leaks, and OOB access during local development. Headless
-; (no SDL/X11/libinput) so it stays cheap to build. Mirrors the shape of
-; [env:native-tft-debug] but without the TFT/X11 dependencies.
-;
-;   pio run -e native-macos-debug
-;   .pio/build/native-macos-debug/meshtasticd -s
-;
-; ASan runtime tuning (set in the shell before launching):
-;   ASAN_OPTIONS=detect_leaks=1:halt_on_error=0:abort_on_error=1
-;   MallocStackLogging=1   # macOS: nicer stack traces in malloc reports
-; ---------------------------------------------------------------------------
-[env:native-macos-debug]
-extends = native_base
-build_type = debug
-build_unflags = ${env:native-macos.build_unflags}
-build_flags = ${env:native-macos.build_flags}
-  -O0
-  -g
-  -fsanitize=address
-  -fno-omit-frame-pointer
-build_src_filter = ${env:native-macos.build_src_filter}
-lib_ignore = ${env:native-macos.lib_ignore}

variants/stm32/CDEBYTE_E77-MBL/platformio.ini

@@ -1,7 +1,7 @@
 [env:CDEBYTE_E77-MBL]
 extends = stm32_base
 board = ebyte_e77_dev
-board_upload.maximum_size = 247808 ; reserve the last 14KB for filesystem (reduced from 20KB in LittleFS.cpp)
+board_upload.maximum_size = 233472 ; reserve the last 28KB for filesystem
 board_level = extra
 build_flags =
   ${stm32_base.build_flags}

variants/stm32/milesight_gs301/platformio.ini

@@ -4,7 +4,7 @@
 extends = stm32_base
 board = wiscore_rak3172 ; Convenient choice as the same USART is used for programming/debug
 board_level = extra
-board_upload.maximum_size = 247808 ; reserve the last 14KB for filesystem (reduced from 20KB in LittleFS.cpp)
+board_upload.maximum_size = 233472 ; reserve the last 28KB for filesystem
 build_flags =
   ${stm32_base.build_flags}
   -Ivariants/stm32/milesight_gs301

variants/stm32/rak3172/platformio.ini

@@ -2,7 +2,7 @@
 extends = stm32_base
 board = wiscore_rak3172
 board_level = pr
-board_upload.maximum_size = 247808 ; reserve the last 14KB for filesystem (reduced from 20KB in LittleFS.cpp)
+board_upload.maximum_size = 233472 ; reserve the last 28KB for filesystem
 build_flags =
   ${stm32_base.build_flags}
   -Ivariants/stm32/rak3172

variants/stm32/russell/platformio.ini

@@ -8,7 +8,7 @@
 extends = stm32_base
 board = wiscore_rak3172
 board_level = extra
-board_upload.maximum_size = 247808 ; reserve the last 14KB for filesystem (reduced from 20KB in LittleFS.cpp)
+board_upload.maximum_size = 233472 ; reserve the last 28KB for filesystem
 build_flags =
   ${stm32_base.build_flags}
   -Ivariants/stm32/russell

variants/stm32/wio-e5/platformio.ini

@@ -2,7 +2,7 @@
 extends = stm32_base
 board = lora_e5_dev_board
 board_level = pr
-board_upload.maximum_size = 247808 ; reserve the last 14KB for filesystem (reduced from 20KB in LittleFS.cpp)
+board_upload.maximum_size = 233472 ; reserve the last 28KB for filesystem
 build_flags =
   ${stm32_base.build_flags}
   -Ivariants/stm32/wio-e5