test: rename HA MQTT test harness

This commit is contained in:
2026-09-07 19:29:21 +10:00
parent 50cf1c36f0
commit 0dbf0ecb75
17 changed files with 107 additions and 107 deletions
@@ -1,4 +1,4 @@
name: Home Assistant MQTT contract name: Home Assistant MQTT test harness
on: on:
workflow_dispatch: workflow_dispatch:
@@ -9,7 +9,7 @@ permissions:
contents: read contents: read
jobs: jobs:
contract: test_harness:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
fail-fast: false fail-fast: false
@@ -23,8 +23,8 @@ jobs:
expect_disabled_cleanup: "1" expect_disabled_cleanup: "1"
env: env:
HA_VERSION: ${{ matrix.home_assistant }} HA_VERSION: ${{ matrix.home_assistant }}
CONTRACT_EXPECT_DISABLED_CLEANUP: ${{ matrix.expect_disabled_cleanup }} TEST_HARNESS_EXPECT_DISABLED_CLEANUP: ${{ matrix.expect_disabled_cleanup }}
COMPOSE_FILE: tests/ha-contract/compose.yaml COMPOSE_FILE: tests/ha-test-harness/compose.yaml
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Start broker and Home Assistant - name: Start broker and Home Assistant
@@ -34,10 +34,10 @@ jobs:
- name: Restart Home Assistant from retained data - name: Restart Home Assistant from retained data
run: docker compose restart homeassistant run: docker compose restart homeassistant
- name: Check retained discovery after restart - name: Check retained discovery after restart
run: docker compose run --rm -e CONTRACT_MODE=retained-restart tests run: docker compose run --rm -e TEST_HARNESS_MODE=retained-restart tests
- name: Collect Home Assistant logs on failure - name: Collect Home Assistant logs on failure
if: failure() if: failure()
run: docker compose logs --no-color homeassistant mqtt run: docker compose logs --no-color homeassistant mqtt
- name: Remove contract volumes - name: Remove test harness volumes
if: always() if: always()
run: docker compose down -v run: docker compose down -v
+3 -3
View File
@@ -1,7 +1,7 @@
# Compatibility baseline # Compatibility baseline
This maintenance line begins from fork commit `84cc0037b1c0` (release `v3.0.2`). This maintenance line begins from fork commit `84cc0037b1c0` (release `v3.0.2`).
It intentionally tracks Home Assistant's MQTT discovery contract without merging It intentionally tracks Home Assistant's MQTT discovery test harness without merging
upstream development wholesale. upstream development wholesale.
| Reference | Audited revision / target | | Reference | Audited revision / target |
@@ -10,7 +10,7 @@ upstream development wholesale.
| Upstream main | `1d333ab229b2` (`v2.1.0`) | | Upstream main | `1d333ab229b2` (`v2.1.0`) |
| Upstream develop | `a7039fad810b` (unreleased WIP 2.2.0) | | Upstream develop | `a7039fad810b` (unreleased WIP 2.2.0) |
| Device discovery minimum | Home Assistant `2024.11.0` | | Device discovery minimum | Home Assistant `2024.11.0` |
| Contract matrix | `2024.11.3`, current `stable`, current `dev` | | Test harness matrix | `2024.11.3`, current `stable`, current `dev` |
The only imported upstream code fix is the four missing-device-ID guards from The only imported upstream code fix is the four missing-device-ID guards from
upstream commit `9c9d074`. Device discovery migration, JSON validation, upstream commit `9c9d074`. Device discovery migration, JSON validation,
@@ -37,4 +37,4 @@ git diff --stat main...upstream/develop
Port only independently reviewed changes with regression tests; treat open Port only independently reviewed changes with regression tests; treat open
upstream pull requests as proposals, not release inputs. Run the native, upstream pull requests as proposals, not release inputs. Run the native,
board-compile, and HA contract gates before publishing a new release. board-compile, and HA test harness gates before publishing a new release.
@@ -1,7 +1,7 @@
# HA/MQTT contract testkit # HA/MQTT test harness testkit
This small Python package supplies generic test transport primitives for a This small Python package supplies generic test transport primitives for a
disposable Home Assistant + MQTT contract environment: disposable Home Assistant + MQTT test harness environment:
- Home Assistant onboarding, MQTT config-entry setup, WebSocket registry access, - Home Assistant onboarding, MQTT config-entry setup, WebSocket registry access,
REST state/service helpers, and readiness waiting; REST state/service helpers, and readiness waiting;
@@ -10,7 +10,7 @@ disposable Home Assistant + MQTT contract environment:
It deliberately contains no ArduinoHA discovery fixture, DeviceFramework It deliberately contains no ArduinoHA discovery fixture, DeviceFramework
import, entity expectation, or release policy. ArduinoHA owns its migration import, entity expectation, or release policy. ArduinoHA owns its migration
fixture in `tests/ha-contract`; DeviceFramework owns its hardware fixture and fixture in `tests/ha-test-harness`; DeviceFramework owns its hardware fixture and
Docker adapter in its own repository. Other projects can reuse this package by Docker adapter in its own repository. Other projects can reuse this package by
providing their own retained MQTT messages and assertions. providing their own retained MQTT messages and assertions.
@@ -3,9 +3,9 @@ requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta" build-backend = "setuptools.build_meta"
[project] [project]
name = "ha-mqtt-contract-testkit" name = "ha-mqtt-test-harness-testkit"
version = "0.1.0" version = "0.1.0"
description = "Generic Home Assistant MQTT contract-test support" description = "Generic Home Assistant MQTT test-harness support"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [ dependencies = [
"paho-mqtt==2.1.0", "paho-mqtt==2.1.0",
@@ -1,4 +1,4 @@
"""Generic Home Assistant MQTT contract-test support. """Generic Home Assistant MQTT test harness-test support.
This package deliberately knows nothing about ArduinoHA, DeviceFramework, or This package deliberately knows nothing about ArduinoHA, DeviceFramework, or
any particular discovery schema. Consumers own their fixtures and assertions; any particular discovery schema. Consumers own their fixtures and assertions;
@@ -8,10 +8,10 @@ the package owns only HA/MQTT transport, readiness, and artifact primitives.
from .artifacts import write_json_artifact, write_json_lines from .artifacts import write_json_artifact, write_json_lines
from .home_assistant import HomeAssistantClient from .home_assistant import HomeAssistantClient
from .mqtt import MqttObserver, RetainedPublisher from .mqtt import MqttObserver, RetainedPublisher
from .wait import ContractError, wait_until from .wait import TestHarnessError, wait_until
__all__ = [ __all__ = [
"ContractError", "TestHarnessError",
"HomeAssistantClient", "HomeAssistantClient",
"MqttObserver", "MqttObserver",
"RetainedPublisher", "RetainedPublisher",
@@ -10,11 +10,11 @@ from typing import Any
import requests import requests
import websocket import websocket
from .wait import ContractError, wait_until from .wait import TestHarnessError, wait_until
class HomeAssistantClient: class HomeAssistantClient:
"""Authenticated HA API client with a deliberately small contract surface.""" """Authenticated HA API client with a deliberately small test harness surface."""
def __init__(self, base_url: str, token: str): def __init__(self, base_url: str, token: str):
self.base_url = base_url.rstrip("/") self.base_url = base_url.rstrip("/")
@@ -32,9 +32,9 @@ class HomeAssistantClient:
base_url: str, base_url: str,
state_dir: str | pathlib.Path, state_dir: str | pathlib.Path,
*, *,
owner_name: str = "HA MQTT Contract Owner", owner_name: str = "HA MQTT Test Harness Owner",
username: str = "ha-mqtt-contract", username: str = "ha-mqtt-test-harness",
password: str = "ha-mqtt-contract-password", password: str = "ha-mqtt-test-harness-password",
) -> "HomeAssistantClient": ) -> "HomeAssistantClient":
"""Create or load an isolated owner token for a disposable HA volume.""" """Create or load an isolated owner token for a disposable HA volume."""
base_url = base_url.rstrip("/") base_url = base_url.rstrip("/")
@@ -44,7 +44,7 @@ class HomeAssistantClient:
if token_file.exists(): if token_file.exists():
return cls(base_url, token_file.read_text(encoding="utf-8").strip()) return cls(base_url, token_file.read_text(encoding="utf-8").strip())
client_id = "http://ha-mqtt-contract.local/" client_id = "http://ha-mqtt-test-harness.local/"
user = { user = {
"client_id": client_id, "client_id": client_id,
"name": owner_name, "name": owner_name,
@@ -58,7 +58,7 @@ class HomeAssistantClient:
) )
auth_code = created.get("auth_code") auth_code = created.get("auth_code")
if not auth_code: if not auth_code:
raise ContractError("Home Assistant onboarding did not return an auth_code") raise TestHarnessError("Home Assistant onboarding did not return an auth_code")
token_response = cls._response_json( token_response = cls._response_json(
requests.post( requests.post(
f"{base_url}/auth/token", f"{base_url}/auth/token",
@@ -69,7 +69,7 @@ class HomeAssistantClient:
) )
token = token_response.get("access_token") token = token_response.get("access_token")
if not token: if not token:
raise ContractError("Home Assistant token exchange did not return an access_token") raise TestHarnessError("Home Assistant token exchange did not return an access_token")
client = cls(base_url, token) client = cls(base_url, token)
for path, payload in ( for path, payload in (
@@ -78,7 +78,7 @@ class HomeAssistantClient:
): ):
response = requests.post(f"{base_url}{path}", headers=client._headers, json=payload, timeout=10) response = requests.post(f"{base_url}{path}", headers=client._headers, json=payload, timeout=10)
if response.status_code not in (200, 201, 400, 404): if response.status_code not in (200, 201, 400, 404):
raise ContractError( raise TestHarnessError(
f"Home Assistant onboarding step {path} failed: {response.status_code} {response.text}" f"Home Assistant onboarding step {path} failed: {response.status_code} {response.text}"
) )
state.mkdir(parents=True, exist_ok=True) state.mkdir(parents=True, exist_ok=True)
@@ -89,19 +89,19 @@ class HomeAssistantClient:
def from_state(cls, base_url: str, state_dir: str | pathlib.Path) -> "HomeAssistantClient": def from_state(cls, base_url: str, state_dir: str | pathlib.Path) -> "HomeAssistantClient":
token_file = pathlib.Path(state_dir) / "ha-token" token_file = pathlib.Path(state_dir) / "ha-token"
if not token_file.exists(): if not token_file.exists():
raise ContractError("Home Assistant token has not been bootstrapped") raise TestHarnessError("Home Assistant token has not been bootstrapped")
return cls(base_url, token_file.read_text(encoding="utf-8").strip()) return cls(base_url, token_file.read_text(encoding="utf-8").strip())
@staticmethod @staticmethod
def _response_json(response: requests.Response, context: str) -> dict[str, Any]: def _response_json(response: requests.Response, context: str) -> dict[str, Any]:
if not response.ok: if not response.ok:
raise ContractError(f"{context} failed ({response.status_code}): {response.text}") raise TestHarnessError(f"{context} failed ({response.status_code}): {response.text}")
try: try:
value = response.json() value = response.json()
except ValueError as error: except ValueError as error:
raise ContractError(f"{context} returned invalid JSON: {error}") from error raise TestHarnessError(f"{context} returned invalid JSON: {error}") from error
if not isinstance(value, dict): if not isinstance(value, dict):
raise ContractError(f"{context} returned unexpected JSON: {value}") raise TestHarnessError(f"{context} returned unexpected JSON: {value}")
return value return value
@classmethod @classmethod
@@ -126,7 +126,7 @@ class HomeAssistantClient:
if flow.get("type") == "create_entry": if flow.get("type") == "create_entry":
return return
if flow.get("type") != "form" or not flow.get("flow_id"): if flow.get("type") != "form" or not flow.get("flow_id"):
raise ContractError(f"unexpected MQTT config-entry flow result: {flow}") raise TestHarnessError(f"unexpected MQTT config-entry flow result: {flow}")
user_input: dict[str, Any] = {"broker": host, "port": port} user_input: dict[str, Any] = {"broker": host, "port": port}
schema_names = { schema_names = {
field.get("name") for field in flow.get("data_schema", []) if isinstance(field, dict) field.get("name") for field in flow.get("data_schema", []) if isinstance(field, dict)
@@ -147,7 +147,7 @@ class HomeAssistantClient:
"MQTT config-entry flow configuration", "MQTT config-entry flow configuration",
) )
if configured.get("type") != "create_entry": if configured.get("type") != "create_entry":
raise ContractError(f"MQTT config-entry flow did not create an entry: {configured}") raise TestHarnessError(f"MQTT config-entry flow did not create an entry: {configured}")
time.sleep(3) time.sleep(3)
def _ensure_socket(self) -> websocket.WebSocket: def _ensure_socket(self) -> websocket.WebSocket:
@@ -159,12 +159,12 @@ class HomeAssistantClient:
required = json.loads(socket.recv()) required = json.loads(socket.recv())
if required.get("type") != "auth_required": if required.get("type") != "auth_required":
socket.close() socket.close()
raise ContractError(f"unexpected Home Assistant WebSocket greeting: {required}") raise TestHarnessError(f"unexpected Home Assistant WebSocket greeting: {required}")
socket.send(json.dumps({"type": "auth", "access_token": self.token})) socket.send(json.dumps({"type": "auth", "access_token": self.token}))
authenticated = json.loads(socket.recv()) authenticated = json.loads(socket.recv())
if authenticated.get("type") != "auth_ok": if authenticated.get("type") != "auth_ok":
socket.close() socket.close()
raise ContractError(f"Home Assistant WebSocket authentication failed: {authenticated}") raise TestHarnessError(f"Home Assistant WebSocket authentication failed: {authenticated}")
self._socket = socket self._socket = socket
return socket return socket
@@ -183,23 +183,23 @@ class HomeAssistantClient:
if result.get("id") != message_id: if result.get("id") != message_id:
continue continue
if not result.get("success"): if not result.get("success"):
raise ContractError(f"WebSocket {message_type} failed: {result}") raise TestHarnessError(f"WebSocket {message_type} failed: {result}")
return result.get("result") return result.get("result")
def entity_registry(self) -> list[dict[str, Any]]: def entity_registry(self) -> list[dict[str, Any]]:
result = self.call("config/entity_registry/list") result = self.call("config/entity_registry/list")
if not isinstance(result, list): if not isinstance(result, list):
raise ContractError(f"entity registry returned unexpected value: {result}") raise TestHarnessError(f"entity registry returned unexpected value: {result}")
return result return result
# A concise compatibility spelling for contract suites; this remains schema-neutral. # A concise compatibility spelling for test harness suites; this remains schema-neutral.
def registry_entries(self) -> list[dict[str, Any]]: def registry_entries(self) -> list[dict[str, Any]]:
return self.entity_registry() return self.entity_registry()
def device_registry(self) -> list[dict[str, Any]]: def device_registry(self) -> list[dict[str, Any]]:
result = self.call("config/device_registry/list") result = self.call("config/device_registry/list")
if not isinstance(result, list): if not isinstance(result, list):
raise ContractError(f"device registry returned unexpected value: {result}") raise TestHarnessError(f"device registry returned unexpected value: {result}")
return result return result
def entities_for_device(self, device_id: str) -> list[dict[str, Any]]: def entities_for_device(self, device_id: str) -> list[dict[str, Any]]:
@@ -209,7 +209,7 @@ class HomeAssistantClient:
def find() -> dict[str, Any] | None: def find() -> dict[str, Any] | None:
matches = [entry for entry in self.entity_registry() if entry.get("unique_id") == unique_id] matches = [entry for entry in self.entity_registry() if entry.get("unique_id") == unique_id]
if len(matches) > 1: if len(matches) > 1:
raise ContractError(f"duplicate entity-registry entries for {unique_id}: {matches}") raise TestHarnessError(f"duplicate entity-registry entries for {unique_id}: {matches}")
return matches[0] if matches else None return matches[0] if matches else None
return wait_until(f"entity registry entry {unique_id}", find, timeout=timeout) return wait_until(f"entity registry entry {unique_id}", find, timeout=timeout)
@@ -235,11 +235,11 @@ class HomeAssistantClient:
timeout=15, timeout=15,
) )
if not response.ok: if not response.ok:
raise ContractError(f"service {domain}.{service} failed ({response.status_code}): {response.text}") raise TestHarnessError(f"service {domain}.{service} failed ({response.status_code}): {response.text}")
try: try:
result = response.json() result = response.json()
except ValueError as error: except ValueError as error:
raise ContractError(f"service {domain}.{service} returned invalid JSON: {error}") from error raise TestHarnessError(f"service {domain}.{service} returned invalid JSON: {error}") from error
if not isinstance(result, list): if not isinstance(result, list):
raise ContractError(f"service {domain}.{service} returned unexpected JSON: {result}") raise TestHarnessError(f"service {domain}.{service} returned unexpected JSON: {result}")
return result return result
@@ -1,4 +1,4 @@
"""MQTT publishing and observation helpers for retained-discovery contracts.""" """MQTT publishing and observation helpers for retained-discovery test harnesses."""
from __future__ import annotations from __future__ import annotations
@@ -10,7 +10,7 @@ from typing import Any
import paho.mqtt.client as mqtt import paho.mqtt.client as mqtt
from .wait import ContractError, wait_until from .wait import TestHarnessError, wait_until
class _MqttClient: class _MqttClient:
@@ -25,7 +25,7 @@ class _MqttClient:
self._client.loop_start() self._client.loop_start()
if not self._connected.wait(timeout=15): if not self._connected.wait(timeout=15):
self.close() self.close()
raise ContractError(f"MQTT client {client_prefix} did not connect within 15 seconds") raise TestHarnessError(f"MQTT client {client_prefix} did not connect within 15 seconds")
def _on_connect(self, _client: mqtt.Client, _userdata: Any, _flags: Any, reason_code: Any, _properties: Any) -> None: def _on_connect(self, _client: mqtt.Client, _userdata: Any, _flags: Any, reason_code: Any, _properties: Any) -> None:
if reason_code == 0: if reason_code == 0:
@@ -40,13 +40,13 @@ class RetainedPublisher(_MqttClient):
"""A QoS-1 retained publisher used by schema/migration fixtures.""" """A QoS-1 retained publisher used by schema/migration fixtures."""
def __init__(self, host: str, port: int): def __init__(self, host: str, port: int):
super().__init__(host, port, client_prefix="contract-publisher") super().__init__(host, port, client_prefix="test-harness-publisher")
def publish(self, topic: str, payload: str, *, retain: bool = True) -> None: def publish(self, topic: str, payload: str, *, retain: bool = True) -> None:
info = self._client.publish(topic, payload, qos=1, retain=retain) info = self._client.publish(topic, payload, qos=1, retain=retain)
info.wait_for_publish(timeout=10) info.wait_for_publish(timeout=10)
if not info.is_published(): if not info.is_published():
raise ContractError(f"MQTT publish timed out for {topic}") raise TestHarnessError(f"MQTT publish timed out for {topic}")
def retained_payload(self, topic: str, *, timeout: float = 30) -> str: def retained_payload(self, topic: str, *, timeout: float = 30) -> str:
received: list[str] = [] received: list[str] = []
@@ -75,17 +75,17 @@ class MqttObserver(_MqttClient):
self._messages: list[dict[str, Any]] = [] self._messages: list[dict[str, Any]] = []
self._lock = threading.Lock() self._lock = threading.Lock()
self._subscription_ack = threading.Event() self._subscription_ack = threading.Event()
super().__init__(host, port, client_prefix="contract-observer") super().__init__(host, port, client_prefix="test-harness-observer")
self._client.on_message = self._on_message self._client.on_message = self._on_message
self._client.on_subscribe = self._on_subscribe self._client.on_subscribe = self._on_subscribe
for topic in topics: for topic in topics:
result, _mid = self._client.subscribe(topic, qos=1) result, _mid = self._client.subscribe(topic, qos=1)
if result != mqtt.MQTT_ERR_SUCCESS: if result != mqtt.MQTT_ERR_SUCCESS:
self.close() self.close()
raise ContractError(f"unable to subscribe to MQTT topic {topic}") raise TestHarnessError(f"unable to subscribe to MQTT topic {topic}")
if not self._subscription_ack.wait(timeout=15): if not self._subscription_ack.wait(timeout=15):
self.close() self.close()
raise ContractError(f"MQTT subscription to {topic} was not acknowledged within 15 seconds") raise TestHarnessError(f"MQTT subscription to {topic} was not acknowledged within 15 seconds")
self._subscription_ack.clear() self._subscription_ack.clear()
def _on_subscribe( def _on_subscribe(
@@ -1,4 +1,4 @@
"""Bounded wait helpers shared by integration contracts.""" """Bounded wait helpers shared by integration test harnesses."""
from __future__ import annotations from __future__ import annotations
@@ -7,8 +7,8 @@ from collections.abc import Callable
from typing import TypeVar from typing import TypeVar
class ContractError(RuntimeError): class TestHarnessError(RuntimeError):
"""An external Home Assistant or MQTT contract could not be satisfied.""" """An external Home Assistant or MQTT test harness could not be satisfied."""
T = TypeVar("T") T = TypeVar("T")
@@ -33,4 +33,4 @@ def wait_until(
last_error = error last_error = error
time.sleep(interval) time.sleep(interval)
suffix = f" (last error: {last_error})" if last_error else "" suffix = f" (last error: {last_error})" if last_error else ""
raise ContractError(f"timed out waiting for {description}{suffix}") raise TestHarnessError(f"timed out waiting for {description}{suffix}")
-10
View File
@@ -1,10 +0,0 @@
FROM python:3.12-slim
WORKDIR /tests
COPY test-support/ha-mqtt-contract /opt/ha-mqtt-contract
COPY tests/ha-contract/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install --no-cache-dir /opt/ha-mqtt-contract
COPY tests/ha-contract/test_contract.py .
CMD ["python", "test_contract.py"]
+10
View File
@@ -0,0 +1,10 @@
FROM python:3.12-slim
WORKDIR /tests
COPY test-support/ha-mqtt-test-harness /opt/ha-mqtt-test-harness
COPY tests/ha-test-harness/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install --no-cache-dir /opt/ha-mqtt-test-harness
COPY tests/ha-test-harness/test_harness.py .
CMD ["python", "test_harness.py"]
@@ -1,4 +1,4 @@
# Home Assistant MQTT contract tests # Home Assistant MQTT test harness tests
This harness runs retained MQTT discovery messages through Mosquitto and a real This harness runs retained MQTT discovery messages through Mosquitto and a real
Home Assistant container. It verifies the behavior that firmware unit tests Home Assistant container. It verifies the behavior that firmware unit tests
@@ -23,11 +23,11 @@ device-component tombstone/omission sequence, and makes the current
From the repository root: From the repository root:
```bash ```bash
HA_VERSION=2024.11.3 docker compose -f tests/ha-contract/compose.yaml up -d mqtt homeassistant HA_VERSION=2024.11.3 docker compose -f tests/ha-test-harness/compose.yaml up -d mqtt homeassistant
HA_VERSION=2024.11.3 docker compose -f tests/ha-contract/compose.yaml run --rm tests HA_VERSION=2024.11.3 docker compose -f tests/ha-test-harness/compose.yaml run --rm tests
HA_VERSION=2024.11.3 docker compose -f tests/ha-contract/compose.yaml restart homeassistant HA_VERSION=2024.11.3 docker compose -f tests/ha-test-harness/compose.yaml restart homeassistant
CONTRACT_MODE=retained-restart HA_VERSION=2024.11.3 docker compose -f tests/ha-contract/compose.yaml run --rm tests TEST_HARNESS_MODE=retained-restart HA_VERSION=2024.11.3 docker compose -f tests/ha-test-harness/compose.yaml run --rm tests
HA_VERSION=2024.11.3 docker compose -f tests/ha-contract/compose.yaml down -v HA_VERSION=2024.11.3 docker compose -f tests/ha-test-harness/compose.yaml down -v
``` ```
Use `HA_VERSION=stable` and `HA_VERSION=dev` for the current supported and Use `HA_VERSION=stable` and `HA_VERSION=dev` for the current supported and
@@ -38,7 +38,7 @@ as a compatibility audit. When it is on the repository's default branch, GitHub
exposes it for manual dispatch and the weekly schedule. It is not a tag-release exposes it for manual dispatch and the weekly schedule. It is not a tag-release
gate. gate.
For a local development-image test, add `CONTRACT_EXPECT_DISABLED_CLEANUP=1` to For a local development-image test, add `TEST_HARNESS_EXPECT_DISABLED_CLEANUP=1` to
both `docker compose run ... tests` commands. That lane checks that initially both `docker compose run ... tests` commands. That lane checks that initially
disabled device components are cleaned. The test uses only ephemeral named volumes; disabled device components are cleaned. The test uses only ephemeral named volumes;
`down -v` removes its broker data, Home Assistant config, owner token, and `down -v` removes its broker data, Home Assistant config, owner token, and
@@ -46,12 +46,12 @@ registry state.
## Shared support ## Shared support
The schema-neutral [HA/MQTT contract testkit](../../test-support/ha-mqtt-contract/README.md) owns only Docker-side Home Assistant onboarding, MQTT transport, registry/service access, retries, and artifact helpers. It has no ArduinoHA discovery assertions. This suite owns ArduinoHA migration fixtures; DeviceFramework carries its own fixtures and hardware adapter while reusing the testkit. The schema-neutral [HA/MQTT test harness testkit](../../test-support/ha-mqtt-test-harness/README.md) owns only Docker-side Home Assistant onboarding, MQTT transport, registry/service access, retries, and artifact helpers. It has no ArduinoHA discovery assertions. This suite owns ArduinoHA migration fixtures; DeviceFramework carries its own fixtures and hardware adapter while reusing the testkit.
## Scope ## Scope
The firmware's native Unity suite covers JSON escaping, invalid topic tokens, The firmware's native Unity suite covers JSON escaping, invalid topic tokens,
serializer preflight, migration ordering, component removal, and lifecycle serializer preflight, migration ordering, component removal, and lifecycle
behavior. This container suite covers Home Assistant's persistence contract. It behavior. This container suite covers Home Assistant's persistence test harness. It
does not need a physical board: fixture discovery documents mirror retained does not need a physical board: fixture discovery documents mirror retained
payloads emitted by ArduinoHA and isolate Home Assistant/MQTT compatibility. payloads emitted by ArduinoHA and isolate Home Assistant/MQTT compatibility.
@@ -24,15 +24,15 @@ services:
tests: tests:
build: build:
context: ../.. context: ../..
dockerfile: tests/ha-contract/Dockerfile dockerfile: tests/ha-test-harness/Dockerfile
environment: environment:
HA_URL: http://homeassistant:8123 HA_URL: http://homeassistant:8123
MQTT_HOST: mqtt MQTT_HOST: mqtt
MQTT_PORT: "1883" MQTT_PORT: "1883"
CONTRACT_MODE: ${CONTRACT_MODE:-migration} TEST_HARNESS_MODE: ${TEST_HARNESS_MODE:-migration}
CONTRACT_EXPECT_DISABLED_CLEANUP: ${CONTRACT_EXPECT_DISABLED_CLEANUP:-0} TEST_HARNESS_EXPECT_DISABLED_CLEANUP: ${TEST_HARNESS_EXPECT_DISABLED_CLEANUP:-0}
volumes: volumes:
- contract_state:/state - test_harness_state:/state
depends_on: depends_on:
mqtt: mqtt:
condition: service_healthy condition: service_healthy
@@ -41,4 +41,4 @@ services:
volumes: volumes:
ha_config: ha_config:
contract_state: test_harness_state:
@@ -1,4 +1,4 @@
"""Home Assistant MQTT discovery contract checks. """Home Assistant MQTT discovery test harness checks.
The test intentionally uses retained MQTT messages, like a deployed firmware The test intentionally uses retained MQTT messages, like a deployed firmware
node. It inspects HA's entity registry over the authenticated WebSocket API so node. It inspects HA's entity registry over the authenticated WebSocket API so
@@ -11,8 +11,8 @@ import os
import pathlib import pathlib
import time import time
from ha_mqtt_contract import ( from ha_mqtt_test_harness import (
ContractError, TestHarnessError,
HomeAssistantClient, HomeAssistantClient,
RetainedPublisher as SharedRetainedPublisher, RetainedPublisher as SharedRetainedPublisher,
wait_until, wait_until,
@@ -22,16 +22,16 @@ from ha_mqtt_contract import (
HA_URL = os.environ.get("HA_URL", "http://homeassistant:8123").rstrip("/") HA_URL = os.environ.get("HA_URL", "http://homeassistant:8123").rstrip("/")
MQTT_HOST = os.environ.get("MQTT_HOST", "mqtt") MQTT_HOST = os.environ.get("MQTT_HOST", "mqtt")
MQTT_PORT = int(os.environ.get("MQTT_PORT", "1883")) MQTT_PORT = int(os.environ.get("MQTT_PORT", "1883"))
MODE = os.environ.get("CONTRACT_MODE", "migration") MODE = os.environ.get("TEST_HARNESS_MODE", "migration")
STATE = pathlib.Path("/state") STATE = pathlib.Path("/state")
TOKEN_FILE = STATE / "ha-token" TOKEN_FILE = STATE / "ha-token"
DEVICE_ID = "contract_device" DEVICE_ID = "test_harness_device"
EDGE_DEVICE_ID = "contract_edge_device" EDGE_DEVICE_ID = "test_harness_edge_device"
EXPECT_DISABLED_CLEANUP = os.environ.get("CONTRACT_EXPECT_DISABLED_CLEANUP") == "1" EXPECT_DISABLED_CLEANUP = os.environ.get("TEST_HARNESS_EXPECT_DISABLED_CLEANUP") == "1"
def fail(message): def fail(message):
raise ContractError(message) raise TestHarnessError(message)
def legacy_topic(object_id, device_id=DEVICE_ID): def legacy_topic(object_id, device_id=DEVICE_ID):
@@ -47,7 +47,7 @@ def component(object_id, unique_id=None, device_id=DEVICE_ID, **extra):
"p": "sensor", "p": "sensor",
"name": object_id.replace("_", " ").title(), "name": object_id.replace("_", " ").title(),
"uniq_id": unique_id or f"{device_id}_{object_id}", "uniq_id": unique_id or f"{device_id}_{object_id}",
"stat_t": f"contract/{object_id}/state", "stat_t": f"test-harness/{object_id}/state",
} }
payload.update(extra) payload.update(extra)
return payload return payload
@@ -55,15 +55,15 @@ def component(object_id, unique_id=None, device_id=DEVICE_ID, **extra):
def device_payload(components, device_id=DEVICE_ID): def device_payload(components, device_id=DEVICE_ID):
return { return {
"dev": {"ids": [device_id], "name": "ArduinoHA contract device"}, "dev": {"ids": [device_id], "name": "ArduinoHA test harness device"},
"o": {"name": "ArduinoHA", "sw": "contract"}, "o": {"name": "ArduinoHA", "sw": "test harness"},
"cmps": components, "cmps": components,
} }
def legacy_payload(object_id, unique_id=None, device_id=DEVICE_ID, **extra): def legacy_payload(object_id, unique_id=None, device_id=DEVICE_ID, **extra):
payload = component(object_id, unique_id, **extra) payload = component(object_id, unique_id, **extra)
payload["dev"] = {"ids": [device_id], "name": "ArduinoHA contract device"} payload["dev"] = {"ids": [device_id], "name": "ArduinoHA test harness device"}
payload.pop("p") payload.pop("p")
return payload return payload
@@ -83,24 +83,24 @@ def wait_for_entry(ws, unique_id):
) )
def migration_contract(): def migration_test_harness():
publisher = SharedRetainedPublisher(MQTT_HOST, MQTT_PORT) publisher = SharedRetainedPublisher(MQTT_HOST, MQTT_PORT)
ws = HomeAssistantClient.bootstrap( ws = HomeAssistantClient.bootstrap(
HA_URL, HA_URL,
STATE, STATE,
owner_name="ArduinoHA Contract Owner", owner_name="ArduinoHA Test Harness Owner",
username="arduinoha-contract", username="arduinoha-test-harness",
password="arduinoha-contract-password", password="arduinoha-test-harness-password",
) )
ws.configure_mqtt(MQTT_HOST, MQTT_PORT) ws.configure_mqtt(MQTT_HOST, MQTT_PORT)
try: try:
# Existing single-component entity and a user-owned registry customization. # Existing single-component entity and a user-owned registry customization.
unique = f"{DEVICE_ID}_temperature" unique = f"{DEVICE_ID}_temperature"
publisher.publish(legacy_topic("temperature"), json.dumps(legacy_payload("temperature", unique))) publisher.publish(legacy_topic("temperature"), json.dumps(legacy_payload("temperature", unique)))
publisher.publish("contract/temperature/state", "21.5") publisher.publish("test-harness/temperature/state", "21.5")
original = wait_for_entry(ws, unique) original = wait_for_entry(ws, unique)
original_id = original["id"] original_id = original["id"]
renamed_entity_id = "sensor.contract_temperature_user_name" renamed_entity_id = "sensor.test_harness_temperature_user_name"
ws.call( ws.call(
"config/entity_registry/update", "config/entity_registry/update",
entity_id=original["entity_id"], entity_id=original["entity_id"],
@@ -149,7 +149,7 @@ def migration_contract():
# Direct publication is deliberately not a migration protocol. It must not # Direct publication is deliberately not a migration protocol. It must not
# create a second registry entry for the same stable unique ID. Keep it # create a second registry entry for the same stable unique ID. Keep it
# on a separate device topic so it cannot invalidate the primary # on a separate device topic so it cannot invalidate the primary
# retained-payload/restart contract. # retained-payload/restart test harness.
direct_unique = f"{EDGE_DEVICE_ID}_direct" direct_unique = f"{EDGE_DEVICE_ID}_direct"
publisher.publish( publisher.publish(
legacy_topic("direct", EDGE_DEVICE_ID), legacy_topic("direct", EDGE_DEVICE_ID),
@@ -233,11 +233,11 @@ def migration_contract():
if find_unique(ws.registry_entries(), f"{DEVICE_ID}_malformed"): if find_unique(ws.registry_entries(), f"{DEVICE_ID}_malformed"):
fail("malformed discovery payload created an entity") fail("malformed discovery payload created an entity")
# This fixture documents the current HA field contract: def_ent_id is # This fixture documents the current HA field behaviour: def_ent_id is
# allowed on first creation; obsolete obj_id is intentionally absent. # allowed on first creation; obsolete obj_id is intentionally absent.
default_payload = legacy_payload("default_name", def_ent_id="contract_default_name") default_payload = legacy_payload("default_name", def_ent_id="test_harness_default_name")
if "obj_id" in default_payload: if "obj_id" in default_payload:
fail("contract fixture accidentally contains obsolete obj_id") fail("test harness fixture accidentally contains obsolete obj_id")
publisher.publish(legacy_topic("default_name"), json.dumps(default_payload)) publisher.publish(legacy_topic("default_name"), json.dumps(default_payload))
wait_for_entry(ws, f"{DEVICE_ID}_default_name") wait_for_entry(ws, f"{DEVICE_ID}_default_name")
@@ -248,14 +248,14 @@ def migration_contract():
publisher.close() publisher.close()
def retained_restart_contract(): def retained_restart_test_harness():
if not (STATE / "migration-complete").exists(): if not (STATE / "migration-complete").exists():
fail("retained-restart mode requires the migration contract to run first") fail("retained-restart mode requires the migration test harness to run first")
publisher = SharedRetainedPublisher(MQTT_HOST, MQTT_PORT) publisher = SharedRetainedPublisher(MQTT_HOST, MQTT_PORT)
ws = HomeAssistantClient.from_state(HA_URL, STATE) ws = HomeAssistantClient.from_state(HA_URL, STATE)
try: try:
migrated = wait_for_entry(ws, f"{DEVICE_ID}_temperature") migrated = wait_for_entry(ws, f"{DEVICE_ID}_temperature")
if migrated["entity_id"] != "sensor.contract_temperature_user_name": if migrated["entity_id"] != "sensor.test_harness_temperature_user_name":
fail("HA restart lost the user-owned entity rename") fail("HA restart lost the user-owned entity rename")
retained = json.loads(publisher.retained_payload(device_topic())) retained = json.loads(publisher.retained_payload(device_topic()))
if "cmps" not in retained: if "cmps" not in retained:
@@ -268,12 +268,12 @@ def retained_restart_contract():
def main(): def main():
HomeAssistantClient.wait_until_ready(HA_URL) HomeAssistantClient.wait_until_ready(HA_URL)
if MODE == "migration": if MODE == "migration":
migration_contract() migration_test_harness()
elif MODE == "retained-restart": elif MODE == "retained-restart":
retained_restart_contract() retained_restart_test_harness()
else: else:
fail(f"unknown CONTRACT_MODE: {MODE}") fail(f"unknown TEST_HARNESS_MODE: {MODE}")
print(f"Home Assistant MQTT contract mode {MODE} passed") print(f"Home Assistant MQTT test harness mode {MODE} passed")
if __name__ == "__main__": if __name__ == "__main__":