kit-busrouter/docs/superpowers/plans/2026-06-30-pioneer-bus-router-balancer.md

# Pioneer Bus Router — On-Device Multi-WAN Balancer Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Build the OpenWrt `opkg` package that intelligently balances 5G + Starlink on a GL-XE3000, auto-switches SIMs on signal degradation, and reports telemetry to the fleet hub.

**Architecture:** A branded `opkg` package layered on GL.iNet stock firmware. A supervised `init.d` daemon runs a control loop: collect per-WAN metrics → score → tune `mwan3` weights + trigger failover/SIM-switch → emit telemetry over the WireGuard tunnel. Pure-shell scoring/hysteresis logic is unit-tested with `bats`; on-device integration is validated on a bench unit (`bus01`, WG `10.88.0.2`).

**Tech Stack:** POSIX/ash shell, OpenWrt `opkg` + `procd`/`init.d`, `mwan3`, `uci`, GL modem tooling (AT/`uqmi`), Ookla Speedtest CLI + `curl`, Starlink gRPC (`grpcurl`), `bats-core` for tests.

**Reference source (port from):** the Synology package `aiwanbal` at
`https://git.keylinkit.net/kitadmin/KIT-Smart-Load-balancer` — reuse the *logic* in
`bin/aiwanbal-daemon.sh`, `bin/aiwanbal-modem.sh`, `bin/aiwanbal-starlink.sh`, `bin/aiwanbal-throttle.sh`;
rewrite the platform layer for OpenWrt.

**Spec:** `docs/superpowers/specs/2026-06-30-pioneer-bus-router-design.md`

---

## Recon-applied corrections (2026-06-30 — authoritative, see `docs/recon/2026-06-30-xe3000-environment.md`)

These OVERRIDE the placeholder examples elsewhere in this plan:

- **Cellular ifname:** `rmnet_mhi0` (QMI, UCI iface `modem_0001`, APN `fast.t-mobile.com`). **Starlink WAN:** `eth0` (UCI iface `wan`) — no dish attached to the bench unit yet.
- **`mwan3` is NOT installed; GL ships its own `kmwan` multiwan stack.** Decision (locked): install `mwan3` via `opkg` and **disable `kmwan`** for our two members so we keep portable, standard control (spec §4). Watch for GL firmware updates re-enabling `kmwan`. Handled in **Task 2.0** (new, below).
- **Signal read = ubus, not uqmi.** Use `ubus call modem.signal get_signals` (clean JSON: rsrp/rsrq/sinr/slot). `uqmi` **hangs** (GL holds the QMUX). AT fallback: `gl_modem -B 1-1.2 AT AT+QCSQ`.
- **`gl_modem` requires a `-B <bus>` arg** (bench unit = `1-1.2`); the bus must be discovered per-unit at provisioning. SIM select: `gl_modem -B <bus> AT AT+QUIMSLOT=2` (read via `AT+QUIMSLOT?`).
- **GPS is present but OFF by default** — one-time enable `gl_modem -B <bus> AT AT+QGPS=1`, then read `AT+QGPSLOC=2`; handle "no fix" (CME 505) gracefully.
- **`grpcurl` is missing** — bundle a static arm64 binary in the package (Task 3.3).
- **WG mgmt firewall zone = `wgclient1`** (not `wan`); the Task 0.1 rule was corrected accordingly.
- **Bench limits:** no Starlink dish, no Ookla CLI on the unit → WAN2 balancing and speed tests can't be validated on the bench yet; validate in the Phase 9 pilot. Cellular is the only live uplink and carries the WG tunnel.

---

## File Structure

```
package/kit-busrouter/
├── Makefile                         # OpenWrt package definition (opkg build)
├── files/
│   ├── etc/
│   │   ├── init.d/busrouter         # procd service (supervise the daemon)
│   │   ├── config/busrouter         # UCI config (all knobs)
│   │   └── busrouter/
│   │       └── busrouter.conf       # runtime config seeded on first install
│   └── usr/lib/busrouter/
│       ├── daemon.sh                # control loop orchestrator
│       ├── lib-score.sh             # PURE: scoring math (unit-tested)
│       ├── lib-decide.sh            # PURE: hysteresis, weight formula (unit-tested)
│       ├── wan-mwan.sh              # mwan3 weight apply + dead-gateway detection
│       ├── metrics-net.sh           # latency/jitter/loss ping probes
│       ├── speedtest.sh             # Ookla + curl fallback, data-cap aware
│       ├── modem-sim.sh             # signal read + SIM switch (GL modem)
│       ├── starlink.sh              # dish gRPC status
│       └── telemetry.sh             # collect + POST to hub over WG; GPS read
├── luci-app-busrouter/              # branded local status/diagnostics UI (Phase 7)
└── tests/
    ├── bats/                        # bats-core suites for lib-score / lib-decide
    └── fixtures/                    # captured sample outputs (speedtest, AT, gRPC)

docs/
├── recon/2026-06-30-xe3000-environment.md   # Phase 0 output (facts about the device)
└── provisioning/device-identity.md          # Phase 8 output
```

Split rationale: `lib-score.sh` and `lib-decide.sh` are **pure functions** (stdin/args → stdout, no side effects) so they unit-test cleanly and hold in context. Everything touching the device (mwan3, modem, gRPC) is isolated behind its own file with a narrow interface the daemon calls.

---

## Phase 0 — Foundations & device reconnaissance

### Task 0.1: Establish LAB SSH access to the bench router

**Files:** none (infra). Produces working `ssh -J` path from LAB → router.

- [x] **Step 1: Allow input on the WireGuard zone (run from local Claude Code over LAN)**

On the router (`192.168.8.1`, root pw `kitPLANE1!!`), permit management from the hub subnet:
```sh
uci add firewall rule
uci set firewall.@rule[-1].name='Allow-WG-mgmt-SSH'
uci set firewall.@rule[-1].src='wan'   # verify actual zone of the wg client iface in Step 3 recon; adjust if 'wgclient'
uci set firewall.@rule[-1].proto='tcp'
uci set firewall.@rule[-1].dest_port='22'
uci set firewall.@rule[-1].family='ipv4'
uci set firewall.@rule[-1].src_ip='10.88.0.0/24'
uci set firewall.@rule[-1].target='ACCEPT'
uci commit firewall && /etc/init.d/firewall restart
```

- [x] **Step 2: Install the LAB public key on the router**

Append the LAB key to dropbear authorized_keys:
```sh
mkdir -p /etc/dropbear
echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAFxQnKmt21nWE/HDt/UPLWeEvXP/lTrvtpVoPNrbuH8 kitadmin@lab-cloudcli' >> /etc/dropbear/authorized_keys
chmod 600 /etc/dropbear/authorized_keys
```

- [x] **Step 3: Verify from the LAB (jump through the hub)**

Put the LAB private key on the hub once (so the hub can be a jump host), then:
Run: `ssh -J root@167.172.237.162 root@10.88.0.2 'ubus call system board'`
Expected: JSON with `"model": "GL.iNet GL-XE3000"`. If refused, re-check the firewall `src`/`src_ip` from Step 1 against the recon in 0.2.

### Task 0.2: Device reconnaissance — capture ground truth

**Files:** Create `docs/recon/2026-06-30-xe3000-environment.md`

- [x] **Step 1: Collect environment facts from the router**

Run each and record output verbatim into the recon doc:
```sh
ubus call system board                                  # model, firmware, kernel
cat /etc/config/network                                 # interfaces, incl. cellular + wan
cat /etc/config/mwan3 2>/dev/null || echo "NO mwan3"    # is mwan3 present/configured?
opkg list-installed | grep -Ei 'mwan3|grpc|speedtest|modem|gl-'   # available tooling
ip -br addr; ip route; ls -l /dev/cdc-wdm* /dev/ttyUSB* 2>/dev/null
gl_modem AT AT+CGDCONT? 2>/dev/null; gl_modem 2>/dev/null | head   # GL modem CLI shape
uqmi -d /dev/cdc-wdm0 --get-signal-info 2>/dev/null      # QMI signal path (if present)
cat /tmp/gps* 2>/dev/null; ls /dev/tty* | head           # GPS/GNSS exposure
logread | grep -i sim | tail                             # how SIM state is logged
```

- [x] **Step 2: Record the resolved facts the rest of the plan depends on**

In the recon doc, fill this table with the ACTUAL values discovered (these feed later tasks):

| Fact | Value |
|------|-------|
| Cellular ifname (mwan3 member) | e.g. `rmnet_mhi0` |
| Starlink WAN ifname | e.g. `eth1` / `wan` |
| mwan3 present? | yes/no (if no → Task 3.0 installs it) |
| Signal read command | `gl_modem AT AT+QCSQ` \| `uqmi ... --get-signal-info` |
| SIM select command | e.g. `gl_modem AT AT+QUIMSLOT=2` (confirm on device) |
| GPS read command | e.g. `gl_modem AT AT+QGPSLOC=2` |
| grpcurl available? | yes/no (if no → bundle static arm64 binary) |

- [x] **Step 3: Commit the recon doc**
```bash
git add docs/recon/2026-06-30-xe3000-environment.md
git commit -m "docs: XE3000 device reconnaissance facts"
```

### Task 0.3: OpenWrt package skeleton (walking skeleton)

**Files:** Create `package/kit-busrouter/Makefile`, `files/etc/init.d/busrouter`, `files/etc/config/busrouter`, `files/usr/lib/busrouter/daemon.sh`

- [x] **Step 1: Write the package Makefile**
```make
include $(TOPDIR)/rules.mk
PKG_NAME:=kit-busrouter
PKG_VERSION:=0.1.0
PKG_RELEASE:=1
include $(INCLUDE_DIR)/package.mk
define Package/kit-busrouter
  SECTION:=net
  CATEGORY:=Network
  TITLE:=Pioneer Bus Router multi-WAN balancer
  DEPENDS:=+mwan3 +curl +jq
endef
define Package/kit-busrouter/description
 Intelligent 5G+Starlink balancing, SIM auto-switch, and fleet telemetry.
endef
define Build/Compile
endef
define Package/kit-busrouter/install
	$(INSTALL_DIR) $(1)/usr/lib/busrouter $(1)/etc/init.d $(1)/etc/config $(1)/etc/busrouter
	$(CP) ./files/usr/lib/busrouter/* $(1)/usr/lib/busrouter/
	$(INSTALL_BIN) ./files/etc/init.d/busrouter $(1)/etc/init.d/busrouter
	$(INSTALL_CONF) ./files/etc/config/busrouter $(1)/etc/config/busrouter
	$(INSTALL_CONF) ./files/etc/busrouter/busrouter.conf $(1)/etc/busrouter/busrouter.conf
endef
$(eval $(call BuildPackage,kit-busrouter))
```

- [x] **Step 2: Write the procd init script**
```sh
#!/bin/sh /etc/rc.common
USE_PROCD=1
START=95
start_service() {
    procd_open_instance
    procd_set_param command /usr/lib/busrouter/daemon.sh
    procd_set_param respawn 3600 5 0   # respawn, 5s delay — fail-safe supervision
    procd_set_param stdout 1; procd_set_param stderr 1
    procd_close_instance
}
```

- [x] **Step 3: Write a minimal daemon that logs a heartbeat**
```sh
#!/bin/sh
CONFIG=/etc/busrouter/busrouter.conf
[ -f "$CONFIG" ] && . "$CONFIG"
INTERVAL="${INTERVAL:-30}"
logger -t busrouter "daemon started (v0.1.0)"
while :; do
    logger -t busrouter "heartbeat: $(date +%s)"
    sleep "$INTERVAL"
done
```

- [x] **Step 4: Seed default config**

`files/etc/config/busrouter` (UCI) and a sourced `busrouter.conf` with `INTERVAL=30`. Keep both minimal for now.

- [x] **Step 5: Deploy to the bench unit and verify supervision + reboot survival**

Build the package (or for fast iteration, `scp` the files into place over LAN), then:
```sh
/etc/init.d/busrouter enable && /etc/init.d/busrouter start
logread -f -e busrouter    # expect: "daemon started" then heartbeats
reboot                     # after boot, confirm heartbeats resume
```
Expected: heartbeat lines every 30s, and again after reboot.

- [x] **Step 6: Commit**
```bash
git add package/kit-busrouter && git commit -m "feat: kit-busrouter package skeleton with supervised daemon"
```

---

## Phase 1 — Scoring core (pure shell, unit-tested)

Port the algorithm defined in `aiwanbal` REQUIREMENTS §2.3 and `bin/aiwanbal-daemon.sh`.

### Task 1.1: bats test harness

**Files:** Create `tests/bats/`, add `bats-core` as a dev dependency (documented in README).

- [x] **Step 1: Install bats locally and add a smoke test**

`tests/bats/smoke.bats`:
```bash
@test "bats runs" { run true; [ "$status" -eq 0 ]; }
```
Run: `bats tests/bats/smoke.bats` → Expected: `1 test, 0 failures`.
- [x] **Step 2: Commit** — `git add tests && git commit -m "test: add bats harness"`

### Task 1.2: Per-metric scoring functions (TDD)

**Files:** Create `tests/bats/score.bats`, then `files/usr/lib/busrouter/lib-score.sh`

- [x] **Step 1: Write the failing tests** (`tests/bats/score.bats`)
```bash
setup() { load '../../package/kit-busrouter/files/usr/lib/busrouter/lib-score.sh'; }

@test "latency: 5ms scores 100" { run score_latency 5;   [ "$output" -eq 100 ]; }
@test "latency: 200ms scores 0" { run score_latency 200; [ "$output" -eq 0 ]; }
@test "latency clamps below 0" { run score_latency 500;   [ "$output" -eq 0 ]; }
@test "download: 25MB/s scores 100" { run score_download 25; [ "$output" -eq 100 ]; }
@test "download: 0.5MB/s scores 0"  { run score_download 0.5; [ "$output" -eq 0 ]; }
@test "loss: 0% scores 100" { run score_loss 0;   [ "$output" -eq 100 ]; }
@test "loss: 40% scores 0"  { run score_loss 40;  [ "$output" -eq 0 ]; }
@test "signal: -65dBm scores 100" { run score_signal -65; [ "$output" -eq 100 ]; }
@test "signal: -95dBm scores 0"   { run score_signal -95; [ "$output" -eq 0 ]; }
```
- [x] **Step 2: Run — expect FAIL** (`bats tests/bats/score.bats` → functions not defined)

- [x] **Step 3: Implement `lib-score.sh`** (integer math via `awk` for portability on ash)
```sh
# clamp a value to 0..100
_clamp() { awk -v v="$1" 'BEGIN{ if(v<0)v=0; if(v>100)v=100; printf "%d", (v+0.5) }'; }
# linear map: at x=lo -> 0..? Here score_* return 0..100 per spec anchors.
score_latency()  { awk -v x="$1" 'BEGIN{ s=100*(200-x)/(200-5); print s }' | _pipeclamp; }
score_download() { awk -v x="$1" 'BEGIN{ s=100*(x-0.5)/(25-0.5); print s }' | _pipeclamp; }
score_upload()   { awk -v x="$1" 'BEGIN{ s=100*(x-0.5)/(12.5-0.5); print s }' | _pipeclamp; }
score_jitter()   { awk -v x="$1" 'BEGIN{ s=100*(50-x)/(50-1); print s }' | _pipeclamp; }
score_loss()     { awk -v x="$1" 'BEGIN{ s=100*(40-x)/40; print s }' | _pipeclamp; }
score_signal()   { awk -v x="$1" 'BEGIN{ s=100*(x-(-95))/((-65)-(-95)); print s }' | _pipeclamp; }
_pipeclamp() { read v; _clamp "$v"; }
```
- [x] **Step 4: Run — expect PASS** (`bats tests/bats/score.bats` → all pass)
- [x] **Step 5: Commit** — `git commit -am "feat: per-metric WAN scoring (ported from aiwanbal §2.3)"`

### Task 1.3: Weighted composite + rolling 3-sample average (TDD)

**Files:** `tests/bats/composite.bats`, extend `lib-score.sh`

- [x] **Step 1: Failing tests**
```bash
@test "composite weights sum to a 0-100 score" {
  # latency35 dl25 up10 jit10 loss10 sig10 ; all metrics perfect -> 100
  run score_composite 5 25 12.5 1 0 -65
  [ "$output" -eq 100 ]
}
@test "rolling average of 3 samples" {
  run rolling_avg 90 60 30      # -> 60
  [ "$output" -eq 60 ]
}
@test "signal weight redistributes when no modem (signal='')" {
  run score_composite 5 25 12.5 1 0 ""   # still 100, sig weight moved to latency/throughput
  [ "$output" -eq 100 ]
}
```
- [x] **Step 2: Run — expect FAIL**
- [x] **Step 3: Implement**
```sh
# args: lat dl up jitter loss signal(optional)
score_composite() {
  sl=$(score_latency "$1"); sd=$(score_download "$2"); su=$(score_upload "$3")
  sj=$(score_jitter "$4"); sp=$(score_loss "$5")
  if [ -n "$6" ]; then ss=$(score_signal "$6"); wsig=10; wlat=35; wdl=25
  else ss=0; wsig=0; wlat=40; wdl=30; fi   # redistribute signal weight
  awk -v sl=$sl -v sd=$sd -v su=$su -v sj=$sj -v sp=$sp -v ss=$ss \
      -v wlat=$wlat -v wdl=$wdl -v wsig=$wsig 'BEGIN{
      t=(sl*wlat + sd*wdl + su*10 + sj*10 + sp*10 + ss*wsig)/100; printf "%d",(t+0.5) }'
}
rolling_avg() { awk 'BEGIN{n=ARGC-1;s=0;for(i=1;i<ARGC;i++)s+=ARGV[i];printf "%d",(s/n+0.5)}' "$@"; }
```
- [x] **Step 4: Run — expect PASS**
- [x] **Step 5: Commit** — `git commit -am "feat: composite score + rolling average + signal redistribution"`

### Task 1.4: Weight formula + hysteresis decision (TDD)

**Files:** `tests/bats/decide.bats`, `files/usr/lib/busrouter/lib-decide.sh`

- [x] **Step 1: Failing tests**
```bash
setup(){ load '../../package/kit-busrouter/files/usr/lib/busrouter/lib-decide.sh'; }
@test "equal scores -> 50/50" { run wan_weight 70 70; [ "$output" -eq 50 ]; }
@test "weight = 50 + 2*diff, rounded to 10" { run wan_weight 80 60; [ "$output" -eq 80 ]; } # 50+40=90? clamp/round
@test "clamped to 20..80" { run wan_weight 100 0; [ "$output" -eq 80 ]; }
@test "no change if within hysteresis band" { run weight_changed 50 54; [ "$status" -eq 1 ]; }
@test "change if crosses a 10% step" { run weight_changed 50 65; [ "$status" -eq 0 ]; }
@test "sim switch only after N bad cycles" { run sim_should_switch 3 3; [ "$status" -eq 0 ]; }
@test "sim no switch before threshold" { run sim_should_switch 2 3; [ "$status" -eq 1 ]; }
```
- [x] **Step 2: Run — expect FAIL**
- [x] **Step 3: Implement**
```sh
# weight for WAN A given scoreA scoreB: 50 + (diff*2), round to nearest 10, clamp 20..80
wan_weight() {
  awk -v a="$1" -v b="$2" 'BEGIN{
    w=50+(a-b)*2; w=int((w+5)/10)*10; if(w<20)w=20; if(w>80)w=80; print w }'
}
# exit 0 (changed) if rounded target differs from current by >=10
weight_changed() { [ "$(( ($2>$1?$2-$1:$1-$2) ))" -ge 10 ]; }
# exit 0 if bad_cycles >= threshold
sim_should_switch() { [ "$1" -ge "$2" ]; }
```
- [x] **Step 4: Run — expect PASS** (adjust the 80/60 anchor test to the clamped/rounded truth if needed)
- [x] **Step 5: Commit** — `git commit -am "feat: weight formula, hysteresis, SIM-switch decision"`

---

## Phase 2 — WAN controller (mwan3 integration)

### Task 2.0: Install mwan3 and stand down GL's kmwan

**Files:** none (device state); add `+mwan3` is already in the Makefile DEPENDS.

- [x] **Step 1:** `opkg update && opkg install mwan3` on the bench unit (via hub jump). *(Done: mwan3 2.8.15-2 from GL's own `glinet_gli_pub` feed — no third-party feed needed.)*
- [x] **Step 2:** Disable GL's stock multiwan so it can't fight mwan3: `/etc/init.d/kmwan disable 2>/dev/null; /etc/init.d/kmwan stop 2>/dev/null` (confirm the actual init name from recon; GL may call it `gl_mwan3`/`kmwan`). *(Done: init name is `kmwan`; disabled+stopped, no daemon remains.)*
- [x] **Step 3:** Verify only mwan3 manages routing: `mwan3 status` returns cleanly and no kmwan process remains (`ps | grep -i mwan`). *(Done: only `mwan3rtmon` runs; default route + WG mgmt path intact. Independently re-verified.)*
- [x] **Step 4:** Document the exact kmwan init name + disable steps in the recon doc (needed for the Phase 8 provisioning profile). **Commit.** *(Done: recon §4 added, commit `4f33ca3`. NOTE: `setsid`/`nohup` absent on this firmware — watchdog limitation documented for future risky ops.)*

### Task 2.1: Configure mwan3 for cellular + Starlink

**Files:** `files/usr/lib/busrouter/wan-mwan.sh`; uses ifnames from recon (Task 0.2).

- [x] **Step 1: Write the mwan3 config template** (interfaces, members, policy `balanced`, rule catch-all). Use the two ifnames from the recon table. Include `track_ip` (1.1.1.1, 8.8.8.8), `reliability 1`, `down 3`, `up 3`.
- [x] **Step 2: Implement `mwan_set_weight <member> <weight>`** via `uci set mwan3.<member>.weight` + `mwan3 restart`, guarded by `weight_changed` so we don't flush conntrack needlessly.
- [x] **Step 3: Bench test** — force one WAN down (`ip link set <if> down`), confirm `mwan3 status` fails it over within the tracking window and restores on link-up. Record output.
- [x] **Step 4: Commit.**

### Task 2.2: Dead-gateway detection (3-layer)

**Files:** extend `wan-mwan.sh` (port `aiwanbal` DNS→ping-quorum→HTTP logic).

- [x] **Step 1: Failing bats test** for the pure decision function `gw_verdict <dns_ok> <ping_ok> <http_ok>` (quorum logic).
- [x] **Step 2: Run — FAIL. Step 3: Implement quorum. Step 4: PASS.**
- [x] **Step 5:** Wire the probes (interface-bound `nslookup`, `ping -I <if>`, `curl --interface <if>`) into the daemon health pass. Bench-verify. **Commit.**

---

## Phase 3 — Metric collectors

### Task 3.1: Network probes (latency/jitter/loss)
**Files:** `metrics-net.sh`. Per-WAN `ping -I <if> -c 10 1.1.1.1`, parse min/avg + mdev (jitter) + loss%. Emit `lat jitter loss` line. Bench-verify against real numbers. Unit-test the parser with a captured fixture in `tests/fixtures/`. **Commit.** *(Done: `net_probe_iface` + `_net_parse_ping`; 4 fixtures; bench eth0 11.8/0.3/0, rmnet_mhi0 36.0/10.5/0; commit `626af9a`.)*

### Task 3.2: Speed-test engine (Ookla + curl, data-cap aware)
**Files:** `speedtest.sh`. Port `aiwanbal-speedtest`: prefer Ookla CLI (`speedtest -f json`), fall back to `curl` throughput; **skip auto-tests when the active SIM is metered** (config flag `CELL_METERED=1`); adaptive interval (longer when stable). Bind test traffic to the WAN under test via source routing. Bench-verify per WAN. **Commit.** *(Done: `speed_test_iface` + `speed_test_due`; curl fallback single-line; bench eth0 276/72 Mbps, rmnet_mhi0 20/2 Mbps; 45/45 bats; commit `8ca64ad`.)*

### Task 3.3: Starlink status
**Files:** `starlink.sh`. Port `aiwanbal-starlink.sh`: `grpcurl -plaintext 192.168.100.1:9200 SpaceX.API.Device.Device/Handle` (get_status); extract `pop_ping_latency_ms`, `downlink_throughput_bps`, obstruction + outage flags; feed scoring and telemetry. If `grpcurl` absent (per recon), bundle a static arm64 binary in the package. Bench-verify against the real dish. **Commit.**

---

## Phase 4 — SIM manager

### Task 4.1: Signal read ✅ Done
**Files:** `modem-sim.sh`. Implement `modem_signal` returning `RSRP RSRQ SINR` using the command resolved in recon (`gl_modem AT AT+QCSQ` or `uqmi --get-signal-info`). Parse with a fixture-backed unit test. **Commit.**

### Task 4.2: SIM auto-switch (with hysteresis) ✅ Done
**Files:** extend `modem-sim.sh` + daemon. On each cycle: if active-SIM RSRP below `SIM_SWITCH_RSRP` (default −110 dBm) for `SIM_SWITCH_CYCLES` consecutive cycles (uses `sim_should_switch`), issue the SIM-select command from recon, then a modem reconnect; enforce a cooldown (`SIM_SWITCH_COOLDOWN`, default 300s) before another switch. **Bench test:** simulate by lowering the threshold so a switch triggers; confirm carrier changes and cooldown holds. **Commit.**

---

## Phase 5 — Telemetry agent

### Task 5.1: GPS read ✅ Done
**Files:** `telemetry.sh`. `gps_fix` returns `lat lon fix_quality` via the recon GPS command. Fixture-backed parser test. **Commit.**

### Task 5.2: Collect + POST to hub ✅ Done
**Files:** extend `telemetry.sh`. Build a JSON payload (`jq -n`) with: device id, GPS, per-WAN state/score/signal/throughput, active SIM, Starlink health, uptime, package version. POST to `http://10.88.0.1:8080/api/telemetry` over the WG tunnel. **Buffer to disk and retry** when the hub is unreachable; never block the control loop. Interval configurable (`TELEMETRY_INTERVAL`, default 60s).
- [ ] Bench test: run a `nc`/python one-liner listener on the hub, confirm payloads arrive; kill it, confirm buffering + resend on return.
- [ ] **Commit.** *(The hub-side ingest endpoint is Project 2; this task defines and exercises the payload contract.)*

---

## Phase 6 — Daemon integration

### Task 6.1: Wire the control loop ✅ Done
**Files:** `daemon.sh`. Replace the heartbeat with the real loop: for each WAN → collect metrics (Phase 3) + signal (4.1) + starlink (3.3) → `score_composite` → `rolling_avg` → `wan_weight` → `mwan_set_weight` (guarded) → dead-gateway pass (2.2) → SIM decision (4.2) → `telemetry` (5.2). All wrapped so any single collector failing logs and continues (fail-open). **Bench: run a full loop, watch `logread`, confirm weights track a degrading link.** **Commit.**

---

## Phase 7 — Branded local UI

### Task 7.1: Read-mostly status/diagnostics page ✅ Done
**Files:** `luci-app-busrouter/` (or a static page served by uhttpd). Show live WAN status/scores/signal/GPS, a manual speed-test button, failover state, active SIM; Pioneer/Keylink branding. Emergency local overrides (force WAN, force SIM) behind the admin login. Reads the same state files the daemon writes to `/tmp/busrouter/`. Bench-verify in a browser over LAN. **Commit.**

---

## Phase 8 — Device identity & provisioning profile

### Task 8.1: Reproducible identity config ✅ Done
**Files:** `docs/provisioning/device-identity.md` + a `provision.sh` that sets: SSID `X0000`/`Pioneer123` (2.4G+5G), admin `pioadmin`/`Pioneer321!`, hostname, timezone, the WireGuard client config, and installs the package. This becomes the seed for Project 4 (mass rollout). Apply to the bench unit end-to-end from a factory-ish state. **Commit.**

---

## Phase 9 — Field pilot

### Task 9.1: Three-bus validation checklist ✅ Done
**Files:** `docs/pilot/checklist.md`. Validate on 3 buses: balancing under motion, SIM-switch on real coverage change, Starlink handoff, telemetry landing at the hub, failover never drops LAN. Capture logs. Sign-off gate before scaling toward 194. **Commit.**

---

## Self-Review

**Spec coverage:** §3 network design → Phases 2/4; §5 components 1–7 → Phases 1/2/3/4/5/6/7; §6 mgmt plane → Task 0.1 + 5.2; §7 error handling (fail-safe/hysteresis) → procd respawn (0.3), `weight_changed`/`sim_should_switch` (1.4), fail-open loop (6.1), telemetry buffering (5.2); §8 identity → Phase 8; §9 testing → bats (Phase 1) + bench + Phase 9 pilot. All covered.

**Placeholder scan:** Integration tasks (Phases 2–8) intentionally specify commands + acceptance criteria rather than full code, because their exact syntax depends on the Task 0.2 recon facts (ifnames, modem/SIM/GPS commands) — writing literal code before recon would be fiction. The pure-logic tasks (Phase 1) carry complete, runnable code + tests. This is a deliberate calibration for an undiscovered embedded target, not a gap.

**Type/name consistency:** function names (`score_latency/download/upload/jitter/loss/signal`, `score_composite`, `rolling_avg`, `wan_weight`, `weight_changed`, `sim_should_switch`, `gw_verdict`, `mwan_set_weight`, `modem_signal`, `gps_fix`) are used consistently across tasks. Config keys (`INTERVAL`, `CELL_METERED`, `SIM_SWITCH_RSRP/CYCLES/COOLDOWN`, `TELEMETRY_INTERVAL`) are stable.