feat: kit-busrouter v2.0 — complete Synology + GL fleet router platform

Includes: - package/ GL-XE3000 kit-busrouter (opkg) - scripts/provision.sh (GL) and provision-synology.sh (Synology) - syno-balance/ — new WAN balancer replacing aiwanbal (SmartWAN adapter) - kit-connect/ — unified connectivity SPK (Tailscale + reverse SSH) - docs/deployment/synology-rt2600ac-checklist.md — 62-point checklist - docs/provisioning/device-identity.md — fleet identity spec - docs/pilot/checklist.md — field pilot validation - x4078_20260721.dss — reference config backup Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 00:07:14 +00:00
commit f15ac69925
41 changed files with 5083 additions and 0 deletions
@@ -0,0 +1,238 @@
+# Synology RT2600ac — Fleet Deployment Checklist
+
+**Version:** 1.1 · **Last updated:** 2026-07-21 · **Audited against:** x4078 DSS config backup
+
+Every Synology RT2600ac bus router MUST pass ALL sections before departing the yard. Items flagged 🔴 were missed on x4078 and caused remote-access emergencies.
+
+---
+
+## 0. Pre-flight (before touching SRM)
+
+- [ ] **0.1** Insert SD card into router (required for Package Center and third-party packages)
+- [ ] **0.2** Power on router; connect LAN port to a PC for setup
+- [ ] **0.3** Verify default LAN IP `192.168.1.1` is reachable
+
+---
+
+## 1. SRM First-Time Setup Wizard
+
+- [ ] **1.1** 🔴 **Admin password:** `kitPLANE1!!` (user: `kitadmin`)
+- [ ] **1.2** 🔴 **Timezone:** `(GMT-06:00) Central Time (US & Canada)` — Chicago, CST/CDT
+  - NOT UTC, NOT Central America (Mexico City)
+- [ ] **1.3** 🔴 **WiFi Main SSID:** `<DEVICE_ID> Pioneer Admin` (e.g. `x4078 Pioneer Admin`)
+  - Security: WPA2-PSK · Password: `Pioneer321!`
+  - Smart Connect: **ON** (single SSID for both bands)
+  - Do NOT add `_2.4G` or `_5G` suffixes to the SSID
+- [ ] **1.4** 🔴 **WiFi Guest SSID:** `<DEVICE_ID> Pioneer` (e.g. `x4078 Pioneer`)
+  - Security: WPA2-PSK · Password: `Pioneer123`
+  - Client isolation: ON
+  - Smart Connect: ON
+  - Delete any default `SynologyGuest_*` networks
+- [ ] **1.5** **LAN subnet:** Keep default `192.168.1.0/24` (lbr0)
+- [ ] **1.6** **Guest subnet:** `192.168.2.0/24` (gbr0, SRM default)
+- [ ] **1.7** **Hostname:** Set to `<DEVICE_ID>` (e.g. `x4078`)
+- [ ] **1.8** **Router login port:** Keep default HTTP 8000 / HTTPS 8001
+
+---
+
+## 2. SRM Control Panel — System Settings
+
+- [ ] **2.1** 🔴 **QuickConnect:** Enable · ID = `<DEVICE_ID>` (e.g. `x4078`)
+  - Sign in with Synology Account (create `pioneercoach@keylinkit.net` if needed)
+  - Verify: `https://<DEVICE_ID>.us2.quickconnect.to/webman/index.cgi` loads
+- [ ] **2.2** 🔴 **DSM Auto-Update:** Control Panel → System → Update & Restore → Update Settings
+  - **Automatically check for updates:** ON
+  - **Automatically install important updates:** ON
+  - **Update schedule:** Daily, off-peak (e.g. 03:00)
+- [ ] **2.3** 🔴 **Database auto-update:** Security Advisor → Settings → Auto-update definitions
+- [ ] **2.4** 🔴 **Notifications:** Control Panel → Notification → Enable push/email notifications
+  - Enable: System health, WAN status changes, package events
+  - Email: configure SMTP or use SRM notification relay
+- [ ] **2.5** **SSH:** Control Panel → Terminal & SNMP → Enable SSH on port 2223
+- [ ] **2.6** **Regional Options:** Verify language, time format, and date format
+
+---
+
+## 3. SmartWAN Configuration
+
+- [ ] **3.1** Network Center → Smart WAN → **Load Balancing with Failover** mode
+- [ ] **3.2** 🔴 **WAN1 (eth0):** Primary — this will be Eyeride Eyenet
+- [ ] **3.3** 🔴 **WAN2 (eth2):** Secondary — this will be Starlink
+- [ ] **3.4** **Health check targets:** `1.1.1.2`, `9.9.9.9`, `8.8.8.8`
+  - Cloudflare malware-filtering (1.1.1.2) + Quad9 (9.9.9.9) + Google (8.8.8.8 fallback)
+- [ ] **3.5** **Policy routing:** All traffic balanced; no manual pinning yet
+
+---
+
+## 4. Package Center Setup
+
+- [ ] **4.1** Open Package Center → Accept terms of service when prompted
+- [ ] **4.2** Package Center → Settings → Trust Level → **"Any publisher"**
+- [ ] **4.3** Verify SD card is recognized (Package Center → Storage)
+
+---
+
+## 5. aiwanbal Package Install
+
+- [ ] **5.1** Download `aiwanbal-1.3-0013.spk` from Gitea:
+  ```
+  https://git.keylinkit.net/kitadmin/KIT-Smart-Load-balancer/releases/download/v1.3-0013/aiwanbal-1.3-0013.spk
+  ```
+- [ ] **5.2** SCP to router: `scp -O -P 2223 aiwanbal-*.spk kitadmin@192.168.1.1:/tmp/`
+- [ ] **5.3** Install via SSH: `echo "kitPLANE1!!" | sudo -S synopkg install /tmp/aiwanbal-*.spk`
+- [ ] **5.4** Start: `echo "kitPLANE1!!" | sudo -S synopkg start aiwanbal`
+- [ ] **5.5** 🔴 **Configure for mobile/bus environment:**
+  ```sh
+  # In /etc/aiwanbal/aiwanbal.conf, set:
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf ENVIRONMENT mobile
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf SPEEDTEST_AUTO_ENABLED 0
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf DEVICE_NAME x4078
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf CHECK_INTERVAL 30
+  ```
+- [ ] **5.6** 🔴 **Set Eyeride GPS config** (if Eyeride password is available):
+  ```sh
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf EYERIDE_ENABLE 1
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf EYERIDE_IP 192.168.10.1
+  echo "kitPLANE1!!" | sudo -S synosetkeyvalue /etc/aiwanbal/aiwanbal.conf EYERIDE_PASSWORD <eyeride-password>
+  ```
+- [ ] **5.7** 🔴 **Verify ISP detection:** Check the aiwanbal UI at `/webman/3rdparty/aiwanbal/index.html`
+  - WAN1 should show carrier/ISP (Eyeride → T-Mobile/Verizon depending on SIM)
+  - WAN2 should show Starlink
+  - If not detecting, run auto-detection: `sudo /var/packages/aiwanbal/target/bin/aiwanbal-detect.sh eth0; sudo /var/packages/aiwanbal/target/bin/aiwanbal-detect.sh eth2`
+
+---
+
+## 6. kittunnel Package Install (Reverse SSH Tunnel)
+
+- [ ] **6.1** Assign a unique port for this bus (see fleet port registry)
+- [ ] **6.2** Download hardened kittunnel SPK (v1.1+) from Gitea
+- [ ] **6.3** SCP to router and install via synopkg
+- [ ] **6.4** Verify tunnel.conf has correct `REMOTE_PORT`
+- [ ] **6.5** Start: `sudo synopkg start kittunnel`
+- [ ] **6.6** Verify tunnel is connected: check port is LISTENING on VPS (162.243.83.36)
+- [ ] **6.7** Verify end-to-end: `ssh -J root@172.18.0.1 -p <port> kitadmin@127.0.0.1 "hostname"`
+
+---
+
+## 7. Tailscale Install (Management Tailnet)
+
+- [ ] **7.1** Download Tailscale static `linux_arm` binary
+- [ ] **7.2** Install to `/usr/local/bin/tailscale` and `/usr/local/bin/tailscaled`
+- [ ] **7.3** Start tailscaled in userspace mode:
+  ```sh
+  /usr/bin/setsid /usr/local/bin/tailscaled \
+    --statedir=/var/packages/Tailscale/var/state \
+    --tun=userspace-networking \
+    --socket=/var/packages/Tailscale/var/run/tailscaled.sock &
+  ```
+- [ ] **7.4** 🔴 Join bus fleet tailnet:
+  ```sh
+  tailscale up --auth-key tskey-auth-... --hostname <DEVICE_ID> --accept-routes=false --accept-dns=false
+  ```
+- [ ] **7.5** 🔴 Install rc.local with **correct paths** (`/usr/bin/setsid`, `/bin/mkdir`):
+  ```sh
+  echo '#!/bin/sh' | sudo tee /usr/syno/etc/rc.d/rc.local
+  echo '/bin/mkdir -p /var/packages/Tailscale/var/state /var/packages/Tailscale/var/run' | sudo tee -a /usr/syno/etc/rc.d/rc.local
+  echo '/usr/bin/setsid /usr/local/bin/tailscaled --statedir=/var/packages/Tailscale/var/state --tun=userspace-networking --socket=/var/packages/Tailscale/var/run/tailscaled.sock &' | sudo tee -a /usr/syno/etc/rc.d/rc.local
+  sudo chmod +x /usr/syno/etc/rc.d/rc.local
+  ```
+- [ ] **7.6** 🔴 **Reboot test:** `sudo reboot` → wait 2 min → verify tailscaled is running
+- [ ] **7.7** Verify Tailscale status shows router online at `pioneercoach@`
+
+---
+
+## 8. Device Identity & Hostname (Persistent)
+
+- [ ] **8.1** 🔴 Create `/etc/busrouter/device-id` with `<DEVICE_ID>`:
+  ```sh
+  echo "<DEVICE_ID>" | sudo tee /etc/busrouter/device-id
+  ```
+- [ ] **8.2** 🔴 Create `/etc/busrouter/version` with `0.0`:
+  ```sh
+  echo "0.0" | sudo tee /etc/busrouter/version
+  ```
+- [ ] **8.3** 🔴 **Persistent hostname** — set in ALL locations:
+  ```sh
+  sudo hostname <DEVICE_ID>
+  echo "<DEVICE_ID>" | sudo tee /etc/hostname
+  ```
+  Also check SRM UI: Control Panel → Network → Router Name
+- [ ] **8.4** 🔴 **DHCP hostname:** SRM → Network Center → Internet → WAN → edit → Hostname = `<DEVICE_ID>`
+  (Without this, DHCP sends `SynologyRouter` to upstream)
+
+---
+
+## 9. Telemetry Agent
+
+- [ ] **9.1** SCP `telemetry-synology.sh` to `/usr/lib/busrouter/telemetry-synology.sh`
+- [ ] **9.2** Configure `TELEMETRY_HUB` to fleet hub endpoint:
+  ```sh
+  # In the script or via environment: http://167.172.237.162:8080/api/telemetry
+  ```
+- [ ] **9.3** 🔴 Add cron job (every 60s):
+  ```sh
+  echo '* * * * * root TELEMETRY_HUB=http://167.172.237.162:8080/api/telemetry /usr/lib/busrouter/telemetry-synology.sh 2>/dev/null' | sudo tee -a /etc/crontab
+  ```
+- [ ] **9.4** Dry-run test: `sudo /usr/lib/busrouter/telemetry-synology.sh`
+
+---
+
+## 10. Reboot & Final Verification
+
+- [ ] **10.1** Reboot: `sudo reboot`
+- [ ] **10.2** After reboot, verify ALL of these survive:
+
+| Check | Command | Expected |
+|-------|---------|----------|
+| Hostname | `hostname` | `<DEVICE_ID>` |
+| Device ID | `cat /etc/busrouter/device-id` | `<DEVICE_ID>` |
+| aiwanbal | `sudo synopkg status aiwanbal` | started |
+| kittunnel | `sudo synopkg status kittunnel` | started |
+| Tailscale | `ps \| grep tailscaled` | running |
+| Tailscale status | `tailscale status` | online, pioneercoach@ |
+| WiFi SSID | Check phone WiFi scan | `<DEVICE_ID> Pioneer Admin` |
+| Guest WiFi | Check phone WiFi scan | `<DEVICE_ID> Pioneer` |
+| QuickConnect | Browser → `https://<DEVICE_ID>.us2.quickconnect.to` | SRM login loads |
+| SSH tunnel | From lab: `ssh -J root@172.18.0.1 -p <port> kitadmin@127.0.0.1 "hostname"` | `<DEVICE_ID>` |
+| Tailscale path | From lab: `ssh -J root@172.18.0.1 -p 2223 kitadmin@<tailscale-ip> "hostname"` | `<DEVICE_ID>` |
+| WAN1 carrier | aiwanbal UI | Shows ISP |
+| WAN2 carrier | aiwanbal UI | Shows Starlink |
+| Telemetry cron | `grep telemetry /etc/crontab` | present |
+| Auto-update | SRM UI → Update & Restore | Auto-check ON |
+| Notifications | SRM UI → Notification | Enabled |
+
+- [ ] **10.3** 🔴 **Run the reboot test before the bus leaves the yard.** There is no second chance.
+- [ ] **10.4** Store credentials in vault under `busrouter/<DEVICE_ID>`
+- [ ] **10.5** Record device in fleet registry with IPs, port, keys
+
+---
+
+## Fleet Port Registry
+
+| Device | VPS Port | Tailscale IP | QuickConnect URL |
+|--------|----------|-------------|-------------------|
+| x4078 | 2226 | `100.116.71.43` | `https://x4078.us2.quickconnect.to` |
+| x5925 | 2230 | TBD | `https://x5925.us2.quickconnect.to` |
+| *(next)* | 2231 | TBD | TBD |
+| *(next)* | 2231 | TBD | TBD |
+
+---
+
+## x4078 Audit (2026-07-21) — Findings from DSS backup
+
+🔴 **Critical — must fix remotely or on next physical access:**
+
+| Gap | Severity | Fix |
+|-----|----------|-----|
+| SSID: `x4078 Admin` → should be `x4078 Pioneer Admin` | Medium | SRM Wi-Fi settings |
+| Guest SSID: `x4078 Pioneer Guest` → should be `x4078 Pioneer` | Medium | SRM Wi-Fi settings |
+| Band SSIDs have `_2.4G`/`_5G` suffixes | Low | Disable per-band SSIDs, use SmartConnect only |
+| Old `SynologyGuest_*` networks still present | Low | Delete in SRM Wi-Fi settings |
+| Timezone was Central America, not CST/CDT (Chicago) | High | Fixed via QuickConnect SRM UI |
+| Database auto-update OFF | High | Fixed via QuickConnect SRM UI |
+| Notifications not enabled | High | Fixed via QuickConnect SRM UI |
+| DHCP hostname sends `SynologyRouter` | Medium | SRM Network Center → WAN → Hostname |
+| Hostname persistence failing on reboot | High | Verify `/etc/hostname` + SRM Network Center |
+| ISP/carrier not detecting per WAN | Medium | Run `aiwanbal-detect.sh` on each WAN |
+| rc.local has wrong `/bin/setsid` path → `/usr/bin/setsid` | High | Fixed already |
+| Tailscale not boot-tested | Critical | Must test on next physical access |