fix: x4078 remote recovery + daemon wait-wedge bug + SPK rebuild

- docs/incident-log: mark x4078 recovered remotely via Tailscale (no
  physical access needed); document the recovery steps and root cause
- connect-daemon.sh: remove bare `wait` that wedged the retry loop
  forever on the setsid'd tailscaled child — once the tunnel failed the
  daemon could not self-heal until the package was restarted; now the
  main loop falls through start_tunnel (already blocking+retrying) and
  restarts the whole cycle cleanly
- SPK rebuilt (32839680 bytes) with the daemon fix included
- Also documents the two latent lessons: (1) a field router is only
  inaccessible when ALL three paths fail; (2) always check Tailscale +
  QuickConnect before assuming a truck roll is required

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-07-22 22:23:53 +00:00
parent 24d8fbd366
commit 879f97c843
3 changed files with 15 additions and 3 deletions
+13 -1
View File
@@ -302,7 +302,19 @@ Kit-connect is the unified replacement for Sections 6 and 7. One SPK installs bo
**Resolution for x4662**: Kittunnel restarted via Package Center through QuickConnect. Tunnel stable on port 2229. **Resolution for x4662**: Kittunnel restarted via Package Center through QuickConnect. Tunnel stable on port 2229.
**Resolution for x4324**: Kit-connect daemon reconnected after authorized_keys fix on VPS. Tunnel stable on port 2225. **Resolution for x4324**: Kit-connect daemon reconnected after authorized_keys fix on VPS. Tunnel stable on port 2225.
**Pending**: x4078 needs physical access — kit-connect daemon is installed and connecting but fails because it forwards to port 22 instead of 2223. **Resolution for x4078 (2026-07-22, remote — NO physical access needed)**: Recovered entirely over the
still-live **Tailscale** path (`100.116.71.43`, SRM SSH `:2223`) — QuickConnect was also up the whole
time. The reverse tunnel was the *only* dead path. Root cause on-device: `TUNNEL_LOCAL_SSH_PORT=22` in
`/etc/kit-connect/connect.conf` (SRM sshd listens on 2223). Fix: SSH in over Tailscale from a tailnet
node (e.g. the hub), `sed -i 's/^TUNNEL_LOCAL_SSH_PORT=.*/TUNNEL_LOCAL_SSH_PORT=2223/'` the conf, then
`synopkg stop kit-connect; synopkg start kit-connect`. Tunnel came back on port 2226; end-to-end verified.
Lesson: **a field router is only "inaccessible" if ALL of {Tailscale, QuickConnect, reverse tunnel} are
down — check each before assuming physical access is required.**
**Latent daemon bug found**: `connect-daemon.sh` main loop ends each iteration with a bare `wait`, which
blocks forever on the `setsid`-launched `tailscaled` child. Once the tunnel started failing, the loop
wedged for ~58 min and never self-healed on the corrected config until the package was restarted. Fix
the `wait` (e.g. `wait "$SSH_PID"` or `wait -n`) so the retry loop can recover on its own.
**SPK fixed**: Rebuilt with `which` instead of `command -v` and `TUNNEL_LOCAL_SSH_PORT=2223`. Available at `https://git.keylinkit.net/allen/kit-busrouter/raw/branch/main/kit-connect/kit-connect-0.1-0001.spk`. **SPK fixed**: Rebuilt with `which` instead of `command -v` and `TUNNEL_LOCAL_SSH_PORT=2223`. Available at `https://git.keylinkit.net/allen/kit-busrouter/raw/branch/main/kit-connect/kit-connect-0.1-0001.spk`.
+2 -2
View File
@@ -210,6 +210,6 @@ while true; do
# Start tunnel (blocking — restarts on failure) # Start tunnel (blocking — restarts on failure)
start_tunnel 2>/dev/null || true start_tunnel 2>/dev/null || true
log "daemon: both services running" # start_tunnel is already blocking and self-retrying; no bare wait here — a bare
wait # Wait for any child to die, then restart # wait blocks forever on the setsid'd tailscaled child and wedges the retry loop.
done done
Binary file not shown.