feat(server): phase 4 polish and observability (SYM-24)

[Swiftyos]

Summary

• Ships the Phase 4 hardening pass for agentprobe start-server: in-process metrics + spans + structured logger, SSE heartbeat/retry/terminal-event guarantees, dashboard keyboard navigation and empty/error/loading polish, plus deterministic latency-budget and soak harnesses with the supporting ops docs.
• Every behaviour required by the SYM-24 acceptance criteria is backed by a new or updated test or a new repo entrypoint; no major refactors of the Phase 1–3 layout.

Key changes

• Observability adapters under src/runtime/server/observability/ (metrics registry, span recorder, structured logger, redaction helpers, server.startup log). Wired into startAgentProbeServer (server.http.requests, server.runs.active, server.runs.started_total, server.runs.finished_total, server.sse.connections) and the RunController (run.started, run.finished, run.error plus span scopes for runStartValidation, runControllerExecute, and runSuiteBoot).
• SSE hardening in src/runtime/server/routes/sse.ts and streams/events.ts: heartbeat comments, retry: 2000 on connect, Last-Event-ID honored from header and last_event_id query, explicit terminal kinds (run_finished / run_cancelled / run_failed), and terminal replay for historical runs whose ring buffer is gone.
• Dashboard polish (dashboard/src/**): useKeyboardShortcuts dispatcher with j/k///g r/g p/g s, filter input on the Runs page with its own empty state, expanded empty/error/loading affordances on Suites/Scenarios, visible focus rings.
• Latency budget + soak harness (scripts/latency-budget.ts, scripts/soak.ts): deterministic local checks that exit non-zero when p95 exceeds budgets; soak harness runs fast by default and flips to ~1h manual mode. Exposed as bun run latency-budget and bun run soak.
• Docs: docs/RELIABILITY.md now lists shipped metric/span names and budgets; docs/playbooks/agent-probe-server.md adds proxy SSE + nginx buffering, backup/restore, migration recovery, dashboard cache notes, request-id troubleshooting, and the keyboard reference; platform product specs gain Phase 4 scenarios.

Test plan

• bun run docs:validate
• bun run typecheck
• bun run test (173 passing, 36 files)
• bun run test:e2e
• bun run dashboard:build
• bun run fast-feedback
• bun run soak --duration-ms 5000 --runs 30 --sse-connections 2 — 0 active runs at shutdown, 0 failures
• bun run latency-budget --samples 5 --report-only — all surfaces well under documented budgets on loopback
• Manual ~1h soak (bun run soak --manual) on the reviewer's machine if the CI soak is not enough evidence
• Manual dashboard pass at desktop + narrow mobile widths for Runs (search + empty-filter), Start, Presets, Compare, Settings (auth state), empty states, and SSE-driven live run detail

Notes for reviewers

• Targets Significant-Gravitas/AgentProbe:dev per the SYM-24 PR-target directive.
• Metric and span names become part of the operational contract per the issue scope.
• The metrics adapter is in-process only; no external collector is required or added.

🤖 Generated with Claude Code