Skip to content

Commit bce0a01

Browse files
authored
phase a/fix playwright webserver (#48)
* fix(e2e): wire playwright webServer to boot live code-iq backend Adds a webServer block to src/main/frontend/playwright.config.ts so npm run test:e2e boots a real code-iq serve against the spring-petclinic fixture before any spec runs, removing the root cause of the 0-pass / 575-fail Playwright audit result. Design choices: - /api/stats is the readiness probe, not /actuator/health (today returns 503 OUT_OF_SERVICE; tracked separately on phase-a/fix-graph-health). - bash -c "exec java ..." so Playwright can signal the JVM cleanly on teardown, and so the *-cli.jar glob expands. - Overridable via BASE_URL (skip webServer entirely), E2E_FIXTURE (pick a different pre-enriched repo), E2E_PORT (avoid port conflicts). - baseURL unchanged for spec code — still http://localhost:8080 by default, overridable via BASE_URL. Verified on this machine: - webServer boots the backend; /api/stats returns 200 within 120s. - Playwright connects to the backend. Remaining system-level blocker (not a config issue): chromium headless shell is missing 8 shared libraries on this host (libatk-1.0, libXcomposite, libgbm, libatspi, ...). Fix once per host: sudo npx playwright install-deps chromium Once that's run, npm run test:e2e exercises the full 131-test chromium suite against a real backend. The original baseline 0p/575f number was the sum across 7 Playwright projects. Documented in BASELINE.md. * docs(baseline): capture real Playwright chromium pass rate (33/131) With the webServer wiring in place + system deps installed via `sudo npx playwright install-deps chromium`, ran the full chromium project (19.2 min wall time). Results: 33 passed / 94 failed / 4 skipped Huge jump from the "0 passed / 575 failed" baseline — the 575 was the sum across 7 Playwright projects (4 browsers × many responsive breakpoints), and all of it was caused by not having a backend. With a live backend against the spring-petclinic fixture, a third of the chromium suite passes. The remaining 94 failures are NOT environmental; they're genuine test ↔ UI divergence: - expect(locator).toBeVisible / element(s) not found — selectors targeting UI elements that don't render in the current build (component/CSS-class renames). - strict mode violations on `getByText(/0/)` — tests assume unique stat values; spring-petclinic's 691-node graph produces many zero-valued cards, so the match resolves to 8+ elements. - toHaveLength mismatches — expected N list items, got different count. - "Button has no aria-label" in accessibility.spec.ts — real a11y regressions, not a test bug. - 30s timeouts on locator.focus / locator.boundingBox — elements that never appear. Treating the chromium 33/94/4 split as the honest Phase A baseline. Test-suite maintenance (update selectors to match current UI, or generate a deterministic fixture) is scoped out of Phase A.
1 parent a131a80 commit bce0a01

2 files changed

Lines changed: 44 additions & 3 deletions

File tree

docs/superpowers/baselines/2026-04-17/BASELINE.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -222,6 +222,19 @@ Ordered by severity. Each item cites the raw artifact it was derived from.
222222

223223
- **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`.
224224
- Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`.
225+
- **RESOLVED — config + system-deps + run (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200). Installed system libs via `sudo npx playwright install-deps chromium`. Ran full chromium suite against the live backend (19.2 min wall time).
226+
227+
**New chromium baseline**: `33 passed / 94 failed / 4 skipped` out of **131 tests**. Huge improvement from "0 passed / 575 failed" (the 575 was the sum across 7 Playwright projects — chromium/firefox/edge/webkit + 3 responsive breakpoints). The 94 chromium failures are **not environmental** — they're real test ↔ UI divergence. Signature histogram of the 22 detailed failure blocks analyzed:
228+
229+
| Pattern | Interpretation |
230+
|---|---|
231+
| `expect(locator).toBeVisible() failed` / `element(s) not found` | Selectors targeting UI elements that no longer render (renamed / removed). |
232+
| `strict mode violation: getByText(/0/) resolved to 8 elements` | Tests assumed unique stats values; spring-petclinic's 691-node graph produces many zero-valued stat cards. |
233+
| `expect(received).toHaveLength(expected)` | List count mismatches (menu items, nav links, stats rows). |
234+
| `Error: Button ... has no aria-label` | Real a11y regressions in `accessibility.spec.ts`. |
235+
| `locator.focus: Test timeout 30000ms exceeded` | UI interactions hanging on elements that never appear. |
236+
237+
Test-suite maintenance / fixture generation is a follow-up (not a Phase A blocker). Candidate approaches: (1) rebuild the Ant-Design selectors to match current UI taxonomy; (2) pin a synthetic fixture with known, deterministic stats counts; (3) relax strict-mode selectors to `.first()` where tests genuinely target "any such element". 22 of the 94 failures have full error blocks captured in `raw/frontend/playwright.log`.
225238

226239
### High
227240

src/main/frontend/playwright.config.ts

Lines changed: 31 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,14 @@ import { defineConfig, devices } from '@playwright/test';
55
* Playwright E2E test configuration for Code IQ UI Redesign (Phase 7).
66
*
77
* Prerequisites:
8-
* - `code-iq serve` running on localhost:8080 (Neo4j graph loaded)
9-
* - Or use `webServer` below to start the Vite dev server pointing at a real backend
8+
* - A pre-built CLI jar under ../../../target/code-iq-*-cli.jar
9+
* (run `mvn -DskipTests package` from the repo root if missing).
10+
* - A pre-populated fixture at <repo-root>/.seeds/spring-petclinic
11+
* with .code-iq/cache + .code-iq/graph populated by
12+
* `./scripts/baseline/run-pipeline.sh spring-petclinic`. The webServer
13+
* block below boots `code-iq serve` against this fixture automatically.
14+
* - Set `BASE_URL` / `E2E_FIXTURE` to override defaults when running against
15+
* a different backend or fixture.
1016
*
1117
* Run all tests: npm run test:e2e
1218
* Run headed: npm run test:e2e:headed
@@ -22,8 +28,30 @@ export default defineConfig({
2228
workers: process.env.CI ? 1 : undefined,
2329
reporter: [['html', { outputFolder: 'playwright-report' }], ['line']],
2430

31+
// Boot the real code-iq backend against a pre-enriched fixture before any
32+
// spec runs. Skipped if BASE_URL points elsewhere (developer has their own
33+
// backend) or if a server is already listening on the target port locally.
34+
webServer: process.env.BASE_URL ? undefined : {
35+
command: [
36+
'bash -c',
37+
// Use shell so the target/*-cli.jar glob expands. `exec` hands the PID
38+
// to Playwright so it can kill the process cleanly on test teardown.
39+
`"exec java -jar ../../../target/code-iq-*-cli.jar serve `
40+
+ `${process.env.E2E_FIXTURE ?? '../../../.seeds/spring-petclinic'} `
41+
+ `--port ${process.env.E2E_PORT ?? '8080'}"`,
42+
].join(' '),
43+
// /actuator/health is not a reliable readiness signal today (see
44+
// BASELINE.md §GraphHealthIndicator). /api/stats returns 200 iff the
45+
// server has finished starting and the graph has loaded.
46+
url: `http://localhost:${process.env.E2E_PORT ?? '8080'}/api/stats`,
47+
timeout: 120_000,
48+
reuseExistingServer: !process.env.CI,
49+
stdout: 'pipe',
50+
stderr: 'pipe',
51+
},
52+
2553
use: {
26-
baseURL: process.env.BASE_URL ?? 'http://localhost:8080',
54+
baseURL: process.env.BASE_URL ?? `http://localhost:${process.env.E2E_PORT ?? '8080'}`,
2755
trace: 'on-first-retry',
2856
screenshot: 'only-on-failure',
2957
video: 'retain-on-failure',

0 commit comments

Comments
 (0)