Skip to content

fix(ui): per-phase upload progress feedback#94

Merged
aksOps merged 1 commit intomainfrom
fix/upload-progress-feedback
May 3, 2026
Merged

fix(ui): per-phase upload progress feedback#94
aksOps merged 1 commit intomainfrom
fix/upload-progress-feedback

Conversation

@aksOps
Copy link
Copy Markdown
Contributor

@aksOps aksOps commented May 3, 2026

Summary

When a user uploads a file via the UI, today they see ~4 plaintext messages over the whole indexing job and silent failures on LLM 5xx during extraction. This PR replaces the upload SSE channel with structured per-phase events.

  • Pipeline (internal/pipeline/pipeline.go) — extends ProgressEvent with File, ChunksDone, ChunksTotal. indexFile now emits at every phase boundary: load → chunk → embed (with chunk counters) → extract_entities → extract_relationships → extract_claims → structure. Send is non-blocking so a slow consumer never stalls indexing. Existing CLI callers pass nil Progress and remain unaffected.
  • Upload handler (internal/api/handlers.go) — per-job structured event log (jobProgressLog w/ sync.Cond). The upload goroutine creates a buffered chan pipeline.ProgressEvent, passes it to IndexOptions, and a relay goroutine fans events into the log. The SSE handler at GET /api/upload/progress?job_id=... now writes data: {json}\n\n frames where {json} is {job_id, file, phase, chunks_done, chunks_total, message, done, error}. Indexing failures emit a terminal phase: "error" event. Legacy plain-text relay preserved when ?job_id= is omitted. Route path unchanged.
  • UI hook (ui/src/hooks/api/useUploadProgress.ts) — fetch + ReadableStream-based SSE consumer with a shared store accessed via useSyncExternalStore. Robust frame parser (parseSseChunk) handles split frames, comment/retry/id lines, and falls back gracefully on plain-text legacy frames.
  • Modal (ui/src/routes/documents/UploadModal.tsx) — per-file row per phase (Linear-style restraint, semantic shadcn classes, aria-live="polite"), embed N/M chunks counter, inline text-destructive for failures.

End-to-end verified against the agent's ollama/bge-m3 + llama3.2 setup: 11 distinct phases observed for a single-file upload, terminating on done:true. Production server on :37777 was not touched.

Test plan

  • make build clean
  • make vet clean
  • make test — all packages green
  • cd ui && npm run build — TypeScript + Vite clean
  • cd ui && npm test -- --run — 102/102 tests pass (incl. new useUploadProgress.test.tsx)
  • Existing TestUploadProgress_JobIDFiltering regression test extended; new TestUploadProgress_StructuredEventFormat asserts the JSON wire shape and ≥6 distinct phases
  • Live smoke (build → :37788 → POST upload → SSE stream): 11 distinct phases, done:true received, server cleanly stopped

Caveats

  • Vitest covers parseSseChunk (frame parsing) but not the full streaming hook against a mock ReadableStream; deferred since the parser is the load-bearing part and the live smoke covers wiring.
  • Job's per-file event log retains the full history in memory (cleared on terminal). For very large multi-file uploads this could grow; not a regression vs. today's behaviour and acceptable for the typical UI use case.

🤖 Generated with Claude Code

Upload SSE now streams structured per-phase JSON events (queued, load,
chunk, embed, extract_entities, extract_relationships, extract_claims,
structure, finalize, done/error) instead of 1-4 plaintext messages over
an entire job. The pipeline emits non-blocking ProgressEvents at every
phase boundary; the upload handler relays them on a per-job channel,
JSON-encoded over `data: {...}\n\n` frames. The React modal subscribes
via a shared store-backed hook (parseSseChunk + useSyncExternalStore),
renders a per-file row with the current phase plus chunk counters
during embed, and inline errors when the pipeline reports a failure.

Verified end-to-end against ollama/bge-m3 + llama3.2 — 11 distinct
phase events observed for a single-file upload, terminating on
done:true. Existing job-id-filtering regression test extended to
assert the new JSON wire format.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@aksOps aksOps enabled auto-merge (squash) May 3, 2026 17:33
@aksOps aksOps merged commit afdacc5 into main May 3, 2026
16 of 18 checks passed
@aksOps aksOps deleted the fix/upload-progress-feedback branch May 3, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant