Skip to content

fix: WebFetch fails ~30% of the time due to TLS fingerprint mismatch with spoofed browser UA #302

@anandgupta42

Description

@anandgupta42

Problem

The WebFetch tool uses a fake Chrome User-Agent string, but Bun's TLS fingerprint doesn't match a real Chrome browser. Anti-bot systems (Cloudflare, Anubis, etc.) detect the mismatch and return 403/404 as a soft block, causing ~30% of fetches to fail.

Root Cause

Proposed Fix

Invert the UA strategy: honest bot UA first (altimate-code/1.0), browser UA as fallback on 403/406. This matches industry practice (ChatGPT, Perplexity use honest bot UAs).

Benchmark Results

  • Honest UA solo: 20/20 (100%) vs Browser UA solo: 14/20 (70%)
  • New retry strategy: 19/20 (95%) vs Old retry strategy: 17/20 (85%)
  • 0 regressions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions