Skip to content

When a site blocks you

Your real browser

When walls keep stopping automated browsers, let your installed Chrome or Edge try the page.

Some walls recognise automated browsers, however stealthy. A real browser, the Chrome or Edge you browse with, looks like what it is. Turn this on and a read that keeps meeting walls tries the page in your real browser (a real, non-headless window, kept off your screen) before giving up.

  • Opt-in. Off unless a call allows it (allow_real_browser) or you set FRANKENSURF_REAL_BROWSER=1 for every read.
  • Never your profile. It uses FrankenSurf’s own browser profile, so your cookies, history and sign-ins stay yours. Cookies that profile earns (a challenge it passed) are kept, as in a person’s browser, so later visits get through more easily.
  • Never clicks. A challenge that clears by itself (“Just a moment…”) is waited out, up to real_browser_wait_seconds (15). One that needs a person still fails as a wall, and a handoff can follow.
  • Out of your way. On Linux and WSL it runs on a private virtual display (Xvfb: sudo apt install xvfb): a real, non-headless browser that never appears on your screen or takes your cursor or focus, and that works on a server with no display too. Without Xvfb, and on macOS and Windows, the window opens off-screen on your display: out of sight, but it may still take focus there. Set FRANKENSURF_REAL_BROWSER_WINDOW=visible to watch it on your screen (on a virtual display it draws without your GPU, which a few walls notice). A browser you attach with FRANKENSURF_REAL_BROWSER_CDP is yours to place: its windows show wherever that browser runs.
  • One page at a time per profile, in a window that opens and closes. A read that finds the profile busy (another read, or another FrankenSurf process) skips the real browser rather than waiting.

It joins the end of an automatic read’s route (before an archived copy, when that is allowed too). Once the read has met walls (escalate_after_walls, 4), it moves to the front of what is left, ahead of any paid tools: it is free, and walls rarely tell it from a person. The receipt says so: receipt.method is real_browser, and receipt.routing.escalated_to names it when walls moved it ahead.

A walled site pauses other reads for 15 minutes (pacing), but not this one: a read that allows the real browser while the site is cooling down after a wall goes straight to it (a rate limit’s pause still holds). So an agent can retry a walled page with allow_real_browser, or try_harder_than and allow_real_browser together, and get a real attempt.

A handoff, when the call allows one, still comes last and uses its own profile: a sign-in you complete there never reaches the real browser’s anonymous reads.

# Try your real browser when walls keep blocking
page = await web.read(url, policy_overrides={"allow_real_browser": True})
# Straight to your real browser
page = await web.read(url, provider="real_browser")

FrankenSurf looks for Google Chrome, then Microsoft Edge, where they install themselves on Linux, macOS and Windows. Point it elsewhere with FRANKENSURF_REAL_BROWSER_PATH (any Chromium-based browser, such as Brave).

To use a browser that is already running, start it with remote debugging and a profile of its own, and set FRANKENSURF_REAL_BROWSER_CDP:

Terminal window
google-chrome --remote-debugging-port=9222 --user-data-dir="$HOME/.frankensurf-chrome"
export FRANKENSURF_REAL_BROWSER_CDP=http://127.0.0.1:9222

Each read then opens a fresh private window there and closes it afterwards: no cookies go in or out, so the browser’s own sign-ins never reach a read. Chrome refuses remote debugging on your everyday profile, which is why it needs its own.

Option Default Description
allow_real_browser false Let your real browser join automatic reads.
real_browser_wait_seconds 15 How long a challenge gets to clear by itself.
FRANKENSURF_REAL_BROWSER unset 1 turns it on for every read (an MCP server, say).
FRANKENSURF_REAL_BROWSER_PATH Chrome, then Edge The browser to launch.
FRANKENSURF_REAL_BROWSER_PROFILE ~/.local/share/frankensurf/real-browser-profile FrankenSurf’s own profile for it.
FRANKENSURF_REAL_BROWSER_CDP none Open tabs in a real browser you already run.
FRANKENSURF_REAL_BROWSER_WINDOW hidden visible shows the window on your screen.