Three approaches, three different cost structures. Here is what actually works, what it costs, and how to choose for your workload.
requests failsA request from the requests library arrives with a TLS fingerprint and HTTP/2 settings that do not match any real browser. Cloudflare reads that fingerprint before a single byte of HTML is rendered, and answers with a 403 or a JavaScript challenge.
Even when the fingerprint is masked, the challenge page still needs JavaScript execution to produce a clearance cookie. requests cannot execute JavaScript, so it never gets past that step. This is the reason most "Cloudflare blocked my scraper" problems are not about headers — they are about the client itself.
A headless browser executes JavaScript, so it can pass a simple challenge. That is where the advantages end.
navigator.webdriver, canvas and WebGL rendering, and behavioural signals. Stock Selenium is detected almost immediately.Headless browsers make sense when you need the full interaction: logging in, clicking through a flow, filling forms on an unprotected site. For bulk page fetching they are the most expensive option per page.
Residential IPs fix the reputation problem: the request comes from a real home connection instead of a datacenter range. Prices usually run from one to eight dollars per gigabyte.
But a clean IP does not execute JavaScript. On a page protected by a managed challenge, a residential proxy alone still returns the challenge page. Proxies are a necessary ingredient, not a complete solution — they work best when paired with a client that can pass the challenge.
A managed API combines everything above behind one HTTP call: rotating residential IPs, a browser with a realistic fingerprint, JavaScript rendering and challenge handling. You send a URL and receive the page.
import requests
response = requests.post(
"https://cloudflare-bypass.shop/v1/solve",
headers={"X-API-Key": "sk_live_YOUR_KEY"},
json={"url": "https://target-site.com/page"},
timeout=200,
)
html = response.text
print(len(html), "chars")
The response is not wrapped in JSON: you get the raw HTML or bytes, the original Content-Type and the original status code, the same way the target server sent them. HTML responses also carry the header x-bypass-status: success.
| Approach | Speed | Cost | Maintenance | Managed challenges |
|---|---|---|---|---|
| requests, no proxy | fast | free | none | blocked |
| requests + residential proxy | fast | $1–8 / GB | medium | sometimes |
| Selenium / Playwright | 3–10 s per page | server + dev time | high | unreliable |
| Managed bypass API | 1–5 s per page | $0.03 / request | minimal | yes |
requests is fine.requests.10 free bypasses on signup. $0.03 per successful request afterwards, refund if we fail. Same balance covers residential proxies, VPS and VLESS VPN keys.
Create an account →Rarely. Modern Cloudflare checks the TLS fingerprint and runs a JavaScript challenge, and requests can do neither.
Partly. It executes JavaScript, but automation signals, RAM usage and captchas make it unreliable at scale.
From a few thousand pages a day, a managed API at $0.03 per request is usually cheaper than servers, proxies and engineering time combined.
A transparent one returns raw HTML or bytes with the original Content-Type and status code — exactly what the target server returned.
No. The provider handles IP rotation and rendering. You send a URL and receive the page.
It depends on the target and jurisdiction. You are responsible for complying with the target site's terms and local law.