Updated November 2026 · 8 min read
Nobody wants to get blocked. But most guides on this topic are from 2019 and give outdated advice. This is the practical 2026 playbook — what works, what doesn't, and how to decide whether to build or buy your bypass layer.
Before you touch proxies or browsers, respect the rate limit. A single IP making 1 request per second to a mid-size site is usually fine. 10 per second gets you banned. 100 per second gets you banned plus a firewall rule against your IP range.
import time
import requests
for url in urls:
r = requests.get(url, headers=HEADERS)
process(r)
time.sleep(1.0) # 1 req/sec — safe on most targets
Default python-requests/2.x in the User-Agent is a giant "block me" sign. Copy a real Chrome header set. But don't stop there — order matters, and headers must match the browser you claim to be.
HEADERS = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Upgrade-Insecure-Requests": "1",
}
Cloudflare, Akamai, DataDome and PerimeterX all fingerprint your TLS handshake (JA3/JA4). Python's requests and httpx produce a fingerprint that no real browser has. You can fake it with curl_cffi:
from curl_cffi import requests as cffi_requests r = cffi_requests.get(url, impersonate="chrome120")
This alone gets you past ~40% of Cloudflare-protected sites. Not all — managed challenges still need JS.
Datacenter proxies (Hetzner, DigitalOcean IPs) get challenged immediately. Residential proxies cost $2–5/GB but actually work. In 2026, the ranking of proxy types by "gets past Cloudflare" is:
If a site uses Cloudflare's managed challenge (Turnstile), none of the above is enough — you must actually run JavaScript and solve a proof-of-work. You have two options:
Option A: DIY with Playwright + proxy
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(proxy={"server": "http://proxy:port"})
page = browser.new_page()
page.goto("https://target.com", wait_until="networkidle")
html = page.content()
Costs: ~2 GB RAM per browser, ~3–5 seconds per request, and you still need to solve Turnstile somehow (services like 2Captcha charge $1–3 per 1000 solves).
Option B: bypass API
import requests
r = requests.post(
"https://cloudflare-bypass.shop/v1/solve",
headers={"X-API-Key": "YOUR_KEY"},
json={"url": "https://target.com"},
timeout=200
)
html = r.text
Costs: $0.03 per success. No RAM, no proxies, no browser. The API handles all of the above on its side.
Do the math on a 3-month horizon:
| Scenario | DIY cost | Bypass API cost |
|---|---|---|
| 10K requests, no blocks | $0 | $300 |
| 10K requests, 50% blocked | $30 + 20 hrs work | $150 |
| 100K requests, Cloudflare | $500 proxies + $300 server + 60 hrs work | $3000 |
Rule: if your time is worth more than $20/hour, buy the bypass for anything Cloudflare-related. Build for simple targets.
Once you're past the block, don't second-guess. A 200 OK with > 10,000 characters is a real page. If you're seeing 200 OK with 500 characters, you got a challenge page — retry or escalate to a bypass API.
Rate limits and bot-like TLS fingerprints. Sending too many requests from one IP gets you banned within minutes. Even at low rates, Python's default requests fingerprint is flagged by Cloudflare, Akamai and DataDome as a known bot signature.
Yes, on sites that score IP reputation. Datacenter IPs (Hetzner, DigitalOcean) are challenged immediately; residential proxies pass most of them. But residential proxies do not solve the JavaScript challenge — for that you still need a browser or a bypass API.
Build only if you have time to maintain it. Cloudflare changes its challenge every few weeks, and self-built bypasses break. If your time is worth more than $20 per hour, a bypass API at $0.03 per success is cheaper than DIY.
Check three things: 200 OK status, body length over 10,000 characters for real pages, and a Content-Type that matches the target. Do not scan the HTML for "cloudflare" — every protected page contains that string, so you will get false positives.
POST the URL to a bypass API that returns raw HTML and charges per success. CF Bypass charges $0.03 per bypass, refunds failures, and requires no subscription. For a few thousand requests a month, it is cheaper than any proxy + captcha solver combination.
Related: Python guide · ZenRows alternatives · Scrapy + Cloudflare · API comparison 2026 · requests without Selenium