How to Identify Which Anti-Bot Protection a Site Uses

Anti-bot products fail differently, so they have to be handled differently.

Cloudflare's managed challenge is mostly a browser-environment test: give it a real browser with a consistent TLS and JavaScript fingerprint and you're usually through. Kasada runs a client-side proof-of-work, so a real browser is mandatory and no amount of header tuning substitutes for the CPU cycles it costs. DataDome leans hard on IP reputation and behaviour, which means the browser can be perfect and a datacenter IP will still lose. Akamai scores a long-lived _abck cookie across a whole session, so a strategy of rotating your IP on every request actively hurts you there.

Pick the wrong model and you'll spend days tuning the one variable nobody is measuring. That's the real cost of guessing.

Here is the same point as a decision table. The middle column is the signal each product weights most heavily, and the right column is what to change first once you've named it:

Two rows in that table point in opposite directions, which is the clearest argument for identifying before acting. Against DataDome, rotating to a fresh IP is often the fix. Against Akamai, rotating is the thing that breaks you, because the _abck cookie is scored over the life of a session and a new IP mid-session reads as a hijack. The same action helps in one case and causes the block in the other.

There's a second reason to name the vendor, and it matters more than it sounds. A 403 with no vendor fingerprint at all usually isn't a managed anti-bot. It's an origin-level deny rule on your IP or your ASN, which is a different problem with a much simpler fix. Knowing the difference saves you from building a headless browser farm to solve something that only needed cleaner address space.

Four signals, ordered by how quickly they pay off:

1. Response headers. Vendor-specific headers are the fastest tell and the hardest to strip without breaking the product.
2. Cookie names. Nearly every vendor sets at least one uniquely named cookie, even on a block.
3. Block-page body. Error pages carry incident IDs, vendor text and support URLs.
4. Injected script path. The challenge script's URL names the vendor, and sometimes the tier.

Collect all four with one request and read the output instead of guessing:

curl -s -D - -o /tmp/body.html \
  -A 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36' \
  'https://target.example/some-page'

-D - prints response headers to stdout and -o keeps the body for inspection. Use a realistic User-Agent, because some vendors serve a generic 403 to obvious tooling and a named challenge to a browser-shaped request. A bare curl/8.x User-Agent can hide the very fingerprint you're trying to read.

If you check targets regularly, wrap the first four signals in one function and stop doing it by hand. This returns every vendor it can find rather than the first match, which is what you want for the stacked case further down:

import re
import requests

SIGNATURES = {
    "Cloudflare":        {"cookies": [r"^__cf_bm$", r"^cf_clearance$", r"^__cfruid$"],
                          "headers": ["cf-ray"],
                          "body":    [r"attention required", r"challenges\.cloudflare\.com"]},
    "Akamai":            {"cookies": [r"^_abck$", r"^ak_bmsc$", r"^bm_sz$", r"^bm_sv$"],
                          "headers": [], "body": [r"reference #[0-9a-f.]+"]},
    "DataDome":          {"cookies": [r"^datadome$", r"^dd_cookie_test$"],
                          "headers": ["x-datadome", "x-datadome-cid"],
                          "body":    [r"captcha-delivery\.com", r"\bdd\.js\b"]},
    "PerimeterX/HUMAN":  {"cookies": [r"^_px", r"^_pxhd$", r"^_pxvid$"],
                          "headers": [],
                          "body":    [r"access to this page has been denied", r"perimeterx"]},
    "Imperva Incapsula": {"cookies": [r"^incap_ses", r"^visid_incap", r"^nlbi_"],
                          "headers": ["x-iinfo"],
                          "body":    [r"incapsula incident id", r"_incapsula_resource"]},
    "Kasada":            {"cookies": [r"^KPSDK"],
                          "headers": ["x-kpsdk-ct", "x-kpsdk-cd", "x-kpsdk-r"],
                          "body":    []},
    "F5 BIG-IP":         {"cookies": [r"^TSPD_101", r"^TS[0-9a-f]{6,}$"],
                          "headers": [], "body": []},
    "AWS WAF":           {"cookies": [r"^aws-waf-token$"],
                          "headers": ["x-amzn-waf-action"],
                          "body":    [r"token\.awswaf\.com"]},
    "Sucuri":            {"cookies": [r"^sucuri_cloudproxy_uuid"],
                          "headers": ["x-sucuri-id", "x-sucuri-cache"],
                          "body":    [r"sucuri website firewall"]},
}

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36")


def fingerprint(url, timeout=30):
    r = requests.get(url, headers={"User-Agent": UA}, timeout=timeout)
    cookie_names = {c.name for h in list(r.history) + [r] for c in h.cookies}
    header_keys = {k.lower() for k in r.headers}
    body = r.text[:200_000].lower()

    found = []
    for vendor, sig in SIGNATURES.items():
        if (any(re.search(p, n) for p in sig["cookies"] for n in cookie_names)
                or any(h in header_keys for h in sig["headers"])
                or any(re.search(p, body) for p in sig["body"])):
            found.append(vendor)
    return {"status": r.status_code, "vendors": found,
            "cookies": sorted(cookie_names)}


print(fingerprint("https://target.example/"))

A result of {"status": 403, "vendors": [], ...} is a finding, not a failure. That combination is the origin deny rule described above, and the sections below explain how to read each signal the function collects.

Headers are where the vendor advertises itself. Pull out the ones that matter:

curl -sI 'https://target.example/' | grep -iE 'server|x-cdn|x-iinfo|x-datadome|x-kpsdk|x-amzn-waf|x-sucuri|cf-ray|akamai|via|set-cookie'

What you're looking for:

• cf-ray and server: cloudflare put the request behind Cloudflare. On its own this only proves Cloudflare is the CDN, not that bot management is switched on.
• x-iinfo, or x-cdn: Incapsula (sometimes x-cdn: Imperva), means Imperva.
• x-datadome or x-datadome-cid means DataDome.
• x-kpsdk-ct, x-kpsdk-cd or x-kpsdk-r means Kasada.
• x-amzn-waf-action means AWS WAF.
• x-sucuri-id and x-sucuri-cache mean Sucuri.
• server: AkamaiGHost, or an akamai-prefixed header, points at Akamai.

One caveat on server headers. Cloudflare and Sucuri announce themselves reliably, Akamai often does, and Imperva frequently strips or rewrites it. Absence proves nothing. Treat server as a hint and the cookies as the evidence.

Cookie names are the most dependable fingerprint, because these products need them to carry state and can't rename them without breaking the protection. Extract just the names:

curl -sI 'https://target.example/' | grep -i '^set-cookie:' | sed -E 's/^[Ss]et-[Cc]ookie: *([^=]+)=.*/\1/'

A Python version that walks the redirect chain, which matters for the stacked case later:

import requests

UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
      "(KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36")

r = requests.get(
    "https://target.example/",
    headers={"User-Agent": UA},
    allow_redirects=True,
    timeout=30,
)

for i, hop in enumerate(list(r.history) + [r]):
    names = sorted({c.name for c in hop.cookies})
    print(f"hop {i}  {hop.status_code}  {hop.url}")
    print(f"        cookies: {names or 'none'}")

Printing per hop rather than from the merged session jar is the whole point. The merged jar tells you which vendors are present. The per-hop view tells you which one actually blocked you.

Two rows need a caveat. F5's BIG-IP sets TS-prefixed cookies across several of its application-security modules, so TS01a2b3c4 alone only tells you BIG-IP is in front of the origin; the TSPD_101 variant is the one tied to Bot Defense specifically. And Kasada randomises its script path on purpose, so match the x-kpsdk-* headers rather than trying to pin a URL.

That leads to the thing most diagnoses get wrong, which is confusing presence with enforcement. Four false positives account for nearly all of it:

1. Cloudflare is present but bot management is off. cf-ray and __cf_bm appear on a huge share of the web, because __cf_bm ships with Bot Fight Mode and basic bot handling on plans that have no managed challenge configured at all. If you're getting 200 responses with complete HTML, Cloudflare is proxying the site and nothing more. Don't build a browser farm for a site that isn't challenging you.
2. A CDN banner is not a bot product. server: AkamaiGHost means Akamai delivers the content. Akamai Bot Manager is a separate product on top of that. The _abck cookie, not the server header, is what tells you Bot Manager is switched on.
3. TS cookies from plain load balancing. BIG-IP uses them for session persistence as well as for security modules, so a generic TS cookie on an enterprise site often means nothing more than a hardware load balancer.
4. A CAPTCHA on one route only. Turnstile or hCaptcha on /login or a contact form says nothing about /products. Fingerprint the exact path you intend to scrape, not the homepage, because protection is routinely applied per route and per tier.

The reliable test for enforcement is behavioural rather than structural: request the path you actually want, from the IP range you actually plan to use, and see whether the content arrives. Fingerprints tell you who is standing at the door. Only a real request tells you whether the door is locked.

When headers are stripped, the body usually still talks. Grep it for vendor strings:

grep -ioE 'incapsula incident id|_incapsula_resource|powered by imperva|access to this page has been denied|perimeterx|sucuri website firewall|attention required|datadome|captcha-delivery|reference #[0-9a-f.]+' /tmp/body.html | sort -u

What the distinctive strings mean:

• Incapsula incident ID, usually inside an iframe, plus subject=WAF Block Page in the HTML: Imperva.
• Attention Required! | Cloudflare with a ray ID in the footer: Cloudflare.
• Access to this page has been denied: the classic PerimeterX wording.
• Access Denied - Sucuri Website Firewall: Sucuri.
• Reference # followed by a dotted hex string: Akamai's error format.

Keep those incident and reference IDs. If you're scraping a site you have a relationship with, quoting the incident ID to their support team gets a block reviewed far faster than describing the symptom does.

If the page rendered but the content is missing, the challenge is running in JavaScript. List the script sources:

grep -oE '<script[^>]+src="[^"]+"' /tmp/body.html | sed -E 's/.*src="([^"]+)".*/\1/' | sort -u

CAPTCHA widgets are unambiguous, and they tell you which pathway applies:

• challenges.cloudflare.com/turnstile/v0/api.js: Cloudflare Turnstile.
• www.google.com/recaptcha/api.js: reCAPTCHA.
• js.hcaptcha.com/1/api.js: hCaptcha.
• client-api.arkoselabs.com: Arkose Labs FunCaptcha.
• static.geetest.com: GeeTest.

A CAPTCHA widget and a bot-management product sit at different layers, and plenty of sites run both. Turnstile on the login form tells you nothing about what guards the product pages.

The code alone won't name a vendor, but it rules options out.

That 429 row catches people out. A 429 carrying x-kpsdk-ct is a Kasada proof-of-work challenge, not a request to slow down, and backing off your rate won't clear it.

Some vendors only reveal themselves once JavaScript executes. Load the page in a real browser and watch where it talks:

from playwright.sync_api import sync_playwright

VENDOR_HOSTS = ("perimeterx", "captcha-delivery", "datadome", "awswaf",
                "kasada", "challenges.cloudflare.com", "arkoselabs",
                "hcaptcha", "recaptcha", "geetest", "incapsula")

def log_vendor_call(request):
    if any(host in request.url for host in VENDOR_HOSTS):
        print("vendor call:", request.url[:120])

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.on("request", log_vendor_call)
    page.goto("https://target.example/", wait_until="networkidle")
    print("cookies:", sorted({c["name"] for c in page.context.cookies()}))
    browser.close()

The cookie set after a real page load is the complete picture. A bare HTTP request often sees only the edge layer, because the application-layer product doesn't engage until its script runs.

Bear in mind that an unpatched headless browser is itself detectable, so a vendor may serve you a challenge it wouldn't serve a normal visitor. See headless browser detection for what gives automation away.

You can run the same check through an API that renders the page for you, which helps when your own IPs are already burned and you want a clean read. Mirroring the target's status code and asking for the JSON envelope gives you headers alongside the body:

import requests

r = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={
        "url": "https://target.example/",
        "render_js": "true",
        "transparent_status_code": "true",
        "json_response": "true",
    },
    timeout=120,
)
print(r.json())

transparent_status_code passes the target's real status through instead of the API's own, so a 403 stays a 403 and you can see what the site actually returned. The full parameter list is in the Scraping API docs.

Most guides assume one vendor per site. In practice a CDN-layer product and an application-layer product on the same host is common, and the combination changes what you have to solve.

A typical stack sets __cf_bm from Cloudflare at the edge and _px3 from PerimeterX at the application layer. Both cookies show up in a merged cookie jar, which tells you nothing about which one refused you. The per-hop output from step 2 does:

• Vendor cookies on the first hop, then a redirect, then a challenge on a later hop: the edge let you through and the application layer stopped you. Solve the application-layer product.
• A challenge on the first response, before any redirect: the edge stopped you and the application layer never ran. Nothing behind it matters yet.

There's one more distinction, and it's the one that should change your fix. Look at whether the blocking response sets a fresh challenge cookie or rejects a token you were already carrying:

• A fresh Set-Cookie on the block means you were never admitted. That's a cold-start problem: fingerprint, IP reputation, or a challenge you didn't solve.
• A rejection while you held a valid token means your session was retired mid-flight. That's a behavioural or rate signal, and rotating identity makes it worse rather than better. Slow down and keep the session.

Finally, the negative result, which is just as informative. If you see a 403 with no vendor cookies, no vendor headers, no challenge script and a short generic body, you're almost certainly looking at an origin or firewall deny rule keyed to your IP or ASN rather than a bot-management product. Browser automation will not fix that. Cleaner address space will. Datacenter ranges with poor reputation get caught by exactly this kind of blunt rule, which is covered in how to bypass IP fingerprinting with clean datacenter subnets.

Each product has its own guide:

• Cloudflare and Cloudflare Turnstile
• Akamai Bot Manager
• DataDome
• Kasada
• Imperva Incapsula
• PerimeterX
• AWS WAF
• Arkose Labs FunCaptcha, hCaptcha, reCAPTCHA and GeeTest

Two cross-cutting pieces apply whatever the vendor, because nearly every product listed scores both: TLS fingerprinting and browser fingerprinting.

Re-run the triage after any change you make. Vendors get swapped, tiers get upgraded, and a site that was plain Cloudflare last quarter may have an application-layer product in front of its pricing pages today. The fingerprint is cheap to re-read. The stale assumption is expensive to carry.