How to Bypass Shape Security (F5) When Web Scraping

"Bypass" here means one narrow thing: making legitimate, automated access to public data look like the ordinary browser traffic it already is, so a heuristic doesn't wrongly flag it. It does not mean breaking into anything protected. Shape sits in front of airline booking, online banking, retail checkout, and account login precisely because those flows attract credential stuffing and fraud, and those are the flows to leave alone. Set the ground rules before you write a line of code.

• Public data only. Anything behind a login, a paywall, or that exposes someone's personal information is off-limits without explicit permission. Shape guards fraud-sensitive paths on banks and airlines. Do not touch them.
• Read robots.txt and the Terms of Service. If a path is disallowed or the ToS prohibits automated collection, respect it. A Crawl-delay is a rate the site is asking you to honor, not a suggestion to route around.
• Rate-limit yourself. Bot defenses exist in part because scrapers hammer origins. Slow, considerate crawling is both more ethical and, conveniently, far less detectable.
• Prefer the official API. Many airlines and retailers publish a partner feed or a public API. If one exists, use it. It is faster, cleaner, and it is the access path the site actually sanctions.
• No guarantees. Shape rebuilds its client bytecode on every deployment and retrains its models on worldwide telemetry, so a technique that works this morning can fail this afternoon. Anyone selling a "permanent Shape bypass" is selling snake oil.

The goal is not to defeat security. It is to stop a bot filter from misclassifying a well-behaved, public-data crawler as an attacker. If a site clearly does not want to be scraped, the right answer is to stop, not to escalate.

Everything below is about fingerprint realism and good manners, not about defeating authentication or collecting data that was never meant to be public.

Shape Security started in 2011 as an anti-automation company, and F5 bought it for roughly one billion dollars in cash, closing the deal on January 24, 2020. The technology now ships as F5 Distributed Cloud Bot Defense (the on-premises variant was long sold as Shape Enterprise Defense). You will meet it in front of the login and booking flows of large US airlines, major retail banks, big-box retailers, and telecoms. These are the highest-value targets for credential stuffing, gift-card cracking, scalping, and aggregation, so they buy the most expensive defense on the market.

Two design choices make Shape harder to deal with than a generic WAF, and they are what set it apart from Cloudflare, DataDome, Akamai, or PerimeterX. First, Shape leans on behavioral analysis at scale. F5 feeds signals from traffic across its whole customer base into models that spot retooling in near real time, so the moment your automation looks different from a human it gets scored down, and the moment thousands of scrapers adopt the same trick the model learns it. Second, its client code is a JavaScript virtual machine. F5 describes it as the first VM-based obfuscation defense in JavaScript: the code that runs in your browser is a private bytecode interpreter, not readable JavaScript, and it encrypts the telemetry it collects so you cannot see which signals it reads.

The upshot: you cannot bypass Shape by faking a header or replaying a cookie. The VM has to actually run, produce a valid encrypted payload, and be backed by behavior that looks human. Only a real browser driven like a real person clears all three.

Shape does not make one yes-or-no check. It scores several independent layers on every request to a protected path and folds them into a trust decision, and a single strong mismatch can sink the whole request. That is why swapping in a "better proxy" rarely fixes anything on its own. The IP is one row in the table.

The mental model that matters: Shape cross-checks these layers against each other and against what a human on that site normally looks like. A Chrome User-Agent paired with a Python TLS handshake, no VM payload, and a form submitted in 40 milliseconds is far more suspicious than any single tell, because the inconsistencies compound. To pass, every layer has to tell the same story, and one of those layers is a computation only a real JS engine can perform. That principle drives the whole proxy-block avoidance playbook, and it applies double when an encrypted VM is watching how you move.

Shape's client script is not ordinary, minified JavaScript. It is a custom virtual machine written in JavaScript that interprets a private bytecode instruction set, and F5 randomizes those opcodes and rebuilds the bundle on a frequent rotation. Reverse-engineering last week's payload buys you nothing this week, because the instruction set itself has changed. Community decompiler projects exist, but they go stale within days, which is the entire point of the design.

Here is the flow that decides whether your scraper lives or dies on a protected path:

1. Your browser loads the page and pulls in Shape's script, usually served first-party from a path on the site's own domain so ad blockers and third-party blocks do not stop it.
2. The VM boots, fingerprints the device, and starts recording behavior: pointer movement, scroll, key timing, focus and blur events, and dozens of environment checks.
3. The VM encrypts that signal bundle and attaches it to protected requests, or posts it to a Shape collector endpoint, which returns a session token.
4. If the score is clean, the protected request (login, search, fare lookup) is allowed. If not, you get a 403 or a challenge, and no readable reason.

Because the telemetry is encrypted inside a rotating VM, you cannot inspect which signals passed or failed, and you cannot hand-build a valid payload. The design deliberately hides its own tells. This is the difference between Shape and a fingerprinting library like the ones covered in what browser fingerprinting is: the signals are similar, but here they are collected and sealed by an interpreter you cannot read. Trying to port the VM's math into your own client is a treadmill that F5 resets on every deployment.

Even though the payload is opaque, Shape leaves observable artifacts, and knowing them tells you what you are up against before you waste a single residential IP. The most reliable is the reese84 token. The VM must run and mint a valid reese84 before a protected request is accepted, the token expires in minutes, and it is bound tightly to a single session and IP.

The obfuscated header names are worth a note, because they trip people up. Shape does not use a fixed header like some vendors. It ships request headers whose prefix is a per-deployment hash, with suffixes such as -a, -b, and -c carrying different encrypted chunks. The prefix you see on one airline will not match the next, so you detect the pattern, not a literal string. A quick, read-only probe confirms Shape without touching anything protected:

import requests

def detect_shape(url):
    """Confirm Shape / F5 Bot Defense from its artifacts. Read-only, no bypass."""
    r = requests.get(url, timeout=20)
    cookie_names = set(r.cookies.keys())
    obf_headers = [h for h in r.headers if h.lower().startswith("x-") and len(h) > 8]
    is_shape = (
        "reese84" in cookie_names
        or any(n.startswith("TS") for n in cookie_names)
        or "_shapesec_" in r.text
    )
    print(f"status={r.status_code} shape={is_shape} "
          f"cookies={sorted(cookie_names)} obfuscated_headers={obf_headers}")
    return is_shape

detect_shape("https://example.com/login")

If you see a reese84 or TS* cookie, an obfuscated x- header family, or _shapesec_ in the body, you are dealing with Shape, and the VM plus behavior, not the HTTP status, is your real obstacle.

Most guides that promise a "Shape bypass" fail at one of two points, and it helps to see exactly where. The table maps each common approach to what Shape actually receives.

The first three rows are the important lesson. A plain HTTP client cannot run the VM, so it never produces a reese84 and never sends a valid encrypted payload. Adding curl_cffi fixes your TLS fingerprint, which matters against the network layer described in TLS fingerprinting, but it does not run JavaScript, so the token is still missing and the protected request still 403s. A naive headless browser finally runs the VM, so it clears the "did the script execute" check, then loses on the next layer: navigator.webdriver is true, the Chrome DevTools Protocol leaves Runtime.enable traces, the viewport is a headless default, and there is no mouse movement or realistic timing for the behavioral model to accept. Shape gets its token and a bot signature in the same request.

That is the core reason Shape is harder than token-only defenses. Producing the token is necessary but not sufficient. The behavior behind the token has to look human too.

The VM and the behavioral score are layers a plain HTTP library cannot satisfy, so the workable path starts with an actual browser. Launch Chromium through Playwright or Puppeteer, patch the obvious automation leaks with a stealth layer, and then, this is the part people skip, drive it with real interaction so the behavioral model has something human to score.

from playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync

with sync_playwright() as p:
    browser = p.chromium.launch(
        headless=True,
        proxy={"server": "http://gateway.sparkproxy.io:11000",
               "username": "user", "password": "pass"},
    )
    context = browser.new_context(
        viewport={"width": 1440, "height": 900},
        locale="en-US",
        timezone_id="America/New_York",   # match the proxy's country
    )
    page = context.new_page()
    stealth_sync(page)                    # hide navigator.webdriver and CDP tells

    page.goto("https://example.com/", wait_until="networkidle")  # VM boots, mints reese84
    page.mouse.move(240, 180)             # real pointer movement feeds the behavioral model
    page.wait_for_timeout(1500)           # let the VM record signals and settle its telemetry
    page.mouse.wheel(0, 900)              # a human scrolls before clicking
    page.goto("https://example.com/flights", wait_until="networkidle")
    html = page.content()
    browser.close()

Two details carry the weight. The stealth layer hides the headless and automation tells that the device-fingerprint row flags, so navigator.webdriver and the CDP artifacts stop leaking. The mouse.move, wait_for_timeout, and mouse.wheel calls are not decoration: they generate the pointer paths and timing the behavioral model expects, which is the layer that quietly sinks a "clean" headless bot. Set the viewport, locale, and timezone to consistent human values rather than the headless defaults, and warm up on a light page so the VM mints reese84 before you request anything protected. Puppeteer with a stealth plugin follows the same pattern.

A real browser is heavy. Run one only for targets that genuinely challenge you, and reserve lightweight clients for the easy pages that do not run Shape at all.

IP reputation is the fastest way to earn an instant block. Requests to a bank or airline login from a well-known datacenter ASN (AWS, GCP, Azure, OVH, Hetzner) start with a heavy suspicion penalty, because almost nobody checks a mileage balance from an AWS address. You can run a flawless browser and still get 403ed purely on the exit IP.

Residential and mobile proxies route through real consumer ISPs and carrier networks, so the exit IP carries the reputation of an ordinary home or phone connection. If you are new to the differences between IP families, the residential proxy explainer breaks down when each type fits. Against Shape, the IP is necessary but it has to agree with everything else:

• One session, one IP. The reese84 token is bound to the session and the IP that earned it. Do not mint a token on one address and replay it from a rotating pool; that token-to-IP mismatch is exactly what Shape watches for.
• Geo-match the audience. Scraping a US carrier's public fares? Exit from a US residential IP with a US locale and timezone. A German IP with an en-US browser hitting a US-only login is an easy anomaly.
• Keep the fingerprint stable per session. The viewport, User-Agent, timezone, and language should not change mid-session. A device that "teleports" between fingerprints inside one session is a bigger tell than any single value.

A residential IP is not a standalone bypass. It clears the reputation row, but the VM, the fingerprint, and the behavioral rows still have to pass. Pair a clean IP with a real browser and human pacing, never one alone.

This is where scraping Shape differs from beating a static cookie, and it is the step most scrapers get wrong. The reese84 token lives for only a few minutes and is tied to your session, so you cannot solve once and batch a thousand requests off one token. Shape expects the VM to keep running and to refresh its telemetry as you move through the site, roughly per meaningful action. A live browser session does this for you; a grab-the-token-and-leave approach does not.

• Warm up. Load a light page first and let the VM mint reese84 before you touch a protected path. Jumping straight to a fare-search endpoint with no prior session is a classic bot pattern.
• Move before you act. Real pointer movement, a scroll, and a short dwell before a click feed the behavioral model. A form that fills and submits in one machine-timed instant is the loudest tell there is.
• Add jittered delays. Random pauses between actions beat a fixed interval. A perfectly regular heartbeat is itself robotic.
• Keep the session alive. Let the VM refresh the token naturally inside one browser context instead of tearing down and reconnecting for every request. Reconnecting from a fresh IP every call throws away the trust you just built.
• Cap concurrency per IP. A handful of parallel sessions across the pool, not hundreds behind one address. Volume from a single IP is what pushes the behavioral score toward "bot."

import random, time

def human_pause(lo=2.0, hi=6.0):
    """Jittered delay between actions. Real users are not metronomes."""
    time.sleep(random.uniform(lo, hi))

If the site publishes a Crawl-delay in robots.txt, treat it as a hard floor. Slower crawling costs you throughput today and saves you from a blanket ban tomorrow, and against a behavioral model it is the cheapest anti-detection technique you have. There is a matching honesty check here: if a target only yields to fast, aggressive automation, that is usually a sign the data was not meant to be taken that way.

Standing up a real browser farm that runs Shape's VM, a residential pool with clean reputation, current TLS fingerprints, and human-like behavior is a full-time engineering job, because F5 keeps moving the target on every deployment. A scraping API collapses those layers into request parameters and keeps them current on the provider's side. The scraping API versus self-managed proxies comparison covers when that trade is worth it. For a VM-plus-behavior target like Shape, it usually is.

The SparkProxy Scraping API takes a target URL and handles the proxy, the real browser render, the VM execution, and the pacing for you. Each parameter maps to one of the detection rows above:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://example.com/flights" \
  --data-urlencode "render_js=true" \
  --data-urlencode "premium_proxy=true" \
  --data-urlencode "country_code=US" \
  --data-urlencode "stealth=true" \
  --data-urlencode "human=true" \
  --data-urlencode "wait_for=#results" \
  --data-urlencode "format=json"

The same request in Python, with each flag annotated by the layer it satisfies:

import requests

def fetch(url):
    r = requests.get(
        "https://scrape.sparkproxy.io/api/v1",
        headers={"X-API-Key": "YOUR_API_KEY"},
        params={
            "url": url,
            "render_js": "true",       # real Chromium runs Shape's VM and mints reese84
            "premium_proxy": "true",   # residential exit, not a flagged datacenter ASN
            "country_code": "US",       # geo-match the site's expected audience
            "stealth": "true",          # extra automation-leak patching the VM fingerprints
            "human": "true",            # simulated mouse movement and interaction delays
            "wait_for": "#results",     # hold until content past the challenge renders
            "format": "json",           # envelope with status_code + body + credits_used
        },
        timeout=120,
    )
    return r.json()

render_js=true runs a genuine browser, so Shape's VM executes and a valid reese84 gets minted. premium_proxy=true routes through a residential IP, which clears the reputation row. stealth=true adds extra automation-leak patching so the device fingerprint reads as a real machine. human=true simulates mouse movement and interaction delays, which is what feeds the behavioral model. wait_for holds until your real content appears so you never capture a challenge page by mistake, and format=json returns an envelope carrying status_code, body, and credits_used. On the SparkProxy price sheet a residential request with JS rendering costs 25 credits, and stealth and country_code add 5 credits each, so turn them on for hard targets like Shape and leave them off for pages that do not run it.

Shape is not the only vendor you will meet. The fundamentals carry across Kasada and DataDome, but Shape stands out for its VM-based obfuscation and its heavy behavioral scoring, so the human flag that shapes realistic interaction carries more weight here than a flag that only replays a cookie.

Tie it together with a loop that branches on the decoded status instead of blindly retrying. With format=json, the envelope's status_code mirrors what the target returned, so you can treat a 403 block differently from a 5xx render failure.

import time
import requests

API = "https://scrape.sparkproxy.io/api/v1"
KEY = "YOUR_API_KEY"

def scrape(url, max_tries=4):
    params = {
        "url": url,
        "render_js": "true",
        "premium_proxy": "true",
        "country_code": "US",
        "stealth": "true",
        "human": "true",
        "wait_for": "#results",
        "format": "json",
    }
    for attempt in range(1, max_tries + 1):
        env = requests.get(API, headers={"X-API-Key": KEY},
                           params=params, timeout=120).json()
        code = env.get("status_code")

        if code == 200:
            return env["body"]               # rendered HTML, past the challenge

        if code in (403, 429):               # blocked or challenged: fresh IP + session, retry
            time.sleep(2 ** attempt)
            continue

        if code >= 500:                      # render or upstream failed: back off and retry
            time.sleep(1.5 * attempt)
            continue

        raise RuntimeError(f"Unexpected Shape status {code} for {url}")

    raise RuntimeError(f"Gave up on {url} after {max_tries} tries")

The loop already runs with premium proxy, stealth, and human interaction on, so a 403 retry gets a fresh residential IP and a new browser session that runs the VM and rebuilds behavior from scratch rather than replaying a dead reese84. Exponential backoff matters here because Shape's models react to volume, and hammering a login path is the single fastest way to move your whole IP pool onto a watchlist. When nothing clears after a few tries, the honest move is to stop, respect the site's signal, and revisit whether the data is worth pursuing at all.