How to Bypass Arkose Labs FunCaptcha When Scraping

This guide is for engineers collecting data they are permitted to access: public pages, data covered by an agreement, or their own accounts and tenants. Circumventing an access control can breach a site's terms of service, and in some jurisdictions it touches computer-misuse law, including the Computer Fraud and Abuse Act in the United States and the Computer Misuse Act 1990 in the UK. The Ninth Circuit's hiQ Labs v. LinkedIn opinion drew a line around genuinely public data, and Arkose Labs sits mostly on the other side of it, in front of signup, login, checkout and account recovery. Those flows are not a scraping target.

That is the last word on it. Everything below assumes you are allowed to fetch the data and the challenge is simply misclassifying you.

Every solver-shaped approach to Arkose depends on the current shape of a system whose vendor ships changes continuously and whose commercial pitch is that the shape keeps moving. Arkose Labs sells attack economics, documented in its own Bot Manager overview, so the product is explicitly designed for the cost of an automated attempt to rise as suspicion rises. Difficulty is not fixed. It ramps against you.

Think about what that does to a cost curve. In normal scraping, unit cost falls as you scale, because you amortise infrastructure across more requests. Against an adaptive challenge, unit cost rises as you scale, because volume is itself the suspicion signal that raises difficulty. A pipeline built on solving gets more expensive at exactly the moment it starts working, then breaks outright on the next release. Avoidance has a flat cost curve. That asymmetry is the whole argument, and it is why this guide ships no solver.

The engineering answer is boring and it holds: fix the inputs to the risk score so the challenge is never served.

Arkose Labs is the bot-mitigation vendor behind FunCaptcha, the interactive challenge most people recognise as the rotate-the-animal puzzle, and its successor challenge suite Arkose MatchKey. You meet it on account creation, login and recovery for large consumer platforms rather than on product catalogues, which is the first clue about what it is defending.

The important structural fact is that FunCaptcha is not the detector. It is the enforcement action. The detector is Arkose Bot Manager, a risk engine that scores a session and then decides which of three things happens: pass silently, serve a challenge, or deny. The challenge widget only appears once that decision has already been made.

Why that distinction changes your approach

If you aim at the widget, you are negotiating with a component that has no authority. It cannot let you through on its own merits, because the difficulty it hands you was chosen by the scorer that already distrusts you. Aim upstream instead. A MatchKey challenge that renders at low difficulty for a clean session and at punishing difficulty for a dirty one is not one obstacle with two settings. It is a readout of a decision you lost several requests ago.

Arkose also ships a dedicated scraping protection product line, which tells you plainly that scraping-shaped traffic is a first-class detection category, not an edge case the vendor forgot about.

A challenge is served when the accumulated risk score crosses a threshold. These are the inputs, in rough order of how often they turn out to be the actual cause in production.

Read that table as a checklist of things you can fix, because every row is under your control. None of them requires defeating anything. The same pattern shows up across every major vendor, and our guide on how to avoid CAPTCHAs when web scraping covers the general case. Arkose is simply the strictest place to apply it.

Detection first. Before changing anything, confirm Arkose is actually what you hit, using a read-only probe.

import re
import requests

ARKOSE_MARKERS = (
    "client-api.arkoselabs.com",
    "arkoselabs.com/v2/",
    "funcaptcha",
    "data-pkey",
)

def detect_arkose(url, timeout=20):
    """Read-only probe. Confirms Arkose enforcement without interacting with it."""
    r = requests.get(url, timeout=timeout)
    body = r.text.lower()
    hits = [m for m in ARKOSE_MARKERS if m in body]
    pkey = re.search(r'data-pkey="([0-9A-Fa-f-]{36})"', r.text)
    return {
        "status": r.status_code,
        "arkose": bool(hits),
        "markers": hits,
        "public_key_present": bool(pkey),
    }

print(detect_arkose("https://example.com/login"))

If arkose comes back true on a page you expected to fetch plainly, you have a configuration problem to solve, not a puzzle.

This is the part that trips up engineers arriving from simpler defences. A WAF rule evaluates one request. Arkose Bot Manager evaluates a session: the sequence of requests carrying the same identity, across time, from the same exit, with the same fingerprint. Scores accumulate.

Three consequences follow, and they are not obvious.

Retrying the failed request is the worst available move. The request did not fail. The session did. Firing the same request again from the same session adds one more suspicious event to a score that was already over threshold, which is exactly why naive "retry on challenge" loops spiral into hard denials.

Rotating only the IP resets nothing. If the cookie jar, the browser profile and the fingerprint carry over, you have changed one column in a table where five others still say the same thing. Worse, you have now demonstrated that this identity moves between addresses.

A session with no history is suspicious on its own. Real browsers accumulate cookies, revisit pages, and carry a session that predates the request you care about. A cookie jar created milliseconds ago that immediately requests a sensitive endpoint is a shape almost no human produces. Our guide to handling cookies and sessions in web scraping covers the mechanics of keeping that state properly.

A useful metric falls out of this: track your challenge-served rate as an operational SLO, not as a cost of doing business.

from collections import defaultdict

class ChallengeRate:
    """Challenge-served rate per exit IP. Treat >2% as a broken config, not bad luck."""

    def __init__(self, threshold=0.02, min_samples=50):
        self.counts = defaultdict(lambda: {"total": 0, "challenged": 0})
        self.threshold = threshold
        self.min_samples = min_samples

    def record(self, exit_ip, was_challenged):
        c = self.counts[exit_ip]
        c["total"] += 1
        c["challenged"] += 1 if was_challenged else 0

    def unhealthy(self):
        bad = []
        for ip, c in self.counts.items():
            if c["total"] < self.min_samples:
                continue
            rate = c["challenged"] / c["total"]
            if rate > self.threshold:
                bad.append((ip, round(rate, 4), c["total"]))
        return sorted(bad, key=lambda x: -x[1])

Most teams never measure this, so they cannot tell a bad proxy pool from a bad fingerprint and end up guessing. Two percent is a workable alarm line for permitted, well-paced collection. Sit above it consistently and something in the trigger table is wrong, which no amount of retrying will fix.

Before any of the technical tuning below, exhaust the routes where the challenge is not in play at all. They are ordered by how much engineering they save you.

1. Official or partner APIs

Most platforms defended by Arkose publish a documented API, a partner programme, or a bulk data export. These paths are authenticated, rate-limited, versioned, and completely outside the challenge flow. They also survive redesigns, which HTML scraping does not.

2. Authenticated access under terms

If you hold an account and the terms permit programmatic access, use the credentialed path. An authenticated session that behaves consistently is the highest-trust identity you can present, and the risk engine treats it accordingly. Do not use this to reach data the account is not entitled to.

3. Negotiated rate limits

This route is badly undervalued. Contacting the site and asking for a documented crawl allowance, an increased quota, or a feed works more often than engineers expect, particularly for research, price comparison and academic use. It costs one email. Compare that to maintaining an adversarial pipeline indefinitely.

4. Reduce what you actually need

Ask whether you need every page daily or a sample weekly. Requirements are usually inherited rather than justified, and cutting the requirement removes the velocity signal at its source.

Velocity is the signal you can fix fastest and the one most pipelines get wrong by an order of magnitude. Start by capping yourself below whatever the site would tolerate, then measure.

Honour robots.txt as a rate instruction, not just an allow list. If it publishes a Crawl-delay, that number is the site telling you its preferred pace.

import time
import random
import threading

class PolitePacer:
    """Token bucket with jitter. One instance per exit IP, not one per process."""

    def __init__(self, rps=0.2, burst=1):
        self.interval = 1.0 / rps
        self.burst = burst
        self.tokens = burst
        self.updated = time.monotonic()
        self.lock = threading.Lock()

    def acquire(self):
        with self.lock:
            now = time.monotonic()
            self.tokens = min(self.burst, self.tokens + (now - self.updated) / self.interval)
            self.updated = now
            if self.tokens < 1:
                wait = (1 - self.tokens) * self.interval
            else:
                self.tokens -= 1
                wait = 0
        # Jitter breaks the machine-perfect cadence a fixed sleep produces.
        time.sleep(wait + random.uniform(0.4, 1.6))

Two details matter more than the rate itself. First, the pacer has to be scoped per exit IP, because that is the unit the scorer counts against. A single global limiter fanned out across ten proxies gives each of them ten times your intended rate, which is how teams end up running at 2 rps while believing they run at 0.2. Second, jitter is not decoration. A request every 5.000 seconds is a stronger bot signal than a request every 4 to 7 seconds at the same average.

Concurrency deserves the same discipline. One in-flight request per exit IP is the safe default against an Arkose-defended origin. If that is too slow, add exits rather than concurrency.

Reputation is the highest-weight input and the cheapest to fix. Arkose scores where your traffic exits: the address, its history, and the ASN it belongs to. Cloud and hosting ranges start from a deficit, because almost nothing legitimate browses a consumer signup page from an EC2 instance.

The nuance most guides skip: a residential IP is necessary but not sufficient, and a badly used residential IP is worse than a datacenter one, because you are burning an address that other people also depend on. Pin one exit per logical session, keep it for that session's lifetime, and let it rest afterwards. The reasoning behind that choice is laid out in our breakdown of what a residential proxy is and when to use one.

Geography has to agree with the rest of the story. A German exit, a de-DE locale, Europe/Berlin as the timezone and Accept-Language: de-DE form one coherent client. Mix and match, and the mismatch itself becomes the signal.

The scorer is not looking for a perfect fingerprint. It looks for contradictions between layers, because a real client cannot contradict itself. That reframing is useful: the goal is not to be undetectable, it is to be internally consistent.

Four layers have to tell the same story.

A plain HTTP client cannot satisfy layer four at all, which is why Arkose-defended pages are a browser problem before they are a proxy problem. If you run Playwright, make the four layers agree explicitly instead of accepting headless defaults.

from playwright.sync_api import sync_playwright

def coherent_context(playwright, proxy_server, proxy_user, proxy_pass):
    """A German consumer profile where every layer tells the same story."""
    browser = playwright.chromium.launch(
        headless=True,
        proxy={"server": proxy_server, "username": proxy_user, "password": proxy_pass},
        args=["--disable-blink-features=AutomationControlled"],
    )
    return browser.new_context(
        locale="de-DE",
        timezone_id="Europe/Berlin",
        viewport={"width": 1512, "height": 852},   # a real laptop, not 800x600
        device_scale_factor=2,
        color_scheme="light",
        extra_http_headers={"Accept-Language": "de-DE,de;q=0.9,en;q=0.7"},
    )

with sync_playwright() as p:
    ctx = coherent_context(p, "http://gateway.sparkproxy.io:11000", "user", "pass")
    page = ctx.new_page()
    page.goto("https://example.com/", wait_until="domcontentloaded")
    print(page.title())

The viewport value is deliberate. A 1512x852 logical resolution at a device scale factor of 2 matches a common real laptop, whereas the 800x600 default appears in almost no human traffic. Every default you leave in place is a free identifier. For the first layer specifically, see what TLS fingerprinting is.

When a challenge appears anyway, treat it as telemetry. It is the risk engine telling you this identity is spent. The correct response has three steps and none of them is "try again".

1. Stop the session. Not the request, the whole identity: cookies, browser profile, exit IP.
2. Cool off the exit. Park that address for a meaningful interval, on the order of tens of minutes, so its recent history ages out.
3. Re-warm a fresh identity. Start on a low-risk entry page, let cookies settle naturally, then approach the path you want.

Re-warming is the step people skip. A brand-new session that goes straight to a sensitive endpoint has zero history, which is one of the shapes the trigger table flags. Landing on a homepage first, waiting a realistic beat, then following an internal link produces the history a normal client would already have.

import random
import time

COOL_OFF_SECONDS = (900, 2400)   # 15 to 40 minutes, jittered

class SessionManager:
    def __init__(self, exits):
        self.available = list(exits)
        self.resting = {}          # exit -> timestamp it becomes available again

    def _reclaim(self):
        now = time.time()
        for exit_ip, ready_at in list(self.resting.items()):
            if now >= ready_at:
                del self.resting[exit_ip]
                self.available.append(exit_ip)

    def checkout(self):
        self._reclaim()
        if not self.available:
            return None            # no clean exit: pause the job, do not force one
        return self.available.pop(random.randrange(len(self.available)))

    def burn(self, exit_ip):
        """Called when a challenge was served. Rest the exit, discard the session."""
        self.resting[exit_ip] = time.time() + random.uniform(*COOL_OFF_SECONDS)

def warm_up(ctx, entry_url, target_url):
    page = ctx.new_page()
    page.goto(entry_url, wait_until="domcontentloaded")
    time.sleep(random.uniform(3.0, 8.0))      # read the page the way a person would
    page.goto(target_url, referer=entry_url, wait_until="domcontentloaded")
    return page

Notice what checkout does when nothing is available: it returns None and the job pauses. That is the honest behaviour. Forcing a request through an exit that is still cooling off is how a soft challenge becomes a hard denial. General retry mechanics live in retry and backoff strategies for web scraping. Where a target answers with 429 Too Many Requests and a Retry-After header as defined in RFC 9110, that is an explicit instruction worth obeying literally rather than approximating.

Keeping residential exits clean, fingerprints coherent, pacing honest and sessions rotating is a standing job. The SparkProxy Scraping API collapses those layers into request parameters, each one mapping to a row of the trigger table.

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://example.com/catalog" \
  --data-urlencode "render_js=true" \
  --data-urlencode "premium_proxy=true" \
  --data-urlencode "country_code=DE" \
  --data-urlencode "stealth=true" \
  --data-urlencode "human=true" \
  --data-urlencode "wait_for=.product-grid" \
  --data-urlencode "json_response=true"

The same call in Python, annotated by which signal each flag addresses:

import requests

API = "https://scrape.sparkproxy.io/api/v1"

def fetch(url):
    r = requests.get(
        API,
        headers={"X-API-Key": "YOUR_API_KEY"},
        params={
            "url": url,
            "render_js": "true",        # real Chromium, so the JS fingerprint layer is genuine
            "premium_proxy": "true",    # residential exit instead of a flagged hosting ASN
            "country_code": "DE",       # geo that agrees with locale and Accept-Language
            "stealth": "true",          # homepage pre-warm, forced referrer, longer idle delays
            "human": "true",            # mouse movement and randomised interaction delays
            "wait_for": ".product-grid",# hold until real content renders, not an interstitial
            "json_response": "true",    # envelope with job_id, status_code, duration_ms, body
        },
        timeout=120,
    )
    return r.json()

Per the SparkProxy API documentation, premium_proxy=true costs 25 credits with rendering and 10 without, while country_code and stealth add 5 credits each. stealth=true is the re-warm pattern from the previous section implemented server-side: it visits the homepage first, forces a search-engine referrer, and stretches idle delays to between 1.5 and 3.5 seconds.

One gotcha worth knowing

session_id is a log label only. Every request gets a fresh browser profile, and cookies do not persist between API calls. If your workflow genuinely needs continuity across requests, carry the state yourself with the cookies parameter:

import json

jar = [{"name": "consent", "value": "1", "domain": "example.com"}]

r = requests.get(
    API,
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={
        "url": "https://example.com/catalog?page=2",
        "render_js": "true",
        "premium_proxy": "true",
        "cookies": json.dumps(jar),   # explicit state, since session_id persists none
        "json_response": "true",
    },
    timeout=120,
)

Teams routinely assume session_id gives them sticky sessions, then cannot work out why their history-based trust never accumulates. It is a naming trap, and the docs are explicit about it. Whether to run all of this yourself or buy it is the usual trade, laid out in web scraping API versus self-managed proxies.

None of this defeats Arkose, and it is not meant to. It keeps permitted traffic below the threshold where a challenge gets served, the only outcome that stays true after the next release.