Session-Aware Proxy Rotation for JavaScript Sites

Before comparing vendors, decide which rotation mode your target needs. That single choice determines the port you connect to, the plan size, and whether datacenter IPs are viable at all.

The honest trade-off: pinning an exit reduces IP diversity. Every request in that session lands on the same address, so your per-IP request budget becomes the binding constraint instead of your pool size. If you pin ten sessions to ten IPs and push 500 requests through each, you are asking one address to look like a very busy human. The counterweight is to shorten sessions, not to abandon pinning. Managing per-IP request limits covers how to set that budget.

Session-aware proxy rotation binds one exit IP to one logical session, a browser context, an account, or a cart, and rotates only when that session ends rather than on every HTTP request.

The reason this matters more for rendered sites than for plain HTTP is arithmetic. A requests.get() is one request. A single navigation in Chromium is one document plus anywhere from thirty to well over a hundred subresources, followed by the XHR and fetch calls that carry the data you came for. If your unit of rotation is the HTTP request, you have just spread one page view across dozens of unrelated IP addresses.

There is a subtler trap. Browsers pool connections aggressively: HTTP/2 multiplexes an origin's requests onto one TCP connection, and HTTP/1.1 opens up to six per origin and reuses them. Many proxy gateways assign an exit per TCP connection, not per HTTP request. So a rotating gateway often looks sticky during a quick test, because most of the page rode one connection. Then a pool eviction, an idle timeout, or an HTTP/2 GOAWAY rebuilds the connection mid-session and you get a new IP with no error and no log line. Accidental stickiness is not stickiness. If the flow needs the same exit, ask for it explicitly. What a sticky session proxy is covers the mechanism, and how proxy rotation works covers the alternative.

These are the failure modes that show up in real logs, roughly ordered by how often they get misdiagnosed as "the proxies are bad".

Soft blocks deserve their own alarm. A challenge page returns HTTP 200 with a body your parser will happily accept, so the dataset fills with nulls while the dashboard reports a healthy success rate. Assert on a content marker, a price node, a product title, some element that only exists on a real page, and treat its absence as a failure. Proxy error codes explained separates gateway-side failures from target-side ones, which is the first fork in any debugging session.

Here is the rule that saves the most debugging time: derive the session label from the identity you are modelling, never from the worker index.

session-worker-3 looks tidy and fails badly. When worker 3 crashes and restarts on a different account, it inherits a warm exit IP that the previous account's cookies were bound to. Now two identities share one address, which is exactly the pattern account-security systems are built to catch.

On SparkProxy the label lives in the username, and the sticky port is 11002:

http://USER-session-acct42:PASS@gateway.sparkproxy.io:11002
http://USER-session-acct43:PASS@gateway.sparkproxy.io:11002

Same label, same exit. New label, new exit. Port 11000 is the rotating HTTP and HTTPS entry point, and 13000 speaks SOCKS5 over TCP if your tooling prefers it.

Derive labels deterministically so a restart lands on the same exit as before:

import hashlib

def session_label(identity: str) -> str:
    """Stable label per identity: the same account always maps to the same exit."""
    return "s" + hashlib.sha1(identity.encode()).hexdigest()[:10]

def proxy_url(identity: str, user: str, password: str) -> str:
    return (f"http://{user}-session-{session_label(identity)}:{password}"
            f"@gateway.sparkproxy.io:11002")

Deterministic does not mean permanent. Rotate the label when the session genuinely ends, when you hit a hard block, or after a per-IP request budget you set yourself, whichever comes first. A label that never changes turns a sticky session into a static IP, with all the exposure that implies.

The two frameworks differ in exactly the way that matters here: the scope at which a proxy can be set.

Playwright gives you the cleanest model. One context is one identity: its own cookie jar, its own storage, its own exit IP.

const { chromium } = require('playwright');

// Chromium reads proxy settings from the browser process, so Playwright wants a
// launch-level entry before per-context proxies take effect. Check your version's
// docs; newer builds have relaxed this.
const browser = await chromium.launch({ proxy: { server: 'per-context' } });

async function contextFor(identity) {
  return browser.newContext({
    proxy: {
      server: 'http://gateway.sparkproxy.io:11002',
      username: `USER-session-${sessionLabel(identity)}`,
      password: 'PASS',
    },
    locale: 'en-US',
    timezoneId: 'America/New_York',
  });
}

Puppeteer sets --proxy-server once for the whole browser process, so the obvious reading is one browser per session, which is expensive. The workaround most teams land on keeps the gateway host constant and varies only the credentials, because Puppeteer answers the proxy's 407 challenge with per-page credentials:

const puppeteer = require('puppeteer');

const browser = await puppeteer.launch({
  args: ['--proxy-server=http://gateway.sparkproxy.io:11002'],
});

async function pageFor(identity) {
  const page = await browser.newPage();
  await page.authenticate({
    username: `USER-session-${sessionLabel(identity)}`,
    password: 'PASS',
  });
  return page;
}

Verify that on your own stack with an IP echo before trusting it at scale, because pages inside one browser process still share state, and a shared cookie jar defeats the point of separate exits. If you need genuine isolation in Puppeteer, use one browser per identity or a separate user data directory per identity.

One more coherence rule, cheap to get right: pin the context locale and timezone to the exit country. An exit in Frankfurt paired with America/New_York and en-US is a mismatch a junior engineer could write a rule for, and plenty of sites have. Headless browser detection covers the rest of that surface, and web scraping with Playwright and proxies has the full setup. If you are still choosing a framework, Puppeteer vs Playwright compares them on more than proxy handling.

This is where money is won or lost, and where per-gigabyte pricing punishes rendered scraping specifically. A rendered page pulls images, fonts, analytics beacons, and sometimes video. On a per-gigabyte plan you pay for every one of those bytes. On a thread-based plan you pay for concurrency and the bytes are free, which is why SparkProxy plans are priced on threads with unlimited bandwidth and 30 days validity:

Speed caps are ceilings under the fair usage policy, not guaranteed rates, and for browser work latency matters far more than throughput anyway.

Now the number people miss. A thread is a concurrent connection, and one headless browser is not one connection. A modern page touches five to twenty origins; HTTP/1.1 origins get up to six sockets each, HTTP/2 origins get one multiplexed socket. At peak load a single browser context can hold something in the range of five to twenty open connections through the gateway, falling to one or two while idle. Budget with a multiplier, not one to one:

Treat that 10x figure as a planning heuristic, not a measurement of your target. Measure your own on the scraper host while a real run is in flight:

ss -tn state established '( dport = :11002 )' | wc -l

You can cut the number hard by refusing to download what you never parse:

await context.route('**/*.{png,jpg,jpeg,webp,gif,svg,woff,woff2,mp4}',
                    route => route.abort());

That drops connections, bytes, and page load time in one line. Be a little careful: blocking every last request is itself a signal on sites that check whether their tracking pixel fired, so block media and fonts, and leave the analytics call alone when the target is sensitive.

Whitelist slots matter more than they look. IP whitelisting binds to the machine's address, so if you spread browsers across thirty VMs and your plan carries five slots, use username and password authentication instead. That is also the only auth mode that can carry a session label, so for session-aware work it is the default anyway.

Pro and Pro+ tiers exist in the fair usage policy at 1500 and 2000 threads with 200 and 250 Mbps caps, and custom plans reach 1 Gbps. Those are quoted rather than listed publicly, so ask before you build a budget around them.

Sticky is a promise about the common case, not a physical guarantee. Gateways restart, TTLs expire, exits go unhealthy. Treat a mid-session IP change as a detectable fault rather than something you hope never happens.

async function exitIp(context) {
  const page = await context.newPage();
  await page.goto('https://ipinfo.io/json', { waitUntil: 'domcontentloaded' });
  const body = await page.textContent('pre, body');
  await page.close();
  return JSON.parse(body).ip;
}

// Wrap the flow: same context, same exit, or the run is void.
const before = await exitIp(ctx);
await runFlow(ctx);
const after = await exitIp(ctx);
if (before !== after) {
  await ctx.close();                 // burn the cookie jar along with it
  metrics.increment('session.rotated_midflight');
}

When the IP does change, do not carry on. Discard the context and its cookies and restart the flow under a fresh label, because a cookie jar built across two addresses is the exact artifact that gets accounts flagged. Track the mid-flight rotation rate as a first-class metric. A rate that climbs means your flows now run longer than the sticky TTL, which is a scheduling problem rather than a proxy problem. Handling cookies and sessions while scraping covers the state side of that.

Measure the TTL before you commit to a vendor. Poll one label every 30 seconds for an hour and log the first change:

for i in $(seq 1 120); do
  curl -s -x http://USER-session-ttltest:PASS@gateway.sparkproxy.io:11002 \
    https://ipinfo.io/json | grep -o '"ip": *"[^"]*"'
  sleep 30
done

If the exit holds for the full hour, sessions longer than your flows are safe. If it flips at minute seven, you now know the maximum length of any flow you can build on it, which is the single most useful number in the whole evaluation.

Running your own browsers buys control. A managed endpoint buys your time back. The split is cleaner than most vendors admit.

The arithmetic is easy to run. SparkProxy Scraping API Growth is $99 for 1,000,000 credits per month at 100 concurrent requests. A rendered page costs 5 credits, so that is 200,000 rendered pages, and a plain fetch is 1 credit when the target needs no browser. Starter is $49 for 250,000 credits at 50 concurrent, Pro is $249 for 3,000,000 at 200 concurrent, and Scale is $599 for 8,000,000 at 400 concurrent. Screenshots and PDFs cost 10.

Compare that against self-managing honestly: the proxy plan is the small line item and headless Chromium compute is usually the large one, plus the engineering hours to keep stealth current. If your workload is a few hundred thousand independent page renders a month, the managed path is almost always cheaper once you count salaries. If it runs to millions of renders, or your flows are long and stateful, owning the browsers wins. Scraping API versus self-managed proxies works through that comparison in more detail, and scraping dynamic JavaScript websites covers the rendering side.

The API takes the session decision off your plate for single-page work:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.sparkproxy.io/pricing" \
  --data-urlencode "render_js=true" \
  --data-urlencode "country_code=us"

There are 1,000 free credits with no card, which is enough to run the comparison below against a real target instead of a vendor demo page.

Set your acceptance bar before you run this, in writing, then hold to it. Deciding what counts as good after seeing the numbers is how teams talk themselves into the wrong vendor.

1. Get the trial or the free credits. Do not evaluate against a synthetic target; use the site you actually need.
2. Measure the sticky TTL with the polling loop above. Write the number down.
3. Run your real flow 50 times through sticky exits, one fresh label per run.
4. Count soft blocks separately from non-200 responses. Assert on a content marker, and treat a 200 with no marker as a failure.
5. Watch established connections during the run and record the peak. That is your thread requirement, not a guess.
6. Check geo coherence: do the rendered currency, language, and shipping estimate match the exit country you asked for?
7. Run the same 50 URLs through the managed API with render_js=true and compare success rate, wall-clock time, and total cost.

The output is four numbers: sticky TTL, soft-block rate, peak concurrent connections, and cost per successful page on each path. Any vendor that makes those four hard to collect is telling you something.