Using Proxies for Distributed Load Testing

Distributed load generation is indistinguishable from a distributed denial of service attack at the packet level. The only thing that separates the two is authorisation. So authorisation is the first artefact of the test, not a formality you handle afterwards.

The rule is simple: you may load test infrastructure you own, or infrastructure you hold written authorisation to test. Nothing else. Sending synthetic load at a third party without permission is a criminal offence in most jurisdictions, including under 18 U.S.C. section 1030(a)(5) in the United States, sections 3 and 3A of the UK Computer Misuse Act 1990, and national laws implementing EU Directive 2013/40/EU on attacks against information systems. It also violates the SparkProxy Acceptable Use Policy, which prohibits unauthorised penetration testing and any form of cyberattack against third-party systems. Accounts used that way get terminated.

What a real authorisation package contains

Teams that have been through a security review already have this. If you do not, build it once and reuse it:

• Written scope. Named hostnames, IP ranges and paths. Anything not listed is out of scope, including third-party endpoints your own pages call. That last one catches people out: a checkout flow that posts to a payment provider will hammer the payment provider unless you stub it.
• A signed authorisation from whoever owns the system, with a start and end time. Verbal approval in a standup is not an authorisation.
• Named contacts on both sides, reachable during the window, with a kill switch procedure and a stated maximum time to abort.
• Advance notice to your hosting provider, CDN and bot mitigation vendor. Many require it. AWS, for example, asks customers to submit a Simulated Events form before network stress tests that push past roughly 1 Gbps sustained. Check the current policy for every provider in your path before you plan the run, because the thresholds move.
• A shared-infrastructure check. On shared hosting or a shared database cluster, your authorised test can take down somebody else's authorised workload. That is still an incident.

Then start small. A smoke pass at 1 percent of expected peak proves the wiring works before you have committed to anything you cannot stop quickly.

Run 500 virtual users from one cloud instance and three defence layers activate before your application code does any meaningful work.

Per-IP rate limits fire first

Most origins throttle by client address. An nginx limit_req_zone $binary_remote_addr bucket set at 100 requests per second per IP starts returning 503 at request 101 no matter how idle the app is. Your graph shows a hard ceiling at exactly 100 rps and a wall of errors above it. That is a correct measurement of the limiter and a useless measurement of the application. The same applies to API gateway quotas, Redis-backed token buckets keyed on IP, and fail2ban-style ban rules that will blackhole your generator halfway through the run.

CDN edge caching flatters the result

Every request from one IP resolves to one POP. After the first miss, the edge serves from cache, hit ratio climbs toward 100 percent, and origin sees almost nothing. You have load tested a cache that already worked. Real traffic arrives at 30 or 40 POPs with independent cache states, so the origin fill rate under real load can be an order of magnitude higher than your test suggested.

WAF and bot management distort the tail

Sustained high-rate traffic from a single address, especially from a well-known cloud ASN, triggers challenges, tarpitting or outright blocking. Response times balloon, p99 goes vertical, and the natural conclusion is that the application degrades under load. It does not. It is being slowed on purpose by a device in front of it.

There is a fourth failure that is not the target's fault at all. A single Linux load generator runs out of ephemeral ports at around 28,000 concurrent outbound connections on the default net.ipv4.ip_local_port_range of 32768 to 60999, and conntrack tables fill well before that. Sockets stuck in TIME_WAIT make it worse. You hit a client-side ceiling and record it as a server-side one.

Distributing across IPs solves the rate-limit and cache problems. Distributing across countries solves a different one: the path itself is part of your product's performance.

Round-trip time is not a detail you can add back later. A TLS 1.3 handshake costs two round trips on a fresh connection, TLS 1.2 costs three. At a 230 ms RTT between Sydney and Frankfurt, that is 460 to 690 ms before a single byte of your response moves. Your Frankfurt-to-Frankfurt test measured 40 ms and told you nothing about the Australian experience.

Regional distribution also exposes routing behaviour a local test cannot reach:

• Geo-DNS and anycast send different regions to different POPs and sometimes to different origin clusters entirely.
• Read replicas in one region may be lagging or under-provisioned relative to the primary.
• Regional feature flags, currencies, consent banners and localisation middleware change the response size and the render path. A cookie-consent interstitial in the EU adds a request and a repaint that your US test never saw. If that is your main concern, proxies for localization testing covers the QA side of it in detail.

Weight your regional mix by real traffic, not evenly. Pull the country breakdown from analytics and reproduce it. If 62 percent of sessions are US, 18 percent UK and 9 percent Germany, a test that splits load equally across six countries is a different experiment from the one you meant to run. SparkProxy covers 80+ countries, so the constraint is your analytics data rather than availability. Country selection through a proxy gateway works the same way it does for any geo-targeted request, which what geo-targeting means in proxies explains from the routing side.

Load testing is the one proxy workload where you are not hiding from anybody. You own the target. You told them you were coming. What you need is stable, high-throughput, predictable egress across many addresses, which is exactly what datacenter proxies are for.

Residential IPs are the wrong tool here and it is worth saying plainly why: they add jitter you cannot separate from application latency, they are metered in a way that makes sustained load expensive, and pushing test traffic through consumer connections is not something to do at volume. SparkProxy is datacenter only with unlimited bandwidth, which is the combination load testing actually needs, since a serious run moves tens of gigabytes and per-GB pricing makes that unaffordable before it makes it slow.

There is exactly one host, gateway.sparkproxy.io, and three ports:

The rotating and sticky distinction matters more in load testing than people expect. A stateless read benchmark wants port 11000: every request from a different address, maximum spread across your rate limiters and edge nodes. A journey test that logs in, adds to a cart and checks out needs the virtual user to look like one visitor for the duration, or you will trip session-fixation defences, break sticky-session load balancing at the origin, and generate a login storm that is not in your traffic model. Route those virtual users through port 11002.

Port 13000 speaks SOCKS5 over TCP. That covers HTTP, HTTPS, WebSocket, raw TCP protocols and JVM tooling that prefers a SOCKS proxy. It does not carry UDP, so QUIC and HTTP/3 load generation will not traverse it. Test HTTP/3 endpoints directly from regional generators instead.

Two configuration notes that save an afternoon. First, an HTTP proxy on port 11000 uses CONNECT for HTTPS, so TLS stays end to end and certificate validation still works. If your tool suddenly demands --insecure, something else is wrong. Second, decide between credential auth and IP whitelisting before you scale out ephemeral generators, because whitelisting a fleet of autoscaled runners is painful. How proxy authentication works covers the tradeoff.

This is the part that ruins otherwise good tests, so it gets the most space.

A proxy plan allocates threads, meaning concurrent open connections through the gateway, plus a bandwidth ceiling. Neither is a requests-per-second number. Converting between them is Little's Law, and it is the single most useful formula in this workload:

concurrency = throughput x average response time

# 3,000 rps at a 400 ms mean response time
concurrency = 3000 x 0.4 = 1,200 concurrent connections

Invert it to read a plan as a throughput ceiling. At a 250 ms mean response time each thread completes 4 requests per second, so the thread-bound ceiling is threads multiplied by 4. Then compute the bandwidth-bound ceiling separately, because it is almost always the lower of the two:

Mbps required = rps x average response bytes x 8 / 1,000,000

# 2,000 rps at a 40 KB average response
2000 x 40960 x 8 / 1e6 = 655 Mbps

Read the two right-hand columns against the fourth. For a 40 KB HTML page, bandwidth binds far below the thread ceiling on every tier. For a 2 KB JSON API response, threads and bandwidth land in roughly the same place. Your effective ceiling is the lower of the two, and it is a property of your payload, not of the plan alone.

The practical consequence: test lean endpoints through the proxy. Hit your JSON APIs, enable gzip or brotli, and block static assets in the generator rather than downloading every image 200,000 times. Full-page fetches with all sub-resources belong in a small browser-based test alongside the protocol-level one, not in the volume run.

The failure mode this prevents

Exceeding your allocated threads may lead to bandwidth restriction under the Fair Usage Policy. When that happens mid-test the run does not fail loudly. It plateaus. You add virtual users, throughput stays flat, connect times climb, and you write up a conclusion like "the application saturates at 850 rps" that is entirely about the proxy tier.

The tell is a disagreement between the two ends of the pipe. If your generator reports 850 rps and rising latency while origin access logs show 850 rps at unchanged server-side latency and 20 percent CPU, the bottleneck is between them. Instrument both ends and compare request counts before you believe any ceiling. Understanding concurrent connections in proxies goes deeper on what counts as a connection.

One more counting trap. A thread is an open connection, not a virtual user. With HTTP keep-alive, one VU holds roughly one connection and the two numbers match. Turn keep-alive off, or use a tool that opens a fresh connection per request, and actual concurrency can run several times your VU count while connections churn through TIME_WAIT. Check your tool's connection reuse setting before sizing anything. TCP keepalive and connection pooling for proxies covers the mechanics.

All four accept a proxy. They differ in how much per-user control they give you.

k6

k6 has no in-script proxy option. It honours the standard Go environment variables, which is enough for most runs:

export HTTPS_PROXY="http://USERNAME:PASSWORD@gateway.sparkproxy.io:11000"
export HTTP_PROXY="$HTTPS_PROXY"
export NO_PROXY="localhost,127.0.0.1"

k6 run --vus 500 --duration 10m load.js

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '2m', target: 100 },   // ramp
    { duration: '5m', target: 500 },   // hold
    { duration: '2m', target: 0 },     // recover
  ],
  thresholds: {
    // Keep transport failures separate from application failures.
    'http_req_failed': ['rate<0.01'],
    'http_req_duration{expected_response:true}': ['p(95)<800'],
    'http_req_connecting': ['p(95)<200'],
  },
};

export default function () {
  const res = http.get('https://staging.sparkproxy.io/api/v1/status', {
    tags: { endpoint: 'status' },
  });
  check(res, { 'status 200': (r) => r.status === 200 });
  sleep(1);
}

Watch http_req_connecting and http_req_tls_handshaking separately from http_req_waiting. The first two are the proxy path. The third is your server.

For several proxy configurations in one run, launch one k6 process per configuration with its own environment, or use the k6 Operator to schedule regional runners. There is no per-VU proxy assignment.

JMeter

JMeter takes the proxy on the command line, which keeps credentials out of the committed JMX file:

jmeter -n -t plan.jmx -l results.jtl \
  -H gateway.sparkproxy.io -P 11000 \
  -u USERNAME -a PASSWORD \
  -N "localhost|127.0.0.1" \
  -Jsummariser.interval=10

For SOCKS5 on port 13000, use JVM system properties instead:

export JVM_ARGS="-DsocksProxyHost=gateway.sparkproxy.io -DsocksProxyPort=13000 -DsocksProxyVersion=5"
jmeter -n -t plan.jmx -l results.jtl

Different Thread Groups can carry different HTTP Request Defaults, so JMeter is the easiest of the four for splitting one plan across the rotating and sticky ports.

Locust

Locust's client is a requests.Session subclass, so a proxies dict set on it applies to every request that user makes. That gives genuine per-user proxy control:

import os
from locust import HttpUser, task, between

PROXY = os.environ["SPARK_PROXY"]  # http://USER:PASS@gateway.sparkproxy.io:11002

class CheckoutUser(HttpUser):
    host = "https://staging.sparkproxy.io"
    wait_time = between(1, 4)

    def on_start(self):
        # Sticky port: this user keeps one exit IP for its whole session.
        self.client.proxies = {"http": PROXY, "https": PROXY}
        self.client.trust_env = False      # ignore ambient env proxies
        self.client.verify = True          # CONNECT tunnel, TLS still end to end

    @task(3)
    def browse(self):
        self.client.get("/catalog", name="GET /catalog")

    @task(1)
    def add_to_cart(self):
        self.client.post("/cart", json={"sku": "SP-1001", "qty": 1}, name="POST /cart")

Install requests[socks] if you want port 13000, then use a socks5h:// scheme so DNS resolves at the proxy rather than on the generator. Run distributed with locust --master and workers started with a different SPARK_PROXY value per region.

Gatling

Gatling exposes the proxy on the HTTP protocol object:

val httpProtocol = http
  .baseUrl("https://staging.sparkproxy.io")
  .proxy(
    Proxy("gateway.sparkproxy.io", 11000).http
      .credentials("USERNAME", "PASSWORD")
  )
  .shareConnections   // reuse connections so VUs map ~1:1 to threads

Swap .http for .socks5 to use port 13000. Set shareConnections deliberately: without it Gatling can open far more connections than you have virtual users, which is the keep-alive trap from the previous section.

Verifying what the target actually served

Protocol-level generators do not render pages, so they cannot tell you whether the site still looked right at peak. A cheap out-of-band canary during the run covers that gap. The SparkProxy Scraping API can fetch and screenshot your own page from a chosen country while the test is in flight:

curl -s "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  -G \
  --data-urlencode "url=https://staging.sparkproxy.io/catalog" \
  --data-urlencode "country_code=DE" \
  --data-urlencode "render_js=true" \
  --data-urlencode "wait_for=.product-grid" \
  --data-urlencode "format=screenshot" \
  --output canary-de-peak.jpg

Fire it once a minute against your own staging host. If the screenshot degrades before the latency graph does, the failure is in a dependency the protocol test never called.

You cannot attribute latency you have not measured separately. Run three passes before the real test, all at low concurrency where nothing is saturated:

1. Direct, same network. Generator inside the same VPC or region as the origin. This is the application's floor.
2. Direct, from the generator's own public IP. Adds the real internet path.
3. Through the proxy, same low concurrency. The delta against pass 2 is your proxy overhead.

Use a trivial endpoint for all three, something like /healthz returning a fixed short body, so the application contributes as close to zero as possible. Record the full distribution rather than a mean: p50, p95 and p99. Overhead through a well-provisioned datacenter gateway is usually a small and stable number, but the p99 is what shows up in your headline result, and it is the one worth knowing. Understanding proxy latency and speed breaks down where the milliseconds go.

Percentiles do not subtract

This is the mistake that survives peer review because it looks like arithmetic. You measured p95 proxy overhead at 55 ms in the baseline. Your loaded p95 is 900 ms. It is tempting to report application p95 as 845 ms.

That number is wrong. The p95 of a sum is not the sum of the p95s. Those two percentiles come from different distributions whose tails do not line up, and the request that was slowest through the proxy is generally not the request that was slowest inside the application. Depending on the correlation between them, subtracting can overshoot or undershoot by a wide margin. Medians survive the treatment slightly better, and only when the two components are close to independent.

The fix is to stop inferring and start measuring. Log server-side timings at the origin and treat those as ground truth for application latency:

log_format timing '$remote_addr $status $body_bytes_sent '
                  'rt=$request_time urt=$upstream_response_time '
                  'ua="$upstream_addr" ct=$upstream_connect_time';

access_log /var/log/nginx/load-test.log timing;

$upstream_response_time is your application. $request_time includes reading the client request and writing the response back, so the gap between the two is the network side. Join those logs to the generator's per-request records using a correlation header you inject from the test, and you get the real split per request instead of a subtraction across distributions.

Before you compute an error rate, classify the errors. A run reported as "0.8 percent errors" is meaningless if half of them were the transport.

Retry logic deserves a deliberate decision rather than a default. Aggressive client retries inflate your apparent request rate and hide the moment of failure, and a retry storm inside the generator can produce a load curve that never existed. Log retries as a separate series so they cannot silently pad throughput. Proxy timeouts and retry logic covers sensible backoff for the transport half.

Three more habits make results defensible. Report percentiles rather than averages, because a mean response time hides the exact tail your users complain about. State the concurrency at which each number was taken, since "2,000 rps" without a VU count and a response-time distribution is not a result. And record the breaking point as a layer, not a number: "database connection pool exhausted at 1,150 concurrent users" is actionable in a way that "peak 4,600 rps" never is.

A load test answers one question well: does this fall over, and where. It is not a substitute for application performance monitoring, and it does not model reality.

It is not APM. Load testing gives you black-box timings from outside. It cannot tell you that 60 percent of the latency is one N+1 query in the catalogue serializer. Distributed traces can. Run the load test with tracing enabled and read the traces, or you will find the cliff without finding the cause.

Synthetic traffic is not real user behaviour. Your script visits a handful of paths in a fixed ratio with a fixed think time. Real users have a long-tailed page popularity distribution, unpredictable pauses, abandoned sessions, back-button behaviour, browser caches in every state, and a device and network mix your generators do not have. A test that is 100 percent cache-cold or 100 percent cache-warm is wrong in a different direction from real traffic.

It says nothing about growth over time. A query that returns in 12 ms against 100,000 rows and does a full scan at 10 million rows will pass every load test you run today. Data volume, not concurrency, is the failure.

Protocol-level tests skip the browser. Real page load includes third-party tags, fonts, hydration and main-thread work. Field metrics such as INP and LCP come from real user monitoring, not from a request generator.

Use load testing to find capacity limits and break points. Use APM to explain them. Use RUM to know whether any of it matters to actual users. Then run a soak test at 60 to 70 percent of peak for several hours, because memory leaks, connection pool drift and log-disk exhaustion only appear with time on the clock.

1. Write the scope and get it signed. Hostnames, paths, time window, named contacts, kill switch, abort criteria.
2. Notify your host, CDN and bot mitigation vendor. Ask what they need and how much lead time.
3. Pull the country mix from analytics. Set your regional weights from real numbers.
4. Compute required concurrency with Little's Law, then compute the bandwidth ceiling for your actual payload size. Pick the plan against the lower of the two, with headroom.
5. Baseline at low concurrency: direct in-region, direct public, then through the proxy. Record p50, p95 and p99 for each.
6. Turn on server-side timing logs and inject a correlation header so you can join both ends per request.
7. Ramp in stages at 1, 10, 25, 50, 100 and 150 percent of expected peak. Note which layer degrades first at each step. That list is the deliverable, not the peak number.
8. Classify every error as transport or application before computing any rate. Then repeat the run once, because a single load test is an anecdote.

If you are sizing a first run, the SparkProxy 24-hour trial is enough time to baseline, calibrate concurrency against the thread ceiling, and confirm your tooling actually routes through the gateway before you commit to a plan.