TCP Keepalive and Connection Pooling for Proxies

These share a name and nothing else. Getting them mixed up is why people enable one and expect the other's behaviour.

TCP keepalive is a transport-layer probe defined in RFC 9293, Section 3.8.4. When a connection has been idle, the stack sends a probe segment carrying no data, or one garbage byte, with a sequence number the peer has already acknowledged. A live peer answers with an ACK. A dead one answers with nothing, or with an RST. The specification is deliberately conservative: the keepalive interval must be configurable and must default to no less than two hours, and an implementation must not treat a single unanswered probe as proof the connection is dead.

HTTP keep-alive is an application-layer convention. HTTP/1.1 connections are persistent by default under RFC 9112, Section 9.3, which means the socket stays open after a response so the next request can use it. The Connection: keep-alive header is the HTTP/1.0 opt-in, still sent by many clients out of habit. It sends no probes and detects nothing. It just declines to close the socket.

That last row is worth reading twice. RFC 9110, Section 7.6.1 classifies Connection header options as hop-by-hop and requires intermediaries to strip them before forwarding. So when a forward proxy handles a plain http:// request, your Connection: keep-alive applies only to the leg between you and the proxy. Whether the proxy keeps its own connection to the origin alive is entirely the proxy's decision, and you cannot see it.

For https://, the picture flips. Your client opens a CONNECT tunnel, the proxy relays bytes without parsing them, and your HTTP keep-alive semantics run end-to-end with the origin. The proxy is not reading your headers, so it cannot strip them. This is why HTTPS-through-proxy reuse behaves more predictably than plain-HTTP reuse, and why almost every scraping stack sees steadier numbers on HTTPS targets.

TCP keepalive, meanwhile, never crosses the proxy. Your socket option applies to your socket. It probes the proxy gateway, not the origin. If the exit node's connection to the target dies, your client-to-proxy keepalive will keep answering happily while the far half of the path is already gone.

Here is the full setup sequence for one fresh HTTPS request through an HTTP proxy, with nothing cached:

1. TCP handshake to the proxy. SYN, SYN-ACK, ACK. One RTT on the client-to-proxy leg.
2. CONNECT tunnel setup. Request, credential validation, 200 Connection Established. One more RTT on the same leg. A 407 Proxy Authentication Required challenge and retry adds a third.
3. TLS handshake to the target. TLS 1.3 needs one RTT measured client-to-target through the tunnel, so it costs the client-to-proxy leg plus the proxy-to-target leg. TLS 1.2 needs two.

Three round trips minimum, four if the origin is still on TLS 1.2, five if your client waits for a 407 before sending credentials. Put concrete numbers on it. Client-to-proxy RTT of 30 ms, proxy-to-target RTT of 50 ms, end-to-end 80 ms:

TCP handshake to proxy      30 ms
CONNECT tunnel              30 ms
TLS 1.3 handshake           80 ms
---------------------------------
Setup before request byte  140 ms

Now the part that decides your job duration. Suppose the origin takes 300 ms to build the page.

The pooled rows include the amortised handshake: 140 ms divided across N requests. The saving is 31% of wall-clock time on identical infrastructure. Same proxy, same plan, same target. Nothing changed except that the socket survived.

Two things follow. First, the return on reuse is steeply front-loaded. Going from 1 request per connection to 50 captures 98% of the available saving, and going from 50 to 500 buys almost nothing. You do not need heroic connection lifetimes. You need to stop opening a new one every time. Second, the size of the win scales with the ratio of handshake to service time. A slow origin dilutes the benefit, a fast one magnifies it. Scraping a 60 ms JSON endpoint through a 140 ms setup means reuse more than doubles your throughput.

A connection pool is a keyed cache of open sockets. When your client finishes reading a response, instead of calling close(), it returns the socket to the pool. The next request that matches the key checks it out.

The key is the part people get wrong. In practice a pool entry is identified by the tuple:

(scheme, proxy host:port, proxy credentials, target host:port)

Change any element and you get a different pool. That includes the proxy credentials, which matters enormously with providers that encode the session or the target country in the username. Switching from user-country-de to user-country-fr is a different pool key, a different tunnel, and a fresh handshake, even though the gateway hostname never changed.

Do not confuse this with a proxy pool. A proxy pool is a set of IP addresses you can exit from. A connection pool is a set of open TCP sockets your process is holding. They interact, and badly, which is the subject of the last two sections here.

Two limits appear in nearly every HTTP client, and they mean different things:

The default that catches most people: Go's http.DefaultTransport sets MaxIdleConnsPerHost to 2. Run 50 goroutines against one host and 48 of them open and discard a connection on every request while the pool quietly holds two. The symptom is a scraper that gets slower per request as you add workers, and it looks exactly like proxy throttling.

Stop guessing at pool numbers. Little's Law gives the answer directly:

Connections needed = arrival rate (requests per second) x average service time (seconds)

Service time here is the full in-flight duration of a request, TTFB plus body transfer, not just the network hop. Worked example: you want 50 requests per second sustained, and your measured p50 service time through the proxy is 400 ms.

50 req/s x 0.4 s = 20 connections in flight

Twenty. Then add headroom for variance, because p50 is not p95. A 20% to 50% margin over the Little's Law figure covers normal jitter without waste:

import math

TARGET_RPS = 50
P50_SERVICE_S = 0.40
HEADROOM = 1.25
PROVIDER_CONCURRENCY_CAP = 100

base = TARGET_RPS * P50_SERVICE_S
pool_size = min(math.ceil(base * HEADROOM), PROVIDER_CONCURRENCY_CAP)
print(pool_size)   # 25

That cap on the last line is not optional. Every provider plan has a ceiling on simultaneous open connections, and a pool larger than the ceiling does not give you more throughput, it gives you refused connections and 429 responses. The mechanics of those limits, and how they differ from thread counts, are covered in understanding concurrent connections in proxies.

An undersized pool announces itself clearly in Python. urllib3 logs this when the pool is full and a returned connection has nowhere to go:

WARNING urllib3.connectionpool: Connection pool is full, discarding connection:
gateway.sparkproxy.io. Connection pool size: 10

Read that carefully. It does not mean requests were blocked. It means a perfectly good warm socket was thrown away because there was no slot to store it, so the next request pays a full handshake. By default requests does not block when the pool is exhausted, it opens an extra connection, uses it, and discards it. You lose reuse silently while the request count looks fine. If you want the pool to be an actual limit rather than a suggestion, mount an adapter with pool_block=True and let callers queue.

The intuition that a bigger pool cannot hurt is wrong in four specific ways, and the first one is the expensive one.

Idle sockets still count against your concurrency quota. Provider gateways count open TCP connections, not active HTTP requests. A pool holding 200 sockets when you are only running 25 requests at a time is consuming 200 slots of a plan you are paying for. If your plan caps you at 100, you have locked yourself out of your own capacity with connections that are doing nothing. This is the most common cause of "we bought a bigger plan and throughput did not move."

Stale draws rise with pool size. A socket's chance of having been reaped somewhere in the path scales with how long it sat idle. Larger pool, lower request rate per socket, longer average idle time, higher probability that the connection you check out is already dead. A 200-socket pool serving 25 requests per second gives each socket an 8 second average rest. A 2,000-socket pool gives it 80 seconds, which is past the idle timeout of plenty of load balancers.

Ephemeral port exhaustion. Linux defaults to the range 32768 to 60999, about 28,000 source ports. Sockets in TIME_WAIT hold their port for 60 seconds after close. A pool that churns aggressively against one gateway can reach EADDRNOTAVAIL. Check your range with sysctl net.ipv4.ip_local_port_range.

Memory and file descriptors. Each open TLS connection carries kernel socket buffers plus userspace TLS state, commonly 30 to 60 KB in total. Ten thousand idle connections is several hundred megabytes doing nothing, plus 10,000 file descriptors against a default ulimit -n of 1024.

The honest summary: pool size is a Goldilocks parameter with a real optimum, and both directions of error produce the same visible symptom, which is throughput that refuses to scale with worker count.

A half-open connection is one where your side still believes the state is ESTABLISHED and the peer no longer has any record of it. TCP has no heartbeat by default, so an idle connection is indistinguishable from a healthy one until you write to it.

Then one of three things happens:

• The peer sends RST and you get ConnectionResetError: [Errno 104] Connection reset by peer.
• The peer closed cleanly while you were idle and you get RemoteDisconnected('Remote end closed connection without response').
• A middlebox dropped its state table entry without telling anyone, so your packets vanish and the request hangs until your read timeout fires. This is the expensive case, because it costs the full timeout instead of an instant error.

Through a proxy you are exposed to more state tables than usual. A residential request can traverse your local NAT, the provider's gateway load balancer, the exit node's home router, and carrier-grade NAT, each with its own idle timer. The shortest timeout in the chain governs your safe idle period, and you control almost none of them.

Published values worth calibrating against:

Note the gap between the standard and reality. RFC 5382 says a NAT must not abandon an established TCP session before 2 hours 4 minutes of idleness, but consumer equipment and CGNAT deployments routinely ignore that, and residential exit nodes sit behind exactly that equipment. Treat anything over 60 seconds of idle time on a residential path as an assumption rather than a guarantee.

The practical fix is a ladder, applied in order:

1. Set your pool's idle expiry below the shortest timeout you can observe. If you know an NLB is in the path, expire idle sockets at 300 s rather than 350 s. When you do not know, 60 s is a safe default that still captures nearly all the reuse benefit.
2. Enable TCP keepalive with a short interval so the connection generates traffic and the state tables stay warm. Details in the next section.
3. Retry once on a connection error when the socket came from the pool. A reset on a checked-out connection is an infrastructure event, not an application failure, and it deserves an immediate retry on a fresh socket rather than a backoff. Wider failure-mode planning is in understanding proxy uptime and reliability.

Idempotency matters at step 3. Retrying a GET on a reset is free. Retrying a POST that may already have reached the origin is a duplicate write, so key those with an idempotency token or accept a lower retry ceiling.

The Linux defaults make keepalive useless for proxy work. Check them:

sysctl net.ipv4.tcp_keepalive_time    # 7200  seconds before the first probe
sysctl net.ipv4.tcp_keepalive_intvl   # 75    seconds between probes
sysctl net.ipv4.tcp_keepalive_probes  # 9     unanswered probes before giving up

Worst-case detection time is 7200 + (9 x 75) seconds, which is 7,875 seconds, roughly 2 hours 11 minutes. Every NAT in your path will have reaped the connection long before the first probe is even sent. That is not a bug, it is RFC 9293 conformance, and it is why keepalive has a reputation for not working.

Set it per socket instead of globally, so you tune the connections that matter without touching the whole host. In Python, requests and urllib3 do not enable SO_KEEPALIVE by default, so you mount an adapter:

import socket
import requests
from requests.adapters import HTTPAdapter

class KeepAliveAdapter(HTTPAdapter):
    """TCP keepalive tuned for proxy paths with short NAT timers."""
    def init_poolmanager(self, connections, maxsize, block=False, **kw):
        kw["socket_options"] = [
            (socket.IPPROTO_TCP, socket.TCP_NODELAY, 1),
            (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),
            (socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 60),   # first probe after 60s idle
            (socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 10),  # then every 10s
            (socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 3),     # dead after 3 misses
        ]
        return super().init_poolmanager(connections, maxsize, block=block, **kw)

session = requests.Session()
session.proxies = {
    "http":  "http://user:pass@gateway.sparkproxy.io:8000",
    "https": "http://user:pass@gateway.sparkproxy.io:8000",
}
session.mount("https://", KeepAliveAdapter(pool_connections=10, pool_maxsize=25))

Detection now takes 60 + 30 = 90 seconds instead of 2 hours 11 minutes. TCP_KEEPIDLE is Linux only. On macOS the equivalent is TCP_KEEPALIVE and it takes the idle time directly, so guard the option list by platform if your code runs in both places.

Defaults across runtimes vary more than people expect:

The Node change is worth knowing if you maintain older services. Before Node 19 the global agent had keepAlive: false, so every http.request opened a new socket unless you passed your own agent. Upgrading the runtime silently fixed a throughput problem in a lot of codebases. Async Python clients need their own attention too, since limits are set per client instance rather than globally, and the patterns for that are in using proxies with Python requests and aiohttp.

Here is the conflict nobody prices. A pooled connection is a tunnel to one exit IP. If your configuration assigns a new exit IP on every request, the old tunnel is useless by definition, and you pay the full handshake every time. Per-request rotation and connection reuse are mutually exclusive. You cannot have both, and most providers do not tell you which one you have selected.

The tax has a closed form. With handshake cost H and service time T:

per-request rotation:   H + T          per request
sticky, N per session:  T + (H / N)    per request

throughput tax = H / (H + T)

At H = 140 ms and T = 300 ms, the tax is 140 / 440, which is 31.8%. Roughly a third of your capacity goes to re-establishing tunnels you already had. On a fast API target where T = 80 ms, the same handshake costs you 63.6%.

Read the bottom row as permission. If you are rendering JavaScript and each page takes two and a half seconds, per-request rotation costs you almost nothing and you should rotate freely. Read the top row as a warning: hammering a fast JSON endpoint with per-request rotation means you are paying for roughly three requests to complete one. The general mechanics of rotation strategies are covered in what proxy rotation is and how it works.

With SparkProxy's Scraping API the choice is one parameter. Passing a session_id pins the exit IP across calls so the underlying tunnel is reused, and omitting it lets the API pick a fresh IP per request:

# Sticky: same exit IP and warm connection across the whole crawl
curl -s -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: sk-your-key" \
  --data-urlencode "url=https://www.sparkproxy.io/pricing" \
  --data-urlencode "session_id=crawl-2026-08-18" \
  --data-urlencode "country_code=de" \
  --data-urlencode "premium_proxy=true"

# Rotating: new exit IP per call, full handshake per call
curl -s -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: sk-your-key" \
  --data-urlencode "url=https://www.sparkproxy.io/pricing" \
  --data-urlencode "country_code=de" \
  --data-urlencode "premium_proxy=true"

The full parameter reference lives in the SparkProxy Scraping API docs.

Rotation exists for a reason, so the question is not which is faster but which produces more successful responses per unit of time. Compare effective cost per success. Let b be the block rate under per-request rotation and B the block rate under sticky sessions:

rotation cost per success = (H + T) / (1 - b)
sticky cost per success   = (T + H/N) / (1 - B)

With a reasonably long session, H/N rounds to zero, and sticky wins whenever:

(1 - B) / (1 - b) > T / (H + T)

Sticky sessions with reuse beat per-request rotation while your sticky success rate stays above T / (H + T) of your rotating success rate. Plug in the common case, H = 140 ms and T = 300 ms: the threshold is 0.682. If rotation gets you a 100% success rate, sticky still wins as long as it succeeds more than 68% of the time. You can give up 32 percentage points of success rate and still finish more pages per hour.

Measure both arms against the same target for an hour and compute successes per hour, not success rate. Success rate alone will always favour rotation and always mislead you, because it ignores the time cost of the handshakes rotation forces.

Three cases where the rule does not apply and rotation wins regardless of the arithmetic:

• The target rate-limits per IP. If a single IP is cut off after 20 requests, session length is capped externally and reuse has nowhere to go. Set your session length just under the observed limit and take the partial benefit.
• Login or cart state must not leak between accounts. One identity per session is a correctness requirement, not a performance choice.
• The target scores IP behaviour over time. Some anti-bot systems flag steady per-IP request rates that no human produces, and a long-lived session is exactly that signal.

The middle ground that works for most crawls: sticky sessions of 30 to 100 requests, rotated on a fixed count or on the first 403, with the connection pool sized to the number of concurrent sessions rather than the number of workers. You amortise the handshake across a useful run and still refresh the IP often enough to stay unremarkable. The window mechanics and typical durations are in what a sticky session proxy is.