aiohttp ClientProxyConnectionError: Fix Every Proxy Error

aiohttp's proxy exceptions don't share one parent class, and that's the root of most broken retry code. ClientProxyConnectionError inherits from ClientConnectorError, which is a ClientConnectionError. ClientHttpProxyError inherits from ClientResponseError instead, so an except aiohttp.ClientConnectionError block never sees a 407. The SOCKS errors from aiohttp-socks don't inherit from aiohttp.ClientError at all. We printed each class's MRO from the installed packages to confirm this.

Two details in those strings mislead people. The ssl:default fragment describes the request's TLS setting, not TLS to the proxy, so it appears even for a plain http:// proxy. And the timeout message names the target URL rather than the proxy, so the obvious suspect is a website that is perfectly fine. On Linux the bracketed OS text differs (a refusal is errno 111, Connection refused); we didn't reproduce the Linux wording, so match on the class name, not the bracket.

Test each layer a request crosses, in order, and stop at the first failure: DNS for the proxy host, then a raw TCP connect, then the CONNECT through aiohttp. The session below sets trust_env=False explicitly so nothing from your environment muddies the result.

import asyncio, os, socket, sys

import aiohttp

HOST = os.environ.get("SPARK_HOST", "gateway.sparkproxy.io")
PORT = int(os.environ.get("SPARK_PORT", "11000"))
USER, PASS = os.environ.get("SPARK_USER", ""), os.environ.get("SPARK_PASS", "")
TARGET = "https://www.sparkproxy.io/robots.txt"


async def main() -> None:
    loop = asyncio.get_running_loop()
    try:
        info = await loop.getaddrinfo(HOST, PORT, type=socket.SOCK_STREAM)
        print("1 DNS   ok", info[0][4][0])
    except socket.gaierror as e:
        sys.exit(f"1 DNS   FAIL {e} -> hostname typo, VPN or container resolver")
    try:
        _, writer = await asyncio.wait_for(asyncio.open_connection(HOST, PORT), 5)
        writer.close()
        await writer.wait_closed()
        print("2 TCP   ok")
    except (OSError, asyncio.TimeoutError) as e:
        sys.exit(f"2 TCP   FAIL {e!r} -> refused: wrong port | timeout: firewall")
    hdrs = {"Proxy-Authorization": aiohttp.encode_basic_auth(USER, PASS)} if USER else None
    timeout = aiohttp.ClientTimeout(total=30, connect=10)
    async with aiohttp.ClientSession(timeout=timeout, trust_env=False) as s:
        try:
            async with s.get(TARGET, proxy=f"http://{HOST}:{PORT}", proxy_headers=hdrs) as r:
                print("3 PROXY ok, target answered", r.status)
        except aiohttp.ClientHttpProxyError as e:
            sys.exit(f"3 PROXY FAIL {e.status} {e.message} -> credentials or IP whitelist")
        except aiohttp.ClientError as e:
            sys.exit(f"4 TARGET FAIL {type(e).__name__}: {e}")


asyncio.run(main())

Four real runs, changing one environment variable each time:

1 DNS   ok 65.109.70.130
2 TCP   ok
3 PROXY FAIL 407 Proxy Authentication Required -> credentials or IP whitelist

1 DNS   ok 65.109.70.130
2 TCP   FAIL ConnectionRefusedError(22, 'The remote computer refused the network connection', None, 1225, None) -> refused: wrong port | timeout: firewall

1 DNS   FAIL [Errno 11001] getaddrinfo failed -> hostname typo, VPN or container resolver

1 DNS   ok 65.109.70.130
2 TCP   ok
4 TARGET FAIL ServerDisconnectedError: Server disconnected

The first run is the default port with no credentials. The second used port 11009, the third a misspelled hostname, and the fourth pointed HTTP at the SOCKS5 port 13000. The first one already answers "is the proxy down?": it's up, reachable and replying. It simply doesn't know who we are.

aiohttp raises ClientProxyConnectionError from exactly one place: the TCP connect to the proxy, in _create_proxy_connection, which passes client_error=ClientProxyConnectionError to the direct-connection helper. If you see this class, the TCP handshake to the proxy host never completed. A refusal comes back instantly, because a machine answered and said nothing listens on that port.

On SparkProxy the host is always gateway.sparkproxy.io, and the port picks the mode: Random Rotation lives on 11000, Sticky Sessions (one exit IP kept for five minutes) on 11002, and SOCKS5 on 13000. Any other number, typically copied from another provider's sample code, produces the refusal string in the table.

The second cause is the scheme inside the proxy URL. https:// there tells aiohttp to speak TLS to the proxy itself, and a plain HTTP proxy answers in cleartext, so OpenSSL reports WRONG_VERSION_NUMBER as a ClientConnectorSSLError (a sibling class, but the same fix). HTTPS targets still travel encrypted with an http:// proxy URL, because aiohttp opens a CONNECT tunnel first; the CONNECT method explained walks through that handshake.

import asyncio

import aiohttp

PROXY = "http://gateway.sparkproxy.io:11000"   # http:// even for https:// targets


async def main() -> None:
    timeout = aiohttp.ClientTimeout(total=30, connect=10)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        async with session.get("https://www.sparkproxy.io/robots.txt", proxy=PROXY) as r:
            print(r.status, (await r.text())[:80])


asyncio.run(main())

The third cause of a surprise ClientProxyConnectionError is a proxy you didn't pass at all. In our run with HTTPS_PROXY=http://127.0.0.1:9 set and trust_env=True, the error named 127.0.0.1:9. If the host in the message isn't one you recognise, skip to the trust_env section.

Code that catches only ClientProxyConnectionError misses two proxy-side failures.

DNS. When the proxy hostname doesn't resolve, aiohttp raises ClientConnectorDNSError, which is a ClientConnectorError but not a ClientProxyConnectionError. We checked with issubclass() and got False. Check the name from the same machine: nslookup gateway.sparkproxy.io should return an address (ours was 65.109.70.130). Inside Docker, read /etc/resolv.conf in the container, not on the host.

Timeouts. A blackholed proxy address (we used 10.255.255.1) raised ConnectionTimeoutError: Connection timeout to host https://www.sparkproxy.io/robots.txt with no mention of the proxy. Egress firewalls that only allow ports 80 and 443 are the usual reason; Test-NetConnection gateway.sparkproxy.io -Port 11000 on Windows or nc -vz -w 5 gateway.sparkproxy.io 11000 elsewhere settles it without Python. A longer connect timeout won't get through a firewall drop. For sizing connect and read timeouts separately, see proxy timeouts and retry logic.

aiohttp's default session timeout is ClientTimeout(total=300, sock_connect=30), read from client.py in 3.14.5. Five minutes is a long time for a dead proxy to hold an asyncio task, so set your own.

aiohttp.client_exceptions.ClientHttpProxyError: 407, message='Proxy Authentication Required', url='http://gateway.sparkproxy.io:11000'

That's SparkProxy's gateway answering our unauthenticated, non-whitelisted CONNECT. Port 11002 replied the same way. The exception carries the proxy's response headers in e.headers, and ours included Proxy-Authenticate: Basic realm="SparkProxy" and X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0, which confirm an auth refusal rather than a network fault.

Plain http:// targets behave differently. The same request to http://www.sparkproxy.io/robots.txt raised nothing: aiohttp returned a normal response with status 407. Code that skips raise_for_status() then parses the proxy's error page as data. HTTPS raises, HTTP returns; both are one auth problem.

SparkProxy supports two auth methods, so there are two fixes:

• IP whitelist. Add the public egress IP of the machine running the scraper. It breaks when that IP changes: a cloud instance restarted without a static IP, a CI runner, a VPN switched on.
• Username and password. Percent-encode them. A raw @, : or # in the password made aiohttp raise InvalidURL before any connection; urllib.parse.quote(pw, safe='') fixed it, and our fake proxy decoded the original p@ss:w/rd#1.

One version note: aiohttp 3.13.3 (2026-01-03) fixed "proxy authorization headers not being passed when reusing a connection, which caused 407 (Proxy authentication required) errors" (issue #2596 in the changelog). If you get 407s only after the first few successful requests on an older aiohttp, upgrade before touching credentials. The 407 flow itself is covered in how proxy authentication works.

Never retry a 407. It won't fix itself on attempt three.

aiohttp gives you three places to put proxy credentials, and in 3.14 they don't agree. We pointed each combination at a local fake proxy that logs the Proxy-Authorization header it receives:

Mixing two methods never raised an error. One just won quietly, and which one depends on the pair.

Version 3.14.0 also deprecated proxy_auth and BasicAuth (removal planned for 4.0). Both now emit a DeprecationWarning pointing to proxy_headers={'Proxy-Authorization': aiohttp.encode_basic_auth(login, password)}. The changelog adds that encode_basic_auth() "defaults to utf-8, not latin1", and we saw the difference on the wire: a password containing ä reached the proxy as the byte 0xE4 via the URL or BasicAuth, and as 0xC3 0xA4 via encode_basic_auth(). Same password, different bytes. Switching methods during the upgrade can turn a working non-ASCII password into a 407.

On per-request versus session-wide settings: a proxy used to be per-request only, and ClientSession(proxy=..., proxy_auth=...) arrived in 3.11.0 (2024-11-13). But ClientSession in 3.14.5 has no proxy_headers argument. Passing one raised TypeError: ClientSession.__init__() got an unexpected keyword argument 'proxy_headers', although the stable docs page showed that form when we read it on 2026-10-30. So the one non-deprecated way to set credentials once for a whole session is the URL:

import asyncio, os
from urllib.parse import quote

import aiohttp

USER = quote(os.environ["SPARK_USER"], safe="")
PASS = quote(os.environ["SPARK_PASS"], safe="")
ROTATING = f"http://{USER}:{PASS}@gateway.sparkproxy.io:11000"   # Random Rotation
STICKY = f"http://{USER}:{PASS}@gateway.sparkproxy.io:11002"     # Sticky Sessions


async def main() -> None:
    async with aiohttp.ClientSession(proxy=ROTATING,
                                     timeout=aiohttp.ClientTimeout(total=30)) as s:
        async with s.get("https://www.sparkproxy.io/robots.txt") as r:   # session default
            print("rotating", r.status)
        async with s.get("https://www.sparkproxy.io/", proxy=STICKY) as r:  # per-request override
            print("sticky", r.status)


asyncio.run(main())

One logging hazard: ClientHttpProxyError prints the proxy URL, credentials included. Our bad-credentials run logged url='http://baduser:badpass@gateway.sparkproxy.io:11000'. If exceptions go to a shared log or an error tracker, use per-request proxy_headers (the URL then carries no password) or scrub the message before logging.

aiohttp ignores HTTP_PROXY, HTTPS_PROXY and NO_PROXY unless you create the session with trust_env=True, and the default is False. That's the reverse of requests. With HTTPS_PROXY=http://127.0.0.1:9 exported, our default session fetched the page directly (status 200) while the trust_env=True session failed with ClientProxyConnectionError: Cannot connect to host 127.0.0.1:9.

Both directions bite. A script that works in your shell fails in a container that injects HTTPS_PROXY only if someone added trust_env=True. More often it's the other way round: the proxy you set in the environment never gets used, and the scraper quietly goes out from your own IP. The aiohttp docs add that trust_env=True also reads proxy credentials from ~/.netrc, another source of a credential you didn't expect.

An explicit proxy= argument takes precedence; the source only consults the environment when proxy is None. For a scraper, pass the proxy explicitly and leave trust_env off, so a stray variable can't reroute it.

aiohttp has no native SOCKS support; its docs describe HTTP proxies and CONNECT tunnels only. What surprised us is that it doesn't reject the scheme either. proxy="socks5://gateway.sparkproxy.io:13000" sent an HTTP request to the SOCKS port, and the server hung up: ServerDisconnectedError: Server disconnected. Nothing in that message mentions SOCKS.

Use aiohttp-socks (0.12.0, released 2026-08-12 per PyPI, built on python-socks 3.2.0 released 2026-10-27). It replaces the connector rather than taking a proxy= argument:

import asyncio, os
from urllib.parse import quote

import aiohttp
from aiohttp_socks import ProxyConnector, ProxyError

USER = quote(os.environ["SPARK_USER"], safe="")
PASS = quote(os.environ["SPARK_PASS"], safe="")


async def main() -> None:
    connector = ProxyConnector.from_url(
        f"socks5://{USER}:{PASS}@gateway.sparkproxy.io:13000", rdns=True)
    try:
        async with aiohttp.ClientSession(connector=connector,
                                         timeout=aiohttp.ClientTimeout(total=30)) as s:
            async with s.get("https://www.sparkproxy.io/robots.txt") as r:
                print(r.status)
    except ProxyError as e:   # not an aiohttp.ClientError subclass
        print("SOCKS refused:", e)


asyncio.run(main())

Without credentials, our non-whitelisted box got ProxyError: Connection not allowed by ruleset, which is the SOCKS5 reply code 0x02 from RFC 1928. Treat it like a 407. The reverse mistake, a socks5:// connector aimed at HTTP port 11000, raised asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 2 expected bytes, which names neither the proxy nor SOCKS. rdns=True resolves the target hostname on the proxy side, the async equivalent of socks5h://. The wider SOCKS setup is in our SOCKS5 setup guide.

If ServerDisconnectedError shows up only after you raise concurrency, the far end is closing sockets it doesn't want. To see the shape of it, we ran a local server that accepts three concurrent connections and closes anything beyond that, then fired eight requests with asyncio.gather:

[200, 200, 200, 'ClientOSError: [WinError 64] The specified network name is no longer available', 'ServerDisconnectedError: Server disconnected', 'ServerDisconnectedError: Server disconnected', 'ServerDisconnectedError: Server disconnected', 'ServerDisconnectedError: Server disconnected']

Three succeeded and five failed with two different exception classes for the same cause. A proxy plan with a concurrency ceiling, or a target that limits connections per IP, produces the same pattern. aiohttp's own connector doesn't disconnect you: TCPConnector defaults to limit=100 and limit_per_host=0, and extra requests wait in a queue.

That queue has its own trap. We set TCPConnector(limit=1) and ClientTimeout(connect=2) against a server that takes three seconds to answer. The first request returned 200; the other two raised ConnectionTimeoutError after exactly 2.0 seconds without ever opening a socket. Time spent waiting for a pool slot counts against connect. With only sock_connect=2 set, all three succeeded at 3, 6 and 9 seconds. So at high concurrency a "connection timeout" may be your own pool, not the proxy.

Also note that limit_per_host is keyed on aiohttp's ConnectionKey, which includes the target host, port and proxy settings. It doesn't cap what one proxy endpoint sees across many target sites. Cap that with limit or a semaphore:

import asyncio

import aiohttp

PROXY = "http://gateway.sparkproxy.io:11000"   # IP-whitelisted machine
MAX_CONCURRENT = 20   # at or below your plan's concurrency


async def fetch(session: aiohttp.ClientSession, sem: asyncio.Semaphore, url: str) -> int:
    async with sem:
        async with session.get(url, proxy=PROXY) as r:
            await r.read()
            return r.status


async def main(urls: list[str]) -> None:
    connector = aiohttp.TCPConnector(limit=MAX_CONCURRENT, keepalive_timeout=10)
    timeout = aiohttp.ClientTimeout(total=60, sock_connect=10, sock_read=30)
    async with aiohttp.ClientSession(connector=connector, timeout=timeout) as s:
        sem = asyncio.Semaphore(MAX_CONCURRENT)
        results = await asyncio.gather(*(fetch(s, sem, u) for u in urls),
                                       return_exceptions=True)
        print(results)


asyncio.run(main(["https://www.sparkproxy.io/robots.txt"] * 5))

Matching limit to the semaphore keeps queued tasks from burning their timeout budget. For the pooling side, TCP keepalive and connection pooling covers idle sockets, and understanding concurrent connections explains how providers count them. Semaphores and gather patterns in general are in async web scraping in Python.

Order matters, because subclasses must come before their parents. This classifier returns a decision instead of retrying blindly:

import asyncio

import aiohttp
from aiohttp_socks import ProxyError as SocksProxyError


def classify(exc: BaseException) -> str:
    if isinstance(exc, aiohttp.ClientHttpProxyError):
        return "fatal: 407 or other proxy refusal, fix credentials or whitelist"
    if isinstance(exc, SocksProxyError):
        return "fatal: SOCKS refusal, fix credentials or whitelist"
    if isinstance(exc, aiohttp.ClientConnectorDNSError):
        return "fatal: proxy hostname does not resolve"
    if isinstance(exc, aiohttp.ClientConnectorSSLError):
        return "fatal: check for https:// in an HTTP proxy URL"
    if isinstance(exc, aiohttp.ClientProxyConnectionError):
        return "fatal if instant (wrong port), retry once if intermittent"
    if isinstance(exc, (aiohttp.ServerDisconnectedError, aiohttp.ClientOSError)):
        return "retry with backoff, and lower concurrency if it repeats"
    if isinstance(exc, (aiohttp.ConnectionTimeoutError, asyncio.TimeoutError)):
        return "retry; if constant, test the port or check pool waits"
    if isinstance(exc, aiohttp.ClientResponseError):
        return f"target answered {exc.status}: a blocking problem, not a proxy fault"
    return "unknown: log and inspect"

ClientOSError sits beside ServerDisconnectedError because our capped-server run produced both for one cause. ClientProxyConnectionError is also a ClientOSError, which is why it's checked first. Backoff timing and rotate-on-retry belong in your retry loop; using proxies with requests and aiohttp has one.

Only the last two rows involve the proxy service itself. Everything above them is configuration on your side.

If port 11000 is blocked and nobody will open it, SparkProxy's Scraping API runs over standard HTTPS on 443 and does the proxying on its own network:

import asyncio, os

import aiohttp


async def main() -> None:
    params = {"url": "https://www.sparkproxy.io/robots.txt",
              "render_js": "false",       # plain fetch, 1 credit (default true costs 5)
              "proxy_type": "premium"}
    headers = {"X-API-Key": os.environ["SPARK_API_KEY"]}
    async with aiohttp.ClientSession(timeout=aiohttp.ClientTimeout(total=90)) as s:
        async with s.get("https://scrape.sparkproxy.io/api/v1",
                         params=params, headers=headers) as r:
            print(r.status, (await r.text())[:200])


asyncio.run(main())

A new API account starts with 1,000 free credits. Asking for json_response=true wraps the result in a JSON envelope with job_id, status_code, duration_ms and credits_used, and the page itself sits base64-encoded in body, so decode it before you look for markers in the HTML. The format parameter is unrelated: it converts content (html, md, json and others) and never adds that envelope.

The proxy plans are priced as packs, "Starting from $75/month" in USD as listed on sparkproxy.io on 2026-10-09, and bandwidth is unlimited. Practically, that means a burst of failed retries while you debug costs nothing extra. With the 24-hour free trial (no credit card), you can run the four-step diagnostic from each server, CI runner and laptop that will scrape, and add every egress IP to the whitelist before going live.