Python Requests: Tunnel Connection Failed 407 Fix

Here is the full error, captured verbatim from requests 2.34.2 hitting our gateway with a deliberately wrong password:

requests.exceptions.ProxyError: HTTPSConnectionPool(host='www.sparkproxy.io', port=443):
Max retries exceeded with url: / (Caused by ProxyError('Unable to connect to proxy',
OSError('Tunnel connection failed: 407 Proxy Authentication Required')))

Read it from the inside out. The innermost OSError comes from the standard library, not from requests. CPython's http.client raises it in _tunnel() when the proxy answers a CONNECT with anything other than 200; the line is raise OSError(f"Tunnel connection failed: {code} {message.strip()}") in Lib/http/client.py. urllib3 wraps that in its own ProxyError, and requests wraps it again.

Two facts fall out of this:

• The target site never saw your request. The failure happened on the CONNECT www.sparkproxy.io:443 line, before TLS started. Changing user agents, cookies or headers for the target does nothing.
• "Max retries exceeded" is noise here. With default settings there were zero retries. The phrase is a fixed part of urllib3's message.

The 407 status itself is defined in RFC 9110, section 15.5.8. If you want the general meaning of 407 next to its neighbours (403, 429, 502), our proxy error codes reference covers that. This post stays on the part specific to requests.

One more thing worth knowing before you debug: the same bad credentials produce a different symptom on a plain http:// URL. There's no tunnel, so requests returns a normal Response with status_code == 407 and no exception at all. We measured exactly that against the gateway. If your script fetches a mix of http:// and https:// URLs, you may see quiet 407 responses on one and a ProxyError on the other from the same misconfiguration.

A browser and requests authenticate to a proxy in different ways.

When the browser gets a 407, it reads the Proxy-Authenticate header, shows you a login prompt, and you type the password as raw text. Our gateway's challenge looks like this (from curl -v, 30 October 2026):

< HTTP/1.1 407 Proxy Authentication Required
< Proxy-Authenticate: Basic realm="SparkProxy"

The scheme is Basic, so no NTLM or Digest library is involved; the client just needs to send Proxy-Authorization: Basic on the CONNECT.

requests never shows a prompt and never retries after a challenge. It sends the header up front, and it builds that header from the proxy URL you gave it: it parses the URL, runs unquote() on the username and password (see get_auth_from_url in requests/utils.py), and Base64-encodes the result. So the URL is a second encoding layer the browser never had. Anything that survives the browser prompt but gets mangled by URL parsing produces a 407 in Python only.

curl behaves like the browser if you pass --proxy-user 'user:pass', and like requests if you embed the credentials in the -x URL. That's why "it works in curl" depends on how you called curl.

Run these in order. Each one takes under a minute and rules out a whole class of cause.

Step 1 isolates the credentials from Python entirely. --proxy-user takes the password raw, exactly as a browser prompt would:

curl -s -o /dev/null \
  -x http://gateway.sparkproxy.io:11000 \
  --proxy-user 'YOUR_USER:YOUR_PASS' \
  -w "connect=%{http_connect} code=%{http_code}\n" \
  https://www.sparkproxy.io/

connect=407 code=000 means the gateway rejected the credentials themselves, and nothing in your Python code will help. That's the exact output we got with a dummy login. connect=200 with a real status code means the credentials are fine and the problem lives in your script. Use single quotes so your shell doesn't expand $ or ! in the password.

Step 2 asks requests which proxy it's going to use, after it has merged the environment in:

import requests
from urllib.parse import urlsplit

session = requests.Session()
session.proxies = {"https": "http://user:pass@gateway.sparkproxy.io:11000"}

url = "https://www.sparkproxy.io/"
effective = session.merge_environment_settings(url, {}, None, None, None)["proxies"]
for scheme, proxy in effective.items():
    parts = urlsplit(proxy)
    print(scheme, "->", f"{parts.scheme}://{parts.hostname}:{parts.port}", "user:", parts.username)

If the host or username printed there isn't the one in your code, skip straight to Cause 3.

The userinfo part of a URL has its own rules (RFC 3986, section 3.2.1), and generated proxy passwords love symbols. The folklore says "@ and : break it". Our measurements say the real offenders are different. We put each character into an unencoded password and logged what our test proxy received on the CONNECT:

Two separate failure modes hide in that table:

• #, / and ? end the authority part of the URL, so requests can't find the host and raises InvalidURL: Failed to parse: 'spuser:p' is not a valid host or port. Loud, but it isn't a 407, so people chase the wrong error.
• A literal % followed by two hex digits is the quiet one. unquote() treats it as an escape, the proxy receives a different password, and you get exactly Tunnel connection failed: 407. No warning anywhere.

@ works unencoded because both urllib3 (authority.rpartition("@")) and urllib.parse split the authority on the last @. That's a parser detail, not a promise in any spec, and the fix below makes the question irrelevant.

The fix is to percent-encode both parts with safe="". The default safe="/" in urllib.parse.quote leaves slashes alone, which is precisely one of the characters that breaks the URL:

import os
from urllib.parse import quote

user = quote(os.environ["SPARKPROXY_USER"], safe="")
password = quote(os.environ["SPARKPROXY_PASS"], safe="")
proxy_url = f"http://{user}:{password}@gateway.sparkproxy.io:11000"

We ran this with the password p#ss%41@x against a strict test proxy, and the CONNECT carried spuser:p#ss%41@x byte for byte. One edge case encoding can't rescue: a colon in the username. RFC 7617 states that "a user-id containing a colon character is invalid", because the server splits user:pass on the first colon. Our proxy received sp:user:goodpass whether or not we encoded it.

This one appears in a lot of copied snippets:

import base64
import requests

token = base64.b64encode(b"user:pass").decode()
headers = {"Proxy-Authorization": f"Basic {token}"}
proxies = {"http": "http://gateway.sparkproxy.io:11000",
           "https": "http://gateway.sparkproxy.io:11000"}

requests.get("http://www.sparkproxy.io/", headers=headers, proxies=proxies)   # works
requests.get("https://www.sparkproxy.io/", headers=headers, proxies=proxies)  # 407

For the http:// URL there's no tunnel. requests sends GET http://www.sparkproxy.io/ to the proxy with your headers attached, the proxy sees Proxy-Authorization, and the request passes. For the https:// URL, urllib3 first opens a tunnel with CONNECT, and the headers you passed belong to the request that travels inside that tunnel. Our test proxy logged the CONNECT with auth=None and answered 407. Even if the tunnel had opened, the header would have gone to the target site, which is a credential leak.

Comments in the requests source say the same thing: HTTPAdapter.proxy_headers exists "to ensure that they are correctly sent to the proxy, rather than in a tunnelled request if CONNECT is being used", and Session.rebuild_proxies only adds Proxy-Authorization when the scheme isn't https, to "avoid appending this to TLS tunneled requests where it may be leaked".

Put the credentials in the proxy URL (Cause 1's encoding applies). If you need them out of the URL, for instance because a logger prints proxy URLs, override proxy_headers on a custom adapter. Those headers are attached to the CONNECT:

import base64
import requests
from requests.adapters import HTTPAdapter


class ProxyAuthAdapter(HTTPAdapter):
    def __init__(self, user, password, **kwargs):
        token = base64.b64encode(f"{user}:{password}".encode()).decode()
        self._proxy_auth = f"Basic {token}"
        super().__init__(**kwargs)

    def proxy_headers(self, proxy):
        return {"Proxy-Authorization": self._proxy_auth}


session = requests.Session()
session.proxies = {"http": "http://gateway.sparkproxy.io:11000",
                   "https": "http://gateway.sparkproxy.io:11000"}
adapter = ProxyAuthAdapter("YOUR_USER", "raw#password%41")
session.mount("http://", adapter)
session.mount("https://", adapter)

The password goes in raw here, with no URL encoding, because it never passes through a URL parser. We confirmed the CONNECT carried it unchanged.

This is the cause behind most "but I set the proxy correctly" reports. requests.Session has trust_env = True by default, and on every request it merges proxies from the environment before it looks at session.proxies. In Session.merge_environment_settings, environment proxies are added to the per-request dict, and only then is the result merged with self.proxies, where the per-request side wins.

The requests documentation says this in a warning box: values in session.proxies "will be overwritten by environmental proxies (those returned by urllib.request.getproxies)". The long-running discussion is psf/requests issue #2018.

We reproduced it. With HTTPS_PROXY set to a proxy holding stale credentials and session.proxies set to a valid one, the stale proxy got the CONNECT and returned 407. Your correct credentials were never sent.

Where the stray proxy comes from:

• HTTPS_PROXY, HTTP_PROXY, ALL_PROXY or their lowercase forms, often set by a corporate shell profile, a Docker image, a CI runner or an old .env file.
• On Windows, the registry. If no proxy environment variables exist, urllib.request.getproxies falls back to the system proxy settings in the registry (getproxies_environment() or getproxies_registry() in CPython 3.14). On macOS it reads the system configuration the same way.

Three fixes, from most to least explicit:

import requests

proxy_url = "http://YOUR_USER:YOUR_PASS@gateway.sparkproxy.io:11000"
proxies = {"http": proxy_url, "https": proxy_url}

# 1. Ignore the environment for this session
session = requests.Session()
session.trust_env = False
session.proxies = proxies

# 2. Pass proxies on every call; per-request values beat the environment
other = requests.Session()
other.get("https://www.sparkproxy.io/", proxies=proxies, timeout=30)

# 3. Or clear the variables for the process that runs the script
env -u HTTPS_PROXY -u https_proxy -u HTTP_PROXY -u http_proxy python scraper.py

Setting trust_env = False has side effects: requests also stops reading .netrc, REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE. If you depend on a custom CA bundle, pass verify= explicitly.

The proxies dict is keyed by the scheme of the URL being fetched, not the scheme of the proxy. A dict with only an "http" key covers http:// URLs and nothing else. That looks harmless until a redirect changes the scheme.

During redirects, Session.resolve_redirects calls rebuild_proxies for each hop, which looks up the new URL's scheme and fills any missing key from the environment. We tested a session with proxies = {"http": ...} fetching an http:// URL that answers 301 to https://:

The second row is worse than a 407. The request "succeeds" from your real IP, so the bug hides until a target starts blocking you. The repair is boring and complete: always set both keys, and pair it with trust_env = False if anything on the machine might export a proxy.

proxies = {"http": proxy_url, "https": proxy_url}

Both values use the http:// scheme, because port 11000 speaks plain HTTP to your client and carries HTTPS through CONNECT. Writing https://gateway.sparkproxy.io:11000 makes urllib3 attempt TLS to the proxy itself. Against the gateway that failed with SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number') inside a ProxyError, not a 407, and urllib3 appended a link to its "https-proxy-error-http-proxy" docs note.

If step 1's curl test also returns connect=407, the problem isn't in Python. Common account-side causes:

We haven't verified which status code the gateway returns once a plan expires, so treat row three as a hypothesis to check in the dashboard, not a guaranteed 407.

Switching proxy providers fixes none of the four requests causes above; the same bytes would fail anywhere. And a working proxy can't fix a target that blocks you after the tunnel opens. That shows up as a 403 or a CAPTCHA, not as Tunnel connection failed.

If you're moving this code to aiohttp, note that its proxy_auth= parameter takes credentials outside the URL. Our requests and aiohttp proxy guide covers that setup. For background on how CONNECT tunnels work at the protocol level, see what an HTTP tunnel is, and for the 407 challenge flow across auth methods, how proxy authentication works.

This combines every fix: encoded credentials, both scheme keys, the environment ignored, and a check that fails loudly if the tunnel is refused.

import os
import sys
from urllib.parse import quote

import requests

user = quote(os.environ["SPARKPROXY_USER"], safe="")
password = quote(os.environ["SPARKPROXY_PASS"], safe="")
proxy_url = f"http://{user}:{password}@gateway.sparkproxy.io:11000"

session = requests.Session()
session.trust_env = False  # ignore HTTP(S)_PROXY, ALL_PROXY and the OS proxy settings
session.proxies = {"http": proxy_url, "https": proxy_url}

try:
    response = session.get("https://www.sparkproxy.io/", timeout=30)
except requests.exceptions.ProxyError as exc:
    if "407" in str(exc):
        sys.exit("Proxy rejected the credentials: re-run the curl --proxy-user check")
    raise

print(response.status_code, len(response.content), "bytes")

How we verified it on 30 October 2026: with dummy credentials against gateway.sparkproxy.io:11000, the script reached the gateway and exited on the 407 branch, which proves the host, port and CONNECT path. With the gateway host swapped for our strict local test proxy, a password full of #, % and @, and a stale HTTPS_PROXY deliberately left in the environment, it returned 200 and the proxy logged the password byte for byte. Set SPARKPROXY_USER and SPARKPROXY_PASS from your dashboard and it runs as-is.

Port 11000 is the Random Rotation endpoint. For Sticky Sessions, where the same IP holds for 5 minutes, use port 11002 with the same credential handling. The pool covers 1M+ premium IPs in 80+ countries over HTTP, HTTPS and SOCKS5, plans start from $75/month (USD, read from sparkproxy.io on 9 October 2026) with unlimited bandwidth, and there's a 24-hour free trial with no credit card.

If you only need page content, the SparkProxy Scraping API removes proxy authentication from your code entirely. There is no CONNECT from your side: you call an HTTPS endpoint with an X-API-Key header, and the API handles the proxy hop.

import os
import requests

response = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": os.environ["SPARKPROXY_API_KEY"]},
    params={
        "url": "https://www.sparkproxy.io/",
        "render_js": "false",  # plain fetch, 1 credit; the default true renders JS for 5
    },
    timeout=60,
)
response.raise_for_status()
print(response.text[:200])

New accounts get 1,000 free credits. If you add json_response=true for metadata such as status_code and credits_used, remember that the body field comes back Base64-encoded and must be decoded before you search it.