The HTTP CONNECT Method Explained

CONNECT is defined in RFC 9110 section 9.3.6 (IETF, June 2022), which replaced the older RFC 7231 text. The specification is short because the behaviour is simple. The client names a host and a port. The proxy opens a TCP connection to that host and port. If the connection succeeds, the proxy sends a 2xx response and then blindly copies bytes in both directions until one side closes.

Three properties follow from that, and they explain most CONNECT behaviour you will ever debug.

The request has no path. There is nothing to route on beyond host:port. A proxy cannot apply per-URL rules to a CONNECT because the URL does not exist yet at the time of the request.

The response has no body and no Content-Length. After the blank line that terminates the 2xx response headers, every subsequent byte belongs to the tunnel. A client that over-reads the response buffer will swallow the first bytes of the TLS ClientHello and then hang forever waiting for a handshake that already left.

The method is not cacheable, not safe, and not idempotent. Nothing about a CONNECT can be stored or replayed, which is why proxies that cache aggressively for plain HTTP get exactly zero cache hits on HTTPS.

CONNECT is also protocol-agnostic below HTTP. The payload can be TLS, SSH, SMTP, or anything else that runs on TCP. That is why corporate proxies restrict which ports CONNECT may target rather than which protocols it may carry: the port is the only signal available before the pipe opens.

Here is the complete exchange when a client fetches https://example.sparkproxy.io/pricing through an HTTP proxy at gate.sparkproxy.io:8080. Byte for byte, this is what crosses the first hop.

The client opens a plain TCP connection to the proxy and writes:

CONNECT example.sparkproxy.io:443 HTTP/1.1
Host: example.sparkproxy.io:443
Proxy-Authorization: Basic dXNlci0xMjM0NTpzM2NyM3Q=
Proxy-Connection: Keep-Alive
User-Agent: curl/8.7.1

Four things in that block are worth naming precisely.

CONNECT example.sparkproxy.io:443 HTTP/1.1 is the authority-form request target described in RFC 9112 section 3.2.3. No https://, no trailing slash, no path. Writing CONNECT https://example.sparkproxy.io/ HTTP/1.1 is a malformed request and well-behaved proxies answer 400.

The port is not optional. HTTP has default ports for schemes, but authority-form has no scheme, so there is no default to fall back on. CONNECT example.sparkproxy.io HTTP/1.1 is invalid.

Proxy-Connection never appeared in any RFC. It was a workaround from the Netscape era for broken intermediaries, and it survives because everyone implements it. Connection: keep-alive is the standards-track equivalent.

The request ends with a bare \r\n and carries no body. CONNECT with a payload is undefined.

The proxy resolves the hostname, opens TCP to port 443, and on success writes back:

HTTP/1.1 200 Connection established
Proxy-Agent: SparkProxy-Gateway/2.1

That blank line is the boundary. The client must read exactly up to and including \r\n\r\n, then hand the untouched socket to its TLS layer. The next bytes on the wire are the client's ClientHello:

16 03 01 02 00 01 00 01 fc 03 03 ...
|  |     |     |     |        |
|  |     |     |     |        +-- client random (32 bytes)
|  |     |     |     +----------- handshake body length
|  |     |     +----------------- handshake type 0x01 = ClientHello
|  |     +----------------------- TLS record length
|  +----------------------------- legacy record version 0x0301
+-------------------------------- record type 0x16 = handshake

From this point the proxy is a byte pump. It never parses those bytes, never inserts an X-Forwarded-For, and never sees the GET /pricing HTTP/1.1 that eventually travels inside the encrypted records.

One subtlety trips up hand-rolled clients: some proxies return 200 and then close moments later because the origin reset. A 200 is not a guarantee that the origin will still be there when your ClientHello arrives, so treat the first read after the handshake as a failure point too.

A forward proxy has two entirely different operating modes, and the target scheme decides which one you get. Plain HTTP goes through in absolute-form, where the proxy is a full HTTP participant:

GET http://example.sparkproxy.io/pricing?page=2 HTTP/1.1
Host: example.sparkproxy.io
Proxy-Connection: keep-alive
Accept-Encoding: gzip, deflate

Compare that to what the same client sends when talking straight to the origin, in origin-form:

GET /pricing?page=2 HTTP/1.1
Host: example.sparkproxy.io

Four request-target forms exist, and mixing them up produces confusing 400s:

The operational difference between absolute-form and CONNECT is much larger than the syntax suggests:

That last row is the commercial consequence nobody writes about. A provider carrying HTTPS traffic literally cannot count your requests, because request boundaries live inside the encrypted stream. All it can meter is bytes through the tunnel. This is why residential proxy pricing is denominated in gigabytes while API products, which terminate the request themselves, price per call. The pricing model is downstream of the protocol.

For a map of which port carries which mode, see proxy ports explained.

The obvious question is why a proxy cannot relay HTTPS the way it relays HTTP, forwarding the request and returning the response. Two independent reasons make that impossible.

There is no request to forward. In absolute-form mode the proxy parses a text request line, reads headers, and acts. Over HTTPS the first bytes after the TCP handshake are a TLS record, not text: no request line to parse, no Host header, no path to route on. The proxy would have to complete a TLS handshake first, which brings up the second reason.

The certificate is bound to the origin, not the proxy. TLS server authentication works because the client validates the presented certificate against the hostname it meant to reach and a chain ending in a trusted root. A proxy terminating TLS for example.sparkproxy.io would have to present a certificate for that name, and it has no private key for it. Even if it did, TLS record integrity means any byte it modified in transit would fail authentication at the far end.

So the proxy's only honest option is to carry the bytes untouched, and CONNECT is the request that asks for exactly that. The tunnel exists so that the security properties of TLS survive the presence of the middlebox.

A useful mental model is that CONNECT changes what the proxy is. Before the 200 it is an HTTP server. After the 200 it is a layer-4 relay that happens to have been configured over HTTP.

Proxy credentials travel in the Proxy-Authorization header, which is hop-by-hop and is consumed by the proxy rather than forwarded. On a CONNECT there is nothing downstream to forward it to anyway, since the tunnel payload is opaque.

The common scheme is Basic, defined in RFC 7617, which is base64 of username:password:

printf 'user-12345:s3cr3t' | base64
# dXNlci0xMjM0NTpzM2NyM3Q=

CONNECT example.sparkproxy.io:443 HTTP/1.1
Host: example.sparkproxy.io:443
Proxy-Authorization: Basic dXNlci0xMjM0NTpzM2NyM3Q=

Here is the part that deserves more attention than it gets. Base64 is an encoding, not a cipher. On a standard proxy port such as 8080 or 3128 the client-to-proxy leg is plain TCP, so those credentials cross the network in trivially reversible form on every new tunnel you open. Your tunnelled traffic is protected by TLS. Your proxy password is not. A scraper opening 200 connections per second is broadcasting it 200 times per second.

Two mitigations exist. Use IP whitelisting where the provider supports it, which removes the secret from the wire entirely. Or connect to the proxy over HTTPS so the first hop is TLS-wrapped before the CONNECT is written, which some gateways offer on a separate port. The full set of options is covered in how proxy authentication works.

There is also a latency question. Clients differ on whether they send credentials preemptively or wait to be challenged:

The challenge-response path costs one extra round trip on every new connection. With per-request IP rotation, where connection reuse is impossible by design, that overhead lands on every single request in the job. If your proxy latency looks 50 to 150 ms worse than it should, check whether your HTTP client is doing the 407 dance.

Rotating gateways add one more wrinkle. Because a CONNECT line carries only host:port, there is no room in it for routing instructions. Providers therefore encode session and geo directives into the username field:

user-12345-country-us-session-a7f3d9:s3cr3t

The entire routing decision for a tunnelled request lives in the Proxy-Authorization header. That is a direct consequence of authority-form having no query string.

A CONNECT response tells you which hop broke, which saves hours, because one application-layer symptom ("my scraper times out") maps to at least five distinct causes.

Some of these deserve expansion.

405 is almost always a topology mistake. Reverse proxies, load balancers, and origin servers have no reason to implement CONNECT and correctly refuse it. A 405 means you pointed a forward-proxy configuration at something that is not a forward proxy, usually by setting HTTPS_PROXY to a CDN endpoint or an API base URL.

407 versus 401 matters. 407 arrives with Proxy-Authenticate and refers to the proxy. 401 arrives with WWW-Authenticate and refers to the origin, so a 401 answering a CONNECT is anomalous and usually means an intercepting middlebox replied on the proxy's behalf. Many commercial gateways also return 407 when your balance or bandwidth quota is exhausted, not only when the password is wrong, so a sudden wall of 407s on credentials that worked an hour ago is a billing signal before it is an auth signal.

502 and 504 are the same hop with different failure modes. A 502 means the proxy got a definite negative: connection refused, RST, or NXDOMAIN. A 504 means it got nothing and gave up. Refused points at the origin's port or firewall. Silence points at packet filtering, often geographic.

403 on CONNECT is usually a port ACL. Squid ships with http_access deny CONNECT !SSL_ports, and SSL_ports defaults to 443 alone. That one default line is why tunnelling to 8443, 9443, or SSH on 22 returns 403 Forbidden on a proxy that handles ordinary HTTPS perfectly. Configuration, not breakage.

For codes that appear after the tunnel is up, which are origin responses rather than proxy responses, see proxy error codes explained.

The fastest diagnostic is verbose curl. The * lines are curl's commentary, > is what it sent, < is what came back:

curl -v -x http://user-12345:s3cr3t@gate.sparkproxy.io:8080 \
     https://example.sparkproxy.io/pricing -o /dev/null

*   Trying 203.0.113.20:8080...
* Connected to gate.sparkproxy.io (203.0.113.20) port 8080
* CONNECT tunnel: HTTP/1.1 negotiated
* Establish HTTP proxy tunnel to example.sparkproxy.io:443
* Proxy auth using Basic with user 'user-12345'
> CONNECT example.sparkproxy.io:443 HTTP/1.1
> Host: example.sparkproxy.io:443
> Proxy-Authorization: Basic dXNlci0xMjM0NTpzM2NyM3Q=
> User-Agent: curl/8.7.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* CONNECT phase completed
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):

Read that output as two phases. Everything up to CONNECT phase completed is the proxy hop; everything after it is the origin hop, tunnelled. A failure before the boundary is a proxy problem. A failure after it, such as SSL certificate problem or handshake failure, is a TLS problem the proxy neither caused nor can fix. Most proxy support tickets are TLS failures misfiled as proxy failures because nobody checked which side of that line they landed on.

To force a tunnel for a plain http:// URL, useful for testing whether CONNECT works at all without TLS in the way, add -p. OpenSSL can also drive the tunnel itself with -proxy, available since OpenSSL 1.1.1, which isolates TLS problems from HTTP client behaviour:

curl -v -p -x http://user-12345:s3cr3t@gate.sparkproxy.io:8080 \
     http://example.sparkproxy.io/health

openssl s_client -connect example.sparkproxy.io:443 \
  -proxy gate.sparkproxy.io:8080 \
  -servername example.sparkproxy.io -brief

When you need certainty about the exact bytes, write the CONNECT by hand. This snippet is deliberately literal about the one detail most implementations get wrong:

import base64
import socket
import ssl

PROXY = ("gate.sparkproxy.io", 8080)
TARGET = ("example.sparkproxy.io", 443)
CREDS = base64.b64encode(b"user-12345:s3cr3t").decode()

sock = socket.create_connection(PROXY, timeout=15)
sock.sendall(
    f"CONNECT {TARGET[0]}:{TARGET[1]} HTTP/1.1\r\n"
    f"Host: {TARGET[0]}:{TARGET[1]}\r\n"
    f"Proxy-Authorization: Basic {CREDS}\r\n"
    f"Proxy-Connection: Keep-Alive\r\n"
    f"\r\n".encode()
)

# Read ONE BYTE AT A TIME until the blank line. Anything read past the
# header terminator belongs to the tunnel and would be lost from the
# TLS stream, producing a handshake that hangs with no error message.
buf = b""
while not buf.endswith(b"\r\n\r\n"):
    byte = sock.recv(1)
    if not byte:
        raise ConnectionError("proxy closed before completing the tunnel")
    buf += byte

status_line = buf.split(b"\r\n", 1)[0].decode()
print(status_line)                       # HTTP/1.1 200 Connection established
code = int(status_line.split()[1])
if not 200 <= code < 300:
    raise ConnectionError(f"CONNECT refused: {status_line}")

# The socket is now a raw pipe. Everything below is end to end with the origin.
ctx = ssl.create_default_context()
tls = ctx.wrap_socket(sock, server_hostname=TARGET[0])
print(tls.version())                     # TLSv1.3
print(dict(x[0] for x in tls.getpeercert()["issuer"]))

The byte-at-a-time loop looks wasteful, and it is, for roughly 60 bytes. It is also the only way to guarantee you have not consumed part of the ServerHello. A buffered recv(4096) usually works and then fails on one proxy in twenty.

One more configuration trap. In Python's requests, the proxies dictionary keys describe the scheme of the target URL, not the scheme used to reach the proxy:

proxies = {
    "http":  "http://user-12345:s3cr3t@gate.sparkproxy.io:8080",
    "https": "http://user-12345:s3cr3t@gate.sparkproxy.io:8080",
    #         ^^^^ still http:// This is how you TALK TO the proxy.
}

Writing "https": "https://gate.sparkproxy.io:8080" tells the client to run TLS against the proxy port itself. If the gateway is not expecting TLS there, you get a handshake error that reads like a certificate problem and has nothing to do with certificates.

"The proxy sees nothing" is a comfortable simplification and it is wrong in a specific, exploitable way.

After the 200, the first thing the client writes is a TLS ClientHello, and several fields in a ClientHello are plaintext by design. The Server Name Indication extension carries the target hostname in the clear, in both TLS 1.2 and TLS 1.3, unless Encrypted Client Hello is in use. So a proxy operator can read:

The fingerprinting row matters for scraping. A JA3 or JA4 hash is computed entirely from ClientHello fields, so a middlebox, and equally the origin at the far end, can fingerprint your TLS stack through a tunnel without decrypting a single record. Rotating IPs while your HTTP library emits the same unusual ClientHello every time is a well-known way to get blocked on a brand-new IP. See TLS fingerprinting.

Organisations that need to read HTTPS content do it with an explicit man-in-the-middle, and understanding its shape tells you how to detect it.

The inspecting proxy answers the CONNECT with 200 Connection established exactly as a normal proxy would. Then, instead of relaying bytes, it performs its own TLS handshake as the server, presenting a certificate it minted on the spot for the requested hostname, signed by a private CA. Separately it opens a second TLS session to the real origin as the client. Two sessions, decrypted plaintext in the middle, re-encrypted on the way out.

This only works if the client already trusts the proxy's CA, which is why corporate device management pushes a root certificate to every managed machine. Without that root the client sees an untrusted issuer and fails closed. Interception requires prior consent from the client's trust store, not merely a position on the network path.

Detection is one command. Inspect the issuer of the certificate you actually received:

openssl s_client -connect example.sparkproxy.io:443 \
  -proxy gate.sparkproxy.io:8080 -servername example.sparkproxy.io 2>/dev/null \
  | openssl x509 -noout -issuer -subject -dates

A public issuer such as issuer=C = US, O = Let's Encrypt, CN = R11 means the tunnel is genuine. An issuer naming a device vendor or an internal CA means you are being decrypted. Two secondary tells: an inspected connection usually shows a shorter certificate validity window, because the proxy mints leaves that expire in days, and the JA3 fingerprint reaching the origin belongs to the proxy's TLS library rather than to your client.

For scrapers the consequence is sharp. Certificate pinning defeats interception outright, and your TLS fingerprint through an inspecting proxy is not the one you configured. Tune a client to present a convincing Chrome ClientHello, route it through a decrypting middlebox, and the origin sees the middlebox's hello instead.

HTTP/2 has no request line to put an authority-form target on, so the method was remapped onto pseudo-headers in RFC 9113 section 8.5:

:method    = CONNECT
:authority = example.sparkproxy.io:443

:scheme and :path must be omitted, and a peer that includes them has to treat the stream as malformed. The tunnel then lives inside a single HTTP/2 stream: DATA frames carry the payload in both directions, and END_STREAM closes it. One TCP connection can host many concurrent tunnels this way, which removes per-tunnel handshake cost and adds head-of-line blocking as the price.

Extended CONNECT, from RFC 8441 (2018), reintroduces :scheme and :path plus a :protocol pseudo-header. It exists to carry WebSockets over HTTP/2 and is not a general-purpose TCP tunnel. RFC 9220 (2022) ports the same idea to HTTP/3.

Here is the part that surprises people. Even in 2026 the CONNECT you actually send to a commercial gateway is nearly always HTTP/1.1 plaintext, because speaking HTTP/2 to a proxy needs either TLS with ALPN negotiating h2 or prior knowledge over cleartext, and most gateway ports offer neither. Your client negotiates HTTP/2 with the origin, end to end inside the tunnel, while the tunnel setup itself is 1990s-era text. curl's verbose output shows both at once: CONNECT tunnel: HTTP/1.1 negotiated, then a few lines later ALPN: server accepted h2.

Routing an ordinary client through a SparkProxy gateway is a proxy dictionary and nothing else. The CONNECT handshake is handled by the HTTP library:

import requests

proxies = {
    "http":  "http://user-12345:s3cr3t@gate.sparkproxy.io:8080",
    "https": "http://user-12345:s3cr3t@gate.sparkproxy.io:8080",
}

# Session reuse keeps the tunnel open across requests, so the CONNECT
# handshake is paid once instead of once per call.
with requests.Session() as s:
    s.proxies.update(proxies)
    for page in range(1, 6):
        r = s.get(f"https://example.sparkproxy.io/pricing?page={page}", timeout=30)
        print(page, r.status_code, len(r.content))

The SparkProxy Scraping API takes the opposite approach: it terminates the request on its own infrastructure, so your client never issues a CONNECT at all.

import requests

r = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={
        "url": "https://example.sparkproxy.io/pricing",
        "render_js": "false",       # plain HTTP fetch, 1 credit
        "json_response": "true",    # envelope with status_code and duration_ms
    },
    timeout=90,
)
data = r.json()
print(data["status_code"], data["duration_ms"], data["credits_used"])

Because the API terminates the request itself, it reports the origin's status code and per-request timing as structured fields, which is exactly the visibility a CONNECT tunnel destroys. That is the tradeoff in one sentence: tunnels preserve end-to-end encryption and cost you observability, API termination gives you observability and moves the trust boundary to the provider.

If you run your own proxy fleet and want the API to tunnel through it, own_proxy takes a full proxy URL and performs the CONNECT for you:

import requests

r = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={
        "url": "https://example.sparkproxy.io/pricing",
        "own_proxy": "http://user-12345:s3cr3t@gate.sparkproxy.io:8080",
        "render_js": "true",
    },
    timeout=120,
)
print(r.status_code, len(r.text))

The parameter accepts ip:port, ip:port:user:pass, http://user:pass@host:port, and socks5://host:port. SOCKS5 targets require render_js=true, since the headless browser handles that protocol rather than the plain fetch path.