HTTP/2 vs HTTP/1.1 for Web Scraping

The last three rows are the whole article. Everything else is performance trivia that a scraper rarely feels.

HTTP/2 keeps HTTP semantics and replaces the transport syntax. Requests and responses become sequences of binary frames carried on numbered streams, and many streams share one TCP connection. RFC 9113 defines the frame types you care about: HEADERS, DATA, SETTINGS, WINDOW_UPDATE, PRIORITY, RST_STREAM, GOAWAY and PING.

Three practical consequences for scraping:

One connection instead of a pool

HTTP/1.1 clients open several sockets per host because each socket handles one in-flight request. HTTP/2 sends dozens of concurrent requests down a single socket. Fewer TCP and TLS handshakes per page, which matters most on high-latency residential exits.

A mandatory opening ritual

Before any request, an h2 client sends the connection preface (the literal bytes PRI * HTTP/2.0 followed by SM and blank lines) and then a SETTINGS frame, per RFC 9113 section 3.4. That SETTINGS frame is where the fingerprint starts.

Pseudo-headers replace the request line

Method, scheme, authority and path become the pseudo-header fields :method, :scheme, :authority and :path, which must precede all regular fields (RFC 9113 section 8.3.1). The spec constrains their position relative to normal headers but not their order relative to each other, so every implementation picked its own and never changed it.

You negotiate all of this in TLS, not in HTTP. The ALPN extension in the ClientHello offers a token list, and the presence of h2 in that list is what switches the connection to HTTP/2:

curl --http2 -sv https://www.sparkproxy.io/ -o /dev/null 2>&1 | grep -iE "alpn|using http"
# * ALPN: curl offers h2,http/1.1
# * ALPN: server accepted h2
# * using HTTP/2

Server push existed in RFC 7540 and is effectively dead. Chrome removed support in 2022, and modern clients advertise SETTINGS_ENABLE_PUSH = 0. That zero is itself a fingerprint bit.

HTTP/1.1 blocks at the request layer. A connection carries one exchange at a time, pipelining never worked in practice, and browsers compensate by opening roughly six connections per origin. Slow response, blocked connection.

HTTP/2 removes that: a stalled stream does not stall its siblings. What it cannot remove is TCP's own ordering guarantee. All streams ride one byte sequence, so a single lost segment stalls delivery for every stream on that connection until retransmission completes. On a clean datacenter link you never notice. On a mobile or residential exit with two to four percent loss, one dropped packet can pause forty concurrent scrapes at once, where six independent HTTP/1.1 sockets would have lost only the requests on the affected socket.

That is the one case where HTTP/1.1 genuinely wins on speed, and it is worth measuring rather than assuming. RFC 9114 solves the problem properly by moving HTTP/3 onto QUIC, where every stream gets independent loss recovery, but HTTP/3 through most proxy stacks is still awkward because UDP relay support is uneven.

If you are already tuning concurrency, treat h2 stream limits as a separate budget from socket limits. One h2 connection with SETTINGS_MAX_CONCURRENT_STREAMS = 100 is not the same shape of load as 100 HTTP/1.1 sockets, and your async scraping architecture needs to know which one it is running.

HTTP/2 compresses headers with HPACK, defined in RFC 7541. It has three moving parts: a fixed static table of 61 common field entries, a per-connection dynamic table that both peers keep in sync, and optional Huffman coding of literal values.

The compression is real. A second request on the same connection can shrink a 700-byte header block to a few dozen bytes, because most fields are already indexed. The catch for scrapers is that all of that state is per connection.

Here is the part most guides miss. Rotating your exit IP on every request forces a brand new TCP connection, a new TLS handshake and a new h2 preface every time. You throw away the dynamic table and pay full header cost, and more importantly you hand the target a fresh, complete copy of your h2 fingerprint on every single request. Per-request rotation does not hide you at the HTTP layer. It maximises the number of independent observations the defender collects, which is exactly what they need to cluster hundreds of unrelated IPs into one bot identity. Sticky sessions that hold an exit for a batch of requests leak less, not more.

HPACK choices are themselves fingerprintable: whether the client Huffman-encodes literals, whether it inserts fields into the dynamic table or sends them as literals without indexing, and what SETTINGS_HEADER_TABLE_SIZE it advertises. Chrome, Firefox and the Python h2 library each make different calls.

The canonical work here is Akamai's research paper, Passive Fingerprinting of HTTP/2 Clients by Elad Shuster and Ory Segal, presented at Black Hat EU 2017. It defines a compact string built from four fields of the connection opening, joined by pipes:

<SETTINGS>|<WINDOW_UPDATE>|<PRIORITY>|<PSEUDO_HEADER_ORDER>

• SETTINGS: each advertised parameter as id:value, joined by ;, in the order sent.
• WINDOW_UPDATE: the connection-level flow control increment, or 00 if the client sends none.
• PRIORITY: each PRIORITY frame as streamId:exclusivity:dependentStreamId:weight, joined by ,, or 0 if there are none.
• PSEUDO_HEADER_ORDER: the first letters of the pseudo-headers in the order emitted, such as m,a,s,p.

Commonly observed values (these drift with browser releases, so treat them as shapes rather than constants):

Chrome    1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p

Firefox   1:65536;4:131072;5:16384|12517377|3:0:0:201,5:0:0:101,7:0:0:1,
          9:0:7:1,11:0:3:1,13:0:0:241|m,p,a,s

Firefox is instantly recognisable because it opens with a set of PRIORITY frames that build a dependency tree for resource classes. Chrome sends none. Neither behaviour is required by the spec, which is precisely why it identifies them.

The SETTINGS identifiers come from RFC 9113 section 6.5.2:

Two clients can advertise identical values and still hash differently, because the fingerprint preserves the order the parameters were written in. That is the trap for anyone hand-rolling an h2 client: matching Chrome's numbers is not enough, you have to match Chrome's sequence.

One more wrinkle worth knowing. RFC 9113 marks the original stream-priority scheme from RFC 7540 as deprecated, and RFC 9218 defines a replacement that signals urgency through a Priority header field instead of PRIORITY frames. Clients migrated at different times and in different ways, so priority behaviour has become an even sharper discriminator than it was in 2017.

A defender sees three layers on one connection, and they must agree.

1. TLS ClientHello: cipher suites, extensions, curves, ALPN list. This is the JA3 or JA4 layer, covered in our guide to TLS fingerprinting.
2. HTTP/2 opening: SETTINGS, WINDOW_UPDATE, PRIORITY, pseudo-header order.
3. Request content: User-Agent, header set, header order, Client Hints.

Claiming Chrome in layers one and three while emitting a library fingerprint in layer two is a hard contradiction. No real Chrome build has ever produced that combination, so classifying it needs no machine learning. A lookup table does it.

Go's standard library is the clearest example, and you can read the behaviour straight from the source. In golang.org/x/net/http2/transport.go the initial settings list is assembled as ENABLE_PUSH first, then INITIAL_WINDOW_SIZE (transportDefaultStreamFlow, 4 MiB), then MAX_FRAME_SIZE, then optionally MAX_HEADER_LIST_SIZE and HEADER_TABLE_SIZE. It follows with a connection-level WINDOW_UPDATE of transportDefaultConnFlow, 1 GiB, and sends no PRIORITY frames at all. Chrome writes HEADER_TABLE_SIZE first. Go writes it last, or not at all.

The pseudo-header order is even more damning. Go's request encoder emits them like this:

// golang.org/x/net/internal/httpcommon/request.go
f(":authority", host)
f(":method", m)
f(":path", path)
f(":scheme", req.URL.Scheme)
// fingerprint field: a,m,p,s

Chrome emits m,a,s,p. Firefox emits m,p,a,s. Four pseudo-headers, one ordering choice, and a Go binary wearing a Chrome User-Agent is identified before it has sent a byte of request body.

Turning h2 on in httpx is one flag, and it works through a proxy:

import httpx

client = httpx.Client(
    http2=True,
    proxy="http://user:pass@proxy.sparkproxy.io:8000",
    timeout=30.0,
)
r = client.get("https://www.sparkproxy.io/")
print(r.http_version)  # "HTTP/2"

That gets you HTTP/2. It does not get you a browser fingerprint, because httpx delegates framing to the h2 library, which has its own SETTINGS shape. The same distinction runs through the wider requests vs httpx comparison: async and h2 are ergonomics wins, not stealth wins.

For stealth, the impersonating clients are the only shortcut that holds up, because they patch the TLS stack and the h2 layer in one place:

from curl_cffi import requests

r = requests.get(
    "https://www.sparkproxy.io/",
    impersonate="chrome131",
    proxies={"https": "http://user:pass@proxy.sparkproxy.io:8000"},
)
print(r.http_version, r.status_code)

Setup and version pinning are covered in the curl_cffi scraping guide.

Scrapy needs an explicit handler swap, and its h2 support is still documented as carrying limitations:

# settings.py
DOWNLOAD_HANDLERS = {
    "https": "scrapy.core.downloader.handlers.http2.H2DownloadHandler",
}

Node's undici keeps h2 behind a flag:

import { Agent, request } from 'undici'

const agent = new Agent({ allowH2: true })
const { statusCode } = await request('https://www.sparkproxy.io/', { dispatcher: agent })

For years, sticking to HTTP/1.1 was the safe default. That has inverted. Every current browser negotiates h2 or h3, and the HTTP Archive Web Almanac puts HTTP/2 and HTTP/3 at the clear majority of desktop and mobile requests. A connection that claims to be Chrome and then speaks HTTP/1.1 is now the anomaly.

Three specific ways an HTTP/1.1-only client gives itself away:

The ALPN list

Your ClientHello advertises the protocols you support. A copied Chrome JA3 with an ALPN list offering only http/1.1 is internally inconsistent, and it is visible before a single HTTP byte is exchanged.

Silent downgrade

Some clients offer h2 in ALPN, accept the server's h2 selection, and then a proxy layer or a config flag forces them back to HTTP/1.1 anyway. Servers log the mismatch.

Connection-specific headers

RFC 9113 section 8.2.2 forbids Connection, Keep-Alive, Transfer-Encoding, Upgrade and their relatives in HTTP/2 messages. Browsers stopped emitting them habitually. A hand-built HTTP/1.1 request that still sends Connection: keep-alive next to a modern Chrome User-Agent is announcing that its header set was written by a person, not generated by a browser.

The takeaway is not "always use h2". It is that your protocol version has to be consistent with the identity your other layers claim. An honest client that never pretends to be a browser often survives longer on a permissive target than a half-convincing impersonation.

Protocol choice at the proxy layer interacts with h2 in ways that surprise people. Tunnels are transparent. Plain forwarding is not.

HTTPS via CONNECT is fully transparent

An HTTP proxy handling an https:// target opens a tunnel with the CONNECT method (RFC 9110 section 9.3.6) and then relays raw bytes. Your TLS handshake, ALPN negotiation and every h2 frame travel end to end untouched, and the proxy sees only ciphertext. HTTP/2 works, and your h2 fingerprint reaches the origin exactly as your client wrote it. Our explainer on what an HTTP tunnel is walks through the handshake.

SOCKS5 is transparent too

It relays TCP below the HTTP layer, so it has no opinion about protocol versions. The trade-offs between these options are laid out in our guide to HTTP, HTTPS and SOCKS5 proxy protocols.

Plain HTTP forwarding is HTTP/1.1 in practice

For an http:// target, a classic forward proxy receives an absolute-form request target rather than a tunnel. That path is HTTP/1.1 shaped, and you get no h2 to the origin. Scraping plain HTTP through a forward proxy quietly downgrades you.

Check your client, not just your proxy

The tunnel can be perfectly transparent while your own library disables h2 the moment you configure it. Go is the classic trap. The Transport.ForceAttemptHTTP2 field documents that "use of any those fields conservatively disables HTTP/2" when a custom Dial, DialContext, DialTLS or TLSClientConfig is set. Scrapers routinely set TLSClientConfig to pin a cipher list or to skip verification against a MITM proxy, and drop to HTTP/1.1 without noticing:

tr := &http.Transport{
    Proxy:             http.ProxyURL(proxyURL),
    TLSClientConfig:   &tls.Config{MinVersion: tls.VersionTLS12},
    ForceAttemptHTTP2: true, // without this you silently get HTTP/1.1
}

Avoid TLS-terminating middleboxes

Any proxy that decrypts and re-encrypts replaces your ClientHello and re-frames your h2 connection with its own SETTINGS. The origin then fingerprints the middlebox rather than you, and that fingerprint is shared with everyone else behind it.

Do not guess. Capture. Three approaches that need no third-party service:

Verify the negotiated version first:

curl --http2 -sv https://www.sparkproxy.io/ -o /dev/null 2>&1 | grep "using HTTP"

Then run a local h2 server and read the frames. nghttp2 ships nghttpd, whose verbose mode prints every SETTINGS, WINDOW_UPDATE, PRIORITY and HEADERS frame your client sends. Point your scraper at it and read the fingerprint fields directly:

nghttpd -v 8443 server.key server.crt

Finally, decrypt a live capture. Set SSLKEYLOGFILE, capture with tcpdump, then load both into Wireshark with the key log configured. That shows the real frames sent to a real target through your real proxy chain, which is the only test that covers the whole stack:

export SSLKEYLOGFILE=/tmp/keys.log
tcpdump -i any -w /tmp/h2.pcap 'tcp port 443'

Record the fingerprint string for every client in your fleet and diff it after each dependency bump. Library upgrades change SETTINGS values, and a fingerprint that quietly shifted is a common cause of "it worked last month".

That last row is the honest answer for most teams. Keeping a hand-built h2 fingerprint aligned with a moving browser release train is continuous work, not a one-time fix. The SparkProxy Scraping API handles TLS and h2 matching, rotation and retries behind one endpoint:

curl "https://scrape.sparkproxy.io/api/v1?url=https://www.sparkproxy.io/&render_js=false&premium_proxy=true&country_code=US" \
  -H "X-API-Key: YOUR_API_KEY"

For hardened targets, add the browser and the extra anti-bot layers:

import requests

resp = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": "YOUR_API_KEY"},
    params={
        "url": "https://www.sparkproxy.io/pricing",
        "render_js": "true",
        "stealth": "true",
        "premium_proxy": "true",
        "country_code": "US",
        "wait_for": "#pricing-table",
    },
    timeout=120,
)
print(resp.status_code, resp.headers.get("X-Credits-Used"))

Plain HTTP mode (render_js=false) returns X-Job-Id, X-Credits-Used and X-Duration-Ms response headers, which is enough to track cost and latency per target without extra instrumentation. The full parameter reference lives in the SparkProxy Scraping API docs.