Node Fetch Proxy: Fix fetch Ignoring HTTP_PROXY

This is the failure in its purest form. The shell has the variable, the code is the one-liner everyone writes, and the response is fine:

export HTTPS_PROXY="http://USER:PASS@gateway.sparkproxy.io:11000"
node -e "fetch('https://api.ipify.org/?format=json').then(r => r.text()).then(console.log)"
# {"ip":"<your server's own public IP>"}

We reproduced it with a logging proxy on the loopback interface, so we could count what actually arrived at the proxy. With HTTPS_PROXY and HTTP_PROXY both set and no other configuration, Node 24.22.0 sent the request directly and the proxy recorded zero connections. A second variant that hurts more: passing an https-proxy-agent instance as agent to the built-in fetch. Same result, zero connections at the proxy, no warning. The agent option belongs to node-fetch and http.request; the global fetch ignores unknown init keys.

So the question to hold onto through the rest of this page is not "is my proxy broken?" but "did the request ever reach it?" Most of the time, it didn't.

Node's global fetch is not a wrapper around http.request. The Node documentation states that "the implementation is based upon undici, an HTTP/1.1 client written from scratch for Node.js", and exposes the bundled version as process.versions.undici (Node 24.22.0 reports 7.29.1). Fetch left the --experimental-fetch flag in v18.0.0 and stopped being experimental in v21.0.0, according to the same page's change history.

Every undici request goes through a dispatcher. The default global dispatcher is a plain Agent that opens a direct TCP connection to the target origin. It does not consult process.env at all. Environment proxy variables are a convention inherited from curl and wget, not part of the WHATWG Fetch standard that Node's fetch implements, and undici chose to keep that behaviour opt-in.

That leaves three places where the proxy can be wired in, and they map to the three fixes:

Versions in that table come from the undici API docs (ProxyAgent.md, EnvHttpProxyAgent.md) and Node's cli.md, http.md and v24 changelog, all read on 30 October 2026.

Run these in order and stop at the first one that explains it. The early checks cost seconds; the later ones need a code change.

// diag.mjs - run with the SAME launcher your app uses (pm2, systemd, docker, npm script)
console.log({
  node: process.versions.node,
  undici: process.versions.undici,
  HTTPS_PROXY: process.env.HTTPS_PROXY ?? process.env.https_proxy ?? '(unset)',
  NO_PROXY: process.env.NO_PROXY ?? process.env.no_proxy ?? '(unset)',
  NODE_USE_ENV_PROXY: process.env.NODE_USE_ENV_PROXY ?? '(unset)',
  execArgv: process.execArgv,
});

1. Does the process even see the variable? A shell export does not reach a service started by systemd, a container without -e, or a pm2 daemon that was launched before you exported it. sudo drops most of the environment by default. If HTTPS_PROXY prints (unset), no fix in this article helps until it doesn't.
2. Is the target excluded by NO_PROXY? We confirmed that with the env flag on and NO_PROXY=api.ipify.org, the request went direct. Corporate images often ship a broad NO_PROXY.
3. Which client sent the request? Global fetch, node-fetch, axios and got all have different proxy rules. If a library bundles its own HTTP client, none of the undici fixes below apply to it.
4. Is any proxy opt-in present? No NODE_USE_ENV_PROXY, no --use-env-proxy in execArgv, and no setGlobalDispatcher call means fetch is going direct by design.
5. Does the opt-in fail loudly? A TypeError: fetch failed with a cause means the request did reach for a proxy. That is progress, and the cause tells you which section to read next.

The most explicit fix is to hand fetch a dispatcher that knows about the proxy. It works on every Node version that has fetch, it never depends on the environment, and it lets different calls use different exits.

// npm i undici@7   (match the major Node bundles; see the next section)
import { ProxyAgent } from 'undici';

const user = process.env.SPARKPROXY_USER;
const pass = encodeURIComponent(process.env.SPARKPROXY_PASS); // escape @ : # / in passwords

// Create ONCE and reuse. Each ProxyAgent owns its own connection pool.
const viaGateway = new ProxyAgent(`http://${user}:${pass}@gateway.sparkproxy.io:11000`);

const res = await fetch('https://api.ipify.org/?format=json', { dispatcher: viaGateway });
console.log(res.status, await res.json());

Credentials can live in the URL, as above, or in the token option as a ready-made Proxy-Authorization value. The undici docs say the header is derived "in order of precedence, from token, then auth, then the username and password embedded in the proxy uri", that URI credentials "are URL-decoded before being base64-encoded", and that auth is deprecated in favour of token.

import { ProxyAgent } from 'undici';

const viaGateway = new ProxyAgent({
  uri: 'http://gateway.sparkproxy.io:11000',
  token: 'Basic ' + Buffer.from(`${process.env.SPARKPROXY_USER}:${process.env.SPARKPROXY_PASS}`).toString('base64'),
});

The token form sidesteps URL escaping entirely, which matters for the next error. In our test, a password containing @ and # left unescaped in the URI made the ProxyAgent constructor throw InvalidArgumentError: invalid url before any connection was attempted. The same password wrapped in encodeURIComponent connected and authenticated.

If you also need rotation pools, weighted selection or retries across several proxy URLs, that is a separate job covered in rotating proxies in Node.js. Here the goal is narrower: get one fetch to leave through the gateway.

This one costs people an evening because the fix looks correct and fails anyway. Install the latest undici from npm, pass its ProxyAgent to the built-in fetch, and you get:

TypeError: fetch failed
  [cause]: InvalidArgumentError: invalid onRequestStart method
      at assertRequestHandler (...\node_modules\undici\lib\core\util.js:597:11)
      ...
      at ProxyAgent.dispatch (...\node_modules\undici\lib\dispatcher\proxy-agent.js:288:25)
      at node:internal/deps/undici/undici:13891:55 {
    code: 'UND_ERR_INVALID_ARG'
  }

We captured that on Node 24.22.0 (bundled undici 7.29.1) with npm undici@8.12.3, the latest tag on 30 October 2026. The global fetch is undici 7 code building a request handler; the dispatcher you passed is undici 8 code validating it against a newer handler interface. They disagree, and the proxy logged zero connections. With undici@7.30.1 (the seven dist-tag, published 2026-10-15) the identical code connected through the proxy and returned 200.

Two clean ways out. Either pin undici to the same major as process.versions.undici, or stop mixing and import fetch from the package too, so both halves come from one version:

import { fetch, ProxyAgent } from 'undici'; // fetch and dispatcher from the same release
const res = await fetch('https://api.ipify.org/?format=json', {
  dispatcher: new ProxyAgent(process.env.HTTPS_PROXY),
});

The fourth row is a measured observation, not a documented guarantee. The global-dispatcher path worked across majors in our run; don't rely on that surviving the next release.

When you want every fetch in the process to honour the standard variables (including calls buried in your dependencies), replace the global dispatcher once at startup. EnvHttpProxyAgent was added in undici v6.14.0 for exactly this, and Node's globals documentation confirms that setGlobalDispatcher() from the npm package "will affect both undici and Node.js".

// proxy-bootstrap.mjs - import this before anything that makes requests
import { EnvHttpProxyAgent, setGlobalDispatcher } from 'undici';

setGlobalDispatcher(new EnvHttpProxyAgent());
// From here on, global fetch reads http_proxy / https_proxy / no_proxy

HTTPS_PROXY="http://USER:PASS@gateway.sparkproxy.io:11000" \
NO_PROXY="localhost,127.0.0.1,.internal.sparkproxy.io" \
node --import ./proxy-bootstrap.mjs server.mjs

The rules it follows are worth knowing exactly, because they differ from some other tools:

That is undici's own description in EnvHttpProxyAgent.md, and we confirmed the first row: with only HTTP_PROXY set, an HTTPS request still arrived at the proxy as a CONNECT. The docs also note that no_proxy is re-read from the environment on demand, so changes after startup apply to later requests, unless you passed noProxy as a constructor option.

Watch the URL format. HTTPS_PROXY=127.0.0.1:28480 (no scheme) produced TypeError: Invalid URL the first time fetch ran. Always include http://.

Newer Node releases can do Fix 2 without any code. Set NODE_USE_ENV_PROXY=1, or start Node with --use-env-proxy, and the runtime "parses the HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables during startup, and routes requests through the specified proxy" (Node cli.md). The v24 changelog adds that "fetch() already supports NODE_USE_ENV_PROXY as of Node.js 24.0.0"; v24.5.0 (released 2025-07-31) extended it to node:http and node:https and added the CLI flag. The 22.x line got both in v22.21.0. When both forms are set, the flag takes precedence.

# Either form works on Node 24.5.0+ and 22.21.0+
NODE_USE_ENV_PROXY=1 HTTPS_PROXY="http://USER:PASS@gateway.sparkproxy.io:11000" node app.mjs
HTTPS_PROXY="http://USER:PASS@gateway.sparkproxy.io:11000" node --use-env-proxy app.mjs

For a Dockerfile or a systemd unit, set it next to the proxy variables so they travel together:

# /etc/systemd/system/scraper.service (excerpt)
[Service]
Environment=NODE_USE_ENV_PROXY=1
Environment=HTTPS_PROXY=http://USER:PASS@gateway.sparkproxy.io:11000
Environment=NO_PROXY=localhost,127.0.0.1

If you can't touch the launcher, Node v25.4.0 and v24.14.0 added http.setGlobalProxyFromEnv(), which applies the same configuration at runtime and returns a function that restores the previous agent and dispatcher. The docs recommend calling it before any requests are made. It worked in our Node 24.22.0 run.

import http from 'node:http';
const restore = http.setGlobalProxyFromEnv(); // reads process.env by default
const res = await fetch('https://api.ipify.org/?format=json');
console.log(await res.text());

Three cautions from the docs and our runs. The feature is marked Stability: 1.1, Active Development, so expect refinements. The built-in support accepts HTTP and HTTPS proxy URLs only; a socks5:// value is not on the documented list, so the SOCKS5 port needs a different client (see the SOCKS5 setup guide). And a schemeless HTTPS_PROXY with the env flag on crashed our process at startup with TypeError: Invalid URL, before a single line of application code ran.

Don't trust a 200. Compare the IP the target sees with and without the proxy, from the same process, in the same run. This is the check that would have caught the original symptom on day one.

// egress-check.mjs
import { ProxyAgent } from 'undici';

const ECHO = 'https://api.ipify.org/?format=json';
const proxied = new ProxyAgent(process.env.HTTPS_PROXY);

const direct = (await (await fetch(ECHO, { dispatcher: undefined })).json()).ip;
const viaProxy = (await (await fetch(ECHO, { dispatcher: proxied })).json()).ip;

console.log({ direct, viaProxy });
if (direct === viaProxy) {
  console.error('FAIL: proxied request left from the server IP');
  process.exit(1);
}

Run it under the same launcher as the real service, as a startup assertion or a CI step. If you rely on Fix 2 or Fix 3 instead of a per-request agent, swap the second call for a plain fetch(ECHO) and run the first one with NO_PROXY=api.ipify.org so it goes direct. For browser, curl and Python versions of the same idea, plus latency checks, see how to test if your proxy is working.

One more thing our logging proxy showed: for an HTTPS target, undici sent CONNECT api.ipify.org:443, the hostname rather than a resolved address. Name resolution for the target happens at the gateway, not on your server, so a local DNS resolver is not leaking the destination's lookup in that path.

SparkProxy supports two authentication modes: user:pass credentials, and an IP whitelist where your server's address is authorised and no credentials are sent. Both fail the same way from Node's point of view, and the message does not say "407".

Our test proxy answered the CONNECT with HTTP/1.1 407 Proxy Authentication Required, and undici surfaced it as a cancelled request in every path we tried. So when you see Request was cancelled. on a proxied call, check credentials first, then whether the server's IP is on the whitelist, then whether the account is active. Log err.cause rather than err; the top-level fetch failed is identical for every row above. For what the other gateway status codes mean, see proxy error codes explained.

try {
  await fetch(url, { dispatcher: viaGateway });
} catch (err) {
  const cause = err.cause ?? err;
  console.error(cause.code ?? '-', cause.message); // e.g. "ECONNREFUSED connect ECONNREFUSED ..."
  throw err;
}

Once fetch goes through the proxy, the next complaint is usually "random rotation is not rotating". The cause is on the client side. The undici docs say a ProxyAgent sets up the proxy connection "lazily when the first request to an origin is dispatched, then reused for subsequent requests to the same origin". For an HTTPS target, that connection is a CONNECT tunnel, and a tunnel is one TCP stream from the gateway to the target. No gateway, ours included, can change the exit IP in the middle of it.

We counted CONNECTs at the logging proxy for five requests to the same HTTPS origin:

That is a single small run against one echo service, so treat the sequential count as illustrative. The direction is the point: keep-alive makes sequential requests share exits. Pick the behaviour you want deliberately:

• A fresh exit per request on the rotating port (11000): open a new tunnel each time, either with a short-lived ProxyAgent that you close() afterwards, or by sending requests in parallel.
• The same exit across a login or a multi-page flow: use the Sticky Sessions port (11002), which holds an exit for SparkProxy's 5-minute window, and keep one shared ProxyAgent so keep-alive works for you rather than against you.

import { ProxyAgent } from 'undici';
const GW = `http://${process.env.SPARKPROXY_USER}:${encodeURIComponent(process.env.SPARKPROXY_PASS)}@gateway.sparkproxy.io`;

// New tunnel per call on the random-rotation port
async function fetchFreshExit(url) {
  const d = new ProxyAgent(`${GW}:11000`);
  try { return await (await fetch(url, { dispatcher: d })).text(); }
  finally { await d.close(); }
}

// One long-lived agent on the Sticky Sessions port for a session-bound flow
const sticky = new ProxyAgent(`${GW}:11002`);

Opening a tunnel per request costs a TCP and TLS handshake each time, so it is slower. It won't change your SparkProxy bill: plans are packs with unlimited bandwidth, billed by plan and concurrency rather than per GB, starting from $75/month (USD, as listed on sparkproxy.io on 9 October 2026), with 1M+ premium IPs across 80+ countries and a 24-hour free trial with no credit card.

Getting fetch onto the gateway fixes one class of problem: which IP the target sees. That covers per-IP rate limits, geo-restricted content, and an IP that a site has already blocked. It doesn't fix:

• Bot detection on the request itself. undici's TLS handshake and default headers are recognisably Node, whatever the exit. A site that rejects that fingerprint rejects it from a residential IP too.
• JavaScript-rendered pages. fetch returns the initial HTML. If the data arrives later, parsing with Cheerio finds nothing; see web scraping with Cheerio and Node.js for where that boundary sits.
• Clients that are not undici. axios, got and node-fetch have their own proxy handling; none of the three fixes here touch them.
• A misconfigured proxy URL. The errors in the authentication table are yours to fix; a different provider will not change them.

When you can't change the process's dispatcher at all (a third-party SDK, a locked-down serverless runtime), call the SparkProxy Scraping API with an ordinary fetch and let it make the outbound request. Base URL https://scrape.sparkproxy.io/api/v1, header X-API-Key, and 1,000 free credits on a new account. render_js defaults to true at 5 credits; render_js=false is a plain fetch at 1 credit.

const params = new URLSearchParams({
  url: 'https://api.ipify.org/?format=json',
  render_js: 'false',      // plain fetch, 1 credit (the default true costs 5)
  proxy_type: 'premium',
});
const res = await fetch(`https://scrape.sparkproxy.io/api/v1?${params}`, {
  headers: { 'X-API-Key': process.env.SPARKPROXY_API_KEY },
});
console.log(res.status, await res.text()); // the target's response, fetched from SparkProxy's side

If you add json_response=true to get the envelope (job_id, status_code, credits_used), remember its body field is base64 and must be decoded before you parse it. format=json is a content conversion, not that envelope.