How to Rotate Proxies in Node.js

Send a few hundred requests from one IP and the target starts rate-limiting you, serving CAPTCHAs, or blocking the address outright. A rotating pool spreads traffic across many IPs so no single address crosses the site's detection threshold. If you want the conceptual background before the code, read what proxy rotation is and how it works.

The strategy you pick changes both throughput and how likely you are to get blocked:

Per-request rotation is the default for scraping content that does not care about session state. Sticky rotation is mandatory the moment a site tracks you across requests through cookies or a login. Choosing per-request rotation on a logged-in site is the fastest way to trip a security challenge.

If you also write Python, the same four strategies map almost one to one onto how to rotate proxies in Python. The concurrency model is the part that differs, and that difference matters below.

Node's HTTP clients route through a proxy by way of an Agent. For scraping, almost every target is https://, and HTTPS through a forward proxy uses the CONNECT tunneling method. The https-proxy-agent package handles that tunnel.

npm install https-proxy-agent axios got node-fetch

import { HttpsProxyAgent } from 'https-proxy-agent';

// Proxy credentials go in the URL: http://user:pass@host:port
const proxyUrl = 'http://user:pass@proxy-1.sparkproxy.io:10000';
const agent = new HttpsProxyAgent(proxyUrl);

The single most common mistake here is picking the wrong agent for the target scheme. The rule:

Use https-proxy-agent for https:// targets even though the proxy URL itself starts with http://. The http:// scheme describes how you reach the proxy; the CONNECT tunnel it opens carries your TLS handshake end to end to the destination. Reaching for http-proxy-agent on an https:// URL produces a broken request or a plaintext leak, not a helpful error.

Round-robin walks the pool in a fixed repeating order, giving every proxy equal use. In Node this is a bare integer index, no locking required.

const PROXIES = [
  'http://user:pass@proxy-1.sparkproxy.io:10000',
  'http://user:pass@proxy-2.sparkproxy.io:10000',
  'http://user:pass@proxy-3.sparkproxy.io:10000',
];

let index = 0;
function nextProxy() {
  const proxy = PROXIES[index];
  index = (index + 1) % PROXIES.length;
  return proxy;
}

Here is a Node-specific detail worth internalizing. Python's itertools.cycle is not thread-safe and needs a lock under ThreadPoolExecutor. Node runs your JavaScript on a single-threaded event loop, so index = (index + 1) % PROXIES.length never interleaves with itself: two in-flight requests cannot corrupt the counter. You get correct round-robin for free.

The caveat is cluster and worker_threads. Each worker is a separate V8 isolate with its own copy of index, so an 8-worker cluster runs 8 independent round-robins. That is usually fine for even distribution across a large pool, but if you need one global sequence you must share state through Redis or a parent-process message channel, not a module-level variable.

Random selection avoids the predictable request ordering that some anti-bot systems fingerprint. It is stateless, which makes it the simplest option across workers.

function randomProxy() {
  return PROXIES[Math.floor(Math.random() * PROXIES.length)];
}

Weighted rotation is what you want with a mixed pool. Give reliable, low-latency IPs a higher weight so they take more traffic while flaky ones still get occasional use.

const WEIGHTED = [
  { url: 'http://user:pass@proxy-1.sparkproxy.io:10000', weight: 5 },
  { url: 'http://user:pass@proxy-2.sparkproxy.io:10000', weight: 3 },
  { url: 'http://user:pass@proxy-3.sparkproxy.io:10000', weight: 1 },
];

const totalWeight = WEIGHTED.reduce((sum, p) => sum + p.weight, 0);

function weightedProxy() {
  let r = Math.random() * totalWeight;
  for (const p of WEIGHTED) {
    r -= p.weight;
    if (r < 0) return p.url;
  }
  return WEIGHTED.at(-1).url;
}

The loop subtracts each weight from a random point on the cumulative range and returns the first proxy that crosses zero. A proxy with weight 5 is selected five times as often as one with weight 1. Feed the weights from live metrics (success rate, median latency) and the pool tunes itself.

Some jobs must keep the same IP for a sequence of requests: a login followed by paginated data, or a cart flow. Map a session key to a proxy and reuse it until the session ends.

import { HttpsProxyAgent } from 'https-proxy-agent';

// Cache one agent per proxy URL. See the note below.
const agentCache = new Map();
function agentFor(proxyUrl) {
  if (!agentCache.has(proxyUrl)) {
    agentCache.set(proxyUrl, new HttpsProxyAgent(proxyUrl, { keepAlive: true }));
  }
  return agentCache.get(proxyUrl);
}

const sessionProxies = new Map();
function stickyProxy(sessionId) {
  if (!sessionProxies.has(sessionId)) {
    sessionProxies.set(sessionId, nextProxy());
  }
  return sessionProxies.get(sessionId);
}

Notice the agentCache. This is the leak nobody warns you about: a new HttpsProxyAgent per request opens a new socket pool every time and never reuses connections, so under load you burn through file descriptors and eventually hit EMFILE: too many open files. Cache one keepAlive agent per unique proxy URL and let it pool sockets. With a 200-proxy rotation you hold 200 agents, not one per request.

Each client wires the agent in differently, and two of them have traps.

axios (version 1.7+) needs proxy: false alongside the agent. Without it, axios applies its own proxy layer on top of your agent and the request either bypasses the proxy or double-tunnels.

import axios from 'axios';

const res = await axios.get('https://www.sparkproxy.io/ip', {
  httpsAgent: agentFor(nextProxy()),
  proxy: false, // let the agent own the tunnel; disable axios's built-in handling
  timeout: 15000,
});
console.log(res.data);

got (version 14, ESM only) takes an agent object keyed by protocol:

import got from 'got';
import { HttpsProxyAgent } from 'https-proxy-agent';

const res = await got('https://www.sparkproxy.io/ip', {
  agent: { https: new HttpsProxyAgent(nextProxy()) },
}).json();

node-fetch (version 3, ESM only) accepts a single agent:

import fetch from 'node-fetch';
import { HttpsProxyAgent } from 'https-proxy-agent';

const res = await fetch('https://www.sparkproxy.io/ip', {
  agent: new HttpsProxyAgent(nextProxy()),
});
console.log(await res.json());

Now the trap that eats the most time. Node's built-in global fetch (powered by undici, stable since Node 21 and standard in Node 22 LTS) does not read the agent option at all. Pass an agent to it and your request goes out on your real IP with zero errors and zero warnings. The proxy is simply ignored. The fix is undici's ProxyAgent, passed as dispatcher:

import { ProxyAgent } from 'undici';

const dispatcher = new ProxyAgent(nextProxy());

const res = await fetch('https://www.sparkproxy.io/ip', { dispatcher }); // global fetch
console.log(await res.json());

If you would rather route every fetch through a proxy without threading dispatcher through each call, set it globally with import { setGlobalDispatcher } from 'undici'. Just remember that a global dispatcher is one proxy, so for real rotation you still pass a per-request dispatcher from your pool.

A proxy will fail. It times out, returns 407, or the target throws a 429. The pattern that works is: catch the failure, rotate to the next proxy, and back off before retrying so you do not hammer a rate limit.

import axios from 'axios';

const RETRYABLE = new Set([407, 408, 429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function fetchWithRotation(url, { maxRetries = 4 } = {}) {
  let lastError;
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const proxyUrl = nextProxy(); // fresh proxy every attempt
    try {
      const res = await axios.get(url, {
        httpsAgent: agentFor(proxyUrl),
        proxy: false,
        timeout: 15000,
        validateStatus: (s) => s < 400 || !RETRYABLE.has(s),
      });
      return res.data;
    } catch (err) {
      lastError = err;
      const status = err.response?.status;
      const retryable = !status || RETRYABLE.has(status);
      if (!retryable || attempt === maxRetries) break;
      const backoff = Math.min(1000 * 2 ** attempt, 15000);
      const jitter = Math.random() * 250; // spread out concurrent retries
      await sleep(backoff + jitter);
    }
  }
  throw lastError;
}

Two decisions make this reliable. Rotate the proxy on every attempt, not just the first, so a dead IP never gets a second try in the same call. Add jitter to the backoff so that when 50 concurrent requests all hit a 429 at once, they do not retry in lockstep and re-trigger the limit. Doubling the delay (1s, 2s, 4s, 8s) capped at 15 seconds keeps retries polite. For the broader picture on why requests fail and how to stay under the radar, see how to avoid getting your proxy blocked.

Copying rotation logic into every script gets old. This class bundles the strategy, agent caching, retry, and a cooldown that benches proxies that fail so they get a rest before rejoining the pool.

import { HttpsProxyAgent } from 'https-proxy-agent';
import axios from 'axios';

export class RotatingProxyPool {
  constructor(proxies, { strategy = 'round-robin', maxRetries = 4 } = {}) {
    this.proxies = proxies.map((p) =>
      typeof p === 'string' ? { url: p, weight: 1 } : p
    );
    this.strategy = strategy;
    this.maxRetries = maxRetries;
    this.index = 0;
    this.agents = new Map();
    this.cooldown = new Map(); // url -> timestamp when it may be used again
  }

  agentFor(url) {
    if (!this.agents.has(url)) {
      this.agents.set(url, new HttpsProxyAgent(url, { keepAlive: true }));
    }
    return this.agents.get(url);
  }

  live() {
    const now = Date.now();
    return this.proxies.filter((p) => (this.cooldown.get(p.url) ?? 0) < now);
  }

  pick() {
    const pool = this.live();
    if (pool.length === 0) throw new Error('No live proxies available');
    if (this.strategy === 'random') {
      return pool[Math.floor(Math.random() * pool.length)].url;
    }
    if (this.strategy === 'weighted') {
      const total = pool.reduce((s, p) => s + p.weight, 0);
      let r = Math.random() * total;
      for (const p of pool) { r -= p.weight; if (r < 0) return p.url; }
      return pool.at(-1).url;
    }
    const p = pool[this.index % pool.length];
    this.index++;
    return p.url;
  }

  bench(url, cooldownMs = 60000) {
    this.cooldown.set(url, Date.now() + cooldownMs);
  }

  async get(url, config = {}) {
    let lastError;
    for (let attempt = 0; attempt <= this.maxRetries; attempt++) {
      const proxyUrl = this.pick();
      try {
        const res = await axios.get(url, {
          httpsAgent: this.agentFor(proxyUrl),
          proxy: false,
          timeout: 15000,
          ...config,
        });
        return res.data;
      } catch (err) {
        lastError = err;
        const status = err.response?.status;
        if (!status || status === 407) this.bench(proxyUrl); // dead or auth-failing
        if (attempt === this.maxRetries) break;
        await new Promise((r) => setTimeout(r, Math.min(1000 * 2 ** attempt, 15000)));
      }
    }
    throw lastError;
  }
}

Using it is three lines:

const pool = new RotatingProxyPool(
  [
    'http://user:pass@proxy-1.sparkproxy.io:10000',
    'http://user:pass@proxy-2.sparkproxy.io:10000',
    'http://user:pass@proxy-3.sparkproxy.io:10000',
  ],
  { strategy: 'weighted' }
);

const data = await pool.get('https://www.sparkproxy.io/ip');

The cooldown map is the piece most homemade pools skip. A proxy that returns a connection error or 407 gets benched for 60 seconds instead of being retried immediately, which stops one bad IP from poisoning every request. When the whole pool is benched, live() returns empty and pick() throws, which is the signal to widen your pool or slow down. For managing per-IP request budgets across a larger pool, the patterns in this guide to rotating proxies and per-IP request limits pair well with this class.

Managing a pool yourself gives you full control. Sometimes you would rather not. The SparkProxy Scraping API rotates the exit IP on every request server-side, so there is no pool, no agent, and no retry loop to maintain. You send a target URL, it returns the response.

import axios from 'axios';

const res = await axios.get('https://scrape.sparkproxy.io/api/v1', {
  headers: { 'X-API-Key': process.env.SPARKPROXY_API_KEY }, // key format: sk-...
  params: {
    url: 'https://example.com/products',
    render_js: false,    // 1 credit; set true for JS-heavy pages (5 credits)
    country_code: 'us',  // geo-target the exit IP
    premium_proxy: true, // route through residential IPs
  },
  timeout: 60000,
});

console.log(res.data);

Authentication is the X-API-Key header carrying a key of the form sk-... from your dashboard. country_code picks the exit geography, render_js runs a real browser when the page needs JavaScript, and premium_proxy upgrades to residential IPs for tougher targets. Because the API assigns a fresh IP per call, the rotation, retries, and health checks you built above are handled for you. A practical split: run your own RotatingProxyPool for high-volume, simple targets where per-request cost matters, and send the hard, JavaScript-heavy, or heavily defended pages to the Scraping API. If you are comparing that trade-off in Python land, the async scraping with requests and aiohttp guide covers the same decision from the client-managed side.