IP Whitelisting for Proxies: How It Works & Setup

An IP whitelist (also called an IP allowlist) is a list of IP addresses that are explicitly permitted to connect to a system. Any address not on the list is denied by default. This is the opposite of a blacklist, which blocks known bad addresses while allowing everything else.

For proxies, the whitelist lives in the provider's access control layer. When a client initiates a connection, the proxy reads the source IP from the TCP connection header before any HTTP request is processed. If that IP appears in the whitelist, the connection is accepted. If it doesn't, the proxy returns 407 Proxy Authentication Required, even though no amount of credential-sending would help.

Allowlist (whitelist) vs denylist (blacklist): why the terminology matters

The security industry has moved toward "allowlist" and "denylist" as preferred terms over "whitelist" and "blacklist". The underlying concept is the same: an allowlist defines what is permitted; everything else is denied. A denylist defines what is blocked; everything else is permitted. For access control, allowlisting (defining permitted IPs explicitly) is the stronger security posture, since unknown addresses are blocked by default rather than allowed until blocked.

For the full picture of how proxies verify who is connecting, see our guide on how proxy authentication works, which covers IP whitelisting and username-password options side by side.

---

IP whitelisting for proxies operates at the TCP connection layer, before any HTTP headers are examined. Here is the sequence for an inbound connection from a whitelisted client:

1. Client → Proxy:   TCP SYN (source IP: 198.51.100.42)
2. Proxy:            Read source IP from TCP header
3. Proxy:            Check 198.51.100.42 against whitelist → MATCH
4. Proxy → Client:   TCP SYN-ACK (connection accepted)
5. Client → Proxy:   HTTP request (no Proxy-Authorization header needed)
6. Proxy → Target:   Forward request
7. Target → Proxy:   Response
8. Proxy → Client:   Relay response

For a non-whitelisted client:

1. Client → Proxy:   TCP SYN (source IP: 45.33.32.156)
2. Proxy:            Read source IP → NOT in whitelist
3. Proxy → Client:   407 Proxy Authentication Required
4. Connection ends

The key technical detail: the proxy reads the source IP from the actual TCP connection, not from any header in the HTTP request. Headers like X-Forwarded-For or X-Real-IP are not used for whitelist authentication. They are set by intermediary servers and could be spoofed. Only the real source IP of the TCP connection is authoritative.

---

Whitelisting an IP address for a proxy is done in your provider's dashboard, not in your code. The process has three steps regardless of provider:

Step 1: Find your current public IP

Your proxy provider sees the public IP of the machine that connects to the proxy. This is the IP that needs to be whitelisted. Find it by running:

# curl-based
curl https://ifconfig.me

# or via IP info
curl https://ipinfo.io/ip

This returns your public-facing IP (e.g. 198.51.100.42). Note that this may differ from your machine's local IP (192.168.x.x) if you're behind a NAT router.

Step 2: Add the IP in your provider dashboard

In your provider account, navigate to the IP Whitelist or Access Control section. Enter the IP address you found in Step 1. Some providers also accept CIDR notation for ranges (covered in the next section).

Step 3: Test the connection

# Test without credentials, should succeed if IP is whitelisted
curl --proxy proxy.sparkproxy.io:10000 https://httpbin.org/ip

# Expected: returns your proxy's egress IP, not your machine's IP

If you receive a 407 error, the whitelist entry hasn't propagated yet (some providers take 1 to 5 minutes), or the IP used in Step 1 doesn't match what the proxy sees.

Step 4: Use the proxy in your code, no credentials needed

import requests

# No username or password in the URL
proxies = {
    "http":  "http://proxy.sparkproxy.io:10000",
    "https": "http://proxy.sparkproxy.io:10000",
}

response = requests.get("https://httpbin.org/ip", proxies=proxies)
print(response.json())

# curl, no --proxy-user flag required
curl --proxy proxy.sparkproxy.io:10000 https://httpbin.org/ip

New to how a proxy reads your connection? Our primer on what an IP address is and why it matters for proxies explains the source IP that the whitelist checks on every request.

---

If you have multiple machines that need access, or if your infrastructure uses a range of IP addresses, entering each IP individually doesn't scale. Most proxy providers accept CIDR notation for whitelist entries, which lets you cover an entire IP range with a single entry.

CIDR notation expresses an IP range as an address and a prefix length: address/prefix. As defined in RFC 4632, "a prefix is shown as a 4-octet quantity, just like a traditional IPv4 address or network number, followed by the '/' character, followed by a decimal value between 0 and 32 that describes the number of significant bits" (RFC 4632, Classless Inter-domain Routing).

The prefix length determines how many addresses the range covers:

Practical example: whitelisting a cloud provider IP range

If your scraping infrastructure runs on cloud VMs that are assigned IPs from the block 198.51.100.0/27 (32 addresses), you enter that single CIDR string in your dashboard rather than 32 individual IPs. If the VM pool scales up within that range, new IPs are covered automatically.

# Check whether an IP falls in a CIDR range (Python)
import ipaddress

whitelist = ipaddress.ip_network("198.51.100.0/27")
test_ip   = ipaddress.ip_address("198.51.100.19")

print(test_ip in whitelist)  # True

Important: CIDR ranges should match your actual IP allocation. Your ISP or cloud provider assigns you a block. Don't whitelist larger-than-necessary ranges. A /24 (256 IPs) where you only need /32 (1 IP) increases the attack surface if any of those addresses is later used maliciously.

---

Neither method is universally better. The choice depends on your infrastructure:

• IP whitelisting excels on dedicated servers with static IPs where the IP is stable, managed, and unlikely to change. It keeps credentials out of your codebase entirely.
• Username-password excels on distributed, dynamic, or cloud-scaled infrastructure where machines come and go. A single credential set works across all of them without dashboard changes.

---

IP whitelisting is a strong fit for some setups and a poor fit for others. Here is the honest tradeoff before you commit to it.

Pros:

• No credentials in your code. There is nothing to leak, rotate, or accidentally commit to a repository.
• Simpler connection strings. Your proxy URL carries no username or password.
• Strong default-deny posture. Any IP that is not on the list is blocked automatically.
• Fast revocation. Remove an IP from the dashboard and access stops right away.
• Clean audit trail. Access maps to a known set of IPs that you control.

Cons:

• It needs a static IP. A changing IP breaks access with no clear error.
• It is weak in shared environments. One NAT entry can grant access to every device behind it.
• It suits auto-scaling poorly. New cloud instances often get IPs the whitelist has never seen.
• It is not portable. A developer moving between office, home, and a cafe hits a different IP at each location.
• It fails silently. A 407 does not tell you the IP changed, so debugging takes longer than it should.

The short version: whitelisting rewards stable, controlled infrastructure and punishes anything that moves.

---

The most underappreciated limitation of IP whitelisting is what happens in shared network environments. Network Address Translation (NAT) maps many private IP addresses to a single public IP. In practice, this means:

• All devices in an office share one public IP
• All VMs behind a cloud NAT gateway share that gateway's IP
• All devices on a coworking space network share one public IP
• A mobile device tethering to a laptop shares one carrier-assigned IP

The hidden tradeoff: When you whitelist your office's public IP, you're not granting access to your specific machine. You're granting access to every device behind that NAT. That includes every laptop, every mobile device, every IoT device, and any guest on the office WiFi, as long as they share the same public IP.

Username-password authentication doesn't have this problem. Each machine or scraping job can have its own credential set. IP whitelisting can't distinguish between machines behind the same NAT.

The bottom line: IP whitelisting is most secure (and most reliable) when used on a single machine or tightly-controlled cluster where each public IP maps to exactly your infrastructure and no one else's.

---

Before you whitelist an IP address, you need to know whether it's static or dynamic.

Static IP: The IP address assigned to your machine or connection does not change between sessions. It's the same after a reboot, after a lease renewal, and tomorrow morning. Dedicated servers, cloud instances with static/Elastic IPs, and business internet plans from many ISPs provide static IPs.

Dynamic IP (DHCP): The IP address is assigned from a pool each time your machine connects and can change between sessions. Most residential internet connections, mobile data connections, and cloud VMs on shared NAT use dynamic IPs.

How to check if your IP is static:

# Record your IP now
curl https://ifconfig.me

# Disconnect and reconnect your internet connection (or reboot)
# then check again
curl https://ifconfig.me

# If both outputs are the same: likely static (or DHCP with long lease)
# If different: dynamic, do NOT rely on IP whitelisting

A more reliable method: contact your ISP or cloud provider and confirm whether the IP is statically assigned, or check your cloud console for "Static IP" / "Elastic IP" / "Reserved IP" assignment.

Cloud provider static IP options:

---

The mechanics of IP whitelisting are identical for IPv4 and IPv6, but they're managed separately because the address formats are completely different.

IPv4 whitelist entry: 198.51.100.42 or 198.51.100.0/24

IPv6 whitelist entry: 2001:db8::42 or 2001:db8::/32

IPv6 complicates whitelisting in two ways:

1. Address length and readability: IPv6 addresses are 128-bit and written in hexadecimal. A typo in an IPv6 whitelist entry is much easier to make and harder to spot than in a 32-bit IPv4 address.

1. Privacy extensions (RFC 4941): Many modern operating systems use IPv6 Privacy Extensions by default, which randomizes the interface identifier portion of the IPv6 address on each connection. This means your IPv6 address can change even on a network with a stable prefix. A whitelisted IPv6 prefix entry (e.g. /64) that covers the entire subnet is usually more practical than a /128 host entry.

Practical guidance:

• If your proxy provider supports IPv6, whitelist both your IPv4 and IPv6 addresses
• Use a subnet prefix (e.g. /64) for IPv6 entries if your OS uses privacy extensions
• If you're unsure about your IPv6 address stability, use username-password auth and avoid IPv6 whitelist dependency

---

Whitelisting is a security control, so it pays to treat it like one.

• Whitelist the narrowest range that works. A /32 for a single host beats a /24 you do not fully control. Every extra address in the range is extra attack surface.
• Prefer dedicated IPs over shared NAT. On a dedicated server or cloud VM with a reserved IP, the whitelisted address maps to your infrastructure and nobody else's. The difference between shared and dedicated resources matters here.
• Review your entries on a schedule. IP allocations change hands over time. An address you whitelisted last year may belong to a stranger now.
• Combine methods where it helps. Many providers let you run IP whitelisting and username-password together, so stable servers use the whitelist while mobile or cloud workers fall back to credentials.
• Never trust forwarded headers for identity. Whitelisting checks the real TCP source IP, not X-Forwarded-For, which is exactly why it resists header spoofing. Keep it that way.

The guiding idea follows OWASP's default-deny principle: permit exactly what you trust, and block everything else by default.

---

Diagnosing a whitelist IP mismatch:

# What IP does the proxy see for YOUR connection?
# (Run this from the machine that's failing)
curl https://ifconfig.me

# Compare to your dashboard whitelist entries
# They must match exactly (or the IP must fall within a whitelisted CIDR range)

If the IPs don't match, the fix is to either:

• Update the whitelist with the new IP, or
• Switch to username-password authentication if the IP is dynamic

---

IP whitelisting fits best when your scraping or automation infrastructure runs on machines with stable, static public IPs that you own and control, such as dedicated servers, cloud VMs with reserved IPs, or enterprise internet connections with static assignments.

It's the wrong choice for developer machines, dynamic cloud auto-scaling groups, cellular/mobile connections, or any environment where the public IP changes without deliberate action.

The clearest signal to use IP whitelisting: you can answer "yes" to all three questions:

1. Does this machine have a static public IP?
2. Do I control that IP assignment (not a shared NAT)?
3. Does my team size and deployment scale allow managing an IP list rather than rotating credentials?

If any answer is "no," username-password authentication is the more operationally reliable option, and the one less likely to produce a silent auth failure at an inconvenient time.

SparkProxy supports IP whitelisting with single-IP entries and CIDR ranges, plus username-password authentication as an alternative or fallback. If you are weighing the two methods, our guide on how proxy authentication works walks through both.

---