Regional vs Global Proxy Pools: Effective Depth

Pool counts are advertised as a single global integer because that integer is the largest true number a provider can print. It describes the union of every address the network has seen recently, across every country, every hour of the day, and every target. Your job never gets to use that union. It gets one slice of it, at one moment, against one site.

The mechanics of how a pool assigns and recycles addresses are covered in what is a proxy pool, and the country-selection layer itself is covered in what geo-targeting means in proxies. This article is about neither. It is about the size of the slice that survives after those two layers run.

Three things shrink the headline number, and they multiply rather than add.

Registration is power-law distributed

Residential and mobile addresses accumulate where devices and bandwidth-sharing installs accumulate. In practice a handful of markets take most of the pool. The United States, Brazil, India, Indonesia, the United Kingdom, and Germany routinely account for the bulk of a global residential network, and the long tail of 150-plus remaining countries divides what is left. A country holding 0.1% of a 50 million IP pool has 50,000 registered addresses. A country holding 0.002% has 1,000.

Registered is not the same as reachable

A residential IP belongs to a real device that is powered on some of the time. At any given instant only a fraction of a country's registered addresses are online and accepting a tunnel. Datacenter pools do not have this problem, since the machines never go home, which is one of the underrated reasons a datacenter pool's advertised number sits much closer to its usable number.

Clean is not the same as reachable

An address that is reachable and in the right country can still be worthless if your target has already seen it misbehave. Reputation is per target, so the same IP can be pristine for one site and burned for another.

Write the number you actually get as a product. Call it E, the effective depth for a specific country, a specific hour, and a specific target.

E = N × f_geo × a × c

N     = headline pool size (total IPs advertised)
f_geo = fraction of the pool registered in your target country
a     = fraction of that country slice online and assignable in your window
c     = fraction of those still clean against your specific target

Every term after N is a fraction below 1, so E collapses fast. Worked example against a global residential network, targeting Portugal:

Effective depth: 4,500 addresses out of a headline 50 million. That is 0.009% of the number on the pricing page, and it is the only number your rotation logic will ever touch.

Availability is diurnal, and most jobs are scheduled against the trough

The a term is not a constant. A residential country slice is deepest when that country's devices are awake and idle, roughly 19:00 to 23:00 local, and thinnest in the small hours when phones are off and home routers have dropped idle sessions. The swing between peak and trough inside a single market commonly runs a factor of two to four.

That matters because scheduled jobs default to overnight. A nightly crawl set for 03:00 in the target country is deliberately running against the worst effective depth of the day, and the resulting block rate gets blamed on the provider. If your job is geo-constrained and your rotation is thin, move the window into the target country's evening instead of your own. That one change often recovers more depth than upgrading the plan does.

Regional pools invert the first two terms

A regional provider concentrating on, say, Southeast Asia or Brazil has a small N, perhaps 800,000 addresses. But f_geo for Indonesia might be 35% rather than 0.4%, and a is often higher because the provider sources locally instead of through a global aggregation layer. Run the same arithmetic: 800,000 × 0.35 × 0.20 × 0.60 = 33,600. A pool 62 times smaller on paper delivers more than seven times the usable depth in that country.

That inversion is the whole regional versus global argument in one line.

Providers do not trade depth against breadth for fun. Acquiring addresses costs money, and a fixed acquisition budget spread across 195 markets cannot also go deep in any of them.

The subnet row is the one people forget. Two hundred addresses spread across four /24 blocks is not two hundred identities to a site that rate-limits by network prefix, and thin country slices are usually thin at the block level too. If your target blocks at the prefix rather than the address, read what subnet proxies are before counting IPs at all, because the right denominator may be distinct /24s.

A coverage claim is a set-membership claim. It usually means the network has observed at least one address in that country within some recent window. It promises nothing about depth, concurrency, or city spread. Run the effective-depth arithmetic on a genuine tail market and the gap between claim and reality is stark.

Take a country holding 0.002% of a 50 million IP global pool:

Twelve addresses. The marketing page and your invoice both still say 50 million, and neither is lying. They are answering a different question than the one your job asks.

The city filter cuts hardest and trusts the weakest data

Filtering from country to city is where depth falls off a cliff, and it is also where the underlying data is least reliable. MaxMind publishes a per-country accuracy comparison for GeoIP City showing what share of addresses it places within 50 km of their true location, and the spread across markets is enormous. Some territories sit at 100%, while Finland is around 35% and Hong Kong around 28%. Your provider labels IPs from one database. Your target resolves them with a different one. Request a specific city in a market with weak city data and you are filtering on a label that frequently disagrees with what the target concludes.

Country labels are far more stable than city labels, which is why country targeting works reliably and city targeting is where geo jobs quietly fail. Where network operators publish authoritative data they do it through geofeeds: RFC 8805 defines the CSV format mapping prefixes to country, region, and city, and RFC 9092 defines how consumers discover those feeds from RIR records and validate them against RPKI. Publication is voluntary and partial, so commercial databases keep guessing for everything else.

Once you know E you can predict when a job starts reusing addresses. Most people get this wrong in the same two ways.

Mistake one: assuming repeats start late

With random assignment from E addresses, the expected number of requests before you draw the same IP twice is not E, and it is not E/2. It is roughly the birthday-problem quantity:

first repeat ≈ sqrt(pi × E / 2)

For E = 4,500 that is about 84 requests. Not 4,500. Eighty-four. A slice that "should last all day" produces its first collision inside the first two minutes of a modest crawl. Repeats are not a symptom of a broken pool, they are the ordinary statistics of drawing from a finite set, which is why rotation policy is built around per-IP pacing rather than around never repeating.

Mistake two: sizing against the average

The average tells you nothing about who gets blocked, because sites rate-limit individual addresses. What matters is the busiest one. For R requests spread randomly over E addresses, with R at least as large as E, the busiest address sees approximately:

mean hits per IP    = R / E
busiest IP (approx) = R/E + sqrt(2 × (R/E) × ln E)

At R = 10,000 requests in one hour:

Read the last column against your target's tolerance. If the site starts challenging an address at 6 requests per hour, E = 10,000 is marginal and E = 1,000 is hopeless, even though the average at E = 1,000 looks like a harmless 10.

The naive sizing rule, E = R / T where T is the tolerated requests per IP per hour, gives 1,667 for R = 10,000 and T = 6. Solving the busiest-IP inequality instead gives roughly 8,000. The average understates the depth you need by about five times at this scale, and the gap widens as E shrinks. Solve it numerically rather than by hand:

import math

def required_depth(requests_per_hour: int, tolerated_per_ip: float) -> int:
    """Smallest effective depth E where the busiest IP stays under the target's limit."""
    lo, hi = 2, requests_per_hour * 100
    while lo < hi:
        mid = (lo + hi) // 2
        load = requests_per_hour / mid
        busiest = load + math.sqrt(2 * load * math.log(mid))
        if busiest <= tolerated_per_ip:
            hi = mid
        else:
            lo = mid + 1
    return lo

print(required_depth(10_000, 6))    # ~8,000 usable IPs, not 1,667
print(required_depth(2_000, 6))     # a slower crawl needs far less depth

Now compare that requirement against the E you computed for your target country. If required depth exceeds available depth, no amount of retry logic rescues the job. You lower R, raise T by slowing down and behaving better, or change pool architecture.

Geo-targeting a global pool solves the identity problem and creates a routing one. The request path is a double hop: scraper to provider gateway, gateway to exit IP, exit IP to target origin. When the gateway sits in Virginia, the exit sits in Jakarta, and the target origin also sits in Jakarta, you pay the Pacific crossing twice, once outbound and once coming back.

Light in fiber travels at roughly two-thirds of c, about 200,000 km per second, so every 1,000 km of path adds around 10 ms of round-trip time before any equipment touches the packet. Real routes run 1.5 to 2 times the great-circle distance. Frankfurt to Singapore is roughly 10,000 km straight, which floors the round trip near 100 ms and lands near 170 ms in practice.

Multiply that by the round trips one page needs. A TCP handshake costs one. TLS 1.3 adds one more for a full handshake, so two round trips pass before the first request byte leaves, and TLS 1.2 spends three. A rendered page then serialises perhaps four more through its resource dependency chain. Six is a reasonable floor.

The throughput cost is real, but the billing cost is usually bigger, because concurrency is what most plans meter. Needing three to six times the workers to hold the same request rate multiplies your concurrent-connection footprint by the same factor, which is worth reading against how concurrent connections work in proxies before assuming the cheaper per-GB price wins.

A regional pool usually terminates its gateway inside the region, so you cross the ocean once instead of twice. On a single request that saves about 80 ms. On a 500,000-request monthly job it saves roughly 11 hours of pure wall-clock waiting.

No provider can tell you E, because E depends on your country, your hour, and your target. You can measure it yourself on a trial account in about an hour, using a technique borrowed from wildlife population estimation. You are, after all, trying to count a population you can only sample.

Step 1: capture two independent batches

Send a batch of requests to an IP echo endpoint with your country filter applied, record every exit address you are handed, wait long enough for the pool's assignment state to turn over, then send a second batch and record those.

import time, requests
from collections import Counter

API = "https://scrape.sparkproxy.io/api/v1"
KEY = "YOUR_API_KEY"

def sample(country: str, n: int) -> Counter:
    seen = Counter()
    for _ in range(n):
        r = requests.get(
            API,
            headers={"X-API-Key": KEY},
            params={
                "url": "https://api.ipify.org",
                "country_code": country,   # ISO 3166-1 alpha-2
                "proxy_type": "premium",   # the pool you intend to buy
                "render_js": "false",      # no browser needed to read an IP back
            },
            timeout=60,
        )
        r.raise_for_status()
        seen[r.text.strip()] += 1
    return seen

batch_a = sample("PT", 500)
time.sleep(1800)                 # 30 minutes, let assignment state turn over
batch_b = sample("PT", 500)

Step 2: estimate depth from the overlap

The overlap between the two batches is the signal. If the country slice is large, two 500-request samples barely intersect. If it is tiny, they look almost identical. The Chapman-corrected Lincoln-Petersen estimator turns that intuition into a number:

def chapman_estimate(a: Counter, b: Counter) -> float:
    """Chapman-corrected Lincoln-Petersen: population from two-sample overlap."""
    n_a, n_b = len(a), len(b)
    m = len(set(a) & set(b))                        # addresses seen in both batches
    return ((n_a + 1) * (n_b + 1)) / (m + 1) - 1

def chao1(pooled: Counter) -> float:
    """Lower-bound richness estimate from singletons and doubletons."""
    d  = len(pooled)
    f1 = sum(1 for v in pooled.values() if v == 1)
    f2 = sum(1 for v in pooled.values() if v == 2)
    return d + (f1 * f1) / (2 * f2) if f2 else d + f1 * (f1 - 1) / 2

pooled = batch_a + batch_b
print("distinct seen:", len(pooled))
print("chapman E:", round(chapman_estimate(batch_a, batch_b)))
print("chao1   E:", round(chao1(pooled)))
print("distinct /24s:", len({ip.rsplit(".", 1)[0] for ip in pooled}))

Chapman is the workhorse. Chao1 is a useful cross-check, since it infers the unseen population from how many addresses appeared exactly once versus exactly twice, and the two estimates agreeing tells you the sample was big enough. If they disagree by more than roughly a factor of two, sample more.

Step 3: interpret it against your job

Run the same script against every provider on your shortlist, in the country you actually need, at the hour you actually run. Then compare each measured E against the output of required_depth(). That comparison is the buying decision, and it costs about two hours and a few thousand trial credits to make.

The question resolves cleanly once you classify the job by how constrained its geography is.

Broad crawls: global wins

A crawl touching thousands of domains with no country requirement gets the full union. f_geo is effectively 1, so E is close to N and breadth is pure upside. Global pools also carry wider subnet diversity in aggregate, which matters when you spread load across many unrelated targets rather than hammering one. Search-result collection, link-graph building, and general content crawling all sit here.

Country-locked monitoring: regional wins

Price monitoring for one national market is the archetype. You hit a small set of hosts, repeatedly, all day, from one country, which is exactly the profile that exhausts a thin slice and trips per-IP thresholds. Depth in that single country is the entire requirement, and 33,000 Indonesian addresses beat 50 million global ones that happen to include 4,000 Indonesian. The same logic drives datacenter proxies for price comparison sites, where predictable in-country depth beats breadth you cannot use.

Multi-market localization checks: global wins on breadth, not depth

Verifying that a site renders the right currency, language, and legal notices across 40 countries needs one or two successful requests per market, not thousands. Low R means low required depth, so tail-country thinness is survivable and breadth is the actual requirement. That workflow is covered in proxies for localization testing.

Mixed portfolios: run both, route by country

For teams running several jobs the practical answer is not to pick one. Route tail markets and broad crawls through a global pool, route the two or three countries carrying most of your volume through regional pools or a provider with proven depth there, and keep the measured E per country in a config file so the router chooses by number rather than by habit. Re-measure quarterly, because supply moves.

SparkProxy's Scraping API exposes the two parameters this analysis depends on, country_code for the geo filter and proxy_type for which pool you draw from, so you can measure depth per country per pool with the same call you will use in production:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://api.ipify.org" \
  --data-urlencode "country_code=PT" \
  --data-urlencode "proxy_type=premium" \
  --data-urlencode "render_js=false"

Set render_js=false for sampling runs. Rendering is not needed to read back an exit address, and turning it off drops the request from the 25-credit rendered premium tier to the much cheaper non-rendered one, so a 1,000-request depth measurement stays affordable. country_code takes an ISO 3166-1 alpha-2 code and adds 5 credits, and proxy_type accepts premium or ad_free. Add json_response=true if you want the reply wrapped with metadata so your script can log status codes alongside each address. The full parameter list and credit table live in the Scraping API docs.