ScrapeBox Proxies: Load, Test, and Harvest Google

ScrapeBox's multi-threaded harvester mostly shows you counters. The detail lives in the single-threaded harvester, and its per-query lines are what the official ScrapeBox FAQ (scrapeboxfaq.com, linked from the main scrapebox.com navigation) uses to explain failures. Its "Scraping not working or returning no results" entry gives two shapes, quoted verbatim:

Results 0 completed using proxy xxx.xxx.xxx.xxx:xxx
Results 0 Error xxx received using proxy xxx.xxx.xxx.xxx:xxx

The first means the engine returned nothing. The second carries a status code, mapped by the FAQ:

Two of those six rows are not proxy problems at all. More IPs won't touch a Completed But Error wave.

The official FAQ answer to "What format do I put my username and password in for proxies?" is short:

IP:PORT
IP:PORT:USERNAME:PASS
127.0.0.1:80:username:password

One proxy per line, colons only, no scheme. Four rules follow:

• No http:// prefix and no user:pass@host form. Those are curl and Python conventions. ScrapeBox splits on colons, so http://user:pass@host:port becomes garbage fields.
• A colon inside the password breaks the line. The parser can't tell where the password ends. Regenerate a password without colons rather than trying to escape it.
• SOCKS needs an S prefix. The FAQ's SOCKS answer: "you need to append a S to the beginning of each proxy", so 195.195.195.195:80 becomes S195.195.195.195:80. Changelog 2.0.0.62 also says the proxy manager "can now determine the type of proxy (http or socks) during test".
• Use the HTTP port, not an "HTTPS proxy". The FAQ says ScrapeBox "lacks the elements to support and function with HTTPS proxies", meaning a proxy you talk TLS to. A normal HTTP proxy still carries HTTPS pages through CONNECT, which is what SparkProxy's port 11000 does.

For SparkProxy the fields map like this (copy exact values from your dashboard):

The format calls the first field "IP", and the safest thing is to give it one. ScrapeBox has handled hostnames since changelog 2.0.0.42 ("Fixed an issue with non-ip proxies"), but country filters, sorting and some add-ons key on the address. Resolve the host once and keep the hostname version as a fallback:

nslookup gateway.sparkproxy.io
# Name:    gateway.sparkproxy.io
# Address: 65.109.70.130   (what we saw on 2026-10-30; re-check, don't hardcode forever)

If the address changes later, every line stops at once with Error code 0, so re-run the lookup whenever an entire list dies together.

A rotating gateway is one endpoint with many exit IPs behind it. ScrapeBox sees one line and treats it as one proxy, so all its failure bookkeeping lands on that line.

The changelog shows the harvester has dead-proxy logic ("Fixed a bug related to dead proxy handling in harvester", 2.0.0.38; "Fixed a bug in dead proxy monitoring code", 2.0.0.39) and retries counted per proxy. When Google answers one exit with a 302, ScrapeBox can't know the next request on the same line would leave from a different IP. Our reading, which you should confirm in your own build, is that a single gateway line collects every failure from every exit and gets retired like any other bad proxy, after which the harvest stalls or runs out of proxies.

The fix is to load the same line many times. Each copy is a separate entry to ScrapeBox, so a failure on one copy retires only that copy while the gateway keeps handing out fresh exits on the rest. Sixty identical lines give the harvester sixty strikes before it runs dry.

This script writes the list and refuses the two format mistakes that break it:

# make_proxy_list.py - write a ScrapeBox proxy list for a rotating gateway
import socket
import sys

HOST = "gateway.sparkproxy.io"
PORT = 11000            # Random Rotation; 11002 for Sticky Sessions
USER = "user-country-us"  # copy the exact username from your dashboard
PASSWORD = "YOUR_PASS"
COPIES = 60             # keep at or below your harvester connection count
USE_IP = True           # resolve once so the first field is a real IP


def build_lines():
    if ":" in USER or ":" in PASSWORD:
        sys.exit("Username or password contains ':' and will split the line.")
    host = socket.gethostbyname(HOST) if USE_IP else HOST
    line = f"{host}:{PORT}:{USER}:{PASSWORD}"
    return [line] * COPIES


if __name__ == "__main__":
    lines = build_lines()
    with open("proxies.txt", "w", encoding="ascii", newline="\r\n") as fh:
        fh.write("\n".join(lines) + "\n")
    print(f"wrote {len(lines)} lines, first: {lines[0].rsplit(':', 1)[0]}:****")

Then in ScrapeBox: Manage proxies, Load from file (or "Load from ClipBoard", added in 2.0.0.28), pick proxies.txt.

Now the trap. ScrapeBox ships several de-duplication helpers: "Remove Duplicate Proxies" in the proxy grid menu (2.0.0.57), "Compare with existing list and remove duplicates" in the proxy manager filter (2.0.0.13), and an Automator command for the same job. Any of them collapses your sixty lines back to one, which is why a gateway list "keeps shrinking". Skip dedupe on gateway lists, or keep them in their own file and reload it.

How many copies? Match your harvester connection count. More copies than connections adds nothing; far fewer and a bad minute burns through them.

Don't start with the Google test. First prove format and credentials against a page that never blocks you. ScrapeBox's own proxy-harvester page describes a "Custom Test" option where you "add any URL you want the proxy tester to check against" and "specify something on the webpage to check for", such as a unique piece of text.

Set it up in the proxy manager configuration:

Custom Test URL:     https://www.sparkproxy.io/
Must contain text:   SparkProxy
Connections:         10
Timeout:             15 seconds

Ten connections is plenty for a test of sixty identical lines; you're checking the plumbing, not the pool. Read the result like this:

• All pass. Format, port and login are correct. Any Google failure later is about Google, not your setup.
• All fail fast. Wrong port, wrong host, or the first field isn't parsing. Re-check the line and that you used 11000, not 13000 without the S prefix.
• All fail with an auth error. Username or password is wrong, or you're on IP whitelist auth and the machine's public IP isn't whitelisted. Starter allows 5 whitelisted IPs, so a ScrapeBox VPS has to be one of them. The IP whitelisting explainer covers when to drop credentials entirely.

Latency, anonymity and geolocation checks are covered in our proxy testing guide.

ScrapeBox describes the Google test plainly on its proxy-harvester page: it tests "if proxies are working with Google by conducting a search query on Google and seeing if search results are returned." Then you can "keep only Google passed proxies" with a filter.

That fits static public proxies, one fixed IP per line. For a gateway it means something else:

1. The test samples one exit, once. Your line passed because the exit it got at that moment returned results. The next request on that line leaves from another IP. "Google passed" on a rotating line is a single draw, not a property of the line.
2. Testing sixty copies is a sample of sixty exits. The pass ratio across copies is a rough snapshot of how Google is treating the pool right now. It's useful as a trend if you run it daily, useless as a filter.
3. "Keep Google passed" deletes copies of a working endpoint. Failed copies are byte-identical to passing ones. Filtering them only shrinks your list, which brings back the run-out problem.
4. A result page has to parse. The test only passes if ScrapeBox recognises results in what came back. We fetched https://www.google.com/search?q=scrapebox+proxies&hl=en&gl=us from a server with plain curl on 2026-10-30 and got HTTP 200 with a page titled "Google Search" that says "Please click here if you are not redirected within a few seconds" and links to /httpservice/retry/enablejs. No results, no block, no error code. Any client that doesn't run JavaScript gets that shape, whatever IP it uses. We have not tested how the current ScrapeBox build handles that page, so treat this as a reason to check the engine, not as a verdict on the tool.

The scrapeboxfaq.com proxy guide says something consistent with point 4 in its own words: back-connect proxies "used to be extremely valuable for search engine scraping, but after the 2025 shakeup they aren't as effective for Google and Bing." That's the FAQ site's claim, not our measurement, but it matches what the curl test shows: the obstacle moved from the IP to the client.

So use the Google test for one thing: to see whether the pool is being 302'd en masse today. Don't use it to prune a gateway list.

When a harvest goes bad, walk this list top to bottom. Each step takes a minute and rules out a class of cause.

1. Run the query in a browser. If inurl:car inurl:house site:purple.com purple cows returns nothing for a human, the harvester will return nothing too. The FAQ uses almost exactly that example.
2. Run the Custom Test from the previous section. If it fails, stop: this is format, port or auth.
3. Switch to the single-threaded harvester. The FAQ's own advice: in Settings, untick "use multi threaded harvester", harvest a handful of keywords, and read each line with the proxy and result.
4. Count the outcomes. Paste the lines into a file and tally them. A quick script beats eyeballing 300 rows:

# tally_harvest_lines.py - count ScrapeBox single-threaded harvester outcomes
import collections
import re
import sys

ERR = re.compile(r"Results\s+\d+\s+Error\s+(\d+)\s+received using proxy\s+(\S+)")
DONE = re.compile(r"Results\s+(\d+)\s+completed using proxy\s+(\S+)")


def tally(path):
    codes = collections.Counter()
    empty = found = 0
    with open(path, encoding="utf-8", errors="replace") as fh:
        for line in fh:
            m = ERR.search(line)
            if m:
                codes[m.group(1)] += 1
                continue
            m = DONE.search(line)
            if m:
                if int(m.group(1)) == 0:
                    empty += 1
                else:
                    found += 1
    return codes, empty, found


if __name__ == "__main__":
    codes, empty, found = tally(sys.argv[1])
    print(f"with results: {found}  completed-empty: {empty}")
    for code, n in codes.most_common():
        print(f"error {code}: {n}")

1. Read the dominant outcome against the fixes below. One dominant code means one cause. A mix of 302 and successes means rate, not setup.

Mostly Error 302. Google is redirecting exits to its block page. This is the one a proxy pool can genuinely help with, and the fixes are about load per IP: lower harvester connections, make sure you're on the Random Rotation port so each new connection gets a new exit, and spread queries across more time. If one IP is fetching ten result pages for every keyword, cut the pages per keyword. Pacing math for larger runs lives in proxies for SERP scraping at scale; this post won't repeat it.

Mostly Error 407. Credentials aren't reaching the gateway. Check for a colon in the password, a trailing space from copy-paste, or a line with only three fields. If you use whitelist auth, the harvesting machine's current public IP must be on the list.

Mostly Error code 0 or Error 404. The connection never got a proper answer. Causes in order of likelihood: the first field doesn't resolve (stale IP after a gateway change), the wrong port, more connections than your plan's threads, or local security software. The FAQ's locked-threads entry blames anti-virus and firewalls for stalls and tells you to whitelist the whole ScrapeBox folder.

Mostly Completed But Error, or 200s with zero results across every query. No proxy fixes this. The page arrived and ScrapeBox couldn't find its results marker, either because Google changed its markup or because the response is the JavaScript interstitial shown above. Update ScrapeBox (the FAQ: "often times it is already fixed in a new version"), and if it persists, move SERP collection off the harvester (see the Scraping API section).

Results 0 completed on specific queries only. Your footprint has no results. Rewrite the query.

Harvest stops at around 1,000 URLs per keyword. That's the engines, not you. The FAQ: "1000 urls is the max number of urls you can scrape per keyword per engine." More keywords, not more proxies.

ScrapeBox will let you run absurd concurrency. Its search-engine-scraper page says you "could run 3000 connections at once", and changelog 2.0.0.32 added an "Adv." checkbox that raises the harvester limit to 3500. Those numbers describe the software, not what Google or your proxy plan will tolerate.

Two ceilings matter more:

• Your plan's threads. On SparkProxy, threads are concurrent connections: Starter is 100, Core 250, Boost 500, Plus 1000 (sparkproxy.io, read 2026-10-30). Harvester connections plus anything else using the plan at the same time (a proxy test in another window, a second ScrapeBox instance, the poster) must fit under it. Go over and the extra connections fail, which shows up as Error code 0 noise that looks like dead proxies.
• What Google accepts per IP. No published number exists, so find yours empirically: start low, read the 302 share from the tally script, step up.

A starting configuration for a Google harvest on a 100-thread plan:

The retry settings exist because of changelog 2.0.0.33: "Added proxy retries, error retries and redirect retries to harvester (under Connections and Timeouts)". The same release "Added proxy timeout for harvester and poster". If your build labels them differently, the mapping still holds.

Raise connections in steps of 10, run a few hundred keywords at each, and stop when the 302 share climbs. That's your ceiling for this pool, country and week.

What about the delay field? The official FAQ says the delay "only works with: Single Threaded Harvester, Ping Mode, Page Rank Checker, Email Grabber", and changelog 2.0.0.48 "Changed the way how the delay works in detailed harvester." So connection count is your throttle on the multi-threaded harvester; a single-threaded harvest with a delay is the slow fallback for small jobs.

One stall cause from the FAQ: proxies "that keep returning small amounts of data" never trigger the timeout, which locks threads and greys out Stop. Lower connections and check security software before blaming the pool.

Random Rotation (port 11000) suits the Google harvester. Each new connection takes a new exit, so the load spreads across the pool and a 302 on one exit doesn't follow the next query. This is the mode the repeated-lines setup assumes.

Sticky Sessions (port 11002) hold an exit for up to five minutes. That's the wrong shape for SERP harvesting, because one IP pulling page after page is exactly the pattern Google rate-limits. It suits ScrapeBox jobs that need a stable identity: a manual poster session, or a check where you want consecutive requests from the same address. The sticky session explainer covers the tradeoff in general terms.

Country goes in the username (user-country-us style; copy the format from your dashboard). Match it to the Google market you're harvesting. Results differ by IP location, which the FAQ's rank-tracker entry makes a point of: "IPs in the same state/country can show considerably different results." If you harvest google.de footprints, use German exits; mixing countries inside one list gives you a blended result set you can't reproduce.

A note on proxy type. SparkProxy's pool is datacenter IPs, and we won't quote a Google pass rate for it that we haven't measured for your queries, your country and this week. Treat the Google test ratio and the 302 share from your own runs as the numbers that count.

If the tally is mostly Completed But Error, the harvester's Google engine is the weak link, not the IPs. You can keep ScrapeBox for everything downstream (dedupe, link checking, the poster, the Automator) and collect Google URLs elsewhere, then load them through the URL list import.

The SparkProxy Scraping API renders the page in a browser, which is what the JavaScript interstitial demands. render_js defaults to true and costs 5 credits per request; country_code adds 5. json_response=true returns an envelope whose body is base64, so decode it before looking for links:

# serp_to_scrapebox.py - collect Google result URLs and write a ScrapeBox URL list
import base64
import time
from html.parser import HTMLParser
from urllib.parse import quote_plus, urlparse

import requests

API = "https://scrape.sparkproxy.io/api/v1"
KEY = "YOUR_API_KEY"


class ResultLinks(HTMLParser):
    """Keep hrefs of anchors that wrap an <h3>, the organic result pattern."""

    def __init__(self):
        super().__init__()
        self.stack, self.links = [], []

    def handle_starttag(self, tag, attrs):
        if tag == "a":
            self.stack.append(dict(attrs).get("href", ""))
        elif tag == "h3" and self.stack and self.stack[-1].startswith("http"):
            self.links.append(self.stack[-1])

    def handle_endtag(self, tag):
        if tag == "a" and self.stack:
            self.stack.pop()


def serp(query, country="us"):
    target = f"https://www.google.com/search?q={quote_plus(query)}&hl=en&gl={country}"
    r = requests.get(API, headers={"X-API-Key": KEY}, timeout=120, params={
        "url": target, "render_js": "true", "country_code": country,
        "json_response": "true"})
    r.raise_for_status()
    env = r.json()
    html = base64.b64decode(env["body"]).decode("utf-8", "replace")
    parser = ResultLinks()
    parser.feed(html)
    return env.get("credits_used"), [u for u in parser.links if "google." not in urlparse(u).netloc]


if __name__ == "__main__":
    seen = set()
    for kw in open("keywords.txt", encoding="utf-8").read().split("\n"):
        if not kw.strip():
            continue
        credits, urls = serp(kw.strip())
        seen.update(urls)
        print(f"{kw!r}: {len(urls)} urls, {credits} credits")
        time.sleep(1)
    with open("harvested_urls.txt", "w", encoding="utf-8", newline="\r\n") as fh:
        fh.write("\n".join(sorted(seen)) + "\n")

Load harvested_urls.txt into ScrapeBox's URL list. Each keyword costs 10 credits with a country set, so the 1,000 free credits cover 100 SERP pages. Google's markup changes, so verify the wrapping