How to Scrape Yellow Pages Data (Business Lead Lists)

To scrape Yellow Pages data into a usable local lead list, you don't need a headless browser or a residential proxy pool for most of the job. YellowPages.com serves its listings as plain server-rendered HTML with human-readable class names, so a single HTTP fetch plus a set of CSS rules pulls the business name, phone, address, and category straight off each result card. This guide walks the whole pipeline: how the search-by-category-and-location URLs are built, how to page through every result, how to extract and normalize the card fields, how to dedupe the list, and where US and EU law draws the line once you start contacting those businesses. Every request runs through SparkProxy's Scraping API, so IP rotation and geo-targeting are request parameters instead of infrastructure you babysit.

Yellow Pages listings are public. Anyone can open a search page without logging in, so the access question is settled. The permission question is separate, and the two don't move together, so get the framing straight before you write code.

In the United States, the Ninth Circuit's decision in hiQ Labs v. LinkedIn (2022) held that scraping data that's publicly accessible, meaning no login and no authentication wall, generally does not violate the Computer Fraud and Abuse Act. That ruling is about unauthorized access, not a license to copy and redistribute whatever you like. YellowPages.com's Terms of Use separately prohibit automated collection and bulk harvesting, and its robots.txt restricts crawler paths. So a scrape can clear the CFAA bar and still breach the site's terms and stated crawl policy. Different questions, different answers.

There's no free, Fusion-style public API for YellowPages.com the way Yelp offers one, which is part of why people scrape it directly. The listings themselves are aggregated from public business records, and the individual facts on a card (a plumber's shop name, its phone, its street address) are not copyrightable as facts. A large verbatim copy of the compiled database is a different matter, so treat the data as facts to extract, not a database to clone.

Guardrails that keep a Yellow Pages project defensible:

• Collect public business data only: name, category, phone, address, website, hours. That is contact and firmographic data for an organization, not private personal data.
• Rate-limit hard and back off on errors so you never degrade the service for the people using it. A directory is a shared resource.
• Keep provenance. Store where and when each record came from, so you can honor a later removal request and prove the source.
• The legal exposure on a Yellow Pages project usually isn't the scraping. It's what you do next, when you email or call the businesses you collected. That part has its own rules, covered in the compliance section below. This is engineering guidance, not legal advice; run a commercial program past a lawyer.

A Yellow Pages search result card is dense. Each organic listing carries the fields a sales team actually wants, and because the markup uses descriptive class names rather than generated hashes, you can address each field with a plain CSS selector. Here's the reference set worth pulling, with the stable selector for each as of mid-2026. Verify these against a live page before a long run, since any site can reshuffle its markup.

The reason this target is friendly: the name, phone, and both address lines each have their own class, so you extract them without parsing a wall of nested

Two things drive a Yellow Pages search, and both are query parameters you control: the category (or keyword) and the location. The canonical search URL is:

https://www.yellowpages.com/search?search_terms=&geo_location_terms=&page=

• search_terms is the category or keyword, for example plumbers, dentists, hvac contractors.
• geo_location_terms is the location, usually City, ST or a ZIP code. URL-encode the comma and space.
• page is the 1-indexed pagination cursor. Page one returns roughly 30 organic listings; page two is &page=2, and so on.

There's a second, cleaner form when you're targeting one category in one city. Yellow Pages exposes browse paths like:

https://www.yellowpages.com/los-angeles-ca/plumbers

The browse path returns the same card markup as the search form, so your parser doesn't care which URL produced the page. Use the search form when you're driving categories and cities from a list; use the browse path when you're hand-picking a single niche in a single metro. Build the search URL from parts you control:

from urllib.parse import urlencode

def yp_search_url(category, location, page=1):
    q = urlencode({
        "search_terms": category,
        "geo_location_terms": location,
        "page": page,
    })
    return f"https://www.yellowpages.com/search?{q}"

# yp_search_url("plumbers", "San Francisco, CA", 2)
# -> https://www.yellowpages.com/search?search_terms=plumbers&geo_location_terms=San+Francisco%2C+CA&page=2

Keep a spreadsheet of the (category, city) pairs you want. That grid, run one page at a time, is your whole crawl plan.

The SparkProxy Scraping API takes a target URL, routes it through a proxy, and hands back the HTML (or a structured extract, if you pass rules). For Yellow Pages, the important choice is how little you need. Because the listings are server-rendered, you can skip the headless browser and fetch plain HTML, which costs less and runs faster.

• render_js=false: Yellow Pages puts the result cards in the initial HTML, so you don't need a browser to paint them. A plain fetch costs 1 credit against 5 for a rendered page. Flip this to true only if a live check shows the cards are missing from the raw HTML.
• country_code=US: route through a US exit so the site serves US listings and formatting. Yellow Pages is a US directory, so keep this pinned.
• Rotating proxies are the default. You do not need premium_proxy=true for most Yellow Pages work; escalate to it only if plain rotating IPs start drawing blocks under load.

The base endpoint is https://scrape.sparkproxy.io/api/v1 and auth is a single header, X-API-Key. A minimal request in cURL:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.yellowpages.com/search?search_terms=plumbers&geo_location_terms=San+Francisco%2C+CA" \
  --data-urlencode "render_js=false" \
  --data-urlencode "country_code=US"

The full parameter list and response fields live in the Scraping API docs. If you're weighing this against building and rotating your own pool, Web Scraping API vs Self-Managed Proxies lays out that build-versus-buy trade-off honestly. The rest of the code here reuses one small helper:

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = "YOUR_API_KEY"

def fetch(url, extract_rules=None):
    params = {
        "url": url,
        "render_js": "false",
        "country_code": "US",
    }
    if extract_rules is not None:
        import json
        params["extract_rules"] = json.dumps(extract_rules)
        params["json_response"] = "true"
    resp = requests.get(
        API, headers={"X-API-Key": API_KEY}, params=params, timeout=120
    )
    resp.raise_for_status()
    return resp

Fetch a search page and you get one HTML document with about 30 organic listing cards in it, plus a few ads at the top that you'll want to skip. The organic cards live under .search-results.organic and each card is a .result element. Two ways to get the fields out follow: a quick server-side grab with extract_rules, and a per-card parse that keeps every field aligned to its business. Start with the fetch:

resp = fetch(yp_search_url("plumbers", "San Francisco, CA"))
html = resp.text   # raw HTML, because we passed no extract_rules

If a live check ever shows an empty shell instead of cards, the page started requiring a browser; set render_js to true in the helper and re-run. That single switch is the whole difference between the cheap path and the rendered one.

The API can do the CSS work for you. Pass an extract_rules object and it returns the matched text as JSON, so you never touch an HTML parser:

rules = {
    "names":      {"selector": "a.business-name", "type": "list"},
    "phones":     {"selector": ".phones.phone.primary", "type": "list"},
    "streets":    {"selector": ".adr .street-address", "type": "list"},
    "localities": {"selector": ".adr .locality", "type": "list"},
    "links":      {"selector": "a.business-name", "type": "href"},
}
data = fetch(yp_search_url("plumbers", "San Francisco, CA"), rules).json()
extracted = data["extracted"]   # {"names": [...], "phones": [...], ...}

This is fast and needs no dependencies, but it has one sharp edge: it returns parallel lists, and the lists can drift out of sync. If one card has no phone, the phones array is shorter than names, and every phone after the gap now belongs to the wrong business. Parallel extraction is fine when you only need one field (say, a list of detail-page URLs). When you need a coherent record per business, parse each card as a unit.

For that, fetch the raw HTML and walk the cards with selectolax, a fast C-backed parser (pip install selectolax). Reading one .result at a time keeps a missing field local to its own card:

from selectolax.parser import HTMLParser

def text_of(node, selector):
    el = node.css_first(selector)
    return el.text(strip=True) if el else None

def parse_search_cards(html):
    tree = HTMLParser(html)
    rows = []
    for card in tree.css(".search-results.organic .result"):
        name_el = card.css_first("a.business-name")
        if not name_el:
            continue  # ad slot or non-listing block
        href = name_el.attributes.get("href", "") or ""
        website = card.css_first("a.track-visit-website")
        rows.append({
            "name": name_el.text(strip=True),
            "detail_path": href,
            "listing_id": href.rsplit("-", 1)[-1] if "-" in href else None,
            "phone": text_of(card, ".phones.phone.primary"),
            "street": text_of(card, ".adr .street-address"),
            "locality": text_of(card, ".adr .locality"),
            "categories": [a.text(strip=True) for a in card.css(".categories a")],
            "website": website.attributes.get("href") if website else None,
        })
    return rows

cards = parse_search_cards(html)

Now clean the two fields that arrive as messy strings. The phone comes formatted for humans, and the locality packs city, state, and ZIP into one line. Normalize both so your lead list is queryable:

import re

def normalize_phone(raw):
    if not raw:
        return None
    digits = re.sub(r"\D", "", raw)
    if len(digits) == 11 and digits.startswith("1"):
        digits = digits[1:]
    return f"+1{digits}" if len(digits) == 10 else None

def split_locality(raw):
    # "San Francisco, CA 94103" -> ("San Francisco", "CA", "94103")
    if not raw:
        return (None, None, None)
    m = re.match(r"^(.*),\s*([A-Z]{2})\s*(\d{5})?", raw.strip())
    if not m:
        return (raw, None, None)
    return (m.group(1), m.group(2), m.group(3))

for c in cards:
    c["phone"] = normalize_phone(c["phone"])
    c["city"], c["state"], c["zip"] = split_locality(c.pop("locality"))

One search page is about 30 listings. A busy category in a large city runs many pages, so you loop page until the results run out. The reliable stop condition is simple: if a page returns zero .result cards, you've gone past the last page. Don't hard-code a page count, because it varies by category and city:

import time

def scrape_search(category, location, max_pages=50, pause=1.5):
    all_rows, page = [], 1
    while page <= max_pages:
        resp = fetch(yp_search_url(category, location, page))
        rows = parse_search_cards(resp.text)
        if not rows:
            break  # ran past the last page, or got blocked
        all_rows.extend(rows)
        page += 1
        time.sleep(pause)  # be polite; back off harder on errors
    return all_rows

plumbers = scrape_search("plumbers", "San Francisco, CA")

The max_pages cap is a safety valve, not a target. It stops a runaway loop if the site keeps serving a non-empty page forever. Set it comfortably above the real page count for your densest category. If you're collecting a lot of cities, keep the pause honest and read How to Avoid Getting Your Proxy Blocked for the pacing and backoff patterns that keep a long run clean.

The search card gives you enough for a first-pass lead list. The detail page (Yellow Pages calls it the MIP, or "more info page") adds the fields that make a lead worth calling: a general phone plus alternate numbers, the full hours, the website, and sometimes an email or contact link. Follow each detail_path and parse the richer page:

def scrape_detail(detail_path):
    url = "https://www.yellowpages.com" + detail_path
    tree = HTMLParser(fetch(url).text)
    hours = {}
    for row in tree.css(".hours-table tr, table.hours tr"):
        cells = [c.text(strip=True) for c in row.css("th, td")]
        if len(cells) >= 2 and cells[0]:
            hours[cells[0]] = cells[1]
    site = tree.css_first("a.website-link, a.custom-link[href^='http']")
    return {
        "hours": hours,
        "website": site.attributes.get("href") if site else None,
        "extra_phones": [p.text(strip=True) for p in tree.css(".phone")],
    }

Enrichment costs one extra request per business, so gate it. Pull the whole search set cheaply first, filter to the leads you actually want (a category, a rating floor, a ZIP list), then enrich only those. Scraping the detail page for 20,000 listings you'll never call is wasted budget.

Duplicates are guaranteed on a directory. The same shop appears under two categories, in two neighboring city searches, and sometimes twice on one page as a paid slot plus an organic one. Dedupe on the stable key first, then fall back to a fuzzy key for records that lack an ID:

def dedupe(rows):
    seen, out = set(), []
    for r in rows:
        key = r.get("listing_id")
        if not key:  # fallback: normalized name + phone
            name = re.sub(r"\W+", "", (r.get("name") or "").lower())
            key = f"{name}|{r.get('phone')}"
        if key in seen:
            continue
        seen.add(key)
        out.append(r)
    return out

leads = dedupe(plumbers)

The listing_id is the trustworthy key because it survives a name that's punctuated two ways ("Joe's Plumbing" vs "Joes Plumbing"). The name-plus-phone fallback catches records scraped from a surface that didn't expose an ID. After dedupe, drop rows with no phone and no website (a lead you can't contact isn't a lead), and write the set to CSV:

import csv

FIELDS = ["listing_id", "name", "phone", "street", "city",
          "state", "zip", "website", "categories"]

def write_csv(rows, path="leads.csv"):
    with open(path, "w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=FIELDS, extrasaction="ignore")
        w.writeheader()
        for r in rows:
            r = {**r, "categories": "; ".join(r.get("categories") or [])}
            w.writerow(r)

write_csv([r for r in leads if r.get("phone") or r.get("website")])

That CSV is a real local lead list: one row per business, deduped, with normalized phones and split address fields ready for a CRM import. If a lead list is the goal of a larger prospecting program, Proxies for Lead Generation covers the intent signals and firmographic sources that layer on top of a directory pull.

Here's the part most scraping tutorials skip, and it's where the actual liability lives. Collecting public business facts is low risk. Cold-emailing or cold-calling 10,000 of them is regulated activity, and the rules differ by channel and by country.

US email, CAN-SPAM. CAN-SPAM covers commercial email, and it applies to business-to-business messages, not just consumer ones. It does not require prior consent to send a cold email, but it does require every message to use accurate headers and a non-deceptive subject line, identify itself as an advertisement, include a valid physical postal address, and offer a working opt-out that you honor within 10 business days. Civil penalties run up to $53,088 per email under the FTC's 2024 figure, and each non-compliant message is a separate violation, so a bulk send multiplies fast.

US phone and text, TCPA. You scraped phone numbers, so the Telephone Consumer Protection Act is in play the moment you dial or text at scale. Autodialed or prerecorded calls and marketing texts carry consent requirements and statutory damages of $500 to $1,500 per violation, and you have to scrub against the National Do Not Call Registry. Business lines have narrower exemptions than residential ones, but "it's a business number" is not a blanket pass, especially for a sole proprietor whose cell is the business line.

EU and UK, GDPR and PECR. A business phone or email that identifies a named person, or belongs to a sole trader, is personal data under GDPR even though it's professional. If you contact businesses in the EU or UK, you need a lawful basis (legitimate interest is the usual route for B2B, and it requires a documented balancing test), you must identify yourself and offer an opt-out, and you must honor objection and erasure requests. PECR (the ePrivacy rules) governs the marketing message itself, and corporate subscribers get more leeway than individuals, though not a free pass.

Practical posture that keeps a program defensible:

• Suppress before you send. Maintain a do-not-contact list, scrub against the DNC registry, and never re-add someone who opted out.
• Keep every message compliant by construction: real sender identity, a physical address, and a one-click unsubscribe in every email.
• Store provenance and honor deletion. If a business asks to be removed, you should be able to find and purge its record and prove where it came from.
• Don't hoard personal data you won't use. A business phone and category is a lead; a named owner's personal details you scraped and stored "just in case" is risk with no upside.

None of this is legal advice. It's the shape of the rules so you know which questions to bring to counsel before a campaign goes out.

Yellow Pages is gentler than a target like Yelp or Google Maps, but a directory still throttles a client that hammers it. A few habits keep a large run clean and your data complete:

• Rate-limit and back off. A short sleep between requests plus exponential backoff on soft failures does more for your success rate than any single trick. Detect a block page and retry rather than saving it as a row.

BLOCK_MARKERS = ("captcha", "unusual traffic",
                 "access denied", "are you a human")

def looks_blocked(html):
    low = html.lower()
    return len(html) < 2000 or any(m in low for m in BLOCK_MARKERS)

• Keep concurrency modest. With a scraping API the provider rotates the exit IP per request, so your real ceiling is your plan's rate limit, not a pool you own. Steady and moderate beats spiky and blocked.
• Escalate only when you must. Start on plain rotating IPs with render_js=false. If a category or city starts returning block pages, flip premium_proxy=true for residential exits, and turn on render_js=true only if the raw HTML stops carrying the cards. Paying for rendering and residential on every request when the cheap path works is money set on fire.
• Persist as you go. Write each page's rows to storage as they land, so a mid-run block costs you one page, not the whole metro.

For a broader comparison of directory targets, How to Scrape Google Maps Data covers a harder local-business source where rendering and residential IPs are mandatory, not optional. Yellow Pages sits at the easy end of that spectrum, which is exactly why it's a good first source for a local lead list.