How to Scrape Crunchbase Company Data the Right Way

Crunchbase is a company graph. Each organization record links to people, funding rounds, investors, acquisitions, and categories. The fields that matter for lead generation and sales intelligence are the firmographics and the funding signals, and they map cleanly to the entity fields the API returns.

Two details on that table are worth calling out because they trip people up. num_employees_enum is not a raw number. It's a range enum such as c_00051_00100 or c_00251_00500, so you filter on buckets, not exact headcount. And funding_total is a money object with a value, a currency, and a value_usd, not a bare integer, so parse the sub-field, not the wrapper.

Investors and full funding-round history are the fields most likely to be gated behind a Crunchbase Pro subscription or the API. Treat those as licensed data, not something to lift from a page you had to log in to see.

Crunchbase publishes a REST API (v4), and it's the route the company sanctions for programmatic access. If you need funding history, investor networks, or bulk firmographics, this is the clean way to get them. It returns structured JSON, so you skip HTML parsing entirely.

The base URL and auth are fixed:

Base URL:  https://api.crunchbase.com/v4/data/
Auth:      user_key query parameter, or the X-cb-user-key request header

An Entity Lookup call fetches one organization by its permalink. Ask for the fields you want with field_ids and for related data (funding rounds, investors, people) with card_ids:

curl -s "https://api.crunchbase.com/v4/data/entities/organizations/stripe\
?field_ids=identifier,short_description,website_url,categories,location_identifiers,num_employees_enum,funding_total,founded_on\
&card_ids=founders,raised_funding_rounds" \
  -H "X-cb-user-key: INSERT_YOUR_KEY"

The same in Python, with the key on the header so it never lands in a log or a browser history:

import requests

CB_BASE = "https://api.crunchbase.com/v4/data"
CB_KEY  = "INSERT_YOUR_KEY"  # from your Crunchbase account

def cb_org(permalink, fields, cards=None):
    params = {"field_ids": ",".join(fields)}
    if cards:
        params["card_ids"] = ",".join(cards)
    r = requests.get(
        f"{CB_BASE}/entities/organizations/{permalink}",
        headers={"X-cb-user-key": CB_KEY},
        params=params,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

org = cb_org(
    "stripe",
    fields=["identifier", "short_description", "website_url",
            "categories", "location_identifiers",
            "num_employees_enum", "funding_total", "founded_on"],
    cards=["founders", "raised_funding_rounds"],
)

For building a list rather than looking up one company, use the Search endpoint. It's a POST with a predicate query, and it's how you pull every organization that matches an ICP filter:

def cb_search(fields, predicates, limit=100, after_id=None):
    body = {
        "field_ids": fields,
        "query": predicates,
        "order": [{"field_id": "rank_org", "sort": "asc"}],
        "limit": limit,
    }
    if after_id:
        body["after_id"] = after_id  # cursor from the last row of the previous page
    r = requests.post(
        f"{CB_BASE}/searches/organizations",
        headers={"X-cb-user-key": CB_KEY, "Content-Type": "application/json"},
        json=body,
        timeout=30,
    )
    r.raise_for_status()
    return r.json()

# US-ish SaaS companies, 51-250 staff, founded 2020 or later
page = cb_search(
    fields=["identifier", "website_url", "num_employees_enum",
            "categories", "funding_total"],
    predicates=[
        {"type": "predicate", "field_id": "num_employees_enum",
         "operator_id": "includes",
         "values": ["c_00051_00100", "c_00101_00250"]},
        {"type": "predicate", "field_id": "founded_on",
         "operator_id": "gte", "values": ["2020-01-01"]},
    ],
)

Paginate by passing the last row's uuid as after_id on the next call. The API is rate-limited per key and metered per plan, so batch your field requests (ask for everything you need in one call) rather than making a round trip per field.

One thing to plan for: Crunchbase removed free API access in 2025. As of 2026 the API ships with paid tiers, a limited Basic license exposing a handful of endpoints, and full search plus firmographic access on Pro or an Enterprise/Applications data license (enterprise contracts commonly start in the five figures). Price the API into the project before you write a line of code, because it changes the build-versus-buy math against scraping.

Here is the rule that keeps a crunchbase data scraping project defensible: collect only what an anonymous visitor can load, and never authenticate to reach more.

Crunchbase serves a public organization page at https://www.crunchbase.com/organization/. An anonymous visitor sees the company name, description, website, categories, location, and a trimmed funding summary. The deeper layers, full funding-round tables, complete investor lists, contact emails, and people detail, are gated behind a Crunchbase Pro login and a paywall. That gate is the line.

• Public page fields (name, domain, description, category, location, funding summary) are collectable, subject to the site's terms.
• Gated fields behind the login or the Pro paywall are off limits to scraping. The compliant way to get them is the API or a data license.

Two habits keep you on the right side of it. First, read robots.txt before you crawl and honor its Disallow rules:

curl -s "https://www.crunchbase.com/robots.txt"

Second, remember that Crunchbase's Terms of Service restrict automated crawling of the site. US courts have generally declined to treat scraping public pages as unauthorized computer access (hiQ Labs v. LinkedIn, 9th Cir. 2022; Van Buren v. United States, 2021), but a site's ToS is a contract, and breaching it carries its own risk of account termination and civil action. For anything at volume, the API is not just cleaner, it's the route Crunchbase actually offers you.

Crunchbase sits behind a CDN with bot management, and its public pages are heavily JavaScript-rendered. A plain requests.get gets you a challenge page or a near-empty HTML shell, not the company data. Three things break a naive scrape:

1. JavaScript hydration. The firmographic fields load client-side after the initial HTML. You need a real browser that runs JS, not a raw HTTP fetch.
2. Bot fingerprinting. The CDN scores TLS fingerprints, header order, and browser signals. Default automation stacks (a bare headless Chrome, an obvious client) get flagged fast.
3. IP rate limits. Fire a few hundred requests from one address and you collect 429s, then a block. Datacenter ranges get filtered harder than residential ones on defended sites.

The fix is the same discipline that works on any protected target: rotate the exit IP, present a real browser fingerprint, pace requests, and back off on 429. Our guide on how to avoid getting your proxy blocked walks through the header hygiene, rotation, and retry patterns in detail. The rest of this section shows the managed way to get all of that in one call.

Running your own rotating pool plus a headless browser plus stealth patches is a lot of moving parts to maintain. The SparkProxy Scraping API folds them into one request: it rotates the exit IP server-side, renders JavaScript in a real Chromium, and ships randomized fingerprints by default. Authentication is the X-API-Key header carrying a key of the form sk-... from your dashboard.

A thin helper wraps the endpoint:

import requests

SPARK_API = "https://scrape.sparkproxy.io/api/v1"
SPARK_KEY = "sk-your-api-key"  # from your SparkProxy dashboard

def scrape(url, country="us", rules=None, render=True, premium=True, stealth=True):
    payload = {
        "url": url,
        "render_js": render,       # Crunchbase hydrates client-side, so keep this on
        "premium_proxy": premium,  # residential exit, passes datacenter-range filters
        "stealth": stealth,        # extra anti-bot layers for CDN-protected pages
        "country_code": country,
        "format": "json",
    }
    if rules:
        payload["extract_rules"] = rules
    r = requests.post(
        SPARK_API,
        headers={"X-API-Key": SPARK_KEY, "Content-Type": "application/json"},
        json=payload,
        timeout=90,
    )
    r.raise_for_status()
    return r.json()

Point it at a public organization page and pull the public fields with extract_rules, which returns JSON keyed by your rule names instead of raw HTML:

# Public fields only. Crunchbase selectors change often, so treat these as a
# starting point and re-check them against the live DOM before a big run.
public = scrape(
    "https://www.crunchbase.com/organization/stripe",
    rules={
        "name":        "h1",
        "description": "span.description",
        "website":     "a[href^='http']@href",
        "location":    {"selector": "span.location", "type": "list"},
        "industries":  {"selector": "chips-container a", "type": "list"},
    },
)

If a cookie or consent banner blocks the content, drive past it with a js_scenario and wait for the field you care about to appear before capture:

consent = scrape(
    "https://www.crunchbase.com/organization/stripe",
    rules={"name": "h1", "description": "span.description"},
)
# Or use a scenario for stubborn interstitials:
payload = {
    "url": "https://www.crunchbase.com/organization/stripe",
    "render_js": True, "premium_proxy": True, "stealth": True, "format": "json",
    "js_scenario": {"instructions": [
        {"click": "#onetrust-accept-btn-handler"},
        {"wait_for": "h1"},
    ]},
    "extract_rules": {"name": "h1", "description": "span.description"},
}

Two honest caveats. Crunchbase obfuscates and rotates its CSS class names, so any selector set is a maintenance cost, not a one-time write. And the public page will never give you the gated funding-round detail or investor lists no matter how good your scraper is, because that data is not on the anonymous page. When you need those, go back to the API.

Raw pages and raw API responses are not a usable dataset. The gap is normalization, and it's where most homegrown company data scraping pipelines fall apart. Collapse everything to a single record keyed on the canonical domain, because the domain is the only identifier that stays stable across sources.

from urllib.parse import urlparse

def canonical_domain(url):
    host = urlparse(url).netloc.lower()
    return host[4:] if host.startswith("www.") else host

def to_record(api_org=None, scraped=None):
    props = (api_org or {}).get("properties", {})
    website = props.get("website_url") or (scraped or {}).get("website", "")
    return {
        "domain":      canonical_domain(website),  # primary key
        "name":        props.get("identifier", {}).get("value")
                       or (scraped or {}).get("name"),
        "description": props.get("short_description")
                       or (scraped or {}).get("description"),
        "employees":   props.get("num_employees_enum"),   # a range bucket, not a count
        "funding_usd": (props.get("funding_total") or {}).get("value_usd"),
        "industries":  [c.get("value") for c in props.get("categories", [])],
        "collected":   "2026-07-30",  # stamp every record for freshness decay
    }

Four rules make the output trustworthy:

1. Normalize the domain: strip www, lowercase, resolve redirects, so Stripe.com, www.stripe.com, and stripe.com/ become one key.
2. Deduplicate on the domain, never the name. "Acme, Inc.", "Acme Inc", and "ACME" are three rows until you resolve them to acme.com.
3. Parse the sub-fields. Read funding_total.value_usd, not funding_total, and treat num_employees_enum as a band.
4. Stamp and decay. Tag each record with a collection date and re-pull on a schedule, because B2B data goes stale fast.

The same field-and-metadata discipline is the backbone of any collection program. Our guide on using proxies for market research and data collection goes deeper on capturing structured fields and metadata so downstream enrichment stays reliable.

This is a build-versus-buy call, and it turns on volume, budget, and which fields you actually need. Neither answer is universal.

The clean split: use the API for anything that needs funding, investors, or scale, and reserve scraping for light, public firmographic checks where a license is overkill. If you are weighing a managed API against running your own proxy pool more broadly, our breakdown of a web scraping API vs self-managed proxies lays out the per-request cost and the maintenance trade-off in full.

Crunchbase data feeds sales and marketing, which means it carries personal data (founder and executive names, sometimes emails) and real regulatory obligations. "Public" and "compliant to use" are two different tests, and a defensible program passes both.

• Terms of Service. Crunchbase's ToS restricts automated crawling of the site. The API and data licenses are the sanctioned access routes. Breaching the ToS risks account termination and civil claims, even where the underlying access is not criminal.
• GDPR (EU/UK). A named founder or a business email like jane@acme.com is personal data. You can process it for B2B prospecting under the legitimate-interest basis (Article 6(1)(f); Recital 47 contemplates direct marketing), but you owe the person notice, data minimization, and an easy way to object. Honor objections promptly.
• CCPA/CPRA (California). Adds a notice-at-collection duty and deletion or opt-out rights for California residents.
• Access line (CFAA, US). Courts have declined to treat scraping public pages as unauthorized access, but bypassing a login, paywall, or CAPTCHA to reach gated Crunchbase data is a different category. Do not cross it.

The safe posture is consistent: prefer the licensed API, scrape only genuinely public pages, store a lawful basis and a collection date for every record, and make opt-out instant. None of this is legal advice, so check your specific use case and jurisdiction with counsel.

Clean Crunchbase records are raw material for two jobs: building a target list that matches your ICP, and scoring it by intent so reps call the right accounts first.

Firmographics do the fit filtering. Headcount band, industry category, HQ location, and company age narrow the universe to accounts worth a rep's time. Funding signals do the timing. A recent raise means fresh budget and a hiring wave, which is exactly when many vendors want to reach a company. Pair the funding date with the funding stage and you can rank a list by buying readiness before anyone fills out a form.

The mechanics are the same competitive-monitoring patterns that e-commerce teams already run. Our guide on how e-commerce companies use proxies for competitive intelligence shows the collect-normalize-score loop pointed at pricing; here you point it at funding, headcount, and category instead. Deduplicate on the domain, enrich from the API where the license allows, layer in job-posting and tech-stack signals from other public sources, and you have a scored account list built from data you can stand behind.

Collect public company data without the block wall

SparkProxy runs rotating datacenter and residential pools with 40+ country geo-targeting, plus a managed Scraping API that rotates IPs and renders JavaScript server-side. Pull the public firmographics you're entitled to, cleanly and at scale.

Start collecting company data today