How to Scrape Alibaba Product Data

Both sites belong to Alibaba Group and share an anti-bot stack, which is where the similarity ends. AliExpress is B2C: one buyer, one unit, one price, a review count, an order count. Alibaba.com is B2B: one buyer negotiating a container, a price that falls as the quantity rises, and a counterparty you may wire five figures to. The data model differs enough that reusing a retail scraper produces silently wrong output.

The practical consequence: a product row scraped from Alibaba is incomplete on its own. Two listings quoting "$2.10" tell you nothing until you know at what quantity, in what unit, and from whom. If your target is actually consumer retail pricing, the companion guide How to Scrape AliExpress Product Data covers the B2C model, window.runParams, and the slider CAPTCHA in detail. This one assumes you're sourcing, not shopping.

Settle the framing before writing code, because B2B scraping brushes against company data more than retail scraping does.

In the United States, the Ninth Circuit in hiQ Labs v. LinkedIn (2022) held that scraping publicly accessible data, meaning pages with no login and no authentication wall, generally does not violate the Computer Fraud and Abuse Act. That ruling addresses unauthorized access. It is not a license. Alibaba's Terms of Use separately prohibit automated data collection, so a scrape can breach a contract even where it clears the CFAA. Two different questions, two different answers.

Guardrails that keep a sourcing project defensible:

• Stay on public listing and company-profile pages. Nothing behind a login, no message threads, no buyer contact details.
• Company data is not automatically free of privacy law. A supplier's business name and business type are corporate facts; a named sales contact plus their direct email is personal data under the GDPR if that supplier is in scope, so drop contact fields unless you have a lawful basis.
• Never automate RFQ submission or supplier messaging. Those require an account, which means you've agreed to the terms, and they generate real inbound work for real people.
• Rate-limit and back off. A sourcing dataset refreshed weekly does not need aggressive concurrency.

Price benchmarking, category research, supplier discovery, and landed-cost modeling are ordinary uses of public Alibaba data. For commercial deployment, get counsel. This is engineering guidance.

Alibaba product pages live at https://www.alibaba.com/product-detail/_.html. The numeric productId at the end is your primary key; the slug portion changes when a supplier edits the title and should never be part of your key.

Like AliExpress, the page serializes its state into a JavaScript global rather than scattering it across CSS classes. Alibaba's global names drift between front-end releases and A/B buckets, so treat the exact variable name as a moving target and the shape as stable. The fields below reflect the detailData shape as of mid-2026; confirm the paths against a live dump before you trust them in production.

Note that lead time is a ladder too, keyed on quantity, and almost every tutorial ignores it. A supplier who is cheapest at 5,000 units and quotes 45 days may lose to one who is four cents dearer at 20 days. Capture both ladders or your model is only half a model.

The SparkProxy Scraping API takes a target URL and returns the rendered page, handling proxy rotation, browser rendering, and the anti-bot layer. Base endpoint: https://scrape.sparkproxy.io/api/v1. Auth is one header, X-API-Key.

Four parameters carry an Alibaba scrape:

• render_js=true: the detail page assembles its modules client-side, so a plain fetch can return a shell with no product state.
• premium_proxy=true: residential exits. Alibaba's defenses flag datacenter ranges quickly on repeated product-detail hits.
• stealth=true: fingerprint hardening against the baxia challenge that Alibaba shares with AliExpress. Requires render_js=true.
• country_code: ISO alpha-2 exit country, which drives the currency shown and occasionally the logistics block.

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.alibaba.com/product-detail/example_1600123456789.html" \
  --data-urlencode "render_js=true" \
  --data-urlencode "premium_proxy=true" \
  --data-urlencode "stealth=true" \
  --data-urlencode "country_code=US" \
  --data-urlencode "wait_for=.price-range"

wait_for takes a CSS selector and blocks capture until it appears, which is cheaper insurance than a fixed wait. Full parameters are in the Scraping API docs. If you're deciding between this and running your own pool, Web Scraping API vs Self-Managed Proxies has the cost math.

Wrap the fetch so every call carries the same parameters, then treat the response body as suspect until proven otherwise.

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = "YOUR_API_KEY"

def fetch_product(product_id: str, country: str = "US") -> str:
    url = f"https://www.alibaba.com/product-detail/x_{product_id}.html"
    resp = requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": url,
            "render_js": "true",
            "premium_proxy": "true",
            "stealth": "true",
            "country_code": country,
        },
        timeout=120,
    )
    resp.raise_for_status()
    return resp.text

The slug before the underscore is cosmetic; Alibaba resolves on the numeric id, so x_ works as a placeholder when you only have ids.

Alibaba's block pages return HTTP 200. A slider challenge, a punish interstitial, or an empty shell all come back "successful", so resp.ok proves nothing. Detect blocks by content:

BLOCK_MARKERS = (
    "_____tmd_____/punish",   # Alibaba punish interstitial
    "x5secdata",              # baxia security token page
    "nc_1_wrapper",           # slider CAPTCHA widget
    "Please slide to verify",
)

def is_blocked(html: str) -> bool:
    if any(marker in html for marker in BLOCK_MARKERS):
        return True
    # a real detail page always carries product state
    return "productLadderPrices" not in html and "detailData" not in html

That last line is the one that saves you. A 200 with no product state is a soft block, and if you skip the check you'll write thousands of empty rows and only notice a week later.

Next, pull the JSON out. Because the global's name moves, scan for candidates instead of hard-coding one:

import json, re

CANDIDATE_GLOBALS = ("detailData", "__INIT_DATA__", "__PAGE_DATA__", "runParams")

def extract_state(html: str) -> dict | None:
    for name in CANDIDATE_GLOBALS:
        m = re.search(rf"window\.{name}\s*=\s*(\{{)", html)
        if not m:
            continue
        start = m.start(1)
        depth, in_str, esc = 0, False, False
        for i in range(start, len(html)):
            ch = html[i]
            if in_str:
                if esc:          esc = False
                elif ch == "\\": esc = True
                elif ch == '"':  in_str = False
                continue
            if ch == '"':   in_str = True
            elif ch == "{": depth += 1
            elif ch == "}":
                depth -= 1
                if depth == 0:
                    try:
                        return json.loads(html[start:i + 1])
                    except json.JSONDecodeError:
                        break
    return None

Brace matching beats a greedy regex here. The Alibaba payload nests deeply and contains braces inside strings (product descriptions love {}), so a re.search(r"\{.*\}") grabs the wrong span roughly as often as the right one. The string-aware walk above costs a few milliseconds and never mis-slices.

Traverse the result defensively, since modules disappear on some listings:

def dig(obj, path: str, default=None):
    cur = obj
    for key in path.split("."):
        if isinstance(cur, list):
            try: key = int(key)
            except ValueError: return default
            if key >= len(cur): return default
            cur = cur[key]
        elif isinstance(cur, dict):
            if key not in cur: return default
            cur = cur[key]
        else:
            return default
    return cur if cur is not None else default

For a broader treatment of finding these payloads on any site, How to Scrape Hidden JSON API Endpoints covers the network-tab method that also works here.

Here's the field that separates a useful Alibaba dataset from a decorative one. productLadderPrices is an array of quantity bands, each with a floor, an optional ceiling, and a unit price:

[
  { "min": 2,    "max": 99,   "price": "3.50" },
  { "min": 100,  "max": 499,  "price": "2.80" },
  { "min": 500,  "max": 1999, "price": "2.35" },
  { "min": 2000, "max": null, "price": "2.10" }
]

The card in search results reads "$2.10 - 3.50". Those are the ladder's endpoints, nothing more. Sorting suppliers by that displayed low price ranks them by how deep a discount they'll give at volumes you may never buy.

def parse_product(state: dict) -> dict:
    comp = dig(state, "componentsVO", {})
    ladder_raw = dig(comp, "priceModule.productLadderPrices", []) or []
    ladder = [
        {
            "min_qty": int(t.get("min") or 0),
            "max_qty": int(t["max"]) if t.get("max") else None,
            "unit_price": float(str(t.get("price", "0")).replace(",", "")),
        }
        for t in ladder_raw
    ]
    ladder.sort(key=lambda t: t["min_qty"])

    return {
        "product_id": str(dig(state, "globalData.product.productId", "")),
        "title": dig(state, "globalData.product.subject", ""),
        "currency": dig(comp, "priceModule.currencyCode", "USD"),
        "ladder": ladder,
        "moq_qty": dig(comp, "tradeModule.moqNumber"),
        "moq_unit": (dig(comp, "tradeModule.moqUnit", "") or "").lower(),
        "lead_time": dig(comp, "shippingModule.leadTime", []),
        "company_id": str(dig(comp, "companyModule.companyId", "")),
        "company_name": dig(comp, "companyModule.companyName", ""),
    }

Some listings ship a single price with no ladder, which is common for machinery and for suppliers who quote by negotiation only. Handle the empty array rather than assuming index 0 exists.

Two things must happen before any cross-supplier comparison is valid: units get normalized, and price gets evaluated at your quantity.

MOQ units are free text chosen by the seller. "Pieces", "Piece", "pcs", "Sets", "Bags", "Cartons", "Kilograms", "Meters", "Pairs" all appear, and a set of 12 is not a piece.

UNIT_ALIASES = {
    "piece": "piece", "pieces": "piece", "pcs": "piece", "pc": "piece",
    "unit": "piece", "units": "piece", "item": "piece", "items": "piece",
    "set": "set", "sets": "set",
    "pair": "pair", "pairs": "pair",
    "carton": "carton", "cartons": "carton", "box": "carton", "boxes": "carton",
    "bag": "bag", "bags": "bag",
    "kilogram": "kg", "kilograms": "kg", "kg": "kg", "kgs": "kg",
    "ton": "ton", "tons": "ton", "metric ton": "ton",
    "meter": "m", "meters": "m", "metre": "m", "m": "m",
    "square meter": "sqm", "square meters": "sqm",
}

def normalize_unit(raw: str) -> str:
    key = (raw or "").strip().lower().rstrip(".")
    return UNIT_ALIASES.get(key, key or "unknown")

Keep the unnormalized string alongside the normalized one. When a new alias shows up you want the raw value in your table, not a silent bucket of unknown.

Now the calculation nobody publishes. Given a target quantity, walk the ladder to the applicable band and return the effective unit price, flagging listings whose MOQ puts them out of reach:

def effective_price(product: dict, target_qty: int) -> dict:
    ladder = product.get("ladder") or []
    moq = product.get("moq_qty") or 0

    if not ladder:
        return {"unit_price": None, "reason": "no_ladder"}
    if target_qty < moq:
        return {"unit_price": None, "reason": f"below_moq({moq})"}

    tier = None
    for t in ladder:
        if target_qty >= t["min_qty"] and (t["max_qty"] is None or target_qty <= t["max_qty"]):
            tier = t
            break
    if tier is None:                       # above the top band's floor
        tier = ladder[-1]

    return {
        "unit_price": tier["unit_price"],
        "extended_cost": round(tier["unit_price"] * target_qty, 2),
        "tier": f"{tier['min_qty']}-{tier['max_qty'] or '+'}",
        "reason": "ok",
    }

Run that across a category at, say, 1,000 units and the ranking often inverts against the displayed prices. A supplier advertising "$1.80" from a 10,000-unit floor is irrelevant at 1,000, while one advertising "$2.60 - 3.10" may quote $2.60 at exactly your band. That inversion is the entire value of scraping Alibaba rather than eyeballing it.

Store the ladder as its own table, one row per band, keyed on product_id. Flattening it into price_min and price_max columns discards the shape and you'll have to re-scrape to get it back. How to Store Scraped Data walks through schema choices for this kind of nested output.

Alibaba's badges are not equivalent, and treating them as one "trust score" is the most common analytical error in sourcing datasets. Here's what each one actually certifies:

Two things follow. First, "Gold Supplier" in your dataset is close to a constant, so it carries almost no discriminating information; drop it from any scoring model or it will just add noise. Second, businessType is self-declared and the most valuable field to verify, because a trading company reselling a factory's goods sits a margin layer above the source. Cross-check it: trading companies typically list a wide catalog spanning unrelated categories, while a real factory concentrates in one. You can compute that from data you already have.

def supplier_signals(state: dict) -> dict:
    comp = dig(state, "componentsVO", {})
    cm = dig(comp, "companyModule", {}) or {}
    rate = str(cm.get("responseRate", "")).replace("%", "").strip()
    return {
        "company_id": str(cm.get("companyId", "")),
        "company_name": cm.get("companyName", ""),
        "business_type": cm.get("businessType", ""),
        "years_active": int(cm.get("year") or 0),
        "verified_supplier": bool(cm.get("verifiedSupplier")),
        "trade_assurance": bool(dig(comp, "tradeModule.tradeAssurance", False)),
        "response_rate": float(rate) if rate.replace(".", "").isdigit() else None,
        "country": cm.get("countryName", ""),
    }

A workable composite, weighted toward the signals that cost effort rather than money: 0.35 verified + 0.25 response_rate_normalized + 0.20 min(years / 10, 1) + 0.20 trade_assurance. Tune it against outcomes you actually observe, and keep the components in your table so you can re-weight later without re-scraping.

Category and search pages are how you build the candidate list. The URL pattern is https://www.alibaba.com/trade/search?SearchText=&page=, and useful filters ride along as query parameters, including a server-side price band (priceFrom, priceTo) that cuts your crawl volume before you spend credits.

For list pages, the Scraping API's extract_rules parameter returns structured JSON directly, so you skip HTML parsing entirely:

import json

RULES = {
    "cards": {
        "selector": ".organic-list .organic-gallery-offer-outter",
        "type": "list",
        "output": {
            "url":      {"selector": "a.elements-title-normal", "output": "@href"},
            "title":    {"selector": "a.elements-title-normal", "output": "@title"},
            "price":    {"selector": ".elements-offer-price-normal__price"},
            "moq":      {"selector": ".element-offer-price-normal__minorder"},
            "supplier": {"selector": ".organic-gallery-offer__seller-company"},
        },
    }
}

def search_page(query: str, page: int = 1) -> list[dict]:
    resp = requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": f"https://www.alibaba.com/trade/search?SearchText={query}&page={page}",
            "render_js": "true",
            "premium_proxy": "true",
            "stealth": "true",
            "country_code": "US",
            "extract_rules": json.dumps(RULES),
        },
        timeout=120,
    )
    resp.raise_for_status()
    return resp.json().get("cards", [])

Search-card selectors change more often than the detail-page JSON shape, so validate the extraction on every run and alert on an empty cards array instead of writing zero rows quietly.

Pagination stops when a page returns nothing new. Track ids, not page numbers, because Alibaba re-ranks between requests and the same listing can appear on pages 3 and 5:

import re, time, random

ID_RE = re.compile(r"_(\d{10,})\.html")

def crawl_search(query: str, max_pages: int = 20) -> dict[str, dict]:
    seen: dict[str, dict] = {}
    for page in range(1, max_pages + 1):
        cards = search_page(query, page)
        if not cards:
            break
        new = 0
        for c in cards:
            m = ID_RE.search(c.get("url") or "")
            if not m:
                continue
            pid = m.group(1)
            if pid not in seen:
                seen[pid] = c | {"product_id": pid, "found_page": page}
                new += 1
        if new == 0:                    # fully overlapping page, stop
            break
        time.sleep(1.5 + random.random())
    return seen

Then hydrate the ids you kept with detail-page fetches. Search cards give you a price range and a supplier name; only the detail page gives you the ladder, and the ladder is the point.

This is where an Alibaba dataset stops resembling a retail scrape. In sourcing, the row you care about is the supplier, and product rows are evidence about that supplier. Group by company_id and derive:

from collections import defaultdict
from statistics import median

def build_supplier_table(products: list[dict], target_qty: int = 1000) -> list[dict]:
    by_company = defaultdict(list)
    for p in products:
        if p.get("company_id"):
            by_company[p["company_id"]].append(p)

    rows = []
    for cid, items in by_company.items():
        prices = [
            r["unit_price"]
            for r in (effective_price(p, target_qty) for p in items)
            if r["unit_price"] is not None
        ]
        moqs = [p["moq_qty"] for p in items if p.get("moq_qty")]
        rows.append({
            "company_id": cid,
            "company_name": items[0].get("company_name", ""),
            "listings_in_category": len(items),
            "median_price_at_qty": round(median(prices), 3) if prices else None,
            "min_moq": min(moqs) if moqs else None,
            "reachable_listings": len(prices),          # priced at your quantity
            "moq_units": sorted({p.get("moq_unit", "") for p in items}),
        })
    return rows

reachable_listings divided by listings_in_category is a quick screen: a supplier with 40 listings of which 3 are buyable at your quantity is aiming at a different buyer than you. Mixed moq_units inside one supplier is a mild trading-company tell, since factories usually sell in the unit they produce in.

Dedupe matters more here than on retail sites. The same factory frequently posts one product under several titles to occupy more search results, and separate resellers list the identical item using the supplier's own photos. Hash the first gallery image and cluster on (company_id, image_hash) to collapse the first case, and on image_hash alone to spot the second. When one image hash appears under six company ids, five of them are resellers.

Where RFQ fits. Alibaba's Request for Quotation flow is the other half of B2B, and it splits cleanly. Submitting an RFQ requires an account and generates real work for suppliers, so don't automate it. The public RFQ marketplace is a different matter: buyer-posted requests are public listings with a product description, a target quantity, and a destination country. That is demand-side data, and almost nobody collects it. Scraped over a few months it shows which categories buyers are actively sourcing and at what order sizes, a leading indicator that no supply-side price scrape can give you. Keep it to the public request text and quantity, and drop any buyer identity fields.

For the broader business framing of this kind of dataset, How E-commerce Companies Use Proxies for Competitive Intelligence covers how sourcing and pricing teams operationalize it.

Alibaba's defenses are the same baxia stack AliExpress runs, so the failure modes are familiar: slider challenges, punish redirects, and empty shells, all served with HTTP 200.

That last row is important: retrying a selector change wastes credits forever. Distinguish "blocked" from "changed" by checking whether the page contains product state at all.

def fetch_with_retry(product_id: str, attempts: int = 4) -> str | None:
    for i in range(attempts):
        try:
            html = fetch_product(product_id)
            if not is_blocked(html):
                return html
        except requests.RequestException:
            pass
        time.sleep((2 ** i) + random.random())
    return None

The jitter is doing real work. Without it, a batch that hits a challenge together retries together and re-triggers the same defense in lockstep. Keep concurrency modest, 5 to 15 workers, and persist as you go so a crash at product 40,000 doesn't cost the first 39,999.

For storage, three tables beat one wide file: products (one row per listing per scrape date), price_ladders (one row per band), and suppliers (one row per company_id per scrape date). Stamp every row with scraped_at, country_code, and currency. Sourcing questions are almost always temporal ("did this factory's 1,000-unit price move?"), and you can only answer them if the region was held constant. The proxy-side discipline behind keeping any of this running is covered in How to Avoid Getting Your Proxy Blocked.