How to Scrape eBay Listings and Prices

Public listing pages are the data you can collect responsibly. A search results page and an active item page are served to anyone without a login, and the facts on them (price, condition, shipping cost) are not personal data. That is different from harvesting buyer identities, scraping behind a signed-in account, or replaying data in a way that breaks eBay's User Agreement. Keep three habits and you stay on the defensible side:

• Read robots.txt first. Fetch https://www.ebay.com/robots.txt and honor the disallowed paths. The search endpoint you will use here is public, but check before you widen the crawl.
• Throttle hard. A human does not open 240 listings a second. Add delays and cap concurrency so you never degrade the site for real users.
• Take facts, not people. Collect the listing attributes you need for pricing. Skip anything that ties a real person to behavior, and cache aggressively so you request each page once.

Pricing intelligence is the most common reason teams do this. If you are building a repricer or a competitor tracker, the workflow patterns in how ecommerce companies use proxies for competitive intelligence show where eBay data fits alongside other marketplaces.

Before you write a single selector, check whether eBay will just hand you the data. The Browse API returns active listings and item details as clean JSON, it is free to register for, and it will never get your IP challenged. Reach for scraping only when the API cannot answer your question.

The one thing the standard API does not give you freely is sold-price history, which is exactly what most pricing work needs. Here is how the options compare:

Getting a Browse API token is a two-step OAuth flow. First trade your app credentials for an application access token:

import base64, requests

client_id = "YourApp-PRD-xxxx"
client_secret = "PRD-xxxxxxxx"
basic = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()

token = requests.post(
    "https://api.ebay.com/identity/v1/oauth2/token",
    headers={
        "Authorization": f"Basic {basic}",
        "Content-Type": "application/x-www-form-urlencoded",
    },
    data={
        "grant_type": "client_credentials",
        "scope": "https://api.ebay.com/oauth/api_scope",
    },
).json()["access_token"]

Then search active listings. Note the X-EBAY-C-MARKETPLACE-ID header, which pins the request to a specific eBay site:

r = requests.get(
    "https://api.ebay.com/buy/browse/v1/item_summary/search",
    headers={
        "Authorization": f"Bearer {token}",
        "X-EBAY-C-MARKETPLACE-ID": "EBAY_US",
    },
    params={"q": "mechanical keyboard", "limit": 50},
)
for item in r.json().get("itemSummaries", []):
    print(item["title"], item["price"]["value"], item["price"]["currency"])

If the Browse API covers your use case, stop here. You get structured ebay product data without maintaining a single CSS selector. People still scrape the public site for two reasons: sold-price history sits behind the Marketplace Insights approval wall, and the free API quota is too small for wide, frequent sweeps. The rest of this guide is for those cases.

Every eBay search card carries the same core fields, and each maps to a stable CSS class inside the li.s-item container. eBay does rotate its markup, so treat these selectors as a starting point and re-verify them when a scrape returns empty:

Two of these trip people up. The sold count on an active listing ("1,024 sold") is a lifetime popularity badge, not a record of individual sales. The sold archive (what a specific unit closed at, and when) lives behind the sold-listings filter covered later. And condition is eBay's own label, so normalize the strings ("Pre-Owned", "Used", "Open Box") into your own enum before you compare across listings.

Two libraries are enough: requests for fetching and parsel for CSS selection. parsel is the same selector engine Scrapy uses, and its .css() and ::text syntax reads cleanly.

pip install requests parsel

Send a real browser User-Agent and an Accept-Language header. eBay serves different markup and currency to a bare Python client, and a missing language header is one of the first bot signals it checks:

import requests
from parsel import Selector

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

url = "https://www.ebay.com/sch/i.html?_nkw=mechanical+keyboard&_ipg=240"
html = requests.get(url, headers=HEADERS, timeout=20).text
sel = Selector(text=html)

_ipg=240 asks for 240 items per page, the maximum eBay allows, which cuts the number of requests you make by a factor of four versus the default 60.

Pull the fields from the table into a dictionary, then filter out the placeholder card eBay injects at the top of every result set:

def parse_card(card):
    return {
        "title": card.css(".s-item__title::text").get(),
        "price": card.css(".s-item__price::text").get(),
        "bids": card.css(".s-item__bids::text").get(),
        "condition": card.css(".SECONDARY_INFO::text").get(),
        "seller": card.css(".s-item__seller-info-text::text").get(),
        "shipping": card.css(".s-item__shipping::text").get(),
        "sold": card.css(".s-item__quantitySold::text").get(),
        "url": card.css("a.s-item__link::attr(href)").get(),
    }

cards = sel.css("li.s-item")
listings = [parse_card(c) for c in cards]
# eBay makes the first card a "Shop on eBay" placeholder, so drop it
listings = [l for l in listings if l["title"] and l["title"] != "Shop on eBay"]

Prices come back as messy strings, and you cannot compare "$79.00 to $99.00" to "$89.99" until you parse them into numbers. This helper handles single prices, ranges, and thousands separators, and it is where you scrape eBay prices into something you can actually sort on:

import re

def parse_price(text):
    if not text:
        return None
    text = text.replace(",", "")
    nums = re.findall(r"\d+\.\d{2}", text)
    if not nums:
        return None
    low, high = float(nums[0]), float(nums[-1])   # "$79.00 to $99.00" -> two values
    return {"low": low, "high": high, "is_range": low != high}

Run every raw price string through parse_price before you store it. A range keeps both bounds so downstream math (median, min, spread) stays honest instead of silently dropping the high end.

eBay's search URL is fully query-driven, which means you can build any view without clicking through the UI. These are the parameters that matter:

Walk the pages by incrementing _pgn until a page returns no cards, which is the natural stop signal:

import time

def scrape_all_pages(keyword, max_pages=10):
    results = []
    for page in range(1, max_pages + 1):
        url = (
            f"https://www.ebay.com/sch/i.html?_nkw={keyword}"
            f"&_ipg=240&_pgn={page}"
        )
        html = requests.get(url, headers=HEADERS, timeout=20).text
        cards = Selector(text=html).css("li.s-item")
        page_items = [
            parse_card(c) for c in cards
            if c.css(".s-item__title::text").get() not in (None, "Shop on eBay")
        ]
        if not page_items:
            break                     # no more results, stop early
        results.extend(page_items)
        time.sleep(2)                 # stay polite, avoid rate limits
    return results

One eBay-specific limit to plan around: a single query stops returning fresh results after roughly 10,000 items, no matter how high you push _pgn. If your category is bigger than that, split it. Add a price band (_udlo and _udhi) or a sub-category so each query stays under the ceiling, then merge and de-duplicate the results by item ID afterward.

Sold listings are the gold in ebay scraping, because a closed sale is the only price that proves what a buyer actually paid. Add two filters to any search to switch from active items to the sold archive:

# Sold and completed listings: LH_Sold=1 and LH_Complete=1
sold_url = (
    "https://www.ebay.com/sch/i.html"
    "?_nkw=mechanical+keyboard"
    "&LH_Sold=1&LH_Complete=1"
    "&_ipg=240&_sop=13"          # _sop=13 sorts by most recently ended
)

On these pages the price shown is the final sale price, and each card carries the date it ended. The card structure is the same li.s-item layout, so your existing parse_card works without changes. Two constraints shape what you get. The public sold filter only retains roughly the last 90 days, and eBay increasingly gates deeper sold history behind a signed-in view, so scrape it soon after sales close rather than expecting a long tail.

The sanctioned route for the same data is the Marketplace Insights API, which returns sold items from the last 90 days as JSON. It is a Limited Release that needs business approval, but if you can get it, it is far more stable than parsing HTML:

# Sanctioned sold-data route (Limited Release, requires eBay approval)
r = requests.get(
    "https://api.ebay.com/buy/marketplace_insights/v1_beta/item_sales/search",
    headers={
        "Authorization": f"Bearer {token}",
        "X-EBAY-C-MARKETPLACE-ID": "EBAY_US",
    },
    params={"q": "mechanical keyboard", "limit": 50},
)

If you have used eBay scraping tutorials from a few years ago, they likely called the Finding API's findCompletedItems. eBay has been retiring the Finding API, so do not build new pipelines on it. Marketplace Insights is its replacement, and the public sold filter is the fallback when you lack approval.

eBay fronts its pages with commercial bot management, so aggressive scraping trips challenges instead of returning HTML. The failures are usually easy to recognize once you know the pattern:

Detect the soft failures explicitly. A 200 response with no listings is not success, it is a block wearing a disguise, and your parser will happily record zero results as if the query were empty:

def is_blocked(html):
    markers = ("Pardon Our Interruption", "Checking your browser", "unusual activity")
    return any(m.lower() in html.lower() for m in markers)

resp = requests.get(url, headers=HEADERS, timeout=20)
if resp.status_code == 429 or is_blocked(resp.text):
    print("Blocked or rate-limited, rotate the IP and back off")

The durable fixes are the same ones that keep any scraper alive: realistic headers, human-paced request timing, and a pool of clean IPs so no single address crosses eBay's threshold. Datacenter IPs handle the light pages; the hardest challenges need residential exits. For the full playbook on staying under the radar, see how to avoid getting your proxy blocked.

Rotating your own proxy pool, retrying failures, and rendering JavaScript for challenge pages is real infrastructure to own. The SparkProxy Scraping API handles all three server-side: it rotates the exit IP on every request, can run a headless browser when a page needs it, and lets you pin the exit country. You send a target URL and get the HTML back.

import requests
from parsel import Selector

resp = requests.get(
    "https://scrape.sparkproxy.io/api/v1",
    headers={"X-API-Key": "sk-xxxxxxxxxxxxxxxx"},   # key format: sk-...
    params={
        "url": "https://www.ebay.com/sch/i.html?_nkw=mechanical+keyboard&_ipg=240",
        "render_js": "false",     # eBay search HTML is server-rendered, 1 credit
        "country_code": "us",     # match the eBay marketplace you target
        "premium_proxy": "true",  # residential IPs for the hardest pages
    },
    timeout=60,
)
sel = Selector(text=resp.text)

Authentication is the X-API-Key header carrying a key of the form sk-... from your dashboard. Keep render_js off for eBay search pages, since they arrive server-rendered and a plain fetch costs 1 credit instead of 5. Turn premium_proxy on only when a target keeps challenging you, because residential IPs cost more per call. The practical split is to run cheap datacenter fetches for the wide, easy sweeps and reserve the API's residential path for the pages that fight back. If you are weighing that build-versus-buy decision in detail, web scraping API vs self-managed proxies lays out the cost and maintenance trade-offs.

Raw listings are not analysis. Three cleanup passes turn them into something you can price against:

1. Normalize prices through parse_price so every row holds numbers, not strings.
2. De-duplicate by item ID parsed from the url field, because the same listing can appear on more than one page during a long crawl.
3. Map shipping so "Free shipping" becomes 0.0 and folds into a landed-cost total.

Then write it out. CSV is enough to load into a sheet or a database:

import csv

with open("ebay_listings.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=listings[0].keys())
    writer.writeheader()
    writer.writerows(listings)

With sold listings normalized, the median closing price of the last 90 days is a far better anchor than any single active asking price. That number drives repricing, sourcing decisions, and demand tracking. If eBay is one input in a broader monitoring stack, the architecture in datacenter proxies for price comparison websites covers how to run the same collection across many retailers at once.