How to Scrape Steam Game Data and Prices

You can scrape store.steampowered.com/app/ as HTML, but you should not. The store page renders an age gate on mature titles, shifts its markup on promotions, and buries prices inside script tags. The endpoints behind it return clean, structured JSON that is far cheaper to parse and far less likely to break when Valve reskins the store.

Two facts make this practical. First, appdetails returns the full product record (name, description, developers, genres, price, Metacritic score) for any app ID. Second, appreviews returns paginated review text plus an aggregate summary. Neither requires a Steam Web API key. You only need a key for the official stats endpoints, and that key is free.

The trade you make is that these endpoints are undocumented. Valve can change them without notice, and they rate limit aggressively. Build for that from the start: cache responses, pace your requests, and expect the occasional null payload rather than a clean error.

Four sources cover almost every Steam data need. Keep this table handy.

Every app on Steam has a numeric app ID. Dota 2 is 570, Counter-Strike 2 is 730, Team Fortress 2 is 440. You find IDs from the store URL (/app/570/), from SteamSpy's all listing, or from the app-list endpoint at api.steampowered.com/ISteamApps/GetAppList/v2/, which dumps every app ID and name as one large JSON file.

The core call is a single GET request. Pass one app ID and you get the whole record back.

curl "https://store.steampowered.com/api/appdetails?appids=570&cc=us&l=english" \
  -H "User-Agent: Mozilla/5.0 (research; contact@sparkproxy.io)"

The response is keyed by the app ID, and each entry has a success flag and a data object. Always check success before reading data, because a delisted or region-blocked app returns {"570": {"success": false}} with no data at all.

import requests

def get_app(appid: int, cc: str = "us", lang: str = "english") -> dict | None:
    resp = requests.get(
        "https://store.steampowered.com/api/appdetails",
        params={"appids": appid, "cc": cc, "l": lang},
        headers={"User-Agent": "Mozilla/5.0 (research; contact@sparkproxy.io)"},
        timeout=20,
    )
    resp.raise_for_status()
    payload = resp.json()[str(appid)]
    if not payload.get("success"):
        return None
    return payload["data"]

app = get_app(570)
if app:
    print(app["name"], app["release_date"]["date"])
    print("Genres:", [g["description"] for g in app.get("genres", [])])
    print("Metacritic:", app.get("metacritic", {}).get("score"))

Useful fields inside data include name, steam_appid, is_free, short_description, developers, publishers, release_date.date, genres, categories, platforms, recommendations.total, and metacritic.score. For a catalog scrape, that single object replaces a dozen brittle CSS selectors.

To trim the payload when you only need part of the record, add filters. For example filters=basic returns just the core identity fields, and filters=price_overview returns only pricing. Smaller responses parse faster and count the same against the rate limit, so filter aggressively on large runs.

This is where scrape Steam prices gets interesting, and where most guides give you a subtly broken recipe. Pricing lives in data.price_overview:

"price_overview": {
  "currency": "USD",
  "initial": 999,
  "final": 499,
  "discount_percent": 50,
  "initial_formatted": "$9.99",
  "final_formatted": "$4.99"
}

Two things to internalize. Prices are integers in the currency's minor unit, so 499 means 4.99, not 499. Divide by 100 yourself. And price_overview is omitted entirely for free-to-play titles, so guard for its absence rather than assuming the key exists.

def get_price(appid: int, cc: str = "us") -> dict | None:
    resp = requests.get(
        "https://store.steampowered.com/api/appdetails",
        params={"appids": appid, "cc": cc, "filters": "price_overview"},
        headers={"User-Agent": "Mozilla/5.0 (research; contact@sparkproxy.io)"},
        timeout=20,
    )
    data = resp.json()[str(appid)]
    if not data.get("success") or "price_overview" not in data["data"]:
        return None  # free game, or no price in this region
    p = data["data"]["price_overview"]
    return {
        "currency": p["currency"],
        "final": p["final"] / 100,
        "discount": p["discount_percent"],
    }

You can batch prices to save requests. Pass comma-separated IDs with filters=price_overview, and Steam returns a price block for each ID in one call:

curl "https://store.steampowered.com/api/appdetails?appids=570,730,440&cc=de&filters=price_overview&l=english"

Now the catch that breaks region scrapers. The cc parameter asks for a region's pricing, but Steam cross-checks the request's IP address. From a US IP, cc=ar (Argentina) or cc=tr (Turkey) frequently returns USD pricing or falls back to your own region instead of the region you asked for. The store treats your network location as the source of truth for currency and availability, and cc only nudges it. If you need accurate Argentine, Turkish, or Brazilian prices, and those are exactly the regions people track because they run cheapest, you must send the request from an exit IP inside that country. That is not a nicety. It is the difference between real regional data and silent US fallbacks. See the SparkProxy section for the routing pattern.

The appreviews endpoint returns both an aggregate query_summary and the individual reviews. Add json=1 or you get an HTML fragment back.

curl "https://store.steampowered.com/appreviews/570?json=1&language=english&filter=recent&num_per_page=100"

The query_summary on the first page gives you the headline numbers without reading a single review body:

"query_summary": {
  "num_reviews": 100,
  "review_score": 8,
  "review_score_desc": "Very Positive",
  "total_positive": 1642885,
  "total_negative": 231004,
  "total_reviews": 1873889
}

Pagination is cursor based, not page numbered. The first request uses cursor=*. Each response returns a cursor value you feed into the next request, and the cursor contains characters that must be URL encoded. Stop when a page returns zero reviews or repeats the previous cursor.

import requests
from urllib.parse import quote

def scrape_reviews(appid: int, max_pages: int = 20) -> list[dict]:
    reviews, cursor, seen = [], "*", set()
    session = requests.Session()
    session.headers["User-Agent"] = "Mozilla/5.0 (research; contact@sparkproxy.io)"

    for _ in range(max_pages):
        resp = session.get(
            f"https://store.steampowered.com/appreviews/{appid}",
            params={
                "json": 1,
                "filter": "recent",
                "language": "english",
                "num_per_page": 100,
                "purchase_type": "all",
                "cursor": cursor,
            },
            timeout=20,
        )
        data = resp.json()
        batch = data.get("reviews", [])
        if not batch or cursor in seen:
            break
        seen.add(cursor)
        reviews.extend(batch)
        cursor = quote(data["cursor"])  # URL-encode for the next call
    return reviews

rows = scrape_reviews(570)
print(f"Pulled {len(rows)} reviews")
for r in rows[:3]:
    print(r["voted_up"], r["votes_up"], r["review"][:80])

Each review object carries recommendationid, review (the text), voted_up (the thumb), votes_up, timestamp_created, and an author block with steamid, num_games_owned, num_reviews, and playtime_forever in minutes. Useful filter values are recent, updated, and all; set language=all to pull every language or a specific code to narrow it. For sentiment work, filter_offtopic_activity=0 keeps review-bombing spikes in the dataset instead of Steam's default filtering.

There are two ways to get a Steam player count, and they measure different things.

The official Steam Web API returns the exact number of players in a game right now. It needs no key for this method:

def current_players(appid: int) -> int:
    resp = requests.get(
        "https://api.steampowered.com/ISteamUserStats/GetNumberOfCurrentPlayers/v1/",
        params={"appid": appid},
        timeout=15,
    )
    return resp.json()["response"]["player_count"]

print(current_players(730), "players in CS2 right now")

That gives you a live concurrent number, nothing historical. To track a trend, poll it on a schedule (every 10 to 15 minutes is plenty) and store each reading with a timestamp.

SteamSpy answers a different question: how many people own a game and how long they play. Its numbers are modeled estimates, not Valve's ground truth, so treat owner counts as ranges rather than exact figures.

curl "https://steamspy.com/api.php?request=appdetails&appid=570"

The response includes owners (a banded range like "100,000,000 .. 200,000,000"), ccu (peak concurrent users the previous day), average_forever and median_forever (playtime in minutes), and a price in US cents. SteamSpy caps appdetails requests at roughly one per second, and its bulk all endpoint at one request per 60 seconds, so it is built for periodic snapshots, not real-time polling.

The practical split: use the official GetNumberOfCurrentPlayers for live concurrency, and SteamSpy for ownership scale and playtime context. Together they give you both the pulse and the population.

The JSON endpoints ignore the age gate, which is one more reason to prefer them. But if you do need to fetch a mature title's store HTML (for tags or media the JSON omits), Steam redirects to an agecheck interstitial and withholds the content. You clear it with cookies, no login required.

Set birthtime to a Unix timestamp for a birthdate comfortably over 18, and add the mature-content flags:

curl "https://store.steampowered.com/app/1091500/" \
  -H "User-Agent: Mozilla/5.0" \
  --cookie "birthtime=631152000; lastagecheckage=1-January-1990; wants_mature_content=1; mature_content=1"

Here 631152000 is 1 January 1990 as a Unix timestamp, which reads as an adult in any current year. In Python with a session:

session = requests.Session()
session.cookies.update({
    "birthtime": "631152000",
    "lastagecheckage": "1-January-1990",
    "wants_mature_content": "1",
    "mature_content": "1",
})
html = session.get("https://store.steampowered.com/app/1091500/", timeout=20).text

Send those cookies on every store-page request in the session and the age gate stops interrupting you.

The Steam storefront endpoints are rate limited to roughly 200 requests per 5 minutes per IP, a figure the community has measured repeatedly rather than one Valve publishes. Cross it and you get HTTP 429s, then temporary IP blocks that widen if you keep pushing.

A few habits keep a scraper healthy:

• Pace to about one request per second. That keeps you comfortably under 200 per 5 minutes and looks nothing like a burst attack.
• Cache aggressively. Prices change on sale boundaries, not by the minute. Cache appdetails for a day or more, and only refresh player counts on the interval you actually chart.
• Batch prices with appids=a,b,c&filters=price_overview so one request covers many titles.
• Send a real User-Agent with a contact address. A blank or scripted agent is the first thing a bot filter flags.
• Back off on 429. Read any Retry-After hint, sleep, and resume rather than hammering through the block.

A minimal rate-aware wrapper looks like this:

import time

def fetch(session, url, params, tries=4):
    for attempt in range(tries):
        r = session.get(url, params=params, timeout=20)
        if r.status_code == 429:
            wait = int(r.headers.get("Retry-After", 2 ** attempt))
            time.sleep(wait)
            continue
        r.raise_for_status()
        return r
    raise RuntimeError("rate limited after retries")

For anything beyond a few hundred apps, one IP is not enough. You need to spread requests across many IPs, and for regional pricing you need those IPs in specific countries. That is a proxy job.

Two problems push a Steam scraper toward proxies: the 200-per-5-minute cap on a single IP, and the region lock on pricing. A pool of rotating IPs solves the first, and country-targeted exit IPs solve the second. The SparkProxy Scraping API handles both from one endpoint, so you pass a target URL and a country instead of managing IPs yourself.

curl "https://scrape.sparkproxy.io/api/v1?url=https://store.steampowered.com/api/appdetails%3Fappids%3D570%26cc%3Dtr%26filters%3Dprice_overview&country_code=tr&render_js=false" \
  -H "X-API-Key: YOUR_API_KEY"

The same call in Python, pulling Turkish pricing from a Turkish exit IP so cc=tr is actually honored:

import requests

def steam_price_in_region(appid: int, region: str) -> requests.Response:
    target = (
        "https://store.steampowered.com/api/appdetails"
        f"?appids={appid}&cc={region}&filters=price_overview&l=english"
    )
    return requests.get(
        "https://scrape.sparkproxy.io/api/v1",
        headers={"X-API-Key": "YOUR_API_KEY"},
        params={
            "url": target,
            "country_code": region,   # exit IP in the pricing region
            "render_js": "false",     # JSON endpoint, no browser needed
        },
        timeout=60,
    )

resp = steam_price_in_region(570, "tr")
print(resp.text)

Set render_js=false for the JSON endpoints, since they need no browser and the flag keeps each request cheaper. Match country_code to the cc value in your target URL so the exit IP and the requested region agree, which is the whole point. To compare a title across regions, loop the region codes and let the API place each request in the right country. If you would rather run your own pool, the same pattern works with SparkProxy rotating proxies, and the Scraping API docs list every parameter. For a deeper build-versus-buy breakdown, see Web Scraping API vs Self-Managed Proxies.

Scraping Steam sits in a gray area, so scrape responsibly. A few ground rules:

• The Steam Subscriber Agreement governs the storefront, and heavy automated access can violate it. Public price and review data is low risk to collect for analysis, but respect the intent behind the rate limits rather than trying to defeat them.
• Do not resell or republish review text or full catalog dumps as if they were yours. Aggregate, analyze, and cite instead.
• Personal data lives in reviews. Author steamids and profiles are personal data under regimes like the GDPR. If you store them, have a lawful basis and a retention limit.
• Prefer the official Web API where it covers your need. It is the sanctioned path, and it signals good faith.

The tools here exist to power price trackers, market research, and academic work. Keep your footprint small, cache what you can, and you stay on the right side of both the servers and the rules. For related market-monitoring patterns, see datacenter proxies for price comparison websites.