How to Scrape Ticketmaster Event Listings (Legally)
Scrape Ticketmaster event listings for price research and market analysis: the official Discovery API, its 1,000-item cap, and where the BOTS Act line sits.
If you want to scrape Ticketmaster event listings, the first thing to get right is not a selector, it's a boundary. Reading public event data for price research or market analysis is one activity. Automating ticket purchases is a different one, and in the United States it is a federal offense under the BOTS Act of 2016 that the FTC has actually prosecuted. This guide covers only the first: pulling event metadata, on-sale windows, and published price ranges at scale, using Ticketmaster's own Discovery API where it works and the public event pages where it doesn't. You get working code, the undocumented paging cap that breaks most first attempts, and a clear line under every technique.
The legal line: BOTS Act, plainly
The Better Online Ticket Sales Act of 2016, codified at 15 U.S.C. § 45c, makes two things unlawful. First, circumventing a security measure, access control system, or other technological control on a ticket seller's website when that control exists to enforce posted purchase limits or to protect the integrity of the online purchasing order. Second, selling or offering tickets you know were obtained that way. The FTC enforces it as an unfair or deceptive practice, with civil penalties above $50,000 per violation, adjusted for inflation each year.
This is not a formality that nobody acts on. In January 2021 the FTC brought its first BOTS Act cases against three New York ticket brokers who used software to buy tens of thousands of tickets past posted limits, winning judgments totaling $31 million (suspended to $3.7 million on payment). A March 2025 Executive Order on the live entertainment ticket market directed the FTC and the Justice Department to step up BOTS Act enforcement, and joint action against broker networks followed later that year. Ticket bots are one of the few areas of scraping with an actual federal statute and an actual enforcement record behind it.
Here is where each activity sits:
| Activity | Where it sits |
|---|---|
| Reading public event listings through the Discovery API | Allowed, governed by the API terms |
| Fetching a public event page and parsing displayed prices | Generally lawful reading, subject to site terms |
| Storing price snapshots over time for analysis | Fine, this is market research |
| Academic study of pricing behavior or fee structures | Fine, and a well-trodden research area |
| Automating add-to-cart, checkout, or ticket holds | Prohibited by the BOTS Act |
| Solving a CAPTCHA to reach or complete checkout | Circumventing an access control, prohibited |
| Bypassing a virtual waiting room to reach the buy page | Circumventing an access control, prohibited |
| Rotating accounts or IPs to exceed posted ticket limits | Squarely what the statute was written for |
| Reselling tickets acquired by any of the above | Separately prohibited, and it is how brokers get caught |
The distinction is buying versus reading. Everything in this guide is a GET request that renders a page a member of the public can already open. Nothing here touches a cart, a checkout, a session hold, or a queue token. If your project needs to acquire inventory rather than observe it, this guide is not for you and no proxy configuration will make that legal.
Outside the US the same split applies with different statutes. The UK's Breaching Limits on Ticket Sales Regulations 2018 makes bot purchasing a criminal offense with an unlimited fine. Ireland's Sale of Tickets (Cultural, Entertainment, Recreational and Sporting Events) Act 2021 does the same. None of them restrict reading published prices.
This is engineering guidance, not legal advice. If your data feeds a commercial product, have a lawyer read Ticketmaster's Terms of Use and the Discovery API Developer Terms before you ship.
The numbers you need before you start
The whole pipeline hangs off about a dozen constants, and none of them are guessable from a response body.
| Constant | Value | Why it matters |
|---|---|---|
| Discovery base URL | `https://app.ticketmaster.com/discovery/v2/` | Free JSON catalog, no bot wall in front of it |
| Discovery auth | `apikey` query parameter | No header, no OAuth handshake |
| Default quota | 5,000 calls per day | Request an increase, do not proxy around it |
| Rate ceiling | 5 requests per second | Pace against the `Rate-Limit-*` response headers |
| Max `size` | 200 items per page | Not 500, whatever old answers say |
| Deep-paging cap | `page * size` must stay under 1,000 | The most common cause of silent truncation |
| Timestamp format | `2026-09-01T00:00:00Z` | Milliseconds in the string return a 400 |
| Event page pattern | `https://www.ticketmaster.com/event/ | `eventId` is the same `id` Discovery returns |
| Durable extraction target | `script[type="application/ld+json"]` | schema.org `Event`, survives page redeploys |
| Pricing regime break | 12 May 2025 in the US | Face value before it, all-in after it |
| Page defense | Akamai Bot Manager plus TLS fingerprinting | Rendered request from a residential IP |
| Hard stop | Queue-it virtual waiting room | Access control on buying, skip the event |
Scraping at scale? Skip the blocks.
Fast, unblockable datacentre proxies with unlimited bandwidth.
Start with the official Discovery API
Most guides skip straight to HTML parsing, which is backwards. Ticketmaster publishes a free Discovery API that returns the same event catalog the site is built on, in clean JSON, with no bot wall in front of it. For listings work it is faster, more stable, and more defensible than scraping. Register at developer.ticketmaster.com and take the key from your app.
curl -s "https://app.ticketmaster.com/discovery/v2/events.json\
?apikey=$TM_API_KEY\
&city=Chicago&stateCode=IL\
&classificationName=music\
&startDateTime=2026-09-01T00:00:00Z\
&endDateTime=2026-09-30T23:59:59Z\
&size=200&sort=date,asc" | jq '.page'
The response envelope is consistent: events live under _embedded.events, and page carries size, number, totalPages, and totalElements. A client that survives a real run:
import os, time, requests # requests 2.32, Python 3.12+
TM_BASE = "https://app.ticketmaster.com/discovery/v2"
TM_KEY = os.environ["TM_API_KEY"]
SESSION = requests.Session()
class DiscoveryError(RuntimeError):
pass
def discovery(path, tries=4, **params):
params["apikey"] = TM_KEY
for attempt in range(tries):
r = SESSION.get(f"{TM_BASE}/{path}", params=params, timeout=30)
if r.status_code == 429:
# Rate-Limit-Reset is an epoch timestamp in MILLISECONDS, not a
# delay. Sleeping for its value directly parks the process for
# decades. Treat it as a deadline and clamp it.
reset_ms = int(r.headers.get("Rate-Limit-Reset", 0))
time.sleep(max(1.0, min(60.0, reset_ms / 1000 - time.time())))
continue
if r.status_code == 400:
# Never branch on the fault text: Ticketmaster has reworded it.
raise DiscoveryError(f"400 on {path} {params.get('page')}/"
f"{params.get('size')}: {r.text[:200]}")
r.raise_for_status()
return r.json()
raise DiscoveryError(f"rate limited {tries} times on {path}")
page = discovery(
"events.json",
city="Chicago", stateCode="IL",
classificationName="music",
startDateTime="2026-09-01T00:00:00Z",
endDateTime="2026-09-30T23:59:59Z",
size=200, sort="date,asc",
)
print(page["page"]["totalElements"], "events matched")
Three details trip people up. Timestamps are ISO 8601 UTC with no milliseconds, so 2026-09-01T00:00:00.000Z returns a 400. size caps at 200, not the 500 a 2019 Stack Overflow answer will tell you. And Rate-Limit-Reset is a wall-clock epoch in milliseconds, not a retry delay, which is why the backoff above subtracts time.time() before sleeping. Pass the raw value to sleep(), as plenty of sample code does, and the worker parks for decades while the symptom reads as a network stall.
Flatten each event into a record you can actually store:
def flatten(ev):
venue = (ev.get("_embedded", {}).get("venues") or [{}])[0]
prices = ev.get("priceRanges") or [{}]
sales = ev.get("sales", {}).get("public", {})
return {
"event_id": ev["id"],
"name": ev["name"],
"url": ev.get("url"),
"status": ev.get("dates", {}).get("status", {}).get("code"),
"local_date": ev.get("dates", {}).get("start", {}).get("localDate"),
"local_time": ev.get("dates", {}).get("start", {}).get("localTime"),
"onsale_start": sales.get("startDateTime"),
"onsale_end": sales.get("endDateTime"),
"presale_count": len(ev.get("sales", {}).get("presales") or []),
"venue": venue.get("name"),
"city": venue.get("city", {}).get("name"),
"state": venue.get("state", {}).get("stateCode"),
"country": venue.get("country", {}).get("countryCode"),
"price_min": prices[0].get("min"),
"price_max": prices[0].get("max"),
"currency": prices[0].get("currency"),
"segment": (ev.get("classifications") or [{}])[0]
.get("segment", {}).get("name"),
"ticket_limit": ev.get("ticketLimit", {}).get("info"),
}
records = [flatten(e) for e in page["_embedded"]["events"]]
dates.status.code is the field to watch. It moves through onsale, offsale, cancelled, postponed, and rescheduled, and a diff on that field across two runs is the cheapest event-lifecycle signal you can build.
The 1,000-item cap nobody warns you about
Here is the constraint that quietly ruins most first pipelines. Discovery refuses deep paging: page * size must stay under 1,000. Ask for size=200&page=5 and you get an HTTP 400 about maximum paging depth, not an empty result set. Los Angeles has far more than 1,000 upcoming events, so a naive loop returns five pages and dies, and you conclude the API is broken. It isn't. You are meant to narrow the query instead of paging deeper.
The fix is to slice the query space until every slice fits under 1,000, and the date axis slices most cleanly:
from datetime import datetime, timedelta, timezone
SEGMENTS = ["Music", "Sports", "Arts & Theatre", "Film", "Miscellaneous"]
def iso(d):
return d.strftime("%Y-%m-%dT%H:%M:%SZ")
def count_only(start, end, **filters):
"""One cheap probe: size=1 returns the true total without pulling events."""
return discovery("events.json", size=1, startDateTime=iso(start),
endDateTime=iso(end), **filters)["page"]["totalElements"]
def harvest_window(start, end, depth=0, max_depth=9, **filters):
"""Bisect a date window until every slice fits under the 1,000-item cap."""
total = count_only(start, end, **filters)
if total == 0:
return []
if total >= 1000:
if depth < max_depth:
mid = start + (end - start) / 2
return (harvest_window(start, mid, depth + 1, max_depth, **filters)
+ harvest_window(mid, end, depth + 1, max_depth, **filters))
# The date axis is exhausted: this slice is minutes wide and still over
# the cap. Switch axes instead of silently truncating at 1,000.
if "classificationName" in filters:
raise DiscoveryError(
f"{iso(start)}..{iso(end)} exceeds the cap on both axes; "
"partition by dmaId or venueId next")
rest = {k: v for k, v in filters.items() if k != "classificationName"}
return [e for seg in SEGMENTS
for e in harvest_window(start, end, classificationName=seg, **rest)]
out, page_no = [], 0
while page_no * 200 < total: # total < 1000, so the cap is respected
res = discovery("events.json", size=200, page=page_no, sort="date,asc",
startDateTime=iso(start), endDateTime=iso(end), **filters)
out += res.get("_embedded", {}).get("events", [])
page_no += 1
time.sleep(0.25) # 4 req/s, under the documented 5
return out
def dedupe(events):
"""Bisection boundaries are inclusive on both sides, so overlaps are normal."""
seen, out = set(), []
for e in events:
if e["id"] not in seen:
seen.add(e["id"])
out.append(e)
return out
events = dedupe(harvest_window(
datetime(2026, 9, 1), datetime(2027, 3, 1),
city="Los Angeles", classificationName="music",
))
print(len(events), "events harvested")
Three parts of that are load-bearing. The probe branches on totalElements rather than catching a 400 and reading the fault text, so a reworded error message cannot break the harvester. The max_depth arm refuses to guess: a window halved nine times and still over the cap gets partitioned on classification instead of truncated at 1,000, and if that axis is spent too it raises rather than hand you a quietly incomplete dataset. The dedupe is not optional either, because startDateTime and endDateTime are both inclusive, so every bisection boundary duplicates the events sitting exactly on it.
For the densest markets, dmaId partitions best: a designated market area is fixed geography, not a filter that events drift across.
That time.sleep(0.25) is deliberate: five requests per second is the documented ceiling, and the Rate-Limit-* headers let you pace against real state rather than guesswork. Do not route Discovery calls through rotating proxies to dodge your daily quota. That is quota evasion, it is trivially detectable through key usage patterns, and Ticketmaster grants higher limits to projects that ask. Ask.
What Discovery gives you and what it withholds
The API is generous on catalog data and deliberately quiet on inventory. Knowing the split tells you exactly when you need to touch a page at all.
| Data point | In Discovery API | Notes |
|---|---|---|
| Event name, ID, date, local time | Yes | Canonical, use `id` as your key |
| Venue, city, geo coordinates | Yes | Under `_embedded.venues` |
| Performer or attraction | Yes | Under `_embedded.attractions` |
| Genre and classification tree | Yes | Segment, genre, subgenre |
| On-sale and presale windows | Yes | `sales.public`, `sales.presales[]` |
| Event status transitions | Yes | `dates.status.code` |
| Face-value price range | Partial | `priceRanges[]` is often absent, and it is a static min and max |
| Posted ticket limit | Sometimes | `ticketLimit.info`, free text |
| Live inventory or seat counts | No | Not exposed, by design |
| Section-level or seat-level prices | No | Page only |
| Verified resale listings | No | Page only |
| All-in price including fees | No | Page only |
| Dynamic price movement | No | Page only, and only by snapshotting |
priceRanges is the field people over-trust. When present it is a static face-value band published at announce time, not what a buyer sees today, and for many events it is missing entirely. If your question is about price, Discovery gets you the event universe and the page gets you the number.
Scrape Ticketmaster event listings from the event page
Event pages sit at https://www.ticketmaster.com/event/, and that eventId is the id Discovery already handed you, so you never guess or crawl a search index. The page itself is a JavaScript application behind Akamai Bot Manager, so a plain requests.get returns a challenge rather than content. That is what the Scraping API is for: rendering and residential egress become request parameters instead of infrastructure you maintain.
curl -s -X POST "https://scrape.sparkproxy.io/api/v1" \
-H "X-API-Key: $SPARK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.ticketmaster.com/event/0B006125A9F84B77",
"render_js": true,
"premium_proxy": true,
"country_code": "us",
"stealth": true,
"wait_for": "script[type=\"application/ld+json\"]",
"wait": 2
}'
Wrapped for reuse, with the retry behavior you will need at volume:
import json, os, time, requests
SPARK = "https://scrape.sparkproxy.io/api/v1"
HEADERS = {"X-API-Key": os.environ["SPARK_API_KEY"],
"Content-Type": "application/json"}
def spark_get(url, tries=3, **opts):
body = {
"url": url,
"render_js": True,
"premium_proxy": True,
"country_code": "us",
"stealth": True,
"wait_for": 'script[type="application/ld+json"]',
"wait": 2,
"tag": "ticketmaster-listings",
**opts,
}
for attempt in range(tries):
r = requests.post(SPARK, headers=HEADERS, data=json.dumps(body), timeout=120)
if r.status_code == 200:
return r.text
if r.status_code in (429, 530, 503):
time.sleep(2 ** attempt * 3)
continue
r.raise_for_status()
raise RuntimeError(f"failed after {tries} attempts: {url}")
Geo matters more here than on most targets. Ticketmaster serves different currencies, fee disclosures, and even different catalogs by country, so pin country_code to the market you are researching and hold it constant across a series. Comparing a us snapshot against a gb one and calling the delta a price change wrecks a dataset quietly.
Parse the Event JSON-LD
Ticketmaster publishes schema.org Event markup on its event pages, because that markup powers event rich results in Google. It is the most durable extraction target on the page. Generated class names churn every deploy. Structured data that feeds search does not, because breaking it costs Ticketmaster organic traffic.
import re
from bs4 import BeautifulSoup # beautifulsoup4 4.13
def parse_event_jsonld(html):
soup = BeautifulSoup(html, "html.parser")
for tag in soup.find_all("script", type="application/ld+json"):
try:
data = json.loads(tag.string or "{}")
except json.JSONDecodeError:
continue
for node in (data if isinstance(data, list) else [data]):
t = node.get("@type", "")
types = t if isinstance(t, list) else [t]
if any("Event" in x for x in types):
return node
return None
def to_record(node):
offers = node.get("offers") or {}
if isinstance(offers, list):
offers = offers[0] if offers else {}
loc = node.get("location") or {}
addr = loc.get("address") or {}
return {
"name": node.get("name"),
"start": node.get("startDate"),
"event_status": node.get("eventStatus"),
"venue": loc.get("name"),
"city": addr.get("addressLocality"),
"region": addr.get("addressRegion"),
"currency": offers.get("priceCurrency"),
"low": offers.get("lowPrice") or offers.get("price"),
"high": offers.get("highPrice"),
"availability": offers.get("availability"),
"offer_url": offers.get("url"),
"valid_from": offers.get("validFrom"),
"performers": [p.get("name") for p in (node.get("performer") or [])
if isinstance(p, dict)],
}
offers.availability is a schema.org URL such as https://schema.org/InStock, LimitedAvailability, or SoldOut, and tracking its transitions across snapshots is a legitimate sell-through signal. It reports what the public page says about the event as a whole. It is not seat inventory, and you should not present it as one.
If you would rather skip the parsing layer entirely, the Scraping API can apply selectors server-side and hand back JSON:
curl -s -X POST "https://scrape.sparkproxy.io/api/v1" \
-H "X-API-Key: $SPARK_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.ticketmaster.com/event/0B006125A9F84B77",
"render_js": true, "premium_proxy": true, "country_code": "us",
"extract_rules": {
"title": "h1",
"structured": {"selector": "script[type=\"application/ld+json\"]", "type": "list"}
}
}'
Keep the raw HTML for a sampled fraction of runs. When a parser starts returning nulls three weeks from now, archived HTML is the only thing that tells you whether the page changed or your code did. The same discipline that applies to any hidden JSON API endpoint applies here.
All-in pricing changed what you are scraping
This is the piece most Ticketmaster scraping guides have not caught up with, and it will silently corrupt any longitudinal price dataset that spans 2025.
The FTC's Rule on Unfair or Deceptive Fees, 16 CFR Part 464, took effect on 12 May 2025 and covers live-event tickets directly. It requires that the total price a consumer will pay, service fees included, be displayed up front and more prominently than any other price figure. Ticketmaster moved US pages to all-in pricing to comply. The practical consequence: the headline number on a US event page after May 2025 is the fee-inclusive total, where the same page before that date showed face value with fees revealed at checkout.
If you are comparing 2024 snapshots to 2026 snapshots, you are comparing two different quantities. A 25% "increase" may be the fee disclosure moving, not the price. Store both figures separately and label the pricing regime on every row:
FEE_RE = re.compile(r"(fees? included|includes fees|all[- ]in pric)", re.I)
def price_record(soup, node_record, country_code, snapshot_ts):
"""Tag every row with the pricing regime it was captured under."""
text = soup.get_text(" ", strip=True)
all_in = bool(FEE_RE.search(text))
return {
**node_record,
"country_code": country_code,
"price_basis": "all_in" if all_in else "face_value",
"captured_at": snapshot_ts,
"source": "event_page",
}
Three rules follow. Never mix face_value and all_in rows in the same trend line without normalizing. Treat a face_value row from a US page after May 2025 as a parse failure, not data: it usually means the fee text moved and your detector went blind. And when you publish a comparison against a non-US market, check that market's disclosure regime first, because the EU, the UK, and Canada landed on all-in display through different rules on different dates. This is the nuance that separates a defensible market research dataset from a chart that falls apart under review.
The bot wall, and why the waiting room is the line
Ticketmaster runs one of the heavier defensive stacks on the public web, and the layers behave very differently depending on what you are doing.
| Layer | Where it appears | How to handle it while reading |
|---|---|---|
| Akamai Bot Manager | Sitewide, including event pages | Real browser fingerprint plus residential egress, which `render_js` with `premium_proxy` and `stealth` provides |
| TLS and JA3/JA4 fingerprinting | Edge, before any HTML | A stock HTTP client is fingerprintable regardless of headers, so render |
| Rate and behavior heuristics | Per IP and per session | Slow down, vary intervals, cap concurrency |
| Geo and currency gating | Per country | Pin `country_code`, do not rotate it mid-series |
| CAPTCHA challenge | Under load or on suspicion | Back off, widen the interval, retry later |
| Queue-it virtual waiting room | High-demand onsales only | Stop. Do not proceed. |
That last row is the whole ethical architecture of this topic in one line. A virtual waiting room exists for exactly one reason: to fairly order humans who are trying to buy. It is an access control on the purchase funnel. If your scraper finds itself in a queue, it has stopped reading a listing and started standing in a buying line, and circumventing it is the conduct 15 U.S.C. § 45c describes. The correct handling is not a clever bypass, it is raise and skip the event.
class BlockedError(RuntimeError):
"""Bot wall. Back off, do not escalate."""
class PurchaseFunnelError(PermissionError):
"""Waiting room. Out of scope by design, never retried."""
QUEUE_MARKERS = ("queue-it.net", "queue_it", "waitingroom", "you are now in line")
WALL_MARKERS = ("access denied", "pardon the interruption", "reference #")
def guard(html, url):
"""Detect the purchase funnel and refuse to continue."""
low = html.lower()
if any(m in low for m in QUEUE_MARKERS):
raise PurchaseFunnelError(f"waiting room detected, skipping {url}")
if any(m in low[:4000] for m in WALL_MARKERS):
raise BlockedError(f"bot wall on {url}")
return html
The two exception types are separate on purpose, because your retry policy has to treat them as opposites. BlockedError is a transport problem: back off and try the event again tomorrow. PurchaseFunnelError is a scope decision, permanent for that run, and it must never reach a retry queue. Collapse them into one generic exception and the first well-meaning "retry everything that failed" loop marches your crawler back into the queue it just declined to enter.
Call that guard on every single response. It keeps the crawler out of the purchase funnel structurally rather than by good intentions, and it leaves an audit trail. CAPTCHA gets the same treatment: back off and retry later, do not solve it. Solving a challenge that guards a listing page is a gray area, and solving one that guards checkout is not gray at all.
For the sitewide layer, the durable approach is the same one that works against Akamai Bot Manager anywhere else: present a consistent, real browser rather than a patched HTTP client, and keep your request rate somewhere a human could plausibly be.
Building a price-monitoring pipeline
The useful output is not a table of events, it is a time series. Structure it as append-only snapshots keyed by (event_id, captured_at), never a mutable current-price row, because the movement is the whole point.
The snapshot table
The shape of the table is where most of these projects go wrong, and the two columns people leave out are the two that decide whether the series means anything.
CREATE TABLE event_snapshot (
event_id text NOT NULL,
captured_at timestamptz NOT NULL,
country_code char(2) NOT NULL, -- never compare across markets
price_basis text NOT NULL -- never compare across regimes
CHECK (price_basis IN ('face_value', 'all_in')),
low_price numeric(10,2),
high_price numeric(10,2),
currency char(3),
availability text, -- InStock | LimitedAvailability | SoldOut
event_status text, -- from dates.status.code
source text NOT NULL, -- 'discovery' | 'event_page'
PRIMARY KEY (event_id, captured_at)
);
CREATE INDEX event_snapshot_series
ON event_snapshot (event_id, country_code, price_basis, captured_at DESC);
country_code and price_basis sit in the series index because every meaningful query has to filter on both. Group by event_id alone and you will average a US all-in total against a UK face value, rendering a trend that never happened. Indexing them makes the correct query the convenient one.
The daily cycle
import random
from datetime import datetime, timedelta, timezone
def now():
# datetime.utcnow() has been deprecated since Python 3.12 and returns a
# naive value that silently breaks timestamptz comparisons.
return datetime.now(timezone.utc)
def daily_cycle(conn, filters, country="us", page_sample=200):
# 1. Cheap and complete: refresh the catalog from Discovery
events = dedupe(harvest_window(now(), now() + timedelta(days=180), **filters))
catalog = {e["id"]: flatten(e) for e in events}
upsert_catalog(conn, catalog.values())
# 2. Expensive and selective: only page-scrape events worth watching
watchlist = select_watchlist(conn, catalog, limit=page_sample)
rows = []
for event_id in watchlist:
url = f"https://www.ticketmaster.com/event/{event_id}"
try:
html = guard(spark_get(url, country_code=country), url)
except PurchaseFunnelError as exc:
log_skip(conn, event_id, "purchase_funnel", str(exc), retry=False)
continue
except BlockedError as exc:
log_skip(conn, event_id, "blocked", str(exc), retry=True)
continue
soup = BeautifulSoup(html, "html.parser")
node = parse_event_jsonld(html)
if not node:
log_skip(conn, event_id, "no_jsonld", url, retry=True)
continue
rows.append(price_record(soup, to_record(node), country, now()))
time.sleep(random.uniform(3, 7))
validate_run(rows, attempted=len(watchlist))
insert_snapshots(conn, rows)
Note the order of the last two lines. Validate before you write, not after, so a bad run fails without contaminating the warehouse.
Where the budget goes
select_watchlist is where the money is saved. Score the catalog and take the top slice: events inside their on-sale window, events whose dates.status.code changed since the last run, events with an on-sale in the next 48 hours, plus a small random control sample so the dataset is not entirely selection-biased. The arithmetic on a 20,000-event, 180-day catalog:
| Work | Requests per day | Notes |
|---|---|---|
| Bisect probes (`size=1`) | ~39 | About 20 leaf slices means roughly 39 nodes in the recursion tree |
| Catalog pages (`size=200`) | 100 | 20,000 events at 200 per page |
| Discovery total | ~140 | Roughly 3% of the free 5,000-call daily quota |
| Rendered page snapshots | 200 | The scored watchlist only |
| Retries and control sample | ~20 | Blocked events and a random 5% |
The catalog half is nearly free and the rendered half is what you pay for, which is the entire reason the cycle is split. Page-scraping all 20,000 instead of a scored 1% costs 100 times more and answers the same questions, because 19,800 of those events did not move today.
Fail loudly
A parser that quietly returns nulls is worse than one that crashes, because nulls survive review and land in the warehouse looking like data.
def validate_run(rows, attempted):
coverage = len(rows) / max(1, attempted)
null_price = sum(1 for r in rows if r["low_price"] is None) / max(1, len(rows))
bases = {r["price_basis"] for r in rows}
if coverage < 0.80:
raise AssertionError(f"only {coverage:.0%} of the watchlist returned")
if null_price > 0.05:
raise AssertionError(f"price null rate {null_price:.1%}, parser likely stale")
if len(bases) > 1:
raise AssertionError(f"run mixed pricing regimes in one market: {bases}")
return {"coverage": coverage, "null_price": null_price}
The third check is the one nobody writes and the one that saves the dataset. One market in one run should produce exactly one price_basis. Two means the fee text moved and the detector is half-blind, or the run crossed a market boundary. Both are reasons to stop rather than average.
For anything longer-running, hand the fetch off asynchronously rather than holding a connection open per event:
requests.get(SPARK, headers={"X-API-Key": os.environ["SPARK_API_KEY"]}, params={
"url": f"https://www.ticketmaster.com/event/{event_id}",
"render_js": "true", "premium_proxy": "true", "country_code": "us",
"callback_url": "https://hooks.sparkproxy.io/tm-snapshot",
})
# 202 {"job_id": "...", "status": "queued"}
# Results land at /api/v1/files/<job_id> and the webhook fires on completion
Then compute the deltas, which are the only part worth having: the price move between announce and on-sale, the sell-through curve implied by availability transitions, fee ratio by venue once you hold both bases, and how far ahead of the event date prices start climbing. None of that survives a single scrape, which is why snapshot discipline matters more than the parser. Wire the cycle into a scheduled scraper and let it accumulate.
Scale politely
Ticketmaster's infrastructure carries real load during onsales, and a noisy crawler during one is both bad citizenship and a fast route to a permanent block. Limits that have held up:
- Respect the paths
robots.txtdisallows. Search and cart paths are off limits, and honoring that is a documented sign of good faith. - Cap concurrency to single digits per domain. Two to four parallel rendered requests carries a monitoring workload.
- Randomize intervals instead of firing on a clean cron tick. Fixed-period traffic is trivial to classify.
- Avoid onsale windows. Snapshot during quiet hours in the venue's local timezone: cleaner data, and no competing with real buyers for capacity.
- Back off exponentially on 429 and 530, and treat a sustained block as a reason to pause rather than rotate harder. The reasons proxies get blocked are mostly behavioral.
- Cache the stable fields. Venue, performer, and date almost never change; refetch price and availability only.
- Log every skip with its reason. Waiting-room skips are your evidence that the system stayed on the reading side of the line.
The habit underneath all of it is unglamorous: every request is a GET against a page the public can already open, and the door marked "buying" stays shut in code rather than in a policy document. That is what makes the dataset defensible a year later, when someone asks how it was collected.
Frequently asked questions
FAQ
Reading publicly accessible event listings, dates, and displayed prices for research or price monitoring is generally lawful in the US, and the Ticketmaster Discovery API exists specifically to serve that data. What is illegal under the BOTS Act is circumventing purchase limits or access controls to buy tickets, and reselling tickets obtained that way. Site terms are a separate contractual question from the statute, so read them before commercial use.
No. The priceRanges field is a static face-value band published at announce time, it is missing for a large share of events, and it does not move with demand. For current displayed pricing you need the public event page, where the schema.org Event JSON-LD carries offers.lowPrice, offers.highPrice, and offers.availability.
You cannot page past it, so you partition instead. Bisect the query by date window until each slice returns fewer than 1,000 results, then page normally inside each slice, and dedupe on event_id at the end. Adding classificationName, dmaId, or venueId as a second partition axis handles the densest markets.
For the Discovery API, no, and routing it through proxies to dodge your daily quota is quota evasion rather than scraping. For the public event pages, yes: they sit behind Akamai Bot Manager with TLS fingerprinting, so a rendered request from a residential IP is what returns real HTML. Pin the country code to the market you are studying so currency and fee disclosure stay consistent.
Stop and skip the event. A virtual waiting room is an access control on the purchase funnel, and working around it is exactly the conduct the BOTS Act prohibits. Detect the queue markers in the response and raise, rather than treating it as a challenge to solve.
You can read what a public event page displays about verified resale listings, and analyzing that over time is legitimate price research. What you cannot do is automate purchases, hold inventory, or use the data to acquire tickets past posted limits. Store observations, not carts, and keep the collection layer strictly read-only.
Get 20% off your first month
Premium datacentre proxies with unlimited bandwidth. Use the code at checkout.
Save up to 15% more on quarterly, half-yearly and yearly plans
Related articles

XPath and CSS Selectors: Scrapers That Don't Break
Scrape data with XPath and CSS selectors that survive redesigns: label-anchored XPath, stable attributes, namespace fixes, and a selector-drift detector.

Stealth Plugins for Puppeteer and Playwright: What Works
Stealth plugins patch the known automation tells in Puppeteer and Playwright. Which ones still matter, why the patch set is a signature, and how to test yours.

How to Scrape Zomato and Swiggy Data (Menus and Prices)
Scrape Zomato and Swiggy data that means something: pin the delivery coordinates, key every row by restaurant, pin and timestamp, and split the fee stack.
