How to Scrape TikTok Public Data With Proxies

Public means one thing here: content a logged-out browser can load. No session cookie from a real account, no login wall, no private profile, no follower-only content. If you have to authenticate to see it, it's out of scope for this guide and for a defensible project.

That line matters more on TikTok than on a product catalog, because almost every field describes a person or their creative work. A view count is a metric. A creator's handle, bio, and face are personal data in most of the world. The engineering below assumes your answer is aggregates and metadata, not dossiers.

Three targets cover most legitimate TikTok projects:

• Creator profiles for influencer vetting and partnership screening: handle, display name, bio, verified flag, follower and video counts.
• Video metadata for trend research and campaign measurement: caption, hashtags, sound, duration, and the engagement counters.
• Hashtag and search surfaces for share-of-voice tracking: which videos rank for a term, and how that shifts over time.

Notably absent: comments at scale, identifiers assembled across posts, and the video files themselves. Comments are the fastest route to a privacy complaint, and downloads carry a copyright problem that has nothing to do with scraping law. Skip both without a specific mandate and legal sign-off.

Check the sanctioned surfaces before writing a scraper. TikTok runs three, and each fits a narrow profile.

The Research API (open.tiktokapis.com/v2/research/) is the real deal for anyone who qualifies. It exposes video query by keyword, hashtag, region, and date, plus user info and comment lists, with a documented field set and no anti-bot fight. The catch is eligibility: qualifying universities and non-profit academic institutions in the US, EEA, UK, Switzerland, and Brazil only, commercial use explicitly excluded. Approved apps get roughly 1,000 requests per day at up to 100 records each, so about 100,000 records daily.

curl -X POST "https://open.tiktokapis.com/v2/research/video/query/?fields=id,video_description,create_time,region_code,view_count,like_count,comment_count,share_count,hashtag_names" \
  -H "Authorization: Bearer $TIKTOK_RESEARCH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
        "query": {
          "and": [
            {"operation":"IN","field_name":"hashtag_name","field_values":["proxies","webscraping"]}
          ]
        },
        "start_date": "20260701",
        "end_date":   "20260731",
        "max_count":  100
      }'

One wrinkle worth knowing before you build on it: the Research API queries an archived index rather than live state, so counters lag the live page. For "what did engagement look like in July" that's fine. For "what is this video's view count right now" it isn't.

The Display API (open.tiktokapis.com/v2/video/list/) covers accounts that authorized your app through Login Kit. Fine for a dashboard where creators connect their own account, useless for arbitrary third-party profiles.

The Commercial Content API and ad library exposes paid content in the EU under the Digital Services Act, with a public web interface plus an API for approved researchers. If your project is ad transparency or competitive paid-media research in Europe, start there.

If none of the three fit, meaning you need live public counters for arbitrary creators, the HTML layer is the only route left. Everything below covers that path.

A logged-out page load carries far more than the visible DOM suggests. Here's the reference set and where each field lives.

The engagement counters are the fields most projects care about, and they're the ones the archived Research API can't give you live. That gap is the entire reason HTML scraping exists for TikTok.

Four defenses do nearly all the damage, and they fail in ways that look like success.

Request signing on the private API. TikTok's own front end fetches a profile's video grid from /api/post/item_list/, and every call carries an msToken cookie plus signature query parameters (X-Bogus, and in current builds X-Gnarly) generated by heavily obfuscated JavaScript. Reproducing that signer outside a browser is a maintenance treadmill: TikTok rotates the algorithm, your reimplementation dies, and you find out through a silent drop in row counts. The practical answer is not to sign anything, which the pagination section covers.

The 200-status challenge. This is the one that ruins datasets. Hit a profile from a flagged IP and TikTok frequently returns HTTP 200 with a complete-looking HTML document. The hydration script tag is even present. But inside it, __DEFAULT_SCOPE__ contains only webapp.app-context and none of the data scopes. Your raise_for_status() passes, your "does the script tag exist" check passes, and you write an empty row. Validate on the scope key, not the status code and not the tag.

IP reputation. Datacenter ranges are flagged fast, faster than on most e-commerce targets. Mobile IPs do best, which makes sense for an app whose real traffic is overwhelmingly cellular.

Regional divergence. TikTok serves different content and different gates by exit country, and the US and EU stacks have drifted further apart since the US joint-venture restructuring in early 2026. Pin country_code for reproducibility rather than accepting whatever exit you land on.

That fourth row is the one people misdiagnose most often. A profile that reports 340 videos but renders an empty grid isn't blocked, it just hasn't run the signed XHR yet. The deeper pattern behind this class of endpoint is covered in How to Scrape Hidden JSON API Endpoints.

TikTok server-renders a single JSON document into every public page so the client can hydrate without a second round trip. It lives in a script tag with id="__UNIVERSAL_DATA_FOR_REHYDRATION__" and type application/json. Older tutorials reference SIGI_STATE, which TikTok retired. If a guide still says SIGI_STATE, it's stale.

The structure is a top-level __DEFAULT_SCOPE__ object whose keys name page scopes:

{
  "__DEFAULT_SCOPE__": {
    "webapp.app-context":  { "region": "US", "appId": 1988, "...": "..." },
    "webapp.user-detail":  { "userInfo": { "user": {}, "stats": {} }, "statusCode": 0 },
    "webapp.video-detail": { "itemInfo": { "itemStruct": {} }, "statusCode": 0 }
  }
}

webapp.app-context is always present, on every page, including the ones that gave you nothing. The data scopes are conditional. That asymmetry is your health check.

Here's the extraction, with the validation baked in:

import json
from selectolax.parser import HTMLParser

SCRIPT_ID = "__UNIVERSAL_DATA_FOR_REHYDRATION__"

def hydration(html: str) -> dict:
    """Return __DEFAULT_SCOPE__, or {} if the page carried no data scopes."""
    tree = HTMLParser(html)
    node = tree.css_first(f'script#{SCRIPT_ID}')
    if node is None or not node.text():
        return {}
    try:
        blob = json.loads(node.text())
    except json.JSONDecodeError:
        return {}
    return blob.get("__DEFAULT_SCOPE__", {})

def is_challenge(scope: dict) -> bool:
    """TikTok answers a soft block with HTTP 200 and app-context only."""
    if not scope:
        return True
    data_scopes = {k for k in scope if k != "webapp.app-context"}
    return len(data_scopes) == 0

Install the parser with pip install selectolax. It's C-backed and roughly an order of magnitude faster than BeautifulSoup on the default parser, which matters at tens of thousands of pages.

The important consequence: this blob is server-rendered, so it exists in the raw HTML before any JavaScript runs. That opens a much cheaper request tier, quantified below.

Proxy type decides whether a TikTok run survives past the first few hundred requests.

Datacenter proxies are the wrong tool: TikTok flags known hosting ASNs quickly, so even with aggressive rotation you burn IPs faster than you collect rows. Residential proxies route through real consumer ISP connections and carry ordinary home-user reputation, which makes them the sensible default at any meaningful volume. Mobile proxies route through cellular carrier IPs, and because carriers share a small pool across thousands of subscribers behind CGNAT, banning one hurts real users, so TikTok tolerates them far more. They cost more, so reserve them for targets that resist residential. What Is a Mobile Proxy explains the CGNAT mechanics behind that tolerance.

The SparkProxy Scraping API collapses all of this into parameters. Base endpoint is https://scrape.sparkproxy.io/api/v1, auth is a single X-API-Key header. Five parameters carry the weight on TikTok:

• premium_proxy=true: routes through the residential pool.
• country_code: ISO alpha-2 exit country, so results are reproducible.
• render_js: run headless Chromium. Defaults to true, and for TikTok you'll often want it explicitly false.
• stealth=true: extra anti-detection layers, worth it on profile grids.
• device=mobile: present as a mobile client, the profile TikTok trusts most.

A minimal request against a public video page:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.tiktok.com/@nasa/video/7231234567890123456" \
  --data-urlencode "render_js=false" \
  --data-urlencode "premium_proxy=true" \
  --data-urlencode "country_code=US"

render_js=false is deliberate. The hydration blob is already in the server response, so paying for a browser on a video page buys you nothing. The full parameter reference lives in the SparkProxy Scraping API docs. If you're weighing this against running your own rotation and browser farm, Web Scraping API vs Self-Managed Proxies works through the build-versus-buy math.

Wrap the request once so every call carries the TikTok-specific parameters, then parse the webapp.user-detail scope.

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = "YOUR_API_KEY"

def fetch(url: str, *, render: bool = False, country: str = "US",
          stealth: bool = False) -> requests.Response:
    return requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": url,
            "render_js": str(render).lower(),
            "premium_proxy": "true",     # residential exits
            "country_code": country,     # reproducible geo
            "stealth": str(stealth).lower(),
            "device": "mobile",
        },
        timeout=90,
    )

Parsing is a straight walk down the scope. TikTok nests user and stats separately, so flatten them into one row:

def parse_profile(html: str) -> dict | None:
    scope = hydration(html)
    if is_challenge(scope):
        return None                       # soft block, retry on a new IP
    detail = scope.get("webapp.user-detail") or {}
    if detail.get("statusCode") != 0:
        return {"status": detail.get("statusCode")}   # non-zero = user not found
    info  = detail.get("userInfo", {})
    user  = info.get("user", {})
    stats = info.get("stats", {})
    return {
        "handle":    user.get("uniqueId"),
        "name":      user.get("nickname"),
        "bio":       user.get("signature"),
        "verified":  user.get("verified", False),
        "private":   user.get("privateAccount", False),
        "region":    user.get("region"),
        "followers": stats.get("followerCount"),
        "following": stats.get("followingCount"),
        "likes":     stats.get("heartCount"),
        "videos":    stats.get("videoCount"),
    }

Check privateAccount before anything downstream. A private account still returns a shell with real counts, and collecting against it is exactly the boundary you said you wouldn't cross. Drop the row.

If you'd rather not carry a parser, push extraction server-side with extract_rules, a map of your field names to CSS selectors that returns JSON:

import json

resp = requests.get(API, headers={"X-API-Key": API_KEY}, params={
    "url": "https://www.tiktok.com/@nasa",
    "render_js": "true",
    "premium_proxy": "true",
    "stealth": "true",
    "extract_rules": json.dumps({
        "handle":    "h1[data-e2e='user-title']",
        "name":      "h2[data-e2e='user-subtitle']",
        "bio":       "h2[data-e2e='user-bio']",
        "followers": "strong[data-e2e='followers-count']",
        "likes":     "strong[data-e2e='likes-count']",
    }),
}, timeout=90)
print(resp.json()["extracted"])

The tradeoff is real. DOM selectors return abbreviated display strings (4.3M), while the blob returns exact integers (4312884). Use extract_rules for quick one-off pulls and the blob for anything you'll chart.

Video pages carry webapp.video-detail, and the payload is richer than the profile scope.

def parse_video(html: str) -> dict | None:
    scope = hydration(html)
    if is_challenge(scope):
        return None
    detail = scope.get("webapp.video-detail") or {}
    code = detail.get("statusCode")
    if code == 10204:
        return {"status": "unavailable"}     # deleted or region-blocked
    if code != 0:
        return {"status": code}
    item   = detail.get("itemInfo", {}).get("itemStruct", {})
    stats  = item.get("stats", {})
    author = item.get("author", {})
    return {
        "video_id":   item.get("id"),
        "caption":    item.get("desc"),
        "created_at": item.get("createTime"),          # Unix seconds
        "author":     author.get("uniqueId"),
        "duration":   item.get("video", {}).get("duration"),
        "sound":      item.get("music", {}).get("title"),
        "hashtags":   [c.get("title") for c in item.get("challenges", []) or []],
        "plays":      stats.get("playCount"),
        "likes":      stats.get("diggCount"),
        "comments":   stats.get("commentCount"),
        "shares":     stats.get("shareCount"),
        "saves":      stats.get("collectCount"),
    }

Separate statusCode: 10204 from a block. It means the video is genuinely gone or geo-restricted for your exit country, which is data, not an error, and retrying it on a new IP wastes credits forever.

Then there's the endpoint most TikTok guides never mention. TikTok runs a public oEmbed service at https://www.tiktok.com/oembed, unauthenticated, no key, no proxy required:

curl -s "https://www.tiktok.com/oembed?url=https://www.tiktok.com/@nasa/video/7231234567890123456"

{
  "version": "1.0",
  "type": "video",
  "title": "Perseverance sent back new images from Jezero Crater",
  "author_url": "https://www.tiktok.com/@nasa",
  "author_name": "NASA",
  "thumbnail_url": "https://p16-sign-va.tiktokcdn.com/....jpeg",
  "html": "<blockquote class=\"tiktok-embed\" ...>",
  "provider_name": "TikTok"
}

It won't give you engagement counters, so it can't replace the blob. What it gives you is caption, author, and thumbnail for a known video URL at zero credits and near-zero block risk. Two patterns make good use of that:

1. Cheap liveness checks. Before spending a premium request on a video you already have counters for, hit oEmbed. A non-200 means the video is gone, so skip it.
2. Backfill. When a row is missing a caption because the page came back thin, fill it from oEmbed instead of re-running an expensive request.

import requests

def oembed(video_url: str) -> dict | None:
    r = requests.get("https://www.tiktok.com/oembed",
                     params={"url": video_url}, timeout=15)
    return r.json() if r.status_code == 200 else None

Rate-limit yourself here anyway. It's an undocumented courtesy endpoint, not a contract, and hammering it is the fastest way to get it closed.

This is where most TikTok scrapers die. The profile blob gives you the user and their stats reliably, but not the video list, because the grid arrives from /api/post/item_list/ as a signed XHR. Reimplementing X-Bogus or X-Gnarly means chasing an obfuscated signer that TikTok rewrites on its own schedule.

Don't reimplement it. Let a real browser make the signed call, then read the resulting DOM. Every rendered tile is an anchor to /@handle/video/, so the IDs sit in the HTML once the grid paints.

import re

VIDEO_HREF = re.compile(r"/@[\w.\-]+/video/(\d+)")

def profile_video_ids(handle: str, country: str = "US") -> list[str]:
    resp = fetch(
        f"https://www.tiktok.com/@{handle}",
        render=True,        # browser makes the signed XHR for us
        stealth=True,       # grids are the most defended surface
        country=country,
    )
    ids = VIDEO_HREF.findall(resp.text)
    seen, ordered = set(), []
    for vid in ids:                      # de-dupe, preserve grid order
        if vid not in seen:
            seen.add(vid)
            ordered.append(vid)
    return ordered

The Scraping API's scroll parameter defaults to true, which auto-scrolls to trigger lazy loading, and wait adds up to 30 seconds of settle time after load. For deeper grids, wait_for blocks on a CSS selector, and js_scenario drives explicit click and scroll sequences. Check the docs for the scenario grammar before you build on it.

One rendered pass typically yields the first 30 to 60 tiles depending on how far the auto-scroll gets, which covers most monitoring workloads since you care about recent uploads. If you need the full back catalogue of a creator with thousands of videos, this route gets expensive, and that's the honest signal that you want the Research API instead.

Hashtag pages (/tag/) and search behave the same way: hydration gives page context, the result grid needs rendering. Search is the most defended surface on the site, so expect a lower success rate and budget retries accordingly.

Then hydrate each ID cheaply:

def scrape_creator(handle: str, limit: int = 30) -> list[dict]:
    rows = []
    for vid in profile_video_ids(handle)[:limit]:
        url  = f"https://www.tiktok.com/@{handle}/video/{vid}"
        resp = fetch(url, render=False)          # blob is server-rendered
        row  = parse_video(resp.text)
        if row:
            rows.append(row | {"url": url})
    return rows

One expensive rendered request per creator, then N cheap ones per video. That asymmetry is the whole architecture, and the next section prices it.

SparkProxy prices requests by what they actually consume, and on TikTok the spread between tiers is 25x:

The instinct is to reach for the top tier because TikTok is hard. That instinct costs money, because raw credit cost is the wrong number to optimize. The number that matters is credits per successful page:

effective cost = base credits / success rate

Work an example. Premium plus JS plus stealth plus geo comes to 35 credits (25 + 5 + 5). Say it succeeds on 95% of video pages, giving 36.8 credits per good page. Now the cheap tier: premium, no JS, plus geo is 15 credits (10 + 5). Because the hydration blob is server-rendered it doesn't need a browser at all, so its failure mode is purely IP reputation. Even at a mediocre 55% success rate it lands at 27.3 credits per good page, still cheaper. The break-even sits near 41% success. Below that, render. Above it, don't.

Run that measurement yourself rather than trusting anyone's numbers, because success rates move with your exit countries and your target mix. A useful shape for the loop:

from collections import Counter

def measure(urls: list[str], **kw) -> float:
    c = Counter()
    for u in urls:
        ok = parse_video(fetch(u, **kw).text) is not None
        c["ok" if ok else "fail"] += 1
    return c["ok"] / max(1, sum(c.values()))

sample = urls[:200]
cheap = measure(sample, render=False)                 # 15 credits/request
rich  = measure(sample, render=True, stealth=True)    # 35 credits/request
print(f"cheap {15/max(cheap,0.01):.1f} vs rich {35/max(rich,0.01):.1f} credits per good page")

The practical policy that falls out of this: cheap first, expensive on failure.

def fetch_video_adaptive(url: str) -> dict | None:
    row = parse_video(fetch(url, render=False).text)                # 15 credits
    if row is not None:
        return row
    return parse_video(fetch(url, render=True, stealth=True).text)  # 35 credits

If the cheap tier clears 55%, this pattern costs about 31 credits per page against 36.8 for always-render, and it degrades gracefully when TikTok tightens. Grids stay on the expensive tier, because there the browser is doing work you genuinely cannot replicate.

TikTok applies pressure per IP, per fingerprint, and per surface. Profile and video pages tolerate far more than search. Four habits keep a run healthy.

Rotate exits per request. With premium_proxy=true you get a fresh residential IP each call, so a single flagged exit costs you one request instead of a batch.

Keep concurrency modest. Four to ten workers is the sane band. TikTok punishes bursts harder than sustained volume, and your own concurrency ceiling returns 429 before TikTok ever sees the pressure.

Back off with jitter, and cap retries. Without jitter, a batch of simultaneous failures retries in lockstep and reproduces the burst that caused them.

import time, random

def with_retry(fn, *args, attempts: int = 3, **kw):
    for i in range(attempts):
        try:
            out = fn(*args, **kw)
            if out is not None:
                return out
        except requests.RequestException:
            pass
        time.sleep(2 ** i + random.random())   # 1s, 2s, 4s plus jitter
    return None

Cache like the data is expensive, because it is. A creator's follower count doesn't move meaningfully in an hour. Set a TTL per field class: profile stats daily, video counters every 6 to 12 hours for active content and weekly for archives. Caching policy alone cuts most teams' TikTok credit burn substantially, before touching a single request parameter.

Stamp every row with a scraped_at timestamp. Engagement counters only mean anything as a time series, and a row without a collection time is close to worthless six months later. The general ban-avoidance playbook is in How to Avoid Getting Your Proxy Blocked.

Four separate questions, and "it's public" only answers one of them.

Unauthorized access. In the US, the Ninth Circuit's decision in hiQ Labs v. LinkedIn (2022) held that scraping data available without authentication generally doesn't violate the Computer Fraud and Abuse Act. That's about access, not a license covering what you do next.

Contract. TikTok's Terms of Service prohibit automated collection, so scraping public pages can breach that contract even where no computer-crime statute is implicated. Two questions, two exposures.

Data protection. Under GDPR, a handle, bio, photo, and posting history are personal data, and Article 4 defines it broadly. Public availability is not an exemption. In August 2023, twelve data-protection authorities led by the UK's ICO issued a joint statement saying exactly that. If any data subjects sit in the EU or UK, you need a lawful basis under Article 6, usually legitimate interest backed by a documented balancing test, plus a transparency plan. Keep clear of Article 9 special categories, which TikTok content surfaces constantly.

Biometrics. Do not run face recognition on scraped TikTok frames. France's CNIL fined Clearview AI 20 million euros in 2022 for scraping public photos into a facial-recognition database, and other EU regulators issued comparable penalties. That case is the boundary marker for anyone who thinks public images are unrestricted.

Practices that keep a TikTok project defensible: public data only, aggregates over individual dossiers, no comments without a mandate, no biometric derivatives, honor deletion by dropping content the creator removed, a DPIA before a large collection, and rate limits that never degrade the service. The business-side patterns are in Using Proxies for Social Media Monitoring, and the parallel platform walkthrough is How to Scrape Instagram Public Data. This is engineering guidance, not legal advice. Get a lawyer before anything commercial.