How to Scrape TripAdvisor Reviews and Ratings

Sort the framing out before you write a line of code, because "the reviews are public" only answers half the question.

In the United States, the Ninth Circuit's ruling in hiQ Labs v. LinkedIn (9th Cir. 2022) held that scraping data that is publicly accessible, meaning no login and no authentication wall, generally does not violate the Computer Fraud and Abuse Act. That decision is about unauthorized access, not a blanket license to do anything with the data. TripAdvisor's Terms of Use separately prohibit automated collection, so scraping can still be a breach of contract even where it is not a CFAA violation. Those are two different legal questions, and only one of them was settled by hiQ.

There is a second layer here that price or product scraping does not carry. A review contains a person's name, hometown, and posting history. That makes review data personal data under the EU's GDPR and similar laws, regardless of the fact that TripAdvisor publishes it. We handle the practical side in the GDPR section below, but keep it in mind from the first request.

Guardrails that keep a project defensible:

• Collect public review content only: the rating, title, text, dates, trip type, and any management response. Treat reviewer identity as sensitive.
• Rate-limit yourself and back off on errors so you are not degrading TripAdvisor's service for real users.
• Do not republish full review text or reviewer identities beyond what fair use and privacy law allow.
• Prefer the official TripAdvisor Content API when you need reviews for a property you represent. It is the compliant route and returns clean JSON with no anti-bot fight, though it caps you at a handful of the most recent reviews per location.
• If the data feeds a commercial product, run it past a lawyer. This is engineering guidance, not legal advice.

Brand monitoring, competitor sentiment tracking, and travel market research on public reviews are common, legitimate uses. The infrastructure and business context for that work sit in Using Proxies for Review Monitoring and Sentiment Analysis.

A public TripAdvisor listing page exposes a consistent set of fields per review, even though the class names wrapping them change constantly. The trick is to anchor on things that do not churn: stable HTML attributes (data-reviewid), accessibility labels (aria-label), the embedded JSON-LD, and the visible field labels the page prints for humans ("Trip type:", "Date of stay:"). Here is the reference set worth pulling and where each one actually lives.

The review ID is the anchor. Store it as your primary key so re-running a scrape updates existing rows instead of duplicating them. The two fields you will not find on Trustpilot or Yelp are trip type and date of stay, and they are the most valuable part of TripAdvisor data for segmentation. More on that in the sentiment section.

TripAdvisor is one of the tougher public review targets. Four traits break naive scrapers.

Aggressive bot management. A plain requests.get from a datacenter IP gets a 403 or an "Access to this page has been denied" interstitial, sometimes a CAPTCHA, before you ever see a review. TripAdvisor fingerprints the request well beyond the User-Agent string.

Rotating CSS class hashes. The review markup uses short, obfuscated class names that rehash on redesigns. A selector like .biGQs._P.pZUbB works today and returns nothing next month. This is the single biggest reason scraped TripAdvisor pipelines quietly rot.

Offset pagination in the URL path. TripAdvisor does not use ?page=2. It injects an offset token, -or10-, into the path between the Reviews segment and the listing name. Miss this and you re-scrape page one on every iteration.

Truncated review bodies. Long reviews render with a "Read more" cutoff. A scraper that grabs the visible text captures half a review ending in an ellipsis.

A managed scraping API absorbs the first row for you. The other three are extraction discipline, and the rest of this guide is about getting them right. For the proxy-side theory behind clearing bot walls, How to Avoid Getting Your Proxy Blocked covers TLS fingerprinting and header consistency in depth.

The SparkProxy Scraping API takes a target URL and handles the proxy, rotation, and anti-bot layer for you. You send one request, you get the HTML back. TripAdvisor needs a heavier configuration than a lightweight SSR site, because the bot management is the real obstacle.

Two parameters carry the weight, with a third in reserve:

• premium_proxy=true: routes through residential IPs, which clear TripAdvisor's bot management where datacenter IPs get denied.
• render_js=true: run a real headless browser. TripAdvisor's review content is React-rendered and the anti-bot layer checks for browser behavior, so rendering is the safer default here (unlike a purely server-rendered target).
• stealth=true: add advanced anti-detection layers. Turn it on only if premium_proxy plus render_js still draws a challenge, since it costs more.

TripAdvisor also localizes content by country, and you can pin the locale with country_code. Set the review language in the URL itself with filterLang=en. The base endpoint is https://scrape.sparkproxy.io/api/v1 and auth is a single header, X-API-Key. A minimal request in cURL:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.tripadvisor.com/Hotel_Review-g60763-d93589-Reviews-The_New_Yorker_Hotel-New_York_City_New_York.html" \
  --data-urlencode "render_js=true" \
  --data-urlencode "premium_proxy=true"

A rendered request on the residential pool is the top credit tier, so scrape deliberately rather than crawling every page you can find. The full parameter list and current credit costs live in the Scraping API docs. If you are weighing this against running your own residential pool, Web Scraping API vs Self-Managed Proxies lays out the trade-off.

Wrap the call so every request carries the TripAdvisor-specific parameters:

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = "YOUR_API_KEY"

def fetch_page(url: str) -> str:
    """Fetch one TripAdvisor listing page through the Scraping API.
    render_js clears the React render + behavior checks; premium_proxy
    gives a residential exit IP that TripAdvisor's bot wall accepts."""
    resp = requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": url,
            "render_js": "true",
            "premium_proxy": "true",
        },
        timeout=90,
    )
    resp.raise_for_status()
    return resp.text

TripAdvisor does not show stars. It shows five bubbles, and the score comes in half-bubble steps from 1.0 to 5.0. There are two ways it encodes that value in the HTML, and both survive a class rehash better than a class selector does.

The newer React UI puts the score in an accessibility label, for example aria-label="4.5 of 5 bubbles". The legacy widget uses a class like ui_bubble_rating bubble_45, where the trailing number is the rating times ten (so bubble_45 is 4.5). Read the aria-label first, fall back to the class number:

import re
from selectolax.parser import HTMLParser

def parse_rating(card) -> float | None:
    # Newer UI: aria-label reads e.g. "4.5 of 5 bubbles"
    node = card.css_first('[aria-label*="of 5 bubbles"]')
    if node:
        m = re.match(r"([0-5](?:\.\d)?)", node.attributes.get("aria-label", ""))
        if m:
            return float(m.group(1))
    # Legacy widget: class "ui_bubble_rating bubble_45" -> 4.5
    node = card.css_first('[class*="bubble_"]')
    if node:
        m = re.search(r"bubble_(\d{2})", node.attributes.get("class", ""))
        if m:
            return int(m.group(1)) / 10
    return None

For the property-level average, do not compute it from the scraped page. TripAdvisor embeds a schema.org JSON-LD block for Google's rich results, and it carries the aggregateRating (the average and the total review count) reliably, plus often a few of the most recent review objects. Read it instead of averaging a partial scrape:

import json

def extract_jsonld(html: str) -> list[dict]:
    """Return every schema.org JSON-LD block on the page."""
    blocks = []
    for node in HTMLParser(html).css('script[type="application/ld+json"]'):
        try:
            blocks.append(json.loads(node.text()))
        except json.JSONDecodeError:
            continue
    return blocks

def aggregate_rating(blocks: list[dict]) -> dict | None:
    for b in blocks:
        agg = b.get("aggregateRating") if isinstance(b, dict) else None
        if agg:
            return {"average": agg.get("ratingValue"), "count": agg.get("reviewCount")}
    return None

Now you have the headline number without trusting a class name, and a resilient way to read each review's own bubble score.

Each review sits in a container that carries a data-reviewid attribute, which is the one anchor TripAdvisor keeps stable across redesigns. Iterate those containers, then pull each field from an anchor that does not churn: the aria-label for the rating, the profile link for the reviewer, and the visible labels for trip type and date of stay.

The label trick is the part most guides miss. TripAdvisor prints "Trip type: Business" and "Date of stay: March 2026" as literal text for human readers. You can match that label string instead of any CSS class, and it keeps working through every frontend rewrite:

def field_after_label(text: str, label: str) -> str | None:
    """Grab the value TripAdvisor prints after a visible label."""
    m = re.search(rf"{re.escape(label)}\s*:?\s*([^\n]+)", text)
    return m.group(1).strip() if m else None

def review_body(card) -> str | None:
    """Class names rotate, so take the longest text run in the card
    as the review body. It is reliably the longest thing there."""
    runs = [n.text(strip=True) for n in card.css("span")]
    runs = [r for r in runs if len(r) > 40]
    return max(runs, key=len) if runs else None

def parse_reviews(html: str) -> list[dict]:
    tree = HTMLParser(html)
    rows = []
    for card in tree.css("div[data-reviewid]"):
        block = card.text(separator="\n", strip=True)
        profile = card.css_first('a[href*="/Profile/"]')
        rows.append({
            "review_id": card.attributes.get("data-reviewid"),   # stable primary key
            "rating": parse_rating(card),                        # from aria bubbles
            "text": review_body(card),
            "trip_type": field_after_label(block, "Trip type"),  # Business/Family/...
            "date_of_stay": field_after_label(block, "Date of stay"),
            "reviewer_name": profile.text(strip=True) if profile else None,  # personal data
        })
    return rows

Two things worth knowing. The data-reviewid gives you a clean dedupe key for free, so you never store the same review twice across overlapping filter runs. And the label-based extraction for trip_type and date_of_stay is deliberate insurance: those two fields have no stable class or attribute of their own, so matching the printed label is the only approach that does not break on the next redesign. If you would rather have the API do extraction server-side, the extract_rules parameter maps fields to selectors and returns JSON, but on TripAdvisor the same class-churn problem applies to any selector you feed it, so the attribute-and-label approach above is sturdier.

This is where generic scrapers fail on TripAdvisor. The offset lives in the URL path, not a query parameter. A hotel review URL looks like this:

https://www.tripadvisor.com/Hotel_Review-g60763-d93589-Reviews-The_New_Yorker_Hotel-New_York_City_New_York.html

The g60763 is the geographic ID and d93589 is the listing ID. To get the second page, TripAdvisor inserts -or10- right after the Reviews segment:

https://www.tripadvisor.com/Hotel_Review-g60763-d93589-Reviews-or10-The_New_Yorker_Hotel-...html

The offset increments by the page size, and the page size depends on the listing type. Match them or you skip or repeat reviews.

Build the paginated URL by inserting the token, then loop until a page adds nothing new. TripAdvisor clamps an out-of-range offset and re-serves the last page, so a dedupe-and-stop check is what ends the loop cleanly:

import time
import random

def review_page_url(base_url: str, offset: int) -> str:
    """Insert the -or{offset}- token TripAdvisor uses for pagination."""
    if offset == 0:
        return base_url
    return re.sub(r"-Reviews-", f"-Reviews-or{offset}-", base_url, count=1)

def is_blocked(html: str) -> bool:
    markers = (
        "access to this page has been denied",
        "captcha-delivery",
        "px-captcha",
        "please verify you are a human",
    )
    low = html.lower()
    return len(html) < 1500 or any(m in low for m in markers)

def scrape_all_reviews(base_url: str, page_size: int = 10, max_pages: int = 20) -> list[dict]:
    seen, out = set(), []
    for i in range(max_pages):
        html = fetch_page(review_page_url(base_url, i * page_size))
        if is_blocked(html):
            time.sleep(5)
            continue
        fresh = [r for r in parse_reviews(html) if r["review_id"] not in seen]
        if not fresh:                       # last page reached, or offset clamped
            break
        seen.update(r["review_id"] for r in fresh)
        out.extend(fresh)
        time.sleep(random.uniform(3, 6))    # pace yourself between pages
    return out

To pull more than the first stretch of pages on a busy listing, narrow with TripAdvisor's own URL filters (a rating filter, or filterLang=en for one language), run each filtered set to its end, and dedupe on review_id afterward since the sets overlap.

Long reviews render truncated with a "Read more" link, and the visible text ends in an ellipsis. Sometimes the full body is already in the server HTML and only hidden by CSS, in which case your review_body heuristic picks it up and you are done. Check that first before adding complexity.

When the full text really is loaded on click, use a JavaScript scenario to expand every review before the page is captured. The Scraping API runs the interaction inside the same rendered session:

import json

def fetch_page_expanded(url: str) -> str:
    resp = requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": url,
            "render_js": "true",
            "premium_proxy": "true",
            "js_scenario": json.dumps({"steps": [
                {"wait_for": "div[data-reviewid]"},        # wait for reviews to mount
                {"click": "span:has-text('Read more')"},   # expand truncated bodies
                {"wait": 800},
            ]}),
        },
        timeout=120,
    )
    resp.raise_for_status()
    return resp.text

Check the Scraping API docs for the exact js_scenario action keys, since the shape can change. The pattern is what matters: wait for the review cards, click each Read more, wait a beat for the DOM to settle, then capture. Swap fetch_page for fetch_page_expanded in the pagination loop only when you confirm the bodies come back truncated, because rendering plus interaction is slower and costs more credits per page.

At volume, three habits keep the pipeline healthy: retries on soft blocks, exponential backoff so you do not spike the bot wall, and modest concurrency across different listings rather than hammering one property's pages. Because the Scraping API rotates the exit IP for you, your ceiling is your plan's rate limit, not the number of proxies you own.

from concurrent.futures import ThreadPoolExecutor, as_completed

def scrape_listing(base_url: str, attempts: int = 3) -> list[dict]:
    for i in range(attempts):
        try:
            rows = scrape_all_reviews(base_url)
            if rows:
                return rows
        except Exception:
            pass
        time.sleep(2 ** i + random.random())   # exponential backoff + jitter
    return []

def scrape_many(urls: list[str], workers: int = 4) -> dict[str, list[dict]]:
    out = {}
    with ThreadPoolExecutor(max_workers=workers) as pool:
        futs = {pool.submit(scrape_listing, u): u for u in urls}
        for fut in as_completed(futs):
            out[futs[fut]] = fut.result()
    return out

Keep worker counts sane. Four concurrent listings is plenty for review monitoring, and the jitter matters because it staggers retries so a batch of failures does not retry in lockstep and re-trigger the same block. The general patterns behind high-volume collection, connection reuse, and error handling are covered in Using Datacenter Proxies for Web Scraping.

Raw reviews are not analysis-ready. Load them into a dataframe, drop rating-only rows with no text to score, then run a first-pass sentiment model.

import pandas as pd
from nltk.sentiment import SentimentIntensityAnalyzer
# one-time: python -m nltk.downloader vader_lexicon

df = pd.DataFrame(scrape_all_reviews(BASE_URL))
df["text"] = df["text"].fillna("").str.strip()
df = df[df["text"] != ""]                      # drop rating-only reviews

sia = SentimentIntensityAnalyzer()
df["sentiment"] = df["text"].apply(lambda t: sia.polarity_scores(t)["compound"])

VADER is a lexicon model tuned for short, informal text, so it scores each review with no GPU or training data. For heavier work, swap it for a transformer such as a fine-tuned DistilBERT on the same text column.

Here is the analysis angle TripAdvisor gives you that Trustpilot and Yelp do not: the trip type field. Segment sentiment by who was traveling, and the gaps tell you which guest a property underserves.

by_trip = df.groupby("trip_type")["sentiment"].mean().sort_values()

If business travelers score a hotel a full point below families, the problem is usually workspace, wifi, or check-in speed, not the pool. That segmentation is impossible on review sites that do not capture the travel context, and it is the single most actionable thing in TripAdvisor data. Pair it with the star-versus-text gap: a 5.0 review with negative text usually flags one specific incident worth reading. The end-to-end pipeline design, from collection cadence to aggregation, is covered in Using Proxies for Review Monitoring and Sentiment Analysis.

This is the part that separates review scraping from product scraping. A reviewer's name, hometown, and posting history are personal data under the GDPR, and the fact that TripAdvisor publishes them does not change that. If any reviewer is an EU resident, processing their data brings obligations even when your scraper only touches public pages.

Handling that keeps a review dataset defensible:

• Data minimization first. Most sentiment and reputation work needs the rating, text, trip type, and dates, not the person's name. If you do not need identity, do not store it.
• Pseudonymize when you only need to dedupe. If you need to count distinct reviewers or catch repeat posters but not name them, hash the name and drop the raw value:

import hashlib

def pseudonymize(name: str | None) -> str | None:
    if not name:
        return None
    return hashlib.sha256(name.encode("utf-8")).hexdigest()[:16]

df["reviewer_id"] = df["reviewer_name"].apply(pseudonymize)
df = df.drop(columns=["reviewer_name"])    # keep the hash, drop the identity

• Have a lawful basis. For competitive or market research, "legitimate interests" (GDPR Article 6(1)(f)) is the usual basis, and it requires a documented balancing test weighing your interest against the reviewer's privacy.
• Respect data subject rights. Be able to delete a person's records on request, which is another reason to key on a pseudonymous review_id rather than scattering names across tables.
• Do not republish identities. Aggregate sentiment, rating trends, and anonymized quotes are far safer to surface than a searchable copy of named reviews.

None of this blocks legitimate analysis. It shapes what you keep. A pipeline that stores ratings, cleaned text, trip type, dates, and a pseudonymous reviewer key answers almost every business question while holding almost no personal data at rest. When you need reviews tied to real identities for a property you represent, the TripAdvisor Content API is the route built for it.