How to Scrape G2 Reviews with Proxies

Settle the framing before you write a line of code, because "the reviews are public" answers only half of the question.

In the United States, the Ninth Circuit's ruling in hiQ Labs v. LinkedIn (2022) held that scraping data that is publicly accessible, meaning no login and no authentication wall, generally does not violate the Computer Fraud and Abuse Act. That decision is about unauthorized access, not a license to take anything. G2's Terms of Use separately prohibit automated collection, so scraping can be a breach of contract even where it is not a CFAA violation. Those are two different legal questions, and you can lose the second while winning the first.

There is a second layer that price or product scraping does not carry: a G2 review names the reviewer, their job title, their employer's size and industry, and sometimes their region. That makes review data personal data under the EU's GDPR and personal information under California's CCPA, regardless of the fact that G2 published it. The privacy section covers the practical handling, but keep it in mind from the first request.

Guardrails that keep a G2 project defensible:

• Collect public review content only: ratings, titles, the structured pros and cons, dates, and firmographic segments. Treat reviewer names as sensitive.
• Rate-limit and back off on errors so you are not degrading G2's service.
• Do not republish full review text or reviewer identities beyond what fair use and privacy law allow.
• For your own product's reviews, use G2's licensed review syndication and data programs. That is the compliant route and it hands you clean data with no anti-bot fight.
• If the data feeds a commercial product, run it past a lawyer. This is engineering guidance, not legal advice.

Competitive intelligence, category benchmarking, and market research on public software reviews are common, legitimate uses. The pipeline design for ongoing collection sits in Using Proxies for Review Monitoring and Sentiment Analysis, and the sibling walkthrough for a Next.js target is How to Scrape Trustpilot Reviews.

A public G2 reviews page lives at https://www.g2.com/products//reviews, for example /products/salesforce-sales-cloud/reviews. G2 renders the page server-side with a Rails stack and marks the reviews up with schema.org data, which is the part that makes this tractable. Two sources matter: a top-level JSON-LD Product block that carries the aggregate score, and per-review microdata (itemprop attributes) on each review card. Here is the reference set worth pulling.

Two fields carry more signal than most scrapers realize. The structured pros and cons are the reason G2 reviews beat a generic star site for product research: instead of one paragraph, you get the reviewer's specific likes, dislikes, and use case as separate answers, which feeds feature-gap analysis directly. And the review source label matters, because incentivized reviews (the reviewer got a gift card) skew positive. Store that flag so you can filter to organic reviews when you need clean sentiment. The exact selectors and labels are current as of mid-2026; confirm them against a live page before a large run, because G2 redesigns often.

G2 is near the top of the difficulty scale for a public review site. Four traits break naive scrapers.

Cloudflare-grade bot management. G2 fingerprints the TLS handshake and browser environment and serves a "Just a moment..." interstitial, a Turnstile-style challenge, or a flat 403 before a single review loads. A plain requests.get from a datacenter IP is flagged almost immediately. This is the wall the whole rest of the guide is built to clear.

JavaScript challenges. The challenge page runs JavaScript that must execute to mint the clearance cookie. That means an HTML-only fetch is not enough on its own; you need a real browser to run the challenge, which is why render_js is on for G2 even though the reviews themselves are server-rendered.

Utility class churn. G2's frontend uses its "Elevate" design system, so review cards carry hashed, prefixed utility classes like elv-bg-neutral-0 that shift between redesigns. Any scraper keyed on those classes returns empty fields after a deploy. The fix is to read the schema.org markup instead, which is far more stable.

Deep-page friction. Request review pages too fast and you draw soft blocks and 429s. Page deep enough and results thin out or start to repeat, so pulling a busy product's full history takes filter-splitting rather than one long crawl.

A managed scraping API absorbs the first two lines for you. The class-churn line is solved by reading the markup, and the deep-page line is a planning problem you handle with filters. For the proxy-side theory behind clearing challenge pages, How to Bypass Cloudflare in Web Scraping goes deep on TLS fingerprinting, the clearance cookie, and header consistency.

The SparkProxy Scraping API takes a target URL and handles the proxy, rotation, browser, and anti-bot layer for you. You send one request, you get the rendered HTML back. For G2, the parameter choice is heavier than for a soft target, and that is deliberate: this is a 9-out-of-10 anti-bot site, so you arm the request fully.

Three parameters carry the weight:

• premium_proxy=true: routes through residential IPs, which clear G2's Cloudflare where datacenter IPs get a 403.
• render_js=true: runs a real Chromium so the challenge JavaScript executes and the reviews render.
• stealth=true: adds extra stealth layers (a homepage pre-warm and a forced Google referrer) for the cases where the residential route alone still draws a challenge.

That combination costs credits. A premium residential request with JS rendering is 25 credits, and stealth adds 5, so a fully-armored G2 request runs about 30 credits. That is the price of the hardest tier of target, and you earn it back by not maintaining a rotating browser farm. If you are weighing this against running your own residential pool and headless fleet, Web Scraping API vs Self-Managed Proxies lays out the trade-off in detail.

The base endpoint is https://scrape.sparkproxy.io/api/v1 and auth is a single header, X-API-Key. A minimal request in cURL:

curl -G "https://scrape.sparkproxy.io/api/v1" \
  -H "X-API-Key: YOUR_API_KEY" \
  --data-urlencode "url=https://www.g2.com/products/salesforce-sales-cloud/reviews" \
  --data-urlencode "render_js=true" \
  --data-urlencode "premium_proxy=true" \
  --data-urlencode "stealth=true"

The full parameter list and response fields live in the Scraping API docs.

Wrap the call so every request carries the G2-specific parameters and a page number. Adding wait_for a review element makes the API hold until the reviews actually render, so you do not parse a half-loaded challenge page.

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = "YOUR_API_KEY"

def fetch_reviews_page(slug: str, page: int = 1) -> str:
    """Fetch one G2 reviews page through the armored path.
    G2 is a hard anti-bot target, so we run residential + JS + stealth."""
    resp = requests.get(
        API,
        headers={"X-API-Key": API_KEY},
        params={
            "url": f"https://www.g2.com/products/{slug}/reviews?page={page}",
            "render_js": "true",        # challenge JS must execute
            "premium_proxy": "true",    # residential IPs clear G2's Cloudflare
            "stealth": "true",          # homepage pre-warm + Google referrer
            "wait_for": '[itemprop="review"]',  # hold until reviews render
        },
        timeout=90,
    )
    resp.raise_for_status()
    return resp.text

Detect a block before you try to parse it. A cleared page contains review markup; a challenge page contains one of a few known markers.

def is_blocked(html: str) -> bool:
    markers = (
        "Just a moment...",              # Cloudflare interstitial
        "cf-challenge",
        "Attention Required!",
        "Please verify you are a human",
        "challenge-platform",
    )
    return any(m in html for m in markers) or 'itemprop="review"' not in html

If is_blocked fires, the fix is a retry on a fresh IP rather than a tweak to your parser. The rotation happens on the API side, so a plain retry lands on a different exit.

Before parsing individual reviews, grab the aggregate. G2 embeds a JSON-LD Product block that carries the mean rating and the total review count, which is exactly what you want for tracking a competitor's score over time. Reading it is one fetch, no pagination.

import json
from selectolax.parser import HTMLParser  # pip install selectolax

def extract_product_ld(html: str) -> dict:
    """Find the schema.org Product JSON-LD block and return it."""
    tree = HTMLParser(html)
    for node in tree.css('script[type="application/ld+json"]'):
        try:
            data = json.loads(node.text())
        except json.JSONDecodeError:
            continue
        # G2 sometimes wraps blocks in a @graph array
        candidates = data.get("@graph", [data]) if isinstance(data, dict) else data
        for obj in candidates:
            if isinstance(obj, dict) and obj.get("@type") == "Product":
                return obj
    return {}

def aggregate_rating(html: str) -> dict:
    product = extract_product_ld(html)
    agg = product.get("aggregateRating") or {}
    return {
        "product": product.get("name"),
        "rating": agg.get("ratingValue"),      # e.g. "4.4"
        "review_count": agg.get("reviewCount"),
    }

For a lightweight rating tracker, this is the whole job: fetch page one of each competitor weekly, read aggregate_rating, and store the number. You never touch pagination, so the credit cost stays flat no matter how many reviews a product has. The JSON-LD may also carry a small review sample; the full per-page set comes from the review cards, which is the next section.

This is where G2 pays off over a generic star site. Each review card is marked up with itemprop attributes for the stable fields, and the body holds the three-question structure that makes G2 valuable. Map each card to a flat record you control, and keep the pros, cons, and use case as separate fields.

from selectolax.parser import HTMLParser

def _text(node, selector: str) -> str | None:
    hit = node.css_first(selector)
    return hit.text(strip=True) if hit else None

def _attr(node, selector: str, name: str) -> str | None:
    hit = node.css_first(selector)
    return hit.attributes.get(name) if hit else None

def _reviewer_meta(card, key: str) -> str | None:
    """Read a firmographic value from the reviewer meta block by its label.
    Confirm the meta selectors on a live page; G2 renames these often."""
    labels = {"role": "role", "company_size": "business", "industry": "industry"}
    meta = card.css_first('[data-testid="reviewer-info"]') or card
    for row in meta.css("div"):
        text = row.text(strip=True).lower()
        if labels[key] in text:
            return row.text(strip=True)
    return None

def parse_reviews(html: str) -> list[dict]:
    """Parse each G2 review card via schema.org microdata.
    Selectors are current-as-of; verify against a live page."""
    tree = HTMLParser(html)
    out = []
    for card in tree.css('[itemprop="review"]'):
        body = _split_questionnaire(card)
        out.append({
            "rating": _attr(card, '[itemprop="ratingValue"]', "content"),
            "title": _text(card, '[itemprop="name"]'),
            "author": _text(card, '[itemprop="author"]'),   # personal data
            "date": _attr(card, '[itemprop="datePublished"]', "datetime"),
            "likes": body.get("like"),        # "What do you like best?"
            "dislikes": body.get("dislike"),  # "What do you dislike?"
            "use_case": body.get("problem"),  # "What problems is it solving?"
            "role": _reviewer_meta(card, "role"),
            "company_size": _reviewer_meta(card, "company_size"),
            "industry": _reviewer_meta(card, "industry"),
            "incentivized": "Incentivized" in card.text(),
            "validated": "Validated Reviewer" in card.text(),
        })
    return out

The one non-obvious helper is splitting the review body by its question labels. G2 renders each answer under a heading like "What do you like best?", so match on those labels rather than assuming a fixed element order.

def _split_questionnaire(card) -> dict:
    """Split the review body into like / dislike / problem by question label."""
    labels = {
        "like": "what do you like best",
        "dislike": "what do you dislike",
        "problem": "what problems is",
    }
    text = card.text(separator="\n")
    lines = [l.strip() for l in text.split("\n") if l.strip()]
    result, current = {}, None
    for line in lines:
        low = line.lower()
        matched = next((k for k, lbl in labels.items() if lbl in low), None)
        if matched:
            current = matched
            result[current] = ""
        elif current:
            result[current] += (" " + line)
    return {k: v.strip() for k, v in result.items()}

If you would rather have the API extract fields server-side, the extract_rules parameter maps names to CSS selectors and returns JSON, which works well for the stable itemprop fields. For the three-question body, the label-splitting logic above is more reliable than a single selector, because the answers share one container.

Here is the analysis payoff most G2 guides miss. Because every review carries a company-size segment, you can measure how sentiment splits by buyer size, which Trustpilot and Google reviews cannot tell you. Enterprise and Small-Business reviewers often rate the same product very differently, and that gap is a competitive-intelligence signal.

import pandas as pd

df = pd.DataFrame(parse_reviews(fetch_reviews_page("salesforce-sales-cloud")))
df["rating"] = pd.to_numeric(df["rating"], errors="coerce")
organic = df[~df["incentivized"]]                       # drop gift-card reviews
by_segment = organic.groupby("company_size")["rating"].agg(["mean", "count"])
print(by_segment.sort_values("mean"))

Dropping incentivized reviews first keeps the segment averages honest, and grouping by company_size surfaces where a product wins or loses by buyer tier. The wider pipeline for turning this into an ongoing sentiment feed is in Using Proxies for Review Monitoring and Sentiment Analysis.

G2 paginates review pages with ?page=N and loads a batch per page (confirm the exact count on a live page; it has recently been around 25). Loop until a page returns no reviews, pace yourself, and treat a block as a retry rather than a crash.

import time
import random

def scrape_all_reviews(slug: str, max_pages: int = 20) -> list[dict]:
    all_reviews, seen = [], set()
    for page in range(1, max_pages + 1):
        html = fetch_reviews_page(slug, page)
        if is_blocked(html):
            time.sleep(5)
            continue
        reviews = parse_reviews(html)
        if not reviews:                      # last page reached
            break
        fresh = [r for r in reviews if (r["author"], r["date"], r["title"]) not in seen]
        for r in fresh:
            seen.add((r["author"], r["date"], r["title"]))
        all_reviews.extend(fresh)
        time.sleep(random.uniform(3, 6))     # pace to avoid soft blocks
    return all_reviews

At volume, three habits keep the pipeline healthy: retries on soft blocks, exponential backoff with jitter so a batch of failures does not retry in lockstep, and modest concurrency across different products rather than hammering one product's pages. Because the API rotates the exit IP for you, your ceiling is your plan's rate limit, not the number of proxies you own.

from concurrent.futures import ThreadPoolExecutor, as_completed

def scrape_product(slug: str, attempts: int = 3) -> list[dict]:
    for i in range(attempts):
        try:
            reviews = scrape_all_reviews(slug)
            if reviews:
                return reviews
        except Exception:
            pass
        time.sleep(2 ** i + random.random())   # backoff + jitter
    return []

def scrape_many(slugs: list[str], workers: int = 4) -> dict[str, list[dict]]:
    out = {}
    with ThreadPoolExecutor(max_workers=workers) as pool:
        futures = {pool.submit(scrape_product, s): s for s in slugs}
        for fut in as_completed(futures):
            out[futures[fut]] = fut.result()
    return out

Keep worker counts low; four concurrent products is plenty for competitive tracking, and each is running the expensive rendered path. To reach more of a busy product's history than a straight ?page=N crawl surfaces, split the query with G2's filter facets (star rating, company size, industry, region), run each filtered set to its own page cap, and dedupe. The broader patterns for high-volume collection and error handling are in How to Avoid Getting Your Proxy Blocked.

Reviews are one of three data shapes on G2, and the other two answer different questions. Ratings pages give you the aggregate score and criteria breakdown. Comparison pages put two products head to head. Both render the same way, so you fetch them through the same armored path.

A comparison page lives at https://www.g2.com/compare/-vs- and carries side-by-side star ratings, per-criterion scores, and reviewer segment splits. Pull the two aggregate scores straight from the microdata or JSON-LD:

def compare_products(a: str, b: str) -> dict:
    url = f"https://www.g2.com/compare/{a}-vs-{b}"
    html = requests.get(
        API, headers={"X-API-Key": API_KEY},
        params={"url": url, "render_js": "true",
                "premium_proxy": "true", "stealth": "true"},
        timeout=90,
    ).text
    tree = HTMLParser(html)
    scores = {}
    for block in tree.css('[itemtype*="schema.org/Product"]'):
        name = _attr(block, '[itemprop="name"]', "content") or _text(block, '[itemprop="name"]')
        rating = _attr(block, '[itemprop="ratingValue"]', "content")
        if name and rating:
            scores[name] = rating
    return scores   # {"Product A": "4.4", "Product B": "4.1"}

Category Grid pages at https://www.g2.com/categories/ rank every product in a category by Satisfaction and Market Presence, which is the fastest way to build a competitor list before you scrape each product's reviews. Read the product slugs off the Grid, feed them to scrape_many, and you have a full category's review corpus. G2 renders comparison and category pages with JavaScript, so the render step matters here as much as on review pages; the general approach for JS-built pages is in How to Scrape Dynamic JavaScript Websites.

This is the part that separates software-review scraping from product scraping. A G2 review ties a name to a job title, an employer segment, and an industry. Individually those look harmless. Together they can single out one person, especially at a small company, which is exactly what makes them personal data under the GDPR and personal information under the CCPA. G2 publishing them does not remove that status.

Handling that keeps a G2 dataset defensible:

• Minimize first. Most competitive and product work needs the ratings, structured pros and cons, dates, and firmographic segments, not the reviewer's name. If you do not need identity, do not store it.
• Pseudonymize when you only need to dedupe. If you need to catch repeat reviewers but not name them, hash the name and drop the raw value.

import hashlib

def pseudonymize(name: str | None) -> str | None:
    if not name:
        return None
    return hashlib.sha256(name.encode("utf-8")).hexdigest()[:16]

df["reviewer_id"] = df["author"].apply(pseudonymize)
df = df.drop(columns=["author"])   # keep the segment fields, drop the identity

• Have a lawful basis. For competitive or market research, "legitimate interests" (GDPR Article 6(1)(f)) is the usual basis, and it requires a documented balancing test weighing your interest against the reviewer's privacy.
• Respect data-subject rights. Be able to delete a person's records on request, which is another reason to key on a pseudonymous reviewer_id rather than scattering names across tables. Under the CCPA, be ready to honor deletion and opt-out duties for California residents if you sell or share the data.
• Do not republish identities. Aggregate ratings, segment breakdowns, and anonymized pros and cons are far safer to surface than a searchable copy of named reviews.

None of this blocks legitimate analysis; it shapes what you keep. A pipeline that stores ratings, cleaned pros and cons, dates, and firmographic segments answers almost every product question while holding almost no personal data at rest. When you need reviews tied to real identities for your own product, G2's licensed syndication and data programs are the route built for it.