How to Scrape Target Product Data

A single Target product page exposes more structured data than most retailers. Every field below comes back as JSON from the RedSky API, so you never have to parse HTML for it.

RedSky renames and reshapes these paths every few months, so treat the table as a map, not a contract. Always inspect one live response before you wire a parser to it.

Target.com is a Next.js app. Its first HTML response ships a skeleton plus a large __NEXT_DATA__ script, and the browser then fires background XHR calls to redsky.target.com to fill in price, promotions, and live inventory. The most valuable, most volatile fields (current price and per-store stock) are the ones that arrive over those XHR calls, not in the initial HTML.

You have two ways to collect this data:

1. Render the page with Playwright or Puppeteer, wait for hydration, and read the DOM. This works, but it is slow, memory-heavy, and easy to break every time Target ships a CSS change.
2. Call RedSky directly. The site's own front end is doing this on every page load. You send the same request, get back the same JSON, and skip the browser entirely.

The second path is faster and cheaper by an order of magnitude. A rendered page can take three to six seconds and a full headless Chromium; a RedSky call returns in a few hundred milliseconds and costs a single request. For any serious volume of target scraping, the JSON API is the only sensible starting point. If you are weighing a hosted API against running your own browser fleet and proxy pool, this trade-off is the whole argument in web scraping API vs self-managed proxies.

RedSky is Target's public product aggregation layer. The company has written about it openly on its engineering blog as a self-service platform for aggregated APIs. Every endpoint lives under one base path:

https://redsky.target.com/redsky_aggregations/v1/web/

The endpoints you need for product data are these:

Two parameters trip people up. key is a 40-character public web key that Target embeds in its own JavaScript bundle. It is not a secret you register for; it is the same key every browser uses, and it rotates every so often. visitor_id is any stable hex string; the API accepts a random one. channel is always WEB for the desktop site.

Because the key rotates, hardcoding it guarantees breakage. Pull it from the live page instead:

import re, json, requests

SCRAPE_ENDPOINT = "https://scrape.sparkproxy.io/api/v1"
SPARK_KEY = "sk-your-sparkproxy-key"   # from your SparkProxy dashboard

def scrape_raw(target_url: str, render_js: str = "false") -> str:
    """Fetch a URL through the SparkProxy Scraping API and return the raw body."""
    r = requests.get(
        SCRAPE_ENDPOINT,
        headers={"X-API-Key": SPARK_KEY},
        params={
            "url": target_url,
            "render_js": render_js,   # RedSky returns JSON, so no browser is needed
            "country_code": "US",     # Target serves US traffic only
        },
        timeout=45,
    )
    r.raise_for_status()
    return r.text

def get_web_key(sample_tcin: str = "84770895") -> str:
    """The public RedSky web key is embedded in Target's page bundle."""
    html = scrape_raw(f"https://www.target.com/p/-/A-{sample_tcin}")
    match = re.search(r'"apiKey":"([0-9a-f]{40})"', html)
    if not match:
        raise RuntimeError("Web key not found; the bundle layout changed.")
    return match.group(1)

TARGET_KEY = get_web_key()

Routing every request through the SparkProxy Scraping API gives you a clean US exit IP, automatic retries, and a single X-API-Key header instead of a proxy pool you have to babysit. Set render_js to false for RedSky JSON (one credit); reserve true for the rare case where you scrape the rendered page itself.

Every RedSky product call is keyed on a TCIN (Target.com Item Number). You get it two ways.

From a product URL, it is the number after A-:

import re

def tcin_from_url(url: str) -> str | None:
    match = re.search(r"/A-(\d+)", url)
    return match.group(1) if match else None

tcin_from_url("https://www.target.com/p/apple-airpods-pro-2nd-gen/-/A-84770895")
# "84770895"

If you are starting from a keyword or a category rather than a known URL, pull TCINs from the search endpoint (covered in the search section). Each result in the grid carries its own tcin, which you then feed into the PDP and fulfillment calls.

Build the pdp_client_v1 URL, fetch it through the Scraping API, and read the JSON. One helper builds the URL, another parses the fields you care about.

from urllib.parse import urlencode

REDSKY = "https://redsky.target.com/redsky_aggregations/v1/web"

def scrape_json(target_url: str) -> dict:
    return json.loads(scrape_raw(target_url))

def pdp_url(tcin: str, pricing_store_id: str = "1859") -> str:
    params = {
        "key": TARGET_KEY,
        "tcin": tcin,
        "is_bot": "false",
        "pricing_store_id": pricing_store_id,
        "has_pricing_store_id": "true",
        "visitor_id": "0193A1B2C3D4E5F60718293A4B5C6D7E",
        "channel": "WEB",
        "page": f"/p/A-{tcin}",
    }
    return f"{REDSKY}/pdp_client_v1?{urlencode(params)}"

def parse_product(payload: dict) -> dict:
    p = payload["data"]["product"]
    item = p.get("item", {})
    price = p.get("price", {})
    stats = p.get("ratings_and_reviews", {}).get("statistics", {}).get("rating", {})
    return {
        "tcin": p.get("tcin"),
        "title": item.get("product_description", {}).get("title"),
        "brand": item.get("primary_brand", {}).get("name"),
        "price": price.get("current_retail"),
        "reg_price": price.get("reg_retail"),
        "on_sale": price.get("current_retail") != price.get("reg_retail"),
        "rating": stats.get("average"),
        "review_count": stats.get("count"),
    }

product = parse_product(scrape_json(pdp_url("84770895")))
print(product)
# {'tcin': '84770895', 'title': 'Apple AirPods Pro (2nd Generation)',
#  'brand': 'Apple', 'price': 199.99, 'reg_price': 249.99, 'on_sale': True,
#  'rating': 4.6, 'review_count': 12847}

Use .get() all the way down rather than direct key access. RedSky omits keys for items that lack promotions or reviews, and a KeyError mid-crawl will kill an otherwise clean run.

Here is the detail most Target scraping tutorials miss, and the one that matters most if you monitor prices: the current price is store-scoped. Change pricing_store_id and the same TCIN can return a different current_retail, because clearance and markdown are localized to individual stores. Scrape only the default store and your dataset silently misses regional deals.

Availability works the same way and comes from a separate endpoint, pdp_fulfillment_v1, which takes a physical location:

def fulfillment_url(tcin: str, store_id: str, zip_code: str,
                    state: str, lat: float, lon: float) -> str:
    params = {
        "key": TARGET_KEY,
        "tcin": tcin,
        "is_bot": "false",
        "store_id": store_id,
        "store_positions_store_id": store_id,
        "has_store_positions_store_id": "true",
        "zip": zip_code,
        "state": state,
        "latitude": f"{lat:.3f}",
        "longitude": f"{lon:.3f}",
        "pricing_store_id": store_id,
        "has_pricing_store_id": "true",
        "channel": "WEB",
    }
    return f"{REDSKY}/pdp_fulfillment_v1?{urlencode(params)}"

def parse_stock(payload: dict) -> list[dict]:
    p = payload["data"]["product"]
    rows = []
    for opt in p.get("fulfillment", {}).get("store_options", []):
        pickup = opt.get("order_pickup", {})
        rows.append({
            "store": opt.get("location_name"),
            "store_id": opt.get("location_id"),
            "pickup_status": pickup.get("availability_status"),
            "on_hand": opt.get("location_available_to_promise_quantity"),
        })
    return rows

To compare a product across markets, loop over a small table of stores and collect both price and stock per location:

STORES = [
    # store_id, zip,   state, lat,     lon
    ("1859", "98801", "WA", 47.430, -120.320),
    ("3991", "10001", "NY", 40.750,  -73.997),
    ("2775", "78701", "TX", 30.267,  -97.743),
]

def price_across_stores(tcin: str) -> list[dict]:
    out = []
    for store_id, zip_code, state, lat, lon in STORES:
        prod = parse_product(scrape_json(pdp_url(tcin, pricing_store_id=store_id)))
        stock = parse_stock(scrape_json(
            fulfillment_url(tcin, store_id, zip_code, state, lat, lon)))
        out.append({"store_id": store_id, "zip": zip_code,
                    "price": prod["price"], "stock": stock})
    return out

That per-store spread is exactly what price-intelligence teams build on. If cross-retailer comparison is your goal, the same pattern extends to Walmart, Best Buy, and others, as covered in datacenter proxies for price comparison websites.

pdp_client_v1 already returns the rating average and total review count in its ratings_and_reviews.statistics block, which is enough for most dashboards. When you need the full review text, Target paginates reviews through their own aggregation call with count and offset:

def reviews_url(tcin: str, count: int = 50, offset: int = 0) -> str:
    params = {
        "key": TARGET_KEY,
        "tcin": tcin,
        "count": count,
        "offset": offset,
        "sort": "most_recent",
        "channel": "WEB",
    }
    return f"{REDSKY}/product_reviews_v1?{urlencode(params)}"

def all_reviews(tcin: str, page_size: int = 50, max_reviews: int = 500) -> list:
    reviews, offset = [], 0
    while offset < max_reviews:
        data = scrape_json(reviews_url(tcin, count=page_size, offset=offset))
        batch = data.get("data", {}).get("reviews", {}).get("results", [])
        if not batch:
            break
        reviews.extend(batch)
        offset += page_size
    return reviews

Stop as soon as a page returns fewer results than page_size or an empty list. That is the signal you have reached the end, and it avoids hammering the endpoint for pages that do not exist.

To collect TCINs at scale, work from plp_search_v2. It accepts either a keyword or a category node, plus count (results per page, default 24) and offset. Walk the offset forward until the grid runs dry.

def search_url(keyword: str, count: int = 24, offset: int = 0,
               store_id: str = "1859") -> str:
    params = {
        "key": TARGET_KEY,
        "keyword": keyword,
        "count": count,
        "offset": offset,
        "store_id": store_id,
        "pricing_store_id": store_id,
        "visitor_id": "0193A1B2C3D4E5F60718293A4B5C6D7E",
        "channel": "WEB",
        "page": f"/s/{keyword}",
    }
    return f"{REDSKY}/plp_search_v2?{urlencode(params)}"

def search_all(keyword: str, page_size: int = 24, max_items: int = 240) -> list[dict]:
    items, offset = [], 0
    while offset < max_items:
        data = scrape_json(search_url(keyword, count=page_size, offset=offset))
        products = data.get("data", {}).get("search", {}).get("products", [])
        if not products:
            break
        for prod in products:
            items.append({
                "tcin": prod.get("tcin"),
                "title": prod.get("item", {})
                             .get("product_description", {}).get("title"),
                "price": prod.get("price", {}).get("current_retail"),
            })
        offset += page_size
    return items

wireless_earbuds = search_all("wireless earbuds")

Target caps how deep search paging goes (typically a few hundred results per query), so narrow broad terms into tighter queries or category nodes rather than trying to page past the ceiling. Deeper coverage comes from more specific searches, not larger offsets.

The main target.com domain sits behind Akamai Bot Manager, which drops sensor cookies (_abck, bm_sz) and scores browser behavior. The RedSky host is lighter, but it still rate-limits per IP and rejects requests with a stale or missing key. Here is what each defense looks like and how to get past it cleanly.

Two practical habits keep a Target crawler alive. First, refresh the web key on the fly instead of assuming it lasts forever:

def scrape_json_resilient(target_url: str) -> dict:
    global TARGET_KEY
    try:
        return scrape_json(target_url)
    except (requests.HTTPError, json.JSONDecodeError):
        TARGET_KEY = get_web_key()                     # key rotated; refresh once
        refreshed = re.sub(r"key=[0-9a-f]{40}", f"key={TARGET_KEY}", target_url)
        return scrape_json(refreshed)

Second, pace yourself. RedSky tolerates steady traffic far better than bursts. A short randomized delay between calls does more for your success rate than any single header trick. For the full anti-detection playbook, including TLS fingerprinting and header consistency, read how to avoid getting your proxy blocked.

Scraping public product data is common practice, and courts have generally treated openly available web data differently from data behind a login. That is not a license to ignore the rules. Keep your collection defensible:

• Read Target's robots.txt and Terms of Use first. Fetch https://www.target.com/robots.txt, honor its disallowed paths, and respect any Crawl-delay. Target's Terms restrict certain automated access, so understand what you are agreeing to.
• Only touch public data. Product pages, prices, and public reviews are visible to any visitor. Never scrape anything behind a login, a cart, or a member account.
• Collect no personal data. Reviewer nicknames and profiles are personal information. Aggregate ratings and counts, not identities.
• Rate-limit like a good citizen. One request every one to three seconds per IP is plenty for price and stock monitoring. You are not trying to overload anyone's servers.
• Use the data for legitimate purposes. Price monitoring, assortment research, and market analysis are standard. This is the same category of work described in how ecommerce companies use proxies for competitive intelligence.

import urllib.robotparser

rp = urllib.robotparser.RobotFileParser()
rp.set_url("https://www.target.com/robots.txt")
rp.read()

print(rp.can_fetch("*", "/p/-/A-84770895"))   # check before you crawl a path
print(rp.crawl_delay("*"))                      # respect any declared delay

None of this is legal advice. Rules and terms change, so review Target's current policies and your own jurisdiction before you run a crawler at scale.