Proxies for App Store Optimization (ASO) Data

App store optimization runs on four questions, and only one of them is answerable from your own dashboards.

1. Where do we rank for the keywords that matter, in each market we sell in?
2. Where do we sit in the category and overall charts, and for how long?
3. What are users actually saying, per country and per app version?
4. What are competitors shipping, how often, and what did they change in their listing?

App Store Connect and Google Play Console answer none of these about anyone but you. They report impressions, product page views, and conversion rate for your own app. They will not tell you that a rival moved from position 14 to position 3 for "budget tracker" in Germany last Tuesday, and they do not give you your own ranked position for a keyword either. That gap is why ASO teams collect public store data.

Here is the mapping from question to collection job, which is the part most ASO posts skip:

Design the warehouse around that split early. Retrofitting a rank history onto a table that only stores "current rank" costs you the months of data you actually wanted.

Both stores decide what to show using two inputs: the country parameter in the request and the IP the request came from. When those disagree, the failure is quiet. You do not get an error, you get a response that looks fine and describes the wrong market.

The specific ways it goes wrong:

• Storefront mismatch. A US exit asking for the Japan storefront can receive a partial catalogue, English metadata where a Japanese listing exists, or a redirect back to the default storefront. The JSON parses. The numbers are wrong.
• Geo-filtered availability. Apps are published per territory. An app absent from the US storefront is simply missing from your results, which reads as "not ranking" instead of "not distributed here."
• Rate limiting per IP. Apple documents the iTunes Search API at approximately 20 calls per minute, and Google Play's review RPC throttles hard per source address. A keyword matrix of any size hits that ceiling on a single IP within minutes.
• Datacenter range blocks. Google Play serves rendered store pages differently, or not at all, to flagged datacenter ranges, so a slice of your fleet returns empty JSON-LD blocks and your parser records nulls.
• Price localization. Price fields, offers, and in-app purchase tiers localize by storefront, so a pricing comparison built from one exit IP describes one country.

Routing each request through an exit in the storefront you are asking about removes the whole class of problems at once. With the SparkProxy Scraping API that is a single parameter:

import requests

API = "https://scrape.sparkproxy.io/api/v1"
KEY = "YOUR_API_KEY"

def fetch(target, country="US", render=False):
    r = requests.get(API, headers={"X-API-Key": KEY}, params={
        "url": target,
        "country_code": country,     # exit IP in this country
        "render_js": str(render).lower(),
        "format": "json",
    }, timeout=60)
    r.raise_for_status()
    return r.json()                  # {"status_code": ..., "body": "...", ...}

country_code takes an ISO 3166-1 alpha-2 code, so the parameter you send Apple or Google and the IP the request exits from stay in agreement. If the concept is new, what geo-targeting means in proxies covers the mechanics underneath.

Keyword rank tracking is the backbone of ASO data collection and the job with the worst signal-to-noise ratio if you are careless about method.

The task itself is simple: for each keyword in your tracked set, in each storefront, find the position of your app ID and of each competitor ID in the result list. Apple's Search API returns a ranked array, so position is array index plus one.

import json
from urllib.parse import quote

def keyword_positions(term, watch_ids, country="US", limit=100):
    target = (f"https://itunes.apple.com/search?term={quote(term)}"
              f"&country={country}&entity=software&limit={limit}")
    env = fetch(target, country=country, render=False)
    results = json.loads(env["body"])["results"]
    index = {str(a["trackId"]): pos + 1 for pos, a in enumerate(results)}
    return {app_id: index.get(app_id) for app_id in watch_ids}   # None = outside top N

Now the caveat that decides whether your numbers are worth anything. The Search API is a catalogue search endpoint, and its result ordering is closely related to, but not guaranteed identical to, the ranked results a user sees inside the App Store app on a device. Treat the API position as a tracked index rather than an exact reproduction of store search. Two rules keep that index honest:

• Freeze the methodology. Same limit, same entity, same storefront, same time of day, permanently. A change in request shape produces a rank movement that has nothing to do with your app.
• Calibrate periodically. Once a month, check a handful of keywords by hand on a device in one or two priority markets and record the delta against your index. If the delta is stable, the series is trustworthy for trend work even where the absolute number is off by a position or two.

Which keywords to track follows from how each store indexes text. Apple gives you a 30-character title, a 30-character subtitle, and a 100-character keyword field, and it does not index the description. Google Play indexes the 30-character title, the 80-character short description, and the 4,000-character full description. That difference means the same app usually needs two keyword sets and two tracking matrices, and the Google Play set is typically larger because the store indexes far more text.

Language is a separate axis from country. Canada, Switzerland, and Belgium each run one storefront with several localizations, so "budget tracker" and "gestionnaire de budget" in the Canadian storefront are two different rows. Teams that model rank as a (keyword, country) pair silently overwrite half their data the day they add a second language. Use (keyword, country, locale) from day one.

The workflow will feel familiar to anyone who has built web rank tracking, and the infrastructure lessons transfer directly from datacenter proxies for SEO rank tracking: fixed schedule, distributed exits, one row per observation, never overwrite.

Chart position answers a different question than keyword rank. Keyword rank tells you whether the store thinks you are relevant; chart rank tells you whether you are getting installs. They move on different clocks, and the gap between them is one of the more useful things an ASO pipeline can measure.

Apple publishes charts as RSS feeds where array order is the ranking, one feed per country and feed type. A daily snapshot per market gives you your own position and the full competitive set for that category.

def chart_snapshot(country="us", feed="top-free", limit=100):
    target = f"https://rss.applemarketingtools.com/api/v2/{country}/apps/{feed}/{limit}/apps.json"
    env = fetch(target, country=country.upper(), render=False)
    rows = json.loads(env["body"])["feed"]["results"]
    return [{"rank": i + 1, "id": a["id"], "name": a["name"],
             "publisher": a["artistName"], "country": country.upper()}
            for i, a in enumerate(rows)]

Three derived metrics are worth more than the raw rank:

• Time in top N. Count the days an app held a position at or above your threshold, per country. A competitor that spikes to rank 8 for two days ran a burst campaign. One that holds rank 20 for six weeks has organic pull.
• Publisher share of chart. Group a category chart by artistName and you see which studios own the shelf in each market. That is the honest competitive set for that market, which is often not the list your product team assumed.
• Rank-to-chart lead time. Track how many days pass between a keyword rank improvement and a chart movement for the same app. In most categories keyword rank moves first, because visibility precedes installs. Once you know your own lead time, a keyword gain becomes a forecast instead of a vanity metric.

None of that works from one country. Chart composition varies wildly between markets.

Reviews are the only ASO input that tells you why the numbers moved. They are also the most storefront-specific data in the store, because a Japanese review pool and a German review pool for the same app share nothing except the app ID.

Apple's customer reviews feed puts the country in the URL path and returns the version each reviewer was running, which is the field that makes sentiment analysis actionable:

from collections import defaultdict

def reviews_by_version(track_id, country="us", max_pages=10):
    buckets = defaultdict(list)
    for page in range(1, max_pages + 1):
        target = (f"https://itunes.apple.com/{country}/rss/customerreviews/"
                  f"page={page}/id={track_id}/sortby=mostrecent/json")
        env = fetch(target, country=country.upper(), render=False)
        entries = json.loads(env["body"]).get("feed", {}).get("entry", [])
        rated = [e for e in entries if "im:rating" in e]
        if not rated:
            break
        for e in rated:
            buckets[e["im:version"]["label"]].append({
                "stars": int(e["im:rating"]["label"]),
                "text": e["content"]["label"],
                "country": country.upper(),
                "updated": e["updated"]["label"],
            })
    return buckets

Grouping by version rather than by date is the trick. Star averages by calendar week blur across releases and hide what you need to know, which is whether build 4.7.1 broke something for one market. Group by (version, country) and a crash that hits one carrier or language shows up as a rating cliff in that storefront while the global average barely twitches.

Watch the two rating fields together as well. averageUserRating is lifetime, while averageUserRatingForCurrentVersion resets when a developer starts a fresh ratings cycle with a new version. A competitor's headline score jumping two tenths overnight is usually a ratings reset plus an in-app prompt campaign, not a sudden quality leap. Comparing the two fields tells you which one it was.

Apple's feed stops at 10 pages per country, roughly 500 reviews, so poll most-recent on a schedule and accumulate rather than backfill. The analysis side, including several languages in one sentiment model, is covered in using proxies for review monitoring and sentiment analysis.

This is the section most ASO tooling never builds, and it is where a self-run pipeline beats a subscription dashboard.

Every daily metadata snapshot you already collect contains a competitor's title, subtitle, description, release notes, version string, release date, and screenshot URLs, per storefront. Diff yesterday's snapshot against today's and you get a change log of your competitors' ASO work:

import hashlib

TRACKED = ["trackName", "version", "currentVersionReleaseDate",
           "releaseNotes", "description", "primaryGenreName", "formattedPrice"]

def snapshot(track_id, country):
    target = f"https://itunes.apple.com/lookup?id={track_id}&country={country}"
    r = json.loads(fetch(target, country=country)["body"])["results"][0]
    shots = "|".join(r.get("screenshotUrls", []))
    row = {k: r.get(k) for k in TRACKED}
    row["screenshot_hash"] = hashlib.sha1(shots.encode()).hexdigest()[:12]
    row["country"] = country
    return row

def diff(previous, current):
    return {k: (previous.get(k), current.get(k))
            for k in current
            if k != "country" and previous.get(k) != current.get(k)}

What that surfaces, in rough order of how often it gets missed:

• Localized metadata experiments. A title or subtitle that changes in Germany and Japan but not the US is a competitor testing positioning in secondary markets first. You see the hypothesis while it runs, not after the case study.
• Creative tests. A screenshot_hash that churns repeatedly in one storefront while staying stable elsewhere usually means a product page test is live. Apple's Product Page Optimization supports up to three treatments against the original, and Google Play offers store listing experiments, so single-market churn is a strong signal one of them is running.
• Release cadence. Deltas between currentVersionReleaseDate values give you a competitor's shipping rhythm. A team that moved from monthly to weekly releases is either fighting fires or investing hard, and their release notes usually say which.
• Price and tier changes. formattedPrice moving in a subset of storefronts is regional price testing, which matters a lot if you compete on subscription pricing.

Keyword rank tracking tells you the result. Metadata diffing tells you the input that caused it. Running both against the same daily snapshot costs almost nothing extra, because you already fetched the record.

ASO collection volume is a straightforward product, and it grows faster than teams expect:

daily requests = (keywords x storefronts x locales) + (apps tracked x storefronts) + (chart feeds x storefronts) + review polls

Three realistic programs:

Put the portfolio row against Apple's roughly 20 calls per minute per IP and the arithmetic is unforgiving: 9,300 requests from one address needs about eight hours of clean running, and it gets throttled long before that. Split across 20 country-matched exits, each carries roughly 465 requests, under half an hour per market, inside one collection window.

Cost follows the render flag, not the request count. Apple's JSON feeds and RSS charts are plain HTTP and need no browser, so keep render_js=false for those. Google Play's details page needs a rendered fetch, which costs more per request, so restrict rendering to Play and to any screenshot capture you genuinely need:

# Apple: plain JSON, no rendering, exit IP in Germany
curl -X GET "https://scrape.sparkproxy.io/api/v1?url=https%3A%2F%2Fitunes.apple.com%2Flookup%3Fid%3D123456789%26country%3DDE&country_code=DE&render_js=false&format=json" \
  -H "X-API-Key: YOUR_API_KEY"

# Google Play: rendered page, premium exit for a stubborn storefront
curl -X GET "https://scrape.sparkproxy.io/api/v1?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.example.app%26gl%3DJP%26hl%3Dja&country_code=JP&render_js=true&premium_proxy=true&format=json" \
  -H "X-API-Key: YOUR_API_KEY"

Capturing the product page as a local user sees it is a format=screenshot call through the same country-matched exit, the cheapest way to keep visual evidence of a competitor's creative test before they roll it back.

Not every ASO request needs the same class of IP, and paying residential rates for Apple's public JSON feeds is money set on fire.

The practical setup is a hybrid: datacenter carries the bulk daily sweep, residential handles Google Play and anything that failed validation, and you escalate into the expensive pool instead of defaulting to it. The trade-offs are laid out in our comparison of residential, datacenter, and mobile proxy types.

An ASO dataset is only useful if a number from March and a number from August were measured the same way. Four rules do most of the work.

Collect at a fixed local time per storefront. Ranks and charts drift through the day as installs accumulate on local clocks. Sampling Japan at 09:00 JST one week and 22:00 JST the next produces movement that is pure sampling artifact. Store every timestamp in UTC, but schedule against local time, and record which local window each row belongs to.

Freeze the request shape. Same parameters, same result limit, same locale, same rendering flag. Version your collector config and stamp that version on every row. When a series breaks, the first question is whether the app changed or the collector did, and a config version answers it in seconds.

Run a canary app. Pick a stable, well-established app in each tracked category whose rank should be close to flat week over week, and collect it alongside your real targets. If the canary jumps 40 positions overnight in every storefront, the store did not reshuffle, your pipeline broke. Canaries catch parser drift, storefront mismatches, and silent geo-routing failures faster than any alert on your own metrics, because you already know the right answer.

Persist raw payloads. Keep the original JSON next to the parsed row for a rolling window. When you decide in three months that you want the star histogram or the in-app purchase list, you backfill from storage instead of losing that history.

Scheduling this reliably, with retries, backoff on 429, and jitter so retries do not synchronize into a fresh burst, is its own problem. The patterns in how to schedule and automate web scrapers apply directly to a daily multi-storefront ASO run.

ASO collection reads public listing pages, which puts it in the better-understood half of scraping law, but "public" does not settle every question.

In the United States, the Ninth Circuit's decision in hiQ Labs v. LinkedIn (2022) held that scraping publicly accessible data, with no login and no authentication wall, generally does not violate the Computer Fraud and Abuse Act. That addresses access. Apple's and Google's terms of service are a separate, contractual matter, and Google's restrict automated collection. Both are true at once.

Practical guardrails:

• Collect public listing data only: ranks, ratings, review text, version notes, category, price. Nothing behind a sign-in.
• Treat reviewer identity as personal data. Aggregate sentiment, drop names at ingest, and put a retention policy on what survives.
• Rate-limit below what the endpoints tolerate. Country-matched IPs exist to make collection correct and finish on schedule, not to hit an endpoint harder than a real client would.
• If the data feeds a commercial product rather than internal decisions, have a lawyer read the terms. This is engineering guidance, not legal advice.

The teams that get burned are rarely the ones tracking ranks. They are the ones who bolted review scraping onto the same pipeline and never wrote a retention policy for reviewer names.