Proxies for SaaS Pricing and Paywall Research

Tracking published list prices across countries? Buy country-targeted datacenter exits and sample each country many times rather than once. SaaS marketing sites want to be crawled and indexed, so they are lightly defended compared with a marketplace, and the expensive requirement is sample count rather than exit quality.

Trying to see prices that only exist inside a logged-in account? That is a different build. You need persistent sessions you control, one identity per account, and a stable address behind each one. No amount of rotation helps, and a managed scraping endpoint that gives every request a fresh browser will actively fight you.

Researching paywalls? Almost always, what you want is the offer page and it is public. Read the section on paywalls before you spend anything, because the most common purchase in this category is proxies bought to solve a cookie problem.

A pricing page decides what to show you by stacking four independent gates. Miss any one and your comparison is between two things that are not comparable.

The region gate. Geo-IP detection sets the displayed currency, and often more than the currency. Plan lineups differ by market, some tiers are hidden in some countries, and local payment methods change what appears at checkout. Some vendors read the IP, others read an explicit country selector stored in a cookie, and a few read the browser's Accept-Language. Which mechanism a vendor uses determines whether a country-targeted exit is enough or whether you also need to set a cookie. Our explainer on geo-targeting in proxies covers how the detection works from the network side.

The currency gate. Displayed currency and charged currency are not always the same, and the exchange rate a vendor uses is rarely the spot rate. A price shown in euros may be a rounded local price, a converted USD price, or a genuinely different commercial decision. You cannot tell which from the number alone, so record the currency as a field rather than normalising at collection time.

The account gate. Published pricing is the shop window. The prices that matter are frequently inside the product: in-app upgrade pricing, seat-tier thresholds, usage overage rates, and the numbers a sales team quotes against a published "contact us". Public pages will not give you these, and no proxy configuration changes that.

The experiment gate. This is the one that makes SaaS different. Vendors run continuous tests on pricing pages: different price points, different tier names, different anchoring, different annual discounts. Your request gets assigned a bucket, that assignment sticks to a cookie, and the page you see is one draw from a distribution rather than the price.

Here is the failure mode, and it is quiet enough that teams ship dashboards built on it.

You fetch a competitor's pricing page once from each of eight countries. Six show $49, one shows $59, one shows $45. You write up a finding about regional price discrimination. In fact the vendor charges $49 everywhere and is running a test with a $59 variant at 15 percent of traffic and a $45 variant at 10 percent. Your two "regional differences" are experiment buckets, and the countries they landed in are arbitrary.

This is worse than having no data, because a single observation carries no signal that it might be an outlier. A missing number prompts a question. A wrong number prompts a decision.

The fix is not a better proxy. It is more samples per cell, drawn independently, and a summary statistic that survives the variance. Three rules follow:

1. Report the mode, not the value. The price most commonly served is the price. A mean across experiment buckets is a number that no customer has ever been shown.
2. Report the variant set alongside it. The fact that a vendor is testing a higher price point is usually more interesting to your own pricing team than the current list price.
3. Never compare single observations across countries. If you only have budget for one sample per country, you do not have budget for a geo comparison. You have budget for a presence check.

The sample count falls straight out of how small a variant you want to be able to catch.

If a variant is served to a fraction p of visitors, the chance of missing it entirely in n independent samples is (1 - p)^n. Set that below 5 percent and solve for n. The arithmetic is plain, and the results are smaller than people fear:

So 29 samples per country catches any variant running at 10 percent or more, with a one-in-twenty chance of missing it. That is the number we would design around for most competitive pricing work, because variants below 10 percent of traffic are usually early tests that will either scale up (and get caught next cycle) or be abandoned.

Two honest caveats. This assumes your samples really are independent draws, which the next section is about. And it tells you whether a variant exists, not its exact share: estimating the split accurately needs considerably more samples than detecting it.

Experiment assignment is almost always written to a cookie, sometimes derived from a hash of a client identifier, and only occasionally keyed on the IP address. That ordering matters, because it inverts the usual instinct.

Rotating the exit address while reusing a cookie jar gives you 29 requests that all land in the same bucket. You will have paid for 29 samples and collected one. Conversely, clearing cookies while reusing one address usually does give you 29 independent draws, because the assignment had nowhere to persist.

So the requirement is, in priority order:

1. A fresh cookie jar per sample. Non-negotiable. This is the variable the assignment actually rides on.
2. A fresh exit address per sample. Needed when the vendor hashes the IP into the bucket, and useful anyway for rate-limit headroom.
3. A varied client fingerprint. Relevant only for the minority of vendors that assign on a device hash.

This is one of the few workloads where the SparkProxy Scraping API's session behaviour is exactly what you want rather than a limitation. The documented behaviour is that each request gets a fresh browser profile and cookies do not persist across separate API requests, even when you pass a session_id, which labels the job in per-request logs rather than binding state. For sampling an experiment, that means every request is independent by construction and you cannot accidentally reuse a bucket.

The same property is the reason it is the wrong tool for the logged-in half of the job. If you need to hold a session, you either inject the cookies yourself on every call through the API's cookies parameter, or you run your own browser against a proxy and keep the jar. Our guides to handling cookies and sessions and scraping behind a login cover the second path.

Paywall research gets bundled into this work because the mechanics look similar. They are not, and the distinction saves money.

Hard paywall. No content without an active subscription. There is nothing to sample and nothing a proxy fixes. What is public is the subscription offer page, which is usually what you wanted.

Metered paywall. A fixed number of free items per period. The counter is kept in a cookie first, with the IP and a device fingerprint as corroborating or fallback signals. Because the cookie is primary, rotating addresses without clearing cookies does not reset the meter, and clearing cookies frequently does. Teams buy proxies to solve this and then discover the fix was one line of cookie handling.

Registration wall. Content is free but requires an account. This is an account problem, not a network problem, and it lands you back in the logged-in-session build.

For pricing research specifically, you nearly always want the paywall's metadata rather than its contents: what tiers exist, what each costs, what is gated behind which, whether there is a trial, and how the annual discount is framed. All of that sits on a public page by design, because the vendor is trying to sell it. Circumventing a paywall to obtain and reuse the gated content is a different activity with different legal consequences, covered below.

Pricing datasets rot because collection throws away the context that makes the number interpretable. Store these fields on every single sample, not on every sweep:

• Full URL, including any query parameters, unshortened.
• UTC timestamp recorded by your collector.
• Exit country actually used, read back from the response rather than assumed from your request.
• Displayed currency, as a separate field from the numeric price.
• Numeric price, billing period, and what one unit of the price buys (per seat, per workspace, flat).
• Annual discount as published, plus whether the headline figure is the monthly-billed or the annual-billed-monthly-equivalent price.
• Whether the tier shows a number or a "contact sales" style call to action.
• Any experiment or variant identifier the page exposes in a cookie, a data attribute or a script payload.
• A hash of the stored rendering.

Two normalisation rules worth writing into the pipeline rather than the analysis. Never convert currencies at read time: store the original and the FX rate with its date, so a later correction is possible. And always normalise to a per-seat-per-month figure at a single billing period before comparing anything, because comparing a monthly-billed price against an annual-billed monthly equivalent is the most common error in SaaS pricing decks and it inflates the gap by exactly the annual discount.

Change detection is the other half. Pricing pages change rarely and then all at once, so a diff-based pipeline saves far more than it costs. Our post on incremental scraping and change detection covers the pattern, and it applies cleanly here because a stable price is the normal case.

Concrete numbers, built on stated assumptions rather than any test we ran.

Scenario: 25 competitor products, 8 countries, sampled enough to catch a 10 percent variant, swept weekly.

• 25 products x 8 countries x 29 samples = 5,800 page fetches per sweep.
• Weekly sweeps across a month = 23,200 fetches.

Modern SaaS pricing pages are usually client-rendered, with the currency switch and the plan grid assembled in the browser, so most of these need render_js. Priced through the SparkProxy Scraping API, where the base cost is the proxy tier multiplied by the rendering mode and country_code, stealth, js_scenario and screenshot or PDF output are the only add-ons at 5 credits each:

The last row is where credit arithmetic usually goes wrong. Setting premium_proxy=true replaces the base cost rather than stacking on top of it, so a rendered residential fetch is 25 credits, and adding country targeting makes it 30. It is not 5 plus 25 plus 5.

Two observations from the table. First, the whole programme fits the $49 Starter plan at 232,000 credits a month with room to spare, which is a smaller number than most teams expect for 23,200 samples across 25 competitors. Second, adding a screenshot to every sample costs 50 percent more and is worth it here: a pricing claim you cannot show a picture of will be argued with, and the archive is the deliverable that outlives the dashboard.

If you find the JSON payload behind the plan grid, which many of these pages have because the grid is client-rendered, the plain-fetch row applies instead and the bill drops from 232,000 to 139,200 credits, a 40 percent reduction. Our guide to finding hidden JSON API endpoints is the relevant technique, and on client-rendered pricing pages it works more often than not. Every SparkProxy account starts with 1,000 free credits and no card, which is enough to test whether your specific targets need rendering at all before you choose a plan.

SaaS pricing pages are marketing pages. The vendor's entire intent is for them to be crawled, indexed, linked and shared. That makes them one of the softest targets in commercial scraping, and it changes the buy.

The practical consequence is that this is a cheaper workload than retail price monitoring, where residential exits are often genuinely required. Our comparison of residential and datacenter proxies sets out where the line falls. SparkProxy's own datacenter network covers 80+ countries through gateway.sparkproxy.io on port 11000, and plans start at $75 a month for 100 threads with unlimited bandwidth. Test a handful of your targets from a hosting address before you assume you need anything more expensive.

One structural note for the in-app case: if you do need logged-in pricing, size it as an account-management problem rather than a scraping problem. One account per competitor product, one stable exit per account, sessions you hold and refresh deliberately. The automated price monitoring pattern does not transfer, because its whole design assumes stateless requests.

Two activities get confused here and they sit on opposite sides of a line.

Observing what a vendor publishes on a public page, including the fact that different visitors see different prices, is ordinary competitive research. The information is published deliberately, to the public, for commercial purposes.

Obtaining content that sits behind a paywall or an account gate, by circumventing the gate, is a different matter. It may breach the terms you accepted, and where the content is copyrighted, reusing it carries its own exposure. Our overview of whether proxies are legal for business use sets out the general framework, and the specific advice for this workload is narrow: collect the offer, not the product.

Three practices that keep the programme defensible. Keep request rates low enough that your collection is indistinguishable from ordinary traffic, because a pricing sweep at 29 samples per country per week genuinely is small. Identify your collector where the target asks for it. And never create accounts using false identity information to reach gated pricing, which converts a research activity into a misrepresentation.

For adjacent SaaS intelligence that does not touch any of these questions, public review platforms carry a lot of what teams actually want, and our guide to scraping G2 reviews covers that source. Vendor changelogs, status pages and job postings are similarly public and similarly underused.