New Listing Alert Scraper: Rental Alerts in Minutes

A portal's alert email is the last link in a chain you don't control: the agent uploads the listing, the portal indexes it, a batch job matches it against millions of saved searches, and a mail queue sends it. Each step has its own schedule. Your scraper skips the last three. The moment a listing appears on the search results page, the next poll sees it.

Be clear about the limits before writing code. Polling can't see a listing before the portal publishes it, and it can't see listings that only exist behind a login (some portals show "early access" or member-only stock). It also can't win you the flat. It only gets the alert to you faster, so you still need to be ready to call or book a viewing from your phone.

Read the portal's terms first, poll only public search pages, and keep the request rate in the range of a person refreshing a tab. Our guide to scraping real estate listings covers the terms-of-service side and the general field extraction, so this post sticks to the alert job.

The whole system is one small process with five steps per search, repeated on a timer:

1. Fetch the saved-search URL, sorted newest first, through the Scraping API or a rotating proxy.
2. Parse the result cards into {id, url, price, beds, text} records.
3. Check each ID against a SQLite table of IDs already seen. Anything new is inserted and kept.
4. Filter the new records against your rules (price ceiling, minimum beds, area keywords, exclusions).
5. Push a notification for each survivor, with a tap-through link to the listing.

Two guards sit between parse and push. One stops alerts when the parser suddenly finds far fewer cards than usual, because that means a block or a markup change, not a quiet market. The other stops alerts when nearly every ID looks new at once, which usually means the portal changed its ID format. Both page you, the operator, instead of the apartment hunter.

Build the search in the portal's normal web UI with every filter it supports: city or map area, rent range, bedrooms, property type. Then change the sort to "newest" or "most recent" and copy the URL from the address bar. That URL is your saved search. The portal's own filters cut what you download, so let it do as much work as it can.

Sorting by newest is what makes polling cheap. New listings land at the top of page 1, so you only ever need page 1. If a portal won't keep the sort in the URL, check whether it has a sort parameter you can add by hand, or look for a "listed in the last 24 hours" filter that does the same job.

Keep every search in a config file so you can add one without touching code. The URLs below are placeholders on our own domain; paste your portal's real saved-search URLs in their place.

# searches.yaml
portals:
  portal_a:
    base: "https://listings.sparkproxy.io"
    card: "li.result-card"
    link: "a.result-link"
    price: ".result-price"
    id_pattern: "/rent/(?P<id>\\d+)"
    render: false
  portal_b:
    base: "https://homes.sparkproxy.io"
    card: "article[data-testid='listing']"
    link: "a"
    price: "[data-testid='price']"
    id_pattern: "/property/(?P<id>[A-Za-z0-9]+)"
    render: true

searches:
  - name: "2 bed east side"
    portal: portal_a
    url: "https://listings.sparkproxy.io/search?city=springfield&beds=2&max=2400&sort=newest"
    every_min: 3
    min_cards: 5
    rules: {max_price: 2400, min_beds: 2, exclude: ["no pets", "sublet"], areas: []}
  - name: "studio downtown"
    portal: portal_b
    url: "https://homes.sparkproxy.io/rent?area=downtown&type=studio&sort=date_desc"
    every_min: 5
    min_cards: 3
    rules: {max_price: 1600, min_beds: 0, exclude: [], areas: ["downtown", "old town"]}

The id_pattern is the most important line per portal. It's a regex with a named id group, run against each card's link. Listing URLs are far more stable than CSS class names, which front-end builds often regenerate, so pinning detection on the URL keeps the scraper alive through most redesigns.

You have two ways to fetch a search page through SparkProxy, and the choice comes down to whether the portal ships its results in plain HTML.

Check first. Open the saved search, view the page source (not the inspector), and search for a price you can see on screen. If it's in the source, a plain fetch works. If the source is an empty shell and the cards arrive by JavaScript, you need rendering, or you need the JSON the page loads. Many portals built on Next.js embed the full result set in a __NEXT_DATA__ script tag; our Zillow scraping guide walks through that pattern, and it applies to other portals too.

The Scraping API path handles retries and browser rendering for you. Per SparkProxy's docs (read 30 October 2026), render_js defaults to true and costs 5 credits, so set it to false explicitly for plain-HTML portals (1 credit). For rendered pages, ask for the json_response envelope and decode its body, which is base64.

import base64
import os

import requests

API = "https://scrape.sparkproxy.io/api/v1"
API_KEY = os.environ["SPARK_API_KEY"]


class FetchError(Exception):
    pass


def fetch_api(url, render=False, wait_for=None, country=None):
    params = {"url": url, "render_js": "true" if render else "false"}
    if country:
        params["country_code"] = country      # +5 credits
    if render:
        params["json_response"] = "true"      # envelope with base64 body
        if wait_for:
            params["wait_for"] = wait_for     # CSS selector, gives up after 30 s
    r = requests.get(API, params=params, headers={"X-API-Key": API_KEY}, timeout=420)
    if r.status_code == 429:
        retry = r.json().get("retry_after_seconds", 60)
        raise FetchError(f"api rate limit, retry after {retry}s")
    if r.status_code != 200:
        raise FetchError(f"api {r.status_code}: {r.text[:200]}")
    if not render:
        return r.text                         # plain fetch: raw HTML body
    env = r.json()
    if env.get("status_code") != 200:
        raise FetchError(f"target returned {env.get('status_code')}")
    return base64.b64decode(env["body"]).decode("utf-8", errors="replace")

Two details from the docs shape that function. A failed job returns 530 and the credits are refunded, so a block costs you time, not money. And the API makes up to three automatic retries (a 90-second first attempt, then 120 s and 180 s), which is why the client timeout is long. For a poller, that means one stuck search can delay a cycle by several minutes; run searches in separate threads if that matters to you.

The raw proxy path sends your own requests call through gateway.sparkproxy.io. Port 11000 is the HTTP/HTTPS gateway, and the username carries the country (the homepage example is user-country-us). Bandwidth is unlimited on every plan, so page weight never shows up on the bill.

import os
import random

import requests

USER = os.environ["SPARK_PROXY_USER"]     # e.g. user-country-us
PASSWORD = os.environ["SPARK_PROXY_PASS"]
ROTATING = f"http://{USER}:{PASSWORD}@gateway.sparkproxy.io:11000"

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
    "(KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 "
    "(KHTML, like Gecko) Version/18.6 Safari/605.1.15",
]


class FetchError(Exception):
    pass


def fetch_proxy(url, proxy=ROTATING):
    headers = {
        "User-Agent": random.choice(USER_AGENTS),
        "Accept": "text/html,application/xhtml+xml",
        "Accept-Language": "en-US,en;q=0.9",
    }
    r = requests.get(url, headers=headers, timeout=30,
                     proxies={"http": proxy, "https": proxy})
    head = r.text[:20000].lower()
    if r.status_code in (403, 429) or "captcha" in head or "are you a robot" in head:
        raise FetchError(f"blocked: HTTP {r.status_code}")
    r.raise_for_status()
    return r.text

SparkProxy's pools are datacenter IPs. They're the right fit for a few polls an hour on most listing sites, and the reasoning behind that is in datacenter proxies for real estate data aggregation. Portals with heavy bot defences are better handled through the Scraping API with rendering, where proxy_type=premium is available (10 credits without JS, 25 with JS, per the docs).

Diffing the whole page sounds simpler and fails constantly. Ad slots rotate, "listed 4 minutes ago" becomes "listed 7 minutes ago", and promoted cards shuffle position on every load. Every one of those changes looks like news. A listing ID doesn't change when the page around it does.

Parse each card into a small record, take the ID from the link with the portal's regex, and keep the price and bed count for filtering. The parser below uses selectolax. One trap if you copy older tutorials: selectolax 1.0.0 (PyPI, 3 October 2026) deprecated its Modest backend, and from selectolax.parser import HTMLParser now raises ImportError with a message telling you to switch. We hit that running this code on 1.0.2 (released 22 October 2026). The lexbor backend has the same API.

import re
from urllib.parse import urljoin

from selectolax.lexbor import LexborHTMLParser

BEDS = re.compile(r"(\d+)\s*(?:bd|bds|bed|beds|br|bedroom)s?\b", re.I)


def to_int(text):
    digits = re.sub(r"[^\d]", "", text or "")
    return int(digits) if digits else None


def parse_cards(html, portal):
    tree = LexborHTMLParser(html)
    id_re = re.compile(portal["id_pattern"])
    found = {}
    for card in tree.css(portal["card"]):
        link = card.css_first(portal["link"])
        if link is None:
            continue
        href = link.attributes.get("href") or ""
        m = id_re.search(href)
        if not m or m.group("id") in found:
            continue
        text = card.text(separator=" ", strip=True)
        price_el = card.css_first(portal["price"])
        price = to_int(price_el.text(strip=True)) if price_el else None
        beds_m = BEDS.search(text)
        beds = int(beds_m.group(1)) if beds_m else (0 if "studio" in text.lower() else None)
        found[m.group("id")] = {
            "id": m.group("id"),
            "url": urljoin(portal["base"], href),
            "price": price,
            "beds": beds,
            "text": text[:400],
        }
    return list(found.values())

The to_int helper strips everything but digits, so "$2,150/mo" becomes 2150. Watch for price ranges ("$1,900 - $2,300") on buildings with several units: the helper would glue those into one huge number. If a portal shows ranges, split on the dash and keep the lower figure.

Relisted units are the one case ID matching misses. A landlord who deletes and re-posts a listing to bump it to the top gets a new ID, and you get a second alert. If that bothers you, add a secondary key built from the normalised address plus bed count, and skip alerts for a key you've seen in the last week.

The seen store is one SQLite table with a primary key on portal plus listing ID. INSERT OR IGNORE does the detection: if the row went in, the listing is new. Keying on the portal rather than the search means a flat that matches two of your saved searches alerts once.

import sqlite3
import time

db = sqlite3.connect("seen.db")
db.execute("""
CREATE TABLE IF NOT EXISTS seen (
    portal      TEXT NOT NULL,
    listing_id  TEXT NOT NULL,
    first_seen  REAL NOT NULL,
    price       INTEGER,
    url         TEXT,
    PRIMARY KEY (portal, listing_id)
)""")
db.execute("CREATE TABLE IF NOT EXISTS seeded (search TEXT PRIMARY KEY)")


def is_seeded(search_name):
    row = db.execute("SELECT 1 FROM seeded WHERE search = ?", (search_name,)).fetchone()
    return row is not None


def record(portal, listings):
    new = []
    with db:
        for item in listings:
            cur = db.execute(
                "INSERT OR IGNORE INTO seen VALUES (?, ?, ?, ?, ?)",
                (portal, item["id"], time.time(), item["price"], item["url"]),
            )
            if cur.rowcount == 1:
                new.append(item)
    return new


def mark_seeded(search_name):
    with db:
        db.execute("INSERT OR IGNORE INTO seeded VALUES (?)", (search_name,))

The first run is where most homemade alert bots annoy their owners. A new search has never seen anything, so every card on page 1 looks new and you get 25 notifications for flats that have been on the market for a week. The fix is a seed pass: the first time a search runs, record every ID without alerting, then mark the search as seeded. Alerts start from the second poll.

Price drops are a cheap extra once the table exists. Before record, read the stored price for IDs you already know, and alert when the new price is lower by more than a threshold you choose. Update the row afterwards.

Portal filters are coarse. They'll do a rent ceiling and a bed count, but rarely "no ground floor", "not a sublet", or "only these three neighbourhoods inside the city boundary". Run those rules locally on the card text, after detection, so a listing that fails today is still marked seen and won't alert tomorrow either.

def passes(item, rules):
    if item["price"] is None:
        if not rules.get("alert_unknown_price", True):
            return False
    elif item["price"] > rules["max_price"]:
        return False
    if item["beds"] is not None and item["beds"] < rules.get("min_beds", 0):
        return False
    text = item["text"].lower()
    if any(word in text for word in rules.get("exclude", [])):
        return False
    areas = rules.get("areas") or []
    if areas and not any(area in text for area in areas):
        return False
    return True

Keep the price rule a little looser than the portal filter, or drop the portal's ceiling entirely and filter here. Depending on how a portal builds its price filter, a listing with no stated rent may never appear in a filtered search, and filtering locally lets you decide what happens to those.

For coordinates, some portals put latitude and longitude in a data attribute or in embedded JSON. If yours does, a point-in-polygon check against a hand-drawn area beats any keyword match, because listing text spells neighbourhood names inconsistently.

Email is the thing you're trying to beat, so push straight to the phone. ntfy is the lowest-effort option: you publish with an HTTP POST and subscribe to the topic in its app. Per ntfy's publishing docs (read 30 October 2026), the Title, Priority, Tags and Click headers set the notification title, a priority from 1 (min) to 5 (max), emoji tags, and the URL opened on tap. The same page warns that on the public server "the topic is essentially a password", so pick a long random topic name.

import os

import requests

NTFY_TOPIC = os.environ["NTFY_TOPIC"]   # long random string, treat it as a secret


def push(item, search_name):
    price = f"${item['price']:,}" if item["price"] else "price n/a"
    beds = "studio" if item["beds"] == 0 else f"{item['beds']} bd" if item["beds"] else "beds n/a"
    body = f"{price} | {beds}\n{item['text'][:200]}"
    requests.post(
        f"https://ntfy.sh/{NTFY_TOPIC}",
        data=body.encode("utf-8"),
        headers={
            "Title": f"New listing: {search_name}"[:120],
            "Priority": "4",
            "Tags": "house",
            "Click": item["url"],
        },
        timeout=10,
    ).raise_for_status()

Header values travel as Latin-1 in requests, so keep titles plain ASCII; put anything fancier in the body. Telegram works just as well if you'd rather get a chat message: call sendMessage on https://api.telegram.org/bot/ with chat_id and text, and keep the text under the documented limit of 4096 characters.

Send operator alerts (blocks, parse guards) to a different topic with a lower priority. Mixing "the scraper is broken" with "a flat just listed" trains you to ignore both.

Your worst-case alert delay is roughly the poll interval plus one fetch. A three-minute interval means you hear about a listing within about four minutes of it appearing. Going to one minute triples the request count for a small gain, and it's the kind of rate that gets noticed. Start at three to five minutes for hot searches and fifteen for slow ones.

Don't poll on exact clock minutes. A cron entry that fires at :00, :03, :06 produces a request pattern no human makes. Give each search its own next-due time with random jitter, and back off exponentially when a search fails.

import random
import time

import yaml

cfg = yaml.safe_load(open("searches.yaml", encoding="utf-8"))
state = {s["name"]: {"due": time.time() + random.uniform(0, 60), "fails": 0}
         for s in cfg["searches"]}


def schedule_next(search, ok):
    st = state[search["name"]]
    base = search["every_min"] * 60
    if ok:
        st["fails"] = 0
        st["due"] = time.time() + base * random.uniform(0.75, 1.25)
    else:
        st["fails"] += 1
        delay = min(base * 2 ** st["fails"], 3600)
        st["due"] = time.time() + delay * random.uniform(0.9, 1.1)


def run_forever(poll_once):
    while True:
        now = time.time()
        for search in cfg["searches"]:
            if state[search["name"]]["due"] <= now:
                try:
                    poll_once(search)
                    schedule_next(search, ok=True)
                except Exception as exc:
                    print(f"[{search['name']}] {exc}")
                    schedule_next(search, ok=False)
        time.sleep(5)

Rotation follows the same logic. Each poll is an independent page view, so SparkProxy's Random Rotation (a fresh IP per request, the default gateway behaviour) is the right mode: no single address builds a long history of hitting one search URL every three minutes. Pin the country in the proxy username, or with country_code on the API, to the portal's own market, since some portals geo-redirect or block foreign traffic.

Use Sticky Sessions (the same IP for 5 minutes) only when a portal needs two requests to look like one visitor. The common case is a search page that sets a cookie and a JSON endpoint that rejects requests without it. Make both calls with one requests.Session routed through the Sticky Sessions port in your dashboard (11002 on current accounts), and they share both the cookie and the exit IP. What is a sticky session proxy explains the mechanics.

Concurrency barely matters here. Six searches polled sequentially use one connection at a time. The Starter proxy plan's 100 threads, or the Starter API plan's 50 concurrent requests, are far beyond what an alert job needs.

The dangerous failure in an alert scraper is silence. A block page, a cookie wall, or a redesign all produce the same output from a naive parser: zero new listings. That looks exactly like a quiet afternoon, and you won't find out until you notice a week without alerts. The poll_once function below treats "too few cards" and "too many new IDs" as errors, not results.

class ParseGuard(Exception):
    pass


def poll_once(search):
    portal = cfg["portals"][search["portal"]]
    if portal["render"]:
        html = fetch_api(search["url"], render=True, wait_for=portal["card"])
    else:
        html = fetch_proxy(search["url"])
    listings = parse_cards(html, portal)

    if len(listings) < search.get("min_cards", 3):
        operator_alert(f"{search['name']}: only {len(listings)} cards parsed")
        raise ParseGuard("card count below floor")

    if not is_seeded(search["name"]):
        record(search["portal"], listings)
        mark_seeded(search["name"])
        return

    new = record(search["portal"], listings)
    if len(new) >= max(5, int(0.8 * len(listings))):
        operator_alert(f"{search['name']}: {len(new)}/{len(listings)} IDs new, alerts held")
        return

    for item in new:
        if passes(item, search["rules"]):
            push(item, search["name"])


def operator_alert(message):
    requests.post(f"https://ntfy.sh/{os.environ['NTFY_OPS_TOPIC']}",
                  data=message.encode("utf-8"), headers={"Priority": "2"}, timeout=10)

Set min_cards per search to a bit under the usual page-1 count. A narrow search in a slow market might legitimately show only four results, so a fixed global floor would false-alarm.

Here's how to read the symptoms and which ones a proxy fixes:

Keep one raw HTML snapshot of every failed fetch on disk. When a parse guard fires, the saved page tells you in ten seconds whether you're looking at a block page or a redesign.

The arithmetic is simple. A search polled every three minutes runs 480 times a day, about 14,400 times in a 30-day month. Six searches make 86,400 fetches a month. Prices below are SparkProxy's published list prices in US dollars, read 30 October 2026, excluding VAT/GST.

Failed jobs are refunded, so blocks don't inflate the credit column. Batch mode is worth a look for plain-HTML portals: the docs say you can pass several comma-separated URLs in url with render_js=false, and "the whole batch costs 1 credit regardless of URL count." Polling all six searches in one call per cycle would be about 14,400 credits a month. The docs don't describe how batch results come back, so test the response shape on your account before you build on it, and note that batching forces every search onto the same interval.

The proxy plan costs more than the Starter API plan for this job alone, but it's billed by threads, not traffic, so the same plan also covers any other scraping you run. Proxy plans take 5%, 10% or 15% off for 3, 6 or 12-month terms. To try either route free, the proxies have a 24-hour trial with no credit card, and the API gives 1,000 free credits, roughly eight hours of six plain-fetch searches at a three-minute interval.

Run the loop on any always-on box. A small systemd unit restarts it after crashes and reboots:

# /etc/systemd/system/listing-alerts.service
[Unit]
Description=New rental listing alert scraper
After=network-online.target

[Service]
WorkingDirectory=/opt/listing-alerts
EnvironmentFile=/opt/listing-alerts/.env
ExecStart=/opt/listing-alerts/venv/bin/python alerts.py
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target

If you later want the same pipeline to track rent levels rather than individual listings, the approach in our automated price monitoring guide carries over.