Web Scraping With Cheerio and Node.js

Cheerio is a server-side HTML parser with a jQuery-style API. You hand it a string of HTML, it builds a document tree, and you query that tree with CSS selectors. Under the hood it uses parse5 and htmlparser2 for parsing and css-select for matching, so it is small, quick, and dependency-light. Cheerio 1.0 shipped as a stable release in 2024 with first-class ESM support and a cleaner API than the old 0.x line.

It helps to be precise about what Cheerio is not. It is not an HTTP client, so it never downloads a page. It is not a browser, so it never runs JavaScript, applies CSS, or fires events. That narrow scope is the point: parsing is the only job, and it does that job in milliseconds.

If a target serves its data in the initial HTML response, Cheerio is the right tool and it will run circles around a headless browser. If the data only appears after JavaScript executes, you render the page first (covered below) and still let Cheerio do the extraction. For the wider picture of how scraping fits together, see what web scraping is. Cheerio is the Node.js counterpart to Java's jsoup, so if you also work in the JVM the patterns in web scraping in Java with jsoup map across almost line for line.

---

One package, no browser binaries.

npm install cheerio

Load an HTML string and you get back a query function, conventionally named $, bound to that one document:

import * as cheerio from 'cheerio';

const html = `
  <ul class="products">
    <li class="product" data-id="a1"><a href="/p/a1">Widget</a><span class="price">$9.99</span></li>
    <li class="product" data-id="b2"><a href="/p/b2">Gadget</a><span class="price">$19.99</span></li>
  </ul>`;

const $ = cheerio.load(html);
console.log($('.product').length); // 2

Cheerio 1.0 is ESM-first, so import * as cheerio from 'cheerio' is the modern form. It still ships a CommonJS build, so const cheerio = require('cheerio') works too; both expose the same cheerio.load. The $ you get back is not global jQuery. It is a small function scoped to this specific document, which is what lets you parse many pages in parallel without them stepping on each other.

For a full HTML page (with , , ), cheerio.load(html) is correct. If you only have a fragment and do not want Cheerio to wrap it in a document skeleton, pass cheerio.load(fragment, null, false) to skip the implied html/body wrapping.

---

Cheerio needs a string, and getting that string is a separate step. Use any HTTP client. axios is the common pick:

import axios from 'axios';
import * as cheerio from 'cheerio';

const { data: html } = await axios.get('https://www.sparkproxy.io/blog', {
  headers: { 'User-Agent': 'Mozilla/5.0 (compatible; SparkProxyBot/1.0)' },
  timeout: 15000,
});

const $ = cheerio.load(html);
console.log($('title').text());

Node's built-in fetch (stable since Node 21, standard in Node 22 LTS) or node-fetch works the same way:

const res = await fetch('https://www.sparkproxy.io/blog', {
  headers: { 'User-Agent': 'Mozilla/5.0 (compatible; SparkProxyBot/1.0)' },
});
const $ = cheerio.load(await res.text());

One habit saves a lot of confusion: always set a real User-Agent. Many servers return a 403 or an empty body to clients that send the default axios or Node user agent, and then you spend twenty minutes debugging a selector that was never going to match because the HTML was blank to begin with. When you scale past a handful of requests, that same fetch step is where a proxy goes, which is section 8.

---

Selection is pure CSS. If you can write it in document.querySelectorAll, Cheerio understands it.

$('h1')                         // by tag
$('.price')                     // by class
$('#main')                      // by id
$('a[href^="/p/"]')             // attribute starts-with
$('ul.products > li.product')   // direct child
$('li:first-child')             // structural pseudo-class
$('.product:has(.price)')       // relational

Then traverse from any match with the jQuery methods you already know:

const firstProduct = $('.product').first();

firstProduct.find('a');      // descendants matching a selector
firstProduct.children();     // direct children
firstProduct.parent();       // parent element
firstProduct.next();         // next sibling
firstProduct.closest('ul');  // nearest ancestor matching

Cheerio matches with css-select, which means CSS selectors only. It does not support XPath. That is rarely a real limit, because CSS combinators plus .find(), .parent(), .next(), and .closest() cover nearly everything XPath does for extraction work.

---

Once you have a match, four methods pull out almost everything you need:

const first = $('.product').first();

first.find('a').text();          // "Widget"  -> visible, decoded text
first.find('.price').text();     // "$9.99"
first.attr('data-id');           // "a1"      -> a single attribute
first.find('a').attr('href');    // "/p/a1"
first.html();                    // inner HTML of the element
$.html(first);                   // outer HTML, tag included

.text() returns HTML-entity-decoded text, so & comes back as & and é as an accented e. .attr('name') reads one attribute; calling .attr() with no argument returns a plain object of every attribute on the first match.

Here is the extraction gotcha that bites everyone once. .text() on a selection that matches multiple elements returns the concatenation of all of them with no separator:

$('.price').text();   // "$9.99$19.99"  <-- both prices mashed together, not just the first

That is by design (jQuery behaves the same way), but it surprises people who expect the first match. When you want one field, narrow to a single element with .first() before calling .text(). When you want every field, iterate, which is the next section.

---

Two idioms cover almost all iteration. Use .each() when you want side effects, and .map() when you want to build an array.

// .each: run a callback per element, push into your own array
const products = [];
$('.product').each((i, el) => {
  const $el = $(el);
  products.push({
    id: $el.attr('data-id'),
    name: $el.find('a').text().trim(),
    price: $el.find('.price').text().trim(),
  });
});

// .map: transform matches into an array in one expression
const names = $('.product a')
  .map((i, el) => $(el).text().trim())
  .get();   // -> ["Widget", "Gadget"]

Two details make or break this. Inside the callback, el is a raw DOM node, not a Cheerio object, so wrap it with $(el) before you call .find(), .attr(), or .text() on it. And .map() returns a Cheerio collection, not a JavaScript array, so you finish with .get() (or .toArray()) to materialize a plain array. Forgetting .get() is the single most common reason a .map() result "looks wrong" when you log it.

---

Cheerio returns the href exactly as written in the HTML. Sites almost always write links relative to the page, so you get /p/a1 or ../reviews?page=2, not a full URL. Turn them absolute with the WHATWG URL constructor, which every modern Node version ships globally:

const base = 'https://www.sparkproxy.io/blog';

const links = $('a')
  .map((i, el) => {
    const href = $(el).attr('href');
    return href ? new URL(href, base).href : null;
  })
  .get()
  .filter(Boolean);

new URL(href, base) resolves against the base for you and handles every form correctly: root-relative (/p/a1), path-relative (../x), query-only (?page=2), protocol-relative (//cdn.sparkproxy.io/img.png), and already-absolute URLs (which it passes through unchanged). Do not build links by string-concatenating the base and the href. That breaks the moment a path starts with /, contains .., or is already absolute, and it silently produces malformed URLs you will chase later.

---

This is where people look for a Cheerio proxy option and never find one, because there is nothing to find. Cheerio has no network layer. You proxy the HTTP request, then hand the returned HTML to Cheerio unchanged.

import axios from 'axios';
import * as cheerio from 'cheerio';
import { HttpsProxyAgent } from 'https-proxy-agent';

const agent = new HttpsProxyAgent('http://user:pass@proxy-1.sparkproxy.io:10000');

const { data: html } = await axios.get('https://www.sparkproxy.io/blog', {
  httpsAgent: agent,
  proxy: false, // let the agent own the tunnel; disable axios's own proxy layer
  headers: { 'User-Agent': 'Mozilla/5.0 (compatible; SparkProxyBot/1.0)' },
  timeout: 15000,
});

const $ = cheerio.load(html);
const titles = $('h2.post-title').map((i, el) => $(el).text().trim()).get();

The proxy: false line matters. axios 1.7+ ships its own proxy handling that competes with the agent, so without it your request can route around the tunnel. Use https-proxy-agent for https:// targets even though the proxy URL itself begins with http://; that scheme describes how you reach the proxy, not the destination. At volume you rotate across many IPs so no single address trips a rate limit. The mechanics of that (round-robin, weighted, sticky sessions, retry with backoff) live in how to rotate proxies in Node.js, and the reasons requests get blocked in the first place are in how to avoid getting your proxy blocked. Cheerio itself never changes across any of it: proxy the fetch, parse the result.

---

Cheerio parses the HTML you give it and stops. There is no JavaScript engine, no CSS engine, and no second network round to load sub-resources. Two consequences follow, and understanding both is what separates a working scraper from an hour of confusion.

First, content that a site hides with CSS is still fully scrapable. Cheerio never applies stylesheets, so an element with style="display:none", an off-screen tab panel, or a JSON blob tucked inside a data- attribute is right there in the tree and your selector will match it. Data being invisible in a browser does not mean it is invisible to Cheerio.

Second, and the opposite trap: content that JavaScript injects after the page loads is simply not present. A React or Vue single-page app often returns little more than

in its initial HTML, and the product grid you see in the browser is drawn client-side after hydration. Cheerio loads that near-empty shell, your selectors match nothing, and everything looks broken even though the code is correct.

There is a fast way to tell which case you are in. Open the page with view-source: in your browser, or curl the URL, and search the raw HTML for the value you want. If it is in the source, Cheerio can extract it. If it only appears in the rendered DOM under DevTools Elements, the data is JS-generated and you need to render the page before Cheerio ever sees it. That rendering step is covered next, and the broader topic is in how to scrape dynamic JavaScript websites.

---

When the data is JavaScript-injected, you have two options: run your own headless browser, or let a rendering API do it. The SparkProxy Scraping API runs a real headless Chromium server-side, waits for the content to appear, and returns fully rendered HTML. You then parse that HTML with Cheerio exactly as before. You get browser-grade rendering without running or scaling a browser fleet, and you keep Cheerio's fast, expressive extraction.

import axios from 'axios';
import * as cheerio from 'cheerio';

const { data: html } = await axios.get('https://scrape.sparkproxy.io/api/v1', {
  headers: { 'X-API-Key': process.env.SPARKPROXY_API_KEY },
  params: {
    url: 'https://www.sparkproxy.io/pricing',
    render_js: true,          // run headless Chromium so the SPA hydrates
    wait_for: '.price-card',  // wait until the elements you need exist
    country_code: 'us',       // geo-target the exit IP
    premium_proxy: true,      // route through residential IPs for tough targets
  },
  timeout: 60000,
});

// Same Cheerio code as a static page. Only the fetch changed.
const $ = cheerio.load(html);
const plans = $('.price-card').map((i, el) => ({
  name: $(el).find('.plan-name').text().trim(),
  price: $(el).find('.plan-price').text().trim(),
})).get();

The request goes to https://scrape.sparkproxy.io/api/v1 with your key in the X-API-Key header. render_js: true runs the browser, wait_for blocks until the given selector exists (so you do not parse a half-loaded page), and country_code plus premium_proxy control the exit IP. The response body is the rendered HTML, which drops straight into cheerio.load. Notice the extraction block is identical to a static-page scrape: render_js is heavier on credits than a plain fetch, so use it only for pages that truly need JavaScript, and keep Cheerio for the parsing either way. For the full parameter list, see the Scraping API docs.

---

---