How to Use Residential Proxies in Puppeteer (Including page.authenticate)

Puppeteer drives real Chromium, which is exactly what you want when a target renders its content with JavaScript, gates data behind interaction, or fingerprints anything that is not a real browser. Adding a residential proxy is a couple of lines, but Puppeteer splits the job across two different places in a way that trips up almost everyone the first time: the proxy address goes in the launch arguments, and the credentials go somewhere else entirely.

This is the Puppeteer entry alongside residential proxies with Playwright; if you want plain HTTP clients instead of a full browser, see proxies in Node.js. Here we focus on the browser-specific traps.

Everything below uses Shifter’s residential gateway: one endpoint, p.shifter.io:443, with all targeting encoded in the username. Swap the host and credentials for another provider; the shape is the same.

The gateway model in one paragraph

The proxy username carries your authentication and your targeting. You do not switch endpoints to change country or session, you change the username string:

customer-USERNAME-country-us-sid-abc123-ttl-600

country-us targets the United States, sid pins a sticky session, ttl holds that IP for N seconds. Omit sid/ttl and every new connection rotates. The password stays constant. In Puppeteer, the host goes in a launch flag and that username goes into page.authenticate.

The setup: proxy in launch args, credentials in page.authenticate

Chromium takes the proxy server as a command-line flag, --proxy-server, passed through Puppeteer’s args. It will not accept user:pass@host there, Chromium does not read credentials from that flag. Instead you provide them per page with page.authenticate, which answers the proxy’s 407 challenge with the right Proxy-Authorization.

import puppeteer from 'puppeteer';

const browser = await puppeteer.launch({
  headless: true,
  args: ['--proxy-server=http://p.shifter.io:443'],   // host only, no credentials
});

const page = await browser.newPage();
await page.authenticate({
  username: `${process.env.SHIFTER_USER}-country-us`,  // targeting lives here
  password: process.env.SHIFTER_PASS,
});

await page.goto('https://api.ipify.org');
console.log(await page.evaluate(() => document.body.innerText));   // a US residential IP
await browser.close();

Two things to internalize. The username includes the targeting flags (-country-us), because the geo lives there, not in the URL. And page.authenticate is not optional for an authenticated proxy, without it every navigation dies on a 407. This split, host in the flag and credentials in page.authenticate, is the single most common Puppeteer-proxy mistake.

Rotating geo and sessions through one gateway

Here is the useful consequence of that design. The proxy host is fixed at browser launch and you cannot change it per page, but you do not need to. Because targeting lives in the username and page.authenticate is set per page, giving each page a different username routes it through a different identity on the same gateway. One browser, many geos, no relaunching.

async function pageFor(browser, country, sid) {
  const page = await browser.newPage();
  const user = `${process.env.SHIFTER_USER}-country-${country}`
    + (sid ? `-sid-${sid}-ttl-600` : '');
  await page.authenticate({ username: user, password: process.env.SHIFTER_PASS });
  return page;
}

const de = await pageFor(browser, 'de', 'job-42');   // German sticky session
const us = await pageFor(browser, 'us', null);        // rotating US

Give each logical unit of work its own sid and rotate between units, not mid-session (sticky vs rotating covers the distinction), and map work to identities the way the load-balancing post describes. For cookie and storage isolation between identities, put each on its own browser context:

const context = await browser.createBrowserContext();  // isolated cookies/storage
const page = await context.newPage();
await page.authenticate({ username: userFor('gb'), password: pass });

Note the name: recent Puppeteer calls this createBrowserContext(), older versions called it createIncognitoBrowserContext(). Same idea, renamed.

Trap 1: reuse the browser, never launch one per request

Launching Chromium is heavy, a fresh browser per request pays hundreds of milliseconds of startup plus real memory every time, the overhead the latency guide exists to remove. Launch one browser at startup and reuse it, opening pages or contexts for concurrent work and closing them when done. A pool of pages under one long-lived browser is the right shape.

Trap 2: block the resources you do not need

A browser fetches everything a real one does: images, fonts, media, stylesheets, analytics. If you only want the HTML or a few fields, that is bandwidth you are paying for and time you are waiting on. Puppeteer lets you intercept requests and abort the ones you do not need, which cuts both your page-load time and your bandwidth bill substantially.

await page.setRequestInterception(true);
page.on('request', (req) => {
  const blocked = ['image', 'font', 'media', 'stylesheet'];
  if (blocked.includes(req.resourceType())) req.abort();
  else req.continue();
});

Be selective: some sites will not render the content you want without their CSS or a specific script, so block aggressively, then confirm the data still appears. When it does, this is the cheapest speed-up available in a headless browser.

Trap 3: the browser is only half of not getting blocked

A residential IP handles the network half of looking human, but Puppeteer is still driving headless Chromium, and sites fingerprint the browser too, navigator.webdriver, headless-specific quirks, and automation signals. A clean IP with good reputation keeps you out of a lot of challenges, but it does not disguise an obviously automated browser. Use a current Puppeteer and Chromium so you get the modern headless mode rather than the old, easily-detected one, keep viewport and user-agent realistic, and drive the page at a human pace. The mistakes that trigger detection apply to the browser layer as much as the IP layer, and the two have to line up.

Trap 4: bound your concurrency

Every open page is a real browser tab holding real memory, so you cannot open thousands the way you might fire off HTTP requests. Keep a bounded pool of pages or contexts and reuse them, and cap in-flight work per target host so one fragile site is not hammered while a permissive one is starved. More parallelism past a target’s tolerance buys blocks and out-of-memory crashes, not throughput (how to avoid getting blocked).

Verify you are actually on the proxy

Before you benchmark or debug anything else, confirm the exit IP from inside the page:

await page.goto('http://ip-api.com/json');
console.log(await page.evaluate(() => document.body.innerText));  // expect the targeted country

Your own IP means the --proxy-server flag did not apply. A hang on a 407 dialog means page.authenticate is missing or the credentials are wrong. A general hang means local egress is blocked. All three are covered in the timeout diagnosis guide.

FAQ

Why does putting user:pass@host in --proxy-server not work? Chromium does not read proxy credentials from the --proxy-server flag. Pass only the host there and provide credentials with page.authenticate({ username, password }), which handles the proxy’s 407 challenge. Because the gateway encodes targeting in the username, the full username (with -country-...) goes into page.authenticate.

How do I use a different country per page if the proxy is set at launch? You do not change the proxy host, you change the page.authenticate username. Since targeting lives in the username and the gateway host is constant, each page can authenticate with a different username and exit through a different identity. One browser serves many geos.

Can I rotate the proxy without relaunching the browser? Yes, for identity and geo, by varying the page.authenticate username per page or context. You would only relaunch if you needed a genuinely different proxy host, which you do not with a single gateway endpoint.

How do I cut bandwidth in Puppeteer? Enable request interception and abort resource types you do not need (images, fonts, media, often stylesheets). Confirm the target still renders the data you want, then keep the blocks. It is the biggest single lever on both speed and cost in a headless browser.

Puppeteer or Playwright? Both drive real browsers and both work well with residential proxies. Playwright takes the proxy (with credentials) directly in its context options; Puppeteer splits it into the launch flag plus page.authenticate. Choose on ecosystem fit and existing code; the proxy concepts are the same.

The bottom line

Puppeteer plus residential proxies is straightforward once the split clicks: the proxy host goes in --proxy-server at launch, and the credentials, carrying your targeting in the username, go in page.authenticate per page. Rotate geo and sessions by varying that username rather than relaunching, isolate identities with browser contexts, reuse one long-lived browser, block the resources you do not need, and remember the browser fingerprint has to look as human as the IP does.

Get that right and Puppeteer handles the JavaScript-heavy targets that plain HTTP clients cannot. Point it at the residential gateway, and remember that pool quality decides how often you are challenged at all (IP reputation). The pricing page has the per-GB plans to test it against your own targets.