How to Set Up Residential Proxies in Node.js with Axios, Fetch and Got

Getting a single request through a proxy in Node is a ten line problem. Every tutorial ends there, and then the project grows: a second HTTP client appears, credentials get hardcoded in three files, someone needs Germany instead of the United States, and nobody can answer whether the requests are actually leaving from where they are supposed to.

This is a setup guide rather than a syntax guide. If what you need is the exact agent configuration each client expects, including the reason Axios needs proxy: false alongside its agent, that is covered in detail in using residential proxies in Node.js with Axios and Got. What follows is how to arrange a project so that wiring stays correct as the project grows.

Decide which clients you actually support

Node has three HTTP clients in common use and they take proxies through different mechanisms. Supporting all three is normal in a codebase of any age, but it should be a decision rather than an accident.

Axios needs an explicit agent, because its own proxy option does not handle HTTPS tunneling the way people expect. Install https-proxy-agent.

Got takes the agent in an agent.https slot rather than a top-level option.

Native fetch goes through undici, so it needs ProxyAgent as a dispatcher. Undici ships with Node, but you will want it as an explicit dependency if you are constructing dispatchers yourself.

The practical consequence is that your dependency list is driven by which clients you keep, so keeping two clients means maintaining two proxy paths. Most codebases would be better off standardizing on one and converting the stragglers, and the ones that genuinely need two should at least know why.

Credentials belong in the environment, not the source

The gateway takes targeting in the username, which means the credential string and the routing configuration are the same string. That is convenient, and it is also how proxy credentials end up committed.

Keep the pieces separate in configuration:

PROXY_HOST=p.shifter.io
PROXY_PORT=443
PROXY_USER=customer-USERNAME
PROXY_PASS=your-password
PROXY_COUNTRY=us

Then assemble the username at runtime. The rule that saves you later is that no file outside your proxy module should contain a literal proxy URL. Once a credential string with embedded targeting appears inline in a scraper, it gets copied, and the copies drift.

Do not put the password in a URL you log. Proxy URLs are the single most common way credentials reach a log aggregator, because the natural debug line is the whole URL.

One module builds the agents

The structural decision that matters most is having exactly one place that turns configuration into an agent. It is a small module and it prevents a large class of problems.

import { HttpsProxyAgent } from "https-proxy-agent";

const { PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS } = process.env;

export function proxyUrl({ country, session, ttl } = {}) {
  const parts = [PROXY_USER];
  if (country) parts.push("country", country);
  if (session) parts.push("sid", session);
  if (session && ttl) parts.push("ttl", String(ttl));
  const user = parts.join("-");
  return `https://${user}:${PROXY_PASS}@${PROXY_HOST}:${PROXY_PORT}`;
}

export function agentFor(opts) {
  return new HttpsProxyAgent(proxyUrl(opts));
}

Two details in that snippet are worth stating explicitly. Targeting flags are built from named options rather than by concatenating strings at each call site, which is what stops a typo in a flag name from becoming a 407 that looks like a credentials problem. And ttl is only added when a session is present, because a time to live has nothing to keep alive without a session identifier.

Callers then never see a URL:

const agent = agentFor({ country: "de", session: "job-141", ttl: 600 });

// axios
await axios.get(url, { httpsAgent: agent, proxy: false });

// got
await got(url, { agent: { https: agent } });

Native fetch is the one that does not take an agent, since undici wants a dispatcher instead:

import { ProxyAgent, fetch } from "undici";
import { proxyUrl } from "./proxy.js";

const dispatcher = new ProxyAgent(proxyUrl({ country: "de" }));
await fetch(url, { dispatcher });

Keeping proxyUrl exported separately is what lets the fetch path share configuration with the other two without duplicating the string building.

Reuse agents, and decide sessions up front

An agent holds a connection pool. Constructing one per request throws that pool away every time, which shows up as latency you will misdiagnose as the network.

Build agents once per configuration you use and cache them. If you are rotating through five countries, that is five agents held for the life of the process, not one per request.

Sessions are the related decision. Without a session identifier the gateway rotates, which is what you want for independent requests. With sid you get a sticky IP, held for the ttl you specify, which is what you want for anything with continuity: a paginated result set, a multi-step flow, anything where the second request needs to come from the same place as the first. Default sticky time is 120 seconds if you do not set one.

Choosing this at setup time rather than per call site is the difference between a coherent rotation policy and a codebase where half the requests happen to be sticky. The trade-offs are laid out in sticky vs rotating residential proxies.

Verify before you build on it

Write the verification script before the scraper. It takes five minutes and it turns an entire category of future confusion into an immediate answer.

import { agentFor } from "./proxy.js";
import axios from "axios";

const agent = agentFor({ country: "de" });
const { data } = await axios.get("https://api.ipify.org?format=json", {
  httpsAgent: agent,
  proxy: false,
});
console.log(data);

If that prints your own address, the agent is not attached and every request in the project is going out direct. This is a genuinely common state to be in for days, because nothing fails: the code works, it is just not using the proxy. Checking the country actually matches what you requested is the second half of the check, and the method for doing that properly is in testing proxy speed, success rate and location accuracy.

Make it a script in package.json so anyone can run it when something looks wrong.

Map the errors once

Proxy errors are specific enough to be diagnosed automatically, and doing that in one place beats interpreting them repeatedly at three in the morning:

• 407 means the credentials are wrong or a targeting flag is malformed. A misspelled flag lands here, which is why it is worth checking your flag names before your password.
• 502 means no exits match your filter. The filter is too narrow rather than the network being down.
• 509 means the bandwidth allowance is exhausted.
• Connection refused usually means a legacy host or port left over from an older configuration.

Only transient failures deserve a retry. Retrying a 407 in a loop burns your rate limit against a problem that will never resolve on its own, and retrying a 509 does nothing at all. Wrap retries in real backoff rather than a fixed delay, as covered in rate limiting and request throttling.

Bound concurrency deliberately

Node will happily start ten thousand requests, and none of the three clients will stop you. Under a proxy this is worse than usual, because every one of those requests is a tunnel that has to be established.

Use a concurrency limiter from the start rather than adding one after the first incident. A modest limit with steady throughput outperforms an unbounded burst that triggers defenses and then spends its time retrying.

Dev, CI and production differ

Three practical notes that come up in every Node project once proxies are in it.

Residential traffic is billed by bandwidth, so a test suite that hits real targets through the proxy is a recurring bill for no benefit. Mock the HTTP layer in unit tests and keep a small number of real requests in a separate, manually triggered check.

Local development should use the same module and the same environment variables as production, with different values. Configuration that only exists in production is configuration nobody has tested.

And container images should not bake credentials. Pass them at runtime, the same as any other secret.

FAQ

Do I need https-proxy-agent if I only use fetch?

No. Undici’s ProxyAgent covers that path. You need the separate agent package for Axios and Got.

Why does Axios need proxy: false when I have already given it an agent?

Because Axios will otherwise try to apply its own proxy handling on top of the agent, and the two do not compose. Setting it false hands the work entirely to the agent.

Can I set targeting per request instead of per agent?

You can, but each distinct configuration is a distinct agent, so building one per request is what costs you the connection pool. Cache them by configuration key.

Should the country be a config value or a per call argument?

Both, in practice. A default in configuration, overridable per call for the jobs that need a specific region. What you want to avoid is the country appearing as a literal inside individual scrapers.

The bottom line

The proxy wiring in Node is small and well understood. What determines whether a project stays maintainable is the arrangement around it: one module that builds agents from configuration, credentials that live in the environment, agents reused rather than rebuilt, a session policy chosen deliberately, a verification script that exists before the scraper, and error handling that knows the difference between a filter that is too narrow and a password that is wrong.

Set that up once and adding a client library or a new region is a configuration change. Skip it and every new requirement is a search through the codebase for hardcoded strings. Gateway details are on the residential proxies page, with bandwidth rates on the pricing page.