This is the pre-flight list for a first residential proxy project: the things to confirm, in the order that makes each one cheap to fix. It assumes you know roughly what a proxy is, and if you do not, start with residential proxies for beginners and come back.
Work down it and you will avoid the failures that usually cost a first project several weeks: a plan sized for the wrong workload, a job that looks like it works while collecting nothing, and a bill nobody predicted.
Before you write code
1. Confirm you actually need residential. If your targets do not scrutinise traffic, cheaper infrastructure will do the same job. Test one target from a plain connection and from a datacenter address first; if both work, you have saved yourself the premium. The comparison is in residential versus datacenter.
2. Decide rotating or static. Rotating for collection at volume, static ISP addresses when something needs to persist, such as an account or a long session. Getting this wrong is not fixable by buying more of the wrong one, per shared versus dedicated.
3. List your markets. Which countries, and whether any work needs city precision. This determines both your targeting and whether pool depth in those markets is something to check, per country availability.
4. Estimate bandwidth before choosing a plan. Response size times request volume, plus headroom for retries. The single biggest factor is whether you fetch data endpoints or render full pages, which can differ by a hundredfold, so settle that first, per estimating monthly bandwidth.
First connection
5. Get a bare request working. No targeting flags, no session, nothing clever:
curl -x customer-USERNAME:PASSWORD@p.shifter.io:443 https://ipinfo.io/jsonIf this fails, nothing else matters yet. A 407 means credentials or a malformed username; strip everything and add it back one piece at a time, per fixing 407 errors.
6. Confirm rotation. Run that command several times and check the address changes. If it does not, and especially if it does from the command line but not from your code, the cause is almost certainly connection reuse in your HTTP client rather than the proxy, per IP not rotating.
7. Confirm geography. Request a country and verify two things: that the exit address reports that country, and more importantly that a geo-sensitive target behaves as though you are there, with local currency and language. Databases and a target’s own opinion do not always agree, and the target’s is the one that matters.
8. Set both proxy entries and handle special characters. In code, set the HTTP and HTTPS entries, not just one, and percent-encode a password containing characters that mean something in a URL. The format reference is in how to connect.
Before you scale up
9. Match your request to your exit. Send a full browser-shaped header set rather than a bare User-Agent, and tie Accept-Language to the exit country so the two cannot contradict each other. If you drive a browser, set locale and timezone to match as well. Details in setting the right headers.
10. Validate response bodies, not status codes. This is the check that separates a working pipeline from one that quietly collects nothing. Define, per target, a marker that only appears on a genuinely good page, and count a response as success only if it passes. A challenge page returning 200 is the most expensive failure in this work, per detecting blocked or fake content.
11. Add pacing and sane retries before volume, not after. A per-target rate limit with jitter, a concurrency cap, and retry logic that classifies failures rather than looping. Specifically: wait on rate signals rather than rotating addresses to keep the same pace, and never retry a terminal error. See rate limiting and throttling and retry logic.
12. Put a cost guardrail in place. Track bytes per request and set an alert well before your plan allocation, because the common first-project surprise is a rendering job consuming a month of bandwidth in three days. The levers are in cutting bandwidth costs.
The five-minute verification
Before leaving anything running unattended, confirm all of these in one short run:
- The exit address is not your own
- It changes between requests when you have not asked for a session
- The country matches what you requested, and the target agrees
- A deliberately bad request is classified as a failure rather than counted as success
- Bytes per request is roughly what you estimated
- A rate limit response causes a wait rather than an immediate retry
If any of those is wrong, fix it now. Each becomes considerably more expensive once there is a month of data built on top of it.
Common first-project mistakes
Four that account for most of the trouble.
Going full speed immediately. New pipelines are usually written to run as fast as the code allows, which is the fastest route to being blocked. Start slow and increase deliberately.
Trusting status codes. Covered above, and worth repeating because it is the one people skip and the one that corrupts a dataset silently.
Rendering pages that did not need rendering. Check whether the data is available from an underlying endpoint before automating a browser, per when you need a headless browser.
Treating a sticky session as guaranteed. Sessions are best effort on real household connections, so a flow must tolerate the address changing mid-sequence, per sticky versus rotating.
The bottom line
Confirm you need residential at all, pick rotating or static deliberately, list your markets, and size bandwidth before choosing a plan. Then get a bare request working before adding anything, confirm rotation and geography, and set both proxy entries in code. Before scaling, make your headers agree with your exit, validate response bodies rather than status codes, add pacing and classified retries, and put a cost alert in place. Run the five-minute verification before leaving anything unattended. Almost every expensive first-project failure is one of those steps skipped, and each is cheaper to do now than to retrofit.
The product these steps configure is residential proxies, one gateway with country and city targeting and sticky sessions when a flow needs one, billed per GB so a small first project stays a small first bill.