Most comparisons between residential proxy networks are decided before the first request is sent. One provider is tested on Monday and another on Friday. One gets an easy target, another a hard one. One is judged on status codes, another on whether the page actually loaded. The result looks like a measurement, but it mostly reflects how the test was set up.
This guide is about setting it up properly. Whether you are trialling Shifter’s residential proxies against your current provider or choosing between three networks you have never used, the rules are the same, and they are the rules Atlas follows for every network it tests.
The one rule: same requests, same targets, same window
Everything else in this guide follows from one principle. Every network in the comparison must receive the same requests, sent to the same targets, from the same locations, at the same time.
The “same time” part is the one people skip. Target sites change their defences hour to hour. A residential pool is a moving set of consumer connections that grows and shrinks through the day. If you test network A at noon and network B at midnight, you are comparing two moments, not two networks. Run them concurrently, or interleave their requests in the same run, so whatever the target and the internet are doing affects both equally.
Atlas applies this as a hard condition: every network gets the same schedule, and results are grouped into time slots that each weigh the same, so a network cannot look better simply by being tested at a quiet hour. The window rules on the methodology page spell out exactly how.
Write the pass rule before you test
A comparison needs a definition of success that does not depend on the network. Decide it per target, in writing, before you send anything.
- For an HTML page, success is a specific element you can parse: a price, a product title, a result list with at least one result.
- For an API, it is a JSON response with the fields you need.
- For a search page, it is results, not an empty page or a consent wall.
Everything that fails the rule is a failure, whatever its status code. This matters because a CAPTCHA or interstitial usually arrives as HTTP 200. A test that counts status codes will report near-perfect success for networks that are being challenged on a large share of requests. Classify each failure as you go (challenge, block, timeout) because the mix tells you more than the total.
Pick targets that look like your work
An undefended test endpoint will tell you every network is excellent. That is true and useless. Differences between residential pools only appear on sites that actively defend themselves, and they appear differently on each one.
Choose a small set of targets that resemble what you will actually scrape or verify, across a range of difficulty: an open page as a control, a marketplace, a search engine, and the hardest site you depend on. Read the results per target, not only as an average. The success rate on each Atlas test target shows how much the order of networks can change from one site to the next.
Measure from the target’s side
Two things that look like details decide whether your results are real.
Record the exit IP the target saw. Have each request return or log the address that reached the target, not the address the provider’s dashboard reports. Dashboards can be cached, can describe the gateway rather than the exit, or can simply be wrong. Everything you later say about pool size, location accuracy and overlap depends on this one field.
Look the exit up yourself. Map each address to country, city and ASN with a public database at the time of the request. That is how you find out whether a pool sold as residential contains hosting addresses, or whether a request for one country exits in another.
Test at the volume you will run
Latency at idle is not latency at work. Many gateways are fast for a handful of requests and slow down noticeably at sustained concurrency, which is the condition you will actually be in.
Run at least part of the test at the concurrency you plan to use in production, and report percentiles rather than an average. The median tells you about a typical request; the slow tail tells you whether your job will finish on time. Keep each provider’s session and rotation settings at their defaults unless you will change them in production, so you measure the product as sold rather than a configuration nobody else gets.
Collect enough data to mean something
A few dozen requests per network will produce a ranking, but not one that survives a rerun. Small samples are dominated by chance: one bad minute on one target can swing the result.
Spread requests evenly across your targets and across a long enough window to include quiet and busy hours. Rerun the comparison on a second day before you act on a small difference. If the order flips, the networks are close, and price or support should decide it. Count the distinct exit IPs each network handed you while you are at it. If you plan to use two providers for redundancy, intersect those sets, because two networks that share most of their addresses are not two sources.
Compare cost last
Only once success has been measured fairly does price mean anything. Divide what each network would cost for your traffic by the number of requests that passed your rule. Failed requests still bill, so a low rate card with a low pass rate can be the most expensive option. Our guide to cost per successful request walks through the arithmetic.
A fair-test checklist
- Same requests, same targets, same locations, same window for every network.
- A written pass rule per target, decided before testing.
- Failures classified as challenge, block or timeout.
- Targets that resemble your real workload, including a hard one.
- Exit IP recorded from the target’s side and looked up independently.
- Part of the test at production concurrency, with latency percentiles.
- Provider defaults for session and rotation settings.
- Enough requests, over enough hours, to survive a rerun.
- Cost per successful request computed last.
If you want working code for the measurements themselves, our post on testing residential proxy speed, success rate and location accuracy has scripts you can adapt. Atlas runs this whole checklist continuously, and the side-by-side comparison page lets you put any networks it tests side by side before you spend a trial on them.