More unique IPs than any network on the planet, in the world's largest residential pool. That's Shifter.More unique IPs than any network, in the world's largest pool. That's Shifter.

See the numbers

How to Compare Residential Proxies: A Fair-Test Guide

The conditions that make a comparison between residential proxy networks fair, the mistakes that quietly decide the result before the test starts, and how to run a side-by-side test you can trust.

James Meadow Product Marketing · Updated November 15, 2026 · 6 min read
test windowABCtime
Same requests, same window. The amber lane ran at another time.
Buying
On this page
  1. The one rule: same requests, same targets, same window
  2. Write the pass rule before you test
  3. Pick targets that look like your work
  4. Measure from the target’s side
  5. Test at the volume you will run
  6. Collect enough data to mean something
  7. Compare cost last
  8. A fair-test checklist
  9. Frequently asked questions

Most comparisons between residential proxy networks are decided before the first request is sent. One provider is tested on Monday and another on Friday. One gets an easy target, another a hard one. One is judged on status codes, another on whether the page actually loaded. The result looks like a measurement, but it mostly reflects how the test was set up.

This guide is about setting it up properly. Whether you are trialling Shifter’s residential proxies against your current provider or choosing between three networks you have never used, the rules are the same, and they are the rules Atlas follows for every network it tests.

The one rule: same requests, same targets, same window

Everything else in this guide follows from one principle. Every network in the comparison must receive the same requests, sent to the same targets, from the same locations, at the same time.

The “same time” part is the one people skip. Target sites change their defences hour to hour. A residential pool is a moving set of consumer connections that grows and shrinks through the day. If you test network A at noon and network B at midnight, you are comparing two moments, not two networks. Run them concurrently, or interleave their requests in the same run, so whatever the target and the internet are doing affects both equally.

Atlas applies this as a hard condition: every network gets the same schedule, and results are grouped into time slots that each weigh the same, so a network cannot look better simply by being tested at a quiet hour. The window rules on the methodology page spell out exactly how.

Write the pass rule before you test

A comparison needs a definition of success that does not depend on the network. Decide it per target, in writing, before you send anything.

  • For an HTML page, success is a specific element you can parse: a price, a product title, a result list with at least one result.
  • For an API, it is a JSON response with the fields you need.
  • For a search page, it is results, not an empty page or a consent wall.

Everything that fails the rule is a failure, whatever its status code. This matters because a CAPTCHA or interstitial usually arrives as HTTP 200. A test that counts status codes will report near-perfect success for networks that are being challenged on a large share of requests. Classify each failure as you go (challenge, block, timeout) because the mix tells you more than the total.

Pick targets that look like your work

An undefended test endpoint will tell you every network is excellent. That is true and useless. Differences between residential pools only appear on sites that actively defend themselves, and they appear differently on each one.

Choose a small set of targets that resemble what you will actually scrape or verify, across a range of difficulty: an open page as a control, a marketplace, a search engine, and the hardest site you depend on. Read the results per target, not only as an average. The success rate on each Atlas test target shows how much the order of networks can change from one site to the next.

Measure from the target’s side

Two things that look like details decide whether your results are real.

Record the exit IP the target saw. Have each request return or log the address that reached the target, not the address the provider’s dashboard reports. Dashboards can be cached, can describe the gateway rather than the exit, or can simply be wrong. Everything you later say about pool size, location accuracy and overlap depends on this one field.

Look the exit up yourself. Map each address to country, city and ASN with a public database at the time of the request. That is how you find out whether a pool sold as residential contains hosting addresses, or whether a request for one country exits in another.

Test at the volume you will run

Latency at idle is not latency at work. Many gateways are fast for a handful of requests and slow down noticeably at sustained concurrency, which is the condition you will actually be in.

Run at least part of the test at the concurrency you plan to use in production, and report percentiles rather than an average. The median tells you about a typical request; the slow tail tells you whether your job will finish on time. Keep each provider’s session and rotation settings at their defaults unless you will change them in production, so you measure the product as sold rather than a configuration nobody else gets.

Collect enough data to mean something

A few dozen requests per network will produce a ranking, but not one that survives a rerun. Small samples are dominated by chance: one bad minute on one target can swing the result.

Spread requests evenly across your targets and across a long enough window to include quiet and busy hours. Rerun the comparison on a second day before you act on a small difference. If the order flips, the networks are close, and price or support should decide it. Count the distinct exit IPs each network handed you while you are at it. If you plan to use two providers for redundancy, intersect those sets, because two networks that share most of their addresses are not two sources.

Compare cost last

Only once success has been measured fairly does price mean anything. Divide what each network would cost for your traffic by the number of requests that passed your rule. Failed requests still bill, so a low rate card with a low pass rate can be the most expensive option. Our guide to cost per successful request walks through the arithmetic.

A fair-test checklist

  1. Same requests, same targets, same locations, same window for every network.
  2. A written pass rule per target, decided before testing.
  3. Failures classified as challenge, block or timeout.
  4. Targets that resemble your real workload, including a hard one.
  5. Exit IP recorded from the target’s side and looked up independently.
  6. Part of the test at production concurrency, with latency percentiles.
  7. Provider defaults for session and rotation settings.
  8. Enough requests, over enough hours, to survive a rerun.
  9. Cost per successful request computed last.

If you want working code for the measurements themselves, our post on testing residential proxy speed, success rate and location accuracy has scripts you can adapt. Atlas runs this whole checklist continuously, and the side-by-side comparison page lets you put any networks it tests side by side before you spend a trial on them.

Frequently asked questions

What is the fairest way to compare residential proxy providers?

Test every provider under the same conditions: the same targets, request types, locations, time window and pass rule. Then compare the same measurements for each, such as success rate, latency and cost per successful request.

How many residential proxy providers should I test at once?

Two or three keeps the comparison manageable. Test each against the same targets and conditions, then add more providers if you need a broader view.

Why should I test proxies on the same websites?

Websites differ in their access rules, anti-bot systems and checks. Testing every provider on the same targets removes the difference caused by the sites themselves, so what remains reflects the networks.

How many requests are needed for a reliable proxy comparison?

There is no single number, but a few dozen requests per network produce an order that rarely survives a rerun. Spread requests evenly across your targets and across quiet and busy hours, then rerun on a second day before you act on a small difference.

Get started

Add Shifter to your side-by-side test

Start a free trial of Shifter's residential network and run it in the same window, against the same targets, as the providers on your shortlist.

3-day money-back guarantee. Cancel anytime.