How to Use Residential Proxies in C# with HttpClient

.NET is a workhorse for backend data collection: fast async I/O, strong typing, and a runtime that handles high concurrency comfortably. Wiring a residential proxy into HttpClient is a few lines, but C# has its own trap that has nothing to do with proxies and everything to do with how HttpClient is meant to be used. Get the client lifetime right and the proxy part is easy.

This is the C# entry in the same series as residential proxies with Python, in Go, and in Node.js: the code that works, plus the .NET-specific pitfalls.

Everything below uses Shifter’s residential gateway: one endpoint, p.shifter.io:443, with all targeting encoded in the username. Swap the host and credentials for another provider; the shape is the same.

The gateway model in one paragraph

The proxy username carries your authentication and your targeting. You do not switch endpoints to change country or session, you change the username string:

customer-USERNAME-country-us-sid-abc123-ttl-600

country-us targets the United States, sid pins a sticky session, ttl holds that IP for N seconds. Omit sid/ttl and every new connection rotates. The password stays constant. In .NET, that username goes into a NetworkCredential, not into the proxy URL.

The basic setup: WebProxy and a handler

You configure the proxy on a message handler and pass it to HttpClient. The credentials go on a WebProxy via NetworkCredential, not inline in the URI.

using System.Net;

var user = Environment.GetEnvironmentVariable("SHIFTER_USER") + "-country-us";
var pass = Environment.GetEnvironmentVariable("SHIFTER_PASS");

var proxy = new WebProxy("http://p.shifter.io:443")
{
    Credentials = new NetworkCredential(user, pass)   // not user:pass@ in the URL
};

var handler = new SocketsHttpHandler
{
    Proxy = proxy,
    UseProxy = true,
    PooledConnectionLifetime = TimeSpan.FromMinutes(2)  // see gotcha below
};

var client = new HttpClient(handler) { Timeout = TimeSpan.FromSeconds(30) };

var ip = await client.GetStringAsync("https://api.ipify.org");
Console.WriteLine(ip);   // a US residential IP

Two things to internalize. The user string includes the targeting flags (-country-us), because the geo lives there. And the credentials go on the WebProxy as a NetworkCredential, HttpClient will not read user:pass@host from a proxy URI the way a browser might.

Prefer SocketsHttpHandler (the default handler on modern .NET) over the older HttpClientHandler. It is the modern, cross-platform implementation and it exposes PooledConnectionLifetime, which you need for the reason below.

Trap 1: the HttpClient lifetime problem

This is the .NET-specific gotcha, and it bites in two opposite directions.

Create a new HttpClient() for every request and you leak sockets: each client holds its own connection pool, and disposed clients leave sockets stuck in TIME_WAIT. Under load this exhausts available ports and your app starts throwing SocketException. The well-known advice is therefore to reuse a single HttpClient.

But a single client held forever has the opposite problem: it caches DNS resolutions for the life of the connection pool and never notices when an endpoint’s IP changes. The fix is not to choose between the two, it is PooledConnectionLifetime on SocketsHttpHandler, which recycles pooled connections on a schedule so you keep connection reuse without pinning stale DNS. Set it to a couple of minutes and reuse the client.

Reusing the client also keeps connections warm through the proxy, which is exactly the overhead the latency guide exists to remove.

The ASP.NET Core way: IHttpClientFactory

If you are in a DI-based app, do not manage HttpClient by hand at all. IHttpClientFactory handles handler lifetime and connection recycling for you, and lets you configure the proxy once on a named client.

services.AddHttpClient("shifter")
    .ConfigurePrimaryHttpMessageHandler(() => new SocketsHttpHandler
    {
        Proxy = new WebProxy("http://p.shifter.io:443")
        {
            Credentials = new NetworkCredential(
                Environment.GetEnvironmentVariable("SHIFTER_USER") + "-country-us",
                Environment.GetEnvironmentVariable("SHIFTER_PASS"))
        },
        UseProxy = true
    });

// then inject IHttpClientFactory and call factory.CreateClient("shifter")

The factory rotates the underlying handlers on its own schedule, so you get pooling and fresh DNS without touching PooledConnectionLifetime yourself. This is the recommended path for any long-running service.

Trap 2: the proxy is on the handler, so rotate with a client per identity

Because the proxy and its credentials are baked into the handler, you cannot vary the proxy username per request on a single HttpClient. A different identity means a different handler, and therefore a different client. The efficient pattern is to cache one client per identity so each session keeps its own connection pool, the same idea as an agent-per-identity map in other stacks.

using System.Collections.Concurrent;

static readonly ConcurrentDictionary<string, HttpClient> Clients = new();

static HttpClient ClientFor(string country, string sid)
{
    var key = $"{country}:{sid ?? "rotate"}";
    return Clients.GetOrAdd(key, _ =>
    {
        var u = Environment.GetEnvironmentVariable("SHIFTER_USER") + "-country-" + country
              + (sid != null ? $"-sid-{sid}-ttl-600" : "");
        var handler = new SocketsHttpHandler
        {
            Proxy = new WebProxy("http://p.shifter.io:443")
            {
                Credentials = new NetworkCredential(u, Environment.GetEnvironmentVariable("SHIFTER_PASS"))
            },
            UseProxy = true,
            PooledConnectionLifetime = TimeSpan.FromMinutes(2)
        };
        return new HttpClient(handler);
    });
}

Give each logical unit of work its own sid and rotate between units, not mid-stream (sticky vs rotating covers the distinction), and map work to identities the way the load-balancing post describes. Cache and reuse these clients, do not build one per request.

Trap 3: bound your concurrency

.NET makes it trivial to fire thousands of requests at once with Task.WhenAll, and nothing stops you from opening every connection simultaneously, which exhausts sockets on your side and looks like an attack to the target. Bound in-flight requests with a SemaphoreSlim:

var gate = new SemaphoreSlim(8);   // at most 8 concurrent requests

async Task<string> Fetch(HttpClient client, string url)
{
    await gate.WaitAsync();
    try { return await client.GetStringAsync(url); }
    finally { gate.Release(); }
}

Cap concurrency per target host, not just globally, so one fragile site is not hammered while a permissive one is starved. More parallelism past a target’s tolerance buys blocks, not throughput (how to avoid getting blocked).

Verify you are actually on the proxy

Before you benchmark or debug anything else, confirm the exit IP:

var json = await client.GetStringAsync("http://ip-api.com/json");
Console.WriteLine(json);   // expect a residential IP in the targeted country

Your own IP means the handler is not being applied (usually UseProxy left false, or credentials on the wrong object). A hang means local egress is blocked. Both are covered in the timeout diagnosis guide.

FAQ

Why do my proxy credentials not work when I put them in the URL? HttpClient does not read user:pass@host from a proxy URI. Put the credentials on the WebProxy object as a NetworkCredential. Because the gateway encodes targeting in the username, that full username (with -country-... flags) is the NetworkCredential username, and the password is constant.

HttpClientHandler or SocketsHttpHandler? Prefer SocketsHttpHandler on modern .NET. It is the default managed handler, it is cross-platform, and it exposes PooledConnectionLifetime, which you need to avoid stale DNS on a long-lived client. HttpClientHandler still works and delegates to it under the hood.

My .NET scraper throws SocketException under load. Why? You are almost certainly creating a new HttpClient() per request and exhausting sockets. Reuse a single client (or use IHttpClientFactory), and set PooledConnectionLifetime so reuse does not pin stale DNS.

How do I rotate IPs per request in C#? Vary the proxy username, which means a different handler and therefore a different client. Cache one HttpClient per identity in a ConcurrentDictionary so each session keeps its own pool, and pick the client per unit of work. Omit the sid in the username to rotate on every new connection.

Does IHttpClientFactory work with proxies? Yes. Configure the proxy on the primary handler via ConfigurePrimaryHttpMessageHandler. The factory manages handler lifetime and DNS freshness for you, so it is the cleanest option in any DI-based service.

The bottom line

C# plus residential proxies is quick once you respect two things: put the credentials on a WebProxy as a NetworkCredential rather than in the URL, and get the HttpClient lifetime right. Use SocketsHttpHandler with PooledConnectionLifetime, or IHttpClientFactory in a DI app, so you reuse connections without pinning stale DNS and without leaking sockets. Rotate geo and sessions by caching one client per identity, and bound concurrency with a SemaphoreSlim.

Get that right and .NET handles concurrent collection as well as anything. Point it at the residential gateway, and remember that pool quality decides how often you retry at all (IP reputation). The pricing page has the per-GB plans to test it against your own targets.