Why Your MCP Server Needs a Proxy Layer

The Model Context Protocol solved the wrong half of the problem first, and that’s not a criticism, it’s just the order things happened. MCP gave us a clean, standard way to connect a language model to tools and data. What it deliberately doesn’t do is solve what happens when one of those tools reaches out to the open web and the open web reaches back with a 403.

If you’ve built or run an MCP server that fetches URLs, scrapes pages, hits search results, or pulls live data for a model, you’ve probably already met this wall. The server works perfectly on your laptop. You deploy it to a cloud box, point your agent at it, and suddenly half the fetches come back as CAPTCHA pages, “access denied,” or geo-redirects to the wrong country’s site. Nothing about your MCP code changed. The IP did.

This post is about that gap: why web-fetching MCP servers get blocked, why it’s not an MCP problem to fix, and how a residential proxy layer closes it with a few lines of code.

A quick refresher on where the web access happens

MCP has a clean separation of roles. A host (Claude Desktop, an IDE, an agent runtime) runs an MCP client, which talks to one or more MCP servers. Each server exposes tools, resources, and prompts. When the model decides to use a tool, the call flows host → client → server, the server does the actual work, and the result flows back.

The important part for this discussion: the server is where real-world side effects happen. When you build a fetch tool, a search tool, or a scrape_page tool, the outbound HTTP request originates from the MCP server process, wherever that process happens to be running. The model doesn’t make the request. The host doesn’t. The server does.

So the IP the target website sees is the MCP server’s IP. And that’s the whole problem.

Why web-fetching MCP servers get blocked

Three things stack up, and they stack up specifically against MCP servers in a way they don’t against a human with a browser.

Your MCP server is almost certainly on a datacenter IP. If you deployed it to AWS, GCP, Fly, Render, a VPS, or any cloud host, its outbound IP belongs to a hosting-provider ASN. Anti-bot systems (Cloudflare, Akamai, DataDome) treat datacenter ASNs as guilty until proven innocent. A request from an AWS IP to a protected site is flagged before the server even sends a header. This is the single biggest reason “it worked locally” turns into “it’s blocked in prod”, your home connection is residential, your cloud box is not.

Agent traffic is bursty and concentrated. An agent doesn’t browse like a person. It fires a sequence of requests in a tight window, often to the same domain, then goes quiet. From one IP, that pattern reads as automation immediately. We wrote about the shape of AI agent traffic separately, but the short version is: rate limits and behavioral detection that a human never trips, an agent trips constantly, because all the traffic comes from one address.

You have no geo control. Your MCP server runs in one region. If the model needs to see a product page as a German shopper, a search result as a Tokyo user, or a price as a US customer, a single-region server physically cannot. The site geolocates the server’s IP and serves the wrong country’s content, silently. The model then reasons over data it thinks is correct and isn’t.

None of these are bugs in your MCP server. They’re properties of where it runs and how it’s shaped. MCP is doing exactly its job, carrying the tool call. It was never meant to launder the IP.

Why this isn’t MCP’s job to solve

It’s worth being explicit, because people sometimes wait for the protocol to grow a feature that isn’t coming. MCP is a protocol for model-to-tool communication. It standardizes how a tool is described, called, and how results come back. The transport between client and server (stdio, or Streamable HTTP) is about the agent’s plumbing, not about how the server reaches the internet.

How your fetch tool talks to the outside world is an implementation detail of your server, exactly like which HTTP library you use or how you parse the response. The protocol shouldn’t dictate it, and doesn’t. Which means the web-access layer is yours to add. The good news is that it’s a small, well-understood layer: route the server’s outbound requests through residential IPs.

The fix: a residential proxy behind the fetch tool

The pattern is simple. Instead of your MCP server’s tool making a direct request from its datacenter IP, it makes the request through a residential proxy gateway. The target site sees a real consumer IP in the right country, with the trust profile of an actual home connection, and the request goes through.

Here’s a minimal Python MCP server with a fetch_url tool that routes through the Shifter gateway. The MCP scaffolding is standard FastMCP; the part that matters is the proxies wiring on the HTTP client.

import os
import httpx
from mcp.server.fastmcp import FastMCP

mcp = FastMCP("web-fetch")

# One gateway endpoint, targeting encoded in the username.
# Keep credentials in env, never in the tool definition.
USER = os.environ["SHIFTER_USER"]      # e.g. "customer-<id>"
PASS = os.environ["SHIFTER_PASS"]
GATEWAY = "p.shifter.io:443"

def proxy_url(country: str = "us") -> str:
    return f"http://{USER}-country-{country}:{PASS}@{GATEWAY}"

@mcp.tool()
async def fetch_url(url: str, country: str = "us") -> str:
    """Fetch a URL through a residential IP in the given country."""
    proxies = proxy_url(country)
    async with httpx.AsyncClient(proxy=proxies, timeout=30) as client:
        r = await client.get(url, follow_redirects=True)
        r.raise_for_status()
        return r.text

if __name__ == "__main__":
    mcp.run()

That’s the whole change. The model calls fetch_url("https://example.com/product", country="de"), the request leaves through a residential IP in Germany, and the site serves the German page to what looks like a German shopper. Swap country and the model sees the same page through a different market’s eyes, with no redeploy and no second server.

Geo-targeting is the part agents actually need most

The blocking problem is the one people notice first, but the geo problem is the one that quietly corrupts results. An MCP server pinned to one region is an LLM looking at the world through one keyhole.

Because the Shifter gateway encodes targeting in the username, your fetch tool can take a country (or state, city, even ASN) argument and let the model choose per call. That turns a single-region server into a global one. A research agent comparing prices across markets, a localization QA agent checking how a site renders in five countries, a SERP-monitoring agent pulling Google results as a local user, all of them need exactly this and can’t get it from a vanilla cloud-hosted fetch tool.

For multi-step flows where the agent needs the same IP across several calls (a login, then a sequence of authenticated pages), add a sticky session by including a session id and TTL in the username, the same IP holds for the lifetime of the TTL:

def sticky_proxy_url(country: str, session: str, ttl: int = 600) -> str:
    return f"http://{USER}-country-{country}-sid-{session}-ttl-{ttl}:{PASS}@{GATEWAY}"

Now a browse_session tool can carry a stable identity across the steps of a task instead of hopping IPs mid-flow and tripping every session check on the site.

What to keep in mind

A proxy layer isn’t a magic “never blocked” switch, and treating it like one is how people waste bandwidth. A few practical notes:

Cache aggressively. Agents re-fetch the same URLs constantly. A small cache in front of your fetch tool cuts proxy bandwidth (and your cost) dramatically, and is faster for the model too. Don’t proxy a request you already have the answer to.

Pass the country through, don’t hardcode it. The whole value is per-call geo control. Make country a tool argument the model can set, with a sensible default, rather than baking one region into the server.

Respect the target. A proxy changes which IP the request comes from, not whether you should be making it. Honor robots.txt where it’s load-bearing, keep request rates sane, and don’t hammer a site just because the blocks went away. Our acceptable use policy is the source of truth for what’s allowed.

Keep credentials out of the tool schema. Proxy username and password live in environment variables on the server, never in the tool definition the model sees. The model should choose a country, not hold your gateway password.

Datacenter is fine for unprotected, region-neutral fetches. If a tool only ever hits your own APIs or public endpoints that don’t block and don’t geolocate, it doesn’t need the proxy. Reserve the residential layer for the open-web fetches that actually face blocking or need geo. (For a fuller comparison of when each IP type fits, see residential vs datacenter proxies.)

FAQ

Does MCP have built-in proxy support? No, and it shouldn’t. MCP standardizes model-to-tool communication, not how a tool reaches the internet. Outbound web access is an implementation detail of your server. You add a proxy at the HTTP-client level inside the tool, exactly as shown above.

Why not just run my MCP server from a residential connection? You could, but it doesn’t scale, doesn’t give you per-request geo control, and ties your agent’s reliability to one home connection’s uptime and single IP. A residential proxy gateway gives you a large rotating pool and per-call country targeting without hosting anything on a residential line.

Will a proxy stop my agent from getting blocked entirely? It removes the biggest cause (the datacenter IP) and gives you geo control, but behavior still matters. Bursty request patterns, missing or inconsistent headers, and ignoring rate limits can still get you flagged. The proxy is necessary, not sufficient, pair it with sane request behavior.

Does this work with TypeScript MCP servers too? Yes. The concept is identical, configure your HTTP client (fetch with an agent, axios, undici, got) to route through the gateway. The proxy URL format and per-request targeting are the same regardless of language.

Can the model choose the country per request? Yes, that’s the recommended pattern. Expose country (and optionally state/city) as a tool argument with a default. The model sets it based on the task, and your server encodes it into the proxy username on the fly.

Does routing through a proxy change how Shifter bills me? No. It’s the standard residential gateway at the usual per-GB pricing. An MCP server is just another client connecting to the same endpoint. Bandwidth is bandwidth.

The bottom line

MCP is a clean answer to “how does a model call a tool.” It was never meant to answer “how does that tool reach a hostile open web from a flagged datacenter IP in the wrong country.” That second question is real, it’s yours to answer, and the answer is a thin residential proxy layer behind your fetch tools.

The wiring is a few lines. The payoff is an MCP server whose web tools actually work in production, return the right country’s data, and don’t fall over the first time an agent points them at a protected site. If you’re building web-connected agents, start with the residential gateway and put it behind your tools from day one, it’s a lot easier than retrofitting it after the blocks start. For more on giving agents reliable web access, see the best proxies for AI agents that browse the web.