Guides, Tutorials & Insights
Learn about residential proxies, web scraping, data collection strategies, and industry best practices from the Shifter team.
Introducing Automatic Wallet Top-Up
Set a threshold and refill amount, and your Shifter wallet refills itself so usage-based charges never pause your proxies. Monthly cap, off anytime.
James Meadow · July 30, 2026 How to Use Residential Proxies in Node.js with Axios and Got
Proxies in Node.js: why Axios needs https-proxy-agent and proxy:false, Got's agent option, undici's ProxyAgent for fetch, and per-request geo rotation.
Chris Collins · July 29, 2026 How to Scale Residential Proxy Scraping Workloads with Kubernetes
Run scraping on Kubernetes: proxy creds as Secrets, per-pod session identity via the Downward API, queue-driven autoscaling, and graceful shutdown.
Chris Collins · July 28, 2026 Web Scraping Best Practices: How to Collect Data Without Harming the Sites You Scrape
Responsible web scraping: honor robots.txt, rate-limit, back off on 429s, cache, and scrape off-peak. Being a good citizen also gets you blocked far less.
Chris Collins · July 27, 2026 How to Use Residential Proxies in PHP with cURL and Guzzle
Proxies in PHP: cURL's CURLOPT_PROXY and CURLOPT_PROXYUSERPWD, Guzzle's proxy option, CURLOPT_HTTPPROXYTUNNEL, and per-request geo rotation.
Chris Collins · July 26, 2026 How to Use Residential Proxies in Java with OkHttp and Apache HttpClient
Proxies in Java: OkHttp's proxyAuthenticator gotcha, Apache HttpClient's default-5 per-route pool limit, entity consumption, and per-request geo rotation.
Chris Collins · July 25, 2026 How to Use Residential Proxies in Go with net/http and Colly
Proxies in Go: transport setup, the body-drain and MaxIdleConnsPerHost gotchas that kill connection reuse, per-request geo rotation, and Colly's SetProxyFunc.
Chris Collins · July 24, 2026 Why Your Residential Proxy Requests Are Timing Out
A diagnostic guide to proxy timeouts: connect vs read, over-tight geo filters, timeouts tuned for datacenter, and the checks that find the real cause fast.
Matt Brown · July 23, 2026 Residential Proxy Failover: Building Reliable Multi-Region Data Pipelines
A retry handles a bad request. Failover handles a bad dependency. How to design proxy-backed pipelines that survive block waves, geo degradation, and outages.
Chris Collins · July 22, 2026