Guides, Tutorials & Insights
Learn about residential proxies, web scraping, data collection strategies, and industry best practices from the Shifter team.
Writing a Company Data Collection Policy: A Working Template
Most teams that collect web data have no written rules for it. A working policy template: scope, approvals, sources, personal data and takedowns.
Matt Brown · October 4, 2026 Accessibility Auditing Across Markets: Testing What Users in Each Country Actually See
An audit of your US homepage says little about your German or Japanese one. We audited 23 sites from four countries: a quarter had market-only problems.
James Meadow · October 4, 2026 Is Your Target Blockable? An Anti-Bot Stack Lookup
Before building a scraper, find out what protects the target. We checked the top 1,000 domains: which anti-bot vendors they use and how a plain client fared.
Chris Collins · October 3, 2026 Rate Parity Monitoring: Catching Your Own Inventory Undercut
Hotels lose direct bookings when their own rooms appear cheaper elsewhere. How to monitor rate parity across channels and markets without false alarms.
Matt Brown · October 3, 2026 Proxy Selection as a Multi-Armed Bandit Problem
Which proxy configuration should the next request use? Treat it as a bandit problem. In our simulation, adaptive selection came within 2% of perfect hindsight.
James Meadow · October 2, 2026 Designing Idempotent Scrapers: Restarting Without Duplicating
Scrapers crash. We killed one ten times mid-crawl: a naive version stored up to 4.3 times the rows, an idempotent one exactly 500. How to build the second.
Chris Collins · October 2, 2026 How to Detect Fake Reviews and Review Fraud at Scale
Fake reviews are banned in the US, UK and EU, and platforms remove hundreds of millions. How to detect review fraud across many products, with tested code.
Matt Brown · October 2, 2026 Using hreflang to Find Every Language and Country Version of a Page
hreflang lists every localized version of a page. We checked how top sites use it, what breaks, and why some versions only open from inside a country.
James Meadow · October 1, 2026