How to Scrape Amazon Product Data (Prices, Reviews, Rankings) with Proxies

Amazon is the largest product dataset on the open web, and scraping it, prices, reviews, rankings, availability, powers price monitoring, digital shelf analytics, market research, and competitive intelligence. It’s also one of the most aggressively defended sites you’ll ever point a scraper at. Get the setup wrong and you meet the “Robot Check” page, a 503, or wrong-country data within a handful of requests.

This is a practical guide to doing it right: what you can pull from an Amazon product page, why it breaks without the right infrastructure, and how to collect it cleanly through residential proxies. The short version: Amazon is geo-split by marketplace and hard on bots, so you have to fetch as a real shopper in the right country, which is exactly what residential proxies are for.

What you can scrape from an Amazon product page

An Amazon listing is dense with public, structured data. The fields teams collect most:

• Price and deal info — current price, list price, discounts, and coupons.
• Availability and buy box — in stock or not, and which seller wins the buy box.
• Best Sellers Rank (BSR) — Amazon’s ranking within a category, a demand proxy.
• Ratings and reviews — star rating, review count, and review text.
• Product detail — title, bullets, description, images, variations, and specs.
• Seller info — the seller and shipping details on the offer.

All of it is on the public page. The catch is that Amazon decides which page to show you based on where you appear to be and whether you look like a real shopper.

Why Amazon is hard: it’s a proxy problem

Three properties of Amazon turn scraping it into an access problem that lands on the proxy layer.

Amazon is split by marketplace, and geo-personalized. Amazon runs separate marketplaces per country, amazon.com, amazon.de, amazon.co.uk, amazon.co.jp, and each has different prices, availability, sellers, and even catalog. On top of that, a single marketplace personalizes price and delivery by the shopper’s location. Fetch amazon.de from a US IP and you may get a redirect, a different experience, or availability a German shopper would never see. To capture a marketplace accurately, you have to come from that country, which is what country and city targeting is for.

Amazon defends hard against bots. Data-center IPs are flagged fast and get the CAPTCHA “Robot Check,” a 503, or throttling, so you record the bot version, not the shopper’s page. (Why scrapers get blocked covers the mechanics.) Residential IPs carry real-user trust, so Amazon serves you the real listing.

Coverage is high-volume and repeated. Tracking many ASINs across marketplaces every day is a lot of requests. From a handful of IPs you trip rate limits fast and get a partial, block-riddled sample. A large rotating pool is what keeps the crawl complete.

The fix for all three is the same: fetch from residential IPs that match the marketplace’s country, rotating at scale.

Step 1: route through a residential IP in the right country

On the Shifter gateway, you pick a country by encoding it in the proxy username, one endpoint, no IP lists. Match the proxy country to the marketplace domain: amazon.com from a US IP, amazon.de from a German IP.

# amazon.com as a US shopper
curl -x customer-USERNAME-country-us:PASSWORD@p.shifter.io:443 \
  "https://www.amazon.com/dp/B0EXAMPLE"

# amazon.de as a German shopper
curl -x customer-USERNAME-country-de:PASSWORD@p.shifter.io:443 \
  "https://www.amazon.de/dp/B0EXAMPLE"

Every product has a stable ASIN (the B0... code in the /dp/ASIN URL), so once you have your ASIN list, the URL pattern is predictable across marketplaces.

Step 2: fetch and parse in code

You rarely need a browser for Amazon product data, most of it is in the initial HTML, so a plain HTTP client is faster and far cheaper on bandwidth. Here’s the shape in Python (see residential proxies with Python for the full client setup):

import os, requests
from bs4 import BeautifulSoup

USER, PASS = os.environ["SHIFTER_USER"], os.environ["SHIFTER_PASS"]
GATEWAY = "p.shifter.io:443"

def proxy(country="us"):
    url = f"http://{USER}-country-{country}:{PASS}@{GATEWAY}"
    return {"http": url, "https": url}

def scrape_product(asin, country="us", tld="com"):
    url = f"https://www.amazon.{tld}/dp/{asin}"
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
        "Accept-Language": "en-US,en;q=0.9",
        "Accept-Encoding": "gzip, br",
    }
    r = requests.get(url, headers=headers, proxies=proxy(country), timeout=30)
    r.raise_for_status()
    soup = BeautifulSoup(r.text, "html.parser")
    title = soup.select_one("#productTitle")
    price = soup.select_one(".a-price .a-offscreen")
    rating = soup.select_one("#acrPopover")
    return {
        "asin": asin,
        "title": title.get_text(strip=True) if title else None,
        "price": price.get_text(strip=True) if price else None,
        "rating": rating["title"].strip() if rating and rating.has_attr("title") else None,
    }

print(scrape_product("B0EXAMPLE", country="us", tld="com"))

Two things matter more than the selectors: send a realistic User-Agent and Accept-Language that match your proxy country, and expect Amazon’s markup to shift, so treat every selector as optional and validate what you extract. For reviews, request the reviews pages and paginate; for BSR, it’s usually in the “Product information” / detail section.

Step 3: rotate, and hold sessions where it helps

• Rotate per request for broad catalog scraping across many ASINs, omit any session id so each request gets a fresh IP.
• Hold a sticky session when you paginate through a single product’s reviews or a multi-step flow, so the sequence looks like one shopper. Add -sid-<id>-ttl-<seconds> to the username.

Because IP quality decides how often you see the real page versus a Robot Check, understanding IP reputation helps you keep the crawl clean.

Step 4: handle blocks and keep bandwidth low

When Amazon serves a CAPTCHA/Robot Check or a 503, don’t hammer, rotate to a fresh IP, back off, and retry. Consistent blocks (not occasional ones) point to IP quality or request behavior, covered in how to avoid getting blocked when scraping.

If a page genuinely needs JavaScript and you reach for a browser, block images, media, and fonts so you’re not paying per-GB for pixels you never parse, the technique is in how to cut proxy bandwidth costs. For most Amazon product fields, plain HTTP is enough and cheapest.

Using it responsibly

Scrape public product data only, listings, prices, ratings, and review text any shopper can see, and steer clear of personal data, reviewer identities and any personal information attached to reviews aren’t fair game. Honor Amazon’s terms and rate limits, don’t degrade the site, and be aware Amazon’s terms of service restrict automated access, so keep collection to public data and get legal advice for anything uncertain (see is web scraping legal). A proxy changes which IP a request comes from, not whether you should be making it; our acceptable use policy is the source of truth for what’s allowed on Shifter.

FAQ

Can I scrape Amazon without proxies? Not reliably at any scale. Amazon flags data-center and repeated-IP traffic fast with CAPTCHAs and 503s, and you’d only ever see one country’s marketplace. Residential proxies let you fetch as a real shopper in each target country.

Why do I need the proxy country to match the marketplace? Because Amazon marketplaces are per-country and geo-personalized. Fetching amazon.de from a US IP can trigger redirects or show data a German shopper wouldn’t see. Match the IP country to the domain (amazon.de from a DE IP) for accurate data.

Do I need a headless browser for Amazon? Usually not. Most product fields, price, title, rating, availability, are in the initial HTML, so a plain HTTP request is faster and much cheaper on bandwidth. Reserve a browser for genuinely JavaScript-dependent pages.

Residential or datacenter proxies for Amazon? Residential. Amazon detects and blocks data-center IPs aggressively, so datacenter gives you Robot Checks and 503s. Residential IPs see the real, geo-accurate listing a genuine shopper would.

Is scraping Amazon legal? Public product data is generally public-facing, and collecting it is broadly fine when done responsibly, respecting terms and rate limits and avoiding personal data. Amazon’s ToS restricts automated access, so keep it to public data and get legal advice for anything uncertain. A proxy doesn’t change the legality of the underlying activity.

The bottom line

Amazon is a goldmine of public product data, but it’s split by marketplace, geo-personalized, and one of the hardest sites to scrape without getting blocked. The recipe is straightforward: route through a residential IP that matches the marketplace’s country, fetch the HTML with a realistic client, parse defensively, rotate at scale, and handle Robot Checks by switching IPs rather than pushing harder. Do that and you get the real prices, reviews, and rankings a shopper sees, in every marketplace you track.

The piece that makes or breaks it is IP quality, clean residential IPs mean the real listing instead of a Robot Check. A quality residential proxy network is the access layer that makes Amazon scraping reliable, and it’s the same foundation behind price monitoring and digital shelf analytics. The pricing page has the per-GB plans to trial it against the ASINs and marketplaces that matter to you.