Skip to content
Login Sign up

Rendering JavaScript

SPAs and JS-heavy pages need real browser execution. Web Scraping API handles this behind one flag.

Enable rendering

Section titled “Enable rendering”

Add render_js=1 to any request. Shifter spins up a fresh headless Chrome instance, navigates to the URL, and returns the post-render DOM.

Terminal window
curl "https://scrape.shifter.io/v1?api_key=YOUR_API_KEY&url=https://example.com&render_js=1"

JS rendering is billed at the same 1 credit per successful request as static fetches. No surcharge.

Waiting for content

Section titled “Waiting for content”

Most SPAs finish loading after a specific element appears. Use wait_for_css to block until that selector renders:

Terminal window
curl "https://scrape.shifter.io/v1?api_key=YOUR_API_KEY&url=https://shop.example.com&render_js=1&wait_for_css=.product-price"

The request completes as soon as .product-price is in the DOM, or fails on timeout (default 30s).

Forcing timeouts

Section titled “Forcing timeouts”

Cap the total time the browser can spend on a page with timeout (in milliseconds):

Terminal window
curl "https://scrape.shifter.io/v1?api_key=YOUR_API_KEY&url=https://slow.example.com&render_js=1&timeout=10000"

Useful for scraping slow-loading dashboards where you want whatever is visible after 10 seconds rather than a full render.

JavaScript instructions

Section titled “JavaScript instructions”

Scroll, click, or execute arbitrary steps before capture using js_instructions (URL-encoded JSON). Common patterns:

Scroll to load infinite feeds:

[{"action": "scrollTo", "selector": "footer", "timeout": 5000, "block": "start"}]

Click a button before scraping:

[{"action": "click", "selector": "button.load-more", "timeout": 3000}]

Chain multiple steps:

[
  {"action": "click", "selector": "button.accept-cookies"},
  {"action": "scrollTo", "selector": "div.results"},
  {"action": "wait", "duration": 2000}
]

Pass the array as a URL-encoded string in js_instructions.

Screenshots

Section titled “Screenshots”

Take a full-page screenshot instead of DOM by adding screenshot=1. Response is a binary PNG. See Advanced.

When to skip rendering

Section titled “When to skip rendering”
  • Static HTML pages (news articles, blog posts, RSS-style content)
  • JSON APIs (use a direct fetch, not the scraping endpoint)
  • Files and binaries

Skipping render_js is cheaper in latency, though credits are the same.

Next

Section titled “Next”