Errors & limits
Credits
Section titled “Credits”| Event | Charged? |
|---|---|
| Successful request (HTTP 200) | 1 credit |
Rendering-enabled success (render_js=1) | 1 credit |
| Failed request (timeouts, connection errors) | 0 credits |
| HTTP 4xx or 5xx from the target | 0 credits |
| Retries within one call | Counted as the single parent call |
You are billed only for responses that reach you with a 2xx status and a non-empty body.
Concurrency
Section titled “Concurrency”| Plan | Max concurrent requests |
|---|---|
| Starter | 20 |
| Growth | 50 |
| Business | 100 |
| Enterprise | 500 |
Requests above the cap return 429 Too Many Requests. Back off with exponential delay and retry.
Error codes
Section titled “Error codes”| Status | Meaning | What to do |
|---|---|---|
200 | Success, response body attached | Process normally |
400 | Bad request (malformed params) | Fix the extract_rules or url parameter |
401 | Invalid or missing api_key | Check the key in the panel |
403 | Plan does not allow this feature | Upgrade or remove the flag (e.g. premium_proxy on Starter) |
408 | Target site timed out | Retry with a higher timeout=, or simpler options |
422 | Render failed (JS crash, navigation error) | Retry; check the site works in a real browser |
429 | Concurrency cap exceeded | Back off and retry |
500 | Shifter internal error | Retry after a short delay, or contact support if persistent |
509 | Plan credits exhausted | Upgrade or wait for the next billing cycle |
Automatic retries
Section titled “Automatic retries”Failed fetches, CAPTCHAs, and transient 5xx from the target are retried automatically (up to 3 times) with different proxies before the endpoint returns an error. You are not charged for retries.
Webhook delivery
Section titled “Webhook delivery”For long-running scrapes, pass webhook=<URL> to have the response POSTed to your endpoint when ready:
curl "https://scrape.shifter.io/v1?api_key=YOUR_API_KEY&url=https://example.com&webhook=https%3A%2F%2Fyour.app%2Fcallback"You still pay 1 credit per successful delivery.
Monitoring
Section titled “Monitoring”Real-time credit usage and error rate are visible in the panel under Web Scraping API → Usage.