Browser Crawl Engine

Errors

Failures are JSON with "ok": false and a message written for a human. The status code tells you whether retrying can possibly help.

{"ok": false, "error": "Human-readable message", "code": "machine_readable"}

Status codes

CodeCauseRetry?
400Malformed request: bad URL, too many URLs, a value out of rangeNo — fix the request
401Missing, invalid, expired or revoked keyNo
403Valid key, missing scopeNo
404No such resource — or it is not yoursNo
409Wrong state, e.g. HTML requested for a task that is not DONEAfter the state changes
429Rate limit exceededYes, after Retry-After
503No capacity, or a circuit is open for that hostYes, after Retry-After
500A defect on our sideOnce, then tell us
404 is deliberate for resources you do not own. A 403 would confirm that the identifier exists. The API will not tell you about another account's data, including whether it is there.

Task failures are not HTTP failures

A job can return 201 and still contain tasks that end FAILED. The HTTP code describes your request; the task status describes the page. Always read failedTasks.

Common task errors

ErrorWhat happenedWhat to do
signal:access deniedThe site served a block pageTry another country; the site may simply refuse automation
Execution context was destroyedThe page navigated while being readRaise postLoadDelayMs to 3000–5000
Navigation timeoutThe page never finished loadingRaise navigationTimeoutMs, or set blockAssets true
expectedText not foundThe page loaded without your markerWorking as intended — a wrong page was rejected
No eligible Worker…No worker can serve that host and countryDrop geo, or ask us to add capacity

Blocked pages fail on purpose

A challenge page usually arrives as HTTP 200 with a body that contains no content. Storing that as a success would hand you a file that looks fine and is worthless, so we detect the block, quarantine the exit IP and retry elsewhere. If every attempt is blocked the task ends FAILED — which is the honest answer.

The strongest guard you control is expectedText: name a string that only the real page contains, and anything else is rejected before it reaches you.

A retry policy that works

429, 503  → wait Retry-After, then retry
500       → retry once after a few seconds
400,401,403,404,409 → do not retry; the next attempt fails identically

Per-task retries are already handled server-side by maxAttempts, including rotating to a different exit IP. You do not need to resubmit a job because some tasks failed — check whether the attempts were exhausted first.