Browser Crawl Engine

Browser sessions

A session holds one Chrome browser context open, keeps its cookies and storage, and runs same-origin fetch() calls inside the loaded page. Use it when state between requests matters.

Most work does not need a session. Fetching many independent pages is what crawl jobs are for, and they parallelise across the fleet. A session occupies one entire worker for its lifetime — while it is open, that worker does no other work.

Jobs or sessions?

Crawl jobBrowser session
Good forMany independent pagesA sequence inside one site
CookiesFresh per pageKept for the session
ParallelismAcross the whole fleetOne worker, one at a time
ReturnsRendered HTMLResponse bodies from in-page fetch

Lifecycle

POST /v1/browser-sessionsjobs:create
curl -sS -X POST "$API/v1/browser-sessions" \
  -H "Authorization: Bearer $API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"entryUrl": "https://example.com/", "geo": {"country": "de"},
       "ttlSeconds": 1800, "idleTimeoutSeconds": 300}'

The session starts PENDING, becomes READY once a worker has loaded the entry page, and ends when you close it, its TTL expires, or it goes idle past idleTimeoutSeconds.

GET /v1/browser-sessions/{sessionId}jobs:read
POST /v1/browser-sessions/{sessionId}/fetchesjobs:create
GET /v1/browser-sessions/{sessionId}/fetches/{fetchId}jobs:read
DELETE /v1/browser-sessions/{sessionId}jobs:create

Entry URLs are allowlisted

Sessions may only be opened against hostnames an operator has enabled, and the match is exact — www.example.com does not cover example.com. An entry URL outside the list returns 400. Crawl jobs are not restricted this way.

Ask us to add a hostname before you build against it.

Limits worth knowing