Batch Scrape

You can now scrape an explicit list of URLs in a single request. No crawling, no polling loop per URL: send the list, get a batch ID back, and collect results when the batch finishes.

What you get

• POST /v2/batch/scrape takes up to your plan's item limit of URLs (100 during trial) and returns a batch id and a status_url.
• GET /v2/batch/scrape/{id} returns progress (total, completed, failed, canceled) and a paginated list of items with links to each page's markdown, cleaned or HTML output.
• GET /v2/batch/scrape/{id}/errors lists only the URLs that failed, with error codes.
• DELETE /v2/batch/scrape/{id} cancels a running batch. Pages that did not start are not charged.
• Pass webhook_url to get one call when the whole batch finishes.
• Invalid and duplicate URLs are skipped and reported in invalid_urls, or set ignore_invalid_urls to false to reject the whole request.
• Use max_concurrency to cap how many URLs run in parallel.

Options

Same options as single scrape: output_formats, main_content_only, clean_selectors, max_age and respect_robots_txt.

Pricing

Each page is billed at the same per-page rate as a crawled page. Cached pages follow the usual caching rules.

Read the batch scrape guide and the API reference to get started.