Scraping fetches a single URL and returns the result in the same response. No job to poll, no webhook to set up. Use it when you know the exact page you want.
First request
bash
curl --request POST \
--url https://api.webcrawlerapi.com/v2/scrape \
--header 'Authorization: Bearer <PASTE YOUR API KEY HERE>' \
--data '{
"url": "https://stripe.com/",
"output_formats": ["markdown"]
}'Get your key in Authentication & API keys.
Clean up the content
- Want only the main content, without menus, footers and banners? Use LLM-cleaned Markdown.
- Want fields instead of text? Use Structured outputs.
- Need to tweak what is removed? See Content cleaning.
Need more than one page?
Use a crawl job to follow links across a whole website.
Full parameters and response: POST /scrape.