LLM-Cleaned Markdown Endpoint

The /markdown endpoint is now fully documented and supported across all SDKs, with published pricing.

What it does

GET/POST /markdown scrapes a single page and returns its content as markdown, cleaned with an LLM to keep only the main content - article body, product details, docs content - while stripping navigation, sidebars, ads, footers, and other boilerplate. The LLM cleaning step adds only one to two seconds of extra latency on top of a normal scrape.

If the LLM cleaning step fails for any reason, the endpoint falls back to returning the raw, uncleaned markdown instead of erroring out.

Quick example

curl --request GET \
  --url 'https://api.webcrawlerapi.com/markdown?url=https://example.com/blog/some-article' \
  --header 'Authorization: Bearer YOUR_API_KEY'

Response:

{
  "success": true,
  "markdown": "# Some Article\n\nThis is the cleaned main content of the page..."
}

Pricing

Current pricing is always available on the pricing page and in your dashboard's billing page.

If you just need plain page markdown without LLM cleaning, POST /scrape with output_formats: ["markdown"] is cheaper since no LLM call is involved.

See the API documentation for request/response examples and error codes.