GET/POST /markdown

Scrapes a single page and returns its markdown content, cleaned with an LLM to keep only the main content — article body, product details, docs content — and strip navigation, sidebars, ads, footers, and other boilerplate.

Method: GET or POST

If you just need plain page markdown without LLM cleaning, use POST /scrape with output_formats: ["markdown"] instead — it's cheaper since no LLM call is involved.

Request example#

curl --request POST \
  --url https://api.webcrawlerapi.com/markdown \
  --header 'Authorization: Bearer <YOUR_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "https://example.com/blog/some-article"
  }'

Or as GET, with the URL as a query parameter:

curl --request GET \
  --url 'https://api.webcrawlerapi.com/markdown?url=https://example.com/blog/some-article' \
  --header 'Authorization: Bearer <YOUR_API_KEY>'

Request params#

• url - (required) The URL of the webpage to scrape and clean.

Response format#

• Content-Type: application/json

{
    "success": true,
    "markdown": "# Some Article\n\nThis is the cleaned main content of the page..."
}

If LLM extraction fails for any reason, the endpoint falls back to returning the raw, uncleaned markdown instead of erroring out.

Error responses#

If the request fails, the response has success: false with an error_code and error_message:

{
    "success": false,
    "error_code": "website_timeout",
    "error_message": "The website took too long to respond"
}

• 401 Unauthorized - Invalid or missing API key
• 402 Payment Required - Insufficient account balance
• 403 Forbidden - Website access denied
• 429 Too Many Requests - Rate limit exceeded
• 500 Internal Server Error - Server-side error

Read more about error codes in the Errors section.

Notes#

• This endpoint scrapes the page and runs LLM cleaning in the same synchronous request, so it can take a few extra seconds compared to a plain scrape.
• For the plain, non-LLM-cleaned markdown of a page, use POST /scrape with output_formats: ["markdown"].