n8n is a powerful workflow automation tool that allows you to connect various services and automate tasks. You can use n8n to integrate WebCrawlerAPI for crawling websites and extracting data, which can then be used for training large language models (LLMs) or other purposes.

The simplest way to use WebcrawlerAPI in n8n is to use the HTTP Request node to make API calls.

1. In your workflow add a new node and select the HTTP Request node.

2. Then tap "Import cURL" and paste the following snippet, using your API key (opens in a new tab) and the URL you want to crawl:

curl --request POST \
  --url https://api.webcrawlerapi.com/v2/scrape \
  --header 'Authorization: Bearer <YOUR API KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
	"url": "https://webcrawlerapi.com"
}'

Tap "Test step" to make sure everything is working correctly. You should see the response with markdown from the API in the output panel.