SDKs and Code Examples
Make (formerly Integromat) WebcrawlerAPI integration
How to get website content for LLM training using Make and WebCrawlerAPI.
Make (formerly Integromat) is a powerful automation platform that allows you to connect various apps and services to automate tasks. You can use Make to integrate WebCrawlerAPI for crawling websites and extracting data, which can then be used for training large language models (LLMs) or other purposes.
-
First, you need to create a WebCrawlerAPI account and get your API key.
-
Next, log in to your Make account and search for the WebcrawlerAPI app in the app directory.
-
Click on the WebcrawlerAPI app to open its details page, then click "Create a connection".
- In the connection settings, enter your WebCrawlerAPI API key and click "Save".
- Now you can use the WebcrawlerAPI app in your Make scenarios. To scrape a webpage, add the WebcrawlerAPI module to your scenario and configure it with the URL you want to scrape and any additional parameters you need.
