Remote MCP server

Connect Claude, Cursor, VS Code and ChatGPT to WebCrawlerAPI with OAuth and read any web page as Markdown.

The WebCrawlerAPI remote MCP server lets AI assistants read any web page as clean Markdown. You connect it by URL and sign in with your WebCrawlerAPI account. There is nothing to install and no API key to paste.

Server URL: https://mcp.webcrawlerapi.com/mcp
Transport: Streamable HTTP
Sign-in: OAuth through https://auth.webcrawlerapi.com
Tool: get_markdown

This page covers the hosted OAuth server. If you prefer to run a local server with an API key, see MCP server (local).

Connect your client

The first time you connect, a browser window opens. Sign in, choose the organization to use, and approve access. Requests are billed to that organization.

Claude Code

bash

claude mcp add --transport http webcrawlerapi https://mcp.webcrawlerapi.com/mcp

Run /mcp in Claude Code, select webcrawlerapi, and choose Authenticate.

Claude.ai and Claude Desktop

Open Settings, then Connectors.
Choose Add custom connector.
Enter the name WebCrawlerAPI and the URL https://mcp.webcrawlerapi.com/mcp.
Click Connect and approve access.

Cursor

Add to ~/.cursor/mcp.json:

json

{
  "mcpServers": {
    "webcrawlerapi": {
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}

VS Code

Add to .vscode/mcp.json:

json

{
  "servers": {
    "webcrawlerapi": {
      "type": "http",
      "url": "https://mcp.webcrawlerapi.com/mcp"
    }
  }
}

ChatGPT

Turn on Developer mode in Settings, then Connectors. Create a connector with the URL https://mcp.webcrawlerapi.com/mcp and OAuth authentication. Menu names vary by plan.

MCP Inspector

bash

npx @modelcontextprotocol/inspector

Choose Streamable HTTP, enter the server URL, and complete the OAuth flow.

Tool: get_markdown

Fetches a public web page and returns its main content as Markdown. JavaScript-rendered pages are supported, and navigation, ads and boilerplate are removed.

Parameter	Type	Required	Description
`url`	string	yes	Absolute `http://` or `https://` page URL

On failure the tool returns an error result with a short message, for example when the site cannot be reached, blocks crawlers via robots.txt, times out, or your organization has no balance left. See Errors for the underlying error codes.

Authentication

The server uses OAuth 2.1. Clients find the authorization server automatically through https://mcp.webcrawlerapi.com/.well-known/oauth-protected-resource.

The only scope is markdown:read.
Access is tied to the organization you approve.
You can review and revoke connected apps in the dashboard.

Pricing

Each get_markdown call is billed to your organization. See pricing for current prices.