Feed Management

Manage your feeds by pausing, resuming, force-running, or deleting them.

Pause Feed

Temporarily stop a feed from running scheduled crawls.

PUT https://api.webcrawlerapi.com/v2/feed/:id/pause

Response

{
    "id": "abc123",
    "status": "paused",
    "message": "Feed paused successfully"
}

Paused feeds will not run until resumed. Any scheduled runs will be skipped.

Error Responses

400 Bad Request - Invalid operation (e.g., trying to pause a canceled feed)
404 Not Found - Feed not found or does not belong to your organization

Resume Feed

Resume a paused feed to continue scheduled crawls.

PUT https://api.webcrawlerapi.com/v2/feed/:id/resume

Response

{
    "id": "abc123",
    "status": "active",
    "next_run_at": "2024-01-02T12:00:00Z",
    "message": "Feed resumed successfully"
}

When resumed, the feed will be scheduled to run based on its interval from the current time.

Error Responses

400 Bad Request - Invalid operation (e.g., trying to resume a canceled feed)
404 Not Found - Feed not found or does not belong to your organization

Force Run a Feed

Trigger an immediate feed run outside of the regular 24-hour schedule. Rate limited to one run per hour.

PUT https://api.webcrawlerapi.com/v2/feed/:id/run

Response

{
    "id": "run_xyz789",
    "job_id": "job_abc123",
    "status": "in_progress",
    "message": "Feed run triggered successfully"
}

Important Notes

Feed must be active: Paused or canceled feeds cannot be force-run. Resume the feed first if needed.
Rate limiting: Minimum 1 hour between force-runs to prevent abuse. If you try to force-run too soon, you'll receive a 400 error with a message indicating how long to wait.
Does not affect schedule: Force-runs are "extra" runs that don't change the regular 24-hour schedule. Your feed will still run automatically at its scheduled time.
Same pricing: Force-runs use the same pricing as regular scheduled runs - you pay per page crawled.
Requires balance: Your organization must have sufficient balance (≥10,000 micro-dollars) to trigger a force-run.

Error Responses

400 Bad Request - Feed is not active, or force-run attempted too soon after last run, or invalid request
402 Payment Required - Insufficient balance to run the feed
404 Not Found - Feed not found or does not belong to your organization

Resend Webhook

Resends the webhook notification for the latest completed feed run.

POST https://api.webcrawlerapi.com/v2/feed/:id/webhook/resend

Response

{
    "success": true,
    "message": "Webhook resent successfully",
    "webhook_url": "https://yourserver.com/webhook",
    "status_code": 200
}

Error Responses

400 Bad Request - Invalid request or no webhook configured for this feed
404 Not Found - Feed not found or does not belong to your organization
500 Internal Server Error - Failed to send webhook

Delete Feed

Permanently cancel a feed. This action cannot be undone.

DELETE https://api.webcrawlerapi.com/v2/feed/:id

Response

{
    "id": "abc123",
    "status": "canceled",
    "message": "Feed canceled successfully"
}

Canceled feeds cannot be resumed. Historical data (past runs and detected changes) remains accessible via the RSS and JSON endpoints until the retention period expires (30 days).

Error Responses

400 Bad Request - Invalid request
404 Not Found - Feed not found or does not belong to your organization

Feed Statuses

active - Feed is running and will crawl at the scheduled interval
paused - Feed is temporarily stopped (can be resumed)
canceled - Feed is permanently deleted (cannot be resumed)

Automatic Pausing

Feeds are automatically paused after 3 consecutive errors. This prevents unnecessary charges when a website becomes unreachable. You can resume the feed manually once the issue is resolved.

Trial Expiration

When your trial period ends, all active feeds are automatically canceled. You will need to add credits and create new feeds to continue monitoring.

Feed Management

On this page