Webcrawler API LogoWebCrawlerAPI
APIFeed

GET /feed/:id/rss

Get feed changes in Atom 1.0 (RSS) format

Returns the feed content in Atom 1.0 format with RFC 5005 pagination support.

https://api.webcrawlerapi.com/v2/feed/:id/rss

Format: Atom 1.0 XML Method: GET

Request

URL Parameters:

  • id - (required) the feed ID

Query Parameters:

  • page - (optional, default: 1) page number for pagination
  • page_size - (optional, default: 1000, max: 1000) number of items per page

Headers:

  • Authorization: Bearer {api_key} - (required) your API key

Response

Content-Type: application/atom+xml; charset=utf-8

Example:

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>WebCrawlerAPI Feed: example.com</title>
  <link href="https://api.webcrawlerapi.com/v2/feed/abc123/rss" rel="self" type="application/atom+xml"/>
  <link href="https://example.com/blog" rel="alternate" type="text/html"/>
  <id>urn:webcrawlerapi:feed:abc123</id>
  <updated>2024-01-01T12:05:00Z</updated>
  <generator>WebCrawlerAPI</generator>
  <subtitle>Change tracking feed for example.com</subtitle>

  <entry>
    <title>New: Getting Started with Web Scraping</title>
    <link href="https://example.com/blog/web-scraping-guide" rel="alternate"/>
    <id>urn:webcrawlerapi:feeditem:item123</id>
    <updated>2024-01-01T12:05:00Z</updated>
    <published>2024-01-01T12:05:00Z</published>
    <summary type="text">New page discovered</summary>
    <category term="new" label="New"/>
    <author>
      <name>WebCrawlerAPI</name>
    </author>
  </entry>

  <entry>
    <title>Changed: API Documentation</title>
    <link href="https://example.com/docs/api" rel="alternate"/>
    <id>urn:webcrawlerapi:feeditem:item456</id>
    <updated>2024-01-01T12:05:00Z</updated>
    <published>2024-01-01T12:05:00Z</published>
    <summary type="text">Page content has changed</summary>
    <category term="changed" label="Changed"/>
    <author>
      <name>WebCrawlerAPI</name>
    </author>
  </entry>
</feed>

Change Types

Each entry includes a category element indicating the type of change:

  • new - A new page was discovered that wasn't in the previous crawl
  • changed - The page content has changed since the last crawl
  • unavailable - A page that was previously available is no longer accessible

Pagination

The RSS feed supports pagination with RFC 5005 support. Use the page and page_size query parameters to navigate through changes:

  • Page 1 (default) contains the most recent changes (up to 1000 items)
  • Default page size is 1000 items (max: 1000)
  • If a page has no items, the feed will return an empty feed with just the header

Using the RSS Feed

You can subscribe to this feed using any RSS reader or feed aggregator:

  1. Copy the feed URL: https://api.webcrawlerapi.com/v2/feed/{id}/rss
  2. Add your API key as a query parameter or header (depending on your RSS reader)
  3. Subscribe to the feed in your reader

Example with curl:

# Get first page (default, 1000 items)
curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://api.webcrawlerapi.com/v2/feed/abc123/rss"

# Get specific page with custom page size
curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://api.webcrawlerapi.com/v2/feed/abc123/rss?page=2&page_size=100"

Error Responses

  • 400 Bad Request - Invalid request parameters
  • 404 Not Found - Feed not found or does not belong to your organization