Summary
Across 806 valid URLs, Context.dev returned usable content for 92.1% of pages, Firecrawl for 89.7% and WebCrawlerAPI for 88.0%. WebCrawlerAPI scored 88.0% end to end and had the highest extraction F1 (0.623). It left the most boilerplate out of the markdown while keeping 72.8% of the expected content.
The two tools with the lowest F1, Context.dev (0.176) and Jina Reader (0.184), still posted high recall and coverage (95.0% and 90.2%). They return most of the page, chrome included, which raises recall and lets in the menus, footers and cookie banners. Crawlbase had the most failed requests (96). Median latency ranged from 1.4 s to 10.0 s, and p95 from 15.6 s to 132.7 s.
What was measured
We used the first 1,000 rows of firecrawl/scrape-content-dataset-v1 (1-0-0.csv). Each row holds a URL, the page's main text (truth_text) and a short boilerplate sample that should not be in a clean result (lie_text). 133 rows are marked as broken in the dataset itself and are skipped for every tool. A further 61 URLs are dead: at least one tool got an HTTP 404 for them, which is the correct answer for a missing page. Those are excluded for every tool, which leaves 806 scored URLs. Each tool was called with its default markdown settings, listed in the table below.
| Tool | Request sent | Cache | Non-default settings | Timeout | Retries | Run date |
|---|---|---|---|---|---|---|
| WebCrawlerAPI | POST /markdown · {url} | No cache: the /markdown endpoint does not cache | None | 60s | 1 | 2026-09-28 |
| Firecrawl | POST /v1/scrape · {url, formats: [markdown], maxAge: 0} | Off: maxAge=0 | maxAge=0 (cache off) | 120s | 1 | 2026-10-09 |
| Crawl4AI Cloud | POST /scrape · {url, format: md, bypass_cache: true} | Off: bypass_cache=true (undocumented, non-default) | bypass_cache=true (cache off, undocumented) | 120s | 2 | 2026-10-10 |
| Spider | POST /scrape · {url, return_format: markdown, limit: 1} | No cache parameter documented: could not be turned off | limit=1 (scrape a single page) | 120s | 2 | 2026-10-05 |
| Jina Reader | POST r.jina.ai · {url} · X-Return-Format: markdown, X-Engine: browser, X-Retain-Images: none | Off: X-Cache-Tolerance: 0, X-No-Cache: true | X-Engine: browser, X-Retain-Images: none, cache off | 60s | 1 | 2026-10-02 |
| Context.dev | POST /v1/web/scrape · {url, formats: {markdown: true}, maxAgeMs: 0} | Off: maxAgeMs=0 | maxAgeMs=0 (cache off) | 120s | 2 | 2026-10-09 |
| Crawlbase | GET /?url&format=md&md_readability=true&javascript=true | No cache parameter documented: could not be turned off | javascript=true (headless browser rendering), md_readability=true (readability pass strips nav and sidebars) | 120s | 2 | 2026-10-09 |
| Context.dev (main content only) | POST /v1/web/scrape · {url, formats: {markdown: true}, maxAgeMs: 0, sharedParams: {dismissCookies: false, mainContentOnly: true}} | Off: maxAgeMs=0 | maxAgeMs=0 (cache off), mainContentOnly=true, dismissCookies=false | 120s | 2 | 2026-10-09 |
| Firecrawl (main content only) | POST /v1/scrape · {url, formats: [markdown], maxAge: 0, onlyMainContent: true} | Off: maxAge=0 | maxAge=0 (cache off), onlyMainContent=true set explicitly (already the default) | 60s | 1 | 2026-10-09 |
Timeout and retries are the values each run was started with, so they differ between tools. Cache handling is listed per tool. Spider and Crawlbase do not document a cache or max-age setting, so their pages could not be forced live. WebCrawlerAPI's /markdown endpoint has no cache, so there is nothing to turn off. For Crawl4AI we turned on bypass_cache, which is not its default and is not in its public docs, so every page was fetched fresh instead of served from its archive. The last column lists every setting that differs from the API's defaults. Crawlbase ran with JavaScript rendering and its readability pass on, which is not its default; Jina ran with its browser engine and without images.
WebCrawlerAPI's /markdown endpoint is a dedicated markdown endpoint that cleans the page content and returns only the main content. That is what the endpoint returns to every user, so it is scored as is. The other tools in the main ranking return the default markdown of their own APIs.
Results
Pick a metric to re-rank the tools, or switch to the table to see every number at once.
End-to-end successhigher is better
Share of all 806 scored URLs where the tool returned markdown with at least 10% of the expected content. Errors count as failures.
Hover a bar for the full breakdown.
WebCrawlerAPI vs Firecrawl, Spider, Jina Reader and Crawl4AI
Head-to-head numbers for the tools people compare us with most. Same URLs, same scoring, default settings.
WebCrawlerAPI vs Firecrawl
WebCrawlerAPI scored 88.0% end-to-end success, 0.623 extraction F1 and 8.5 s median latency. Firecrawl scored 89.7%, 0.539 and 3.6 s. On success rate WebCrawlerAPI is behind Firecrawl; on extraction F1 it is ahead of Firecrawl. Firecrawl built the dataset, so its own numbers are the reference point here. With its main-content option on, Firecrawl scored 0.537 F1; see Main content mode.
WebCrawlerAPI vs Spider
WebCrawlerAPI scored 88.0% end-to-end success, 0.623 extraction F1 and 8.5 s median latency. Spider scored 84.0%, 0.520 and 5.4 s. On success rate WebCrawlerAPI is ahead of Spider; on extraction F1 it is ahead of Spider.
WebCrawlerAPI vs Jina Reader
WebCrawlerAPI scored 88.0% end-to-end success, 0.623 extraction F1 and 8.5 s median latency. Jina Reader scored 86.8%, 0.184 and 5.5 s. On success rate WebCrawlerAPI is ahead of Jina Reader; on extraction F1 it is ahead of Jina Reader.
WebCrawlerAPI vs Crawl4AI
WebCrawlerAPI scored 88.0% end-to-end success, 0.623 extraction F1 and 8.5 s median latency. Crawl4AI Cloud scored 88.0%, 0.562 and 1.4 s. On success rate WebCrawlerAPI is tied with Crawl4AI; on extraction F1 it is ahead of Crawl4AI.
Picking a tool for a specific job, such as monitoring, screenshots or markdown for LLMs? See Firecrawl alternatives by use case.
Per-URL outcomes
Averages hide the shape. Each bar splits the 806 URLs by how much of the expected content came back, so you can see whether a tool mostly gets whole pages or lots of partial ones, and how many requests failed.
Quality vs speed
Clean markdown and fast answers pull in different directions. Top left is clean and fast. Tools that skip the browser are quick but tend to keep more boilerplate.
Main content mode
The ranking above uses each API's defaults. Some APIs also have a switch that keeps only the main content and drops navigation, footers and sidebars. We ran Context.dev and Firecrawl again over the same 1,000 URLs with that switch on. These runs are compared here, not mixed into the main ranking.
- Context.dev with
mainContentOnlywent from an F1 of 0.176 to 0.567, and noise-free responses rose from 92.6% to 98.0%. The cost: recall fell from 82.7% to 78.9%, success from 92.1% to 89.7%, and p95 latency rose from 16.0 s to 19.8 s. - Firecrawl already sets
onlyMainContentto true by default, so turning it on explicitly changed little: F1 0.539 vs 0.537, success 89.7% vs 89.2%. The differences are within run-to-run noise. - WebCrawlerAPI is shown with its defaults for reference. Its F1 of 0.623 is still the highest of any run, with or without a main-content option.
| Tool | Mode | Success | Recall | F1 | Noise-free | Errors | median | p95 |
|---|---|---|---|---|---|---|---|---|
| Context.dev | Default | 92.1% | 82.7% | 0.176 | 92.6% | 25 | 2.9 s | 16.0 s |
| Context.dev | Main content only | 89.7% | 78.9% | 0.567 | 98.0% | 26 | 3.4 s | 19.8 s |
| Firecrawl | Default | 89.7% | 79.4% | 0.539 | 98.5% | 28 | 3.6 s | 15.6 s |
| Firecrawl | Main content only | 89.2% | 78.8% | 0.537 | 98.6% | 28 | 3.6 s | 31.3 s |
| WebCrawlerAPI | Default (reference) | 88.0% | 72.8% | 0.623 | 99.1% | 41 | 8.5 s | 49.3 s |
Extraction F1, default vs main content onlyhigher is better
What each metric means
A URL is covered when at least 10% of its expected words appear in the returned markdown. That bar comes from Firecrawl's published method. Everything else builds on it.
End-to-end success
The number to use if you need a page back every time. A URL counts as a success when the API returns markdown that contains at least 10% of the expected main-content words. Timeouts, HTTP errors and empty responses count as failures.
Coverage
Same 10% bar, but errors are removed from the denominator. This matches how Firecrawl publishes coverage. A tool that fails often but scrapes well when it works can score high here, so read it next to the error rate.
Content recall
How much of the real article, product page or doc text made it into the markdown. Both texts are lower-cased, link syntax is stripped and they are split into words. Recall is the share of the dataset's truth_text words that appear anywhere in the output. Averaged over URLs that returned markdown.
Extraction F1
One number for clean and complete content. R is content recall. P is a precision proxy: each dataset row has a short sample of boilerplate (menu, footer, cookie banner, ad text). P is one minus the share of that sample's words found in the output. Dumping the whole page raises recall but lowers P, so F1 rewards tools that keep the content and drop the chrome. Tools that return the full page score low on F1 by design. This is our own approximation, not Firecrawl's F1, so don't compare it to the numbers on their site.
Noise avoidance
A stricter check on the same boilerplate sample: does the exact phrase appear in the output? Higher means fewer nav, footer and cookie blocks leaking into your LLM context.
Error rate
Requests that ended in an API-level failure, a timeout, or empty markdown after the allowed retries. URLs that returned HTTP 404 are not part of the scored set.
Median latency
Time from sending the request to the final answer, measured on our side, including retries and failed attempts. Half the URLs finished faster than this.
Slow-tail latency (p95)
1 in 20 URLs took longer than this. It shows how a tool handles slow, heavy or protected pages and matters most for synchronous pipelines with timeouts.
Methodology
- 1
Same input for everyone
Every tool got the same 1,000 URLs in the same order. No per-tool URL filtering: 133 rows are marked as broken by the dataset itself, and 61 URLs that returned HTTP 404 for at least one tool are dropped for every tool. That leaves 806 URLs, and every number is computed on them.
- 2
Default settings, cache off where possible
Each API was asked for markdown with its default extraction mode, apart from the non-default settings listed per tool in the table (Crawl4AI bypass_cache, Jina engine and image options, Crawlbase JavaScript rendering and readability, Spider single-page limit). Where an API documents a cache or max-age setting, it was set to 0 or turned off. Spider and Crawlbase document none. WebCrawlerAPI's /markdown endpoint has no cache.
- 3
Scored automatically, the same way
The harness compares each response to the dataset's truth_text and lie_text using word overlap. No model judge and no manual relabelling.
- 4
Timed on the client
Latency is wall-clock time measured by the harness, from the first request to the final answer, retries included.
- 5
Run dates
All runs covered the full 1,000 rows and were done between Sep 28 and Oct 10, 2026. Each tool's run date is in the settings table. Context.dev and Firecrawl were also run with their main-content option on.
How to read these numbers
- Use end-to-end success if you need a result for every URL. Use F1 and noise avoidance if the markdown goes straight into an LLM prompt or a knowledge base.
- Read coverage next to the error rate. Coverage leaves failed requests out.
- Some dataset URLs are now dead. WebCrawlerAPI, Crawl4AI and Spider return an error for HTTP 404 pages instead of the error page as markdown. A 404 for a missing page is a correct answer, not a failure, so the 61URLs that returned a 404 for any tool are left out of every tool's numbers (success, coverage, recall, F1, errors and latency). All tools are scored on the same 806 URLs. Tools that return content for dead pages, from a cache or a soft-404 page, are not rewarded for it, and tools that return a 404 are not penalised. Only a tool that actually returned a 404 can mark a URL as dead, so a URL that is dead but where every tool returned something is still scored.
- Latency depends on where you run from. Compare tools against each other, not against your own SLA.
What this benchmark does not prove
- ✕It does not cover every kind of site. It measures a 1,000-URL sample from one public dataset.
- ✕Precision is a proxy. The dataset has a short boilerplate sample per page, not the full page, so true precision cannot be computed.
- ✕Word overlap does not judge markdown structure: headings, tables and code blocks are not scored.
- ✕It does not measure price, full-site crawling, support or uptime over time.
- ✕WebCrawlerAPI's /markdown endpoint is a dedicated markdown endpoint with extra content cleaning, which helps its F1 and noise scores. The other tools in the main ranking return their default markdown unless noted.
- ✕We build one of the tools tested. That is why the dataset and scoring are third-party and every setting is listed above.
Scoring code
The harness and the exact scoring function it runs on every response are open source. You can apply them to the dataset and any API output yourself: WebCrawlerAPI/scraping-benchmark on GitHub.
FAQ
Which scraping API had the highest success rate?+
Context.dev returned usable content on the most URLs (92.1%), ahead of Firecrawl (89.7%) and WebCrawlerAPI (88.0%).
Which tool returned the cleanest markdown?+
WebCrawlerAPI had the highest extraction F1 (0.623), so it kept the most main content while leaving out the most boilerplate. Crawl4AI Cloud was second at 0.562.
Why is coverage higher than success rate?+
Coverage ignores failed requests and only scores pages that came back. Success rate counts every valid URL. A tool with many errors can still post high coverage, so read both.
Does a main-content-only setting change the results?+
For Context.dev, a lot: F1 goes up more than threefold when its main-content option is on, at a small cost in recall and success. Firecrawl already extracts main content by default, so turning the option on explicitly changes almost nothing. See the Main content mode section.
Firecrawl vs WebCrawlerAPI: which is better?+
WebCrawlerAPI scored 88.0% end-to-end success, 0.623 extraction F1 and 8.5 s median latency. Firecrawl scored 89.7%, 0.539 and 3.6 s. Both ran on the same 1,000 URLs with default markdown settings. Firecrawl ran with its cache off; WebCrawlerAPI’s /markdown endpoint has no cache.
Who built the dataset?+
Firecrawl published it on Hugging Face as firecrawl/scrape-content-dataset-v1, together with the scoring method. We used it as published and did not add, remove or relabel URLs.
You build one of the tools. Why trust this?+
Every tool got the same URLs and default markdown settings, with the cache turned off wherever the API lets you, and every setting is listed on this page. The dataset and scoring are someone else's, and every number on this page comes from the per-URL results. Where WebCrawlerAPI loses, we show it.
Found a problem? Tell us
If you spot an issue in this benchmark, please contact us at benchmark@webcrawlerapi.com. We will review it and fix or re-run what is needed. Examples:
- A mistake in the methodology, scoring code or published numbers.
- A misconfigured provider, such as wrong parameters, a wrong endpoint or a wrong plan.
- An unfair comparison, for example a feature that was available but not enabled.
- A dataset or URL problem, such as a bad truth or noise snippet.