HTML to Markdown in Python: markdownify vs html2text vs html-to-markdown vs MarkItDown

For HTML to markdown in Python, a library does the job when the HTML is already in hand. On 10 real pages, html-to-markdown was the fastest (median 9 ms per page), markdownify and MarkItDown produced nearly the same output, and html2text was fastest of the older libraries but dropped the most links and tables. None of the four removes navigation, footers or cookie text. That part needs a selector or an extraction step. If the HTML is not in hand yet, the web scraping API fetches the page and returns markdown directly.

This matters when crawled pages feed a support bot or an AI knowledge base. Every leaked footer costs tokens on each embedding and prompt, and boilerplate repeated across pages can pull retrieval toward the wrong chunk.

This is the Python counterpart of HTML to markdown in JavaScript, and it sits inside the larger website to markdown knowledge base pipeline. All numbers below come from one run on 2026-10-28: 10 pages, 4 libraries, one Apple Silicon laptop (Darwin arm64, Python 3.11.2).

Copy-paste answer

from markdownify import markdownify

md = markdownify(html, heading_style="ATX")
print(md)

With html-to-markdown the same call is one line: html_to_markdown.convert(html).content. With html2text it is html2text.html2text(html). That is enough for a clean article body. The rest of this post covers what breaks.

HTML to markdown in Python: results at a glance

Input is the full page HTML. Time is the median across the 10 pages of each page's median over 20 runs. Leakage is the share of output tokens that do not come from the hand-labeled main region (method below).

Tables, code and links were scored on the pre-extracted main region, so the input is the same for every library. The API row is compared on tokens and leakage only, for reasons given in the API section.

Three verdicts from the data:

• Speed: html-to-markdown is about 6x faster than markdownify.
• Output: markdownify and MarkItDown are the same engine and score the same.
• Leakage: every library lets 5% to 6% of tokens through from outside the main content. Pre-extracting the region cut that to about 1% for three of them.

Reruns moved medians by around 10%, so read small gaps as ties.

How the benchmark was run

Versions came from pip freeze in a venv: markdownify 1.2.3, html2text 2025.4.15, html-to-markdown 3.19.0, markitdown 0.1.9, tiktoken 0.14.0.

Each page was fetched once, after a robots.txt check, with a descriptive User-Agent and a 3 second pause. The HTML was cached on disk, so the timings contain no network.

robots.txt was checked for every host before the fetch and allowed all 10 paths. All pages were fetched between 14:46 and 14:47 UTC.

Metrics, one line each:

1. Time: 1 warmup plus 20 runs per page per library, in-process. Import time is measured separately.
2. Tokens: cl100k_base only, counted on the full output.
3. Tables: source <table> count and rows against pipe tables and rows in the output.
4. Code: source <pre> blocks against fenced blocks, and whether the text and indentation survived.
5. Links: source <a href> against [text](url).
6. Leakage: see below. The unit is tokens, chosen before running.

For each page one label was set by hand: a CSS selector for the main region (main, article, div[role=main]). Ground truth is the text of that element. Output words are matched against it as a multiset after dropping link targets and undoing markdown escapes. Leakage is the token share of output words that have no match. This is one person's labeling over 10 pages with a text-match heuristic. It is not a formal extraction metric. A word present in both navigation and the article is absorbed by the article count, so leakage is slightly undercounted.

Underscores are not counted as word characters, because html2text writes emphasis as _word_ and would otherwise look leakier than it is. The method follows Python HTML parsers compared. The core of the run is below.

converters = {
    "markdownify": lambda h: markdownify.markdownify(h, heading_style="ATX"),
    "html2text": lambda h: html2text_nowrap(h),  # HTML2Text with body_width = 0
    "html-to-markdown": lambda h: html_to_markdown.convert(h).content,
    "markitdown": lambda h: md_it.convert_stream(io.BytesIO(h.encode()), stream_info=info).markdown,
}
for name, fn in converters.items():
    fn(html)  # warmup
    times = [timed(fn, html) for _ in range(20)]
    md = fn(html)
    tokens = len(tiktoken.get_encoding("cl100k_base").encode(md))

The four libraries

markdownify

markdownify walks a BeautifulSoup tree. Defaults emit setext headings (underlined), and heading_style="ATX" switches to #. The token difference was 98 of 146,388 across all pages. It kept 27 of 27 tables, but an empty header row is added when the source has no <thead> (229 output rows for 219 source rows).

html2text

The default wraps lines at 78 columns. Setting body_width = 0 cut total tokens from 140,792 to 137,031 (2.7%) and median time from 29 ms to 22 ms. It writes code as 4-space indented blocks (1 fence in 353 blocks) and writes tables as a | b rows with a ---|--- separator and no outer pipes. It drops links that point to a #fragment on the same page (its skip_internal_links default is on), which covers most table-of-contents links on docs pages. That is why only 70.6% of unique link targets survived.

html-to-markdown

The import name is html_to_markdown and the call is html_to_markdown.convert(html).content. It is maintained under the xberg-io GitHub organization (Kreuzberg). It is Rust-backed: the wheel is built with maturin, ships a compiled _html_to_markdown.abi3.so, and the binary contains pyo3 and Cargo crate paths.

Two defaults cost tokens. It prepends YAML frontmatter with page metadata, and it turns inline <svg> into base64 data:image/svg+xml images. The Guardian page produced 26,989 tokens with defaults and 7,802 with extract_metadata=False, strip_tags=["svg"] (the "tuned" variant). Across all 10 pages the tuned total is 139,140 tokens against 160,296 by default. A third default causes real damage: on Fowler's article it dropped 12.6% of the labeled words, because its preprocessing removes elements with the class sidebar, and the article keeps real content in <aside class='sidebar'> boxes.

markitdown python

MarkItDown (Microsoft) is built to convert many file types and takes a file or stream, not a string. For HTML it parses with BeautifulSoup (html.parser), removes script and style, and runs a markdownify.MarkdownConverter subclass on <body>. This was checked by reading the installed 0.1.9 source (markitdown/converters/_html_converter.py and _markdownify.py). The output was within 0.3% of markdownify in tokens, and table and code scores match to within one block.

The extra cost is startup: importing and creating MarkItDown() took 738 ms in the benchmark venv (installed with extras) and about 240 ms in a clean venv with only pip install markitdown, against about 55 ms for markdownify.

Results

Speed

The slowest page was the 331 KB Python built-in functions reference (222 ms in markdownify, 34 ms in html-to-markdown). Fetching the same 10 pages took a median of 155 ms each (68 to 995 ms). In a crawl the fetch dominates, so conversion speed matters for bulk jobs over cached HTML.

Tables

27 <table> elements and 219 rows sat in the labeled regions, 23 of them on the Wikipedia page. Cell text survived everywhere (99.6% to 100% of cells found). The shape did not.

The Flask "converter types" table has 5 rows and no header. markdownify and MarkItDown add an empty header row. html-to-markdown and html2text promote string to the header. For the Python functions table, whose header cell spans 4 columns, markdownify emits | Built-in Functions | | | |, and html2text emits no table at all, only text. html2text kept 23 of 27 tables as tables with a separator row. For html table to markdown work, markdownify and MarkItDown are the safest.

Code blocks and links

There were 353 <pre> blocks. markdownify, html-to-markdown and MarkItDown wrote 335 fenced blocks each, and none of those fences carried a language tag, even though the Sphinx pages mark the language on a wrapper class. Exact text and indentation matched on 322, 323 and 321 blocks. html2text indents code instead of fencing it and matched 234 exactly, though the text was found in 350.

Links: 1,869 <a href> in the regions. html-to-markdown kept 99.1% of unique targets, markdownify 93.4%, MarkItDown 92.3% and html2text 70.6%. The misses for the first three were under 8% and not every one was traced.

Tokens

Totals over the 10 full pages, cl100k_base: html2text without wrapping 137,031, tuned html-to-markdown 139,140, html2text default 140,792, markdownify (ATX) 146,290, MarkItDown 146,772 and html-to-markdown with defaults 160,296. Differences come from wrapping, escaping (markdownify writes FIRST\_COMPLETED), image syntax and the two html-to-markdown defaults. See HTML vs markdown as an output format for why markdown is worth the conversion.

The problem none of them fix: boilerplate

Pooled over 10 pages, between 5.1% and 6.4% of output tokens came from outside the labeled main region (4.6% for tuned html-to-markdown). Medians are lower (3.6% to 6.1%) because one page drags the pool. On the Guardian front page 45% to 53% of tokens were navigation, footers and promo text, with only about 6,000 characters of headlines in the labeled region.

Pre-extracting the region with select_one(selector) and converting only that cut pooled leakage to 1.2% for markdownify, html2text and MarkItDown, and 2.7% for html-to-markdown. Median tokens fell from 12,090 to 10,962 for markdownify.

The catch is the selector: one per site, written by hand, broken by markup changes. The 10 pages from 8 sites needed 4 distinct selectors, and each page had to be opened to pick one. For a crawl of 500 domains it is a maintenance job.

Same pages through the crawl API

The same 10 URLs went through the POST /v2/scrape endpoint, one call per page, 10 calls in total:

curl --request POST \
  --url https://api.webcrawlerapi.com/v2/scrape \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "https://martinfowler.com/articles/microservices.html",
    "output_formats": ["markdown"],
    "main_content_only": true,
    "respect_robots_txt": true
  }'

import requests

r = requests.post(
    "https://api.webcrawlerapi.com/v2/scrape",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"url": url, "output_formats": ["markdown"],
          "main_content_only": True, "respect_robots_txt": True},
)
markdown = r.json()["markdown"]

clean_selectors accepts extra CSS selectors to strip, as described in the cleaning guide.

The comparison has a caveat. The libraries read HTML cached at 14:47 UTC, and the API fetched the live pages 12 minutes later (14:58 to 14:59 UTC). A headline page like the Guardian may have changed in between. For that reason the API is compared on tokens and leakage only, not on tables, code or links.

On the Guardian page the API returned 3,274 tokens against 7,887 for markdownify. It also lost some content: recall on that page was 93.5%, and on Fowler's article 86.1% (html-to-markdown scored 87.4% there, for the sidebar reason above). If your labeling counts sidebars as main content, the API trades some recall for lower leakage. It returned no images in any of the 10 responses, which matches img being in the default clean_selectors list. It also glued a kicker into a headline on the Guardian page ("UKAI bots flooded..."), a defect the libraries did not have.

Latency was a median of 3.1 s per page (2.7 to 4.7 s), against 9 to 60 ms for conversion alone. That is not a like-for-like comparison, because the API fetches and renders the page. The response carries no cost field. By the pricing docs (read 2026-10-28) pay as you go is $0.002 per page, which makes 10 pages $0.02.

A library with a hand-picked selector beat the API on leakage: pre-extracted markdownify leaked 1.2% pooled against the API's 2.3% on full pages.

Which one to use

• HTML in hand, bulk conversion, offline: html-to-markdown, with extract_metadata=False and strip_tags=["svg"]. Check that your pages do not put content in elements with a sidebar class.
• Smallest surprises, tables matter: markdownify with heading_style="ATX".
• Already using MarkItDown for PDFs and Office files: it is fine for HTML, with the same output as markdownify.
• html2text: only for plain-text-style output. It lost 4 of 27 tables and 29% of unique link targets here.
• Known site, one selector, local control: a library plus select_one. It beats the API on leakage and costs nothing per page.
• You need the fetch, JavaScript rendering and cleaning anyway: the crawl API returns markdown in one call. /tools/url-to-md does a single page without code, and the web scraping API page has the full set. Set respect_robots_txt: true; the default is false.

Next steps

• Crawl a whole site to files: Crawl a website to markdown files.
• Cut the output into pieces: Chunking strategies for a knowledge base.
• The JavaScript side of the same problem: HTML to markdown in JavaScript.
• Get an API key at webcrawlerapi.com/crawl-api and read the scrape endpoint docs. The web scraping API page covers single pages.