Embedding Models for Website Content: OpenAI vs BGE vs Voyage vs Ollama on Real Docs

The best embedding model for website docs is, on this test, a tie. Seven models (eight setups, because bge-m3 ran both hosted and on Ollama) embedded the same 1,389 chunks from 130 crawled pages and were scored on the same 93 questions. Six of the other seven setups landed within noise of text-embedding-3-small, including the 6.5 times pricier text-embedding-3-large. One was clearly worse. None was clearly better. The pick under the rule fixed before the run is text-embedding-3-small, and it stays for continuity and cost, not because it won.

"Ollama" in this post means running nomic-embed-text and bge-m3 on a laptop. This is the measured version of the embedding choice in the website to markdown knowledge base guide, and it picks up after the chunking test, whose recursive splitter produced the chunks used here.

Best embedding model for website docs: quick answer

Tests run and numbers read November 1, 2026. text-embedding-3-small is the anchor, the model used in the earlier posts, and the other models are compared with it. Section recall@1 is the share of questions where the top returned chunk holds the answer, and recall@5 is the same for the top five. MRR@5 averages 1 divided by the rank of the first correct chunk (0 if none is in the top five). Recall figures are pooled over both sites. "Tied" means the paired 95% interval against text-embedding-3-small included zero.

voyage-4 had the highest recall@1 (66.7%) and qwen3-embedding-8b the next (64.5%). Neither separated from the anchor on 93 questions. voyage-4 also led on recall@5, MRR@5 and on the second question set. Those leads are exploratory (picked after scoring, no intervals on the second set). If you test one model against your own pages, test voyage-4.

How the test works

Corpus, questions and chunking

The pages are the 130 pages from the chunking test, crawled with the WebCrawlerAPI crawl endpoint: 80 from the FastAPI docs and 50 from the htmx docs. The questions are the same 93 (46 FastAPI, 47 htmx), each with a gold passage inside one page. Each question was written by gpt-4.1-mini from a whole page, with the answer quote copied from that page.

Chunks come from the unchanged chunker B of the previous post (see chunking strategies for the alternatives): LangChain RecursiveCharacterTextSplitter with the markdown separators, 400 tokens, 60 overlap. That gives 1,389 chunks, about 10.7 per page, with a median of 260 and a maximum of 399 tokens (cl100k). Only the model changes between runs. Hosted models were reached through OpenRouter with the upstream provider pinned and fallbacks off, so every vendor name below means "as served by OpenRouter", not the vendor's own API.

Metrics and how differences are judged

The primary metric was fixed before any model ran: pooled section recall@1. A chunk counts as a hit when it covers at least half of the answer quote. Recall@5, MRR@5 and page recall are reported but carry no claims.

Differences were judged with a paired bootstrap over gold pages (1,000 resamples, seed 0). The list of comparisons was fixed: each of the seven other setups against text-embedding-3-small, plus bge-m3 on Ollama against bge-m3 hosted. A model counts as better than the anchor only if the 95% interval of the paired recall@1 difference excludes zero and the sign agrees on both sites. Otherwise it is tied within noise. No correction for the eight comparisons was applied.

With 93 questions the intervals are wide. The paired interval for voyage-4 against the anchor spans 19.7 points. Differences under about 10 points cannot be resolved here.

Before scoring anything else, text-embedding-3-small was run again through the same pipeline. It reproduced the previous post exactly: 63.4% recall@1, 83.9% recall@5, 0.710 MRR@5, with 0 of 93 questions changing rank.

The models

Prices are from the OpenRouter model pages and limits from the model cards and the Ollama pages, all read on November 1, 2026. Word pieces are the sub-word tokens a model's own tokenizer produces. They are counted differently from the cl100k tokens used for chunking. Two extra rows are descriptive only and left out of every comparison: MiniLM with max_seq_length=512 and text-embedding-3-large with dimensions=1024.

The format traps decide whether a model gets a fair test. Each was probed on three short strings before any scoring, and the formats were frozen before the first run. The strings are copied from the model cards: nomic's two prefixes, and Qwen3's Instruct: Given a web search query, retrieve relevant passages that answer the query plus \nQuery: before the question. Chunks get no instruction.

OpenAI embeddings: text-embedding-3-small and text-embedding-3-large

text-embedding-3-small is the anchor and the model used in the earlier posts of this series. It scored 63.4% recall@1 and 94.6% page recall@5 and cost $0.0053 to embed the corpus and the questions.

text-embedding-3-large scored 60.2% recall@1, 3.2 points lower, with a paired interval of -11.5 to +6.0 and opposite signs on the two sites. That is a tie, at 6.5 times the price. Its recall@5 was higher (88.2% against 83.9%), a point that comes back in the results.

The dimensions parameter went through OpenRouter. With dimensions=1024, the model returned 1,024 numbers, and they matched the full vector shortened to 1,024 and re-normalized (cosine 0.99999996). Recall@1 stayed at 60.2% and MRR@5 moved from 0.720 to 0.712. For this corpus, a shortened 3-large costs nothing measurable in recall and takes a third of the storage.

BGE-M3

bge-m3 was run twice: hosted by DeepInfra through OpenRouter, and locally through Ollama (F16, 567M parameters). Both scored 57.0% recall@1 and 84.9% recall@5. The chunk vectors agreed to a mean cosine of 1.0000 (minimum 0.9995), and the top-1 chunk was identical for all 93 questions. One question differed in section rank, which moved MRR@5 from 0.686 to 0.688. Hosted and local bge-m3 are the same model for retrieval purposes.

Against the anchor, bge-m3 was 6.5 points lower (hosted interval -15.2 to +2.1, Ollama -16.5 to +2.2), with the same sign on both sites. It is tied within noise, but it is the lowest of the tied group and the interval almost clears zero.

The surprise came from serving, not the model. DeepInfra answered HTTP 404 "No endpoints found" to batches of 32 or more chunks, while single chunks passed. The run uses batches of 16 and splits a refused batch in half.

Ollama embeddings with nomic-embed-text

nomic-embed-text on Ollama scored 61.3% recall@1 and 86.0% recall@5, 2.1 points below the anchor (interval -10.9 to +5.6). It needs the two prefixes. Without them the model gets different text on each side, so they are part of the setup, not a tuning option.

Ollama reports a context of 2,048 for the model. In nomic's own tokenizer the longest of the 1,389 chunks (with the prefix) is 871 word pieces and none exceeds 2,048, so nothing was truncated. Chunks tokenize longer in word pieces than in cl100k (median 329 with the prefix, against 260), which is the number to watch on other corpora.

On an M2 Max it embedded 75 chunks per second and answered a single query in 12 ms at the median. The first call after loading took 0.55 seconds. Disk size is 274 MB for nomic-embed-text and 1.2 GB for bge-m3.

Voyage embeddings

voyage-4 had the highest recall@1 (66.7%), recall@5 (91.4%) and MRR@5 (0.773) of all eight setups, and the paired interval against the anchor was -6.7 to +13.0 points with the same sign on both sites. By the rule fixed in advance this is a tie. The leads on recall@5 and MRR@5 are descriptive.

OpenRouter did pass input_type to Voyage. The vectors for the same text change (cosine 0.857 against no parameter for document, 0.960 for query), so chunks were sent as document and questions as query. The model is labelled "as served by OpenRouter" because the route, not Voyage's own API, was tested.

Embedding the corpus and questions cost $0.0159. At $0.06 per million tokens that is three times the anchor.

Sentence transformers and MiniLM

all-MiniLM-L6-v2 is the sentence-transformers model used in the pgvector post. At 400-token chunks it is the wrong fit. The card says input longer than 256 word pieces is truncated, and 854 of the 1,389 chunks (61.5%) are longer than that in its tokenizer. On average only 78.7% of a chunk reaches the model.

It scored 44.1% recall@1, 19.4 points below the anchor, with an interval of -29.5 to -10.6 and the same sign on both sites. This is the one clear difference of the test.

Raising max_seq_length to 512 did not help: 40.9% recall@1 (descriptive row, no interval computed). Only 45 chunks (3.2%) exceed 512 word pieces, so the longer window would have helped few chunks. The fix for MiniLM is smaller chunks, a change to the chunker that this test did not make. It ran on the Mac's GPU (mps) at 656 chunks per second, the fastest of all.

Open source embedding models vs hosted

Among the open weights models, qwen3-embedding-8b scored 64.5% recall@1 and 91.4% recall@5, nomic-embed-text 61.3% and bge-m3 57.0%. All three are tied with the hosted anchor. qwen3-embedding-8b was run on DeepInfra, because an 8B model is not a laptop job here. Its instruction goes on queries only, and its vectors have 4,096 dimensions.

The price gap is in operations, not in the per-token rate. Hosted Qwen3 is $0.01 per million tokens. The local models cost $0 in this table because the hardware is not counted: a machine to run, update and keep warm is the cost.

Results

Retrieval quality

The bars show each model's own interval, which is wide and overlaps heavily. They are not the paired intervals used to judge differences. The table has those.

Details per model. The two extra rows are descriptive and not compared with the anchor. The full tables, including page recall, are in results.md.

Which differences are real

Of the seven registered comparisons against the anchor, six are tied within noise and one (MiniLM) is worse. Nothing is better. The bge-m3 pair, hosted against Ollama, differs by 0.0 points.

For tiers, the best point estimate is voyage-4. Tier 1 is every model whose paired interval against it includes zero: voyage-4, text-embedding-3-small, text-embedding-3-large, qwen3-embedding-8b, nomic-embed-text and bge-m3 on Ollama. Hosted bge-m3 falls below (+9.7 points, interval +1.0 to +18.8), and so does MiniLM. The line is arbitrary where it splits bge-m3: the Ollama copy, with the same recall@1, has an interval starting at +0.0. Tiers are not transitive, so read them as "these six cannot be told apart from the leader", not as a ranking.

Recall@5 shows something that recall@1 hides. The intervals against the anchor for text-embedding-3-large, voyage-4 and qwen3-embedding-8b sit above zero on recall@5 (differences of +4 to +8 points). Recall@5 was not the registered metric, and no correction was applied across the many comparisons, so this is a lead for a follow-up and not a result. Some models (3-large, voyage-4, qwen3-embedding-8b) seem to put the right chunk in the top 5 more often without moving it to the top. That matters if a reranker or an LLM reads all five.

Checking for question-writer bias

The 93 questions were written by an OpenAI model, which could favour OpenAI embeddings. A second set was generated and frozen before any model was scored: the same 32 sampled pages, the same filters, and a different writer, anthropic/claude-haiku-5.5. It produced 113 questions (59 FastAPI, 54 htmx), and 12 were hand-checked and kept. It is a direction check, never pooled with the first set, and carries no intervals.

The order is mostly stable: Spearman correlation 0.90 for recall@1 across the eight main setups (the two extra rows are excluded). voyage-4 and qwen3-embedding-8b are first and second on both sets, and MiniLM is last on both. The anchor did not look inflated by its writer: it led bge-m3 by 6.5 points on set 1 and by 3.6 on set 2. Against nomic-embed-text the sign flipped, +2.1 on set 1 and -0.9 on set 2. That is what a tie looks like.

A second limit applies to both sets. The questions were written with the answer quote in view, so they share words with the gold chunk. That favours models that match surface words and compresses the differences between models.

Where the models fail

Five of the 93 questions had the gold section in no model's top 5, and 14 had it at rank 1 for no model. Three of the five show it:

• "How can I convert input data to JSON-compatible format before saving it with FastAPI?" The gold passage is in the PUT section of the body-updates page. text-embedding-3-small returned the page about jsonable_encoder, which is arguably a better page for the question.
• "What should I be careful about when returning a Response object directly?" The gold passage is on the additional-status-codes page. The top result was the page titled "Return a Response Directly".
• "How can I show a confirmation dialog before an htmx request is sent?" The gold passage is one row of the htmx reference table for hx-confirm, and the top result was the general docs page.

Page-level recall@5 is between 91% and 96% for every model, so the models find the right page. They miss when a neighbouring page covers the same topic or when the gold passage is a single table row. MiniLM's miss is different: its recall@5 is 74.2%, and the cut-off at 256 word pieces removes the end of the 61.5% of chunks that are longer.

Cost, speed and storage

Embedding cost per 1,000 pages

Billed tokens were read from the API usage field for the 130 pages (the 93 questions are subtracted). Tokenizers differ, so billed tokens are not comparable across rows: bge-m3 counts 2,608 tokens per page against 2,018 for OpenAI. Per 1,000 pages is an extrapolation from two docs sites.

The token count per page depends on markdown size, which is why markdown costs fewer tokens than HTML.

Crawl plus embed per 1,000 pages

The crawl is the larger cost. WebCrawlerAPI pricing is $0.002 per page on pay as you go, $0.0015 on Professional and $0.001 on Business, which is $2.00, $1.50 and $1.00 per 1,000 pages. Embedding adds between $0.02 and $0.26 to that.

The most expensive model here, text-embedding-3-large, adds 13% to the pay-as-you-go crawl. The anchor adds 2%. The plan moves the bill more than the model does.

Speed

Hosted and local numbers measure different things and are in two tables. Each is the median of three repeats on 128 chunks at batch 64 (the 128 chunks are every tenth chunk of the corpus), plus 20 warm single-query calls. Hosted figures include the network and OpenRouter routing from one laptop, not the vendor's raw speed.

For a search box the single-query number is the one that counts. A hosted call costs about half a second to nearly a second, a local call 6 to 37 ms. The three repeats ran within a few minutes of each other, so network variation over a day is not captured. ollama ps was empty before every timing run.

Vector size and pgvector limits

Storage is chunks per page (10.7) times dimensions times 4 bytes, before database overhead (pgvector adds 8 bytes per vector).

The pgvector README (read November 1, 2026) says a vector column can be indexed up to 2,000 dimensions and halfvec up to 4,000. So text-embedding-3-large at 3,072 dimensions needs halfvec or the dimensions parameter, and qwen3-embedding-8b at 4,096 sits past the halfvec limit. The Qwen3 card says it supports user-defined dimensions, which this test did not run. The pgvector post covers the schema, and the vector database comparison lists the limits of the other stores.

Which embedding model to use

Stay on text-embedding-3-small. Under the rule fixed before the run, nothing beat it, so the series default stays for continuity and cost. Had a model beaten it, price and dimensions would have decided.

• Hosted default: text-embedding-3-small. 63.4% recall@1, $0.040 to embed 1,000 pages, 1,536 dimensions.
• Free and local: nomic-embed-text on Ollama, with both prefixes. It is tied with the anchor, runs at 75 chunks per second on a laptop and has 768 dimensions. Check that your chunks fit its 2,048 context. It is also the option when pages cannot leave your network.
• Worth a retest on your pages (exploratory, selected after scoring): voyage-4, first on both question sets, at three times the anchor's price. qwen3-embedding-8b is second on both and cheaper per token, but has 4,096 dimensions.
• Pay for text-embedding-3-large: not for recall@1 here. Its recall@5 was higher, so test it if a reranker or an LLM reads the top 5, and shorten it with dimensions.
• Avoid: all-MiniLM-L6-v2 with 400-token chunks, unless the chunks are cut to fit 256 word pieces.

Changing the model means re-embedding everything, since vectors from different models do not mix. Key the cache by model id and format string, as in the embedding cache of the build post.

Not tested: rerankers, hybrid search, languages other than English, code-heavy corpora, long documents, and Cohere, Gemini, Jina and Mistral embeddings.

Limitations

• 93 questions across 130 pages. Intervals about 20 points wide hide any difference under roughly 10 points.
• Two English docs sites with short technical prose, one chunker, one run per model. Hosted models can change without notice, and vendors can change prices.
• The first question set was written by one OpenAI model. The second set is a direction check with a different writer and has no intervals. Both were written with the answer quote visible.
• Latency comes from one laptop and one network. Extrapolated costs assume other sites tokenize like these two.
• bge-m3 hosted was embedded twice after a batch-size failure. The reported run is the full second one, with one question differing in rank from the first (details in the code README).
• The comparisons have no multiplicity correction. This blog belongs to WebCrawlerAPI, whose crawl price appears in the cost table. No model vendor was involved in the test.

Run it on your own site

Crawl your docs to markdown with WebCrawlerAPI (crawl docs, crawl API), cut them with the chunker from the chunking post, and score your shortlist with the scripts in the code folder. The adapter below is the useful part: formats per model, one call for hosted and one for Ollama. It was run against all three models shown.

import os, requests
TASK = "Given a web search query, retrieve relevant passages that answer the query"
FORMATS = {  # model -> (document format, query format)
    "openai/text-embedding-3-small": ("{t}", "{t}"),
    "qwen/qwen3-embedding-8b": ("{t}", "Instruct: " + TASK + "\nQuery:{t}"),
    "nomic-embed-text": ("search_document: {t}", "search_query: {t}"),
}

def embed(model, texts, kind="doc"):  # kind is "doc" or "query"
    fmt = FORMATS[model][0 if kind == "doc" else 1]
    inputs = [fmt.format(t=t) for t in texts]
    if model == "nomic-embed-text":  # local Ollama, /api/embed returns unit-length vectors
        r = requests.post("http://localhost:11434/api/embed", json={"model": model, "input": inputs, "truncate": True}, timeout=600)
        r.raise_for_status()
        return r.json()["embeddings"]
    r = requests.post("https://openrouter.ai/api/v1/embeddings", timeout=180, json={"model": model, "input": inputs},
                      headers={"Authorization": "Bearer " + os.environ["OPENROUTER_API_KEY"]})
    r.raise_for_status()
    return [x["embedding"] for x in sorted(r.json()["data"], key=lambda x: x["index"])]

Normalize hosted vectors yourself (L2) before taking dot products. The scripts do this, and the full embed.py also tracks spend and aborts above $2.

Next step: store the vectors

Embeddings have to be stored somewhere. The vector database comparison and the pgvector walkthrough cover that step, and the pillar guide covers the whole pipeline. The follow-up posts on specific databases are planned in this series.