Weaviate Tutorial: Hybrid Search Over a Crawled Docs Site

I wanted to know whether adding BM25 to vector search fixes the questions that vectors miss. This post loads 52 crawled docs pages (304 chunks) into Weaviate 1.40.3 running in Docker, queries them three ways, and scores 28 labeled questions. The short answer: hybrid search at alpha 0.5 found the right page in the top 5 for 27 of 28 questions. Vector search alone found 26, and BM25 alone found 21. The gap between hybrid and vector is one question. The gap between hybrid and BM25 is six, all of them paraphrased questions.

Vectors come from a local embedding model that I supply myself ("bring your own vectors"), so Weaviate needs no vectorizer module. The same chunk file feeds the Qdrant and Supabase pgvector tutorials, and the whole pipeline is described in the website to markdown knowledge base guide.

Tests run and numbers read November 2, 2026. I run WebCrawlerAPI, which crawls the pages that feed this kind of knowledge base, and I have no deal with Weaviate. This ran on a laptop. Each configuration was scored once on the frozen question set, and the query was repeated three times; rankings were identical. I did not tune anything after seeing results.

Weaviate hybrid search Python example

This is the whole query path, assembled from the scripts that produced the numbers below (the final print loop is new, the functions are unchanged) (ingest.py and search.py in the code folder). It assumes a Weaviate collection DocsChunk that is already loaded (steps below) and a local Ollama server with nomic-embed-text.

import requests
import weaviate
from weaviate.classes.query import HybridFusion, MetadataQuery

OLLAMA = "http://localhost:11434/api/embed"
MODEL, DIM = "nomic-embed-text", 768

def embed(texts, prefix):
    resp = requests.post(OLLAMA, json={"model": MODEL, "input": [prefix + t for t in texts]}, timeout=300)
    resp.raise_for_status()
    return resp.json()["embeddings"]

def embed_query(text):
    """Embed a search query (uses the nomic 'search_query: ' prefix)."""
    return embed([text], "search_query: ")[0]

PROPS = ["heading", "text"]  # BM25 searches these two properties, not url or chunk_id
RET = ["url", "heading"]

def hybrid(docs, q, alpha=0.5, fusion=HybridFusion.RELATIVE_SCORE, k=20):
    return docs.query.hybrid(query=q, vector=embed_query(q), alpha=alpha, fusion_type=fusion,
                             query_properties=PROPS, limit=k, return_properties=RET,
                             return_metadata=MetadataQuery(score=True)).objects

with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:
    docs = client.collections.get("DocsChunk")
    for o in hybrid(docs, "name_not_resolved", 0.5)[:3]:
        print(o.metadata.score, o.properties["url"], o.properties["heading"])

The query vector must use the same model and the search_query: prefix, or results degrade quietly (numbers in the gotchas section).

What is Weaviate and when to pick it

Weaviate is a vector database that also ships a BM25 keyword index, so one collection can answer keyword, vector and hybrid queries. The repository says most of its code is under the BSD 3-Clause License, with files in a wl/ folder under a commercial license (GitHub README, read November 2, 2026). It also sells a managed cloud. I did not create a cloud account for this post, so I make no claims about it. The vector database comparison has dated limits and prices for the cloud tiers.

I would pick Weaviate when hybrid search is the main requirement and I want it in the database rather than in my own code. For a pure vector search on a few hundred chunks, a numpy array is enough, as shown in build a knowledge base from a website in Python.

How Weaviate hybrid search works: BM25 plus vectors

That Python post ended with a miss. A differently worded question about running code after a response ranked a middleware section first, and the background tasks page was not in the top five. The post said hybrid search or a reranker could help and tried neither. That corpus was the FastAPI docs. This one is the WebCrawlerAPI docs, so this is the same kind of failure on different pages. I did not re-run the FastAPI question.

The two methods fail in opposite ways. BM25 needs the question to share words with the page, so a paraphrase gets nothing. A vector model matches meaning (semantic search), but a rare identifier such as an environment variable name may land near the wrong neighbours. For how BM25 scores terms, see putting 1,000 pages of documentation into qmd. I skip the internals here.

Hybrid search in Weaviate runs both queries and fuses the scores. The alpha parameter weights them: 0 is BM25 only, 1 is vector only.

Run Weaviate locally with Docker Compose

The compose file is the sample from the Weaviate Docker installation page (read November 2, 2026), with the image tag pinned, both ports bound to 127.0.0.1, no vectorizer module and telemetry off. Weaviate collects telemetry by default, according to the same page. Anonymous access stays on because nothing leaves my machine.

name: kb-weaviate
services:
  weaviate:
    command: ["--host", "0.0.0.0", "--port", "8080", "--scheme", "http"]
    image: cr.weaviate.io/semitechnologies/weaviate:1.40.3
    ports:
      - "127.0.0.1:8080:8080"
      - "127.0.0.1:50051:50051"
    volumes:
      - weaviate_data:/var/lib/weaviate
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'none'
      DISABLE_TELEMETRY: 'true'
      CLUSTER_HOSTNAME: 'node1'
volumes:
  weaviate_data:

Before starting I checked that ports 8080 and 50051 were free (lsof -i :8080 -i :50051 printed nothing), because another stack on this machine already runs its own Weaviate. The project name keeps this one separate.

docker compose -p kb-weaviate up -d
curl -s -o /dev/null -w '%{http_code}' localhost:8080/v1/.well-known/ready   # 200 when ready

The screenshot omits the image pull lines. The cleanup later is docker compose -p kb-weaviate down -v, which removes this project's container, network and volume and nothing else.

Python client v4

I used weaviate-client 4.20.0 on Python 3.11. The v4 client talks REST on 8080 and gRPC on 50051, so both ports are passed in.

import weaviate

with weaviate.connect_to_local(port=8080, grpc_port=50051) as client:
    print(client.is_ready())

The context manager closes the connection.

Load the crawled chunks with your own vectors

The data is the WebCrawlerAPI docs site, crawled to markdown and split by headings: 304 chunks from 52 pages. Each chunk carries url, heading, text and a 768-dimension vector from nomic-embed-text, served by a local Ollama. Why this chunker and this model: chunking markdown by headers and embedding models for website content. The file has no secrets and needs no API key, because the embeddings are local.

The dataset came from the webcr command-line crawler. This is the crawl step of ingest.py, which is what produced the 52 pages:

def crawl():
    # Shells out to the webcr CLI: saves every crawled page as <url>.md in ./pages
    subprocess.run(["webcr", SITE, "-l", "60", "-d", "3", "-w", "/docs", "-o", str(PAGES)], check=True)

SITE is https://webcrawlerapi.com/docs. For your own site, use the WebCrawlerAPI crawl API to get markdown pages the same way: items_limit is required, output_formats defaults to [markdown] (crawl docs), and pay-as-you-go pages cost $0.002 each (pricing). Repeat crawls inside the default 7-day cache window return cached results unless you set max_age to 0 (caching docs). I did not run the crawl API itself for this post, only the CLI above. The crawl-and-chunk code that I did run against the API is in build a knowledge base from a website in Python.

load.py creates the collection and inserts the chunks:

docs = client.collections.create(
    "DocsChunk",
    vectorizer_config=Configure.Vectorizer.none(),
    vector_index_config=Configure.VectorIndex.hnsw(distance_metric=VectorDistances.COSINE),
    properties=[
        Property(name="chunk_id", data_type=DataType.TEXT, tokenization=Tokenization.FIELD),
        Property(name="url", data_type=DataType.TEXT, tokenization=Tokenization.FIELD),
        Property(name="heading", data_type=DataType.TEXT),
        Property(name="text", data_type=DataType.TEXT),
    ],
)
with docs.batch.dynamic() as batch:
    for r in rows:
        batch.add_object(
            properties={"chunk_id": r["id"], "url": r["url"], "heading": r["heading"], "text": r["text"]},
            vector=r["embedding"],
        )
print("failed objects:", len(docs.batch.failed_objects))
print("count in Weaviate:", docs.aggregate.over_all(total_count=True).total_count)

Cosine is set because it is the usual metric for text embeddings. url and chunk_id use field tokenization, so each is one token and neither is searched by BM25. heading and text use the default word tokenization and are the two properties every BM25 query below searches.

I check failed_objects and the stored count after every load.

Query it three ways

search.py holds all three modes. The query string is embedded with the search_query: prefix that nomic-embed-text expects, using the same helper that built the index.

BM25 only

docs.query.bm25(query=q, query_properties=["heading", "text"], limit=20,
                return_metadata=MetadataQuery(score=True))

Weaviate's BM25 defaults are word tokenization, k1 1.2, b 0.75 and the or operator, so a chunk needs only one query token to match (Weaviate docs, read November 2, 2026).

Vector only

docs.query.near_vector(near_vector=embed_query(q), limit=20,
                       return_metadata=MetadataQuery(distance=True))

The query vector has to come from the same model and prefix as the documents. I ran the vector search once with the prefix left off as a check. Results are in the gotchas section.

Hybrid with alpha and fusion

docs.query.hybrid(query=q, vector=embed_query(q), alpha=0.5,
                  fusion_type=HybridFusion.RELATIVE_SCORE,
                  query_properties=["heading", "text"], limit=20,
                  return_metadata=MetadataQuery(score=True))

Alpha weights the two legs. Weaviate offers two fusion methods, and the client has both in the HybridFusion enum. Relative score fusion normalizes each leg's scores and adds them with the alpha weights. Ranked fusion uses only each result's rank. Relative score is the default since v1.24 (hybrid search docs, read November 2, 2026). I always passed alpha explicitly, because the docs say the default depends on the client.

Here is one question run through all three:

All three put the right chunk first. The scores are not comparable across modes: BM25 scores are 5.672 and 5.349, vector scores are 1 minus a cosine distance, and the hybrid top hit is exactly 1.000 because relative score fusion rescales each query to 0 to 1. A hybrid score of 1.000 says "best in this query", not "confident".

BM25 vs vector search vs hybrid: what the run showed

How the test works

The set has 28 questions, each with one or two gold pages: 10 keyword_exact (a verbatim token from the gold chunk, such as an error code or a parameter name), 12 paraphrase and 6 mixed (one exact token plus plain words). A script checked that every exact token appears in a gold chunk and that each paraphrase shares at most 2 salient words with its gold chunk (check_overlap.py). One draft question shared four words and was reworded before the freeze.

I froze the question set (sha256 b0a68f3c...) and the parameters before the first scored run. The comparison fixed in advance was BM25, vector and hybrid at alpha 0.5. Hybrid at 0.25 and 0.75 and ranked fusion at 0.5 are a descriptive sweep.

Scoring is page level. Each query returns the top 20 chunks, which I collapse to pages in rank order, and a question counts as a hit at k if any gold page is in the first k pages. I report recall@1, recall@3, recall@5 and MRR@5. Each configuration was scored once on the frozen set, and its queries were repeated three times with identical rankings.

Question numbers below are the id values in eval_set.json. The limits are large enough that they belong before the numbers:

• An LLM (Claude) wrote the questions and the gold labels from the same chunks, which flatters keyword overlap. The same model re-read 14 gold chunks and changed none. No human reviewed the labels.
• With 52 pages, picking 5 at random would hit about 10% of the time. That is a coarse scale for a retrieval score.
• On 28 questions one question is 3.6 points. The test cannot separate methods that differ by a few points.
• The corpus has near-duplicate pages (/docs/job and /docs/api/job, for example), so some questions have two gold pages.

Results

Pooled over all 28 questions:

The chart below shows BM25, vector and the three hybrid alphas. The ranked-fusion row is only in the table.

By question type, with recall@5 as questions found out of the group size:

Counting discordant questions (one method hit, the other missed) tells more than the pooled totals. At the top 5, hybrid 0.5 found the gold page on six questions where BM25 did not (questions 11, 13, 15, 16, 17 and 20), and BM25 beat hybrid on none. Against vector, hybrid won one (question 7) and lost none. At rank 1 the picture is mixed: hybrid 0.5 against BM25 was 5 to 1 in hybrid's favour, and hybrid against vector was 1 to 2 in vector's favour.

Hybrid at 0.5 kept BM25's exact-token safety net and most of vector's paraphrase handling, and vector alone was already good on this corpus. I expected exact-token queries to favour BM25 clearly. They did not. At rank 1 both found 8 of 10 keyword_exact questions, and vector missed only one at the top 5.

Where each method fails

• An exact token that vector misses (question 7). WEBCRAWLER_API_KEY should lead to the MCP page. Vector put it at rank 6. BM25 found it at rank 3 and hybrid at rank 5, barely inside the top 5. Hybrid did not rescue it by much, because BM25 itself ranked /docs/access-key first. The word tokenizer splits WEBCRAWLER_API_KEY at underscores, so "webcrawler", "api" and "key" match many pages. I did not test field tokenization on those properties.
• A paraphrase that BM25 misses (question 11). "Abort an in-flight request early so unfinished work is not paid for" has no word in common with the cancel page's key terms. The cancel page was not among BM25's top 20 chunks. Vector ranked it third. Hybrid 0.5 ranked it fourth, below /docs/getting-started and two markdown endpoint pages. This is the failure type from the FastAPI post: hybrid recovered the page, but not at the top.
• Hybrid worse than vector (question 13). "How much money is left on my account" had the balance page first under vector. BM25 brought in /docs/errors, and hybrid 0.5 dropped the balance page to rank 4. Hybrid 0.75 put it first again, and 0.25 lost it from the top 20.
• Everyone misses (question 14). "Reuse an earlier answer instead of paying twice for the same page" never found the caching page in any method. The page says "cache" and "max_age", and the question says neither.

Alpha values and fusion type

At the top 5, alpha 0.25 found 22 of 28, and 0.5 and 0.75 both found 27. At rank 1 the counts were 17, 18 and 20. On paraphrases at rank 1, 0.75 found 7 of 12 and 0.5 found 5. Put plainly: in this run vector alone (19/28 at rank 1, MRR 0.775) and hybrid 0.75 (20/28, MRR 0.799) had equal or better rank-1 and MRR than hybrid 0.5 (18/28, MRR 0.742). Hybrid 0.5 was the default I fixed before the run, not a value the data picked. The gaps are a question or two, so none of this separates the settings. Tune alpha on your own questions.

Ranked fusion at 0.5 found 25 of 28 at the top 5 against 27 for relative score, and the two paraphrase questions it lost were the difference. Only one alpha was tested for ranked fusion, so I cannot say whether the gap is about the method or about the alpha.

Tips and gotchas from the run

• A missing query prefix cost two questions. Vector search with the query embedded without search_query: found 24 of 28 at the top 5, against 26 with it. Recall@1 was the same, and MRR@5 fell from 0.775 to 0.756. The failure is quiet: results still look plausible.
• Snake_case tokens are split by the default tokenizer. See question 7 above. Identifiers that matter should be checked against the tokenization setting.
• Hybrid scores are relative. The top hit is always 1.000 under relative score fusion. Do not set a similarity threshold on it.

Weaviate vs Qdrant, Pinecone and pgvector

Weaviate fits when you want BM25 and vectors in one query without writing the fusion yourself. Qdrant is the pick for payload filters, and the Qdrant Docker tutorial uses this same chunk file. If you already run Postgres, use pgvector or the hosted Supabase pgvector version. Pinecone is closed source and managed only. The comparison post has licences, free tiers and prices side by side.

Keep it fresh when the docs change

Recrawl on a schedule (how often to recrawl), and mind the 7-day default cache described in the caching docs. Upsert chunks by chunk_id and delete the objects of any URL that disappeared. A hash-based re-embed step is planned and is not covered here.

What this test did not cover

This was one docs site with 52 pages and 28 questions, written by an LLM from the same chunks the labels point to. The labels are page level, with a chance level near 10%. It used one embedding model (nomic-embed-text) and one chunker. One Weaviate version on a laptop. No reranker, no Weaviate vectorizer modules or hosted embeddings, no filters, no multi-tenancy and no scale test. I measured no latency, so none is claimed.

Run it on your own site and what I would pick

Crawl your docs, chunk and embed the markdown, then run load.py and evaluate.py from the code folder against your own 25 questions. It holds the scripts, the frozen question set, results and run logs.

My recommendation is a judgment from one small run, not a finding. If exact identifiers (environment variable names, error codes) matter little for your questions, use hybrid with a higher alpha such as 0.75, or vector alone, since both matched or beat 0.5 here. If those identifiers matter a lot, use a lower alpha such as 0.5 or below, and check the tokenization of snake_case tokens (question 7). Either way, the measured gain of hybrid over vector in this run was one question out of 28 at the top 5.

Next in the series is Qdrant with Docker and payload filters, on the same chunk file.