Benchmarks

Fifteen measurements over 65 rows, every one against an external reference. What matters is not the best-case recall — it is the distance to the exact scan, and what you gain in exchange. Each table links to the notebook that produced it.

1. What was measured, and against what

Every number on this page has two references: the exact scan over the same query, and an approximate index on the same corpus. Without both, a recall figure means nothing.

2. Every number has a notebook

No figure on this page is copied from a report. Each one comes from a public notebook that installs the published packages, runs the benchmark and writes the output shown here.

3. Recall by real dataset — routed configuration

Cinco datasets reais com configuração roteada — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-madhava-1913-five-datasets-fusoes

4. Random projection vs PCA — the measured effect

Projeção aleatória vs PCA, ground truth externa — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-1913-scale-v2-gt-externo

5. Comparison with FAISS — recall, RAM and auditability

Madhava contra FAISS: recall, RAM, auditabilidade — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-1913-ndcg-vs-faiss

6. Indexing — Madhava against HNSW, IVF and PQ

Indexação contra HNSW, IVF e PQ — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-1913-madhava-certificado

7. Scale — from 100 thousand to 100 million

Escala de 100 mil a 100 milhões — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-madhava-1913-scale-timed

8. Tracer by domain — real corpus

Tracer por domínio em corpus real — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-1913-certificado-v2-jus-med

9. Aggregated metrics

Métricas agregadas com certificado — notebook no Kaggle ↗

Notebook: https://www.kaggle.com/code/kleniopadilha/winnex-1913-madhava-certificado

10. What these numbers do not say

An honest benchmark declares where it loses.

• Latency is the axis where we lose. Indexing is far faster than HNSW (2.57 s against 40 s at 1M), but the query costs more — the bound is work an approximate search does not do.
• Word2Vec shows the limit of the method. Recall 0.7333 with 0% bound-pruned: on a weak manifold PCA degrades. That is why the router sends this dataset to a random projection.
• A random projection empties the proof. In the projection table, random reaches 100% recall with 0.4% provable exclusion on arXiv. The answer is right and the proof is empty.
• 100M uses mmap streaming. The 24.4 s build at that scale is with a memory map, not a corpus in RAM — the FAISS comparison on that row is not apples to apples.

Every benchmark runs from the packages published on PyPI. The notebooks are on Kaggle and reproduce line by line.

11. Raw real datasets — no routing

Before the router existed, the engine ran on raw datasets with the default configuration. These are the numbers without tuning.

12. Build speed — Madhava against HNSW

Where the engine wins decisively is index construction. The bound is computed once at build; the approximate index has to insert every vector into a graph.

13. Engine routing inside Maestro

The ERP runs three different workloads, and each gets a different engine configuration. The choice is per workload, not per product.

14. Prompt-injection bound check — cost

winnex-madhava-sec uses the same bound to detect injected tool calls. This is what the check costs per query.

15. Red ocean vs blue ocean

This is positioning, not a benchmark — but it belongs here, because every number above only matters in the column on the right.