PageIndex: The RAG System That Throws Out Vector Search Entirely

PageIndex builds a tree index from documents and uses LLM reasoning to navigate to relevant sections — no vectors, no chunking, no approximate nearest neighbor. The architecture is genuinely different. Whether it matters for your use case is the real question.

Here's the dirty secret of production RAG in 2026: most teams have given up on making vector search actually work and started adding more vectors. More embedding models, more chunking strategies, more hybrid approaches, more rerankers — all to compensate for the fact that semantic similarity is a proxy for relevance, not the thing itself.

PageIndex is the first RAG framework I've seen that takes this criticism seriously and builds something genuinely different from the ground up. No vector database. No chunking. No approximate nearest neighbor search. Just a tree structure and a model that reasons its way to the right document sections.

What Vector Search Gets Wrong

The premise is worth stating clearly because it rarely is: vector similarity and relevance are different things. When you embed a document chunk and store it in a vector database, you're betting that the geometric distance between vectors in high-dimensional space correlates with how relevant that chunk is to a given query. Sometimes it does. Often it doesn't.

The failure modes are well-documented at this point. A chunk that contains the answer but is surrounded by irrelevant context will retrieve because its vector is semantically similar to the query — and then the model has to extract the answer from a sea of noise. A chunk that contains the answer but lives in section 47 of a 200-page document will often lose to more "topically central" chunks whose vectors happen to cluster near the query embedding. And queries that require multi-step reasoning — find the section of the contract that addresses the interaction between clauses 3.1 and 7.4 — reduce to vector similarity theater because no embedding captures that kind of relational reasoning.

Hybrid search helps. Rerankers help. More sophisticated chunking strategies help. None of them solve the fundamental problem: you're optimizing for similarity when you actually need relevance, and those two objectives diverge more often than the industry wants to admit.

PageIndex's Alternative

PageIndex builds a hierarchical tree structure — essentially a table of contents, but optimized for LLMs — from a document, and then uses the LLM itself to reason over that tree to find relevant sections. Not to find the nearest neighbor in a vector space. To think about where the answer probably lives.

The mechanism is inspired by AlphaGo's tree search: instead of evaluating every possible move, the model searches the tree intelligently, following promising paths and pruning branches that don't lead where it needs to go. For document retrieval, this means the model can use domain knowledge and query context to navigate — it knows that a question about executive compensation is likely in the proxy statement section, not the notes to the financial statements, even if those sections have similar vectors.

The tree index is built from the document's natural structure: headings, sections, subsections. No arbitrary chunking. The nodes are summaries of their content with page references. Retrieval is a reasoning process over the tree, not a similarity lookup.

The FinanceBench Numbers

PageIndex claims 98.7% accuracy on FinanceBench — a benchmark of 150 financial documents with question-answer pairs that require reasoning over complex financial filings. For comparison, the best vector-based RAG systems typically score in the 60-75% range on the same benchmark.

I'm skeptical of single-benchmark claims, and you should be too. FinanceBench is a specific domain with specific document types. The results won't generalize uniformly to all document types and all query patterns. But a 20+ point gap on a domain that demands precise, reasoning-heavy retrieval is worth taking seriously — not as a settled fact, but as evidence that the approach has genuine merit and the benchmark selection isn't entirely cherry-picked.

The reasoning-based architecture also has a structural advantage for the exact document types where vector RAG struggles most: long legal contracts, financial filings, regulatory documents, academic papers. The cases where you need to answer "which section addresses X and how does it interact with Y" are where tree search beats similarity search. The cases where a quick semantic lookup works fine — FAQ-style questions, short documents, queries with obvious topical matches — are where the overhead of building and searching a tree isn't obviously worth it.

What's Actually Different Technically

The "no chunking" claim requires some nuance. PageIndex doesn't use sliding-window or overlap-based chunking — it organizes content by natural document structure. If a section is 40 pages, it becomes a node with nested sub-nodes. The model can traverse the tree to find the right granularity rather than being handed arbitrary 512-token blocks.

The retrieval is also context-aware in a way that vector search isn't. When you retrieve against a vector database, the query is your user query and maybe a handful of turns of conversation history. PageIndex can incorporate the full conversation context, domain knowledge, and query history into the traversal decision. The retrieval decision for "what happened in Q3" is different when the conversation has already established that we're discussing the 2024 filing vs. when it's a standalone question. Vector search has no mechanism for this; PageIndex does.

The tradeoff: reasoning-based traversal is slower than a vector lookup. If your bottleneck is retrieval speed and your use case works fine with semantic similarity — and for many document Q&A scenarios, it does — the latency cost of tree traversal is real. PageIndex isn't claiming to beat vector databases on speed. It's claiming to beat them on accuracy for complex retrieval tasks. Whether that tradeoff matters depends entirely on your use case.

The Agentic RAG Angle

The more interesting part of the PageIndex story is the agentic vectorless RAG example they've published, which uses the OpenAI Agents SDK with PageIndex as the retrieval backend. The agent can plan a multi-step retrieval strategy: "first find the relevant contract section, then look up the related amendment, then compare the dates." Each step involves tree traversal, not vector similarity — and the model can course-correct if a retrieval step doesn't yield what it expected.

This is where the tree structure pays off over flat vector spaces. An agent navigating a tree can make strategic decisions about where to look next. An agent retrieving from a vector database gets back top-k results and has to work with whatever came back. The first is a reasoning loop. The second is a retrieval loop. For complex, multi-step document analysis tasks, the reasoning loop wins.

The Honest Limitations

PageIndex isn't a replacement for vector search in most production systems today. The overhead of tree construction — you need to process the document upfront and build the index — means it's not a zero-setup solution. The latency of reasoning-based retrieval means it's not ideal for user-facing systems where sub-100ms retrieval is a hard requirement. And the dependency on a capable LLM for the traversal reasoning means your retrieval quality is bounded by your model's reasoning ability in ways that vector search doesn't impose.

The self-hosting option is worth noting: you can run PageIndex against your own PDFs with an OpenAI-compatible API endpoint, no cloud service required. The open-source implementation handles the tree building and traversal locally. This matters for compliance and data privacy use cases where sending documents to third-party APIs isn't acceptable.

What PageIndex represents is a genuine architectural alternative in a field where most "improvements" are incremental tweaks to the same vector search paradigm. Whether the specific implementation is the right approach for your document retrieval problem is a question worth asking — because the assumption that vector search is the default answer to RAG has been producing mediocre results for long enough that alternatives deserve serious evaluation.

The idea that retrieval should be reasoning-based rather than similarity-based is older than the current RAG paradigm. PageIndex is one of the first implementations that's serious enough about it to publish benchmarks and code. That's worth paying attention to, even if the final verdict on production readiness takes another six months of community testing to emerge.

---

PageIndex: open-source vectorless RAG framework. No vector DB required. Tree-index-based reasoning retrieval. 98.7% on FinanceBench. GitHub: VectifyAI/PageIndex.