Case study
Why we are building sift.
Enterprise knowledge work fails before the LLM ever sees the data. sift exists to fix the path from messy documents to trustworthy answers — for people and for agents.
The problem is upstream of chat.
Teams buy “RAG” and get a wrapper around a vector store. The hard work — faithful extraction, review, citations, and agent access — is left as homework. That is the case for sift.
What keeps breaking
Parsing is treated as a solved problem
Most stacks dump PDFs into a chunker and hope. Tables collapse, equations vanish, and structure that humans rely on never makes it into the index.
No one reviews what the model will trust
High-stakes teams still need a human to approve low-confidence extractions before they become answers. Chatbots rarely offer that gate.
Answers float free of sources
Without citations tied to pages and sections, an answer is just fluent text. Regulated work needs a trail back to the document.
Agents get bolted on last
Internal agents need scoped search, markdown export, and APIs — not a chat box. Knowledge products that ignore that become dead ends.
What we are betting on instead.
01
Own the full path
Upload → extract → validate → human review → chunk → embed → index. Then serve humans in chat and agents over REST, MCP, and export.
02
Run where the documents live
Privacy is not a toggle. sift is self-hostable and air-gap capable so sealed corpora never have to leave the building.
03
Ground every answer
Retrieval is hybrid and reranked. Replies carry citations. Review catches bad parses before they enter the corpus.
sift is open source under Apache 2.0. The product is the full shell — UI, CLI, API, MCP — so privacy-sensitive teams can run document intelligence without sending the corpus out.