RAG for Engineers Who Already Know Databases

1 min. Name the two paths once. Index time is the write. Query time is the read. Point at Prompt plus chunks and say that is the JOIN. Then: we are not recapping embeddings. The rest of this session is the five failure stages on that retrieve box.

The spine of the session and the slide people screenshot. Run the poll properly, read three answers out loud by name. Costs 40 seconds, buys the room. Almost everyone says 1 or 2. Do not correct anyone, say "hold that thought" and advance. The reveal is the next slide.

This slide earns the advanced framing. Slow down here. If you are behind later, take it from slide 5 or 7, never this one. The line to land is the callout, say it twice. Card 3 sets up slide 8, so flag that you will come back to it.

The diagram is the failure: a table splits across two chunks, numbers in one, headers in the other, both retrieve and neither answers. Everyone here has normalised a schema badly, so this lands fast. The tradeoff to narrate: small chunks match precisely but shred context, large chunks keep context but dilute the signal until the embedding averages toward nothing. Anthropic numbers are from their own engineering blog, state them flat. Costs about one dollar per million document tokens with prompt caching. If asked about late chunking: embed the whole document first, pool per chunk after, same goal, cheaper at query time.

Say hybrid is blind to much less, at the cost of two queries and a fusion step. It is not blind to nothing: it still inherits both indexes’ recall ceilings and adds a fusion tuning problem, and someone in this room will push back if you say otherwise. The error code example is what lands. Ask them to picture a customer pasting ERR_CONN_REFUSED_5031 into support search. Cosine similarity sees a generic error shaped token sequence and returns generically error shaped documents. BM25 sees a rare term and goes straight to it. Contextual embeddings plus contextual BM25 took Anthropic to 49 percent fewer failures, neither alone gets close. Why 60: empirical, from the 2009 paper, it damps the top ranks so one confident list cannot dominate. This slide compresses if you are behind.

Click once to apply the filter: the right panel fades in and the path breaks after one hop. The two ways a filter fails: filter after searching discards non matches, so a selective filter returns fewer results than you asked for; filter before searching builds the eligible set first, and with too few eligible nodes the paths between them break and the walk dead ends early. Highest value slide for anyone running multi tenant retrieval and almost nobody measures it. The instinct to break: adding WHERE can only narrow, so it can only help. Those numbers are Qdrant benchmarking their own engine, say so. Yours will differ, the shape of the failure will not. Practical fix if asked: partition or namespace by tenant instead of filtering by tenant, so the constraint becomes which index you query. Some engines build filterable index structures and do neither naive approach, so the real instruction is to benchmark yours.

The funnel: retrieve wide with hybrid search, fuse, take 50 to 200 and optimise purely for recall. Rerank narrow with a cross encoder that reads query and document together, not precomputed vectors. Pass the top 20 to the model; Anthropic found 20 beat both 5 and 10. The distinction that matters: the bi encoder embedded that document months ago with no idea what would be asked, the cross encoder sees both at once. Reported cost is 100 to 150 ms on CPU for 50 candidates, 30 to 50 ms on GPU, and 5 to 10 nDCG points of gain. Say reported. Stack the whole pipeline for the big number: contextual retrieval plus hybrid plus rerank is 67 percent fewer retrieval failures, 5.7 down to 1.9. If behind schedule, this compresses to 90 seconds: just the two stage shape and the callout.

This is card 3 from slide 3 cashing out, call that back. The line that gets a reaction: an embedding model upgrade is a full index migration and everyone treats it as a config change. One reported case saw recall drift 0.92 to 0.74 with nothing in the codebase changing, say reported. Practical: keep 50 real queries with known correct documents and rerun weekly. That is the whole monitoring story and it takes an afternoon.

This is the payoff, give it the full three minutes, do not rush it to protect the closing slide. Reveal one at a time with the arrow key and read chat between each. D gets the biggest reaction because everyone has lived it: nothing in your code changed, the world did. If chat is quiet, call on someone by name, it is late and people need permission to type.

Tell them explicitly to screenshot the table. One spoken mention of the bootcamp, then straight to questions, do not sell. Maven adds every signup to the waitlist and emails the recording in 48 hours, the work is done. Held for Q&A: GraphRAG, multi hop and agentic retrieval, ColBERT, quantisation, fine tuning embeddings.