Session 2: RAG and Embeddings - grounding in your own data
Make your /ask endpoint answer from your documents, not training data. Running example: Northwind Robotics policy corpus throughout.
11 lessons · Back to full syllabus
What you keep
How to make a model answer from your data instead of its training, and why naive RAG fails - plus Claude Code skills, MCP, and eval habits that make the build repeatable.
You ship
Week 1 /ask extended with a vector store, POST /ingest for new docs, RAG query with citations + refusal, a Streamlit UI for ingest + ask, and a golden-set eval on your live Render URL.
Recommended reading (async)
RAG foundations - watch & read (async)
Before or after the live Session 2 deck, these videos and articles give you the industry vocabulary for retrieval, chunking, and production RAG - complementary to the Northwind hands-on work in the lessons below.
Videos
What is Retrieval-Augmented Generation (RAG)?
IBM Technology - plain-English overview of why RAG exists and how it fits together.
Retrieval-Augmented Generation (RAG) Patterns and Best Practices
InfoQ - practitioner patterns for designing RAG systems that hold up beyond demos.
Articles
- What is Retrieval-Augmented Generation (RAG)? - Google CloudEnterprise framing: grounded generation, when to use RAG vs fine-tuning.
- Introduction to LLM RAG - WeaviateDeep dive on RAG architecture, pipelines, and evaluation.
- Building Enterprise AI: Hard-Won Lessons from 1200+ Hours of RAG - ByteVagabondProduction lessons: chunking, hybrid search, and what breaks at scale.
Live session resources
Week 2 live session notebook (GitHub)
Hands-on RAG notebook: embeddings, chunking, vector DBs, LangChain Document Q&A build, and evaluation. Clone and run with your OpenAI key.
Open resourceWeek 2 assignment build guide
Step-by-step Path A / Path B guide with copy-paste prompts for RAG ingest, citations, refusal, and required Streamlit UI.
Open resourceLessons
Embeddings intuition
Similar meanings sit close together in vector space - retrieval quality depends on this mental picture.
Build naive RAG, and watch it fail
Embed, store, retrieve, stuff into prompt - then break it instructively on the Northwind corpus.
Chunking
How you split documents decides what retrieval can find - most bad RAG is bad chunking.
Reranking and hybrid search
Reranking reorders candidates by relevance; hybrid search catches exact terms embeddings miss.
RAG evals
Measure retrieval and generation separately - wrong chunks vs right chunks ignored.
Graph RAG
When knowledge has structure and relationships, plain vector RAG leaves value on the table.
Multimodal RAG
Retrieve over images, tables, and diagrams - not just prose.
Chunking video for multimodal RAG
Video chunking is two independent decisions - where to cut, and what to put in each chunk - and getting either wrong is the one RAG mistake no reranker fixes.
Claude Code layers - skills, MCP, subagents, and plugins
One reference for which layer does which job: CLAUDE.md, skills, MCP, subagents, hooks, plugins.
Session 2 assignment support: RAG on your Session 1 endpoint
A grouped guide with copy-paste prompts for the basic RAG submission (including Streamlit UI), plus optional retrieval add-ons.
Session 2 homework: RAG on your Session 1 endpoint
Extend POST /ask with basic RAG: ingest text, retrieve before generation, cite sources, refuse when docs are missing, and ship a Streamlit UI.
Lessons in this module