Async

Session 2 assignment support: RAG on your Session 1 endpoint

1. Start here: what this assignment is

This guide has two paths. Path A is the basic RAG submission: enough to pass. Do that first. Path B is add-ons: choose one or two only if you want to push further.

You already shipped `POST /ask` in Session 1. Session 2 extends that same service with simple document ingest, retrieval, citations, and refusal — plus a Streamlit UI for ingest + ask. You do not need to build every advanced RAG feature for this submission.

Critical

The Golden Rule (read this first)

Whenever anything goes wrong, copy the entire error message and paste it into Claude Code or Cursor with: "I got this error, please fix it and explain what happened in simple terms."

  • That habit fixes most problems.
  • You are never stuck. You just haven't pasted the error yet.

Basic RAG submission: what counts

Basic RAG submission: enough to pass

  • Extend your Session 1 service. Do not start from a new unrelated app.
  • Add `POST /ingest` for plain text + `document_id`.
  • Store chunks in one vector store. Pinecone is the recommended beginner route.
  • Prove retrieval works before generation, using a debug route or script.
  • Upgrade live `/ask` so it answers from ingested docs and cites `document_id` or chunk IDs.
  • Show one successful cited answer and one refusal when the docs do not contain the answer.
  • Ship a Streamlit UI for ingest + ask that calls your API (screenshot for Maven).

2. Before you build: what you are building

  • POST /ingest - you send text plus a `document_id`; the service chunks, embeds, and stores it.
  • Retrieval debug - before calling the LLM, you can print the top chunks and scores for a question.
  • POST /ask with RAG - same endpoint as Session 1, but now it searches your documents first, answers only from retrieved context, cites sources, and refuses when the docs do not contain the answer.
  • Streamlit UI - a simple front end for ingest + ask that calls your API (required).
  • Done means the basic ingest + ask flow works on your public Render URL, not just localhost, and the Streamlit UI demos it.

Preflight: what you need

Session 1 service deployed

Your live Render URL with POST /ask working. Session 2 extends this repo - do not start from scratch.

Vector DB account (recommended: Pinecone)

Sign up at pinecone.io, create an index, copy API key + index name. Free tier is enough for the assignment.

One small document corpus

Use the Northwind sample docs first. Your own capstone docs can come later.

Same OpenAI key as Session 1

You need embeddings + generation. Lock one embedding model (e.g. `text-embedding-3-small`) - switching later means re-indexing.

Critical

Test retrieval BEFORE you wire the LLM

The #1 mistake in Session 2 is debugging generation when retrieval is wrong. Add a debug route or script that prints top-k chunks + scores for a question. Only upgrade /ask to RAG once the right passages come back.

3. Path A: Basic RAG submission

Do these steps first. If you get the cited answer and refusal working on your live URL, plus a Streamlit UI for ingest + ask, you have passed the Session 2 assignment. Stop there if you are tired.

Step 1: Open your Session 1 project

PROMPT 1: Orient on the Session 1 codebase
I am extending my Session 1 FastAPI /ask service for a Session 2 RAG assignment. Please read this repo and explain in plain English:
1. Where POST /ask is implemented
2. How structured output, tokens_used, and cost_usd are returned
3. Where I should add POST /ingest and vector-store logic
4. What env vars I will need for Pinecone (or ChromaDB) and OpenAI embeddings

Do not write code yet - just map the project.

Step 2: Choose and wire your vector store

Recommended for the basic path: Pinecone. It keeps the storage problem outside Render and is the least confusing route for beginners.

PROMPT 2: Add vector store config
Add Pinecone vector store support to this FastAPI project.

Requirements:
- Config via environment variables (no secrets in code)
- Use text-embedding-3-small for embeddings (same model at ingest and query time)
- Add a small health/debug function I can call to confirm Pinecone is reachable

Explain what env vars I must set locally and on Render.

Step 3: Build POST /ingest

PROMPT 3: Ingest endpoint
Build POST /ingest on this FastAPI app.

Requirements:
- Accept JSON with text + document_id (and optional metadata like source filename)
- Chunk with RecursiveCharacterTextSplitter - chunk_size ~800, overlap ~100 (make these configurable)
- Embed each chunk with text-embedding-3-small
- Upsert into the vector store with metadata: document_id, chunk_index, source
- Return JSON: document_id, chunks_indexed, status

Include a curl example in comments. Handle empty input with a clear 400 error.
Smoke-test ingest locally
curl -s -X POST http://127.0.0.1:8000/ingest \
  -H "Content-Type: application/json" \
  -d '{"text": "Remote work: up to 3 days per week with manager approval.", "document_id": "handbook"}'

Step 4: Test retrieval alone

PROMPT 4: Retrieval debug route
Before changing POST /ask, add GET /debug/retrieve?q=... (or a small script) that:
1. Embeds the question
2. Returns top-5 chunks with similarity scores and document_id metadata
3. Does NOT call the LLM

I will use this to verify retrieval before wiring generation.

Ingest the Northwind handbook (or one doc you know well). Ask a question you know the answer to. If the wrong chunks appear, fix chunking or embedding before Step 5.

Step 5: Upgrade POST /ask to RAG

PROMPT 5: RAG on /ask
Upgrade POST /ask to use retrieval-augmented generation:
1. Embed the question
2. Retrieve top-k chunks (start with k=5)
3. Build a grounding prompt: answer ONLY from context, cite document_id for each chunk used, refuse if context is insufficient
4. Call the existing Session 1 generation path
5. Preserve tokens_used and cost_usd in the response where possible
6. Include retrieved chunk IDs in the response JSON

Show me the grounding prompt template you used.
Grounding prompt pattern
Answer using ONLY the context below.
If the context does not contain the answer, say:
"I don't have enough information to answer that."
Cite the document_id of each chunk you used.

Context:
{retrieved_chunks}

Question: {question}

Step 6: Ingest your full corpus

For the basic path, ingest the Northwind sample docs first. You can switch to your own capstone corpus as an add-on.

PROMPT 6: Batch ingest
Help me ingest the Northwind sample docs via POST /ingest.
For each file: read text, assign a stable document_id, call ingest, print chunk count.
At the end, print total chunks in the vector store.

Step 7: Deploy to Render

Add new env vars to Render (Pinecone API key + index name, or Chroma path). Redeploy the same Web Service as Session 1.

  1. Push changes to GitHub.
  2. In Render → your Session 1 service → Environment: add vector DB secrets.
  3. Redeploy and wait for Live.
  4. Run ingest against the live URL, then ask a doc-grounded question.
Live curl - ingest + ask (replace YOUR-SERVICE)
# Ingest on live URL
curl -s -X POST https://YOUR-SERVICE.onrender.com/ingest \
  -H "Content-Type: application/json" \
  -d '{"text": "...", "document_id": "handbook"}'

# RAG ask
curl -s -X POST https://YOUR-SERVICE.onrender.com/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "What is the remote work policy?"}'

Step 8: Prove one answer and one refusal

PROMPT 7: Basic proof
Help me prove the basic RAG submission.

Use my live URL and show:
1. One question that should be answered from the ingested docs, with cited document_id or chunk IDs
2. One question that is NOT in the docs and should trigger the refusal response
3. The curl commands and full JSON responses for both

Keep the explanation simple enough to paste into Maven.

Step 9: Streamlit UI for ingest + ask (required)

Build a simple Streamlit UI that calls your live API: one path to ingest text + `document_id`, one path to ask a question and show the cited answer (or refusal). Keep the FastAPI service as the source of truth — Streamlit is the demo front end, not a second RAG implementation.

PROMPT 8: Streamlit RAG UI
I have a live FastAPI service with POST /ingest and POST /ask (RAG). Please build a minimal Streamlit UI that:
1. Lets me paste text + document_id and call /ingest
2. Lets me ask a question and call /ask
3. Shows citations / refusal clearly in the response
4. Points at my live Render URL via an env var or sidebar input (not hardcoded secrets)

Keep the API as the source of truth. Do not reimplement RAG inside Streamlit. Tell me the exact run command and what screenshot to take for Maven.

4. Stop point: basic RAG submission is done

Ready when

Stop here if you need to

If your live service can ingest text, retrieve the right chunks, answer with citations, refuse when context is missing, and you have a Streamlit UI for ingest + ask, the basic Session 2 submission is done.

Before you move to add-ons, confirm this

  • `POST /ingest` accepts text plus `document_id`.
  • Retrieval works by itself before the LLM is involved.
  • `POST /ask` answers from retrieved context and cites sources.
  • The service refuses when the docs do not contain the answer.
  • Both ingest and ask work on the live Render URL.
  • Streamlit UI can ingest and ask against the live API (screenshot ready).

5. Path B: Optional add-ons

Add-on 1: Golden-set eval

Recommended

Create 5 known-answer questions and track retrieval hit, faithfulness, and correctness.

Add-on 2: Chunking experiment

Compare two chunk sizes and document which one retrieves better on your questions.

Add-on 3: Hybrid search

Combine keyword/BM25 with vector similarity using weighted fusion or Reciprocal Rank Fusion.

Add-on 4: Metadata filtering

Filter retrieval by `document_id`, source, tag, or capstone area.

Add-on 5: Batch ingest

Add a script or endpoint that ingests a folder or multiple files at once.

Add-on 6: Reranking

Retrieve top-k, then reorder with a rerank model or API.

Add-on prompt: golden-set eval

ADD-ON PROMPT: Eval spreadsheet
Help me create a golden-set eval for my RAG service with at least 5 questions I know the answers to.

For each question record:
- question
- expected answer (short)
- retrieval hit? (right chunk in top-5)
- faithfulness? (answer grounded in retrieved text)
- correctness? (matches expected)
- one question that SHOULD trigger refusal (answer not in docs)

Format as a markdown table I can paste into README.

Add-on prompt: choose a retrieval improvement

ADD-ON PROMPT: Pick one retrieval add-on
I have completed the basic Session 2 RAG assignment (including Streamlit UI). Help me choose ONE add-on based on my current code and corpus:
1. Chunk size comparison
2. Hybrid search
3. Metadata filtering
4. Batch ingest
5. Reranking

Recommend the most useful one, explain why, then implement it in small steps with a simple before/after test.

6. Share + troubleshoot

Critical

Do NOT share your live URL publicly

Never post your Render URL on LinkedIn or any public page. Anyone with the link can call /ask and /ingest and burn your API credits.

  • Maven submission channel: live URL, ingest + ask curls, Streamlit screenshot, one cited answer, and one refusal.
  • If you did add-ons: include eval scores, chunking notes, or screenshots as extra proof.
  • LinkedIn: screenshots or screen recording only - no live URL.

Basic RAG submission checklist

  • Session 1 POST /ask still works (extended, not replaced blindly).
  • POST /ingest accepts plain text + `document_id`, chunks it, embeds it, and upserts it.
  • Retrieval tested alone - you can show top-k for a known question.
  • POST /ask retrieves, cites chunk IDs, and refuses when docs lack the answer.
  • Both endpoints work on live Render URL (not just localhost).
  • You have one successful cited answer and one refusal response.
  • Streamlit UI for ingest + ask works against the live API (screenshot for Maven).
  • Submitted live URL + proof in Maven only.

Optional add-ons

  • Golden set ≥5 questions with retrieval + faithfulness + correctness in README.
  • Chunking experiment, hybrid search, metadata filtering, batch ingest, or reranking attempted.
  • README explains architecture, how to ingest new docs, and what broke.
  • LinkedIn post uses screenshots only, never the live URL.

Troubleshooting

  • Wrong chunks retrieved - fix chunk size/overlap before touching the prompt. Log what retrieval returns.
  • Empty Pinecone index - confirm ingest ran on live URL after deploy, not only locally.
  • Answers hallucinate despite good retrieval - strengthen grounding prompt; add explicit refusal instruction.
  • Render deploy fails on new deps - paste build log into your coding agent; pin versions in requirements.txt.
  • Embedding dimension mismatch - index dimension must match your embedding model.