Introduction
What is AI engineering?
AI engineering is building reliable products on top of frontier models you did not train - grounding them in your data, giving them tools, measuring whether they work, and shipping something that holds up when real users touch it. Coding agents write most of the code; your job is systems thinking and judgment.
Read the full introduction in pre-courseStart with a step-by-step guide to what AI engineering is, then install your coding agent and get ready for Session 1.
What you keep
A clear mental model of AI engineering - reliability around models you did not train - plus a working coding-agent workflow.
You ship
Cursor or Claude Code installed, GitHub + OpenAI key ready, and a capstone one-liner before the first live session.
What is AI engineering? (start here)
A step-by-step guide: what AI engineering is, what we assume, what we teach, and what to do next before Session 1.
Software components for beginners
APIs, REST, POST, FastAPI, Docker, scaling, and the other building blocks Session 1 assumes. Plain-English map for newcomers.
Install Cursor or Claude Code (do this next)
Pick one AI coding agent, install it, open a project folder, and confirm you can chat with it. This is the highest-leverage pre-course step.
Python, Git, and API keys
Let your coding agent check Python, create a venv, and scaffold FastAPI. Create a GitHub account, add your OpenAI API key, and keep secrets out of chat.
Seed your capstone
Pick the one project you will build across five live sessions - your Session 1 `/ask` endpoint is its first capability.
Building in public
Why we share builds openly, where to post (LinkedIn, Substack, share it), and how to talk about the skills you learn.
Pre-session check-in
Share your setup status and capstone one-liner before Session 1.
Session 1 checklist
What you must have ready before the first live session - and what is fine to finish during or after it.
What you keep
How to turn a model call into a dependable, observable piece of software - plus context engineering, token optimisation, and a free-resources bridge to mastery.
You ship
A working `/ask` endpoint that answers reliably, plus a Streamlit UI that demos it - the reasoning core of your capstone.
Watch through (async)
Think Like an AI Engineer (free 10-part series)
- 1How LLM Actually Works (in Plain English)Watch
- 2How to Prompt AI Like a Pro: The 5-Part FormulaWatch
- 3How to Pick the Right AI Model for the JobWatch
- 4Using AI With Your Files, Images and DataWatch
- 5Context Is Everything: The #1 Skill for Using AIWatch
- 6Vibe Coding for Non-Engineers (and Engineers)Watch
- 7What AI Agents Actually Are (Hype vs Reality)Watch
- 8Build AI Workflows With No Code (n8n, Zapier, Make)Watch
- 9AI Evals: How to Know If Your AI Actually WorksWatch
- 10The Fastest Way to Stay Current in AIWatch
Live session resources
Week 1 session deck (12 Aug 2026)
Live Session 1 slides for Cohort 13: WhatsApp join, tool stack, three layers, harness, prompting, and the /ask build path.
Open resourceWeek 1 session deck (31 Jul 2026)
Prior cohort Session 1 slides (same teaching spine).
Open resourceWeek 1 `/ask` demo code (5 stages)
FastAPI starter repo: bare /ask through structured output, guardrails, model routing, and cost readout. Includes Streamlit demo UI and stage smoke tests.
Open resourceWeek 1 assignment build guide
Step-by-step Path A / Path B guide with copy-paste prompts for the basic live /ask submission, Streamlit UI, and optional add-ons.
Open resourcePrompting lab (bad vs good)
Five OpenAI Playground pairs: specificity, structured output, few-shot, chain of thought, constraints. One-tap copy for each weak and strong prompt.
Open resourceSoftware components for beginners
APIs, REST, POST, FastAPI, Docker, horizontal vs vertical scaling, and the rest of the building blocks Session 1 assumes. Read before or alongside the live build.
Open resourceFrom model call to reliable component
Production means the right shape, at a known cost, with a plan for when it fails - every time.
The OpenAI API and the Playground
Experiment in the Playground first: system vs user messages, temperature, max tokens, and the token counter.
Module 1.1 - Build your first LLM service
Stand up a real POST endpoint with typed models before any theory - the living thing everything else improves.
Module 1.2 - Prompting that ships
Structured outputs, few-shot, and decomposition - three techniques to make output deterministic enough to build on.
Module 1.3 - Output guardrails and validation
Schema, semantic, and safety validation - plus retry patterns - between the model and everything downstream.
Module 1.4 - Token economics and context budgets
Tokens are money and latency. Treat the context window as a budget, not free space.
Module 1.5 - Choosing the right model
Match capability, cost, and latency - and route different steps to different models in one system.
Module 1.6 - Context engineering: the throughline
Prompting, cost, and model choice are all facets of one skill: engineering what the model sees.
Module 1.D1 - Advanced structured outputs and function-calling schemas
Nested schemas, enums, and function calling - when the shape must be airtight and feed code directly.
Module 1.D2 - The provider landscape beyond OpenAI
Anthropic, Gemini, open-weight models - per-task fit and portability across providers.
Module T - Token Optimisation
Nine concrete techniques to cut cost and latency without trading away quality you need.
Module B - Mastering AI Engineering: the free-resources bridge
A curated map from bootcamp graduate to self-directed mastery - almost everything is free.
Session 1 assignment support: Ship Your First AI Endpoint
A grouped guide with copy-paste prompts for the basic Session 1 submission (including Streamlit UI), plus optional add-ons.
Make your /ask endpoint answer from your documents, not training data. Running example: Northwind Robotics policy corpus throughout.
What you keep
How to make a model answer from your data instead of its training, and why naive RAG fails - plus Claude Code skills, MCP, and eval habits that make the build repeatable.
You ship
Week 1 /ask extended with a vector store, POST /ingest for new docs, RAG query with citations + refusal, a Streamlit UI for ingest + ask, and a golden-set eval on your live Render URL.
Recommended reading (async)
RAG foundations - watch & read (async)
Live session resources
Week 2 live session notebook (GitHub)
Hands-on RAG notebook: embeddings, chunking, vector DBs, LangChain Document Q&A build, and evaluation. Clone and run with your OpenAI key.
Open resourceWeek 2 assignment build guide
Step-by-step Path A / Path B guide with copy-paste prompts for RAG ingest, citations, refusal, and required Streamlit UI.
Open resourceEmbeddings intuition
Similar meanings sit close together in vector space - retrieval quality depends on this mental picture.
Build naive RAG, and watch it fail
Embed, store, retrieve, stuff into prompt - then break it instructively on the Northwind corpus.
Chunking
How you split documents decides what retrieval can find - most bad RAG is bad chunking.
Reranking and hybrid search
Reranking reorders candidates by relevance; hybrid search catches exact terms embeddings miss.
RAG evals
Measure retrieval and generation separately - wrong chunks vs right chunks ignored.
Graph RAG
When knowledge has structure and relationships, plain vector RAG leaves value on the table.
Multimodal RAG
Retrieve over images, tables, and diagrams - not just prose.
Chunking video for multimodal RAG
Video chunking is two independent decisions - where to cut, and what to put in each chunk - and getting either wrong is the one RAG mistake no reranker fixes.
Claude Code layers - skills, MCP, subagents, and plugins
One reference for which layer does which job: CLAUDE.md, skills, MCP, subagents, hooks, plugins.
Session 2 assignment support: RAG on your Session 1 endpoint
A grouped guide with copy-paste prompts for the basic RAG submission (including Streamlit UI), plus optional retrieval add-ons.
Session 2 homework: RAG on your Session 1 endpoint
Extend POST /ask with basic RAG: ingest text, retrieve before generation, cite sources, refuse when docs are missing, and ship a Streamlit UI.
What you keep
What an agent really is, how to build the loop by hand, and how to ship one on Google ADK (with LangGraph as an accepted alternative for explicit graph control).
You ship
Your assistant turned into an agent that plans and uses tools, plus a Streamlit UI that demos the loop.
Recommended reading (async)
Multi-agent foundations - watch & read (async)
Live session resources
Week 3 session: Agents with ADK
Agent loop, MCP, multi-agent, and A2A. Live session deck for Engineering Week 3.
Open resourceWeek 3 session: Agents with ADK (updated 2026-07-29)
Same live session deck, plus a bonus "New Kids on the Block" section on graph, loop, harness, and context engineering.
Open resourceWeek 3 assignment build guide
Step-by-step Path A / Path B guide - Google ADK preferred, LangGraph accepted - with Streamlit UI required and copy-paste prompts.
Open resourceClaude Agent SDK overview
Build production agents with Claude Code as a library - same tools, agent loop, and context management in Python and TypeScript.
Open resourceADK sample code (preferred)
Routing, MCP, and full-system demos with Google ADK. Clone, run Demo 1, then adapt to your capstone.
Open resourceLangGraph sample code (accepted alternative)
Same three demos as the ADK repo, re-implemented in LangGraph. Use if ADK is blocked or you prefer graphs.
Open resourceGoogle ADK docs
ADK agents, tools, MCP, multi-agent, and A2A (default assignment stack).
Open resourceLangGraph docs + Academy
Accepted alternative stack: overview, quickstart, persistence, and free Intro to LangGraph course.
Open resourceThe agent loop, by hand with the raw SDK
Think, pick a tool, call it, observe, decide again - no framework mystification.
Orchestration with LangGraph
Nodes, edges, shared state - durable execution and human-in-the-loop checkpoints.
Tools and MCP
MCP is how modern agents get tools - connect to servers instead of hand-wiring every integration.
Multi-agent and A2A
Several specialists when roles are genuinely separable - most problems do not need multiple agents.
Prompt injection, where guardrails get dangerous
Malicious instructions in retrieved content can hijack tool-using agents.
Build your own MCP server
Tools, resources, and prompts, then a real Python FastMCP server you can wire into Claude Code today.
Build your own A2A integration
Agent Card, skills, and tasks - a minimal A2A server and the client call that reaches it.
LangGraph in depth
Reducers, Command routing, subgraphs, checkpointers, time travel - what the live session does not have time for.
Agent security mitigations
Least privilege, human-in-the-loop, structural separation of instructions from content, sandboxing, and an audit trail.
Session 3 assignment support: Turn your capstone into an agent
Step-by-step: pick the job, build a Google ADK agent with one real tool and a Streamlit UI, prove Think → Act → Observe, then optional stretch (LangGraph accepted as alt).
Turn your capstone into an agent
Google ADK agent with at least one real tool and a Streamlit UI - trace each step of the decision loop. LangGraph accepted as alternative. Stretch: HITL, MCP, A2A.
What you keep
The full TRACE loop as a system you build - Trace, Read, Analyze, Codify, Enforce. Code assertions first; validate every LLM judge with TPR/TNR.
You ship
An eval suite your capstone runs against, visible from a Streamlit UI.
Live session resources
Week 4 session: Evaluation, Monitoring & Shipping
TRACE live deck - golden sets, rule checks, LLM-as-judge, observability, and shipping discipline.
Open resourceWeek 4 assignment build guide
Path A / Path B TRACE homework: taxonomy, code assertions, one fix, Streamlit; stretch validated judge and goldens.
Open resourceSample traces pack (Harmony Apartments)
20 SMS leasing-bot traces + knowledge base for Path A when you do not have capstone traffic yet.
Open resourceHow to Build AI Evals in 2026 (YouTube)
Step-by-step evals framing - watch before or after the live session.
Open resourceMastering AI Evaluation: Playground to Production (YouTube)
From ad-hoc playground checks to production eval loops.
Open resourceBraintrust
Primary classroom tooling - traces, human labels, scorers.
Open resourceLangfuse
Open-source backup for tracing and evals.
Open resourcePromptfoo
CI/CD evals and OpenAI's migration path off hosted Evals.
Open resourceAnthropic: Demystifying evals for AI agents
Task, trial, grader, transcript vs outcome.
Open resourceThe vibe-check trap
"It works on the examples I tried" is not evaluation - and for non-deterministic systems it is dangerous.
Trace and Read: error analysis
Capture full records, then read traces by hand - qualitative research on real failures.
Analyze: cluster, count, prioritize
Binary pass/fail judgments, ranked by frequency and impact - the spec for your eval tooling.
Codify: build the eval suite
Code-based assertions plus validated LLM-as-judge - an unvalidated judge is just another vibe check.
Enforce: evals in the loop
Wire evals into the development loop so every change gets measured automatically.
Evaluating agents: trajectories, not just final answers
A polite final message can hide a wrong tool call, a silent retry loop, or an action that never actually happened.
Aligning an LLM-as-judge to human labels
The actual calibration workflow: blind labels, held-out validation, TPR/TNR, and re-validating after every model change.
Agent trajectory and world-state grading
Score the path the agent took and what actually changed in the environment, not just what it said.
Session 4 assignment support: Build and run TRACE on your capstone
Path A: traces, taxonomy, code assertions, one fix, Streamlit. Path B: validated judge, goldens, CI sketch.
Build and run TRACE on your capstone
Trace, read, analyze, codify, and wire evals to run on demand - show a metric moving after a fix.
What you keep
How to give an agent memory that lasts and improves, and how to run it persistently.
You ship
Your agent remembering across sessions and deployed to run for real, with a Streamlit UI that demos memory.
Recommended reading (async)
Session 5 recommended reading and video
- The Four Types of Memory Every AI Agent Needs
- Architecting Agent Memory: Principles, Patterns, and Best Practices
- Google's OKF - The New Way to Structure Your Knowledge for Agents
- Anthropic: Effective context engineering for AI agents
- Anthropic: Effective harnesses for long-running agents
- Machine Learning Mastery: Choosing the right AI agent memory strategy
- Hugging Face: KV Caching Explained
- Karpathy llm-wiki gist
- Open Knowledge Format (OKF) spec
- LangGraph: add memory
- LangGraph: persistence (checkpointers vs stores)
- gbrain (Garry Tan)
- gstack (Garry Tan)
Live session resources
Week 5 session: Anatomy of Agentic Memory
Live deck: memory vs context, LLM wiki / OKF, compaction, write gates, durable execution, poisoning demos, and Demo Day prep.
Open resourceWeek 5 assignment build guide
Path A / Path B guide: durable store, cross-session recall, public URL, UI proof, and copy-paste prompts.
Open resourceWeek 5 Agentic Memory Lab (GitHub)
Next.js interactive UI + notebook: context budget, vector vs graph, compaction, memory store, crash/resume, poisoning. Local equivalent of gbrain ideas.
Open resourceOpen Knowledge Format (OKF v0.1)
Open markdown + YAML standard for portable agent knowledge wikis (Google Cloud Data Cloud).
Open resourceKarpathy llm-wiki gist
Where the LLM wiki pattern was coined: raw sources compiled by an LLM into a living markdown knowledge base.
Open resourcegbrain · gstack (Garry Tan)
gbrain = agent brain / knowledge layer. gstack = Claude Code role toolkit. Skim; dig in later.
Open resourceThe memory spectrum
Working, semantic, episodic, and procedural memory. Match the store to the job.
Graph plus vector hybrid memory (and gbrain)
Vector recalls by similarity; graph recalls by relationship. Hybrid earns its cost when both matter.
Compaction, write gates, and durable execution
Context fills up. Compaction loses rules. Writes need gates. Checkpoints are not full durability.
Persistent runtime deployment
From "I built an agent" to "I deployed an agent": stays up, holds memory, ready for Demo Day.
Skills and the compounding loop
Procedural memory you can write down: capture a process once, and the agent gets better at it every time after.
Course wrap and your capstone from here
Reasoning, RAG, agents, evals, memory - the full arc, what carries forward, and where the capstone goes after Demo Day.
Rolling your own memory store
A real, minimal memory store: Postgres + pgvector, a write gate, hybrid retrieval, and an actual forgetting policy.
Skills authoring patterns
A skill is only as good as its trigger. Structure, scope, and how to actually test one before you trust it.
Session 5 assignment support: Give your capstone memory and deploy it
Path A: durable store, cross-session recall, public URL, UI proof. Path B: graph, consolidation, provenance, crash drill, LLM wiki.
No new teaching. Two weeks to finish so your system ships instead of stalling.
What you keep
A demo-ready brief, a rehearsal habit (including a recorded backup), and a production checklist.
You ship
A publishable capstone URL, a 5 to 10 minute Demo Day walkthrough plan, and proof that memory, evals, and the core task still work for a stranger.
Live session resources
Demo Day expectations (from Session 5)
Problem or product, architecture, stack and tools, live demo, plus a recorded backup. Reach out if unsure the project is strong enough.
Open resourceWeek 5 memory assignment (if still open)
Cross-session recall + public URL before you polish the talk track.
Open resourceAnthropic: Effective harnesses for long-running agents
Clean state between sessions, progress files, and incremental ship habits.
Open resourceBuild-support office hours
Bring a concrete blocker: a failing eval, a flaky tool, a deployment error, or a memory write bug.
Final system brief and production-readiness checklist
From working to demo-ready: one-page brief, 5 to 10 minute walkthrough, recorded backup, ship checklist.
Final portfolio brief and ship checklist
The production-readiness checklist gets your system demo-ready. This gets your capstone interview-ready.
Mandatory live system + URL. Optional presentation. 5 to 10 minutes if you present.
What you keep
A public trail of your capstone and a story you can reuse in interviews.
You ship
Submitted live URL (mandatory). Optional 5 to 10 minute presentation with problem, architecture, stack, and live or recorded demo.
Live session resources
Demo Day
Submit live system + URL (mandatory). Present the grounded, agentic, evaluated, memory-holding stack (opt-in).
Course resources
- Week 1 session deck (12 Aug 2026)
Live Session 1 slides for Cohort 13 (12 Aug).
- Week 1 session deck (31 Jul 2026)
Prior cohort live Session 1 slides.
- Prompting lab (bad vs good, copy-paste)
Five OpenAI Playground pairs for the live demo: weak and strong prompts with one-tap copy.
- Week 1 /ask demo code (5 stages, GitHub)
- Week 1 assignment build guide
Path A / Path B guide with copy-paste prompts for the basic live /ask submission and required Streamlit UI.
- Week 2 RAG + Vector Databases slides
- Week 2 live session notebook (GitHub)
- Week 2 assignment build guide
Path A / Path B guide for RAG ingest, citations, refusal, and required Streamlit UI on your Session 1 endpoint.
Videos and articles - also embedded on the Week 2 module page.
- Full series: Think Like an AI Engineer (10 parts)
- Part 1: How LLM Actually Works (in Plain English)
- Part 2: How to Prompt AI Like a Pro: The 5-Part Formula
- Part 3: How to Pick the Right AI Model for the Job
- Part 4: Using AI With Your Files, Images and Data
- Part 5: Context Is Everything: The #1 Skill for Using AI
- Part 6: Vibe Coding for Non-Engineers (and Engineers)
- Part 7: What AI Agents Actually Are (Hype vs Reality)
- Part 8: Build AI Workflows With No Code (n8n, Zapier, Make)
- Part 9: AI Evals: How to Know If Your AI Actually Works
- Part 10: The Fastest Way to Stay Current in AI
- Software components for beginners
API, REST, POST, FastAPI, Docker, scaling, and the other building blocks Session 1 assumes.
- OpenAI API
- FastAPI
- Structured outputs
- Pydantic
Agents: raw loop, Google ADK (default assignment), MCP, multi-agent / A2A, prompt injection. ADK and LangGraph docs plus the Week 3 session deck.
- Week 3 session: Agents with ADK
- Week 3 assignment build guide
Path A / Path B guide - Google ADK preferred, LangGraph accepted, Streamlit UI required.
- The Multi-Agent Architecture That Actually Ships (video)
Luke Alvoeiro / Factory - production multi-agent patterns. Embedded on the Week 3 syllabus page.
- Multi Agent Systems Explained (video)
IBM Technology - how agents and LLMs work together. Embedded on the Week 3 syllabus page.
- Claude Agent SDK overview
Claude Code as a library - agent loop, tools, and context management in Python and TypeScript.
- ADK sample code (preferred)
- LangGraph sample code (accepted alternative)
- Google ADK docs
- Google ADK, MCP tools
- A2A protocol
- LangGraph overview (accepted alternative)
- LangGraph quickstart
- LangChain Academy, Intro to LangGraph
- Your AI product needs evals (Hamel)
- Why error analysis matters (Hamel FAQ)
- LLM evals FAQ (binary judgments, judge validation)
- Anthropic: Demystifying evals for AI agents
Task, trial, grader, transcript vs outcome; code / model / human graders.
- Addy Osmani: An Engineer's Guide to AI Code Model Evals
Goldens, hill climbing, pass rates, avoid overfitting the suite.
- How to Build AI Evals in 2026 (YouTube)
- Mastering AI Evaluation: Playground to Production (YouTube)
- Braintrust (primary classroom tooling)
Free Starter, unlimited users; traces, human labels, scorers in one place.
- Langfuse (open-source backup)
- Promptfoo (CI/CD evals; OpenAI migration path)
YAML + GitHub Action. OpenAI hosted Evals shut down Nov 30, 2026 - method lasts, vendor UIs move.
- Sample traces pack: Harmony Apartments (Week 4 Path A)
20 traces + KB + CSV. Use when you do not have capstone traffic yet.
- Week 5 Anatomy of Agentic Memory slides
- Week 5 assignment build guide
Path A / Path B: durable store, cross-session recall, public URL, UI proof.
- Week 5 Agentic Memory Lab (GitHub)
UI + notebook. Local equivalent of gbrain ideas.
Videos and articles on memory, compaction, and LLM wikis.
- Anthropic: Effective context engineering for AI agents
- Anthropic: Effective harnesses for long-running agents
- Machine Learning Mastery: Choosing the right AI agent memory strategy
- Hugging Face: KV Caching Explained
- Karpathy llm-wiki gist
- Karpathy LLM Knowledge Bases (X)
- Open Knowledge Format (OKF) spec
- LangGraph: add memory
- LangGraph: persistence
- The Four Types of Memory Every AI Agent Needs (YouTube)
- Architecting Agent Memory: Principles, Patterns, and Best Practices (YouTube)
- Google's OKF for agent knowledge (YouTube)
- gbrain (Garry Tan)
- gstack (Garry Tan)
- Week 5 module (embedded videos + articles)
- Capstone Build Sprint module
Office hours, system brief, production-readiness checklist, backup recording.
- Demo Day module
Mandatory live URL. Optional 5 to 10 minute presentation.
- How to give a great tech talk (Pragmatic Engineer)
AI Engineering learning resources
A curated reference for students. Every link is a primary source or a practitioner-written guide, favouring free material where possible.
- LLM Engineer roadmap (GitHub)
Best curated roadmap. Follow the LLM Engineer path (not LLM Scientist) plus the LLM Engineer's Handbook.
- The AI Engineering Curriculum: 5 Tracks, Ranked and Sequenced
Production-focused, updated in the last 12 months. Blunt take: RAG is how most companies ship AI in 2026.
- Neural Networks: Zero to Hero (Andrej Karpathy, YouTube)
Build LLMs and transformers from scratch - understand what breaks and why.
- LLMs from scratch (rasbt, GitHub)
Pairs with Karpathy for hands-on code alongside the videos.
- Generative AI for Beginners (Microsoft, GitHub)
21 lessons on prompt engineering, responsible AI, and LLMOps with code samples.
- Generative AI Engineering with LLMs (IBM / Coursera)
Free to enroll. Hugging Face, PyTorch, RAG, LoRA and QLoRA. Needs working Python and ML basics.
- LLM Engineering (Ed Donner, Udemy)
Most popular paid hands-on option - 8 weeks, 20+ models, RAG, QLoRA, agents.
- LLM evaluations course roundup (Evidently AI)
Free ~1 hour track: evaluation methods, LLM judges, guardrails, building eval datasets.