本页目录
- At a glance
- See the results
- Start in 5 minutes
- Learning path
- Portfolio projects
- 📚 Course 1: Hand-written RAG (9 lessons)
- 🤖 Course 2: Hand-written Agents (9 lessons)
- 🔧 Course 3: Framework Engineering (9 lessons)
- 🔀 Course 4: Workflow & Multi-Agent Orchestration (9 lessons)
- 🛡️ Course 5: LLMOps in Production (13 lessons)
- 📄 Course 6: Multimodal Document Intelligence (10 lessons)
- 🧠 Course 7: Agent Frontiers (13 lessons)
- 🖥️ Course 8: GUI Agent / Computer Use (13 lessons)
- 🛡️ Course 9: Agent Production Reliability / AgentOps (10 lessons)
- 🌙 Course 10: Ambient / Always-on Proactive Agents (10 lessons)
- 🧠 Course 11: Agent Harness & Context Engineering (10 lessons)
- Course 12: DeepSeek Harness Hands-On (6 labs + 12 deep-reading chapters)
- Course 13: DeepSeek Harness Design-Decision Case Studies (8 cases)
- Verification
- 📁 Directory Structure
- 💡 Study Tips
- Contributing
Awesome Agent Engineering
From hand-written RAG and ReAct loops to Agent systems that can be evaluated, observed, and deployed.
中文 | English
A hands-on LLM application engineering course for Python developers. Its 135 lessons follow one continuous path: hand-write the core mechanisms, translate them into LangChain and LangGraph, study a real open-source Harness from runtime internals through protocols and product surfaces, then integrate the ideas into two tested projects with evaluation, APIs, and Docker support.
This repository goes beyond API recipes. It asks: Why choose this design? What are the trade-offs? How can an experiment prove that a new mechanism actually helps?
Start in 5 minutes · Learning path · Portfolio projects · Contributing
At a glance
| Courses | Portfolio apps | Automated tests | Languages |
|---|---|---|---|
| 13 / 135 lessons | 2 | 592 + 20 TS suites | Chinese + English |
- Principles before frameworks: hand-written RAG, Function Calling, and ReAct are paired with framework implementations.
- Evidence before claims: RAGAS, ablations, trajectory evaluation, and local mini-benchmarks run through the curriculum.
- Engineering beyond demos: auth, rate limiting, tracing, caching, load testing, MCP, and Docker land in the portfolio apps.
- Current topics: multimodal documents, Agent memory, CodeAct, long-running tasks, GUI Agents, production reliability, ambient Agents, context engineering, and open-source Harness internals.
See the results
| Enterprise Knowledge Base QA | AI Research Assistant |
|---|---|
![]() |
![]() |
| Hybrid retrieval, reranking, citations, multimodal parsing | Parallel research, review loops, memory, browser evidence |
Screenshots were rendered from the local interfaces with illustrative data. External APIs are called only when you explicitly run an API-backed example.
Start in 5 minutes
Run the dependency-free, API-key-free tour first:
python quickstart.py
python quickstart.py "How many annual-leave days after 5 years?"
The tour uses local character n-gram retrieval and a deterministic answerer. It exposes the RAG data flow without pretending to be a real LLM. Then run Lesson 01 with Zhipu AI:
python -m venv .venv
# Windows
.\.venv\Scripts\python.exe -m pip install -r requirements-quickstart.txt
# macOS / Linux
# .venv/bin/python -m pip install -r requirements-quickstart.txt
copy .env.example .env # macOS / Linux: cp .env.example .env
# Set ZHIPUAI_API_KEY in .env
.\.venv\Scripts\python.exe rag-lessons\01_getting_started\code.py
Install requirements.txt only when you need the complete course stack; browser, OCR, and voice dependencies are intentionally not required for Lesson 01.
Learning path
| Stage | Course | Main outcome | Progress |
|---|---|---|---|
| Foundations | Hand-written RAG | Embeddings, retrieval, chunking, evaluation | 9/9 |
| Foundations | Hand-written Agents | Function Calling, ReAct, planning, memory | 9/9 |
| Engineering | Framework Engineering | LangChain, LangGraph, state, HITL | 9/9 |
| Architecture | Multi-Agent Orchestration | Supervisors, swarms, subgraphs, parallelism | 9/9 |
| Production | LLMOps | Observability, security, MCP, performance and cost | 13/13 |
| Applied | Multimodal Documents | PDF, OCR, tables, charts, citation provenance | 10/10 |
| Frontier | Agent Frontiers | Memory, reflection, CodeAct, trajectory evaluation | 13/13 |
| Frontier | GUI Agents | Browser control, vision, reliability, security | 13/13 |
| Production | Agent production reliability | Step/cost budgets, circuit breaker, idempotent approvals, durable resume, chaos eval | 10/10 |
| Frontier | Ambient Agents | Scheduled triggers, change detection, incremental research, interruption policy, always-on operations | 10/10 |
| Frontier | Agent Harness | Context ledger, disciplined compaction, memory files, subagent isolation, steering & progressive disclosure | 10/10 |
| Hands-on | DeepSeek Harness hands-on source study | Run it, extend it, take it apart in a real checkout: 6 keyless labs + 12 deep-reading chapters | 6+12 |
| Advanced | DeepSeek Harness design decisions | Eight decision case studies: protocols, attribution, projections, composition, ownership, approval, testing, evolution | 8/8 |
Portfolio projects
| Project | Verifiable capabilities | Tests |
|---|---|---|
| Enterprise Knowledge Base QA | Hybrid retrieval + reranking, citations, RAGAS, auth, rate limits, MCP, multimodal parsing | 143 |
| AI Research Assistant | LangGraph multi-Agent flow, SSE, reviews, memory, CodeAct, trajectory evaluation, browser evidence, production reliability, ambient mode, long-haul research | 449 |
Both projects expose FastAPI services, Docker setups, tests, and fallback paths when optional external capabilities are disabled. They are engineering references, not universal production-capacity guarantees; load-test and validate them in your own deployment environment.
Expand the complete 135-lesson catalog
📚 Course 1: Hand-written RAG (9 lessons)
Follows the real RAG data flow, adding one stage per lesson:
| # | Lesson | You'll learn |
|---|---|---|
| 01 | Get it running: your first RAG | Run the full pipeline end-to-end and build the big picture |
| 02 | Deep dive into Embeddings | How vectors represent semantics, cosine similarity |
| 03 | Vector Retrieval | Top-K, ANN, and using Chroma |
| 04 | Chunking | The trade-offs of chunk_size / overlap |
| 05 | Prompt Engineering | Anti-hallucination prompts, citation grounding |
| 06 | Advanced Retrieval | Hybrid retrieval + reranking |
| 07 | Query Rewriting | HyDE, multi-query expansion |
| 08 | RAG Evaluation | The three RAGAS dimensions |
| 09 | Engineering: Capstone | An interactive QA assistant integrating everything |
All 9 lessons done 🎉. Each lesson has a principle walkthrough + runnable code + exercises.
🤖 Course 2: Hand-written Agents (9 lessons)
Builds up agent capability layer by layer—each lesson adds one ability (tools → loop → memory → planning → collaboration):
| # | Lesson | You'll learn |
|---|---|---|
| 01 | Meet the Agent: from Q&A to action | Run a minimal agent; grasp "LLM + tools + decision" |
| 02 | Function Calling in depth | Understand the function-calling mechanism; hand-write a generic tool dispatcher |
| 03 | ReAct: the think-act-observe loop | Hand-write a minimal ReAct loop (no framework; an interview staple) |
| 04 | Multiple tools & tool design | Trade-offs across 5+ tools; how description quality affects selection |
| 05 | Memory: remembering context | Multi-turn dialogue, context-window limits and handling strategies |
| 06 | Planning & task decomposition | The Plan-and-Execute paradigm vs ReAct, and when to use which |
| 07 | Agentic RAG: Agent + RAG | Wrap RAG as a tool; let the agent decide when to retrieve |
| 08 | Multi-agent collaboration | Multiple agents, each with a role, cooperate on complex tasks |
| 09 | Capstone: smart research assistant | Web search + structured research report (résumé-grade) |
All 9 lessons done 🎉. Each lesson has a principle walkthrough + runnable code + exercises.
🔧 Course 3: Framework Engineering (9 lessons)
Re-implement what you hand-wrote in the first two courses with LangChain / LangGraph, comparing "hand-written vs framework" each lesson:
| # | Lesson | You'll learn |
|---|---|---|
| 01 | LCEL & the framework landscape | Hand-written RAG vs LCEL—see what the framework does for you |
| 02 | The trio: Models + Prompts + Parsers | Standardized building blocks for calling models, writing prompts, parsing output |
| 03 | Documents: Loaders + Splitters + VectorStores | The engineering pipeline for getting data in |
| 04 | Retrievers + RAG Chain | Compose blocks with \| into a full RAG chain |
| 05 | Advanced retrieval engineering | Ensemble + MultiQuery—where the framework really pays off |
| 06 | LangGraph basics | Rewrite ReAct with StateGraph (the pivot from LangChain to LangGraph) |
| 07 | Framework-level Agents | @tool decorator + create_agent—dozens of hand-written lines in a few |
| 08 | State, memory & human-in-the-loop | Checkpointer persistence + interrupt HITL (LangGraph's killer feature) |
| 09 | Capstone: LangGraph research assistant | Multi-node graph + Checkpointer, integrating all framework skills |
All 9 lessons done 🎉. Each lesson has a principle walkthrough + runnable code + exercises.
🔀 Course 4: Workflow & Multi-Agent Orchestration (9 lessons)
The first three courses cover "single agent + single flow." This course moves into multi-agent orchestration—a core skill for the AI architect track. LangGraph is the backbone for 6 classic topologies, then CrewAI / AutoGen are used for cross-paradigm comparison on the same problem:
| # | Lesson | You'll learn |
|---|---|---|
| 01 | Supervisor pattern | Centralized dynamic routing (vs the hard-coded loop from hand-written L08) |
| 02 | Swarm & Handoff | Decentralized swarm + state handoff (vs hand-written string concatenation) |
| 03 | Subgraphs | Embed a compiled graph as a node for modular reuse |
| 04 | Parallel Map-Reduce | fan-out burst + reducer merge (parallelism hand-writing can't do) |
| 05 | Shared-state communication | Compare messaging / shared state / blackboard |
| 06 | Multi-model routing & topology | Star / ring / mesh / hierarchical topologies + cost control |
| 07 | CrewAI comparison | Role-driven declarative orchestration vs LangGraph supervisor |
| 08 | AutoGen comparison | Conversation-driven group chat vs LangGraph swarm |
| 09 | Capstone: multi-agent research system | supervisor + parallel + shared state + multi-model (résumé-grade) |
All 9 lessons done 🎉. Each lesson keeps the "hand-written Agent L08 pipeline vs framework multi-agent" side-by-side. The L09 capstone integrates all of L01–L08 and is a résumé-grade piece.
🛡️ Course 5: LLMOps in Production (13 lessons)
The first four courses teach you to build an AI app; this one teaches you to operate it—answering the interviewer's "and after your project goes live? How do you know it's good, defend against attacks, get integrated by other systems, control cost?" All changes land directly on knowledge-base-qa, upgrading it from "running demo" to "ops-ready v2." Four modules, progressively:
| # | Lesson | You'll learn |
|---|---|---|
| 01 | Structured logging | From print to queryable JSON event streams + trace_id across the chain |
| 02 | Langfuse end-to-end tracing | Visualize per-query retrieval / rerank / generation latency, tokens, cost |
| 03 | Online eval loop | Real-query sampling + automated ragas scoring + bad-answer queue |
| 04 | API auth & rate limiting | Key auth + per-key rate limiting—prevent open access and runaway bills (401/429/200) |
| 05 | Prompt injection offense/defense | Indirect injection (malicious instructions hidden in docs) + build an attack test set and run a breach baseline |
| 06 | I/O guardrails | Material isolation + instruction/data separation + output filtering, hardened into CI |
| 07 | What is MCP | The "USB port" for AI apps: M×N→M+N; hand-write a minimal server/client |
| 08 | Wrap the KB as an MCP Server | Expose kb-qa retrieval as a standard tool; any host integrates with zero code |
| 09 | Agent as MCP Client | research-assistant calls kb-qa—connecting the two portfolio projects |
| 10 | Semantic caching | Cache hits on synonymous queries, skipping retrieval + generation |
| 11 | Load testing & concurrency | QPS / P95 / P99 baselines; locate the bottleneck at the upstream API limiter |
| 12 | Cost/quality trade-offs | Quantify glm-4 vs flash on eval data; per-stage model selection to cut cost |
| 13 | Capstone: ops-ready v2 | An ops dashboard + a production launch checklist tying all 12 lessons together |
All 13 lessons done 🎉. Teaching
code.pyfiles are all zero-dependency or have mock fallback paths so they run standalone; production changes go into kb-qa with a "## rollout checklist." Places that can't run real external services (Langfuse / Docker / load testing) are honestly marked as unverified with a fallback path.
📄 Course 6: Multimodal Document Intelligence (10 lessons)
The first five courses built a RAG pipeline that only eats "clean plain text"—but real enterprise knowledge bases are full of scans, tables, and charts, which a text-only pipeline is blind to. This course teaches converged engineering knowledge (document parsing / OCR / table handling have mature industry practices), upgrading kb-qa from "text-only" to a "multimodal document intelligence system v3 that eats scans / tables / charts with citations traceable to page + region." The tone aligns with the ops course (standard practices + trade-offs); every lesson has a "## approach comparison" section. Two through-lines: ① cost-accuracy (every multimodal decision trades cost for accuracy); ② provenance (citations upgrade from chunk text to doc name + page + region).
| # | Lesson | What you learn |
|---|---|---|
| 00 | Overview & baseline | Real enterprise doc composition + quantifying the text-RAG ceiling (scan/table/chart questions at 0%) + poison doc set + bare baseline |
| 01 | PDF anatomy & layout parsing | PDF three-layer structure + a layout-aware parser (Element with type and bbox) + classification routing |
| 02 | Tables: from serial text to structured | pdfplumber extraction + a markdown/HTML/serial three-representation comparison experiment + whole-table chunking with header redundancy |
| 03 | Scans: three OCR routes | Local RapidOCR vs VLM direct read vs confidence-routed hybrid (a textbook cost-accuracy case) |
| 04 | Charts & image understanding | glm-4v-plus two-stage (description cache for indexing + live image read for answering) + hash dedup |
| 05 | Multimodal retrieval | Description indexing makes charts searchable + element_type routing + CLIP dual-tower comparison |
| 06 | Citation provenance upgrade | Citations upgrade from chunk text to page + region (bbox) + region clip images + credibility trilogy step 3 |
| 07 | Voice entry (exploratory) | ASR → kb-qa → TTS full pipeline + latency breakdown (voice is an entry point, not the core) |
| 08 | Multimodal evaluation: gains table | Per-mechanism switch matrix + anti-regression control + ragas multimodal blind spot + ingest cost column |
| 09 | Capstone: v3 + renumbering | All mechanisms协同 on the hard task + kb-qa v3 finalization + repo-wide course renumbering |
All 10 lessons done 🎉. Two through-lines: ① cost-accuracy (VLM direct read is expensive but strong, local OCR is cheap but brittle—classification routing is the engineering answer); ② provenance (citations upgrade from chunk text to doc name + page + region, the third step of the credibility trilogy). All new mechanisms default off (
enable_multimodal_ingestetc.), existing tests stay green, every lesson has a "## approach comparison" + at least one "design experiment" exercise.
🧠 Course 7: Agent Frontiers (13 lessons)
The first six courses teach converged knowledge (how to chunk for RAG, how to write ReAct). This course teaches not-yet-converged frontiers—agent memory, reflection, Code Agents, trajectory evaluation, context engineering—where the industry has no standard answer. So the style changes: the README doesn't lecture "the standard way," it lays out "which schools of thought exist, what the trade-offs are, and why we picked X…"; the code is "hand-write the core mechanism + a design experiment to test whether it helps." All changes land on research-assistant, growing it from a one-shot "search → write report" system into a cross-session Deep Research Agent v2. Six modules:
| # | Lesson | You'll learn |
|---|---|---|
| 00 | Method warm-up | The three-pass paper reading method + reading LangGraph source + running an amnesiac baseline (reference throughout) |
| 01 | Memory tiers | Episodic (Chroma) + semantic (list) MemoryStore; researcher gets recall |
| 02 | Reflective writes | reflect_and_store distills memory + consolidate reinforces + forgetting policy |
| 03 | Skills & context engineering | Progressive skill_loader; unify memory / skills / RAG / MCP under context engineering |
| 04 | Hand-written Reflexion | Three-component loop + blind-retry vs reflective-retry comparison + ablation |
| 05 | Reflection into the research loop | Dual-channel reviewer (text + facts) + conflict detection + targeted re-research and correction |
| 06 | Hand-written CodeAct | Code as the action space + process-level sandbox (import allowlist / timeout / truncation) |
| 07 | Code interpreter lands | code_interpreter wired into writer; report numbers become reproducible |
| 08 | Trajectory evaluation | TrajectoryEvaluator: success rate / steps / loops / attribution + mechanism-trigger detection |
| 09 | Eval Harness | Switch matrix × task set = mechanism-gains table (regression-style eval) |
| 10 | Long-horizon tasks | TaskLedger: TODO tree + resume-from-checkpoint + incremental briefings |
| 11 | Capstone | Deep Research v2: five mechanisms in concert + architecture doc + gains table |
| 12 | Frontier-tracking method | Full three-pass reading method + framework evaluation checklist + minimal multi-agent memory-sharing repro |
All 13 lessons done 🎉. Two through-lines: ① an evaluation main line (L00 sets the baseline → L08 builds the evaluator → L09 harness quantifies every mechanism's gain); ② a context-engineering main line (memory / skills / RAG / MCP unified under the one question "what goes in the window"). Each lesson's README has a "schools of thought" section + at least one "design experiment to validate" exercise. 104 unit tests green; all new mechanisms default-off, with intact fallback paths.
🖥️ Course 8: GUI Agent / Computer Use (13 lessons)
The first seven courses grew research-assistant into a deep agent that thinks—but it only has a brain, no hands: its sole channel to the world is search snippets. This course teaches a frontier that is still unconverged in 2025–2026: letting the agent operate a browser directly (open pages, click, paginate, extract, download), growing research-assistant a pair of hands that are steady, safe, and measurable. The style continues Course 7: READMEs lay out "the three schools of thought (text / vision / dedicated models), their trade-offs, and why we pick X…"; the code is "hand-write the core mechanism + a design experiment to test whether it helps." All changes land on research-assistant; enable_browser defaults to off, and all 123 tests stay green.
| # | Lesson | You'll learn |
|---|---|---|
| 00 | Landscape & baseline | Map of the three schools + WebArena/SeeAct/OSWorld primer + hard-task definition + run the bare baseline (what search snippets can't get you) |
| 01 | Playwright foundations | Deterministic BrowserSession control (auto-wait / timeout fallback / context manager) + slow-load & popup pages |
| 02 | Observation space | page_to_obs with three page representations (raw HTML / numbered element list / plain text) + token comparison (9x savings) |
| 03 | Action space | Constrained action DSL (click/type/scroll/back/finish) + parse & validate + structured error feedback for illegal actions |
| 04 | Minimal GUI Agent | observe→think→act loop + sliding-window context trimming + mock-LLM zero-API run |
| 05 | Vision route | SoM-annotated screenshots into glm-4v-plus + text/vision/hybrid same-task comparison (tokens / success rate) |
| 06 | Reliability engineering | Failure-mode checklist + loop detection (observation hashing) + strategy switching + tricky-page before/after |
| 07 | Web injection offense & defense | GUI injection is an order of magnitude worse than RAG (doing wrong vs saying wrong) + action-layer defense (allowlist / sensitive-action confirmation / injection scanning) |
| 08 | Evaluation mini-benchmark | The WebArena idea: self-hosted local task set + functional acceptance + two-layer eval with the trajectory evaluator |
| 09 | Landing: growing "hands" | browser_tool.py wired into researcher (async + security on by default + fallback chain + 17 tests) |
| 10 | Deep browsing & evidence chains | deep_browse multi-step evidence gathering + evidence chains (URL + access time + snapshot) + revisitable report citations |
| 11 | Capstone | A web-browsing Deep Research Agent: four layers in concert + architecture doc + gains table (success rate 75%→100%) |
| 12 | Frontier tracking | Dedicated models vs general VLM + scaffolding: a three-axis framework + a minimal SoM-ablation repro |
All 13 lessons done 🎉. Two through-lines: ① an evaluation main line (L00 bare baseline → L08 mini-benchmark → L11 gains table quantifying every mechanism); ② an observation–action interface main line (L02 observation space → L03 action DSL → L04 loop closure—the context-engineering theme extended to GUI). Each lesson's README has a "schools of thought" section + at least one "design experiment to validate" exercise. Landing adds 19 browser tests to research-assistant (123 total, all green);
enable_browserdefaults to off with intact fallback paths.
🛡️ Course 9: Agent Production Reliability / AgentOps (10 lessons)
ops-lessons protects a single request (auth, rate limiting, guardrails); this course protects a trajectory—an Agent that loops many times and decides its own next step, so the failure modes are fundamentally different: infinite loops, cost blowouts, fault propagation, dangerous side effects, mid-run crashes. kb-qa is a linear chain that doesn't need these mechanisms; research-assistant is a loop body that can't ship without them—this asymmetry is itself evidence of the boundary. The style follows ops-lessons (teaching "the standard practice + the trade-offs"); each lesson has a "comparison of approaches" section. All changes land on research-assistant, upgrading it from the capable "Deep Research Agent v2" into a production-reliable v3: survives faults, gates dangerous actions, recovers from crashes, and has SLO numbers. Ten modules:
| # | Lesson | What you learn |
|---|---|---|
| 00 | Landscape & baseline | Request-guard vs trajectory-guard boundary + risk map + six-fault chaos suite + bare baseline (all blast radii unbounded) |
| 01 | Steps & loops | Global step budget (add_int reducer) + action-signature loop detection + honest truncation (partial result, not a crash) |
| 02 | Cost budget | Trajectory-level token wallet (usage_metadata metering) + soft-budget downgrade / hard-budget truncate + per-node cost table |
| 03 | Timeout, circuit breaker & honest degradation | Hand-written 3-state circuit breaker + structured degradation protocol + fallback chain |
| 04 | Side effects & idempotency | Side-effect classification + idempotency key (thread_id + content hash) + sqlite registry + dry-run + optional publish node |
| 05 | Human-in-the-loop approval | langgraph interrupt/resume gate + policy layering (first_only reuses the idempotency key) + cross-process resume |
| 06 | Durable resume | jobs registry + checkpoint resume (completed nodes not re-run) + recover_orphans + boundary vs frontier-L10 ledger |
| 07 | Trajectory observability | One-line run-summary health report + threshold alerts + three-layer split with request logs / evaluation |
| 08 | Reliability evaluation | Chaos gains matrix (six faults × all-off/all-on) + SLO card + clean-run zero-tax regression (success 33%→100%) |
| 09 | Capstone | End-to-end with all mechanisms + research-assistant v3 finalization + seven-mechanism governance + repo-wide Course 9 registration |
All 10 lessons done 🎉. Two through-lines: ① a blast-radius main line (L00 measures five unbounded failure modes → each lesson bounds one: loops→step-bounded, cost→budget-bounded, faults→degradation-bounded, side-effects→idempotent+approval-bounded, crashes→redo-bounded); ② an autonomy-vs-control main line (every protection trades autonomy/latency/human-effort for safety—too tight and the Agent is useless, too loose and it's reckless; each lesson gives the "when tight, when loose" criterion). Each lesson's README has a "comparison of approaches" section + at least one "design experiment" exercise. Landing adds 96 tests to research-assistant (219 total, all green); all new mechanisms default off with zero tax on clean runs.
🌙 Course 10: Ambient / Always-on Proactive Agents (10 lessons)
The first nine courses build conversational (pull) Agents: a human initiates, the Agent answers, the run ends. This course inverts the paradigm (push): the Agent lives in the background, wakes on schedule, decides what changed in the world, and only interrupts when it is worth your attention. An active 2025–2026 frontier (OpenAI scheduled tasks / Pulse, LangChain ambient agents, Claude Code background tasks all point here). All landings go to research-assistant, upgrading it from "production-reliable v3" to an ambient-proactive v4 that wakes itself, researches only what changed, knows when to speak, and cannot die silently.
| # | Lesson | What you learn |
|---|---|---|
| 00 | Landscape & baseline | Five-step paradigm inversion + request→trajectory→service boundary + 5-day simulated timeline + manual-watch bare baseline |
| 01 | Triggers & scheduling | Hand-written scheduler: fixed shift grid + countable missed shifts + injectable clock (5 days of scheduling tested in seconds) |
| 02 | Sources & change detection | Item-level content hashing + snapshot diff + two disciplines ("no change" is a first-class result / "couldn't see" ≠ "no change") |
| 03 | Incremental research loop | Focus-as-subtopics (skip re-splitting) + prior-conclusion injection + ✏️ correction briefs (TaskLedger joins the runtime main path) |
| 04 | Interruption policy | major/minor/none grading + hoard-don't-drop degradation + daily interruption quota + three-policy comparison |
| 05 | Inbox & agency levels | Five-channel inbox + overnight approvals (interrupt waits in the checkpoint) + agency ladder (notify/propose/act) |
| 06 | Daemon lifecycle | AmbientDaemon wires everything + one failed cycle never kills the daemon + two-layer recovery + overlap skip + graceful stop |
| 07 | Period budget & ambient observability | Third budget layer (daily total) + adaptive backoff + heartbeat absence detection + one-line daily report |
| 08 | Ambient evaluation | Five configs × six-metric gains matrix (cron tier = baseline, proven / tokens −79% / interruptions 5→1 / absence detectable) |
| 09 | Capstone | "A week of v4" end-to-end with all eight switches on + research-assistant v4 finalization + repo-wide Course 10 registration |
All 10 lessons done 🎉. Two through-lines: ① a paradigm-inversion main line (five steps—who initiates / what to research / who diffs / when to speak / who keeps watch—move from human to machine lesson by lesson; "the cron tier ties the baseline on all six metrics" proves inversion ≠ mere timing automation); ② an attention-economy main line (an ambient Agent spends two currencies that belong to others: the user's attention and overnight tokens—grading + quotas + the period wallet make both auditable). Each lesson's README has a "comparison of approaches" section + at least one "design experiment" exercise. Landing adds 112 tests to research-assistant (331 total, all green); all eight switches default off with zero tax on clean runs, and every test runs with zero real waiting (injectable clock).
🧠 Course 11: Agent Harness & Context Engineering (10 lessons)
The first ten courses all rest on a hidden assumption: the context window of a single run is always big enough. Once a task gets long (30 sources, hundreds of tool calls, work spanning sessions), the window becomes the new ceiling: overflow, silent-truncation amnesia, lost-in-the-middle drift, context poisoning. What lifts the ceiling is not a bigger model but a better harness—the real 2025–2026 battleground of the application layer (Claude Code's compaction/memory/subagents/skills, LangChain deepagents, Manus's context engineering, MemGPT). All landings go to research-assistant, upgrading it from "ambient-proactive v4" to a long-haul research v5 that finishes 30 sources inside an 8k window without forgetting, accepts mid-run steering, and remembers you across sessions.
| # | Lesson | What you learn |
|---|---|---|
| 00 | Landscape & baseline | Window physics + a 30-source long-haul task + four bare baselines (naive run dies at S11 / hard truncation keeps 8/20 facts—alive but amnesiac) |
| 01 | Context ledger | Injectable tokenizer + four-bucket accounting (tool results measured at 75–95%) + three watermark zones |
| 02 | Disciplined compaction | Register-summarize-verify: pinned facts survive mechanically (even a malicious summarizer) + every compaction leaves an audit row |
| 03 | Cross-session memory files | Three kinds of memory + a write-discipline gate + index-resident/body-on-demand + pollution-guarded recall |
| 04 | Tool-result shaping | Truncate/paginate/reference + omission must be explicit (the liar case: silent truncation passes half a document off as the whole) |
| 05 | Subagent isolation | Processes stay in sub-windows, only conclusions return (main-window peak 5,272→706) + structured failure ≠ empty conclusion |
| 06 | File workspace | Pointers instead of payloads (2,763 chars stand in for 91,366) + recitation against drift + crash-resume fetches 30 vs 48 |
| 07 | Steering & permission gate | Safe-point negotiated merges (10 sources of work never die with a re-run) + honest partial reports on soft-stop + three red lines blocked 100% |
| 08 | Progressive disclosure | Extends frontier-L03's skill_loader into a three-layer instruction architecture (core/index/on-demand, 58% saved over 30 calls) |
| 09 | Capstone | v5 end-to-end (steering applied + preferences survive sessions) + five-tier gains matrix + repo-wide Course 11 registration |
All 10 lessons done 🎉. Two through-lines: ① a window-economy main line (the fourth budget layer—space: every token pays rent to stay in attention; the ledger measures rent, shaping cuts it, compaction reclaims, externalization ends the lease—the same coin as Course 10's attention economy, spent on the model instead of the human); ② an externalization main line (virtual memory: window=RAM, files=disk, compaction=swap, subagents=process isolation, indexes=lazy loading). Killer row: "truncation buys survival, not memory." Landing adds 118 tests to research-assistant (449 total, all green); all nine switches default off with zero tax on clean runs, every test deterministic and reproducible (injectable tokenizer, byte-identical double runs).
Course 12: DeepSeek Harness Hands-On (6 labs + 12 deep-reading chapters)
Third redefinition (2026-08): the simulator version taught our toys; the pure reading-guide version lost to upstream's own docs. This version stands on the only defensible ground — you run, extend, and tear down a real checkout yourself. Pinned at
99f6f02(rc.7, shared with Course 13); every lab's commands and outputs were executed by the course author on the same SHA, keyless throughout: in Lab 2 you write a 50-line scripted LLM adapter whose scripted model drives the real loop, tool pipeline, and session log (same shape as upstream's own keyless test lane — the system is real; only the model boundary is pluggable, which is exactly dsh's architectural promise).
| Stage | Content | Entry |
|---|---|---|
| Hands-on (core) | ① observer plugin watches a whole turn's events ② scripted adapter drives the real loop keyless ③ defineTool real tool through the five-stage pipeline ④ unpack the append-only session log ⑤ policy gate & fail-closed denial ⑥ delegate a real subagent, find its durable child session |
labs/ |
| Deep reading (reference) | 12 chapters: "how the conventional design breaks → the mechanism in source → what it buys, what it costs", matched to labs | deep-reading.md |
| Capstone | one real extension in the real checkout, passing upstream's own tests | chapter 11 |
Course 13: DeepSeek Harness Design-Decision Case Studies (8 cases)
Shares the Course 12 checkout (
99f6f02). Every case follows one procedure: dilemma → wounded options → upstream's choice with evidence (source comments, test names, Known Limitations) → hands-on verification → costs → transfer. The signature move is break-it: temporarily remove the decision, run upstream's own test, watch it go red, restore — A05's fail-closed normalization has the author's fully executed transcript (38 green → delete one line →normalizes a rogue non-vocabulary answer to unavailablered withexpected 'yolo' to be 'unavailable'→ restore → 38 green). Shared method across all eight: among wounded options, pick the one whose harm is reversible.
| # | Case | Dilemma in one line |
|---|---|---|
| 00 | Honest protocol boundaries | Cancel won admission: orphan attachment or a late-queued message? |
| 01 | Attribution by receipt, not result | Why does the server refuse to answer "what happened"? |
| 02 | Layered projections | Three redundant layers: waste or division of labor? |
| 03 | Transactional composition | Why may a producing session never swap compositions? |
| 04 | Ownership fences | Secrecy or authorization? Is restart loss documented? |
| 05 | Approval and capability evidence | Nobody answers: allow or deny? (break-it executed) |
| 06 | Evidence layers | What does each test lane prove — and never prove? |
| 07 | Capstone: real evolution | Which decisions did rc.5→rc.7→master drift confirm? |
Verification
python -m pytest portfolio-projects/knowledge-base-qa/tests -q
python -m pytest portfolio-projects/research-assistant/tests -q
./deepseek-harness-lessons/scripts/run_tests.sh
./deepseek-harness-advanced-lessons/scripts/run_tests.sh
The Python projects retain 592 tests; Courses 12 and 13 add 20 dependency-free TypeScript test suites (anchor-checker logic plus course-material consistency) run under Node.js 24 in CI. Both DeepSeek Harness courses answer to their locked SHAs: after scripts/prepare_upstream.sh checks out the real source, each lesson's checker re-verifies its anchors against it, and scripts/check_upstream_drift.sh re-verifies the whole course. Real-model evaluation, full upstream composition, browser performance, and platform sandbox results require separate validation and must not be inferred from offline checks.
📁 Directory Structure
RAG-test/
├── README.md ← Course index (Chinese)
├── README.en.md ← You are here: thirteen courses + portfolio overview (English)
├── requirements.txt ← Python course and project dependencies
├── .env.example ← API key config template
├── data/sample_docs/ ← Sample docs for exercises (shared across courses)
├── data/multimodal_docs/ ← Multimodal course poison doc set (scan/table/chart PDF + golden questions)
├── rag-lessons/ ← Course 1: Hand-written RAG (9 lessons, done)
├── agent-lessons/ ← Course 2: Hand-written Agents (9 lessons, done)
├── framework-lessons/ ← Course 3: Framework Engineering (9 lessons, done)
├── workflow-lessons/ ← Course 4: Workflow & Multi-Agent Orchestration (9 lessons, done)
├── ops-lessons/ ← Course 5: LLMOps in Production (13 lessons, done)
├── doc-intelligence-lessons/ ← Course 6: Multimodal Document Intelligence (10 lessons, done)
├── frontier-lessons/ ← Course 7: Agent Frontiers (13 lessons, done)
├── gui-agent-lessons/ ← Course 8: GUI Agent / Computer Use (13 lessons, done)
├── agent-ops-lessons/ ← Course 9: Agent Production Reliability / AgentOps (10 lessons, done)
├── ambient-agent-lessons/ ← Course 10: Ambient / Always-on Proactive Agents (10 lessons, done)
├── harness-lessons/ ← Course 11: Agent Harness & Context Engineering (10 lessons, done)
├── deepseek-harness-lessons/ ← Course 12: DeepSeek Harness hands-on (6 keyless labs, author-executed, + 12 deep-reading chapters)
├── deepseek-harness-advanced-lessons/ ← Course 13: DeepSeek Harness design-decision case studies (8 cases, break-it executed)
├── portfolio-projects/ ← 🚀 Production-grade portfolio projects (landings after the courses; main battleground for ops/docint/frontier/gui/agentops/ambient)
│ ├── knowledge-base-qa/ ← Enterprise KB QA (RAG, multimodal document intelligence v3)
│ └── research-assistant/ ← AI Research Assistant (multi-agent + FastAPI + Docker, long-haul research v5)
└── docs/ ← Design docs and implementation plans
Each lesson includes ① a principles README (the why and trade-offs) + ② runnable code (code.py for the Python courses and code.ts for Courses 12–13) + ③ exercises. Both DeepSeek Harness courses include a tests/ directory per lesson.
Portfolio projects use a modular engineering layout (src/ + api/ + tests/ + Docker), organized to production standards.
💡 Study Tips
- Run the code. Don't just read. A lot of RAG intuition comes from changing parameters yourself and watching the output change.
- Learn in order—each lesson builds on the previous one.
- Each lesson contains a principles README, runnable code, and experiments. Run the baseline before changing parameters.
- Use the GitHub issue templates for bugs and lesson feedback.
Contributing
Corrections, cross-platform fixes, model adapters, and reproducible experiments are welcome. Read CONTRIBUTING.md before opening a pull request, see CHANGELOG.md for release changes, and report security issues privately according to SECURITY.md.
MIT License · Thanks to the Linux.do community for its support.

