From Software Engineer to AI Engineer
A structured path through generative AI, RAG and agentic systems, following the arc of my book Designing Systems That Think — five parts, 19 chapters, 186 sections. Every chapter ends with something that runs, building on eight years of shipping production platforms.
The book behind the journey
Foundations
5 chapters · 46 sectionsThe environment every later chapter extends, the physics of what a model can do, and fluency with hosted APIs, open weights and engineered prompts.
Environment and Ground Rules
Five decisions made in the first hour determine how expensive the next two years are — this chapter makes all five and builds the repository every later chapter extends.
Deliverable: The repository the whole book extends: compose stack, the four seams, secrets hygiene and cost guardrails.
Core Foundations of Generative AI
Four quantities decide what is architecturally possible: the token, the context window, the quadratic cost of attention, and the geometry of embedding space.
Deliverable: A custom BPE tokenizer built from scratch, and a context-window budget with overflow as an observable event.
Working with Model APIs
How a request reaches a model is decided in an afternoon, usually by whoever writes the prototype, and it constrains everything for two years — decided here three times: good, better, best.
Deliverable: A provider gateway — one seam, one normalised result — with router, cache, ledger and a degraded path.
Running Models Locally
Self-hosting is the decision most often made for the wrong reason and abandoned for the right one — this chapter ends with the arithmetic that tells you which side of the break-even you are on.
Deliverable: A tiered Ollama + FastAPI serving stack, plus the local-versus-hosted utilisation sweep.
Prompting and Instruction Formats
The prompt is the highest-leverage and least-governed object in most AI systems — it determines quality more than model choice, yet usually has no version, no test and no owner.
Deliverable: A versioned prompt registry with typed contracts, regression checks and gated artifacts.
Retrieval
3 chapters · 33 sectionsGrounding models in real documents — the full RAG loop end to end, the retrieval-quality work that separates a demo from production, then scale.
Chat with PDF, End to End
RAG is the most built and least engineered pattern in the field — the demo takes an afternoon; the version that survives documents you did not choose takes a quarter.
Deliverable: A chat-with-PDF system with an ingestion contract, grounded and cited answers, an abstain path, and deletion that deletes.
Retrieval Quality
The highest-leverage chapter in the book — most teams reach for a better model when the failure is three stages upstream, in a component nobody measures.
Deliverable: A hybrid BM25 + dense retriever with reciprocal rank fusion and cross-encoder reranking, measured on a labelled case set.
Scaling with Async Queues
Generation takes seconds to minutes; web request handling is designed for milliseconds — almost every scaling failure in an AI product comes from ignoring that mismatch one release too long.
Deliverable: An ARQ worker fleet behind FastAPI with SSE delivery, idempotency, dead letters and age-based scaling.
Agents
5 chapters · 53 sectionsFrom a bare reason–act–observe loop to orchestrated, persistent, tool-wielding and voice-capable agents — where the earlier builds start composing into systems.
Agent Fundamentals
An agent is a model whose output determines what happens next — a small-sounding change that converts a bounded call into an unbounded process that decides for itself when to stop.
Deliverable: A bounded ReAct agent with tool schemas as contracts, step and cost budgets, and a permission chokepoint.
Orchestration with Explicit Graphs
The free agent's variance is the price of its freedom — this chapter asks how much of that freedom the system actually needs. Usually less than the design grants.
Deliverable: A declared graph with typed state, per-node budgets, checkpoints, human-in-the-loop interrupts and per-node evaluation.
Memory and Persistence
Memory is state that outlives a run — the feature most often built as a buffer and later found to have no provenance and no deletion path.
Deliverable: Governed memory: typed facts with a gated write path, supersession over overwrite, deletion, and poisoning defences.
Tools, MCP and the Permission Model
Capabilities the system does not own: tools written by other people, whose schemas can change without consent and whose output goes straight into the prompt.
Deliverable: A governed tool supply chain — MCP with per-node scoping, an idempotency ledger, and result handling that treats output as untrusted input.
Voice and Conversational Agents
Text tolerates a slow answer; conversation does not — silence past roughly eight hundred milliseconds reads as a broken system, and the whole pipeline has to fit inside it.
Deliverable: A fully streamed voice pipeline inside the latency budget, with barge-in, grounded first sentences and measured quality.
Production
4 chapters · 39 sectionsThe gap between a demo and a system — statistical evaluation, guardrails on what output may cause, real deployment — closed by a capstone that puts the whole system on one page.
Evaluation and Regression Testing
Correctness here is statistical, so testing is sampling — the pipeline is easy; the case set is the work, and it is the thing that makes every earlier decision defensible.
Deliverable: A gated eval suite: a real case set, deterministic scorers first, a calibrated judge, and canary with cheap rollback.
Safety, Guardrails and Human-in-the-Loop
Evaluation tells you what the system does; this chapter is about what it is permitted to do — you cannot control what the model reads, only what its output is allowed to cause.
Deliverable: An output-policy chokepoint with designed degradation, redaction across five surfaces, audit trails and rehearsed incident response.
Deployment, Scaling and Cost
What changes when the system is not yours to restart at will — mostly mechanisms built earlier for other reasons, now answering to someone at three in the morning.
Deliverable: A production posture: cost attribution, canaried rollout without a deploy, SLOs that are not uptime, and runbooks that work.
Capstone: The Whole System
Sixteen chapters assembled on one page, then reduced to something more portable — a small number of habits that work on systems built with tools that do not exist yet.
Deliverable: The whole system on one page, a decision log, the twenty-minute review, and the architect's checklist.
Practice
2 chapters · 15 sectionsArgument turned into measurement — exercises designed to contradict expectation, then projects where the constraints arrive together and interact.
Exercises
Seventeen chapters of argument are worth about one afternoon of measurement — these exercises are the afternoon, and several are designed to contradict what you expect.
Deliverable: Three artefacts of measurement per part — the ceiling in your own numbers, a broken gateway, an honestly priced task.
Project Ideas and Real-World Problems
Exercises isolate one argument each; projects remove the bounds — the difficulty of real work is that the constraints arrive together and interact.
Deliverable: A scoped, measured, written-up project — climbing a ten-project skills ladder from a weekend build producing one number to a one-quarter agentic application.
How I work through it
Order matters
Parts build on each other — foundations feed retrieval, retrieval feeds agents, everything lands in production and gets practised. Skipping ahead is how demos get built and systems don't.
Build, don't watch
Every chapter ends with something running on my machine. The section list is a proxy; the working build is the bar.
Production is the point
Evaluation, guardrails and deployment aren't an appendix — they're a full stage, because that's the gap between a demo and a system.
A note on this content
The book and its chapters are my personal learning notes — compiled from online research and hands-on practice, with most of the content AI-generated from that research and learning. It is not a peer-reviewed publication, and I make no claim that it is 100% error-free. If you spot a mistake, I'd genuinely appreciate hearing about it — contact me.