Have an interview date?Answer a few questions and get a week-by-week plan with a countdown, what to skim and when to run your mock loop.Build my plan →

Applied AI Engineer Playbook. Build production LLM systems: prompting, agents, retrieval, evals, latency and cost, then the system design round. For: backend and ML engineers.

Applied AI Engineer map

Build it for production.Defend it in the loop.

0%
complete
0/61
graded tasks done
56h
to go

Start here

E1 · Prompting

Anatomy of a Production Prompt

Leo Martins is waitingBegin →

Ready for the full loop?

Loop mode is closed book: no tutor, hints or reveals, and your first graded attempt on each round is the score.

Modules

0 of 114 lessons
  1. E1·PromptingFull course

    Production Prompting & Structured Outputs

    System prompts, JSON schemas, extraction reliability and prompt regression tests.

    Leo Martins, Staff engineer6 lessons · 2.7h
    0/6
  2. E2·AgentsFull course

    Tool Use & Agents

    Tool loops, MCP servers, sub-agents, failure handling and traces.

    Leo Martins, Staff engineer8 lessons · 3.6h
    0/8
  3. E3·RetrievalFull course

    Retrieval at Scale

    Chunking, hybrid search, reranking, retrieval evals and citations.

    Leo Martins, Staff engineer6 lessons · 2.7h
    0/6
  4. E4·EvalsFull course

    Evals as Engineering

    Golden sets, LLM-as-judge calibration and regression gates in CI.

    Leo Martins, Staff engineer6 lessons · 2.8h
    0/6
  5. E5·ProductionFull course

    Production Concerns

    Streaming, latency budgets, prompt caching, rate limits, fallbacks, observability and cost.

    Leo Martins, Staff engineer6 lessons · 2.4h
    0/6
  6. E6·System designFull course

    LLM System Design Interview

    Design rounds for LLM systems: requirements, architecture, evals, failure modes and cost, graded against an interviewer's rubric.

    Leo Martins, Staff engineer9 lessons · 5.2h
    0/9
  7. E7·Mock loopFull course

    Mock Interview Loop

    A full practice loop: timed progressive coding, system design, project deep dive and values.

    Leo Martins, Staff engineer11 lessons · 6.7h
    0/11
  8. E8·Gannet Home EnergyFull course

    Agent Evals and Reliability

    Evaluate agents the way they fail in production: end-state and trajectory grading in a sandbox, pass^k with clustered intervals, simulated users, error analysis into a ranked taxonomy, overfitting-safe hill-climbing, synthetic and adversarial suites with attack success rate as a gate, rerun policies for flaky cases, and a broken harness to debug. Builds on E4.

    Teodora Kessler, Head of AI Quality15 lessons · 7.9h
    0/15
  9. E9·Tallowmere Industrial Supply (fictional)Full course

    Multimodal Input, Streaming and Throughput at Scale

    Route PDFs and phone photos into Claude with page-verified extraction and an abstain path, ground answers on the native Citations API, parse the raw streaming protocol with cancellation and TTFT, run async pools and Message Batches jobs that respect rate limits, and find the exact byte that breaks a prompt cache.

    Imani Ferrowell, VP of Order Operations12 lessons · 6h
    0/12
  10. E10·Lumenfold Media (fictional)Full course

    Inference, Embeddings and Model Choices

    What happens below the API: prefill and decode, the KV cache, batching and rate-limit pools; embeddings and approximate search; when fine-tuning or distillation is justified; OpenTelemetry for LLM calls; and choosing and migrating between Opus 5.5, Sonnet 5.5, Haiku 5.5 and Fable 5.1.

    Noor Haddad, Head of ML Platform12 lessons · 5.5h
    0/12
  11. E11·Fernhollow Travel (fictional)Full course

    TypeScript for Claude

    The TypeScript side of the job, run locally with Node (the browser course runs Python only): typed Messages API calls, streaming to a Next.js route handler, tool loops, structured outputs and prompt caching with @anthropic-ai/sdk, and an MCP server with @modelcontextprotocol/sdk. Practised through code-reading checks, a code review, a Python-to-TypeScript port and a local lab.

    Ada Wexley, Staff Engineer9 lessons · 3.7h
    0/9
  12. E12·Kestrel Code (fictional)Full course

    LLM Inference & Serving in depth

    One layer below E10: the inference pipeline and the roofline, KV-cache size math, GQA, paged KV and prefix caching, continuous batching and chunked prefill, speculative decoding and quantization, tensor, pipeline and expert parallelism, the levers you control on a managed API like Claude's, and the latency and cost playbook that decides between self-hosting and the API.

    Ines Moreau, VP of Infrastructure14 lessons · 6.6h
    0/14

Unlock every track.The first lessons of every track are free. Get all four tracks, every graded task and three mock loops with a one-time payment.

Get full access · $249