AI Engineer Notebooks – free, framework-free RAG/agents/evals on Colab

Learn the applied-LLM stack the way you'll actually be interviewed on it — framework-free, on a free API, from prompting all the way to serving, fine-tuning, and a red-team benchmark.

Runnable Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set: building working systems on top of foundation models — model APIs, RAG, evals, agents, adaptation, serving — using raw APIs, not frameworks.

Framework-free, on purpose. You write the agent loop, RAG, and evals from raw API calls first — so you understand what LangChain/LlamaIndex actually do before you reach for them (and can judge whennotto). Patterns are durable; wrappers churn.Evals are the spine."Measure before you tune" is installed early and returns in every section — the habit that separates an engineer who shipped a system from one who built a demo.Free to run, end to end. Everything runs on thefree(no credit card). The two topics Groq can't host — LoRA fine-tuning (06) and self-hosted serving (09) — are concept-first with optional, fenced Colab-GPU appendices,GroqAPIverified on a real Colab T4.** Real case studies, not toy demos.**Three end-to-endcase studiesshow the skills combined under real constraints — a support assistantdebugged in production, a pipeline-vs-agent cost showdown, and a red-team robustness benchmark.OpenAI-compatible throughout, so every pattern transfers directly to OpenAI and (with small changes) Anthropic — the seam is swappable, the skills aren't.

Built as the hands-on companion to Plan: Transitioning to Forward Deployed Engineer / AI Engineer. The plan explains what to learn and why; these notebooks are where you run it.

Backend or full-stack engineers moving into AI Engineer, FDE, Applied AI, or Solutions Engineer (AI) roles — different titles, largely the same job. You can ship production code; you want the applied-model layer on top.

Work top to bottom. Each notebook is self-contained (installs its own dependencies, reads API keys from Colab secrets) and ends with exercises.

Structured outputTool callingStreamingContext & caching| Notebook | What you'll learn | |---|---| |

beforebuilding anything you'd need to tune. Evals is the spine; it returns in every section after this| Notebook | What you'll learn | |---|---| |

Embeddings & retrievalHybrid & rerankingChunkingWhy RAG fails| Notebook | What you'll learn | |---|---| |

LLM as judgeRegression evals| Notebook | What you'll learn | |---|---| |

Tool designGuardrails & budgetsMCP & the tool ecosystemSkills & progressive disclosureSKILL.md

pattern, the context-budget payoff, and Tools/MCP/Skills as one storyHarness engineeringaroundthe call — context assembly & compaction, tool-result shaping, and verification loops. Names the discipline the section has been teaching piece by piece| Notebook | What you'll learn | |---|---| |

Reliability & fallbacksExperiment tracking & registryWhere the free Groq API can't run the topic (these frameworks need a GPU), the notebook teaches it concept-first and fences an optional Colab-GPU appendix — the same pattern as the section-06 LoRA appendix.

[Inference performance](/calmrocks/ai-engineer-notebooks/blob/main/09-serving-inference/02-inference-performance.ipynb)| Notebook | What you'll learn |
|---|---|

|

Where the skills come together into projects. First a case study — one realistic scenario worked end to end, runnable — then the capstone, the

deployed repo you build yourself. (Section overview.)

debugged in production: a vague ask becomes a deployed, evaluated RAG+agent assistant, then a live quality regression (a stale index after a corpus migration) that you diagnose and fix. A build-to-debug arc threading sections 02–11Case study B — Contract extraction: pipeline vs agentsameextraction task as both an agent and a pipeline, then prove with accuracy + token cost that the pipeline wins when the steps are knownCase study C — Red-team robustness benchmarkkindof system — a harness that evaluates a model instead of serving one: an attacker→target→judge (PAIR) loop that measures attack success rate, composing the agent loop, LLM-judge, security, and evalsCapstone: the brief for the deployed project that goes on your resume — a real repo with a serving component and an eval report. Case studies are for learning; the capstone is for hiring.

Raw model APIs, no frameworks. Patterns are durable; wrappers churn.One shared corpus(data/

) across RAG and eval sections, so evals measure the retrieval you actually built.Self-contained notebooks. First cell installs, second cell callsfrom aien import setup; client, MODEL = setup()

to load your key from Colab secrets (or a local env var). No hidden state between notebooks.aien

is the tiny shared-setup package in this repo — one place to change credential — installed automatically by the first cell.Every notebook ends with exercises— do them before moving on.

• Get a free API key at console.groq.com— no credit card required. - In Colab: the key icon in the left sidebar → add GROQ_API_KEY

as a secret, and toggle notebook access on. - Open any notebook via its badge and run top to bottom.

Running locally instead: pip install -r requirements.txt && pip install -e .

(the second installs the aien

setup helper), export GROQ_API_KEY=...

, open with Jupyter.

Plan: Transitioning to FDE / AI Engineer— the roadmap these notebooks implementGuide: Building a Real LLM Project for Your Resume— the capstone's requirements barWalkthrough: Designing a RAG System— the systems view of section 03Walkthrough: Designing an AI Agent Orchestration System— the systems view of section 05