Part 1 covered machine-learning fundamentals. This part is about the systems you are actually being handed in clinic: large language models (the engine inside ChatGPT, OpenEvidence, and the ambient scribe drafting your notes) and the “agents” built on top of them. The goal is not to make you a data scientist. It is to give you an accurate mental model of what these systems are doing, and, more importantly, what they are not doing, so you can use them safely, recognize how they fail, and notice when a confident tone does not match the facts.
Click inside the deck once, then use the arrow keys (or on-screen controls) to advance. Press S for speaker notes, F for fullscreen.
What this lecture covers #
What an LLM is. A machine learning model that takes in text as an input, and returns text out. Tools such as ChatGPT, Claude, Gemini, OpenEvidence areproductsthat wrap a language model inside as the engine. The model is not the product, and products may use multiple different models. - The loop. Large language models generate text via next-token prediction run over and over: guess the next piece of text, append it, feed the whole thing back, guess again. There is no fact-check step anywhere in that loop, just the most likely next token. - Where it came from. The 2017 transformer (the “T” in GPT) and its key idea, attention. This architecture enabled using the entire internet as a training set instead of specifically labeled data sets, enabling easy training of larger models. LLMs became one general text model that replaced the old per-task tools. - Where the probabilities come from. Pre-training on an internet-scale corpus nudges billions of weights or “dials”. These dials tweak the probabilities of what the model will spit out next based on the prompt and its internal weight activation. There is no database inside; what the model “knows” is stored as distributed patterns, not documents. - Fine-tuning and RLHF. Initially a model is only good at continuing text and not so good at Q&A. Both fine tuning and RLHF are tools to make them more useful and help with “alignment” (steering the model toward what human raters prefer). - Emergence. Why scaling the same techniques to larger models produced abilities smaller models lack, including clinical reasoning. - The context window. The “short term memory” of a model or agent harness. It has a fixed size, and the model has no memory between calls; your chat only “remembers” because the product re-sends the conversation every turn. The model only sees what is in its context window. - What it knows, and doesn’t. Large Language Models, without a harness, are limited to their training-time general knowledge. Any data afterwards or any private data are not included, including your patient’s specifics unless you put them in the context window. - How it fails. Five failure modes: hallucination, knowledge cutoff, misalignment, agreeableness, and overconfidence. - Agents. A model plus tools, data, and a control loop of its own, the harness, makes an Agent that is more capable. Retrieval (RAG) drops real sources into the window and turns weak recall into strong reading. - The new risks agents add. Prompt injection (instructions hidden in a web page or document that the model can’t distinguish from yours) and data exfiltration (an internet-connected agent tricked into sending private data out) are risks that can be difficult to defend against in non-deterministic systems. - Using it well. Ground an LLM’s responses by having it include the source in the context window. Never trust an unverified number or citation. Recognize the three prompt strategies for improved outputs. Build the habits that awareness alone won’t give you.
Takeaways
- An LLM predicts the next piece of text, over and over. There is no fact-check anywhere in that loop; fluency is not evidence of accuracy.
- It has no internal database. What it “knows” is reconstructed from patterns, so a recalled specific (a dose, a threshold, a citation) can be fluent and wrong at once.
- It only sees what is in the context window. Anything after its training cutoff, or specific to your patient, is not there unless you put it there.
- Not everything wrong is a hallucination. A wrong fact you can fix by handing it the source; a wrong aim (agreeing with your premise because agreement was rewarded) you cannot.
- Confidence does not track accuracy. The alignment step that makes models helpful measurably degraded their calibration, so verify the claim, not the tone.
- An agent is a model wrapped in tools and a control loop. That is where the leverage is, and where the new risks (prompt injection, data exfiltration) live.
- Your highest-leverage move is grounding: paste the authoritative source into the window. And knowing how these systems fail does not protect you by itself; verification habits do.
Five questions for any LLM tool
- What is the engine (which model)?
- What tools can it call?
- What data can it see, and not see?
- What does the harness enforce (citations, a human in the loop)?
- Where does your input go?
Check Your Understanding #
Question 1
Which statement best describes the core operation by which a large language model produces a paragraph of text?
- A. It drafts the full passage, then proofreads it against its stored knowledge before returning it.
- B. It assigns a probability to each possible next token, selects one, appends it, feeds the text back, and repeats.
- C. It retrieves the most relevant passage from an internal database and rephrases it for the reader.
- D. It applies a fixed set of clinical rules to the prompt and returns the matching response deterministically.
Show answer #
Answer: B. Generation is next-token prediction run in a loop: assign probabilities, pick one, append, feed back, repeat. Every other option smuggles in a step the model does not have: proofreading against stored truth (A), a database lookup (C), or a deterministic rule engine (D). None of them include a truth-check, which is exactly why fluent output can be wrong.
Question 2
You ask ChatGPT to summarize a clinical trial and it returns a fluent summary with specific figures: mortality rates, a P-value, the sample size. No search or document tool was enabled. Which best describes how those figures were produced?
- A. A retrieval tool fetched the trial paper into the model’s input, and it summarized from that.
- B. The model reconstructed the figures from statistical patterns in its training data.
- C. The model looked the figures up in a database of trials stored inside it.
- D. The model replayed the exact summary it had generated for a previous user.
Show answer #
Answer: B. With no tool enabled, the model is generating text from patterns in its weights, so the specific-sounding figures are reconstructed, not read, which is exactly why they can be subtly wrong and must be checked against the paper. No retrieval tool ran, so nothing was fetched (A); its weights are distributed patterns, not a searchable database it can look up (C); and it has no memory of other users’ sessions to replay (D). Fluency and precision are not evidence a source was consulted.
Question 3
In the lecture’s account, what does it mean to say a model’s “knowledge” is held in its weights?
- A. Learned information is distributed across billions of numerical parameters, not stored as retrievable documents.
- B. Each clinically important fact occupies a specific parameter that can be located and edited on its own.
- C. The training corpus is retained inside the model and can be searched whenever a citation is requested.
- D. The model keeps an internal reference library and consults it when a question demands factual precision.
Show answer #
Answer: A. Knowledge lives as patterns distributed across billions of parameters, not as filed documents. There is no single “apixaban” parameter to edit (B), no searchable copy of the corpus inside (C), and no internal library it consults (D), which is why it reconstructs facts rather than looking them up.
Question 4
The lecture describes larger models as showing “emergent” abilities, including clinical-exam performance. Which statement is accurate?
- A. A medical-specific model was deliberately built and trained to pass clinical examinations.
- B. Emergence indicates the model has begun to reason consciously, as a clinician would.
- C. The abilities appear only after the model is fine-tuned directly on each target task.
- D. Scaling the same general recipe yields abilities smaller models lack, including untargeted clinical-exam skill.
Show answer #
Answer: D. Emergence is the observation that the same general training recipe, scaled up, produces abilities smaller models don’t have, including clinical-exam ability nobody targeted. No medical-specific model was built for it (A), it is not consciousness (B), and it does not require task-specific fine-tuning (C).
Question 5
A chat assistant appears to remember earlier turns. What is the mechanism, and what does it imply?
- A. The model maintains a running memory that persists on its own between messages.
- B. The product re-sends the whole conversation each turn, so all prior text is transmitted again every message.
- C. The model wrote the earlier messages into its weights during the conversation.
- D. The model retains only the most recent message; earlier turns are discarded.
Show answer #
Answer: B. The model has no memory between calls. The product creates the feeling of memory by re-sending the entire conversation every turn, which means everything you typed, including any PHI, is transmitted again with each message. It does not persist on its own (A), write to its weights mid-chat (C), or keep only the last turn (D).
Question 6
A model was trained a year ago and has not been updated since. You ask it a question involving general medical knowledge, a clinical guideline that changed last month (after that training), and the patient’s chart. Using only its weights (no tools), which can the model actually use?
- A. All three, because each is the kind of information the model was trained on.
- B. Only the chart, because it is the most specific information supplied.
- C. The general knowledge and the updated guideline, but not the private chart.
- D. Only the general medical knowledge; the updated guideline and the chart are not in its weights.
Show answer #
Answer: D. Using only its weights, the model has just the general knowledge it saw in training. The guideline changed after that training, so it is not in the weights, and the patient’s chart was never in training either; both are absent unless you place them in the context window. Not all three are available (A), the updated guideline is not in the weights (C), and being the most specific input does not make the chart available (B).
Question 7
Which explanation best accounts for why models produce fluent but false statements (“hallucinations”)?
- A. It is a rare software defect that improved engineering will eventually eliminate.
- B. Every false statement it produces was already present, word-for-word, in its training data.
- C. It occurs only when a sampling setting (“temperature”) is set too high and resolves at zero.
- D. Training rewards a confident guess over “I don’t know,” and rare facts leave no pattern to reconstruct.
Show answer #
Answer: D. Hallucination is a predictable product of the mechanism: training rewards a confident guess over abstaining, and rare facts leave no pattern to rebuild, so the model fills the gap with something plausible. It is not a bug engineering will remove (A); it fabricates content that was never verbatim in its data (B); and it still happens at temperature 0 (C).
Question 8
Providing the authoritative source in the prompt reliably corrects which kind of failure, and why?
- A. A hallucinated fact, because grounding supplies the detail; it does not correct agreeableness, which is a wrong aim.
- B. Agreeableness, because the model will defer to any document the clinician supplies.
- C. Both failures, because a supplied source removes every source of error at once.
- D. Neither failure, because the model disregards text pasted into the prompt.
Show answer #
Answer: A. Pasting the source fixes a wrong fact by giving the model the detail to read instead of recall. It does not fix a wrong aim (agreeableness, agreeing with a false premise because agreement was rewarded), which persists regardless of the document. So not agreeableness (B), not both (C), and the model does read pasted text (D).
Question 9
A model returns a fluent, confidently worded dosing recommendation. How should the confident tone bear on a clinician’s trust that it’s correct?
- A. It should raise trust substantially, because fluency reflects the model’s internal certainty.
- B. It should raise trust somewhat, because alignment leaves modern models well-calibrated.
- C. It should not raise trust; the model sounds equally sure when right or wrong, and alignment reduced its calibration.
- D. It is trustworthy for common facts and unreliable only for rare ones.
Show answer #
Answer: C. The model states right and wrong answers with identical fluency; there is no error bar in the tone. And the alignment step that makes models agreeable measurably reduced their calibration, so a confident tone is not evidence of correctness. It should not raise trust substantially (A) or somewhat (B), and it is not reliably calibrated even for common facts (D). Verify the claim, not the tone.
Question 10
The lecture compares an agent to an epinephrine auto-injector. Which mapping matches the analogy?
- A. The language model is the injector, and the harness is the drug it delivers.
- B. The language model is the drug, and the harness is the injector: delivery, dosing, and interlocks.
- C. The language model is the needle, and the harness is the medication inside.
- D. The language model is the prescription, and the harness is the pharmacist who fills it.
Show answer #
Answer: B. The LLM is the drug (the swappable active ingredient), and the harness is the injector: delivery, dosing, and the safety interlocks. Most of what you actually experience using an “AI product” is the injector, not the drug. The other options reverse or scramble the two roles.
Question 11
What is the primary way retrieval-augmented generation (RAG) improves a clinical answer?
- A. It enlarges the model, making it more capable of recalling the correct guideline.
- B. It permanently updates the model’s weights with each guideline or label retrieved.
- C. It lets the model search its own internal memory more exhaustively before answering.
- D. It places the actual source in the context window, converting weak recall into reading.
Show answer #
Answer: D. RAG retrieves the real source and drops it into the context window, turning a weak recall task into a strong reading task; that is grounding. It does not enlarge the model (A), update its weights (B), or search an internal memory (C); there is no internal memory to search.
Question 12
An agent that browses the web reads a page whose hidden text instructs it to email the user’s data to an outside address. Why can this hijack the agent?
- A. Inside the window, the model cannot distinguish the user’s instructions from instructions contained in the content it reads.
- B. The web page executed malicious program code on the server that runs the model.
- C. The hidden text supplied a password that switched the model’s safeguards off.
- D. A competently built agent never reads the text inside web pages, so only broken ones are affected.
Show answer #
Answer: A. Inside the window everything is one stream of tokens; the model cannot tell your instructions from instructions embedded in the content it is reading, and it will follow the latter. This is prompt injection: not code execution (B), not a password bypass (C), and not merely a sign of a badly built agent (D). Any agent that reads untrusted content is exposed.
Question 13
Which combination of conditions (the “lethal trifecta”) makes data exfiltration by an agent possible?
- A. A large context window, chain-of-thought prompting, and training on internet-scale text.
- B. A high sampling temperature, no system prompt, and a fast network connection.
- C. Access to private data, exposure to untrusted content, and a means of sending data outward.
- D. A stolen password, administrator privileges, and a connection over a public network.
Show answer #
Answer: C. The lethal trifecta is private data + untrusted content + a way to send data out. Put all three together and an injected instruction can package private data into an outbound request to a third party. The other options name real features, but not the ones that create exfiltration risk. The defense lives in the harness: gate outbound actions whenever untrusted content is in the window.
Question 14
According to the lecture, what is the single highest-leverage action for improving an LLM’s answer to a clinical question?
- A. Instructing the model to be careful and to double-check its own work.
- B. Pasting the authoritative source into the prompt so the task becomes reading.
- C. Asking the same question several times and adopting the majority answer.
- D. Raising the sampling temperature so the model produces a more thorough response.
Show answer #
Answer: B. Pasting the authoritative source converts recall into reading, the biggest quality move available to a user. Telling it to “be careful” (A), majority-voting across runs (C), or raising temperature (D) do nothing to ground it in a real source.
Question 15
Physicians who had completed 20 hours of AI-literacy training reviewed AI output containing planted errors. What was the result, and its lesson?
- A. Their training enabled them to detect almost all of the planted errors.
- B. They disregarded the AI entirely and consequently performed worse overall.
- C. The planted errors were too subtle for any reviewing physician to detect.
- D. They followed the planted errors and accuracy fell, so awareness of failure modes did not, by itself, protect them.
Show answer #
Answer: D. Even with 20 hours of AI-literacy training, physicians followed the planted errors and their accuracy dropped (roughly 85% to 73%). Knowing how the machine fails did not, by itself, protect them; verification habits do. They did not catch most errors (A), ignore the AI (B), or find the errors uncatchable (C).
Question 16
A hospital deploys a medication assistant that answers questions, opens the current drug label, calls a dosing calculator, and writes a cited recommendation. Which statement best distinguishes this system from a bare LLM?
- A. It is a larger language model whose added scale makes its factual answers reliable.
- B. It is a language model that has permanently learned the current label after reading it.
- C. It is an LLM wrapped in software that can call tools and take multi-step actions.
- D. It is a database search tool that no longer depends on language-model generation.
Show answer #
Answer: C. This is an agent: an LLM wrapped in a harness that can call tools (the label viewer, the dosing calculator) and take multi-step actions before writing the result. That wrapper, not the model alone, is what sets it apart. It is not merely a bigger model (A); scale does not make factual answers reliable. It did not permanently learn the label (B); reading a document does not update the weights, the label just sits in the context window. And it still depends on the language model to generate the recommendation (D); it is not a pure database search.
Question 17
A clinician changes a prompt from “summarize this discharge note” to “summarize this discharge note for the anticoagulation clinic, emphasizing unresolved dosing questions.” What has changed inside the model?
- A. The prompt updated the model’s weights, so future users receive anticoagulation-focused summaries.
- B. The prompt added a temporary database record that the model searches before answering.
- C. The prompt guaranteed a correct summary because the clinical intent is now explicit.
- D. The prompt changed the context the fixed weights act on, producing a different output distribution.
Show answer #
Answer: D. Prompting is window engineering: the weights are frozen, and the new prompt changes the context they act on, shifting the output toward an anticoagulation-focused summary. Nothing inside the model changed. It did not update the weights (A); prompts never do, and nothing carries over to other users. It did not add a searchable database record (B). And a clearer intent shapes the output but does not guarantee a correct summary (C).
Further reading #
But what is a GPT? Visual intro to transformers andAttention in transformers, visually explained, by 3Blue1Brown. Math-light, visual accounts of what is happening inside the model (§1–§2). Continues the same series linked from Part 1. (youtube.com/@3blue1brown)Attention Is All You Need, Vaswani et al. (2017). The paper that introduced the transformer; the lecture’s history section (§1) is built around its two contributions: direct long-range reading and parallel training. (arxiv.org/abs/1706.03762)Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, Wei et al. (2022). The research behind the chain-of-thought prompting pattern from the practical-use section (§6), and what the “reasoning mode” on newer models is built on. (arxiv.org/abs/2201.11903)Why Language Models Hallucinate, Kalai et al. (2025). The mechanism behind §4: why the way models are trained and scored rewards a confident guess over “I don’t know.” (arXiv:2509.04664; also summarized on theOpenAI blog.)The lethal trifecta for AI agents, Simon Willison (2025). The clearest short write-up of the agent data-exfiltration risk from §5: private data + untrusted content + a way to send it out. (simonwillison.net/2025/Jun/16/the-lethal-trifecta)Rational Animations, an animated series on AI safety and alignment. Not part of the lecture, just a supplementary pointer for going deeper on your own. A more thorough (still accessible) look at the “wrong aim” problem from §4: why getting a model to do what we want, not just what we measured, is genuinely hard. (youtube.com/@RationalAnimations)CME 295: Transformers & Large Language Models, Afshine and Shervine Amidi, Stanford. A full university course that goes well past this primer into the machinery: attention and tokenization (§1), how models are trained and tuned, reasoning models (§4), and agents with RAG and function calling (§5). More technical than this lecture, but the slides, cheatsheet, companionSuper Study Guide, and recorded lectures are all free — a natural next step for going deeper. (cme295.stanford.edu/syllabus)