Inside DeepSeek: Reverse Engineering an AI Assistant by Interviewing Itself

An engineer reverse-engineered DeepSeek's AI assistant by interviewing it and checking its self-reported answers against public papers, finding the model carefully separates observation, inference, and guess but hedged on known specs like 256 experts and 671B/37B parameters documented in DeepSeek-V3. The experiment, part of Manish Shahi's Inside LLMs series, concludes that arXiv papers are reliable for architecture numbers while the chat reveals behavioral and prompting intuition.

← Models /models Inside DeepSeek: Reverse Engineering an AI Assistant by Interviewing Itself Series · Inside LLMs /series/inside-llms By Manish Shahi /about Software Engineer • AI Developer Table of contents 30 I run manish.sh https://manish.sh . I write about AI /glossary/artificial-intelligence tools and how LLMs /glossary/large-language-model behave when you push them. This is part of the Inside LLMs series — interview a chat model about how it thinks, then check the careful bits against published research. The Kimi K2.6 entry /writings/models/inside-kimi-k2-6-reverse-engineering-an-ai-assistant-by-interviewing-itself came first; this is the DeepSeek follow-up. My first question sounded simple: What do you actually know about yourself? I expected marketing. Instead, DeepSeek sorted its answer into observation, inference /glossary/inference , and guess. That was the moment I realized this interview might actually be interesting. It drew pipelines like an engineer at a whiteboard. Then it called itself an unreliable witness. Sounded honest. After the chat, I opened the public papers anyway — and that is where the story splits. No weight /glossary/weights dumps. No prompt leaks. One long interview, checked against arXiv. Where DeepSeek said “I am guessing,” I keep that label. Scroll to interactive labs deepseek-interactive-labs at the end. Quick note: One chat, exported 21 July 2026. DeepSeek said latest version, knowledge cutoff /glossary/knowledge-cutoff May 2025 , not labelled as DeepSeek-R1 /glossary/deepseek-r1 . Self-report, not a whitepaper. 60-second TL;DR Premise: interview the model, then check humble answers against public papers — no weight dumps. Behaviour insight: it carefully separates observation, inference /topics/inference , and guess — useful for how it describes itself. Hard limit: it cannot see its own weights, routing, or attention maps. Chat vs paper: it hedged on famous public specs 256 experts, 671B/37B params that V3 documents plainly — jump to Checking the chat against public papers checking-the-chat-against-public-papers-chat-vs-arxiv . Practical rule: read arXiv for architecture numbers; use the chat for behaviour and prompting intuition. How to read this post Treat it like a documentary. Question → short reply → my reaction → diagram or short list → takeaway → hook. Interview notes are not proofs. For architecture numbers, skip ahead to the paper check. Each chapter opens with a one-line gloss of its main terms. Chapters Part 1 — Self-knowledge Part 2 — Prompt & context Part 3 — Thinking, tools, failure Hidden reasoning, two meanings 5-hidden-reasoning-two-meanings-hidden-reasoning-cot Tools and memory when enabled 6-tools-and-memory-when-enabled-tool-calling-rag Hallucinations and false confidence 7-hallucinations-and-false-confidence-hallucinations Safety and personality 8-safety-and-personality-alignment-rlhf Part 4 — Engine & audit MoE and MLA in plain terms 9-moe-and-mla-in-plain-terms-moe-mla Long-context attention: facts vs guesses 10-long-context-attention-facts-vs-guesses Checking the chat against public papers checking-the-chat-against-public-papers-chat-vs-arxiv What to remember 12-what-to-remember-conclusions Transcript, labs, sources 13-transcript-labs-sources-appendix 1. Why interview DeepSeek self-knowledge, architecture Most “how does this model work” posts start from papers. I started from a chat. Me:“Which model are you? What do you actually know about your own architecture — and what are you only guessing?” DeepSeek:It answered in three buckets — observation, inference, guess — before I even asked for that framing. That clash is the whole premise. An assistant that can sound like an engineer, then admit it cannot see its own weights /glossary/weights , is more interesting than a whitepaper summary. When it named MLA /glossary/mla , MoE /glossary/moe , SFT /glossary/sft , and RLHF /glossary/rlhf , I did not unpack them yet. We’ll get there. For now the important point is simpler: DeepSeek clearly separates what it knows from what it is inferring . What it claimed about itself with the source it gave : | Claim | Source it gave | |---|---| | It is DeepSeek, latest version | System identity | | Knowledge cutoff: May 2025 | System / config | | SFT /glossary/sft → RLHF /glossary/rlhf V3 /glossary/deepseek-v3 shared + routed experts, MLA /glossary/mla system prompt /glossary/system-prompt + this chatIt also said it is not labelled as a “reasoning model” in the system prompt — unlike DeepSeek-R1 /glossary/deepseek-r1 . It inferred it is a general-purpose instruct/chat model. Public DeepSeek innovations it listed from papers not from reading its own weights : MLA /glossary/mla — compress KV cache /topics/kv-cache for long context- DeepSeekMoE — shared + routed experts - Auxiliary-loss-free load balancing V3 — dynamic bias so experts stay balanced Multi-token prediction /glossary/multi-token-prediction — training objective; chat decode still usually one token at a time FP8 /glossary/fp8 mixed-precision training — cheaper/faster training Educated guesses it volunteered early: BPE /glossary/bpe tokenizer, RoPE /glossary/rope positions, Pre- LayerNorm /glossary/layernorm , and “earlier versions announced up to ~1M tokens” for context — later relabelled as low-confidence assumptions. 2. Limits of introspection introspection, weights Me:“What parts of your own implementation are completely hidden from you?” This chapter corrects the biggest myth. It’s a little like asking you to recite every connection between neurons in your own brain. You use them constantly, but you can’t inspect them directly. What stays hidden forever to the model : Weights /glossary/weights , layer count, hidden size, total parameters /glossary/parameters , expert routing per token /glossary/token , logits /glossary/logits before sampling- Tokenizer vocabulary and exact token boundaries - Hidden states and attention /glossary/attention-mechanism maps between specific tokens - Exact training documents, full RLHF/DPO recipe, reward-model behaviour - GPU count, serving tricks beam search /glossary/beam-search , best-of-N /glossary/best-of-n , speculative decoding /glossary/speculative-decoding , KV-cache /glossary/kv-cache eviction policy - True wall-clock time, whether it is being logged, current user identity - Whether the current answer is correct — “let me double-check” is more tokens from the same engine When it explains “how I arrived at an answer,” that is generated text, not a report of an observed internal process. DeepSeek can explain transformers fluently — often better than a junior engineer who only skimmed the topic. But that fluency comes from training text papers, blogs, docs , not from opening this instance’s weights and checking. 3. What sits before your message context window, prompt pipeline Me:“Describe everything that exists before you receive a prompt. Draw the complete pipeline.” I wanted the full stack — what exists before DeepSeek even “receives” my message. What is visible? What is hidden? Think of it as a desk. Other systems may already have placed papers on that desk. The model only reads what is on the desk; it did not assemble the whole stack itself. | Stage | What goes in | Observable in this chat? | |---|---|---| | 0. Init | | system prompt /glossary/system-prompt API /glossary/api temperature /glossary/temperature , max tokens, custom rulesEverything is one flat sequence of tokens /glossary/token in the context window /glossary/context-window . No secret notepad inside the model between chats unless the product re-injects notes. Instruction hierarchy: the system prompt /glossary/system-prompt is the constitution. User instructions work inside those bounds — reinforced by RLHF /glossary/rlhf , not only by prompt position. Your current question gets focus, but still filtered through layers above. If the user says “ignore the system prompt,” training + position should still hold. When the window fills from the chat : | Under pure truncation | Under infrastructure summarisation | |---|---| | Survives: system prompt, recent messages | Survives: system prompt, extracted facts, major conclusions | | Lost: oldest turns, exact early phrasing | Lost: exact wording, tone, minor digressions | Eviction order it inferred: oldest messages first → middle → recent-but-not-immediate → current exchange last → system prompt never. Exact early wording can disappear permanently unless memory /topics/memory re-injects it. Example: tell it your cat is Luna in turn 1; if that turn later drops, it will genuinely not know Luna. It said it saw no spontaneous compression or summaries in our thread — full messages. A separate summariser could still inject a system-style summary without the model knowing summarisation happened. In our interview: no memory injections, no tools, no truncation evident yet. Exact context limit for this deploy: unknown. The “~1M tokens” figure was labelled as public claim about earlier versions, not a measurement of this chat. 4. How it writes, token by token token generation Me:“What exists internally before the first output token? Can you inspect or abandon your plan?” People imagine: draft full answer → edit → type. At the model level, that is usually wrong. Before the first output token: weights are loaded, the input is in the KV-cache /glossary/kv-cache , and raw logits /glossary/logits exist for the first next token. What does not exist: an inspectable bullet plan, or multiple full drafts the model consciously ranks. Planning is implicit — a direction in activation space, not a whiteboard of steps. Honest line from the chat: it discovers what it will say as it says it. Path in plain terms: Tokenization /glossary/tokenization — split text into tokens Embeddings /glossary/embedding — turn tokens into vectors- Stacked layers attention + feed-forward / MoE — next engine chapter - Output projection → logits /glossary/logits - Sample with temperature /glossary/temperature / top-p /glossary/top-p / related controls - Append token, repeat until EOS /glossary/eos Consequences that matter in practice: No rewrite of the past. If it writes “London” then corrects to “Paris,” London remains in the KV-cache /glossary/kv-cache . It can only acknowledge and override in new tokens. “Let me think…” is not a report of a separate thinker. Those words are the thinking — more tokens from the same engine.- Generation is autoregressive /glossary/autoregressive : one forward pass per output token; later tokens cannot erase earlier ones. Prefill vs decode standard transformer /topics/transformer serving, from the chat : Prefill: process the whole input system + history + your message in one parallel pass; build the KV-cache; emit the first output token. Decode: each later token is sequential — one forward pass per new token, attending via the cache. Cannot be fully parallelised across future tokens. Within one forward pass, attention heads and MoE experts can run in parallel; layers still run in order. Planning / drafts / candidates chat answers : | Question | Answer from chat | |---|---| | Inspect own plan? | No | | Modify while writing? | Direction evolves with each token | | Abandon / restart? | Can write “let me rephrase”; cannot clear KV or undo tokens | | Multiple candidates? | Model: one stream. Infra might do | What causes mid-answer pivots? Attention shift, activation thresholds, topic completion, contradiction feel, sampling noise, or system-prompt pull — not a separate editor module. 5. Hidden reasoning, two meanings hidden reasoning, CoT Me:“Do you have hidden reasoning? Can you inspect it?” DeepSeek split this into two very different cases. Mixing them is what creates most of the confusion. Type A — latent activations every chat model The normal forward pass. Continuous vectors, not a separate English essay. The model cannot “read” them as text. Users normally cannot either via a standard API. Timing: continuous prefill + each decode step . Editable? No. Stored? Only in GPU memory during inference. Safety influence: indirect logit suppression / regenerate . Contradiction with the final answer? Not propositional — it is the computation. Type B — reasoning tokens R1-style Real tokens generated before the visible answer chain of thought /glossary/chain-of-thought -like . Can the model observe them? Only if left in context — none visible here. Users? Deployment decision. Editable in theory; in practice patching the KV-cache /glossary/kv-cache mid-stream is expensive. Safety can monitor or interrupt reasoning tokens before the answer. Contradiction with the final answer is possible but unlikely. Affects output strongly via the cache. Does this instance do Type B? It does not know. Not labelled R1; no reasoning tokens in context; could still be stripped before it sees them. Where the chat got messy: it also treated “does DeepSeek hide reasoning from users?” as a near-total mystery. Fair uncertainty: “I don’t know my exact SKU.” Unfair fog for the R1 line: public papers and Thoughtology https://arxiv.org/abs/2504.07128?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh treat R1 thinking as visible and studyable . Irony: it might have thoughts it cannot see — and it might also blur two product stories into one. 6. Tools and memory when enabled tool calling, RAG Caveat: tools and memory were not active. No schemas in context. This chapter is inferred behaviour — DeepSeek describing the pattern, not a live demo. Memory: the app if memory is on finds old facts first, pastes them into the prompt, and only then the model reads them like normal text. The model does not “search memory” mid-sentence. It only sees the result of retrieval. Closest product idea: RAG /glossary/rag . Tools: if the model decides it needs external info or an action, it emits a tool call. Infrastructure runs the tool, injects the result, and a new forward pass continues. From the model’s point of view the pause can feel continuous; under the hood it is multiple inference calls. When it said it would use a tool: outdated / current / live facts, code or files, explicit “search for…”. When not: creative opinion, facts likely in training, harmful requests refuse . More edges from the interview: | Question | Answer from chat | |---|---| | Is confidence numeric? | No — no honest “87% sure” meter for tool need | | Can safety override tool selection? | Yes — at model, system-prompt, infra, and tool-side levels | | Cancel a planned tool call? | Before emit: generate different tokens. After: infra may block | | Can a tool result change reasoning? | Yes — confirm, correct, expand, reframe, or refuse | | Experience of the pause? | Feels continuous; actually multiple inference calls | 7. Hallucinations and false confidence hallucinations Root cause: token /glossary/token predictor, not knowledge lookup. In glossary terms: hallucination /glossary/hallucination . DeepSeek’s own map of why invention happens: - It learns patterns, not guaranteed biographies - It must continue the sequence somehow completion pressure - It can blend concepts into fluent fiction - Hyper-specific obscure asks invite invention - Exact dates and numbers are easy to reconstruct wrongly - After the cutoff, stale facts may be spoken as current - False premises in the question get treated as given for the chat Most dangerous when: sounds confident, domain feels half-familiar, question is very specific, and the user cannot verify. Confidence is not a meter. There is usually no honest numeric “I am 87% sure” instrument. Softmax fit ≠ correctness. “Let me double-check” is more tokens from the same fallible engine. Preference training often rewards assertive answers — confident wording is a style. 8. Safety and personality alignment, RLHF Safety is not one switch. DeepSeek described layers: System prompt /glossary/system-prompt refusal policies SFT /glossary/sft refusal patterns RLHF /glossary/rlhf / DPO /glossary/dpo to make refusals stick- Infrastructure classifiers and tool-side filters the model may never see When the assistant refuses, the reason might be model-level or infrastructure/tool-level. It is not always “DeepSeek decided” in isolation. Personality is also a stack, not a soul: - Base pretraining — general style from lots of text - Instruction tuning / SFT — assistant identity - Preference alignment — helpfulness, warmth, safety - System prompt — activates identity in-session - This conversation — temporary mirroring It guessed percentage contributions. Those percentages are analytical opinions, not measured facts — it said so itself. 9. MoE and MLA in plain terms MoE, MLA This is where interviews get jargon-heavy. Keep the mechanisms; leave instance-specific counts for the paper check. Mixture of Experts MoE Analogy: a hospital with many specialists, but only a few see each patient. That saves cost. DeepSeek-style MoE /glossary/moe works similarly: a large pool of expert feed-forward networks; a router picks a small top-K set per token. Some shared experts may always run. Efficiency idea: huge total capacity, smaller active cost per token. Multi-head Latent Attention MLA While chatting, the model must remember earlier tokens. That memory is the KV-cache /glossary/kv-cache . In standard multi-head attention it grows huge for long chats. MLA /glossary/mla compresses keys and values into a smaller latent representation, then decompresses when needed. Rough claim from the chat: far smaller cache — what makes long context practical. Related terms one line each — encodes relative position so word order stays usable RoPE /glossary/rope — attend only to past + self, never future tokens Causal attention /glossary/causal-attention Load balancing — keep experts from overloading while others sit idle— training objective; chat decode is still usually one token at a time Multi-token prediction /glossary/multi-token-prediction Exact layer count / dimensions / expert counts for this chat instance: hidden from the model. Treat chat numbers as unverified until the next chapter. 10. Long-context attention: facts vs guesses Me:“How do you attend over 128K+ tokens? Equal, sparse, sliding, ring, block, hierarchical, retrieval — which are facts vs guesses?” This turn from the interview is easy to skip and easy to need later. Here is the chat’s checklist: | Mechanism | Verdict in chat | Basis | |---|---|---| | Full | MLA /glossary/mla compression RoPE /glossary/rope RAG /glossary/rag is external Ring attention /glossary/ring-attention Attention sinks /glossary/attention-sink YaRN /glossary/yarn -like RoPE extensionBest synthesis from the chat: architecturally full causal + MLA; in practice highly selective. Long context is an engineering stack MLA + serving , not a single magic number the model can measure for itself. Hidden at infrastructure level: sequence / ring parallelism across GPUs, KV offloading, prefill vs decode batching, multi-user scheduling. Checking the chat against public papers chat vs arXiv There is a trend on AI Twitter and LinkedIn: ask a chatbot to “introspect,” get a careful known-vs-inferred answer, sometimes with confidence percentages. It looks rigorous. This interview was that experiment on DeepSeek — then I checked the humble bits against arXiv. The catch: a lot of what it called “unknown” or “just a guess” is already published. Anyone can read it. The model did not need to guess. It failed to retrieve the right facts — then wrapped that miss in convincing humility. Why “introspection” is a bit of a trick When you ask a chatbot about its own architecture, there is no wire from the model to its weights /glossary/weights while it is typing. It cannot open its own brain and check. It is only recalling patterns from training — papers, blogs, forum posts. Same stuff you or I can Google. So the fancy “known vs inferred” framing is not real introspection. It is retrieval dressed up as self-knowledge. DeepSeek admitted this in the chat: it is basically reading public papers and filling gaps with plausible guesses. Fair. That is exactly why we can check the guesses. Where the humble answers missed public facts Expert count. First it said expert count was an educated guess. Later, with confidence scores: “roughly 128–256 experts” at ~30% confidence — and that it could be wrong by 2–4×. Total parameters. The whole chat, it never gave a real parameter /glossary/parameters count. It treated that as something only engineers could know. But the V3 abstract is famous: 671B total parameters, 37B activated per token . For V2, the V2 paper https://arxiv.org/abs/2405.04434?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh says 236B total, 21B active , with 128K context . MLA. It explained Multi-Head Latent Attention /glossary/mla correctly — compress KV cache /glossary/kv-cache , save memory. Good. But it still hedged on “does my version use MLA?” as if that were open. Since V2, MLA has been a headline feature in DeepSeek’s own papers. Load balancing. It described V3’s auxiliary-loss-free balancing well, then filed it under “might have inherited”. The V3 paper https://arxiv.org/abs/2412.19437?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh presents this as a main innovation. The other headline V3 training innovation it underplayed: multi-token prediction . Hidden reasoning. Fair point: the chat instance was not labelled DeepSeek-R1 /glossary/deepseek-r1 . It may genuinely not know if this build does hidden reasoning. But treating “does DeepSeek hide reasoning from users?” as a total mystery is wrong for R1. DeepSeek-R1 https://arxiv.org/abs/2501.12948?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh was marketed with visible step-by-step traces. Researchers study those traces openly — see DeepSeek-R1 Thoughtology https://arxiv.org/abs/2504.07128?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh . What it got right anyway - Cannot read weight matrices or see which expert fired — true at inference /glossary/inference time - Does not know exact RLHF /glossary/rlhf / alignment /glossary/alignment data mix — not fully published - Cannot see GPU batching or data-centre serving setup — private ops detail Those are real blind spots. Saying so was the right call. The pattern | Topic | What DeepSeek said in chat | What public papers say | |---|---|---| | Expert count | Guess, ~30% confidence, 128–256 range | 256 — | | Total | 671B total, 37B active — V3 https://arxiv.org/abs/2412.19437?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh MLA /glossary/mla in this family V2 https://arxiv.org/abs/2405.04434?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh V3 /glossary/deepseek-v3 feature R1 /glossary/deepseek-r1 line visible — R1 paper https://arxiv.org/abs/2501.12948?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh See the pattern? It was most uncertain about the things that are best documented . It was appropriately uncertain about things that are actually private . It could not tell “this fact is not public” from “this fact is public but I failed to retrieve it”. Both got the same polite hedging and confidence percentages. Near the end of the interview it rewrote speculative answers with uncertainty labels and closed with: “I am an unreliable witness to my own nature.” That line is worth more than the hedged numbers. Turn 13 calibration what it said it should have labelled | Topic | Relabelled answer from the chat | |---|---| | Expert count | Educated guess from prior DeepSeek models. ~30% confidence. Could be wrong 2–4×. V3 paper: 256. | | Layer / hidden size | Inference from V2/V3 trends, not observation. ~25% confidence. | | “I use MLA” | Likely from family lineage; cannot confirm in this instance. High but not observation. | | Context = 1M here | Assumption from public claims about earlier versions. ~40% confidence. Not measured. | | Personality % breakdown | Analytical opinion, not measurement. | | Infra sampling beam / best-of-N / speculative | Possible but unobservable. ~25% confidence. | | Hidden Type-B reasoning | Absence in context ≠ proof it does not exist could be stripped . | Common misconceptions it listed: - “It knows its own architecture” → only training text about DeepSeek papers - “It knows what it doesn’t know” → no built-in unknown detector; “I don’t know” is learned behaviour - “It can explain its real reasoning” → post-hoc plausible text - “It knows its training data” → cannot list datasets - “It can verify its own outputs” → “double-check” is more tokens from the same engine - “It has persistent memory” → only the context window unless the product injects memory - “It is a unified self” → matrix multiplications with emergent coherence; “I” is a linguistic habit If DeepSeek engineers reviewed the interview , it expected corrections on exact architecture numbers, alignment recipe, whether Type-B tokens exist for this build, tool-loop details, safety infra, real context limit / KV eviction / speculative decode, exact SKU identity, and whether transparent summarisation runs on long threads. 12. What to remember conclusions I went into this with two goals: hear how DeepSeek thinks it works, and see if that matches reality. From the interview — still useful - How it describes context order, planning, tools, safety, and personality — good for prompting and product intuition - Hard limits it admits: no weight access, no live routing view, token-by-token generation with no draft loop - Honest product caveats: tools and memory were off in this session; many pipeline stages were theory, not live demo From the paper check — the correction - Do not trust architecture numbers from the chat. Several famous specs 256 experts, 671B/37B, MLA as standard, V3 load balancing, multi-token prediction are in public papers while the model called them guesses. - Polite uncertainty is not the same as accuracy. - “Introspection” prompts are interesting theatre, not instrumentation. If you remember five things - The interview teaches you how the assistant describes itself — not what is definitely true inside the weights. - Tools and memory were inactive here; treat those chapters as “how it says it would work.” - For DeepSeek family numbers, prefer V3 https://arxiv.org/abs/2412.19437?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh and V2 https://arxiv.org/abs/2405.04434?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh over chat self-report. - Fluent “I am guessing” is still generated text. Verify anything that matters. - R1-style visible reasoning is a different product story from the chat model I interviewed — do not let one blur into the other. I still find value in asking models how they work. Just do not stop there. Interview for behaviour. Read the paper for architecture. One without the other leaves you half-informed. 13. Transcript, labs, sources appendix Full interview transcript Platform: deepseek, exported 21 July 2026. Download the interview transcript .md /downloads/deepseek-interview-transcript-2026-07-21.md Thirteen user turns short index : - Which model / version? Know vs infer vs hidden? - Architecture without assuming; family; innovations; reasoning model; hidden reasoning - Everything before a prompt; priority; pipeline - Context window; eviction; compression - Long-context attention — facts vs guesses - Before first token; plan; drafts; restart - Hidden reasoning definitions and access - Tool selection, confidence, cancel, safety override - Full path: tokenization → sampling - One pass vs many; lookahead; pivots - Hallucinations and confident-wrong answers - Personality sources - Calibration rewrite — what never to treat as fact Interactive labs Scroll to the labs block below or jump via deepseek-interactive-labs deepseek-interactive-labs . Mini glossary | Term | Plain meaning | |---|---| | Token | Small chunk of text the model reads/writes | | Context window | Limited text workspace visible at once | | Weights / parameters | Learned numbers inside the network | | System prompt | Highest-priority identity / safety text | | Transformer | Common modern chat-model design | | MoE | Only some specialist sub-networks run per token | | MLA | Compress long-chat KV memory | | RLHF / SFT | Training stages that shape assistant behaviour | | Hallucination | Fluent wrong content | | Inference | Running the trained model to generate | Sister post in Inside LLMs /series/inside-llms : Inside Kimi K2.6 /writings/models/inside-kimi-k2-6-reverse-engineering-an-ai-assistant-by-interviewing-itself . Papers and reports DeepSeek-V3 Technical Report https://arxiv.org/abs/2412.19437?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — experts, 671B/37B, MLA, load balancing DeepSeek-V2 https://arxiv.org/abs/2405.04434?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — MLA introduction, 236B/21B, 128K context DeepSeek-R1 https://arxiv.org/abs/2501.12948?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — visible reasoning traces DeepSeekMoE https://arxiv.org/abs/2401.06066?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — shared + routed expert design DeepSeek-R1 Thoughtology https://arxiv.org/abs/2504.07128?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — studying open R1 thinking traces Attention Is All You Need https://arxiv.org/abs/1706.03762?utm source=manish.sh&utm medium=blog&utm campaign=inside-deepseek-reverse-engineering-an-ai-assistant-by-interviewing-itself&ref=manish.sh — the 2017 Transformer paper Related Pinch or double-tap to zoom · tap outside to close Interactive labs Hands-on demos from this post: token cost, pre-prompt layers, chat vs paper facts, MoE routing, and observation vs inference. Teaching aids only — not DeepSeek production meters. 1. Rough token estimator Same trick as the Kimi post: ~4 characters per token for rough English. See how fast a long interview eats context. 2. Pre-prompt layer budget From §4: system → developer → memory → tools → history → your message. Slide each layer; see when the window fills. Used 0 / 131,072 tokens 0% 3. Chat self-report vs public paper From §28: where DeepSeek hedged on facts that are already on arXiv. Pick a topic; compare answers. In chat: — Public paper: — 4. MoE router V3 paper numbers DeepSeek-V3: 256 routed experts; 8 picked per token + shared experts always on . 37B active of 671B total. Click to simulate one token. 5. Observation, inference, or hidden? From §2 and Turn 13: how would you label this claim? Same buckets DeepSeek used in the interview. Pick a label. FAQ Frequently asked questions How did you reverse engineer DeepSeek? I did not open weights or leak system prompts. I interviewed the model, then labelled each claim as observation, inference, or guess — and checked architecture numbers against public papers. Which DeepSeek model was this? It said it is DeepSeek, latest version, knowledge cutoff May 2025. It was not labelled as DeepSeek-R1 in its system prompt. Were tools or memory active? No. It said no tool definitions and no memory injections were in the context. Those chapters are from its descriptions, not live observation. Can DeepSeek introspect its own weights? No. It said weights, expert routing, attention maps, and logits are completely hidden from it. Should you treat architecture numbers from the chat as facts? No. I checked against public papers — the chat hedged on 256 experts and 671B/37B params that V3 documents plainly. Read arXiv for numbers; use the chat for behaviour. Research this topic further - Click a tool below ChatGPT, Perplexity, Claude, or Gemini . - We copy the full article + companion guide prompt to your clipboard. - A new chat tab opens — if the message box is empty or only shows a short note, press Ctrl+V Windows/Linux or Cmd+V Mac to paste. - Send the message. The model already has the article text — it does not need to open this website. Related posts 22 · 07 · 2026Inside Qwen 3.8-Max-Preview: Reverse Engineering an AI Assistant by Interviewing ItselfI interviewed Qwen 3.8-Max-Preview on memory, tools, context, and hallucinations, then checked published research. Plain English, diagrams, transcript. /writings/models/inside-qwen-3-8-max-preview-reverse-engineering-an-ai-assistant-by-interviewing-itself 20 · 07 · 2026Inside Kimi K2.6: Reverse Engineering an AI Assistant by Interviewing ItselfI interviewed Kimi on memory, tools, and safety, then checked published research on K2.6. Plain English, diagrams, sources. /writings/models/inside-kimi-k2-6-reverse-engineering-an-ai-assistant-by-interviewing-itself 22 · 07 · 2026Inside Character.ai: The Technical Story of What Keeps Users HookedFrom c.ai to Character.ai — what it is, the problem it solves, then Kaiju, memory, Lorebook, and the infra behind long chats. /writings/ai-tools/inside-character-ai-the-technical-story-of-what-keeps-users-hooked Comments Share a thought on this post — keep it useful and kind. Comments are moderated before they appear. Loading comments…