{"slug": "ace-the-ai-engineer-interview-llm-fundamentals", "title": "Ace the AI Engineer Interview: LLM Fundamentals", "summary": "A practical cheat sheet for AI engineer interviews covers core LLM concepts, including data augmentation techniques. It distinguishes rule-based methods (paraphrasing, noise injection) from generative methods (rephrasing, text generation), noting trade-offs in cost, controllability, and generalization. The guide advises preparing through hands-on projects and understanding trade-offs to build transferable skills.", "body_md": "A Practical Cheat Sheet for Core LLM Concepts, Architecture, Training, and Evaluation for AI Engineer Interviews\n\n0. Prep Principals\n\nGive a man a fish and you feed him for a day; teach a man to fish and you feed him for a lifetime.\n\nThe AI era is evolving furiously. I will do my best to keep this blog updated, but it is impossible to cover every potential interview question for every company. So, I hope this note serves as a source of inspiration rather than a bible to follow mechanically.\n\nPrepare as if you were a real engineer. Go through a toy AI project, such as a chatbot, and think deeply about the following:\n\nBreak down each step and identify the practical problems that may arise.\n\nExplore what techniques are commonly used to solve those problems.\n\nThink through the trade-offs between different techniques, and how you would make decisions under different scenarios.\n\nI believe that, by doing this, YOU CAN eventually build a strong, transferable skill set, shine, and ACE the AI Engineer Interviews.\n\n1. NLP Data Augmentation\n\nA common problem in LLM training is data insufficiency. Thus data augmentation is a frequented tested point. The data augmentation in NLP area typically can be classified into two categories: rule-based (more classic), and generative (more modern). The details are shown below:\n\n1.1. Rule-based\n\nRevise a small portion of the document using explicit rules; in many cases, simple functions with dictionaries or a small LLM are sufficient.\n\nParaphrasing/Extension\n\nSynonym replacement(random swap) : E.g. A good person -> A nice person.)\n\nFormat change: E.g. datetime yyyy-mm-dd -> mm/dd/yyyy, money USD 1.5k -> $1,500)\n\nRandom insertion/deletion: E.g. A dog -> An old dog.\n\nNoise injection：\n\nWorld/lexical level: E.g. spelling errors ,including keyboard proximity, ocr error.\n\nSentence/syntactic level: E.g.grammar/syntax errors like word order, tense, singular/plural, upper/lower case.\n\nDocument level: E.g. adding irrelevant sentences.\n\nComparison: Paraphrasing vs Noise Injection\n\nParaphrasing/Extension — increase generalization (LLM can handle broader valid cases)\n\nNoise injection — increase robustness (LLM can handle erroneous inputs.)\n\n1.2. Generative\n\nGenerate a large portion of the document, typically relying on the strong general-purpose intelligence of a large LLM.\n\nParaphrasing\n\nFrom given textual data, produce alternative surface forms that preserve the same semantic information. For example:\n\nRephrasing/rewriting: express the same sentence in a different way.\n\nBack-translation: translate into another language and then translate back.\n\nPartial rewriting: E.g. for a (Q, A) pair, create multiple paraphrased versions of A for the same Q.\n\nText Generation:\n\nIntroduce new semantic information based on existing textual or non-textual inputs. For example:\n\nFrom-scratch: generate text from non-text data. E.g., given relational DB query result like “capital (America)-> Washington”, generate “The capital of America is Washington.”\n\nExpansion: turn a short document into a longer one. E.g. Summary Expansion — Expanding a summary into a detailed passage.\n\nPartial generation: fill in missing parts of a document. E.g. Pseudo User Query — generate Q based on a real fact A; Data labeling — generate A for an existing Q.\n\n1.3. Comparison: Rule-based vs Generative\n\nRule-based:\n\nPro: cheaper and more controllable in terms of quality and correctness.\n\nCon: limited coverage and weaker generalization.\n\nBest for: small datasets and cold-start (initial training/data augmentation) phases.\n\nGenerative\n\nCon: less controllable; quality depends on a strong model, with higher cost and more complex validation.\n\nPro: stronger at improving generalization.\n\nBest for: scenarios where a powerful model is already available.\n\nTrade-off summary: think in terms of cold-start vs mature stage, the required level of controllability, cost and validation complexity.\n\n1.4. Risk/Cons of Data Augmentation\n\nOverfitting: the model may memorize specific augmented samples instead of learning general patterns.\n\nData quality degradation: the model may pick up noise and errors from augmented data and ultimately generate lower-quality outputs.\n\n2. LLM Architecture\n\n2.0. Why LLM? (Comparison: Transformer vs RNN)\n\nNotation: Assume the sequence length is O(T).\n\nTraining\n\nLLM’s pro vs RNN\n\nLong‑term dependency: Transformers model global token‑to‑token dependencies, whereas in RNNs, even with a memory state, long‑term information typically decays over time.\n\nFaster training (and inference prefilling) speed: processing existing input tokens can be parallelized in Transformers, while it is sequential in RNNs. I.e., sequential wall-time complexity: Transformer O(1) vs RNN O(T), assuming enough parallel hardware.\n\nLLM’s con vs RNN\n\nHigh memory usage: For RNNs, the model parameters are shared across tokens, but during training, we need to store per-token hidden states/activations for backpropagation (BP), so the memory consumption over the sequence is O(T). For Transformers, storing per-token activations is also O(T), but the peak memory often comes from sequence-level pairwise attention computation, such as the QKᵀ attention matrix, which takes O(T²) memory. In short, for a sequence of length T, memory complexity: Transformer O(T²) vs RNN O(T).\n\nHigher FLOPs complexity: Standard Transformer self-attention computes pairwise interactions between tokens. For each token, attending to all other tokens costs O(T), so for a sequence of length T, the total compute complexity is O(T²). In contrast, an RNN processes one token at a time together with a hidden state that compresses information from previous tokens, leading to O(1) compute per token and O(T) compute for the full sequence. In short, for a sequence of length T, FLOPs complexity: Transformer O(T²) vs RNN O(T).\n\nInference\n\nLLM’s pro vs RNN\n\nLong-term dependency: Same explanation as training.\n\nLLM’s tie vs RNN\n\nWall-time complexity: Both vanilla Transformers and RNNs follow the autoregressive generation pattern, decoding one token at a time. Therefore, sequence generation wall-time complexity for both models: O(T).\n\nLLM’s con vs RNN\n\nHigh memory usage: For RNNs, decoding each token only requires keeping the current hidden state, leading to O(1) memory complexity with respect to sequence length. For Transformers, memory usage depends on whether KV cache is used. Without KV cache, Transformer decoding recomputes attention over the full prefix, and the attention matrix QKᵀ can take O(T²) memory at sequence length T. With KV cache, Transformers store K/V states for all previous tokens, leading to O(T) resident memory during decoding. In short, inference memory complexity: Transformer O(T) (w/ KV cache, O(T²) w/o KV cache) vs RNN O(1).\n\nHigher FLOPs complexity: For RNNs, each token requires O(1) compute to process the current token and update the hidden state, leading to O(T) compute for a sequence of length T. For Transformers, even with KV cache, each newly decoded token still attends to all previous tokens, which costs O(T) at sequence length T. Therefore, sequence-level FLOPs complexity: Transformer O(T²) vs RNN O(T).\n\n2.1. Transformer Structure\n\nGeneral components in Transformer:\n\nEmbedding layer / lookup table, positional encoding, Transformer blocks, linear layer, and softmax head.\n\nSimilarity between encoder and decoder models?\n\nBoth encoder and decoder models mostly use token-level cross-entropy loss in pre-training. (The difference is just in the prediction objective: encoders often predict masked tokens in MLM, while decoders predict the next token in CLM / autoregressive language modeling.)\n\nDifference between encoder and decoder model?\n\nEncoder\n\nTraining method / mask: MLM\n\nPrediction target: representation / embedding of the input token or sequence.\n\nMulti-head self-attention: No intentional causal mask for valid tokens; only uses an attention mask for padding tokens.\n\nTraining method / mask: CLM / Autoregression. Prediction target: probability distribution of the next token after the input tokens.\n\nMasked multi-head self-attention: Self-attention with a causal mask, so each token can only attend to previous tokens and itself.\n\nCross-attention: Cross-attention means Q comes from the model’s current sequence, while K/V come from another sequence, modality, or external memory. By definition, an encoder is not forbidden from using cross-attention. However, in a common seq2seq setting, the encoder is used to interpret sequence A, while the decoder uses cross-attention to acquire information from sequence A and generate sequence B.\n\nTo enrich the representation and increase model expressivity. This is similar in spirit to the SVM kernel trick: a structure that is hard to separate in a lower-dimensional space may become easier to represent or separate in a higher-dimensional space.\n\n2.2. Attention Mechanism\n\nDescribe the (self) attention mechanism:\n\nThe output is a weighted sum of all tokens’ value vectors (V), where the weights are computed by applying softmax to the dot products between the current token’s query (Q) and other tokens’ keys (K), scaled by sqrt(dim).\n\nNote: The “self” in self-attention means that Q, K, and V are all generated from the same sequence. In contrast, in cross-attention, Q comes from one sequence, while K and V come from another.\n\nDescribe the self-attention formula:\n\nQKV generation: The Q, K, and V matrices are generated from the input token embeddings through linear transformations.\n\nQK product: Each token’s query vector (Q) takes a dot product with all tokens’ key vectors (K) to produce the raw attention logits.\n\nScaling: The QK logits are scaled by sqrt(d_k).\n\nCausal mask: Add -inf to the logits corresponding to future tokens relative to the current position, so they receive zero attention weight after softmax.\n\nSoftmax: Apply softmax to the logits to obtain normalized attention weights.\n\nWeighted sum: The attention output is a weighted sum of all tokens’ value vectors (V), using the attention weights.\n\nWhy scale the dot product by sqrt(d_k)?\n\nThe dot product is scaled by sqrt(d_k) to keep its variance roughly constant, reducing the chance of extremely large attention logits.\n\nExtremely large logits can saturate the softmax, making the output probabilities close to 0 or 1. In these saturated regions, gradients become very small, which can lead to vanishing gradients and slower or less stable training.\n\nWhy does the attention mechanism work? (i.e. what are its advantages over traditional language models?)\n\nAutoregressive task: Attention vs. RNN\n\nLong-term dependency: RNNs compress previous information into a recurrent hidden state, which makes long-range dependencies harder to preserve. Transformers can directly attend to tokens across the full available context.\n\nParallelism: RNNs process tokens sequentially, while Transformer training can process tokens in parallel across sequence positions.\n\nExplainability: The hidden state or memory cell in an RNN/LSTM is difficult to interpret directly, while a Transformer’s attention weights can be visualized to inspect which tokens attend to which others.\n\nEmbedding task: Transformer vs. Static embedding techniques\n\nContextualized representation: Classical embeddings such as Word2Vec and GloVe are static, meaning a word has the same embedding regardless of context; While modern transformer-based embeddingsare contextualized, meaning the representation of a token changes depending on the surrounding context.\n\nNote: the Transformer’s input embedding lookup table is also static. However, in most downstream applications, we use the contextualized hidden states produced by the Transformer rather than the raw lookup-table embeddings.\n\nWhat is Multi-Head Attention (MHA)? What are its advantages over single-head attention?\n\nDefinition\n\nThe input embeddings are sent to multiple attention heads. Each head has its own Q, K, and V projection matrices and computes attention independently. The outputs from all heads are then concatenated and passed through a final output projection.\n\nAdvantages:\n\nRicher representation space: Different heads can attend to different representation subspaces and different token positions, potentially capturing different types of information, such as semantic relationships, syntactic patterns, and short- or long-range dependencies.\n\nMultiple attention patterns: Compared with a single head, MHA allows the model to represent multiple attention patterns in parallel.\n\nTypical number of heads: Common choices include 8, 12, 16, 32, or more, depending on the model size. Very large models may use substantially more heads.\n\n2.3. Tokenization\n\nWhat is tokenization?\n\nTokenization is the process of splitting a text string into smaller units called tokens, such as words, subwords, or characters.\n\nAdvantage of subword tokenization over word-level tokenization?\n\nFewer OOV tokens\n\nSmaller vocabulary size\n\nTrade-off of using a larger vocabulary in subword tokenization?\n\nPro: Shorter sequences and more direct representation of common words/subwords\n\nCon: Higher memory and computation cost; rare tokens require more data to learn well\n\nWhat are common subword tokenization algorithms, and how do they differ?\n\nByte-Pair Encoding (BPE)\n\nAlgorithm:\n\nStart with a base vocabulary of small units, typically characters or bytes.\n\nCount adjacent token-pair frequencies in the training corpus.\n\nIteratively merge the most frequent adjacent pair into a new token.\n\nStop when the target vocabulary size or number of merges is reached.\n\nKey idea:\n\nFrequency-based bottom-up merging. Frequent adjacent token pairs are gradually combined into larger subword tokens.\n\nMerge criterion: Typically select the most frequent adjacent pair: Score(w1, w2) = Count(w1, w2)\n\nBPE was adapted for subword tokenization by iteratively merging frequent symbol pairs.\n\nWordPiece\n\nAlgorithm:\n\nSimilar to BPE, WordPiece builds a subword vocabulary by combining smaller units into larger units.\n\nHowever, instead of selecting merges purely by pair frequency, it prefers merges that better improve the language-model likelihood / vocabulary objective.\n\nKey idea:\n\nLikelihood-oriented merging rather than pure frequency-based merging.\n\nCommon intuition for merge score:Score(w1, w2) ≈ Count(w1, w2) / (Count(w1) * Count(w2))\n\nThis favors pairs that occur together more strongly relative to the individual frequencies of their components.\n\nCommon usage:\n\nWordPiece is strongly associated with Google NLP systems and is used by BERT.\n\nUnigram Language Model\n\nAlgorithm:\n\nUnlike BPE and WordPiece, Unigram does not build the vocabulary primarily through iterative merging.\n\nStart with a large candidate subword vocabulary.\n\nAssign a probability to each candidate token.\n\nModel the probability of a tokenized sequence as the product of its token probabilities.\n\nIteratively remove tokens whose removal causes the smallest degradation in the corpus likelihood.\n\nStop when the target vocabulary size is reached.\n\nKey idea:\n\nTop-down vocabulary pruning based on a probabilistic language-model objective.\n\nDifference from BPE:\n\nBPE: start small and merge tokens.\n\nUnigram: start large and prune tokens.\n\nThe Unigram LM method explicitly defines a probabilistic segmentation model and iteratively prunes candidate pieces while preserving corpus likelihood as much as possible.\n\nSentencePiece\n\nSentencePiece is not a tokenization algorithm itself. It is a tokenization framework / library.\n\nIt can train and apply multiple tokenization models, especially:\n\nBPE\n\nUnigram Language Model\n\nA key feature is that it can train directly from raw text rather than requiring language-specific pre-tokenization.\n\nSentencePiece’s official implementation supports both BPE and Unigram segmentation models.\n\n2.4. Positional Encoding\n\nWhy do we need positional encoding (PE)?\n\nPE helps the model distinguish tokens at different positions.\n\nWithout PE, the same token at different positions would have the same input embedding, and self-attention itself has no inherent notion of token order.\n\nHow does positional encoding work (i.e., help distinguish same tokens at different positions)?\n\nFor different positions, it incorporates different positional information into the token representations, either directly or indirectly, allowing the model to distinguish the same token appearing at different positions.\n\nWhat are the main types of positional encoding? Explain their key ideas and compare their differences.\n\nAbsolute Positional Encoding (APE):\n\nEach absolute position is assigned a specific positional representation.\n\nCon: Learned APE is typically tied to a predefined maximum position range and does not naturally provide translation invariance.\n\nRelative Positional Encoding (RPE):\n\nEncodes the relative position or distance between token pairs rather than only their absolute positions.\n\nE.g. A common approach is to add a relative-position term or bias to the attention score: softmax(QK^T / sqrt(d_k) + B_relative)\n\nPro: Translation invariant: shifting both tokens by the same offset preserves their relative position.\n\nCon: The positional signal is relatively simple and depends on the chosen distance-bias slopes.\n\nRotary Positional Embedding (RoPE):\n\nApplies position-dependent rotations to Q and K based on their absolute positions.\n\nThe resulting Q-K dot product naturally depends on their relative position.\n\nPro: Combines absolute-position-based transformations with relative-position-aware attention, providing a richer positional signal than a simple scalar distance bias.\n\nWhen should we choose which positional encoding?\n\nAPE: Does not have translation invariance, so it is less suitable for long text. E.g. Learnable APE: Simple, suitable for short text with clear positional patterns.\n\nPro: Trainable, potentially more accurate.\n\nCon: Limited context window.\n\nExample: Short-text encoder.\n\nRPE (e.g., ALiBi / T5): Suitable for long context with relatively simple positional patterns.\n\nCon: The positional effect is often mainly distance-based, which may be too simple.\n\nExample: Sentence-level translation.\n\nRoPE: Combines long-range decay with local oscillation, making it suitable for more complex positional patterns.\n\nExample: Agents, long documents.\n\n2.5. Normalization\n\nWhy do we need normalization layers?\n\nDNNs, including Transformers, have inherent training stability problems:\n\nInternal covariate shift:\n\nParameter updates change the distribution (e.g., mean and variance) of intermediate activations.\n\nSuch distribution shifts can be amplified through deeper layers.\n\nThis may lead to unstable gradients, including gradient explosion or vanishing.\n\nBatch randomness:\n\nDifferent batches may have different data distributions, making training unstable, slower, or even causing divergence.\n\nHow does normalization help?\n\nStabilizes the input distribution of each layer (e.g., around zero mean and unit variance).\n\nMakes optimization more stable and gradient flow smoother, leading to more efficient training.\n\nWhat normalization does the Transformer use? Explain it.\n\nLayerNorm: Normalizes across the feature / embedding dimensions within each token position independently.\n\nWhy is BatchNorm (BN) not suitable for autoregressive (AR) tasks?\n\nBN requires: 1. i.i.d. samples, and 2. a large batch size.\n\nI.i.d.:\n\nTokens within the same sequence are highly correlated, so they are not i.i.d.\n\nThe same token positions across different sequences may also have different distributions due to variable sequence lengths and [PAD] tokens.\n\nBatch size:\n\nLong sequences and large autoregressive models consume significant memory, often limiting batch size. A smaller batch provides fewer samples for estimating mean and variance, making BN statistics noisier and training less stable.\n\nWhy does LayerNorm (LN)not have the above problems for AR tasks?\n\nLayerNorm normalizes each token independently across its feature / embedding dimensions, so it does not depend on other tokens in the sequence or other samples in the batch.\n\nWhat is the core difference between Pre-LN and Post-LN?\n\nPost-LN: LayerNorm is applied to both the transformation output and the skip connection.\n\nPre-LN: LayerNorm is applied only to the transformation output, leaving the skip connection unaffected. This helps preserve a cleaner gradient path through the skip connection, leading to smoother gradient flow and better scalability for deeper Transformer architectures.\n\nWhat is the difference between RMSNorm andLayerNorm?\n\nLayerNorm: Normalizes using both the mean and variance:\n\nmu = mean(emb)\n\nsigma = sqrt(mean((emb - mu)^2))\n\nemb' = gamma * (emb - mu) / sigma + beta\n\nRMSNorm: A simplified variant of LayerNorm that does not subtract the mean:\n\nRMS(emb) = sqrt(mean(emb^2))\n\nemb' = gamma * emb / RMS(emb)\n\nPros of RMSNorm:\n\nLower computation cost because no mean subtraction is required.\n\nSlightly fewer learnable parameters if the LayerNorm bias term is omitted.\n\nOften achieves performance comparable to LayerNorm.\n\n3. Pre-training\n\n3.1. Pre-training Configuration\n\nWhat regularization techniques are commonly used in LLM training?\n\nWeight decay (L2 regularization)\n\nDropout\n\nEarly stopping\n\nData augmentation\n\nOther related training-stability techniques?\n\nGradient clipping\n\nLayer Normalization\n\nHow to train a new LLM from scratch? What are the key hyperparameters / configurations?\n\nInitialization\n\nXavier initialization: Keeps input and output variance at a similar scale. More suitable for sigmoid / tanh-like activations.\n\nKaiming (He) initialization: Uses a larger variance to account for ReLU truncating negative activations. Suitable for ReLU-like activations.\n\nNote: For Transformers, initialization is often architecture-specific rather than simply choosing Xavier or Kaiming.\n\nOptimizer\n\nAdam: Adaptive learning rate + momentum.\n\nAdamW: Adam with decoupled weight decay; commonly used for LLM training.\n\nLearning Rate (LR)\n\nModel- and scale-dependent; commonly around 1e-3 to 1e-4 for smaller models, and often lower for larger models.\n\nLR Scheduler\n\nWarmup: Gradually increase the global LR at the beginning to avoid early training divergence.\n\nCosine decay: Gradually decrease the LR after warmup to reduce oscillation and improve convergence.\n\nBatch Size\n\nFor LLM pretraining, global batch size is often measured by the number of tokens per optimization step, rather than only the number of sequences.\n\nLarge-scale training may use millions of tokens per step; the exact value depends strongly on model size and compute scale.\n\nRegularization and training-stability techniques\n\nSee questions above\n\nQuantization\n\nCommonly BF16 mixed-precision training.\n\nLower-precision formats such as FP8 may also be used on supported large-scale training systems.\n\nWhat is stored in the optimizer state, e.g., AdamW?\n\nFirst moment (exp_avg): Exponential moving average of gradients; acts like a momentum-smoothed gradient or “velocity.”\n\nSecond moment (exp_avg_sq): Exponential moving average of squared gradients; used to adapt the effective learning rate for each parameter.\n\nStep count: Number of optimization steps; used for bias correction of the first and second moment estimates.\n\n3.2. Scaling Law\n\nExplain the scaling law?\n\nThe scaling law states that test loss decreases approximately as a power law when increasing:\n\nCompute budget\n\nDataset size\n\nModel size (number of parameters)\n\nEquivalently, in log-log space, test loss decreases approximately linearly with these scaling factors.\n\nWhy do we need scaling laws?\n\nLLM training is expensive. Scaling laws help us understand and predict the trade-off between model performance (e.g., loss) and cost factors such as data size, model size, and compute budget.\n\n3.3. Memory-Bound vs. Compute-Bound\n\nWhat is the difference between memory-bound and compute-bound?\n\nMemory-bound: More time is spent moving / accessing data than performing computation; compute units may wait for data.\n\nCompute-bound: More time is spent performing computation; performance is limited mainly by data throughput.\n\nWhat are common indicators of memory-bound vs. compute-bound?\n\nGPU utilization:\n\nHigh GPU utilization → more likely compute-bound.\n\nLow GPU utilization → more likely memory-bound.\n\nExplanation: Lower GPU utilization may indicate that more time is spent waiting for data movement for the same amount of work.\n\nAmount of data generated:\n\nMore data generated → more likely memory / I/O-bound.\n\nLess data generated → more likely compute-bound.\n\nExample: If 10 numbers generate 100 numbers, more output data needs to be moved, making the operation more likely memory-bound. If 10 numbers generate 1 number, less time is spent on data movement, making it more likely compute-bound.\n\nWhen is a Transformer memory-bound vs. compute-bound?\n\nTransformer-block wise:\n\nFFN → Compute-bound: Input is O(T × d), output is also O(T × d), while substantial matrix multiplication is performed.\n\nAttention → Memory-bound: Input is O(T × d), while the attention matrix is O(T²), causing significantly more data movement / I/O.\n\nSequence-length wise:\n\nLong sequence → More memory-bound: QK^T scales as O(T²), causing more attention data movement.\n\nShort sequence → More compute-bound: Less time is spent on attention I/O relative to computation.\n\nBatch-size wise:\n\nLarge batch size → More compute-bound: Higher GPU utilization and better compute saturation.\n\nSmall batch size → More memory-bound: Lower GPU utilization, with relatively more time spent waiting on data movement.\n\nMethods for compute-bound and memory / IO-bound problems?\n\nKV Cache: Compute-memory trade-off\n\nReduces FLOPs by storing and reusing previous K and V states instead of recomputing them.\n\nTrade-off: Increases memory usage and KV-cache reads.\n\nNote: KV cache does not make decoding I/O-bound simply because “cache is expensive.” It significantly reduces recomputation, so compute cost drops; at long context, reading the growing KV cache can then become the bottleneck.\n\nGradient Checkpointing: Memory-compute trade-off\n\n(Used only during training, not inference.) Stores only selected activations during forward propagation and recomputes the missing activations during backward propagation.\n\nReduces memory usage by increasing FLOPs.\n\nConceptually, somewhat opposite to KV cache: recompute instead of store.\n\nFlashAttention: I/O optimization\n\nReduces HBM I/O by tiling attention computation and avoiding materialization of the full attention matrix. May use some recomputation / extra on-chip computation to reduce expensive memory traffic.\n\nTrade-off: More local computation for significantly less HBM I/O.\n\nModel compression (e.g., quantization, pruning, distillation):\n\nCan reduce both computation and memory / I/O cost.\n\nExact benefit depends on the method and hardware support.\n\nDistributes memory and computation across multiple devices.\n\nReduces per-device memory and compute workload.\n\nDoes not necessarily reduce total FLOPs or total I/O; it mainly improves wall-clock time at the cost of more hardware and communication overhead.\n\n3.4. Flash Attention\n\nWhat is FlashAttention?\n\nFlashAttention splits large Q, K, and V tensors into blocks and computes attention block by block in fast on-chip memory (SRAM).\n\nIt fuses attention operations and avoids materializing the full O(L²) attention matrix in HBM, significantly reducing memory I/O.\n\nWhy do we need FlashAttention, and how does it work?\n\nProblem in attention calculation process:\n\nStandard attention generates large intermediate matrices: 1. QK^T 2. Softmax(QK^T) 3. Softmax(QK^T) @ V\n\nThese intermediate results may require expensive reads / writes between HBM and on-chip memory, causing an I/O bottleneck.\n\nHow does FlashAttention solve the problem:\n\nFlashAttention can be viewed as a fused, I/O-aware attention algorithm.\n\nSplits large Q, K, and V tensors into blocks so that blocks of Q_i, K_j, and V_j fit into fast on-chip SRAM.\n\nComputes each block’s attention contribution locally without writing the intermediate Q_i K_j^T or attention-probability matrix back to HBM.\n\nUses online softmax to maintain running normalization statistics and correctly combine contributions from different K/V blocks.\n\nTherefore, it computes the exact attention result while significantly reducing HBM I/O.\n\nWhy can it compute the same result without storing the full L × L attention matrix?\n\nBecause the intermediate QK^T and softmax attention matrix do not need to be explicitly materialized in HBM.\n\nFlashAttention processes them block by block and uses online softmax to incrementally update the normalized output.\n\n4. SFT\n\n4.1 PEFT Approaches Overview\n\nWhy is PEFT needed?\n\nFull fine-tuning of LLMs is computationally expensive and requires large amounts of GPU memory and training data. PEFT adapts the model by updating only a small fraction of parameters, making fine-tuning practical on limited hardware (often a single GPU).\n\nCompared with full fine-tuning, PEFT generally reduces the risk of catastrophic forgetting.\n\nMajor PEFT approaches?\n\nSelective: Fine-tune only selected existing layers; the chosen parameters are updated directly.\n\nReparameterization: Fine-tune selected layers through a low-dimensional parameterization (e.g., low-rank updates), greatly reducing the number of trainable parameters.\n\nAdditive: Keep the original model frozen and add trainable modules or virtual parameters.\n\nExample techniques for different PEFT approaches?\n\nSelective fine-tuning:\n\nShallow-layer fine-tuning for data-distribution adaptation\n\nDeeper-layer fine-tuning for task adaptation\n\nReparameterization:\n\nLoRA\n\nAdditive methods:\n\nAdapters: Add small trainable layers after the attention or FFN sublayers.\n\nPrompt tuning / soft prompting: Prepend trainable virtual tokens to the input embeddings.\n\nWhat are the application scenarios for different PEFT approaches?\n\nAdditive vs. Selective / Reparameterization\n\nAdditive methods: Better preserve the pretrained model because the original parameters remain frozen. Suitable when one shared base model needs to support multiple customized versions, such as an enterprise chatbot personalized for different customers.\n\nSelective / Reparameterization methods: Better when stronger adaptation to a specialized task or domain is required, potentially at the cost of some generalization ability.\n\nSelective vs. Reparameterization\n\nSelective fine-tuning: Suitable when only one specialized model is needed, such as on-device deployment or infrequently updated models. However, each task may require storing and transferring a larger modified checkpoint.\n\nReparameterization (e.g., LoRA): Suitable when one server needs to support multiple tasks or domains. Multiple lightweight adapters can share the same frozen base model and be loaded or switched efficiently.\n\n4.2 LoRA\n\nExplain the LoRA method.\n\nFreeze the original weight matrices.\n\nFor each selected weight matrix, add a trainable low-rank update.\n\nInstead of learning a full weight matrix, the update is parameterized as the product of two low-rank matrices: ΔW = αAB\n\nThe effective weight becomes: W = W_frozen + αAB\n\nHow does LoRA reduce the number of trainable parameters?\n\nThis is easier to illustrate with a demo. Suppose the original weight matrix has shape: 100 × 100\n\nFull fine-tuning:\n\n100 × 100 = 10,000 trainable parameters.\n\nLoRA with rank = 10:\n\nA: 100 × 10\n\nB: 10 × 100\n\nTotal trainable parameters = 2,000\n\n80% fewer trainable parameters.\n\nWhat are the trade-offs of LoRA compared with full fine-tuning?\n\nPro: fewer trainable parameters, therefore: requires less training data, lower risk of overfitting, lower GPU memory consumption, faster training and less catastrophic forgetting.\n\nCon: lower performance ceiling\n\nVariants of LoRA?\n\nQLoRA\n\nQLoRA = LoRA + quantization of the frozen base model.\n\nThe frozen model is stored in low precision (typically 4-bit) to reduce GPU memory.\n\nComputation and LoRA adapters are still performed in higher precision (e.g., BF16).\n\nAdditional dequantization is performed during computation.\n\nLoRA+\n\nLoRA+ = different learning rates for A and B.\n\nTypically trains faster and achieves better convergence.\n\nAdaLoRA\n\nAdaLoRA = LoRA + adaptive rank allocation.\n\nThe total rank budget remains approximately constant.\n\nDuring training, ranks are dynamically reallocated across layers according to their estimated importance.\n\nMore important layers receive higher ranks, while less important layers receive lower ranks.\n\n5. RL\n\n5.1. RL Overview\n\nChoosing between RL and SFT?\n\nSFT: Better for learning fixed output patterns with relatively rigid inter-token dependencies, such as QA and tool-use tasks.\n\nRL: Better for optimizing a high-level objective that can be achieved through multiple valid routes, where the optimal path to the best answer is ambiguous.\n\nCommon sources of RL reward signals?\n\nRLHF (Human Feedback): The source of the supervision signal is human annotators.\n\nRLAIF (AI Feedback): The source of the supervision signal is an AI judge, typically another LLM.\n\nRLVR (Verifiable Rewards): The source of the supervision signal is an automated evaluation system, such as unit tests, code execution, mathematical answer checking, or rule-based verifiers.\n\n5.2. Reward Design\n\nPossible cause of RL training divergence or failure to learn?\n\nSparse reward problem: Reward is received only at the end (or very infrequently), making it difficult for the agent to identify which actions contributed to success.\n\nSolutions to the sparse reward problem?\n\nLevel reward: Assign reward scores to intermediate states or milestones. (E.g., fully achieving the goal -> 1.0, while partially achieving it -> 0.3.)\n\nProcess Reward Model (PRM): Train a model to evaluate whether each intermediate action contributes to a successful final outcome. (E.g. after a planning step, if the agent generates a correct, incorrect, or no tool-call query, the planning step may receive rewards of +0.5, -0.1, or 0, respectively. )\n\nWhat is reward hacking?\n\nThe agent learns to exploit flaws in the reward function, achieving a high reward through undesired behaviors instead of accomplishing the intended objective.\n\nCauses and solutions for reward hacking?\n\nCause 1: Reward-objective misalignment — The reward does not fully or accurately represent the true objective. E.g. A recommendation model maximizes clicks by generating clickbait.\n\nMulti-objective reward design: 1. Add guardrail and safety-related reward terms. 2. Add explicit penalties for undesired behaviors.\n\nCause 2: Reward-model overfitting — The reward model overfits its training data, and the policy exploits erroneous regions of the reward landscape.\n\nReduce reward-model overfitting: Use reward-model ensembles and add more diverse or adversarial training data.\n\nRegularize the policy during training: Mix in real / supervised data or add a KL penalty to constrain deviation from the reference policy.\n\nControl actions during inference: Restrict the allowed action space through explicit instructions or infrastructure-level tool-use constraints.\n\n5.3. RL Algorithms\n\nDescribe the process of RLHF with the PPO algorithm.\n\nGenerate multiple responses (completions) for the same query (prompt) and obtain feedback from human annotators.\n\nPrepare the samples in a pairwise format and train the reward model, typically using a pairwise ranking loss.\n\nDuring policy training, a prompt is passed to the LLM to generate a completion. The full (prompt, completion) pair is then sent to the reward model to obtain a reward score. The reward score is consumed by the PPO algorithm to update the LLM.\n\nWhat are the limitations of PPO?\n\nRelies on a reward model: Training is more complex and may introduce reward hacking.\n\nRelies on a value function/model: Training is more computationally expensive and may amplify reward hacking, since the value function estimates the discounted cumulative future reward.\n\nHow do alternative algorithms address these limitations?\n\nReward model is expensive to train → DPO\n\nRemoves the reward model.\n\nReformulates the two-stage reward modeling + RL pipeline into a one-stage preference classification / optimization problem.\n\nValue function is expensive to train → GRPO\n\nRemoves the value function.\n\nInstead of estimating the advantage using a learned value function, GRPO computes the advantage from the relative rewards within a group of sampled responses.\n\nWhen should we use PPO, DPO, or GRPO?\n\nAlgorithm choice largely depends on the reward-signal format and available computational resources.\n\nPPO\n\nSuitable when a large volume of preference data is available, so the reward model can be trained accurately.\n\nAlso suitable when there are only one or a few simple reward functions, resulting in a relatively simple reward landscape and making the value function easier to train for accurate advantage estimation.\n\nRequires abundant computational resources for training both the reward model and the value function.\n\nDPO\n\nSuitable when only a relatively small amount of preference data is available and training a reliable reward model for PPO is difficult.\n\nRequires fewer computational resources because it avoids separate reward-model training and directly optimizes preference pairs.\n\nGRPO\n\nSuitable when there are multiple reward functions, making the value function difficult to train reliably with PPO.\n\nAvoids training a value function, making it easier to train and more computationally efficient than PPO.\n\n5.4. RL Metrics\n\nModel Training Monitoring\n\nActor loss: Measures how well the policy is being optimized. It is used to monitor whether the policy is learning and whether policy optimization is stable.\n\nCritic loss: Measures the prediction error of the value function (critic), i.e., how accurately it estimates the expected cumulative reward. A lower critic loss generally indicates more accurate advantage estimation.\n\nPolicy Model Comparison\n\nCumulative Reward: The total reward obtained by the agent when completing a task or episode. Variants include average episodic reward, average reward per step, and discounted cumulative reward.\n\nSuccess Rate: The percentage of tasks or episodes completed successfully.\n\nStability: Measures whether performance is consistent across different tasks or episodes, often represented by the variance of cumulative rewards.\n\nRL Algorithm Comparison\n\nSample Efficiency: The number of samples or environment interactions required to reach a given performance level.\n\nConvergence Rate: How quickly the algorithm approaches a stable and effective policy during training.\n\n6. Evaluation\n\n6.1. Evaluation Methods\n\nHuman Evaluation\n\nForms: Binary pairwise comparison (preference) and absolute scoring.\n\nPros: Accurate and flexible for evaluating open-ended tasks.\n\nCons: Expensive, slow, difficult to scale, and may suffer from annotation inconsistency.\n\nLLM-as-a-Judge: Use a powerful LLM to evaluate another model.\n\nMetrics: Binary pairwise comparison and absolute scoring (typically based on explicit evaluation criteria).\n\nPros: Inexpensive, scalable, and flexible for open-ended tasks.\n\nCons: Less reliable than human evaluation or reference-based evaluation.\n\nPublic Benchmarks / Datasets\n\nMetrics: Reference-based or reference-free metrics, depending on the benchmark.\n\nPros: Accurate, inexpensive, and standardized.\n\nCons: Static, limited in scope, and may not fully align with the target application.\n\nUse-case capability: Whether the model supports the target task, such as chat, summarization, reasoning, or tool use.\n\nCost: Inference cost, model size, and available compression options such as quantization.\n\nLatency: Whether the model satisfies the application’s response-time requirements.\n\nBenchmark performance: Performance on relevant public benchmarks and, more importantly, task-specific evaluations.\n\nHow can we handle context-window limits for long-text generation?\n\nPositional-encoding scaling:\n\nRule-based relative PE (e.g., ALiBi): More naturally scalable to longer contexts.\n\nRoPE scaling: Extends the context window by modifying or rescaling RoPE frequencies.\n\nAbsolute PE: Uses positional interpolation or extrapolation.\n\nContext-window operations:\n\nTruncation: Remove part of the text so the input fits within the context window.\n\nSliding window: Process overlapping text segments while moving the window by a fixed stride.\n\nPrompt engineering / summarization:\n\nContextual summarization: Sequentially summarize earlier chunks and combine the running summary with the next chunk.\n\nHierarchical summarization: Summarize chunks in parallel, then recursively combine and summarize adjacent summaries.\n\nPrompt chaining: Break a large prompt into smaller sub-prompts and process them sequentially.\n\nRAG: Chunk and index the document, then retrieve only the relevant chunks when needed.\n\nHow can we evaluate LLM applications with insufficient human annotations?\n\nData augmentation for human annotations: Generate additional evaluation samples using generative AI or rule-based transformations, then use the augmented samples as evaluation data.\n\nLLM-as-a-judge calibrated with human annotations: Run the LLM judge on human-annotated samples and use human–LLM disagreement signals to calibrate the evaluation. One example is Prediction-Powered Inference (PPI).\n\nProxy labels: Use user behavior or product signals as indirect labels, such as clicks, retention, task completion, or user corrections.\n\nLabel-free evaluation: Evaluate properties that do not require ground-truth labels. For example, measure whether the model’s decisions are self-consistent or stable across repeated runs or prompt variations.\n\nThis article may not be reproduced, distributed, republished, or adapted, in whole or in part, without the author’s prior written permission, except as permitted by applicable copyright law. Sharing the original Medium link is welcome.", "url": "https://wpnews.pro/news/ace-the-ai-engineer-interview-llm-fundamentals", "canonical_source": "https://pub.towardsai.net/ace-the-ai-engineer-interview-llm-fundamentals-ac889a80aad9?source=rss----98111c9905da---4", "published_at": "2026-08-23 22:01:01+00:00", "updated_at": "2026-08-23 22:43:05.370859+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "natural-language-processing"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/ace-the-ai-engineer-interview-llm-fundamentals", "markdown": "https://wpnews.pro/news/ace-the-ai-engineer-interview-llm-fundamentals.md", "text": "https://wpnews.pro/news/ace-the-ai-engineer-interview-llm-fundamentals.txt", "jsonld": "https://wpnews.pro/news/ace-the-ai-engineer-interview-llm-fundamentals.jsonld"}}