📐 Mathematics for AI — Foundation Course

A developer published a guide covering the essential mathematical concepts that power modern AI and large language models, explaining how text is tokenized, embedded, and processed through a transformer pipeline to predict the next token. The guide maps each step of the pipeline—from tokenization and embeddings to attention layers and probability distributions—to foundational math topics such as number systems, vectors, linear algebra, and probability.

Before you can truly understand how AI systems think, learn, and generate responses, you need to understand the math that powers them. This guide covers the essential mathematical concepts that form the backbone of modern Artificial Intelligence and Large Language Models LLMs . Why does this matter?Every aspect of AI — from how text is encoded, to how a model predicts the next word, to how it improves itself during training — is driven by mathematics. Skipping this foundation means you will only ever use AI as a black box, without understandingwhyit works. Before diving into each math concept individually, here's the big picture of how text flows through a Large Language Model from input to output. Every section in this guide maps to a step in this pipeline: ┌─────────────────────┐ │ Your Prompt │ "What is gravity?" └──────────┬──────────┘ ↓ ┌─────────────────────┐ │ Tokenizer │ Splits text into chunks BPE algorithm └──────────┬──────────┘ → Section 1: Number Systems & Encoding ↓ ┌─────────────────────┐ │ Token IDs │ Each token → a number e.g., "gravity" → 17942 └──────────┬──────────┘ → Section 1: Number Systems & Encoding ↓ ┌─────────────────────┐ │ Embedding Model │ Each token ID → a dense vector of numbers └──────────┬──────────┘ → Section 3: Vectors & Embeddings ↓ ┌─────────────────────┐ │ Vectors │ 0.12, -0.87, 0.45, ... per token │ + Positional Info │ → Section 3 & 6: Embeddings & Linear Algebra └──────────┬──────────┘ ↓ ┌─────────────────────┐ │ Transformer │ Multi-Head Attention + Feed-Forward layers │ ×N layers │ repeated 32-96+ times └──────────┬──────────┘ → Section 4, 6: Algebra & Linear Algebra ↓ ┌─────────────────────┐ │ Probability │ Softmax converts final output to │ Distribution │ probabilities over entire vocabulary └──────────┬──────────┘ → Section 2 & 6: Probability & Softmax ↓ ┌─────────────────────┐ │ Next Token │ Sampling picks one token │ Sampling │ using Temperature, Top-K, Top-P └──────────┬──────────┘ → Section 2: Probability & Prediction ↓ ┌─────────────────────┐ │ Append & Repeat │ Add the new token to the sequence, │ Autoregressive │ feed it back in, and repeat until done └─────────────────────┘ Key Insight:The model generates textone token at a time. Each time it produces a token, it adds it to the input and runs the entire pipeline again for the next token. This is calledautoregressive generation. How do computers understand human language? Computers only understand numbers. So before any AI can process text, every character must be converted into a number. A = 65 . 💡 Example — How "Hello AI" becomes numbers: Text Input: "Hello AI" Step 1 – Tokenize using BPE : "Hello", " AI" Step 2 – Assign Token IDs lookup in vocabulary table : "Hello" → 9906 " AI" → 15592 Step 3 – Model Input: 9906, 15592 The model never sees letters — only numbers. 💡 Example — One-Hot Encoding vs. Embeddings: Vocabulary: cat, dog, fish, car One-Hot sparse, no meaning : "cat" → 1, 0, 0, 0 "dog" → 0, 1, 0, 0 "fish" → 0, 0, 1, 0 "car" → 0, 0, 0, 1 Problem: "cat" and "dog" look equally different from each other as "cat" and "car". No notion of similarity. Embedding dense, captures meaning : "cat" → 0.82, -0.15, 0.47 ← close to "dog" "dog" → 0.79, -0.12, 0.51 ← close to "cat" "fish" → 0.55, 0.33, 0.42 ← animal but different "car" → -0.71, 0.88, -0.22 ← very different from animals How does AI "know" what to say next? AI models do not look up facts in a database. Instead, they predict the most probable next token based on everything they have processed so far. This is purely a probability problem. 1.0 100% . P word n | word 1, word 2, ..., word n-1 . This chain of dependencies is what makes language models "contextual." softmax logits / T . 0.2 sharpens the distribution — the top word dominates. 1.5 flattens the distribution — more words become viable. 0 → always picks the single most probable word called K most likely words. This prevents very unlikely words from ever being chosen. P e.g., 90% . This is a more flexible and nuanced approach. N best partial sequences at each step and choose the best overall. Slower, but often produces higher-quality output for tasks like translation. 💡 Example — Predicting the next word for "The sky is ": Model's probability scores: "blue" → 55% "clear" → 25% "cloudy" → 15% "falling" → 3% "delicious"→ 0.1% Temperature = 0.2 low → almost always picks "blue" safe, predictable Temperature = 1.5 high → might pick "cloudy" or even "falling" creative Top-K = 2 → only considers "blue" and "clear" Top-P = 0.80 → only considers "blue" + "clear" 80% combined 📊 Top-k vs. Top-p Comparison: Aspect Top-k Top-p Nucleus Selection methodFixed number of tokens Dynamic based on probability mass AdaptabilityPoor Excellent When model is confidentStill forces k tokens Uses very few tokens When model is uncertainLimits to k tokens Can use many tokens Risk of bad tokensMedium Lower Best forSimple control Natural, high-quality text 💡 Example — Greedy vs. Beam Search: Input: "I want to" Greedy pick best at each step : Step 1: "go" 40% → "I want to go" Step 2: "to" 35% → "I want to go to" Step 3: "the" 50% → "I want to go to the" Result: "I want to go to the" — decent but narrow Beam Search track top 3 paths : Path A: "go" 40% → "go home" 18% → score: 7.2 Path B: "eat" 30% → "eat dinner" 25% → score: 7.5 ← winner Path C: "know" 20% → "know more" 15% → score: 3.0 Result: "I want to eat dinner" — better overall sentence How does AI understand meaning, not just words? A computer cannot inherently understand that "King" and "Queen" are related, or that "cat" and "dog" are both animals. Embeddings solve this problem. 0.2, -0.5, 0.8, ... . In AI, every word or concept is converted into a vector with hundreds or thousands of numbers. Each number in the vector represents some learned aspect of the word's meaning. 1.0 means identical direction very similar , 0 means unrelated, and -1 means opposite. "Paris" - "France" + "Germany" ≈ "Berlin" . This is how AI understands relationships between concepts. 💡 Example — Words as coordinates in space: Imagine a 2D map where similar words are placed close together: Royalty Axis ↑ Queen • | • King | ──────────┼──────────→ Gender Axis | Woman • | • Man | "King" - "Man" + "Woman" ≈ "Queen" The math works because similar concepts cluster together in vector space. 💡 Example — Cosine Similarity in practice: Embedding for "dog": 0.8, 0.3, -0.1 Embedding for "puppy": 0.75, 0.35, -0.05 Embedding for "laptop": -0.2, 0.9, 0.6 cosine similarity "dog", "puppy" = 0.99 → very similar ✓ cosine similarity "dog", "laptop" = 0.12 → very different ✗ This is how a search engine knows that a query for "puppy care" should also return results about "dog health" — even though the exact words don't match. The math inside a neural network. At its core, a neural network is just a series of mathematical functions applied one after another. y = wx + b x = the input your data w = weights values the model learns during training to adjust its behavior b = bias an offset value that helps the model fit the data better y = the output the model's prediction y = w₁x₁ + w₂x₂ + w₃x₃ + ... + b Σ : y = Σ wᵢxᵢ + b Common Activation Functions: | Function | Formula | Range | Used For | |---|---|---|---| | ReLU | max 0, x | 0, ∞ | Hidden layers most common | | Sigmoid | 1 / 1 + e⁻ˣ | 0, 1 | Binary classification outputs | | Tanh | eˣ - e⁻ˣ / eˣ + e⁻ˣ | -1, 1 | Hidden layers, LSTMs | | Softmax | eˣⁱ / Σeˣʲ | 0, 1 , sums to 1 | Multi-class outputs | | GELU | x · Φ x | -0.17, ∞ | Transformers GPT, BERT | wx + b + activation. The output of one layer becomes the input to the next. 💡 Example — Predicting house price y = wx + b : Input x : 150 house size in m² Weight w : 0.5 learned: each m² adds $500 Bias b : 20 learned: base price is $20k y = 0.5 × 150 + 20 = 95 → Predicted price: $95,000 During training, 'w' and 'b' are adjusted thousands of times until predictions match the real prices in the dataset. 💡 Example — Why non-linearity activation matters: Without activation just stacking y = wx + b : Layer 1: y = 2x + 1 Layer 2: y = 3 2x + 1 + 5 = 6x + 8 → Still just a straight line No matter how many layers. With ReLU activation: Layer 1: y = ReLU 2x + 1 = max 0, 2x + 1 Layer 2: y = 3 · max 0, 2x + 1 + 5 → Now the network can model curves, bends, and complex shapes This is why we need activation functions — they let the network learn patterns that are NOT simple straight lines. Understanding the shape of your data. Statistics are used to analyze and understand the data that is used to train AI models. Knowing the statistical properties of your data directly impacts model quality. 0 . x new = x - min / max - min x new = x - mean / σ 💡 Example — Normal Distribution of user response lengths: Most users write medium-length messages ↓ | ▂▄█▄▂ | ▂██████▂ | ▄██████████▄ | ▄████████████████▄ 5 10 20 30 40 words ↑ ↑ Very Very long short rare rare 68-95-99.7 Rule: 68% of messages are 15-25 words within 1σ of mean=20 95% of messages are 10-30 words within 2σ 99.7% are 5-35 words within 3σ 💡 Example — Why Normalization matters: Raw Features for a model predicting salary: Age: 25, 30, 45, 50 ← range: 25-50 Years of Exp: 2, 5, 18, 22 ← range: 2-22 GitHub Commits: 100, 500, 8000, 12000 ← range: 100-12000 Without normalization: the model thinks "GitHub Commits" is 240x more important than "Age" just because the numbers are bigger After Min-Max Normalization all become 0-1 : Age: 0.0, 0.2, 0.8, 1.0 Years of Exp: 0.0, 0.15, 0.8, 1.0 GitHub Commits: 0.0, 0.034, 0.66, 1.0 Now all features are on equal footing ✓ How AI handles data at massive scale. Training AI on millions of data points simultaneously requires extremely efficient computation. Matrices make this possible. Scalar, Vector, Matrix, Tensor: 5 . 1, 2, 3 . Matrix Multiplication: The core operation in neural networks. When a model processes a batch of inputs through a layer, it's performing matrix multiplication: Output = Input × Weights + Bias . GPU Parallelism: Graphics Processing Units GPUs are extremely efficient at performing thousands of matrix operations simultaneously in parallel , which is why they are essential for AI training. A CPU processes operations one-by-one; a GPU processes entire matrices at once. Dot Product: The sum of element-wise multiplication of two vectors. It measures how "aligned" two vectors are. This is the fundamental building block of attention. 1, 2, 3 · 4, 5, 6 = 1×4 + 2×5 + 3×6 = 32 Transpose: Flipping a matrix so rows become columns and vice versa. Written as Aᵀ . Essential for many neural network operations e.g., computing attention scores . Softmax Function: A function applied to the final output layer of a model that converts a list of raw scores logits into a probability distribution all values sum to 1 , making it easy to select the most likely prediction. Attention Mechanism Query, Key, Value : The core of the Transformer architecture. This is perhaps the most important concept in modern AI. Attention Q, K, V = softmax QKᵀ / √dₖ × V QKᵀ = dot product of queries and keys → produces attention scores how relevant is each token to every other token / √dₖ = scaling factor to prevent scores from getting too large dₖ = dimension of key vectors softmax ... = normalizes scores to probabilities × V = weighted sum of values using those probabilities N times in parallel with different learned projections "heads" , then concatenates the results. This lets the model focus on different types of relationships simultaneously e.g., one head tracks grammar, another tracks meaning, another tracks position . Positional Encoding: Since Transformers process all tokens simultaneously not sequentially like RNNs , they have no inherent sense of word order. Positional encodings are special vectors added to the input embeddings to tell the model the position of each token in the sequence. 💡 Example — Matrix Multiplication in a neural network layer: Input batch 2 sentences, 3 features each : Weights 3 inputs → 2 outputs : X = | 1 2 3 | W = | 0.5 0.1 | | 4 5 6 | | 0.3 0.7 | 2×3 | 0.8 0.2 | 3×2 Output = X × W | 1×0.5+2×0.3+3×0.8 1×0.1+2×0.7+3×0.2 | | 3.5 2.1 | | 4×0.5+5×0.3+6×0.8 4×0.1+5×0.7+6×0.2 | = | 8.3 5.1 | 2×2 2×2 Both sentences processed AT THE SAME TIME — this is why GPUs are fast. 💡 Example — Softmax converting raw scores to probabilities: Raw model scores logits : "cat" → 3.2 "dog" → 1.8 "car" → 0.5 After Softmax: "cat" → 68% ← highest probability, model picks this "dog" → 26% "car" → 6% Total = 100% ✓ 💡 Example — Attention in the sentence "The animal didn't cross the street because itwas too tired": When the model processes "it", attention helps it look back: "The" 2% "animal" 60% "didn't" 3% "cross" 5% "the" 2% "street" 8% "because" 5% "it" 15% The model correctly identifies "it" = "animal", not "street" — because attention assigns higher weight to "animal". 💡 Example — Multi-Head Attention simplified : Sentence: "The bank was flooded after the river overflowed" Head 1 Grammatical : "bank" attends to → "was" subject-verb Head 2 Semantic : "bank" attends to → "river" meaning = river bank Head 3 Positional : "bank" attends to → "The" nearby context Combined: The model understands "bank" = river bank, not financial bank. Multiple perspectives = better understanding. How does a model actually learn? Learning in AI is the process of gradually adjusting the model's internal parameters weights to make better predictions. Calculus is the tool used to determine how to adjust them. Common Loss Functions: | Loss Function | Used For | What It Measures | |---|---|---| | MSE Mean Squared Error | Regression predicting numbers | Average of squared differences | | Cross-Entropy Loss | Classification predicting categories | Difference between predicted and actual probability distributions | | Binary Cross-Entropy | Yes/No classification | Cross-Entropy for 2 classes | | Contrastive Loss | Embedding models | Distance between similar vs. dissimilar pairs | Derivative / Gradient: The derivative tells us the rate of change — how much the loss changes if we slightly adjust a weight. The gradient is a vector of all partial derivatives across all weights. Backpropagation: The algorithm that efficiently calculates gradients for every weight in the network by working backward from the output layer to the input layer, using the chain rule of calculus. y = f g x , then dy/dx = f' g x × g' x . In a deep network with many layers, the chain rule is applied repeatedly to propagate the error signal back through each layer. Gradient Descent: The core optimization algorithm. After calculating the gradients, we take a small step in the direction that reduces the loss. Repeat this process millions of times, and the model progressively improves. Variants of Gradient Descent: | Variant | Description | Pros/Cons | |---|---|---| | Batch GD | Uses ALL training data per step | Accurate but very slow on large datasets | | Stochastic GD SGD | Uses ONE random sample per step | Very fast but noisy/unstable | | Mini-Batch GD | Uses a small batch e.g., 32 samples | Best balance — standard in practice | | Adam | Adaptive learning rate per parameter | Most popular optimizer; fast convergence | Learning Rate α : A hyperparameter that controls how large each step in gradient descent is. Too large, and the model overshoots and may never converge. Too small, and training is extremely slow. Vanishing / Exploding Gradients: In very deep networks, gradients can become extremely small vanishing or extremely large exploding as they pass through many layers during backpropagation. 💡 Example — Gradient Descent as rolling a ball downhill: Imagine a valley the loss landscape : Loss | \ / | \ / | \ / | \ / | \ / ← Minimum loss best model weights └──────────────── Weights The "ball" = model's current weights Gradient = slope at the ball's current position which way is "down"? Step size = Learning Rate Too large a step → overshoots the valley, bounces around forever Too small a step → takes forever to reach the bottom Just right → smoothly rolls to the minimum ✓ 💡 Example — Backpropagation Chain Rule applied to a 3-layer network : Forward Pass: Input → Layer 1 → Layer 2 → Layer 3 → Prediction → Loss = 5.2 Backward Pass backpropagation : Loss = 5.2 ← How much did Layer 3 contribute to this error? → adjust Layer 3 weights ← How much did Layer 2 contribute? → adjust Layer 2 weights ← How much did Layer 1 contribute? → adjust Layer 1 weights Each layer's contribution is calculated using the Chain Rule: ∂Loss/∂w₁ = ∂Loss/∂Layer3 × ∂Layer3/∂Layer2 × ∂Layer2/∂w₁ This is how error signals "propagate back" through the network. Measuring uncertainty in AI. Information theory provides mathematical tools to quantify how much uncertainty exists in a probability distribution — crucial for evaluating model quality. Entropy H : Measures the average amount of uncertainty or "randomness" in a system. H X = -Σ p x × log₂ p x Cross-Entropy: Measures the difference between the true distribution and the predicted distribution. This is the most commonly used loss function for classification tasks and language models. H P, Q = -Σ p x × log₂ q x where P is the true distribution and Q is the predicted. Perplexity: A direct measure of how well a language model predicts a sample of text. Mathematically, it's 2^ cross-entropy . KL Divergence Kullback-Leibler Divergence : Measures the difference between two probability distributions. It is NOT symmetric: KL P||Q ≠ KL Q||P . KL P||Q = Σ p x × log p x / q x Mutual Information: Measures how much knowing one variable tells you about another. If two variables share a lot of mutual information, they are strongly related. Used in feature selection and understanding what a model has learned. 💡 Example — Entropy comparison: Fair coin flip: P heads = 0.5, P tails = 0.5 H = - 0.5 × log₂ 0.5 + 0.5 × log₂ 0.5 = 1.0 bit → Maximum uncertainty could be either outcome Biased coin: P heads = 0.99, P tails = 0.01 H = - 0.99 × log₂ 0.99 + 0.01 × log₂ 0.01 = 0.08 bits → Very low uncertainty almost always heads AI Application: When a model is well-trained, its predictions should have LOW entropy on correct answers. 💡 Example — Perplexity comparison: Sentence: "The cat sat on the " Good Model low perplexity : "mat" → 60%, "floor" → 20%, "roof" → 10% ... Model is NOT surprised. Perplexity ≈ 5 Bad Model high perplexity : "mat" → 8%, "democracy" → 7%, "blue" → 6% ... Model is very confused. Perplexity ≈ 500 Lower perplexity = better language understanding ✓ 💡 Example — KL Divergence in RLHF: Original Model before fine-tuning : P "thank you" = 30%, P "thanks" = 25%, P "ok" = 20% Fine-tuned Model after RLHF : Q "thank you" = 80%, Q "thanks" = 15%, Q "ok" = 2% KL P||Q = 0.82 → Significant drift from original behavior If KL threshold → apply penalty to prevent the model from becoming too different could lose general capabilities Putting it all together — how a model actually trains from start to finish. Understanding each math concept individually is important, but the real power comes from seeing how they all work together in the training loop: ┌─────────────────────────────────────────────────────────────────┐ │ THE TRAINING LOOP │ │ │ │ 1. INPUT │ │ Raw text → Tokenize → Token IDs → Embeddings vectors │ │ Encoding BPE Lookup Learned vectors │ │ │ │ 2. FORWARD PASS │ │ Embeddings + Positional Encoding │ │ → Multi-Head Attention Q, K, V + Softmax │ │ → Feed-Forward layers wx + b → Activation → wx + b │ │ → Repeat N times e.g., 96 layers for GPT-4 │ │ → Final Softmax → Probability distribution over vocab │ │ Linear Algebra + Algebra + Probability │ │ │ │ 3. COMPUTE LOSS │ │ Compare prediction vs. actual next token │ │ Loss = Cross-Entropy predicted probs, actual token │ │ Information Theory + Statistics │ │ │ │ 4. BACKWARD PASS Backpropagation │ │ Calculate gradients for ALL weights using Chain Rule │ │ ∂Loss/∂w for every weight in every layer │ │ Calculus │ │ │ │ 5. UPDATE WEIGHTS │ │ w new = w old - learning rate × gradient │ │ Using Adam or SGD optimizer │ │ Gradient Descent / Optimization │ │ │ │ 6. REPEAT steps 1-5 for millions of batches │ │ until loss is sufficiently low │ └─────────────────────────────────────────────────────────────────┘ Understanding these common issues will save you hours of debugging: | Concept | What Happens | Symptom | Solution | |---|---|---|---| Overfitting | Model memorizes the training data instead of learning general patterns | Training loss ↓, Validation loss ↑ | Dropout, Data Augmentation, Early Stopping, Regularization | Underfitting | Model is too simple to capture the patterns in the data | Both training and validation loss remain high | Increase model size, Train longer, Better features | 💡 Example — Overfitting vs. Underfitting: Imagine fitting a curve through data points: Underfitting: Good Fit: Overfitting: straight line through smooth curve through wiggly line through scattered points the general trend EVERY point exactly • • • • •──• ───────── • ╱ ╲ ╱ ╲ • • • • • • ╲╱ • "Too simple" "Just right" "Memorized noise" High train error Low train error Zero train error High test error Low test error High test error 0 sparse model . Useful for feature selection. 💡 Example — Dropout: Normal forward pass: With Dropout 20% : N1 ── N4 ── N7 N1 ── ── N7 ← N4 turned off N2 ── N5 ── N8 ── N5 ── N8 ← N2 turned off N3 ── N6 ── N9 N3 ── N6 ── ← N9 turned off Each training step randomly disables different neurons. This prevents the network from "memorizing" via specific neurons and forces it to learn more robust, general patterns. | Concept | What it powers in AI | |---|---| Number Systems & Encoding | How text is converted into data a model can process | Probability | How models generate and sample their responses | Vectors & Embeddings | How AI understands semantic meaning and similarity | Algebra & Activation | The core computations inside every neural network layer | Statistics & Normalization | Analyzing data quality and ensuring stable training | Linear Algebra & Attention | Enabling massively parallel GPU computation and context understanding | Calculus & Optimization | How models learn from mistakes and improve over time | Information Theory | Measuring and evaluating model confidence and quality | Training Loop | How all concepts connect into an end-to-end learning system | Regularization | Preventing models from memorizing data instead of learning | For Aspiring AI Engineers:Mastering these fundamentals is more important than jumping directly into complex model architectures. Once you understandwhythe math works, complex topics like Transformers and fine-tuning become significantly easier to learn. Once you are comfortable with the concepts above, consider exploring these more advanced mathematical topics used in AI research: