While working on machine learning projects, it might look like a world full of algorithms, datasets, and model tuning — but behind everything lies a powerful mathematical foundation. It is essential to understand the math behind Machine Learning to understand and work with data and models effectively. The key components you must master in mathematics for machine learning are as follows:
If you’ve ever wondered how ML models magically make predictions, they rely heavily on mathematics. I’ve written a series of blogs explaining the core concepts of mathematics for machine learning. If you are someone who’s working with ML models or simply interested to learn about them, feel free to check it out.
List: Math behind ML | Curated by Rajendran S | Medium
In this blog, we’ll look into the need for calculus in machine learning. You don't need to be a math expert; you have to connect the dots and make sense of things. I’ll try to break down the concepts that every machine learning practitioner must understand. By the end of this blog, you’ll see calculus as not an intimidating thing, but as a toolkit for powerful ML models and deep learning systems.
Calculus is an important tool used in building machine learning algorithms and models. It offers a mathematical framework for describing how machines learn and optimize their performance. It is mainly used in optimization, understanding algorithms, and function approximation.
Calculus might seem like a tedious topic, but it is everywhere. While we rarely write code on differentiation or integration, the ML algorithms have theoretical roots in calculus. You need calculus to understand algorithms to their roots. It also helps to read and understand research papers in a better way.
Calculus is the branch of mathematics that studies change, focusing on limits, functions, derivatives (rate of change), and integrals (accumulation/area). The core concepts in calculus are boiled down into five components:
-
Differentiation
-
Partial derivatives
-
Gradient and Gradient Descent
-
Chain rule
-
Jacobian and Hessian Matrices
Once we master these concepts, we are good to understand the ML terms such as linear regression, logistic regression, neural networks, support vector machines, etc.
Let’s begin with the most fundamental topic: functions. If you’re familiar with programming, you must have come across this term. A function is a rule or method that assigns each input exactly one output
All possible inputs in the domain ‘x’ are known as the independent variable, and the values in the range ‘f(x)’ are called as dependent variable. The function helps to assign a single value in the range to each value of the domain.
Understanding the functions helps in analysing the loss surfaces, activation functions, and transformations in neural networks.
import numpy as npimport matplotlib.pyplot as plt def linear(x, m=2, c=1): return m*x + cdef exponential(x): return np.exp(x)def logarithmic(x): return np.log(x)def sigmoid(x): return 1 / (1 + np.exp(-x))x = np.linspace(-5, 5, 200)y = np.linspace(-5, 5, 100)X, Y = np.meshgrid(x, y)plt.figure(figsize=(15, 10))# linear plt.subplot(2, 3, 1)plt.plot(x, linear(x))plt.title("Linear function")plt.grid(True)# exponential plt.subplot(2, 3, 2)plt.plot(x, exponential(x))plt.title("Exponential function")plt.grid(True)# logarithmplt.subplot(2, 3, 3)plt.plot(x, logarithmic(x))plt.title("Logarithm function")plt.grid(True)# sigmoid plt.subplot(2, 3, 4)plt.plot(x, sigmoid(x))plt.title("Sigmoid function")plt.grid(True)plt.tight_layout()plt.show()
A limit describes the value that a function approaches as the input approaches a particular point. “Limits” refer to the boundary or maximum level of something allowed or possible. A limit is the value of a function that gets arbitrarily close to as the input approaches a particular number.
Limits are fundamental to understanding derivatives and smooth optimizations. It is fundamentally used to study the gradient descent, activation behavior, continuity, or loss functions, and one-sided limits in ReLU.
The derivative of a function measures how fast the function’s output changes with respect to its input.
In machine learning, derivatives are used to describe the gradient of the loss function, which is the direction in which the weights are updated. It is mainly used in backpropagation in neural networks. Imagine a hiker (model) on a foggy mountain (loss landscape) trying to reach the lowest valley (minimum error).
By repeatedly calculating the derivative and adjusting parameters (weights/biases) based on the gradient, the model iteratively minimizes its error and learns to make better predictions.
Derivatives are used in almost every algorithm for optimization, gradient descent, backpropagation, parameter updates, loss functions, etc. Essentially, derivatives help models learn by telling them how much and in which direction to adjust their internal settings (parameters) to improve accuracy.
The higher-order derivatives, such as the second derivative, help to measure the curvature of the function.
Second derivative helps to adjust the learning rates adaptively (eg, in Adam or Newton’s method).
Second-order information, on the other hand, allows us to make a quadratic approximation of the objective function and approximate the right step size to reach a local minimum …
In essence, while first derivatives steer you down the hill (gradient descent), higher-order derivatives (like the Hessian) give you a topographical map to navigate the optimization landscape more intelligently and efficiently. It is mainly used for optimization, Hessian matrix, feature importance, sensitivity, model analysis, and improvement of convergence.
When a function depends on multiple variables, a partial derivative measures the rate of change with respect to one variable, keeping others constant. Partial derivatives are the mathematical engine that allows the ML models to “learn” by iteratively tweaking the parameters to minimize the errors, recognize patterns, and make predictions.
The fundamental principle of the backpropagation algorithm in adjusting the weights in a network is that each weight in a network should be updated in proportion to the sensitivity of the overall error of the network to changes in that weight.
This sensitivity of the overall error of the network to changes in any one particular weight is measured in terms of the rate of change, which, in turn, is calculated by taking the partial derivative of the error with respect to the same weight.
Partial derivatives play a major role in gradient descent, backpropagation, multivariable functions, and activation functions. The usage of partial derivatives is vital in calculating the rate of change, showing how much the loss changes if the single weight changes slightly.
Jacobian contains all the first-order partial derivatives. Jacobians help to understand how small input changes affect the output and are mainly used in backpropagation through layers.
The back-propagation algorithm consists of performing such a Jacobian-gradient product for each operation in the graph.
This means that the backpropagation algorithm can relate the sensitivity of the network error to changes in the weights, through a multiplication by the Jacobian matrix.
It generalizes the gradient and is crucial for backpropagation (calculating gradients), sensitivity analysis, understanding model behavior (like in creative AI), and performing transformations in vector spaces, helping to fine-tune models and analyze complex interactions between inputs and outputs. It is the derivative for functions that map multiple inputs to multiple outputs (e.g., a neural network layer mapping input features to multiple neuron activations).
The Hessian is the square matrix for second-order partial derivatives. It helps to capture the curvature of the loss surface.
The Hessian matrix plays an important role in many machine learning algorithms, which involve optimizing a given function. While it may be expensive to compute, it holds some key information about the function being optimized. It can help determine the saddle points and the local extremum of a function. It is used extensively in training neural networks and deep learning architectures.
Integration is the inverse operation of differentiation. It measures the area under a curve. Integral calculus is fundamentally about summing up infinitesimal parts to find whole quantities.
Integration is particularly used to aggregate values, compute expectations, or normalize probability distributions. In essence, while differential calculus (derivatives) helps find how fast functions change (gradients for optimization), integral calculus helps find cumulative totals, which is vital for understanding probabilities, expectations, and overall model behavior.
Improper integrals are integrals where the limits go to infinity or the function has singularities.
They are used in continuous probability distributions: Gaussian, exponential, Laplace, etc.
With that, we have reached the end of the blog. Calculus is more than a mathematical formality — it intrinsically drives how our models learn, adapt, and optimise. A good understanding of calculus helps to absorb the knowledge of the inner workings of the algorithms we use every day. I’m also planning to write detailed articles on probability and statistics for machine learning, so stay tuned for more posts that will help you strengthen your mathematical foundation for ML.
If this series helped you, consider **following me **to get updates — there’s plenty more interesting stuff on the way.
**Note: **Most of the content is represented as images due to the lack of LaTeX compatibility in Medium.
Calculus You Actually Need for Machine Learning was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.