Introduction to Calculus for ML Beginner

Calculus is the mathematics of change. In machine learning, we need to understand how small changes in model parameters affect the model's predictions and errors. This understanding is what allows neural networks to learn — by calculating which direction to adjust parameters to reduce errors.

Why Calculus for Machine Learning?

Every time you train a neural network, calculus is working behind the scenes. The training process can be summarized in three calculus-driven steps:

1. Compute the loss

   Measure how wrong the model's predictions are using a loss function (a mathematical function that outputs a single number).

2. Compute the gradient

   Use calculus (specifically, the chain rule) to compute how the loss changes with respect to each model parameter. This is backpropagation.

3. Update parameters

   Adjust each parameter in the direction that reduces the loss. This is gradient descent.

Calculus Concepts in ML

A Taste of What's Coming

Here is a preview of gradient descent — the algorithm that trains every neural network:

import numpy as np

# Simple function: f(x) = x^2 (minimum at x=0)
def f(x):
    return x ** 2

def df(x):
    return 2 * x  # Derivative: tells us the slope

# Gradient descent: start at x=5, find the minimum
x = 5.0
learning_rate = 0.1

for i in range(20):
    gradient = df(x)
    x = x - learning_rate * gradient  # Move opposite to gradient
    print(f"Step {i}: x = {x:.4f}, f(x) = {f(x):.4f}")
# x converges to 0.0 (the minimum)