Maths for machine learning

Four areas cover almost everything

  1. Linear algebra: vectors, matrices, dot products, matrix multiplication. Data and neural network layers are all matrices, so this is the most important one.
  2. Calculus: derivatives, partial derivatives, the chain rule. This is how models learn (gradient descent and backpropagation).
  3. Probability and statistics: distributions, expectation, variance, Bayes’ rule. Needed for loss functions, uncertainty, and evaluating models.
  4. Optimization: gradient descent and its variants. It’s mostly calculus applied to “how do I make the error smaller?”

You do not need a full maths degree to start. You need a small working kit: enough linear algebra to read a layer, enough calculus to see how training moves, and enough probability to understand a loss.

Most of machine learning is this loop: make a prediction, measure the error, then move the parameters a little to make the error smaller. The symbols on the next pages are the shorthand for that loop.

Proofs can wait. First you want to know what a formula is asking.

Reading order

  1. ML math symbols, explained easy — start here. One card for each common symbol (\(\nabla\), \(\mathbb{E}\), \(\odot\), a norm, \(\arg\max\), and a few extras), with a tiny numerical example and a NumPy line.
  2. Linear algebra essentials — vectors, matrices, the dot product, and why a neural network layer is a matrix multiply.
  3. Calculus essentials — slope, partial derivatives, the gradient, the chain rule, and one step of gradient descent. Ties in with the gradient primer on the activation-functions page.
  4. Probability and statistics essentials — expectation, variance, Bayes, likelihood, and why a cross-entropy loss is an average over the data. Ties in with Naïve Bayes.
  5. Optimization essentials — the loss as a landscape, the update \(\theta \leftarrow \theta - \eta\nabla L\), the learning rate, mini-batch, momentum, and Adam.

flowchart TD
  A[Symbols] --> B[Linear algebra]
  A --> C[Calculus]
  A --> D[Probability and statistics]
  C --> E[Optimization]

Read them in any order if you only need one idea. If you are new, the list above is the shortest path.

See also

Back to top