Lab#FU01-2: Taxi fare, NumPy and scikit-learn

Lab

📘 Lab#FU01-2: Taxi fare, NumPy and scikit-learn

In this lab you will:

  1. Load more receipts, with a pickup fee as well as a rate per kilometre.
  2. Run gradient descent in NumPy for both \(w\) and \(b\).
  3. Plot the loss, and how \(w\) and \(b\) settle.
  4. Compare your result with LinearRegression from scikit-learn.

Open In Colab   Code on GitHub

Lab#FU01-1 learned one number: the fare per kilometre. A real taxi also has a flag fall — the bit you pay before the metres start to run. The model is now

\[ \text{price} = w \cdot \text{km} + b \]

True values on these receipts: \(w = 1.50\), \(b = 2.00\). The program is not told those two numbers. The loop does not change: predict, measure the error, nudge each parameter, repeat.

The editor below is live Python in the browser. The first load takes a few seconds, because it also fetches NumPy and scikit-learn.

1 Creating the algorithm

The question. A taxi charges a rate per kilometre and a fixed pickup fee. We have eight receipts, from 1 km at €3.50 to 12 km at €20. What does 7 km cost? We don’t know the rate or the fee, so we let the receipts teach both to us.

# TODO: find the price of a 7 km trip from eight receipts

Why one number isn’t enough. In Lab#FU01-1 the fare was just price = w · km. That cannot fit these receipts: 1 km at €3.50 would mean a rate of €3.50 / km, but 12 km at €20 would mean about €1.67 / km. We need a second number, the flag fall. The model is now price = w · km + b. Two unknowns: the rate w and the base fare b.

# TODO: price = w * km + b; both w and b are unknown

What we know and what we don’t. We know the kilometres (the input) and the prices (the answers). We store them as NumPy arrays, so we can add and multiply the whole list in one go, with no for loop. We don’t know w or b.

import numpy as np

km    = np.array([1.0, 2.0, 3.0, 5.0, 6.0, 8.0, 10.0, 12.0])  # inputs
price = np.array([3.50, 5.00, 6.50, 9.50, 11.00, 14.00, 17.00, 20.00])  # answers
# TODO: w and b are unknown, the things to learn

A first guess. We have no idea, so we start badly on purpose and watch both numbers improve.

w = 0.0                  # a bad guess on purpose
b = 0.0

Predict. With our guess, what would each trip cost? With w = 0 and b = 0, every trip comes out free. NumPy multiplies the whole array at once.

pred = w * km + b        # [0, 0, 0, 0, 0, 0, 0, 0]

How wrong are we? We compare each prediction with the real receipt, square the gaps, and average them. That number is the loss, and we want it small. NumPy does it in one line.

loss = np.mean((pred - price)**2)

Which way to move? Every prediction was too low, so both w and b must go up. We now have two gradients. The one for w still multiplies by kilometres, because a change in the rate matters more on a long trip. The one for b does not: the flag fall is the same on every ride, short or long.

grad_w = np.mean(2 * (pred - price) * km)
grad_b = np.mean(2 * (pred - price))

A small step. We nudge both numbers a little that way. The learning rate is the step size: too small and we crawl, too big and we bounce past the bottom.

lr = 0.01
w = w - lr * grad_w
b = b - lr * grad_b

Again and again. We predict, measure and nudge, over and over. It takes many more steps than Lab 1: w and b pull on each other, so a step that helps the rate can undo some of the fee, and the other way around. After 500 steps we are close; after 2000 steps we are there.

for step in range(2000):
    # predict, measure, find both directions, step
    ...

When to stop. We stop when the loss barely changes, or after a fixed number of steps.

# TODO: stop when loss < 0.0001, or after N steps

Check the answer. w ≈ 1.50 and b ≈ 2.00, so 7 km costs about €12.50. We also ask scikit-learn’s LinearRegression for the same fit: a trusted reference, in closed form, with no loop. It lands on the same two numbers. Nobody told the program the rate or the fee; it found them from examples. That’s learning.

from sklearn.linear_model import LinearRegression
model = LinearRegression().fit(km.reshape(-1, 1), price)
# model.coef_ ≈ 1.50,  model.intercept_ ≈ 2.00
print(w * 7 + b)         # ≈ 12.50

2 More receipts, then two parameters

Eight rides, all generated from \(1.5 \cdot \text{km} + 2\). The derivative of the mean squared error with respect to \(w\) is the average of \(2 \cdot \text{error} \cdot \text{km}\). With respect to \(b\), it is the average of \(2 \cdot \text{error}\). Start at \(w = 0\), \(b = 0\), and take \(2000\) steps of size \(0.01\). Two parameters take more steps than one.

Try it

Change learning_rate and press Run. Try a smaller rate, or a larger one. You can also try fewer steps and watch \(w\) and \(b\) stop short of 1.50 and 2.00.

After 500 steps the fit is already close (\(w \approx 1.514\), \(b \approx 1.887\)). After 2000 steps, \(w = 1.500004\) and \(b = 1.999971\). LinearRegression lands on the true \(w\) and \(b\). The two lines sit on top of each other. That is the point: your loop and the library are doing the same job. The library is faster and more stable; the loop is the one you can see.

3 What you should notice

Two parameters are coupled, so the walk toward the true \(w\) and \(b\) is slower than Lab 1. A learning rate of \(0.01\) is still safe here, but you need many more steps. In later labs you will scale the inputs and use better updates.

4 See also

Back to top