Advanced·20 min·0/47 done

From NumPy to PyTorch tensors

Tensors, gradients, and the training loop pattern behind fine-tuning code.

Why it matters: PyTorch is the default for training and fine-tuning. A tensor is like a NumPy array with gradients and a device.

Basics

Arrays and tensor ideas

A PyTorch tensor acts like a NumPy array, plus:

  1. Device placement - move data to GPU with .to("cuda")
  2. Autograd - with requires_grad=True, .backward() computes gradients

Convert with torch.from_numpy(arr) and tensor.numpy().

Common mistake: model on GPU, batch still on CPU - "tensors on different devices".

Tiny example

A NumPy array shows the same shape idea as a tensor.

import numpy as np
x = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
print(x.shape, x.dtype)
print(x * 2)

Basics

One training step

Every training step repeats:

  1. optimizer.zero_grad() - clear old gradients
  2. forward - outputs = model(inputs)
  3. loss - loss = criterion(outputs, targets)
  4. loss.backward() - gradients
  5. optimizer.step() - update weights

For inference: model.eval() and with torch.no_grad():.

Tiny example

One hand-written gradient step on y = w * x.

w = 0.0
x, y = 2.0, 4.0
pred = w * x
w = w - 0.1 * (pred - y) * x
print("w", w)

Basics

Gradient descent in NumPy

Training means: guess parameters, measure error, nudge parameters to reduce error, repeat.

The learning rate is the step size. Too large and loss blows up. Too small and training crawls.

The browser lab uses NumPy to show the same learning process. PyTorch adds automatic derivatives, but the training steps stay the same.

Tiny example

A few NumPy steps toward the right weight.

import numpy as np
x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 6.0])
w = 0.0
for _ in range(20):
    pred = w * x
    w -= 0.1 * float(((pred - y) * x).mean())
print(round(w, 3))

Then AI / ML

AI / ML examples

Same idea, used in real AI work. Press Try in lab to run it.

AI / ML example

AI / ML: fit a line with NumPy GD

Fits y = 3x + 2. This is what .backward() automates.

In AI / ML: Linear regression is the smallest version of a neural training loop.

example.py

Python

import numpy as np

rng = np.random.default_rng(0)
x = rng.uniform(-1, 1, size=(200, 1)).astype(np.float32)
y = 3.0 * x + 2.0 + rng.normal(0, 0.05, size=(200, 1)).astype(np.float32)
w = np.zeros((1, 1), dtype=np.float32)
b = np.zeros((1,), dtype=np.float32)
lr = 0.5

for epoch in range(1, 61):
    pred = x @ w + b
    err = pred - y
    loss = float((err ** 2).mean())
    w -= lr * (2.0 * (x.T @ err) / len(x))
    b -= lr * (2.0 * err.mean(axis=0))
    if epoch % 20 == 0:
        print(f"epoch {epoch} loss={loss:.4f} w={w[0,0]:.3f} b={b[0]:.3f}")
print("learned:", round(float(w[0, 0]), 2), "* x +", round(float(b[0]), 2))

AI / ML example 2

AI / ML: understand the PyTorch loop

See the five training steps in a simple runnable form.

In AI / ML: Fine-tuning scripts use this zero_grad / backward / step pattern.

example.py

Python

training_steps = [
    "zero_grad",
    "forward",
    "loss",
    "backward",
    "step",
]
print(" -> ".join(training_steps))

Takeaways

  • 1A tensor is like a NumPy array plus device and autograd.
  • 2zero_grad, forward, loss, backward, step is the training loop.
  • 3Use model.eval() and torch.no_grad() for inference.
Certificate