scikit-learn: regression
Fit, predict, split, and score regression models with scikit-learn.
Why it matters: fit/predict is the interface behind most ML tools. Regression teaches overfitting with clear numbers.
Basics
The estimator interface
scikit-learn models share three steps:
model = Estimator(...)model.fit(X_train, y_train)model.predict(X_test)
X is 2D (rows = samples, columns = features). y is 1D.
Even one feature needs .reshape(-1, 1).
Tiny example
Fit a one-feature line with plain NumPy math.
import numpy as np
x = np.array([1.0, 2.0, 3.0])
y = np.array([2.0, 4.0, 6.1])
w = float(np.dot(x, y) / np.dot(x, x))
print("slope", round(w, 3))Basics
Train/test split
Never score on the same data you trained on.
train_test_split(X, y, test_size=0.2, random_state=42) holds out 20%.
Fit scalers on train only, then transform test. A Pipeline does this for you.
For time series, split by time, not randomly.
Tiny example
Shuffle indices and split into train and test.
import numpy as np
idx = np.arange(10)
rng = np.random.default_rng(0)
rng.shuffle(idx)
cut = int(0.8 * len(idx))
print("train", idx[:cut], "test", idx[cut:])Basics
Regression metrics
- MAE - average absolute error (easy to explain)
- RMSE - penalises large errors more
- R² - share of variance explained (1.0 is perfect)
Report MAE with R². A high R² can still mean a large absolute error.
High train score and low test score means overfitting.
Tiny example
Compute MAE on a tiny prediction list.
y_true = [1.0, 2.0, 3.0]
y_pred = [1.1, 1.8, 3.2]
mae = sum(abs(a - b) for a, b in zip(y_true, y_pred)) / len(y_true)
print("mae", round(mae, 3))Then AI / ML
AI / ML examples
Same idea, used in real AI work. Press Try in lab to run it.
AI / ML example
AI / ML: linear regression end to end
First run may download scikit-learn.
In AI / ML: Baselines for latency, cost, and score prediction often start linear.
example.py
Python
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, r2_score
rng = np.random.default_rng(42)
X = rng.uniform(0, 10, 200).reshape(-1, 1)
y = 3.5 * X.ravel() + 12 + rng.normal(0, 2, 200)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)
model = LinearRegression().fit(X_tr, y_tr)
pred = model.predict(X_te)
print("coef", round(float(model.coef_[0]), 3))
print("MAE", round(mean_absolute_error(y_te, pred), 3))
print("R2", round(r2_score(y_te, pred), 4))AI / ML example 2
AI / ML: polynomial overfitting check
High degree can ace train and fail test.
In AI / ML: Compare train and test R² before shipping any regressor.
example.py
Python
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
rng = np.random.default_rng(3)
X = np.sort(rng.uniform(-3, 3, 60)).reshape(-1, 1)
y = 0.5 * X.ravel() ** 3 - 2 * X.ravel() + rng.normal(0, 2.5, 60)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=1)
for degree in [1, 3, 15]:
model = make_pipeline(PolynomialFeatures(degree), LinearRegression()).fit(X_tr, y_tr)
print(degree, "train", round(r2_score(y_tr, model.predict(X_tr)), 3),
"test", round(r2_score(y_te, model.predict(X_te)), 3))Takeaways
- 1Every estimator is fit(X, y) then predict(X); X is 2D and y is 1D.
- 2Split before fitting; fit scalers on training data only.
- 3Report MAE with R²; rising train and falling test means overfitting.