We’ll fit a Quadratic (degree 2) curve to a simple dataset and compare it to a straight line. You’ll see the code, the math intuition, and a live chart.
We model a smooth curve: Y = 4 + 0.8X + 0.15X². Below are evenly spaced points (no noise) we’ll use in code and chart.
| X | Y | X | Y | X | Y |
|---|---|---|---|---|---|
| 0 | 4.00 | 2 | 6.60 | 4 | 12.40 |
| 6 | 21.40 | 8 | 33.60 | 10 | 49.00 |
| 12 | 67.60 | 14 | 89.40 | 16 | 114.40 |
| 18 | 142.60 | 20 | 174.00 | — | — |
(We’ll fit degree 1 and degree 2 models; degree 2 should match this curve almost perfectly.)
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
# Deterministic curved data (matches the chart)
X = np.arange(0, 21).reshape(-1, 1) # 0,1,2,...,20
y = (4 + 0.8*X.flatten() + 0.15*(X.flatten()**2)).astype(float)
df = pd.DataFrame({"X": X.flatten(), "Y": y})
print(df.head())
What’s happening? We generate a clean curved dataset using the formula above so your Python results match the JS chart exactly.
def fit_poly_degree(X, y, degree):
poly = PolynomialFeatures(degree=degree, include_bias=False)
X_poly = poly.fit_transform(X) # [X, X^2, X^3, ...]
model = LinearRegression().fit(X_poly, y)
y_hat = model.predict(X_poly)
r2 = r2_score(y, y_hat)
return model, poly, y_hat, r2
models = {}
for d in [1, 2, 3]:
mdl, fe, yhat, r2 = fit_poly_degree(X, y, d)
models[d] = {"model": mdl, "fe": fe, "yhat": yhat, "r2": r2}
print(f"Degree {d} → R^2: {r2:.4f}")
Why this step? We learn three models and compute R² for each. For this dataset, degree 2 is the true generating curve, so it should yield R²≈1.00.
plt.figure(figsize=(8,6))
plt.scatter(X, y, label="Data", edgecolor="black", color="gold")
for d, sty in zip([1,2,3], ["--", "-", ":"]):
plt.plot(X, models[d]["yhat"], sty, linewidth=2, label=f"Degree {d} (R²={models[d]['r2']:.3f})")
plt.title("Polynomial Regression: Linear vs Quadratic vs Cubic")
plt.xlabel("X"); plt.ylabel("Y"); plt.legend(); plt.grid(True, ls="--", alpha=.4)
plt.show()
How to read this: Gold points are actual data; dashed/solid/dotted lines are deg 1/2/3 fits. The deg 2 line should pass right through the points.
# Choose degree (2 is the correct curve here)
degree = 2
fe = models[degree]["fe"]
mdl = models[degree]["model"]
# Predict for any X values
X_new = np.array([[5], [12], [20]])
X_new_poly = fe.transform(X_new)
y_pred = mdl.predict(X_new_poly)
for xval, yval in zip(X_new.flatten(), y_pred):
print(f"Predicted Y at X={xval}: {yval:.2f}")
What you’ll see: Predictions that match the formula very closely.
For example: X=5 → Y= 4 + 0.8*5 + 0.15*25 = 11.75, X=12 → Y=67.60, X=20 → Y=174.00.
R² notably below 1 (e.g., ~0.95–0.98).R² ≈ 1.0000, line passes through all points.R² ≈ 1 but adds an unnecessary term (β₃ close to 0).Gold = data (Y = 4 + 0.8X + 0.15X²), Orange = degree 2 fit (same curve).
We’ll fit a Quadratic (degree 2) polynomial to a simple Ad Spend → Sales dataset, then print the equation, R², make predictions, and visualize the result.
| Ad Spend (X) | Sales (Y) |
|---|---|
| 0 | 5 |
| 2 | 15 |
| 4 | 40 |
| 6 | 80 |
| 8 | 130 |
| 10 | 200 |
| 12 | 290 |
| 14 | 400 |
| 16 | 530 |
| 18 | 680 |
| 20 | 850 |
(Notice how sales accelerate as spend increases → the relationship is curved.)
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
# Transform X → [X, X²]
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)
# Train model
model = LinearRegression().fit(X_poly, y)
y_pred = model.predict(X_poly)
print("Equation: Sales = {:.2f} + {:.2f}X + {:.2f}X²"
.format(model.intercept_, model.coef_[0], model.coef_[1]))
print("R² score:", r2_score(y, y_pred))
Gold = actual data, Orange = Polynomial Regression (degree=2)
Same dataset, three models. Notice how Degree 1 underfits, Degree 2 fits well, and Degree 5 starts to wiggle (risk of overfitting).
Start simple. If a straight line underfits, try degree 2 or 3. High degrees can look great on training data but overfit and perform poorly on new data. Use train/test or cross-validation to choose.
Pick a polynomial degree and see how it fits the train data vs the test data. Use Re-split to shuffle the split and observe how high degrees can overfit.
Degree 1 often underfits; Degree 2 fits this dataset well. Very high degrees (e.g., 5+) may look perfect on train but can overfit and show a drop in R² (Test). Always validate with a train/test split.
Polynomial Regression is powerful—but easy to misuse. Use this checklist to avoid underfit and overfit, and keep your results reliable on new data.
If you must try a higher degree (e.g., 5), add Ridge to shrink extreme coefficients and reduce wiggle.
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.linear_model import Ridge
from sklearn.pipeline import Pipeline
from sklearn.metrics import r2_score
import numpy as np
# X, y from Section 3
X = np.array([[0],[2],[4],[6],[8],[10],[12],[14],[16],[18],[20]])
y = np.array([5,15,40,80,130,200,290,400,530,680,850])
# Build a safe high-degree pipeline
model = Pipeline([
("poly", PolynomialFeatures(degree=5, include_bias=False)),
("scaler", StandardScaler(with_mean=False)), # scale polynomial terms
("ridge", Ridge(alpha=10.0)) # try alpha in [0.1, 1, 10, 100]
])
model.fit(X, y)
y_hat = model.predict(X)
print("R² (deg=5 + Ridge):", round(r2_score(y, y_hat), 4))
Tip: Tune alpha (regularization strength) with cross-validation to balance bias–variance.
The best curve is the one that generalizes. Validate on unseen data, keep degree modest, and add regularization if the curve starts to wiggle.
Polynomial Regression shines when growth is non-linear — diminishing returns, U-shapes, or smooth curves. Below are practical applications, a mini project you can code quickly, and interview questions to test yourself.
Ad spend → conversions with diminishing returns; pricing vs demand curves; funnel drop-offs.
Throughput vs utilization (non-linear queueing effects); learning curves; maintenance wear patterns.
Dose–response curves; growth trajectories; non-linear risk scoring.
Calibration curves; drag vs speed; battery discharge profiles.
Ad_Spend (₹) and Conversions.R² or RMSE.Ad_Spend.
import numpy as np, pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error
import matplotlib.pyplot as plt
# Example data (replace with your weekly data)
df = pd.DataFrame({
"Ad_Spend":[0,2,4,6,8,10,12,14,16,18,20],
"Conversions":[5,15,40,80,130,200,290,400,530,680,850]
})
X = df[["Ad_Spend"]].values; y = df["Conversions"].values
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=42)
best = None
for deg in [1,2,3]:
poly = PolynomialFeatures(degree=deg, include_bias=False)
Xtr_poly = poly.fit_transform(Xtr)
mdl = LinearRegression().fit(Xtr_poly, ytr)
ytr_hat = mdl.predict(Xtr_poly)
yte_hat = mdl.predict(poly.transform(Xte))
r2_tr = r2_score(ytr, ytr_hat)
r2_te = r2_score(yte, yte_hat)
rmse_te = mean_squared_error(yte, yte_hat, squared=False)
print(f"deg={deg} R2_train={r2_tr:.3f} R2_test={r2_te:.3f} RMSE_test={rmse_te:.2f}")
if (best is None) or (r2_te > best["r2_te"]):
best = {"deg":deg, "poly":poly, "mdl":mdl, "r2_te":r2_te}
# Final model + plot
xx = np.linspace(df.Ad_Spend.min(), df.Ad_Spend.max(), 200).reshape(-1,1)
yy = best["mdl"].predict(best["poly"].transform(xx))
plt.figure(figsize=(8,5))
plt.scatter(Xtr, ytr, c="gold", edgecolor="black", label="Train")
plt.scatter(Xte, yte, c="silver", edgecolor="black", label="Test")
plt.plot(xx, yy, c="orange", lw=2, label=f"Best curve (deg={best['deg']})")
plt.xlabel("Ad Spend (₹)"); plt.ylabel("Conversions")
plt.title("Polynomial Regression — Train vs Test")
plt.legend(); plt.grid(True, ls="--", alpha=.4); plt.show()
PolynomialFeatures, loop degrees, track R²/RMSE on test set.Pipeline.Use Polynomial Regression for smooth, non-linear trends. Pick a modest degree with train/test validation, and prefer a Pipeline for clean, leak-free modeling.
Quick answers to common doubts — from how it works to when to use something else.
Yes. It’s linear in the coefficients (β’s). We expand features to [X, X², X³, …] and then run a normal Linear Regression on those columns.
When the relationship between X and Y is a smooth curve (e.g., diminishing returns, U‐shape). If a straight line underfits and residuals show curvature, try degree 2 or 3.
Use a train/test split or cross-validation. Start with degree 2, compare test R² / RMSE across degrees, and pick the one that performs best on unseen data.
Rule of thumb: have at least 10–20 data points per coefficient. Degree d uses d+1 coefficients → aim for ≥ (d+1)×10 samples for stability.
Higher degrees add flexibility to pass near every point—including noise. You’ll see excellent train scores but poor test scores. Keep degrees low and validate.
For plain OLS it’s optional, but with high degrees or when you add regularization (Ridge/Lasso), scaling helps stabilize coefficients and training.
You can use polynomial expansion on multiple inputs (e.g., X1, X2) to create interaction terms like X1·X2 and powers like X1², but feature count grows fast—use with care.
Polynomial: sums of powers of X (good for smooth curves). Exponential: Y grows/decays proportionally (use log transform on Y). Power law: Y = a·Xᵇ (use log–log transform). Choose based on domain theory & residual checks.
If relationships are piecewise, have thresholds, or involve many categorical features, try Decision Trees / Random Forest / Gradient Boosting (XGBoost/LightGBM) instead of high-degree polynomials.
Yes. Use Ridge (L2) or Lasso (L1) with polynomial features (via a Pipeline), tuning alpha via CV. This reduces coefficient blow-ups and improves generalization.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV, KFold
pipe = Pipeline([
("poly", PolynomialFeatures(include_bias=False)),
("scale", StandardScaler(with_mean=False)),
("ridge", Ridge())
])
params = {
"poly__degree": [1,2,3,4],
"ridge__alpha": [0.1, 1, 10, 100]
}
cv = KFold(n_splits=5, shuffle=True, random_state=42)
grid = GridSearchCV(pipe, params, scoring="r2", cv=cv)
grid.fit(X, y)
print("Best:", grid.best_params_, "R2:", round(grid.best_score_, 3))
Use the best degree + alpha for a stable, generalizable curve.
You just learned how Polynomial Regression extends Linear Regression by adding powers of X to fit smooth curves. With careful degree selection and validation, it becomes a powerful tool for non-linear trends in business and analytics.
Refresh the basics of straight-line modeling, assumptions, and interpretation.
हिंदी में आसान तरीके से लीनियर रिग्रेशन — शुरुआती सीखने वालों के लिए बढ़िया संसाधन।
Move on to Logistic Regression in Python to model Yes/No outcomes (conversion, churn, fraud). We’ll cover data prep, decision boundary, evaluation (AUC/ROC), and regularization—Vista Academy style.
Questions & options are shuffled every time. Navigate with Prev/Next. Submit at the end to see your score and explanations. Retake anytime.
Vista Academy • Polynomial Regression MCQ • Shuffled + Paginated