Polynomial Regression

Polynomial regression extends the linear regression model by including higher-degree terms of a predictor variable. This allows the model to capture curved relationships between a predictor and an outcome, while retaining the core idea of estimating coefficients within a regression framework. The general form of a polynomial regression model of \(n\)-th degree is:

\[\hat{y} = b_0 + b_1 x + b_2 x^2 + \dots + b_n x^n\]

The following plot shows some example data and allows you to fit polynomial models of varying degree to the data. Please take some time to explore these models:

Learning break

What happens if you increase the order of the polynomial? Can you observe an interesting behavior?

Hint: The data consists of 30 observations.

Click to show solution

As you probably expected, the explained variance (\(R^2\)) increases as the order of the polynomial increases, eventually reaching 1 for a model of degree 29. At this point, the model passes exactly through every single data point.

This does not happen by chance. Any dataset with \(n\) observations can be perfectly interpolated by a polynomial of degree \(n−1\). However, such a high-order model is usually not a good choice in practice, as it was overfit to the data and will generalise poorly to new observations. Although this will be more thoroughly discussed in the psy112 module next semester, we will already cover it a bit in today’s session.