Multiple Regression & Inferenceintermediate

Sampling Distribution and the Classical Linear Model

Adding MLR.6 normality of the errors, $u\sim\mathcal{N}(0,\sigma^2)$ independent of the regressors, to the Gauss-Markov set defines the classical linear model (CLM). Under the CLM the OLS estimators are themselves normally distributed, so the standardized statistics follow exact $t$ and $F$ distributions in any sample size. This exact-distribution result is what licenses the textbook t test, confidence interval, and F test.

Try it yourself

Sampling distribution of the OLS slope

Unbiased means the estimator centres on the truth across MANY samples — not that one sample is right.

Fix a known process y = β₀ + β₁·x + u with u ~ Normal(0, σ). We draw M fresh samples of size n, and for each one compute the OLS slope β̂₁. The blue histogram is the sampling distribution of those estimates; the gold line marks the true β₁. The mean of the estimates lands on it even though no single sample does.

mean of β̂₁ vs true β₁1.992 vs 2.00

mean β̂₁ 1.992true β₁ 2.00SD β̂₁ (empirical SE) 0.184theoretical SE 0.184

True slope β₁2.00

Error SD σ3.00

Sample size n30

Repeated samples M400

Across 400 samples the estimates average 1.992, a gap of just -0.008 from the true 2.00 (0.04 sampling SEs). That is unbiasedness: E[β̂₁] = β₁ across samples, even though individual estimates scatter around it. The spread is the sampling SE: empirical 0.184 tracks the theoretical σ/√Σ(xᵢ−x̄)² = 0.184, and the histogram approaches the normal curve (the CLT).

Setup: x is a fixed evenly-spaced grid on [1, 11] (so Sxx = Σ(xᵢ−x̄)² is known), errors are i.i.d. Normal(0, σ), and the estimator is β̂₁ = Σ(xᵢ−x̄)(yᵢ−ȳ)/Σ(xᵢ−x̄)². Larger n widens Sxx and shrinks the SE; larger σ widens it.

Discuss

Set n small and watch the histogram spread out while its centre stays on the true β₁. If you only ever collected ONE sample, your single estimate could land far out in that spread. So how can an estimator be "unbiased" and yet be wrong in the one sample you actually have? What does unbiasedness promise you, and what does it not?

Why it matters

Normality of $u$ passes straight through to the estimators because each $\hat{\beta}_j$ is a linear combination of the errors. That gives clean, exact small-sample distributions rather than approximations. When normality is doubtful you lean on the large-sample asymptotic results instead.

Formulas

MLR.6 normality (CLM)

u\sim\mathcal{N}(0,\sigma^2)\ \text{independent of}\ (x_1,\dots,x_k)

Bundles zero mean, constant variance, and a normal shape.

Exact distribution of the t ratio

\frac{\hat{\beta}_j-\beta_j}{\mathrm{se}(\hat{\beta}_j)}\ \sim\ t_{\,n-k-1}

Holds exactly under the CLM, with

(n-k-1)

degrees of freedom.

Worked examples

Scenario

You run `regress lwage educ exper` with $n=30$ and want to trust the reported t values exactly.

Solution

With such a small sample the exact $t_{\,n-k-1}$ distribution relies on MLR.6 normality, so a normal-looking residual histogram or a passing `sktest` of residuals supports using the textbook t critical values rather than asymptotic ones.

NoteIn tiny samples the normality assumption does real work; in large samples it is dispensable.

Common mistakes

✗Believing OLS estimation requires normal errors. Estimation and unbiasedness do not; only the exact small-sample t and F distributions do.
✗Thinking MLR.6 is needed in large samples. Asymptotic normality replaces it once $n$ is big.
✗Reading MLR.6 as a claim about the distribution of $y$ unconditionally. It is about the error $u$ conditional on the regressors.
✗Assuming a high $R^2$ implies the errors are normal. Goodness of fit says nothing about the shape of the error distribution.

Revision bullets

•MLR.6 adds normal errors and defines the classical linear model.
•Under the CLM, OLS estimators are normal and t, F statistics have exact t, F distributions.
•Exact small-sample inference relies on MLR.6.
•Large samples replace MLR.6 with asymptotic normality.
•MLR.6 concerns the error $u$ , not the unconditional distribution of $y$ .

Quick check

What does the normality assumption MLR.6 add beyond the Gauss-Markov assumptions?

In the classical linear model, the statistic $(\hat{\beta}_j-\beta_j)/\mathrm{se}(\hat{\beta}_j)$ follows:

Connected topics

MLR assumptions Gauss-Markov Asymptotics t test Conf. intervals F test

Sources

Wooldridge, Introductory Econometrics, Ch. 4
Wooldridge (2019), Introductory Econometrics: A Modern Approach, 7th ed., Sec. 4.1 (CLM)
Hill, Griffiths & Lim, Ch. 3
Hill, Griffiths & Lim (2018), Principles of Econometrics, 5th ed., Ch. 3

How to cite this page

Dr. Phil's Quant Lab. (2026). Sampling Distribution and the Classical Linear Model. Derivatives Atlas. https://phucnguyenvan.com/concept/efm-sampling-distribution

← Back to the atlas See in the network →