Standard Error of the Mean – Derivation

The standard error (SE) is an amazingly useful statistical device for defining confidence intervals. In layman terms standard error is measure of how far a sample statistic is from it’s true value. This post will go through the process of deriving the SE of the mean.

$SE = \frac{\sigma}{\sqrt{n}}$

I have always wanted to dig deeper into where the $\sqrt{n}$ comes from.

The mathematical derivation is pretty straight forward. First, is to note that the mean $\bar{X}$ is a sample mean of a population.

$\bar{X} = \frac{\displaystyle\sum_{i=1}^n{X_i}}{n}$

We know that the variance is equal to the expected value for the square difference from the mean.

$Var(X) = E[(X - E[X])^2] = \sigma^2$

Replacing $X$ with $\bar{X}$ yields.

$Var(\bar{X}) = E[(\bar{X} - E[\bar{X}])^2]$

$Var(\bar{X}) = E[(\frac{\sum_{i=1}^n{X_i}}{n} - E[\frac{\sum_{i=1}^n{X_i}}{n}])^2]$

$Var(\bar{X}) = \frac{1}{n^2}E[(\sum_{i=1}^n{X_i} - E[\sum_{i=1}^n{X_i}])^2]$

$Var(\bar{X}) = \frac{1}{n^2}E[(\sum_{i=1}^n{X_i} - \sum_{i=1}^nE[\bar{X}])^2]$

$Var(\bar{X}) = \frac{1}{n^2}\sum_{i=1}^nE[({X_i} - \bar{X})^2]$

$Var(\bar{X}) = \frac{\sum(\sigma^2)}{n^2}$

since $\sum_{i=1}^n{\sigma^2} = \sigma^2 +\sigma^2 + ... + \sigma^2 = n\sigma^2$

$Var(\bar{X}) = \frac{\sigma^2}{n}$

$\sqrt(Var(\bar{X})) = \frac{\sigma}{\sqrt(n)}$

And there we have it.

Most commonly, standard error is a calculated from a sample $n$ for a sample mean $\bar{x}$.

For example the SE for a 95% confidence interval (alpha of 5%) from a normal distribution would be.

$lower limit = \bar{X} - 1.96 * SE$
$upper limit = \bar{X} + 1.96 * SE$

I found this a helpful exercise in gaining confidence if the formula’s I am using. The key was to understand that the SE is looking at the sample mean and not an individual sample value.