Standard Error of the Mean – Derivation

The standard error (SE) is an amazingly useful statistical device for defining confidence intervals. In layman terms standard error is measure of how far a sample statistic is from it’s true value. This post will go through the process of deriving the SE of the mean.

SE = \frac{\sigma}{\sqrt{n}}

I have always wanted to dig deeper into where the \sqrt{n} comes from.

The mathematical derivation is pretty straight forward. First, is to note that the mean \bar{X} is a sample mean of a population.

\bar{X} = \frac{\displaystyle\sum_{i=1}^n{X_i}}{n}

We know that the variance is equal to the expected value for the square difference from the mean.

Var(X) = E[(X - E[X])^2] = \sigma^2

Replacing X with \bar{X} yields.

Var(\bar{X}) = E[(\bar{X} - E[\bar{X}])^2]

Var(\bar{X}) = E[(\frac{\sum_{i=1}^n{X_i}}{n} - E[\frac{\sum_{i=1}^n{X_i}}{n}])^2]

Var(\bar{X}) = \frac{1}{n^2}E[(\sum_{i=1}^n{X_i} - E[\sum_{i=1}^n{X_i}])^2]

Var(\bar{X}) = \frac{1}{n^2}E[(\sum_{i=1}^n{X_i} - \sum_{i=1}^nE[\bar{X}])^2]

Var(\bar{X}) = \frac{1}{n^2}\sum_{i=1}^nE[({X_i} - \bar{X})^2]

Var(\bar{X}) = \frac{\sum(\sigma^2)}{n^2}

since \sum_{i=1}^n{\sigma^2} = \sigma^2 +\sigma^2 + ... + \sigma^2 = n\sigma^2

Var(\bar{X}) = \frac{\sigma^2}{n}

\sqrt(Var(\bar{X})) = \frac{\sigma}{\sqrt(n)}

And there we have it.

Most commonly, standard error is a calculated from a sample n for a sample mean \bar{x}.

For example the SE for a 95% confidence interval (alpha of 5%) from a normal distribution would be.

lower limit = \bar{X} - 1.96 * SE
upper limit = \bar{X} + 1.96 * SE

I found this a helpful exercise in gaining confidence if the formula’s I am using. The key was to understand that the SE is looking at the sample mean and not an individual sample value.