Unbiased Estimates (Cambridge (CIE) A Level Maths: Probability & Statistics 2): Revision Note

Unbiased Estimates

What is an unbiased estimator of a population parameter?

• An estimator is a statistic that is used to estimate a population parameter

  • When a sample is used with the estimator, the value that it produces is called an estimate

• An estimator is called unbiased if the expected value of the estimator is equal to the population parameter

  • An estimate from an unbiased estimator is called an unbiased estimate

  • This means that the mean of the unbiased estimates will get closer to the population parameter as more samples are taken

What are the unbiased estimates for the mean and variance of a population?

• If you had the data for a whole population you could find the actual population mean and variance using

  • $\mu =\frac{\mathrm{\Sigma }x}{n}$

  • ${\sigma }^2=\frac{\mathrm{\Sigma }{(x-\mu )}^2}{n}=\frac{\mathrm{\Sigma }{x}^2}{n}-{\mu }^2$

• If you are using a sample to estimate the mean of a population then an unbiased estimate is given by

  • $\overline{x}=\frac{\mathrm{\Sigma }x}{n}$ 

  • This is the same formula for the population mean

• If you are using a sample to estimate the variance of a population then an unbiased estimate is given by

  • $s^2=\frac{\mathrm{\Sigma }{(x-\overline{x})}^2}{n-1}$ 

  • This can be written in different ways

  • $s^2=\frac{\Sigma {\left(x-\overline{x}\right)}^2}{n-1}=\frac{1}{n-1}\left(\Sigma x^2-\frac{{\left(\Sigma x\right)}^2}{n}\right)=\frac{n}{n-1}\left(\frac{\Sigma x^2}{n}-{\overline{x}}^2\right)$

  • This is a different formula to the population variance

  • The last formula shows a method for finding an unbiased estimate for the variance

    • Find the variance of the sample (treating it as a population)

    • Multiply this by $\frac{n}{n-1}$

Is there an unbiased estimate for the standard deviation?

• Unfortunately square rooting an unbiased variance does not result in an unbiased standard deviation

• There is not a formula for an unbiased estimate for the standard deviation that works for all populations

  • Therefore it is better to just work with the variance and not the standard deviation

• If you need an estimate for the standard deviation then you can use:

  • You can use the square root of your unbiased estimate for the population variance

    • $s=\sqrt{\frac{\mathrm{\Sigma }{(x-\overline{x})}^2}{n-1}}=\sqrt{\frac{1}{n-1}\left(\Sigma x^2-\frac{{\left(\Sigma x\right)}^2}{n}\right)}$

  • This won’t be unbiased but it will be a good estimate

How do I calculate unbiased estimates?

• If you are given the summary statistics $\mathrm{\Sigma }x$ and $\mathrm{\Sigma }{x}^2$ then you can simply use the formulae in the formula booklet

  • $\overline{x}=\frac{\mathrm{\Sigma }x}{n}$

  • $s^2=\frac{1}{n-1}\left(\mathrm{\Sigma }{x}^2-\frac{{\left(\mathrm{\Sigma }x\right)}^2}{n}\right)$

• If you are given the raw data then you will first need to calculate $\mathrm{\Sigma }x$ and $\mathrm{\Sigma }{x}^2$