Bias of a Point Estimator

Let

Then, the bias is the expected error associated with the estimator, i.e.,

Notes:

• Unbiased estimators have zero bias for all values of the parameter (or estimand). Such point estimators are appealing since, on average, they tend to be "on target".
• An equivalent condition which an unbiased estimator must satisfy is
  For example, the sample mean derived from a random sample is an unbiased estimator of the the population mean since
  
  
  
• Biased estimators are those point estimators whose bias are not equal to zero for some value(s) of the parameter. Thus, for certain value(s) of the population parameter, a biased estimator tends to be "off target" on average.
• An equivalent condition which a biased estimator satisfies is
  
  
  The following intuitive estimator of the population variance
  
  is biased since
  if the above variance estimator is based on a random sample.
• The figure below shows the probability densities for the errors associated with three different estimators. The diagram indicates that two of the three estimators are biased while the remaining one is unbiased.
  
  
  
• When the bias is a linear function of the parameter (or estimand), one can easily perform a bias correction to derive an unbiased estimator. Essentially, all you need to do is to start with the expression for the expected value of the estimator and then rearrange terms so that the parameter is isolated on the right hand side of the equal sign. The correction process will require you to exploit certain properties (e.g., linearity) of expectations. For example, when the biased variance estimator given above is subjected to such a correction, we obtain
  
  and hence deduce the following biased corrected estimator
  
  
  As suggested by the preceding equation, the unbiased estimator we derived is none other than the usual sample variance.