Table of contents |
2 The theorem 3 Example 4 Idempotence of the Rao-Blackwell process 5 When is the Rao-Blackwell estimator the best possible? |
One case of Rao-Blackwell theorem states:
The essential tools of the proof besides the definition above are the law of total expectation and the fact that for any random variable Y, E(Y2) cannot be less than [E(Y)]2. That inequality is a case of Jensen's inequality, although in a statistics course it may be shown to follow instantly from the frequently mentioned fact that
Some prerequisite definitions
The theorem
In other words
A more general version of the theorem will be mentioned below.
The more general version of the Rao-Blackwell theorem speaks of the "expected loss"
where the "loss function" L may be any convex function. For the proof of the more general version, Jensen's inequality cannot be dispensed with.
The improved estimator is unbiased if and only if the original estimator is unbiased, as may be seen at once by using the law of total expectation. The theorem holds regardless of whether biased or unbiased estimators are used.
The theorem seems very weak: it says only that the allegedly improved estimator is no worse than the original estimator. In practice, however, the improvement is often enormous, as an example can show.
Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers of phone calls that arrived during n successive one-minute periods are observed. It is desired to estimate the probability e−λ that the next one-minute period passes with no phone calls. The answer given by Rao-Blackwell may perhaps be unexpected.
A extremely crude estimator of the desired probability is
The sum
In case the sufficient statistic is also a complete statistic, i.e., one which "admits no unbiased estimator of zero", the Rao-Blackwell process is idempotent, i.e., using it to improve the already improved estimator does not do so, but merely returns as its output the same improved estimator.
If the improved estimator is both unbiased and complete, then the Lehmann-Scheffé theorem implies that it is the unique "best unbiased estimator."Example
i.e., this estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise.
can be readily shown to be a sufficient statistic for λ, i.e., the conditional distribution of the data X1, ..., Xn, given this sum, does not depend on λ. Therefore, we find the Rao-Blackwell estimator
After doing some algebra we have
Since the average number X1+ ... + Xn of calls arriving during the first n minutes is nλ, one might not be surprised if this estimator has a fairly high probability (if n is big) of being close to
So δ1 is clearly a very much improved estimator of that last quantity.Idempotence of the Rao-Blackwell process
When is the Rao-Blackwell estimator the best possible?