It is a common characteristic of experiments in widely diverse fields of research that, when they are repeated, the effects of the experimental treatments vary from trial to trial. This variation introduces a degree of uncertainty into any conclusions that are drawn from the results. Even after a number of repetitions, or replications as they are called, the investigator still does not know by how much his results would be changed if the experiment were repeated further under the same conditions. Successive trials may be so discrepant in their results that it is doubtful which of two treatments would turn out better in the long run.
As an illustration of this variation, data are given in table 1.1 from a simple experiment. The data are the times (minus 2 minutes) required to perform a routine statistical calculation, that of finding the sum of squares of 27 observations. Each sum of squares was computed separately by the same person on both of two standard computing machines. In all, 10 different sums of squares were worked, making 10 trials or replications of the experiment. It will be noted that the differences in speed range from 17 seconds in favor of machine B to 2 seconds in favor of machine A. Some experimenters may comment that the results of this experiment are remarkably well behaved, and exhibit nothing like the variation with which they have to contend. The results will, however, serve our purpose.
| TABLE 1.1 Time in seconds (Minus 2 minutes) required for computing sums of squares | |||
|---|---|---|---|
| Replication | Machine A | Machine B | Difference (A-B) |
| 1 | 30 | 14 | 16 |
| 2 | 21 | 21 | 0 |
| 3 | 22 | 5 | 17 |
| 4 | 22 | 13 | 9 |
| 5 | 18 | 13 | 5 |
| 6 | 29 | 17 | 12 |
| 7 | 16 | 7 | 9 |
| 8 | 12 | 14 | -2 |
| 9 | 23 | 8 | 15 |
| 10 | 23 | 24 | -1 |
| Means | 21.6 | 13.6 | 8.0 |
The object of the experiment is, of course, to compare the speeds of the two machines for this calculation. More specifically, two objects might be stated. The first is to answer the question: is there any difference in speed? or, to put it another way, to test the hypothesis that there is no difference in speed. The second object, which is related to the first, is to estimate the size of the difference in speed. Almost all experiments are carried out for one or both of these purposes -- the testing of hypotheses and the estimation of differences in the effects of different treatments.
Adapted from:
Cochran, William G., and Cox, Gertrude M., "Experimental Designs", John Wiley & Sons Inc, 1957, pp.1-2.