Review

The central reason we do hypothesis testing is to decide whether or not the sample data are consistent with the null hypothesis. In the second step of the procedure we identify the kind of data that is expected if the null hypothesis is true. Specifically, we identify the mean we expect if the null hypothesis is true. Decisions: If the mean of the sample of data obtained in the experiment is consistent with the mean under the null hypothesis, we believe the null hypothesis is true: We "do not reject (we retain) the null hypothesis". If the mean of the sample of data obtained in the experiment is inconsistent with the null hypothesis, we decide it is not true: We "reject the null hypothesis". We can be wrong in either decision we reach.

 

Errors in Hypothesis Testing

We can come to one of two decisions:

1. We "do not reject (we retain) the null hypothesis".
2. We "reject the null hypothesis".

Since there are two decisions, there are two ways to be wrong (and two ways to be right).

Errors in Hypothesis Testing
Actual Situation No Effect Ho True Effect Exists Ho False Decision: Reject Ho Type I Error Decision Correct Decision: Retain Ho Decision Correct Type II Error

1. Type I Error: A type I error consists of rejecting the null hypothesis when it is actually true. This is a very serious error that we want to seldomly make. We don't want to be very likely to conclude the experiment had an effect when it didn't.

   The experimental results look really different than we expect according to the null hypothesis. But it could come out the way it did just because by chance we have a wierd sample.

   Example

   Control

   We have a control group in which a random sample of rats receives daily doses of water during pregnancy. At birth, we measure the weight of the sample of newborn rats. The weights, in grams, are shown in the table.The mean weight is 23 grams. We observe that the rat pups are really heavy and conclude that prenatal exposure to water increases birthweight. In reality (let's assume we know this) water has no effect on weight, but what we see in our sample is that it increases weight. We conclude, erroneously, that the mothers drinking increased water causes them to have heavier pups! There could be another reason. Perhaps the mother has unusual genes. We have made a Type I error, but we don't know it!

   Sample Mean: 23

2. Type II Error: A type II error consists of failing to reject the null hypothesis when it is actually false. This error has less grevious implications, so we are more willing to err in this direction (of not concluding the experiment had an effect when it, in fact, did). Of course, we may be missing an important effect.

   The experimental results don't look different than we expect according to the null hypothesis, but they are, perhaps because the effect isn't very big, or perhaps because our sample is too small.

   Example

   Experiment 2

   A second experimenter repeats the experiment, using alcohol, and gets the data shown at the right. The rat pups weigh 16.5 grams and we conclude there is no effect. We do not reject the null hypothesis. But "really" (if we only knew!) alcohol does reduce weight, we just don't have a big enough effect to see it. We have not rejected the null hypothesis, but it was really false and should have been rejected. We have made a Type II error, but we don't know it!

   Sample Mean = 16.5

---