On p-value

Albert Schweitzer said: “Example is not the main thing in influencing others. It is the only thing.”, so I start with it.

Generate two random samples from Normal distribution.
Test the hypothesis number one: \mu_1 = \mu_2 = 0, do not reject, then test the more general hypothesis number two: \mu_1 = \mu_2.
Logic states that since there is at least one value we do not reject for (in this case, 0), the more general statement would also not be rejected.

This is not always the case.

Since the code involves simulations, results may differ between users, despite the repetition procedure applied.

So in about 14% = 70/500 cases, we can not reject hypothesis number one, but we reject the more general hypothesis number two. Note, using the same data.

What is the point to take from this?
Well, that there is no logical contradiction here, we just need to be careful with the interpretation of p-values:
“one agent that does not believe in a proposition does NOT imply the she believes in its negation” (Dubois and Prade, 2001).

The idea for the post came from reading the paper: A classical measure of evidence for general null hypotheses. The author suggest an alternative measure to p-value.

Related:

2 comments on “On p-value”

Leave a Reply

Your email address will not be published. Required fields are marked *