Recently, p-value is a hot term due to the God particle. The following is from the post:

The follow list summarizes five criticisms of significance testing as it is commonly practiced.

  1. Andrew Gelman: In reality, null hypotheses are nearly always false. Is drug A identically effective as drug B? Certainly not. You know before doing an experiment that there must be some difference that would show up given enough data.
  2. Jim Berger: A small p-value means the data were unlikely under the null hypothesis. Maybe the data were just as unlikely under the alternative hypothesis. Comparisons of hypotheses should be conditional on the data.
  3. Stephen Ziliak and Deirdra McCloskey: Statistical significance is not the same as scientific significance. The most important question for science is the size of an effect, not whether the effect exists.
  4. William Gosset: Statistical error is only one component of real error, maybe a small component. When you actually conduct multiple experiments rather than speculate about hypothetical experiments, the variability of your data goes up.
  5. John Ioannidis: Small p-values do not mean small probability of being wrong. In one review, 74% of studies with p-value 0.05 were found to be wrong.

Related posts:
Statistically significant but incorrect
False positives for medical papers

Please also refer to the comments of the above post.

So-called Bayesian hypothesis testing is just as bad as regular hypothesis testing

How loud is the evidence?

Advertisements