Significance level - exact level of significance

Exact level of significance

Levels of significance can be used as a proxy to inform about how probable the results are if the null hypothesis were true, without necessarily settling for a cutoff level beforehand nor making any decisions regarding whether to reject or not the null hypothesis.

This interpretation of significance levels would call for the reporting of exact levels of significance (eg, p = .49, or p = .65), instead of conventional levels of significance. In this case, the exact level of significance is just the probability result. This approach is, perhaps, more suitable for descriptive studies than prescriptive ones.

Fisher rejected conventional levels of significance and embraced exact levels of significance instead in 1956 (quoted by Gigerenzer, 2004). In any case, exact levels of significance may be better understood within Fisher's theoretical positioning within statistics:

  • Inference is based on a frequentist approach to statistics, thus random sampling and controlled experiments are a necessary prerequisite to reduce non-random error. Given these conditions then, the level of significance is a property of the data themselves.
  • Only the null hypothesis is tested. Thus, non-significant results are ignored, while significant results may be considered for rejecting the null hypothesis. Significant results, however, can say nothing about an alternative hypothesis, other than as opposition to the null hypothesis (eg, we may be able to reject the null hypothesis that there is no correlation between a pair of variables, but we can say nothing regarding whether the correlation is real nor whether it is due to the particular variables being correlated).
  • The null hypothesis cannot be proved.

See also [http://petulantskeptic.blogspot.com/2011/02/you-do-not-understand-what-p-value-is-p.html]
See also [http://labstats.net/articles/pvalue.html]
See also [http://www.experiment-resources.com/p-value.html]
See also [http://www.childrensmercy.org/stats/definitions/pvalue.htm]
See also [http://www.graphpad.com/articles/pvalue.htm]
See also [http://www.sportsci.org/resource/stats/pvalues.html]
See also [http://xa.yimg.com/kq/groups/18751725/803387166/name/twelve+P+value+misconceptions.pdf]

Properties

  • The significance level is a probability figure between '0' and '1' associated to some statistical tests.
  • This figure is used to report the probability of the data supporting a "null hypothesis" (eg, that there is no difference between groups), but not for necessarily rejecting it nor drawing any conclusions about unstated or untested "alternative hypotheses".
  • A result is said to be significant according to the researcher reporting them. The decisions made by the researcher may not be the ones made by others, yet the significance is reported exactly, rather than by using more obscure conventional cut-off points. Thus, significance levels should be expressed exactly (or rounded) as, for example, p = .55, or p = .002).

Calculation

References
1. NOYMER Andrew (undated). Alpha, significance level of test. In, Paul J LAVRAKAS (undated). Encyclopedia of survey research methods. ISBN 9781412918084.

Contributors to this page

Authors / Editors

JDPerezgonzalezJDPerezgonzalez


Other interesting sites
320
Journal KAI
105px-Stylised_Lithium_Atom.png
Wiki of Science
120px-Aileron_roll.gif
AviationKnowledge
Artwork-194-web.jpg
A4art
Artwork-162-web.jpg
The Balanced Nutrition Index
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License