Significance level - alpha

Research methods & Statistics » Significance level - disambiguation » Significance level - alpha

Table of Contents

Alpha (α) is the probability of making a Type I error, in the long run, when testing hypotheses.

This probability is associated to a particular region of a test statistic (eg, a region of 1.96 in the 'z' table, or the 't' table) and, thus, is not a data-based probability (ie, "a random variable whose distribution is uniform over the interval [0, 1]", Hubbard & Bayarri, 2003). The probability of alpha only means the probability of rejecting the hypothesis in the long run under repeated experimentation.

The main confusion between 'p' and 'alpha' as indicative of levels of significance is probably due to the historical enmity between Fisher and Neyman, and a probable attempt by the latter to discredit the former by using similar concepts and, thus, creating confusion:

Fisher was the first in using the concepts of "inductive inference" and "significance testing", and in using a probability (eg, 0.05 or 0.01) as a convenient "level of significance" for making a decision whether to reject the null hypothesis or not.
Neyman-Pearson latter used the concepts of "inductive behavior" and "hypothesis testing", set a probability for alpha similar to the cutoff points suggested by Fisher (ie, 0.05 or 0.01), and, then, referred to alpha as the "level of significance".

References

1. NOYMER Andrew (undated). Alpha, significance level of test. In, Paul J LAVRAKAS (undated). Encuclopedia of survey research methods. ISBN 9781412918084.