Fisher-Gigerenzer's tests of significance

Table of Contents

Fisher-Gigerenzer's significance testing

Fisher's procedure for performing tests of significance was reduced down to three basic steps by Gigerenzer et al (2004¹).

Procedure

Set up a statistical null hypothesis².
Report the exact probability of the data³ (eg, p=0.011, or p=0.51). Do not use a conventional 5% level⁴, and do not talk about accepting or rejecting hypotheses.
Use this procedure only if you know very little about the problem at hand.

Properties

Gigerenzer et al's modification of Fisher's procedure does not lead to a proper test of significance. The procedure simply obtains the probability of the data under the null hypothesis, but does not indicate when to interpret a result as being statistically significant.
It is conceivable that Gigerenzer et al meant for 'significance' to be a decision made by the researcher rather than an automatic decision based on conventional levels of significance.
The second point above may be supported on Gigerenzer et al calling the probability of the data the "exact level of significance". Yet, here they mixed up two concepts: one thing is the probability of the data itself, another thing is the decision to whether such probability is statistically significant or not. Gigerenzer et al create confusion by mixing these two concepts together.
In any case, Gigerenzer et al's modification of Fisher's procedure was an attempt to highlight the limitations of statistical tests of significance. Thus, it needs to be understood as a reaction against the spread of the pseudoscientific null hypothesis significance testing procedure, and, overall, against the popularity of significance testing over other procedures, such as Bayes's, which may be more relevant in fields such as psychology.

References

1. GIGERENZER Gerd, Stefan KRAUSS & Oliver VITOUCH (2004). The null ritual: what you always wanted to know about significance testing but were afraid to ask. Chapter 21 in David KAPLAN [ed] (2004), The SAGE handbook of quantitative methodology for the social sciences. SAGE (California, USA), 2004. ISBN 9780761923596.

+++ Footnotes +++

2. The null does not need to be a nil hypothesis

3. Gigerenzer et al (2004) calls it the "exact level of significance"

4. ie, sig<0.05