[Data] | [<Normal page] [PEREZGONZALEZ Jose D (2014). Misinterpretation of the p-value and of the level of significance (2000). Knowledge (ISSN 1177-4576), 2014, pages 24-25.] [DOI] |

# Misinterpretation of 'sig' and 'p'

Oakes *(1986 ^{3})* was the first in exploring the extent to which British academic users misinterpreted inferential statistics, namely 'statistical significance' and the 'p-value'. His research was later replicated by Falk & Greenbaum in Israel

*(1995*, and Haller & Krauss

^{1})*(2000*in Germany. This article offers the results of a meta-analysis done on the collated data.

^{2})Results (see illustration 1) show that, overall, about 91% of academic users of statistics (including researchers, teachers and students) misunderstood the concepts of probability and statistical significance in the context of the so-called "null hypothesis testing".

Illustration 1. Collated frequencies and percentages of misinterpretations regarding tests of significance | |||||
---|---|---|---|---|---|

Common misinterpretations | f | % | (n) | ||

Statistical significance proves the alternative hypothesis | 21 | 8.9% | (236) | ||

Statistical significance disproves the null hypothesis | 27 | 11.4% | (236) | ||

The p-value informs of the probability of the null hypothesis | 96 | 40.7% | (236) | ||

The p-value informs of the probability of the alternative hypothesis | 97 | 41.1% | (236) | ||

The p-value informs of the probability of the results if replicated | 90 | 49.2% | (183) | ||

The p-value informs of the probability of making a wrong decision when rejecting the null | 138 | 75.4% | (183) | ||

(Participants who answered that all of above were false) |
20 | 8.5% | (236) | ||

f = frequencies, or times a misinterpretation was chosen; n = collated sample size |

As far as misinterpretations go, misunderstandings regarding the meaning of the p-value appeared more pronounced than those regarding proving or disproving hypotheses. About 9% of academic users thought that achieving statistical significance proved the alternative hypothesis, while a slightly larger percentage (11%) thought achieving statistical significance disproved the null hypothesis.

As per the p-value (or the probability of data given that the null hypothesis is true), 75% of academic users misinterpreted it as the probability of making the so-called 'Type I error' (rejecting the null hypothesis when it is true), and almost half of them misinterpreted it as the probability of obtaining similar results if the study were to be replicated. To a lesser degree, they misinterpreted the p-value as the probability of the null hypothesis being true (41%) or as the probability of the alternative hypothesis being false (41%).

# Methods

### Research approach

Meta-analysis done on data collated from three exploratory studies.

### Sample

Frequency of responses from 236 users of statistics for six of the items, and from 183 users for the remaining two items^{4}. The users were a mix of university lecturers, researchers and students from three countries: the UK *(Oakes, 1986 ^{3})*, Israel

*(Falk & Greenbaum, 1995*, and Germany

^{1})*(Haller & Krauss, 2000*.

^{2})### Materials

Six statements reflecting common misinterpretations of the p-value and tests of significance.

### Analysis

Percentages.

**FALK Ruma & Charles W GREENBAUM (1995).**

*Significance tests die hard: the amazing persistence of a probabilistic misconception.*Theory & Psychology, 1995, volume 5, number 1, pages 75-98. DOI 10.1177/0959354395051004.

**HALLER Heiko & Stefan KRAUSS (2000).**

*Misinterpretations of significance. A problem students share with their teachers.*Methods of Psychological Research Online, 2002, volume 7, number 1, pages 1-20.

**OAKES Michael (1986).**

*Statistical inference: a commentary for the social and behavioral sciences.*John Wiley & Sons (Chichester, UK), 1986.

**Notes**+++

^{1}).

# Want to know more?

**WikiofScience - Null hypothesis significance testing procedure**- This WikiofScience page offers an introduction to the pseudoscientific use of the so-called 'null hypothesis testing'.
**WikiofScience - Related studies**- You can find more information on the studies used for this meta-analysis on WikiofScience: the study done by Oakes in 1986; the replication done by Falk & Greenbaum in 1995; and the replication done by Haller & Krauss in 2000.
**WikiofScience - Tests of significance**- This page offers a correct procedure for testing the research data against a null hypothesis.

## Author

**Jose D PEREZGONZALEZ (2014).** Massey University, Turitea Campus, Private Bag 11-222, Palmerston North 4442, New Zealand. (JDPerezgonzalez).

Other interesting sites |

Knowledge |
WikiofScience |
AviationKnowledge |
A4art |
The Balanced Nutrition Index |