Misinterpretations of 'p' and 'sig'
Haller & Krauss (20002) carried out a study on common misinterpretations of tests of significance among German psychology students and academics, which partly replicates one done by Oakes (19863). Typically, most of these misinterpretations confuse p-values (ie, the probability of the data when assuming that the null hypothesis is true) and, especially, statistical significance7, with the probability of proving or disproving hypotheses (be this the null hypothesis or an alternative hypothesis). Another misinterpretation is the so-called "replication fallacy", which occurs when the probability of the data is assumed to represent the probability of finding similar results if the research were to be repeated.
Haller & Krauss found that most participants held at least one misinterpretation out of the six presented (see table 1). They also found that, overall, 100% of psychology students held one or more misinterpretations (mean=2.5), almost 90% of psychology researchers also held one or more misinterpretations (mean=2), and 80% of instructors of statistics in psychology also held one or more misinterpretations (mean=1.9). The authors thought worrisome the high percentage of instructors with misinterpretations, as these may pass those misinterpretations down to students. Another interesting result, one not highlighted by the authors, though, is the high percentage of researchers (including instructors when carrying out and publishing research) with misinterpretations, as these would perpetuate those when publishing, peer-reviewing others' publications, and making research-informed decisions (such as chairing committees, granting funding, etc).
|Table 1. Percentages of misinterpretations regarding tests of significance|
|Significance disproves the null hypothesis||3||10%||6||15%||15||34%|
|The p-value informs of the probability of the null hypothesis||5||17%||10||26%||14||32%|
|Significance proves the alternative hypothesis||3||10%||5||13%||9||20%|
|The p-value informs of the probability of the alternative hypothesis||10||33%||13||33%||26||59%|
|'P' informs of the probability of a wrong decision when rejecting the null||22||73%||26||67%||30||68%|
|The p-value informs of the probability of the results if replicated||11||37%||19||49%||18||41%|
|(Participants who answered that all of above were false)||6||20%||4||10%||0||0%|
Jose D PEREZGONZALEZ (2013). Massey University, Turitea Campus, Private Bag 11-222, Palmerston North 4442, New Zealand. (JDPerezgonzalez).
Want to know more?
- WikiofScience - Hypotheses testing (disambiguation)
- This WikiofScience page lists alternative methods for testing the probability of data or hypotheses.
- WikiofScience - Null hypothesis significance testing
- This WikiofScience page reflects on the pseudoscientific bases of the null hypothesis significance testing (NHST) procedure typically used in the social sciences and medicine.
- WikiofScience - Related studies
- You can find more information on two related studies on WikiofScience. One was the original study done by Oakes in 1986; the other study was a replication done by Falk and Greenbaum in 1995.