20120425 - Misinterpretation of 'p' (1995) (2e)

[Data]

[<Normal page] [PEREZGONZALEZ Jose D [ed] (2012). Misinterpretation of 'p' (1995) (2e)⁶. Journal of Knowledge Advancement & Integration (ISSN 1177-4576), 2012, pages 144-145.]

Table of Contents

Misinterpretations of 'p' and 'sig'

Generalization potential

Misinterpretations of 'p' and 'sig'

Falk and Greenbaum (1995²) carried out a study on common misinterpretations of the logic of tests of significance among Israeli psychology students, which partly replicates one by Oakes (1986⁴). Typically, most of these misinterpretations confuse p-values (ie, the probability of the data when assuming that the null hypothesis is true) and, especially, statistical significance, with the probability of proving or disproving hypotheses (be this the null hypothesis or an alternative hypothesis).

Falk and Greenbaum found that almost 87% of the students held at least one misinterpretation out of the four presented (see table 1). Most of the students misinterpreted p-values as the probability of the null hypothesis being true.

Table 1. Frequencies and percentages of misinterpretations regarding tests of significance
Common misinterpretations⁷	f	%
Significance disproves the null hypothesis	2	3.8%
The p-value informs of the probability of the null hypothesis	42	79.2%
Significance proves the alternative hypothesis	0	0.0%
The p-value informs of the probability of the alternative hypothesis	2	3.8%

(Participants who answered that all of above were false)	7	13.2%

Methods

Research approach

Not much detail. It appears to have been a confirmatory study with a hint of 'quasi-experimental' assumption (the quasi-experiment being that students should had being familiar with Bakan's 1965¹ paper, as it had been one of the readings for their Experimental Psychology course).

Sample

A convenient sample of 53 psychology students from the Hebrew University of Jerusalem. The participants had taken two courses in statistics and one course in experimental psychology.

Materials

Not much detail about the materials used. Plausibly a tool consisting of either a verbal or written scenario regarding the results of a test with a nominal p-value acting also as a predetermined level of significance (akin to a similar scenario used by Oakes, 1986⁴), and a one-item questionnaire with five multiple-choice options. These options presented several interpretations of the results, and the participants could choose as many options as they thought correct. (Unbeknownst to the participants, four statements were false, representing four common misinterpretations of tests of significance. The last statement negated all others.)

Analysis

Descriptive statistics.

Generalization potential

This particular research appears to be limited to the population of (undergraduate) psychology students at the Hebrew University of Jerusalem. Yet the results might, at least, serve as a working hypothesis for generalizing to other populations such as the following (in order of decreasing generalization scope):

Israeli psychology academics and graduands from that university.
Israeli psychology students, academics, researchers and graduands, in general.
Professional psychologists trained in Israeli universities.
(See also Oakes, 1986⁴, original study in Britain, and a partial replication of that study by Haller and Krauss, 2000³, in Germany, for a potential generalization beyond Israel).

References

1. BAKAN David (1965). The test of significance in psychological research. Psychological Bulletin, 1966, volume 66, pages 423-437.

2. FALK Ruma & Charles W GREENBAUM (1995). Significance tests die hard: the amazing persistence of a probabilistic misconception. Theory & Psychology, 1995, volume 5, number 1, pages 75-98. DOI 10.1177/0959354395051004.

3. HALLER Heiko & Stefan KRAUSS (2000). Misinterpretations of significance. A problem students share with their teachers. Methods of Psychological Research Online, 2002, volume 7, number 1, pages 1-20.

4. OAKES Michael (1986). Statistical inference: a commentary for the social and behavioral sciences. John Wiley & Sons (Chichester, UK), 1986.

5. PEREZGONZALEZ Jose D [ed] (2011). Misinterpretation of 'p' (1995). Journal of Knowledge Advancement & Integration (ISSN 1177-4576), 2011, pages 107-109.

+++ Notes +++

6. This second edition updates the original edition⁵ by reducing confusion between p-values and statistical significance (see tests of significance).

7. The original research statements have been rephrased here.

Want to know more?

Wiki of Science - Hypotheses testing (disambiguation): This Wiki of Science page lists alternative methods for testing data or hypotheses.
Wiki of Science - Null hypothesis significance testing: This Wiki of Science page reflects on the pseudoscientific bases of the null hypothesis significance testing (NHST) procedure.
Wiki of Science - Related studies: You can find more information on two related studies in Wiki of Science: Oakes (1986) and Haller and Krauss (2000).