Importance

Table of Contents

Importance (or relevance), in the context of science, refers to the relative weight of an effect in practical terms. It is otherwise referred to as 'practical significance'.

Statistically speaking, a results may be statistically significant and still lack practical significance (aka, be unimportant or irrelevant). This is the case when, for example, trivial differences between groups come up as statistically significant because the sample used is very large.

Practical significance may be used to refer to another statistical tool, the effect size (eg, Cohen, ###). 'Importance' (or 'relevance'), however, is slightly different to effect size in two accounts: firstly, a result may have a sizable effect size and still be rather irrelevant or trivial; secondly, importance is more bound to raw results (eg, unstandardised effects sizes) than to standardized results (ie, it has to make real sense and be easily understood).

A good example of the difference between 'importance' and 'effect size' is the interpretation of ordinal scales in psychology or the social sciences. For example, a Likert-scale measuring degree of agreement may comprise 5 anchors, such as '1-Disagree fully, 2-Disagree slightly, 3-Neutral, 4-Agree slightly, 5-Agree fully'. However, people vary in their levels of agreement more than the five anchors that the scale provides, thus, it is reasonable to expect that, for example, being neutral covers not only anchor '3' but also half-way into the nearest anchors (ie, from about 2.6 to 3.4, approximately), so that people who feel they agree '3.3' rather than a perfect '3' would still select '3' as their answer as this is the closest anchor to their real level of agreement.
Given this, the minimum distance that would make a 'difference' in the scale is that distance that could move agreement to another level. For example, a distance of about '0.5' is the minimum required to move a level of agreement from a perfect '3' to about '3.5', which, for some people, may represent a 'slight agreement'. Any distance lesser than this can be reasonably considered random variation in decision making. Thus, '0.5' is the minimum required for a reasonable 'small' difference of relevance (including movement within the same level). A large difference may be that large enough to move a level of agreement two levels (for example, from a perfect '3' to about '4.5', which is from 'neutral' to 'fully agree'). A medium difference would be in between those two distances.
This reasoning, however good it could be, may not necessarily find a correlate with effect sizes. An effect size depends on the standard deviation, so it is possible to find large differences which, in practice, are rather small or trivial, a fluke of decision making and the nature of the scale, with little impact (aka, importance or relevance) in real life. This is why is important to distinguish 'importance' from 'effect size' and '(statistical) significance'.

References

1. full reference in the following format AUTHOR (date work).Title. Reference location, date publication. ISBN/ISSN.

+++ Notes +++

2. ###