1. Clinical vs. Statistical Significance
The findings of one scientific experiment can have statistical significance but not clinical significance. Statistical Significance is the degree by which a value is greater or smaller than what would be expected by chance. Clinical significance is related to the practical importance of the findings. Data analysis may indicate that the control and experimental groups are statistically significantly different, but the findings have no clinical significance. The difference is not sufficiently important to warrant clinical benefits (Burns & Grove 2007).
It is difficult to judge how much change would constitute clinical significance. There is no common agreement in nursing about how to evaluate the clinical significance of a finding. It is not determined how much improvement do subjects need to demonstrate clinical significance. Clinical significance is ultimately a value judgment (LeFort, 1993).
Measurements of Statistical and Clinical Significance
Statistical significance is easily calculated by statistical equations. However, clinical significance requires a judgment by the research team about what constitutes clinical significance in that particular area of study.
Measurements of Statistical Significance
Statistical significance is usually determined through the proposition of a null and alternative hypothesis. Acceptance or rejection of this null hypothesis is determined by comparing the computer generated p-value to an alpha value predetermined before the experiment. The p-value is the probability of getting the results if the null were actually true. For example, if the alpha value to reject or accept the null hypothesis is set to .5, then if the p-value is less than .05, the null hypothesis is rejected and the alternative hypothesis is accepted. Otherwise, if the p-value is greater than .05, there is insufficient evidence to reject the null hypothesis or accept the alternative hypothesis.
Measurements of Clinical Significance
A number of measures are used to conclude clinical significance of experiments. One of the best statistics used is the Absolute risk reduction (ARR). ARR is the difference between the control group’s event rate Control event rate (CER) and the experimental group’s event rate Experimental event rate (EER). CER is the proportion of patients responding to control treatment. For example, if 25 patients are in a control group and the event being studied is observed in 15 of those patients, the control event rate would be 15/25 = 0.60. EER is the proportion of patients responding to the experimental treatment. For example, if 26 patients are in an experimental group and the event being studied is observed in 22 of those patients, the experimental event rate would be 22/26 = 0.85. Another statistic is the Number needed to treat (NNT) which is the number of patients that must be treated to prevent one outcome. NNT is the inverse of the ARR; NNT = 1/ARR Robert F. (2004)
Relative risk reduction (RRR) is another very popular clinical statistic which is reported in nearly every journal article. RRR is the percent reduction in events in the experimental group compared to the control group event rate. Unlike ARR, RRR amplifies small differences between the between the experimental and control interventions. RRR is not considered a very good statistic as it makes insignificant findings appear significant and does not reflect the baseline risk of the outcome event. However, RRR is very popular as it makes weak results look good Robert F. (2004)
What is more important: Statistical significance or clinical significance?
Clinical significance can not exist except after statistical significance is confirmed for the experiment. Statistical significance indicates the presence of a relationship between two events that is not a result of chance. However clinical significance indicates the level of importance from the clinical point of view of this relationship. Clinical Significance is a measure of whether a research result “matters” in the real world.
In the nursing field, clinical significance is more important that statistical significance. Relationships discovered in clinical experiments may prove to be statistically significant, but they may not be clinically significant. Clinical significance determines the worthiness of the statistical significance detected within one experiment.
References
Burns, N. & Grove, S. (2007). Understanding Nursing Research: Building an Evidence-based Practice. Fourth Edition. University of Texas at Arlington, Arlington, Texas, U.S.A.
Robert F. (2004). A simple method for evaluating the clinical literature. Fam Prac Mgt, May 2004; 47- 52. Available online at: www.aafp.org/fpm.
