Confidence intervals (CI) and p-values are two different approaches in determining whether the result of an experiment is statistically significant or not. CI, which is sometimes called the acceptance region, provides a statistical range with a specified probability that a given estimator lies within the range (Gujarati, 2003). P-value, on the other hand, is defined as the “lowest significance level at which a null hypothesis can be rejected.”
Part I
CI is very useful in epidemiological research, both in study design and result interpretation (Macfarlane & Silman, 2002). Clinical experiments with small sample size tend to yield results with high statistical variability which the CI can easily express. Aside from setting sample sizes, CIs are also considered “informative data summaries” which can complement or even replace the role of p-values in reporting statistical results. Given the sample size and observed effect, it can isolate which results are statistically compatible with the data from those that are not. Moreover, CIs can put light to vague results of studies which try to rule out differences saying that these are function of the design, not the results (McNeil, 1996). CIs can distinguish between the differences that are statistically discarded based on what was actually observed and those that are not.
P-value aids researcher in measuring the evidence against the null hypothesis or the hypothesis of no change or no effect (Gujarati, 2003). Instead of preselecting an arbitrary significance level, such as one percent, five percent or 10 percent, the researcher can obtain the exact level of significance of a test statistic. The smaller the p-value, the more likely the null hypothesis will be rejected.
For example, a group of medical researchers want to determine whether cervical cancer vaccine significantly prevents HPV infection. The null hypothesis states that the vaccine has no effect while the alternative hypothesis holds that the effect is not equal to zero.
H0 : μ = 0
H1 : μ ≠ 0
Where μ = effectiveness of the cervical cancer vaccine
If the vaccine is effective, the study can reject the null hypothesis; if it is not, then the researchers can assume that the alternative hypothesis is true. Assuming that the p-value is evaluated against the criterion α set at 0.05, then a p-value of .05 or less is required to reject the null hypothesis and establish statistical significance. If α= 0.05, the confidence interval is 95% where μ may or may not be included. Assuming that μ is included, researchers cannot reject the null hypothesis.
Part II
Sample size, percentage and population size determine the size of the CI at a given confidence level (Mann, 2007). It is often impossible to conduct a study using the entire population. But a large sample, with randomly selected participants, is enough to draw a conclusion for the entire population after the experiment is conducted. For a given confidence level, as the sample size increases, the CI decreases though sample size is not directly proportional to CI. The smaller the CI, the more certain are the results.
An inadequate sample size may provide support to the null hypothesis. Though the small p-value supports the alternative hypothesis, a large p-value does not necessarily mean that the null hypothesis holds true. In some situations, however, large sample size misleads researchers to think that even unimportant differences have statistical significance. Careful discretion has to be applied, especially in epidemiological research. There has to be a balance between the cost and side effects of a treatment against the benefits that the therapy provides. The larger the sample size, the more reliable are the findings.
References
Gujarati, Damodar (2003). Basic Econometrics. Asia: McGraw-Hill Education
Macfarlane, Gary and Silman, Alan (2002). Epidemiological Studies: A Practical Guide. Cambridge: Cambridge UP.
Mann, Prem S. (2007). Introductory Statistics. England: John Wiley & Sons Ltd.
McNeil, Don (1996). Epidemiological Research Methods. England: John Wiley & Sons Ltd.
