A guide to obtaining confidence intervals for effect sizes

Effect sizes, specify the magnitude of a statistical comparison. However, this does not tell us how precisely it is measured.

There are [http://www.latrobe.edu.au/psy/esci/ details by Geoff Cumming], Keselman et al (2008) and Steiger (2004) on combining these concepts by giving confidence intervals for effect sizes. Michael Smithson [https://www.anu.edu.au/psychology/people/smithson/details/CIstuff/CI.html has syntax] in SPSS and other statistical software to do the computations. There are also some [https://www.anu.edu.au/psychology/people/smithson/details/CIstuff/Noncoht2.pdf workshop notes] to explain what's going on. See also Smithson (2001).

This [http://www.danielsoper.com/statcalc3/calc.aspx?id=28 on-line calculator] will work out CIs for R-squareds.This can be also done in R using the CI.Rsq function which uses the standard error for R-squared equal to sqrt((4*rsq*(1-rsq)^2*(n-k-1)2)/((n^2-1)*(n+3))) for a R-squared (rsq) with k predictors and a sample size of n.

There is also [http://core.ecu.edu/psyc/wuenschk/SPSS/SPSS-Programs.htm SPSS syntax], with an example, for obtaining a confidence interval for Cohen's d and [:FAQ/Rcis: R syntax] for an alternative more robust nonparametric bootstrap estimate (see for example Keselman et al (2008)) for confidence intervals for Cohens' d and correlations which come from non-Normal distributions.

Keselman et al (2008) have some [http://www.apa.org/journals/supplemental/met_13_2_110/met_13_2_110_supp.html SAS V9.1 code with examples] to produce bootstrap confidence intervals for effect sizes (ie based on repeated random sampling) for a robust (winsorised) version of Cohen's d in mixed anovas (replacing the lowest and highest 20% of outcome data by their respective least extreme values. This paper is available free to CBSUers using the APA internet link.

* [:FAQ/cdse:A suggestion for directly computing the s.e. of Cohen's d for use in constructing confidence intervals]

References

Fidler, F and Thompson, B (2001) Computing correct confidence intervals for ANOVA Fixed- and random-effects effect sizes. Educational and Psychological Measurement 61 575-604. CIs computed using step-by-step standard output from ANOVAs.

Keselman, HJ, Algina, J, Lix, LM, Wilcox, RR, Deering, KN (2008) A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes Psychological Methods 13(2) 110-129.

Smithson, M (2001) Correct confidence intervals for various regression effect sizes and parameters: the importance of noncentral distributions in computing intervals. Educational and Psychological Measurement 61 605-632.

Steiger, JH (2004) Beyond the F Test: Effect Size Confidence Intervals and Tests of Close Fit in the Analysis of Variance and Contrast Analysis. Psychological Methods 9(2) 164-182.