Diff for "FAQ/meta" - CBU statistics Wiki
location: Diff for "FAQ/meta"
Differences between revisions 77 and 78
Revision 77 as of 2012-06-28 15:31:48
Size: 9886
Editor: PeterWatson
Comment:
Revision 78 as of 2012-08-21 16:39:00
Size: 10075
Editor: PeterWatson
Comment:
Deletions are marked like this. Additions are marked like this.
Line 63: Line 63:
Viechtbauer, W. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. ''Statistics in Medicine'' '''26(1)''' 37-52. Illustrates using the R procedure 'metafor'.

Meta-analysis issues: 1) How do I measure publication bias? 2) How do I obtain a confidence interval for pooled estimates from sets of log odds, Cohen's d and Pearson correlations using meta-analysis?

Measuring Publication Bias

This [attachment:james.pdf paper] of Peters et al. (2006) proposes using a weighted linear regression to assess publication bias in reporting log odds ratios as an alternative to the usual funnel plot. Funnel scatterplots can be drawn plotting differences/associations which are either standardized (effect size/s.e.) or unstandardized on the x-axis against precision (1/s.e. of effect size) on the y-axis. If there is no publication bias the plot should be symmetric forming a funnel shape. (As an aside Forest plots are also commonly presented in meta-analysis which plot effect size confidence intervals for each study or set of studies on a single plot (Peters et al, 2010)).

The method of Peters et al. (2006) corresponds to a weighted linear regression using the log odds, obtained using a 2x2 table of frequencies from each study, as the outcome and the reciprocal of the sample size of each study as the only predictor with a variance measure, $$\(\frac{1}{n_text{11}+n_text{12}}+\frac{1}{n_text{21}+n_text{22}}\)^text{-1}$$, used as a study weight where $$n_text{ij}$$ are the frequencies in the i,jth cells in each study (Peters et al. 2010). This may be carried out using the WLS weight option in SPSS LINEAR REGRESSION in SPSS or the metabias procedure from the meta library in R. It follows from the form of the weights that Peters regression procedure, unlike that of Egger, can only be used for odds ratios from 2x2 tables.

Peters et al. (2006) criticise the usual publication bias approach of Egger et al. (1997) ([http://www.bmj.com/content/315/7109/629.full details here)] who proposed a test for asymmetry of the funnel plot. In particular Peters et al. (2006) say the p-values of Egger et al.'s test are not as reliable as theirs because they have inflated type I errors. They, instead assess the regression coefficient showing the association between sample size and the log odds ratio and etst to check this is zero. Both these methods may be fitted using this [attachment:eggpe.xls spreadsheet.]

Egger et al.'s test is a test for the Y intercept = 0 from a linear regression of normalized effect estimate (estimate divided by its standard error) against precision (reciprocal of the standard error of the estimate). Note that this regression may also be equivalently carried out using the unstandardized effect size as outcome, its standard error as predictor and weighting each effect size by the reciprocal of its squared standard error. The power of this method to detect bias will be low with small numbers of studies.

Obtaining pooled effect sizes

It is also usually of interest to obtain a pooled estimate of the effect size. This is usually done using the inverse of the effect size variance. In the case of the log odds its variance is found by simply summing the inverses of each of the four cells in the table of frequencies which are used to compute the odds ratio for a particular study (see an example [:FAQ/oddsr:here.])

Schmidt, Oh and Hayes (2009) illustrate methods for performing meta-analyses combining estimates of Cohen's d from various studies assuming (a) the same population value underlies all the studies being combined (fixed) and (b) different population values underlie the studies being combined (random) and conclude that, although the majority of studies in the literature use fixed effects, in many cases combined estimates based on pooling random terms have a better coverage of the population group differences. Both combined effects use the variance of Cohen's d suggested in the 'distributions of effect sizes' section just short of half-way down the page given [http://en.wikipedia.org/wiki/Effect_size here.]

The above article also mentions briefly combining study correlations using Fisher's z transform. An involved iterative method for producing a random effects meta-analysis using the WLS weight option in SPSS on correlations is described in the introductory pdf article [attachment:metaSPSS.pdf here.]

Wilson (2005) has written a comprehensive set of SPSS macros for meta-analysis pooling Cohen's d, the logged odds ratio or the Fisher transformed Pearson correlation, anova effect sizes and regression effect sizes which assume either fixed or random effects. These macros and the spreadsheets mentioned below also perform a test for random effects using the Q statistic (Peters et al, 2010). The macros are available for download from [http://mason.gmu.edu/~dwilsonb/ma.html here.] Note that you may need to add a full stop to comment lines (starting with an asterisk) in the downloaded macros to make them work.

There is also a 'help' file detailing how to use these macros on that website (or [attachment:wilson.pdf here if the link is broken]). The help file explains and illustrates the uses of simple formulae for adjusting Cohen's d for small samples and computing its variance which is inverted to give the study weights prior to running the macro.

These macros are used by Lindberg et al (2010) to obtain a pooled Cohen's d pooling across 242 studies! A [attachment:metacd.xls spreadsheet] performs Wilson's inverse variance pooling of Cohen's D using the formulae in his MeanES.sps macro.

Formulae used in Wilson's macros are also explained more fully [http://www.zoology.ubc.ca/~schluter/bio548/workshop-meta.html here.]

An alternative procedure, the Mantel-Haenszel method, which is available in the CROSSTABS procedure in SPSS is recommended by the Cochrane handbook (Higgins and Green, 2008)(see above primer and [attachment:cochraneor.pdf here]) for pooling odds ratios (ORs) with the weightings of each odds ratio related to the number of observations on which that odds ratio is based. The approach used by Wilson is to weight the log of each odds ratio by its inverse variance (also related to sample sizes but using different weights to the Mantel-Haenszel). The log odds ratios are used because their variance has [:FAQ/oddsr:a simple form]. Results can be backtransformed to odds ratios if desired. This method is the one computed in the above macros or using this [attachment:metaor.xls spreadsheet.]

The variance of Fisher transformed Pearson correlations also has a simple form (see [:FAQ/corrs: here]) and can, therefore, be pooled using the inverse variance method of Wilson(2005) in this [attachment:metafr.xls spreadsheet.]

In addition to Wilson's Metareg SPSS macro the WLS weight option in SPSS LINEAR REGRESSION can be used for assessing the influence of covariates on effect sizes combining summary measures using fixed and random variances as the weight on each summary measure. For an application of this using the log odds ratio effect size see the powerpoint slides located [attachment:WLSmeta.ppt here.]

A further primer on meta-analysis (in pdf format) is given [attachment:PLane.pdf here.]

References

Egger, M., Smith, G. D., Schneider, M. and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal 315 629-634.

Field, A. P. and Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology 63 665-694.

The Field and Gillett article is a useful primer with illustrations of the main issues and presentation of results. This [http://www.statisticshell.com/meta_analysis website] contains further information files including SPSS and R programmes which can be downloaded to carry out aspects of a standard meta-analysis including pooled effect size calculations and funnel plots.

Higgins, J. P. T. and Green, S. (2008). Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.1. Cochrane Collaboration:Oxford. (Available from http://www.cochrane-handbook.org).

Lindberg, S. M., Hyde, J. S., Linn, M. C. and Petersen, J. L. (2010). New trends in gender and mathematics performance: a meta-analysis. Psychological Bulletin 136(6) 1123-1135.

(A PDF COPY OF THE ABOVE ARTICLE IS AVAILABLE FOR FREE DOWNLOAD VIA SCIENCE DIRECT FOR CBSU USERS)

Morris, S. B. and Deshon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods 7(1) 105-125.

(A PDF COPY OF THE ABOVE ARTICLE IS AVAILABLE FOR FREE DOWNLOAD VIA SCIENCE DIRECT FOR CBSU USERS)

Peters, J. L., Sutton, A. J., Jones, D. R. and Abrams K. R. (2010). Assessing publication bias in meta-analyses in the presence of between-study heterogeneity. Journal of the Royal Statistical Society A 173(3) 575-591. (see [http://onlinelibrary.wiley.com/doi/10.1111/j.1467-985X.2009.00629.x/full here.])

(THE ABOVE JOURNAL ARTICLE IS ALSO AVAILABLE FOR CBSUERS FROM PETER WATSON UPON REQUEST AND IS A GOOD PRIMER PROVIDING AN OVERVIEW OF META-ANALYSES INCLUDING ILLUSTRATIONS OF THE ABOVE IDEAS)

Schmidt, F. L., Oh, I. S. and Hayes, T. L. (2009). Fixed- versus random-effects models in meta-analysis: model properties and an empircial comparison of differences in results. British Journal of Mathematical and Statistical Psychology 62 97-128.

VanDerwerken, D. (2012). What Petri dishes have to do with your research. Significance 9(3) 40-42. Runner-up in a young statisticians writing competition, this article illustrates advantages such as increased power and precision afforded by meta-analysis (using 1/variance as weighting) in an example from microbiology.

Viechtbauer, W. (2007). Confidence intervals for the amount of heterogeneity in meta-analysis. Statistics in Medicine 26(1) 37-52. Illustrates using the R procedure 'metafor'.

Wilson, D. B. (2005). Meta-analysis macros for SAS, SPSS, and Stata [Computer software]. Retrieved from http://mason.gmu.edu/~dwilsonb/ma.html

None: FAQ/meta (last edited 2019-01-07 15:33:11 by PeterWatson)