Meta-Analysis Resources

Tools for Those Who Summarize the Evidence Base

Resources and networking for those who conduct or interpret meta-analyses related to any phenomenon that is gauged in multiple studies.

Assessing the sensitivity of meta-analysis to selection bias: a multiple imputation approach

Assessing the sensitivity of meta-analysis to selection bias: a mul...


James Carpenter, Gerta Rucker, Guido Schwarzer, Department of Medical Statistics, London School of Hygiene & Tropical Medicine. 

 

The assessment of publication bias has gained popularity over the last decade.  Graphical and statistical methods such as the funnel plot, radial plot, trim-and-fill procedure and the Copas selection model have become standard steps in data analysis for contemporary
meta-analysis.  However, despite the widespread use of these methods, Carpenter et al. argue these methods have drawbacks, published in the journal Biometrics.  The trim-and-fill method is based on the assumption of funnel plot symmetry.  The Copas selection model is difficult to employ and interpret, and frequently encounters statistical estimation
problems.  Carpenter et al propose a logistic selection model.  Furthermore, they show how treatment effects can be estimated via multiple imputation (timely for Stata users with the 11.1 mi command). 

 The proposed logistic selection model and multiple imputation procedure impute missing studies under a missing at random assumption, then re-weights the imputed data to allow for non-random selection.  For example, a contrast to the non-parametric trim-and-fill imputing “reflections” of studies on a funnel plot.  Carpenter states symmetry assumptions are strong and often questionable.  The less common, but still popular, Copas selection model avoids the assumptions made in the trim-and-fill, rather, using a probit-selection model.  In practical use the Copas selection model is difficult to employ for the following reasons; 1) the results cannot be visually inspected on a funnel plot unlike the trim-and-fill method, and 2) around 20% of the time, there are numerical problems in estimating the model.  This makes the trim-and-fill method a frequent choice for meta-analysts. 

To this end, Carpenter et al employed a multiple imputation approach to sensitivity analysis for assessing funnel plot asymmetry.  Carpenter et al evoke their proposed method in a re-analysis of Serenoa Rapens data fitting an appropriate model in WinBUGS. The authors compare and contrast their proposed method with statistical and graphical summaries with the trim-and-fill method and the Copas selection model. 

Carpenter et al conclude the local logistic sensitivity analysis for funnel plot asymmetry is very flexible, sidesteps practical issues that arise with other methods, and allows for graphical depiction of imputed missing studies. 

To my knowledge there is no Stata macro employing this procedure.  It would be timely to develop this macro given the new flexibility of Stata 11.1 to perform multiple imputations with the mi command. 



Assessing the sensitive of meta-analysis to selection bias: a multiple imputation approach

James Carpenter, Gerta Rucker, Guido Schwarzer, Department of Medical Statistics, London School of Hygiene & Tropical Medicine. 

 

 The assessment of publication bias has gained popularity over the last decade.   Popular graphical and statistical methods such as the funnel plot, radial plot, trim-and-fill procedure and the Copas selection
model have become standard steps in data analysis for contemporary
meta-analysis.  However, despite the
widespread use of these widely used methods, Carpenter et al. argue these methods have drawbacks in the journal Biometrics.  The trim-and-fill method is based on the
assumption of funnel plot symmetry, and the Copas selection model is difficult
to employ and interpret, and frequently encounters statistical estimation
problems.  Carpenter et al propose a logistic selection model.  Furthermore, they show how treatment effects
can be estimated via multiple imputation (timely for Stata users with the 11.1 mi command). 

 The proposed logistic selection model and multiple imputation procedure imputes missing studies under a missing at random, then reweights the imputed data to allow for non-random selection.  For example, different from the
non-parametric trim-and-fill imputing “reflections” of studies on a funnel
plot.  Carpenter states  symmetry assumptions are strong and often
questionable.  The less common, but still
popular, Copas selection model avoids the assumptions made in the trim-and-fill,
rather, using a probit-selection model. 
In practical use the Copas selection model is difficult to employ for
the following reasons; 1) the results cannot be visually inspected on a funnel
plot unlike the trim-and-fill method, and 2) around 20% of the time, there are
numerical problems in estimating the model. 
This makes the trim-and-fill method a frequent choice for
meta-analysts. 

To this end, Carpenter et al employed a multiple imputation approach to sensitivity analysis for assessing funnel plot asymmetry.  Carpenter et al evoke their proposed method in a re-analysis of Serenoa Rapens data fitting an appropriate model in WinBUGS. 
The compare and contrast their proposed method with statistical
and graphical summaries with the trim-and-fill method and the Copas selection
model.  Carpenter et al conclude the local logistic sensitivity analysis for funnel
plot asymmetry is very flexible, sidesteps practical issues that arise with
other methods, and allows for graphical depiction of imputed missing
studies.  To my knowledge there is no
Stata macro employing this procedure.  It
would be timely to develop this macro given the new flexibility of Stata 11.1
to perform multiple imputations with the
mi
command. 

Views: 430

© 2017   Created by Blair T. Johnson.   Powered by

Badges  |  Report an Issue  |  Terms of Service