Tools for Those Who Summarize the Evidence Base
James Carpenter, Gerta Rucker, Guido Schwarzer, Department of Medical Statistics, London School of Hygiene & Tropical Medicine.
The assessment of publication bias has gained popularity over the last decade. Graphical and statistical methods such as the funnel plot, radial plot, trim-and-fill procedure and the Copas selection model have become standard steps in data analysis for contemporary
meta-analysis. However, despite the widespread use of these methods, Carpenter et al. argue these methods have drawbacks, published in the journal Biometrics. The trim-and-fill method is based on the assumption of funnel plot symmetry. The Copas selection model is difficult to employ and interpret, and frequently encounters statistical estimation
problems. Carpenter et al propose a logistic selection model. Furthermore, they show how treatment effects can be estimated via multiple imputation (timely for Stata users with the 11.1 mi command).
The proposed logistic selection model and multiple imputation procedure impute missing studies under a missing at random assumption, then re-weights the imputed data to allow for non-random selection. For example, a contrast to the non-parametric trim-and-fill imputing “reflections” of studies on a funnel plot. Carpenter states symmetry assumptions are strong and often questionable. The less common, but still popular, Copas selection model avoids the assumptions made in the trim-and-fill, rather, using a probit-selection model. In practical use the Copas selection model is difficult to employ for the following reasons; 1) the results cannot be visually inspected on a funnel plot unlike the trim-and-fill method, and 2) around 20% of the time, there are numerical problems in estimating the model. This makes the trim-and-fill method a frequent choice for meta-analysts.
To this end, Carpenter et al employed a multiple imputation approach to sensitivity analysis for assessing funnel plot asymmetry. Carpenter et al evoke their proposed method in a re-analysis of Serenoa Rapens data fitting an appropriate model in WinBUGS. The authors compare and contrast their proposed method with statistical and graphical summaries with the trim-and-fill method and the Copas selection model.
Carpenter et al conclude the local logistic sensitivity analysis for funnel plot asymmetry is very flexible, sidesteps practical issues that arise with other methods, and allows for graphical depiction of imputed missing studies.
To my knowledge there is no Stata macro employing this procedure. It would be timely to develop this macro given the new flexibility of Stata 11.1 to perform multiple imputations with the mi command.