Tools for Those Who Summarize the Evidence Base
I want to start again a discussion about a problem i already had a few years ago:
I'm working on a meta-analysis where i combine different designs: independent groups (treatment and control group) on the one hand and only dependent pre post without control group on the other. For both study types I computed hedges' g according to Borenstein et al. (2009).
As prescribed for studies that use pre post scores I used the correlation between pre- and post-scores to compute the standard deviation within groups and the variance of d. To compute the weight for each study I used the inverse of that study’s variance.
And here is my problem/question:
If the pre- / postscore-correlation is relativ high (e.g. .06) the weight of this study shoots up extremly. One study showed a pre post correlation of .08 and its weight is more than 20th times the most precise study with independent groups has. I tried to use an averaged pre post correlation but also then as result some of these studies have a much higher weight than the independent ones. I read a few papers that adressed this problem but i didnt find a solution. I also read the chapter in Borenstein et al. about the factors that affect precision. But I'm not sure how to deal with that.
Thanks for any reply