Tools for Those Who Summarize the Evidence Base
We aim to do a meta-analysis on mean proportions using. Thus, individual proportions (accuracy scores) within a study are already averaged to a mean proportion when I distract them from studies.
We now struggle with the following: We want to calculate the variance but the denominator in our proportions reflect the number of math calculations attempted not the number of people in a study sample. Thus, denominators in calculations for e.g. Risk difference reflect the number of math calculations attempted. However, for calculation of the variance, this isn’t right as the variance will be used to weight the effect size and thus needs to reflect the number of people in a study sample. Normally one would calculate the variance of proportions as p(1-p) but I doubt whether this is appropriate as our distribution of mean proportions is more likely to be normally distributed than binomially distributed. Variance is generally not given in studies included as those studies report on the number attempted and the number correct (with accompanying variance), not on the proportion.
Do you have any advice on how to obtain the variance necessary to weight the studies in our situation?