computing 95%CI of the I^2 statistic; and publication bias - Meta-Analysis Resources2017-08-18T08:53:45Zhttp://meta-analysis.ning.com/forum/topics/computing-95-ci-of-the-i-2-statistic-and-publication-bias?groupUrl=heterogeneity&xg_source=activity&feed=yes&xn_auth=noSincerely thanks so muc…tag:meta-analysis.ning.com,2017-05-10:5367515:Comment:244372017-05-10T20:33:54.849ZKevriahttp://meta-analysis.ning.com/profile/GloriaMa
<p></p>
<br />
<br />
<br />
<p></p>
<br />
<p class="MsoNormal" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">Sincerely thanks so much for the informative and concise explanation!! May I further ask few follow-up questions?</span></p>
<p class="MsoNormal" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">4) Should I^2 be computed when k = 2,…</span></p>
<p></p>
<br />
<br />
<br />
<p></p>
<br />
<p class="MsoNormal" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">Sincerely thanks so much for the informative and concise explanation!! May I further ask few follow-up questions?</span></p>
<p class="MsoNormal" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">4) Should I^2 be computed when k = 2, but both studies are from the same article? e.g. two independent samples reported in the same article? Should we use the criteria of k > 1 or the degrees of freedom of Q > 1. It is because the 95% CI of I^2 cannot be computed in the following two situations as well, but I haven’t been able to figure out the root problems.</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">1 article (2 studies): Cohen’s d = 0.21, SE = 0.06, 95% CI = 0.10, 0.32<br/></span><span style="font-family: 'Times New Roman';">Q = 0.147, df = 1, I^2 = 0.00</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">2 articles (2 studies): Cohen’s d = 0.26, SE = 0.08, 95% CI = 0.10, 0.41</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span style="font-family: 'Times New Roman';">Q = 0.167, df = 1, I^2 = 0.00</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span style="font-family: 'Times New Roman';">5) In fact, how do we conceptualize negative values of I^2 statistic?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">6) Moreover, when we statistically round up negative I^2 values up to zero, what does this mean conceptually? If we round up the value to zero, do we need to specify this in any parts of a manuscript?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">Much thanks again for the explanation of PET-PEESE and it helped me a lot. May I further ask:</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">7) Does it mean we can directly put the effect sizes as DV and standard error (or variance, depending on the results you specified last time) as the single predictor in linear regression model in SPSS? Then the “constant” in the SPSS linear regression output will be the intercept we are looking for?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">8) When we compare the naive mean and the estimates obtained from PET-PEESE, is there any thumb of rule or criteria to determine how close together the two values are to imply the degree of small-studies effect? What if when the two estimates are close together, but the 95% CI of the estimates from PET-PEESE cut across zero (which suggests non-significance?)?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">9) I’m indeed a bit confused. Based on my understanding, publication bias could be a reason for small-study effects. But it seems to me these two terms are used interchangeably sometimes. So for PET-PEESE, Egger’s Test, Begg’s Test, and the trim-and-fill method, what does each of this test assess actually? Publication bias or small-studies effect?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">10) Referring to your stock advice, for “complex model that account for heterogeneity” — when there is high heterogeneity, should we try to employ statistical models to account for it, or select a relatively “homogeneous” group of studies from the original batch to conduct a separate meta-analysis?</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;"><span lang="EN-US" style="font-family: 'Times New Roman';" xml:lang="EN-US">Sincerely much thanks again for the explanation and great help!!</span></p>
<p class="MsoNormalCxSpMiddle" style="line-height: 16.0pt; mso-line-height-rule: exactly;">Kevria</p> With 1), right I^2 can only b…tag:meta-analysis.ning.com,2017-05-09:5367515:Comment:245342017-05-09T23:02:44.607ZBlair T. Johnsonhttp://meta-analysis.ning.com/profile/BlairTJohnson
<p>With 1), right I^2 can only be calculated when k>1.</p>
<p>With 2), according to Higgins and Thompson's (2002) formulation, I^2 values below zero are rounded up to zero.</p>
<p>With the second 2), it looks like a calculation error.</p>
<p>With 3), for PET-PEESE, (a) put the SE in a meta-regression as a single predictor of the effect sizes. If the intercept is not significant, then the intercept is the estimate of your effect, adjusted for small-studies effects, along with its 95%…</p>
<p>With 1), right I^2 can only be calculated when k>1.</p>
<p>With 2), according to Higgins and Thompson's (2002) formulation, I^2 values below zero are rounded up to zero.</p>
<p>With the second 2), it looks like a calculation error.</p>
<p>With 3), for PET-PEESE, (a) put the SE in a meta-regression as a single predictor of the effect sizes. If the intercept is not significant, then the intercept is the estimate of your effect, adjusted for small-studies effects, along with its 95% CI.</p>
<p>(b) If intercept <strong>is</strong> significant, then run a second model and replace SE with its squared form, SE^2. Now the intercept and its 95% CI are your adjustment for small-sample effects.</p>
<p>(c) Compare the values to the naive mean (with no predictors). Obviously, estimates that are close together imply less "publication bias" (or small-studies effects).</p>
<p>(d) My stock advice: Beware heterogeneity! Under heterogeneity, such estimates are prone to error, as what is needed are more complex models that account for heterogeneity.</p>