Tools for Those Who Summarize the Evidence Base
Resources and networking for those who conduct or interpret meta-analyses related to any phenomenon that is gauged in multiple studies.
Hi Everyone! I'm currently planning a project that I'd like get underway but it's still unclear to me whether or not it's even possible. Enter this group! I want to look at researcher decisions when using structural equation modeling (SEM). Specifically, I want to examine which decisions contribute to changes in overall model fit. I could simply look at reported chi square values but I'd like to roll this up into a meta-analytic framework.
I am conceptualizing the overall model chi square as the effect size where instead of variable X relating with variable Y, it represents the "correlation" between the theoretical and observed model. A chi square of zero represents a perfect "correlation" between theory and data and large chi square represents a low correlation.
My question is if it's possible to calculate standardized effect sizes and variances from a chi square goodness of fit statistic from an SEmodel? I've read about Cramer's V for 2X2 contingency tables and that it *could* be applied to goodness of fit statistics, but I have not be able to find anything further about this. Could this be applied to chi-square goodness of fit for SEM?
I've also read about the possibility of using the log likelihood value in some meta-analyses but I don't think it was as an effect size. Would it be possible to use the log likelihood value as an effect size that's then converted into a standardized metric? If so, I believe the log likelihood function could be estimated from the chi-square value and used to create standardized effect sizes but I really do not know. So with that I have come to seek the experts' advice. Do you think this is even possible?
I also plan to post this on the MASEM page and get Dr. Cheung's thoughts.
Thanks!
Garett
Tags:
Hi Garett,
It sounds interesting. But it appears that there are lots of issues that you need to address. The followings are some thoughts.
1. log likelihood is *not* an effect size. The difference between the log likelihood of the proposed model and the saturated model is the chi-square statistic reported in SEM.
2. Similar to t statistic and F statistic, chi-square statistic in *not* an effect size. It depends on the sample size.
3. Independent and dependent t statistics can be converted into d because it compares two means only (simple hypothesis). Similar to the F statistic, the chi-square statistic in SEM is testing a composite hypothesis.
4. There are a couple of factors affecting the chi-square statistic: (a) sample size; (b) model complexity (degrees of freedom); (c) degree of misspecification; and (d) violation of distribution.
If you want to examine what factors (not decisions) contribute to changes in overall model fit, you may do a computer simulation.
Hope it helps.
Mike
Hello Dr. Cheung,
Thank you for your response - it was very helpful.
Garett
© 2022 Created by Blair T. Johnson. Powered by