Tools for Those Who Summarize the Evidence Base
I have several questions regarding pros, cons, and/or special considerations related to including different study designs in a single meta-analysis. Any literature and/or guidance related to these questions will be much appreciated.
Specifically, I am curious about a scenario in which one is interested in elucidating the consistency and strength of an association between exposure X and outcome Y.
(1) Does it make sense to include both cross-sectional, retrospective, and prospective study designs to address this issue?
(2) Alternately, could one argue that including only prospective studies is preferable given that it strengthens the ability to draw causal inferences?
(3) If one were to include studies with alternate study designs, could study design be analyzed as a moderator variable?
(4) What are the implications if the ratio of cross-sectional to prospective studies is large?
Thank you in advance for consideration of this question.
Mixing designs in a single meta-analysis is a controversial topic.
I suppose at least you could start with them in the same analysis, and at least it can tell you how parallel the results are across these design types, yes, using study design as a moderator variable.
Traditionally, prospective studies have been regarded as superior to retrospective and cross-sectional studies, but it is possible that other methods flaws would be worse. You could consider gauging the quality of the methods using one or more of the scales that are out there for this purpose. As a rule, other factors equal, trust the results of the higher quality studies more.
That advice might lead you to look solely at the highest quality studies (which are probably prospective), but if you do that you will not learn how parallel the results are in the others. Perhaps your literature already offers guidance on design type, in which case others have already grappled with the question. If they have used high-quality review strategies, then you can trust their results and knowledge inferences more. If the literature has not grappled with the question with high-quality review methods, then it would seem that looking at all of the types of studies is indicated, again pursuing the best quality-methods. (Always the best!)
If you have very few studies of a particular design type it is harder to draw inferences about the results from that category, especially if the results differ widely from the others.
Other members could weigh in on your question, but there's a bit of advice. Hope it is helpful!