×

Testing for homogeneity in meta-analysis. I. The one-parameter case: standardized mean difference. (English) Zbl 1217.62173

Summary: Meta-analysis seeks to combine the results of several experiments in order to improve the accuracy of decisions. It is common to use a test for homogeneity to determine if the results of the several experiments are sufficiently similar to warrant their combination into an overall result. W. G. Cochran’s [J. R. Stat. Soc. 4, Suppl., 102–118 (1937; Zbl 0019.13003)] \(Q\) statistic is frequently used for this homogeneity test. It is often assumed that \(Q\) follows a chi-square distribution under the null hypothesis of homogeneity, but it has long been known that this asymptotic distribution for \(Q\) is not accurate for moderate sample sizes. We present an expansion for the mean of \(Q\) under the null hypothesis that is valid when the effect and the weight for each study depend on a single parameter, but for which neither normality nor independence of the effect and weight estimators is needed. This expansion represents an order \(O(1/n)\) correction to the usual chi-square moment in the one-parameter case.
We apply the result to the homogeneity test for meta-analyses in which the effects are measured by the standardized mean difference (Cohen’s \(d\)-statistic). In this situation, we recommend approximating the null distribution of \(Q\) by a chi-square distribution with fractional degrees of freedom that are estimated from the data using our expansion for the mean of \(Q\). The resulting homogeneity test is substantially more accurate than the currently used test. We provide a program available at the Paper Information link at the Biometrics website http://www.biometrics.tibs.org for making the necessary calculations.

MSC:

62P10 Applications of statistics to biology and medical sciences; meta analysis
62E20 Asymptotic distribution theory in statistics
62N03 Testing in survival analysis and censored data
65C60 Computational problems in statistics (MSC2010)

Citations:

Zbl 0019.13003
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Biggerstaff, The exact distribution of Cochran’s heterogeneity statistic in one-way random effects meta-analysis, Statistics in Medicine 27 pp 6093– (2008) · doi:10.1002/sim.3428
[2] Biggerstaff, Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis, Statistics in Medicine 16 pp 753– (1997) · doi:10.1002/(SICI)1097-0258(19970415)16:7<753::AID-SIM494>3.0.CO;2-G
[3] Cochran, Problems arising in the analysis of a series of similar experiments, Journal of the Royal Statistical Society 4 pp 102– (1937) · Zbl 0019.13003
[4] Cohen, Statistical Power Analysis for the Behavioral Sciences (1988) · Zbl 0747.62110
[5] Firth, Bias reduction of maximum likelihood estimates, Biometrika 80 pp 27– (1993) · Zbl 0769.62021 · doi:10.1093/biomet/80.1.27
[6] Hedges, Statistical Methods for Meta-analysis (1985) · Zbl 0666.62002
[7] Hróbjartsson, Placebo interventions for all clinical conditions, Cochrane Database of Systematic Reviews (2004) · doi:10.1002/14651858.CD003974.pub2
[8] Jackson, The power of the standard test for the presence of heterogeneity in meta-analysis, Statistics in Medicine 25 pp 2688– (2006) · doi:10.1002/sim.2481
[9] James, The comparison of several groups of observations when the ratios of the population variances are unknown, Biometrika 38 pp 324– (1951) · Zbl 0043.14002 · doi:10.1093/biomet/38.3-4.324
[10] Kulinskaya, Robust weighted one-way ANOVA: Improved approximation and efficiency, Journal of Statistical Planning and Inference 137 pp 462– (2007) · Zbl 1102.62075 · doi:10.1016/j.jspi.2006.01.008
[11] Kulinskaya , E. Dollinger , M. Bjørkestøl , K. 2009 Testing for homogeneity in meta-analysis I. The one parameter case: Standardized mean difference · Zbl 1217.62173
[12] Kulinskaya, A Welch-type test for homogeneity of contrasts under heteroscedasticity with application to meta-analysis, Statistics in Medicine 23 pp 3655– (2004) · doi:10.1002/sim.1929
[13] Olkin, Unbiased estimation of certain correlation coefficients, Annals of Mathematical Statistics 29 pp 201– (1958) · Zbl 0094.14403 · doi:10.1214/aoms/1177706717
[14] Petitti, Approaches to heterogeneity in meta-analysis, Statistics in Medicine 20 pp 3625– (2001) · doi:10.1002/sim.1091
[15] Sánchez-Meca, Homogeneity tests in meta-analysis: A Monte Carlo comparison of statistical power and type I error, Quality & Quantity 31 pp 385– (1997) · doi:10.1023/A:1004298118485
[16] Sánchez-Meca, Testing the significance of a common risk difference in meta-analysis, Computational Statistics & Data Analysis 33 pp 299– (2000) · Zbl 1061.62564 · doi:10.1016/S0167-9473(99)00055-9
[17] Tuunainen, Light therapy for non-seasonal depression, Cochrane Database of Systematic Reviews (2004) · doi:10.1002/14651858.CD004050.pub2
[18] Viechtbauer, Hypothesis tests for population heterogeneity in meta-analysis, British Journal of Mathematical and Statistical Psychology 60 pp 29– (2007) · doi:10.1348/000711005X64042
[19] Welch, On the comparison of several mean values: An alternative approach, Biometrika 38 pp 330– (1951) · Zbl 0043.14101 · doi:10.1093/biomet/38.3-4.330
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.