×

Testing homogeneity of effect sizes in pooling \(2\times2\) contingency tables from multiple studies: a comparison of methods. (English) Zbl 1426.62310

Summary: Meta-analysis is a statistical methodology that combines the outcomes of several independent studies. Either a fixed-effects or a random-effects model is used to combine the results of the independent studies. The choice depends on whether the outcomes can be considered homogeneous or not. Several methods have been developed to test the homogeneity of effect sizes. Although many of them have been compared already, they have not been studied in more realistic practical settings where individuals all have their own risk of an event and a complete overview is lacking. In this article, we investigate the performance of 10 statistical methods to test homogeneity of a binary exposure on a binary clinical outcome, using an extensive simulation study and a real life meta-analysis. We evaluated the Type I error and the statistical power. The fixed-effects regression model for treatment and study interaction was found to be a slightly liberal test, while the \(Q\)-statistic and the Bliss statistic can be considered conservative tests. The random-effects regression model, the Peto statistic and the \(I^2\) perform rather poorly compared to the other methods. All chi-square tests that are based on the calculation of a specific odds ratio and the fixed-effects logistic regression analysis perform best. Among these chi-square tests, we recommend the Breslow-Day test, but only for convenience purposes, since it is available in most statistical software packages and the other tests are not.

MSC:

62P10 Applications of statistics to biology and medical sciences; meta analysis
62H17 Contingency tables
62J12 Generalized linear models (logistic models)
62H15 Hypothesis testing in multivariate analysis
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Agresti, A., Categorical data analysis (2002), New York, NY: John Wiley & Sons, New York, NY · Zbl 1018.62002
[2] Bagheri, Z.; Ayatollahi, S. M. T.; Jafari, P., Comparison of three tests of homogeneity of odds ratios in multicenter trials with unequal sample sizes within and among centers, BMC Medical Research Methodology, 11, 58 (2011)
[3] Breslow, N. E.; Day, N. E., Statistical methods in cancer research 1980; Volume I—the design and analysis of case-control studies, Vol 32 of IARC Scientific Publications (1980), Lyon: International Agency for Research on Cancer, Lyon
[4] Bliss, C. L., The statistics of Bioassay: With special reference to the vitamins (1952), New York, NY: Academic Press Inc, New York, NY
[5] Breslow, N. E., Statistics in epidemiology: The case-control study, Journal of the American Statistical Association, 91, 433, 14-28 (1994) · Zbl 0870.62082 · doi:10.1080/01621459.1996.10476660
[6] Brockwell, S. E.; Gordon, I. R., A comparison of statistical methods for meta-analysis, Statistics in Medicine, 20, 6, 825-840 (2001) · doi:10.1002/(ISSN)1097-0258
[7] Cochran, W. G., The combination of estimates from different experiments, Biometrics, 10, 101-129 (1954) · Zbl 0059.12505 · doi:10.2307/3001666
[8] Dersimonian, R.; Laird, N., Meta-analysis in clinical trials, Control Clinical Trials, 7, 3, 177-188 (1986) · doi:10.1016/0197-2456(86)90046-2
[9] Gart, J. J., Alternative analyses of contingency tables, Journal of the Royal Statistical Society Series B (Methodological), 28, 1, 164-179 (1966) · Zbl 0138.13502
[10] Gavaghan, D. J.; Moore, R. A.; Mcquay, H. J., An evaluation of homogeneity tests in meta-analyses in pain using simulations of individual patient data, Pain, 85, 3, 415-424 (2000) · doi:10.1016/S0304-3959(99)00302-4
[11] Halperin, M.; Ware, J. H.; Byar, D. P.; Mantel, N.; Brown, C. C.; Koziol, J.; Green, S. B., Testing for interaction in an I×J×K contingency table, Biometrika, 64, 271-275 (1977) · Zbl 0376.62036
[12] Higgins, J. P. T.; Thompson, S. G., Quantifying heterogeneity in a meta-analysis, Statistics in Medicine, 21, 11, 1539-1558 (2002) · doi:10.1002/sim.1186
[13] Higgins, J. P. T.; Thompson, S. G.; Deeks, J. J.; Altman, D. G., Measuring inconsistency in meta-analyses, British Medical Journal, 327, 7414, 557-560 (2003) · doi:10.1136/bmj.327.7414.557
[14] Hoaglin, D. C., Misunderstandings about Q and ‘Cochran’s Q test’ in meta-analysis, Statistics in Medicine, 35, 485-495 (2016) · doi:10.1002/sim.v35.4
[15] Huedo-Medina, T. B.; Sánchez-Meca, J.; Marín-Martínez, F.; Botella, J., Assessing heterogeneity in meta-analysis: Q statistic or I2 index, Psychological Methods, 11, 2, 193-206 (2006) · doi:10.1037/1082-989X.11.2.193
[16] Jackson, D., The power of the standard test for the presence of heterogeneity in meta-analysis, Statistics in Medicine, 25, 2688-2699 (2006) · doi:10.1002/(ISSN)1097-0258
[17] Jones, M. P.; O’Gorman, T. W.; Lemke, J. H.; Woolson, R. F.; Monte Carlo, A., Investigation of homogeneity tests of the odds ratio under various sample size configurations, Biometrics, 45, 1, 171-181 (1989) · Zbl 0715.62217 · doi:10.2307/2532043
[18] Kulinskaya, E.; Dollinger, M. B.; Bjørkestøl, K., Testing for homogeneity in meta-analysis. I. The one parameter case: Standardized mean difference, Biometrics, 67, 203-212 (2011) · Zbl 1217.62173 · doi:10.1111/j.1541-0420.2010.01442.x
[19] Liang, K. Y.; Self, S. G., Tests of homogeneity of odds ratio when the data are sparse, Biometrica, 72, 353-358 (1989) · Zbl 0576.62060 · doi:10.1093/biomet/72.2.353
[20] Mantel, N.; Haenszel, W., Statistical aspects of the analysis of data from retrospective studies of disease, Journal of the National Cancer Institute, 22, 719-748 (1959)
[21] Mcculloch, C. E.; Searle, S. R.; Neuhaus, J. M., Generalized, linear, and mixed models (2008), New Jersey: Wiley, New Jersey · Zbl 1165.62050
[22] Mzolo, T.; Hendriks, M.; Van Den Heuvel, E., A comparison of statistical methods for combining relative bioactivities from parallel line bioassays, Pharmaceutical Statistics, 12, 6 (2013)
[23] Pasipanodya, J. G.; Srivastava, S.; Gumbo, T., Meta-analysis of clinical studies supports the pharmacokinetic variability hypothesis for acquired drug resistance and failure of antituberculosis therapy, Clinical Infectious Diseases, 55, 2, 169-177 (2012) · doi:10.1093/cid/cis353
[24] Paul, S. R.; Donner, A., Small sample performance of tests of homogeneity of odds ratios in K 2 × 2 tables, Statistics in Medicine, 11, 2, 159-165 (1992) · doi:10.1002/sim.4780110203
[25] Sánchez-Meca, J.; Marín-Martínez, F., Homogeneity tests in meta-analysis: A Monte Carlo comparison of statistical power and Type I error, Quality and Quantity, 31, 385-399 (1997) · doi:10.1023/A:1004298118485
[26] Jg, Skellam, A probability distribution derived from the binomial distribution by regarding the probability of success as variable between the sets of trials, Journal of the Royal Statistical Society, 10, 2, 257-261 (1948) · Zbl 0032.41903
[27] Sutton, A. J.; Higgins, J. P., Recent developments in meta-analysis, Statistics in Medicine, 27, 5, 625-650 (2008) · doi:10.1002/sim.2934
[28] Tarone, R. E., On heterogeneity tests based on efficient scores, Biometrika, 72, 91-95 (1985) · doi:10.1093/biomet/72.1.91
[29] Verbeke, G.; Molenberghs, G., The use of score tests for inference on variance components, Biometrics, 59, 254-262 (2003) · Zbl 1210.62013 · doi:10.1111/1541-0420.00032
[30] Wolfinger, R.; O’ Connell, M., Generalized linear mixed models: A pseudo-likelihood approach, Journal of Statistical Computation and Simulation, 4, 233-243 (1993) · Zbl 0833.62067 · doi:10.1080/00949659308811554
[31] Woolf, B., On estimating the relationship between blood group and disease, Annual Human Genetics, 19, 251-253 (1955) · doi:10.1111/j.1469-1809.1955.tb01348.x
[32] Yusuf, S.; Peto, R.; Lewis, J.; Collins, R.; Sleight, P., Beta blockade during and after myocardial infarction: An overview of the randomized trials, Progress in Cardiovascular Diseases, 27, 5, 335-371 (1986) · doi:10.1016/S0033-0620(85)80003-7
[33] Zelen, M., The analysis of several 2 × 2 contingency tables, Biometrika, 58, 129-137 (1971) · Zbl 0218.62061
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.