Small studies are more heterogeneous than large ones: a meta-meta-analysis. 2015

Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
Radboud Institute for Health Sciences (RIHS), Radboud University Medical Center, Mailbox 133, P.O. Box 9101, Nijmegen 6500 HB, The Netherlands. Electronic address: joanna.inthout@radboudumc.nl.

OBJECTIVE Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). METHODS We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. RESULTS Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. CONCLUSIONS Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.

UI MeSH Term Description Entries
D008962 Models, Theoretical Theoretical representations that simulate the behavior or activity of systems, processes, or phenomena. They include the use of mathematical equations, computers, and other electronic equipment. Experimental Model,Experimental Models,Mathematical Model,Model, Experimental,Models (Theoretical),Models, Experimental,Models, Theoretic,Theoretical Study,Mathematical Models,Model (Theoretical),Model, Mathematical,Model, Theoretical,Models, Mathematical,Studies, Theoretical,Study, Theoretical,Theoretical Model,Theoretical Models,Theoretical Studies
D012107 Research Design A plan for collecting and utilizing data so that desired information can be obtained with sufficient precision or so that an hypothesis can be tested properly. Experimental Design,Data Adjustment,Data Reporting,Design, Experimental,Designs, Experimental,Error Sources,Experimental Designs,Matched Groups,Methodology, Research,Problem Formulation,Research Methodology,Research Proposal,Research Strategy,Research Technics,Research Techniques,Scoring Methods,Adjustment, Data,Adjustments, Data,Data Adjustments,Design, Research,Designs, Research,Error Source,Formulation, Problem,Formulations, Problem,Group, Matched,Groups, Matched,Matched Group,Method, Scoring,Methods, Scoring,Problem Formulations,Proposal, Research,Proposals, Research,Reporting, Data,Research Designs,Research Proposals,Research Strategies,Research Technic,Research Technique,Scoring Method,Source, Error,Sources, Error,Strategies, Research,Strategy, Research,Technic, Research,Technics, Research,Technique, Research,Techniques, Research
D002986 Clinical Trials as Topic Works about pre-planned studies of the safety, efficacy, or optimum dosage schedule (if appropriate) of one or more diagnostic, therapeutic, or prophylactic drugs, devices, or techniques selected according to predetermined criteria of eligibility and observed for predefined evidence of favorable and unfavorable effects. This concept includes clinical trials conducted both in the U.S. and in other countries. Clinical Trial as Topic
D004812 Epidemiologic Methods Research techniques that focus on study designs and data gathering methods in human and animal populations. Epidemiologic Method,Epidemiological Methods,Methods, Epidemiologic,Epidemiological Method,Method, Epidemiologic,Method, Epidemiological,Methods, Epidemiological
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D001499 Bayes Theorem A theorem in probability theory named for Thomas Bayes (1702-1761). In epidemiology, it is used to obtain the probability of disease in a group of people with some characteristic on the basis of the overall rate of that disease and of the likelihood of that characteristic in healthy and diseased individuals. The most familiar application is in clinical decision analysis where it is used for estimating the probability of a particular diagnosis given the appearance of some symptoms or test result. Bayesian Analysis,Bayesian Estimation,Bayesian Forecast,Bayesian Method,Bayesian Prediction,Analysis, Bayesian,Bayesian Approach,Approach, Bayesian,Approachs, Bayesian,Bayesian Approachs,Estimation, Bayesian,Forecast, Bayesian,Method, Bayesian,Prediction, Bayesian,Theorem, Bayes

Related Publications

Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
February 2019, Nature,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
October 2020, Science advances,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
December 2018, Annals of botany,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
September 2022, Proceedings of the National Academy of Sciences of the United States of America,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
May 2021, Science advances,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
January 2016, Biology of sex differences,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
October 2019, Harefuah,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
March 2000, BMJ (Clinical research ed.),
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
January 2013, PloS one,
Joanna IntHout, and John P A Ioannidis, and George F Borm, and Jelle J Goeman
April 2021, ACS chemical neuroscience,
Copied contents to your clipboard!