FIELD_Psychological_Methods_NOV_2021.pdf (1.14 MB)
Estimating the change in meta-analytic effect size estimates after the application of publication bias adjustment methods
journal contribution
posted on 2023-06-10, 01:50 authored by Martina SladekovaMartina Sladekova, Lois E A Webb, Andy FieldAndy FieldPublication bias poses a challenge for accurately synthesizing research findings using meta-analysis. A number of statistical methods have been developed to combat this problem by adjusting the meta-analytic estimates. Previous studies tended to apply these methods without regard to optimal conditions for each method’s performance. The present study sought to estimate the typical effect size attenuation of these methods when they are applied to real meta-analytic data sets that match the conditions under which each method is known to remain relatively unbiased (such as sample size, level of heterogeneity, population effect size, and the level of publication bias). Four-hundred and 33 data sets from 90 articles published in psychology journals were reanalyzed using a selection of publication bias adjustment methods. The downward adjustment found in our sample was minimal, with greatest identified attenuation of b = –.032, 95% highest posterior density interval (HPD) ranging from –.055 to –.009, for the precision effect test (PET). Some methods tended to adjust upward, and this was especially true for data sets with a sample size smaller than 10. We propose that researchers should seek to explore the full range of plausible estimates for the effects they are studying and note that these methods may not be able to combat bias in small samples (with less than 10 primary studies). We argue that although the effect size attenuation we found tended to be minimal, this should not be taken as an indication of low levels of publication bias in psychology. We discuss the findings with reference to new developments in Bayesian methods for publication bias adjustment, and the recent methodological reforms in psychology.
History
Publication status
- Published
File Version
- Accepted version
Journal
Psychological MethodsISSN
1082-989XPublisher
American Psychological AssociationExternal DOI
Department affiliated with
- Psychology Publications
Full text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2021-11-23First Open Access (FOA) Date
2021-11-23First Compliant Deposit (FCD) Date
2021-11-22Usage metrics
Categories
No categories selectedLicence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC