Publication bias in psychology: A closer look at the correlation between sample size and effect size

Audrey Helen Linden, Thomas V. Pollet, Johannes Hönekopp*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Downloads (Pure)

Abstract

Previously observed negative correlations between sample size and effect size (n-ES correlation) in psychological research have been interpreted as evidence for publication bias and related undesirable biases. Here, we present two studies aimed at better understanding to what extent negative n-ES correlations reflect such biases or might be explained by unproblematic adjustments of sample size to expected effect sizes. In Study 1, we analysed n-ES correlations in 150 meta-analyses from cognitive, organizational, and social psychology and in 57 multiple replications, which are free from relevant biases. In Study 2, we used a random sample of 160 psychology papers to compare the n-ES correlation for effects that are central to these papers and effects selected at random from these papers. n-ES correlations proved inconspicuous in meta-analyses. In line with previous research, they do not suggest that publication bias and related biases have a strong impact on meta-analyses in psychology. A much higher n-ES correlation emerged for publications’ focal effects. To what extent this should be attributed to publication bias and related biases remains unclear.
Original languageEnglish
Article numbere0297075
Number of pages15
JournalPLoS One
Volume19
Issue number2
DOIs
Publication statusPublished - 15 Feb 2024

Cite this