DIENES_Advances in Methods and Practices in Psychological Science_DEC_2019_author_copy.pdf (420.55 kB)
Why bayesian “evidence for H1” in one condition and bayesian “evidence for H0” in another condition does not mean good-enough bayesian evidence for a difference between the conditions
journal contribution
posted on 2023-06-09, 20:11 authored by Bence Palfi, Zoltan DienesZoltan DienesPsychologists are often interested in whether an independent variable has a different effect in condition A than in condition B. To test such a question, one needs to directly compare the effect of that variable in the two conditions (i.e., test the interaction). Yet many researchers tend to stop when they find a significant test in one condition and a nonsignificant test in the other condition, deeming this as sufficient evidence for a difference between the two conditions. In this Tutorial, we aim to raise awareness of this inferential mistake when Bayes factors are used with conventional cutoffs to draw conclusions. For instance, some researchers might falsely conclude that there must be good-enough evidence for the interaction if they find good-enough Bayesian evidence for the alternative hypothesis, H1, in condition A and good-enough Bayesian evidence for the null hypothesis, H0, in condition B. The case study we introduce highlights that ignoring the test of the interaction can lead to unjustified conclusions and demonstrates that the principle that any assertion about the existence of an interaction necessitates the direct comparison of the conditions is as true for Bayesian as it is for frequentist statistics. We provide an R script of the analyses of the case study and a Shiny app that can be used with a 2 × 2 design to develop intuitions on this issue, and we introduce a rule of thumb with which one can estimate the sample size one might need to have a well-powered design.
History
Publication status
- Published
File Version
- Accepted version
Journal
Advances in Methods and Practices in Psychological ScienceISSN
2515-2459Publisher
SAGE PublicationsExternal DOI
Issue
3Volume
3Page range
300-308Department affiliated with
- Psychology Publications
Full text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2020-01-09First Open Access (FOA) Date
2020-01-09First Compliant Deposit (FCD) Date
2020-01-08Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC