When should researchers cite study differences in response to a failure to replicate? 2022

David Colaço, and John Bickle, and Bradley Walters
Munich Center for Mathematical Philosophy, LMU Munich, Munich, Germany.

Scientists often respond to failures to replicate by citing differences between the experimental components of an original study and those of its attempted replication. In this paper, we investigate these purported mismatch explanations. We assess a body of failures to replicate in neuroscience studies on spinal cord injury. We argue that a defensible mismatch explanation is one where (1) a mismatch of components is a difference maker for a mismatch of outcomes, and (2) the components are relevantly different in the follow-up study, given the scope of the original study. With this account, we argue that not all differences between studies are meaningful, even if they are difference makers. As our examples show, focusing only on these differences results in disregarding the representativeness of the original experiment's components and the scope of its outcomes, undercutting other epistemic aims, such as translation, in the process.

UI MeSH Term Description Entries

Related Publications

David Colaço, and John Bickle, and Bradley Walters
July 2016, Perspectives on psychological science : a journal of the Association for Psychological Science,
David Colaço, and John Bickle, and Bradley Walters
January 1991, The Florida nurse,
David Colaço, and John Bickle, and Bradley Walters
May 2007, Journal of the experimental analysis of behavior,
David Colaço, and John Bickle, and Bradley Walters
November 1984, Journal of personality and social psychology,
David Colaço, and John Bickle, and Bradley Walters
April 2016, Attention, perception & psychophysics,
David Colaço, and John Bickle, and Bradley Walters
July 2019, Ecology letters,
David Colaço, and John Bickle, and Bradley Walters
January 2015, The Hastings Center report,
David Colaço, and John Bickle, and Bradley Walters
July 1992, Journal of studies on alcohol,
David Colaço, and John Bickle, and Bradley Walters
January 1974, Social biology,
David Colaço, and John Bickle, and Bradley Walters
October 2019, Nature,
Copied contents to your clipboard!