Other Meta-Research

FINISHED META-RESEARCH PROJECTS THAT ARE NOT ABOUT PREREGISTRATION OR STATISTICAL INFERENCE:

Schweinsberg, M., Feldman, M., Staub, N., …, Van den Akker, O. R., …, & Uhlmann, E.L. (under review). Radical dispersion in estimates when independent scientists operationalize and test the same hypothesis with the same data.

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientist gender and status on verbosity in intellectual debates. Not only the analytic approach but also the operationalization of key variables was left unconstrained and up to individual researchers. Researchers reported radically dispersed estimates, in a number of cases obtaining significant effects in opposite directions for the same research question.