Statistical Inference

FINISHED PROJECTS ABOUT HOW RESEARCHERS INTERPRET STATISTICAL RESULTS:

Van den Akker, O. R., Dominguez Alvarez, L., Bakker, M., Wicherts, J. M., Van Assen, M. A. L. M. (under review). How do academics assess the results of multiple experiments?

Among other things, we carried out a preregistered analysis of individual researcher data to examine which heuristics academics use to assess the results of multiple experiments. Only 6 out of 312 (1.9%) participants used the normative method of Bayesian inference, whereas the majority of participants used vote counting approaches. Our findings highlight that most academics structurally undervalue the validity of the underlying theory when assessing the results of scientific papers with multiple experiments.

Preprint

Data and materials

Van Assen, M. A. L. M., Van den Akker, O. R., Augusteijn, H. E. M., Bakker, M., Nuijten, M. B., Olsson-Collentine, A., Stoevenbelt, A. H., Wicherts, J. M., Van Aert, R. C. M. (under review). The meta-plot: A graphical tool for interpreting the results of a meta- analysis.

The meta-plot is a descriptive visual tool for meta-analysis that provides information on the primary studies in the meta-analysis and the results of the meta-analysis. More precisely, the meta-plot portrays (i) the precision and statistical power of the primary studies in the meta-analysis, (ii) the estimate and confidence interval of a random-effects meta-analysis, (iii) the results of a cumulative random-effects meta-analysis yielding a robustness check of the meta-analytic effect size with respect to primary studies’ precision, and (iv) evidence of publication bias.

Olmo van den Akker
Olmo van den Akker

Olmo van den Akker is a PhD-student at the Meta-Research center at Tilburg University, where he is currently studying the efficacy of preregistration and the way researchers interpret statistical results.