FINISHED PROJECTS ABOUT HOW RESEARCHERS INTERPRET STATISTICAL RESULTS:
Van den Akker, O. R., Dominguez Alvarez, L., Bakker, M., Wicherts, J. M., Van Assen, M. A. L. M. (under review). How do academics assess the results of multiple experiments?
Among other things, we carried out a preregistered analysis of individual researcher data to examine which heuristics academics use to assess the results of multiple experiments. Only 6 out of 312 (1.9%) participants used the normative method of Bayesian inference, whereas the majority of participants used vote counting approaches. Our findings highlight that most academics structurally undervalue the validity of the underlying theory when assessing the results of scientific papers with multiple experiments.
Van Assen, M. A. L. M., Van den Akker, O. R., Augusteijn, H. E. M., Bakker, M., Nuijten, M. B., Olsson-Collentine, A., Stoevenbelt, A. H., Wicherts, J. M., Van Aert, R. C. M. (under review). The meta-plot: A graphical tool for interpreting the results of a meta- analysis.
The meta-plot is a descriptive visual tool for meta-analysis that provides information on the primary studies in the meta-analysis and the results of the meta-analysis. More precisely, the meta-plot portrays (i) the precision and statistical power of the primary studies in the meta-analysis, (ii) the estimate and confidence interval of a random-effects meta-analysis, (iii) the results of a cumulative random-effects meta-analysis yielding a robustness check of the meta-analytic effect size with respect to primary studies’ precision, and (iv) evidence of publication bias.