![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
The self-correcting nature of psychological and educational science has been seriously questioned. Recent special issues of Perspectives on Psychological Science and Psychology of Aesthetics, Creativity, and the Arts have roundly condemned current organizational models of research and dissemination and have criticized the perverse incentive structure that tempts researchers into generating and publishing false positive findings. At the same time, replications are rarely attempted, allowing untruths to persist in the literature unchallenged. In this article, the editors of the Journal of Advanced Academics consider this situation and announce new policies for quantitative submissions. They are (a) an explicit call for replication studies; (b) new instructions directing reviewers to base their evaluation of a study’s merit on the quality of the research design, execution, and written description, rather than on the statistical significance of its results; and (c) an invitation to omit statistical hypothesis tests in favor of reporting effect sizes and their confidence limits.
References
|
Begley, C. G., Ellis, L. M. (2012). Raise standards for preclinical cancer research. Nature, 483, 531-533. Google Scholar | Crossref | Medline | ISI | |
|
Bennett, C. M., Baird, A. A., Miller, M. B., Wolford, G. L. (2010). Neural correlates of interspecies perspective taking in the post-mortem Atlantic salmon: An argument for proper multiple comparisons correction. Journal of Serendipitous and Unexpected Results, 1, 1-5. Retrieved from http://faculty.vassar.edu/abbaird/about/publications/pdfs/bennett_salmon.pdf Google Scholar | |
|
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003. Google Scholar | Crossref | ISI | |
|
Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York, NY: Routledge. Google Scholar | |
|
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25, 7-29. Google Scholar | SAGE Journals | ISI | |
|
Eich, R. (2014). Business not as usual. Psychological Science, 25, 3-6. doi:10.1177/0956797613512465 Google Scholar | SAGE Journals | ISI | |
|
Ferguson, C. J., Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7, 555-561. Google Scholar | SAGE Journals | ISI | |
|
Ioannidis, J. P. A. (2005). Why most published research findings are false. PloS MEDICINE, 2, 696-701. Google Scholar | Crossref | ISI | |
|
Ioannidis, J. P. A. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological Science, 7, 645-654. Google Scholar | SAGE Journals | ISI | |
|
John, L. K., Loewenstein, G., Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524-532. doi:10.1177/0956797611430953 Google Scholar | SAGE Journals | ISI | |
|
Kerr, N. L. (1998). HARKing: Hypothesizing after results are known. Personality and Social Psychology Review, 2, 196-217. Google Scholar | SAGE Journals | |
|
Makel, M. C. (2014). The empirical march of science: Making science better at self-correction. Psychology of Aesthetics, Creativity, and the Arts, 8, 2-7. Google Scholar | Crossref | |
|
Makel, M. C., Plucker, J. A. (2012, April). Replications in giftedness and creativity research. Paper presented at the annual meeting of the American Educational Research Association (AERA), San Francisco, CA. Google Scholar | |
|
Makel, M. C., Plucker, J. A., Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7, 537-542. Google Scholar | SAGE Journals | ISI | |
|
Matthews, M. S., Gentry, M., McCoach, D. B., Worrell, F. C., Matthews, D., Dixon, F. (2008). Evaluating the state of a field: Effect size reporting in gifted education. Journal of Experimental Education, 77, 55-68. doi:10.3200/JXE.77.1.55-68 Google Scholar | Crossref | ISI | |
|
Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147-163. doi:10.1037/1082-989X.9.2.147 Google Scholar | Crossref | Medline | ISI | |
|
McBee, M., Matthews, M. S. (2014). Change starts with journal editors: A commentary on Makel (2014). Psychology of Aesthetics, Creativity, and the Arts, 8, 8-10. Google Scholar | Crossref | |
|
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638-641. Google Scholar | Crossref | ISI | |
|
Schimmack, U. (2012). The ironic effect of significant results on the credibility of multiple-study articles. Psychological Methods, 17, 551-566. doi:10.1037/a0029487 Google Scholar | Crossref | Medline | ISI | |
|
Simmons, J. P., Nelson, L. D., Simonsohn, U. (2011). False–positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. Google Scholar | SAGE Journals | ISI | |
|
Thompson, B. (2007). Effect sizes, confidence intervals, and confidence intervals for effect sizes. Psychology in the Schools, 44, 423-432. doi:10.1002/pits.20234 Google Scholar | Crossref |
