The Meta Research reading list for peer reviewers

This list was developed by the eLife Ambassadors Meta-research team and was originally posted on ECR Life.

Looking for ways to improve your manuscripts and peer reviews?

This “science of science” reading list can help!

When reviewing papers or writing your own papers, it’s important to remember that the fact that something is standard practice for your field doesn’t mean that it’s the best way of doing things. Scientists, journals and funding agencies are increasingly recognizing the limitations of many existing practices and are implementing new policies to improve transparency, rigor and reproducibility. The eLife Ambassadors meta-research group has prepared a list of meta-research articles for authors and peer reviewers. These “science of science” papers will help peer reviewers learn to identify and understand the problems with some very common practices.

This collection of articles also offer constructive solutions that make it easier for authors to improve transparency, rigor and reproducibility. Citing relevant meta-research articles in reviews is one step that peer reviewers can take to improve the quality of the scientific literature. While the articles in the list focus on practices that are common in the basic biomedical sciences, these practices are also used in many other fields that typically have small sample sizes. Each topic is clearly labeled, so it’s easy to identify articles that are relevant to your discipline.

If you participate in a pre-print or peer review journal club (such as ASAPBio), you may want to try discussing some of these articles at one of your sessions. Sharing information about best practices with your colleagues may improve the quality of future discussions and reviews.

Meta-research Articles to Improve Transparency & Reproducibility

  1. Why you shouldn’t use bar graphs to show continuous data (and what to do instead) http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002128

    Free online tools for creating better graphics for scientific publications
  2. The problem with underpowered studies
    (Low power is a problem even when you find a significant difference)
    https://www.nature.com/articles/nrn3475

  3. Why it’s important to report all excluded observations & the reasons for exclusion
    http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002331

  4. P-values are often reported incorrectly:
    Why it’s important to present the information needed to verify the test result
    https://link.springer.com/article/10.3758/s13428-015-0664-2

  5. Unblinded studies find larger effects
    http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002190

  6. Animal research: Follow the ARRIVE guidelines to improve transparency and reproducibility
  7. Animal research: Multi-lab studies may improve reproducibility
    http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2003693

  8. Check for clusters of non-independent data (replicates, mice from the same litter, etc.):
    Did the authors account for non-independence in their analysis? https://bmcneurosci.biomedcentral.com/articles/10.1186/1471-2202-11-5

  9. Beware of image manipulation
    http://mbio.asm.org/content/7/3/e00809-16.long

 


Educational Articles

  1. Common misconceptions about data analysis and statistics http://jpet.aspetjournals.org/content/351/1/200.full.pdf+html

  2. Small samples are more likely to give spurious results
    http://www.jstor.org/stable/27858964

  3. Transparency is the key to quality
    http://www.jbc.org/content/290/50/29692.full.pdf+html

  4. Research methods: Know when your numbers are significant http://www.nature.com/nature/journal/v492/n7428/full/492180a.html