Jump to page content

A shorter version of this list was originally developed by the eLife Ambassadors Meta-research team and posted on ECR Life.

Looking for ways to improve your manuscripts and peer reviews?

This “science of science” reading list can help!

When reviewing papers or writing your own papers, it’s important to remember that the fact that something is standard practice for your field doesn’t mean that it’s the best way of doing things. Scientists, journals and funding agencies are increasingly recognizing the limitations of many existing practices and are implementing new policies to improve transparency, rigor and reproducibility. The eLife Ambassadors meta-research group has prepared a list of meta-research articles for authors and peer reviewers. These “science of science” papers will help peer reviewers learn to identify and understand the problems with some very common practices.

This collection of articles also offer constructive solutions that make it easier for authors to improve transparency, rigor and reproducibility. Citing relevant meta-research articles in reviews is one step that peer reviewers can take to improve the quality of the scientific literature. While the articles in the list focus on practices that are common in the basic biomedical sciences, these practices are also used in many other fields that typically have small sample sizes. Each topic is clearly labeled, so it’s easy to identify articles that are relevant to your discipline.

If you participate in a pre-print or peer review journal club (such as ASAPBio), you may want to try discussing some of these articles at one of your sessions. Sharing information about best practices with your colleagues may improve the quality of future discussions and reviews.

Meta-research Articles to Improve Transparency & Reproducibility

  1. Why you shouldn’t use bar graphs to show continuous data (and what to do instead)

    Follow-up paper with information on how to recognize & fix other common visualization problems: https://www.ahajournals.org/doi/10.1161/CIRCULATIONAHA.118.037777

    Free online tools for creating better graphics for scientific publications
  2. How to create clear and informative image-based figures (photos, microscopy, electron microscopy, clinical imaging techniques)
  3. The problem with underpowered studies
    (Low power is a problem even when you find a significant difference)
  4. Why it’s important to report all excluded observations & the reasons for exclusion
  5. P-values are often reported incorrectly:
    Why it’s important to present the information needed to verify the test result
  6. Why we need to report more than “Data were analyzed by t-tests or ANOVA”:
  7. Why you can’t conclude that two experimental effects are different because one is statistically significant, and the other is not: https://www.nature.com/articles/nn.2886
  8. When is a case-control study not a case-control study?
  9. Unblinded studies find larger effects
  10. Animal research: Follow the ARRIVE guidelines to improve transparency and reproducibility
  11. Animal research: Multi-lab studies may improve reproducibility
  12. Check for clusters of non-independent data (replicates, mice from the same litter, etc.):
    Did the authors account for non-independence in their analysis? https://bmcneurosci.biomedcentral.com/articles/10.1186/1471-2202-11-5
  13. Beware of image manipulation

Educational Articles

  1. Common misconceptions about data analysis and statistics
  2. Strategies for improving data analysis and reporting in experimental biology: http://jpet.aspetjournals.org/content/372/1/136
  3. Small samples are more likely to give spurious results
  4. Transparency is the key to quality
  5. Research methods: Know when your numbers are significant