Using Meta-Research to Improve Science: A Participant-Guided "Learn by Doing" Course

Concerned about the reproducibility crisis? Wondering what you can do to improve scientific rigor & reproducibility? Want to learn skills to identify practices that contribute to poor reproducibility in your own field, and develop solutions? Looking for hands on experience?

Apply for the QUEST Center’s meta-research course, where students will learn about meta-research by working as a team to design, conduct and publish a meta-research study.

What is meta-research?

Meta-research is research on research. Meta-research, or science of science studies, can help us to improve science by identifying common problems with the design, conduct, analysis and reporting of scientific studies. This allows us to develop targeted solutions to improve scientific research. Meta-research may also examine other topics that influence research practices, including hiring and promotion practices, educational programmes, journal guidelines and funding agency policies.

Please note that this is not a course on meta-analysis. Meta-analysis refers to the process of systematically identifying all studies that address a particular research question (systematic review), and combining the results of those studies to estimate the size and direction of an effect. While meta-research borrows some techniques from systematic reviews, there are clear differences.

Description

In this participant guided learn by doing course, students will work as a team to complete a meta-research study. All participants will be listed as authors on a meta-research paper, which will be submitted to a peer-reviewed journal for publication at the conclusion of the course. After completing the course, students will be able to apply the skills that they have learned to address problems in their own fields.

Example papers:

Jambor H, Antonietti A, Alicea B, Audisio TL, Auer S, Bhardwaj V, et al. (2021) Creating clear and informative image-based figures for scientific publications. PLoS Biol 19(3): e3001161. https://doi.org/10.1371/journal.pbio.3001161

Kroon C, Breuer L, Jones L, An J, Akan A, Mohamed Ali EA, et al. (2022) Blind spots on western blots: Assessment of common problems in western blot figures and methods reporting with recommendations to improve them. PLoS Biol 20(9): e3001783. https://doi.org/10.1371/journal.pbio.3001783

Learning objectives

After completing the course, students will be able to:

1. Define meta-research, and explain how scientists can use meta-research to improve science
2. Critique study designs for systematic review/literature survey style meta-research studies by evaluating scientific rigour and feasibility
3. Write screening, abstraction and analysis protocols for a meta-research study
4. Implement procedures to train the study team in data screening and analysis
5. Conduct a systematic review/literature survey style meta-research study
6. Create a public repository containing the results of the meta-research study, including protocols, data and meta-data
7. Use the results of the meta-research study to develop targeted solutions to the problems identified
8. Prepare a communication strategy to explain the results of the meta-research study to active researchers
9. Write a manuscript detailing the results of the meta-research study

Prerequisites

Course participants must have some research experience.

Certificates, ECTS & time commitment

We will provide a packet (signed certificate/information on the course etc.) to help students obtain ECTS credit from their institution or department (ECTS: 6 credits). As this course is open for students from various programmes, obtaining formal approval from each programme is not feasible. Students themselves are responsible for ensuring that they will receive credit for the course.

Students will participate in a 90-minute class video conference each week, which will combine a seminar to teach new material with active discussions about how to proceed with each phase of the meta-research project. Students will interact virtually through Microsoft Teams throughout the course. It is estimated that students will spend, on average, one day per week throughout the course working on course activities. The amount of time required may change depending on the project and study design that students select.

Lecturer, instructor

Main lecturer
Tracey Weissgerber, PhD, Meta-researcher, group leader

Dr. Weissgerber is a meta-researcher at QUEST (Quality | Ethics | Open Science | Translation) in Berlin. She completed her graduate work in physiology at Queen’s University (Canada), a post-doctoral fellowship at the Magee-Women’s Research Institute (USA), and was an Assistant Professor at Mayo Clinic (USA). Her 2015 paper on bar graphs has been viewed more than 375,000 times. This paper contributed to policy changes in many journals that encourage authors to replace bar graphs of continuous data with more informative graphics (dot plots, box plots, violin plots).

Additional lecturer
Camila Baselly, student assistant

Eligibility

This course is open to students in the biomedical or biological sciences at Charité - Universitätsmedizin Berlin, Humboldt-Universität zu Berlin, Freie Universität Berlin, or Technische Universität Berlin. The course is open to PhD students, medical students and MD/PhD students who actively participate in research. Master's students who have significant research experience may also apply.