Apply for the QUEST Center’s meta-research course, where students will learn about meta-research by working as a team to design, conduct and publish a meta-research study.
What is meta-research?
Meta-research is research on research. Meta-research, or science of science studies, can help us to improve science by identifying common problems with the design, conduct, analysis and reporting of scientific studies. This allows us to develop targeted solutions to improve scientific research. Meta-research may also examine other topics that influence research practices, including hiring and promotion practices, educational programs, journal guidelines and funding agency policies.
In this participant guided learn by doing course, students will work as a team to complete a meta-research study. All participants will be listed as authors on a meta-research paper, which will be submitted to a peer-reviewed journal for publication at the conclusion of the course. After completing the course, students will be able to apply the skills that they have learned to address problems in their own fields.
Course language: English
This course is funded by the Berlin University Alliance and is open to students in the biomedical or biological sciences at Charité - Universitätsmedizin Berlin, Humboldt-Universität zu Berlin, Freie Universität Berlin or Technische Universität Berlin. The course is open to PhD students, medical students and MD/PhD students who actively participate in research. Masters students who have significant research experience may also apply.
Tracey L. Weissgerber, Ph.D. Meta-researcher QUEST – Quality | Ethics | Open Science | Translation BIH Center for Transforming Biomedical Research Charité - Universitätsmedizin Berlin Berlin Institute of Health (BIH) Email: email@example.com
Dr. Weissgerber is a meta-researcher at QUEST (Quality | Ethics | Open Science | Translation) in Berlin. She completed her graduate work in physiology at Queen’s University (Canada), a post-doctoral fellowship at the Magee-Women’s Research Institute (USA), and was an Assistant Professor at Mayo Clinic (USA). Her 2015 paper on bar graphs has been viewed more than 350,000 times. This paper contributed to policy changes in many journals that encourage authors to replace bar graphs of continuous data with more informative graphics (dot plots, box plots, violin plots).
Student Assistant: TBD
ECTS & Time commitment
Interested in receiving ECTS for this course? We will provide a packet (signed certificate/information on the course etc.) to help students obtain ECTS credit from their institution or department (estimated ECTS: 5-6 credits). As this course is open for students from various programs, obtaining formal approval of the program is not feasible and students themselves are responsible for ensuring that they will receive credit for the course.
This elective course will run from late June to the end of November.
Students will participate in a 90-minute class video conference each week, which will combine a seminar to teach new material with active discussions about how to proceed with each phase of the meta-research project. Students will interact virtually through Microsoft Teams throughout the course. It is estimated that students will spend, on average, one day per week throughout the course working on course activities.
After completing the course, students will be able to:
- Define meta-research, and explain how scientists can use meta-research to improve science
- Critique study designs for systematic review/literature survey style meta-research studies by evaluating scientific rigor and feasibility
- Write screening, abstraction and analysis protocols for a meta-research study
- Implement procedures to train the study team in data screening and analysis
- Conduct a systematic review/literature survey style meta-research study
- Create a public repository containing the results of the meta-research study, including protocols, data and meta-data
- Use the results of the meta-research study to develop targeted solutions to the problems identified
- Prepare a communication strategy to explain the results of the meta-research study to active researchers
- Write a manuscript detailing the results of the meta-research study
June 29 to July 5
What is meta-research? How can we use meta-research to change science?
Preparation: Select and read at least three articles from the science of science reading list.
Designing a meta-research study
Preparation: Review detailed protocols for two meta-research studies
2 & 3
Virtual brainstorming: We will hold one Virtual Brainstorming Day to discuss each of the ideas proposed by the instructors, and an additional Virtual Brainstorming Day to discuss proposals from the class and select “wild card” ideas in weeks 2 and 3 of the course. Class members will divide into groups to draft protocols for each proposed project.
You can find information on Virtual Brainstorming Days here: https://osf.io/c5gyz/. You’ll be asked to check in on Teams 2-3 times over the course of each brainstorming day to share your thoughts and respond to others’ comments.
Assessing feasibility: Students will complete feasibility assessments for each proposed project.
Selecting a project
Preparation: Rank the project ideas in order, and provide reasons for your ranking. Consider the following criteria:
1. Quality of the study design
2. Feasibility (time & resources needed to complete the project)
3. Is the project a good fit for the expertise of the team?
4. Potential to change scientific practice (Are solutions to the problem available? How might the project lead to new solutions?)
5. Personal interest
July 27-Aug 2
Creating screening, abstraction & analysis protocols
Class members will divide into three groups to work on the screening, data abstraction and data analysis protocols. Each group will develop and test their protocol, revise their protocol, and create a training set of 25 articles.
Deliverable: Screening and abstraction protocols
Screening & selection
Preparation: Log onto Rayyan and complete screening for the training articles
Why you need a dissemination strategy, and how to design one that works
Deliverable: During weeks 6 & 7, the team will complete screening, the study flow chart, and the final study protocol
Abstractor training 1: Testing & revising a data abstraction protocol
Creating a repository, Abstractor training 2
Deliverable: Final protocol deposited on OSF, all abstractors completed training, data abstraction plan & timeline
Aug 31-Oct 25
Data abstraction, resolving discrepancies
Weekly call to discuss progress, resolve issues and discrepancies, deal with challenges that arise during data abstraction
Oct 26 – Nov 1
Communication strategies – introduction, methods, visualizations
Deliverable: Draft introduction, methods, educational visualizations
Data figures, writing results
· Revised drafts of introduction, methods, educational visualizations
· First draft of data figures, results
Discussion, Revising the manuscript
Deliverables: Final draft of manuscript
Leadership – Career Development
Students will work in teams to lead different sections of the research project. The leadership team for each phase of the project will be responsible for designing the strategy for completing that phase, and working with all class participants to complete the required tasks.
Phases include the following:
- Organizing Virtual Brainstorming Days for each project
- Project selection
- Protocol development
- Data abstraction
- Data analysis
- Depositing data and protocols 8. Manuscript preparation
Course marks will be assigned based on the following criteria:
- Protocol: 10% (same mark assigned to all course participants)
- Manuscript: 30% (same mark assigned to all course participants)
- Repository: 10% (same mark assigned to all course participants)
- Performance on tasks led by the student: 15%
- Contribution to tasks led by others: 35%