Meta-Science Summer School: Designing and Conducting Studies To Improve Research
Die Teilnehmenden lernen mehr über Meta-Forschung, indem sie in einem kleinen Team Daten für eine Meta-Forschungsstudie entwerfen und zu sammeln beginnen. Die Teilnehmenden können sich dafür entscheiden, nach der Sommerschule weiterzuarbeiten, um ihr Projekt abzuschließen und zu veröffentlichen. (In Englisch)
Concerned about the reproducibility and transparency of research? Interested in learning more about meta-research, or the science of science? Looking for hands-on experience? Excited to work with other early career researchers across disciplines?
Apply for the Meta-Science Summer School to learn more about meta-research by working collaboratively in a small team to design a meta-research study. After the summer school, some groups may decide to continue working with their instructors to complete and publish their study. The Meta-Science Summer School is organized in cooperation between the Berlin Institute of Health (BIH), the German Reproducibility Network (GRN), the META REP Priority Program, iRise and the University of Coimbra EXCELScIOR project ERA chair.
What is meta-research?
Meta-research, or research on research, studies apply the scientific method to study the research process and ecosystem. This may include examining the design, conduct, reporting and dissemination of research, factors that influence the assessment of research or the composition of the research workforce. Meta-researchers may also develop, test, or implement interventions to improve research culture and practice.
Please note: This summer school is not focusing on meta-analysis. Meta-analysis refers to the process of systematically identifying all studies that address a particular research question (systematic review). While some meta-research uses similar techniques, there are clear differences to systematic reviews.
Description
Rather than focusing on a few pre-selected topics during the summer school, instructors will support participants in developing meta-reseach study proposals that are relevant to their own work and will provide value to the research community. An international group of experienced meta-researchers will work closely with each group to develop a study proposal.
Programme
Programme Akkordeon
Virtual preparatory phase
Before the summer school, participants will watch a series of videos introducing meta-research concepts and methods to maximise learning during the in-person sessions. In each 30-minute video, a meta-researcher will briefly describe a study that they conducted, and highlight important design considerations, possible challenges, and errors or design flaws that should be avoided. You can find recordings of past webinars on YouTube or register to attend live webinars.
Online project selection
During this interactive online workshop, participants will brainstorm and critically evaluate meta-research study ideas with support from experienced meta-research instructors. Participants will divide into 3-4 small project teams based on their interests.
In-presence summer school (near Frankfurt, Germany)
Each project team will work with experienced instructors (see below) to define their research question, identify pre-existing literature, design a study, prepare a proposal, develop a possible protocol, and perform feasibility testing. In the final session, each group will present their work, discuss feasibility, and decide whether to continue working with the instructors to complete their project after the summer school.
Hackathon
This half-day hackathon will take place during the in-presence summer school. Participants will learn how to curate data and what to consider when designing an automated tool to screen scientific papers or preprints for a common problem or beneficial practice (e.g. open data). Automated tools can be used to collect data, monitor uptake of good research practices, or as an intervention to improve reporting.
Instructors
Instructors Akkordeon
Instructors are internationally-recognized meta-scientists who specialise in different fields and meta-research methodologies.
Tracey Weissgerber (BIH QUEST Center): Tracey Weissgerber’s paper on the inappropriate use of bar graphs has been viewed more than 400,000 times and contributed to journal policy changes. Her meta-research specialities include data visualization, statistics, and open/reproducible methods reporting. Methods expertise: Systematic review/literature survey, automated screening, innovative educational formats.
Anita Bandrowski (University of California, San Diego): Anita Bandrowski is a founding member of the research resource identifier initiative, and founded a start-up that developed the SciScore automated screening tool. Many publishers use this tool to screen submitted manuscripts for MDAR checklist items. Methods expertise: Automated screening, meta-research using automated tools.
Malcolm Macleod (CAMARADES, University of Edinburgh): Malcolm Macleod is a clinician-scientist who has led or contributed to many transformational science improvement efforts, including the ARRIVE guidelines for animal studies, the Landis criteria, and the MDAR checklist. Methods expertise: Systematic review, meta-analysis, checklist/guideline development, interventional studies, preclinical and translational meta-research.
Malte Elson (ERROR, University of Bern): Malte Elson examines the formation and perpetuation of research norms and procedural standards in psychology. He studies the transparency, reproducibility, robustness, and replicability of scientific findings, and quantifies the credibility, validity, and methodological adequacy of research literature. Methods expertise: Systematic review, content analysis, experimental design.
Vladislav Nachev (BIH QUEST Center): Vladislav Nachev is a data scientist with a background in behavioural biology and neurobiology. He programs automated screening tools and dashboards with metrics of open and responsible research. Methods expertise: Data analysis, automated screening, experimental design.
Rachel Heyard (Center for Reproducible Science, University of Zurich): Rachel Heyard is a biostatistician and meta-researcher. Her research interests include testing new approaches for awarding research funding, statistical methods in research, and replicability.
Ian Hussey (ERROR, University of Bern): Ian Hussey’s work focuses on the meta-science of measurement and assessments of error and trustworthiness in scientific articles. Together with Malte Elson and others, he co-runs the ERROR scientific bug bounty program. Methods expertise: Measurement flexibility, simulation studies, error detection.
Costs
The Summer School is free to attend. A limited amount of travel funding is available to cover transportation costs from participants’ location to the venue. Please indicate whether you would like to apply for travel funding in your application.
ECTS
Students will receive a certificate after completing the summer school. BIH and Charité graduate students can obtain 2.0 ECTS for completing the summer school and preparatory work. As the preparatory work includes watching videos from the Meta-research Methods webinar series, students who participate in the summer school can not receive additional ECTS credit for attending the webinar series.
Scientific coordination and organisation
- Dr. Tracey Weissgerber – Team leader EXCELScIOR project ERA Chair, University of Coimbra, Portugal | Guest researcher BIH QUEST Center
Information
Date: 15 - 20 June 2025
Language: English
Location: near Frankfurt, Germany
Number of participants: 29
Contact: In case of questions or inquiries, please contact Tracey Weissgerber.
Registration: closed
