WorkshopQUEST Center

28.08.2023 - 01.09.2023

Meta-Science Summer School: Designing Studies to Improve Research

Meta-Science Summer School to learn more about meta-research by designing, conducting, and publishing a meta-research study in a small team. (Kurssprache: Englisch)

Concerned about the reproducibility and transparency of research? Interested in learning more about meta-research, or the science of science? Looking for hands-on experience? Excited to work with other early career researchers across disciplines?

Apply for the Meta-Science Summer School to learn more about meta-research by working collaboratively in a small team to design a meta-research study. After the summer school, some groups may decide to continue working with their instructors to complete and publish their study. The Meta-Science Summer School is organized in cooperation between the Berlin Institute of Health (BIH), the German Reproducibility Network (GRN) and the META REP Priority Program.

What is meta-research?

Meta-research, or science of science, is an extremely broad field and includes the detection of common problems in designing, conducting, analysing and reporting scientific studies. This can help us identify and implement targeted solutions to improve scientific research practices.Other topics, such as journal guidelines and funding agency policies, or hiring and promotion practices, can have an influence on these practices as well.

Please note: This summer school is not focusing on meta-analysis. Meta-analysis refers to the process of systematically identifying all studies that address a particular research question (systematic review). While meta-research uses similar techniques, there are clear differences to systematic reviews.

Description

Rather than focusing on a few pre-selected topics during the summer school, instructors will support participants in developing meta-reseach study proposals that are relevant to their own work and will provide value to the research community. An international group of experienced meta-researchers will work closely with each group to develop a study proposal.

Programme

Before the summer school, participants will watch a series of videos introducing meta-research concepts and methods to maximise learning during the in-person sessions. In each 30-minute video, a meta-researcher will briefly describe a study that they conducted, and highlight important design considerations, possible challenges, and errors or design flaws that should be avoided. Find more information about the webinar series here.
After brainstorming and critically evaluating meta-research study ideas, participants will divide into 3-4 small project teams based on participants’ interests. Each group will work with experienced instructors (see below) to define a research question, identify pre-existing literature, design a study, prepare a proposal, develop a possible protocol, and perform feasibility testing. In the final session, each group will present their work, discuss feasibility, and decide whether to continue working with the instructors to complete their project after the summer school.
During the summer school, participants will collaboratively create and test an automated tool to screen scientific papers or preprints for a common problem or beneficial practice (e.g. reporting of sex/gender in abstracts). Instructors will pre-define the tool objective and compile material to be curated in advance, while participants will curate training data, train simple models, and compare model performance.

Instructors

Tracey Weissgerber (BIH QUEST Center): Tracey Weissgerber’s paper on the inappropriate use of bar graphs has been viewed more than 400,000 times and contributed to journal policy changes. Her meta-research specialities include data visualization, statistics, and open/reproducible methods reporting. Methods expertise: Systematic review/literature survey, automated screening, innovative educational formats.
Anita Bandrowski (University of California, San Diego): Anita Bandrowski is a founding member of the research resource identifier initiative, and founded a start-up that developed the SciScore automated screening tool. Many publishers use this tool to screen submitted manuscripts for MDAR checklist items. Methods expertise: Automated screening, meta-research using automated tools.
Malcolm Macleod (CAMARADES, University of Edinburgh): Malcolm Macleod is a clinician-scientist who has led or contributed to many transformational science improvement efforts, including the ARRIVE guidelines for animal studies, the Landis criteria, and the MDAR checklist. Methods expertise: Systematic review, meta-analysis, checklist/guideline development, interventional studies, preclinical and translational meta-research.
Malte Elson (META-REP, Ruhr University Bochum): Malte Elson examines the formation and perpetuation of research norms and procedural standards in psychology. He studies the transparency, reproducibility, robustness, and replicability of scientific findings, and quantifies the credibility, validity, and methodological adequacy of research literature. Methods expertise: Systematic review, content analysis, experimental design.
Vladislav Nachev (BIH QUEST Center): Vladislav Nachev is a data scientist with a background in behavioural biology and neurobiology. He programs automated screening tools and dashboards with metrics of open and responsible research. Methods expertise: Data analysis, automated screening, experimental design.
Maximilian Frank (META-REP, Ludwig-Maximilians-University Munich): Maximilian Frank will be responsible for event coordination and evaluation. He is an experienced Open Science working group leader and meta-scientist.

Costs

The Summer School is free to attend. A limited amount of travel funding is available to cover transportation costs from participants’ location to the venue. Please indicate whether you would like to apply for travel funding in your application.

ECTS

Students will receive a certificate after completing the summer school. BIH and Charité graduate students can obtain 1.5 ECTS for completing the summer school and preparatory work. As the preparatory work includes watching videos from the Meta-research Methods webinar series (https://www.bihealth.org/en/notices/meta-research-methods-a-webinar-series), students who participate in the summer school can not receive additional ECTS credit for attending the webinar series.

Scientific coordination and organisation

Dr. Tracey Weissgerber – BIH QUEST Center
Maximilian Frank, M.Sc. – LMU Munich, META-REP Priority Program

Application

Interested in attending? Submit a brief application. Prior meta-research experience is not required. If you have an idea for a meta-research study that you might like to propose during the summer school, include this in your application. Remember that participants will come from many different fields; hence, projects that apply to many disciplines may resonate more with other participants than field-specific projects.

Current news: application deadline is closed.

Each application will be anonymized and scored by two instructors. Scored applications will be divided into pools according to gender and career stage, and we will select the highest scoring applicants in each pool.
A total number of 29 participants (Master students, PhD students, and postdoctoral fellows) will be selected. We anticipate that we may receive more applications from psychology than from other fields, as meta-research has received more attention in psychology. We will limit the number of participants in psychology to 10 to ensure that participants come from many different disciplines.

Dates: Monday, August 28th - Friday, September 1st

Summer School language: English

Venue: TaunusTagungsHotel (close to Frankfurt am Main), Lochmühlenweg 3, 61381 Friedrichsdorf, https://www.taunustagungshotel.de.

Number of participants: 29

Contact: In case of questions or inquiries, please contact

Application: Application deadline is closed.

Telefon:	+49 30 450 543 009
E-Mail:	tracey.weissgerber@bih-charite.de

Meta-Science Summer School: Designing Studies to Improve Research

What is meta-research?

Description

Programme

Instructors

Costs

ECTS

Scientific coordination and organisation

Application

Funded by Volkswagen Foundation

Organizational support

Dr. Tracey Weissgerber

Meta-Science Summer School: Designing Studies to Improve Research

What is meta-research?

Description

Virtual preparatory phase

In-presence summer school

Hackathon (Developing automated screening tools)

Instructors are internationally-recognized meta-scientists who specialise in different fields and meta-research methodologies.

Costs

ECTS

Scientific coordination and organisation

Application

Evaluating applications

Dr. Tracey Weissgerber