New platform makes genome analyses scalable and reproducible
Biomedical data analysis is often difficult to reporduce and can only be applied to larger data sets with considerable effort. A group led by Sven Olaf Twardziok at the Berlin Institute of Health at Charité (BIH), together with a team from the German Cancer Research Center (DKFZ), has developed WESkit, a system that automates complex computational workflows based on open standards. By implementing the Workflow Execution Service (WES) specification of the Global Alliance for Genomics and Health (GA4GH), they created a scalable and interoperable solution that enables high-throughput genome analyses across institutional boundaries.
What question did you want to answer, what were you particularly interested in?
Our main goal was to solve the major challenges of scalability and reproducibility in biomedical data analysis that today’s research teams are facing. One example is the extraction of genetic information from blood samples to discover new risk factors for diseases such as cancer or diabetes. We aimed to develop a system built entirely on open standards, specifically implementing the Workflow Execution Service (WES) specification of the Global Alliance for Genomics and Health (GA4GH). This ensures that our solution remains compatible with other systems and is not tied to closed or isolated platforms.
Another key focus for us was high-throughput capability. The software we developed, called WESkit, was designed to reliably and automatically handle hundreds of thousands of workflow executions. At the same time, we placed strong emphasis on cloud integration. With built-in support for technologies like Kubernetes, we enable research institutions to flexibly scale their complex data processing in modern cloud environments, facilitating more seamless collaboration across different compute environments.
What specific challenges did you face when developing your approach?
The biggest challenge was ensuring seamless and stable automation for an enormous number of workflow executions. Unlike other solutions, WESkit was primarily designed for use in large service units and research teams, meaning the system had to remain stable even under extremely high workloads.
To guarantee this reliability on a technical level, we implemented an architecture that processes requests asynchronously through dedicated workers and permanently stores every individual analysis in a database. This setup allows continuous monitoring and ensures that all analysis documentation remains intact, even in the event of system interruptions. Our practical results confirm this approach: at the DKFZ and Charité, we successfully processed over 660,000 workflow runs within a year—stably and reliably.
What surprised you and why? What was new?
We were positively surprised by how seamlessly the integration of security protocols for federated access into the cloud network worked. However, what we consider truly innovative and forward-looking is the consistent alignment with the entire GA4GH Cloud API ecosystem. By combining open-source standards , we are creating an interoperable infrastructure that enables complex analyses across institutional and geographic boundaries.
With this approach, WESkit provides the long-sought, standards-based bridge for use in modern environments such as the German Network for Bioinformatics Infrastructure (de.NBI), a federally funded research infrastructure.
What significance could this success have for future patients?
WESkit provides the essential technical foundation for the data-intensive medicine of the future. In fields such as genome sequencing, vast amounts of data are generated that must be processed efficiently and reliably to enable precise analyses, medical advances, and cost savings. By automating these processes, WESkit frees up valuable time for researchers, allowing them to focus on interpreting results and developing new therapeutic approaches. In the long term, this accelerates the identification of disease mechanisms and supports the transition towards personalized treatment strategies.
Where do you see the translational bridge between research and application?
The translational bridge lies in making research results reproducible and standardized at scale. By using GA4GH standards, WESkit ensures that analysis pipelines developed in research can be reliably transferred to larger cohorts or clinical settings. This is particularly relevant for executing analysis pipelines in multicenter studies. It serves as a dependable “engine room” that helps scientific findings move more quickly and robustly into clinical application.