What is the key message of the publication?
Most animal studies in basic biomedical research use so-called block designs. We have been able to show that sequential designs can be more effective and allow researchers to save resources without diminishing the validity of their trials.
What is the difference between “conventional” block designs and sequential designs?
Trials based on a block design conduct tests on a previously defined number of animals and the results are then evaluated statistically. For example, ten animals in one group are compared to ten animals in a control group.
In sequential trials, either groups of animals (blocks) or individual animals are examined one after the other. The trial is stopped once a significant conclusion can be drawn. The nature of this conclusion and other trial criteria are precisely defined in advance. This means that not all animals will necessarily be tested, as it may be possible to end the trial at an earlier stage. This could either be because the expected effect (therapy A is better than therapy B) has not occurred and based on the results produced so far, is not likely to occur at a later stage. Another possibility is that the expected effect is so pronounced that fewer cases are required to prove it.
How does this actually work in practice? And how did you reach these conclusions?
First of all we had a close look at preclinical studies, using a computer algorithm to process and analyze several thousand trials. We found out that they almost all exclusively use conventional block designs. The situation is a bit different in clinical trials but block designs are mainly used there as well.
To prove that trials that are conducted sequentially are more efficient, we simulated thousands of trials in different scenarios on the computer. The results were clear: trials with a sequential design allow researchers to get the same findings using up to 20 percent fewer resources or, alternatively, to produce more powerful findings.
Why is the approach you advocate not yet being applied in practice if it is so much more efficient?
It has a lot to do with the history of statistics and the possibilities of statistical analysis. The block design was how everything started. Researchers are very conservative and there is often a lack of sufficient expertise in statistics. Trials with a sequential design can also be more time-consuming to plan. The method we describe makes more sense and is more efficient in many cases, but not across the board.
What is the next step? How do you intend to bring your findings into the laboratories?
We hope that our publication will spark people’s interest in sequential trial designs. In the publication we clearly show that the designs we propose are feasible in practice, that the analysis is not more complicated, and that using them can save up to 20 percent of resources in terms of the amount of animals, work time, materials, etc. required. We also illustrate our findings with concrete examples. We very much hope that our publication will inspire at least a few researchers to try out different designs and start to rethink their approach to designing trials.
Interviewer: Saskia Blank
*The publication has been made available under open access terms in PLOS Biology and is available here for free download.
Neumann K, Grittner U, Piper SK, Rex A, Florez-Vargas O, Karystianis G, Schneider A, Wellwood I, Siegerink B, Ioannidis JP, Kimmelman J1, Dirnagl U. Increasing efficiency of preclinical research by group sequential designs. PLoS Biol. 2017 Mar 10;15(3):e2001307. doi: 10.1371/journal.pbio.2001307