Selection and Bias--two Hostile Brothers

We consider the situation where in a first stage of a clinical trial several treatments are compared with a single control and the 'best' treatment(s) are selected in an interim analysis to be carried on to the second stage. We quantify the mean bias and mean square error of the conventional estimates after selection depending on the number of treatments and the selection time during the trial. The cases without or with reshuffling the planned sample size of the dropped treatments to the selected ones are investigated. The mean bias shows very different patterns depending on the selection rule and the unknown parameter values. We stress the fact that the quantification of the bias is possible only in designs with planned adaptivity where the design allows reacting to new evidence, but the decision rules are laid down in advance. Finally, we calculate the mean bias which arises in a simple but influential regulatory selection rule, to register a new medical therapy only when two pivotal trials have both proven an effect by a statistical test.