Treatment response prediction: Is model selection unreliable?

Abstract Quantitative modelling has become an essential part of the drug development pipeline. In particular, pharmacokinetic and pharmacodynamic models are used to predict treatment responses in order to optimise clinical trials and assess the safety and efficacy of dosing regimens across patients. It is therefore crucial that treatment response predictions are reliable. However, the data available to fit models are often limited, which can leave considerable uncertainty about the best model to use. Common practice is to select the model that is most consistent with the observed data based on the Akaike information criterion (AIC). Another popular approach is to average the predictions across the subset of models consistent with the data. In this article, we argue that both approaches can lead to unreliable predictions, as treatment responses typically display nonlinear dynamics, so models can be consistent with the observed dynamics, whilst predicting incorrect treatment responses. This is especially the case when predicting treatment responses for either times or dosing regimens that go beyond the observed dynamics. Across a range of experiments on both real laboratory data and synthetically derived data on Neisseria gonorrhoeae response to ciprofloxacin, we show that probabilistic averaging of models results in more reliable treatment response predictions.