Assessment of Actual Significance Levels for Covariate Effects in NONMEM

The objectives of this study were to assess the difference between actual and nominal significance levels, as judged by the likelihood ratio test, for hypothesis tests regarding covariate effects using NONMEM, and to study what factors influence these levels. Also, a strategy for obtaining closer agreement between nominal and actual significance levels was investigated. Pharmacokinetic (PK) data without covariate relationships were simulated from a one compartment i.v. bolus model for 50 individuals. Models with and without covariate relationships were then fitted to the data, and differences in the objective function values were calculated. Alterations were made to the simulation settings; the structural and error models, the number of individuals, the number of samples per individual and the covariate distribution. Different estimation methods in NONMEM were also tried. In addition, a strategy for estimating the actual significance levels for a specific data set, model and parameter was investigated using covariate randomization and a real data set. Under most conditions when the first-order (FO) method was used, the actual significance level for including a covariate relationship in a model was higher than the nominal significance level. Among factors with high impact were frequency of sampling and residual error magnitude. The use of the first-order conditional estimation method with interaction (FOCE-INTER) resulted in close agreement between actual and nominal significance levels. The results from the covariate randomization procedure of the real data set were in agreement with the results from the simulation study. With the FO method the actual significance levels were higher than the nominal, independent of the covariate type, but depending on the parameter influenced. When using FOCE-INTER the actual and nominal levels were similar. The most important factors influencing the actual significance levels for the FO method are the approximation of the influence of the random effects in a nonlinear model, a heteroscedastic error structure in which an existing interaction between interindividual and residual variability is not accounted for in the model, and a lognormal distribution of the residual error which is approximated by a symmetric distribution. Estimation with FOCE-INTER and the covariate randomization procedure provide means to achieve agreement between nominal and actual significance levels.