Comparing Cox PH regression models with random effect

When to use:

Model comparison serves to choose the set of independent variables, as in comparing Cox PH regression models: we compare a full model with a reduced model, or use automatic selection. The random effect (gamma frailty for the indicated grouping variable) is shared by all compared models – in each of them the variance \(\theta\) is re-estimated, but the group structure remains the same. Selection concerns the independent variables only; whether the random effect is needed is decided by the LR test \(\theta=0\) in the Cox regression with random effect report.

Definitions of measures

Models are compared on the basis of the integrated log-likelihood, the information criteria AIC, AICc, BIC (the smaller, the better) and the likelihood ratio test for nested models: \(2(\ell_{full}-\ell_{reduced})\) follows a \(\chi^2\) distribution with degrees of freedom equal to the number of removed variables.

Out-of-sample evaluation of models

The information criteria and the LR test are determined on the same data on which the models were fitted, which makes the assessment of the models too optimistic. As in comparing Cox PH regression models, the analysis window therefore makes it possible to evaluate the models on observations that did not take part in fitting them (the Validation option): \(k\)-fold cross-validation or a random test set, with the split determined by the Seed. For each model the report then gives \(-2\ln PL\) per event (the validation error; the lower, the better) and Harrell's concordance index C, and in the model selection block – the error of the null model (without variables) and the best model by validation; validation error can also be chosen as the automatic selection criterion. In models with a random effect two additional rules apply:

• whole groups are split – all the observations of one group (e.g. one patient) go to the same part of the data. The number of folds \(k\) is limited to the number of groups, and the share of the test set means the percentage of groups (the split is not stratified by status here);
• the prediction for a new group is based on the fixed effects only – the random effect of a group that was not in the training set is unknown, so its expected value is assumed (frailty \(=1\)).

How to set up the analysis

Statistics \(\to\) Mixed models \(\to\) Cox PH regression with random effect – comparing models.

We indicate the survival time, the censored variable with the Failure events value, the variables of the full and reduced models (or the automatic selection criterion) and the Grouping variable (ID), shared by all models. Below the list of selection methods there are the fields: Criterion (AIC, AICc, BIC, p (LR / F test) with a threshold, or validation error), Validation (none, k-fold CV, k = with the number of folds, or test set, % = with the share of the test set as a percentage of the groups) and Seed of the random split (the same seed means the same split).

Results and interpretation

The report contains a fit table of the compared models (integrated log-likelihood, AIC, AICc, BIC, \(\theta\) of each model) with the LR test between them, and the coefficient tables of the models with conditional hazard ratios. When validation is switched on, the report additionally contains the Out-of-sample evaluation block (the method of splitting, the seed, the split unit, the error of the null model, the best model by validation), and for each model – \(-2\ln PL\) per event and Harrell's C.