A goodness-of-fit measure for logistic regression under separation

Logistic regression models have a severe problem called separation. The maximum likelihood estimator does not exist in logistic regression models for data structures under separation. Under separation, the forcibly estimated maximum likelihood estimate may have an extremely large value. Separation often occurs when the size of dataset is small. Consequently, goodness-of-fit measures based on the likelihood ratio and those based on covariance functions using the maximum likelihood estimate indicate that the model is excessively good regardless of the cause of the separation. The Firth and exact logistic regression methods are valid estimation methods for separation problems. Therefore, we propose methods to reasonably evaluate the goodness-of-fit measures of statistical models under separation with dataset of a small sample size with the abovementioned methods. The goodness-of-fit measures based on covariance functions which are a generalization of the multiple correlation coefficient, referred to as the regression correlation coefficient and the entropy coefficient of determination are then used combined with the abovementioned methods for the separation data. In addition, we conducted a data analysis using the definition of the non separation ratio based on the regression depth.