ES-DoS: Exhaustive search and density-of-states estimation as a general framework for sparse variable selection

In this paper, we propose an exhaustive search with density-of-states estimation (ES-DoS) method for sparse variable selection in a wide range of learning tasks with various learning machines. We applied this ES-DoS method to synthetic and real data as an example of the regression and classification problems and discuss the results in this paper. The most important aspect of our ES-DoS method is to extract not only the optimal solution but also density of states (DoS) in terms of machine learning and data-driven science. Mapping the solutions of various approximate methods or scientists' hypotheses onto the DoS, we can comprehensively discuss and evaluate these methods and hypotheses. Our ES-DoS method opens the way for sparse variable selection in various fields, which promotes the high-dimensional data-driven science.