Orthant-Wise Limited-memory Quasi-Newton Optimizer for L1-regularized Objectives

The Orthant-Wise Limited-memory Quasi-Newton algorithm (OWL-QN) is a numerical optimization procedure for finding the optimum of an objective of the form {smooth function} plus {L1-norm of the parameters}. It has been used for training log-linear models (such as logistic regression) with L1-regularization. The algorithm is described in "Scalable training of L1-regularized log-linear models" by Galen Andrew and Jianfeng Gao. This implementation includes built-in capacity to train logistic regression or least-squares models with L1 regularization. It is also possible to use OWL-QN to optimize any arbitrary smooth convex loss plus L1 regularization by defining the function and its gradient using the supplied "DifferentiableFunction" class, and passing an instance of the function to the OWLQN object. For more information, please read the included file README.txt. Also included in the distribution are the ICML paper and slide presentation.