A robust regularization path for the Doubly Regularized Support Vector Machine

Antoine Lachaud (1) , David Mercier (1) , S. Canu (2) , Frédéric Suard (1)

1 LIMA - Laboratoire Information, Modèles, Apprentissage [Gif-sur-Yvette]
2 LITIS - Laboratoire d'Informatique, du Traitement de l'Information et des Systèmes

The Doubly Regularized SVM (DrSVM) is an extension of SVM using a mixture of $L2$ and $L1$ norm penalties. This kind of penalty, sometimes referred as the elastic net, allows to perform variable selection while taking into account correlations between variables. Introduced by Wang, an efficient algorithm to compute the whole DrSVM solution path has been proposed. Unfortunately, in some cases, this path is discontinuous, and thus not piecewise linear. To solve this problem, we propose here a new sub gradient formulation of the DrSVM problem. This led us to propose an alternative L1 regularization path algorithm. This reformulation effciently addresses the aforementioned problem and makes the initialization step more generic. The results show the validity of our sub-gradient formulation and the efficiency compared to the initial formulation.

Keywords: Algorithms; Artificial intelligence; Learning systems; Neural networks; Piecewise linear techniques; Support vector machines; Elastic net; Initialization step; Piecewise linear; Regularization paths; Solution path; Variable selection; Problem solving; Machine learning

Domains: Artificial Intelligence [cs.AI]; Data Analysis, Statistics and Probability [physics.data-an]

Antoine Lachaud, David Mercier, S. Canu, Frédéric Suard. A robust regularization path for the Doubly Regularized Support Vector Machine. i6doc.com publication, pp.313-318, 2014, 978-287419095-7. ⟨cea-01844455⟩