Phase Transitions in Sparse PCA

Abstract : We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error and the one achieved by AMP in the limit of large sizes. For a special case of rank one and large enough density of non-zeros Deshpande and Montanari [1] proved that AMP is asymptotically optimal. We show that both for low density and for large rank the problem undergoes a series of phase transitions suggesting existence of a region of parameters where estimation is information theoretically possible, but AMP (and presumably every other polynomial algorithm) fails. The analysis of the large rank limit is particularly instructive.
Type de document :
Pré-publication, Document de travail
6 pages, 3 figures. 2015
Liste complète des métadonnées

https://hal-cea.archives-ouvertes.fr/cea-01140712
Contributeur : Emmanuelle De Laborderie <>
Soumis le : jeudi 9 avril 2015 - 12:01:56
Dernière modification le : mercredi 23 janvier 2019 - 14:39:04

Lien texte intégral

Identifiants

  • HAL Id : cea-01140712, version 1
  • ARXIV : 1503.00338

Citation

Thibault Lesieur, Florent Krzakala, Lenka Zdeborova. Phase Transitions in Sparse PCA. 6 pages, 3 figures. 2015. 〈cea-01140712〉

Partager

Métriques

Consultations de la notice

207