Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing

Abstract : In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Liste complète des métadonnées

Littérature citée [43 références]  Voir  Masquer  Télécharger

https://hal-cea.archives-ouvertes.fr/cea-01553517
Contributeur : Emmanuelle De Laborderie <>
Soumis le : lundi 3 juillet 2017 - 16:04:57
Dernière modification le : vendredi 16 novembre 2018 - 01:56:22

Fichier

1706.00705.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : cea-01553517, version 1
  • ARXIV : 1706.00705

Citation

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová. Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing. T14/110. 19 pages, 4 figures. 2017. 〈cea-01553517〉

Partager

Métriques

Consultations de la notice

205

Téléchargements de fichiers

58