Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing - CEA - Commissariat à l’énergie atomique et aux énergies alternatives Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2017

Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing

Résumé

In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Fichier principal
Vignette du fichier
1706.00705.pdf (1.2 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

cea-01553517 , version 1 (03-07-2017)

Identifiants

Citer

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová. Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing. 2017. ⟨cea-01553517⟩
387 Consultations
334 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More