HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing

Abstract : In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Complete list of metadata

Cited literature [43 references]  Display  Hide  Download

https://hal-cea.archives-ouvertes.fr/cea-01553517
Contributor : Emmanuelle de Laborderie Connect in order to contact the contributor
Submitted on : Monday, July 3, 2017 - 4:04:57 PM
Last modification on : Thursday, March 17, 2022 - 10:08:10 AM

File

1706.00705.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : cea-01553517, version 1
  • ARXIV : 1706.00705

Citation

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová. Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing. 2017. ⟨cea-01553517⟩

Share

Metrics

Record views

374

Files downloads

310