Understanding deep learning requires rethinking generalization, 2016. ,
Exploring generalization in deep learning, Advances in Neural Information Processing Systems, pp.5947-5956, 2017. ,
The Nature of Statistical Learning Theory, 1998. ,
Rademacher and gaussian complexities: Risk bounds and structural results, Journal of Machine Learning Research, vol.3, pp.463-482, 2002. ,
Understanding machine learning: From theory to algorithms, 2014. ,
Statistical mechanics of learning from examples, Physical review A, vol.45, issue.8, p.6056, 1992. ,
The statistical mechanics of learning a rule, Reviews of Modern Physics, vol.65, issue.2, p.499, 1993. ,
Statistical mechanics of complex neural systems and high dimensional data, Journal of Statistical Mechanics: Theory and Experiment, issue.03, p.3014, 2013. ,
High-dimensional dynamics of generalization error in neural networks, 2017. ,
The committee machine: Computational to statistical gaps in learning a two-layers neural network, Advances in Neural Information Processing Systems, pp.3223-3234, 2018. ,
URL : https://hal.archives-ouvertes.fr/cea-01933130
The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression, 2018. ,
Surprises in high-dimensional ridgeless least squares interpolation, 2019. ,
The generalization error of random features regression: Precise asymptotics and double descent curve, 2019. ,
, Modelling the in uence of data structure on learning in neural networks, 2019.
Mean-eld message-passing equations in the hop eld model and its generalizations, Physical Review E, vol.95, issue.2, p.22117, 2017. ,
On lazy training in di erentiable programming, Advances in Neural Information Processing Systems, vol.32, pp.2933-2943, 2019. ,
Neural tangent kernel: Convergence and generalization in neural networks, Advances in neural information processing systems, pp.8571-8580, 2018. ,
URL : https://hal.archives-ouvertes.fr/hal-01824549
Disentangling feature and lazy learning in deep neural networks: an empirical study, 2019. ,
A random matrix approach to neural networks, The Annals of Applied Probability, vol.28, issue.2, pp.1190-1248, 2018. ,
URL : https://hal.archives-ouvertes.fr/hal-01957656
Random features for large-scale kernel machines, Advances in Neural Information Processing Systems, vol.20, pp.1177-1184, 2008. ,
Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications, World Scienti c Publishing Company, vol.9, 1987. ,
Statistical mechanics of learning, 2001. ,
Statistical physics of inference: Thresholds and algorithms, Advances in Physics, vol.65, issue.5, pp.453-552, 2016. ,
Annals of mathematics, pp.221-263, 2006. ,
Optimal errors and phase transitions in high-dimensional generalized linear models, Proceedings of the National Academy of Sciences, vol.116, pp.5451-5460, 2019. ,
Generative adversarial nets, Advances in neural information processing systems, pp.2672-2680, 2014. ,
, , 2013.
Random matrix theory proves that deep learning representations of gan-data behave as gaussian mixtures, 2020. ,
Gradient descent provably optimizes overparameterized neural networks, 2018. ,
A convergence theory for deep learning via overparameterization, International Conference on Machine Learning, pp.242-252, 2019. ,
Kernel and deep regimes in overparametrized models, 2019. ,
Fastfood-approximating kernel expansions in loglinear time, Proceedings of the international conference on machine learning, vol.85, 2013. ,
, ACDC: A structured e cient linear layer, 2015.
The generalization error of max-margin linear classi ers, High-dimensional asymptotics in the overparametrized regime, 2019. ,
Reconciling modern machine-learning practice and the classical bias-variance trade-o, Proceedings of the National Academy of Sciences, vol.116, pp.15849-15854, 2019. ,
A jamming transition from under-to over-parametrization a ects generalization in deep learning, Journal of Physics A: Mathematical and Theoretical, vol.52, issue.47, p.474001, 2019. ,
The unreasonable e ectiveness of structured random orthogonal embeddings, Advances in Neural Information Processing Systems, pp.219-228, 2017. ,
Three un nished works on the optimal storage capacity of networks, Journal of Physics A: Mathematical and General, vol.22, issue.12, 1983. ,
A typical reconstruction limit for compressed sensing based on lp-norm minimization, Journal of Statistical Mechanics: Theory and Experiment, issue.09, p.9003, 2009. ,
Statistical-physics-based reconstruction in compressed sensing, Physical Review X, vol.2, issue.2, p.21005, 2012. ,
URL : https://hal.archives-ouvertes.fr/hal-00716897
The spectrum of random inner-product kernel matrices, Random Matrices: Theory and Applications, vol.02, p.1350010, 2013. ,
Nonlinear random matrix theory for deep learning, Advances in Neural Information Processing Systems, vol.30, pp.2637-2646, 2017. ,
, Kernel random matrices of large concentrated data: the example of gan-generated images, ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.7480-7484, 2019.
, Simon Lacoste-Julien, and Ioannis Mitliagkas. A modern take on the bias-variance tradeo in neural networks, 2018.
Scaling description of generalization with number of parameters in deep learning, 2019. ,
Deep double descent: Where bigger models and more data hurt, vol.2020, 2019. ,
Neural networks and the bias/variance dilemma, Neural computation, vol.4, issue.1, pp.1-58, 1992. ,
Re ections after refereeing papers for nips. The Mathematics of Generalization, pp.11-15, 1995. ,
Statistical mechanics of generalization, Models of neural networks III, pp.151-209, 1996. ,
Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE transactions on electronic computers, issue.3, pp.326-334, 1965. ,
Learning with kernels: support vector machines, regularization, optimization, and beyond, 2002. ,
Falkon: An optimal large scale kernel method, Advances in Neural Information Processing Systems, pp.3888-3898, 2017. ,
Optimal rates for the regularized least-squares algorithm, Foundations of Computational Mathematics, vol.7, issue.3, pp.331-368, 2007. ,
Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates, The Journal of Machine Learning Research, vol.16, issue.1, pp.3299-3340, 2015. ,
Random projections through multiple optical scattering: Approximating kernels at the speed of light, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.6215-6219, 2016. ,
Kernel computations from large-scale random features obtained by optical processing units, 2019. ,
Practical and optimal lsh for angular distance, Advances in neural information processing systems, pp.1225-1233, 2015. ,
Structured adaptive and random spinners for fast machine learning computations, 2016. ,
URL : https://hal.archives-ouvertes.fr/hal-02010086
Deterministic equivalents for certain functionals of large random matrices, Ann. Appl. Probab, vol.17, issue.3, pp.875-930, 2007. ,
URL : https://hal.archives-ouvertes.fr/hal-00621793
The spectral norm of random inner-product kernel matrices. Probability Theory and Related Fields, vol.173, pp.27-85, 2019. ,
Scikit-learn: Machine learning in Python, Journal of Machine Learning Research, vol.12, pp.2825-2830, 2011. ,
URL : https://hal.archives-ouvertes.fr/hal-00650905
API design for machine learning software: experiences from the scikit-learn project, ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp.108-122, 2013. ,
URL : https://hal.archives-ouvertes.fr/hal-00856511