Polynomial convergence of iterations of certain random operators in Hilbert space

• Autorzy: Ghosh Soumyadip , Lu Yingdong , Nowicki Tomasz

Identyfikatory

DOI

10.1515/jaa-2023-0115

• Języki publikacji: EN

Abstrakty

EN

We study the convergence of a random iterative sequence of a family of operators on infinitedimensional Hilbert spaces, inspired by the stochastic gradient descent (SGD) algorithm in the case of the noiseless regression. We identify conditions that are strictly broader than previously known for polynomial convergence rate in various norms, and characterize the roles the randomness plays in determining the best multiplicative constants. Additionally, we prove almost sure convergence of the sequence.

Słowa kluczowe

EN

polynomial convergence; random operators; stochastic gradient descent algorithm

• Wydawca: De Gruyter
• Czasopismo: Journal of Applied Analysis
• Rocznik: 2024
• Tom: Vol. 30, nr 2
• Strony: 239--249
• Opis fizyczny: Bibliogr. 5 poz.

Twórcy

autor

Ghosh Soumyadip

• IBM Research, Yorktown Heights, NY, USA

autor

Lu Yingdong

• IBM Research, Yorktown Heights, NY, USA

• https://orcid.org/0000-0002-3672-0865

autor

Nowicki Tomasz

• IBM Research, Yorktown Heights, NY, USA

Bibliografia

• [1] R. Berthier, F. R. Bach and P. Gaillard, Tight nonparametric convergence rates for stochastic gradient descent under the noiseless linear model, in: Proceedings of the 34th International Conference on Neural Information Processing Systems—NIPS’20, NeurIPS, Vancouver (2020), 2576-2586.
• [2] P. Hall, C. Heyde, Z. Birnbaum and E. Lukacs, Martingale Limit Theory and its Application, Communication and Behavior, Elsevier Science, London, 2014.
• [3] M. Ledoux and M. Talagrand, Probability in Banach Spaces: Isoperimetry and Processes, Mod. Surveys Math. Ser., Springer, New York, 1991.
• [4] M. Reed and B. Simon, Functional Analysis, Methods of Modern Mathematical Physics, Elsevier Science, London, 1981.
• [5] S. Sheffield, Gaussian free fields for mathematicians, Probab. Theory Related Fields 139 (2007), no. 3, 521-541.