- Turkish Journal of Mathematics and Computer Science
- Vol: 15 Issue: 1
- On the Convergence of Stochastic Aggregated Gradient Method
On the Convergence of Stochastic Aggregated Gradient Method
Authors : Figen OZTOPRAK TOPKAYA
Pages : 89-95
Doi:10.47000/tjmcs.1037384
View : 12 | Download : 14
Publication Date : 2023-06-30
Article Type : Research Article
Abstract :The minimization problem of the sum of a large set of convex functions arises in various applications. Methods such as incremental gradient, stochastic gradient, and aggregated gradient are popular choices for solving those problems as they do not require a full gradient evaluation at every iteration. In this paper, we analyze a generalization of the stochastic aggregated gradient method via an alternative technique based on the convergence of iterative linear systems. The technique provides a short proof for the $O(\\kappa^{-1})$ linear convergence rate in the quadratic case. We observe that the technique is rather restrictive for the general case, and can provide weaker results.Keywords : Unconstrained optimization, stochastic gradient, incremental methods