TY - GEN
T1 - Projection-based dual averaging for stochastic sparse optimization
AU - Ushio, Asahi
AU - Yukawa, Masahiro
N1 - Funding Information:
This work was partially supported by JSPS Grants-in-Aid (15K06081, 15K13986, 15H02757).
Publisher Copyright:
© 2017 IEEE.
PY - 2017/6/16
Y1 - 2017/6/16
N2 - We present a variant of the regularized dual averaging (RDA) algorithm for stochastic sparse optimization. Our approach differs from the previous studies of RDA in two respects. First, a sparsity-promoting metric is employed, originated from the proportionate-type adaptive filtering algorithms. Second, the squared-distance function to a closed convex set is employed as a part of the objective functions. In the particular application of online regression, the squared-distance function is reduced to a normalized version of the typical squared-error (least square) function. The two differences yield a better sparsity-seeking capability, leading to improved convergence properties. Numerical examples show the advantages of the proposed algorithm over the existing methods including ADAGRAD and adaptive proximal forward-backward splitting (APFBS).
AB - We present a variant of the regularized dual averaging (RDA) algorithm for stochastic sparse optimization. Our approach differs from the previous studies of RDA in two respects. First, a sparsity-promoting metric is employed, originated from the proportionate-type adaptive filtering algorithms. Second, the squared-distance function to a closed convex set is employed as a part of the objective functions. In the particular application of online regression, the squared-distance function is reduced to a normalized version of the typical squared-error (least square) function. The two differences yield a better sparsity-seeking capability, leading to improved convergence properties. Numerical examples show the advantages of the proposed algorithm over the existing methods including ADAGRAD and adaptive proximal forward-backward splitting (APFBS).
KW - online learning
KW - orthogonal projection
KW - proximity operator
KW - sparse optimization
KW - stochastic optimization
UR - http://www.scopus.com/inward/record.url?scp=85023773290&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85023773290&partnerID=8YFLogxK
U2 - 10.1109/ICASSP.2017.7952568
DO - 10.1109/ICASSP.2017.7952568
M3 - Conference contribution
AN - SCOPUS:85023773290
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 2307
EP - 2311
BT - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017
Y2 - 5 March 2017 through 9 March 2017
ER -