Adaptive projected subgradient method and its acceleration techniques

In this paper, we introduce the adaptive projected subgradient method (APSM) that can minimize asymptotically certain sequence of nonnegative unsmooth convex functions over a closed convex set in a real Hilbert space. The main theorem of the method unifies a wide range of set theoretic adaptive filtering schemes, e.g., N L M S, Projected N L M S, Constrained N L M S, APA and Adaptive parallel outer projection algorithm etc., for possibly nonstationary random processes. We also briefly present a pair of acceleration techniques that can improve efficiently the performance of adaptive parallel outer projection algorithm. These include the ideas using optimal supporting hyperplane of the stochastic property set as well as Pairwise Optimal WEight Realization (POWER) of parallel projection. The numerical examples show that the proposed adaptive filtering scheme realizes dramatically fast and stable convergence for highly coloured excited speech like input signals in severely noisy situations.