Home > Adaptation and Learning in Control and Signal Processing > 11th IFAC International Workshop on Adaptation and Learning in Control and Signal Processing, 2013 > Sparsity-Aware Adaptive Learning: A Set Theoretic Estimation Approach

##### Sparsity-Aware Adaptive Learning: A Set Theoretic Estimation Approach

###### Adaptation and Learning in Control and Signal Processing, Volume # 11 | Part# 1

**Location:**University of Caen Basse-Normandie, Caen, France

**National Organizing Committee Chair:**Giri, Fouad, Fadili, Jalal, Duclos, Daniel

**International Program Committee Chair:**Astolfi, Alessandro, Vandergheynst, Pierre, Goerig, Laurent

**Conference Editor:**Giri, Fouad, Van Assche, Vincent

Authors

Theodoridis, Sergios; Kopsinis, Y.; Slavakis, K.; Chouvardas, S.

Digital Object Identifier (DOI)

10.3182/20130703-3-FR-4038.00157

Page Numbers:

748-756

Index Terms

Linear systems; Adaptive and optimal parameter estimators; Convex optimization

Abstract

No abstract available

References

D. Angelosante and G. B. Giannakis. RLS-weighted Lasso for adaptive estimation of sparse signals. In Proceedings of the IEEE ICASSP, pages 32453248, 2009. D. Angelosante, J. A. Bazerque, and G. B. Giannakis. Online adaptive estimation of sparse signals: Where RLS meets the l1-norm. IEEE Trans. Signal Proc., 58(7): 34363447, July 2010. A. Antoniadis and J. Fan. Regularization of wavelet approximations. J. American Statistical Association, 96: 939967, 2001. B. Babadi, N. Kalouptsidis, and V. Tarokh. Asympotic achievability of the Cramer-Rao bound for noisy compressive sampling. IEEE Trans. Signal Processing, 57 (3):12331236, 2009. B. Babadi, N. Kalouptsidis, and V. Tarokh. SPARLS: The sparse RLS algorithm. IEEE Trans. Signal Proc., 58(8): 40134025, Aug. 2010. H. H. Bauschke and J. M. Borwein. On projection algorithms for solving convex feasibility problems. SIAM Review, 38(3):367426, Sept. 1996. H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, 2011. T. Blumensath. Accelerated iterative hard threshoding. preprint, 2011. T. Blumensath and M. E. Davies. Normalized iterative hard thresholding: Guaranteed stability and performance. IEEE Journal of Selected Topics in Signal Processing, 4(2):298309, Apr. 2010. L. M. Bregman. The method of successive projections for finding a common point of convex sets. Soviet Math. Dokl., 6:688692, 1965. E. Cand`es, J. Romberg, and T. Tao. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489509, Feb. 2006. E. Cand`es, M. B. Wakin, and S. P. Boyd. Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl., 14:877905, 2008. F.S. Cattivelli and A.H. Sayed. Diffusion LMS strategies for distributed estimation. IEEE Transactions on Signal Processing, 58(3):1035 1048, March 2010. ISSN 1053- 587X. doi: 10.1109/TSP.2009.2033729. Y. Chen, Y. Gu, and A. O. Hero. Sparse LMS for system identification. In Proceedings of the IEEE ICASSP, pages 31253128, 2009. S. Chouvardas, K. Slavakis, and S. Theodoridis. Adaptive robust distributed learning in diffusion sensor networks. IEEE Transactions on Signal Processing, 59(10):4692 4707, 2011. ISSN 1053-587X. doi: 10.1109/TSP.2011. 2161474. S. Chouvardas, K. Slavakis, S. Theodoridis, and I. Yamada. Stochastic analysis of hyperslab-based adaptive projected subgradient method under bounded noise. To appear in Signal Processing Letters, IEEE, 2013. Symeon Chouvardas, Konstantinos Slavakis, Yannis Kopsinis, and Sergios Theodoridis. A sparsity promoting adaptive algorithm for distributed learning. IEEE Transactions on Signal Processing, 60(10):54125425, October 2012. P. L. Combettes. The foundations of set theoretic estimation. Proc. IEEE, 81(2):182208, 1993. I. Daubechies, M. Fornasier, and I. Loris. Accelerated projected gradient method for linear inverse problems with sparsity constraints. Journal of Fourier Analysis and Applications, 14(5-6):764792, Dec. 2008. David L. Donoho. Compressed sensing. IEEE Trans. Inform. Theory, 52:12891306, 2006. J. Duchi, S. S-Shwartz, Y. Singer, and T. Chandra. Efficient projections onto the l1-ball for learning in high dimensions. In Proceedings of International Conference on Machine Learning (ICML), pages 272279, 2008. D. L. Duttweiler. Proportionate NLMS adaptation in echo cancelers. IEEE Trans. Speech Audio Processing, 8:508 518, 2000. Y.C. Eldar and M. Mishali. Robust recovery of signals from a structured union of subspaces. Information Theory, IEEE Transactions on, 55(11):53025316, 2009. S. Foucart. Hard thresholding pursuit: an algorithm for compressive sensing. SIAM J. Numerical Analysis, 2011. to appear. Ildiko E. Frank and Jerome H. Friedman. A statistical view of some chemometrics regression tools. Technometrics, 35(2):109135, 1993. Hong-Ye Gao. Wavelet shrinkage denoising using the non-negative garrote. Journal of Computational and Graphical Statistics, 7(4):pp. 469488, Dec. 1998. K. Goebel and W. A. Kirk. Topics in Metric Fixed Point Theory. Cambridge University Press, 1990. Yuantao Gu, Jian Jin, and Shunliang Mei. L0 norm constraint LMS algorithm for sparse system identification. Signal Processing Letters, IEEE, 16(9):774 777, September 2009. L. G. Gubin, B. T. Polyak, and E. V. Raik. The method of projections for finding the common point of convex sets. USSR Comput. Math. Phys., 7:124, 1967. O. Hoshuyama, R.A. Goubran, and A. Sugiyama. A generalized proportionate variable step-size algorithm for fast changing acoustic environments. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 2004. Proceedings. (ICASSP '04), volume 4, pages 161 164, May 2004. Y. Kopsinis, K. Slavakis, and S. Theodoridis. Online sparse system identification and signal reconstruction using projections onto weighted l1 balls. IEEE Trans. Signal Proc., 59(3):905930, Mar. 2011. Y. Kopsinis, K. Slavakis, S. Theodoridis, and S. McLaughlin. Generalized thresholding sparsity-aware algorithm for low complexity online learning. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE Interna- tional Conference on, pages 32773280, 2012. Y. Kopsinis, K. Slavakis, S. Theodoridis, and S. McLaughlin. Thresholding-based online algorithms of complexity comparable to sparse lms methods. To be presented at the IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China, May 19-23, 2013, 2013. C.G. Lopes and A.H. Sayed. Diffusion least-mean squares over adaptive networks: Formulation and performance analysis. IEEE Transactions on Signal Processing, 56 (7):31223136, 2008. Y.M. Lu and M.N. Do. A theory for sampling signals from a union of subspaces. Signal Processing, IEEE Transactions on, 56(6):2334 2345, june 2008. R. Mazumder, J. H. Friedman, and T. Hastie. SPARSENET: Coordinate descent with nonconvex penalties. J. American Statistical Association, 2011. to appear. G. Mileounis, B. Babadi, N. Kalouptsidis, and V. Tarokh. An adaptive greedy algorithm with application to nonlinear communications. IEEE Trans. Signal Proc., 58 (6):29983007, June 2010. Y. Murakami, M. Yamagishi, M. Yukawa, and I. Yamada. A sparse adaptive filtering using time-varying soft-thresholding techniques. In Proceedings of the IEEE ICASSP, pages 37343737, Dallas: USA, Mar. 2010. C. Paleologu, J. Benesty, and S. Ciochina. An improved proportionate NLMS algorithm based on the l0 norm. In Acoustics Speech and Signal Processing (ICASSP), 2010 IEEE International Conference on, pages 309312, March 2010a. C. Paleologu, S. Ciochina, and J. Benesty. An efficient proportionate affine projection algorithm for echo cancellation. Signal Processing Letters, IEEE, 17(2):165 168, February 2010b. K. Pelekanakis and M. Chitre. New sparse adaptive algorithms based on the natural gradient and the l0 - norm. Oceanic Engineering, IEEE Journal of, 38(2): 323332, 2013. R. T. Rockafellar and R. J.-B. Wets. Variational Analysis. Springer, Berlin, 2004. A. H. Sayed. Fundamentals of Adaptive Filtering. John Wiley & Sons, New Jersey, 2003. Kun Shi and Peng Shi. Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal. Signal Process., 90(12):32893293, December 2010. K. Slavakis, I. Yamada, and N. Ogura. The adaptive projected subgradient method over the fixed point set of strongly attracting nonexpansive mappings. Numerical Functional Analysis and Optimization, 27(7&8):905 930, 2006. K. Slavakis, S. Theodoridis, and I. Yamada. Low complexity projection-based adaptive algorithm for sparse system identification and signal reconstruction. In Signals, Systems and Computers (ASILOMAR), 2010 Confer- ence Record of the Forty Fourth Asilomar Conference on, pages 703707, 2010. K. Slavakis, Y. Kopsinis, S. Theodoridis, and S. McLaughlin. Generalized thresholding and online sparsity-aware learning in a union of subspaces. To appear in IEEE Trans. Signal Proc., 2013. Konstantinos Slavakis and Isao Yamada. The adaptive projected subgradient method constrained by families of quasi-nonexpansive mappings and its application to online learning. SIAM Journal on Optimization, 23(1): 126152, 2013. H. Stark and Y. Yang. Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics. John Wiley & Sons, New York, 1998. G. Su, J. Jin, Y. Gu, and J.Wang. Performance analysis of l 0 norm constraint least mean square algorithm. Signal Processing, IEEE Transactions on, 60(5):2223 2235, May 2012. S. Theodoridis, K. Slavakis, and I. Yamada. Adaptive learning in a world of projections. Signal Processing Magazine, IEEE, 28(1):97123, Jan. 2011. S. Theodoridis, Y. Kopsinis, and K. Slavakis. Sparsityaware learning and compressed sensing: An overview. To appear in E-Reference Signal Processing, Elsevier, 2013. S. Werner and P.S.R. Diniz. Set-membership affine projection algorithm. Signal Processing Letters, IEEE, 8(8): 231235, August 2001. I. Yamada. Adaptive projected subgradient method: A unified view for projection based adaptive algorithms. The Journal of IEICE, 86(8):654658, Aug. 2003. In Japanese. I. Yamada and N. Ogura. Adaptive projected subgradient method for asymptotic minimization of sequence of nonnegative convex functions. Numerical Functional Analysis and Optimization, 25(7&8):593617, 2004. D. C. Youla and H. Webb. Image restoration by the method of convex projections: part 1theory. IEEE Trans. Med. Imaging, MI-1:8194, Oct. 1982. C.-H. Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38(6): 894942, 2010. H. Zou. The adaptive LASSO and its oracle properties. Journal of the American Statistical Association, 101: 14181429, Dec. 2006.