Compressed Sensing

This page contains resources about Compressed Sensing, Sparse Sampling and Sparse Signal Processing.

Subfields and Concepts

 * Incoherence / Incoherent Sampling / Incoherent bases
 * Canonical/Kroneker basis
 * Fourier basis
 * Random basis
 * Random sequences / codes
 * Inverse Discrete Cosine Transform (IDCT) / Heisenberg
 * Wavelet basis
 * Coherent-based Sampling
 * Coherence / Mutual Coherence
 * Local Coherence
 * Null Space Property
 * Restricted Isometry Property
 * Underdetermined Linear System
 * Uncertainty Principles (between sparsity basis and measurement system)
 * Continuous Uncertainty Principles (Heisenberg)
 * Discrete Uncertainty Principle (Donoho and Stark)
 * Dirac Comb / Picket Fence
 * Quantitative Uncertainty Principle
 * Quantitative Robust Uncertainty Principle
 * Sparse Approximation / Sparse Representation
 * Basis Pursuit
 * Matching Pursuit
 * Sparse Signal Recovery / Sparse Signal Reconstruction
 * Exact Recovery Theorem
 * Stable Recovery / Stability Theorem
 * Sub-Nyquist Sampling
 * Nonlinear Sampling Theorem
 * Iterative Reweighted Least Squares
 * Sparse Principal Component Analysis (PCA)
 * Structure Sparse PCA
 * B-Splines
 * E-Splines
 * Wavelets
 * Bayesian Compressive Sensing
 * Variational Bayesian Compressive Sensing
 * Sparse Bayesian Models
 * Inverse Problems (Optimization)
 * Regularization
 * Regularized least squares
 * L0 penalization / Spike-and-slab prior
 * L1-regularization / LASSO / Laplace prior
 * L2-regularization / Ridge Regression / Gaussian prior
 * Elastic nets
 * Total Variation (TV) Regularization (i.e. L1-norm of the gradient)

Video Lectures

 * Compressive Sensing and Sparse Recovery by Justin Romberg (Youtube )

Lecture Notes

 * Compressive sensing and Sparse optimization by Aswin C Sankaranarayanan
 * Compressed Sensing by Simon Foucart
 * Sparse Structure Recovery: Theory And Computation by Paul E. Hand
 * Compressed Sensing by Terence Tao
 * Computational Methods for Data Analysis by Nathan Kutz

Books and Book Chapters

 * Theodoridis, S. (2015). "Chapter 9: Sparsity-Aware Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
 * Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 10: Signal Approximation and Compressed Sensing". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
 * Eldar, Y. C. (2015). Sampling theory: Beyond bandlimited systems. Cambridge University Press.
 * Carmi, A. Y., L. Mihaylova, & S. J. Godsill (Eds.). (2014). Compressed Sensing and Sparse Filtering. Springer.
 * Rish, I., & Grabarnik, G. (2014). Sparse modeling: theory, algorithms, and applications. CRC Press.
 * Foucart, S., & Rauhut, H. (2013). A mathematical introduction to compressive sensing. Birkhäuser.
 * Murphy, K. P. (2012). "Chapter 13: Sparse linear models". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Baraniuk, R., Davenport, M. A., Duarte, M. F., & Hegde, C. (2011). An introduction to compressive sensing. Connexions e-textbook.
 * Starck, J. L., Murtagh, F., & Fadili, J. M. (2010). "Chapter 11: Compressed Sensing". Sparse image and signal processing: wavelets, curvelets, morphological diversity. Cambridge University Press.
 * Elad, M. (2010). Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer.
 * Mallat, S. (2008). A Wavelet Tour of Signal Processing: The Sparse Way. Academic Press.
 * Saad, Y. (2003). Iterative Methods for Sparse Linear Systems. Siam.
 * MacKay, D. J. (2003). "Part VI: Sparse Graph Codes". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Scholarly Articles

 * Chen, Y., Bhojanapalli, S., Sanghavi, S., & Ward, R. (2014). Coherent matrix completion. In Proceedings of the 31st International Conference on Machine Learning (pp. 674-682).
 * Davenport, M. A., Duarte, M. F., Eldar, Y. C., & Kutyniok, G. (2011). Introduction to compressed sensing. Preprint, 93(1), 2.
 * Fornasier, M., & Rauhut, H. (2011). Compressive sensing. In Handbook of mathematical methods in imaging (pp. 187-228). Springer New York.
 * Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on image processing, 19(11), 2861-2873.
 * Dai, W., & Milenkovic, O. (2009). Subspace pursuit for compressive sensing signal reconstruction. IEEE Transactions on Information Theory, 55(5), 2230-2249.
 * Starck, J. L., & Fadili, M. J. (2009). An overview of inverse problem regularization using sparsity. In Image Processing (ICIP), 16th IEEE International Conference on, 1453-1456.
 * Duarte, M. F., Davenport, M. A., Takhar, D., Laska, J. N., Sun, T., Kelly, K. E., & Baraniuk, R. G. (2008). Single-pixel imaging via compressive sampling.IEEE Signal Processing Magazine, 25(2), 83.
 * Ji, S., Xue, Y., & Carin, L. (2008). Bayesian compressive sensing. IEEE Transactions on Signal Processing, 56(6), 2346-2356.
 * Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21-30.
 * Blu, T., Dragotti, P. L., Vetterli, M., Marziliano, P., & Coulot, L. (2008). Sparse sampling of signal innovations. IEEE Signal Processing Magazine, 25(2), 31-40.
 * Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72-82.
 * Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic resonance in medicine, 58(6), 1182-1195.
 * Dragotti, P. L., Vetterli, M., & Blu, T. (2007). Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang–Fix.IEEE Transactions on Signal Processing, 55(5), 1741-1757.
 * Baraniuk, R. G. (2007). Compressive sensing. IEEE Signal Processing Magazine, 24(4).
 * Candes, E., & Romberg, J. (2007). Sparsity and incoherence in compressive sampling. Inverse problems, 23(3), 969.
 * Candes, E. J., Romberg, J., & Tao, T. (2006). Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information.IEEE Transactions on information theory, 52(2), 489-509.
 * Candes, E. J., Romberg, J. K., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on pure and applied mathematics, 59(8), 1207-1223.
 * Candse, E. J. (2006, August). Compressive sampling. In Proceedings of the international congress of mathematicians (Vol. 3, pp. 1433-1452).
 * Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on information theory, 52(4), 1289-1306.
 * Castro, R., Haupt, J., & Nowak, R. (2006). Compressed sensing vs. active learning. In IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings. (Vol. 3, pp. III-III). IEEE.
 * Elad, M., & Bruckstein, A. M. (2002). A generalized uncertainty principle and sparse representation in pairs of bases. IEEE Transactions on Information Theory, 48(9), 2558-2567.
 * Donoho, D. L., & Stark, P. B. (1989). Uncertainty principles and signal recovery. SIAM Journal on Applied Mathematics, 49(3), 906-931.

Tutorials

 * Sparse Signal Processing by Pier Luigi Dragotti (Part 1) - 2015
 * Sparse Signal Processing by Pier Luigi Dragotti (Part 2) - 2015
 * Tradeoffs between Speed and Accuracy in Inverse Problems by Raja Giryes​

Software

 * pycompsense - Python
 * pyCSalgos - Python
 * Sparco - MATLAB
 * SparseLab - MATLAB
 * SparseMRI - MATLAB
 * SPArse Modeling Software (SPAMS) -  Python and R
 * KL1p - C++

Other Resources

 * Learning Compressed Sensing - Nuit Blanche blog
 * Compressive Sensing - Reddit
 * Compressive Sensing on Google+ - online community
 * Compressive Sensing Resources - Rice
 * Compressive Sensing: The Big Picture
 * Compressive Sensing:A New Framework for Imaging
 * Uncertainty Principle in Quantum Physics and Signal Processing - blog post
 * A Brief Introduction to Compressed Sensing with Scikit-Learn - blog post
 * Compressive sensing: tomography reconstruction with L1 prior (Lasso) - blog post
 * Compressed Sensing in Python - blog post
 * Image reconstruction using compressed sensing - StackExchange