压缩感知的信息论解译

压缩感知(compressed sensing or compressive sampling, CS)是数据采集与信号重构的新体制，其与信息论的关系是，应该且可以从信息论的角度对CS进行分析，而CS丰富和发展着信息论的内涵和外延。换言之，信息论的一些基本概念和原理(如信源、信道、信源编码、信道编码、率失真、Fano不等式、数据处理定理等)为CS研究提供了理论基础，尤其是在性能限(如采样数)的界定等方面;另一方面，CS提供了采集、存贮、 传输、恢复稀疏信号的高效方法，以其独特的理念和算法模式，提供了直接对信息的采样和处理机制，延拓了经典信息论的范畴。本文将梳理和阐释CS和信息论之间的关系，力图从信息论角度揭示CS中的一些基本问题，尤其是CS采样问题，并寻求用信息论指导CS的学习与研究。

Information-theoretic Interpretations of Compressive Sampling

Compressive sampling or compressed sensing(CS)is a new paradigm for data acquisitionand signal recovery. There are two-way relationships between CS and information theory:the formershould and can be analyzed from the perspective of the latter，while the latter’s content and extent areenriched and broadened by the former. Specifically，some basic concepts and theorems in informationtheory，such as source，channel，source coding，channel coding，rate distortion，Fano inequality，andthe data processing theorem，provide theoretical foundation for research on CS，in particular，thatconcerning performance limits(e. g.，sampling rates).CS provides a highly efficient strategy for col-lecting，storing，transmitting，and reconstructing sparse signals through its unique concepts and algo-rithms，such as the sparsity of real signals (enabling CS sampling at a rate lower than Nyquist rate)，the information sensing capacity of random sampling matrices(which preserve information);and in-formation reconstruction based on convex optimization(different from signal reconstruction by Sinckernels in the Shannon-Nyquist sampling theorem).Thus，CS is a mechanism for direct informationsampling and processing，extending the domain of classic information theory. This paper seeks to clar-ify and explain the relationships between CS and information theory，revealing some of the fundamen-tal issues in CS，in particular，those concerning CS sampling，and providing guidance for CS researchdirections.