Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Onsager-corrected deep learning for sparse linear inverse problems

Abstract: 

Deep learning has gained great popularity due to its widespread success on many inference problems. We consider the application of deep learning to the sparse linear inverse problem encountered in compressive sensing, where one seeks to recover a sparse signal from a few noisy linear measurements. In this paper, we propose two novel neural-network architectures that decouple prediction errors across layers in the same way that the approximate message passing (AMP) algorithms decouple them across iterations: through Onsager correction. We show numerically that our "learned AMP" network significantly improves upon Gregor and LeCun's "learned ISTA" when both use soft-thresholding shrinkage. We then show that additional improvements result from jointly learning the shrinkage functions together with the linear transforms. Finally, we propose a network design inspired by an unfolding of the recently proposed "vector AMP" (VAMP) algorithm, and show that it outperforms all previously considered networks. Interestingly, the linear transforms and shrinkage functions prescribed by VAMP coincide with the values learned through backpropagation, yielding an intuitive explanation for the design of this deep network.

up
0 users have voted:

Paper Details

Authors:
Mark Borgerding and Philip Schniter
Submitted On:
6 December 2016 - 10:30am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Mark Borgerding
Paper Code:
1388
Document Year:
2016
Cite

Document Files

slides

(779)

Subscribe

[1] Mark Borgerding and Philip Schniter, "Onsager-corrected deep learning for sparse linear inverse problems", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1359. Accessed: Mar. 30, 2020.
@article{1359-16,
url = {http://sigport.org/1359},
author = {Mark Borgerding and Philip Schniter },
publisher = {IEEE SigPort},
title = {Onsager-corrected deep learning for sparse linear inverse problems},
year = {2016} }
TY - EJOUR
T1 - Onsager-corrected deep learning for sparse linear inverse problems
AU - Mark Borgerding and Philip Schniter
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1359
ER -
Mark Borgerding and Philip Schniter. (2016). Onsager-corrected deep learning for sparse linear inverse problems. IEEE SigPort. http://sigport.org/1359
Mark Borgerding and Philip Schniter, 2016. Onsager-corrected deep learning for sparse linear inverse problems. Available at: http://sigport.org/1359.
Mark Borgerding and Philip Schniter. (2016). "Onsager-corrected deep learning for sparse linear inverse problems." Web.
1. Mark Borgerding and Philip Schniter. Onsager-corrected deep learning for sparse linear inverse problems [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1359