Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS

Abstract: 

Tensor decomposition has been proved to be effective for solving many problems in signal processing and machine learning. Recently, tensor decomposition finds its advantage for compressing deep neural networks. In many applications of deep neural networks, it is critical to reduce the number of parameters and computation workload to accelerate inference speed in deployment of the network. Modern deep neural network consists of multiple layers with multi-array weights where tensor decomposition is a natural way to perform compression. It is achieved by decomposing the weight tensors in convolutional layers or fully-connected layers with specified tensor ranks (e.g. canonical ranks, tensor train ranks). Conventional tensor decomposition in compressing deep neural networks selects the ranks manually that requires tedious human efforts to finetune the performance. To overcome this issue, we propose a novel rank selection scheme, which is inspired by reinforcement learning, to automatically select ranks in recently studied tensor ring decomposition in each convolutional layer. Experimental results validate that our learning based rank selection significantly outperforms hand-crafted rank selection heuristics on a number of benchmark datasets, for the purpose of effectively compressing deep neural networks while maintaining comparable accuracy.

up
0 users have voted:

Paper Details

Authors:
Submitted On:
15 May 2020 - 10:03pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Zhiyu (Edward) Cheng
Document Year:
2020
Cite

Document Files

ICASSP_slides_Cheng.pdf

(16)

Subscribe

[1] , "A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5363. Accessed: Jul. 14, 2020.
@article{5363-20,
url = {http://sigport.org/5363},
author = { },
publisher = {IEEE SigPort},
title = {A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS},
year = {2020} }
TY - EJOUR
T1 - A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS
AU -
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5363
ER -
. (2020). A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS. IEEE SigPort. http://sigport.org/5363
, 2020. A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS. Available at: http://sigport.org/5363.
. (2020). "A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS." Web.
1. . A NOVEL RANK SELECTION SCHEME IN TENSOR RING DECOMPOSITION BASED ON REINFORCEMENT LEARNING FOR DEEP NEURAL NETWORKS [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5363