Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates

Abstract: 

An increasing number of distributed machine learning applications require efficient communication of neural network parameterizations. DeepCABAC, an algorithm in the current working draft of the emerging MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis, has demonstrated high compression gains for a variety of neural network models. In this paper we propose a method for employing DeepCABAC in a Federated Learning scenario for the exchange of intermediate differential parameterizations. Furthermore, we discuss the efficiency of DeepCABAC when compressing trained neural networks. Our experiments on large neural networks show that in both scenarios, DeepCABAC achieves competitive compression rates, without degrading the network accuracy.

up
0 users have voted:

Paper Details

Authors:
David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek
Submitted On:
18 November 2020 - 9:06am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
David Neumann
Document Year:
2020
Cite

Document Files

DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates Presentation Slides.pdf

(11)

Subscribe

[1] David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek, "DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5556. Accessed: Nov. 29, 2020.
@article{5556-20,
url = {http://sigport.org/5556},
author = {David Neumann; Felix Sattler; Heiner Kirchhoffer; Simon Wiedemann; Karsten Müller; Heiko Schwarz; Thomas Wiegand; Detlev Marpe; Wojciech Samek },
publisher = {IEEE SigPort},
title = {DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates},
year = {2020} }
TY - EJOUR
T1 - DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates
AU - David Neumann; Felix Sattler; Heiner Kirchhoffer; Simon Wiedemann; Karsten Müller; Heiko Schwarz; Thomas Wiegand; Detlev Marpe; Wojciech Samek
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5556
ER -
David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek. (2020). DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates. IEEE SigPort. http://sigport.org/5556
David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek, 2020. DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates. Available at: http://sigport.org/5556.
David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek. (2020). "DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates." Web.
1. David Neumann, Felix Sattler, Heiner Kirchhoffer, Simon Wiedemann, Karsten Müller, Heiko Schwarz, Thomas Wiegand, Detlev Marpe, Wojciech Samek. DeepCABAC - Plug&Play Compression of Neural Network Weights and Weight Updates [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5556