Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning

Abstract: 

In this paper, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). Every worker in Q-GADMM communicates only with two neighbors, and updates its model via the group alternating direct method of multiplier (GADMM), thereby ensuring fast convergence while reducing the number of communication rounds. Furthermore, each worker quantizes its model updates before transmissions, thereby decreasing the communication payload sizes. We prove that Q-GADMM converges to the optimal solution for convex loss functions, and numerically show that Q-GADMM yields 7x less communication cost while achieving almost the same accuracy and convergence speed compared to GADMM without quantization.

up
0 users have voted:

Paper Details

Authors:
Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal
Submitted On:
21 May 2020 - 3:34pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Anis A Elgabli

Document Files

icassp2020_final.pdf

(5)

Subscribe

[1] Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal, "Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5428. Accessed: Jun. 06, 2020.
@article{5428-20,
url = {http://sigport.org/5428},
author = {Anis Elgabli; Jihong Park; Amrit Bedi; Mehdi Bennis; Vaneet Aggarwal },
publisher = {IEEE SigPort},
title = {Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning},
year = {2020} }
TY - EJOUR
T1 - Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning
AU - Anis Elgabli; Jihong Park; Amrit Bedi; Mehdi Bennis; Vaneet Aggarwal
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5428
ER -
Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal. (2020). Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning. IEEE SigPort. http://sigport.org/5428
Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal, 2020. Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning. Available at: http://sigport.org/5428.
Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal. (2020). "Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning." Web.
1. Anis Elgabli, Jihong Park, Amrit Bedi, Mehdi Bennis, Vaneet Aggarwal. Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5428