Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS

Abstract: 

Cooperative training methods for distributed machine learning are typically based on the exchange of local gradients or local model parameters. The latter approach is known as Federated Learning (FL). An alternative solution with reduced communication overhead, referred to as Federated Distillation (FD), was recently proposed that exchanges only averaged model outputs. While prior work studied implementations of FL over wireless fading channels, here we propose wireless protocols for FD and for an enhanced version thereof that leverages an offline communication phase to communicate ``mixed-up'' covariate vectors. The proposed implementations consist of different combinations of digital schemes based on separate source-channel coding and of over-the-air computing strategies based on analog joint source-channel coding. It is shown that the enhanced version FD has the potential to significantly outperform FL in the presence of limited spectral resources.

up
0 users have voted:

Paper Details

Authors:
Joonhyuk Kang
Submitted On:
13 May 2020 - 11:03pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Jin-Hyun Ahn
Paper Code:
2868
Document Year:
2020
Cite

Document Files

[ICASSP]JHA_Cooperative Learning via Federated Distillation over Fading Channels.pdf

(31)

Subscribe

[1] Joonhyuk Kang, "COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5211. Accessed: Sep. 20, 2020.
@article{5211-20,
url = {http://sigport.org/5211},
author = {Joonhyuk Kang },
publisher = {IEEE SigPort},
title = {COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS},
year = {2020} }
TY - EJOUR
T1 - COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS
AU - Joonhyuk Kang
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5211
ER -
Joonhyuk Kang. (2020). COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS. IEEE SigPort. http://sigport.org/5211
Joonhyuk Kang, 2020. COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS. Available at: http://sigport.org/5211.
Joonhyuk Kang. (2020). "COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS." Web.
1. Joonhyuk Kang. COOPERATIVE LEARNING VIA FEDERATED DISTILLATION OVER FADING CHANNELS [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5211