Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning

Abstract: 

The growing use of virtual autonomous agents in applications like games and entertainment demands better control policies for natural-looking movements and actions. Unlike the conventional approach of hard-coding motion routines, we propose a deep learning method for obtaining control policies by directly mimicking raw video demonstrations. Previous methods in this domain rely on extracting low-dimensional features from expert videos followed by a separate hand-crafted reward estimation step. We propose an imitation learning framework that reduces the dependence on hand-engineered reward functions by jointly learning the feature extraction and reward estimation steps using Generative Adversarial Networks~(GANs). Our main contribution in this paper is to show that under injective mapping between low-level joint state (angles and velocities) trajectories and corresponding raw video stream, performing adversarial imitation learning on video demonstrations is equivalent to learning from the state trajectories. Experimental results show that the proposed adversarial learning method from raw videos produces a similar performance to state-of-the-art imitation learning techniques while frequently outperforming existing hand-crafted video imitation methods. Furthermore, we show that our method can learn action policies by imitating video demonstrations on YouTube with similar performance to learned agents from true reward signal. Please see the supplementary video submission at https://ibm.biz/BdzzNA.

up
0 users have voted:

Paper Details

Authors:
Daiki Kimura, Asim Munawar, Ryuki Tachibana
Submitted On:
24 September 2019 - 4:46pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Subhajit Chaudhury
Paper Code:
32
Document Year:
2019
Cite

Document Files

mmps_final.pdf

(20)

Subscribe

[1] Daiki Kimura, Asim Munawar, Ryuki Tachibana, "Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4836. Accessed: Oct. 18, 2019.
@article{4836-19,
url = {http://sigport.org/4836},
author = {Daiki Kimura; Asim Munawar; Ryuki Tachibana },
publisher = {IEEE SigPort},
title = {Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning},
year = {2019} }
TY - EJOUR
T1 - Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning
AU - Daiki Kimura; Asim Munawar; Ryuki Tachibana
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4836
ER -
Daiki Kimura, Asim Munawar, Ryuki Tachibana. (2019). Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning. IEEE SigPort. http://sigport.org/4836
Daiki Kimura, Asim Munawar, Ryuki Tachibana, 2019. Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning. Available at: http://sigport.org/4836.
Daiki Kimura, Asim Munawar, Ryuki Tachibana. (2019). "Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning." Web.
1. Daiki Kimura, Asim Munawar, Ryuki Tachibana. Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4836