Sorry, you need to enable JavaScript to visit this website.

Learning Motion Disfluencies for Automatic Sign Language Segmentation

Citation Author(s):
Iva Farag
Submitted by:
Heike Brock
Last updated:
9 May 2019 - 2:18am
Document Type:
Poster
Document Year:
2019
Event:
Presenters:
Heike Brock
Paper Code:
4135
 

We introduce a novel technique for the automatic detection of word boundaries within continuous sentence expressions in Japanese Sign Language from three-dimensional body joint positions. First, the flow of signed sentence data within a temporal neighborhood is determined utilizing the spatial correlations between line segments of inter-joint pairs. Next, a frame-wise binary random forest classifier is trained to distinguish word and non-word frame content based on the extracted spatio-temporal features. The output of the classifier is used to propose an automatic word synthesis that achieves reliable and accurate sentence segmentation with an average frame-wise F1 score of 0.89. Evaluation with a baseline data set furthermore shows that the proposed approach can easily be adapted to distinguish between motion transitions and motion primitives for a coarse-action domain.

up
0 users have voted: