Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE

Abstract: 

In this work, we investigate mapping both natural language food and quantity descriptions to matching USDA database entries. We demonstrate that a convolutional neural network (CNN) model with a softmax layer on top to directly predict the most likely database matches outperforms our previous state-of-the-art approach of learning binary classification and subsequently ranking database entries using similarity scores with the learned embeddings. The softmax model achieves 97.3% top-5 USDA quantity and 91.1% food recall over the full database, compared to only 70.0% quantity and 46.4% food recall with a sigmoid model, where top-5 recall indicates the percentage of test cases in which the correct quantity or food is in the top-5 hits. Evaluated on 9,600 spoken meals over all foods, the softmax model achieves 91.6% top-5 quantity and 80.1% food recall. We also explore jointly learning both mappings with a single CNN to boost quantity mapping, and improve food mapping by reranking the food database entries using the predicted quantity matches.

up
0 users have voted:

Paper Details

Authors:
Mandy Korpusik, James Glass
Submitted On:
12 April 2018 - 12:21pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Mandy Korpusik
Paper Code:
2827
Document Year:
2018
Cite

Document Files

icassp_2018.pdf

(364)

Subscribe

[1] Mandy Korpusik, James Glass, "CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2431. Accessed: Aug. 18, 2019.
@article{2431-18,
url = {http://sigport.org/2431},
author = {Mandy Korpusik; James Glass },
publisher = {IEEE SigPort},
title = {CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE},
year = {2018} }
TY - EJOUR
T1 - CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE
AU - Mandy Korpusik; James Glass
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2431
ER -
Mandy Korpusik, James Glass. (2018). CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE. IEEE SigPort. http://sigport.org/2431
Mandy Korpusik, James Glass, 2018. CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE. Available at: http://sigport.org/2431.
Mandy Korpusik, James Glass. (2018). "CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE." Web.
1. Mandy Korpusik, James Glass. CONVOLUTIONAL NEURAL NETWORKS AND MULTITASK STRATEGIES FOR SEMANTIC MAPPING OF NATURAL LANGUAGE INPUT TO A STRUCTURED DATABASE [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2431