Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

A Novel Method for Human Bias Correction of Continuous-time Annotations

Abstract: 

Human annotations are of integral value in human behavior studies and in particular for the generation of ground truth for behavior prediction using various machine learning methods. These often subjective human annotations are especially required for studies involving measuring and predicting hidden mental states (e.g. emotions) that cannot effectively be measured or assessed by other means. Human annotations are noisy and prone to the influence of several factors including personal bias, task ambiguity, environmental distractions, and health state. We propose a novel method for fusion of continuous real-time human annotations to generate accurate ground truth estimates. We introduce a signal warping method that uses additional comparative rank-based information about specific subsets of the annotations to correct for specific types of human annotation artifacts. This approach is validated using a mechanically simple but perceptually demanding psycho-physical annotation experiment where objective truth labels are known. Our method yields ground truth estimates that are in better agreement with the objective truth than state-of-the-art approaches.

up
0 users have voted:

Paper Details

Authors:
Karel Mundnich, Shrikanth S. Narayanan
Submitted On:
13 April 2018 - 2:06pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Brandon M Booth
Paper Code:
4133
Document Year:
2018
Cite

Document Files

booth_ICASSP_2018.pdf

(26 downloads)

Subscribe

[1] Karel Mundnich, Shrikanth S. Narayanan, "A Novel Method for Human Bias Correction of Continuous-time Annotations", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2735. Accessed: May. 20, 2018.
@article{2735-18,
url = {http://sigport.org/2735},
author = {Karel Mundnich; Shrikanth S. Narayanan },
publisher = {IEEE SigPort},
title = {A Novel Method for Human Bias Correction of Continuous-time Annotations},
year = {2018} }
TY - EJOUR
T1 - A Novel Method for Human Bias Correction of Continuous-time Annotations
AU - Karel Mundnich; Shrikanth S. Narayanan
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2735
ER -
Karel Mundnich, Shrikanth S. Narayanan. (2018). A Novel Method for Human Bias Correction of Continuous-time Annotations. IEEE SigPort. http://sigport.org/2735
Karel Mundnich, Shrikanth S. Narayanan, 2018. A Novel Method for Human Bias Correction of Continuous-time Annotations. Available at: http://sigport.org/2735.
Karel Mundnich, Shrikanth S. Narayanan. (2018). "A Novel Method for Human Bias Correction of Continuous-time Annotations." Web.
1. Karel Mundnich, Shrikanth S. Narayanan. A Novel Method for Human Bias Correction of Continuous-time Annotations [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2735