Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Fast and Easy Crowdsourced Perceptual Audio Evaluation

Crowdsourced Audio Quality Evaluation
Abstract: 

Automated objective methods of audio evaluation are fast, cheap, and require little effort by the investigator. However, objective evaluation methods do not exist for the output of all audio processing algorithms, often have output that correlates poorly with human quality assessments, and require ground truth data in their calculation. Subjective human ratings of audio quality are the gold standard for many tasks, but are expensive, slow, and require a great deal of effort to recruit subjects and run listening tests. Moving listening tests from the lab to the micro-task labor market of Amazon Mechanical Turk speeds data collection and reduces investigator effort. However, it also reduces the amount of control investigators have over the testing environment, adding new variability and potential biases to the data. In this work, we compare multiple stimulus listening tests performed in a lab environment to multiple stimulus listening tests performed in web environment on a population drawn from Mechanical Turk.

up
0 users have voted:

Paper Details

Authors:
Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman
Submitted On:
16 March 2016 - 9:44pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Mark Cartwright
Document Year:
2016
Cite

Document Files

icassp_poster_print.pdf

Subscribe

[1] Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman, "Fast and Easy Crowdsourced Perceptual Audio Evaluation", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/727. Accessed: Mar. 20, 2019.
@article{727-16,
url = {http://sigport.org/727},
author = {Mark Cartwright; Bryan Pardo; Gautham J. Mysore; Matt Hoffman },
publisher = {IEEE SigPort},
title = {Fast and Easy Crowdsourced Perceptual Audio Evaluation},
year = {2016} }
TY - EJOUR
T1 - Fast and Easy Crowdsourced Perceptual Audio Evaluation
AU - Mark Cartwright; Bryan Pardo; Gautham J. Mysore; Matt Hoffman
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/727
ER -
Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman. (2016). Fast and Easy Crowdsourced Perceptual Audio Evaluation. IEEE SigPort. http://sigport.org/727
Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman, 2016. Fast and Easy Crowdsourced Perceptual Audio Evaluation. Available at: http://sigport.org/727.
Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman. (2016). "Fast and Easy Crowdsourced Perceptual Audio Evaluation." Web.
1. Mark Cartwright, Bryan Pardo, Gautham J. Mysore, Matt Hoffman. Fast and Easy Crowdsourced Perceptual Audio Evaluation [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/727