Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Minimax Active Learning via Minimal Model Capacity

Abstract: 

Active learning is a form of machine learning which combines supervised learning and feedback to minimize the training set size, subject to low generalization errors. Since direct optimization of the generalization error is difficult, many heuristics have been developed which lack a firm theoretical foundation. In this paper, a new information theoretic criterion is proposed based on a minimax log-loss regret formulation of the active learning problem. In the first part of this paper, a Redundancy Capacity theorem for active learning is derived along with an optimal learner. Building on this, a new active learning criterion is proposed which naturally induces an exploration - exploitation trade-off in feature selection. In the second part, the linear separator hypotheses class with additive label noise is considered and a low complexity algorithm is proposed which optimizes the active learning criterion from the first part. This greedy algorithm is based on the Posterior Matching scheme for communication with feedback and is shown that for BSC and BEC label noise, the proposed information theoretic criterion decays at an exponential rate.

up
0 users have voted:

Paper Details

Authors:
Meir Feder
Submitted On:
16 October 2019 - 4:02pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Shachar Shayovitz
Document Year:
2019
Cite

Document Files

MLSP_2019_Minimax_Active_Learning_via_Minimal_Model_Capacity.pdf

(29)

Subscribe

[1] Meir Feder , "Minimax Active Learning via Minimal Model Capacity", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4876. Accessed: Nov. 11, 2019.
@article{4876-19,
url = {http://sigport.org/4876},
author = {Meir Feder },
publisher = {IEEE SigPort},
title = {Minimax Active Learning via Minimal Model Capacity},
year = {2019} }
TY - EJOUR
T1 - Minimax Active Learning via Minimal Model Capacity
AU - Meir Feder
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4876
ER -
Meir Feder . (2019). Minimax Active Learning via Minimal Model Capacity. IEEE SigPort. http://sigport.org/4876
Meir Feder , 2019. Minimax Active Learning via Minimal Model Capacity. Available at: http://sigport.org/4876.
Meir Feder . (2019). "Minimax Active Learning via Minimal Model Capacity." Web.
1. Meir Feder . Minimax Active Learning via Minimal Model Capacity [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4876