Sorry, you need to enable JavaScript to visit this website.

Information-theoretic learning (MLR-INFO)

Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization)


In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed.

Paper Details

Authors:
Song Fang, Quanyan Zhu
Submitted On:
24 October 2019 - 4:45pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

postertemplate.pdf

(20)

Subscribe

[1] Song Fang, Quanyan Zhu, "Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization)", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4890. Accessed: Dec. 08, 2019.
@article{4890-19,
url = {http://sigport.org/4890},
author = {Song Fang; Quanyan Zhu },
publisher = {IEEE SigPort},
title = {Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization)},
year = {2019} }
TY - EJOUR
T1 - Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization)
AU - Song Fang; Quanyan Zhu
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4890
ER -
Song Fang, Quanyan Zhu. (2019). Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization). IEEE SigPort. http://sigport.org/4890
Song Fang, Quanyan Zhu, 2019. Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization). Available at: http://sigport.org/4890.
Song Fang, Quanyan Zhu. (2019). "Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization)." Web.
1. Song Fang, Quanyan Zhu. Generic Bounds on the Maximum Deviations in Sequential/Sequence Prediction (and the Implications in Recursive Algorithms and Learning/Generalization) [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4890

Minimax Active Learning via Minimal Model Capacity


Active learning is a form of machine learning which combines supervised learning and feedback to minimize the training set size, subject to low generalization errors. Since direct optimization of the generalization error is difficult, many heuristics have been developed which lack a firm theoretical foundation. In this paper, a new information theoretic criterion is proposed based on a minimax log-loss regret formulation of the active learning problem. In the first part of this paper, a Redundancy Capacity theorem for active learning is derived along with an optimal learner.

Paper Details

Authors:
Meir Feder
Submitted On:
16 October 2019 - 4:02pm
Short Link:
Type:
Event:
Presenter's Name:
Document Year:
Cite

Document Files

MLSP_2019_Minimax_Active_Learning_via_Minimal_Model_Capacity.pdf

(42)

Subscribe

[1] Meir Feder , "Minimax Active Learning via Minimal Model Capacity", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4876. Accessed: Dec. 08, 2019.
@article{4876-19,
url = {http://sigport.org/4876},
author = {Meir Feder },
publisher = {IEEE SigPort},
title = {Minimax Active Learning via Minimal Model Capacity},
year = {2019} }
TY - EJOUR
T1 - Minimax Active Learning via Minimal Model Capacity
AU - Meir Feder
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4876
ER -
Meir Feder . (2019). Minimax Active Learning via Minimal Model Capacity. IEEE SigPort. http://sigport.org/4876
Meir Feder , 2019. Minimax Active Learning via Minimal Model Capacity. Available at: http://sigport.org/4876.
Meir Feder . (2019). "Minimax Active Learning via Minimal Model Capacity." Web.
1. Meir Feder . Minimax Active Learning via Minimal Model Capacity [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4876

Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors


Single-molecule sensors based on carbon nanotubes transducer, enable to probe stochastic molecular dynamics thanks to long acquisition periods and high throughput measurements. With such sampling conditions, the sensor baseline may drift significantly and induce fake states and transitions in the recorded signal, leading to wrong kinetic estimates from the inferred state model.

We present MDL-AdaCHIP a multiscale signal compression technique based on the Minimum Description Length (MDL) principle, combined with an Adaptive piecewise Cubic Hermite Interpolation (AdaCHIP), both implemented into a blind source separation framework to compensate the parasitic baseline drift in single-molecule biosensors

Paper Details

Authors:
Mohamed OUQAMRA, Delphine BOUILLY
Submitted On:
13 October 2019 - 2:00pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

Mohamed_Ouqamra_MLSP2019_Unsupervised Drift Compensation Based on Information Theory for Single molecule Sensors

(19)

Keywords

Additional Categories

Subscribe

[1] Mohamed OUQAMRA, Delphine BOUILLY, "Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4858. Accessed: Dec. 08, 2019.
@article{4858-19,
url = {http://sigport.org/4858},
author = {Mohamed OUQAMRA; Delphine BOUILLY },
publisher = {IEEE SigPort},
title = {Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors},
year = {2019} }
TY - EJOUR
T1 - Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors
AU - Mohamed OUQAMRA; Delphine BOUILLY
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4858
ER -
Mohamed OUQAMRA, Delphine BOUILLY. (2019). Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors. IEEE SigPort. http://sigport.org/4858
Mohamed OUQAMRA, Delphine BOUILLY, 2019. Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors. Available at: http://sigport.org/4858.
Mohamed OUQAMRA, Delphine BOUILLY. (2019). "Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors." Web.
1. Mohamed OUQAMRA, Delphine BOUILLY. Unsupervised Drift Compensation Based on Information Theory for Single-Molecule Sensors [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4858

Deep Learning for Classroom Activity Detection from Audio


Increasingly, post-secondary instructors are incorporating innovative teaching practices into their classrooms to improve student learning outcomes. In order to assess the effect of these techniques, it is helpful to quantify the types of activity being conducted in the classroom. Unfortunately, self-reporting is unreliable and manual annotation is tedious and scales poorly.

Paper Details

Authors:
Robin Cosbey, Allison Wusterbarth, Brian Hutchinson
Submitted On:
10 May 2019 - 4:33pm
Short Link:
Type:
Event:
Paper Code:
Document Year:
Cite

Document Files

[POSTER] Deep Learning for Classroom Activity Detection from Audio

(50)

Subscribe

[1] Robin Cosbey, Allison Wusterbarth, Brian Hutchinson, "Deep Learning for Classroom Activity Detection from Audio", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4404. Accessed: Dec. 08, 2019.
@article{4404-19,
url = {http://sigport.org/4404},
author = {Robin Cosbey; Allison Wusterbarth; Brian Hutchinson },
publisher = {IEEE SigPort},
title = {Deep Learning for Classroom Activity Detection from Audio},
year = {2019} }
TY - EJOUR
T1 - Deep Learning for Classroom Activity Detection from Audio
AU - Robin Cosbey; Allison Wusterbarth; Brian Hutchinson
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4404
ER -
Robin Cosbey, Allison Wusterbarth, Brian Hutchinson. (2019). Deep Learning for Classroom Activity Detection from Audio. IEEE SigPort. http://sigport.org/4404
Robin Cosbey, Allison Wusterbarth, Brian Hutchinson, 2019. Deep Learning for Classroom Activity Detection from Audio. Available at: http://sigport.org/4404.
Robin Cosbey, Allison Wusterbarth, Brian Hutchinson. (2019). "Deep Learning for Classroom Activity Detection from Audio." Web.
1. Robin Cosbey, Allison Wusterbarth, Brian Hutchinson. Deep Learning for Classroom Activity Detection from Audio [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4404

Feature Selection for Multi-labeled Variables via Dependency Maximization


Feature selection and reducing the dimensionality of data is an essential step in data analysis. In this work, we propose a new criterion for feature selection that is formulated as conditional information between features given the labeled variable. Instead of using the standard mutual information measure based on Kullback-Leibler divergence, we use our proposed criterion to filter out redundant features for the purpose of multiclass classification.

Paper Details

Authors:
Salimeh Yasaei Sekeh, Alfred O. Hero
Submitted On:
9 May 2019 - 9:36am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

ICASSP2019-Salimeh-V2.pdf

(43)

Subscribe

[1] Salimeh Yasaei Sekeh, Alfred O. Hero, "Feature Selection for Multi-labeled Variables via Dependency Maximization", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4202. Accessed: Dec. 08, 2019.
@article{4202-19,
url = {http://sigport.org/4202},
author = {Salimeh Yasaei Sekeh; Alfred O. Hero },
publisher = {IEEE SigPort},
title = {Feature Selection for Multi-labeled Variables via Dependency Maximization},
year = {2019} }
TY - EJOUR
T1 - Feature Selection for Multi-labeled Variables via Dependency Maximization
AU - Salimeh Yasaei Sekeh; Alfred O. Hero
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4202
ER -
Salimeh Yasaei Sekeh, Alfred O. Hero. (2019). Feature Selection for Multi-labeled Variables via Dependency Maximization. IEEE SigPort. http://sigport.org/4202
Salimeh Yasaei Sekeh, Alfred O. Hero, 2019. Feature Selection for Multi-labeled Variables via Dependency Maximization. Available at: http://sigport.org/4202.
Salimeh Yasaei Sekeh, Alfred O. Hero. (2019). "Feature Selection for Multi-labeled Variables via Dependency Maximization." Web.
1. Salimeh Yasaei Sekeh, Alfred O. Hero. Feature Selection for Multi-labeled Variables via Dependency Maximization [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4202

ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION


This paper introduces a deep neural network based feature extraction scheme that aims to improve the trade-off between utility and privacy in speaker classification tasks. In the proposed scenario we develop a feature representation that helps to maximize the performance of a gender classifier while minimizing additional speaker

Paper Details

Authors:
Rainer Martin
Submitted On:
8 May 2019 - 2:50am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

ICASSP2019_poster_Nelus.pdf

(44)

Subscribe

[1] Rainer Martin, "ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4027. Accessed: Dec. 08, 2019.
@article{4027-19,
url = {http://sigport.org/4027},
author = {Rainer Martin },
publisher = {IEEE SigPort},
title = {ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION},
year = {2019} }
TY - EJOUR
T1 - ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION
AU - Rainer Martin
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4027
ER -
Rainer Martin. (2019). ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION. IEEE SigPort. http://sigport.org/4027
Rainer Martin, 2019. ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION. Available at: http://sigport.org/4027.
Rainer Martin. (2019). "ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION." Web.
1. Rainer Martin. ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4027

ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION

Paper Details

Authors:
Submitted On:
8 May 2019 - 2:50am
Short Link:
Type:
Paper Code:
Document Year:
Cite

Document Files

ICASSP2019_poster_Nelus.pdf

(32)

Subscribe

[1] , "ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4025. Accessed: Dec. 08, 2019.
@article{4025-19,
url = {http://sigport.org/4025},
author = { },
publisher = {IEEE SigPort},
title = {ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION},
year = {2019} }
TY - EJOUR
T1 - ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION
AU -
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4025
ER -
. (2019). ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION. IEEE SigPort. http://sigport.org/4025
, 2019. ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION. Available at: http://sigport.org/4025.
. (2019). "ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION." Web.
1. . ICASSP 2019 Poster for Paper #3198: PRIVACY-AWARE FEATURE EXTRACTION FOR GENDER DISCRIMINATION VERSUS SPEAKER IDENTIFICATION [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4025

CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING


A simple and scalable denoising algorithm is proposed that can be applied to a wide range of source and noise models. At the core of the proposed CUDE algorithm is symbol-by-symbol universal denoising used by the celebrated DUDE algorithm, whereby the optimal estimate of the source from an unknown distribution is computed by inverting the empirical distribution of the noisy observation sequence by a deep neural network, which naturally and implicitly aggregates multiple contexts of similar characteristics and estimates the conditional distribution more accurately.

Paper Details

Authors:
Jongha Jon Ryu, Young-Han Kim
Submitted On:
8 October 2018 - 3:38am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

icip2018-poster-cude.pdf

(89)

Subscribe

[1] Jongha Jon Ryu, Young-Han Kim, "CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/3616. Accessed: Dec. 08, 2019.
@article{3616-18,
url = {http://sigport.org/3616},
author = {Jongha Jon Ryu; Young-Han Kim },
publisher = {IEEE SigPort},
title = {CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING},
year = {2018} }
TY - EJOUR
T1 - CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING
AU - Jongha Jon Ryu; Young-Han Kim
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/3616
ER -
Jongha Jon Ryu, Young-Han Kim. (2018). CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING. IEEE SigPort. http://sigport.org/3616
Jongha Jon Ryu, Young-Han Kim, 2018. CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING. Available at: http://sigport.org/3616.
Jongha Jon Ryu, Young-Han Kim. (2018). "CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING." Web.
1. Jongha Jon Ryu, Young-Han Kim. CONDITIONAL DISTRIBUTION LEARNING WITH NEURAL NETWORKS AND ITS APPLICATION TO UNIVERSAL IMAGE DENOISING [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/3616

ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS

Paper Details

Authors:
Submitted On:
23 April 2018 - 12:06am
Short Link:
Type:
Event:
Presenter's Name:
Document Year:
Cite

Document Files

GeometryMixtures-Poster-ICASSP2018.pdf

(160)

Subscribe

[1] , "ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/3143. Accessed: Dec. 08, 2019.
@article{3143-18,
url = {http://sigport.org/3143},
author = { },
publisher = {IEEE SigPort},
title = {ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS},
year = {2018} }
TY - EJOUR
T1 - ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS
AU -
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/3143
ER -
. (2018). ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS. IEEE SigPort. http://sigport.org/3143
, 2018. ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS. Available at: http://sigport.org/3143.
. (2018). "ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS." Web.
1. . ON THE GEOMETRY OF MIXTURES OF PRESCRIBED DISTRIBUTIONS [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/3143

RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR


Meta learning of optimal classifier error rates allows an experimenter to empirically estimate the intrinsic ability of any estimator to discriminate between two populations, circumventing the difficult problem of estimating the optimal Bayes classifier. To this end we propose a weighted nearest neighbor (WNN) graph estimator for a tight bound on the Bayes classification error; the Henze-Penrose (HP) divergence. Similar to recently proposed HP estimators [berisha2016], the proposed estimator is non-parametric and does not require density estimation.

Paper Details

Authors:
Morteza Noshad, Alfred Hero
Submitted On:
16 April 2018 - 12:28am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:

Document Files

icassp-poster-v3.pdf

(144)

Subscribe

[1] Morteza Noshad, Alfred Hero, "RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2907. Accessed: Dec. 08, 2019.
@article{2907-18,
url = {http://sigport.org/2907},
author = {Morteza Noshad; Alfred Hero },
publisher = {IEEE SigPort},
title = {RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR},
year = {2018} }
TY - EJOUR
T1 - RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR
AU - Morteza Noshad; Alfred Hero
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2907
ER -
Morteza Noshad, Alfred Hero. (2018). RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR. IEEE SigPort. http://sigport.org/2907
Morteza Noshad, Alfred Hero, 2018. RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR. Available at: http://sigport.org/2907.
Morteza Noshad, Alfred Hero. (2018). "RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR." Web.
1. Morteza Noshad, Alfred Hero. RATE-OPTIMAL META LEARNING OF CLASSIFICATION ERROR [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2907

Pages