ENTROPY ESTIMATION USING THE PRINCIPLE OF MAXIMUM ENTROPY
Machine Learning Methods and Applications
Presented by: Raviv Raich, Author(s): Behrouz Behmardi, Raviv Raich, Oregon State University, United States; Alfred O. Hero III, University of Michigan, United States
In this paper, we present a novel entropy estimator for a given set of samples drawn from an unknown probability density function (PDF). Counter to other entropy estimators, the estimator presented here is parametric. The proposed estimator uses the maximum entropy principle to offer an $m$-term approximation to the underlying distribution and does not rely on local density estimation. The accuracy of the proposed algorithm is analyzed and it is shown that the estimation error is $le {cal O}(sqrt{log n/n})$. In addition to the analytic results, a numerical evaluation of the estimator on synthetic data as well as on experimental sensor network data is provided. We demonstrate an order of magnitude improvement in accuracy relative to other methods.
Lecture Information
| Recorded: | 2011-05-27 14:25 - 14:45, Club H |
|---|---|
| Added: | 21. 6. 2011 17:21 |
| Number of views: | 41 |
| Video resolution: | 1024x576 px, 512x288 px |
| Video length: | 0:20:06 |
| Audio track: | MP3 [6.80 MB], 0:20:06 |
Related Lectures
0:19:38
A LEARNING-BASED APPROACH TO EXPLOSIVES DETECTION USING MULTI-ENERGY X-RAY COMPUTED TOMOGRAPHY
Machine Learning Methods and Applications
Added: 21. 6. 2011 20:24
0:18:12
TIME-FREQUENCY SEGMENTATION OF BIRD SONG IN NOISY ACOUSTIC ENVIRONMENTS
Machine Learning Methods and Applications
Added: 21. 6. 2011 19:03
Comments