Limiting density of discrete points
From Wikipedia, the free encyclopedia
This article or section is in need of attention from an expert on the subject. WikiProject Mathematics or the Mathematics Portal may be able to help recruit one. |
This article is orphaned as few or no other articles link to it. Please help introduce links in articles on related topics. (May 2008) |
The limiting density of discrete points is used to formulate an adjustment made by Edwin Thompson Jaynes to Claude Elwood Shannon's fundamental uniqueness theorem. The adjustment was created to allow Shannon's information measure formulation to be compatible with continuous distributions.
The limiting density of discrete points, as formulated by Jaynes, is proportional to the invariant measure m(x).
Whereas the information entropy, H(X), of the discrete distribution p(x) is as shown below in (1):
the relative entropy of a continuous distribution p(x) would be as shown in (2)
[edit] References
- Jaynes E.T.,Prior Probabilities IEEE, Transaction on System Science and Cybernetics SSSC-4 (1968), 227.
- Jaynes, E. T., 1983, `Papers On Probability, Statistics and Statistical Physics,' Edited by R. D. Rosenkrantz, D. Reidel publishing Co., Dordrecth, Holland