Talk:Entropy in thermodynamics and information theory

From Wikipedia, the free encyclopedia

	This article is within the scope of WikiProject Physics, which collaborates on articles related to physics.	Physics Portal
???	This article has not yet received a rating on the assessment scale. [FAQ]
???	This article has not yet received an importance rating within physics.
Help with this template Please rate this article, and then leave comments here to explain the ratings and/or to identify the strengths and weaknesses of the article.

This article was created from content copied from the information theory article. I believe this topic justifies its own article for the following reasons (130.94.162.64 02:29, 5 December 2005 (UTC)):

• There is popular interest in the topic. (As seen from various comments on talk pages and so forth.)
• There are deep, undeniable connections between thermodynamics and information theory.
• People have been contributing information on this topic into various articles where it doesn't properly belong, and that information should have its place.
• It will take a considerable amount of space to adequately explain the topic.
• It will avoid the need to duplicate this information and clutter up other related articles such as entropy, reversible computing, information theory, and thermodynamics.
• Those who wish to deny certain connections between thermodynamics and information theory will have a place to contribute verifiable information that supports their viewpoint, rather than just deleting information that would tend to oppose it.
• There are without doubt some proposed connections between information theory and thermodynamics that simply are not true. (Zeilinger's principle looks like one example to me.)
• The Wikipedia community has successfully written articles from NPOV about far more controversial topics than this one.

Just a reminder: Wikipedia is not the place for Original Research. If you want to add something, you need verifiable sources to back it up. I'm not that familiar with Wikipedia policy, but it's probably O.K. to mention recent and ongoing research from a Neutral Point Of View, as long as one places it in proper context and does not make grandiose claims about it. (BTW, I'm the same person as 130.94.162.64; my computer just died on me.)-- 130.94.162.61 19:03, 6 December 2005 (UTC)

[edit] Zeilinger's Principle

Does anyone want to write an article on this one? I know we don't have an article on every Principle that comes along, (nor do we want to), but it looks like some researchers went to a lot of trouble to refute it. I'm not sure how important it is. Maybe someone can just explain it a little more in this article. -- 130.94.162.64 03:24, 5 December 2005 (UTC)

On second thought, I do think some more explanation of Zeilinger's principle is in order. I seem to recall it was popular a number of years ago. -- 130.94.162.61 18:55, 6 December 2005 (UTC)

[edit] Difficulty of Research

The whole subject of information theory is murky, difficult to research, and shrouded in secrecy, (especially, for some reason I am as yet unaware of, as it relates to the things discussed in this article). Little academic research of note has been published on it for the last forty years. The journals that once covered information theory now cover mainly coding theory, its canonical application.

I would like to show, by a little example, why I believe this is so. Let me tell you, from an information-theoretic standpoint, that information surely is not quantized. Consider what Shannon's theorem tells us: that information can be transmitted across a noisy channel at any rate less than the channel capacity. That channel capacity can be ever so small as one wishes to make it, so that for example only a thousandth of a bit at a time can be transmitted through it over the noise, and yet by and by that information can be accumulated and put back together at the other end of the channel with near perfect fidelity.

Now consider a certain TLA (Three Letter Agency) that must work continually with infomation that it wishes to keep out of reach of an adversary. Every employee of that TLA (and there are many many thousands of them) is, in effect, a communications channel that conveys classified information to the adversary. Employees are human. They talk in their sleep. They publish papers. They have friends. What they know influences their actions, perhaps in ever so subtle degrees. An academic might avoid mention of a certain subject in a paper or conversation, and inadvertently make it conspicuous by its absence. People continually let little bits of infomation slip that, taken individually by themselves, would be harmless slips.

But consider a top secret memo that must be circulated widely in the TLA. Now the redundancy of that information is very high, and those little bits of information that inevitably slip can by and by be aggregated by the adversary. A powerful adversary or a determined researcher can conceivably find, aggregate, and reconstruct much information in this way. And the aggregate "leakage" channel capacity of all those employees is no doubt high indeed for that TLA.

I hope that one potential use for this theory was made clear that might explain some of the difficulty in researching it.

As another user put it in another talk page:

"Information theory leaks information."

-- 130.94.162.61 08:10, 7 December 2005 (UTC)

[edit] Updated, re-editing needed

I expanded the text on this subject in the Information theory article, then spotted this page.

So I've copied it all across to here. Some editing may be needed to fit it into the evolving structure of the page here; it would then probably also be worth cutting down the treatment in the Information theory.

But I'm calling it a night just for now. -- Jheald 00:01, 10 December 2005 (UTC).

I had to make a slight correction. We cannot speak of a "joint entropy" of two distributions that are not jointly observable. The "joint distribution" formed by considering them as statistically independent random variables is a completely artificial (not to mention rather misleading) construction. (It completely fails to take into account which variable we observe first!) Maybe someone who knows more about this will expand on it. All we can really include in the article is information from verifiable sources, such as Hirschman's paper. If you have Original Research on this topic, you will have to write a paper on it so that we can refer to it here in the article. But you might think obut the Difficulty of Research before you do this. :) -- 130.94.162.64 15:20, 20 December 2005 (UTC)

We need an Expert on the subject to help with this article. -- 130.94.162.64 00:21, 19 March 2006 (UTC)