Talk:Invariant estimator

From Wikipedia, the free encyclopedia

This article is within the scope of WikiProject Statistics, which collaborates to improve Wikipedia's coverage of statistics. If you would like to participate, please visit the project page.

[edit] Equivariant?

Can someone say where the term "equivariant" has been used ... it is not in any of my dictionaries of maths or stats. Melcombe (talk) 09:32, 12 May 2008 (UTC)

It appears, for example, in Lehmann and Casella, Theory of Point Estimation. --Zvika (talk) 18:17, 12 May 2008 (UTC)

[edit] non-Bayesian?

Should such prominence be given to "non-Bayesian"? After all ideas of invariance can be applied to Bayesian estimation just as well. Consider for example HPD (highest posterior density) estimation (either point or interval estimates), which is not invariant to transformation of the parameters. Melcombe (talk) 09:14, 19 May 2008 (UTC)

Can you provide a source dealing with Bayesian equivariant estimators? I haven't encountered one. --Zvika (talk) 18:33, 19 May 2008 (UTC)

I am not sure that it is important to actually find examples of invariant Bayesian estimators, just that the idea of invariance can be applied to estimation in a Bayesian setting, where this would go beyond the impractical definition given in Bayesian estimation since contexts with prescribed loss functions are rare. Thus things like Maximum a posteriori need to be included as Bayesian estimates, and also the expectation of the posterior distribution, both in contexts of no-loss-function. Obviously HPD estimates are not invariant to transformations of the parameter space, and nor are expected values of the posterior, and it is important to be able to say this. Melcombe (talk) 09:41, 30 May 2008 (UTC)

Hi again Melcombe, nice to hear from you. You raise two separate points.

1. As I understand it, there is a difference between Bayesian estimation and Bayes estimator. The fact that currently one redirects to the other is simply a result of the fact that we don't yet have an article on Bayesian estimation. Bayesian estimation refers to the general approach of using of prior information (as opposed to frequentist or classical point estimation); this includes MAP, MMSE, etc. By contrast, a Bayes estimator is an estimator which is optimal under the "minimum Bayes risk" criterion (see, e.g., Lehmann and Casella, p.225).
   So, as far as this issue is concerned, I think the best solution is to create a new article Bayesian estimation which would discuss the various Bayesian approaches to estimation and provide links to Bayes estimator, minimum mean square error, maximum a posteriori, etc. I would be happy to work with you on such an article. I do not think MAP should be included in Bayes estimator because it is not a Bayes estimator under this definition.
2. Concerning the relation between invariance/equivariance and Bayes methods: As I said above, the critical issue is to find a reliable source dealing with this question. Bayes estimators might or might not be invariant, but unless you can find a source saying so, including such statements in Wikipedia constitutes original research. I have not myself looked specifically for such a source, so it may well be present in Bayesian textbooks like Berger's. I urge you to go ahead and find such a source; it will definitely improve the article if you do.

Zvika (talk) 17:52, 30 May 2008 (UTC)

I had forgotten to set a watch on the page. A reference is Bernardo & Smith's "Bayesian Theory" (Wiley, 1994/2000). They cover (briefly): invariance for model formulation, invariance for prior distributions and non-informative priors and invariance in estimation. The last is most directly relevant here: they have about 2/3rds of a page (p454) for example :"From a Bayesian point of view, for invariance to be a relevant notion it must be true that the transformation involved also applies to the prior distribution". So at least it has been considered in Bayesian contexts. It may have been found to be non-useful, but this is perhaps no more than in general statistics, where ideas of "unbiasedess" pervade even though this leads to non-invariant procedures. Thus ideas of "invariance" may be best seen as possibly providing a guide to finding reasonable procedures, but not as being a definitive requirement. Melcombe (talk) 09:03, 2 June 2008 (UTC)

Excellent! In this case, I would definitely support a discussion of the Bayesian aspects of invariance in the article. --Zvika (talk) 14:24, 2 June 2008 (UTC)

Reading through the passage you quoted, I noticed that it appears in Appendix B, "Non-Bayesian Approaches". Furthermore, as you said, the basic conclusion of this passage is that invariance is a rather limited technique from a Bayesian point of view. On the other hand, frequentist books give quite a lot of space to invariance (an 80-page chapter in Lehmann and Casella), including a more general formulation than that which appears in Bernardo and Smith. So perhaps the discussion of Bayesian aspects should be a passing statement saying that invariance is not usually applied in a Bayesian setting, with a short explanation why as explained by B&S. What do you think? --Zvika (talk) 07:06, 3 June 2008 (UTC)

Broadly speaking I agree, at least when dealing with questions of finding "optimal" estimators. However, it may be that (presently) this has too much emphasis in the article and that there needs to be more initial discussion of at least the two special cases of: (i) invariance to permutations of observations in the iid case; (ii) invariance to non-linear transformations of parameters. For case (i), all Bayesian procedures (and all likelihood-based procedures) would be invariant to this type of transformation. There would then remain the question of what to do about the other types of invariance that B&S mention and which are both: not directly about estimation and not referred to in the article Invariant (mathematics). Melcombe (talk) 09:38, 3 June 2008 (UTC)

Concerning (i), I find it difficult to think of any reasonable estimator which is not invariant for such a transformation, so I don't know what would be the goal of mentioning this (except perhaps as a trivial example). I didn't quite understand what you're referring to in (ii) - what other types of invariance do B&S mention? I only see a mention of location invariance. In any case, if you've got a concrete idea of what you think should be changed, go ahead and do it, so we have something tangible to discuss. --Zvika (talk) 12:23, 3 June 2008 (UTC)

As mentioned above B&S discuss invariance for model formulation and invariance for prior distributions and non-informative priors: these are listed in their index. Further, I have found in "Essentials of Statistical Inference" by Young & Smith(Cambridge UP, 2005, p86-87) treatment of an "invariant statistic", with a separate (distinct) definition of "equivariant" which then leads on to "equivariant estimator". They also have "maximal invariant". Thus there may be a need to clarify these (different?) levels of invariance for estimators. Melcombe (talk) 13:31, 3 June 2008 (UTC)

As I say, go ahead and edit the article as you see fit, and then we can discuss concrete issues. It could definitely use a discussion of maximal invariance. Iirc, this is analogous to the concept of maximal sufficiency. --Zvika (talk) 17:21, 3 June 2008 (UTC)