Talk:Positive-definite matrix

From Wikipedia, the free encyclopedia

WikiProject Mathematics
Mathematics Portal
This article is within the scope of WikiProject Mathematics, which collaborates on articles related to mathematics.
Mathematics rating: Start Class Mid Priority  Field: Algebra
One of the 500 most frequently viewed mathematics articles.

Contents

[edit] Old remarks

The statements about the complex/real case were incorrect. Here's a simple counterexample:

\begin{bmatrix} 1 & 2i \\ -2i & 1 \end{bmatrix}

This Hermitian matrix has $x^T A x > 0$ for all real $x$, but not for $x = [1 , i]$.

Would it make any difference if x was in Cn?

shd: In that case it should be something like x*Mx where * refers to complex conjugate transpose

I added that to the article. AxelBoldt

Does this encompass all positive-definite matrices? The Mathworld page leads me to believe that there are positive-definite matrices that are not Hermitian. From the page: 'A necessary and sufficient condition for a complex matrix to be positive definite is that the Hermitian part ... be positive definite.' inferno

If you think about xtMx as a quadratic form in the vector x, and write the square matric M as a sum of a symmetric part M1 and an antisymmetric part M2, you see that the quadratic form is independent of M2. So if you look at a symmetric positive definite M1, you can add any M2 you want, and still be positive definite. If that's useful ... And the same thing if you split up into a hermitian and anti-hermitian part, for complex x and introducing complex conjugation in the form.

Charles Matthews 20:11, 16 Jun 2004 (UTC)


Actually, from the point of view of applications, the restriction of the definition to symmetric matrices (in the real case) is unfortunate, since for instance discretizations of advection problems yield non-symmetric positive definite matrices. I'll change this unless I receive serious complaints. Unfortunately, the article Normal matrix becomes wrong then. -- Guido Kanschat 20:15, 3 November 2005 (UTC)

Why does a PD matrix A need to be Hermitian (or symmetric in real case)? This is very confusing and serious. eakbas 04:24, 2 October 2007 (UTC)

This page is linked to from Niemeier lattices. The usage there is a "positive definite unimodular lattice". So we should probably explain here what "positive definite" means in the context of a lattice. I don't know the answer. A5 06:02, 20 Jun 2005 (UTC)

I'm afraid I can't find the link. The first sentence in Niemeier lattices links to positive definite, which redirect to definite bilinear form. However, the best explanantion of "positive definite" in this context can be found on Lattice (group). -- Jitse Niesen 12:18, 20 Jun 2005 (UTC)

[edit] English

This is all very nice and I'm sure a "rigorous definition" is great for some people. However, for someone who just got redirected from positive-semidefinite and doesn't have a degree in pure math, this is not very usefull.

Agreed - can someone with a better understanding of the subject please try to give a more intuitive description somewhere? 70.93.249.46

I may be a mathematician, but I have no idea what could be made clearer. Of course, you'll likely encounter only real matrices for which the definition is simply that B is symmetric and x^T B x >= 0 for all vectors x. Geometrically, the set of n x n positive semidefinite matrices is a cone (you can add two together and multiply them with a nonnegative number), but that's rather trivial, and the definition of the barrier function that makes them useful in optimization is not helpful either. What sort of "useful" facts would you like to learn? Wandrer2 16:35, 2 March 2006 (UTC)
I've just expanded property 2 in the definition to give a spectral intuition about how positive definite M are special within the set of Hermitian operators. For me, this intuition is fundamental and was not quite there before. Hope it helps. Eclecticos 08:02, 20 September 2006 (UTC)

I completely agree with 70.93.249.46. I have a good background in Maths-for-Natural-Sciences and I currently want to explain what an invertible matrix is to a colleague who also has good MfNS but was never taught about matrices. This page doesn't help either of us very much. Could a link be added to a page about matrix operations at A-level/ US High School/ Baccalaureat level? (Is there one...?) OldSpot61 (talk) 13:41, 9 April 2008 (UTC)

[edit] Proofs of Positive Semidefiniteness

I would like to prove that the difference between two general matrices (each of a certain class) is a positive semidefinite matrix. I am not up to the task without some examples; would anybody mind posting examples of positive semidefinite (or definite) proofs?

[edit] Positive Eigenvalue?

What is ment by "A positive definite if and only if all eigenvalues are positive"?. Is all eigenvalues >0 or is all eigenvalues \geq0?

I have managed to prove the following: Let $A$ be a positive definite $n\times n$-matrix with eigenvalues \lambda_1,\lambda_2,\ldots,\lambda_n

then \lambda_i\geq0, \quad i=1,2,3\ldots,n and there exist a k\in\{1,2,3\ldots,n\} such that λk > 0.

But I havn't managed to prove the following:

Let $A$ be a positive definite $n\times n$-matrix with eigenvalues \lambda_1,\lambda_2,\ldots,\lambda_n then \lambda_i>0,\quad i=1,2,3,\ldots,n.

To be clearer A= \begin{pmatrix} 3&0&0\\ 0&2&0\\ 0&0&0\\ \end{pmatrix} has the eigenvalues λ1 = 3,λ2 = 2,λ3 = 0.

The matix is positive definite since x^tAx=\sum_{i=1}^3\lambda_ix_i^2>0 for all vectors x=(x_1,x_2,x_3)\in\mathbf{R^3},x\neq0.

Therefore A symmetric and positive definite doesn't imply that all eigenvalues of $A$ is positive (in the sence >0).

However maybe this might just be the case when the matrix contain a row $j$ and column $j$ that are both zerovectors.

Can anybody help me??? I don't get it.

/Tobias mathstudent

"A positive definite if and only if all eigenvalues are positive" means that all eigenvalues have to be > 0.
Your matrix A is not positive definite, because xTAx = 0 for x = (0,0,1), which is not the zero vector.
I hope this clarifies the matter. If not, feel free to ask. -- Jitse Niesen (talk) 10:53, 16 August 2006 (UTC)

[edit] Proof of a property mentioned?

If M and N are positive definite, then the sum M + N and the products MNM and NMN are also positive definite; and if MN = NM, then MN is also positive definite.

Can anyone provide a reference or a sketch of a proof to the MN part? I have not found it in any googleable literature and I cannot prove it either. Thanks.

if MN = NM where M = M* and N = N*, then MN can be simultaneously diagonalized. from this the claim follows. Mct mht 13:04, 20 January 2007 (UTC)
OMG, I have been blind... However, is there an argument not requiring real analysis as in the positivity of eigenvectors criterion? I have hoped for something that uses criterii 4. or 5. Thanks anyway! —The preceding unsigned comment was added by 84.163.72.75 (talk) 13:32, 20 January 2007 (UTC).
well, the following can be shown: if MN = NM, then x is an eigenvector of N iff x is an eigenvector of M. that is essentially what i said above. also, there is no analysis involved at all here. a result more general than my first answer is that any commuting family of square normal matrices can be simultaneously diagonalized. Mct mht 13:44, 20 January 2007 (UTC)
if MN = NM, then x is an eigenvector of N iff x is an eigenvector of M. - This is clear. You can define eigenvectors/-values by field extension (eigenvalues are zeroes of the characteristic polynomial and eigenvectors are vectors from the nullspace of a certain matrix), but when speaking of positivity and eigenvalues/-vectors in one sentence (and that's what we do when applying your argument to prove that MN is positive definite), real analysis comes in because you can't define positivity in an algebraic field extension. I was wondering whether a proof of the above property was known which did not use real analysis. Anyway, this is rather a question for a math forum than for the Wikipedia, so sorry for going offtopic.
hm, don't think so, this has nothing to do at all with algebraic field extensions, or any analysis for that matter. MN = NM if and only if they have the same invariant subspaces. the claim that they have the same eigenvectors is an immediate corollary of this (under the given assumptions). Mct mht 14:23, 20 January 2007 (UTC)
Well, over a field like \mathbb{Q}, a matrix (particularly, a positive definite one) may have no nontrivial invariant subspaces at all, and such criteria fail, so we do need the analytic structures of \mathbb{R} and \mathbb{C} for the argument (precisely, we need the fundamental theorem of algebra in \mathbb{C} for the MN = NM <==> equal invariant subspaces criterion and we need some analysis over \mathbb{R} to show that a matrix is positive definite iff all its eigenvalues are positive). Anyway, as I said, this is offtopic here, it's just me searching for a proof matching my criteria of elegance... —The preceding unsigned comment was added by 84.163.72.75 (talk) 17:05, 20 January 2007 (UTC).
ok, fair enough. i thought the running assumption was that the matrix entries are real or complex. the article should have stated this. more precisely, the claim "MN = NM <==> equal invariant subspaces" is true regardless what field the entries are in. it's the existence of invariant subspaces that is in question. i do agree that some analysis would be needed in the case of, say, rational entries. you might wanna add this to the article. Mct mht 17:30, 20 January 2007 (UTC)
It is already stated there well enough. I didn't criticize the article, I just wanted to explain the reason that I am looking for a more elementary proof (but I still like yours!).

[edit] notation for positive definite matrices

Regarding the recent addition by Kkliger, could you provide a reference for this usage? That is, I'm much more familiar with M\geq 0 indicating a nonnegative matrix rather than a positive semi-definite one. Feel free to add it back, but it'd be nice if you also note the potential for confusion (and add a reference to support the usage). Cheers, Lunch 19:58, 14 February 2007 (UTC)

In the (pure mathematics) literature mostly with a positive matrix a positive (semi-) definite matrix is meant and nearly never a matrix which entries are positive real numbers (which are of nearly no use, perhaps in numerics?). This comes, as a positive definite matrix defines a positive linear operator. The corresponding Wikipedia article for "positive matrices" is therefore misleading and moreover inconsistent, as the corresponding Wikipedia article of "positive linear functional" is correct (in my sense). So in Wikipedia a positive matrix induces no positive operator which is not what most people want. I think this difference in the literature (which I was not aware about) should at least be mentioned somewhere.
You can find my notation in many books and articles, for example Rajendra Bhatia "Positive definite matrices" (whose first chapter is also in the internet), Eberhard Freitag "Siegel modular functions" (these are the only books I have at home today) but surely in all books about functional analysis and C^* algebras (e.g. the article of S. Sherman "Order in operator algebras" American Journal of Mathematics, Vol. 73, No. 1 (Jan., 1951)).

kkilger 21:46, 14 February 2007 (UTC)

Okey dokey. Like I said, you're welcome to put it back, but please put a note regarding the potential confusion. A canonical reference would be nice, too. The book by Bhatia might be nice except that it was only published this year (2007). It isn't widely available (or read) just yet nor has it been reviewed.
In the (pure mathematics) literature... What field(s)? which are of nearly no use, perhaps in numerics? Yes, in numerical analysis, but also in probability and operations research. There are countless others. Mathematics is quite a broad subject. Lunch 21:18, 14 February 2007 (UTC)
The notation A > 0 for positive operators is indeed common in functional analysis, at least in my experience. It's for instance in Riesz & Sz.-Nagy, Functional Analysis and in Kato, Perturbation Theory for Linear Operators. However, I think that it's not so common in works that restrict themselves to matrices, i.e., finite-dimensional. I had a look in some entry-level Linear Algebra books and none mentioned the notation (please correct me if I'm wrong).
For that reason, I think it's better to avoid the notation in the article if possible (of course, the notation should be mentioned). On the other hand, in some places the notation is really useful — for instance, in "M > N implies N^{-1} > M^{-1}" — so I guess in those places it's best to use the notation. -- Jitse Niesen (talk) 04:39, 15 February 2007 (UTC)

I've also seen the notation \succ (etc.) used. This is used by Boyd in Convex Optimization, and I think I've seen it elsewhere (systems/control literature), but I can't recall exactly off the top of my head. Whether it's more or less confusing is debatable, though, since for vectors, \succ usually means componentwise greater-than. Overall, it's probably a win, assuming you know what's a matrix and what's a vector. --Paul Vernaza (talk) 04:48, 10 February 2008 (UTC)

[edit] minor question

Recall the property mentioned: 9. If M > 0 is real, then there is a δ > 0 such that M\geq \delta I where I is the identity matrix. I guess, we can as well write M > \delta I (because of the eigenvalue characterization), right? 146.186.132.163 19:43, 12 March 2007 (UTC)

yes. "is real" presumably means symmetric with real entries. it's true for positive matrices with complex entries also. Mct mht 22:18, 12 March 2007 (UTC)

[edit] Hermitian

The leading paragraph states that a positive definite matrix is Hermitian. Isn't that simply wrong? (The same article considers non-Hermitian positive definite matrices, anyway). The last sentence speaks of some "disagreement" about the definition for complex matrices; the article at Mathworld, however, explains it is only that some authors merely restricted the discussion on Hermitian matrices and that the definition have instances in matrices real and complex, Hermitian and non-Hermitian. -- 213.6.23.13 (talk) 16:07, 24 February 2008 (UTC)

You may notice that the "some authors" cited by MathWorld are in fact quite a lot respectable authors. Some of them restrict the definition of positive-definite matrix to Hermitian matrices. I think the end of this article reflects the literature better than MathWorld (which is not surprising given that I wrote it).
You're of course most welcome to improve the article. I agree that it's unfortunate that the leading paragraph states that a positive definite matrix is Hermitian, while there is no agreement in the literature about this. Perhaps we should just remove the word Hermitian in the first sentence? -- Jitse Niesen (talk) 14:41, 29 February 2008 (UTC)