Talk:Pseudoinverse

From Wikipedia, the free encyclopedia

Is this article too much a listing of things now? (Let this... Let that...) - User:Kaol

Looks fixed to me now (it is now a structured listing of things ;-). RainerBlome 22:15, 19 Sep 2004 (UTC)

Contents 1 To Do 2 Generalization to linear mappings 3 Applications 4 Finding the pseudoinverse of a matrix 5 Which categories to choose? 6 Itemization Markup 7 Transposition vs. Conjugation 8 Derivation ?

[edit] To Do

• Show computation using SVD.
• Link to correlation matrix (AA^T), maybe factor that into a separate article.
• Show incremental computation.
• Show numerical complexity of computations.
• Add Examples section
• Mention that there are other generalized inverses, link to prospective page

[edit] Generalization to linear mappings

I removed

The pseudoinverse is defined as above for linear mappings in general (not checked in books, can somebody confirm?).

No its not defined on a linear map between arbitrary vector spaces. Of course one needs at least an inner product space for it to make sense. There are some operators on infinite dimensional Hilbert spaces with generalized inverses. For example construct an operator on l² with a suitable singular value decomposition and use the SVD to construct the generalized inverse. User:Billlion 11:33, 7 Sep 2004

What I would like to do is to move away from the requirement that "A is a matrix". As far as I can see, there is no need to refer to a basis. Would "linear operator on an inner product space of finite dimensionality" be sufficient? RainerBlome 22:15, 19 Sep 2004 (UTC)

I think the last thing this article needs is more generality. The point is that mostly people who are interested in teh pseudoinverse are practical scientists and engineers using it to solve least-sqyuares problems, if you start the article by talking about linear transformations in inner product spaces you have lost most of them already. Fine to explain that it the notion is independent of baisis later on, and hence defined for linear maps, but the article should definitely start of describing the case for a matrix. Billlion 07:59, 20 Sep 2004 (UTC)

Good point. My intent is not really to convert from matrix to LinOp. Rather, I am trying to find out under what conditions the stuff in the article still holds in the case of LinOps. You could say I want to see whether there is a different point of view. If so, it should be shown in the article. However, I have been unable to find out for sure so far.

What would be the first thing that this article needs, in your opinion? RainerBlome 14:25, 20 Sep 2004 (UTC)

When I teach this stuff I start by explaining the need for least squares 'solutions' of overdetermined systems, and least norm solutions of underdetermined ones. In most practical situations where linear algebra is applied the system to be solved is overdetermined as sensible scintists always use more measuremnets that they have unknowns. Without this part of the story it looks the pseudoinverse like it is just contrived as an algebraist's whim. Billlion 19:20, 20 Sep 2004 (UTC)

The idea of a pseudo-inverse extends to bounded linear operators on arbitrary Hilbert spaces (= complete inner product spaces). The only delicate issue compared to finite-dimensional spaces is that in the general the pseudo-inverse of a bounded operator is not necessarily bounded and thus not necessarily continuous, which can pose problems in numerical simulations of inverse problems (such as backward solution of PDEs) --128.130.51.88 16:15, 12 August 2005 (UTC)

[edit] Applications

I just moved this to discusion Billlion 17:09, 5 Dec 2004 (UTC) [Disclaimer: The below is a translation of a German text that I wrote years ago. As far as I remember, at that time I knew what I was doing. However, I am currently unable to verify whether the prose below is valid (the equations should be fine). Please verify it yourself if you think of actually using this.]

[edit] Finding the pseudoinverse of a matrix

Regarding the incremental algorithms, I copied this deleted offer to here RainerBlome 21:09, 26 Aug 2005 (UTC):

I am hesitant to write these [algorithms] down here, as I am not sure whether they provide an advantage over SVD at all. When I worked with them, I was not aware of the SVD method or at least I don't remember having been. Send me a note and I may find the time to write them up for Wikipedia.

Hi, Forgive me if I haven't made this comment in the correct syntax, I'm new to this. In the 'Finding the pseudoinverse of a matrix' section you supply two formulas for the psuedoinverse. Only one of these formula (A+ = inv(A*A) A*) worked for some real example matrices for which I wanted to find the psuedoinverse. The other formula (A+ = A* inv(AA*)) was incompuatable, because the inverse of (AA*) did not exist. It may be that which formula you should use depends on the matrix for which you want the psuedoinverse, in that which formula you use is determined by whether the rank comes from the rows or columns. However the text does not make this clear. regards Desmond Campbell 194.83.139.91 10:27, 7 November 2005 (UTC) d dot campbell at iop dot kcl dot ac dot uk

Right, which formula may be used depends on the column and row ranks. See the special cases section. --RainerBlome 21:21, 7 November 2005 (UTC)

[edit] Which categories to choose?

Jitse, why did you remove the article from the Linear Algebra category? Note that the article's first sentence implies that LA is a category for this article. --RainerBlome 21:35, 10 September 2005 (UTC)

I removed the article from the Linear Algebra category, because it is already in the Matrix Theory category, which implies that it's Linear Algebra. I don't see why it should also be explicitly in the Linear Algebra category, just like it's not in the Mathematics category even though one could just as well say that the first sentence implies that mathematics is a category for this article. -- Jitse Niesen (talk) 21:49, 10 September 2005 (UTC)

[edit] Itemization Markup

Is it really helpful to add punctuation (semicolons and a final full stop) at the end of itemization items? The goals for me are:

1. Clarity -- Make it easy to read.

2. Editability -- Make it easy to maintain, which enhances the chances of future edits being correct, which reduces correction workload, which leaves more time for content development.

To 1.: Mathematics books on my shelf are divided in semicolon usage, some use them, some don't. In my perception, terminating semicolons make equation items harder to read, so I prefer to leave them out. The itemization symbols (bullets) provide enough separation as they are. If an item contains text and not just an equation, a full stop is sometimes more appropriate. When punctuation is used, separating it from the equation by a little whitespace makes the equation easier to read.

To 2.: When items are added or moved, the punctuation may have to be changed. This is often overlooked. (Minor point)

I always use punctuation in lists; full stops if the items in a list are full sentences and semicolons if they are not. Most maths style books say that equations should be treated as normal text and that punctuation should be included (to be frank, I'm not sure that most say that, but I have never come across one that said that you can leave out punctuation). Separating punctuation from the equation seems rather strange to me.

You may want to read a related discussion at Wikipedia talk:WikiProject Mathematics/Archive7#periods at the end of formulas -- request for comment, in which it was suggested that engineers typically don't use punctuation.

In the end, I don't think it matters much whether punctuation is used or not, so feel free to change it to whatever you like if you feel strongly about it. However, I do think that mere bullet points of formulas are generally bad style (regardless of punctuation). One should write in English and not in mathematical formulas. -- Jitse Niesen (talk) 00:43, 11 September 2005 (UTC)

Thanks for pointing me to that discussion. I'm all for full stops after actual sentences, but I'm wary of semicolons. So I had searched for 'semicolon', but did not find any discussion. I agree that good prose is preferable to mere lists of equations. However, the loosely related equations commonly found in the properties section are fine as far as I am concerned. In this case they are not part of a common sentence, so should not be linked with semicolons. Some math books use the semicolon as a form of 'postfix separator'. I think that's fine as long as one doesn't use both a prefix and a postfix separator. Originally, I had listed the properties without separators at all, someone else had introduced the bullets. --RainerBlome 20:50, 11 September 2005 (UTC)

The itemization now uses prose. --RainerBlome 19:21, 15 September 2005 (UTC)

[edit] Transposition vs. Conjugation

The article should be easy to use both for readers who are using real matrices and for readers who are using complex matrices. Currently, this is not so. Some formulas (limiting process, special cases, applications) use transposition and it is not obvious to me whether transpositon may be or has to be replaced by conjugate transposition. When I worked with the pseudoinverse, I never used complex-valued elements, so the question never came up. Does anyone see right away instances of A^TA and such that can be replaced by A ^* A and such?

If the formulas using 'just' transposition are fine for complex matrices, that should be noted (but I don't know whether they are). In general, it should be clear where conjugate transpose is required in the complex case and where not. As long as this is not cleared up, at least there should be some kind of warning, that a formula might only hold in the real case. When I last left a similar warning, it was moved to the talk page. Is this standard practice?

In particular, the 'Special Cases' section gives A ⁺ = A^T(AA^T) ^{− 1} whereas the 'Finding...' section gives A ⁺ = A ^* (AA ^* ) ^{− 1}. Are there any complex A for which the ^T version does not work? Or, since the pseudoinverse is unique, are the two equivalent?

--RainerBlome 20:30, 15 September 2005 (UTC)

Fixed. In the complex case, conjugate transpostion must always be used. Actually, I have a proof only for the first limit expression, but hey, the other one just can't be wrong, can it? --RainerBlome 20:13, 10 October 2005 (UTC)

[edit] Derivation ?

"The pseudoinverse is used to find the best solution (in a least squares sense) to systems of linear equations which are either overdetermined or underdetermined".

I think that overdetermined use left pseudo-inverse while, in undertermined system, we should use right pseudo-inverse, shouldn't we ?

Gtdj 15:18, 26 August 2006 (UTC)

The sentence that you cite was wrong. The method is also applicable to SLE that are neither overdetermined nor underdetermined. The pseudoinverse can be used as the "best" solution to an SLE that has no unique exact solution (either no solution or infinitely many solutions). The SLE (represented by matrix A) may be "square" (be neither over- nor underdetermined), but still have no unique exact solution, because some rows or some columns are linearly dependent. The more general statement is already present in the Introduction and Application sections, so I removed the sentence.

The pseudoinverse is unique, what do you mean by "right/left pseudo-inverse"? If the SLE is overdetermined, it could be that the columns of A are linearly independent (but they don't have to be). If they are, A+ is a left inverse of A (see section "Special_cases"). If the SLE is underdetermined, it could be that the rows of A are linearly independent (but they don't have to be). If they are, A+ is a right inverse of A. --RainerBlome 22:48, 9 September 2006 (UTC)