Talk:Linear independence

From Wikipedia, the free encyclopedia

Contents 1 Projective space of dependencies 2 Vector or tuple? 3 Linear indepence of sets? 4 a survey of English textbooks re: linear independence 5 calculus not required 6 examples 7 Experiment 8 Linear dependences section 9 Linear independence over rings 10 determinant check 11 Example in lead 12 New section

[edit] Projective space of dependencies

I removed the claim that the space of linear dependencies of a a set of vectors is a projectiv space, since I couldn't make sense of it. The space of linear dependencies is a plain vector space, the kernel of the obvious map from Kⁿ to the vector space V. AxelBoldt 15:38 Oct 10, 2002 (UTC)

I've put that comment back, this time with an explanation. Michael Hardy 00:31 Jan 16, 2003 (UTC)

[edit] Vector or tuple?

The section "The projective space of linear dependences" currently starts with "A linear dependence among vectors v1, ..., vn is a vector (a1, ..., an)".

In colloquial speech and java "vector" is sometimes used for a tuple of varying length, but in mathematics a vector is an element of a vectorspace and not just any tuple. "A linear dependence [...] is a vector [...]", thus states that linear dependences form a vector space, which is not the case, as the 0 is missing.

Currently a linear dependence is called a vector just as v₁ to v_n, which can be misunderstood, that a linear dependence is a vector of the same vector space as v₁ to v_n, but it is not.

A linear dependence is, in the first place, a tuple, or more generally a family. It could have become a vector, if linear dependeces formed a vector space, but they don't, they form a projective space, which is a manifold and linear dependences are points of this manifold. Markus Schmaus 09:45, 15 Jun 2005 (UTC)

I do not agree that "A linear dependence [...] is a vector [...]" states that linear dependences form a vector space; it only states that they are elements of a vector space. Similarly, one could say "An eigenvector is a vector v such that ...", even though eigenvectors cannot be zero. -- Jitse Niesen 10:37, 15 Jun 2005 (UTC)

Do we agree that vector simply referes to an element of a vector space?

If so, we can rephrase both statements. "An eigenvector is an element of the vector space, such that ..." makes perfect sense, it is clear which vector space is meant. With "A linear dependence [...] is an element of the (a) vector space [...]" it is not clear what this vector space is, it is not the vector space of the v_i-s, nor is it the vector space of linear dependences, as they form no vector space. We have to specify which vector space we mean and say "A linear dependence [...] is an element of the vector space of n-tuples [...]" or shorter "A linear dependence [...] is an n-tuple [...]".

Compare this with the beginning of quadratic function. "A quadratic function is a vector, ..." would be a bad beginning, as, even though polynomial functions form a vectorspace, a quadratic function is, in the first place, a polynomial function. Markus Schmaus 15:01, 15 Jun 2005 (UTC)

The article says "A linear dependence among vectors v₁, ..., v_n is a vector (a₁, ..., a_n) [...]". It seems clear to me of which vector space the vector (a₁, ..., a_n) is an element of, namely the vector space Kⁿ. Furthermore, one needs the vector space structure to turn it into a projective space. However, I don't think it's worth it to argue about this, so change it to tuple if you insist. -- Jitse Niesen 16:01, 15 Jun 2005 (UTC)

Sorry for insisting, but there's an itching everytime I read the paragraph. Markus Schmaus 20:10, 19 Jun 2005 (UTC)

Please go ahead. Jitse Niesen 20:24, 19 Jun 2005 (UTC)

[edit] Linear indepence of sets?

The article currently starts with "a set of elements of a vector space is linearly independent [...]" I don't think this refers to a set in the mathematical sense. Suppose v₁ = v₂ = (1,0), the set { v₁, v₂ } is linearly independent as it contains only one element (1,0), which isn't zero. But the vectors v₁ and v₂ are not linearly independent, as v₁ - v₂ = 0. Linear independence is the property of a set, but the property of a family (mathematics). Markus Schmaus 15:37, 15 Jun 2005 (UTC)

The article currently starts with "a set of elements of a vector space is linearly independent if none of the vectors in the set can be written as a linear combination of finitely many other vectors in the set". If you take the set S = {v₁,v₂} with v₁ = v₂ = (1,0), then S has only one element, namely (1,0), and none of the elements of S can be written as a linear combination of other elements of S. -- Jitse Niesen 16:14, 15 Jun 2005 (UTC)

I have now been convinced by Markus Schmaus that there is indeed a problem here, see User talk:Markus Schmaus#Families. Consider the following two statements:

• A set of elements of a vector space is linearly independent if there is a nontrivial (finite) linear combination that gives zero;
• A square matrix is invertible if and only if its columns are linearly independent.

Combining these statements gives: "A square matrix is invertible if and only if there is no nontrivial linear combination of the set of vectors, formed by the columns of the matrix." Now, apply this to the matrix

This matrix has two columns, which are equal, hence the set of vectors only has one element. This set is linearly independent, yet the matrix is not invertible; contradiction.

The question is now, how to resolve this problem? Possibilities are

1. Reformulate the definition as: "A family of elements of a vector space ..." This is Markus' preferred solution. I do not like it very much as it is not the standard definition in the English literature, and it uses the rather uncommon term family, but it might be the best solution.
2. Remove the statement "A square matrix is invertible if and only if its columns are linearly independent" in invertible matrix. There might be other statements lurking around that are similarly subtly wrong.
3. Have two definitions, one for finite-dimensional vector spaces ("The vectors v₁, ..., v_n in some vector space ...") and one for infinite-dimensional spaces (in terms of families). This might be the most readable solution, but it is a bit ugly.
4. Do not care about it, since everybody knows what is meant.

I am not happy with any of these solutions, least of all with the last one, so I am soliciting comments from other contributors. -- Jitse Niesen 17:02, 17 Jun 2005 (UTC)

I don't think 4. is acceptable as people are looking on wikipedia, because they don't know what is meant. I'm often sloppy myself, but sloppiness is a luxury, we can't afford on wikipedia. Nor do I think it is an option to remove the "invertability" statement without replacement. It is true in some way and helps to understand the relation between linear independence, bases and matrices.

Currently I would suggest an initial paragraph starting with "A collection, more precisely a set, sequence, or family, of elements of a vectorspace …".

We will need two different formal definitions, as the current definition depends on a family, even though this is not mentioned explicitly. So there is even a subtle error within the same page.

If v₁, v₂, ..., v_n are elements of V, we say that they are linearly dependent over K if there exist elements a₁, a₂, ..., a_n in K not all equal to zero such that:

If we pick v₁ = v₂ as it is the case with the matrix, a₁ = 1 and a₂ = -1 we get

v₁ - v₁ = 0

Hence v₁, v₂ are linearly dependent according to this definition, which is in accordance with the "invertibility" statement but not with the current initial paragraph. Markus Schmaus 15:39, 18 Jun 2005 (UTC)

[edit] a survey of English textbooks re: linear independence

There are 2 notions of linear independence, one whether you regard the collection of vectors as simply a set, the other whether you regard it as an indexed family. Each definition is necessary; e.g. without the "family" definition, the statements about column/row rank become false, whereas a statement such as "every vector space has a basis" has meaning with respect to the "set" definition. So, each is useful...and each should be included in the article.

Here is a survey of the most popular English language algebra textbooks:

• Lang -- gets it right (almost...he's not 100% clear that a linear combination must "combine like terms") he makes clear that there are 2 definitions involved, and he gives the concrete example of v_1 = v_2 = ... = v_n to show where they differ. Hurray for Lang!
• Dummit/Foote -- confuses the 2 definitions
• Hungerford -- splits the difference, takes the set definition but requires the vectors to be unique
• MacLane Birkhoff -- typically, translates the usual definitions into abstract nonsense, but essentially uses the family definition, except the emphasis is on finite dimensions, so I didn't immediately find general definition of basis, e.g.
• Kurosh -- confuses the 2 definitions
• Herstein (Topics) -- takes the set definition, but goes on proving things that should be stated with family definition
• Hoffman/Kunze -- takes the set definition, but seems to avoid incorrect statements by using ordered bases and talking about rank instead of linearly independent columns/rows
• Axler (Linear Algebra Done Right) -- takes the family definition, and dismisses the set definition (with explanation)! An exceptional case.

All in all, there's enough confusion to warrant:

1. Giving both definitions.
2. Explaining the difference, giving an example to show when they are not equal.
3. Explain difference between family and ordered basis (family is indexed, but still unordered)
4. Show when and exactly what way the family definition is or is not needed.

Revolver 16:54, 19 Jun 2005 (UTC)

Have two definitions, one for finite-dimensional vector spaces ("The vectors v_1, ..., v_n in some vector space ...") and one for infinite-dimensional spaces (in terms of families). This might be the most readable solution, but it is a bit ugly.

This doesn't address the issue. The problem remains, whether one restricts to finite dimensional vector spaces or not. Revolver 16:59, 19 Jun 2005 (UTC)

Wow, I never thought that it would be so hard to define linear independence. I am now convinced that we need to include the definition using families. I tried to change the definition in the article to accommodate this discussion. Note that I did not change the very first sentence of the article, since I feel it is okay to be slightly sloppy there. I did not write anything about the differences between the family-based and set-based definition; I do not want to spend too much space on it, but we should probably write a few lines about it. Let me know what you think about it. -- Jitse Niesen 18:39, 19 Jun 2005 (UTC)

In that case I'd prefer collection instead of set, as the first can also refer to a family. Markus Schmaus 19:52, 19 Jun 2005 (UTC)

Thanks for the survey on English text books. Markus Schmaus 19:52, 19 Jun 2005 (UTC)

Revolver, I forgot to do say so, but also thanks from me.

Markus, regarding collection versus set, be bold! (I didn't know that collection can also refer to a family). By the way, thanks for catching my embarrassing mistake in the rewrite of the definition. Jitse Niesen 20:24, 19 Jun 2005 (UTC)

I don't think collection is a precise mathematical term, it's kind of soft, sometimes refering to a set, somtimes to a class, sometimes to a family, and maybe to other things I don't know. There is a pragraph on collection defining it as equivalent to a set but I don't think that's common usage, so I'm going to change that. Markus Schmaus

A term that may be more accessible: Michael Artin's Algebra (ISBN 0130047635) uses "ordered set". I've also seen "list", defined as a mapping from the natural numbers, but that would imply countability. Artin also states a finite-dimensional form, then expands, as suggested above. (Bourbaki might sneer, but students won't.) I believe the infinite-dimensional case is too important to omit in the body, though perhaps the opening can acknowledge it without a definition. Incidentally, Artin, and Mac Lane&Birkhoff (ISBN 0828403309, not the older Survey), take care to say that for some uses a basis can be a set (unordered), but most practical work requires the ordering. --KSmrq^T 03:40, 4 November 2005 (UTC)

[edit] calculus not required

I can do that calculus example without calculus: divide both sides by e^t. then you have a be^t = constant, which is only true when b=0. Why use calculus if we don't need it? I'm going to change it. -Lethe | Talk 01:10, August 9, 2005 (UTC)

[edit] examples

I removed the calculus from the proof of example III. I also think that there are too many uninteresting examples. Example I and II are almost identical, and example III isn't too interesting either. what about an example of an infinite family of independent vectors, like say a basis for l^2. -Lethe | Talk 01:18, August 9, 2005 (UTC)

[edit] Experiment

Just thinking aloud: Is it an improvement to change the current opening from this:

---

In linear algebra, a family of vectors is linearly independent if none of them can be written as a linear combination of finitely many other vectors in the collection. For instance, in three-dimensional Euclidean space R³, the three vectors (1, 0, 0), (0, 1, 0) and (0, 0, 1) are linearly independent, while (2, −1, 1), (1, 0, 1) and (3, −1, 2) are not (since the third vector is the sum of the first two). Vectors which are not linearly independent are called linearly dependent.

---

to something like this:

---

In linear algebra, an ordered set of finite-dimensional vectors is linearly independent if no vector in the set can be written as a linear combination (a weighted sum) of those preceding it. That is, no vector is in the span of its predecessors. More broadly, in a possibly infinite-dimensional vector space, a family of vectors is linearly independent if any vector in the full space can be written in at most one way as a linear combination of a finite number of those in the family. Vectors which are not linearly independent are called linearly dependent.

---

I see both advantages and disadvantages. Mainly, I thought it might help to experiment, to free our thinking a little. --KSmrq^T 08:13, 4 November 2005 (UTC)

Vector spaces, not vectors, have dimensionality. Therefore the phrase "finite-dimensional vectors" has no meaning. Not too keen on predecessors. Prefer family to ordered set. -Lethe | Talk 10:36, 4 November 2005 (UTC)

I think it is fine the way it is - ordering isn't necessary. On a different matter, should the projective spaces of linear dependencies bit really be before the examples of linear independence? JPD (talk) 10:50, 4 November 2005 (UTC)

(JPD, if you have no interest in my question, please take up yours in a different thread, not this one. Thanks.)

Yes, it might be prudent to say "an ordered set of vectors in a finite-dimensional vector space", though the shorthand is often used for convenience. One of my questions is whether order is an advantage or disadvantage. Or is it even better to have both a "procedural" definition and a "universal" definition (bordering on category theory), as illustrated. Lethe, do you want to defer mention of order to the article body, or omit it entirely? And should I assume that you like the "at most one way" definition (borrowed from Mac Lane&Birkhoff)?

I have one definite opinion: The column vector display of independence and dependence works much better than the inline version. (Also, I do think it best to choose vectors that are not orthonormal.) --KSmrq^T 12:33, 4 November 2005 (UTC)

I take it that you actually think I should discuss the ordering in a separate thread, even though I am interested in your question. That's a fair point. I agree that the column vector display is more illuminating, but I don't particularly like any of your other suggested changes. JPD (talk) 13:56, 4 November 2005 (UTC)

I was objecting to "On a different matter…". Order is fair game. So far, no fans. Which is interesting in itself, because it is no more restrictive (in finite dimensions) than unordered, but more practical. It's hard to tell from the brief responses whether the antipathy is a matter of taste or technical difficulties (not that there's a sharp line). But, unless someone speaks up it looks like "family" — despite being unfamiliar to almost anyone needing to learn about linear independence — is here to stay. I'm not surprised, and I can live with "family"; but I am a little disappointed that we can't seem to do better. --KSmrq^T 21:07, 4 November 2005 (UTC)

I'm not one hundred percent sure, but I think in my case, my complaints are purely matters of taste. -Lethe | Talk 00:27, 5 November 2005 (UTC)

I guess you could say my objections are taste as well, but I think it's important that linear dependence does not in any way depend on order. I don't see how the order makes it any more practical. Earlier, someone suggested "collection" rather than family. I guess the point is that linear independence can be defined for sets, or sequences (finite or infinite), in exactly the same way. I doubt that anyone who doesn't already know about linear independence is going to be bothered about whether define it for a set, sequence, family or collection. JPD (talk) 14:45, 5 November 2005 (UTC)

My reading of previous discussion is that "family" is preferred over "collection" (too vague, per Markus Schmaus) and "set" (unacceptable, because a set excludes duplicates). In the finite-dimensional case, "list", a mapping from natural numbers to items (here, vectors), is OK; but we can have an uncountable collection in an infinite-dimensional case.

On order, three comments:

1. In saying one vector is a linear combination of others, we already introduce an artificial asymmetry, if not an order per se. If three vectors are dependent, which is the culprit? The "at most one" definition has no such asymmetry. The asymmetric version also hides a subtle flaw: the zero vector should not be considered independent, even if it is the only vector given.
2. In giving a first definition using order, we do not restrict; it only seems so. For, we can prove that if a list is dependent, so is any reordering of it. This should appear explicitly in the body, because even the family definition currently used does not make this explicit.
3. Stipulate that the empty set is linearly independent, and that its span is the zero space consisting of just the zero vector. Many practical computations involving linear independence, and certainly "basis", will actually use an ordering. For example, Gram-Schmidt orthogonalization in an inner product space converts an independent list into an orthonormal list. Or consider the theorem that if an ordered set spans V then it contains a linearly independent subset that spans V. So whether we start with an ordered definition or not, we will eventually want to derive and use one.

Incidentally, may I assume that nobody likes the "at most one" definition either? --KSmrq^T 22:18, 5 November 2005 (UTC)

I think I see what you mean about the introduced assymetry, for that sort of reason would agree that "one vector is a linear combination of the others" is not my favourite definition of dependence. However, it is easier to grasp for most people, so I was not about to change it. I do not think there is actually any assymmetry there, especially when defining independence rather than dependence. I think the current version and use cover the zero vector validly and equally well, but it might be worth adding a sentence to make it clear.

I agree that we don't restrict by using order, by I don't see any reason to use order in the definition and then show that it doesn't matter, when there are perfectly good definitions which don't use order at all. The family doesn't need to make make the irrelevance of reording explicit, because there is no explicit ordering in th e concept of a family.

I agree that the empty set should be mentioned in the body in some way. I don't think the fact that we often use ordered bases is relevant. The ordering is not part of the definition of basis, and definitely not needed for linear independence. The general definition covers the ordered case, so we do not need to derive a definition, in order to have linear indepedence in an ordered set or a sequence. Lastly, collection is a bit vague, but that is not a completely bad thing. Family is good because it is easily understood in the same vague sense by those who don't want detail, yet does have a detailed definition for those of us who care. JPD (talk) 19:56, 6 November 2005 (UTC)

[edit] Linear dependences section

It seems a bit strange that this section is before the examples. Does anyone agree? JPD (talk) 13:56, 4 November 2005 (UTC)

Yes. If I was teaching students I definitely wouldn't do it in that order! --RFBailey 21:04, 6 November 2005 (UTC)

A strong alternative is to first define a linear relationship or dependency or whatever as a weighted sum of vectors, with not all weights zero, that equals the zero vector. Then define linear independence as the lack of a linear relationship among any of the vectors. That helps several weaknesses at once:

• remove asymmetry from definition
• shows a set with only the zero vector is not independent
• better integrates the digression about projective space
• supports definition of mapping from scalar tuples to vectors, Kⁿ → V, defined by any n-family in V, which is
  1. monomorphism for independent family
  2. epimorphism for spanning family
  3. isomorphism for basis family, thus defining coordinates

Otherwise, the whole discussion of projective space seems unmotivated. --KSmrq^T 05:05, 7 November 2005 (UTC)

Firstly, however indepedence or dependence are defined in the article, I think the examples should come before any mention of the projective space of linear dependences. Apart from that, I agree that KSmrq's suggestion would be a good approach in the Definition section and following. I don't see where the where some of the weaknesses he mentions are, but it would be good to make this section flow better and give a bit more idea of what the relevance of linear independence is. JPD (talk) 15:56, 7 November 2005 (UTC)

[edit] Linear independence over rings

It should at least be mentioned that in a module over a ring, linear independence (in the sense that zero cannot be represented as a non-trivial linear combination) is not equivalent to one vector being contained in the span of the remaining elements.--80.136.131.201 18:37, 20 January 2006 (UTC)

This article is only about linear independence over the real numbers. For general rings, things are messy indeed, and I don't even think that the concept if linear independence is at all useful. Oleg Alexandrov (talk) 23:45, 20 January 2006 (UTC)

Some parts of the usual theory work pretty well: Linear independence of the columns of a matrix corresponds to uniqueness of solutions (if any exist), a subset of a module is a basis iff it is linearly independent and generating. Cf. the characterization based on properties of the induced map mentioned by KSmrq in the preceding section.--80.136.131.201 00:14, 21 January 2006 (UTC)

[edit] determinant check

The article states that a check of independence can use the determinant when the number of vectors equals the dimension. In effect, this is a check to see if the vectors form a basis. But even when the number of vectors, m, is less than the dimension, n, we can check for independence by requiring all m×m subdeterminants to be nonzero. For example, the following three vectors of dimension four are independent.

I'd suggest this is worth mentioning in the article. --KSmrq^T 01:15, 31 January 2006 (UTC)

Sure, put it in. Personally, I always like to err on the side of exhaustiveness, this being an encyclopedia, not a textbook. By the way, both statements are implied by the more high-brow statement that a set of vectors is dependent iff their wedge product is 0. -lethe ^{talk +} 01:54, 31 January 2006 (UTC)

Shouldn't that be 'by requiring at least one of the sub-determinants to be non-zero'? 137.205.139.149 20:58, 19 January 2007 (UTC)

You're a year late to the party. The article has already been augmented, and you can check that the language used there is correct. It says we have dependence (not independence) if all subdeterminants are zero, which (as you correctly observe) means that at least one nonzero subdeterminant indicates independence. But as a practical matter, for numerical computation we would use a rank-revealing decomposition. --KSmrq^T 17:12, 20 January 2007 (UTC)

Can this be proven simply (ie. without involving wedge product and in an uncomplicated fashion)? If so, it would probably be worthwhile to reference some such proof here. Robryk (talk) 22:50, 25 January 2008 (UTC)

[edit] Example in lead

Did anyone actually solve the first example? I think it should be (9,2,3) instead of (4,2,3). —Preceding unsigned comment added by Caelumluna (talk • contribs) 02:38, 20 September 2007 (UTC)

It looks fine to me. -- Jitse Niesen (talk) 04:30, 20 September 2007 (UTC)

[edit] New section

I've removed the newly-added "Formula" section, because it's overly-complicated, and more importantly, redundant.

The article already states that a set of vectors is linearly independent iff the determinant of the matrix they form (let's call it X) is non-zero. However, the matrix formed in the new section (the matrix of dot products) is in fact equivalent to calculating X^TX. Of course, |X^TX| ≠ 0 iff |X| ≠ 0. Therefore, this section says nothing new, I'm afraid. Oli Filth^(talk) 23:25, 8 May 2008 (UTC)

OK, I basically agree, but it is not that trivial: suppose you have a set of 4 vectors which 'live' in and you want to find out if they are linearly independent or not. Then matrix X will have 4 columns and 7 rows: it is not square, so detX is undefined. In such case it is not a redundancy to compute detX^TX. And then, in such case (for non-square X), how does one show (succintly) that detX^TX tests for linear independence? Here is an argument: the
4-D subspace contains some orthonormal basis where . Arrange these vectors in columns to form matrix E, which will be 7 × k. Then E^T is k × 7 and a left-inverse of E, so E^TE = I and EE^T is a 7 × 7 symmetric projection operator which projects vectors in onto the k-dimensional subspace. As such, let Π = EE^T. Then ΠX = X and so where is a 4 × 4 square matrix and X_(E): = E^TX is the matrix X expressed in terms of the basis . Now it is indeed true that where the left side is the usual test for linear independence (as well as giving the 4-volume of the 'parallelotope' whose sides are parallel to ).
[ - because , so . But in such cases (of non-square X) it is easier to compute detX^TX directly rather than first finding E (say, by Gram-Schmidt orthogonalization) and then computing X_(E): = E^TX and then detX_(E).]

Note that since , that is, X^TX is invariant under change of (the subspace's) basis, then X^TX would be a tensor, not unlike the metric tensor.

Anyway, today I found out that X^TX is called Gram matrix, so I added a See also link to it, and that should be enough. There is a nice PDF file on the web about this topic: http://www.owlnet.rice.edu/~fjones/chap8.pdf, followed up with http://www.owlnet.rice.edu/~fjones/chap11.pdf, though this goes well beyond mere linear independence (perhaps I'll add the first one under External links for the Gram matrix article). Anyway, thank you for describing my edit as 'good faith'. —AugPi (talk) 02:10, 10 May 2008 (UTC)

You are absolutely correct that det(X) only exists for square matrices. I realised my mistake as soon as I had performed my edits, and was about to update the existing material on determinants to also discuss det(X^TX), but then got distracted. I will make these changes today! Oli Filth^(talk) 11:24, 10 May 2008 (UTC)