CS6015: Linear Algebra and Random Processes
Lecture 6: Vector spaces, subspaces, independence, span, basis, dimensions
Learning Objectives
What is a vector space and a subspace?
What is the basis of a space/subspace?
What is the span of a set of vectors?
What are independent vectors?
What is the dimension of a space/subspace?
(for today's lecture)
The bigger picture
\begin{bmatrix}
~~~&~~~&~~~\\
~~~&~~~&~~~\\
~~~&~~~&~~~\\
\end{bmatrix}
m<n
m < n
m=n
m=n
\begin{bmatrix}
~~~&~~~&~~~&~~~&~~~\\
~~~&~~~&~~~&~~~&~~~\\
~~~&~~~&~~~&~~~&~~~\\
\end{bmatrix}
\begin{bmatrix}
~~~&~~~&\\
~~~&~~~&\\
~~~&~~~&\\
~~~&~~~&\\
~~~&~~~&\\
~~~&~~~&
\end{bmatrix}
m>n
m > n
rank=
rank =
A
A
A
A
A
A
Q1. For which b's does a solution not exist?
0,1,∞ solutions
0,1,\infty~solutions
Q2. Which x's are a solution for Ax = 0?
Q3. What's the connection between these two Qs?
Ax=b
A\mathbf{x}=\mathbf{b}
n pivots
1 unique solution
Find L,U
Or find
A−1
A^{-1}
Vector spaces
Why this detour?
All b's for which a solution to Ax=b exists
(well, it's not really a detour)
We need some vocabulary to specify:
All x's which are a solution to Ax=0
[Answer: some vector (sub)space]
[Answer: some vector (sub)space]
We will spend some time in learning this vocabulary (concepts)!
Recap: What do you do with vectors?
Linear equations
x=x1x2x3
\mathbf{x} =\begin{bmatrix}
x_1 \\ x_2 \\ x_3
\end{bmatrix}
y=y1y2y3∈R3
\mathbf{y} =\begin{bmatrix}
y_1 \\ y_2 \\ y _3
\end{bmatrix} \in \mathbb{R}^3
Add
x+y=x1+y1x2+y2x3+y3∈R3
\mathbf{x} + \mathbf{y} =\begin{bmatrix}
x_1 + y_1 \\ x_2 + y_2 \\ x_3 + y_3
\end{bmatrix} \in \mathbb{R}^3
Scale
ax=ax1ax2ax3
a\mathbf{x} =\begin{bmatrix}
ax_1 \\ ax_2 \\ ax_3
\end{bmatrix}
=ax1x2x3∈R3
=a\begin{bmatrix}
x_1 \\ x_2 \\ x_3
\end{bmatrix} \in \mathbb{R}^3
Linear combinations
ax+by=ax1+by1ax2+by2ax3+by3
a\mathbf{x} + b\mathbf{y}=\begin{bmatrix}
ax_1 + by_1\\ ax_2 + by_2\\ ax_3 + by_3
\end{bmatrix}
Vector Spaces
Linear equations
u,v,w∈V
\mathbf{u},\mathbf{v},\mathbf{w} \in V
a,b∈R (scalars)
a,b \in \mathbb{R}~(scalars)
(definitions and properties)
V
V
A vector space is a collection of vectors which can be added together and be multiplied by scalars (i.e., we can take linear combinations)
u+(v+w)=(u+v)+w
\mathbf{u}+(\mathbf{v}+\mathbf{w}) = (\mathbf{u}+\mathbf{v})+\mathbf{w}
u+v=v+u
\mathbf{u}+\mathbf{v} = \mathbf{v}+\mathbf{u}
v+0=v
\mathbf{v}+\mathbf{0} = \mathbf{v}
v+−v=0
\mathbf{v}+\mathbf{-v} = \mathbf{0}
Addition properties
a(bv)=(ab)v
a(b\mathbf{v}) = (ab)\mathbf{v}
1(v)=v
1(\mathbf{v}) = \mathbf{v}
a(u+v)=au+av
a(\mathbf{u} + \mathbf{v}) = a\mathbf{u} + a\mathbf{v}
(a+b)v=av+bv
(a + b)\mathbf{v} = a\mathbf{v} + b\mathbf{v}
Multiplication properties
Closure Properties
u+v∈V
\mathbf{u} + \mathbf{v} \in V
au∈V
a\mathbf{u} \in V
au+bv∈V
a\mathbf{u} + b\mathbf{v} \in V
Examples of Vector Spaces
Linear equations
(that we care about)
R
\mathbb{R}
R2
\mathbb{R}^2
R3
\mathbb{R}^3
Rn
\mathbb{R}^n
⋯
\cdots
123
\begin{bmatrix}
1 \\ 2 \\ 3
\end{bmatrix}
214
\begin{bmatrix}
2 \\ 1 \\ 4
\end{bmatrix}
000
\begin{bmatrix}
0 \\ 0 \\ 0
\end{bmatrix}
⋯
\cdots
⋯
\cdots
We will take these for granted
This is what we will really worry about
au+bv∈V
a\mathbf{u} + b\mathbf{v} \in V
Subset of a Vector Space
Linear equations
What if we remove one vector from
au+bv∈V
a\mathbf{u} + b\mathbf{v} \in V
R3
\mathbb{R}^3
(say, the 0 vector) ?
Is this subset a vector space?
(we get a subset of )
R3
\mathbb{R}^3
(clearly not: )
au∈/V for a=0
a\mathbf{u} \notin V~for~a=0
Can we think of a subset of R3 that still satisfies the requirements of a vector space?
Subspaces
Linear equations
A subspace is a non-empty subset of a vector space that still satisfies the requirements of a vector space
Closure Properties
u+v∈S
\mathbf{u} + \mathbf{v} \in S
au∈S
a\mathbf{u} \in S
au+bv∈S
a\mathbf{u} + b\mathbf{v} \in S
if u,v∈S
if~\mathbf{u}, \mathbf{v} \in S
(of course, the addition and multiplication properties should also be satisfied)
Examples of subspaces
Linear equations
Let V=R2
Let~V = \mathbb{R}^2
(i.e., the original space)
We are looking for subspaces of this space
The 0 vector is a subspace (trivial)
u=[44]
u=\begin{bmatrix}
4\\4
\end{bmatrix}
Is S={u} a subspace?
(clearly not: )
au∈/S for a=0
a\mathbf{u} \notin S~for~a=0
What should we add to S to make it a subspace?
0
\mathbf{0}
2u
2\mathbf{u}
3u
3\mathbf{u}
4u
4\mathbf{u}
…
\dots
(all multiples of )
u
\mathbf{u}
Examples of subspaces
Linear equations
What are the possible subspaces of R2 ?
The 0 vector (trivial)
u=[44]
u=\begin{bmatrix}
4\\4
\end{bmatrix}
Any line passing through the origin
The whole of R2
(all multiples of any vector )
u
\mathbf{u}
Note that the 0 vector will always be a part of any subspace
(if u∈S then 0⋅u=0 must be ∈S)
(if~\mathbf{u} \in S~then~0\cdot\mathbf{u}=\mathbf{0}~must~be~\in S)
Examples of subspaces
Linear equations
What are the possible subspaces of R3 ?
u
\mathbf{u}
Examples of subspaces
Linear equations
What are the possible subspaces of R3 ?
The 0 vector (trivial)
The whole of R3
(all multiples of any vector u
Note that the 0 vector will always be a part of any subspace
Any line passing through the origin
(all linear combinations of )
Any plane passing through the origin
u,v
\mathbf{u},\mathbf{v}
(if u∈S then 0⋅u=0 must be ∈S)
(if~\mathbf{u} \in S~then~0\cdot\mathbf{u}=\mathbf{0}~must~be~\in S)
Examples of subspaces
Linear equations
Possible subspaces of R3
Puzzles: complete these subspaces
Linear equations
u=1−12
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2
\end{bmatrix}
v=221
\mathbf{v}=\begin{bmatrix}
2\\
2\\
1
\end{bmatrix}
w=113
\mathbf{w}=\begin{bmatrix}
1\\
1\\
3
\end{bmatrix}
au
a\mathbf{u}
∀a∈R
\forall a \in \mathbb{R}
u=1−12
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2
\end{bmatrix}
au+bv
a\mathbf{u} + b\mathbf{v}
∀a,b∈R
\forall a,b \in \mathbb{R}
v=221
\mathbf{v}=\begin{bmatrix}
2\\
2\\
1
\end{bmatrix}
u=1−12
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2
\end{bmatrix}
au+bv+cw
a\mathbf{u} + b\mathbf{v} + c\mathbf{w}
∀a,b,c∈R
\forall a,b,c \in \mathbb{R}
What kind of a surface would it be?
(line)
(2d plane)
whole of
R3
\mathbb{R}^3
Puzzles: complete these subspaces
Linear equations
u=1−121
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2\\
1
\end{bmatrix}
v=2211
\mathbf{v}=\begin{bmatrix}
2\\
2\\
1\\
1
\end{bmatrix}
w=1131
\mathbf{w}=\begin{bmatrix}
1\\
1\\
3\\
1
\end{bmatrix}
au
a\mathbf{u}
∀a∈R
\forall a \in \mathbb{R}
u=1−121
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2\\
1
\end{bmatrix}
au+bv
a\mathbf{u} + b\mathbf{v}
∀a,b∈R
\forall a,b \in \mathbb{R}
v=2211
\mathbf{v}=\begin{bmatrix}
2\\
2\\
1\\
1
\end{bmatrix}
u=1−121
\mathbf{u}=\begin{bmatrix}
1\\
-1\\
2\\
1
\end{bmatrix}
au+bv+cw
a\mathbf{u} + b\mathbf{v} + c\mathbf{w}
∀a,b,c∈R
\forall a,b,c \in \mathbb{R}
What kind of a surface would each of it be?
(line)
(2d plane)
(3d hyperplane)
Puzzles: what is the subspace?
Linear equations
u=1111
\mathbf{u}=\begin{bmatrix}
1\\
1\\
1\\
1
\end{bmatrix}
v=1234
\mathbf{v}=\begin{bmatrix}
1\\
2\\
3\\
4
\end{bmatrix}
⟹au+bv+cw+dz∈S
\implies a\mathbf{u} + b\mathbf{v} + c\mathbf{w} + d \mathbf{z} \in S
∀a,b,c,d∈R
\forall a,b,c,d \in \mathbb{R}
w=−1−1−1−1
\mathbf{w}=\begin{bmatrix}
-1\\
-1\\
-1\\
-1
\end{bmatrix}
z=−1−2−3−4
\mathbf{z}=\begin{bmatrix}
-1\\
-2\\
-3\\
-4
\end{bmatrix}
u,v,w,z∈S
⟹au+bv+c(−u)+d(−v)∈S
\implies a\mathbf{u} + b\mathbf{v} + c(-\mathbf{u}) + d (-\mathbf{v}) \in S
⟹(a−c)u+(b−d)v∈S
\implies (a-c)\mathbf{u} + (b-d)\mathbf{v} \in S
all linear combinations of 2 vectors: a 2d plane
Puzzles: what is the subspace?
Linear equations
u=1111
\mathbf{u}=\begin{bmatrix}
1\\
1\\
1\\
1
\end{bmatrix}
v=1234
\mathbf{v}=\begin{bmatrix}
1\\
2\\
3\\
4
\end{bmatrix}
⟹au+bv+cw+dz∈S
\implies a\mathbf{u} + b\mathbf{v} + c\mathbf{w} + d \mathbf{z} \in S
∀a,b,c,d∈R
\forall a,b,c,d \in \mathbb{R}
w=−1−1−1−1
\mathbf{w}=\begin{bmatrix}
-1\\
-1\\
-1\\
-1
\end{bmatrix}
z=2345
\mathbf{z}=\begin{bmatrix}
2\\
3\\
4\\
5
\end{bmatrix}
u,v,w,z∈S
⟹au+bv+c(−u)+d(u+v)∈S
\implies a\mathbf{u} + b\mathbf{v} + c(-\mathbf{u}) + d (\mathbf{u}+\mathbf{v}) \in S
⟹(a−c+d)u+(b+d)v∈S
\implies (a-c+d)\mathbf{u} + (b+d)\mathbf{v} \in S
all linear combinations of 2 vectors: a 2d plane
Puzzles: what is the subspace?
Linear equations
u∈R5
\mathbf{u}\in \mathbb{R}^5
⟹au+bv+cw+dy+ez∈S
\implies a\mathbf{u} + b\mathbf{v} + c\mathbf{w} + d \mathbf{y} + e \mathbf{z} \in S
∀a,b,c,d,e∈R
\forall a,b,c,d,e \in \mathbb{R}
u,v,w,y,z∈S
⟹au+bv+c(au)+d(bu+cv)+e(dv)∈S
\implies a\mathbf{u} + b\mathbf{v} + c(a\mathbf{u}) + d (b\mathbf{u}+c\mathbf{v}) + e(d\mathbf{v}) \in S
⟹(a+ac+db)u+(b+dc+ed)v∈S
\implies (a+ac+db)\mathbf{u} + (b+dc+ed)\mathbf{v} \in S
all linear combinations of 2 vectors: a 2d plane
,v∈R5
,\mathbf{v}\in \mathbb{R}^5
,w=au,y=bu+cv,z=dv
,\mathbf{w}=a\mathbf{u}, \mathbf{y}=b\mathbf{u} + c\mathbf{v}, \mathbf{z}=d\mathbf{v}
(5 vectors)
w,y,z are dependent on u,v
Demo: Linear combination
Linear equations
Independent Vectors
Linear equations
We say that v1,v2,v3,…, vn∈Rn are independent if no combination of these vectors gives the 0 vector (except the 0 combination)
c1v1+c2v2+c3v3+⋯+cnvn=0 only if ci=0 ∀i
c_1\mathbf{v_1}+c_2\mathbf{v_2}+c_3\mathbf{v_3}+\dots+c_n\mathbf{v_n}=0~only~if~c_i=0~\forall i
The vector vk is dependent on v1,v2,v3,…,vn if it can be expressed as a linear combination of these vectors
Span
Linear equations
v1,v2,v3,…,vn∈V
\mathbf{v_1}, \mathbf{v_2}, \mathbf{v_3}, \dots, \mathbf{v_n} \in V
We say that v1,v2,v3,…, vn span V if every vector in V can be written as a linear combination of these vectors
Examples:
[10]
\begin{bmatrix}
1\\0
\end{bmatrix}
[01]
\begin{bmatrix}
0\\1
\end{bmatrix}
span R2
100
\begin{bmatrix}
1\\0\\0
\end{bmatrix}
010
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
span R3
001
\begin{bmatrix}
0\\0\\1
\end{bmatrix}
[23]
\begin{bmatrix}
2\\3
\end{bmatrix}
[12]
\begin{bmatrix}
1\\2
\end{bmatrix}
span R2
span R3
111
\begin{bmatrix}
1\\1\\1
\end{bmatrix}
123
\begin{bmatrix}
1\\2\\3
\end{bmatrix}
341
\begin{bmatrix}
3\\4\\1
\end{bmatrix}
Basis of a space
Linear equations
We say that v1,v2,v3,…, vn∈V form the basis of V if
100
\begin{bmatrix}
1\\0\\0
\end{bmatrix}
010
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
span R3
001
\begin{bmatrix}
0\\0\\1
\end{bmatrix}
111
\begin{bmatrix}
1\\1\\1
\end{bmatrix}
123
\begin{bmatrix}
1\\2\\3
\end{bmatrix}
341
\begin{bmatrix}
3\\4\\1
\end{bmatrix}
but we don't need all of them
the last 3 vectors are dependent on the first 3
these vectors are independent
these vectors span the space V
v1,v2,v3,…,vn∈V
\mathbf{v_1}, \mathbf{v_2}, \mathbf{v_3}, \dots, \mathbf{v_n} \in V
Basis of a space
Linear equations
Examples:
[10]
\begin{bmatrix}
1\\0
\end{bmatrix}
[01]
\begin{bmatrix}
0\\1
\end{bmatrix}
100
\begin{bmatrix}
1\\0\\0
\end{bmatrix}
010
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
001
\begin{bmatrix}
0\\0\\1
\end{bmatrix}
[23]
\begin{bmatrix}
2\\3
\end{bmatrix}
[12]
\begin{bmatrix}
1\\2
\end{bmatrix}
111
\begin{bmatrix}
1\\1\\1
\end{bmatrix}
123
\begin{bmatrix}
1\\2\\3
\end{bmatrix}
341
\begin{bmatrix}
3\\4\\1
\end{bmatrix}
Basis of R2
Basis of R3
111
\begin{bmatrix}
1\\1\\1
\end{bmatrix}
123
\begin{bmatrix}
1\\2\\3
\end{bmatrix}
341
\begin{bmatrix}
3\\4\\1
\end{bmatrix}
100
\begin{bmatrix}
1\\0\\0
\end{bmatrix}
Same space can have multiple basis
Every basis of a space will have the same number of vectors (dimension of the space)
A puzzle
Linear equations
u=111
\mathbf{u}=\begin{bmatrix}
1\\1\\1
\end{bmatrix}
v=123
\mathbf{v}=\begin{bmatrix}
1\\2\\3
\end{bmatrix}
What is the span?
Can you think of an alternative basis?
What is the dimension of the spanned subspace?
cu+dv ∀c,d∈R
c\mathbf{u} + d\mathbf{v}~~\forall c,d \in \mathbb{R}
2u,3v
2\mathbf{u}, 3\mathbf{v}
2
2
a 2 dimensional plane
How many vectors in the basis of Rn ?
n
n
Learning Objectives
(achieved)
What is a vector space and a subspace?
What is the basis of a space/subspace?
What is the span of a set of vectors?
What are independent vectors?
What is the dimension of a space/subspace?
CS6015: Linear Algebra and Random Processes Lecture 6: Vector spaces, subspaces, independence, span, basis, dimensions
CS6015: Lecture 6
By Mitesh Khapra
CS6015: Lecture 6
Lecture 6: Vector spaces, subspaces, independence, span, basis, dimensions
- 3,004
