matrix rank

 

many real world matrix has low-rank,

rank of a matrix is the independent vectors (columns), 

https://en.wikipedia.org/wiki/Rank_(linear_algebra)

Matrix Eigen value

spectral theorem

https://youtu.be/KCANLl8z6PI

visual explanation
https://youtu.be/PFDu9oVAE-g

linear dependent row in matrix

https://www.mathworks.com/matlabcentral/answers/108835-how-to-get-only-linearly-independent-rows-in-a-matrix-or-to-remove-linear-dependency-b-w-rows-in-a-m

Yuan method on adjacency matrix controllability

Yuan clearly used weighted matrix, and j->i as direction. So, column to row indicate direction? 

Wikipedia, “In directed graphs, the in-degree of a vertex can be computed by summing the entries of the corresponding column, and the out-degree can be computed by summing the entries of the corresponding row.” The question now is does i->j and j->i matters? It can be tested using a star shaped network with outward and inwarding arrows. A quick exam on these show both star networks should have the same number of minimal control nodes. Well, eigen values of a matrix and its transpose are the same, see  https://yutsumura.com/eigenvalues-of-a-matrix-and-its-transpose-are-the-same/.  

So, i->j and j->i does not matter! This is somewhat shocking to me. 

multiplicity of eigen value

The number of linearly independent eigenvectors associated with a given eigen value $lambda$ is called the geometric multiplicity of eigenvalue $lambda$.

matrix Jordan cannonical form and eigen values

http://mathworld.wolfram.com/JordanCanonicalForm.html

Eigen values in Jordan cannobucak form of matrix

Singular matrix, invertible matrix

https://en.wikipedia.org/wiki/Invertible_matrix

inner product of vectors, dot products, orthogonality

$u \dot v = u^T v$

$u \dot v = v \dot u $

length (or norm) of vector $v$ is the square root of its inner product. This can be seen from the v [a,b], whose length(norm) is sqrt(a^2 + b^2)

u \dot v = ||u|| ||v|| cos \theta
||u-v|| = ||u||^2 + ||v||^2 - 2||u|| ||v|| cos\theta

Two vector $u$ and $v$ are orthogonal if and only if $u \dot v = 0$. 

U has orthonormal columns if and only if U^T U = I

orthogonal projection of vector y to u:
y^hat = \frac{y \dot u}{u \dot u } u

Orthogonal projection of a point y to W space with {u1, u2, ... up} basis can be found by orthogonal projections on each base vector, u1, u2, ..., u_p.

PCA notes

From covariance matrix, the eigen vector is the PCA.

http://youtu.be/5zk93CpKYhg