2017年9月6日星期三

When X'X is invertible

X'X is always positive semidefinite, because for any nonzero a, a'X'Xa = (Xa)'(Xa) = ||Xa||2 >= 0. Moreover, Xa = 0 (and hence ||Xa||2 = 0) if and only if the columns of X are linearly dependent, so if X has full column rank then X'X is positive definite.
Every positive definite matrix is invertible, because if Ax=0 for x =/= 0 then x'Ax = dot(x, 0) = 0 which means A is not positive definite.
Therefore, if X has full column rank then X'X is invertible