Eigenvectors and Eigenvalues Explained

When you multiply a vector by a matrix, the vector usually gets rotated and stretched into a new direction. However, for every square matrix, there are special 'magic' vectors that are not rotated—they only get stretched. These are called Eigenvectors, and their stretch factor is the Eigenvalue.

The Characteristic Equation

For a square matrix $A$, a non-zero vector $\mathbf{v}$ is an eigenvector if applying the matrix only scales it by a constant $\lambda$ (the eigenvalue).

The Math

$$A\mathbf{v} = \lambda \mathbf{v}$$ This equation is the foundation for understanding the 'internal structure' or 'DNA' of a matrix.

The Principal Directions of AI

Eigenvectors represent the axes along which a transformation acts most strongly. This is the secret behind Principal Component Analysis (PCA).

Dimensionality Reduction

In a cloud of data (like a 1000D dataset of customer behaviors), the eigenvectors with the largest eigenvalues point in the directions where the data varies the most. By keeping only these 'Principal Components', we can compress a 1000D dataset into 2D or 3D while keeping 99% of the important information.

Google's PageRank

The original algorithm behind Google Search was essentially an eigenvector problem. The 'importance' of a webpage was the dominant eigenvector of a massive matrix representing the web's link structure.