Eigenvalues and Eigenvectors, Intuitively

When a matrix $A$ acts on a vector $\mathbf{v}$, the result $A\mathbf{v}$ is generally a different vector — different direction, different length. But for some special vectors, $A\mathbf{v}$ points in the same direction as $\mathbf{v}$, only stretched or compressed by a scalar factor. Those vectors are eigenvectors, and the scalar factor is the eigenvalue.

This sounds technical, and the definition is. But the intuition is simple: an eigenvector is a direction that the matrix preserves. When the matrix acts, the eigenvector stays on its own line — it just gets longer or shorter (or possibly flipped to the other side).

This article covers what eigenvalues and eigenvectors mean intuitively, the formal definition, how to compute them, diagonalization (the magic), and the four major applications: principal component analysis, Markov chains, dynamical systems, and Google PageRank.

The intuition: directions that don't change

Consider a 2×2 matrix and its action on the plane. A typical matrix rotates and scales most vectors — a vector $\mathbf{v}$ becomes $A\mathbf{v}$, pointing in a generally different direction.

For most matrices, there are exactly two special directions (in 2D) along which the matrix only stretches or compresses, without rotating. These are the eigenvector directions. A vector pointing along an eigenvector direction stays along that direction after the matrix acts.

Example 1: Consider the matrix $A=\left(\begin{array}{cc}3& 0\\ 0& 2\end{array}\right)$.

The vector ${\mathbf{v}}_1=(1,0)$ satisfies $A{\mathbf{v}}_1=(3,0)=3{\mathbf{v}}_1$. So ${\mathbf{v}}_1$ is an eigenvector with eigenvalue 3.

The vector ${\mathbf{v}}_2=(0,1)$ satisfies $A{\mathbf{v}}_2=(0,2)=2{\mathbf{v}}_2$. So ${\mathbf{v}}_2$ is an eigenvector with eigenvalue 2.

The matrix $A$ stretches the x-axis by 3 and the y-axis by 2. Those two axes are the eigenvector directions.

For general matrices (not diagonal like this example), the eigenvectors are not aligned with the standard axes — they could point in any direction. But the same idea holds: there are special directions that the matrix only stretches.

Formal definition

An eigenvector of $A$ is a nonzero vector $\mathbf{v}$ such that

$A\mathbf{v}=\lambda \mathbf{v}$

for some scalar $\lambda$. The scalar $\lambda$ is the eigenvalue corresponding to $\mathbf{v}$.

Two technicalities:

1. The zero vector is excluded — it trivially satisfies $A0=\lambda 0$ for any $\lambda$, so calling it an eigenvector would be uninteresting.
2. The eigenvalue $\lambda$ can be zero. An eigenvector with eigenvalue zero is a vector that $A$ maps to zero.

How to compute eigenvalues

Rewrite the equation $A\mathbf{v}=\lambda \mathbf{v}$ as $A\mathbf{v}-\lambda \mathbf{v}=0$, or $(A-\lambda I)\mathbf{v}=0$.

For this equation to have a nonzero solution $\mathbf{v}$, the matrix $A-\lambda I$ must be singular — must have determinant zero.

Characteristic equation:

$\det (A-\lambda I)=0$

This is a polynomial equation in $\lambda$. Solve it to find the eigenvalues.

Example 2: Find the eigenvalues of $A=\left(\begin{array}{cc}4& 1\\ 2& 3\end{array}\right)$.

$A-\lambda I=\left(\begin{array}{cc}4-\lambda & 1\\ 2& 3-\lambda \end{array}\right)$

$\det (A-\lambda I)=(4-\lambda )(3-\lambda )-2={\lambda }^2-7\lambda +10=0$

Solve: $\lambda =\frac{7\pm \sqrt{49-40}}{2}=\frac{7\pm 3}{2}$, so $\lambda =5$ or $\lambda =2$.

The eigenvalues are 5 and 2.

How to compute eigenvectors

For each eigenvalue $\lambda$, solve $(A-\lambda I)\mathbf{v}=0$ for $\mathbf{v}$.

Example 3 (continued from Example 2): Find the eigenvectors of $A=\left(\begin{array}{cc}4& 1\\ 2& 3\end{array}\right)$.

For $\lambda =5$:

$A-5I=\left(\begin{array}{cc}-1& 1\\ 2& -2\end{array}\right)$

Solve $\left(\begin{array}{cc}-1& 1\\ 2& -2\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)$.

Both rows give $x=y$. So the eigenvector is $(1,1)$ (or any scalar multiple).

For $\lambda =2$:

$A-2I=\left(\begin{array}{cc}2& 1\\ 2& 1\end{array}\right)$

Both rows give $2x+y=0$, so $y=-2x$. The eigenvector is $(1,-2)$.

So the eigenvectors are ${\mathbf{v}}_1=(1,1)$ (eigenvalue 5) and ${\mathbf{v}}_2=(1,-2)$ (eigenvalue 2). You can verify: $A{\mathbf{v}}_1=(5,5)=5{\mathbf{v}}_1$ and $A{\mathbf{v}}_2=(2,-4)=2{\mathbf{v}}_2$. ✓

Diagonalization

A matrix is diagonalizable if it has a complete set of eigenvectors — enough to form a basis.

For a diagonalizable matrix $A$:

$A=PD{P}^{-1}$

where $D$ is a diagonal matrix with the eigenvalues on the diagonal, and $P$ is a matrix whose columns are the corresponding eigenvectors.

Why this is useful. Diagonal matrices are trivial to work with. Diagonalization expresses the action of a general matrix in a basis where the action is just scaling along each direction.

In particular, $A^n=P{D}^n{P}^{-1}$, and computing $D^n$ is easy — just raise each diagonal entry to the $n$th power.

Example 4 (continued): For $A=\left(\begin{array}{cc}4& 1\\ 2& 3\end{array}\right)$:

$P=\left(\begin{array}{cc}1& 1\\ 1& -2\end{array}\right),D=\left(\begin{array}{cc}5& 0\\ 0& 2\end{array}\right),P^{-1}=\frac{1}{-3}\left(\begin{array}{cc}-2& -1\\ -1& 1\end{array}\right)=\left(\begin{array}{cc}2/3& 1/3\\ 1/3& -1/3\end{array}\right)$

You can verify $A=PD{P}^{-1}$. And to compute $A^{10}$, you compute $D^{10}=\left(\begin{array}{cc}{5}^{10}& 0\\ 0& 2^{10}\end{array}\right)$, then $A^{10}=P{D}^{10}{P}^{-1}$.

Not every matrix is diagonalizable. Matrices with fewer eigenvectors than dimensions (the eigenvalues have less than full geometric multiplicity) cannot be diagonalized — they require a more general decomposition called the Jordan canonical form.

Application 1: Principal component analysis (PCA)

PCA is the most-used technique in data analysis and machine learning for dimensionality reduction.

Given a dataset, compute the covariance matrix — a symmetric matrix whose entry $(i,j)$ is the covariance between feature $i$ and feature $j$. The eigenvectors of this matrix are the principal components — the directions of maximum variance in the data. The corresponding eigenvalues measure how much variance each direction captures.

By projecting the data onto the top few principal components (those with the largest eigenvalues), you reduce dimension while preserving most of the structure. This is how PCA reduces a 1000-feature dataset to 50 features while keeping the meaningful patterns.

Application 2: Markov chains

A Markov chain is a stochastic process that transitions between states with probabilities encoded in a transition matrix $P$. The long-term behavior of the chain is governed by the eigenvalues and eigenvectors of $P$.

The largest eigenvalue is always 1 (for a probability-preserving transition matrix). The corresponding eigenvector is the stationary distribution — the long-run proportions of time spent in each state.

If you simulate a Markov chain for many steps, the distribution converges to the stationary eigenvector. The other eigenvalues control how fast this convergence happens.

Application 3: Dynamical systems

A linear dynamical system has the form ${\mathbf{x}}_{t+1}=A{\mathbf{x}}_t$. Iterating: ${\mathbf{x}}_n=A^n{\mathbf{x}}_0$.

The long-term behavior of this system depends on the eigenvalues of $A$:

• If all eigenvalues have magnitude less than 1, ${\mathbf{x}}_n\to 0$. The system is stable.
• If any eigenvalue has magnitude greater than 1, ${\mathbf{x}}_n$ grows unboundedly along that eigenvector direction. The system is unstable.
• If eigenvalues are exactly 1, the system has a steady state.

Continuous-time dynamical systems $\dot{\mathbf{x}}=A\mathbf{x}$ have similar analysis using the real parts of eigenvalues.

Application 4: Google PageRank

The original PageRank algorithm, which made Google possible, is an eigenvector computation.

Model the web as a graph where pages are nodes and links are directed edges. Define a transition matrix where $P_{ij}$ is the probability of moving from page $j$ to page $i$ (uniform across $j$'s outgoing links). The PageRank vector — the importance score for each page — is the dominant eigenvector of $P$ (with eigenvalue 1).

In essence: a page is important if other important pages link to it. This recursive definition resolves to an eigenvector equation, which is solvable.

The original Google paper (Brin and Page, 1998) is essentially a paper about computing the dominant eigenvector of a very large sparse matrix.