Vector Norms

The norm of a vector is a measure of its length, in geometrical terms. The typical length is the L₂ norm, also called the Euclidean norm or Euclidean length (or sometimes Euclidean distance, which is just the length of a vector obtained by subtracting two other vectors), and is a multidimensional generalization of the Pythagorean theorem:

\[\|\mathbf{x}\|_2 = \sqrt{\sum_i x_i^2}\]

If you see a reference to the “norm” of a vector without further qualification, or the length operator \(\|\mathbf{x}\|\), it is usually referring to the L₂ norm. In my writing and teaching, I try to explicitly name the norm to avoid confusion.

We will often seen the square of the L₂ norm, denoted \(\|\mathbf{x}\|_2^2\): this is just the sum of the squares of the vector’s elements, as the square has the effect of removing the square root. Further, \(\|\mathbf{x}\|_2^2 = \mathbf{x} \cdot \mathbf{x}\).

A unit vector is a vector whose L₂ norm is 1 (\(\|\mathbf{x}\|_2 = 1\)). We can normalize a vector to be a unit vector by dividing it by the norm: \(\hat{\mathbf{x}} = \frac{1}{\|\mathbf{x}\|_2} \mathbf{x}\).

Another common norm is the L₁ norm, also called the Manhattan distance. This is denoted \(\|\mathbf{x}\|_1\), and is the sum of the absolute values: \(\|\mathbf{x}\|_1 = \sum_i |x_i|\).

You may sometimes see reference to the \(L_\infty\) norm, which is the maximum value of the vector.