Orthonormal basis

In mathematics, particularly linear algebra, an orthonormal basis for an inner product space V with finite dimension is a basis for $V$ whose vectors are orthonormal, that is, they are all unit vectors and orthogonal to each other.^[1]^[2]^[3] For example, the standard basis for a Euclidean space $\mathbb {R} ^{n}$ is an orthonormal basis, where the relevant inner product is the dot product of vectors. The image of the standard basis under a rotation or reflection (or any orthogonal transformation) is also orthonormal, and every orthonormal basis for $\mathbb {R} ^{n}$ arises in this fashion.

For a general inner product space $V,$ an orthonormal basis can be used to define normalized orthogonal coordinates on $V.$ Under these coordinates, the inner product becomes a dot product of vectors. Thus the presence of an orthonormal basis reduces the study of a finite-dimensional inner product space to the study of $\mathbb {R} ^{n}$ under dot product. Every finite-dimensional inner product space has an orthonormal basis, which may be obtained from an arbitrary basis using the Gram–Schmidt process.

In functional analysis, the concept of an orthonormal basis can be generalized to arbitrary (infinite-dimensional) inner product spaces.^[4] Given a pre-Hilbert space $H,$ an orthonormal basis for $H$ is an orthonormal set of vectors with the property that every vector in $H$ can be written as an infinite linear combination of the vectors in the basis. In this case, the orthonormal basis is sometimes called a Hilbert basis for $H.$ Note that an orthonormal basis in this sense is not generally a Hamel basis, since infinite linear combinations are required. Specifically, the linear span of the basis must be dense in $H,$ but it may not be the entire space.

If we go on to Hilbert spaces, a non-orthonormal set of vectors having the same linear span as an orthonormal basis may not be a basis at all. For instance, any square-integrable function on the interval $[-1,1]$ can be expressed (almost everywhere) as an infinite sum of Legendre polynomials (an orthonormal basis), but not necessarily as an infinite sum of the monomials $x^{n}.$

Examples

The set of vectors $\left\{e_{1}=(1,0,0),e_{2}=(0,1,0),e_{3}=(0,0,1)\right\},$ which is called the standard basis, forms an orthonormal basis of $\mathbb {R} ^{3}$ .
Proof: A straightforward computation shows that the inner products of these vectors equals zero, $\left\langle e_{1},e_{2}\right\rangle =\left\langle e_{1},e_{3}\right\rangle =\left\langle e_{2},e_{3}\right\rangle =0$ and that each of their magnitudes equals one, $\left\|e_{1}\right\|=\left\|e_{2}\right\|=\left\|e_{3}\right\|=1.$ This means that $\left\{e_{1}=,e_{2},e_{3}\right\}$ is an orthonormal set. All vectors $(x,y,z)\in \mathbb {R} ^{3}$ can be expressed as a sum of the basis vectors scaled $(x,y,z)=x\mathbf {e} _{1}+y\mathbf {e} _{2}+z\mathbf {e} _{3},$ so $\left\{e_{1}=,e_{2},e_{3}\right\}$ spans $\mathbb {R} ^{3}$ and hence must be a basis. It may also be shown that the standard basis rotated about an axis through the origin or reflected in a plane through the origin forms an orthonormal basis of $\mathbb {R} ^{3}$ .
Notice that an orthogonal transformation of the standard inner-product space $(\mathbb {R} ^{n},\langle \cdot ,\cdot \rangle )$ can be used to construct other orthogonal bases of $\mathbb {R} ^{n}$ .
The set $\left\{f_{n}:n\in \mathbb {Z} \right\}$ with $f_{n}(x)=\exp(2\pi inx),$ where $\exp$ denotes the exponential function, forms an orthonormal basis of the space of functions with finite Lebesgue integrals, $L^{2}([0,1]),$ with respect to the 2-norm. This is fundamental to the study of Fourier series.
The set $\left\{e_{b}:b\in B\right\}$ with $e_{b}(c)=1$ if $b=c$ and $e_{b}(c)=0$ otherwise forms an orthonormal basis of $\ell ^{2}(B).$
Eigenfunctions of a Sturm–Liouville eigenproblem.
An orthogonal matrix is a matrix whose column vectors form an orthonormal set.

Basic formula

If $B$ is an orthogonal basis of $H,$ then every element $x\in H$ may be written as $x=\sum _{b\in B}{\frac {\langle b,x\rangle }{\lVert b\rVert ^{2}}}b.$

When $B$ is orthonormal, this simplifies to $x=\sum _{b\in B}\langle b,x\rangle b$ and the square of the norm of $x$ can be given by $\|x\|^{2}=\sum _{b\in B}|\langle x,b\rangle |^{2}.$

Even if $B$ is uncountable, only countably many terms in this sum will be non-zero, and the expression is therefore well-defined. This sum is also called the Fourier expansion of $x,$ and the formula is usually known as Parseval's identity.

If $B$ is an orthonormal basis of $H,$ then $H$ is isomorphic to $\ell ^{2}(B)$ in the following sense: there exists a bijective linear map $\Phi :H\to \ell ^{2}(B)$ such that $\langle \Phi (x),\Phi (y)\rangle =\langle x,y\rangle \quad {\text{ for all }}x,y\in H.$

Incomplete orthogonal sets

Given a Hilbert space $H$ and a set $S$ of mutually orthogonal vectors in $H,$ we can take the smallest closed linear subspace $V$ of $H$ containing $S.$ Then $S$ will be an orthogonal basis of $V;$ which may of course be smaller than $H$ itself, being an incomplete orthogonal set, or be $H,$ when it is a complete orthogonal set.

Existence

Using Zorn's lemma and the Gram–Schmidt process (or more simply well-ordering and transfinite recursion), one can show that every Hilbert space admits an orthonormal basis;^[5] furthermore, any two orthonormal bases of the same space have the same cardinality (this can be proven in a manner akin to that of the proof of the usual dimension theorem for vector spaces, with separate cases depending on whether the larger basis candidate is countable or not). A Hilbert space is separable if and only if it admits a countable orthonormal basis. (One can prove this last statement without using the axiom of choice).

As a homogeneous space

The set of orthonormal bases for a space is a principal homogeneous space for the orthogonal group $O(n),$ and is called the Stiefel manifold $V_{n}(\mathbb {R} ^{n})$ of orthonormal $n$ -frames.^[6]

In other words, the space of orthonormal bases is like the orthogonal group, but without a choice of base point: given an orthogonal space, there is no natural choice of orthonormal basis, but once one is given one, there is a one-to-one correspondence between bases and the orthogonal group. Concretely, a linear map is determined by where it sends a given basis: just as an invertible map can take any basis to any other basis, an orthogonal map can take any orthogonal basis to any other orthogonal basis.

The other Stiefel manifolds $V_{k}(\mathbb {R} ^{n})$ for $k<n$ of incomplete orthonormal bases (orthonormal $k$ -frames) are still homogeneous spaces for the orthogonal group, but not principal homogeneous spaces: any $k$ -frame can be taken to any other $k$ -frame by an orthogonal map, but this map is not uniquely determined.

References

^ Lay, David C. (2006). Linear Algebra and Its Applications (3rd ed.). Addison–Wesley. ISBN 0-321-28713-4.
^ Strang, Gilbert (2006). Linear Algebra and Its Applications (4th ed.). Brooks Cole. ISBN 0-03-010567-6.
^ Axler, Sheldon (2002). Linear Algebra Done Right (2nd ed.). Springer. ISBN 0-387-98258-2.
^ Rudin, Walter (1987). Real & Complex Analysis. McGraw-Hill. ISBN 0-07-054234-1.
^ Linear Functional Analysis Authors: Rynne, Bryan, Youngson, M.A. page 79
^ "CU Faculty". engfac.cooper.edu. Retrieved 2021-04-15.

Rudin, Walter (1991). Functional Analysis. International Series in Pure and Applied Mathematics. Vol. 8 (Second ed.). New York, NY: McGraw-Hill Science/Engineering/Math. ISBN 978-0-07-054236-5. OCLC 21163277.

External links

This Stack Exchange Post discusses why the set of Dirac Delta functions is not a basis of L²([0,1]).

[1] Lay, David C. (2006). Linear Algebra and Its Applications (3rd ed.). Addison–Wesley. ISBN 0-321-28713-4.

[2] Strang, Gilbert (2006). Linear Algebra and Its Applications (4th ed.). Brooks Cole. ISBN 0-03-010567-6.

[3] Axler, Sheldon (2002). Linear Algebra Done Right (2nd ed.). Springer. ISBN 0-387-98258-2.

[4] Rudin, Walter (1987). Real & Complex Analysis. McGraw-Hill. ISBN 0-07-054234-1.

[5] Linear Functional Analysis Authors: Rynne, Bryan, Youngson, M.A. page 79

[6] "CU Faculty". engfac.cooper.edu. Retrieved 2021-04-15.

[1]

[2]

[3]

[4]

[5]

[6]

v t e Linear algebra
Outline Glossary
Basic concepts	Scalar Vector Vector space Scalar multiplication Vector projection Linear span Linear map Linear projection Linear independence Linear combination Multilinear map Basis Change of basis Row and column vectors Row and column spaces Kernel Eigenvalues and eigenvectors Transpose Linear equations
Matrices	Block Decomposition Invertible Minor Multiplication Rank Transformation Cramer's rule Gaussian elimination Productive matrix
Bilinear	Orthogonality Dot product Hadamard product Inner product space Outer product Kronecker product Gram–Schmidt process
Multilinear algebra	Determinant Cross product Triple product Seven-dimensional cross product Geometric algebra Exterior algebra Bivector Multivector Tensor Outermorphism
Vector space constructions	Dual Direct sum Function space Quotient Subspace Tensor product
Numerical	Floating-point Numerical stability Basic Linear Algebra Subprograms Sparse matrix Comparison of linear algebra libraries
Category

v t e Hilbert spaces
Basic concepts	Adjoint Inner product and L-semi-inner product Hilbert space and Prehilbert space Orthogonal complement Orthonormal basis
Main results	Bessel's inequality Cauchy–Schwarz inequality Riesz representation
Other results	Hilbert projection theorem Parseval's identity Polarization identity (Parallelogram law)
Maps	Compact operator on Hilbert space Densely defined Hermitian form Hilbert–Schmidt Normal Self-adjoint Sesquilinear form Trace class Unitary
Examples	Cⁿ(K) with K compact & n<∞ Segal–Bargmann F