Gram-Schmidt Orthonormalization

A procedure which takes a nonorthogonal set of Linearly Independent functions and constructs an Orthogonal Basis over an arbitrary interval with respect to an arbitrary Weighting Function $w(x)$. Given an original set of linearly independent functions $\{u_n\}$, let $\{\psi_n\}$ denote the orthogonalized (but not normalized) functions and $\{\phi_n\}$ the orthonormalized functions.

$$\psi_0(x) \equiv u_0(x)$$ (1)

$$\phi_0(x) \equiv { \psi_0(x)\over \sqrt{\int {\psi_0}^2(x)w(x)\,dx}}.$$ (2)

Take

$$\psi_1(x) = u_1(x)+a_{10}\phi_0(x),$$ (3)

where we require

$$\int \psi_1\phi_0w\,dx = \int u_1\phi_0w\,dx+a_{10}\int {\phi_0}^2w\,dx = 0.$$ (4)

By definition,

$$\int {\phi_0}^2w\,dx=1,$$ (5)

so

$$a_{10} = -\int u_1\phi_0w\,dx.$$ (6)

The first orthogonalized function is therefore

$$\psi_1 =u_1(x)-\left[{\int u_1\phi_0w\,dx}\right]\phi_0,$$ (7)

and the corresponding normalized function is

$$\phi_1 = { \psi_1(x)\over \sqrt{\int {\psi_1}^2w\,dx}}.$$ (8)

By mathematical induction, it follows that

$$\phi_i(x) = { \psi_i(x)\over \sqrt{\int {\psi_i}^2w\,dx}},$$ (9)

where

$$\psi_i(x) = u_i+a_{i0}\phi_0+a_{i1}\phi_1\ldots +a_{i,i-1}\phi_{i-1}$$ (10)

and

$$a_{ij} \equiv - \int u_i\phi_jw\,dx.$$ (11)

If the functions are normalized to $N_j$ instead of 1, then

$$\int^b_a [\phi_j(x)]^2w\,dx = {N_j}^2$$ (12)

$$\phi_i(x) = N_i { \psi_i(x)\over \sqrt{\int {\psi_i}^2w\,dx}}$$ (13)

$$a_{ij} = - { \int u_i\phi_jw\,dx\over {N_j}^2}.$$ (14)

Orthogonal Polynomials are especially easy to generate using Gram-Schmidt Orthonormalization. Use the notation

$$\left\langle{x_i\vert x_j}\right\rangle{}\equiv\left\langle{... ...w\vert x_j}\right\rangle{}\equiv\int_a^b x_i(x)x_j(x)w(x)\,dx,$$ (15)

where $w(x)$ is a Weighting Function, and define the first few Polynomials,

$$p_0(x) \equiv 1$$ (16)

$$p_1(x) = \left[{x-{\left\langle{xp_0\vert p_0}\right\rangle{}\over \left\langle{p_0\vert p_0}\right\rangle{}}}\right]p_0.$$ (17)

As defined, $p_0$ and $p_1$ are Orthogonal Polynomials, as can be seen from

$$\left\langle{p_0\vert p_1}\right\rangle{} = \left\langle{\left[{x-{\left\langle{xp_0\vert p_0}\right\rangle{}\over \left\langle{p_0\vert p_0}\right\rangle{}}}\right]p_0}\right\rangle{}$$

$$= \left\langle{xp_0}\right\rangle{}-{\left\langle{xp_0\vert p_0}\ri... ...over \left\langle{p_0\vert p_0}\right\rangle{}}\left\langle{p_0}\right\rangle{}$$

$$= \left\langle{xp_0}\right\rangle{}-\left\langle{xp_0}\right\rangle{} = 0.$$ (18)

Now use the Recurrence Relation

$$p_{i+1}(x)=\left[{x-{\left\langle{xp_i\vert p_i}\right\rangl... ...ft\langle{p_{i-1}\vert p_{i-1}}\right\rangle{}}}\right]p_{i-1}$$ (19)

to construct all higher order Polynomials.

To verify that this procedure does indeed produce Orthogonal Polynomials, examine

$$\left\langle{p_{i+1}\vert p_i}\right\rangle{} = \left\langle{\left[{x-{\left\langle{xp_i\vert p_i}\right\rangle{}... ...le{p_{i-1}\vert p_{i-1}}\right\rangle{}}p_{i-1} \biggl\vert p_i}\right\rangle{}$$

$$= \left\langle{xp_i\vert p_i}\right\rangle{} - {\left\langle{x p_i\... ...i-1}\vert p_{i-1}}\right\rangle{}}\left\langle{p_{i-1}\vert p_i}\right\rangle{}$$

$$= -{\left\langle{p_i\vert p_i}\right\rangle{}\over\left\langle{p_{i-1}\vert p_{i-1}}\right\rangle{}}\left\langle{p_{i-1}\vert p_i}\right\rangle{}$$

$$= -{\left\langle{p_i\vert p_i}\right\rangle{}\over\left\langle{p_{i... ...{j-2}}\right\rangle{}}\left\langle{p_{j-2}\vert p_{j-1}}\right\rangle{}}\right]$$

$$= \ldots = (-1)^j{\left\langle{p_j\vert p_j}\right\rangle{}\over \l... ...gle{p_0\vert p_0}\right\rangle{}}\left\langle{p_0\vert p_1}\right\rangle{} = 0,$$ (20)

since $\left\langle{p_0\vert p_1}\right\rangle{} = 0$. Therefore, all the Polynomials $p_i(x)$ are orthogonal. Many common Orthogonal Polynomials of mathematical physics can be generated in this manner. However, the process is numerically unstable (Golub and van Loan 1989).

See also Gram Determinant, Gram's Inequality, Orthogonal Polynomials

References

Arfken, G. ``Gram-Schmidt Orthogonalization.'' §9.3 in Mathematical Methods for Physicists, 3rd ed. Orlando, FL: Academic Press, pp. 516-520, 1985.

Golub, G. H. and van Loan, C. F. Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins, 1989.