<?xml version="1.0" encoding="UTF-8"?>

<record version="3" id="1519">
 <title>Inner Products of Functions</title>
 <name>InnerProductsOfFunctions</name>
 <created>2026-10-22 03:36:47</created>
 <modified>2026-10-27 03:29:40</modified>
 <type>Topic</type>
 <creator id="1" name="bloftin"/>
 <modifier id="1" name="bloftin"/>
 <comment>added more detail to section

21 Projection onto several orthogonal functions</comment>
 <author id="1" name="bloftin"/>
 <classification>
	<category scheme="pacs" code="02.30.Nw"/>
	<category scheme="pacs" code="02.30.Lt"/>
	<category scheme="pacs" code="02.10.Ud"/>
 </classification>
 <defines>
	<concept>function inner product</concept>
	<concept>orthogonal functions</concept>
	<concept>function norm</concept>
	<concept>normalized function</concept>
	<concept>orthonormal functions</concept>
	<concept>function projection</concept>
	<concept>projection coefficient</concept>
	<concept>weighted inner product</concept>
	<concept>residual orthogonality</concept>
 </defines>
 <synonyms>
	<synonym concept="Inner Products of Functions" alias="FS02"/>
 </synonyms>
 <keywords>
	<term>Fourier series</term>
	<term>function inner product</term>
	<term>dot product</term>
	<term>orthogonality</term>
	<term>norm</term>
	<term>normalization</term>
	<term>projection</term>
	<term>weighted inner product</term>
	<term>function space</term>
	<term>basis functions</term>
	<term>residual</term>
 </keywords>
 <preamble>% this is the default PhysicsLibrary preamble.  as your knowledge
% of TeX increases, you will probably want to edit this, but
% it should be fine as is for beginners.

% almost certainly you want these
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsfonts}

% used for TeXing text within eps files
%\usepackage{psfrag}
% need this for including graphics (\includegraphics).
% If you use \includegraphics{figure} with no file extension, PhysicsLibrary
% will prefer figure.png before falling back to figure.eps.
\usepackage{graphicx}
\DeclareGraphicsExtensions{.png,.jpg,.jpeg,.pdf,.eps}
% for neatly defining theorems and propositions
%\usepackage{amsthm}
% making logically defined graphics
%\usepackage{xypic}

% there are many more packages, add them here as you need them

% define commands here</preamble>
 <content>\section*{FS02: Inner Products of Functions}

FS01 showed that a complicated periodic waveform can be assembled from harmonics.
The next question is more precise:

\begin{itemize}
\item How can we measure how much of one candidate harmonic is contained in a function?
\end{itemize}

The answer begins with ordinary vector geometry.

For a vector, the dot product measures alignment and extracts components.
Fourier analysis uses the same geometric idea, but the ``vectors'' are functions and the dot product becomes an integral.

The central definition of this lecture will be

\begin{equation}
\langle f,g\rangle
=
\int_a^b f(x)g(x)\,dx.
\label{eq:function_inner_product}
\end{equation}

This lecture develops Equation~\eqref{eq:function_inner_product} slowly rather than treating it as a mysterious rule.
We will build it from:

\begin{itemize}
\item ordinary vector dot products,
\item orthogonality,
\item vector projection,
\item the passage from finite sums to integrals,
\item function norms and normalization,
\item projection of one function onto another,
\item residual orthogonality,
\item weighted inner products.
\end{itemize}

The trigonometric orthogonality identities themselves are postponed to FS03.
By the end of FS02, however, the reason Fourier coefficients are projection coefficients should already be clear.

\section{Review: the dot product of ordinary vectors}

Consider two vectors in three-dimensional Euclidean space:

\begin{equation}
\mathbf a
=
\begin{bmatrix}
a_1\\
a_2\\
a_3
\end{bmatrix},
\qquad
\mathbf b
=
\begin{bmatrix}
b_1\\
b_2\\
b_3
\end{bmatrix}.
\end{equation}

Their dot product is

\begin{equation}
\mathbf a\cdot\mathbf b
=
a_1b_1+a_2b_2+a_3b_3.
\label{eq:vector_dot}
\end{equation}

This operation takes two vectors and returns one scalar.

For example, let

\begin{equation}
\mathbf a
=
\begin{bmatrix}
2\\
-1\\
3
\end{bmatrix},
\qquad
\mathbf b
=
\begin{bmatrix}
4\\
5\\
-2
\end{bmatrix}.
\end{equation}

Then

\begin{align}
\mathbf a\cdot\mathbf b
&amp;=
(2)(4)+(-1)(5)+(3)(-2)
\\
&amp;=8-5-6
\\
&amp;=-3.
\end{align}

The dot product contains geometric information.
It can be written as

\begin{equation}
\mathbf a\cdot\mathbf b
=
\|\mathbf a\|\,\|\mathbf b\|\cos\theta,
\label{eq:dot_angle}
\end{equation}

where $\theta$ is the angle between the vectors.

Thus:

\begin{itemize}
\item a positive dot product indicates an acute angle,
\item a negative dot product indicates an obtuse angle,
\item a zero dot product indicates a right angle.
\end{itemize}

\section{Orthogonal vectors}

Two nonzero vectors are orthogonal when

\begin{equation}
\mathbf a\cdot\mathbf b=0.
\label{eq:vector_orthogonal}
\end{equation}

From Equation~\eqref{eq:dot_angle},

\begin{equation}
0
=
\|\mathbf a\|\,\|\mathbf b\|\cos\theta.
\end{equation}

If both vectors are nonzero, their norms are nonzero, so

\begin{equation}
\cos\theta=0.
\end{equation}

Therefore

\begin{equation}
\theta=90^\circ.
\end{equation}

Orthogonality is therefore the general algebraic form of perpendicularity.

\section{Projection of one vector onto another}

Suppose we want the part of $\mathbf v$ that points along a nonzero vector $\mathbf e$.

Any vector parallel to $\mathbf e$ has the form

\begin{equation}
c\mathbf e,
\end{equation}

for some scalar $c$.

We therefore write

\begin{equation}
\mathbf v
=
c\mathbf e+\mathbf r,
\label{eq:vector_decomposition}
\end{equation}

where $\mathbf r$ is the leftover or residual vector.

For $c\mathbf e$ to be the projection, the residual should contain no component along $\mathbf e$.
Therefore require

\begin{equation}
\mathbf r\cdot\mathbf e=0.
\label{eq:vector_residual_orthogonal}
\end{equation}

Substitute

\begin{equation}
\mathbf r
=
\mathbf v-c\mathbf e.
\end{equation}

Then

\begin{equation}
(\mathbf v-c\mathbf e)\cdot\mathbf e=0.
\end{equation}

Distribute the dot product:

\begin{equation}
\mathbf v\cdot\mathbf e
-c(\mathbf e\cdot\mathbf e)
=0.
\end{equation}

Move the second term to the other side:

\begin{equation}
\mathbf v\cdot\mathbf e
=
c(\mathbf e\cdot\mathbf e).
\end{equation}

Divide by $\mathbf e\cdot\mathbf e$:

\begin{equation}
c
=
\frac{\mathbf v\cdot\mathbf e}
{\mathbf e\cdot\mathbf e}.
\label{eq:vector_projection_coefficient}
\end{equation}

Thus the projection itself is

\begin{equation}
\operatorname{proj}_{\mathbf e}\mathbf v
=
\frac{\mathbf v\cdot\mathbf e}
{\mathbf e\cdot\mathbf e}\mathbf e.
\end{equation}

\begin{center}
\includegraphics[width=0.86\textwidth,keepaspectratio]{FS02_fig01_vector_projection.png}

{\small Figure 1. Vector projection. The coefficient multiplying $\mathbf e$ is chosen so that the residual $\mathbf r=\mathbf v-c\mathbf e$ is perpendicular to $\mathbf e$. This same residual-orthogonality idea will be reused for functions.}
\end{center}

\section{Special case: projection onto a unit vector}

If $\mathbf e$ is a unit vector, then

\begin{equation}
\mathbf e\cdot\mathbf e=1.
\end{equation}

Equation~\eqref{eq:vector_projection_coefficient} becomes

\begin{equation}
c
=
\mathbf v\cdot\mathbf e.
\end{equation}

This is why components in an orthonormal Cartesian basis are obtained by dot products.

For example,

\begin{equation}
v_x
=
\mathbf v\cdot\mathbf e_x.
\end{equation}

Fourier analysis will eventually do the same thing with normalized basis functions.

\section{From vectors to lists of sampled function values}

Suppose we sample two functions $f$ and $g$ at equally spaced points

\begin{equation}
x_1,x_2,\ldots,x_N.
\end{equation}

The function values form lists:

\begin{equation}
\mathbf f
=
\begin{bmatrix}
f(x_1)\\
f(x_2)\\
\vdots\\
f(x_N)
\end{bmatrix},
\qquad
\mathbf g
=
\begin{bmatrix}
g(x_1)\\
g(x_2)\\
\vdots\\
g(x_N)
\end{bmatrix}.
\end{equation}

An ordinary discrete dot product is

\begin{equation}
\mathbf f\cdot\mathbf g
=
\sum_{k=1}^{N}f(x_k)g(x_k).
\end{equation}

If the points have spacing $\Delta x$, multiply by $\Delta x$:

\begin{equation}
\sum_{k=1}^{N}
f(x_k)g(x_k)\Delta x.
\label{eq:riemann_dot}
\end{equation}

Equation~\eqref{eq:riemann_dot} is a Riemann sum.
As the sample spacing becomes smaller and the number of samples increases, the sum approaches an integral:

\begin{equation}
\sum_{k=1}^{N}
f(x_k)g(x_k)\Delta x
\longrightarrow
\int_a^b f(x)g(x)\,dx.
\end{equation}

This provides a natural bridge from the finite-dimensional vector dot product to a continuous function inner product.

\begin{center}
\includegraphics[width=0.9\textwidth,keepaspectratio]{FS02_fig02_samples_to_integral.png}

{\small Figure 2. A function inner product can be motivated by sampling two functions, multiplying corresponding samples, summing them, and then taking the continuous limit of the weighted sum.}
\end{center}

\section{Definition of the inner product of real functions}

For real-valued functions on the interval $[a,b]$, define

\begin{equation}
\langle f,g\rangle
=
\int_a^b f(x)g(x)\,dx.
\end{equation}

The angle brackets distinguish the function inner product from ordinary multiplication.

It is important to read the notation correctly:

\begin{equation}
\langle f,g\rangle
\end{equation}

is one number obtained from two entire functions.

The integration interval is part of the definition.
The same pair of functions can be orthogonal on one interval and not orthogonal on another.

\section{Property 1: symmetry}

For real functions,

\begin{equation}
\langle f,g\rangle
=
\int_a^b f(x)g(x)\,dx.
\end{equation}

Because multiplication of real numbers is commutative,

\begin{equation}
f(x)g(x)=g(x)f(x).
\end{equation}

Therefore

\begin{align}
\langle f,g\rangle
&amp;=
\int_a^b f(x)g(x)\,dx
\\
&amp;=
\int_a^b g(x)f(x)\,dx
\\
&amp;=
\langle g,f\rangle.
\end{align}

Thus

\begin{equation}
\langle f,g\rangle
=
\langle g,f\rangle.
\end{equation}

\section{Property 2: linearity}

Let $\alpha$ and $\beta$ be constants.
Then

\begin{align}
\langle \alpha f+\beta h,g\rangle
&amp;=
\int_a^b
[\alpha f(x)+\beta h(x)]g(x)\,dx
\\
&amp;=
\alpha\int_a^b f(x)g(x)\,dx
+
\beta\int_a^b h(x)g(x)\,dx
\\
&amp;=
\alpha\langle f,g\rangle
+
\beta\langle h,g\rangle.
\end{align}

So the inner product is linear in each argument for real-valued functions.

\section{Property 3: positivity}

Take the inner product of a function with itself:

\begin{equation}
\langle f,f\rangle
=
\int_a^b f^2(x)\,dx.
\end{equation}

Because

\begin{equation}
f^2(x)\ge0,
\end{equation}

we have

\begin{equation}
\langle f,f\rangle\ge0.
\end{equation}

For the ordinary function spaces used in this series, if

\begin{equation}
\langle f,f\rangle=0,
\end{equation}

then $f$ is zero except possibly on sets that do not affect the integral.
For elementary continuous functions, this simply means

\begin{equation}
f(x)=0
\end{equation}

throughout the interval.

\section{Norm of a function}

The Euclidean vector norm satisfies

\begin{equation}
\|\mathbf v\|
=
\sqrt{\mathbf v\cdot\mathbf v}.
\end{equation}

The analogous function norm is

\begin{equation}
\|f\|
=
\sqrt{\langle f,f\rangle}.
\end{equation}

Therefore

\begin{equation}
\|f\|
=
\left[
\int_a^b f^2(x)\,dx
\right]^{1/2}.
\label{eq:function_norm}
\end{equation}

The norm measures the overall size of the function over the interval.

It does not generally equal the maximum value of the function.
It is an integrated measure of magnitude.

\begin{center}
\includegraphics[width=0.88\textwidth,keepaspectratio]{FS02_fig05_norm_and_normalization.png}

{\small Figure 3. The function norm is determined by the area under $f^2(x)$, not by the largest single value of $f$. Normalizing a function rescales its entire graph so that its inner product with itself equals one.}
\end{center}

\section{Example: norm of $f(x)=x$ on $[-1,1]$}

Let

\begin{equation}
f(x)=x.
\end{equation}

Then

\begin{align}
\|f\|^2
&amp;=
\langle f,f\rangle
\\
&amp;=
\int_{-1}^{1}x^2\,dx.
\end{align}

An antiderivative is

\begin{equation}
\int x^2\,dx
=
\frac{x^3}{3}.
\end{equation}

Thus

\begin{align}
\|f\|^2
&amp;=
\left.
\frac{x^3}{3}
\right|_{-1}^{1}
\\
&amp;=
\frac13-\frac{-1}{3}
\\
&amp;=
\frac23.
\end{align}

Therefore

\begin{equation}
\|f\|
=
\sqrt{\frac23}.
\end{equation}

\section{Normalization of a function}

A normalized function has norm one.

Given a nonzero function $f$, define

\begin{equation}
\widehat f
=
\frac{f}{\|f\|}.
\end{equation}

Then

\begin{align}
\langle \widehat f,\widehat f\rangle
&amp;=
\left\langle
\frac{f}{\|f\|},
\frac{f}{\|f\|}
\right\rangle
\\
&amp;=
\frac{1}{\|f\|^2}
\langle f,f\rangle
\\
&amp;=
1.
\end{align}

For $f(x)=x$ on $[-1,1]$,

\begin{equation}
\widehat f(x)
=
\sqrt{\frac32}\,x.
\end{equation}

\section{Orthogonal functions}

Two functions are orthogonal on $[a,b]$ when

\begin{equation}
\langle f,g\rangle=0.
\end{equation}

In integral form,

\begin{equation}
\int_a^b f(x)g(x)\,dx=0.
\label{eq:function_orthogonal}
\end{equation}

The functions do not need to look geometrically perpendicular on an ordinary graph.
The word orthogonal refers to their inner product.

\section{Example: $1$ and $x$ on $[-1,1]$}

Let

\begin{equation}
f(x)=1,
\qquad
g(x)=x.
\end{equation}

Then

\begin{align}
\langle 1,x\rangle
&amp;=
\int_{-1}^{1}x\,dx
\\
&amp;=
\left.
\frac{x^2}{2}
\right|_{-1}^{1}
\\
&amp;=
\frac12-\frac12
\\
&amp;=0.
\end{align}

Therefore $1$ and $x$ are orthogonal on $[-1,1]$.

The cancellation occurs because $x$ is negative on the left half and positive on the right half.

\begin{center}
\includegraphics[width=0.9\textwidth,keepaspectratio]{FS02_fig03_function_orthogonality.png}

{\small Figure 4. Orthogonality of $1$ and $x$ on a symmetric interval. Their product is just $x$, whose negative and positive signed areas cancel exactly over $[-1,1]$.}
\end{center}

\section{The interval matters}

Use the same functions on $[0,1]$:

\begin{align}
\langle 1,x\rangle
&amp;=
\int_0^1x\,dx
\\
&amp;=
\left.
\frac{x^2}{2}
\right|_0^1
\\
&amp;=
\frac12.
\end{align}

Now the inner product is not zero.

Therefore:

\begin{itemize}
\item $1$ and $x$ are orthogonal on $[-1,1]$,
\item $1$ and $x$ are not orthogonal on $[0,1]$.
\end{itemize}

Orthogonality belongs to the functions together with the chosen inner product and interval.

\section{Even and odd symmetry already helps}

Suppose $f$ is even and $g$ is odd on the symmetric interval $[-L,L]$.

Then

\begin{equation}
f(-x)=f(x)
\end{equation}

and

\begin{equation}
g(-x)=-g(x).
\end{equation}

Their product satisfies

\begin{align}
[f g](-x)
&amp;=f(-x)g(-x)
\\
&amp;=f(x)[-g(x)]
\\
&amp;=-f(x)g(x).
\end{align}

Thus $fg$ is odd.

The integral of an odd function over a symmetric interval is zero:

\begin{equation}
\int_{-L}^{L}f(x)g(x)\,dx=0.
\end{equation}

Therefore every even function is orthogonal to every odd function under this standard inner product on a symmetric interval, provided the integral exists.

This symmetry observation will save substantial work later.

\section{Projection of one function onto another}

Now repeat the vector argument using functions.

Suppose we want to approximate $f(x)$ using one nonzero function $\phi(x)$.
Write

\begin{equation}
f(x)=c\phi(x)+r(x),
\label{eq:function_decomp}
\end{equation}

where $r$ is the residual.

For $c\phi$ to contain all of the component of $f$ along $\phi$, require the residual to be orthogonal to $\phi$:

\begin{equation}
\langle r,\phi\rangle=0.
\end{equation}

From Equation~\eqref{eq:function_decomp},

\begin{equation}
r=f-c\phi.
\end{equation}

Therefore

\begin{equation}
\langle f-c\phi,\phi\rangle=0.
\end{equation}

Use linearity:

\begin{equation}
\langle f,\phi\rangle
-c\langle\phi,\phi\rangle
=0.
\end{equation}

Thus

\begin{equation}
\langle f,\phi\rangle
=
c\langle\phi,\phi\rangle.
\end{equation}

Finally,

\begin{equation}
c
=
\frac{\langle f,\phi\rangle}
{\langle\phi,\phi\rangle}.
\label{eq:function_projection_coeff}
\end{equation}

This is one of the central equations of the Fourier-series learning path.

\begin{center}
\includegraphics[width=0.9\textwidth,keepaspectratio]{FS02_fig04_projection_residual.png}

{\small Figure 5. Function projection uses the same geometry as vector projection. The coefficient $c$ is selected so that the residual $r=f-c\phi$ has zero inner product with the basis function $\phi$.}
\end{center}

\section{If the basis function is normalized}

If

\begin{equation}
\langle\phi,\phi\rangle=1,
\end{equation}

then Equation~\eqref{eq:function_projection_coeff} simplifies to

\begin{equation}
c
=
\langle f,\phi\rangle.
\end{equation}

This is the function-space analogue of taking the dot product with a unit vector.

\section{Worked projection example: approximate $x^2$ by a constant}

Let

\begin{equation}
f(x)=x^2
\end{equation}

on $[-1,1]$.

Approximate it using the constant basis function

\begin{equation}
\phi(x)=1.
\end{equation}

We seek

\begin{equation}
f(x)\approx c.
\end{equation}

The projection coefficient is

\begin{equation}
c
=
\frac{\langle x^2,1\rangle}
{\langle1,1\rangle}.
\end{equation}

Compute the numerator:

\begin{align}
\langle x^2,1\rangle
&amp;=
\int_{-1}^{1}x^2\,dx
\\
&amp;=
\frac23.
\end{align}

Compute the denominator:

\begin{align}
\langle1,1\rangle
&amp;=
\int_{-1}^{1}1\,dx
\\
&amp;=2.
\end{align}

Therefore

\begin{align}
c
&amp;=
\frac{2/3}{2}
\\
&amp;=
\frac13.
\end{align}

So the projection of $x^2$ onto the constant functions is

\begin{equation}
\operatorname{proj}_{1}(x^2)
=
\frac13.
\end{equation}

The residual is

\begin{equation}
r(x)
=
x^2-\frac13.
\end{equation}

Check its orthogonality to the constant basis:

\begin{align}
\langle r,1\rangle
&amp;=
\int_{-1}^{1}
\left(x^2-\frac13\right)dx
\\
&amp;=
\frac23-\frac13(2)
\\
&amp;=0.
\end{align}

\begin{center}
\includegraphics[width=0.9\textwidth,keepaspectratio]{FS02_fig06_x2_projection.png}

{\small Figure 6. Projection of $x^2$ onto the constant basis function on $[-1,1]$. The best constant projection in the inner-product sense is $1/3$, leaving a residual whose signed integral is zero.}
\end{center}

\section{Why the projection constant equals the average value}

For the constant basis function $\phi=1$ on $[a,b]$,

\begin{equation}
c
=
\frac{\int_a^b f(x)\,dx}
{\int_a^b1\,dx}.
\end{equation}

But

\begin{equation}
\int_a^b1\,dx=b-a.
\end{equation}

Therefore

\begin{equation}
c
=
\frac{1}{b-a}
\int_a^b f(x)\,dx.
\label{eq:average_projection}
\end{equation}

Equation~\eqref{eq:average_projection} is exactly the average value of $f$ over the interval.

Thus the average or DC component of a function can be understood geometrically as its projection onto the constant function.

This observation will later explain the constant term in a Fourier series.

\section{Projection onto several orthogonal functions}

Suppose

\begin{equation}
\phi_1,
\phi_2,
\ldots,
\phi_N
\end{equation}

are mutually orthogonal:

\begin{equation}
\langle\phi_m,\phi_n\rangle=0
\qquad
(m\ne n).
\end{equation}

Approximate $f$ by

\begin{equation}
S_N
=
\sum_{n=1}^{N}c_n\phi_n.
\label{eq:orthogonal_expansion}
\end{equation}

Require the residual

\begin{equation}
r=f-S_N
\end{equation}

be orthogonal to every basis function:

\begin{equation}
\langle r,\phi_m\rangle=0.
\end{equation}

Substitute the residual:

\begin{equation}
\left\langle
f-
\sum_{n=1}^{N}c_n\phi_n,
\phi_m
\right\rangle
=0.
\end{equation}

Use linearity:

\begin{equation}
\langle f,\phi_m\rangle
-
\sum_{n=1}^{N}
c_n\langle\phi_n,\phi_m\rangle
=0.
\end{equation}

Because the functions are orthogonal, every term with

\begin{equation}
n\ne m
\end{equation}

vanishes.
Only the $n=m$ term remains:

\begin{equation}
\langle f,\phi_m\rangle
-
c_m\langle\phi_m,\phi_m\rangle
=0.
\end{equation}

Therefore

\begin{equation}
c_m
=
\frac{\langle f,\phi_m\rangle}
{\langle\phi_m,\phi_m\rangle}.
\label{eq:orthogonal_coeff_general}
\end{equation}

This is the general projection formula behind real Fourier coefficients.

FS03 will establish that the sine and cosine functions used in Fourier series are orthogonal on the appropriate interval.
Once that is known, Equation~\eqref{eq:orthogonal_coeff_general} will immediately tell us how to isolate one harmonic.

\section{Orthonormal functions}

A collection of functions is orthonormal when

\begin{equation}
\langle\phi_m,\phi_n\rangle
=
\begin{cases}
1,&amp;m=n,\\
0,&amp;m\ne n.
\end{cases}
\end{equation}

This combines two properties:

\begin{itemize}
\item different basis functions are orthogonal,
\item each basis function has norm one.
\end{itemize}

For an orthonormal basis, Equation~\eqref{eq:orthogonal_coeff_general} simplifies to

\begin{equation}
c_m
=
\langle f,\phi_m\rangle.
\end{equation}

This is the cleanest analogue of Cartesian vector components.

\section{Pythagorean relation for orthogonal functions}

For orthogonal functions $f$ and $g$,

\begin{equation}
\langle f,g\rangle=0.
\end{equation}

Compute the squared norm of their sum:

\begin{align}
\|f+g\|^2
&amp;=
\langle f+g,f+g\rangle
\\
&amp;=
\langle f,f\rangle
+
\langle f,g\rangle
+
\langle g,f\rangle
+
\langle g,g\rangle.
\end{align}

The cross terms vanish:

\begin{equation}
\langle f,g\rangle
=
\langle g,f\rangle
=0.
\end{equation}

Therefore

\begin{equation}
\|f+g\|^2
=
\|f\|^2+\|g\|^2.
\label{eq:function_pythagoras}
\end{equation}

Equation~\eqref{eq:function_pythagoras} is the function-space Pythagorean theorem.

Later, Parseval's theorem will be an infinite-dimensional version of this idea.

\section{Angle between functions}

The vector relation

\begin{equation}
\cos\theta
=
\frac{\mathbf a\cdot\mathbf b}
{\|\mathbf a\|\|\mathbf b\|}
\end{equation}

suggests an analogous definition for nonzero real functions:

\begin{equation}
\cos\theta
=
\frac{\langle f,g\rangle}
{\|f\|\|g\|}.
\label{eq:function_angle}
\end{equation}

This provides a geometric interpretation of the inner product.

If

\begin{equation}
\langle f,g\rangle=0,
\end{equation}

then

\begin{equation}
\cos\theta=0,
\end{equation}

so the functions can be regarded as separated by a $90^\circ$ angle in function space.

The graph of one function is not literally drawn at right angles to the graph of the other.
The angle belongs to the abstract geometry defined by the inner product.

\section{Worked example: project $x^3$ onto $x$}

On $[-1,1]$, let

\begin{equation}
f(x)=x^3,
\qquad
\phi(x)=x.
\end{equation}

We seek

\begin{equation}
x^3\approx cx.
\end{equation}

The coefficient is

\begin{equation}
c
=
\frac{\langle x^3,x\rangle}
{\langle x,x\rangle}.
\end{equation}

The numerator is

\begin{align}
\langle x^3,x\rangle
&amp;=
\int_{-1}^{1}x^4\,dx
\\
&amp;=
\left.
\frac{x^5}{5}
\right|_{-1}^{1}
\\
&amp;=
\frac15-\frac{-1}{5}
\\
&amp;=
\frac25.
\end{align}

The denominator is

\begin{equation}
\langle x,x\rangle
=
\frac23.
\end{equation}

Thus

\begin{align}
c
&amp;=
\frac{2/5}{2/3}
\\
&amp;=
\frac25\frac32
\\
&amp;=
\frac35.
\end{align}

Therefore the projection is

\begin{equation}
\operatorname{proj}_{x}(x^3)
=
\frac35x.
\end{equation}

The residual is

\begin{equation}
r(x)=x^3-\frac35x.
\end{equation}

Check orthogonality:

\begin{align}
\langle r,x\rangle
&amp;=
\int_{-1}^{1}
\left(x^3-\frac35x\right)x\,dx
\\
&amp;=
\int_{-1}^{1}
\left(x^4-\frac35x^2\right)dx
\\
&amp;=
\frac25-\frac35\frac23
\\
&amp;=
\frac25-\frac25
\\
&amp;=0.
\end{align}

\section{Weighted inner products}

The ordinary inner product treats every point of the interval with the same weight.
Sometimes a problem naturally assigns greater importance to some parts of the interval than others.

Define a weighted inner product by

\begin{equation}
\langle f,g\rangle_w
=
\int_a^b w(x)f(x)g(x)\,dx,
\label{eq:weighted_inner_product}
\end{equation}

where the weight function satisfies

\begin{equation}
w(x)&gt;0
\end{equation}

throughout the interval, apart from possible special endpoint behavior in more advanced settings.

The associated weighted norm is

\begin{equation}
\|f\|_w
=
\left[
\int_a^b w(x)f^2(x)\,dx
\right]^{1/2}.
\end{equation}

Weighted inner products will become important later in generalized eigenfunction expansions and Sturm-Liouville theory.

\begin{center}
\includegraphics[width=0.9\textwidth,keepaspectratio]{FS02_fig07_weighted_inner_product.png}

{\small Figure 7. A weighted inner product changes how strongly different portions of the interval contribute. Regions with larger $w(x)$ influence the projection and norm more strongly.}
\end{center}

\section{Example of a weighted projection}

On $[0,1]$, let

\begin{equation}
w(x)=2x,
\qquad
f(x)=x,
\qquad
\phi(x)=1.
\end{equation}

The weighted projection coefficient onto the constant function is

\begin{equation}
c
=
\frac{\langle x,1\rangle_w}
{\langle1,1\rangle_w}.
\end{equation}

Compute the numerator:

\begin{align}
\langle x,1\rangle_w
&amp;=
\int_0^1(2x)(x)(1)\,dx
\\
&amp;=
2\int_0^1x^2\,dx
\\
&amp;=
\frac23.
\end{align}

The denominator is

\begin{align}
\langle1,1\rangle_w
&amp;=
\int_0^1 2x\,dx
\\
&amp;=1.
\end{align}

Thus

\begin{equation}
c=\frac23.
\end{equation}

Without the weight, the ordinary average of $x$ on $[0,1]$ is

\begin{equation}
\frac12.
\end{equation}

The weight $w=2x$ emphasizes the right side of the interval, where $x$ is larger, so the weighted projection shifts upward to $2/3$.

\section{Units of a function inner product}

Suppose $f$ and $g$ have physical units $[f]$ and $[g]$, and $x$ has units $[x]$.

Then

\begin{equation}
[\langle f,g\rangle]
=
[f][g][x].
\end{equation}

Therefore an inner product is not necessarily dimensionless.

Likewise,

\begin{equation}
[\|f\|]
=
[f]\sqrt{[x]}.
\end{equation}

In mathematical treatments, variables are often nondimensionalized or the units are suppressed.
In physics applications, the dimensions should still be checked.

\section{Function space: what does it mean to treat a function like a vector?}

A vector in ordinary three-dimensional space is specified by three components.
A function requires infinitely many values to specify its complete shape.

The phrase \emph{function space} refers to a set of functions treated as points or vectors in a larger abstract space.

The inner product supplies geometry to that space.
It lets us define:

\begin{itemize}
\item length through $\|f\|$,
\item orthogonality through $\langle f,g\rangle=0$,
\item angle through Equation~\eqref{eq:function_angle},
\item projection through Equation~\eqref{eq:function_projection_coeff}.
\end{itemize}

This geometric viewpoint is the foundation of Fourier analysis.

A rigorous treatment eventually introduces Hilbert spaces and equivalence classes of functions.
That level of functional analysis is not needed yet.
The elementary integral geometry is enough to derive Fourier series.

\section{Why orthogonality makes coefficient extraction easy}

Suppose

\begin{equation}
f
=
c_1\phi_1+c_2\phi_2+c_3\phi_3.
\end{equation}

Take the inner product with $\phi_2$:

\begin{align}
\langle f,\phi_2\rangle
&amp;=
\langle
c_1\phi_1+c_2\phi_2+c_3\phi_3,
\phi_2
\rangle
\\
&amp;=
c_1\langle\phi_1,\phi_2\rangle
+c_2\langle\phi_2,\phi_2\rangle
+c_3\langle\phi_3,\phi_2\rangle.
\end{align}

If the basis functions are mutually orthogonal,

\begin{equation}
\langle\phi_1,\phi_2\rangle=0
\end{equation}

and

\begin{equation}
\langle\phi_3,\phi_2\rangle=0.
\end{equation}

Therefore

\begin{equation}
\langle f,\phi_2\rangle
=
c_2\langle\phi_2,\phi_2\rangle.
\end{equation}

Hence

\begin{equation}
c_2
=
\frac{\langle f,\phi_2\rangle}
{\langle\phi_2,\phi_2\rangle}.
\end{equation}

The inner product with $\phi_2$ destroys every orthogonal component and leaves only the $\phi_2$ component.

This is exactly what FS04 and FS05 will do with cosine and sine harmonics.

\section{Preview of the Fourier connection}

FS01 introduced a finite harmonic sum of the form

\begin{equation}
S_N(x)
=
\frac{a_0}{2}
+
\sum_{n=1}^{N}
\left[
a_n\cos(n\omega_0x)
+
b_n\sin(n\omega_0x)
\right].
\end{equation}

If the functions

\begin{equation}
1,
\qquad
\cos(n\omega_0x),
\qquad
\sin(n\omega_0x)
\end{equation}

are orthogonal over one full period, then each coefficient can be extracted by projection.

For example, the structure must look like

\begin{equation}
a_m
\propto
\left\langle
f,
\cos(m\omega_0x)
\right\rangle.
\end{equation}

The proportionality factor depends on the norm of the cosine basis function.

We are not yet entitled to write the final Fourier coefficient formula because we have not proved the required trigonometric orthogonality identities.

That is the purpose of FS03.

\begin{center}
\includegraphics[width=0.92\textwidth,keepaspectratio]{FS02_fig08_bridge_to_FS03.png}

{\small Figure 8. FS02 supplies the geometry. FS03 will prove that distinct sine and cosine harmonics are orthogonal, after which FS04 and FS05 can extract individual Fourier coefficients by projection.}
\end{center}

\section{A first three-function orthogonal example}

On $[-1,1]$, consider

\begin{equation}
\phi_0(x)=1,
\qquad
\phi_1(x)=x,
\qquad
\phi_2(x)=x^2-\frac13.
\end{equation}

We already know

\begin{equation}
\langle1,x\rangle=0.
\end{equation}

Now check

\begin{align}
\langle1,\phi_2\rangle
&amp;=
\int_{-1}^{1}
\left(x^2-\frac13\right)dx
\\
&amp;=
\frac23-\frac23
\\
&amp;=0.
\end{align}

Next,

\begin{align}
\langle x,\phi_2\rangle
&amp;=
\int_{-1}^{1}
 x\left(x^2-\frac13\right)dx
\\
&amp;=
\int_{-1}^{1}
\left(x^3-\frac13x\right)dx.
\end{align}

The integrand is odd, so

\begin{equation}
\langle x,\phi_2\rangle=0.
\end{equation}

Thus the three functions are mutually orthogonal.

This example illustrates a general strategy: new basis functions can be constructed by subtracting projections onto earlier basis functions.
A systematic version of this process is the Gram-Schmidt procedure, which will be useful later but is not needed for the first Fourier derivations.

\section{Self-check questions}

\begin{enumerate}
\item What does the inner product of two functions return: another function or a scalar?

\item Why is the integration interval part of the definition of orthogonality?

\item What does it mean geometrically if
\begin{equation}
\langle f,g\rangle=0?
\end{equation}

\item Derive the projection coefficient
\begin{equation}
c=
\frac{\langle f,\phi\rangle}
{\langle\phi,\phi\rangle}
\end{equation}
from residual orthogonality.

\item Why does the denominator disappear for a normalized basis function?

\item What is the projection of a function onto the constant basis function physically related to?

\item Why does an even function have zero inner product with an odd function over a symmetric interval?

\item Why does orthogonality make it possible to isolate one coefficient from an expansion containing many coefficients?

\item What is the difference between an orthogonal set and an orthonormal set?

\item Why are we postponing the Fourier coefficient formulas until after FS03?
\end{enumerate}

\section{Practice problems}

\begin{enumerate}
\item Compute
\begin{equation}
\langle1,x^2\rangle
\end{equation}
on $[-1,1]$.

\item Determine whether $x$ and $x^2$ are orthogonal on $[-1,1]$.

\item Determine whether $1$ and $x$ are orthogonal on $[-2,2]$ and on $[0,2]$.

\item Compute the norm of $f(x)=1$ on $[-L,L]$.

\item Normalize $f(x)=1$ on $[-L,L]$.

\item Compute the norm of $f(x)=x$ on $[-2,2]$.

\item Project $f(x)=x^2$ onto the constant function on $[0,1]$.

\item Project $f(x)=x^4$ onto the constant function on $[-1,1]$.

\item Project $f(x)=x^3$ onto $\phi(x)=x$ on $[-2,2]$.

\item Show directly that $x^2-1/3$ is orthogonal to $1$ on $[-1,1]$.

\item Suppose $\phi_1$ and $\phi_2$ are orthogonal. Prove
\begin{equation}
\|c_1\phi_1+c_2\phi_2\|^2
=
c_1^2\|\phi_1\|^2
+
c_2^2\|\phi_2\|^2.
\end{equation}

\item Using the weighted inner product with $w(x)=x$ on $[0,1]$, project $f(x)=x$ onto the constant function.
\end{enumerate}

\section{Summary}

The ordinary vector dot product

\begin{equation}
\mathbf a\cdot\mathbf b
\end{equation}

has a continuous analogue for real functions:

\begin{equation}
\langle f,g\rangle
=
\int_a^b f(x)g(x)\,dx.
\end{equation}

This inner product defines the geometry needed for Fourier analysis.

Two functions are orthogonal when

\begin{equation}
\langle f,g\rangle=0.
\end{equation}

The norm of a function is

\begin{equation}
\|f\|
=
\sqrt{\langle f,f\rangle}.
\end{equation}

The projection coefficient of $f$ onto a nonzero basis function $\phi$ is

\begin{equation}
c
=
\frac{\langle f,\phi\rangle}
{\langle\phi,\phi\rangle}.
\end{equation}

This coefficient is obtained by requiring the residual

\begin{equation}
r=f-c\phi
\end{equation}

be orthogonal to $\phi$.

For an orthogonal collection $\{\phi_n\}$,

\begin{equation}
c_m
=
\frac{\langle f,\phi_m\rangle}
{\langle\phi_m,\phi_m\rangle}.
\end{equation}

FS03 now supplies the missing ingredient: it proves in detail that distinct sine and cosine harmonics are orthogonal over a complete period.

\section*{References}

\begin{thebibliography}{9}

\bibitem{FS00}
PhysicsLibrary,
\emph{FS00: Fourier Series Roadmap, Motivation, and Learning Path}.

\bibitem{FS01}
PhysicsLibrary,
\emph{FS01: Periodic Functions, Harmonics, and Why Fourier Series Exist}.

\bibitem{Kreyszig}
E. Kreyszig,
\emph{Advanced Engineering Mathematics},
Wiley.

\bibitem{Arfken}
G. B. Arfken, H. J. Weber, and F. E. Harris,
\emph{Mathematical Methods for Physicists},
Academic Press.

\end{thebibliography}</content>
</record>
