• Orthogonal functions

Fourier Series and Transforms

• Orthogonal functions

• Fourier Series

• Discrete Fourier Series

• Fourier Transform

• Chebyshev polynomials

Scope: we are trying to approximate

an arbitrary function and obtain

basis functions with appropriate

coefficients.

Fourier Series

The Problem

we are trying to approximate a function f(x) by another function g_n (x) which consists of a sum over N orthogonal functions Φ(x) weighted by some coefficients a_n .

) ( )

( )

(

0

x a

x g

x f

N

i

i i

N

∑

=

Φ

=

≈

... and we are looking for optimal functions in a least squares (l₂ ) sense ...

... a good choice for the basis functions Φ(x) are orthogonal functions. What are orthogonal functions? Two functions f and g are said to be

orthogonal in the interval [a,b] if

∫

⁼

a

dx x

g x

f ( ) ( ) 0

How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:

The Problem

{ ^{( ) ( )} } ^{Min !}

) ( )

(

2 / 1 2

2

⎥ =

⎦

⎢ ⎤

⎣

⎡ −

=

− ∫

a

N

N

x f x g x dx

g x

f

Orthogonal Functions

... where x₀ =a and x_N =b, and x_i -x_i-1 =Δx ...

If we interpret f(x_i ) and g(x_i ) as the ith components of an N component vector, then this sum corresponds directly to a scalar product of vectors.

The vanishing of the scalar product is the condition for orthogonality of vectors (or functions).

⎟ ⎠

⎜ ⎞

⎝

⎛ Δ

= ∑

∫

N

i

i i

b

a

f x g x dx

f x g x x

1

) ( ) ( lim

) ( ) (

f

g

• = ∑

= 0

i i i

i

g f g

f

Periodic functions

-15 -10 -5 0 5 10 15 20

0 10 20 30 40

Let us assume we have a piecewise continuous function of the form

) ( )

2

( x f x

f + π =

)

( )

2

( x f x x

f + π = =

... we want to approximate this function with a linear combination of 2π periodic functions:

) sin(

), cos(

),..., 2

sin(

), 2 cos(

), sin(

), cos(

,

1 x x x x nx nx

{ }

∑ ⁺

+

=

≈

⇒ f x g

x 1 a

a

cos( kx ) b

sin( kx ) )

( )

(

Orthogonality

... are these functions orthogonal ?

0 ,

0 0

) sin(

) cos(

0 0 ,

, 0

) sin(

) sin(

0 0 2

0 )

cos(

) cos(

>

≥

=

>

=

>

≠

⎪⎩

⎪⎨

⎧

=

>

=

=

=

≠

⎪⎩

⎪⎨

⎧

=

∫

∫

∫

−

−

−

k j

dx kx jx

k j

k j k j

dx kx jx

k j

k j

k j

dx kx jx

π π π π π

π

π π

π

... YES, and these relations are valid for any interval of length 2π.

Now we know that this is an orthogonal basis, but how can we obtain the coefficients for the basis functions?

from minimising f(x)-g(x)

Fourier coefficients

optimal functions g(x) are given if

{ ^{( ) ( )} } ⁰

! Min )

( )

( x − f x

= or

g x − f x

=

g

k

leading to

... with the definition of g(x) we get ...

{ }

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎦

⎢ ⎤

⎣

⎡ + + −

∂

= ∂

∂ −

∂

∫ ∑

− =

dx x

f kx

b kx

a a a

x f x

a g

N

k

k k

k n

k

2

1 0

2 cos( ) sin( ) ( )

2 ) 1

( )

(

2

π π

{ }

∫

∫

∑

−

−

=

=

=

=

=

+ +

=

π π π

π

π π

N k

dx kx x

f b

N k

dx kx x

f a

kx b

kx a

a x

g

k k

N

k

k k

N

,..., 2 , 1 ,

) sin(

) 1 (

,..., 1 , 0 ,

) cos(

) 1 (

with )

sin(

) 2 cos(

) 1 (

1 0

Fourier approximation of |x|

... Example ...

.. and for n<4 g(x) looks like leads to the Fourier Serie

⎭⎬

⎫

⎩⎨

⎧ + + +

−

= ...

5 ) 5 cos(

3 ) 3 cos(

1 ) cos(

4 2

) 1

( ₂ x ₂ x ₂ x

x

g π π

π π ≤ ≤

−

= x x

x

f ( ) ,

-20 -15 -10 -5 0 5 10

0 1 2 3 4

Fourier approximation of x ²

... another Example ...

π 2 0

, )

( x = x

< x <

f

.. and for N<11, g(x) looks like leads to the Fourier Serie

∑

⎩⎨

⎧ −

+

= ^N

k

N kx

kx k x k

g

1

2 2

) 4 sin(

) 4 cos(

3 ) 4

( π π

-1 0 -5 0 5 1 0

-1 0 0 1 0 2 0 3 0 4 0

Fourier - discrete functions

N i x

2 π

=

.. the so-defined Fourier polynomial is the unique interpolating function to the function f(x_j ) with N=2m

it turns out that in this particular case the coefficients are given by

,...

3 , 2 , 1 ,

) sin(

) 2 (

,...

2 , 1 , 0 ,

) cos(

) 2 (

1

*

1

*

=

=

=

=

∑

∑

=

=

k kx

x N f

b

k kx

x N f

a

N

j

j j

N

j

j j

k k

{ }

2 ) 1 sin(

) 2 cos(

) 1

( ^*

1

1

*

*

*

*

0 a kx b kx a kx

a x

g _m

m

k

m = +

∑

=

... what happens if we know our function f(x) only at the points

) ( )

*

(

i i

m

x f x

g =

Fourier - collocation points

... with the important property that ...

... in our previous examples ...

-10 -5 0 5

0 0.5 1 1.5 2 2.5 3 3.5

f(x)=|x| => f(x) - blue ; g(x) - red; x_i - ‘+’

Fourier series - convergence

f(x)=x² => f(x) - blue ; g(x) - red; x_i - ‘+’

-10 -5 0 5 10

-10 0 10 20 30 40 50

N = 4

-10 -5 0 5 10

-10 0 10 20 30 40 50

N = 8

Fourier series - convergence

f(x)=x² => f(x) - blue ; g(x) - red; x_i - ‘+’

-10 -5 0 5 10

-10 0 10 20 30 40 50

N = 16

-10 -5 0 5 10

-10 0 10 20 30 40 50

N = 32

Gibb’s phenomenon

f(x)=x² => f(x) - blue ; g(x) - red; x_i - ‘+’

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 32

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 16

0 0.5

-6 -4 -2 0 2 4 6

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 128

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 256

The overshoot for equi- spaced Fourier

interpolations is ≈14% of the step height.

Chebyshev polynomials

We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.

We depart by observing that cos(nϕ) can be expressed by a polynomial in cos(ϕ):

1 cos

8 cos

8 )

4 cos(

cos 3

cos 4

) 3 cos(

1 cos

2 )

2 cos(

2 4

3 2

+

−

=

−

=

−

=

ϕ ϕ

ϕ

ϕ ϕ

ϕ

ϕ ϕ

... which leads us to the definition:

Chebyshev polynomials - definition

N n

x x

x T T

n ) =

(cos( )) =

( ), = cos( ), ∈ [ − 1 , 1 ], ∈

cos( ϕ ϕ ϕ

... for the Chebyshev polynomials T_n (x). Note that because of x=cos(ϕ) they are defined in the interval [-1,1] (which - however - can be extended to ℜ). The first polynomials are

0 2

4 4

3 3

2 2

1 0

and ]

1 , 1 [ for

1 )

(

where 1

8 8

) (

3 4

) (

1 2

) (

) (

1 ) (

N n

x x

T

x x

x T

x x

x T

x x

T

x x

T x T

n

≤ ∈ − ∈

+

−

=

−

=

−

=

=

=

Chebyshev polynomials - Graphical

The first ten polynomials look like [0, -1]

The n-th polynomial has extrema with values 1 or -1 at

0 0.2 0.4 0.6 0.8

-1 -0.5 0 0.5 1

x

T_n(x)

n k k

x

= cos( π ), = 0 , 1 , 2 , 3 ,...,

Chebyshev collocation points

These extrema are not equidistant (like the Fourier extrema)

n n k

x

= cos( k π ), = 0 , 1 , 2 , 3 ,...,

k

x(k)

Chebyshev polynomials - orthogonality

... are the Chebyshev polynomials orthogonal?

2 0 1

1

, ,

0 0 2

/ 0 1

) ( )

( k j N

j k

for

j k

for

j k

for x

x dx T

x

T

∈

⎪ ⎭

⎪ ⎬

⎫

⎪ ⎩

⎪ ⎨

⎧

=

=

>

=

≠

− =

−

∫ π

π

Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight function

such that

1 2

/

1 −x

... this can be easily verified noting that

) cos(

) ( ),

cos(

) (

sin ,

cos

ϕ ϕ

ϕ ϕ ϕ

j x

T k

x T

d dx

x

=

=

−

=

=

Chebyshev polynomials - interpolation

... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a periodical function but a function which is defined between [-1,1].

We are looking for g_n (x)

) ( )

2 ( ) 1

( )

(

1 0

0

T x c T x

c x

g x

f

n

k

k k

n

∑

=

+

=

≈

... and we are faced with the problem, how we can determine the coefficients c_k . Again we obtain this by finding the extremum

(minimum)

{ } ⁰

1 )

( )

(

1

1

2

⎥ =

⎦

⎢ ⎤

⎣

⎡

− −

∂

∂ ∫

−

x

x dx f

x

c

g

Chebyshev polynomials - interpolation

... to obtain ...

n k

x x dx

T x f

c

, 0 , 1 , 2 ,...,

1 ) ( ) 2

(

1

2

=

= ∫ − π

... surprisingly these coefficients can be calculated with FFT techniques, noting that

n k

d k f

c

2 (cos ) cos , 0 , 1 , 2 ,...,

0

=

= _π ∫

^{ϕ ϕ ϕ}

... and the fact that f(cosϕ) is a 2π-periodic function ...

n k

d k f

c

= 1 ∫ (cos ) cos , = 0 , 1 , 2 ,...,

−

ϕ ϕ π ϕ

π

π

... which means that the coefficients c_k are the Fourier coefficients of the periodic function F(ϕ)=f(cos ϕ)!

Chebyshev - discrete functions

N i

x

π

= cos

... leading to the polynomial ...

in this particular case the coefficients are given by

2 / ,...

2 , 1 , 0 ,

) cos(

) 2 (cos

1

* f k k N

c N

N

j

j

k =

∑

=

ϕ ϕ

∑

+

= ^m

k

k k

m x c T c T x

g

1

*

* 0

* ( )

2 ) 1

( ₀

... what happens if we know our function f(x) only at the points

... with the property

N 0,1,2,..., j

j/N) cos(

x at )

( )

( _j

* x = f x = π =

g_m

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; g_n (x) - red; x_i - ‘+’

-1 -0 .8 -0 .6 -0 .4 -0 .2 0 0 .2 0 .4 0 .6 0 .8

0 0 .2 0 .4 0 .6 0 .8 1

N = 8

0.2 0.4 0.6 0.8 1

N = 1 6

8 points

16 points

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; g_n (x) - red; x_i - ‘+’

32 points

128 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8

0 0.2 0.4 0.6 0.8 1

N = 32

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.

0 0.2 0.4 0.6 0.8 1

N = 128

Chebyshev - collocation points - x ²

f(x)=x² => f(x) - blue ; g_n (x) - red; x_i - ‘+’

8 points

64 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8

0 0.2 0.4 0.6 0.8 1 1.2

N = 8

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1 1.2

N = 64

The interpolating function g_n (x) was shifted by a small amount to be visible at all!

Chebyshev vs. Fourier - numerical

f(x)=x² => f(x) - blue ; g_N (x) - red; x_i - ‘+’

This graph speaks for itself ! Gibb’s phenomenon with Chebyshev?

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

N = 16

0 2

-5 0 5 10 15 20 25 30 35

N =

Chebyshev Fourier

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x-π⁾ => f(x) - blue ; g_N (x) - red; x_i - ‘+’

Chebyshev Fourier

-1 -0.5 0 0.5 1

-1.5 -1 -0.5 0 0.5 1 1.5

N = 16

0 2

-1.5 -1 -0.5 0 0.5 1 1.5

N = 16

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x-π⁾ => f(x) - blue ; g_N (x) - red; x_i - ‘+’

Chebyshev Fourier

-1 -0.5 0 0.5 1

-1.5 -1 -0.5 0 0.5 1 1.5

N = 64

0 2 4

-1.5 -1 -0.5 0 0.5 1 1.5

N = 6

Fourier vs. Chebyshev

Fourier Chebyshev

N i

x_i = 2π i

x_i πN

= cos

periodic functions limited area [-1,1]

) sin(

),

cos( nx nx

ϕ ϕ cos

), cos(

) (

=

= x

n x

T_n

{ }

) 2 cos(

1

) sin(

) cos(

2 ) 1 (

* 1

1

*

*

*

*

0

kx a

kx b

kx a

a x

g

m m

k m

k k

+

+ +

=

∑

=

∑

+

= ^m

k

k k

m x c T c T x

g

1

*

* 0

* ( )

2 ) 1

( ₀

collocation points

domain

basis functions

interpolating function

Fourier vs. Chebyshev (cont’d)

Fourier Chebyshev

coefficients

some properties

∑

∑

=

=

=

=

N

j

j j

N

j

j j

kx x

N f b

kx x

N f a

k k

1

*

1

*

) sin(

) 2 (

) cos(

) 2 (

∑

= ^N

j

j

j k

N f ck

1

* 2 (cos ϕ )cos( ϕ )

• Gibb’s phenomenon for discontinuous functions

• Efficient calculation via FFT

• infinite domain through periodicity

• limited area calculations

• grid densification at boundaries

• coefficients via FFT

• excellent convergence at boundaries

• Gibb’s phenomenon