Function approximation: Fourier, Chebyshev, Lagrange

Function approximation: Fourier, Chebyshev, Lagrange

 Orthogonal functions

 Fourier Series

 Discrete Fourier Series

 Fourier Transform: properties

 Chebyshev polynomials

 Convolution

 DFT and FFT

Scope: Understanding where the Fourier Transform comes

from. Moving from the continuous to the discrete world. The

concepts are the basis for pseudospectral methods and the

spectral element approach.

Fourier Series: one way to derive them

The Problem

we are trying to approximate a function f(x) by another function g _n (x) which consists of a sum over N orthogonal functions (x) weighted by some coefficients a _n .

) ( )

( )

(

0

x a

x g

x

f ^N

i

i i

N 









... and we are looking for optimal functions in a least squares (l ₂ ) sense ...

... a good choice for the basis functions (x) are orthogonal functions . What are orthogonal functions? Two functions f and g are said to be

orthogonal in the interval [a,b] if

 ^{b }

a

dx x g x

f ( ) ( ) 0

How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:

The Problem

 ^{( ) ( )}  ^{Min !}

) ( )

(

2 / 1 2

2  



 



 



  ^b

a

N

N x f x g x dx

g x

f

Orthogonal Functions

... where x ₀ =a and x _N =b, and x _i -x _i-1 =x ...

If we interpret f(x _i ) and g(x _i ) as the ith components of an N component vector, then this sum corresponds directly to a scalar product of vectors.

The vanishing of the scalar product is the condition for orthogonality of vectors (or functions).

 

 



 

 

   

N i

i i

b

a f x g x dx N f x g x x

1

) ( ) ( lim

) ( ) (

f _i g _i    ⁱ  0

i i i

i g f g

f

Periodic functions

-15 -10 -5 0 5 10 15 20

0 10 20 30 40

Let us assume we have a piecewise continuous function of the form

) ( )

2

( x f x

f   

) 2

( )

2

( x f x x

f    

... we want to approximate this function with a linear combination of 2

periodic functions:

) sin(

), cos(

),..., 2

sin(

), 2 cos(

), sin(

), cos(

,

1 x x x x nx nx

 

 ^







 f x g _N x a ₀ ^N a _k cos( kx ) b _k sin( kx ) 2

) 1 ( )

(

Orthogonality

... are these functions orthogonal ?

0 ,

0 0

) sin(

) cos(

0 0 ,

, 0

) sin(

) sin(

0 0 2

0 )

cos(

) cos(















 

 















 

 

















k j

dx kx jx

k j

k j k j

dx kx jx

k j

k j

k j

dx kx jx



















... YES, and these relations are valid for any interval of length 2.

Now we know that this is an orthogonal basis, but how can we obtain the coefficients for the basis functions?

from minimising f(x)-g(x)

Fourier coefficients

optimal functions g(x) are given if

 ^{( ) ( )}  ⁰

! Min )

( )

( x  f x ₂  or _ ^ g x  f x ₂ 

g _{n a n}

k

leading to

... with the definition of g(x) we get ...

 

 





 





 

 



   



 

 

  

 

dx x

f kx

b kx

a a a

x f x

a g

N k

k k

k n

k

2

1 0

2

cos( ) sin( ) ( )

2 ) 1

( )

(





 







































N k

dx kx x

f b

N k

dx kx x

f a

kx b

kx a

a x

g

k k

N k

k k

N

,..., 2 , 1 ,

) sin(

) 1 (

,..., 1 , 0 ,

) cos(

) 1 (

with )

sin(

) 2 cos(

) 1 (

1 0

Fourier approximation of |x|

... Example ...

.. and for n<4 g(x) looks like leads to the Fourier Serie

 



 

   



 ...

5 ) 5 cos(

3 ) 3 cos(

1 ) cos(

4 2

) 1

(

x

x

x

x

g  



  



 x x

x

f ( ) ,

-20 -15 -10 -5 0 5 10 15 20

0

1

2

3

4

Fourier approximation of x ²

... another Example ...

 2 0

, )

( x  x ²  x  f

.. and for N<11, g(x) looks like leads to the Fourier Serie



  

 

 





k

N

kx

kx k x k

g

1 2

2

) 4 sin(

) 4 cos(

3 ) 4

(  

-10 -5 0 5 10 15

-10 0 10 20 30 40

Fourier - discrete functions

N i x _i 2 



.. the so-defined Fourier polynomial is the unique interpolating function to the function f(x _j ) with N=2m

it turns out that in this particular case the coefficients are given by

,...

3 , 2 , 1 ,

) sin(

) 2 (

,...

2 , 1 , 0 ,

) cos(

) 2 (

1

*

1

*

















k kx

x N f

b

k kx

x N f

a

N j

j j

N j

j j

k k

  ^{cos( )}

2 ) 1 sin(

) 2 cos(

) 1

(

1

*

*

*

*

0

a kx b kx a kx

a x

g

k

m

  







... what happens if we know our function f(x) only at the points

) ( )

* (

i i

m x f x

g 

Fourier - collocation points

... with the important property that ...

... in our previous examples ...

-10 -5 0 5 10

0 0.5 1 1.5 2 2.5 3 3.5

f(x)=|x| => f(x) - blue ; g(x) - red; x _i - ‘+’

Fourier series - convergence

f(x)=x ² => f(x) - blue ; g(x) - red; x _i - ‘+’

Fourier series - convergence

f(x)=x ² => f(x) - blue ; g(x) - red; x _i - ‘+’

Gibb’s phenomenon

f(x)=x ² => f(x) - blue ; g(x) - red; x _i - ‘+’

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 32

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 16

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 64

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 128

0 0.5 1 1.5

-6 -4 -2 0 2 4 6

N = 256

The overshoot for equi- spaced Fourier

interpolations is 14% of

the step height.

Chebyshev polynomials

We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly

spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.

We depart by observing that cos(n) can be expressed by a polynomial in cos():

1 cos

8 cos

8 )

4 cos(

cos 3

cos 4

) 3 cos(

1 cos

2 )

2 cos(

2 4

3 2































... which leads us to the definition:

Chebyshev polynomials - definition

N n

x x

x T T

n )  _n (cos( ))  _n ( ),  cos( ),  [  1 , 1 ], 

cos(   

... for the Chebyshev polynomials T _n (x). Note that because of x=cos() they are defined in the interval [-1,1] (which - however - can be extended to ). The first polynomials are

0 2

4 4

3 3

2 2

1 0

and ]

1 , 1 [ for

1 )

(

where 1

8 8

) (

3 4

) (

1 2

) (

) (

1 ) (

N n

x x

T

x x

x T

x x

x T

x x

T

x x

T x T

n    



















Chebyshev polynomials - Graphical

The first ten polynomials look like [0, -1]

The n-th polynomial has extrema with values 1 or -1 at

0 0.2 0.4 0.6 0.8 1

-1 -0.5 0 0.5 1

x

T _ n( x)

n n k

x _k ^{( ext )}  cos( k  ),  0 , 1 , 2 , 3 ,...,

Chebyshev collocation points

These extrema are not equidistant (like the Fourier extrema)

n n k

x _k ^{( ext )}  cos( k  ),  0 , 1 , 2 , 3 ,...,

k

x(k)

Chebyshev polynomials - orthogonality

... are the Chebyshev polynomials orthogonal?

2 0 1

1

, ,

0 0 2

/ 0 ) 1

( )

( k j N

j k

for

j k

for

j k

for x

x dx T

x

T _{k j} 

 

 



 

 













 

  



Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight function

such that

1

/

1  x

... this can be easily verified noting that

) cos(

) ( ),

cos(

) (

sin ,

cos











j x

T k

x T

d dx

x

j

k  







Chebyshev polynomials - interpolation

... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a periodical function but a function which is defined between [-1,1].

We are looking for g _n (x)

) ( )

2 ( ) 1

( )

(

1 0

0 T x c T x

c x

g x

f ⁿ

k

k k

n 









... and we are faced with the problem, how we can determine the coefficients c _k . Again we obtain this by finding the extremum

(minimum)

  ⁰

) 1 ( )

( 2

1

1

2  



 





 



 

 x

x dx f

x

c _k g ⁿ

Chebyshev polynomials - interpolation

... to obtain ...

n x k

x dx T

x f

c _{k k} , 0 , 1 , 2 ,...,

) 1 ( ) 2 ¹ (

1 2 

  

 

... surprisingly these coefficients can be calculated with FFT techniques, noting that

n k

d k f

c _k 2 (cos ) cos , 0 , 1 , 2 ,...,

0



 _ ^  ^{  }

... and the fact that f(cos) is a 2-periodic function ...

n k

d k f

c _k 1 (cos ) cos , 0 , 1 , 2 ,...,



 







 





... which means that the coefficients c _k are the Fourier coefficients

a of the periodic function F()=f(cos )!

Chebyshev - discrete functions

N i

x _i 

 cos

... leading to the polynomial ...

in this particular case the coefficients are given by

2 / ,...

2 , 1 , 0 ,

) cos(

) 2 (cos

1

*

f k k N

c N

j

j

k

 















k

k k

m

x c T c T x

g

1

*

* 0

*

( )

2 ) 1

(

... what happens if we know our function f(x) only at the points

... with the property

N 0,1,2,..., j

j/N) cos(

x at )

( )

(

*

x  f x   

g

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; g _n (x) - red; x _i - ‘+’

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8

1 N = 8

0 0.2 0.4 0.6 0.8

1 N = 16

8 points

16 points

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; g _n (x) - red; x _i - ‘+’

32 points

128 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8

1 N = 32

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8

1 N = 128

Chebyshev - collocation points - x ²

f(x)=x ² => f(x) - blue ; g _n (x) - red; x _i - ‘+’

8 points

64 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

1.2 N = 8

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8 1

1.2 N = 64

The interpolating

function g

(x) was

shifted by a small

amount to be

visible at all!

Chebyshev vs. Fourier - numerical

f(x)=x ² => f(x) - blue ; g _N (x) - red; x _i - ‘+’

0 0.2 0.4 0.6 0.8 1

0 0.2 0.4 0.6 0.8

1 N = 16

0 2 4 6

-5 0 5 10 15 20 25 30 35

N = 16

Chebyshev Fourier

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x- ⁾ => f(x) - blue ; g _N (x) - red; x _i - ‘+’

Chebyshev Fourier

-1 -0.5 0 0.5 1

-1.5 -1 -0.5 0 0.5 1

1.5 N = 16

0 2 4 6

-1.5 -1 -0.5 0 0.5 1 1.5

N = 16

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x- ⁾ => f(x) - blue ; g _N (x) - red; x _i - ‘+’

Chebyshev Fourier

-1 -0.5 0 0.5 1

-1.5 -1 -0.5 0 0.5 1 1.5

N = 64

0 2 4 6 8

-1.5 -1 -0.5 0 0.5 1 1.5

N = 64

Fourier vs. Chebyshev

Fourier Chebyshev

N i x

2 

 i

x _i  N

 cos

periodic functions limited area [-1,1]

) sin(

),

cos( nx nx



 cos

), cos(

) (



 x

n x

T

 

) 2 cos(

1

) sin(

) cos(

2 ) 1 (

* 1 1

*

*

*

*

0

kx a

kx b

kx a

a x

g

m m k m

k k



















k

k k

m

x c T c T x

g

1

*

* 0

*

( )

2 ) 1

(

collocation points

domain

basis functions

interpolating

function

Fourier vs. Chebyshev (cont’d)

Fourier Chebyshev

coefficients

some properties













N j

j j

N j

j j

kx x

N f b

kx x

N f a

k k

1

*

1

*

) sin(

) 2 (

) cos(

) 2 (





j

j

j

k

N f c

1

*

2 (cos  ) cos(  )

• Gibb’s phenomenon for discontinuous functions

• Efficient calculation via FFT

• infinite domain through periodicity

• limited area calculations

• grid densification at boundaries

• coefficients via FFT

• excellent convergence at boundaries

• Gibb’s phenomenon

The Fourier Transform Pair



























 





d e

F t

f

dt e

t f F

t i

t i

) (

) (

) 2 (

) 1

( Forward transform

Inverse transform

Note the conventions concerning the sign of the exponents and the factor.

Some properties of the Fourier Transform

Defining as the FT: f ( t )  F (  )

 Linearity

 Symmetry

 Time shifting

 Time differentiation

) ( )

( )

( )

( _{2 1 2}

1 t bf t aF  bF 

af   

) ( 2 )

(  t   F   f

) ( )

( t t e ^ F 

f    ^{i  t}

) ( ) ) (

( i  F 

t t

f _n

n

n  





Differentiation theorem

 Time differentiation ( ) ( i  ) F (  ) t

t

f _n

n

n  





Convolution



 ^



















 ( ) ( ' ) ( ' ) ' ( ' ) ( ' ) ' )

( t g t f t g t t dt f t t g t dt

f

The convolution operation is at the heart of linear systems.

Definition:

Properties: f ( t )  g ( t )  g ( t )  f ( t ) )

( )

( )

( t t f t

f   





 H t f t dt t

f ( ) ( ) ( )

H(t) is the Heaviside function:

The convolution theorem

A convolution in the time domain corresponds to a multiplication in the frequency domain.

… and vice versa …

a convolution in the frequency domain corresponds to a multiplication in the time domain

) ( ) ( )

( )

( t g t F  G 

f  

) ( )

( )

( )

( t g t F  G 

f  

The first relation is of tremendous practical implication!

Summary

 The Fourier Transform can be derived from the problem of approximating an arbitrary function.

 A regular set of points allows exact interpolation (or derivation) of arbitrary functions

 There are other basis functions (e.g., Chebyshev polynomials, Legendre polynomials) with similar properties

 These properties are the basis for the success of the spectral

element method