Introduction to Digital Signal

Processing (DSP)

Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205

Some material adapted from courses by

Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner

1
 Books

Books:

–  J.G. Proakis and D.G. Manolakis, Digital Signal

Processing 3rd edition, Prentice-Hall.
–  R.G Lyons, Understanding Digital Signal
processing, 2nd edition, , Prentice-Hall.

2
 What is Digital Signal Processing?

Digital: operating by the use of discrete signals to

represent data in the form of numbers

Signal: a parameter (electrical quantity or effect) that can

be varied in such a way as to convey information

Processing: a series operations performed according to

programmed instructions

changing or analysing information

which is measured as discrete
sequences of numbers

3
 Applications of DSP - Biomedical

Biomedical: analysis of biomedical signals,

diagnosis, patient monitoring,
preventive health care, artificial
organs
Examples:

1) electrocardiogram (ECG) signal – provides

doctor with information about the condition of
the patient’s heart

2) electroencephalogram (EEG) signal – provides

Information about the activity of the brain
4
 Applications of DSP - Speech

Speech applications:

Examples
1) noise reduction – reducing background noise
in the sequence produced by a sensing device (microphone)

2) speech recognition – differentiating

between various speech sounds

3) synthesis of artificial speech – text to speech

systems for blind

5
 Applications of DSP - Communications

Communications:

Examples
1) telephony – transmission of information in digital form via
telephone lines, modem technology, mobile phones

2) encoding and decoding of the information

sent over a physical channel (to optimise
transmission or to detect or correct errors in
transmission)

6
 Applications of DSP - Radar

Radar and Sonar:

Examples

1) target detection – position and

velocity estimation

2) tracking

7
 Applications of DSP – Image Processing

Image Processing:

Examples
1) content based image retrieval – browsing,
searching and retrieving images from database

2) image enhancement

2) compression - reducing the redundancy in

the image data to optimise transmission /
storage
8
 Applications of DSP – Music

Music Applications:

Examples:

1) Recording

2) Playback

3) Manipulation (mixing, special effects)

9
 Applications of DSP - Multimedia

Multimedia:
generation storage and
transmission of sound, still
images, motion pictures

Examples:
1) digital TV

2) video conferencing
10
 DSP Implementation - Operations

To implement DSP we must be able to:

Input Digital Digital

Signal Signal
DSP
Output

1) perform numerical operations including, for

example, additions, multiplications, data transfers
and logical operations
either using computer or special-purpose hardware

11
 DSP chips

•  Introduction of the microprocessor in the late 1970's and

early 1980's meant DSP techniques could be used in a
much wider range of applications.
DSP chip – a programmable
device, with its own native
instruction code

designed specifically to meet

numerically-intensive
requirements of DSP

capable of carrying out

millions of floating point
Bluetooth Household Home theatre
headset appliances system operations per second 12
 DSP Implementation – Digital/Analog Conversion

To implement DSP we must be able to:

Digital Digital Analog

Signal Signal Signal
DSP Reconstruction

2) convert the digital information, after being processed

back to an analog signal
- involves digital-to-analog conversion & reconstruction
(recall from 1B Signal and Data Analysis)
e.g. text-to-speech signal (characters are used to generate artificial
sound)
13
 DSP Implementation –Analog/Digital Conversion

To implement DSP we must be able to:

Analog Digital Digital

Signal Signal Signal
Sampling DSP

3) convert analog signals into the digital information

- sampling & involves analog-to-digital conversion
(recall from 1B Signal and Data Analysis)
e.g. Touch-Tone system of telephone dialling (when button is
pushed two sinusoid signals are generated (tones) and
transmitted, a digital system determines the frequences and
uniquely identifies the button – digital (1 to 12) output
14
 DSP Implementation

To implement DSP we must be able to:

Analog Digital Digital Analog

Signal Signal Signal Signal
Sampling DSP Reconstruction

perform both A/D and D/A conversions

e.g. digital recording and playback of music (signal is sensed by

microphones, amplified, converted to digital, processed, and
converted back to analog to be played
15
 Limitations of DSP - Aliasing

Most signals are analog in nature, and have to be sampled

loss of information
•  we only take samples of the signals at intervals and
don’t know what happens in between
aliasing
cannot distinguish between
higher and lower frequencies
(recall from 1B Signal
and Data Analysis)

Sampling theorem: to avoid

aliasing, sampling rate must be
at least twice the maximum
frequency component
(`bandwidth’) of the signal16
Gjendemsjø, A. Aliasing Applet, Connexions, http://cnx.org/content/m11448/1.14
 Limitations of DSP - Antialias Filter

•  Sampling theorem says there is enough information

to reconstruct the signal, which does not mean
sampled signal looks like original one
correct reconstruction is not
just connecting samples with
straight lines

needs antialias filter (to filter out all

high frequency components before
sampling) and the same for
reconstruction – it does remove
information though
Each sample
is taken at a
slightly earlier (recall from 1B Signal
part of a cycle and Data Analysis)
17
 Limitations of DSP – Frequency Resolution

Most signals are analog in nature, and have to be sampled

loss of information

•  we only take samples for a limited period of time

limited frequency
resolution

does not pick up “relatively”

slow changes

(recall from 1B Signal

and Data Analysis)
18
 Limitations of DSP – Quantisation Error

Most signals are analog in nature, and have to be sampled

loss of information
•  limited (by the number of bits available) precision in data
storage and arithmetic

quantisation error

smoothly varying signal

represented by “stepped”
waveform

(recall from 1B Signal

and Data Analysis)
19
 Advantages of Digital over Analog Signal Processing

Why still do it?

•  Digital system can be simply reprogrammed for other

applications / ported to different hardware / duplicated
(Reconfiguring analog system means hadware redesign, testing, verification)
•  DSP provides better control of accuracy requirements
(Analog system depends on strict components tolerance, response may drift with
temperature)
•  Digital signals can be easily stored without deterioration
(Analog signals are not easily transportable and often can’t be processed off-line)
•  More sophisticated signal processing algorithms can be
implemented
(Difficult to perform precise mathematical operations in analog form)
20
Brief Review of Fourier Analysis

Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205

Some material adapted from courses by

Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner

21
 Time domain

Example: speech recognition

difficult to differentiate
between different sounds
in time domain
tiny segment

sound /i/
as in see

sound /a/
as in father
22
 How do we hear?

Cochlea – spiral of tissue with liquid

and thousands of tiny hairs that
Inner Ear gradually get smaller

Each hair is connected to the nerve

The longer hair resonate with lower

frequencies, the shorter hair
resonate with higher frequencies

Thus the time-domain air pressure

signal is transformed into frequency
spectrum, which is then processed
www.uptodate.com by the brain
Our ear is a Natural Fourier Transform Analyser! 23
 Fourier’s Discovery

Jean Baptiste Fourier showed that any

signal could be made up by adding together
a series of pure tones (sine wave) of
appropriate amplitude and phase
(Recall from 1A Maths)

Fourier Series
for periodic
square wave

infinitely large number

of sine waves is
required
24
xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
!
Fourier Transform
∞
he Discrete Time Fourier Transform
X (ω) = (DTFT)xn e a sampled
of −jnωT
. sequen
n }, n = 0, ±1, ±2, . . . , ±∞ is given byn=−∞

!
The Fourier transform
∞
is −jnωT
an equation to
X (ω) = xn e .
calculate the frequency,
n=−∞ amplitude and
phase of each sine wave "∞needed to
make up any X given
(ω)signal
= x(t)e: −jωt
dt.
"∞ −∞
X (ω) = x(t)e−jωt dt.
here T is the sampling interval.
−∞

here T is the(recall from 1B Signal

sampling
and Data Analysis)
interval.
a) Explain how the problems with computing the DT
a) Explain how the
computer canproblems with computing
be overcome the DTFT
by choosing a on a digit
uniform 25

computer can be overcome by choosing a uniformly spaced gr

 Prism Analogy

Analogy:
a prism which splits
white light into a
spectrum of colors
White Spectrum
light of colours
white light consists of all
frequencies mixed
together

Signal Fourier Spectrum

the prism breaks them
apart so we can see the
Transform
separate frequencies
26
 Signal Spectrum

Every signal has a frequency spectrum.

• the signal defines the spectrum
• the spectrum defines the signal

We can move back and forth between

the time domain and the frequency
domain without losing information
27
 Time domain / Frequency domain

•  Some signals are easier to visualise in the

frequency domain
•  Some signals are easier to visualise in the
time domain
•  Some signals are easier to define in the time
domain (amount of information needed)
•  Some signals are easier to define in the
frequency domain (amount of information
needed)

Fourier Transform is most useful

tool for DSP
28
 Fourier Transforms Examples

signal spectrum

cosine t ω

added higher
frequency t ω
component

Back to our sound recognition problem:

peaks correspond to
t the resonances of
sound /a/ ω the vocal tract shape
as in father in logarithmis units of dB

they can be used to

sound /i/ t differentiate between
as in see ω sounds
in logarithmis units of dB
29
n = 0, ±1, ±2, . . . , ±∞ isXgiven
(ω) =by xn e−jnωT1.!
.∞The X
Discrete
(ω) =Timex(t)e
Fourier Transform
dt. (DTFT) of a sampled sequen
{xn }, n =−jnωT ±2, . . . , ±∞ is given by
!
∞ n=−∞X (ω) = xn e 0, ±1,
−∞.
X (ω) = xn e−jnωT . n=−∞ !∞
.
Discrete
3.
n=−∞ Time Fourier Transform (DTFT) X (ω) = xn e−jnωT .
n=−∞
2. "∞ !
∞
X (ω) = x(t)e −jωt 2. xs (t) = x(t)
dt. δ(t − nT )
"∞
What about
"∞
sampled signal?
−∞ n=−∞ "∞
X (ω) = X dt.
x(t)e−jωt (ω) = x(t)e−jωt dt. X (ω) = x(t)e−jωt dt.
. 4. −∞
−∞ −∞
!
∞ 3. "∞
The DTFT
xs (t)is=defined
x(t) asδ(tthe Fourier
− nT ) Xs transform
(ω) = xsof
(t)ethe
−jωtsampled
dt !
3. signal. Define the sampled signal in the
n=−∞ usual
xs (t)way:
= x(t)
∞
δ(t − nT )
!∞ −∞
n=−∞
.
xs (t) = x(t) δ(t − nT ) !
∞
5.n=−∞ xs (t) = x(t)4. δ(t − nT )
"∞ n=−∞ "∞
!
∞ "∞
Take Fourier
Xs (ω)transform
= directly
xs (t)e Xdt
s (ω) =
−jωt
x(t) δ(t − nT
Xs (ω) = )e−jωt dt−jωt dt
xs (t)e
4. "∞ −∞ −∞ n=−∞ −∞

Xs (ω) = xs (t)e
−jωt
dt 1. The Discrete Time Fourier Transform (DTFT) of a sam
. 6.
−∞
"∞ {xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
5.
"∞
"∞ X
∞s (ω) =
! xs (t)e−jωt!
dt
∞
X (ω) = x(t)
!
∞
δ(t!
− nT )e−jωt
∞ dt
s
Xs (ω) = x(t) δ(t − nT
−∞ )e−jωt
Xs dt
(ω) = x(nT−∞)eX (ω) =
−jωnT
n=−∞ xn e
−jnωT
.
n=−∞ n=−∞ n=−∞
"∞ −∞
!∞
−jωt6.
5. Xs (ω) = x(t) δ(t − nT )e dt
where T is the sampling interval.
using 2.
−∞the “sifting
n=−∞ property of the δ-function to reach the
. ∞ last line
!
−jωnT
Xs (ω) = x(nT∞)e
(a)!
∞ "∞
Explain how !
the
∞ problems
with −jωt
computing the DTFT on 30
"
a digital
n=−∞
Xs (ω) =
X (ω) = x(nT )e−jωnT
x(t)can where
δ(t − nT )e dt X (ω) = x(t)e−jωt
dt.
s computer
n=−∞
be overcome by choosing a
T is the sampling interval. uniformly spaced grip
!
∞ of −∞ n=−∞covering the range of ω from 0 to 2π/T and
N frequencies
−∞
 4.
"∞
Discrete Time Fourier Transform
Xs (ω) = – Signal
xs (t)e−jωt dt Samples
−∞

5.
1. The Discrete Time Fourier "∞ Transform (DTFT) of a sampled sequence
Note that this expression known as DTFT ! is a periodic function of
∞
{xn }, n = 0, ±1, ±2,
X .
(ω). . ,=±∞ is given
x(t) byδ(t − nT )e−jωt dt
the frequency usually written as
s
n=−∞
−∞ !
∞
X (ω) = xn e−jnωT .
6. n=−∞
!
∞
The signal sample valuesX may
s (ω)be=expressedx(nT in)eterms
−jωnT of DTFT by
2.
noting that the equation above has n=−∞ the form of Fourier series (as a
function of ω) and hence the sampled "∞ signal can be obtained directly
7. X (ω) = x(t)e−jωt dt.
as
"2π
1 −∞
xn = X(ω)e+jnωT dωT
2π
3. 0

where T is the sampling interval.!

[You can show this for yourself by first noting
∞
that (*) is a complex Fourier
x (t)
series with coefficients however
s = x(t) δ(t − in
it is also covered nTone
) of Part IB
(a) Explain how the problems with
Examples Papers] n=−∞computing the DTFT on a digital
computer can be overcome by choosing a uniformly spaced 31
grip
4. of N frequencies covering the range of ω from 0 to 2π/T and
truncating the summation"∞ in the expression above to just N data
 Computing DTFT on Digital Computer

1. The Discrete Time Fourier Transform (DTFT) of a sampled sequence

{xn }, n = 0, ±1, ±2, . . . , ±∞ is given by
The DTFT
!
∞
X (ω) = xn e−jnωT .
n=−∞

2. expresses the spectrum of a sampled signal in terms of

the signal samples but is not
"∞ computable on a digital
X (ω)
computer for two reasons:= x(t)e−jωt
dt.
−∞

3. 1.  The frequency variable ω is continuous.

2.  The summation involves an ∞ infinite number of
!
samples. xs (t) = x(t) δ(t − nT )
n=−∞

4. 32

"∞
 Overcoming problems with computing DTFT

The problems with computing DTFT on a digital

computer can be overcome by:

Step 1. Evaluating the DTFT at a finite

collection of discrete frequencies.

no undesirable consequences, any

frequency of interest can always be
included in the collection

Step 2. Performing the summation over a

finite number of data points

does have consequences since

signals are generally not of finite duration
33
 ofcan
ofof discrete
discrete
discrete frequencies
frequencies
frequencies withoutany
without
by without any
any undesirable
undesirable
undesirable consequences;
aconsequences;
consequences; sec-
sec-
italbecomputer:
overcome first, evaluating
the frequency the DTFT
variable ω isatcontinuous whichsec-
finite collection
ond,
ond,
ond, thebe
the
the
of can
discrete summation
summation
summation frequencies
overcome involves
involves
involves
by without
evaluating infinite
infinite
infiniteany number
number
number
theundesirable
DTFT of
atofof samples
samples
asamples
consequences;
finite whichsec-
which
which
collection can can
can
problems with calculating bebe
be ond,overcome
overcomeDTFT
of discrete
overcome
the by by
by performing
onperforming
a dig-
performing
frequencies
summation without
involves the the
the any summation
summation
undesirable
summation
infinite numberover over
over aaafinite
consequences;
of samples finite
finitewhich number
number
sec-
number can
frequency variable ofω
ofofbe
The
data
is
ond,
data
data
Discrete
points,
continuous
the summation
points,
overcome
points, which
which
which
by
which
Fourier
performing doeshave
involves
does
does have
have the
Transform
does
infinite
does
doessummation haveconsequences
number
have
have
(DFT)
consequences
ofover
samples
consequences which
a finite since
cansig-
since
number
since sig-
sig-
aluating the DTFT nalsofatbe
nals
nals aovercome
are
are
data
are generally
finite
generally
points,
generally by
collectionperforming
notofof
not
which
not offinite
does finite
finite havetheduration.
summation
duration.
does
duration. have over
IfIfIfwewechoose
we a finite aanumber
choose
choose
consequences uniformly
uniformly
since
a uniformly sig-
without any undesirable spaced
spaced ofconsequences;
datagrip
grip points,
of
of NN which
sec- does havecovering
frequencies
frequencies does havethe
covering consequences
the range
range of
of ωsince
ω from
from sig- 00 to to
nals
spaced are
grip generally
of N not
frequencies of finite duration.
covering the If we
range choose of ω a uniformly
from 0 to
olves infinite number
The 2π/T2π/T
2π/T ofnals
discrete
spaced and
and
and
are truncate
samples
set gripof
generally
which
frequencies
truncate
truncate of N
not
the
can
the
the
of finite duration.
summation
chosen
summation
frequencies
summation is
coveringin in
arbitrary.
in the
the
the
If we
the
choose a uniformly
expression
However,
expression
range
expression of above
since
above
ω
above thetoto
from to just
just
0justto
spaced grip of N frequencies covering the range of ω from 0 to
ming the summation DTFT over
Nis2π/T
periodicapoints
datapoints finite
points number
we
we generally get the
choose DFT: a uniformly spaced grid of N to just
NN 2π/Tand
data
data andtruncatewewe
truncate get
get the the
the
the summation
DFT:
DFT:
summation ininthetheexpression
expression above
above to just
frequencies
oes have does have consequences datacovering the
since range
sig- ωT from 0 to 2π. If the summation is then
NN datapoints points weweget get thetheNNDFT:
DFT:
f finite duration. truncated
If we to just Nadata
choose points
2π
uniformly
2π weN! !!
get
−1−1the DFT
XX =
= X(
X( 2π p)
p) ==
−1
x
x ee −j
−j 2π2π2π
Nnp
np
np
,,ppp== =0,0,0,. ....,..,N
,NN−− −111
encies covering the range of ω fromNN X p pp= X( N p) =
02πto n=0
2π NN!
!
−1
−1x n e
n
n
−j N N
,
−j2π2π npnp
summation in the expression above to N X X = X(
p p = X(
justp)p) == n=0 x e
n=0 xnne NN , ,p p==
−j
0,0,. . . ., N
.,N − 1− 1
N n=0
n=0
he DFT: (c) inverse DFT can be used to obtain the sampled signal values from the
(c)
The
(c)
DFT:(c)multiply
(c) Assume
Assume each you side
you are byinterested
are and sum
interested in
in over
detecting
detecting p=0 tothe N-1presence
the presence of ofa single
a single
N!−1
N!
N
!
−1
continuous-wave NN
!! −1 NN
!!−1
sinusoidal tone 2πin a time-domain N!
N
!
−1
−1 data NN
!! −1
sequence.
−j 2π ! ! ! ! !
np N −1 N −1 N −1 N N −1 2π
= xn e N continuous-wave
−1
, p = 0, . .X X. ,peN
jejNN−
j
2π
2π
2π
pqpq1 =
pq
= sinusoidal
−1 −1
x etone
−j−j 2π
2π in
np a
np
Nnp jej N
N j time-domain
2π
2π
2π pq
pq =
−1
x data
−1 sequence.
jj2π
2π
jeN N
(q−n)p
(q−n)p
(q−n)p
64-pointxxnDFT enep p of enN} can ==bebecomputedxx eep pat at
N −j p Npq p N
n=0 Show Xpphow
Show e howthe = 64-point
the DFT
n N
of e{x
{x } can nn
n
computed the the
p=0
p=0 p=0
p=0of n=0
n=0 n n=0
n=0of {x p=0
p=0
p=0 discrete frequency p=0 n=0 30kHz if the sampling rate n=0 n } was fs =
p=0
discrete frequency of 30kHz if the sampling rate of {x n } was fs =
128kHz. Determine the total number of complex multiplications
(d)Assume
(d) 128kHz.
Assume
Assume you
you Determine
areof
are the total
interested
interested number
indetecting
detecting of thecomplex
thepresence
presence multiplications
ofaaasinglesingle
Orthogonality
(d) property
you
required. are
[20%]. complex exponentials
interested inin detecting the presence ofof single
required.
continuous-wave
continuous-wave −1[20%]. sinusoidal
sinusoidal tone
tone in= aatime-domain
time-domain kHz data sequence.
!
−1 2π
−j N np j pq2π !
N
continuous-wave
Answer.
−1 N! 30j 2π sinusoidal
kHz
(q−n)p corresponds tone toin pin a time-domain
15 since 15 128128 64
data
data = sequence.
sequence.
30kHz
kHz = 30kHz
xn ep e N Show =Show (ax
Answer.
Show how
how
uniformly
n
how 30 e
the
the
the
N
pkHz corresponds
64-point
64-point
spaced
64-point is
gripN if
DFT
DFT
DFTof n=q
N to
=
of and
of
of64 p
{x{x=
{x 0}n15
}}
frequencies since
otherwise
can
can
can be be
be 15 computed
computed
covering
computed64 the range at atthe
at the
the
nn
0 (a
n=0uniformly
discrete p=0
frequency 0spaced grip
of30kHz
30kHz of N =the64 frequencies covering the range
of
discrete ω from
frequency
discrete frequency to 2π/T
ofof 30kHz ifififthe
is considered, the sampling
where fsrate
sampling
sampling rate
=
rate 1/T of{x
ofof ={x128kHz
{xn }
nn}} was
was
was fsfss==
f)34 =
of and
128kHz.
128kHz. ω fromone 0 to 2π/Tthe
obtains
Determine
Determine theis total
considered,
totalnumber number
number whereofcomplex
of complex
complex = 128kHz )
fs = 1/Tmultiplications
multiplications
sted in detecting theand 128kHz.presence Determine of a single the total !63 of multiplications
required.
required. one obtains
[20%].
[20%]. −j 2π
required. [20%]. X = x e 64 n15 ,
The Discrete Fourier Transform Pair

35
uncate the summation inN
N!−1 the expression above to just
n=0 2π N data points we get the n=0 DFT:
−j N np
ts wexget
n e the DFT:
, p = 0, . . . , N − 1 N −1
n=0 2π 2π !N! −1 −j 2π np
) 2π Properties !
N −1 X p = X(
X2π
p =ofNthe
X( p) =
Discrete
p) = xnxen e−j N , ,pp=
N 2π np
= 0,
0, . .. ..,,NN−−1 1
Fourier Transform (DFT)
.
= X( p) = N
xn eN−j−1N np , Np−1
= 0,n=0
. . .n=0
,N −1
N ! N −1!
! ! ! ! !
2π
−jN −1np j 2πn=0
pq N −1j 2π (q−n)p2π N −1 N −1 2π
xn ep −j 2π npe 2π j N=pq! xn! j 2πep(q−n)p N
1 (c)N
(c)N 2π N −1 N −1
N
−j np j 2π
pq j (q−n)p
xn ep N X ejpNepqN = = N xn ep e = N −1 xn ep
N N
n=0 x n e
p=0p
−1 ! −1 ! −1!
N −1 N −1 !
N −1 !
p=0 N! ! ! ! !
−1 N N 2π N −1 N −1 j 2π2π
n=0 j 2π p=0
pqp=0 n=0 2π N np 2π
−j j 2π pq n=0 (q−n)p
p=0
jX2π
pqe N = x−je np je Npq = x e j
N (q−n)p
Xp+N !
N −1=!
N X
−1
p
Xp=0e 2π np =2π p=0N!
p−j N p −1xn epN!
n p
−1 e2π
N N = n=0 xn p=0 ep
n p N

= p+N•  =p=0
Xis
p nperiodic,
ep N eforNp=0 each
pq X j pq p x
n=0 j (q−n)p
x =n=0 n e p
N
Xp+N = Xp n=0 p=0
p=0 n=0 n=0 Xp=0
p+N = Xp
Xp+N = Xp
) • =(d) is X p+N = X
xn+N
x(d)
n+N =xx periodic,
n
forp each n
xn+N = xn
(e)realx data = x
xxn+N
n+N = =
xn
xn
•  for
∗ n+N n
X
) p =(e)
X∗−p = XN∗ −p
∗
= X−p = XN −p Xp = X−p
∗
= XN∗ −p
ted in detecting the presence ofXap single
= X ∗ ∗
= X ∗
d in [You(f)
detecting
X should
Assume
= the
X ∗ check
you that
are
presence
= X ∗ you
Xofp can
interested
a= Xshow
in
single
−p these
=
detectingNX results
−pthe
N from first principles]
∗ presence of a single
dal tone in a ptime-domain
continuous-wave
−p data sequence.
N −psinusoidal −p −p
tone in a time-domain data sequence.
lDFT
tone
(f) ofin{xa ntime-domain
Assume youhow
can
}Show beare
computed
the data sequence.
interested
64-point in
thedetecting
at DFT of {xn } can the be
presence
computed of at
a single
the
) Assume
DFT
kHzare interested
if of
the {x n}
you
sampling canare
in
continuous-wave rate
discrete interested
detecting
be of the
computed
{xn }ofwas
sinusoidal
frequency
presence
30kHz in
sat
ftone=if thedetecting
of
in
the
a single
asampling
time-domaintheof {x
rate presence
data
n } of
sequence.
was fs = a single
wave sinusoidal tone in a time-domain data sequence.
continuous-wave
Hztotal number
if the Show of
how
sampling complex
128kHz. the
rate sinusoidal
multiplications
64-point
of
Determine }DFT
{xnthe was tone
total inn }aofcan
fofsnumber
={x time-domain
be computed
complex data 36sequence.
at the
multiplications
he 64-point DFT of {x
required. n } can be computed at the
[20%].
otalShow discrete
number how of frequency
the ofmultiplications
64-point
complex 30kHzDFT if theofsampling{x n } rate
can ofbe } was fs = at the
{xncomputed
uency of 30kHz if the sampling 128kHz rate of {xn } was fs = 128kHz
DFT Interpolation

37
Zero padding

38
Padded sequence

39
Zero-padding

40
 Zero-padding

just visualisation, not additional information! 41

Circular Convolution

circular convolution

42
 Example of Circular Convolution

Circular convolution of x1={1,2,0} and x2={3,5,4}

clock-wise anticlock-wise
1

3
5 4
0 2 folded sequence

y(0)=1×3+2×4+0×5 y(1)=1×5+2×3+0×4 y(2)=1×4+2×5+0×3

1 1 1

3 5 4
0 spins 1 spin 2 spins
…
5 4 4 3 3 5
0 2 0 2 0 2
x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3 43
Example of Circular Convolution

clock-wise anticlock-wise

44
Example of Circular Convolution

45
Standard Convolution using Circular Convolution

46
 Example of Circular Convolution

clock-wise anticlock-wise
1

3
0 2 5 0
4
folded sequence

y(0)=1×3+2×0+0×4+0×5 y(0)=1×5+2×3+0×0+0×4 0 y(0)=1×4+2×5+0×3+0×0

1 1 1

3 5 4
0 spins 1 spin 2 spins
0 5 0 2 0 4 3 2 0 0 5 2
4 0 3
…
x1(n)x2(0-n)|mod3 x1(n)x2(1-n)|mod3 x1(n)x2(2-n)|mod3 47
0 0 0
Standard Convolution using Circular Convolution

48
 Proof of Validity

Circular convolution of the padded sequence corresponds to the standard

convolution

49
 Linear Filtering using the DFT

FIR filter:

Frequency domain equivalent:

DFT and then IDFT can be used to compute standard convolution

product and thus to perform linear filtering.

50
 Summary So Far

•  Fourier analysis for periodic functions focuses on the

study of Fourier series

•  The Fourier Transform (FT) is a way of transforming

a continuous signal into the frequency domain

•  The Discrete Time Fourier Transform (DTFT) is a

Fourier Transform of a sampled signal

•  The Discrete Fourier Transform (DFT) is a discrete

numerical equivalent using sums instead of integrals
that can be computed on a digital computer

•  As one of the applications DFT and then Inverse DFT

(IDFT) can be used to compute standard convolution
product and thus to perform linear filtering 51
Fast Fourier Transform (FFT)

Elena Punskaya
www-sigproc.eng.cam.ac.uk/~op205

Some material adapted from courses by

Prof. Simon Godsill, Dr. Arnaud Doucet,
Dr. Malcolm Macleod and Prof. Peter Rayner

52
 Computations for Evaluating the DFT

•  Discrete Fourier Transform and its Inverse can be

computed on a digital computer

•  What are computational requirements?

53
 Operation Count

How do we evaluate computational complexity?

count the number of real multiplications and
additions
by ¨real¨ we mean either fixed-point or floating point
operations depending on the specific hardware

subtraction is considered equivalent to additions

divisions are counted separately

Other operations such as loading from memory, storing in

memory, loop counting, indexing, etc are not counted
(depends on implementation/architecture) – present
overhead 54
 Operation Count – Modern DSP

Modern DSP:
a real multiplication and addition is a single machine
cycle y=ax+b called MAC (multiply/accumulate)

maximum number of multiplications and additions must

be counted, not their sum

traditional claim ¨multiplication is much more time-

consuming than addition¨ becomes obsolete

55
 Discrete Fourier Transform as a Vector Operation

Again, we have a sequence of sampled values x n and

transformed values Xp , given by

Let us denote:

Then:

56
 Matrix Interpretation of the DFT

This Equation can be rewritten in matrix form:

complex
as follows: numbers
can be
X0 0 0 0
… 0
x0 computed
X1 0 1 2
…
N-1
x1 off-line
= and
… … …
…
…
…

…
stored
N-1 2(N-1) (N-1) 2
XN-1 0
… x N-1

N x N matrix 57
 DFT Evaluation – Operation Count

Multiplication of a complex N x N matrix by a complex

N-dimensional vector.

No attempt to save operations

N2 complex multiplications (¨x¨s)
N(N-1) complex additions (¨+¨s)

First row and first column are 1 however

2
save 2N-1 ¨x¨s, (N-1) complex ¨x¨s & N(N-1) ¨+¨s

Complex ¨x¨: 4 real ¨x¨s and 2 real ¨+¨s;

complex ¨+¨ : 2 real ¨+¨s
2
Total: 4(N-1) real ¨x¨s & 4(N-0.5)(N-1) real ¨+¨s 58
 DFT Computational Complexity

Thus, for the input sequence of length N the number of

arithmetic operations in direct computation of DFT is
proportional to N 2.

For N=1000, about a million operations are needed!

In 1960s such a number was considered prohibitive

in most applications.

59
 Discovery of the Fast Fourier Transform (FFT)

When in 1965 Cooley and Tukey ¨first¨ announced

discovery of Fast Fourier Transform (FFT) in 1965 it
revolutionised Digital Signal Processing.

They were actually 150 years late – the principle of the

FFT was later discovered in obscure section of one of
Gauss’ (as in Gaussian) own notebooks in 1806.

Their paper is the most cited mathematical paper ever

written.

60
 The Fast Fourier Transform (FFT)

The FFT is a highly elegant and efficient algorithm,

which is still one of the most used algorithms in speech
processing, communications, frequency estimation, etc
– one of the most highly developed area of DSP.

There are many different types and variations.

They are just computational schemes for computing the

DFT – not new transforms!

Here we consider the most basic radix-2 algorithm

which requires N to be a power of 2.
61
 FFT Derivation 1

Lets take the basic DFT equation:

Now, split the summation into two parts: one for even n and one for odd n:

Take
outside

62
 FFT Derivation 2

Thus, we have:

63
 FFT Derivation 3

Now, take the same equation again where we split the summation
into two parts: one for even n and one for odd n:

And evaluate at frequencies p+N/2

64
FFT Derivation 4

simplified

65
FFT Derivation 5

66
 FFT Derivation - Butterfly

Or, in more compact form:

(‘Butterfly’)

67
FFT Derivation - Redundancy

68
FFT Derivation - Computational Load

69
 Flow Diagram for a N=8 DFT

Input: Output:

70
Flow Diagram for a N=8 DFT - Decomposition

71
Flow Diagram for a N=8 DFT – Decomposition 2

72
Bit-reversal

73
 FFT Derivation Summary

•  The FFT derivation relies on redundancy in the calculation

of the basic DFT
•  A recursive algorithm is derived that repeatedly rearranges
the problem into two simpler problems of half the size
•  Hence the basic algorithm operates on signals of length a
power of 2, i.e.

(for some integer M)

•  At the bottom of the tree

we have the classic FFT
`butterfly’ structure

74
 Computational Load of full FFT algorithm

The type of FFT we have considered, where N = 2M, is

called a radix-2 FFT.

It has M = log2 N stages, each using N / 2 butterflies

Complex multiplication requires 4 real multiplications

and 2 real additions

Complex addition/subtraction requires 2 real additions

Thus, butterfly requires 10 real operations.

Hence the radix-2 N-point FFT requires 10( N / 2 )log2 N

real operations compared to about 8N2 real operations
75
for the DFT.
 Computational Load of full FFT algorithm

The radix-2 N-point FFT requires 10( N / 2 )log2 N real

operations compared to about 8N2 real operations for the
DFT.

This is a huge speed-up in typical applications, where N is

128 – 4096:

Direct DFT

FFT

76
 Further advantage of the FFT algorithm

Input Output

77
 The Inverse FFT (IFFT)

The IDFT is different from the DFT:

•  it uses positive power of instead of negative ones

•  There is an additional division of each output value by N

Any FFT algorithm can be modified to perform the IDFT by

•  using positive powers instead of negatives
•  Multiplying each component of the output by 1 / N

Hence the algorithm is the same but computational load

increases due to N extra multiplications 78
 Many types of FFT
The form of FFT we have described is called “decimation in time”; there
is a form called “decimation in frequency” (but it has no advantages).

The "radix 2" FFT must have length N a power of 2. Slightly more efficient
is the "radix 4" FFT, in which 2-input 2-output butterflies are replaced by 4-
input 4-output units. The transform length must then be a power of 4
(more restrictive).

A completely different type of algorithm, the Winograd Fourier Transform

Algorithm (WFTA), can be used for FFT lengths equal to the product of a
number of mutually prime factors (e.g. 9*7*5 = 315 or 5*16 = 80). The
WFTA uses fewer multipliers, but more adders, than a similar-length FFT.

Efficient algorithms exist for FFTing real (not complex) data at about 60%
the effort of the same-sized complex-data FFT.

The Discrete Cosine and Sine Transforms (DCT and DST) are
79
similar real-signal algorithms used in image coding.
 Applications of the FFT
There FFT is surely the most widely used signal processing
algorithm of all. It is the basic building block for a large
percentage of algorithms in current usage

Specific examples include:

•  Spectrum analysis – used for analysing and detecting
signals
•  Coding – audio and speech signals are often coded in the
frequency domain using FFT variants (MP3, …)
•  Another recent application is in a modulation scheme called
OFDM, which is used for digital TV broadcasting (DVB) and
digital radio (audio) broadcasting (DAB).
•  Background noise reduction for mobile telephony, speech
and audio signals is often implemented in the frequency
domain using FFTs 80
 Practical Spectral Analysis

•  Say, we have a symphony recording over 40 minutes

long – about 2500 seconds
•  Compact-disk recordings are sampled at 44.1 kHz and
are in stereo, but say we combine both channels into a
single one by summing them at each time point
•  Are we to compute the DFT of a sequence one hundred
million samples long?
a)  unrealistic – speed and storage
requirements are too high
b)  useless – we will get a very much noiselike
wide-band spectrum covering the range from 20
Hz to 20 kHz at high resolution including all
notes of all instruments with their harmonics 81
 Short-Time Spectral Analysis

•  We actually want a sequence of short DFTs, each

showing the spectrum of a signal of relatively short
interval
•  If violin plays note E during this interval – spectrum
would exhibit energies at not the frequency of note E
(329 Hz) and characteristic harmonics of the violin
•  If the interval is short enough – we may be able to
track not-to-note changes
•  If the frequency resolution is high enough – we will be
able to identify musical instruments playing

Remember: our – ear – excellent spectrum analyser?

82
Short-Time Spectral Analysis – Computational Load

•  If we take a time interval of about 93 milliseconds

(signal of length 4096) the frequency resolution will
be 11 Hz (adequate)
•  The number of distinct intervals in the symphony is
40 x 60/0.093 - about 26,000
•  We may choose to be conservative and overlap the
intervals (say 50%)
•  In total, need to compute 52,000 FFTs of length 4096

the entire computation will be done in

considerably less time than the music
itself
Short-time spectral analysis also use windowing (more on
this later) 83
 Case Study: Spectral analysis of a Musical Signal

Sample rate is
10.025 kHz
(T=1/10,025 s)

Extract a short segment

Load this into Matlab as a vector x

Take an FFT, N=512

X=fft(x(1:512));
Looks almost
periodic over short time 84
interval
Case Study: Spectral analysis of a Musical Signal, FFT

Note Conjugate symmetry

Symmetric as data are real:

Symmetric Anti-Symmetric

85
Case Study: Spectral analysis of a Musical Signal - Frequencies

86
 The Effect of data length N

Low
N=32 resolution
FFT

N=128 FFT

N=1024 FFT High

resolution

87
 The DFT approximation to the DTFT

DTFT at frequency : DFT:

•  Ideally the DFT should be a `good’ approximation to the DTFT

•  Intuitively the approximation gets better as the number of data

points N increases

•  This is illustrated in the previous slide – resolution gets better as

N increases (more, narrower, peaks in spectrum).

•  How to evaluate this analytically?

View the truncation in the summation as a multiplication by a
rectangle window function
88
Analysis

89
Pre-multiplying by rectangular window

90
Spectrum of the windowed signal

91
Fourier Transform of the Rectangular Window

92
 Rectangular Window Spectrum

N=32 Central `Lobe’

Sidelobes

93
 Rectangular Window Spectrum – Lobe Width

N=32

N=16

N=8

N=4

Lobe width inversely

Proportional to N
94
Now, imagine what happens when the sum of two frequency components is DFT-ed:

The DTFT is given by a train of delta functions:

Hence the windowed spectrum is just the convolution of the window spectrum with
the delta functions:

95
 DFT for the data

Both components
separately

Both components
Together

96
ωΤ
 Brief summary

•  The rectangular window introduces broadening of any frequency

components (`smearing’) and sidelobes that may overlap with other
frequency components (`leakage’).
•  The effect improves as N increases
•  However, the rectangle window has poor properties and better choices
of wn can lead to better spectral properties (less leakage, in particular)
– i.e. instead of just truncating the summation, we can pre-multiply by a
suitable window function wn that has better frequency domain
properties.
•  More on window design in the filter design section of the course – see
later

97
 Summary so far …
•  The Fourier Transform (FT) is a way of transforming
a continuous signal into the frequency domain

•  The Discrete Time Fourier Transform (DTFT) is a

Fourier Transform of a sampled signal

•  The Discrete Fourier Transform (DFT) is a discrete

numerical equivalent using sums instead of integrals
that can be computed on a digital computer

•  The FFT is a highly elegant and efficient algorithm,

which is still one of the most used algorithms in
speech processing, communications, frequency
estimation, etc – one of the most highly developed
area of DSP
98
Thank you!

99