Digital Communications

Signal - Space Analysis

Gram – Schmidt orthogonalization
Geometric Representation of Signals
Coherent Detection of Signals in Noise
Correlation Receiver
Probability of Error

Universidade do Minho
Escola de Engenharia
 Signal - Space Analysis
• Gram – Schmidt orthogonalization

– The Gram–Schmidt Process is an algorithm to obtain an

orthogonal (or orthonormal) base from any base

– The Gram–Schmidt process receives a finite set, linearly

independent of vectors B = {v1, …, vn} and returns an
orthogonal set B’ = {w1, …, wn} that generates the same
initial subspace S

Universidade do Minho
Escola de Engenharia 2
 Signal - Space Analysis
• Gram – Schmidt orthogonalization
– The projection operator is defined by:

– This operator projects the vector v orthogonally onto

the line generated by vector u

Universidade do Minho
Escola de Engenharia 3
 Signal - Space Analysis
• Gram – Schmidt orthogonalization

Universidade do Minho
Escola de Engenharia 4
 Signal - Space Analysis
• Gram – Schmidt orthogonalization

Universidade do Minho
Escola de Engenharia 5
 Signal - Space Analysis
• Gram – Schmidt orthogonalization
– If B’ = {w1, …, wn} is an orthogonal base of a space with
an inner product V,

– is an orthonormal base of the same subspace

– The Gram-Schmidt Theorem guarantees the existence of
an infinity of orthonormal bases

Universidade do Minho
Escola de Engenharia 6
 Signal - Space Analysis

Universidade do Minho
Escola de Engenharia 7
 Signal - Space Analysis
• Gram – Schmidt orthogonalization (Example)
– Consider the vectors:

– The space generated by {V1, V2 and V3} is all space R3,

as we have three linearly independent vectors in a
three-dimensional space:
• Check if they are linearly independent
• Check if they are orthogonal

Universidade do Minho
Escola de Engenharia 8
 Signal - Space Analysis
• Gram – Schmidt orthogonalization (Example)

Universidade do Minho
Escola de Engenharia 9
 Signal - Space Analysis
• Gram – Schmidt orthogonalization (Example)
– We conclude that the set of vectors:

– is orthogonal and generates the same space as:

Universidade do Minho
Escola de Engenharia 10
 Signal - Space Analysis
• Gram – Schmidt orthogonalization (Example)
– We also observed that, by normalizing the vectors,
we can obtain an orthonormal basis:

Universidade do Minho
Escola de Engenharia 11
 Signal - Space Analysis
• Geometric representation of signals
– An N-dimensional orthogonal space is characterized
by a set of N linearly independent functions, the so-
called basis functions, {ψj(t)}
– Any arbitrary function, si(t), in this space can be
generated through a linear combination of the basis
functions:

Universidade do Minho
Escola de Engenharia 12
 Signal - Space Analysis
• Geometric representation of signals
– The coefficients are given by:

– The basis functions must satisfy the condition:

• Orthonormal space => Kj = 1

• Orthogonal space => Kj ≠ 0
Universidade do Minho
Escola de Engenharia 13
 Signal - Space Analysis
• Geometric representation of signals
– Example of orthogonal space with N = 3

Universidade do Minho
Escola de Engenharia 14
 Signal - Space Analysis
• Geometric representation of signals
– The signal si(t) is represented by the vector:
– si = [s1 , s2 ,…, sN]T
– The square of the length, or norm, of the vector is
defined as:

– It is the inner product of the vector with itself

Universidade do Minho
Escola de Engenharia 15
 Signal - Space Analysis
• Geometric representation of signals
– Correspondence between the energy of a signal and
its vector representation:

Universidade do Minho
Escola de Engenharia 16
 Signal - Space Analysis
• Geometric representation of signals
– This expression is simplified a lot since:

– and the energy is given by:

– The energy of a signal is equal to the square norm of

the vector that represents it
Universidade do Minho
Escola de Engenharia 17
 Signal - Space Analysis
• Geometric representation of signals
– Euclidean distance between two points:

– The energy of the difference of two signals is equal to

the squared distance between the corresponding
points in space
Universidade do Minho
Escola de Engenharia 18
 Signal - Space Analysis
• Geometric representation of signals
– Correlation between two signals is their inner
product:

– Angle between two signal vectors:

Universidade do Minho
Escola de Engenharia 19
 Signal - Space Analysis
• Geometric representation of signals
– How to get the elements of vector si from si(t) ?

Analyzer
Universidade do Minho
Escola de Engenharia 20
 Signal - Space Analysis
• Geometric representation of signals
– How to get si(t) from the elements of vector si ?

Synthesizer
Universidade do Minho
Escola de Engenharia 21
 Signal - Space Analysis
• Geometric representation of signals
– Example of a set of arbitrary signals defined as
function of orthogonal signals

Universidade do Minho
Escola de Engenharia 22
 Signal - Space Analysis
• Geometric representation of signals
– The set {si(t)} is not orthogonal because (example for
s1(t) and s2(t)):

– to be orthogonal it would have to be:

Universidade do Minho
Escola de Engenharia 23
 Signal - Space Analysis
• Geometric representation of signals
– The set {ψi(t)} is orthogonal because:

– We can express the non-orthogonal set {si(t)}; i = 1,2,3

– as function of the basis waveforms {ψi(t)}; j = 1,2

Universidade do Minho
Escola de Engenharia 24
 Signal - Space Analysis
• Geometric representation of signals
– How to obtain the basis functions (Gram-Schmidt
orthogonalization)?
– From M energy signals si(t), represented by si vectors,
we can obtain N ≤ M basis functions ψi(t) proceeding
as follows:
1. One of the signals is arbitrarily chosen. Let s1(t). The
first basis function is defined like this:

Universidade do Minho
Escola de Engenharia 25
 Signal - Space Analysis
• Geometric representation of signals
2. The first element of s2 is by definition:
 Projection of s2 over the first basis function

3. An intermediate function g2(t) is defined:

4. The second basis function is defined as function of

g2(t) and its energy:

Universidade do Minho
Escola de Engenharia 26
 Signal - Space Analysis
• Geometric representation of signals
– Replacing g2(t):

5. Then:

Universidade do Minho
Escola de Engenharia 27
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – (Notes)
1. This definition implies that:

2. At this point we already have s21. The other

components of s2 are included in the intermediate
function g2(t). Note that g2(t) is orthogonal to ψ1(t)
in the range 0 to T:

Universidade do Minho
Escola de Engenharia 28
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – (Notes)
3. It can be confirmed that:

and
4. This definition encompasses particular cases. For
example:

Universidade do Minho
Escola de Engenharia 29
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – (Notes)
5. The number of basis functions, N, is less than or
equal to the number of signals, M:
 If M signals are linearly independent: N = M
 If they are not linearly independent, then N < M and
the intermediate function gi(t) is null for i > N

Universidade do Minho
Escola de Engenharia 30
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – An example
– What is the orthonormal basis for this set of signals?

Universidade do Minho
Escola de Engenharia 31
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – An example
– 1st basis function:

Universidade do Minho
Escola de Engenharia 32
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – An example
– 2nd basis function:
– By definition:
– Energy of s2(t):

Universidade do Minho
Escola de Engenharia 33
 Signal - Space Analysis
• Geometric representation of signals
– Gram-Schmidt orthogonalization – An example
– 3rd basis function:

Universidade do Minho
Escola de Engenharia 34
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions – 3D

Universidade do Minho
Escola de Engenharia 35
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions - 2D
– Consider the 2-dimensional signal space

Universidade do Minho
Escola de Engenharia 36
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions - 2D
– The two-dimensional signal space is the locus of the
noise-perturbed binary vectors (s1 + n) and (s2 + n)
– The noise vector, n, is a zero-mean random vector
– The received signal vector, r, is a random vector with
mean s1 or s2
– The detector's task after receiving r is to decide which
of the signals (s1 or s2) was actually transmitted

Universidade do Minho
Escola de Engenharia 37
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions - 2D
– The method is usually to decide on the signal
classification that yields the minimum expected PE
although other strategies are possible
– For the case where M equals 2, with s1 and s2 being
equally likely and with the noise being an AWGN
process, we will see that the minimum-error decision
rule is equivalent to choosing the signal class such that
the distance d(r,si) = IIr - siII is minimized. where II x II is
called the norm or magnitude of vector x
Universidade do Minho
Escola de Engenharia 38
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions - 2D
– Let us construct decision regions in the following way:
• Draw a line connecting the tips of the prototype vectors s1
and s2
• Construct the perpendicular bisector of the connecting line
– Notice that this bisector passes through the origin of the space if s1
and s2 are equal in amplitude
• For this M = 2 example, the constructed perpendicular
bisector represents the locus of points equidistant between s1
and s2; hence the bisector describes the boundary between
decision region 1 and decision region 2
Universidade do Minho
Escola de Engenharia 39
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Decision regions – 3D
– The decision rule for the detector, stated in terms of
decision regions, is as follows:
• Whenever the received signal r is located in region 1,
choose signal s1; when it is located in region 2, choose
signal s2
• If the angle θ equals 180°then the signal set s1 or s2
represents BPSK

Universidade do Minho
Escola de Engenharia 40
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver (Introduction)
• We focus particularly on that realization of a matched filter
known as a correlator
• In addition to binary detection, we also consider the more
general case of M-ary detection
• We assume that the only performance degradation is due
to AWGN. The received signal is the sum of the transmitted
prototype signal plus the random noise:

Universidade do Minho
Escola de Engenharia 41
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver

Universidade do Minho
Escola de Engenharia 42
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver
• A correlation receiver comprised of M correlators
transforms a received waveform, r(t), to a sequence of M
numbers or correlator outputs, zi(T) (i = 1, ... , M)
• Each correlator output is characterized by the following
product integration or correlation with the received signal:

Universidade do Minho
Escola de Engenharia 43
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver
• The verb "to correlate" means "to match". The correlators
attempt to match the incoming received signal, r(t), with
each of the candidate prototype waveforms si(t), known a
priori to the receiver
• A reasonable decision rule is to choose the waveform, si(t),
that matches best or as the largest correlation with r(t)
• In other words, the decision rule is:
– Choose the si(t) whose index corresponds to the max zi(T)

Universidade do Minho
Escola de Engenharia 44
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver
– As we saw, any signal set, {si(t)} (i = 1, ... , M), can be
expressed in terms of some set of basis functions,
{ψj(t)} (j = 1, ... , N), where N ≤ M
– Then the bank of M correlators in the previous figure
may be replaced with a bank of N correlators, shown
in next figure, where the set of basis functions {ψj(t)}
form reference signals

Universidade do Minho
Escola de Engenharia 45
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver

Universidade do Minho
Escola de Engenharia 46
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver
– The decision stage of this receiver consists of logic
circuitry for choosing the signal si(t)
– The choice of si(t) is made according to the best
match of the coefficients, aij
– When the prototype waveform set, {si(t)}, is an
orthogonal set, the receiver implementation in the
two figures are identical (differing perhaps by a scale
factor)
Universidade do Minho
Escola de Engenharia 47
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Correlation receiver
– However, when {si(t)} is not an orthogonal set, the
receiver in the last figure, using N correlators instead
of M, with reference signals ψj(t)}, can represent a
cost-effective implementation
– Since N is normally much smaller than M (e.g. 256-
QAM has M = 256 while N = 2) this receiver is
preferable

Universidade do Minho
Escola de Engenharia 48
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Probability of error
• Partition of the observation space into decision regions:
– Zi (i = 1,2, … M)

– Example:
– Constellation with N = 2 and M = 4

Universidade do Minho
Escola de Engenharia 49
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Probability of error
• Observation vector x lies in Zi if euclidean distance:
– IIx – skII
• is minimum for k = i

Universidade do Minho
Escola de Engenharia 50
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Probability of error
– Rotational invariance

Universidade do Minho
Escola de Engenharia 51
 Signal - Space Analysis
• Coherent Detection of Signals in Noise
– Probability of error
– Translational invariance

Universidade do Minho
Escola de Engenharia 52
 Reading Material
• Simon Haykin, Communication Systems, McMaster Univ,
5th Edition, 2009 (ISBN 978-047-16979-0-9)
• A. Bruce Carlson and Paul Crilly. Communication Systems
– An Introduction to Signals and Noise, McGraw-Hill, 5th
edition, 2009 (ISBN 978-007-338-040-7)
• Bernard Sklar and Fredric Harris, Digital
Communications: Fundamentals and Applications,
Pearson, 3rd Edition, 2020 (ISBN-13: 978-0134588568)

Universidade do Minho
Escola de Engenharia 53
 Reading Material

Universidade do Minho
Escola de Engenharia 54