IEEE

TRANSACTIONS ON ACOUSTICS,
SPEECH, AND SIGNAL
PROCESSING, VOL. ASSP-25, NO. 3, JUNE 1977 235

Short Term Spectral Analysis, Synthesis, and

Modification by Discrete Fourier Transform
JONT B. ALLEN

Abstract-A theory of short term spectral analysis, synthesis, and amount of additional filtering. In our method, only a small
modification is presented with an attemptatpointing out certain number of adds (four fo'r a Hamming window) per sample are
practical and theoreticalquestions. The methods discussed here are
required.
useful in designing filter banks when the filter bank outputs are to be
used for synthesis after multiplicative modifications aremade to the The major advantage of the present scheme is that it allows
spectrum. arbitrary modifications of theshorttermspectrum. These
modifications may be directly interpreted in the time domain
as a.filter whose impulse response is given by the Fourier trans-
form of the modification. The price paid for allowing modifi-
I N THIS paper, some practical and theoretical questions are
considered concerning the analysis of and synthesis from a
signal's shorttermspectrum.Theshorttermspectrum, or
cations of the spectrum w i t b e seen to be an increase in the
number of frequency channels required. A modification made
time spectrum, are those signals which result from analyzing prior to the usual method of synthesis, namely, of adding the
a single input signal with a set of filters which areselective filter outputs of a contiguous filter set, does not satisfy the
over a range of frequencies [ l ] -[3]. In the case of analysis by convolution rule.
a spectrum analyzer, the filters are either spaced contiguously
ANALYSISOF SHORTTERM SPECTRA
or one filter is heterodyned over the frequency rangeof in-
terest. Formany applications, this is quiteadequate. How- We have defined the short term spectra as an output derived
ever, when one is interested in both analysis and synthesis, a from a bank of filters. At each filter frequency, we require
more rigorous approachis in order. two filters which have the same magnitude response but differ
In this paper, we shall restrict ourselves to the case of uni- in phase by 90".
formlyspaced,symmetricbandpass filters. We arealso not It is well known [2] , [3] that a filterbank of this form may
concerned with bandwidth reduction. In general, the channel berealized by weighting theinput signal x ( t ) bya sliding
capacity in the short term spectral domain will be greater than low-pass filter impulse responsew ( t ) and Fourier transforming
that of the original signal. We will be interested, however, in the result. Thus, we have
beingable tomodifytheshorttermspectrum in eitherits m

phase or amplitude content without introducing undesireddis- X(f,t ) =/ w (t - 7)x (7)e i 2 n f ~d7 (1)
tortion in the synthesized signal. -m

Previous filter bank analysis-synthesis techniques have been where X(f,t ) is the short term frequency spectrum, t is the
given by Flanagan and Golden [ l ] , Schafer and Rabiner [2], time variable, w ( t - 7) is the shifted window, X ( T ) is the input
and Portnoff [3]. Our approach differs inseveral important signal, and exp (j27r.f~) is the complex exponential.
ways. Previous approaches haveused contiguous filter banks We define W(f)as the Fourier transform of w ( t ) :
in the analysis process. We shall show that this results in an
undersampledspectrum and, as a result, synthesisbecomes
verysensitive to phase or delay modifications. We will then W ( f ) =Jm w(7) ej2nfT d7. (2)
-m
show that by using a properly sampled overlapping filter set,
we may avoid this sensitivity. By recognizing the need for a W(f) is assumed to be small for frequencies above some crit-
greater number of filters, both the analysis and synthesis pro- ical frequency. The short term spectrum X(f,t ) is equivalent
cedures are simplified. However, the number ofsamples of to frequency shifting the frequency band of x(t) centered at
data in the short term spectral domain that results per sample f downtozerofrequencywiththecomplexexponential
of input data is greater than one; thus, the relevance of the exp (j277ft) and low-pass filtering the result with the low-pass
present method to bandwidth reduction remainsunclear. filter w(t). The resulting X(f,t ) is a complex function of time
A second important difference between our approach and (see Fig. 1 and [ l ] -[3]). In the applications considered here,
that of others is during synthesis. All previous authors [ l ] - x ( t ) is to be a sampled data signal x ( k ) and ,
'
(
X t ) will be
[3] have summedthe filter outputsforthe synthesis; we found by replacing the Fourier transform by adiscrete Fourier
synthesize in a way that is similar to the overlap add method transform (DFT).
[4] . As a result, our method does not require an interpolating An important question is relevant at this point, namely, how
filter prior totheadd, and @us, there is a savings in the many frequency and time samples are required to fully repre-
sent the data X(f,t ) in a sampled data system. This question
Manuscript received September 29, 1976;revised January 25, 1977. is answered by applying the Nyquist theorem twice. w ( t ) has
The author is with Bell Laboratories, Murray Hill, NJ 07974. two characteristic "lengths," one in the time domain and one
236 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, JUNE 1977

LOW PASS
FILTER

Fig. 1. Analog equivalent of the windowed Fourier transform method

of simulating a filter bank.

in the frequency domain. These lengths are bounded by the

so-called “uncertainty principle” [5]. We define the length in
time to be that time period T over which w ( t ) is significant.
The frequency length is that frequency range F over which
W( f ) is significant. For the case of the Hamming window,
< t < To12
t
0.54 i-
0.46 COS (2ntjTo), -To12
w(t)= (3)
0, I t l > To12
reasonable definitions of T and F a r e
T = To
F = 4/To. (4)
In Fig. 2 we see the meaning of these definitions graphically. (FREQUENCY)x (WINDOW LENGTH)

w(t) is exactly time limited; thus the definition of T is based f To

on the window’s nonzero length. W ( f ), on the other hand, Fig. 2. Basic definitions of thecharacteristictimeandfrequency
is onlyapproximatelybandlimited. Using a 42 dB criteria, lengths T , F for the case of a Hamming window.
W( f ) is effectively frequency limited with a maximum fre-
quency of 2/To. Note that our definition of the characteristic Properly sampled in both the time and frequency domains,
frequency length includes negative as wellas positive fre- the short term spectra may be found by a discrete Fourier
quencies since the spectrum is symmetric about f = 0. transform (DFT) when the input is a sequence x@). In the
F and T may be used to define sampling rates in the time discrete case, T is normalized by the sequence sampling pe-
and frequency domains, respectively. For a fixed f , X( f , t ) is riod which wecall T,. T, is chosen so that T is an integral
a low-pass signal. Thus, from the Nyquist theorem, X may be power of 2.
sampled in time at a rate greater than or equal to twice its In the following, we denote the time index n and the fre-
highest frequency. An equivalent statement of the Nyquist quency index m as subscripts. There is no significant informa-
theorem is that the density of time samples must be greater tion lost if we replace the short term spectrum X ( f , t ) with its
than the characteristic frequency length. We define this sam- sampled version X,, by sampling it at the two Nyquist pe-
ple period as D. riods l jF and l IT.
D = 1/F. (5)
Xnm = X(m/T,
nlF), n and m integers. (7)
The time samples of interest are then given by
Using this notation, we may specify the formula for the Ny-
t, = nD. (64 quist sampled short term spectrum as
Equation ( 5 ) says that the interval between time samples T- 1
equals the reciprocal of the characteristic frequencylength. X,, = w(nD - k ) x(k) eiznkm’T. (8)
F willbecalled the frame rate. For a Hamming window, D k=O
is equal to one quarter of the window’s length.
By a completely analogous argument, the continuousfre- Equation (8) is the sampled versionof (1). Note that three
quency spectra may be replaced by a discrete set of frequen- different sample periods have been defined: T,, the sample
cies. This isagain the Nyquist theorem; however, now it is period for the input sequence x@); D , the frame period for
applied to the time domain. In this case, the sample density the bandlimited signals at each frequency f ; and 1/T, the
is in frequency and the characteristic length is in time. Thus, frequency sampling period or filter spacing.
we may sample in thefrequency domain witha spacing of If we define F and F-’ as the DFT and its inverse,
1/T. The frequencies of interest are X,, = F{w(nD - k)x(k)}. (9)
f , = m/T. (6b)
In (9), k and m are the transform variables and n is a parame-
Equation (6b) says that the interval between frequency sam- ter. Note that by applying an inverse Fourier transform to (9),
ples is the reciprocal of the characteristic time length T . we may find a relationship which will be useful later:
ALLEN: SPECTRAL ANALYSIS, SYNTHESIS, AND MODIFICATION 237

+ tth
x ( k ) w(nD - k ) = F - l { x n m }
where
(10)
4 D ‘v
n FRAME OUTPUT

1 T-1
F-’ } 67
{xnrn X,, e-j2nkmlT. (1 1) t

m=o OUTPUT DATAARRAY

When X,, is interpreted as the output of a bank of filters,

each n corresponds to atime sample at the lower sampling OUTPUTOVERLAP ADD
SAMPLES
SAMPLES
rate resulting from the
bandlimiting
nature
(“frequency
length”) of each filter. Each m corresponds to a different fil- Fig. 3 . The synthesis is performed in three steps. First, D samples are
ter frequency of the filter set. The frequency domain may be output from the left end of the output buffer. Then the output buf-
fer is shifted D samples to the left to the position shown. Finally, the
sampled because of the finite time lengthof the window. samples x n ( t ) are added to those in the output buffer. The steps are
Next we lookatthe increase in thedata sample density then repeated for the next frame.
defined as the number of samples of short term spectra gen-
erated for each input sample of x @ ) . For a real signal x @ ) , In practice, during analysis the signal x ( k ) is shifted by D
assume that we take a segment T long and compute the num- and the window remains fixed. For synthesis, we define the
ber of samples ofshorttermspectrarequiredduringthat signals x, ( k )
period. A DFT is required every D samples and each DFT has
x,@) = F-I { X n m } .
T unique frequency values (T/2 complex values). Thus, we get
During synthesis, the output buffer x ( k ) is shifted by D and
T
T(samp1esper frame) X - (frames per T interval) (1 2) the new samples x , @ ) are added to x @ ) . (Thismight be
D viewed as a block recursive calculation.) This process is then
samples for every T time samples. The number of samples gen- repeated for the next time frame, each frame being defined as
eratedforeach input sampleis then given by dividing (12) a shift of the window byD samples (Fig. 3).
by T:
SYNTHESIS WITH SPECTRUM MODIFICATIONS
m
I The analysis-synthesis system described above may be gen-
- = TF.
D eralized to allow for modifications of the short term spectrum.
For the case of a Hamming window, this factor becomes4. To do this, we must recognize that spectral modifications are
equivalent to filtering. Since the time characteristic length is
SYNTHESIS WITHOUT MODIFICATION always increased by filtering, the number of frequency points
The synthesis procedure isbased on the following identity must be increased as indicated by (6b). This is done by ap-
proved in the Appendix for any bandlimited windoww(k): pending zeros to the window w ( k ) prior to the analysis DFT.
The number of points in the DFT is also increased to include
m
the appended zeros. This is similar to appending zeros to the
w(nD- k ) = 1 window in the overlap addprocedureaftersectioning [4].
-m
n= The numberofappended zeros, andthustheDFTlength,
must be great enough to accommodate the modification which
where we have assumed with no loss of generality that
is to be made. When the modification has been properly ac-
W(0)= D. (1 5) counted for, time aliasing will not occur during synthesis. In
practice, some time aliasing does occur, but by proper choice
Relation (14) is exactly true if the window is truly bandlim- of the initial DFT length, theerror may be madenegligible.
ited to F/2. Note that (14) is independent of k.
The effect of a fixed spectral modification P,,,may be un-
Ifwe multiply (14) by x ( k ) and use the inverse DFT rela- derstood by factoring it outas follows:
tion (lo), we obtain the synthesisrule
OE

x ( k )= F-’ {Xnm 1 (1 6)
-ea
n=
where
1 T-1 = P(k) * x(k) (20)
~ - {1x n m } = xnm e-i2rkmlT (17) where
m=o

Equation (16) states that x ( k ) is a sum over inverse DFT’s. It ~ ( k= )F - l {pm 1 (21)
issimilar to the overlap add rule, asdiscussed by Stockham and “*” denotesconvolution.Thus,afixedmodification
[4] , which may be used to do continuous convolution using to theshorttermspectrum is equivalent to convolution
FFT’s; it differs in that the sections are taken as overlapping with p ( k ).
and are not rectangular windows. Fixed modifications may not be made in the schemes pre-
238 TRANSACTIONS
IEEE ON ACOUSTICS,
SIGNAL
PROCESSING,
SPEECH,
AND
JUNE 1977

sented by other authors [2], [3]. For example, the synthesis to properly represent the short term spectra. The analysis is
techniques of both authors are particularly sensitive to a 180’ performed in frames by a sliding low-pass filter window and a
phase modification in one channel. Such a modification will DFT, a frame being defined bythe Nyquist period of the
produce zeros in the transition regions between bands, and will bandlimited window. The synthesis is reminiscent of the over-
therefore not result in the desired all-pass modification. lap add process as discussed by Stockham [4], and consists of
The next obvious step is to allow the modification to be- an inverse DFT and a vector add each frame. Spectral modi-
come a function of time, giving rise to the final synthesis rule: fications may be included if zeros are appended to the window
m
function prior to the analysis, the number of zeros being equal
~ ( k= ) F-’ {Prim x n m 1. (22) to the time characteristic length of the modification.
-m Advantages of the new technique are that modifications may
n= be included and no interpolation is necessary during synthesis.
The effect of a time-varying spectral modification is beyond A possible disadvantage is the increased amount of bandwidth
the scope of the present paper. required to transmit the short term spectrum as compared to
Finally, we would like to point out some further differences that required to transmit the original signal.
between the present and previous [2], [3] results. It is true,
in general, that the short term spectra may be subsampled in APPENDIX
either frequency or time and x ( k ) can still be determined from We wish to show that given any.function w ( k ) which is
X,, . When this happens, however, the synthesis is no longer bandlimited to a frequency of 1/(2D) and normalized as given
robust to modifications. That this is true may beseen from by (15), then (14) is true, namely, that the sum of any set of
two examples. samples of w ( k ) taken with aperiod D is one.
Suppose we generate the time subsampled, short term spec- This is easily proved using the Poisson summation formula
tra using a Hamming window which has been shifted by its [5, eq. (3-56), p. 471. If W(f) is the Fouriertransform of
full period w(k),then
x,, =F{w(nTo - k ) x ( k ) } m
1 ”
w ( n o - k ) = - 2 e-i2amklD W(m/D). (All
where w ( k ) is the Hamming window with length T o . x ( k ) -m D -m

may then be recovered by first using (16) and then correcting n= m=

for the window function. However, when amodification P Since W is bandlimited, only the m = 0 term is nonzero.
has occurred, such as a pure delay, the window correction will Thus, (14) follows from (15).
no longer be correct,
The second example is the case of Portnoff [3] . Our anal- ACKNOWLEDGMENT
ysis procedure is equivalent to his if we remove all filters that The author would like to acknowledge the helpful comments
overlap, leaving a contiguous subsampled set. Portnoff has dis- of D. A. Berkley.
cussed in detail a method of synthesis from this undersampled
data set. He requires a restriction on the window function for REFERENCES
his synthesis towork. The extra restriction is thatthe fre- [11 J . L. FlanaganandR. M. Golden, “Phasevocoder,” Bell Syst.
quency response of the filters add to one. This requirement is Tech. J.,vol.45,pp. 1493-1509,Nov. 1966.
particularly sensitive to phase modifications.Portnoff himself L21 R. W. Schaferand L. R. Rabiner,“Designandsimulation of a
speech analysis-synthesis system based on short-time Fourier anal-
points out the problem of this condition not being properly ,
ysis,” IEEE Trans. Audio Electroacoust., vol. AU-21, pp. 165-174,
met: ‘‘. . . the resulting distortion will be perceived as rever- June 1973.
beration in the output signal” [3] . [31 M. R. Portnoff, “Implementation of the digital phase vocoder us-
ing thefastFouriertransform,” IEEETrans. Acoust.,Speech,
Signal Processing, vol. ASSP-24, pp. 243-246, June 1976.
SUMMARY [41 T. G. Stockham, “High-speedconvolution andcorrelation,”re-
Atheory of short term spectral analysis-synthesis with printed in Digital Signal Processing, L. R.Rabinerand C. M.
Rader, Ed. New York: IEEE Press, 1972, p. 330.
modifications has been discussed with particular attention be- A. Papoulis, TheFourierIntegraland ItsApplications. New
ing given to the number of time and frequency points required York: McGraw-Hill, 1962, p. 63.