No CrossRef data available.
ISSUES CONCERNING THE APPROXIMATION UNDERLYING THE SPECTRAL REPRESENTATION THEOREM
Published online by Cambridge University Press: 10 February 2004
Abstract
In many important textbooks the formal statement of the spectral representation theorem is followed by a process version, usually informal, stating that any stationary stochastic process {ξ(t), t ∈ T} is the limit in quadratic mean of a sequence of processes {S(n,t), t ∈ T}, each consisting of a finite sum of harmonic oscillations with stochastic weights. The natural issues, whether the approximation error ξ(t) − S(n,t) is stationary or whether at least it converges to zero uniformly in t, have not been explicitly addressed in the literature. The paper shows that in all relevant cases, for T unbounded the process convergence is not uniform in t (so that ξ(t) − S(n,t) is not stationary). Equivalently, when T is unbounded the number of harmonic oscillations necessary to approximate ξ(t) with a preassigned accuracy depends on t. The conclusion is that the process version of the spectral representation theorem should explicitly mention that in general the approximation of ξ(t) by a finite sum of harmonic oscillations, given the accuracy, is valid for t belonging to a bounded subset of the real axis (of the set of integers in the discrete-parameter case).The author is grateful for very useful suggestions to Francesco Battaglia, Gianluca Cubadda, Domenico Marinucci, Enzo Orsingher, Dag Tjøstheim, and Umberto Triacca and also to an anonymous referee and the Econometric Theory co-editor Benedikt M. Pötscher.
Information
- Type
- Research Article
- Information
- Copyright
- © 2004 Cambridge University Press
1. INTRODUCTION
Let us begin by recalling the statement of the spectral representation theorem (SRT). In the continuous-time case, if ξ(t) is a wide-sense stationary process, then for each fixed t
the integral being defined as the quadratic mean (q.m.) limit of the integral sums
The preceding statement is usually followed by a less formal version, implicitly using the notion of convergence of a sequence of stochastic processes. Important examples are Cramér and Leadbetter (1967, p. 129): “the spectral representation (1) shows how the ξ(t) process is additively built up by elementary and mutually orthogonal harmonic oscillations”; Anderson (1971, p. 398): “any stationary (wide sense) stochastic process can be considered as a weighted sum or integral of trigonometric functions of time with the weights being random variables”; Priestley (1981, p. 245): “any stationary process can be represented as (the limit of) the sum of sine and cosine functions with random coefficients.” (the emphasis on the word process is mine).
Obviously, these process versions of the SRT are perfectly correct if convergence of the sequence of stochastic processes {S(n,t), t ∈ T} to {ξ(t), t ∈ T} simply means that S(n,t) converges to ξ(t) for any t. However—this is the starting point of the present paper—these versions do not address two issues that arise naturally when convergence of stochastic processes is concerned: (i) whether the approximation error ξ(t) − S(n,t) is stationary and, in case it is not, (ii) whether at least ξ(t) − S(n,t) converges to zero uniformly in t. Nor are these issues addressed elsewhere in the literature, the only exception, to the knowledge of the present author, being Doob (1953 p. 528), who maintains that convergence of (2) to (1) is uniform in t:
Thus Example 3 [Example 3, defined on p. 524, is a finite sum of harmonic oscillations with mutually orthogonal complex random amplitudes, such as (2)] can be used to approximate the general case in the sense that to every ε > 0 corresponds a stationary wide sense process of the type in Example 3, with variables {xε(t)}, satisfying
we need only take as xε(t) an appropriate Riemann–Stieltjes sum of the spectral representation of x(t).
The present paper shows that in all relevant cases, i.e., if the spectral distribution function of ξ(t) contains either an absolutely continuous component or a jump function, convergence of the integral sums S(n,t) to ξ(t) is uniform if and only if T is bounded. Among the consequences: ξ(t) − S(n,t) is not stationary; in the discrete-time case, if
is a moving average process, convergence of S(n,t) to ξ(t) is not uniform; Doob's statement must be corrected by assuming that a ≤ t ≤ b, with a and b finite.
The paper is organized as follows. Section 2 gives a summary of the SRT in discrete time and shows that convergence is not uniform for a white noise unless T is bounded. The general results outlined previously are given in Section 3 for the discrete-time case. Section 4 discusses a seemingly paradoxical aspect of the SRT in the discrete-time regular case: a sequence of linearly deterministic processes approximates a process with a positive-variance innovation. A short account of the continuous-time case is given in Section 5. Section 6 concludes.
2. DISCRETE TIME: THE CASE OF A WHITE NOISE
We consider complex wide-sense stationary processes {ξ(t), t ∈ T}, where
(a) The first moment of ξ(t) is zero.
(b) The second moment of ξ(t) is positive.
(c) Unless otherwise stated,
, for continuous- or discrete-time processes, respectively. This is convenient and has no consequences in terms of generality.
Let us begin with discrete time. In this case the SRT has the specification
for any given
, the right-hand side being defined as a q.m. integral.
Let us first recall the basic elements of the proof of the SRT (the presentation here is based on Cramér and Leadbetter, 1967, Sects. 7.4 and 7.5; see also Brockwell and Davis, 1991, Sects. 4.3 and 4.8). Assume that the complex stochastic variables ξ(t), for
, are defined on the probability space
and let
be the minimal closed subspace of
containing all the variables ξ(t), for
. Then consider, first, the spectral distribution function F associated with the process ξ(t), i.e., the nondecreasing right-continuous real function, defined on [−π π], such that
; and, second, the Hilbert space L2([−π π],F) of all functions
, such that
, with the inner product defined as
. It can be shown that the map
, defined by linearly extending Φ(ξt) = exp(it·), is one-to-one and preserves the inner product, i.e., if x and y belong to
, then
(see Brockwell and Davis, 1991, Theorem 4.8.1, p. 144). To define the integral in the SRT, first define the map ζ : [−π π] → Hξ as
where χ((a1 a2]) is the indicator function of the interval (a1 a2] . The term ζ(λ) is a right-continuous (in q.m.) orthogonal-increment process such that ∥ζ(b) − ζ(a)∥2 = E(|ζ(b) − ζ(a)|2) = F(b) − F(a).
Second, define a subdivision of [−π π] as an n-tuple λ = (λ0 λ1 … λn), with −π = λ0 < λ1 < ··· < λn−1 < λn = π. Let d(λ) = maxh(λh − λh−1). With λ is associated the integral sum
(a slight change of notation with respect to (2)). Given a sequence of subdivisions
, where λn = (λ0,n λ1,n … λn,n), we assume throughout that d(λn) → 0 as n → ∞.
As we show subsequently, given
and the sequence of subdivisions Λ, the integral sum sequence
converges in q.m. to ξ(t). Thus the integral in (3) makes sense, as the common q.m. limit of all sequences of integral sums, and (3) is proved.
To see this observe that by (6) and the definition of Φ and ζ,
so that, by (4),
The result follows from uniform continuity of exp(itλ) with respect to
for each fixed t.
The definitions and results briefly recalled here allow the following restatement of the SRT.
SRT1. Given the stationary process
, let Λ be a sequence of subdivisions. For every ε > 0, there exists a positive real δ(ε,t) such that if d(λn) < δ(ε,t), then ∥ξ(t) − S(λn,t)∥2 < ε. (Note that (8) and uniform continuity of exp(itλ) with respect to λ for each fixed t imply that δ(ε,t) can be determined independently of Λ.)
Two issues naturally arise concerning SRT1.
(i) The first is whether ξ(t) − S(λn,t) is stationary.
(ii) The second, arising if the answer to (i) is negative, is whether the convergence of S(λn,t) to ξ(t) is uniform with respect to t, i.e., whether the dependence of δ(ε,t) on t can be dropped.
As regards (i), from the first line in (8):
A cursory inspection of (9) should convince the reader that the answer to (i) is negative in general. However, because obviously a negative answer to (ii) implies a negative answer to (i), and because we are able to construct a simple proof that in all relevant cases S(λn,t) does not converge uniformly to ξ(t), a direct treatment of (i) is not necessary.
The answer to both (i) and (ii) is easy when ξ(t) is a white noise, i.e., when dF(λ) = (σξ2/2π)dλ. Using (9), a straightforward calculation shows that
Given λn, ∥ξ(t) − S(λn,t)∥2 converges to 2σξ2 for |t| → ∞. Thus t cannot be dropped in δ(ε,t). In conclusion, given Λ, S(λn,t) converges in q.m. to ξ(t) for any t but not uniformly in t, and therefore ξ(t) − S(λn,t) is not stationary.
To conclude this section, let us observe that no modification of the preceding arguments is needed if we define the integral sums as
where ρh,n ∈ [λh−1,n λh,n] .
3. DISCRETE TIME: GENERAL RESULTS
The result obtained for a white noise by directly computing the distance between ξ(t) and S(λn,t) can be generalized to any moving average process, i.e., any process ξ(t) such that
where u(t) is white noise and
.
PROPOSITION 1. Let
be a moving average process and let Λ be any sequence of subdivisions. The q.m. convergence of S(λn,t) to ξ(t), as n → ∞, is not uniform in t.
Proof. Moving average processes are characterized by an absolutely continuous spectral distribution function (see, e.g., Doob, 1953, Ch. X, Sect. 8). Let us indicate by f the derivative of F, existing almost everywhere in [−π π] . Given the subdivision λn,
By the Riemann–Lebesgue Lemma (see Apostol, 1974, p. 313), the preceding integral tends to zero as t → ∞. Thus, using (9), for any given λn the squared distance ∥ξ(t) − S(λ,t)∥2 tends to 2σξ2 for t → ∞. █
Let us now turn to the opposite extreme, so to speak, i.e., the case of processes whose spectral distribution is a step function. Precisely, let Q be either the set of the first m integers or the whole
, let {Aj, j ∈ Q} be a set of mutually orthogonal, zero-mean, positive-variance random variables with Aj ∈
, let {φj, j ∈ Q} be a set of real numbers belonging to (−π π] , and let
for
. Defining r(λ) as the set of all integers j ∈ Q such that φj ≤ λ, the spectral distribution function of ξ(t) is
PROPOSITION 2. Let ξ(t) be defined as in (11) and let Λ be any sequence of subdivisions. The q.m. convergence of S(λn,t) to ξ(t), as n → ∞, is not uniform in t.
Proof. Let us define s(h,n) as the set of all integers j ∈ Q such that λh−1,n < φj ≤ λh,n. Moreover, given j, there is exactly one subinterval of the subdivision λn containing φj; we call μj,n the lower extreme of such subinterval (of course, μj,n coincides with one of the lower extremes λh−1,n). Using (9) and defining σk2 = var(Ak),
Now consider the term σ12 cos(t(φ1 − μ1,n)), appearing in (12). Because the subintervals of λn have the form (λh−1,n λh,n] , then φ1 > μ1,n. Thus the ratio ν1,n = π/(φ1 − μ1,n) makes sense. Denote by ν1,n* the greatest integer less or equal to ν1,n. Applying Taylor's formula up to the second derivative, centered at t = ν1,n, to the function g(t) = cos(t(φ1 − μ1,n)) (see, e.g., Apostol, 1974, p. 113),
where τ lies between ν1,n* and ν1,n. On the other hand, φ1 − μ1,n → 0 as n → ∞. Thus, given η, with 0 < η < 2, there exists n(η) such that for n > n(η)
Using (12), for n > n(η),
Thus for n > n(η) the squared distance between ξ(t) and S(λn,t) is greater than 2σ12(2 − η) at least for t = ν1,n*. Therefore the convergence is not uniform. █
It is crucial for the preceding proof that φ1 > μ1,n, which is a consequence of the particular integral sums we are using. However, if the integral sums (10) were used, Proposition 2 would need no more than a minor modification: convergence of the integral sums to ξ(t) is not uniform for sequences of integral sums that are generic with respect to ξ(t), i.e., such that ρh,n ≠ φj for any
.
The following is an easy consequence of Propositions 1 and 2.
PROPOSITION 3. Let F be the nondecreasing, right-continuous spectral distribution function of the stationary process ξ(t). The term F can be decomposed as F = F1 + F2 + F3, where F1 is nondecreasing and absolutely continuous, F2 is a nondecreasing step function (with a finite or countably infinite set of steps), and F3 is nondecreasing and continuous with F3′ = 0 almost everywhere (see, e.g., Riesz and Nagy, 1990, p. 53). If either F1 ≠ 0 or F2 ≠ 0, then q.m. convergence of S(λn,t) to ξ(t) is not uniform in t.
Proof. Assume F1 ≠ 0. Starting with the first line in (8),
The conclusion follows applying Proposition 1. The same argument applies if F2 ≠ 0. █
Note that the spectral distribution function of regular processes, i.e., processes with a positive-variance innovation, contains an absolutely continuous component (see Brockwell and Davis, 1991, Theorem 5.7.2; p. 190). Therefore Proposition 3 implies that convergence is not uniform for regular processes.
Remark. The SRT is sometimes presented as a consequence of the spectral representation of the linear unitary operator
, defined by linearly extending Lξ(t) = ξ(t − 1) (see, e.g., Rozanov, 1967, p. 16). The spectral representation for the operator Lτ is
where {E(a), a ∈ [−π π]} is a spectral family of projections and the right-hand side converges to Lτ in the operator norm (for details and proof, see, e.g., Riesz and Nagy, 1990, Sect. 109). It is possible to show that ζ(b) − ζ(a) = [E(b) − E(a)]ξ(0). Thus defining
, we have S(λn,t) = V(λn,−t)ξ(0) and
As a consequence, if the convergence to zero of the operator norm ∥L−t − V(λn,−t)∥ is uniform in t, then ∥ξ(t) − S(λn,t)∥ converges to zero uniformly in t. Thus if F fulfills the assumptions of Proposition 3, convergence of V(λn,τ) to Lτ is not uniform in τ.
4. HOW CAN LINEARLY DETERMINISTIC PROCESSES APPROXIMATE A REGULAR PROCESS?
Observe first that S(λn,t) is linearly deterministic. If ξ(t) is regular, i.e., has a positive-variance innovation, the SRT implies the somewhat puzzling consequence that, whereas S(λn,t) approximates ξ(t), the innovation of S(λn,t), which is zero, does not converge to the innovation of ξ(t). On the other hand, the case of a regular process is also one in which convergence of S(λn,t) to ξ(t) is not uniform (see the observation following Proposition 3 in Section 3). Thus one may be tempted by the idea that such lack of uniformity is an explanation of the puzzle.
However, it is easy to show that uniform process convergence is neither sufficient nor necessary to ensure convergence of the innovation. Nor does the fact that S(λn,t) is deterministic play any special role in the puzzle. In Lippi (2003) an example is provided in which ξn(t) and ξ(t) are stationary and (1) ξn(t) − ξ(t) is stationary; (2) ξn(t) → ξ(t) in mean square (stationarity of ξn(t) − ξ(t) implies that such convergence is uniform); (3) ξn(t) and ξ(t) are regular moving average processes (thus they contain no deterministic component); (4) however, the innovation of ξn(t) does not converge to the innovation of ξ(t).
In conclusion, the puzzle pointed out in this section is just an example, among many others, in which intuition based on finite-dimensional euclidean spaces is misleading with infinite-dimensional Hilbert spaces. In our case, convergence of ξn(t) to ξ(t), for any t, obviously implies that proj(ξn(t)|ξn(t − 1), ξn(t − 2), …, ξn(m)) converges to proj(ξ(t)|ξ(t − 1), ξ(t − 2), …, ξ(m)), for a given finite m, but does not imply that proj(ξn(t)|ξn(t − j), j > 0) converges to proj(ξ(t)|ξ(t − j), j > 0), irrespective of whether convergence of ξn(t) to ξ(t) is uniform in t or not.
5. CONTINUOUS TIME
Extension to continuous time is straightforward and does not call for a detailed treatment. Let us remember that the SRT, i.e., (1), holds in the continuous-time case under the assumption that limt−s→0 E(|ξ(t) − ξ(s)|2) = 0 (see, e.g., Cramér and Leadbetter, 1967, Ch. 7), with the definition of the spectral distribution function F(λ) and the stochastic measure ζ(λ) being essentially the same as in the discrete-time case, the domain of both functions extending over the whole real line. In the definition of a subdivision the extremes λ0 and λn may be any real numbers, with λ0 < λn. Moreover, if Λ is a sequence of subdivisions we assume that d(λn) → 0, λ0,n → −∞, and λn,n → ∞. The continuous-time version of SRT1 is then obtained using the analogue of (8). The proofs of Propositions 1–3 extend to continuous time with minor modifications.
6. CONCLUSION
The paper has shown what the spectral representation formula (1) is not. Indeed, in relevant situations, the Riemann–Stieltjes sums defining the integral are not jointly stationary with ξ(t). Moreover, if T is unbounded, the variance of the approximation error does not converge to zero uniformly in t.
On the other hand, if T is bounded, uniform convergence of the integral sums to ξ(t) is a trivial consequence of (8) (or its continuous-time analogue), and the SRT can be reformulated as follows (the continuous-time version requiring only minor changes).
SRT2. Given the stationary process
and the integers m1 and m2, with m1 < m2, let Λ be a sequence of subdivisions. For every ε > 0, there exists a positive real δ(ε,m1,m2) such that if d(λn) < δ(ε,m1,m2), then
This is the correct process convergence version of the SRT. Though weaker than Doob's statement (see the Introduction) in that the number of harmonic oscillations necessary to achieve a given approximation accuracy ε increases as the interval [m1 m2] becomes larger, SRT2 is sufficient to provide a firm basis for the idea of harmonic oscillations as building blocks of stationary processes and therefore for the usual interpretation of the spectral measure (the spectral density in the moving average case), for the spectral analysis of linear filters, and for all other frequency-domain results.
References
REFERENCES
Anderson, T.W.
(1971)
The Statistical Analysis of Time Series.
Wiley.
Apostol, T.M.
(1974)
Mathematical Analysis,
2nd ed.
Addison-Wesley.
Brockwell, P.J. &
R.A. Davis
(1991)
Time Series: Theory and Methods.
Springer-Verlag.
Cramér, H. &
M.R. Leadbetter
(1967)
Stationary and Related Stochastic Processes.
Wiley.
Doob, J.L.
(1953)
Stochastic Processes.
Wiley.
Lippi, M.
(2003)
A Note on Convergence of Stochastic Processes.
Working paper,
Dipartimento di Scienze Economiche, Università di Roma
“La Sapienza,”
Rome.
Priestley, M.B.
(1981)
Spectral Analysis and Time Series.
Academic Press.
Riesz, F. &
B.S. Nagy
(1990)
Functional Analysis.
Dover.
Rozanov, Y.A.
(1967)
Stationary Random Processes.
Holden-Day.
You have
Access
- 1
- Cited by