No CrossRef data available.
Published online by Cambridge University Press: 20 March 2003
I review the part played by John Denis Sargan in the formation of the “LSE approach” to dynamic econometric modeling. Despite his unassuming demeanor and his location at LSE—which had earlier dismissed a substantive role for econometric evidence—Sargan nevertheless radically altered the econometric approach of a generation, establishing a powerful approach to empirical modeling of economic time series. His main contributions to econometric methodology, and the subsequent research, are discussed as a complement to the other papers in this memorial volume.Financial support from the U.K. Economic and Social Research Council under grant L138251009 is gratefully acknowledged. I am indebted to Alok Bhargava, Julia Campos, Meghnad Desai, Neil Ericsson, Toni Espasa, Grayham Mizon, Peter Phillips, Timo Teräsvirta, and Ken Wallis for helpful comments and to Peter Phillips for the invitation to contribute this paper to Econometric Theory.
This paper concerns the role of John Denis Sargan in the formation of the “LSE approach” to dynamic econometric modeling.1
It draws on material in Hendry and Wallis (1984), Phillips (1985), Desai et al. (1997), Ericsson, Maasoumi, and Mizon (2001), and Hendry and Pesaran (2001).
The motivation for Denis's research helps explain his general approach (see Phillips, 1985). To quote from Desai, Hendry, and Mizon (1997, p. 1121):
… it was not until he [Denis] visited a bookshop towards the end of the war and found John Maynard Keynes's General Theory that his enthusiasm for economics was aroused. He was so struck with Keynes's ideas for tackling mass unemployment that he decided to use his knowledge of mathematics and statistics to help tackle some of the pressing economic problems that faced society in the post-war years.
It may come as a surprise that Denis first worked on economic theory and only later turned to technical econometrics. When he did so, however, his scientific background enforced an empirical approach, and most of his theoretical research was driven by the need to solve the resulting difficulties in formulating, estimating, evaluating, and analyzing empirical econometric models. This naturally required an understanding of the modeling process itself—which is the aspect of interest here—and also of the properties of the tools of econometricians (estimators and tests) and the tools of the toolmakers (such as Monte Carlo methods and analytic finite-sample investigations). Many of his technical papers are reprinted in Sargan (1988a).
The structure of the present paper is as follows. Section 2 reviews earlier “LSE approaches,” namely, those associated with Robbins (1932) and with the “positivist” strand of philosophy, before considering the precursors to Denis's contributions to methodology. Section 3 focuses on the methodology embodied in Sargan (1964), whereas Section 4 considers his formal statement of general-to-specific (Gets) modeling. Section 5 turns to Denis's contribution to understanding and counteracting “data mining,” in the sense of adventitiously significant coefficients due to testing many effects. Section 6 then considers his contribution to model selection. The papers in these two sections were widely known and influential within LSE but have only recently been published. Section 7 sketches some of the consequential developments in Gets modeling that have occurred more recently. Section 8 concludes.
The first powerfully expressed “LSE approach” was actually that enunciated by Lionel Robbins. Robbins (1932) claimed that economic theory provided general, formal explanations that were applicable always and everywhere that resource allocation impinged. Since the assumptions of economics were “self-evidently true” and the logic impeccable, the conclusions must be correct. However, such insights as economics delivered could not be given quantitative expression, since causes were nonuniform over both time and space. Robbins argued, therefore, that statistical relationships in economics would change when the world did, so the discipline could never boast “statistical laws” like the natural sciences (see, e.g., Hendry and Morgan, 1995). Robbins's general approach was later supported at the LSE by the work of Friedrich von Hayek (Hayek, 1963). More recently, some “freshwater” economists in the United States seem to be the inheritors of the Robbins tradition (see, e.g., Kydland and Prescott, 1991, 1996).
In principle, economic theory could indeed apply even when there were no quantitative laws, although such explanations would be somewhat unsatisfactory—like predicting calm seas as the norm after a storm, to borrow from Keynes. Moreover, Robbins's positive argument for the power of economic theory explanations only becomes a negative one for econometrics on substantiating the claim that useful empirical regularities do not exist (see, e.g., Hendry, 1995a). Certainly, Dr. Blank's nebulous demand function for herrings could only last the test of time if many factors other than price and income were included in his model, but such additional factors are not precluded a priori. Indeed, when Hutchison (1938) launched his critical attack on Robbins, he made a similar argument and proposed instead a move toward “positive economics,” a theme we resume in the next section (for the background to the general arguments, see, e.g., Blaug, 1980). Nevertheless, since Robbins was one of LSE's most eminent economists, the LSE might seem an unpromising location for a group of econometricians, particularly one with an interest in econometric methodology.
However, the postwar period saw a strong move within LSE economics toward implementing a “positivist” philosophical standpoint (for an explanation of the variants of “positivism” see, e.g., Caldwell, 1982). The LSE itself had a strong tradition in the philosophy of science centered on Karl Popper, later supported by others such as Imre Lakatos (see, e.g., Popper, 1959, 1963; Lakatos, 1974). Thus, “falsification” was much under discussion at the LSE.
In economics, activity focused on the Staff Seminar on Methodology, Measurement and Testing (M2T), whose members included Chris Archibald, Kurt Klappholz, and Richard Lipsey, all of whom published on economic methodology (see, e.g., Archibald, 1959; Klappholz and Agassi, 1959; Lipsey, 1966). Here, for example, the famous Phillips curve was first presented and discussed (Phillips, 1958; see, e.g., Holt, 1969). The appointment of an engineer-economist like A.W.H. (Bill) Phillips, who stressed empirical evidence, signaled that the mood was less inimical to econometric modeling as the route ahead for economics than the heritage of Robbins might suggest. Indeed, important antecedents were already in place. Both Bill Phillips and Rex Bergstrom (also then at LSE) considered models that were essentially equilibrium-adjustment mechanisms (see, e.g., Phillips, 1954, 1957; and also Bergstrom, 1962, 1967; and the implementation in Bergstrom and Wymer, 1976).2
Phillips and Loretan (1991) discuss these precursors, and Leeson (2000) provides a detailed analysis of Bill Phillips's contributions, where many of his papers are reprinted.
In any case, Denis Sargan was first appointed in the Statistics Department at the LSE in 1963, where Roy Allen played a major role. Allen admired Phillips's earlier work on control theory (Phillips, 1954, 1957), and he included an explanation of integral, derivative, and proportional control in his textbook (Allen, 1963, Ch. 9). This strand also recurs subsequently, especially in Section 3.2.
Denis himself was a somewhat unlikely candidate to revolutionize econometric methodology. His initial research had been on mathematical economics, followed by econometric theory, particularly seeking to establish the same form of rigorous inferential basis for its application to small samples that Student's famous t-distribution paper had done for statistics (see Student, 1908). Indeed, the majority of his publications were to be in what he called “advanced econometric theory” (see Sargan, 1988b).
However, prior to his arrival at the LSE in 1963, Denis had written two papers on empirical modeling issues, published as Sargan (1957) and Sargan (1964). The former points to his later role in developing methodology; the latter provided several major advances therein.
Sargan (1957) was a discussion of the pathbreaking analysis of the Oxford Savings Survey by Malcolm Fisher (1956).3
Fisher's paper was also discussed by Franco Modigliani, Albert Ando, Milton Friedman, Trygve Haavelmo, Lawrence Klein, and James Tobin.
Sargan (1964), which was to become perhaps his most famous paper, was prepared for the Colston Society conference on National Economic Planning held at Bristol University in 1963 (for an appraisal of its impact, see, e.g., Desai et al., 1997). The next section considers that paper in more detail, emphasizing that almost every aspect of the current “LSE approach” was present.
In this paper, Denis laid out the conceptual foundations of what has become the “LSE approach.” The essential elements that he formalized included
(1) the use of “long-run” economic analysis to specify the equilibrium of the model;
(2) the introduction of “equilibrium-correction” mechanisms into behavioral dynamic econometric models;
(3) the development of a new interpretation of autoregressive errors in time-series models;
(4) the construction of valid misspecification tests after estimating dynamic models;
(5) the use of model comparison procedures for linear against logarithmic specifications;
(6) the investigation of the impact of data transforms on the selection of models;
(7) a nonlinear in parameters instrumental variables estimator for measurement errors;
(8) the development of operational computer programs to implement the new econometric methods;
(9) a proof that his iterative computations would converge with near certainty; and
(10) matching the econometric theory to the substantive empirical modeling problem.
Many of these topics arise in subsequent sections, but we first comment on each of them in turn.
The substantive problem that Denis addressed was empirical modeling of wages and prices (described in more detail in Section 3.10). Previous models had considered the changes in the variables, namely, wage and price inflation. Consider a conditional bivariate model of a variable denoted yt (the log of wages, say) given another variable zt (the log of prices), written in first differences as

Assuming that

and that the differenced variables are stationary with means

, then the long-run steady-state solution to (1) is

As formulated, (1) does not establish any relationship between the levels yt and zt; hence they could drift apart. Denis argued that economic agents would be concerned about the level of real wages, namely, yt − zt, not just price inflation. Thus, he formulated a model with a long-run levels equilibrium, denoted (y − z)e,t and given by

where xt denotes a vector of additional variables, such as unemployment, real productivity, and political factors. A unit coefficient on productivity would make the wage share a component of the left-hand side, but in practice, productivity was approximated by a linear deterministic trend in his wage models.
The disequilibrium is given by

In modern parlance, the levels are

when the differences are

. Consequently, if such a relation as (3) did exist, it would require that vt was

and so entail that yt was cointegrated with zt and xt (see, among many others, Engle and Granger, 1987; Phillips and Loretan, 1991; Banerjee, Dolado, Galbraith, and Hendry, 1993).
Some of the antecedents to the formulation of equilibrium-correction mechanisms were noted in Section 2.2; Section 3.11 comments on the terminology. If (2) is to be the long-run equilibrium, then a mechanism is needed in (1) to reestablish the relationship between the levels of yt, zt, and xt whenever they drift apart. To achieve (2), Denis postulated the explicit adjustment equation,

equivalent to including vt−1 from (3) in (1). Doing so unrestrictedly then entails adding the levels terms (y − z)t−1 and xt−1. When contemporaneous variables are excluded, the resulting equation is

Consequently, when π3 ≠ 0, the long-run levels equilibrium solution to (5) becomes

matching (2).
The model in (5) has both derivative and proportional control (e.g., Δzt−1 and (y − z)t−1), following up Phillips (1954, 1957) (see Salmon, 1982). The latter mechanism also ensures the disequilibrium adjustment, based on the (detrended) log-ratio of two nominal levels (see Bergstrom, 1962). Although Denis formulated (5) without specifying any specific integration properties for the data, the long-run relative stability of the “great ratios” in Klein (1953) was implicitly assumed to justify such transformations (see, e.g., the discussion of Granger and Newbold, 1977, in Hendry, 1977). Granger (1981) related the type of model in (5) to cointegration, and Granger (1986) showed that one of Δyt or Δzt must then depend on the equilibrium correction: the assumption in (4) is that yt adjusts to the disequilibrium. If both variables yt and zt adjust to the disequilibrium, then zt is not weakly exogenous for the {πi} (see Phillips and Loretan, 1991; Hendry, 1995b). It is primarily because of cointegration that equilibrium-correction models like (5) have proved a popular specification.
His empirical model is described in more detail in Section 3.10, but an interesting feature of his two main equations for wages and prices, respectively, was that they had different long-run solutions. Thus, although the integrability of data derives from a reduced-rank condition (fewer corrections than variables; see, e.g., Johansen, 1988), the existence of equilibrium corrections in his two main equations by itself did not preclude that the data were

around a deterministic trend. Alternatively, the variables could have been

by inheriting unit roots from either import prices (from the price equation) or productivity (wage equation).
Prior to Denis's Colston paper, it was common in econometrics to test for residual autocorrelation (e.g., by the Durbin and Watson statistic; see Durbin and Watson, 1950, 1951), and if such was found, to estimate an autoregressive-error process (see Cochrane and Orcutt, 1949). Thus, autoregressive errors were viewed as a “generalization” of white-noise errors. Denis showed that autoregressive errors were a restriction on a more general dynamic model that, when valid, allowed a more parsimonious representation, thereby reversing the conventional interpretation. Further developments on this topic are taken up in Section 4.
Denis stressed that empirical specifications should be stringently evaluated. Drawing on Sargan (1961), he formulated a test for the validity of the instrumental variables used in estimation and derived its distribution under the null. He estimated an autoregressive-error process for each equation by least squares or instrumental variables. To evaluate such nonlinear in parameters models, he developed a test for higher order autoregressive errors based on the residuals from the estimated equations, and he derived its asymptotic distribution. Such a test was, therefore, valid in dynamic equations in general. Nevertheless, the Durbin–Watson test—which did not follow its tabulated distribution if applied to such models—continued to be widely used for many years, although it would often fail to reveal significant misspecification (see, e.g., Nerlove and Wallis, 1966).
Denis also developed a criterion for choice of functional form between log and linear specifications. In essence, he showed that one should simply convert the respective equation standard errors to be proportions of the level of the dependent variable and then select the smaller. The log model automatically had an equation standard error of that form, but the levels equation standard error had to be scaled by the geometric mean of yt.
Previous models of wages given prices, such as Klein, Ball, Hazlewood, and Vandome (1961), modeled annual changes and introduced moving averages of past inflation and unemployment. Denis suggested that doing so would induce autocorrelated errors and would distort measures such as equation standard errors and

. Consequently, he argued for selecting the data transformations that led to an equation with “errors which are independent in different time periods.”
The Colston paper commenced with a discussion of appropriate estimation methods for dynamic equations with autoregressive errors. The analysis was based on Sargan (1959, 1961) and also described the nonlinear in parameters instrumental variables estimator, which allowed for possible data measurement errors and autoregressive residuals.
Despite the somewhat primitive nature of the available computing facilities, the development of an operational program to calculate the new econometric methods was high on Denis's agenda. I learned later from discussions with him how much effort was required in the programming, involving physically rewiring the computer, to ensure that the answers were correct. Denis carefully addressed the logic of the calculations to embed both estimators in a common framework and to ensure as efficient an iterative procedure as possible, including selection of the initial values and step lengths. He tried both stepwise and Newton–Raphson optimizers, analyzed their speeds of convergence—within and between iterations—and also investigated possible multiple optima. Overall, he set a high standard of scholarship for the treatment of the computing aspect of empirical research, in marked contrast to many later investigators, who do not even cite the software they use.
Denis provided a proof that stepwise iterative computations would converge with near certainty to a local optimum when the criterion function was bounded above and every step ensured that the function did not decrease. Such a proof includes, for example, the Cochrane and Orcutt (1949) procedure—although Denis noted that its convergence could be slow in dynamic equations—but does not include many stepwise iterations that seek a “fix-point” solution rather than the optimum of a criterion function.
Having developed the dynamic models, estimators, tests, algorithms, and computer programs, Denis applied this new analytical apparatus to the near-intractable problem of 1960s UK wage-price inflation. The main data set he analyzed was an extension up to 1960(4) of the quarterly seasonally unadjusted time series of Klein et al. (1961) over 1948(1)–1957(4). His wage equation was formulated in terms of real wages—as described in Sections 3.1 and 3.2—unlike, say, the Phillips curve. Denis used a data-based proxy for “inflation expectations,” which he called “an extrapolation of past price movements into the future.” His analysis highlighted the role of real-wage resistance in wage bargains, interpreting the equilibrium correction—the deviation of real wages from a productivity trend—as a “catch-up” mechanism for recouping losses incurred from unanticipated inflation. Permanent and transitory effects were carefully distinguished in his policy discussion to ascertain which changes would fade out (such as devaluations). But first, Denis checked that his model adequately described the evidence by testing for various misspecifications against the hypothesis that the residuals were white noise and independent of the instruments, as discussed in Sections 3.3–3.6.
To illustrate his general findings, it seemed of interest to estimate, test, and forecast with analogous equations on the closely related annual data documented in Hendry (2001). Figure 1 records these annual time series for the period 1945–1965. As can be seen in the top-left panel, wage inflation was above 5% per annum on average and higher than price inflation, and the top-right panel confirms that real wages (w − p) rose substantially over the period. GDP growth was also high after 1947 (bottom-left panel), and correspondingly, the unemployment rate (Ur)was very low, particularly by recent standards (bottom-right panel). In this notation, lowercase letters denote logs.
Inflation, real wages, GDP growth, and unemployment in the UK.
Using the annual data shown in Figure 1, an equation of the following form can be obtained over 1946–1965 (after lagging):
In (6),

is the squared coefficient of multiple correlation,

is the residual standard deviation, coefficient standard errors are shown in parentheses, and SC is the Schwarz (1978) criterion. The diagnostic tests are of the form

, which denotes an approximate

against the alternative hypothesis

for kth-order serial correlation (

; see Godfrey, 1978), kth-order autoregressive conditional heteroskedasticity (

; see Engle, 1982), heteroskedasticity (

; see White, 1980); the RESET test (

; see Ramsey, 1969); and a chi-square test for normality (

; see Doornik and Hansen, 1994). None of these diagnostic tests is significant.
Relative to equation (17) in Sargan (1964), price inflation plays a more important role in (6) but is close to a second difference (surprise inflation). However, unemployment is insignificant, with a small positive coefficient if entered, and so it was omitted. The equilibrium solution for real wages has an annual trend (approximating productivity growth) of about 1.4% p.a., which is close to Denis's finding. The feedback coefficient is important in (6) and clearly worth retaining given the small sample. The overall fit at

is poorer than Denis reported (

, not adjusted for degrees of freedom), suggesting that intrayear information is useful, a finding that is consistent with different groups of workers agreeing on their annual wage settlements in different quarters.
Finally, expressing the equilibrium correction as

and transforming the dependent variable to real wages, leads to the more parsimonious representation
In (7), there is “short-run erosion” of real wages from price inflation, so an “inflation surprise” formulation is in fact rejected. However, the price inflation effect was probably hard to notice at the inflation rates prevalent over most of the sample against the annual growth of 1.5% from productivity gains. The disequilibrium feedback automatically offsets some of that erosion. In any case, the coefficient of Δpt is probably not constant in the face of persistent high inflation but should decline to zero as inflation increases. Indeed, the forecast performance of (7) over the next decade of high inflation is very poor, even for one-step forecasts: testing parameter constancy to 1975 yields

(see Chow, 1960). In fact, most of this forecast failure is due to location shifts, as the equation in differences (i.e., dropping (w − p)e,t−1) delivers

. At the time, and for many years afterward, it was not known just how susceptible “equilibrium-correction models” were to shifts in the coefficients of deterministic terms: changes to the intercept and/or trend have a pernicious effect on their performance (see, e.g., Clements and Hendry, 1999). Income policies, wage freezes, and other related governmental interventions induce such shifts, so it is unsurprising that the original specification fails to characterize the changed process. We resume this aspect of the story in Section 5.
It is always difficult to understand why insights are so slow in coming. The argument from long-run homogeneity by which Denis justified the equilibrium-correction term did not immediately prompt analogies or other applications, perhaps because the Colston volume was an obscure source for economists—and Denis was certainly not a propagandist. Even closer to home, Hendry and Anderson (1977) did not notice that their theoretical derivation of the equilibrium behavior of building societies was an exemplar. Indeed, it was not till after rediscovering such feedback in Davidson, Hendry, Srba, and Yeo (1978) as “the vital ‘initial disequilibrium’ effect” that those authors realized they had an example of Denis's formulation (see, e.g., Hendry, [1993] 2001). Hendry and von Ungern-Sternberg (1981) named the class “error-correction” following Phillips (1957), but its limitation to adjustment within equilibria (and not between) was not foreseen.4
Ken Wallis has pointed out that another precursor is Marglin (1975), who called his consumption function a “disequilibrium hypothesis.” As noted earlier, in Rex Bergstrom's empirical disequilibrium models of economic growth, most of the equations also involved adjustment of an “equilibrium-correction” form around a long-run steady-state growth path, but this author for one did not perceive the generality of the formulation.
Starting from a general specification was at best implicit in the formulation of the “transformed” model against which the autoregressive error form was to be tested. In more recent parlance, that restriction corresponded to a common factor in the lag polynomials characterizing the dynamics of the unrestricted model (see Section 4). However, Denis's “practical methodology” still included trying a variety of specifications of variables, lag lengths, indicator variables, and data transformations, probably because the sample was too small to sustain useful estimates of the nesting model and computer power was then very limited. Like many empirical investigators, Denis scrutinized the internal consistency of his results and the relationships between them, an aspect eventually formalized by encompassing.
It also remained unclear how to generalize the treatment of autoregressive errors to higher orders—the theme of the next section—perhaps because the unrestricted model was derived, rather than postulated initially. Finally, although a variety of tests for model misspecification was developed and implemented, it was not obvious whether these provided a comprehensive evaluation—or were even well behaved (see, e.g., Kiviet, 1986); Section 7 provides an update. Thus, many of the ingredients of the “LSE methodology” were in place, although a great deal of research still remained to be done.
Much of Denis's analysis in the Colston paper concerned the role of autocorrelated errors, which he regarded as likely in models estimated from aggregate quarterly data. Such concerns were prominent in the interests of his colleague Bill Phillips (see, e.g., Leeson, 2000) and were also reflected in the research of his doctoral students (see, e.g., Trivedi, 1970; Hendry and Trivedi, 1972; Hendry, 1974). Both autoregressive and moving-average error processes were of interest, as were vector generalizations thereof. Two issues were involved: selecting the longest lag needed to induce white-noise errors; and “allocating” the dynamics between systematic components (which affected the mean lag) and error dynamics (which did not). The Colston paper had addressed both but essentially in the context of first-order autoregressive errors.
Sargan (1959, 1961) discussed estimating systems of dynamic equations with vector autoregressive errors. When the system was closed, there was an identification problem in extracting the error representation from the endogenous dynamics. Consider the n × 1 system in the endogenous variable yt:

with

. The “solved” form is

Then, Π and R cannot be uniquely separated from information on Ψ1 and Ψ2, as they correspond to the roots—which can be arranged in many ways. Notice that an unrestricted vector autoregression (VAR) was, therefore, considered as the general model.5
Antecedents again occur much earlier (see, e.g., Phillips and Quenouille, 1960).
In 1975, Denis generalized the estimation of scalar dynamic equations with first-order autoregressive errors to higher order errors on equations with longer lags. Espasa (1993) describes the circumstances surrounding this development. Denis read widely, so he almost certainly knew the results in Wald (1947) and also in Anderson (1962) on the optimality of a general-to-simple approach for an ordered sequence of nested hypotheses; he cited the analysis in Anderson (1971) supporting Gets for selecting lag length.
A maximum lag length must first be carefully chosen to reflect the context (e.g., data frequency), data properties, and economic considerations, perhaps restricted by a lack of degrees of freedom. Once that has been done, consider the least restricted autoregressive model:

The Anderson procedure commenced by first testing θk = 0, and if that was not rejected, θk−1 = 0, and so on, assuming that if θm = 0, then θi = 0 for all i > m. The reduction, therefore, ceased with the first rejection.
An autoregressive-distributed lag equation with a maximum lag length of k and n “exogenous” regressors can be written as

Similar considerations could be applied for some aspects of the problem, such as lag length. Denis noted that such an approach also applied to selecting the maximal common factor. Writing (8) in an obvious notation using lag polynomials as

then an autoregressive error of order r ≤ k can be extracted if (9) can be expressed as

where all the βi*(L) (i = 0,…,n) are now polynomials of maximum order k − r ≥ 0. Since

the terms (1 − λj L) are the common factors in the original lag polynomials βi(L) (i = 0,…,n); hence the name COMFAC. Thus, an ordered sequence of hypotheses is postulated, where the existence of a common factor of order m (for m ≤ r) clearly entails the existence of all orders m − l for l > 0. Conversely, rejecting order m ≤ r entails rejecting all higher orders.
The central achievement was developing an operational algorithm to determine the largest r that could be extracted, taking account of the possibility of complex roots in the polynomials (see Sargan and Sylwestrowicz, 1976). The resulting formulation was published as Sargan (1980), although an exposition appeared in advance in Hendry and Mizon (1978).
The most obvious feature of the COMFAC viewpoint is that the initial model (9) must be the most general needed to characterize the dynamics. This is essential, both to allow extraction of the appropriate factors and so that conventional inference procedures will apply validly. Mizon (1977b) (then at the LSE) explicitly investigated model simplification routes for testing the dynamic specification of an econometric model. He considered multipath searches and the nonordered and nonsequential nature of many of the hypotheses to be tested. He stressed the need to commence from the minimally most-general specification that embedded the theory-based model. Thus, Gets was becoming the norm at LSE.
Sargan (1957) was noted previously as one precursor to Denis's interest in the problem of interpreting large numbers of hypothesis tests. There are three aspects to his comments:
(1) the large number of coefficients being compared;
(2) the choice of critical values (or significance levels) for each individual test; and
(3) the role of nonnormal distributions.
These interacted to make him remark that
Having regard to the non-normality of the distributions of the variables and to the large number of regression coefficients that are being compared there is a probability of order 1 per cent that one or more of these coefficients should differ from their population values by more than four times their standard error. (p. 177)
Which nonnormal distribution Denis had in mind is not specified; presumably one with fatter tails than the normal. A simple approximation that remains symmetric but with potentially much fatter tails is a

with few degrees of freedom. For example:

satisfies his statement when even four tests are undertaken (

would do so with one test). By way of contrast, the corresponding probability for a normal is 0.0001, so about 100 tests would be needed to have a probability of 1% of observing a value in excess of 4.
Denis did not explicitly return to the problem of data mining until Sargan (1973).6
My mimeo copy from March 1973 is actually entitled “Model Building and Data Using,” an interesting difference of emphasis.
Second, he considered the possibility that the rejection is due to “overfitting” or “data mining”—to quote:
By conducting numerous tests, all at a conventional 5% confidence level, the econometrician may be biasing his predictions, or alternatively understating the standard errors of his predictions. If the number of parameters estimated is of the same order of magnitude as the sample size, there is indeed a considerable problem if the distribution of the prediction errors is derived from asymptotic sampling theory. Within the limitations imposed by this, it would seem better to formulate as general a model as is required to embrace all possible versions of the relevant economic theories that the economist is prepared to consider. (p. 160)
Personally, I imagine Denis in the LSE coffee bar, sketching his analysis on the proverbial back of an envelope—students have recovered complete lectures sketched that way—with most of the following steps missing. Here they are included, mainly to define notation. Let

where there are n regressors; then

Let

. Then

However, he probably began (and ended!) by using an approximation like

Thus, the asymptotic formula σu2 understates the standard error of prediction by 100q%. One suspects he had a value like q = 0.5 in mind. Nevertheless, he still argued for commencing from the general model.
Denis then proceeded to discuss the “repeated testing” problem, noting the result in Dunn (1959) that the impact of chance significance in repeated testing from a

with (say) 120 degrees of freedom and 50 additional irrelevant variables could be controlled at 5% by setting a critical value of 3.38 (roughly the 0.1% level for each test, as (1 − 0.001)50 ≃ 0.95). Denis appears to be trying to design an “optimal procedure” for estimating a model when a subset of variables may not matter. He proposes a two-step procedure in which a pretest estimator is first calculated but the decision on whether to accept it is based on a loss function. He notes that the estimator associated with (say) Dunn's critical values need not minimize a quadratic loss function. He also considers a Bayesian approach with informative priors, but (from a missing appendix) he appears to conclude that the resulting estimators may be highly affected by changes in the prior distributions. He relates decisions to possible loss functions that might sustain alternative approaches. The issue of model size is noted—this comment suggests that Denis was already at work on Sargan (1975).
Referring back to Sargan (1957), the suggestion in Dunn (1959) necessitated that the errors on the fitted model were indeed approximately normal; as seen above, if fat tails were common, then so would be larger

(as they would, of course, not be drawings from the Student

with the anticipated degrees of freedom). This argues in favor of transformations that allow normality to be a useful approximation, so that a “tight-tailed” distribution can be used, such that reasonable critical values entail stringent significance levels; that was certainly part of the oral tradition among LSE econometricians.
As noted previously, Sargan (1964) viewed autoregressive errors as a restriction on a more general dynamic equation, but he nevertheless estimated a considerable number of separate specifications. He carefully considered the model choice between log and linear representations, and between systematic and error dynamics, but did not formally address other selection issues. However, his continued interest in model selection was reflected in Grayham Mizon's thesis, some of which was published as Mizon (1974, 1977a).
Denis presented an overview paper on model selection to the LSE econometrics workshop in 1981 (Sargan, 1981) that discussed model-selection criteria, focusing on that proposed by Schwarz (1978). The use of

deriving from (11) is reminiscent of the Akaike final prediction error criterion (see, e.g., Akaike, 1969), “penalizing” each additional parameter. Another criterion also bearing a close resemblance is the (probably oldest) model selection criterion, namely, the unbiased residual variance criterion that Theil (1961) proposed—so that aspect was well understood. Denis's decision to investigate this issue even when decisions could not be ordered may have derived from Hannan and Quinn's (1979) ability to establish that the consistency conditions for selection rules applied to autoregressions, although there is no explicit reference to their paper. Denis shows that the Schwarz criterion is not the weakest that ensures a consistent model selection, and his analysis finds no reasons for setting the constant of proportionality (c in Hannan and Quinn, 1979) at unity. Moreover, he references neither of the closely related papers on this issue by Shibata (1981) and Atkinson (1981), which reached similar conclusions.
7Timo Teräsvirta pointed out that the Schwarz criterion was independently derived by Rissanen (1978) using coding theory, but it is unlikely that Denis had seen that paper in 1981 because Rissanen published in Automatica.
Early research on Gets sought to establish optimality properties of selection procedures in structured situations, such as ordered sequences of nested hypotheses (see, e.g., Anderson, 1962, 1971). This was generalized to autoregressive-error extraction in COMFAC, but as Mizon (1977b) stressed, the optimality of simplification approaches could not be shown for unordered sequences. Equally, the nonviability in most settings of (say) simple to general modeling was clear, since it could fail even in cases where Gets was optimal.
The theory of reduction, elucidating the information losses that could arise by (e.g.) conditioning and marginalizing, was being carefully investigated by a group at CORE, Louvain-la-Neuve, Belgium (see, inter alia, Florens, Mouchart, and Richard, 1974, 1976; Florens and Mouchart, 1980, 1985, consolidated in Florens, Mouchart, and Rolin, 1990). Jean-François Richard (from CORE) visited LSE in 1973–1974 and added these ideas to the developing “LSE methodology.” The integrated strand that evolved led to a series of papers on exogeneity (Richard, 1980; Engle, Hendry, and Richard, 1983), model formulation and a taxonomy of evaluation information (Hendry and Richard, 1982, 1983), and the theory of reduction as explaining the origin of empirical models (Hendry, 1987). Another visitor to both institutions, Adrian Pagan, who had earlier also worked on estimating models with autocorrelated errors (see, e.g., Pagan and Nicholls, 1976) and model evaluation, contributed to formalizing the model types and their properties (see Hendry, Pagan, and Sargan, 1984). Useful reviews of this methodology prior to the implementation of computer-search methods are provided by Gilbert (1986) and Ericsson, Campos, and Tran (1990); more recently, see Mizon (1995).
However, the computer automation of simplification procedures did not originate at LSE, perhaps because the approach in Coen, Gomme, and Kendall (1969) was not received very enthusiastically. To my knowledge no one at the LSE attempted to develop an automated Gets approach, although the complete implementation of COMFAC did represent a special case.8
Ken Wallis reminded me that Mike Garside worked at LSE on developing efficient algorithms for computing all possible 2n − 1 regressions.
Two strands of research have since established the fruitfulness of automatic selection. Following up on Phillips (1994), Phillips (1996) enunciated the principles for an automated approach to model reduction with evaluative testing that allows for parameter evolution and (e.g.) reduced rank (equilibrium correction) in systems, in addition to single equation models. His approach has been successfully implemented in empirical models for forecasting the U.S. and various Asia–Pacific economies (see e.g., Phillips, 1995; Schiff and Phillips, 2002, a recent overview is provided in Phillips, 2002).
The other stimulus was the work of Hoover and Perez (1999), who demonstrated the properties of their search algorithm by Monte Carlo simulation. Extensions of their approach are discussed in Hendry and Krolzig (1999, 2001, 2002) and Hoover and Perez (2000). The main finding of these Monte Carlo studies is that the null rejection frequency of the Gets simplification procedure is close to the nominal size selected, and the power is close to that obtained when commencing from the correct specification with no irrelevant variables. Whether the selected model is too large or too small depends on the critical values set by the user and the nature of the correct model, but neither is intrinsic to Gets. The estimated fit (equation standard error) is approximately unbiased (within ±5% depending on the nominal size selected).
One of the issues raised in Sargan (1957) was the interpretation of models after many hypotheses had been tested, representing Denis's worry about the adventitious significance of coefficients. Formally, the overall “size” of a repeated testing exercise with n independent tests each at significance level

when the null is valid is

, which rapidly escalates in n and

. A well-used example is

. This seems an excessive size, especially relative to the simple expedient of first testing the overall null of no effect before conducting individual tests. Alternatively, from the perspective of selecting a model, a more useful interpretation may be the average number of irrelevant variables retained from the large candidate set initially considered, namely,

. For our example,

, interpreted as one variable being retained on two occasions out of five that the general model contains 40 irrelevant regressors, with none retained on the remainder. Thus, dramatic reductions are achieved in both cases. In view of the potentially large costs of omitting relevant variables, Denis's advocacy of Gets seems supported.
Given his quiet, unassuming approach, and his location at what had earlier been a central location for the denigration of econometric evidence, Denis Sargan at the LSE was an unlikely candidate to radically alter the econometric approach of a generation. Yet without doubt he did so, establishing a powerful approach to empirical modeling of economic time series. This paper has outlined the main developments in econometric methodology that he both initiated and undertook, and also some of the subsequent research. But, as the other papers in this volume demonstrate, that represents merely one of his many research strands. Denis has left a splendid legacy of intellectual achievement, in addition to cohorts of well-trained students, many of whom have continued to advance the discipline across, and beyond, the range of topics on which he left his mark.
Inflation, real wages, GDP growth, and unemployment in the UK.