Memoir by Kenneth Arrow

“The National Bureau of Economic Research, then in New York, was a dominant center of research activity when I was a graduate student at Columbia University in 1940–1942. Its leader was Wesley C. Mitchell, a professor at Columbia, and his two chief associates were Arthur F. Burns (then a visiting professor at Columbia, later chairman of the Board of Governors of the Federal Reserve System) and Simon Kuznets, a professor at the University of Pennsylvania, I believe. It was highly empirical and rather antagonistic to theory; its research centered on economic fluctuations, then called “business cycles.”

Somehow, within this framework, an Econometric Seminar was created. It was directed by Jacob Marschak, then a professor at the New School for Social Research. In the fall of 1941, I was invited by Marschak to join the seminar; I presume, though I do not know, that I had been recommended by Abraham Wald. The seminar was dominated by Europeans, recent émigrés, like Marschak and Wald. I attended several of its meetings at the New School; they were devoted to mathematical economics and to econometric issues.

Although the National Bureau's main center was located in midtown Manhattan, the bureau had the use of an estate, called Hillside, in a more rural area of New York City, perhaps ten miles north of the bureau's main center and seven miles north of Columbia University.

In January 1942, I received a notice that the Econometric Seminar was to have a meeting at Hillside, rather than the usual location. It was to discuss the uses of statistical inference in economics, and the topic was to be “introduced” by Trygve Haavelmo. The name was familiar; as I recall he had published a paper in Econometrica on the problem of testing a business cycle theory by comparison with data.

The paper distributed with the notice seemed utterly correct but bland to me. I had been studying statistics with Harold Hotelling and Abraham Wald (at Columbia) and had also been reading mathematical statistics on my own, beginning as an undergraduate. I knew about the work of R.A. Fisher and that of J. Neyman and E.S. Pearson. What Haavelmo was proposing was simply to apply these ideas to economic inference.

The seminar took place on February 14. I do not have the clearest memory of it, of course. I was happy to see a presentation of the view of modern statistics to the National Bureau, whose statistical methods I was learning with considerable disgust. Haavelmo's presentation was very lively, with the combination of informality and depth that I later learned to associate with his work. The only person there who seemed to grasp the full implications of Haavelmo's viewpoint was a very tall and very thin scholar, who was identified as Tjalling Koopmans. I had known him by name, also through an Econometrica paper, on the distribution of the serial correlation coefficient (which Hotelling has been stressing as a major unsolved problem). His questions and remarks made me realize that more than routine modeling and application of standard theory were involved.

I must finally note the presence of Joseph Schumpeter, sitting in what I can only describe as lordly fashion in an armchair (if memory doesn't mislead me). He was acting like a very considerate nobleman among his peasants. His main remark was that Haavelmo was trying to fit relations that were stable over time, but economic activity was dominated by the changes in the production structure. (I had been reading Schumpeter's Business Cycles in Burns's course on business cycles, so I understood how he was fixated on his own work.)

Although I was otherwise preoccupied during the war, I did follow Haavelmo's publications in Econometrica and also the accompanying work of Henry B. Mann and Abraham Wald. In 1946, I was invited to a conference held by the Cowles Commission for Research and Economics (now headed by Marschak, with Koopmans as a leading member). The conference was held at Cornell University (in connection with a meeting of the American Statistical Association). By then, the research program started by Haavelmo had moved very far indeed, with elaborate discussions of full-information maximum likelihood estimates (to be computed on desk calculators!) and the problem of identification.”

Seminar Announcement by Sidney S. Alexander

“Hillside”, West 254th St. and Independence Ave., Riverdale, New York, January 20, 1942

Dear Sir:

The Econometric Seminar will meet at the Hillside office of the National Bureau of Economic Research on Saturday, February 14th, at 10 a.m. The session may be expected to continue through the afternoon. The seminar will engage in a general discussion of the validity of some current objections to the use of statistical inference in economic research. The main emphasis will be centered on the following, with an attempt to define them precisely and to clarify their content:

1. Is there any danger in considering economic data as stochastic variables? Is there any use in so doing?

2. Is it always objectionable to use free-hand methods of curve-fitting?

3.

4. Is it more dangerous, in general, to draw inference from economic time series than from other kinds of statistical information?

Mr. Trygve Haavelmo will introduce the discussion by suggesting answers to these questions. Professor A. Wald will participate in the discussion.

References: Neyman, J.—“Lectures and Conferences on Mathematical Statistics,” Department of Agriculture (Washington, 1937); Koopmans, T.—“Logic of Business Cycle Research” (J.P.E., 1941); Haavelmo, T.—“On the Theory and Measurement of Economic Relations” (mimeographed, Cambridge, 1941). (Copies available at Main Library at Columbia, South Hall; Widener; at Hillside; or at library of National Bureau of Economic Research.)

Lunch will be served at the usual charge of 65 cents. If you expect to attend will you kindly notify me as soon as possible, so that proper arrangements can be made. If you wish to be met at the 8th Avenue subway terminus, northwest corner of Broadway and 211th Street (Isham Street) at 9:30 Saturday morning, or if you wish to stay overnight at Hillside, please inform me accordingly.

TEXT OF THE 1942 HILLSIDE SEMINAR BY TRYGVE HAAVELMO

We are to discuss the following methodological questions:

1. Is there any danger in considering economic data as stochastic variables? Is there any use in so doing?

2. Is it always objectionable to use freehand methods of curve fitting?

3.

4. Is it more dangerous, in general, to draw inference from economic time series than from other kinds of statistical information?

These questions are taken, more or less, directly out of current discussions on methods in economic research. They are questions which, I believe, bother modern economic research workers a great deal. And when the economists ask modern statisticians for help, they are often shown away at the door, so to speak. For the statisticians say, these questions are just imprecise phrases; they are not scientific. And, why should we bother, when you economists cannot even ask clear and intelligent questions? Well, I agree to the first part of this statement: The questions are unclear and imprecise. But I do not agree that they are not worth bothering about, because:

First, I don't believe that so many serious research workers would discuss these questions time and again without there being some real profound ideas contained underneath.

Secondly, even if this were not the case, it would be worthwhile to search for a clarification to save efforts for other purposes, and

Thirdly, to those who, correctly to be sure, scorn the economists for their imprecise formulations, I would say this: In economics, as it stands today, the biggest problem is in fact the very formulation of the problems we are interested in. When we succeed in this, we are, in many cases, very close to the solution also.

From this you will then see what kind of a task I am up against, when it is stated in the program that I am to suggest answers to the questions listed. First, I must say that I can not claim to understand the general ideas behind the questions, although I had a part in posing them. (And this was one of the main reasons why I thought this discussion might be useful.) If I, therefore, tried to start with the answers, I should not know what I was answering. So if I might be able to contribute anything to this discussion, it would be this: I shall try to interpret the questions in a simplified way, which to my mind seems to cover part of their content, and try to draw certain conclusions from this formulation.

For this purpose I shall first have to say a few words about what I believe to be the nature of econometric research and how we are led to the idea of using the tools of probability theory and statistics in this field.

If we think of what we are trying to do in econometric research, we see that it comes to this:

For example: We see that people work, receive money, go to the stores to buy goods at certain prices, put money in banks at certain interest rates, etc. We ask whether—as a result of their actions, in which we believe there is a certain amount of planning and thinking—there will emerge certain more or less stable laws combining the quantities: number of hours worked, wage per hour, amount of goods bought of various kinds, prices etc., etc.

In general, therefore, what we are trying to do in economic research is this: We focus attention, simultaneously, on a group of several such measurements, in order—as we put it—to see whether or not these measurements are somehow linked together.

What is at the bottom of this idea? It is this: We are looking for some kind of mechanism which produces the observed phenomena. We ask: How did this and that constellation of facts come about? That is: We try to repeat, mentally, the process by which the things we see might have been produced. The use of this is that if we had been able to put up such a mechanism before we knew the measurements now at hand, we could have made some prediction about them. This leads us to think that, maybe, we could predict some new measurements by believing in the same mechanism.

It seems natural to do this: We think of each system of measurements as a result of some kind of experiment. Now it would be nice if for each type there were only one mechanism which could have produced them, and such that no two mechanisms produced the same result. But we know that we don't get very far in the direction of useful results if we take this point of view. In fact, we get nothing but a description of a particular set of data. We don't get any possibility of inference. In order to get that, one has to try a more risky procedure, to attribute different looking systems of data to one and the same mechanism, and sometimes, the same type of data to different mechanisms.

We might illustrate this schematically: Suppose, for simplicity, that there are only two different mechanisms which might be at work: mechanisms I and II. And suppose we have two sets of measurements, one produced by each mechanism, but we don't know which. Then the data themselves might not tell us anything but this:

Now we might have the experience, or for a priori reasons we believe, that mechanism I produces data of type 2 only very rarely and mechanism II produces data of type 1 also very rarely, and that, therefore, measurements of type 2 are usually produced by mechanism of type II. We therefore say that, when data of type 2 are at hand, we are almost sure that they are produced by the type II mechanism. Now suppose that we have observed data of type 2. And we want to predict. We say: Mechanism II usually does not produce data of type 1, although it might. I, therefore, believing that mechanism II will continue to work, predict that the new measurements will be of type 2. What are the risks involved in this statement?

They are:

1. The first observations might actually have been the results of mechanism I.
2. It is not known which mechanism is going to produce the next set of data, even if we are practically sure as to which mechanism produced the first set.

In other words: by means of a set of observations we might be able to say something about the possibility of mechanism II producing the results of type 1, but we might not have any idea which mechanism is going to be the producer. Evidently our only hope, whether we use this or that technique, is this: We try to establish which mechanism has been at work in the past and to calculate, somehow, the risk in this decision, and then for prediction, we have to hope that the same mechanism will continue. If we experience that the second hypothesis, as to the persistence of one and the same mechanism, does not work, and we have no additional information as to when and how this shift in mechanism takes place, we simply have to quit.

This risk of predicting the persistence of a certain mechanism will be at hand, no matter what methods we adopt, probability or something else. The common basis for all our attempts to establish laws about observable phenomena must therefore be this: We must accept as an axiom that for each type of measurable phenomenon there is a certain constant, but unknown, mechanism at work. Then we may try to draw inference from observations as to what this mechanism is, try to evaluate the risk involved in this inference when used as a basis for prediction.

Now, the term mechanism is just a phrase. We need a more specific definition of what we mean by a mechanism. We need some conception that permits us to classify all conceivable observation-producing mechanisms. In other words, there must not be any type of observation which cannot be referred to one or the other of the mechanisms contained in our system. The choice might be purely conventional, since this idea of a mechanism, though it has helped science a lot, is, of course, an abstract notion that we create ourselves. But the conception chosen should have objective operational properties, and it should be useful.

Now the notions of probability distributions and stochastic variables meet the requirement of an exhaustive classification of mechanisms, for whatever the type of observations, they can always be referred to some kind of universe or probability distribution.

The question is: What is the use of introducing the purely theoretical notion of probability, for it is a purely theoretical scheme, that we must remember. In other words, what is the link between probability calculations and practical statements about facts? First, what is the nature of the practical statements we wish to be able to make about future observations of certain phenomena? By closer inspection we see that we can transfer practically all such statements back to the following type: Suppose we are concerned with n future measurements of a complex of m variables, i.e., all together m·n = N measurements. The result of these measurements may be represented by a point in an N-dimensional euclidian space. We want to be able to say this: We are almost sure that this observation point will fall into a certain specified subset of the N-dimensional space. For example: Suppose that R is national income and X is total spending (see Figure 1).

Through various kinds of studies and theoretical speculations we may have come to the conclusion that it is a very safe bet to say that the next joint observation of X and R will fall within the belt A. Usually we are not satisfied by merely guessing this directly. We usually try to start way back, with certain simple assumptions that we think are fairly safe, and then we try to deduce the statement mentioned. But as we know, it is not always easy, sometimes it is even humanly impossible, to see the implications of certain original assumptions without having a certain logical operational system to safeguard us against inconsistencies. That is why we have mathematics, and find it useful, even though, as some people put it, mathematics does not yield anything new. Now probability calculus is such a system. In this system we start by giving a name to the variables to be studied; we call them stochastic variables. Then we ascribe to these variables certain attributes called probabilities. At that stage these magnitudes might not have any practical meaning or counterpart. (It is the fortune of mankind to have discovered that such a bold and ultra-factual start might often lead to useful practical conclusions. The history of science is full of examples illustrating this.)

Now, having laid down certain basic hypothetical probabilities, we operate on them by the rules of the probability calculus to reach a statement that can be given a real practical meaning, namely, a statement which, when applied to the observable phenomenon, can be considered as almost certainly true. The counterpart to such a statement in probability terms is: The probability of the statement being true is close to 1. Here is where the logical deduction and the practical conclusion meet.

Let me continue with the example above to illustrate this. If we have the idea that spending is approximately given when we know R this idea can, as far as I see, not mean anything else but this: A priori we assume that there exists some band as illustrated in Figure 1, and as narrow as possible in some sense or another, and such that if we knew it, we could say, almost with certainty of being right, that a pair of observations X, R must fall within the band. In probability terms: There exists a band somewhere in the X-R diagram such that, given R, the probability is almost one that the point, X, R will fall within that band. Now what we want, of course, is to draw some inference as to the location and the “thickness” of this band. But then we realize soon that it is practically impossible to keep track of what we are assuming and doing without adopting some kind of probability model.

For we certainly do not think that the scatter of points observed—and from which we want to draw some inference—is the only possible set of observations. But then it becomes evident that we must adopt some kind of scheme or model that tells us how such data as those of X and R are in general produced. Otherwise there is—at least to my mind—no conceivable hope one can say anything from a set of observations, beyond describing where each of the actually observed points is located. This scheme must have the following properties in order to make sense: First it must be consistent with the scheme that for one and the same income we can have a different amount of spending. Otherwise data will almost certainly reject our scheme in the first round. Secondly, the scheme must be such that, although R being given, X might be anything, some guesses are better than others; i.e., they are more often true than others. Thirdly, the scheme must permit generalizations, such that we can make a statement for any value of R, within a certain range at least. Now, I do not know of any scheme which has these properties and besides is logically self-consistent—except the one of probability. So, although one can never say that nothing better might be found, so far I guess we haven't any choice, if we don't want to quit.

Let me turn now to one of the problems in connection with time series, although, as I see it, not the major problem in this connection. (That I shall come to later.) I am thinking of the frequent jeremiads among economists over the bad fortune that “time series cannot be considered as independent observations.” I think that there is something really profound hidden in this statement, but I think that the real trouble is something different from that I am now going to describe. Still I mention this one, because I have the impression that many economists think this is the trouble.

Let me begin with the most superficial objection against time series, which I have sometimes heard. It is this: Consider our example with X and R above. Suppose that we write

where we by hypothesis assume ε to be a random residual of some kind. Then I have heard this objection: We cannot use the least squares method, because if we look at the series for X and R we see that they are cyclical in time; therefore the probability of δ₁ < X_t < δ₂, say, is not independent of X_t−1,X_t−2, etc. I do not know how this idea has come about, but it certainly has no bearing upon the least squares method. It is, of course, not the variables X and R themselves which have to be independent through time; it is only the ε's that are assumed to have this property, and even this is only in the simplest least squares scheme. (I suggest reading Neyman and David, “Extension of the Markoff Theorem on Least Squares,” in Statistical Research Memo II, London, 1938.) Another, somewhat more serious, objection is that it is, usually, not admissible to assume that the ε's, i.e., ε₁,ε₂,…, ε_t,…ε_N, are independently distributed. This happens rather often, I think, but it is not necessarily any disaster. There are two simple procedures that may often remove this trouble: One is to look for additional independent variables, for instance, in our case the cost of living, or lagged values of R, etc. The other is simply to admit some serial correlation as a characteristic property of the ε's without attempting to exploit explicitly the components that, together, make up the ε's. For instance, in one example, it would be quite natural sometimes to say this: If people at a certain time spend more than normally—which here is supposed to be the amount aR_t + b—they may continue to do so for some time, and then, if the observations are made at short time intervals, the ε's could not be assumed to be independent. But if this appears to be a characteristic behavior of people, there is no reason why we should not try to take account of it explicitly in formulating the theory. For instance we might change our hypothesis by introducing the following idea:

Suppose that people at point of time t have an excess consumption of ε_t. Then, maybe, at t + 1 a certain percentage α is still left of that effect, besides new disturbing effects which might come in at that moment. That is, we would have the following dynamic setup:

After this transformation it might be that even a simple hypothesis about the y's would work, e.g., the assumption that the y's are independent normally distributed with 0 means and the same variance σ². Then we have

Now assuming ε₀, or what amounts to the same—for a constant set of R's—X₀ = constant, we have the distribution of the observable variables X_t, namely,

And then it is a technical matter to find “best estimates” for a and b. Compared with the simpler setup where the ε's themselves are independent, the loss in number of degrees of freedom is just 1, because of the extra parameter α. Of course, in general, the more parameters we have to introduce to describe the distribution of the ε's the more severe the loss in number of degrees of freedom. But the idea which some people have, that we must throw away those observations which are stochastically dependent on those already included, is certainly very inefficient; it is a waste of information.

I don't think that, either from a logical or from a practical point of view, there is anything particular to time series that should prevent the use of methods of statistical inference. The real trouble with time series, from the point of view of economic research, lies in the fact that they are passive observations in a system ruled by many simultaneous relations. This I shall discuss presently.

But let me first at this point give a small discussion about freehand methods of curve fitting. I do not think it is possible to show that those who use freehand methods always get results of less practical value than those who use “exact” methods. In fact, maybe, the freehand people have more luck in predicting actual happenings than those who go through exact procedures. So, on that basis it is probably not possible to object to freehand procedures. But I think one can say this: Freehand methods, just as much as exact methods, depend essentially upon some a priori notion of some probability mechanism. And if the freehand man were forced to describe the meaning of his results in finite terms, this meaning could be transformed into probability terms, and then one may be almost certain that one could prescribe a corresponding exact method that, on the basis of the probability model involved, would lead to fewer erroneous decisions than the application of the freehand result.

Now let us make one broad remark about spuriousness. Of course, there is no such thing as a spurious correlation or a spurious relation between actually observable variables. A result becomes spurious only if we take it to mean something which it looks like, but which it actually isn't. So the spuriousness lies in our interpretation. For example, those strange correlations of Yule are not funny at all, I think. First of all, I think hardly any book on correlations, even old ones, fails to warn against interpreting a correlation as something causal without a theory behind it. Besides, from a macroeconomic point of view, it is quite natural that one could establish relations between almost any things. One might even be able to deduce those strange correlations of Yule.

To get at the real meaning behind such expressions as “Spurious relations,” or “just a statistical coincident,” or “economically meaningful,” I think we have to examine a little more closely the nature of relationships we operate with in economic theory. The relations which interest us most, and those which we are most eager to verify, are so-called behavioristic relations. When we try, theoretically, to deduce such relations, I think we almost always start out by imagining a certain kind of experiment. For example, we say: Suppose that a man was given a certain raise in income; how much of it would he spend?

Or, suppose that we were able to send twice as many customers to a shopkeeper as before; how many more new orders would he place; by how much would he enlarge his personnel and his storeroom, etc. These are the kind of relations that we are interested in, first because from such relations, if we know them, we could deduce certain observable facts, and secondly if we want to estimate the effect of certain active economic policies, these are the kind of relations that we need.

Now, the fundamental question comes: Can we get at these relations, if they are really true, by a passive observation of the game?

To make the whole thing more concrete I shall return to my previous example with the propensity to consume relation, to see what the implications are when this relation is not the only relation between the variables involved. For the time being I shall assume that my relations are exact, no random errors involved, since this is unessential for the general idea I am trying to explain here. Suppose that I consider the following relations:

a, b constant.

The acceleration principle:

k constant.

The closed market condition:

Suppose it is actually true that if consumers were given the income R_t they would consume according to (1). And suppose it is also true that entrepreneurs react to consumers' demand according to the relation (2). Could we infer this from the time series for X_t, R_t, and I_t if we “let the system loose.” The fact is that we could not, and I am going to show this. More precisely: I shall show that, if we consider our system as a hypothesis, and even if it is absolutely true that (1) and (2) describe the behavior of consumers and entrepreneurs, we shall not on the basis of time series for X, R, and I be able to reject other hypotheses which say that consumers and entrepreneurs have different behavior than the one described.

It is sufficient to give one example, and I shall choose one that many economists would consider important. Assuming that my system of relations is true, I can from that deduce the following relations:

or

Therefore, if our first system really is the one that describes the behavior of consumers and entrepreneurs, but we don't know it, then we shall not be able, on the basis of the data, to reject the alternative possibility that consumers in general behave according to (1′).

Suppose then that we decide that consumers behave according to (1′). As long as the system is left alone, there is no harm in this, although the relation (1′) is spurious, if we think it describes how consumers would behave under arbitrary income variations. But the situation becomes serious if we want to use it to predict the effect of government spending, say. Because then, either I is total investment including government expenditure, and is not given by (2), or I is private investment only, and then R is not given by (3). But both (2) and (3) are used to deduce (1′), and there is absolutely no reason why (1′) should continue to hold if (2) or (3) or both cease to be true. But there is no reason why (1) should not hold.

Now, we can in this case improve the situation somewhat by studying other implications of the fact that we have a system of relations and not only one relation in isolation.

For example, from (1′), (2), and (3) we deduce

But from (1), (2), and (3) we deduce

Thus, if (1′), (2), (3) is true and there is no particular restriction on the parameters A and B, then in general no difference equation of order lower than two will fit the R series. That is shown by (4). While, if (1), (2), (3) is true then even a first-order difference equation, namely, (5), would fit. So we can say this: If the basic equation that describes consumers' reactions to income is of the type (1′), the parameters A and B would have to have some very particular values in order to produce a time series for R that would satisfy a first-order difference equation. So if, by considering A and B as free parameters together with k, we did not get such particular values of A and B, we could exclude the possibility (1′). But the possibility, although one might call it unlikely, remains that both (1) and (1′) would satisfy the passively observed time series for X, R, I, while, if the government started to experiment with public spending, we should discover that either (1) or (1′) had to be rejected as a description of consumers' behavior.

What can we do about this? The example above suggests that

1. First of all, in such cases we should draw all the implications of a complete system of relationships and in that way rule out as many hypotheses as possible.
2. Even if that be done, there will usually be arbitrariness left as to the choice of fundamental relations so far as the observations are concerned, and here is where the question of “economically meaningful” comes in. Here I cannot suggest anything else than this: No matter how refined our analytic tools may become, there will always be a difference between good economists and bad ones.

My example above illustrates what I think is the real difficulty with economic time series or, more generally, with drawing inference from purely passive observations. The situation described—a very frequent one in economic research—reminds one of a joke scene I watched in the movies the other night (it is probably familiar). Two fellows, somewhat unsteady on their feet, were making a raid on the icebox in their apartment, and inspired by a few additional stimuli from the box, they got into an argument as to whether actually the light in the box went out when the door was closed. From the outside, of course, it was impossible to reject either the light hypothesis or the blackout hypothesis. Finally, finding it impossible to test either hypothesis by passive observation, one of the fellows entered the box while the other closed the door. And they proved the theorem.

If economists could do something of that sort, I think many of their arguments could be cooled off too.