2. Economic Origins: Models of Discounting

Since the early days of their discipline, economists considered inter-temporal decisions an important topic. They focused on two aspects of inter-temporal choice in particular. First, that people typically prefer immediate to delayed consumption of the same goods bundle, and hence they discount future consumption in accordance with its remoteness. Second, that people's choices are often inter-temporally inconsistent, for example in the sense that people prefer a larger, later consumption bundle over a smaller, sooner one as long as both are sufficiently distant in time, but change their preference to the smaller, sooner bundle as both draw nearer.Footnote ¹ However, these discussions typically proceeded without the help of formal models; and where they did, they could not take recourse to the modern non-hedonic notion of utility. That notion was developed only during the 1930s, with a widely accepted formal basis provided by von Neumann and Morgenstern as late as 1944 (Fishburn Reference Fishburn1989; Moscati Reference Moscati2013). For these reasons, the earlier discussions of inter-temporal decision making never produced a formal model of a temporal discount rate of non-hedonic utility.

That role instead fell to twenty-one year old Paul Samuelson.Footnote ² Samuelson (1915–2009) entered the University of Chicago in 1932 and then moved on to Harvard Graduate School, where he studied under Joseph Schumpeter and the physicist and mathematical economist Edwin Bidwell Wilson, among others. In 1937 while still a graduate student at Harvard, Samuelson published his first scientific article, “A Note on Measurement of Utility.” There, he presented a model of intertemporal choice where the economic agent behaves so as to maximize the sum of all future utilities, appropriately discounted by their distance in time.

Before describing it in more detail, let me first stress that this model was merely a by-product of the 1937 paper. Samuelson's main intention was to provide behaviorist foundations for the measurement of utility, thus contributing to the then vigorous debate over the determinateness of the utility function (Wong Reference Wong2006; Moscati Reference Moscati2013).

One of these “different assumptions” is the exponential discounting model. It specifies a decision maker's inter-temporal preferences over consumption profiles (c_t,..c_T) from time period t to time period T. Such preferences, Samuelson argues, can be represented by an utility function U^t(c_t,..c_T) which takes the following form:

(1)$$\begin{equation} U^t \left( {c_t , \ldots ,c_T } \right) = \mathop \sum \limits_{k = 0}^{T - t} \frac{{u\left( {c_{t + k} } \right)}}{{\left( {1 + \rho } \right)^k }}\end{equation}$$

where ρ>0 is a person's discount rate.

By specifying his model in this way, Samuelson made two trend-setting decisions. First, he offered a unitary concept of time preferences: economic agents chose smaller consumption profiles over greater delayed ones because they have a preference for immediate gratification over delayed gratification, expressed by ρ. The model thus implies that discounting is a stable unitary factor that operates identically to, and applies equally to, all sources of utility, and furthermore that this factor is located in what economists consider the motivational facilities of economic agents – in contrast, for example, to their information-processing facilities.

Second, Samuelson assumed that the rate of discount of future utilities is constant, namely ρ. Thus, his approach explicitly models the discounting of future consumption, but it excludes the possibility of inter-temporal inconsistencies.Footnote ³ Whereas temporally inconsistent agents change their preferences as they progress in time towards the point of consumption, the agents in the exponential discounting model retain their preferences, independent of the specific point in time they find themselves at: “the results are unchanged even if the individual always discounts from the existing point of time rather than from the beginning of the period” (Samuelson Reference Samuelson1937, 160).

Indeed, this perspective-independence seems to have been Samuelson's main reason for proposing (1) as the form of the discounting function. He stressed repeatedly the “arbitrariness of these assumptions,” emphasizing that it is “subject to refutation by the observable facts” (ibid., 156). In line with his declared goal of “bringing out certain theoretical relations” (ibid., 155), rather than “furthering inductive investigation,” he did not provide any empirical evidence: the model describes a (necessary part of a sufficient) condition for the measurability of the marginal utility of income, not a positive fact or a normative requirement.

Nevertheless, Samuelson elsewhere in the paper offered considerations towards the plausibility of the exponential discounting model. The specific form of the inter-temporal utility function, he argued, might well depend upon socially determined parameters, including desire for social prestige, length of human life, or banking and investment structures. Changes in such parameters are unlikely to be “of an equilibrating nature” (ibid., 160), but instead require historical investigations, to which abstract mathematical modeling has little to offer. Only the utility function of form (1) is the result of an equilibration: it is the only kind of discounting which avoids inter-temporal inconsistencies, and hence is more salient than any other form. Samuelson here remained somewhat vague, but it seems that this salience, rather than a genuine normative rationality consideration, motivated him to suggest this particular form of discounting.

In its main intended purpose, the 1937 paper quickly was superseded by Samuelson's seminal 1938 “Note on the Pure Theory of Consumer's Behaviour.” There he put forward his own brand of behaviorism, without any role for the discounting model, and it is this version that most of his contemporaries came to accept as the revealed preference approach to consumer demand. In the next decades the 1937 paper languished in relative obscurity, and the exponential discounting model with it.Footnote ⁴

Nevertheless, the purposes for which these papers were written were similar. In both Samuelson sought to develop behaviorist foundations for preferences and utility, with the intention of “dropping off the last vestiges of [hedonic] utility analysis” (Samuelson Reference Samuelson1938, 62). The purpose of these models was thus more or less directed against psychology (Giocoli Reference Giocoli2003; Wong Reference Wong2006); they substantially contributed, as Giocoli writes, to economics' “escape from psychology . . . the efforts in the first half of the twentieth century to cleanse the neoclassical theory of economic behaviour of any reference to psychic concepts such as pleasure, motivation, utility and so on” (Giocoli Reference Giocoli2003, 41, my emphasis).

This is a remarkable observation: a model that later would be the object of important exchanges between economics and psychology originated as part of a project that was directed against the interaction of these two disciplines. Yet beyond the merely curious, this observation also helps us to understand Samuelson's specifically economic perspective on utility discounting: while he considered it as a theoretical condition for the measurability of utility, he certainly never contemplated measuring this discounting rate itself. The model was meant, to use his own words, to elucidate theoretical relations, not to further inductive investigation.

One of the few economists appreciative of the subtleties already engendered in Samuelson's 1937 paper was Robert Strotz. Robert Strotz (1922–1994) received an undergraduate degree in economics from the University of Chicago in 1942. He then served three years in army intelligence during World War II, including a period as an economist-statistician in Berlin, “estimating necessary food imports to feed the German population.” (Northwestern University 2009) He was awarded a Ph.D. in Economics by the University of Chicago in 1951 for a dissertation on welfare economics. He first joined Northwestern University in 1947, became professor there in 1958, and finally university president in 1970.

In his 1956 paper, Strotz suggested that most people are born with non-exponential discounting functions (Strotz Reference Strotz1956, 177), thus leading to continuously threatening intertemporal inconsistency. He illustrated this proposal with a graphical representation of such a non-exponential discounting curve (labeled λ(t-τ) in fig. 1) amongst a family of exponentially discounted curves. Although Strotz did not specify λ(t-τ) further, the graph in fig. 1 shows a curve that is concave in parts of its interval and convex in others.

Fig. 1.

Strotz's non-exponentially discounting curve (Strotz Reference Strotz1956, 175).

Three observations are noteworthy here. First, the λ(t-τ) curve, as it is non-exponential, does not satisfy the requirements of Samuelson's model. This, of course, was Strotz's intention. Second, when seen from today's perspective, the λ(t-τ) curve is not a direct predecessor of the hyperbolic or quasi-hyperbolic discounting functions proposed by later authors, either. As I will discuss in sections 3 and 4, those discounting functions are steeper than exponential ones, but always convex – while the λ(t-τ) curve is concave in parts of its interval and convex in others. Third, Strotz similarly to Samuelson did not consider measuring the shape or slope of the discounting function. But unlike Samuelson, he drew on introspection to substantiate the possible shape of this function.Footnote ⁵ This introspective manoeuvre certainly was not due to Strotz's ignorance of recent developments in utility theory. In fact, three years earlier, he had written a paper on the new utility concept (Strotz Reference Strotz1953) that received widespread praise for its careful exposition (Fishburn Reference Fishburn1989; Ellingsen Reference Ellingsen, Allais and Hagen1994). Yet in the same paper, he also argued that this new framework does not prove alternative (non-behaviourist) utility concepts to be incorrect or obsolete; and one might speculate that this insight led him to think of introspection as a legitimate source of evidence for the λ(t-τ) curve.

Having thus specified the non-exponential discounting curve, Strotz then focused on the conditions under which “an individual who continuously re-evaluates his planned course of consumption confirm[s] his earlier choices and follow[s] out the consumption plan originally selected” (ibid, 171). His answer was the same as Samuelson's: if and only if the individual discounts with a log-constant, i.e. exponentially. But unlike Samuelson, Strotz gave reasons to believe why one should expect exponential discounting to be prevalent in the real world. In particular, he considered temporal inconsistency an undesirable state, leading to periodic abandonment of previous consumption plans in favor of new ones, which would be in turn abandoned in the next period. One might speculate that Strotz's experience of the vagaries of planning in liberated Germany played a role in these considerations.

Because time inconsistency is undesirable, yet we are born with non-exponential discounting functions, “we are taught to plan [by parental teaching and social pressure] consistently by substituting the log-linear function for the true one” (Strotz Reference Strotz1956, 177). In some cases, this leads to a genuine preference change, such that agents now have the exponential function as their true discounting rate. In others, the true non-exponential function remains, but because agents understand the non-desirability of such a way of discounting, they seek to hedge against later changes of actions through pre-commitment devices. Finally, where pre-commitment is not feasible, the agent “resign[s] himself to the fact of intertemporal conflict” (ibid., 173) and chooses that plan which he will actually follow. Thus there are reasons to believe that at least the educated exhibit behavior consistent with exponential discounting, only occasionally disturbed by “splurges, binges and extravagances” (ibid., 178). Strotz thus followed Samuelson in the representation of the normatively prescribed, rational path of intertemporal choice. But unlike Samuelson, Strotz also argued that actual behavior deviated from this rational path, and he sought to develop tools to represent this actual behavior.

Many behavioral economists of the 1980s and 90s identified Strotz's work on intertemporal choice as an outlier far ahead of its time (including Thaler Reference Thaler1981; Loewenstein Reference Loewenstein1987; Prelec Reference Prelec1989; Laibson Reference Laibson1997; Rabin Reference Rabin1998). His contemporaries did not share this view. Mostly, his paper was ignored, as intertemporal choice in the following decades remained a fringe topic.Footnote ⁶ But where he was cited, authors ignored the non-exponential model and instead focused on his normative evaluation of exponential discounting functions as the rationality standard, seeking to provide this purported standard with a number of axiomatizations.

One of these contemporaries was Tjalling Charles Koopmans (1910 –1985), at that time at Yale University. The motivation of Koopmans' study was “to formulate postulates permitting a sharp definition of impatience, the short term Irving Fisher has introduced for preference for advanced timing of satisfaction” (Koopmans Reference Koopmans1960, 287). He did not cite Samuelson's 1937 paper, but instead referred to Böhm-Bawerk and Fisher as historical antecedents and Strotz for a more recent discussion in the literature.

Koopmans offered a rigorous analysis of the exponential discounting model, by deducing its form from a set of properties of the utility function. The most relevant of these were separability and stationarity. Separability postulates that the particular bundle of commodities to be consumed in the first period has no effect on the preference between alternative sequences of bundles in the remaining future, and conversely. Stationarity postulates that the passage of time has no effect on preference, i.e. for some bundle at time 1 x ₁ and all consumption programs X ₂ = {x ₂,. . .,x _n}, Y ₂ = {y ₂,. . .,y _n}:

(2)$$\begin{equation} U(x_1 ,X_2 ) \ge U(x_1 ,Y_2 )\ {\rm if\, and\, only\, if\, }U(X_2 ) \ge U(Y_2 )\end{equation}$$

Lancaster (Reference Lancaster1963) expanded the formal analysis by deducing the exponential form of Samuelson's model from properties of preferences, thus not only giving sufficient conditions for its form but also for the existence of a continuous inter-temporal utility function of this sort. His model's most important properties were independence from viewpoint and time compounding. Independence from viewpoint postulates that preferences do not change with time. Time compounding states that

(3)$$\begin{equation} {\rm if }(X,t_1 )\sim(Y,t_2 )\ {\rm then }\ (X,t_1 + s)\sim(Y,t_2 + s)\ {\rm for\ all }\ s{\rm > - min(t}_{\rm 1} {\rm ,t}_{\rm 2} {\rm )}{\rm .}\end{equation}$$

where ~ signifies “is indifferent between.” In words, if the agent at t₀ is indifferent between consumption bundle X at t₁ and Y at t₂, then she is also indifferent between these two bundles when they are consumed at different times, but at the same temporal distance. This latter axiom, Lancaster argued, is analogous to Koopmans' stationarity postulate.

Common to all these papers is that they stressed the theoretical primacy of the exponential discounting function, and chose to axiomatically explore the conditions for its existence. If they referred to non-exponential shapes at all, then only to provide reasons for salience or even normatively charged rationality judgments – there is never an attempt to empirically determine its shape or measure its slope. In contrast to this, as I will show in the next section, psychologists since the 1950s had focused on the task of an empirical measurement of the actual discounting function in humans and animals.

3. Psychologists at the Helm: Measuring the Effect of Delay

Psychologists, like economists, have speculated about inter-temporal issues for a long time. Only in the twentieth century, however, did they seek to formulate precise regularities on how the temporal relations between events influence the actions of organisms. Because this endeavor coincided with the rise of behaviorism, it stressed the centrality of (animal) experiments and quantitative measurement from the start. While it shed its behaviorist constraints in the latter half of the century, the project nevertheless retained this empirical focus. This was one of the reasons, I will argue, that later made it so attractive to behavioral economists. In this section, I describe the development of this psychological project up to the early 70s.

Behaviorists were the first to empirically identify and measure the effect of delay on behavior. At Columbia University in 1911, Edward Lee Thorndike (1874 –1949) showed that response acquisition – the rate of responding above its operant level as a function of the stimulus-response relation – was negatively related to the interval between a response and the stimulus. At Johns Hopkins University in 1917, John Broadus Watson (1878 - 1958) studied the effect of delay experimentally by imposing a period of time between rats' responses of digging through sawdust and subsequent access to food. In 1933, Thorndike and Burrhus Frederic Skinner (1904 - 1990) adopted Ivan Petrovich Pavlov's (1849 – 1936) term of reinforcement to denote the strengthening of stimulus-response associations. This terminological adjustment allowed a clearer focus on the effect of delay on the strength of reinforcement. First at Harvard and then at the University of Minnesota in the 1930s, Skinner investigated operant responses in the absence of immediate reinforcement (Skinner Reference Skinner1938). Lever-press responses of experimentally naive rats, he showed, were established when reinforcement followed a delay of up to 8 seconds.

At the same time, theorists endeavored to incorporate these empirical findings into models that connected delay of reinforcement with behavioral responses. At Yale in 1932, Clark Leonard Hull (1884 –1952) proposed the goal-gradient hypothesis, which says that the tendency to approach a goal increases with (spatial or temporal) proximity to the goal. For example, he found that rats in a straight alley ran progressively faster as they proceeded from the starting box to the food. However, subsequent experiments yielded very diverse measurements, leading some to doubt the existence of a unique true temporal gradient. Hull then suggested the existence of a separate gradient of reinforcement as well as a goal gradient, both directly affecting the habit strength of the instrumental response. In contrast, at the State University of Iowa in 1956, Kenneth Wartenbe Spence (1907 – 1967) suggested that delay would indirectly affect conditioning by reducing the incentive motivation. The theoretical question whether a uniform psychological discounting factor existed or whether it fragmented into several distinct factors thus was already raised at this early stage of the discussion.Footnote ⁷

These early discussions were all based on results from animal experiments. At the Harvard Graduate School of Education in 1945, psychologist Orval Hobart Mowrer and psychology student Albert Ullman were the first to apply these results to human behavior, employing the temporal gradient of reinforcement to explain why neurotics often persisted in making responses that led to punishment. A given symptom S, they argued, could lead to immediate reinforcement (e.g. anxiety reduction), but an immediate punishment P may be severe enough to inhibit S. Yet if P is delayed, the steep decline of the gradient might lead to a situation where the net difference between the reinforcement strength of S and delayed P could be reinforcing S. Thus the shape of the goal gradient – specifically the steepness of its decline – became an important explanatory factor of human behavior.

In the 1940s, researchers became interested in the effect of delays not only on conditioning, but also on human preferences. For example, at the University of Pennsylvania in 1943, Francis Irwin (1905–1985) and collaborators offered children the choice between two toys. The interval separating acquisition of the respective toys was three minutes in the case of 50 children, and one week in the case of the remaining 50. The study's aim was to test whether the length of the time interval itself had an effect on the relative preference for the respective toys (which this study could not show to exist).

The experimental and measurement techniques developed by behaviorists thus migrated to other domains of psychology. This migration considerably expanded in the 50s and 60s. As an example of this expansion, I discuss the early work of Walter Mischel here.

Walter Mischel (born 1930) studied at Ohio State University, where his teachers were the psychologist, personality theorist, and therapist George Kelly and the psychologist Julian Rotter. He received his Ph.D. in clinical psychology from Ohio State in 1956. Mischel taught at the University of Colorado from 1956 to 1958, at Harvard University from 1958 to 1962, and at Stanford University from 1962 to 1983. Since 1983, Mischel has been in the Department of Psychology at Columbia University. Mischel, a developmental and personality psychologist by training, would later move on to challenge the very foundations of his home discipline.Footnote ⁸ Yet in his early work, he researched the effect of delay on preferences, and treated this relationship as a personality characteristic. Mischel (Reference Mischel1958, Reference Mischel1961) and Melikian (Reference Melikian1959) ran delayed reinforcement experiments with children of different ethnic groups and with delinquent and non-delinquent children.Footnote ⁹

Mischel and Metzner (1962) proposed the hypothesis that preferences for delayed, larger rewards are negatively related to length of the delay interval. They presented this hypothesis as a “deduction from any expectancy theory which assumes (a) that the effective value of an incentive is a joint function of its ‘real’ value and the expectation of getting it, and (b) that the expectation of getting something decreases with its temporal distance” (Mischel and Metzner Reference Mischel and Ralph1962, 425).

Mischel and collaborators, however, only investigated a binary choice between a smaller immediate and a larger delayed reward. They did not investigate the form of the discounting function, and indeed did not even couch their investigation in terms of a discounting function at all.Footnote ¹⁰ This may not be surprising, as they did not seem to be aware of the economic literature on inter-temporal choice, nor the behavioral literature on delay of reinforcement.

Instead, Mischel and others stressed the systematic differences in delay effects between people. Observing a significant difference in the relative preference for delayed rewards between ethnicities in Trinidad, Mischel (Reference Mischel1958) argued that these differences in turn are influenced by variables like father in the home, or ability to trust. Similar studies by Mahrer (Reference Mahrer1956), Mischel (Reference Mischel1961), and Melikian (Reference Melikian1959) investigated the effects of factors like age, social responsibility, accuracy in time perspective, intelligence and need achievement. How strong delay affects preferences thus became a personality characteristic, which in turn is influenced by social variables. The conclusion from this research, then, was that people have a clear and unchanging relative preference, ceteris paribus, for or against delayed rewards. While this is affected by various kinds of exogenous factors, it remains independent of the specific point in time where the individual finds herself. Inter-temporal inconsistency was thus not considered.

Mischel and collaborators never sought to quantify the effect of delay on preferences. Partly this may have been because they were unaware of the economic literature and its focus on the utility function. Yet as I will show later, these researchers also focused on aspects of inter-temporal choice – mental representations as means of coping with the ensuing conflict – that did not lend themselves to quantification and representation in a discounting function. Even at this early stage, it is plausible that their focus prevented them from seeking to measure the shape of discounting. I will discuss Mischel's later ideas in the last section of this article. Here, I concentrate on efforts of measuring delay effects more precisely, chiefly in Richard Herrnstein's Harvard Pigeon Lab.

Richard Herrnstein arrived at Harvard in 1958 as an assistant professor, and, as Baum (2002, 347) reports, soon “took over responsibility for the Pigeon Lab.” Because the Pigeon Lab predates Herrnstein's arrival by ten years, let me first describe this institution and its founder, Burrhus Frederic Skinner (1904–1990). Skinner enrolled in the graduate program in psychology at Harvard in 1928. As a graduate student, he invented the operant conditioning chamber, in which animals are trained through rewarding or punishing their behavior. This apparatus was instrumental in pursuing the study of operant conditioning, an alternative to the more widely studied classical conditioning à la Pavlov's dog. Skinner received his Ph.D. in 1931 and remained at Harvard to do research until 1936. After a period of teaching elsewhere, Harvard offered him a tenured professorship in 1948. With tenure also came funding for the Harvard Pigeon Lab, which he established the same year. Skinner had worked experimentally with pigeons before, notably on a National Defense Research Committee grant, given in 1943 to train pigeons to guide missiles reliably and accurately to their target (Pickren and Rutherford Reference Pickren and Alexandra2010; see also Rutherford 2009). Yet Skinner's main academic work on pigeons, cumulating in the publication of Schedules of Reinforcement, with C. B. Ferster in 1957, was to happen at the Harvard lab between 1948 and 1954 (Ferster 1970).

The Pigeon Lab was part of the Harvard psychology department, located until 1963 in Memorial Hall, and after that in William James Hall. The department included also two other labs, the laboratory of psychophysics and the Center for Cognitive Studies. Physically, the Pigeon Lab consisted of two main rooms, with additional experiments set up in auxiliary rooms, “so that two or more different experiments could be run in tandem” (Baum Reference Baum2002, 352). Senior faculty at the lab were the tenured professor (first Skinner, then Herrnstein), an assistant professor (a position occupied by a string of people, including Herrnstein 1958–61, Rachlin 1966–70, Baum 1970–73, and Mazur 1973–76), and occasional postdocs (e.g. Ferster 1950–56 and Baum in 1966–70). Most of the lab work was performed by grad students, often on their own projects. Baum (2002, 348) counts 19 grad students under Skinner, and 30 under Herrnstein until 1980, plus 4 ‘transitional’ students. This makes an average of about 5–6 students per year, but in fact the numbers fluctuated more, building up under Skinner to a peak in the mid-50s and under Herrnstein in 1962 reaching 7 to 8 per year (Baum Reference Baum2002, 350). Furthermore, Undergraduates also came to do research, about two or three each semester. Students were supported by two full-time lab workers who tended the pigeons and helped set up and run experiments.

Many authors (Ferster 1970, Baum Reference Baum2002, Catania Reference Catania2002, Heyman Reference Heyman2002) write about the palpable excitement in the laboratory. At weekly “pigeon staff meetings,” new findings were presented, and later, under Herrnstein, also new project proposals discussed (Skinner apparently gave students free reign with their project planning). Night and weekend shifts were frequent. However, when in the early 1970s, the Psychology Department was joined with the larger Social Relations Department, animosity between senior faculty led to a decrease in funding, and the last graduate student at the Pigeon Lab finished in the early 1980s. Herrnstein at around that time “lost interest” (Baum Reference Baum2002, 354), instead focusing more energy on the highly controversial debate about the distribution of I.Q. in the population. When he died in 1994, Harvard administration did not make a new tenured appointment. Four years later, in June 1998, the Harvard Pigeon Lab was officially closed.

Richard Herrnstein (1930–1994) graduated from City College of New York, moved to Harvard and the Pigeon Lab in 1952 and received his PhD from there in 1955. After a three-year stint at the Walter Reed Army Institute of Research (WRAIR), he returned to the Harvard Pigeon Lab as an assistant professor, and received tenure in 1961. The transition period from Skinner to Herrnstein falls within the 1958–62 period. Skinner stopped doing research in the lab after he and Ferster completed the studies that went into Schedules of Reinforcement around 1954 (Ferster 1970). For a few more years, he interacted with the students about their research. When Herrnstein arrived as an assistant professor in 1958, he soon took over intellectual leadership (Baum Reference Baum2002, 348), so that Baum considered some of the students still under Skinner's supervision as being “transitional” in the sense that Herrnstein really advised their thesis.Footnote ¹¹

The continuity from Skinner to Herrnstein is palpable here, not only in the intellectual debt of the student to his mentor, but even in the material constancy of the location, the experimental apparati and the animal subjects. And yet, the new Pigeon Lab under Herrnstein considerably changed the research on the effect of delay, first by focusing more on measurement, second by giving the discounting function a new shape, and thirdly by relaxing the confines of the Skinnerian program. In the remainder of this section, I will discuss these changes in more detail.

First, research at the Pigeon Lab under Herrnstein took a measurement turn. When Skinner's NSF grant for the laboratory officially ended in 1962, “the center of gravity of new work with pigeons shifted from experimental to quantitative analysis, especially in support of Herrnstein's matching law” (Catania Reference Catania2002, 327). Baum (2002, 351) described the enthusiastic mood of that time thus: “we knew we were turning the science that Skinner had founded into a quantitative field.” Herrnstein's matching law, first published in 1961, is a quantitative model expressing how animals distribute their choices among various alternatives. It states that animals match the distribution of their choices to the distribution of the reinforcers for those choices (Herrnstein Reference Herrnstein1961). An important implication of this model was that rates of reinforcements vary dynamically, proportionally to the rate of responding. This perspective in turn emphasized the importance of measuring this relationship, which inspired much of the experimental work performed in the Pigeon Lab.

Secondly, this new quantitative analysis in support of Herrnstein's matching law led to the creation of new experimental designs for the purpose of measurement, and new functional forms for the representation of measurements. For the experiments, Chung and Herrnstein used superimposed delays on a concurrent variable interval schedule: on either side, the first peck after a variable timer had timed out made the key go dark and delivered food after a delay, which varied as a parameter. Early results (Chung Reference Chung1965) unfortunately did not fit the matching law. This was inconvenient, but apparently not a disaster: Boakes (Reference Boakes and Rabbitt2009, 39), for example, stresses the “highly a-theoretical approach” of Herrnstein at Harvard, implying that the newly found measurement techniques somewhat outweigh the inability to confirm the matching law; and Ainslie remembers (almost 50 years later) that “there was no particularly strong theoretical interest that motivated this investigation – as generally, the whole pigeon project was deliberately a-theoretical, rather interested in operationalizing and measuring frequencies, rates, etc.” (Interview 27.03.2013).

In any case, the initial tension between empirical findings and theoretical perspectives was resolved by Chung and Herrnstein (1967). From further experiments, they reported results as shown in fig. 2, leading them to propose an alternative functional form.

Fig. 2.

Relative frequencies of responding to the experimental key, on a logarithmic scale, as a function of the delay of that key. The parameters on the right refer to the duration x of the standard delay (Chung and Herrnstein Reference Chung and Richard1967, 72).

The straight lines in figure 2 give the standard exponential form of the effect of delay on reinforcement. Analyzing the deviations of the data from these straight lines (in particular for x = 0), Chung and Herrnstein found that “the exponential function may not be the best summary of the effects of delay” (Chung and Herrnstein Reference Chung and Richard1967, 72). Additionally, they showed that there is a close match between the relative frequency of responding to the right key R_r and the relative duration of the delay intervals of both keys, as expressed in the matching law:

(4)$$\begin{equation} \frac{{R_r }}{{R_r + R_l }} = \frac{{d_r }}{{d_r + d_l }}\end{equation}$$

where d_r is the reinforcement delay of the right key, and d_l that of the left key. This relationship, Chung and Herrnstein argued, is incompatible with a theory that casts relative frequency of responding as an exponential function of delay. In the end, they considered their data to “slightly favor the matching relation, but not enough to exclude the exponential” (Chung and Herrnstein Reference Chung and Richard1967, 73). Further experiments with pigeons (Shimp Reference Shimp1969; Killeen Reference Killeen1970) yielded similar results. Although not as clear and decisive as the authors might have wanted, this is the first time that direct empirical evidence was provided for the non-exponential form of the discounting function.

The third change that research at the Pigeon Lab underwent under Herrnstein was a softening of the behaviorist tenets. Even if the measured regularities were strictly between observable stimulus and behavioral variables, the vocabulary used in the Pigeon Lab acquired a more cognitive flavor. Some used “reward” instead of “reinforcement” (e.g. Baum in his 1966 Ph.D. thesis), and Herrnstein himself began talking about optimization or maximization.Footnote ¹² In retrospect, Ainslie evaluates the changing atmosphere at the time thus: “Herrnstein was already turning away from stimulus-response bookkeeping. For instance he showed that animals can minimize shock rates without cues as to when the shocks are coming (Herrnstein Reference Herrnstein1969). Once you have the idea of minimizing and maximizing, it is a short leap to ‘utility’” (Ainslie e-mail 29.02.2012). This is not to say that the Pigeon Lab became a hothouse for cognitive psychology. Thirty years later, Baum (2002, 347) still sounded somewhat acrimonious when he stated that “directly and indirectly, the laboratory finally died as a result of the cognitive ‘revolution’.”Footnote ¹³ Yet, notwithstanding these continuing oppositions, the new orientation of the Pigeon Lab under Herrnstein led to change in terminology, which in the longer run opened up their work for interdisciplinary exchange with economists.

But that time had not yet come during the operation of the Pigeon Lab. Although there are many parallels in the development of this psychological research from the 40s to the 60s compared to that of economics during the same time, one has to avoid seeing these two disciplinary efforts as closely linked. In fact, they were not, for at least four reasons. First, the two approaches were separated institutionally. None of the psychologists involved made any explicit comparisons or drew any connections to economics, nor did they quote the economic literature. Second, the two approaches differed conceptually. Despite the changes in terminology noted above, the psychologists' analysis was still conducted within a reinforcement framework. Discounted utility models thus were conceptually distinct from delay of reinforcement frameworks, and hence cannot generally be equated. Third, the two approaches differ methodologically. The different concepts of reward effectiveness – “habit strength” (Hull Reference Hull1943), “incentive value” (Logan Reference Logan1965, 9), “frequency of response” (Chung and Herrnstein Reference Chung and Richard1967) – presented an empirical regularity between a time interval and a closely defined behavioral reaction. The exponential discounting model, in contrast, presented the shape of an evaluative function that was not empirically measured, and for which no measurement procedure had been conceived. Instead, Samuelson, Strotz, Koopmans, and Lancaster presented a conceptual possibility, vaguely supported by plausibility considerations, but highly abstracted and idealized. Fourth, the psychologists performed measurements on animals – and economists until the 1980s did not consider animal experiments relevant for their theory. Thus, despite the conceptual and methodological changes that I described above, which with hindsight can be seen as an interdisciplinary rapprochement, no interaction resulted at the time. Instead, an act of active synthesis was required to bring these two strands of research together. This synthesis I will describe in the next section.

4. Synthesis: Models of Self-Control

To overcome the disciplinary gap and to synthesize both perspectives into a unified approach required a conscious effort to address the institutional, conceptual, and methodological separation. George Ainslie (born 1944) performed this synthesizing role. A clinical psychiatrist by training, he had joined the Pigeon Lab in 1967, after being referred to Richard Herrnstein. He had begun his senior thesis at Yale on delay of gratification, using differential reinforcement of low rates with rats in a maze. His teacher at Yale, Frank Logan, had performed reward delay experiments with rats, and had suggested that the reward effect was inversely proportional to the square root of delay (Logan Reference Logan1965). Ainslie also started exploring the economic literature after conversations with his brother-in-law, Elmer Schaefer, a lawyer who was then taking graduate courses from the economist Richard Zeckhauser (Ainslie email 29.02.2012).Footnote ¹⁴ With this interdisciplinary background between psychiatry, psychology, and economics, Ainslie was in a privileged position to synthesize the two largely independent developments sketched in the two previous sections. Specifically, he contributed to this synthesis in four ways.

First, he proposed to Herrnstein that the matching law implied a hyperbolic discounting function. Here is how he describes this episode himself.

This proposal was based on a number of conceptual and perspectival differences to the views dominant at the Pigeon Lab at the time. For one, although Chung and Herrnstein had plotted relative response frequencies against delays, most lab members did not think of it in terms of a discounting function. From the distance of almost fifty years, Ainslie remembers that “they were rather fixed on concurrent schedules of reinforcement as more ‘natural’ than the discrete trials I [Ainslie] wanted” (Ainslie email 29.02.2012). These discrete trials, furthermore, sought to investigate (reward) dominance at one moment that might be compared to dominance at another moment, rather than looking for average reward effects over time, as Chung and Herrnstein did. Finally, Ainslie's work was motivated by his interest in preference reversals, while his Pigeon Lab colleagues “were not very interested in the difference between matching and exponential formulae because they were not looking for motivational conflict” (ibid.).

Through the sharpening of these interests and consecutive novel experimental designs, Ainslie developed this “vague idea about crossing discount curves” into a new representational form. In his 1975 paper, Ainslie for the first time drew these discounting curves. In this paper, he also coined the term “hyperbolic discounting.” These curves were represented functionally as the ratio of reward to time, yielding the downward sloping part of a hyperbola. Today, these functions are commonly expressed as:Footnote ¹⁵

(5)$$\begin{equation} U^t \left( {c_t ,..c_T } \right){\rm } = {\rm }\mathop \sum \limits_{k = 0}^{T - t} \frac{{u\left( {c_{t + k} } \right)}}{{1 + k}}{\rm }\end{equation}$$

Thus the hyperbolic discounting function was born in psychology. It offered a functional representation different in content from those proposed by Samuelson, Koopmans, and Lancaster, but similar in format: it described the effect of delay on behavior through the aggregation of utilities from future consumption bundles, discounted by the distances of these bundles to the present.

Second, Ainslie created multiple institutional links between psychology and economics. His first two papers (Ainslie Reference Ainslie1974 and 1975) were based on the experiments run in the Pigeon Lab, addressing questions that Herrnstein and colleagues had also pondered. Yet the literature that he drew on in these articles was considerably broader than that of his colleagues. He referred to sociological, psychoanalytical, and economic papers, specifically to Strotz (Reference Strotz1956), while also referring to Mischel's early work (Mischel and Metzner Reference Mischel and Ralph1962). Both these strands were considerably expanded in Ainslie's (1975) paper. Here he cited John Stuart Mill, Jevons, Böhm-Bawerk, Rae, Samuelson and Strotz, and interestingly also adapted Schelling's (1960) work on interpersonal conflicts for models of intrapersonal bargaining (Ainslie Reference Ainslie1975, 482). Furthermore, he not only took note of Mischel's early work extensively, but also referred to his later work on the delay-of-gratification paradigm (Mischel and Ebbesen Reference Mischel and Ebbe1970).

In his publications in the 1980s, Ainslie then moved from animal behavior to human behavior, applying hyperbolic discounting and conflicting-self models to studies on alcoholism and substance abuse. Notably, the title of a 1981 paper (with Elmer Schaefer) is “The application of economic concepts to the motivational conflict in alcoholism,” referring to hyperbolic discounting as an “economic concept.” This institutionally bridging function culminated in his 1991 article, entitled “Intertemporal choice. Derivation from ‘rational’ economic behavior from hyperbolic discounting curves,” published in the American Economic Review, one of the leading economic journals.

Third, Ainslie offered a methodological synthesis that employed the hyperbolically discounted utility function to explain the Pigeon Lab animal behavioral data. The crucial step in this was a shift of focus away from estimating the form of the discounting function directly, towards an abductive inference from the explained behavioral phenomena (in particular, observations of pre-commitment choices) to the form of the discounting function. “I was after the phenomenon of a change of preference over time, and the consequent need to inhibit some potentially harmful behavior” (Interview 27.03.2013).

In various behavioral experiments, it had been shown that pigeons could learn using a pre-commitment device (Rachlin Reference Rachlin1970; Ainslie Reference Ainslie1974).Footnote ¹⁶ For example, pigeons would learn to peck a key that blocked a later key dispensing a sooner, smaller reward. By pecking the pre-commitment key, pigeons forced themselves to wait for the later, larger reward. Ainslie (Reference Ainslie1974) interpreted this result as follows: “Subjects' preferences can often be expected to change in a regular way over time in the absence of any new information about the alternatives. . . [This] poses the problem of impulse control as the need to forestall the temporary effectiveness that some small rewards can be expected to acquire by virtue of their temporal position” (Ainslie Reference Ainslie1975, 464).

The presence of such impulse control tactics, so Ainslie said, shows the threat of a preference change as the agent approaches the realization of the smaller sooner reward.Footnote ¹⁷ Such a preference change is described by the effectiveness curves for the two rewards in fig. 3. Starting at time t₀ on the left, the larger, later reward is more effective than the smaller, sooner one. Yet as the agent approaches the realization of the smaller, sooner reward, its effectiveness increases relative to that of the larger, later reward, to such an extent that the effectiveness curves eventually cross. At this crossing point, the smaller, sooner reward is more effective on behavior than the larger, later one, leading to an “impulsive” choice – unless such a choice has been prevented by appropriate self-controlling tactics like the pigeons' pre-commitment key.

Fig. 3.

“The effectiveness of a duration of access to food versus a duration three times as long beginning 3 sec later, as predicted for pigeons by Chung and Herrnstein's (1967). matching formula” (Ainslie Reference Ainslie1975, 472).

Ainslie thus abandoned the dominant psychological approach – which had been pursued by almost everybody from Watson (Reference Watson1917) to Chung and Herrnstein (1967) – to measure the discounting factor directly as the relative frequencies of choices between smaller, sooner, and larger, later, rewards. Instead, he concluded through (abductive) inference to the best explanation from the observed self-committing behavior to the hyperbolic form of the discounting function. “There is too much ambiguity in the current data from delay of reward experiments to discriminate between exponential curves and nonexponential curves of a roughly similar shape. However, the possibility of highly concave curves has suggested a mechanism of impulse control that is testable by direct experiments” (Ainslie Reference Ainslie1975, 472). This is very much in the spirit of the economic modeling approaches from Samuelson (Reference Samuelson1937) to Strotz (Reference Strotz1956): a highly abstract and idealized factor model is proposed as the best explanation for an observed phenomenon. Unlike the economists, however, Ainslie drew on the empirical psychological literature to construct the functional shape of his model. While Samuelson had chosen an exponential form for the purpose of simplicity, and Strotz (Reference Strotz1956) a free-hand non-exponential form, Ainslie chose the hyperbolic form with reference to the measurement efforts of Logan (Reference Logan1965) and Chung and Herrnstein (1967). Those measurements alone might not have yielded sufficient support for the hyperbolic form, but by synthesizing those methods and their results with the abductive inference, Ainslie offered a new unified methodology to inter-temporal choice, managing to both integrate evidence from economics and psychology as well as contributing novel evidence from his own experiments.

Fourth, Ainslie also pursued a terminological unification of the two approaches. His most important move was to equate the psychological notion of “reward effect” with the economic notion of “utility.” He did so rather casually in 1975, when he summarized Strotz's paper: “he [Strotz] hypothesized that a goal's “utility,” or rewarding effect, decreases with delay according to an inborn function” (Ainslie Reference Ainslie1975, 465; my emphasis).

Note that the explication of economists' utility in terms of rewarding effect is Ainslie's own. So the curves that Logan, Chung, Herrnstein, and Ainslie sought to measure were in fact conceptually identical to the curves that Samuelson and Strotz had drawn! If this was correct, the whole set of psychological results would suddenly be directly accessible for economists to build into their models. And indeed, when Ainslie would later present his results to economists, he took this conceptual identity for granted: “According to conventional utility theory, the value of delayed goods is discounted in an exponential curve. . . However, when behavioural psychologists have conducted parametric studies of choice, they have found a radically different discount function that has become known as Herrnstein's matching law” (Ainslie Reference Ainslie1991, 334).

As I will show in the next section, economists who were instrumental in introducing hyperbolic discounting into their discipline built on Ainslie's claim of this conceptual identity. Ainslie thus can be seen as the main enabler of the transfer of the hyperbolic discounting model into economics. Of course, this enabling role was made possible through a more general change of perspective in the field, which I described in the previous section. As Ainslie remembers it, “there was a tacit struggle in behavioral psychology about using ‘reward,’ which implies utility, instead of Skinner's ‘reinforcement,’ which implies strengthening a learned connection; I just followed Herrnstein in that” (Ainslie email 29.02.2012).

Consequently, Ainslie's terminological synthesis was met positively also amongst his colleagues at the Pigeon Lab.Footnote ¹⁸ Through these four contributions to a synthesis, and through the positive responses he received from his psychology peers, Ainslie enabled the transfer of the hyperbolic discounting function into economics. How this transfer happened in detail will be the topic of the next section.

5. Transfer into Economics

In the early 1980s, Ainslie's synthesizing work met favorable conditions in economics. This brought on a time of genuine interdisciplinary exchange, in which both disciplines collaborated and exchanged experimental designs, models, and theoretical results. However, as I shall argue, this period of genuine interdisciplinary exchange did not last very long. Because some of the initial promises of the exchange could not be met, economists increasingly separated from psychology. In fact, they moved away from an economics-psychology collaboration towards a new kind of economics that took inspiration from psychology, but very much insisted on its economic disciplinary credentials.

But first, let me describe the time of genuine interdisciplinary exchange. Three favorable changes in economics itself facilitated the more ready acceptance of the psychological work on intertemporal choice. First, in the 1980s, some economists began taking seriously the claim (again originating largely from psychology) that humans systematically violated basic rationality postulates in their choices under uncertainty. To be sure, this was an area of research different from intertemporal choice, and the gradual acceptance of these models into economics did not (and still does not) proceed without controversy (for a historical overview of that development, see Heukelom Reference Heukelom2014). But this development showed that as an economist, one could research the difference between actual behavior and rationality postulates – and this inspired some researchers to have a closer look at the relationship between actual intertemporal choice and the postulates of Samuelson, Koopmans and Lancaster, which I discussed in section 2.

To make such comparisons between actual behavior and theoretical postulates required empirical evidence. Economists could not use much of the evidence from psychology, and the pigeon lab specifically, as long as animal experiments were not considered relevant in economics. The second favorable change did away with that problem by making animal data legitimate for economists. In 1981 three of the highest-ranked economics journals, the American Economic Review, the Journal of Political Economy and the Quarterly Journal of Economics, published articles by Battalio, Green, Kagel, and Rachlin on income-leisure trade-offs, commodity choice, and demand curves of pigeons and other laboratory animals. These authors had previously published on animal behavior in economics periodicals, in Economic Inquiry in 1975 and the inaugural volume of Research in Experimental Economics in 1979, but these respectable journals lacked the cachet and impact of AER, JPE and QJE – let alone the impact of all three in the same year (McDonough Reference McDonough2003).

The third favorable change was an increase in institutional interactions – specifically the training of new students in the Pigeon Lab, who would move on to become important behavioral economists. One of them was Drazen Prelec, who in the 1980s collaborated with Herrnstein on the idea that matching was the result of a simple, myopic form of reward maximizing that they called melioration (Herrnstein & Prelec, Reference Herrnstein1991).

Drazen Prelec (born 1955) received his Ph.D. from Harvard in experimental psychology in 1983. He now is a professor of management science and economics in the MIT Sloan School of Management, and thus by affiliation and publication record can be considered more an economist than a psychologist. A similar case is David Laibson (born 1966), who as an undergraduate attended Herrnstein's seminar, with Ainslie as a tutor (Laibson, email 20.8.2013), received his PhD in economics from MIT in 1994, and later became the Robert I. Goldman Professor of Economics at Harvard University. With this personal transfer from psychology to economics, models, experiments, and measurements also traveled more easily.

In these favorable conditions, Ainslie's institutional, methodological, and conceptual synthesis could unfold its potential. Its first effect was a considerable increase in attempts to determine the shape of the human discounting function. A first empirical investigation directed explicitly against the exponential discounting model and its stationarity assumption is found in Thaler (Reference Thaler1981). Thaler asked subjects how much money they would require at different times in the future to make them indifferent to receiving $15 now. He showed that their answers implied an average annual discount rate of 345 percent over a one-month horizon, 120 percent over a one-year-horizon, and 19 percent over a ten-year horizon. He further showed that implied discount rates are much higher for questions about monetary gains than for losses (cf. also Loewenstein Reference Loewenstein1987); and that larger outcomes are discounted at a lower level than smaller ones (cf. also Loewenstein Reference Loewenstein1987 and Benzion et al. Reference Benzion, Amnon and Joseph1989). This was incompatible with an exponential discounting function, but it was compatible with the form of the curve that Strotz (Reference Strotz1956) had drawn (see fig. 1) and with Ainslie's (1975) hyperbolic function.Footnote ¹⁹

From this, a veritable measurement tradition developed. Following Thaler (Reference Thaler1981), and sometimes explicitly referring to the earlier experiments by Mischel and collaborators (e.g. Benzion et al. Reference Benzion, Amnon and Joseph1989), most of these studies were experimental in nature, mostly performing contingent evaluation measures with hypothetical scenarios. Many of these papers also quote some of the psychological literature. Loewenstein (Reference Loewenstein1987), for example, cites Rachlin and Green (1972) and Ainslie (Reference Ainslie1975).Footnote ²⁰

However, this endeavor to measure the human discounting function directly was beset with troubles almost from the start. Intertemporal choice experiments remained methodologically problematic. Other areas – in particular behavioral theories of choice under uncertainty – quickly developed their own experimental quality standards and managed to introduce them as a new research tool into economics. It was crucial, these experimentalists said, to base experiments only on real monetary incentives, thus distinguishing themselves from the experimental methodology of psychologists (for documentation of these differences, see Hertwig and Ortmann Reference Hertwig and Andreas2001 and Guala Reference Guala2004). But with respect to inter-temporal choice, establishing such real monetary incentives created “enormous tactical problems”: “would subjects believe that they get paid in five years?” (Thaler Reference Thaler1981, 207). This forced experimenters to work with other tools, for example hypothetical scenarios, which were not widely accepted in economics. Intertemporal choice experiments in economics thus did not distinguish themselves as clearly from their psychological cousins as choice experiments under uncertainty did, and consequently found less acceptance in the economics community.

Furthermore, the results of this measurement endeavor turned out to be more divergent than many had hoped. Frederick et al. (Reference Frederick, George and Ted2002) lists 42 such measurement papers published before 2002 alone. What soon became clear, however, was that these measurements yielded fantastically divergent results. Discount rates varied with income and other personal characteristics, and also with the kind of goods or events that were discounted. Loewenstein (Reference Loewenstein1987, 667), in fact, found examples for negative discount rates for certain time periods: the prospect of kissing your favorite movie star is, apparently, valued higher when three days away than when merely three hours away. Looking at numbers alone, results as low as 0 to 3 percent (Johannesson and Johansson Reference Johannesson and Per-Olov1997) stand next to those from 96000 percent to infinity (Ainslie and Haendel Reference Ainslie, Varda, Gottheil and Druley1983).

Consequently, there were not only considerable variations in magnitude, but also in the shape of the thus estimated discounting function. All that could be concluded from the measurements were a deviation – in either direction – from the exponential shape of the exponential discounting model.Footnote ²¹ The results could neither license the explanation of a particular individual's behavior across contexts, nor the explanation of different individuals' behavior in the same context. From the hopes implicit in the early measurement papers, this result must have been disappointing. But maybe this disappointment was not all that surprising: it echoed the debate in psychology, chiefly between Hull and Spence, whether there was one universal temporal gradient of reinforcement or many more specialized gradients, each contributing to temporal discounting in different ways. Just like their psychologist colleagues in the 1950s, economists in the late 1980s had to scale back their ambitions and realize that their efforts would not yield a universal discounting function – and not even a universal functional form – pertinent to their explanatory and predictive needs. I will call this the diversity of measurements disappointment. While this was not a single event, but rather a gradual realization amongst the researchers involved, it had a considerable impact on how research on intertemporal choice continued in economics. In particular, the diversity of measurements disappointment led to axiomatization, increased desire for tractable functional form, and perhaps most importantly, a change in explanatory strategy – and all three developments pushed economics away from psychology within the domain of intertemporal research.

In the late 1980s, economists did what economists often do in the face of empirical adversity: they pursued a more abstract modeling strategy. The first step in this strategy was to formulate a new axiomatization of inter-temporal utility (Prelec Reference Prelec1989; Loewenstein and Prelec Reference Loewenstein and Drazen1992). Their axioms were sufficient for the existence of a continuous utility function defined over consumption bundles at different times. The function was assumed to be separable, so that it consists of the timeless utility of the bundles discounted by its temporal dimension. Yet the stationarity axiom was now dropped, so that for sufficiently large parameters, the shape of the function became hyperbolic. For lower parameter values, the function retains an exponential shape, so that the old exponential discounting model became a special case of the new axiomatization.Footnote ²²

A second step was to make the discounting function itself more tractable. In 1997, David Laibson adopted the following equation from Phelps and Pollak (1968) for the purposes of inter-temporal discounting.Footnote ²³

(6)$$\begin{equation} U^t \left( {c_t , \ldots ,c_T } \right) = u\left( {c_t } \right) + \beta \mathop \sum \limits_{k = 1}^{T - t} \delta ^k u\left( {c_{t + k} } \right)\end{equation}$$

where β and δ are between 0 and 1 (Laibson Reference Laibson1997). This function exhibits a steep decline after the first time step, but a relatively mild decline afterwards (see the continuous line in fig. 4).

Fig. 4.

The quasi-hyperbolic discounting function (Laibson Reference Laibson1997, 450).

Laibson called this function the quasi-hyperbolic discounting function. It “mimics the qualitative property of the hyperbolic discounting function, while maintaining most of the analytical tractability of the exponential discount function” (Laibson Reference Laibson1997, 450). This claim merits further attention. Equation (6) consists of two parts: the present utility u(c_t), and the exponentially discounted future utilities, weighted by parameter β. Because β is commonly set at significantly smaller than 1, this function discounts all future utility rapidly after the initial period. But any future utility from t+2 onwards is only discounted exponentially. This functional form differs from genuine hyperbolic discounting (5) in a number of important ways. First, (6) exhibits a linear drop between u(c_t) and u(c_{t +1}). Second, (5) has a much flatter tail than (6) has. Third, intersections of two discounting functions can only occur when the β discounting sets in at different times, but never when it set in at the same time – because the exponentially discounted part ensures inter-temporal consistency. When the β discounting sets in is thus a very important question, one that remains unanswered in Laibson's writing.Footnote ²⁴

Given these considerable differences, it is noteworthy that Laibson insisted on the empirical adequacy of his construct.Footnote ²⁵ Note also that Laibson here spoke of the (single) property that the quasi-hyperbolic curve approximates. In combination, it becomes clear that Laibson solely focuses on the property of inter-temporal preference change, and the consecutive need of agents to control this change in some way. These are the sole features that quasi-hyperbolic discounting focuses on, and thus the only ways in which he saw (6) as empirically justified (Laibson Reference Laibson1997, 445). The consecutive widespread adoption of this quasi-hyperbolic curve in the economics community thus shifted the focus away from the measurement of actual inter-temporal evaluations and towards a family of evaluations that shared one property, namely that of inter-temporal inconsistency.

The third reaction to the diversity of measurements disappointment was a change in explanatory strategy. The aim now became to show how a rational agent dealt with her inter-temporally inconsistent evaluations – in particular, what pre-commitment devices she would use, or what plans she would adopt – in a highly stylized setting, and use the results of such a model for explanations of aggregate phenomena. In principle, such a strategy was not new. Many authors, beginning with Strotz (Reference Strotz1956), Peleg and Yaari (1973), Goldman (Reference Goldman1979), and Thaler and Shefrin (1981), had modeled rational behavior under inter-temporally inconsistent preferences as a strategic equilibrium between temporal selves – specifically a perfect Nash equilibrium of a non-cooperative game where each self chooses that strategy which maximizes its satisfaction given the decisions of all previous selves and the best strategies of all future ones. Schelling described this approach as follows:

Two workshops in Paris in 1980 and 1982 (convened by Jon Elster, with Ainslie, Schelling, and Tversky participating; later published as Elster Reference Elster1986) explored the relation of multiple selves and temporal inconsistency further. After that, Ainslie offered game theoretic analyses as well, talking about “Intra-personal bargaining” in 1986 and then about “repeated prisoners' dilemma” in his 1991 AER paper (Ainslie Reference Ainslie and Elster1986 and 1991).

Yet it was Laibson, in his 1994 thesis Hyperbolic Discounting and Consumption and his 1997 paper, who combined the (quasi-) hyperbolic discounting function with a rigorous game theoretic analysis.Footnote ²⁶ In Laibson (Reference Laibson1997), a consumer was analyzed into multiple temporal selves. Each self determines the liquidity of the next period, but consumes according to liquidity of the present period. Laibson then showed that there is a unique subgame perfect strategy in which each self chooses an investment plan such that the marginal utility from consumption at each period is at least as large as the time-discounted marginal utilities from consuming the returns from investment. This equilibrium result was then used to explain aggregate consumer behavior – for example, the excess co-movement of income and consumption, the existence of asset-specific marginal propensities to consume low levels of precautionary savings, and the correlation of measured levels of patience with age, income, and wealth (Laibson Reference Laibson1997, 454–468).

These game-theoretic models offered a new interpretation of inter-temporal conflict. Each self has a temporal position, from which it evaluates present and future consumption. Consequently, for that self, there is no conflict of preferences: it typically prefers sooner, smaller rewards to larger later ones. Instead, this self conflicts with other selves, who evaluate consumption from their own temporal perspectives. What is a choice between two long-term prospects for one self is a choice between a short-term and a long-term prospect for another, and might therefore be evaluated differently. Laibson's model thus replaced inter-temporal preference conflict with conflict between selves, each of which has a consistent preference ordering.

To economists committed to a unique, consistent preference ordering closely tied to an agent's choices, these game-theoretic models offered an elegant solution to the unpalatable prospect of preference conflict. In the light of this advance, assuming multiple agents seemed to be the smaller and more acceptable evil, in particular as it could be harnessed with established modeling tools: “On one level, the idea of multiple selves . . . is a radical departure from the utility-maximising framework. But because this conceptualisation of intertemporal choice uses a familiar tool – dynamic game theory – it is ready made for adoption by economists interested in improving the behavioural realism of our models” (Rabin Reference Rabin1998, 40).Footnote ²⁷

Hence, from the direct exchange with psychologists in the 1980s, economists adjusted the concept and employment of the discounting function in accord with some underlying characteristics of their discipline. After the project of direct empirical measurement lost its promise, they shifted their focus to an axiomatization, and then developed a strong simplification of the hyperbolically shaped function. Furthermore, they moved away from attempts to directly explain specific behavior with reference to the discounting function to a strategy that derived stylized behavioral phenomena from a multiple-self model, and employed this stylized behavior for their explanations of aggregate phenomena. This implied a preservation of the maximization model based on unique consistent preference as the motor of economic explanations. It also implied a rather intermittent relation of the discounting shape to empirical evidence. What mattered was that the model could explain aggregate data, not whether the discounting function was psychologically realistic. Not unlike Samuelson's view, a certain “arbitrariness” in the assumptions did not pose a problem anymore.

Hyperbolic discounting had thus arrived in the economic mainstream, but it arrived in a multiply adjusted form, considerably changed from the way psychologists had conceived it.Footnote ²⁸ Notably, this arrival also coincided with certain markers of institutional separation: while early behavioral economists freely cited the psychological literature, this practice mostly stopped in the late 1990s, thus signaling an end to the period of genuine interdisciplinary exchange between the two disciplines.

6. Epilogue: Psychology on Its Own Again

The dynamics described in the later part of the previous section was not a one-sided action: psychologists also increasingly left the path of synthesis, exploring decidedly non-economic models. Many psychologists involved in research on discounting diverged from the economists' later modeling strategy. Two avenues of psychological research can be identified. Both exhibit a “non-economic” understanding of the utility function.

The first avenue takes the hyperbolically discounted utility function as a representation of underlying or latent evaluations that can be affected by further mental interventions. Ainslie (2011, 30), for example, has recently argued for a “return to the pure hyperbolic discounting function as originally proposed [by Ainslie Reference Ainslie1975, 1992]” and against multiple-self models: “many people take multiple-self models as dissociated selves – I don't” (Interview 27.3.2013). In particular, he argues that multiple-self models are not capable of explaining self-control without pre-commitment in the right way. Such a mechanism of internal self-control, Ainslie claims, makes crucial use of the cognitive ability to re-frame perceived choices between temporally distinct rewards not as isolated choices, but as bundled streams of future rewards. For example, an alcoholic is more successful in staying sober if she sees her choice of having a drink now not as an isolated choice, but as a prediction of her future choices (Ainslie Reference Ainslie2013b). Ainslie shows that the standard hyperbolic model captures this mechanism: in a single instance, a smaller sooner reward may be preferred to a larger, later one, but if the (hyperbolically discounted) values of these rewards are cumulated over a series of identical choices between these pairs, then this preference is reversed. He also shows that a similar accumulation of exponentially valued rewards does not exhibit such a preference reversal. Hence neither the exponential nor the quasi-hyperbolic model captures the important effect of reframing future choices as bundles of rewards.

Furthermore, Ainslie argues that the hyperbolic function attributed to a single individual (rather than the individual being cast as a set of conflicting multiple selves), captures the cognitive core of the reframing mechanism. In a situation where a smaller, sooner reward is preferred to a larger later reward in isolation, a genuine contest between self-regulation and simultaneous temptation takes place. This genuine contest is decided not by any external prior factor, but alone by the way the individual perceives the rewards: either as part of a self-predictive series of choices, or as isolated instances (Ainslie Reference Ainslie2013a). Ainslie's model thus considers the utility function as a latent evaluation, which may be realized or suppressed by complex internal cognitive mechanisms. It thus does not represent an agent's unique and consistent preference ordering. This, in Ainslie's own view, marks an important difference between psychologists like him and economists working on similar phenomena: “there is an important difference of purpose that divides us. My economist colleagues tell me that their main interest is to find formulae that can make a unique prediction from a given set of priors, whereas the recursive self-prediction on which the bundling strategy rests makes this impossible in principle” (Ainslie email 29.02.2012).

The second avenue that psychologists took after the attempted synthesis is to ask about the underlying cognitive mechanisms that generate behavior consistent with certain shapes of the discounting function. The earliest examples of this are Mischel's delay-of-gratification experiments, popularized in the media as the “Marshmallow test” (Mischel and Ebbesen Reference Mischel and Ralph1970; Mischel et al. Reference Mischel, Ebbe and Antonette1972). These experiments focused on the conflict that agents experience as they struggle to maintain their decision and resist temptation from the immediate, smaller reward – concretely, whether and how children could resist settling for one marshmallow immediately in order to receive two marshmallows later.

Although similar to the experiments described in earlier sections, there are small but crucial differences. For example, the children were never told when the experiment would stop, and when they would receive the larger reward. Consequently, the experiment did not seek to measure a clear preference for the respective rewards-at-a-specific-time. Rather, the children were expected to prefer two marshmallows, but also expected to be tempted by the one marshmallow, and the question was how long, and in particular how, they would manage to hold out before being overwhelmed by the temptation (Mischel and Ebbesen Reference Mischel and Ebbe1970, 330).

The Marshmallow test, thus, elicits behavior that shows how conflicted subjects are: most of them prefer the later, larger reward to the sooner, smaller one at the beginning of the test. But they have difficulty sustaining that preference, often giving up their resolve well into the test and after having spent considerable mental resources in order not to give up. Mischel's analysis of the experiments showed how subjects deal with this continuous conflict, and elicits various strategies. For example, subjects' selectively withdrawing their attention and thoughts from the rewards affected the length of delay they achieved (Mischel et al. Reference Mischel, Ebbe and Antonette1972, 205). Furthermore, training subjects to substitute images of rewards for real rewards, he showed that “children facing pictures of the rewards delayed almost 18 minutes, but they waited less than 6 minutes when they pretended that the real rewards, rather than the pictures, were in front of them” (Mischel et al. Reference Mischel, Shoda and Monica1989, 934).

These mechanisms of cognitive control (cf. Eigisti et al. Reference Eigsti, Vivian, Walter, Yuichi, Ozlem, Mamta, Matthew, Aber and Casey2006) do not postulate a fixed motivational factor that needs to be overcome; rather they assume that motivation is fluid, and is shaped through the behavioral and cognitive strategies a person may or may not have. In various long-term studies, Mischel found that those who had more effective strategies in their repertoire would ten years later appear as “more academically and socially competent than their peers and could cope with frustration and resist temptation” (Mischel et al. Reference Mischel, Shoda and Monica1989, 943).

This focus on the underlying cognitive mechanisms of intertemporal choice can also be found in other areas of psychology. The Simple Heuristics program (Gigerenzer and Todd Reference Gigerenzer and Peter1999), for example, has investigated how attribute-based models and temporal discount functions relate to one another. Attribute-based models represent simple rules that compare between choice options based on a small set of option attributes. Examples include the similarity heuristic, which compares options with respect to their monetary rewards and their delay, and then chooses between them on the basis of that dimension in which they are less similar (e.g. Rubinstein Reference Rubinstein2003); the proportional difference model, where choice is based on that attribute with the highest proportional difference (e.g. González-Vallejo Reference González-Vallejo, Aaron and Joel2002); and the lexicographic strategy, where choices are based on monetary amounts when their difference exceeds the threshold, and otherwise on delays (e.g. Roelofsma and Read Reference Roelofsma and Daniel2000). These investigations show how the form and the rate of discounting depend on the kind of simple rules used, and how these simple rules interact with the environment (see also Pleskac and Hertwig Reference Pleskac and Ralph2014).

This emphasis on the underlying mechanisms also implied a de-emphasis of the (temporally discounted) utility function. This has led some psychologists to draw a rather radical conclusion, at least implicitly directed against the economic modeling approach: “The study of intertemporal choice is currently undergoing a change in emphasis, as has already occurred in the study of decision making under risk and uncertainty. Rather than searching for the holy grail of a single utility function, researchers now take the more pragmatic view that preferences are constructed based on the circumstances of their expression” (Roelofsma and Read Reference Roelofsma and Daniel2000, 172).

Although these positions seem to differ considerably from those of Ainslie, they are in fact quite similar. Ainslie stresses the relevance of a unique hyperbolically discounted utility function, but then introduces complex cognitive mechanisms that create and overcome genuine conflicts of motivations. Positions that deny the usefulness assuming a stable underlying utility function connect similar cognitive mechanisms closer with representations of evaluations. In both cases, however, the attribution of a unique, consistent preference ordering closely related to choice, as practiced by economists, is denied.