3.1. A unifying terminology: Regulator, goal, agent, proxy

To facilitate comparisons across disparate instances of proxy failure, we define four inter-related terms: regulator, goal, agent, and proxy (Box 1). A regulator is any entity or system with an apparent goal. To pursue this goal, the regulator incentivizes or selects agents based on some proxy. The agents in turn pursue the proxy, or are selected on the basis of the proxy. We will now examine each term in sequence.

A regulator is the part of the system that pursues a goal by influencing the agents' behaviour or properties using some form of feedback. The regulator predicates feedback on agents' performance in terms of the proxy. As we will explore below, the regulator may not only be a distributed system (e.g., a market) or an institution (a government), but it can also be a person, a neural subsystem (the dopamine system), or an ecological subpopulation (peahens). Conscious awareness is not required for regulatory feedback, which can occur via selection or reinforcement (rewards and punishments). In the case of the Hanoi rat massacre, the regulator was the French colonial government, which pursued its goal of reducing the rat population through feedback in the form of monetary incentives. In the context of sexual selection, the regulator is the population of potential mating partners and the feedback is access to mating opportunities.

A goal refers to the state of the system that the regulator seeks to establish. In human social contexts, such goals are sometimes explicit and simple (reducing a rat population, increasing shareholder value) and sometimes difficult to precisely define (social welfare, improved education, or scientific progress). The degree to which goals are captured by their respective proxy measures may vary considerably (Braganza, Reference Braganza2022b), even though many proxies were formulated explicitly to quantify abstract goals (e.g., test scores in education or the journal impact factor in academia). In nonhuman contexts, such as in ecology, explicit goals cannot generally be assumed. A peahen may act as if to optimize her offspring's fitness, but she might not consciously represent or monitor this goal. In such cases, imputing goals nevertheless remain an efficient way to account for empirical findings, a perspective known as teleonomic reasoning (Mayr, Reference Mayr1961; Appendix 1).

An agent is an entity, process, or abstract system that is the target of regulation. Agents are subject to competition, selection, or reinforcement on the basis of some output or property. They need not be human or conscious (or even active in any sense), but they must possess multiple ways of producing or expressing a proxy, which can be influenced by feedback from the regulator. In the case of the Hanoi rat massacre, the agents were the people bringing in rat corpses and tails. In educational systems, the agents are teachers who end up “teaching to the test.” In the brain, agents are neural representations “competing” for neural resources (see sect. 4.1). In sexual selection, agents are peacocks trying to appear attractive to choosy peahens.

A proxy is an output or property of each agent that the regulator uses as an approximation of the goal, or the extent to which a goal is achieved. An agent's “performance” in terms of the proxy is the key factor determining the regulator's feedback. Examples of proxies (and their respective goals) include: The number of rat tails (approximating rat population size) in the Hanoi example; test-scores (approximating educational quality); peacock plumage (approximating offspring quality); and dopamine signals (approximating value in decision making). The proxy can typically be expressed as a scalar metric (i.e., a magnitude). This scalar may be aggregated from multidimensional inputs or serve as a summary of multiple metrics. The regulator uses the proxy to adjust its feedback to the agents (reward, ranking, selection, or breeding opportunity). This induces agents to increase proxy production, either actively or via passive selection. The term proxy focuses attention on the fact that the metric should approximate a goal, and that the fidelity of approximation is key.

A regulator with respect to one system may be an agent when viewed as part of a different system. For instance, in many cases of sexual competition, males are regulated by female choices, but females are themselves competing against each other. Furthermore, agents and regulators may be organized in nested hierarchies (Ellis & Kopel, Reference Ellis and Kopel2019; Haig, Reference Haig2020; Kirchhoff, Parr, Palacios, Friston, & Kiverstein, Reference Kirchhoff, Parr, Palacios, Friston and Kiverstein2018), where agents at one level are regulators to a lower level (consider a corporation where general management regulates departments, which in turn regulate subdepartments, etc.; more on this in sect. 5). In this context we should note that we will employ anthropomorphic language in the description of both human and nonhuman systems, simply because it is useful and concise (Ågren & Patten, Reference Ågren and Patten2022; Dennett, Reference Dennett, Beckermann, McLaughlin and Walter2009; Appendix 1).

3.2. Why proxies are imperfect: Limits of legibility and prediction, and the necessity to choose

It is natural to wonder if a regulator could dispense with proxies and focus directly on the goal. We argue that this is often very difficult or impossible. Proxies are almost inevitable in complex goal-oriented systems. A regulator, even if it takes the form of a distributed process, faces three limitations: Restricted legibility; imperfect prediction; and the necessity to choose. These limitations also apply to agents.

First, regulators are limited by legibility (Scott, Reference Scott2008). Many abstract goals cannot be observed directly, given sensory and computational limits; a proxy is an approximation of the goal that can feasibly be observed, rewarded, and pursued. Consider the abstract goal of reducing a rat population. A continuous rat census is simply not feasible in practice. Even if it were, it would not be possible to incentivize individual agents based on this information, for reasons such as the free-rider problem (Hardin & Cullity, Reference Hardin, Cullity and Zalta2020). Therefore, a regulator must operationalize goals based on observable signals that can be incentivized. The Hanoi government chose to measure the number of delivered rat corpses. This proxy is legible by both the agents and the regulator, it can be used as an incentive, and it is a plausible correlate of the underlying goal. The processing of such a signal constitutes a regulatory model (Conant & Ashby, Reference Conant and Ashby1970), which captures how proxies are related to agent actions (or traits), and how well these promote the regulatory goal.

Second, regulators are limited in their ability to predict the future. For a proxy to perfectly capture a complex goal, the regulatory model would often have to represent all relevant factors and their possible consequences. This may be possible in very simple systems, but in complex biological or social systems a perfect proxy implies a “Laplacian demon”: A model that can perfectly predict any future state of the world given any behaviour of an agent (Conant & Ashby, Reference Conant and Ashby1970). Absent the ability to perfectly predict the future, a proxy remains prone to unanticipated failure modes in the future.

Third, regulators are limited by the need to choose. Decision systems across scales can be associated with scalar metrics such as utility, fitness, profit, or impact factor. The reason is that a system tasked with identifying the “better” of any two options in a consistent manner requires options to be rank-ordered. This implies that the options can be described using a scalar metric (see revealed preference approach in economics; Houthakker, Reference Houthakker1950; Samuelson, Reference Samuelson1938; von Neumann & Morgenstern, Reference von Neumann and Morgenstern1944). “Consistent” here means that decisions are coherent and not self-contradictory, for instance that they are transitive (if A > B, and B > C then A > C). Clearly, not all goals can be described in this way (e.g., in socioeconomic systems; Arrow, Reference Arrow1950). But to the degree that consistent regulatory decisions are desirable, a single scalar proxy is implied. It is important to reiterate that this proxy may summarize highly complex, multidimensional information (e.g., multiple contingent input metrics). It may be explicitly constructed (e.g., an objective function in AI, or an academic impact factor), or be implicit, requiring that researchers infer it from an observed set of choices (e.g., utility or fitness) or regulatory system (e.g., voting).

These three limitations shape all regulators, agents, and proxies, because all decision-making systems are practically realized by something physical – and, therefore, something finite. Machine-learning algorithms are limited by computational capacity and training data; peahens and human regulators are limited by the constraints of their eyes and brains. Any proxy reflects the idiosyncrasies, biases, and limitations inherited from the physical system that mediates it.

3.3. Factors that drive or constrain proxy failure

We propose that a pressure towards proxy failure arises whenever two conditions are met:

• • There is regulatory feedback based on the proxy, which has consequences for agents. Agents must be rewarded or selected based on the proxy, which induces optimization of the proxy, either through agent behaviour or through passive evolutionary dynamics. Many commentators emphasize the role of the agent's stakes with respect to proxy performance: The higher the stakes, the higher the pressure for proxy divergence (Muller, Reference Muller2018; O'Neil, Reference O'Neil2016; Smaldino & McElreath, Reference Smaldino and McElreath2016).

• • The system is sufficiently complex. What we mean here is that there must be many causal pathways leading to the proxy that are partially independent of the goal (or vice versa). A system may contain proxy-independent actions that lead to the goal, goal-independent actions that produce the proxy, and/or actions that affect both proxy and goal but unequally (Fig. 1).

Figure 1.

Regulator, goal, agent, proxy, and their potential causal links. Proxy failure can occur when a regulator with a goal uses a proxy to incentivize/select agents. (A) In complex causal networks the causes (arrows) of proxy and goal generally will not perfectly overlap. There may be proxy-independent causes of the goal (c1), goal-independent causes of the proxy (c3), as well as causes of both proxy and goal (c2; note that this subsumes cases in which an additional direct causal link between proxy and goal exists). (B) The regulator makes the proxy a “target” for agents in order to foster (c2). Yet this will tend to induce a shift of actions/properties towards (c3), potentially at the cost of (c1). The causal effects of goals on regulators or proxies on agents are depicted as grey arrows, given that they reflect indirect teleonomic mechanisms such as incentivization or selection. Note that these diagrams are illustrative rather than comprehensive. For instance, the causal diagram of the Hanoi rat massacre would require an “inhibitory” arrow from proxy to goal, as breeding rats directly harmed the goal rather than just diverting resources from it.

The first condition leads to proxy optimization. Note that, although in many systems agent behaviour is the outcome of multiple agent-objectives, the present “generalized” account is best served by focusing on only the proxy-objective and considering competing agent-objectives as constraints on proxy-optimization. Regulatory feedback, which may operate through incentives or selection, will result in amplifying the proxy. The second condition implies that there are actions that optimize the proxy but do not necessarily optimize the goal. We posit that, the greater the causal complexity of the system, the lower the probability that the two optimizations perfectly coincide. Although this hypothesis remains to be rigorously tested, a number of system-theoretic considerations support it. Firstly, more complex systems tend to have more “failure modes” (Manheim, Reference Manheim2018). The more complex a system is, the more difficult it will be for a regulator to map every possible agent action to the entirety of its consequences. Yet, once the proxy becomes a target, agents will begin a search for actions that efficiently maximize the proxy. In practice, actions that optimize only the proxy often seem to be cheaper for agents than actions that simultaneously optimize the goal (Smaldino & McElreath, Reference Smaldino and McElreath2016; Vann, Reference Vann2003). Causal complexity may increase the probability that such actions exist. More complex systems may also tend to produce more unstable and nonlinear relations (Bardoscia, Battiston, Caccioli, & Caldarelli, Reference Bardoscia, Battiston, Caccioli and Caldarelli2017), leading to heavier tails in outcome distributions. This has also been linked to an increased probability of proxy failure (Beale et al., Reference Beale, Battey, Davison and MacKay2020). Intuitively, we can think of an increased probability that some actions exist that very efficiently optimize the proxy, but have detrimental (and potentially catastrophic; Bostrom, Reference Bostrom2014) effects on the goal. Even if agents are passive and exploration occurs through random sampling or mutation, the system will sooner or later “find” those actions or properties that optimize the proxy in the most efficient manner, irrespective of the consequences for the goal.

Another way to think about this is to note that opportunities for proxy failure are latent in the countless number of possible future environments. The regulatory model, and the proxy it gives rise to, can only reflect a finite set of environments. For organisms, this is the environment of evolutionary adaptation (Burnham, Reference Burnham2016); for AI it is the training environment (Hennessy & Goodhart, Reference Hennessy and Goodhart2021). By contrast, there are infinite possible future environments (Kauffman, Reference Kauffman2019), in which the correlation between proxy and goal may not hold. Because of this imbalance, even random change in a complex environment should be expected to promote proxy divergence. But we can go further: Proxy-based optimization actually biases exploration towards such environments. It was precisely the incentive for delivering rat tails that led to rats being bred. An insightful subcategorization of mechanisms underlying this bias has been outlined by Manheim and Garrabrant (Reference Manheim and Garrabrant2018; see Appendix 2).

Importantly, the presence of a pressure towards proxy failure does not guarantee failure: The extent to which a proxy diverges depends on a variety of constraints (which will be elaborated in more detail throughout sects. 4 and 5). For instance, if proxy production diverges sufficiently from goal attainment, the entire system may collapse, as it did in the case of the Hanoi rat massacre (Vann, Reference Vann2003). In other cases, proxy pressure may lead to inflation, because agent “hacks” to produce more proxy are immediately met by an increase in the proxy level required by the regulator. This appears to have driven school grade inflation (McCoy & Haig, Reference McCoy and Haig2020), and it is in fact the natural consequence in any system in which proxy performance matters on a relative rather than an absolute scale (Frank, Reference Frank2011). Some proxies may remain informative, even as they are undergoing inflation, such that the pressure to diverge never leads to actual proxy failure (Frank, Reference Frank2011; Harris, Daon, & Nanjundiah, Reference Harris, Daon and Nanjundiah2020). In other cases, divergence may be precluded because proxies are directly causally linked to the goal, as is assumed for honest (unfakeable) “index” signals in biology. Finally, some decrease (or noise) in the correlation between proxy and goal may be unproblematic for a regulator, depending on its tolerance for such noise (Dawkins & Guilford, Reference Dawkins and Guilford1991).

We will argue that whenever there is some additional selection or optimization pressure that operates on the regulator itself, proxy divergence is constrained. Conversely, in situations where regulators face few constraints, such as in some human social institutions, proxy failure may be persistent and far-reaching (Muller, Reference Muller2018).

4.1. Proxy failure in neuroscience: A new perspective on habit formation and addiction

To pursue objectives, a person implicitly delegates tasks to different parts of their own body and mind. We propose that the dynamics of addiction and other maladaptive habits can be productively understood in terms of proxy failure. Although this framing is novel, its empirical basis is well established (Volkow, Wise, & Baler, Reference Volkow, Wise and Baler2017). The example reiterates that subpersonal agents and regulators can be susceptible to proxy failure even without intentional hacking or gaming by agents. In doing so, it sheds novel light on a key question in “normative behavioural economics” (Dold & Schubert, Reference Dold and Schubert2018), namely why behaviour may systematically diverge from preferences (Box 2).

The brain can be seen as serving the goal of promoting the organism's fitness, utility, or wellbeing (the present account works for each definition of the goal; see Box 2). The variability of the environment and the diversity of actions available to humans make such goals immensely complex to achieve. To manage this complexity, individual neural systems take up subtasks, such as evaluating the relevance of perceived signals or the consequences of possible actions. For this evaluation, the brain must allocate limited resources such as attention or time, and it must associate perceptual inputs and potential actions with some measure of value. This function is performed by a sophisticated “valuation system,” which involves several parts of the limbic system, including medial and orbital prefrontal cortices, the nucleus accumbens, the amygdala, and the hippocampus (Lebreton, Jorge, Michel, Thirion, & Pessiglione, Reference Lebreton, Jorge, Michel, Thirion and Pessiglione2009; Levy & Glimcher, Reference Levy and Glimcher2012). These brain regions are strongly influenced by a neuromodulatory system that plays a key role in neural valuation: the dopamine system (Lewis & Sesack, Reference Lewis, Sesack, Bloom, Björklund and Hökfelt1997; Puig, Rose, Schmidt, & Freund, Reference Puig, Rose, Schmidt and Freund2014).

Many addictive drugs affect the dopamine system (reviewed in Wise & Robble, Reference Wise and Robble2020). Dopamine is often described as a pleasure chemical, but it is more accurately understood as one of the brain's proxies for “value,” broadly construed. The neuromodulator mediates behavioural “wanting” rather than hedonic “liking” (Berridge & Robinson, Reference Berridge and Robinson2016). Berke (Reference Berke2018) argues that dopamine enables dynamic estimates of whether a limited internal resource such as energy, attention, or time should be expended on a particular stimulus or action. Coddington and Dudman (Reference Coddington and Dudman2019) suggest that dopamine signalling reflects a “scalar estimate of consensus” – that is, a proxy – aggregated from complex inputs from across the brain, to estimate “value.” Analogously, the dopamine signal has also been described as approximating “formal economic utility” (Schultz, Reference Schultz2022).

When the dopamine system is functioning appropriately, it contributes to the welfare of the organism. Bursts of dopamine accompany unexpected rewards and other salient events (Bromberg-Martin, Matsumoto, & Hikosaka, Reference Bromberg-Martin, Matsumoto and Hikosaka2010), which enable synaptic credit assignment among neural representations of stimuli or courses of action (Coddington & Dudman, Reference Coddington and Dudman2019; Glimcher, Reference Glimcher2011). Over time, dopamine-modulated synaptic weights in motivation-related brain regions (Reynolds, Hyland, & Wickens, Reference Reynolds, Hyland and Wickens2001; Shen, Flajolet, Greengard, & Surmeier, Reference Shen, Flajolet, Greengard and Surmeier2008), including the nucleus accumbens, come to serve as proxies for the average value of the corresponding representations. Representations compete for internal resources such as energy, attention, or time (Berke, Reference Berke2018; Peters et al., Reference Peters, Schweiger, Pellerin, Hubold, Oltmanns, Conrad and Fehm2004). This class of neural competition can be described as “behaviour prioritization,” rather than the more specific “decision making,” as it spans a spectrum that includes attentional control (Anderson et al., Reference Anderson, Kuwabara, Wong, Gean, Rahmim, Brašić and Yantis2016; Beck & Kastner, Reference Beck and Kastner2009), exploration (DeYoung, Reference DeYoung2013), deliberative weighing of options (Deserno et al., Reference Deserno, Huys, Boehme, Buchert, Heinze, Grace and Schlagenhauf2015), and unconscious habit formation (Yin & Knowlton, Reference Yin and Knowlton2006).

Many drugs of addiction interfere with normal dopamine signalling, effectively hacking the neural proxy for value. This “hijacking” of the dopamine reward signal (Schultz, Reference Schultz2022) causes the brain to overvalue drug-related stimuli and actions (Franken, Booij, & van den Brink, Reference Franken, Booij and van den Brink2005; Wise, Reference Wise2004). Simultaneously, the dopaminergic action of drugs can lead to a decrease in overall dopamine sensitivity, because of well-established biochemical habituation effects (Diana, Reference Diana2011; Volkow et al., Reference Volkow, Wise and Baler2017). This leads to a decreased sensitivity to the drug, eliciting the need to seek progressively increasing dosages. At the same time any nondrug-related behaviours become less “valued.” The resulting neglect of basic social or bodily needs can lead to severe impairments in health and wellbeing. A similar dopamine-mediated phenomenon can occur in the case of any maladaptive habit, such as excessive unhealthy eating (Small & DiFeliceantonio, Reference Small and DiFeliceantonio2019; Volkow et al., Reference Volkow, Wise and Baler2017): The adaptive function of the dopamine system is undermined by the targeted pursuit of dopamine-triggering activities.

Note that the “regulator,” here, is the behaviour prioritization system of the addict's brain, and the “agents” are neural representations, which compete for a limited resource: Neuronal credit assignment (Cruz & Paton, Reference Cruz and Paton2021). The “regulator” consists of a neural circuit linking the basal ganglia, thalamus, and prefrontal cortex, that mediates competition between synaptic weights and representations (Bullock, Tan, & John, Reference Bullock, Tan and John2009; Mink, Reference Mink2018; Seo, Lee, & Averbeck, Reference Seo, Lee and Averbeck2012). The brain's “agents” are subpersonal entities, which cannot “game” the system. Instead, the dopamine system modulates synaptic weights, which ultimately represent the “value” of different “representations” of actions or stimuli. Drug use is an action that directly amplifies these proxies independent of their usual relationship to wider goals. Therefore, use-related synaptic patterns are strengthened, leading to increased drug-seeking.

The key limitations necessitating a proxy system are legibility, prediction, and the need to choose (sect. 3.2). Dopaminergic drugs mediate changes to synapses that are “legible” to the brain's valuation system: They manipulate the “currency” that mediates ordinary decisions. Natural selection cannot possibly anticipate all possible experiences and chemicals that can trigger elevated dopamine, reflecting the limits of prediction in a complex system. Similarly, the pattern-recognition systems in the brain cannot anticipate all possible consequences of the organism's actions. The need to choose implies that the brain must rank-order possible actions at any given moment.

But neural proxy failure is nevertheless contingent on numerous additional factors. A propensity for addiction has been linked to factors in childhood and adolescence (Leyton & Vezina, Reference Leyton and Vezina2014). More generally, most individuals consciously constrain behavioural impulses to use drugs or engage in other potentially maladaptive behaviours on a day-to-day basis. Indeed, the conscious shaping of habits and preferences appears crucial to a self-determined life (Dold & Schubert, Reference Dold and Schubert2018). Such individual efforts are embedded into legal and social constraints that can enable or discourage addiction and related behaviours (Braganza, Reference Braganza2022a).

In summary, addiction and maladaptive habit formation can be viewed as neural instances of proxy failure. To our knowledge, this is the first time that a neural phenomenon has been described in terms of Goodhart-like dynamics. The proxy failure perspective suggests that the risks of addiction and maladaptive habit formation may be natural consequences of pursuing goals via neural representations of (i.e., proxies for) value. If this is the case, a “brain disease” framing of addiction may contribute to excessive medicalization (Satel & Lilienfeld, Reference Satel and Lilienfeld2014) at the expense of exploring structural changes at broader environmental, social, or economic levels (Heather, Reference Heather2017). Later, we will explore a case in which humans appear to harness proxy divergence within themselves in order to support their conscious goals (sect. 5.3).

4.2. Proxy failure in economics: Performance metrics and indicatorism

The disciplines in which proxy failure may have been studied most intensely, albeit under different names, are management and economics (Holmström, Reference Holmström2017; Kerr, Reference Kerr1975; van der Kolk, Reference van der Kolk2022). Consider, for instance, the use of key performance indicators (KPIs). Price and Clark (Reference Price and Clark2009) state in unequivocal terms that: “people who are measured on KPIs have an incentive to play games. […] The indicator ceases to indicate.” A prominent early example was Lincoln Electric's plan to incentivize typists in its secretarial pool through a piece-rate for each key stroked (The Lincoln Electric Company, 1975). This apparently led to typists continuously tapping the same key during lunch hours (Baker, Reference Baker2002). Many additional entertaining examples are collected in Stephen Kerr's management classic: “On the folly of rewarding A, while hoping for B” (Kerr, Reference Kerr1975). More recent examples include the Wells Fargo scandal, where excessive incentives led to fraudulent “cross-selling” (opening additional accounts for customers without their knowledge and against their interest; Tayan, Reference Tayan2019). Many more empirical examples have been described in management and accounting research (Bonner & Sprinkle, Reference Bonner and Sprinkle2002; Bryar & Carr, Reference Bryar and Carr2021; Franco-Santos & Otley, Reference Franco-Santos and Otley2018) – for example, concerning CEO pay (Bénabou & Tirole, Reference Bénabou and Tirole2016; Edmans, Fang, & Lewellen, Reference Edmans, Fang and Lewellen2017).

Inspired by such empirical observations, economic theorists have developed a range of mathematical equilibrium models (Box 3), which describe proxy failure within what is known as the principal–agent framework (Aghion & Tirole, Reference Aghion and Tirole1997; Baker, Reference Baker2002; Bénabou & Tirole, Reference Bénabou and Tirole2016; Holmström, Reference Holmström2017). The task of the principal or “regulator” is cast as the design of an “optimal incentive contract,” given noise or the potential for active “distortion” (Baker, Reference Baker2002; Hennessy & Goodhart, Reference Hennessy and Goodhart2021). In other words, the principal must translate observable agent outputs into regulatory feedback (e.g., monetary incentives or promotions). The recurring result is that the potential for distortion implies a weaker optimal incentive strength (Baker, Reference Baker2002). In other words, whenever proxies do not perfectly capture the goal, a reduction in the reward for the proxy is warranted. In fact, a particularly important question in the early literature was why pure fixed wage contracts (i.e., contracts without performance-contingent pay) are so common in real businesses (Holmström, Reference Holmström2017). Why do companies not generally, or at least more frequently, base wages on measured performance? The answer is: The risk of proxy failure.

Another focus of the economic literature is the role of proxies in large hierarchical organizations. For instance, we can think of KPIs as the communication interface between general management and individual departments. Aghion and Tirole (Reference Aghion and Tirole1997) develop a theory of delegation of authority in hierarchical corporate contexts, exploring the complex roles of information and “incentive congruence” in determining how proxies can be gainfully used. More generally, proxies appear to reflect the division of labour in complex organizations or multistep processes, which may occur on different timescales. Individual employees, particularly in sales and marketing roles, are typically incentivized for achieving short-term objectives such as signing a contract with a customer, or making contact with a potential buyer. Whole departments can then be measured on the aggregate numbers achieved by their employees. Each department or subdivision must regulate its own subdivisions, necessitating further proxy measures. The use of proxies at each level of such a hierarchy naturally entails the risk of proxy failure, which indeed has been extensively documented. For instance, Tardieu, Daly, Esteban-Lauzán, Hall, and Miller (Reference Tardieu, Daly, Esteban-Lauzán, Hall and Miller2020) explore proxy failure in the context of “parochial” KPIs: Those measured by individual departments that are imperfectly related to wider business outcomes. Ittner, Larcker, and Meyer (Reference Ittner, Larcker and Meyer1997) provide an example from a large bank: “A system put in place to provide incentives for branch managers to increase customer satisfaction quickly began to reward some of the most unprofitable branches” (Baker, Reference Baker2002).

In all these examples proxies diverge from the underlying goals of an organization. But proxies are nevertheless useful, indeed unavoidable, in practice. The informational factors necessitating proxy use by organizations are special cases of the three limits that necessitate proxies (sect. 3.2). Agents in a complex organization often have insufficient knowledge of the top-level organizational goal and how to promote it – the “legibility” limit. Proxies are expedient means of communicating the goal through department hierarchies to coordinate agent behaviour. The prediction limit describes that neither employers nor employees can foresee all the consequences of a given behaviour for the organization. The choice limit is reflected in the frequent evaluation of infinite possible employee actions by single scalar metrics such as KPIs. In addition, psychology is at play: Without direct incentives, which are (i) close in time, (ii) relatively certain, and (iii) directly personally relevant, employees may have little motivation to contribute to an organizational goal (Jones & Rachlin, Reference Jones and Rachlin2009; Kobayashi & Schultz, Reference Kobayashi and Schultz2008; Strombach et al., Reference Strombach, Weber, Hangebrauk, Kenning, Karipidis, Tobler and Kalenscher2015).

Another core insight of the economics and business literature concerns a fundamental constraint on proxy failure. If we posit a corporation's goal as competitive success in the market, proxy failure within businesses is naturally constrained by market selection (Braganza, Reference Braganza2022b). If proxies designed to regulate employees diverge too far from the corporation's goal, profit and share prices may plummet – causing behaviour change. In the worst-case scenario, the whole organization may go bankrupt. Market selection thus provides a hard constraint on proxy failure within a business. Whether implemented by savvy managers or market forces, the constraint controls regulatory models themselves. Because of this, the economic literature has tended to view the very existence of a given practice in real corporations (e.g., fixed wages) as evidence of its economic optimality (e.g., Baker, Reference Baker2002; Holmström, Reference Holmström1979, Reference Holmström2017).

This insight also explains why proxy failure is likely even more relevant in nonmarket-based economies, where the hard constraint of market selection is missing. An apocryphal anecdote (Shaffer, Reference Shaffer1963) tells of a Soviet factory rewarded by the Central Committee for the number of nails it produced, resulting in the output of millions of flimsy, useless nails. When the proxy was changed to instead reward the weight produced, the factory retooled to manufacture one single immense nail. Without the market constraint, proxy failure can remain unmitigated. Specifically, market proxies (e.g., profitability) ensure that consumers must actually find a product useful by measuring whether they buy it. However, the assumption that market proxies thus automatically serve our societal goals is also clearly problematic (Box 4).

Finally, many scholars have pointed out that humans are to some degree purpose driven, that is, not solely interested in extrinsic (proxy) incentives and that extrinsic incentives can indeed undermine worker morale and human wellbeing more generally (Muller, Reference Muller2018; van der Kolk, Reference van der Kolk2022). This suggests an approach to mitigating proxy failure that is unique to human contexts: The promotion of intrinsic incentives towards the goal (Mayer, Reference Mayer2021; Pearce, Reference Pearce1982). Some firms advertise “social objectives,” such as to “Accelerate the world's transition to sustainable energy” for Tesla, to provide both guidance and motivation to employees, beyond their material incentives. This could mitigate proxy failure by directly changing agents' utility, as modelled in the multitasking literature (Braganza, Reference Braganza2022b; Holmström, Reference Holmström2017). It may also provide a second-order protective function by promoting employee vigilance towards the possibility of unanticipated avenues for proxy failure. Mayer (Reference Mayer2021) suggests that an explicit corporate purpose, combining both profitability and social/ecological responsibility, can mitigate proxy failure both within a firm and in the larger societal context.

In summary, economics and business research reveals key insights into proxies and their potential for failure. From theories of contract structure and measurement, to the theory of prices as a signal for optimal resource allocation (Friedman & Oren, Reference Friedman and Oren1995), to political debate about the effectiveness of the profit motive – choosing and evaluating proxies is a core activity of economists. The present perspective places this literature within a larger framework, suggesting that the mechanisms explored therein may inform, and be informed by, mechanisms studied in other disciplines.

4.3. Proxy failure in ecology: Sexual selection, nested goals, and countervailing forces

Recently, McCoy and Haig (Reference McCoy and Haig2020) have argued that proxy failure occurs in ecological systems in a manner analogous to many social systems. Here, we introduce and expand their argument and highlight the intriguing similarities to the case of economics.

Proxy failure provides a useful framework to understand evolution. For instance, peahens regulate peacocks via mate choice and mothers regulate offspring via embryo selection. The proxy in such cases is any trait or “signal” expressed by the regulated agent that affects regulation, such as a peacock's tail or an embryo's ability to produce hormones. Individual organisms can respond to such pressures as humans do, by deliberately increasing production of the proxy; for example, males of many species prefer to display near less attractive rivals, apparently so that they can themselves appear better (Gasparini, Serena, & Pilastro, Reference Gasparini, Serena and Pilastro2013). But deliberation is not necessary – animals with greater proxy values may also simply be selected and pass on proxy-producing genes. A discrepancy between a signal and the information it was originally selected to convey, that is, proxy failure, is often termed deception (without implying intentionality). It seems undisputable that both honest and deceptive signalling are widespread in ecological systems (Dawkins & Guilford, Reference Dawkins and Guilford1991; Krebs & Dawkins, Reference Krebs, Dawkins, Krebs and Davies1984; McCoy & Haig, Reference McCoy and Haig2020; Wickler, Reference Wickler1965). However, the question of whether stable ecological signals can safely be considered “honest” and “adaptive” has been a subject of controversy since Darwin (Box 5).

Sexual selection in colourful birds and fish is a useful case study for proxy failure (McCoy et al., Reference McCoy, Shultz, Vidoudez, van der Heide, Dall, Trauger and Haig2021). Many female birds and fish choose a mate based on a proxy signal of quality. A famous proxy is carotenoid-based pigmentation: These red–orange–yellow colours are often considered “honest signals” of fitness, either because they incur a cost (Spence, Reference Spence1973; Zahavi, Reference Zahavi1975) or are causally linked to metabolic processes (Maynard-Smith & Harper, Reference Maynard-Smith and Harper2003; Weaver, Santos, Tucker, Wilson, & Hill, Reference Weaver, Santos, Tucker, Wilson and Hill2018). It is thought that female birds and fish prefer redder males because this trait reliably (i.e., honestly) communicates their fitness. However, some male fish and birds have evolved strategies to maximize the proxy without increasing their carotenoid pigments. Some male fish have evolved the ability to synthesize alternative (potentially cheaper) pigments (pteridines; Grether, Hudon, & Endler, Reference Grether, Hudon and Endler2001); some male birds have evolved light-harnessing structures that make their feathers appear more colourful without requiring more pigment (McCoy et al., Reference McCoy, Shultz, Vidoudez, van der Heide, Dall, Trauger and Haig2021). Females have sensory limits; they cannot always tell what produces the red colour. In other words, the evidence suggests that carotenoid-based signalling is not always honest – male birds and fish can “game” the system, driving proxy failure.

Other apparent instances of proxy failure appear in mother–embryo conflict. Babies are costly; therefore, mothers assess embryo quality early and automatically abort less-fit embryos based on proxy signals. Embryos are incentivized to “hack” this system and ensure their own survival. For instance, marsupial infants are born mostly helpless and must find and latch on to a maternal teat to survive. However, some marsupials birth more offspring than they have nipples available, so the infants must literally race to their mother's nipples. Slower individuals are doomed (Flynn, Reference Flynn1922; Hill & Hill, Reference Hill and Hill1955). For example, Hartman (Reference Hartman1920) observed 18 neonates of Didelphis virginiana reaching the pouch and competing for only 13 teats. Mothers seem to screen infants for quality through this race to the teat; speed is a proxy of quality. In response, infants are born with exceptionally strong forearms – they overinvest in performance at the maternal test. A weaker infant who invested their developmental energy into forearms will win against a healthier rival who invested their developmental energy equally across body parts.

Plants similarly show instances of proxy failure in embryo selection. In many plants, far more initial embryos are produced than ultimately survive, because of various stages of maternal screening and selective abortion, but embryos have evolved many strategies to boost their chances of being selected (Shaanker, Ganeshaiah, & Bawa, Reference Shaanker, Ganeshaiah and Bawa1988). For example, the plant Caesalpinia pulcherrima produces seed pods that have between 1 and 10 seeds per pod, and the mother selectively aborts pods with fewer seeds. However, some of these “less fit” pods evade abortion attempts – and then absorb their siblings, acting in a manner contrary to the evolutionary interests of the mother. Similarly, embryos in the woody tree Syzygium cuminii secrete factors to promote abortion of their peers – potentially a response to maternal selection for healthy embryos (Arathi, Ganeshaiah, Shaanker, & Hegde, Reference Arathi, Ganeshaiah, Shaanker and Hegde1996). For a thorough treatment, see Willson and Burley (Reference Willson and Burley1983). Another example of proxy failure in embryo selection in humans, other primates, and horses is presented in Section 5.1.

So what constrains proxy failure in ecology – which forces can keep signals honest (Andersson & Simmons, Reference Andersson and Simmons2006; Connelly, Certo, Ireland, & Reutzel, Reference Connelly, Certo, Ireland and Reutzel2011; Harris et al., Reference Harris, Daon and Nanjundiah2020; Weaver et al., Reference Weaver, Santos, Tucker, Wilson and Hill2018)? We have already noted the controversial nature of questions regarding when or even if ecological signals are honest (Box 5). Here we sidestep such controversies by simply reviewing proposed mechanisms without attempting to claim a general validity (or falsity) of any specific mechanism. In general, the natural selection of regulators should promote “honest signals” (Maynard-Smith & Harper, Reference Maynard-Smith and Harper2003). Such selection should (i) constrain divergence where it occurs and (ii) systematically lead to the identification of proxies that are more resistant to failure. First, even though we have outlined cases in which presumed “honest signals” were gamed, this does not imply that some proxies may not be more resistant to gaming than others, for example, because they are more tightly causally linked to the actual goal (so-called “index signals”; Maynard-Smith & Harper, Reference Maynard-Smith and Harper2003; Weaver et al., Reference Weaver, Santos, Tucker, Wilson and Hill2018). Another proposed mechanism to keep signals honest, which has been highly influential across ecology and economics (Connelly et al., Reference Connelly, Certo, Ireland and Reutzel2011), is “costly signalling” or the “handicap principle” (Zahavi, Reference Zahavi1975; Box 5). It is the idea that, if the cost of a signal is negatively correlated with fitness, then fitter individuals can afford to produce more signal. Regardless of mechanism, selection should favour those individuals that do not unnecessarily fall prey to deceptive signals. This in turn should lead to the selection of more honest signals whenever possible.

Another intriguing parallel to the business literature is that life is often organized in nested hierarchies (Ellis & Kopel, Reference Ellis and Kopel2019; Haig, Reference Haig2020; Kirchhoff et al., Reference Kirchhoff, Parr, Palacios, Friston and Kiverstein2018; Okasha, Reference Okasha2006). Goals at one level of natural selection can become proxies at another level (Farnsworth, Albantakis, & Caruso, Reference Farnsworth, Albantakis and Caruso2017). Species compete within ecosystems; individuals compete within species; cells compete within individuals; genes compete within genomes; and so on (Okasha, Reference Okasha2006). At the highest level, natural selection regulates all living creatures (agents) according to how well they pass their genetic material on to future generations (the goal – see Appendix 1 for discussion of this choice of word). This genetic goal is operationalized through many simpler behavioural proxies: Most creatures are “wired” to survive predation, find food, attract a mate, invest in healthy offspring, and more. These proxy tasks are not themselves important except insofar as they contribute to the goal of passing on genetic material. Each of these proxies has become a goal, itself operationalized by further proxies.

This nested hierarchy offers many future areas of research into proxy failure in evolution. Consider, for example, “selfish” genetic elements (Ågren & Clark, Reference Ågren and Clark2018; Haig, Reference Haig2020) that disproportionately insert themselves into the genome of an organism's offspring; they have “hacked” the organismal proxy of reproduction by jumping up a level in the hierarchy to the actual goal (transmitting genetic information). Certain weeds have evolved to mimic crop plants through a proxy divergence process known as Vavilovian mimicry (Vavilov, Reference Vavilov1922), requiring a “model” (the crop), “mimic” (weed), and “operator” (human or herbicide) (McElroy, Reference McElroy2014). Similarly, signalling between predators and prey, plants and animals, cleaner fish and their clients, and more, is likely all subject to proxy failure and its consequences.

In summary, proxies and proxy failure appear to be as common and well researched in ecological systems as in economic systems, albeit characterized using different terms, such as deceptive or “runaway” signalling. Although the regulatory feedback mechanism is often selection rather than incentivization, the similarities in the resulting dynamics are striking (McCoy & Haig, Reference McCoy and Haig2020). Yet it should also be emphasized that this does not negate the important differences between biological and human contexts: It seems neither accurate nor desirable to reduce individual and social human goals to biological fitness.

5.1. The proxy treadmill: When agent and regulator race to arms

McCoy and Haig (Reference McCoy and Haig2020) describe a second-order dynamic arising from proxy failure, termed the “proxy treadmill” (Fig. 2). The proxy treadmill describes an arms race between agent and regulator, leading to ratcheting cycles of hacking and counter-hacking. This leads to the emergence of three hallmark phenomena, namely (i) instability, (ii) escalation, and (iii) elaboration. We argue below that these phenomena promote the seemingly inexorable growth of complexity in both biological and social systems.

• • Instability arises in a system when an arms race between regulator and agents prevents stable equilibria from being attained. Optimization implies that agents will never stop “searching” for new ways to hack a signal, and given the complexity of ecological systems, they are likely to find one eventually, triggering a new cycle of counter-hacking by the regulator.

• • Escalation describes a quantitative manifestation of instability, whereby agents find novel ways to produce more of the same proxy, and regulators respond by requiring higher proxy levels. It can account for the size of peacock tails, the inflation of embryonic hormone production, the inflation of test scores in education, or publication counts in academia.

• • Elaboration describes a qualitative manifestation of instability, whereby proxy failure continuously drives the modification of given proxies, or the addition of new proxies. It helps characterize the evolutionary elaboration of the peacock's tail into complex shapes, the chemical modification and addition of hormones secreted by embryos, and the addition of novel criteria in education (extra-curriculars) or academic competition (alt-metrics).

Figure 2.

The proxy treadmill. Illustration of the proxy treadmill and its consequences. (A) An agent “hacks” a proxy by, say, finding cheaper ways to make it. The regulator responds by “counter-hacking,” for example, requiring higher levels of the proxy, or requiring new proxy attributes. Each pursues their own goal (which for the agent, here, is reduced to wanting to maximize the proxy). (B) This results in instability, escalation, and elaboration of the signalling system.

McCoy and Haig (Reference McCoy and Haig2020) introduce the proxy treadmill and its three hallmarks in the context of mother–embryo conflict, reviewing traces it has left in the genomes of both primates and horses. A signal originally used by the mother as a proxy for embryo viability during early pregnancy, luteinizing hormone, was hijacked by the embryo to improve its own chances of survival: Embryos produce a hormone that mimics luteinizing hormone called chorionic gonadotropin or CG (Casarini, Santi, Brigante, & Simoni, Reference Casarini, Santi, Brigante and Simoni2018). Several successive rounds of hacking and counter-hacking appear to have ensued, leading to an escalation to a present in which “heroic” amounts of hormone are produced by embryos (Zeleznik, Reference Zeleznik1998) and required by mothers (McCoy & Haig, Reference McCoy and Haig2020). Simultaneously, elaboration of the signal occurred, as embryos accumulated mutations that changed the structure of the hormone, for instance increasing the half-life and thereby the concentration of the hormone in circulation (Henke & Gromoll, Reference Henke and Gromoll2008). This illustrates how instability, escalation, and elaboration have shaped the present signalling system. Analogous processes occur when embryos evolve new “mimics” of proxy signals or mothers interpret existing signals in new ways (see Haig, Reference Haig1993; McCoy & Haig, Reference McCoy and Haig2020). The authors liken this to the addition of novel selection criteria in job interviews, after grades have inflated to a degree of relative uninformativeness.

A key tenet of the proxy treadmill is that uninformative proxies may sometimes linger for a variety of reasons, despite the pressures of natural selection on both the regulator and the agents. For instance, regulators may keep observing relatively uninformative proxies because of a first-mover disadvantage, which is closely related to Fisherian runaway signalling (Fisher, Reference Fisher1930; McCoy & Haig, Reference McCoy and Haig2020; Prum, Reference Prum2010; Box 6). Fisher argued that any peahen with a preference for peacocks with small tails will have sons who are unattractive to other peahens. In other words, large tail sizes could stabilize, not because they signal fitness, but because the male trait and the female preference for that trait are jointly inherited. By this mechanism, competitive selection can lock in a real selective advantage for an otherwise uninformative, or even a fitness-decreasing, proxy. McCoy and Haig note the similarity with Princeton University's unilateral attempt to halt grade inflation (Stroebe, Reference Stroebe2016): No other universities followed suit, students began ranking Princeton lower, and Princeton eventually abandoned the policy (Finefter-Rosenbluh & Levinson, Reference Finefter-Rosenbluh and Levinson2015). Other reasons why uninformative proxies may linger are simple slack in the selection of regulators or that it can be economical for regulators to employ even poor proxies, if the cost of acquiring more accurate information is high (see Dawkins & Guilford, Reference Dawkins and Guilford1991).

The proxy treadmill bears striking similarities to what might be called the “academic proxy treadmill” (Biagioli & Lippman, Reference Biagioli and Lippman2020), and the “bureaucratic proxy treadmill” (Akerlof & Shiller, Reference Akerlof and Shiller2015). Biagioli and Lippman (Reference Biagioli and Lippman2020) describe how in academia, “new metrics and indicators are being constantly introduced or modified often in response to the perceived need to adjust or improve their accuracy and fairness. They carry the seed of their never-ending tuning and hacking, as each new metric or new tuning of an old one can be subsequently manipulated in different ways.” Similar mechanisms of continuous hacking and counter-hacking appear to occur when a government regulates corporations, which find ways to hack the regulations, triggering novel regulations (Akerlof & Shiller, Reference Akerlof and Shiller2015). The predictable results, analogous to biological cases, are continuously expanding and ever more complex bureaucracies (Muller, Reference Muller2018).

Together, this suggests that the proxy treadmill may act as a driver of complexity in both biological and social systems. Some proxies may rapidly inflate and then completely fail, as was the case for the Hanoi rat massacre. Other proxies may inflate and then remain stable, even though they have become less informative, such as inflated grades (McCoy & Haig, Reference McCoy and Haig2020), publication metrics (Biagioli & Lippman, Reference Biagioli and Lippman2020), or “aesthetic signals” (Prum, Reference Prum2012). Still other proxies may be impossible to inflate because they are causally linked to the goal (index signals) or may remain informative even while inflating (costly signals; Harris et al., Reference Harris, Daon and Nanjundiah2020; Zahavi, Butlin, Guilford, & Krebs, Reference Zahavi, Butlin, Guilford and Krebs1993). Crucially, any change in a proxy or incorporation of a new proxy by the regulator gives rise to novel affordances for the agent, which can lead to new rounds of hacking and counter measures. The result is an ecology of signals, constantly being renegotiated, but tending to increase in complexity and elaboration over time. Note that we do not mean to imply it is the only driver of complexity; as for example in biological systems numerous additional forces have been studied (Box 7).

In summary, proxy failure may lead to a second-order dynamic: The proxy treadmill, an arms race of hacking and counter-hacking that causes instability, inflation, and elaboration. This phenomenon again highlights the question of whether equilibrium analyses may systematically obscure some of the more interesting behaviours of biological and social systems: Signalling systems may remain far from equilibrium because of high levels of instability, and equilibria that are reached may be transient. Proxy failure and the proxy treadmill may be key drivers of complexity in both biological and social systems.

5.2. The proxy cascade: When high-level proxies constrain lower-level proxy failure

Throughout this paper, we have repeatedly noted that proxy-based decision systems are embedded in larger hierarchies, where high-level goals may constrain or shape low-level goals (Heylighen & Joslyn, Reference Heylighen, Joslyn and Meyers2001). For instance, markets contain corporations, which contain departments, which contain individuals (Aghion & Tirole, Reference Aghion and Tirole1997). Individuals are themselves nested hierarchical systems, containing organs such as the brain which in turn are organized in subsystems, down to the level of cells and organelles (Farnsworth et al., Reference Farnsworth, Albantakis and Caruso2017). How, then, might proxies and goals interact across multiple levels? Here, we argue that high-level regulation can constrain and control proxy failure at all lower levels – a cascade of regulation.

An important point to reiterate before we proceed is that the identification of proxy failure is inseparable from the definition of a goal at a specific level. In a hierarchy, the proxy of a “parent” regulator typically becomes the goal for a “child” agent. The failure claim is thus specific to the goal of the regulator but not to that of the agent. We emphasize this point, because considering proxy failure across levels requires particular attention to (i) goals at multiple levels and (ii) the specific goal that underlies the claim of proxy failure.

How can regulation cascade down a sequence of proxies in a nested hierarchy, effectively allowing the pursuit of integrated, high-level goals? Consider a corporation in which management has created a proxy (sales revenue) to promote its goal (say, profitability; Fig. 3). The sales department adopts sales revenue as their goal, and therefore creates a proxy (the number of telephone calls made) to motivate the outputs of its salespeople. The proxy of each higher level becomes the goal of the lower level. Proxy failure can occur at any level. But importantly, the corporation itself is subject to regulation at a still higher level, namely market selection.

Figure 3.

The proxy cascade. Proxies often regulate the interaction between nested modules of complex hierarchical systems, such as corporations. Goals of lower levels tend to be proxies used by higher levels to regulate and coordinate those lower levels. The proxy cascade describes how a higher-level constraint cascades down the levels, constraining proxy failure at all lower levels.

First, assume proxy failure occurs at the department level: The sales department artificially inflates sales revenue numbers, perhaps by an accounting trick. If such department-level proxy failure undermines firm profitability (the firm-level goal) to a large enough extent, then bankruptcy might ensue. In practice, such an outcome may be mitigated by corporate management, which is thus efficiently incentivized to constrain department-level proxy failure. Regardless of the mechanism, market selection provides a higher-level constraint on the degree of proxy divergence at the department level.

Second, assume proxy failure occurs at the subdepartment level. Recall that the sales department chose the number of calls made as its proxy for the goal of increased revenue. Now employees have the goal to maximize the number of calls, but they may do this in ways that do not lead to increased sales. Perhaps employees purposely keep calls short to increase total number, even though they would be more likely to make a sale if they stayed on the line longer with potential clients. This would undermine the department-level goal as well as the corporate-level goal: More calls, lower sales revenue, less profit. Would this lower-level proxy failure also be constrained? It arguably would. First, if the department-level proxy is constrained to function reasonably well, then department managers are efficiently incentivized to mitigate lower-level proxy failure. Second, if, for whatever reason, subdepartment-level proxies do excessively diverge, the firm will go bankrupt just like a corporation with excessively divergent proxies at the department level.

Therefore, wherever proxy failure occurs in a hierarchy, it seems to be constrained by the highest-level proxy – what we term a proxy cascade. In the words of Heylighen and Joslyn (Reference Heylighen, Joslyn and Meyers2001), “higher-level negative feedbacks typically constrain the growth of lower-level positive feedbacks.” Informational limits (sect. 3.2) will still produce the pressure towards proxy failure and runaway dynamics at each level, but regulation of the regulators causes a counter pressure which cascades down the levels. At each level, this may lead to dynamic phenomena such as the proxy treadmill, the identification of more stable or honest proxies, or a mix of such phenomena. Overall, the result is a noisy self-organization of the cascade of proxies, ultimately allowing the integrated pursuit of a high-level goal.

The proxy cascade also seems to capture adaptationist (see Box 5) aspects of the interaction between natural selection and sexual selection. For example, female birds choose males based on how beautiful the males are – a proxy of their quality as a mate. But if the male develops ornaments that are too cumbersome or burdensome, the male (and his offspring) cannot survive the rigours of natural selection. This is not merely theoretical; Prum (Reference Prum2017) describes “evolutionary decadence,” by which female aesthetic choice shapes outrageous ornaments in certain bird species – potentially increasing extinction risk. Beyond mate choice, another instance of the proxy cascade concerns cooperative proxies for intimately associated organisms (such as hosts and symbionts); when cooperation is not properly enforced, or when proxies for cooperation fail, species face extinction risk (Ågren, Davies, & Foster, Reference Ågren, Davies and Foster2019); for example, Wolbachi endosymbionts threaten survival of populations of butterfly by altering sex ratios (Dyson & Hurst, Reference Dyson and Hurst2004). In this manner, natural selection may exert a constraining hand over proxy failure at lower levels in ecology. We offer these as initial ideas of proxy cascades in biology, but note that substantial further theoretical and empirical research is needed.

The proxy cascade suggests that high-level regulation can constrain lower-level proxy failure. It explains why high-level selection systems, such as natural selection or the market, can so efficiently coordinate complex structures towards coherent high-level goals. Further, the cascade of proxies suggests another possible opportunity for our pursuit of human goals – and that is the subject of the next section.

5.3. Proxy appropriation: When high-level goals hack low-level proxies

Finally, we introduce the novel idea that higher-level systems may harness or “appropriate” proxy failure at lower levels to achieve their high-level goals. We illustrate this using the example of artificial sweeteners.

Humans sense sweetness via the oral T1 receptor system, which employs a molecular “lock and key” mechanism (DuBois, Reference DuBois2016; Li et al., Reference Li, Staszewski, Xu, Durick, Zoller and Adler2002). The “goal” of such taste receptors is to enable the organism to base food choices on estimates of the nutritional value of ingested foods. The specific mechanism by which “assessment” takes place resides in the intricate molecular structure of the receptor dimer (the lock; Perez-Aguilar, Kang, Zhang, & Zhou, Reference Perez-Aguilar, Kang, Zhang and Zhou2019), which was shaped by evolution to “identify” valuable substrates (the keys). If “lock” and “key” sufficiently match, the molecules bind and T1 receptors initiate a complex signalling cascade, which ultimately feeds into a value estimate of the neural reward system (Breslin, Reference Breslin2013). Sweetness, as assessed by T1 receptor binding probability, can thus be understood as a proxy for nutritional value. All organisms, from bacteria to humans, rely on similar molecular assessment mechanisms to “infer” the nutritional value of molecular substrates, in order to guide ingestion decisions towards the goals of survival and homoeostasis (Jeckelmann & Erni, Reference Jeckelmann and Erni2020).

If receptor binding probability can be considered a “molecular proxy,” then what constitutes hacking? Some molecules might mimic the specific biophysical properties of the target molecules leading to “non-specific” binding (Frutiger et al., Reference Frutiger, Tanno, Hwu, Tiefenauer, Vörös and Nakatsuka2021). The artificial sweetener aspartame binds to the T1 receptor, eliciting a 200-fold more powerful sweetness effect than conventional sugars (Magnuson, Carakostas, Moore, Poulos, & Renwick, Reference Magnuson, Carakostas, Moore, Poulos and Renwick2016). Although the exact molecular mechanisms remain incompletely understood (DuBois, Reference DuBois2016), the consequence is that subjective sweetness ceases to track nutritional value. In other words, the pursuit of the proxy (whatever the receptor binds) has undermined its relation to the original evolutionary goal of that receptor (identifying energy-dense sugar molecules). Similar accounts emerge for some cases of neural or ecological proxy failure when they are traced down to the molecular level – they often involve “nonspecific” binding of receptors that play key roles in regulatory systems (see sects. 4.1 and 4.3).

The case of aspartame is particularly interesting, however, because it seems to reflect proxy appropriation rather than proxy failure: Proxy failure with respect to the goal of ingesting sugar would seem to imply undernourishment, but consumers choose sweeteners precisely because they allow sweet sensations without high calorific value. The fact that aspartame undermines the original purpose of both the molecular receptors and the neural reward system, thus becomes a feature. Our conscious self can hack our own reward system by intervening at the molecular level. Although the excessive use of artificial sweeteners entails problems of its own (Borges et al., Reference Borges, Louzada, de Sá, Laverty, Parra, Garzillo and Millett2017; Hsiao & Wang, Reference Hsiao and Wang2013), this phenomenon nevertheless illustrates that lower-level proxy failure can be harnessed to promote higher-level goals, allowing us to overcome narrow evolutionary dictates.

Above, we have observed the interaction between proxies from the molecular level to the level of the conscious individual. Remarkably, the ramifications of the proxy nature of the T1 receptor can also be traced further up the levels to social and economic proxy systems. The choices of individuals to consume aspartame translate to the profitability of products containing aspartame. In other words, the molecular proxy performance determines the market-level proxy performance, directing human activities and resources towards the production, processing, distribution, marketing, and regulation of the artificial sweetener. In this example, the goals of conscious individuals are arguably what directs the whole system, harnessing proxy failure at lower levels and guiding proxies at higher levels. To what extent this can be generalized to other chemical components of our remarkably unexplored modern diets remains to be explored (Barabási, Menichetti, & Loscalzo, Reference Barabási, Menichetti and Loscalzo2019). Indeed, corporate practices that harness addictive dynamics (e.g., in fast food; Small & DiFeliceantonio, Reference Small and DiFeliceantonio2019) appear to reflect additional instances, in which proxy failure at a lower level (“disrupted gut–brain signalling”) is harnessed to serve higher-level goals (profitability; Box 8).

Exploring these cases suggests two important observations. The first is that categorizing a phenomenon as proxy failure requires attributing a specific goal to the system. It is often natural to consider the goals of conscious individuals. But it is important to recognize that the system may ultimately be guided by goals at other levels. The second is that proxies at one level tend to be goals at other levels, raising the question to which degree a chain or web of proxies is sufficient to understand proxy failure, and teleonomic behaviour more generally. Goals appear as abstract objects, which help us understand why proxies are the way they are or how they might develop into the future. Yet the only tangible footprints that goals leave behind in the physical world are proxies.