A striking aspect of the Lieder and Griffiths (L&G) article is the broad generality of the proposed “resource-rational” approach (e.g., see Box 2 of the target article). It is intended to be applicable to nearly every domain in cognitive science, including perception, language, memory, attention, reasoning, and decision making. Posing a theory at this level of generality has considerable strengths, but also weaknesses. An important weakness is the lack of emphasis on identifying specific constraints that are constant across domains.
For example, L&G identify constraints operating in the domain of decision making (e.g., a person may need to minimize the amount of time required to reach a decision), which are different than the constraints operating in the domain of memory (e.g., a person may need to minimize the use of limited memory resources), which are different than the constraints operating in the domain of attention (e.g., a person may need to minimize the use of limited attentional resources), which are different than the constraints operating in the domain of reasoning (e.g., a person may need to minimize the number of variables that are reasoned about), which are different than the constraints operating in the domain of language (e.g., a person may need to minimize the amount of communication effort). Our concern is that a lack of constraints on what counts as a constraint leads to theories from different domains that have little or nothing in common. This, in turn, defeats L&G's stated purpose of providing “unifying explanations,” and may lead to over-fitting and “just-so” theorizing.
It does not have to be this way. We have been pursuing a research program meeting many of the desiderata of L&G's rational resource approach but, critically, propose a common constraint across domains, namely an information-theoretic capacity constraint (e.g., Bates et al. Reference Bates, Lerch, Sims and Jacobs2019; Sims Reference Sims2016; Reference Sims2018; Sims et al. Reference Sims, Jacobs and Knill2012). Our program hypothesizes that the need for efficient data compression (i.e., efficient representations) shapes biological systems in many of the same ways that it shapes engineered systems. If true, then the tool engineers use to analyze and design information-processing systems, namely rate-distortion theory, can profitably be used to understand human perception and cognition. In brief, this theory provides an optimal framework relating the limited capacity of a system to its optimal task performance.
Consider, for example, our application of rate-distortion theory to the study of visual working memory. Here, visual working memory is thought of as an information channel. When an observer encodes a visual image in visual working memory, the observer is sending a message to his or her future self. Because the message retrieved from memory will be different than the sent message – because of the memory store's limited capacity and to memory noise – the observer needs to use the retrieved message to make his or her best guess as to the sent message, thereby recalling the image. In this application, the only constraint is the capacity of visual working memory, measured as the mutual information between the sent and retrieved messages (roughly, a measure of how well the sent message can be reconstructed from the retrieved message). If visual working memory has high capacity, then the observer can recall many of the fine perceptual details of the image. In contrast, if it has low capacity, the observer will be able to recall only coarse details, such as category information (e.g., the image depicted a boy eating an apple). Although described here in an intuitive manner, a strength of rate-distortion theory is its rigorous mathematical foundation which have made it commonplace in the field of engineering.
At first glance, it may seem as if rate-distortion theory is relevant only in tasks that can be regarded as involving communication. In fact, it is relevant to any capacity-limited agent (biological or artificial) that needs to form efficient mental representations while seeking to maximize task performance. That is, it is relevant to nearly all of human perception and cognition. Admittedly, it is not always obvious how to apply this theory to many aspects of cognition. In the remainder of this commentary, we briefly describe two recent efforts to expand the application of the theory across domains.
First, we have developed a deep neural network system that approximately implements rate-distortion theory in a task-general manner (Bates & Jacobs Reference Bates and Jacobs2019). It consists of two networks, a memory module that uses the theory to learn efficient latent representations, and a decision module that uses the memory module's latent representations to perform a task. Because of the connection between the memory module's representations and the decision module, the system learns approximately optimal representations which are both capacity-limited and task-dependent. Importantly, the system is trained “end-to-end,” operating on raw perceptual input (e.g., pixels) rather than intermediate levels of abstraction, as is the case with most psychological models.
Second, we are exploring information capacity limits in human reinforcement learning. Here, the goal is to learn a behavioral policy that maximizes task performance. For example, in the game of chess, a (possible) behavioral policy might correspond to a lookup table specifying the optimal move for every board configuration. However, human learners have finite resources, and hence cannot store policies with unlimited complexity or fidelity (see also Botvinick et al. Reference Botvinick, Weinstein, Solway and Barto2015). Instead, humans often learn approximate (compressed) but general policies, such as “control the center of the board.” As applied to reinforcement learning, rate-distortion theory provides a precise mathematical definition of an optimal but capacity-limited policy. Capacity-limited learners necessarily acquire representations that are efficient, resulting in policies that also generalize better to novel situations (Lerch & Sims Reference Lerch and Sims2019).
In conclusion, consistent with the desiderata of L&G's resource-rational approach, the rate-distortion theory framework studies human cognition from an optimality perspective (similar to rational analysis), where optimal task solutions are constrained by people's cognitive architecture (i.e., capacity limits). It does so, however, in a disciplined manner that constrains what counts as a constraint. We believe this approach is necessary to achieve the ambitious and worthwhile goals set out by L&G.
You have
Access
A striking aspect of the Lieder and Griffiths (L&G) article is the broad generality of the proposed “resource-rational” approach (e.g., see Box 2 of the target article). It is intended to be applicable to nearly every domain in cognitive science, including perception, language, memory, attention, reasoning, and decision making. Posing a theory at this level of generality has considerable strengths, but also weaknesses. An important weakness is the lack of emphasis on identifying specific constraints that are constant across domains.
For example, L&G identify constraints operating in the domain of decision making (e.g., a person may need to minimize the amount of time required to reach a decision), which are different than the constraints operating in the domain of memory (e.g., a person may need to minimize the use of limited memory resources), which are different than the constraints operating in the domain of attention (e.g., a person may need to minimize the use of limited attentional resources), which are different than the constraints operating in the domain of reasoning (e.g., a person may need to minimize the number of variables that are reasoned about), which are different than the constraints operating in the domain of language (e.g., a person may need to minimize the amount of communication effort). Our concern is that a lack of constraints on what counts as a constraint leads to theories from different domains that have little or nothing in common. This, in turn, defeats L&G's stated purpose of providing “unifying explanations,” and may lead to over-fitting and “just-so” theorizing.
It does not have to be this way. We have been pursuing a research program meeting many of the desiderata of L&G's rational resource approach but, critically, propose a common constraint across domains, namely an information-theoretic capacity constraint (e.g., Bates et al. Reference Bates, Lerch, Sims and Jacobs2019; Sims Reference Sims2016; Reference Sims2018; Sims et al. Reference Sims, Jacobs and Knill2012). Our program hypothesizes that the need for efficient data compression (i.e., efficient representations) shapes biological systems in many of the same ways that it shapes engineered systems. If true, then the tool engineers use to analyze and design information-processing systems, namely rate-distortion theory, can profitably be used to understand human perception and cognition. In brief, this theory provides an optimal framework relating the limited capacity of a system to its optimal task performance.
Consider, for example, our application of rate-distortion theory to the study of visual working memory. Here, visual working memory is thought of as an information channel. When an observer encodes a visual image in visual working memory, the observer is sending a message to his or her future self. Because the message retrieved from memory will be different than the sent message – because of the memory store's limited capacity and to memory noise – the observer needs to use the retrieved message to make his or her best guess as to the sent message, thereby recalling the image. In this application, the only constraint is the capacity of visual working memory, measured as the mutual information between the sent and retrieved messages (roughly, a measure of how well the sent message can be reconstructed from the retrieved message). If visual working memory has high capacity, then the observer can recall many of the fine perceptual details of the image. In contrast, if it has low capacity, the observer will be able to recall only coarse details, such as category information (e.g., the image depicted a boy eating an apple). Although described here in an intuitive manner, a strength of rate-distortion theory is its rigorous mathematical foundation which have made it commonplace in the field of engineering.
At first glance, it may seem as if rate-distortion theory is relevant only in tasks that can be regarded as involving communication. In fact, it is relevant to any capacity-limited agent (biological or artificial) that needs to form efficient mental representations while seeking to maximize task performance. That is, it is relevant to nearly all of human perception and cognition. Admittedly, it is not always obvious how to apply this theory to many aspects of cognition. In the remainder of this commentary, we briefly describe two recent efforts to expand the application of the theory across domains.
First, we have developed a deep neural network system that approximately implements rate-distortion theory in a task-general manner (Bates & Jacobs Reference Bates and Jacobs2019). It consists of two networks, a memory module that uses the theory to learn efficient latent representations, and a decision module that uses the memory module's latent representations to perform a task. Because of the connection between the memory module's representations and the decision module, the system learns approximately optimal representations which are both capacity-limited and task-dependent. Importantly, the system is trained “end-to-end,” operating on raw perceptual input (e.g., pixels) rather than intermediate levels of abstraction, as is the case with most psychological models.
Second, we are exploring information capacity limits in human reinforcement learning. Here, the goal is to learn a behavioral policy that maximizes task performance. For example, in the game of chess, a (possible) behavioral policy might correspond to a lookup table specifying the optimal move for every board configuration. However, human learners have finite resources, and hence cannot store policies with unlimited complexity or fidelity (see also Botvinick et al. Reference Botvinick, Weinstein, Solway and Barto2015). Instead, humans often learn approximate (compressed) but general policies, such as “control the center of the board.” As applied to reinforcement learning, rate-distortion theory provides a precise mathematical definition of an optimal but capacity-limited policy. Capacity-limited learners necessarily acquire representations that are efficient, resulting in policies that also generalize better to novel situations (Lerch & Sims Reference Lerch and Sims2019).
In conclusion, consistent with the desiderata of L&G's resource-rational approach, the rate-distortion theory framework studies human cognition from an optimality perspective (similar to rational analysis), where optimal task solutions are constrained by people's cognitive architecture (i.e., capacity limits). It does so, however, in a disciplined manner that constrains what counts as a constraint. We believe this approach is necessary to achieve the ambitious and worthwhile goals set out by L&G.