The parallel distributed processing (PDP) approach to modeling cognition has provided a healthy redress of the balance between empiricist and rationalist accounts of human thought. Following Chomsky's demolition of behaviorist theories of thought and language, it was assumed for many years that the mind was a symbol-processing machine, following algorithmic, syntactic rules to solve problems, achieve goals, and so forth. The discovery that PDP networks can behave in systematic rule-following ways has been matched by growing evidence that in many important respects our psychological processes are also only approximately rule-governed, so that a felicitous marrying of model and data has been achieved. In Semantic Cognition, Rogers and McClelland (Reference Rogers and McClelland2004) show how the Rumelhart model can learn to accurately associate properties with their respective noun concepts, while at the same time showing the general influence of the similarity structure of the knowledge being represented. Just as Rosch (Reference Rosch, Rosch and Lloyd1978) proposed, the mind is sensitive to the correlational structure of the world and the concepts we learn correspond to the complex covariation of different properties across semantic domains. The Rumelhart model provides the missing mechanism for how this arises, while at the same time modeling a wide range of now familiar prototype effects such as basic levels, typicality and category-based induction.
As presented, the model does not aim to represent the actual contents of anyone's semantic memory, and so there is still much detail to explore and develop. The following comments are suggestions about directions in which the model could usefully be taken, both to demonstrate its explanatory power and test its limits.
Use of the Context Relation layer
The Context or Relation layer is currently used to determine the type of relation between the noun concept (e.g., pine) and a property (e.g., ISA tree, CAN grow, IS tall). This use of the Context layer appears arbitrary and could lead to difficulties in a more realistic conceptual domain. The Context layer is clearly a vital part of the architecture of the model and cannot be omitted. But perhaps the Context layer might more usefully encode just that – context. Barsalou (Reference Barsalou, Moss and Hampton2003) has reviewed evidence that the properties generated to a noun concept relate to an imagined situational context – so that, for example, very different properties would be generated for a car seen in a parking lot versus a car from the point of view of a driver. Typicality structure can also be highly context dependent (Barsalou Reference Barsalou and Neisser1987; Roth & Shoben Reference Roth and Shoben1983). Output property units could then encode whole properties (can grow, is tall) undifferentiated by their syntax. Syntactic form is a poor guide to the relatedness of properties. In the model most of the “is” relations were visually based. But in real life an “is” relation can encode any number of non-perceptual and abstract properties such as is valuable, is annoying, or is bad for your health. Grouping properties by syntax may not correspond to any real-world structure. An alternative suggestion to try here would be to use the Context layer to input the type of property (part, appearance, function, behavior, origin, etc.) using semantic rather than syntactic criteria to determine types.
Category information is not just another property
Categorical ISA relations have traditionally been treated very differently in studies of semantic memory from other properties. Knowing the category membership of an item will normally provide a much broader range of useful inferences about it than will knowledge of a salient property. The ISA relation captures the kind of thing that the item is, whereas properties just capture a particular property. Category information is also verified more rapidly (Hampton Reference Hampton1984). The model does not reflect this difference structurally, although it is notable that all of the input items reappear as ISA output units. How could the model be asked whether it had learned the properties of superordinate categories – for example, that trees have roots or that fish have gills?
Quantification and modality
A difficulty for any similarity-based model is the handling of extensional reasoning and quantified statements. When it has mastered its knowledge domain, the model will correctly verify that a robin is a robin, a robin is a bird, and a robin is red. It will not be able to explain, however, that a robin is a robin is tautologically true, a robin is a bird is necessarily true (assuming that any non-bird could never resemble a robin sufficiently to belong in that class), whereas a robin is red is generically true – only being true of most adult robins (or in Europe only of adult males). Knowledge of what actually exists is not primarily the job of semantic memory, but the model clearly lacks a way to handle truth under different quantifiers.
Failure to consider the truth of statements extensionally is quite possibly an advantage of the model given that people are also bad at it, and succumb to similarity-based “non-logical” effects when reasoning about category membership (e.g., Hampton Reference Hampton1982; Jönsson & Hampton Reference Jönsson and Hampton2006). But it would be worth exploring whether the model can learn the difference between properties that are necessarily true and those that are typically true.
The reverse side of the coin is whether the model can determine which properties can be expected to co-occur and which may not. Suppose that backpropagation to representation is used to find a representation of an item that is large and yellow, and has petals, as opposed to an item that has roots, gills, and feathers. Can it be demonstrated that some representations are found rapidly and with low residual error (even although the properties have not co-occurred in the training set), whereas others are impossible to represent without a high degree of error. Modal intuitions of necessity and possibility (Rips Reference Rips2001) are an important aspect of semantic cognition, and it would be a bonus for the research program to show how the network can also match such intuitions.
