Metaphor and imagery in design interpretation

Tversky et al. have divided interpretation in design into two sub-processes: construction of mental representations and attribution of meaning to these (Tversky et al., Reference Tversky, Suwa, Agrawala, Heiser, Stolte, Hanrahan, Haymaker and Lindemann2003). As metaphors enable designers to mentally capture complex relations within short, memorable descriptions (Hey and Agogino, Reference Hey and Agogino2007), they intuitively facilitate both of the above sub-processes, and are thus powerful interpretive tools. Indeed, Lakoff and Johnson have elevated the status of conceptual metaphors from a mere decorative linguistic device to a central aspect of human thinking and understanding (Lakoff and Johnson, Reference Lakoff and Johnson2003). The basic principle at work is as follows: by invoking a metaphor (consider their famous example of “time is money”), we are forming a mapping between two domains (time and finance). This enables us to further understand one in terms of the other (e.g., similarly to money, time can also be “spent”). Since such mappings enable to fluidly encode and decode information which is viewed as meaningful from one's perspective, they enhance designers’ capability to cope with design problems in real-time (Casakin, Reference Casakin2006).

When utilizing metaphors, we depart from the literal meaning of a phrase to convey another, non-literal, meaning (Searle, Reference Searle and Martinich1990). These non-literal (intended) meanings can be thought of as our personal interpretation of a certain state of affairs (whether real or imaginary). Since such departure requires us to draw on our imaginative capacities and look beyond the literal meaning, there is a tight link between metaphors and mental images which “accompany” them (and thus plays a cognitive role in interpretation). Consider a team of architects who metaphorically refer to a building as “the beating heart” of their project – in doing so, they project an interpretive image onto a certain part of the design, which then shapes the way the project is understood and thus navigated.

One form of imaginative ability which stands out from design interpretation literature is that of “seeing as” – we look at one object and interpret it as another (Goldschmidt, Reference Goldschmidt1991). For example, a thin 2D rectangle can invoke a mental image of a tall building. Since these cognitive events involve “looking through” the given object (what Liddament described as epistemic transparency; Liddament, Reference Liddament2000), they are associated with metaphoric thinking in design (as we do not take the visual object in front of us “literally”). In this study, we do not limit ourselves to a single type of cognitive phenomenon such as “seeing as”. Instead, we attempt to trace the generation and elaboration of interpretations by focusing on the content of mental images which facilitate these (and other) interpretive events.

A useful starting point for discussing mental imagery is its basic characterization as a “quasi-observational” activity, in which one experiences something which is not present as a perceptual object (Sartre, Reference Sartre1940). However, as imagery refers to a wide range of cognitive phenomena (Richardson, Reference Richardson1969) which facilitate numerous abilities (Stevenson, Reference Stevenson2003), it is essential to further elaborate on our view of imaginative activity. More specifically, Kosslyn has notably advanced the view of imagination as a “screen” onto which mental representations are projected (Kosslyn, Reference Kosslyn1980). Imagining thus consists of a subject “reading-off” information which is temporarily represented in working memory (Dahl et al., Reference Dahl, Chattopadhyay and Gorn1999) for a certain purpose. Such mental reading enhances our ability to think and act, for instance, by facilitating inference-making without the need to manipulate logical terms (Johnson-Laird, Reference Johnson-Laird1983). This immediacy, along with the intuitive nature and power of metaphor, implies of a strong link between metaphoric thinking, mental images, and real-time performance in designing.

Based on the above, we hypothesize that inquiring into such images which appear on the mental “screen” (and thus offer useful information for the designer) can contribute to our knowledge of interpretation activity. We attempt to trace the content of such images by analyzing the ways metaphorical descriptions are externalized in the form of physical objects. Arcangeli has distinguished “seeing-imagining” (close your eye and imagine such a building) from “hearing-imagining” (repeat this with your tune of choice) and from other types of imagery (Arcangeli, Reference Arcangeli2020). This paper focuses on the mental content of imagery, regardless of the modality in which it is generated and experienced by designers. As established by Goel, studying how such content is externalized and utilized in the design process is important for our understanding of design cognition (Goel, Reference Goel1995).

Visualization as a way into the designer's head

Without a doubt, visualization is an effective means for understanding complex processes and events, and as such plays an important role in studying the (often rich) mental world of designers. One notable design visualization method, which takes the view of design as an information-processing activity, can be found in Akin (Reference Akin1989). On the basis of design protocols from architectural design activity, Akin has demonstrated how behavior in design can be described as a sequence of states and transformations between them. States reflect pertinent information dealt with by the designer at a given moment in time, and transformations indicate changes to this information. The diagrams formed by Akin expose the existence of multiple, somewhat concurrent, relations between entities, which shape the way designers engage in problem-solving activity when designing. A more recent method developed by Goldschmidt consists of the creation of “linkographs” – non-directed graphs which link “design moves” (Goldschmidt, Reference Goldschmidt2014). Simply put, design moves are actions/decisions taken by the designer. Linking design moves enables to visualize and trace the structure of the design process, in terms of how one acts based on past events. From a different perspective, Sun et al. have proposed a visualization method termed “creative segment”, which describes sketching in design as a tree with “design paths” for branches. Designers are said to both extend certain paths (by continuously working on a specific idea), add new paths, and revisit these from time to time. The resulting tree structure serves as a visual summary of the process as a whole. As such, it facilitates inquiry into designing – for example, by reflecting how inspirations are combined into a design outcome (Sun et al., Reference Sun, Xiang, Chai, Wang and Huang2014). Finally, taking a different approach, Prats et al. have proposed to trace designers’ thinking by visualizing the progression sketches and identifying transformations between shapes. Their method enables to extract and formalize such transformations as shape rules, which can be used for computational generation of shapes in designing (Prats et al., Reference Prats, Lim, Jowers, Garner and Chase2009). In addition to the above, certain works take an integrated approach – for example, Bilda et al. have used a coding scheme for imagery along with linkographs to examine designer's performance in order to reevaluate the widely accepted view that sketching is essential for conceptual design (Bilda et al., Reference Bilda, Gero and Purcell2006).

We aim to extend this arsenal for inquiry into mental content by offering a high-resolution approach for its visualization and analysis. Our approach thus focuses on the elaboration of mental content at specific moments in time in high detail, as to enable to draw important insights of fleeting information, which would otherwise remain unnoticed.

Situated cognition as an approach for modeling interpretation

Drawing on the situated approach to cognition (Clancey, Reference Clancey1997; Clark, Reference Clark1998; Wilson and Clark, Reference Wilson, Clark, Aydede and Robbins2009), Gero and Kannengiesser have formulated the situated function–behavior–structure framework (situated FBS) (Gero and Kannengiesser, Reference Gero and Kannengiesser2004), as an extension of Gero's original formalization of design in the FBS framework (Gero, Reference Gero1990). In the situated FBS, design is described using a series of transformations between structures (artifacts or their parts), behaviors (what these can do), and functions (what they are meant for). Furthermore, three design worlds are defined: external (outside of the designer), interpreted (an internal reflection of the former), and expected (an internal representation of future intentions). Since it acknowledges the interpretive aspect of designing, we find this framework useful for this study and embrace its distinction between the external and internal worlds. Additionally, we use “structure” to refer to the complete set of entities used by our designers as design representations and further define a “design element” or simply “element” as a single indivisible part of the structure.

Given its general nature, however, this framework demands further elaboration for our task of visualizing interpretive processes in designing. An important aspect of interpretation is the assignment of metaphoric meanings to the artifact (or its representation), a practice which enables conceptual control over the design (Casakin, Reference Casakin2006). Accordingly, it shapes designers’ perspective, as well as the way in which they devise their courses of action. As the transformations in the situated FBS occur between sets of abstract entities (which are independent of the values assigned to them in practice), visualizing such interpretive aspects demands the introduction of additional conceptual tools, which specifically deal with mental content and semantic aspects of designing.

For organizing such mental content, we draw inspiration from Fauconnier's mental space theory (Fauconnier, Reference Fauconnier1994). Mental spaces are virtual environments created by speakers via “space builder phrases”, which establish contexts for conversation. By uttering the phrase “yesterday at the mall”, we can build a mental space for hosting additional details, which may appear in the following conversation. For example, we can say that “John lost his wallet”, which will add both John and his wallet to the space. In this sense, a mental space can be thought of as an abstract container for entities introduced in discourse. Mental spaces enable to visualize interrelations between such entities and are thereby effective for disambiguation of discourse, even in complex cases of conflicting subjective perspectives (Liu and Gao, Reference Liu and Gao2010). In this paper, we describe the designer's internal world using such spaces, which consist of: (1) physical objects serving as design elements, (2) their groupings, and (3) their interpretations.

JRG design and interpretation

JRGs are traditional landscape creations which date back to the Heian period (from ~794 CE), originally designed as spaces for observation or contemplation (Slawson and ZoÌen, Reference Slawson and ZoÌen1991). Despite their description as “gardens”, JRGs often contain little or no vegetation, and instead consist of an arrangement of several rocks, placed carefully on a bed of sand. The rocks commonly have a symbolic value and are assigned with interpretations in varying degrees of complexity (Mansfield and Richie, Reference Mansfield and Richie2009). For example, a single rock may represent a turtle or a crane (these are, in turn, represent longevity), while several rocks may be joined to form miniature “mountain ranges” (possibly reminiscent of the mythological Mount. Sumeru), etc.

We view this tight relation between JRGs and interpretation as an invitation for utilizing it as a context for studying interpretation practices in design. In this paper, we regard interpretation as the assignment of a linguistic description to a physical entity, which metaphorically associates it with another entity. In Figure 1, for example, the middle rock is commonly assigned with the description of a “turtle”, which associates it with a real turtle (in this case, on the basis of resemblance in form).

Fig. 1.

A “turtle” rock at the famous garden of Ryoan-ji in Kyoto, Japan (photograph by the authors).

Into the designer's head

Interpretive descriptions assigned during interpretation activity both reflect and affect how the designer sees the structure, while alterations to the structure may cause changes in the description as well. In this sense, tracking changes in the structure and its interpretation can yield important information regarding the designer's thoughts during the design process.

Figure 10 provides a simplified example of modeling the designer's perspective, using our proposed approach, which draws on Fauconnier's mental spaces (Fauconnier, Reference Fauconnier1994) and on Gero & Kannegiesser's situated FBS framework (Gero and Kannengiesser, Reference Gero and Kannengiesser2004). Two spaces are drawn: M₁ (external reality) and M₂ (metaphoric interpretation world); these are contained in the external and interpreted worlds of the situated FBS framework, accordingly. Recalling the event in Case 1 where the designers have seen the space as a “forest”, we model it as follows: since in external reality several rocks were placed on the tray prior to the appearance of the “forest”, we placed three elements in M₁ which represent these (each corresponding with a single rock). As each rock was interpreted as a “tree”, we connected each element in M₁ with its interpretive counterpart in M₂ (each representing a single tree in the designer's imagination). Furthermore, since the trees were finally seen as a forest, they are grouped within the metaphoric interpretation world and linked to a forest entity. Finally, the fact that the designer had “seen” the forest implies a gestalt grouping of the physical elements, noted by grouping the elements in M₁, as well as by linking this group to the forest in M₂.

Fig. 7.

Case 4; lotus flower with fresh leaves and ones which “already grew”.

Fig. 8.

Case 5; welcoming an unexpected element as a boundary for the design.

Fig. 9.

Case 6; alien structures or islands.

Fig. 10.

Visualizing the designer's perspective using the proposed modeling approach.

The link between an element and its interpretation was named as an “interpretive link”. These links connect physical elements with their imaginary counterparts. Establishing interpretive links may be done by directly naming an element metaphorically (“this is a tree”). Interestingly, such links are not merely referential, but generative as well – the “tree” did not exist as an entity in the metaphoric interpretation world before the rock was assigned with this description. Another way to establish interpretive links is by describing an element's role in the design (“this part serves as a mediator”). In both of these cases, establishing interpretive links shapes the designer's perception of the original object and its contextual space.

Finally, a clarification is due with respect to the entities populating these spaces. Two types of entities are presented in Figure 10: physical elements (white) and interpretive ones (gray/black). The latter are often associated with entities which are external to the design (denoted in parentheses). Furthermore, they are occasionally elaborated into specific kinds – for example, “tree” is later characterized as a “pine tree” or simply “pine”.

Metaphoric interpretation worlds

By using the above modeling approach, we can form maps or diagrams (such as Fig. 10), which can be regarded as “pictures” of the designer's mental content, at certain moments during the design process. What can these pictures tell us about the designer's mental world? Fundamentally, these summarize and visualize: (1) the metaphoric entities which the designer is conceptually employing to understand the structure at a certain moment; (2) certain relations these bear to the structure or its representation. As such, they enable us to make basic and important observations regarding how designers engage in interpretation.

First, as shown in the “Harry and Neal's forest” section, establishing interpretive links populates the metaphoric interpretation world with entities (in this case, “trees”). It is important to notice that the contents of this world should not be seen as an arbitrary agglomeration of independent items. Rather, they are entities chosen based on properties of the structure (e.g., based on the form resemblance between pine trees and elongated rocks), which are organized into meaningful wholes (the “trees” make up a “forest”). In a similar manner, the “leaves” in Case 4 make a “lotus flower” (Fig. 7), etc.

Second, the designer's mental content is not simply projected as a premade interpretation onto the structure. Instead, interpretations seem to be dynamically constructed on-the-fly, as a response both to one's own thoughts and actions and to the changing structure. Continuing with Case 1, after an initial interpretation was assigned (”tree”) by focusing on a single rock (”this looks like a tree-top”), the subjects resolved to freely place several rocks in the space. This was immediately followed by seeing the design as a “forest” (Figs. 3–6). The emergence of this forest, which served as the global interpretation (or setting) of the design, may be further elaborated as follows: first, an interpretation was assigned to an individual element. Then, several elements were placed in the space, inheriting this interpretation, and consequently (also considering their similarity) perceived as a group. Finally, a unifying interpretation was proposed. The global interpretation of the “forest” was thus generated as a consequence of (1) assigning previous interpretation of “trees”; (2) re-perceiving the physical elements under a different gestalt, which demanded adapting the interpretation by relating the parts (trees) with a new whole (forest). This episode supports Suwa and Tversky's suggestion that shifts in focus are important drivers for design (Suwa and Tversky, Reference Suwa and Tversky1997). In this case, a shift in observation in terms of groupings seems associated with the organization of the metaphoric interpretation world as a whole.

Third, rather than strictly bounded or well defined, designers’ interpretations seem flexible and open for further structuring. Surprisingly, the designers in Case 1 resolved to creatively expand the interpretive framework, by detaching from the physical dimension and introducing interpretive entities which do not have any external representation (tree-seeds and birds). This helped in enriching their metaphoric interpretation into a narrative (which is interesting in itself from a perspective of creativity in design), but more importantly, in strengthening its relation with the structure, via its elaboration. The placement of elements in the space was further justified by a chain of causation, in which both the physical and the purely mental played a part. Such flexibility can be observed in Case 5 as well, in the way that the designers serendipitously structure their design, as a response to an accidental line on the sand – the line is embraced and interpreted as a “boundary” between two areas of full and void, thus serving as a dividing line for the setting, which otherwise remains largely undefined (Figs. 6–8).

Finally, a certain type of conflict seems to play a central role in structuring the metaphoric interpretation world, motivating designers to coordinate their perceptions and interpretations. More specifically, when the expectation of designers from elements conflicted with self-generated rules, the designers’ interpretive faculties were triggered into action. An intriguing example of this is evident in the above sample from Case 1: the subjects were building a “forest”, where each rock was interpreted as “tree”. Furthermore, two kinds of trees were identified: “pine” and “opposite of pine” (deciduous). One subject noticed that a rock characterized as “pine” stood physically far from its group and wondered why this is so (i.e., how could it be that it “grew” out of the “pine area”). To resolve this, two new elements were added to the interpretation: a bird which carried a pine seed from across the forest (Fig. 11).

Fig. 11.

Conflict as a driver of interpretation in design.

The above extract can be understood as a discrepancy between two spaces, as shown in Figure 12. M₁ represents the subject's view of external reality at the initial state, where certain rocks were grouped (those in the “pine tree area”) and others remained outside the group (e₁ and e₇). M₂ and ${\rm M}^{\prime}_2$ represent two possible metaphoric interpretation worlds, and M₃ is the one which results from these. In M₂ the grouped rocks, as well as one rock outside the group (e₇), are all assigned with the interpretation of “pine trees”. This space represents how the designer sees the situation on the interpretive level without considering the implicit rule “all pine tree should stand together”. ${\rm M}^{\prime}_2$, on the other hand, represents the understanding of the situation by considering the implicit rule. Accordingly, in this space, e₇ also appears within the “pine tree” group, reflecting the designer's expectation of following the rule. The conflict stems from the fact that, while the designer expects all pine trees to stand together, he perceives a strong grouping which excludes one element (e₇). M₃ shows the designer's perspective after resolving the conflict – the fact that the implicit rule is broken is coped with creatively on the interpretive level, by forming the bird story, which enabled e₇ to both be out of the group on the perceptual level and in the group on the conceptual level.

Fig. 12.

Visualization of negotiation between different perspectives using our approach; drawing on Fauconnier (Reference Fauconnier1994) and Gero and Kannengiesser (Reference Gero and Kannengiesser2004).