1. INTRODUCTION
Examples built into computer tools, intended to assist designers in forming analogous solutions, may be represented in the form of sketches, line drawings, photographs, computer-aided designs (CAD), functional models, pictures, or text descriptions. Drawing inspiration from examples through the use of design by analogy can be a powerful tool for innovative design (Chan et al., Reference Chan, Fu, Schunn, Cagan, Wood and Kotovsky2011). Design by analogy is a method in which designers apply appropriate and relevant features from existing example solutions to solve design problems (Jansson & Smith, Reference Jansson and Smith1991; Qian & Gero, Reference Qian and Gero1996; Goel, Reference Goel1997). Design by analogy increases creativity and expands the solution space. In engineering design, analogies are often used in conceptual design to aid in generating new and novel design ideas (Benyus, Reference Benyus1997; Vogel, Reference Vogel2000; Leclercq & Heylighen, Reference Leclercq and Heylighen2002; Vincent & Mann, Reference Vincent and Mann2002; Eckert et al., Reference Eckert, Stacey and Earl2005; Vattam et al., Reference Vattam, Helms and Goel2007, Reference Vattam, Helms and Goel2008; Helms et al., Reference Helms, Vattam and Goel2009; Wilson et al., Reference Wilson, Rosen, Nelson and Yen2010), and the use of analogies also aims to enhance creativity (Mak & Shu, Reference Mak and Shu2004; Vattam et al., Reference Vattam, Helms and Goel2009; Shu et al., Reference Shu, Ueda, Chiu and Cheong2011).
Since the emergence of the Internet, examples for engineers to use in design and idea generation have become readily available and accessible within mere seconds. These examples and analogues are usually randomly presented through search engines with little attention paid to clustering these examples by representation (e.g., sketches, photos, line drawings, or documents without images). This is not surprising because little attention has been paid to analyzing externally imposed examples by representation. This experiment will do that as well as contribute to the existing literature by analyzing CAD, sketch, and photo representations. Computer tools for analogy or analogical databases have recently emerged to help designers find relevant analogies or examples and to help them map and transfer the analogies appropriately; this is discussed further in the literature review. However, these tools do not group or filter the images of the analogies by representation. A clearer understanding is needed of how the representations and characteristics of examples used as analogues affect idea generation. The findings of this study will offer recommendations for how these analogical databases should be structured and what types of design features would be beneficial.
Larkin and Simon (Reference Larkin and Simon1987) explore the use of diagrams in problem solving, concluding that effective diagrammatic representations (e.g., CAD, sketches, photographs, and line drawings) hold many advantages over textual representations. They found that diagrams group all useful information together, allowing for further processing and avoiding an arduous search for the elements needed to make a problem-solving inference. This grouping of information also allows the problem solver to avoid having to match and understand symbolic labels that purely textual information may give. Likely due to these disadvantages of textual descriptions, diagrammatic representations are more popular for representing engineering ideas and designs. Like any representation, diagrams do have the potential to cause design fixation (Jansson & Smith, Reference Jansson and Smith1991).
Sketching has been a popular method for early idea conceptualization, but with changes in technology, CAD renderings and photographs are increasing in use. With the advent of computer modeling and drafting packages (i.e., CAD), which are readily available and intuitive, engineering students tend to sketch less (Ullman et al., Reference Ullman, Wood and Craig1990; Grenier, Reference Grenier2008; Schmidt et al., Reference Schmidt, Hernandez and Ruocco2012). Grenier's (Reference Grenier2008) study also showed that students did not choose to sketch during the early stages of conceptualization. This result is also seen in a study by Westmoreland et al. (Reference Westmoreland, Ruocco and Schmidt2011), where visual representations are analyzed for their usage in capstone design. Westmoreland found that students rarely used sketches until specifically prompted to do so. Students are also increasingly reluctant to hand in rough sketches when they can quickly transform them to CAD (Westmoreland et al., Reference Westmoreland, Ruocco and Schmidt2011). Photographs are increasingly popular due to the availability of digital cameras and due to the ability to copy images off the Internet.
These various representations allow engineers to convey information to other designers and appear in the examples that designers use when they are developing new ideas. Studies have shown that designers fixate to examples whether they are in the form of sketches, line drawings, photographs, or physical models (Purcell & Gero, Reference Purcell and Gero1996; Kiriyama & Yamamoto, Reference Kiriyama and Yamamoto1998; Christensen & Schunn, Reference Christensen and Schunn2005; Cardoso et al., Reference Cardoso, Badke-Schaub and Luz2009; Linsey et al., Reference Linsey, Tseng, Fu, Cagan, Wood and Schunn2010; Cardoso & Badke-Schaub, Reference Cardoso and Badke-Schaub2011; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a). Previous studies on design fixation have compared other representations, including line drawings to photographs (Cardoso et al., Reference Cardoso, Badke-Schaub and Luz2009; Cardoso & Badke-Schaub, Reference Cardoso and Badke-Schaub2011), sketches to physical models (Youmans, Reference Youmans2011; Viswanathan & Linsey, Reference Viswanathan and Linsey2013b, Reference Viswanathan and Linsey2013c), and sketches to textual representations (McKoy et al., Reference McKoy, Vargas-Hernández, Summers and Shah2001). All of these studies have presented intentionally ineffective examples where design fixation hurts the process.
This paper presents an experiment that investigates how various representations of externally imposed examples (analogues) affect the idea generation process. Specifically, we measure how these various representations influence design fixation and design quality. In this experiment, we compare common representations that existing solutions may occur in. We examine how these various representations impact design fixation, that is, how designers copy features of the example based on the way it is represented. In the following sections, we review the related prior work on design fixation and the impact of various representations or visual stimuli on design.
2. BACKGROUND
A number of different analogue representation options are available to designers, but little is known about how they influence design fixation and creativity. In this section, we discuss the differences among various representations used in idea generation and presented as analogues.
2.1. Representation in idea generation
During the idea generation process, examples (analogues) of existing solutions are commonly used to provide inspiration to designers. These representations could be in the form of sketches, line drawings, CAD, photographs, or even verbal and textual representations. Each conveys different types of information. We will borrow the definitions used by Westmoreland et al. (Reference Westmoreland, Ruocco and Schmidt2011) to describe these visual representations:
Sketch: A sketch is a drawing that is done without concern for detail in order to capture a general idea. A sketch is made without the use of any instruments such as a straight edge.
Line drawing: A line drawing is a picture made of lines created by hand with assistive instruments or by computer.
CAD: A visual image created with a formal computer-aided drawing package (e.g., Pro/ENGINEER, Solid Works, and AutoCAD).
Photograph: A photograph is an image that is produced with the use of a camera. The image is an exact replica of what the human eye would perceive at an instant in time.
We note that these representations might be used in two modes during idea generation. The first mode of representation is external, where examples are shown to design engineers as stimuli for inspiration in the design task. The second mode of representation is how the designers represent their ideas (i.e., self-generated representations). In this study, we are only concerned with varying external representations to see how they influence design fixation; we keep the self-generated mode constant (as sketches).
Sketching is a popular method for developing and representing ideas. Various studies have been done on the role of sketching in design (Yang & Cham, Reference Yang and Cham2007; Yang, Reference Yang2009; Macomber & Yang, Reference Macomber and Yang2011) and state that sketching during idea generation improves the overall quality and realism of the design. Even though there has been a decline in the use of sketching among engineering students, it does offer advantages over other representations (Westmoreland et al., Reference Westmoreland, Ruocco and Schmidt2011). A critical part of generating concepts, sketching promotes creative thought (Goldschmidt, Reference Goldschmidt1994; Goel, Reference Goel1995). Sketching is advantageous because it is economical, simple, and easy to revise (Jonson, Reference Jonson2002). It also allows the designer to obtain immediate visual and kinesthetic feedback (Contero et al., Reference Contero, Varley, Aleixos and Naya2009).
One advantage of a sketch is its inherent ambiguity (Goel, Reference Goel1995; Stacey et al., Reference Stacey, Eckert and McFadzean1999; Jonson, Reference Jonson2002; Contero et al., Reference Contero, Varley, Aleixos and Naya2009). Sketches lack regularity and contain a certain type of looseness or “sketchiness,” which makes them prone to having different interpretations. Rather than inducing uncertainty or confusion, ambiguity in design sketches can be a source of creativity as it allows for the reperceiving and reinterpreting of figures or images (Tversky et al., Reference Tversky, Suwa, Agrawala, Heiser, Stolte, Hanrahan, Phan, Klingner, Daniel and Lee2003) or for alternative interpretations by another team member (Shah, Reference Shah1998). Tversky et al. (Reference Tversky, Suwa, Agrawala, Heiser, Stolte, Hanrahan, Phan, Klingner, Daniel and Lee2003) explain that sketches hold the created constructions in view of the designer, freeing the mind to examine and evaluate.
In contrast to a sketch's potential for ambiguity, photographic and CAD representations possess richer representation. Photographs usually contain colors and visual depth. The same can be said for CAD representations, which in addition have a cleaner, more defined look. Because CAD and photographic representations are by nature more exact representations, the idea they are trying to convey is less subjective to a group of observers (Veisz et al., Reference Veisz, Joshi and Summers2012); that is, as the fidelity of the representation increases, the ambiguity decreases. CAD representations can be advantageous over a photograph because CAD models can contain more dimensional information, show hidden lines, and display hidden components. However, there is research that states that CAD tools, when used to create designs, have the potential to negatively impact the design process (Robertson et al., Reference Robertson, Walther and Radcliffe2007; Veisz et al., Reference Veisz, Joshi and Summers2012). Robertson, in multiple studies (Robertson et al., Reference Robertson, Walther and Radcliffe2007; Robertson & Radcliffe, Reference Robertson and Radcliffe2009) comprising an observational case study of a small engineering team and an extensive survey of 255 CAD users, found that CAD tools may limit the designer through interfering with the designer's intent. The CAD program constrains the thinking and problem solving of the designer (Robertson et al., Reference Robertson, Walther and Radcliffe2007; Robertson & Radcliffe, Reference Robertson and Radcliffe2009). In addition, these studies found that CAD tools might cause premature fixation when the designer resists changing complex or highly detailed models. Robertson et al. (Reference Robertson and Radcliffe2009) also warn that the overuse of CAD tools may decrease motivation and creative abilities. Another disadvantage that CAD may have, compared to sketching, is that digital design is still currently slower than sketching (Thilmany, Reference Thilmany2006).
Studies have shown that the amount and type of information that designers access when interpreting different types of representations varies (Suwa & Tversky, Reference Suwa and Tversky1997; Casakin & Goldschmidt, Reference Casakin and Goldschmidt1999; Kokotovich & Purcell, Reference Kokotovich and Purcell2000; Kavakli & Gero, Reference Kavakli and Gero2001, Reference Kavakli and Gero2002; Menezes & Lawson, Reference Menezes and Lawson2006). A few studies have also examined the impact of design representations on customers. A survey shows that architects preferred to show initial designs to clients using sketches and final versions in CAD (Schumann et al., Reference Schumann, Strothotte, Laser and Raab1996). Sketches encourage discourse about a design, while CAD tends to imply that the image can no longer be altered. Macomber and Yang (Reference Macomber and Yang2011) also found that customers preferred hand drawings with the highest level of finish to the CAD drawings. The complexity and familiarity of an object also influenced perceptions. This study did not capture the usefulness of these various representations but rather merely a visual preference. It is entirely possible that the preference and usefulness of various representations do not necessarily correlate. This study will measure the differences in engineers' behavior when they use various representations of examples to design.
The studies discussed so far have explored how a designer's creativity is enhanced or limited by the representations used during idea generation, how different external representations influence design, and how different representations provide information to the viewer. In this paper, we specifically focus on the former and its influence on design fixation, that is, how external representations affect ideation and creativity.
2.2. Design fixation
Design fixation refers to the adherence of designers to example features and to their own initial ideas, which is sometimes counterproductive (Jansson & Smith, Reference Jansson and Smith1991). The use of any example tends to make designers sensitive to the features of the example because they act as external stimuli. This is especially true for the visual representations such as CAD, photos, and sketches (Goldschmidt & Smolkov, Reference Goldschmidt and Smolkov2006). While the use of these visual examples is intended to provide inspiration to the designers, these examples tend to fixate them to the features of the example and tend to hinder their creativity. There have been numerous studies in engineering design and psychology that have dealt with the topic of fixation (Jansson & Smith, Reference Jansson and Smith1991; Purcell & Gero, Reference Purcell and Gero1996; Wiley, Reference Wiley1998; Christensen & Schunn, Reference Christensen and Schunn2005; Linsey et al., Reference Linsey, Tseng, Fu, Cagan, Wood and Schunn2010; Viswanathan & Linsey, Reference Viswanathan and Linsey2013c), all of which use various examples to induce fixation.
According to Perttula and Liikkanen (Perttula & Likkanen, Reference Perttula, Liikkanen, Jonsson and Unnporson2006; Likkanen & Perttula, Reference Liikkanen and Perttula2010), example exposure may not be necessarily detrimental, and the literature on design by analogy clearly demonstrates the potential to enhance creativity. The benefits of examples or external stimuli have been investigated under the topic of cognitive stimulation, where design and psychology researchers have shown that idea exposure can positively influence one's ability to produce ideas (Brown et al., Reference Brown, Tumeo, Larey and Paulus1998; Coskun et al., Reference Coskun, Paulus, Brown and Sherwood2000; Dugosh & Paulus, Reference Dugosh and Paulus2005; Perttula & Sipilä, Reference Perttula and Sipilä2007). These studies tend to use examples that are very close in domain and are close domain analogies in most cases. Though these studies were not strictly measuring fixation, examples do offer benefits to designers, such as aiding in the convergence of ideas in teams (Fu et al., Reference Fu, Cagan and Kotovsky2010) and helping designers to determine whether existing ideas meet design requirements (Hannah et al., Reference Hannah, Joshi and Summers2012). Purcell and Gero (Reference Purcell and Gero1996) state that the form or representation used in examples, for example, sketch or CAD, appears to establish the conditions for fixation to occur. Thus, exploring the use of various representations in idea generation is very beneficial toward better understanding the dynamics of fixation in design.
Design fixation has also been identified during analogical design. Helms et al. (Reference Helms, Vattam and Goel2009) identified design fixation to biological sources during an experiment about biologically inspired design. In their study, Helms et al. found that in problem-driven design processes, the biological solution used as the analogue becomes a source of design fixation, limiting the source of inspiration to that one analogue.
In design fixation experiments, ineffective examples are typically used to induce fixation and to investigate trends across various parameters. Studies have shown that ineffective examples produce a higher amount of fixation compared with good ones (Fu et al., Reference Fu, Cagan and Kotovsky2010), and common solutions also increase fixation as compared to uncommon solutions (Perttula & Sipilä, Reference Perttula and Sipilä2007). Fixation studies with effective or good examples are usually designed to measure fixation as well as additional trends. For instance, Fu et al. (Reference Fu, Cagan and Kotovsky2010) measured how team convergence is influenced by good and ineffective examples as well as how teams fixate to the examples given. Fixation studies with ineffective examples (Linsey et al., Reference Linsey, Tseng, Fu, Cagan, Wood and Schunn2010; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a, Reference Viswanathan and Linsey2013b) have been used to solely measure fixation to features of the examples, such as designers blindly copying the ineffective features of an example.
Most of the previous research studies on design fixation have used examples that were represented in only one form, predominantly sketches (Jansson & Smith, Reference Jansson and Smith1991; Purcell & Gero, Reference Purcell and Gero1992; Fu et al., Reference Fu, Cagan and Kotovsky2010). Little research has been done in comparing various types of representations to see how they influence fixation.
2.3. Representations and fixation
Only a few studies have specifically explored the influence of representations on design fixation. For example, Cardoso and Badke-Schaub (Reference Cardoso and Badke-Schaub2011) compared a line drawing to a photo measuring differences in quantity, quality, and originality (using a “yes/no” criterion for originality). They found that both line drawings and photographs caused design fixation. There were no significant differences between the line drawing and the photo for quantity, quality, or originality. An experiment by McKoy et al. (Reference McKoy, Vargas-Hernández, Summers and Shah2001), which used teams in an undergraduate course, compared examples represented either as a sketch or as text. They showed that groups who received a sketch had higher novelty and quality scores than the text description example (only quality and novelty were measured). In the digital age, it is important to investigate other types of representations that can be used for the idea generation process. Recent studies have shown that CAD has emerged as an idea generation tool (Jonson, Reference Jonson2005). However, the usefulness of CAD representations in reducing fixation has not been critically studied. This study will do so.
3. METHOD
This paper presents an experiment designed to assess if, and to what extent, fixation occurs in engineering idea generation based on the representation of the example given. We compare CAD, photo, and sketch representations. A good design example was presented, but it was represented in different ways, that is, as a CAD, a photo, and a sketch. A control condition where participants did not receive an example was also included.
3.1. Hypotheses
Hypothesis 1: Fixation. Based on prior literature that states that the ambiguity of sketches helps to promote ideation (Goel, Reference Goel1995; Jonson, Reference Jonson2002; Tversky et al., Reference Tversky, Suwa, Agrawala, Heiser, Stolte, Hanrahan, Phan, Klingner, Daniel and Lee2003; Contero et al., Reference Contero, Varley, Aleixos and Naya2009), we hypothesize that more well-defined/high-fidelity representations (e.g., CAD or photo) will cause designers to fixate more; thus, fixation can be reduced with less well-defined examples (e.g., a sketch).
Hypothesis 2: Identification of working principles of the design. From the study by Hannah et al. (Reference Hannah, Joshi and Summers2012), which found that designers were better able to determine if high-fidelity representations met design or customer requirements compared with low-fidelity representations, we hypothesize that CAD and photo representations will allow designers to be able to better identify the key or working principles of the design. We expect that they will copy these features more in their design concepts.
Hypothesis 3: Feasibility. In line with the second hypothesis, we also hypothesize that the CAD and photo representations will produce more feasible design solutions compared with the sketch condition.
Hypothesis 4: Working principles' effect on novelty and variety. We expect the copying of effective working principles to result in lower novelty and variety scores for CAD and photo representations because the effective working principles will occur frequently in the solutions due to fixation.
3.2. Design task
The design task given to the students was to develop a device to shell peanuts in developing countries. This task has been used in previous studies (Linsey et al., Reference Linsey, Green, Murphy, Wood and Markman2005; Fu et al., Reference Fu, Cagan and Kotovsky2010; Linsey et al., Reference Linsey, Tseng, Fu, Cagan, Wood and Schunn2010, Reference Linsey, Markman and Wood2012; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a), and follows the same approach, which includes a description of the example design, time given to read the problem, and time given to generate ideas. This problem was chosen because it is practical, appropriate for engineers, and able to be solved in diverse ways.
3.3. Experimental conditions
The participants were randomly assigned into four experimental conditions. Each condition received a different representation of the same existing solution for a peanut-shelling device. The peanut sheller example given is the Universal Nut Sheller, designed by inventor and humanitarian Jock Brandis (Connors, Reference Connors2008; Brandis, Reference Brandis2012). This peanut sheller is considered good because it is easy to manufacture, low cost, sustainably powered (human energy), efficient, and effective. It is currently in use, and the design satisfies all of the customer needs.
The four conditions in this experiment are based on the types of representations given: CAD, photo, sketch, and no representation. We designed the experiment so that all conditions would contain basically the same amount of information, but represented in various ways. To do this, all the conditions needed to have a view of the inner workings of the peanut sheller. This was easy to produce via sketch or CAD modeling, but we did not have an inner view of the photo of the sheller. In order to provide a very similar amount of information to the experiment participants, we added a high-fidelity wireframe view, the same given to the CAD, to the photo condition.
The experiment conditions and the representations they received are the following:
• CAD: the example was represented as a CAD model (Fig. 1a).
• Photo: same example represented as a photograph (with a CAD wireframe; Fig. 1b).
• Sketch: same example as the CAD and photo, but represented as a sketch (Fig. 1c).
• Control: no example was given; this condition is used to measure design fixation.
Examples provided to each condition: (a) computer-aided design, (b) photo plus computer-aided design (Nourish-International, 2007; Gazette, Reference Gazette2009), and (c) sketch.
A description for the example solution was also provided on the same sheet of paper as the problem description, customer needs, and example representation. The solution description read
The peanuts are loaded from the top of the system; the user rotates the handle, which pushes the nuts toward the tapered gap between the interior and exterior wall of the machine. The shell of the nut is broken at the point where the gap is sufficiently narrow to cause enough friction to crack open the shells. The kernels and shell fragments fall into a basket and are later separated by winnowing.
3.4. Participants
The participants were 80 senior undergraduate students in mechanical engineering at Texas A&M University with 20 per condition in each of the four conditions.
3.5. Experimental procedure
The experiment occurred in a controlled classroom setting. Half of the participants (in all four conditions) were run in the spring semester and the other half in the fall. Because all students were in the same design class, they had learned the same material. Participation was voluntary, and the students were compensated with either extra credit or a monetary award.
The design task and example were handed to each student on paper. They were then given 5 min to review and understand the design task. During this time, they were encouraged to ask questions concerning the experiment or design task; no questions were asked. After the initial review period, the participants were given 45 min to complete the idea generation section, and all were required to use the entire 45 min. They were asked to sketch each of their design solutions one idea per page and to describe how the design worked by adding short text descriptions and by labeling parts of the design. They were also asked to generate as many solutions as possible. As an incentive to create many solutions, they were told that the person who generated the greatest number of solutions would receive a prize or bonus. This bonus was given to all the participants in the form of a monetary award at the end of the experiment in addition to the compensation for experiment participation.
3.6. Evaluation metrics
To measure fixation, creativity, and the overall effectiveness of the solutions generated, six metrics were used: quantity of nonredundant ideas, number of repeated example features, percentage of example features used, quality of concepts, novelty of concepts, and variety of concepts. Four of these metrics (the quantity [nonredundant], quality, novelty, and variety of ideas) are based on definitions proposed by Shah et al. (Reference Shah, Kulkarni and Vargas-Hernandez2000) and further developed by Linsey et al. (Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011). For the purpose of this study, an idea is defined as a feature of the generated solutions that solves at least one function in the functional basis (Linsey et al., Reference Linsey, Tseng, Fu, Cagan, Wood and Schunn2010). A design concept refers to each solution that a participant generates. Each concept was broken down into ideas and scored using these metrics. To ensure the reliability of the metrics, an interrater agreement was performed by two independent raters, and interrater reliability using either Pearson or Cohen κ correlation was determined.
Table 1 was used as a guide by the raters to determine the features copied from the example. If participants copied a feature from the example they were given, it was counted as a “repeated feature.” The functional features were determined in the same manner as they are for the quantity of ideas. The functional basis approach to functional modeling was used (Hirtz et al., Reference Hirtz, Stone and McAdams2002). The functional basis has been empirically demonstrated by two independent universities to be effective (Ahmed & Wallace, Reference Ahmed and Wallace2003; Sen et al., Reference Sen, Caldwell, Summers and Mocko2010). The interrater agreement was done on 50% of the data for all metrics. Studies have shown that independent experts with domain knowledge can reliably assess quantity and quality of ideas (Linsey et al., Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011), as well as the creativity (novelty and variety) in engineering design (Christiaans, Reference Christiaans1992, Reference Christiaans2002; Linsey et al., Reference Linsey, Green, Murphy, Wood and Markman2005, Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011; Kudrowitz & Wallace, Reference Kudrowitz and Wallace2013; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a, Reference Viswanathan and Linsey2013c). A detailed description of the metrics used and evaluation performed is given below.
Quantity of nonredundant ideas: This measure of fixation gauges how a participant's ideas are limited due to exposure to an example. It measures the quantity of ideas generated minus ideas taken from the example and any repeated ideas. A control condition is used as a baseline to measure fixation. If the conditions with examples produce fewer ideas than the control, then fixation is occurring. A Pearson correlation of 0.83 between the scores of the two raters was obtained, which shows the measure is reliable.
Number of repeated example features: This metric is also a measure of fixation that assesses how often the participants copy or fixate to ideas or features of the example given. The control condition also acts as a baseline for measuring fixation. If the participants in the conditions with examples have solutions with more ideas from those examples than the control group, then fixation to the example is occurring. The Pearson correlation for this metric is 0.80, which shows the measure is reliable.
Percentage of example features used: This metric also measures fixation, but to the features of the example given. It measures how many of the features of the example (out of all the available features) are used in the design solutions. The Pearson correlation for this metric is also 0.80, the same as the number of repeated example features metric.
Quality (feasibility) of ideas: Quality is measured based on the feasibility of the design concept and how well it meets design specifications or customer needs (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003). A 3-point rating scale developed by Linsey et al. (Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011) is used to measure the quality of design concepts generated. A score of zero is given for designs that are not technically feasible and do not meet any of the customer needs. A score of one is given if the design partially meets the customer needs (one to three customer needs). A score of two is given for designs that meet most or all of the customer needs (four to five customer needs). A Cohen κ value of 0.57 was obtained. This value is an acceptable level of agreement (Clark-Carter, Reference Clark-Carter1997).
Novelty: Novelty measures how unusual or unexpected a concept is (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003; Nelson et al., Reference Nelson, Wilson, Rosen and Yen2009). Each idea is sorted into bins, and the novelty is calculated as one minus the frequency of ideas in a bin (Linsey et al., Reference Linsey, Green, Murphy, Wood and Markman2005; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a). See Linsey et al. (Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011) for more details on the blind sorting procedure for the novelty (and variety) scores. The formula used is given by Eq. (1). The Pearson correlation is 0.95.
(1)
$$\eqalign{\hbox{novelty} & = 1 - \hbox{frequency of ideas} \cr & = 1 - \displaystyle{\hbox{number of ideas in a bin} \over \hbox{total number of ideas per participant}}}. $$Variety: Variety measures the solution space explored during the idea generation process (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003; Nelson et al., Reference Nelson, Wilson, Rosen and Yen2009). The variety is calculated as the number of bins a participant's ideas occupy divided by the total number of bins (Linsey et al., Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a). The formula is given by Eq. (2). The Pearson correlation is 0.95.
(2)$$\hbox{variety} = \displaystyle{\hbox{number of bins a participant's ideas occupy} \over \hbox{total number of bins}}. $$
Functions and features contained in the example solution (Full Belly peanut sheller)
3.7. Results
3.7.1. Quantity of nonredundant ideas
The results from the quantity of nonredundant ideas generated shows that fixation to their own ideas is present due to the example analogue (Fig. 2). The three example conditions (CAD, photo, and sketch) generated fewer ideas than the control, indicating fixation is present. The results were analyzed using one-way analysis of variance (ANOVA), where F (3, 79) = 7.39, p < 0.001, and MSerror = 0.05. Figure 2 and the pairwise t tests among the CAD, photo, and sketch conditions show that the number of ideas generated by these three conditions are not statistically significant when compared to each other. These results show that the representations evaluated in this experiment do not significantly influence the degree of design fixation, and all representations evaluated in this experiment cause fixation to about the same extent. This shows that Hypothesis 1 is not supported.
The mean quantity of nonredundant ideas across conditions. All error bars show (±1) standard error.
These results are consistent with those found by Cardoso and Badke-Schaub (Reference Cardoso and Badke-Schaub2011), who showed that there were no significant differences when comparing the quantity of ideas of only the photo and line drawing conditions. However, there were significant differences when comparing both conditions to the control condition, indicating that fixation is occurring.
3.7.2. Number of repeated example features and percentage of example features used
Figure 3 shows the distribution of the mean number of repeated example features, and the mean percentage of example features used follows a similar pattern. The ANOVA results for the number of repeated features and percentage of examples features used are F (3, 79) = 2.52, p = 0.065, and MSerror = 44.33 and F (3, 79) = 3.69, p = 0.015, and MSerror = 0.037, respectively. Figure 3 shows that fixation is again present. The CAD, photo, and sketch conditions are copying more features from the example than the control condition. Even though the control group has not seen the example, features from the example will appear in their designs. The repetition of example features in the designs supports the quantity results and shows that the type of representation used does not influence fixation to the example features, but that fixation is present.
The mean number of repeated example features across conditions.
3.7.3. Quality (feasibility) of design concepts
The results from the quality of concepts metric (Fig. 4) show that the CAD and photo condition produced significantly higher quality ideas compared to the control and sketch conditions. Figure 5 shows an example of a high-quality solution from a CAD participant. The t test pairwise comparisons for CAD to control and sketch conditions are p = 0.035 and 0.018, respectively; the t test pairwise comparisons for photo to control and sketch conditions are p = 0.05 and 0.039, respectively [ANOVA, F (3, 79) = 3.25, p = 0.021; and MSerror = 0.089]. The CAD and photo quality scores were not significantly different from each other. The control and sketch conditions were also not significantly different from each other. These results support our hypothesis that the CAD and photo conditions would produce a higher quality compared to the sketch.
The mean quality of design concepts across conditions.
A typical example of one computer-aided design participant's solution. The text associated with the concept was typed for clarity.
3.7.4. Novelty and variety
The results for the mean novelty and variety metrics show that there are no statistically significant differences [ANOVA novelty F (3, 79) = 0.72, p = 0.55, and MSerror = 0.037; variety F (3, 79) = 1.56, p = 0.20, and MSerror = 0.038]. In addition, the most novel ideas were also evaluated. Each participant's maximum novelty score was taken and analyzed. The data for the maximum mean novelty also shows no statistical differences [ANOVA F (3, 79) = 1.56, p = 0.20, and MSerror = 0.038]. Prior studies (Linsey et al., Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011; Viswanathan & Linsey, Reference Viswanathan and Linsey2013a, Reference Viswanathan and Linsey2013c) have also not seen differences in novelty and variety in idea generation studies. It is possible that the novelty and variety metrics are not sensitive enough to detect differences. Cardoso and Badke-Schaub (Reference Cardoso and Badke-Schaub2011) also saw no differences in originality for line drawing compared to photos.
3.7.5. Effective principles copied from the example
Because this experiment uses an effective example, we hypothesized that the various representations would offer different benefits regarding the participants' abilities to identify the working or effective principles of the design. As discussed earlier, being able to identify and copy these key features is not necessarily a negative consequence of fixation. A given principle may be accomplished by more than one set of features. It is possible to have the same working principle being implemented, but with a different set of features. For the Full Belly peanut sheller, we identified the principles and features of the design from Table 1 that made the design effective: the double taper, taper, rotation, friction, and sufficient gap. In case of this data, all instances of implementing the effective principles also involved the use of the same features.
Figure 6 shows the mean percentage of all of the five effective principles copied from the example for four conditions [ANOVA F (3, 15) = 1.79; p = 0.05, and MSerror = 0.21]. We see that participants in the CAD and photo conditions copied significantly more of the effective principles from the example compared to those in the control and sketch conditions. The t test pairwise comparison for CAD to control and sketch conditions are p = 0.02 and 0.04, respectively, and the t test pairwise comparison for photo to control and sketch conditions are p = 0.04 and 0.05, respectively. There are no significant differences between the control and sketch conditions or between the CAD and photo conditions. These results show that participants in CAD and photo conditions were able to better identify the effective principles of the given examples based on their representations; these results support Hypothesis 3. Figure 7 shows the breakdown of each of the principles that were copied; the graph also shows that participants in the CAD and photo conditions copied more of each of the principles than those in the other conditions.
The mean percentage of effective principles copied from the example.
The percentage of each of the copied principles.
4. DISCUSSION
The data from the three measures of design fixation (quantity of ideas, number of repeated features, and percentage of repeated features) show consistent results that all three representations (CAD, photo, and sketch) do result in design fixation to the example analogue, but the degree of fixation is not significantly different across the three representations evaluated in this experiment. The hypothesis stating that more well-defined or high-fidelity representations cause a higher degree of fixation is not supported. These results are consistent with the Cardoso and Badke-Schaub (Reference Cardoso and Badke-Schaub2011) study. Combining these results with results from the studies by Robertson (Robertson et al., Reference Robertson, Walther and Radcliffe2007; Robertson & Radcliffe, Reference Robertson and Radcliffe2009) indicates that the limits inflicted by CAD may be due to the use of CAD as opposed to being inherent in a more exact representation. The premature fixation and limits to creativity with CAD tools may be the same effect seen with prototyping, the sunk-cost effect. The sunk-cost effect occurs when an individual tends not to change paths due to the time, money, or effort already entered in a particular course of action even when it would be more logical to go in a different direction (Kahneman & Tversky, Reference Kahneman and Tversky1979; Arkes & Blumer, Reference Arkes and Blumer1985). The sunk-cost effect has been shown to be the reason for the apparent fixation in prototyping (Viswanathan & Linsey, Reference Viswanathan and Linsey2013c, Reference Viswanathan and Linsey2014).
This study intentionally kept the information across the representations as similar as possible to measure the influences inherent in the representations. This work does not necessarily contradict previous research that indicates that sketches, likely due to their greater capability for ambiguous representation, may provide more opportunities for creativity and reinterpretation (Suwa & Tversky, Reference Suwa and Tversky1997; Shah, Reference Shah1998; Tversky et al., Reference Tversky, Suwa, Agrawala, Heiser, Stolte, Hanrahan, Phan, Klingner, Daniel and Lee2003). It also does not contradict the studies by Robertson (Robertson et al., Reference Robertson, Walther and Radcliffe2007; Robertson & Radcliffe, Reference Robertson and Radcliffe2009) on the limits due to using CAD. It is entirely possible that designers should use sketches in the early phases of design because they have more potential for ambiguity and that CAD systems require too much time and effort to be useful early in the process. This warrants further investigation.
The results from the quality of design concepts metric provide interesting results. Here, the CAD and photo conditions were initially shown to have produced a statistically significant higher quality of design concepts compared to the control and sketch conditions. The results of the percentage of effective principles copied from the example also produced similar results; that is, participants in the CAD and photo conditions copied significantly more of the effective principles than did those in the control and sketch conditions. Though the quality and percentage of effective principles copied from the example for the CAD condition were higher compared to the photo condition, they were not significantly different. This data shows that high-fidelity representations such as CAD and photographs allow for a clearer depiction of the working principles of the example. This in turn leads to higher quality ideas as designers copy these features. Prior work by Cardoso and Badke-Schaub (Reference Cardoso and Badke-Schaub2011) used a negative example, and they did not see any differences in quality. In addition, they only compared line drawings and photos, not sketches.
Even though fixation still occurs with a good example, regardless of the representation, CAD and photo conditions (high-fidelity representations) allow good features to be reused more frequently. This experiment suggests CAD and photo representations are preferable over sketches in the early design stages design when idea generation is taking place if designers are attempting to build from and recombine effective examples (analogues). The results from this experiment also indicate that analogical databases of effective design examples should include CAD and photolike images of the design, and also offer a way to filter these examples by representation.
5. CONCLUSIONS
The experiment performed in this study investigated how fixation is affected by the representations of an example analogue given during an idea generation task. CAD, photo, and sketch representations were explored. This study evaluated the effects when the example given was good and effective, which is in contrast to prior studies that used an ineffective example where design fixation was not desirable. Computer-based tools for analogy would most likely contain effective examples. This experiment has confirmed the presence of design fixation for all three representations used in this study. The results from this study show that the type of representation used, when the information contained is similar, does not significantly affect the degree of fixation for the examples chosen in this study. Based on prior literature, the various representations explored do tend to produce different effects. Simply presenting designers with a sketched representation does not guarantee a reduction in fixation. An intriguing finding from this study is that high-fidelity representations (i.e., CAD or photo) allow for the effective principles of the example to be identified and copied. These two representations also produced higher quality scores compared to the control and sketch conditions, likely a result of the fixation to effective design features.
The results from this experiment have provided greater insight into the dynamics of fixation, specifically on how various representations should be used during conceptual design including analogical design. Further studies to investigate other types of representations presented alone and in combination are already in progress. We especially want to investigate if there is any bias toward a certain type of example based on the way that it is represented.
ACKNOWLEDGMENTS
Partial support for this work was provided by National Science Foundation Awards CMMI-100095 and CMMI-1322335. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Olufunmilola Atilola received her PhD from the Department of Mechanical Engineering at the Georgia Institute of Technology. She attained her BS from the Georgia Institute of Technology, and her MS from the University of South Carolina, both in mechanical engineering. Dr. Atilola's current research is on exploring how different design representations affect engineering idea generation and creativity. Her other research interests include product development and cognitive design methods.
Julie S. Linsey is an Assistant Professor in the George W. Woodruff School of Mechanical Engineering at the Georgia Institute of Technology. She received her PhD in mechanical engineering at the University of Texas. Her research area is design cognition including systematic methods and tools for innovative design with a paricular focus on concept generation and design by analogy. Her research seeks to understand designers' cognitive processes with the goal of creating better tools and approaches to enhance engineering design. Dr. Linsey has authored over 100 technical publications including more than 25 journal papers and 6 book chapters. She also holds two patents.
