2. MOTIVATING CASE STUDIES: IMPACT OF OBSERVED PROBLEM ON CURRENT PRACTICE

To demonstrate the effect of our observed problem on industry practice, we take two industry case studies: first, the case of the California High Speed Rail Authority's planning process for the South Bay Area, and second, the Stanford Green Dorm Feasibility Study.

As part of this proposed route of the California high-speed rail, the High Speed Rail Authority required a station to be placed either in Palo Alto or Redwood City. In September 2008, Palo Alto, in a parallel study to the High Speed Rail Authority process, investigated undergrounding the existing tracks in hopes of selling “air rights” above the existing right of way to offset the increased cost of tunneling (Dong, Reference Dong2008). Our analysis of this conceptual design process illustrates shortcomings in current practice.

Palo Alto narrowed the solution space one decision at a time, considering different and nonexplicit sets of project objectives at different points in the process. Faced with two decisions (the choice of above or below ground trains in conjunction with either high or low-density development along the right of way), the city had essentially four alternatives to consider. Instead of evaluating each with respect to an agreed upon set of project objectives, the city designers chose the underground option for reasons of noise and aesthetics, and subsequently, based on that imposed constraint, chose high-density development for reasons of cost (extra retail space would help pay for the tunnel). In reality, both decisions had consequences for both of these sets of objectives. For example, it is likely that the amount of development needed to offset the cost of tunneling would need to be of greater density and building height than existing Palo Alto zoning ordinances created for aesthetic and noise reasons similar to those that prompted the initial decision to underground the tracks in the first place. Without a systematic method for exploring this solution space, Palo Alto overconstrained the trade space with a decision lacking explicit rationale and potentially resulting in rework (in the form of many community and city council meetings) later in the design process.

The second case study was launched in November 2003, when the Department of Civil and Environmental Engineering at Stanford University endeavored to build an “evolving,” “influential,” “flexible,” and “desirable” environmentally sustainable facility for student housing and for faculty research (Stanford Green Dorm, 2006). The group's admirable initial description of project objectives should be noted, especially as they will relate to subsequent observations about the actual design process. As part of feasibility study (EHDD Architects, 2006) from August 2005 to March 2006, Green Dorm designers needed to select a combination of options for the project and then demonstrate that it could be built to meet project requirements. It should be noted that the feasibility study team was not explicitly tasked with finding a “best” alternative. However, this is not the explicit task conceptual design, nor will it be the goal of our proposed framework, which is meant to encourage the explicit exploration of a large number of options and objectives. Thus, it is relevant that without an explicit conceptual design framework, the Green Dorm Team neglected to consider many potential combinations of options and their effect on many objectives.

The project team generated two potential alternatives, called “baseline green” and “living lab.” Although these alternatives were innovative given industry norms, the exploration of fortuitous combinations was not. Given the documented information and the names of the two alternatives, it seems as though there were only three factors driving the creation of the trade space (novelty of technologies, environmental responsibility, and cost) when the design team had explicitly noted several more objectives in their initial meeting, as discussed above. The baseline green alternative minimized cost at the expense of novelty and maintaining a minimum of environmental responsibility, whereas the living lab alternative maximized novelty and environmental responsibility at the expense of costs. Furthermore, by taking an “all or none” approach on this axis, designers neglected to consider possible combinations that could satisfy a broader range of project value (expressed on other, implicit axes).

The analysis performed by students in a sustainable design class at Stanford formally calculated the value of some possible different alternatives. Using the same method that was used to evaluate the “living lab” against the “baseline green” alternative by Haymaker and Chachere (Reference Haymaker, Chachere and Smith2006) and by also considering the interactions between options, the students found a “solar electric” alternative with potentially higher overall project value (Corcoran et al., Reference Corcoran, Walton, Ehrich and Stoutenburg2008). By eliminating certain technologies with high costs and overlapping energy production functions, this alternative maximized trade-offs between research potential, cost, and environmental benefit, thereby achieving greater value. Although this work lacks the necessary precision and rigor to draw complete conclusions, it seems a more systematic methodology for assembling options may provide a path to finding more innovative and integrated alternatives.

3. THEORETICAL FRAMEWORK: REQUIREMENTS AND METRICS FOR CONCEPTUAL DESIGN

This section defines a theoretical framework for solving the observed problem. We aim to create a method to help design teams explicitly and flexibly document and explore the interactions among the largest number of options with respect to the large number of objectives. Improving these design processes, however, requires understanding the types of information such a process will need to manage and clearly defining metrics for measuring improvement.

3.2. Metrics for evaluation of design space exploration

Building on a theoretical review of design space metrics described in Clevenger and Haymaker (Reference Clevenger and Haymaker2010), we use five metrics described below and summarized in Table 1 to help understand the extent to which a particular design process achieves these aims.

Table 1.

List of metric

Note: G is the number of goals considered, G _ideal is the number of goals in the “ideal” set, d is the alternatives considered, C is the number of alternatives possible, I _analyzed is the interactions explicitly analyzed, I _total is the number of interactions possible, IG _analyzed is the number of goals affected in I _analyzed, and T is the time.

Objective space quality (OSQ) measures how well a design process explored a broad and well-defined set of project objectives. This concept encompasses two measures: the number of objectives involved and the rationality of those objectives, as covered in Edvardsson and Hansson (Reference Edvardsson and Hansson2005). Because our specific work does not focus on the generation, and in turn the quality or rationality, of project objectives, we will focus only on the number of objectives considered. OSQ is calculated as a percentage of objectives considered compared to an ideal set of objectives; this method gives more information than simply counting the number of objectives included in analysis, although both convey generally the same idea. To specify this “ideal” set, we defer to previously discussed methods of Green (Reference Green1994) and Haymaker and Chachere (Reference Haymaker, Chachere and Smith2006), as these methods do the most to engage a defined set of stakeholders in search of a broadest possible conception of project objectives.

Design space sampling (DSS) is the fraction of alternatives considered divided by the total number of alternatives possible. Although exploring a greater percentage of the design space does not explicitly guarantee finding better alternatives, past research shows that this is true in many cases (Weber & Condoor, Reference Weber and Condoor1998; Akin, Reference Akin, Eastman, Newstetter and McCracken2001; Sutton, Reference Sutton2002; Ïpek et al., Reference Ïpek, McKee, Caruana, de Supinski and Schulz2006).

Interaction quotient (IQ) is the percentage of interactions noted by designers compared to the total number of interactions possible, both for options in different topics and for those in the same topic (total therefore defined as the product of the number of options in each topic added to the sum of number of 2-combinations for the options in both topics). Even though we seek a methodology that gives design teams the ability to note every interaction, they are certainly not required to do so, and thus IQ provides some measure of how thorough a process is in documenting Interactions.

Interaction-goal quotient (IGQ) measures how many different objectives are deemed by designers to be of consequence for each interaction noted. We seek the ability to note different consequences for every potential objective in search of greater value. IGQ is calculated as the average number of objectives analyzed as part of interactions.

Time measures how long a method requires, given the number of options, objectives, and interactions. Any conceptual design process must weigh the benefits in increased design knowledge with the time and resources that must be devoted to the method. Given the scope of our research, however, the most important application of this metric will be to understand whether it is the mechanics of the methodology or in the time needed to complete the methodology that underlie values obtained for the previous four metrics.

Related to transformation, flow, and value, the first metric helps quantify how well we have managed the transformation view, the second how well we have managed the value view, and last three how well we have managed the flow view.

For each of these three engineering views, there are examples of tools that successfully manage them. Through analysis of these tools with respect to the view characteristics, we can pinpoint their existing strengths and shortcomings and, in the context of our theoretical framework, describe how a process might be “better.”

4. POINTS OF DEPARTURE: DESIGN MANAGEMENT SYSTEMS

After defining exactly what aims we wish to address, at which point in the design process we wish to do so, and with which language we will discuss them, we can now formulate our research question:

“Multidisciplinary” conveys the importance of communication between disciplines, as supported by flexible methods. “Conceptual design” encompasses the significance of increased design knowledge and freedom at the beginning stages of design, as well as grounded notions of conceptual design expressed in mechanical engineering and cognitive psychology literature. “Search, manage, and evaluate” mirror the conversion, flow, and value views of engineering, because we mean to search sets of options defined in the conversion view, manage interactions in the flow view, and evaluate integrated alternatives in the value view. Finally, the distinction between solution spaces and trade spaces defines the starting and ending points of our method in terms of the stages of a design process.

Multiattribute, collaborative design, analysis, and decision integration (MACDADI; Haymaker & Chachere, Reference Haymaker, Chachere and Smith2006; Haymaker et al., in press), as mapped in Figure 2, specifies the building of six design models to explicitly define organizations, objectives, weights, options, impacts, and value. MACDADI manages the value view by engaging with stakeholders to define objectives and then defines normalizing metrics for designers to evaluate potential alternatives. MACDADI helps to break down decisions and responsibilities into alternatives and topics, but does not specifically facilitate the recombination and evaluation of different combinations of options, thus only partially managing the conversion view. MACDADI does provide explicit frameworks for teamwork and communication, but it lacks a method to note interactions between hierarchically defined tasks and elements, thus not fully satisfying the flow view. Nonetheless, MACDADI's explicit value management capabilities will serve as the organizing framework of our new method. Essentially, we seek to insert into MACDADI's value management capabilities a systematic way of thinking about the potential value of interactions between options.

Fig. 2.

A schematic of the multiattribute, collaborative design, analysis, and decision integration process (http://www.MACDADI.com; Haymaker et al., in press). [A color version of this figure can be viewed online at journals.cambridge.org/aie]

Pugh's (Reference Pugh1981) controlled convergence method uses a matrix to rate potential “concepts” (alternatives) against a predetermined set of project criteria, thus satisfying the value view. After evaluation, the method encourages multiple iterations where designers combine concepts that may have complementary strengths and weaknesses in search of a concept that best satisfies all project objectives. Although the method encourages the recombination of options to form alternatives, like MACDADI, it does not provide an explicit or systematic method for doing so. Furthermore, it does not systematically note rationale (e.g., interactions) for combinations of options that compose these alternatives. Thus, it cannot completely satisfy the conversion view or the flow view. By inserting the concept of value into the notion of interactions, we are making explicit the reasons for considering different alternatives. MAID can help make transparent the consideration of different combinations of options throughout the process.

Strategy generation tables (Howard, Reference Howard1988) satisfy the conversion view by hierarchically breaking down and representing decisions as made up of topics, their constituent options, and aggregated alternatives. For example, in Figure 3 below, choosing one option from each of the topics labeled in the top row composes an alternative focused on the idea of “service business,” as written at left. By expressing this “strategy theme,” strategy generation tables acknowledge that different alternatives present different value propositions. However, they do not use an explicit framework to rationally justify such labels with respect to project objectives. Thus, we argue that the method only partially manages the value view. We also note that strategy generation tables do not represent interactions (or more generally, information flow) between options, therefore failing to manage the flow view. In essence, we hope to create a framework that fills in the absence of explicit objectives and explicit interactions currently implicit in the lines and boxes within a strategy table.

Fig. 3.

An example of a strategy-generation table based on Howard (Reference Howard1988).

The DSM methodology uses matrices to visually represent ideal information flow within a project (e.g., Yassine & Braha, Reference Yassine and Braha2003). It visually represents what tasks or decisions contain information that should be used in other tasks or decisions. For example, in Figure 4, an “x” in any given box represents that information from the element denoted by the column heading is needed by the element denoted by the column heading. As shown, the “x” in column B and row C denotes that C needs information from B, or more simply, that elements B and C interact in at least one important way and require some level of integrated decision making. DSM explicitly acknowledges interactions between decomposed project elements, and in doing so, helps to make sense of a hierarchical breakdown structure. Thus, it provides a concise way of satisfying both conversion and flow views of a process. However, DSM does not manage the value view. We seek to create a method that conveys specific information regarding the nature of noted interactions with respect to any project objectives.

Fig. 4.

An example of a design structure matrix based on Whitney et al. (Reference Whitney, Anderson, Cadet, Fine, Gossard, Thornton, Groover, Nagel, Ozsoy, Stenger, Anderson, Chang, Cunningham, Gutwald, Hardy, Laukaitis, Lee, Mantripragada, Pomponi, Soman, Baykan, Greif, Johnston, Kim, Marquette, Meixell and Roth1995).

Morphological matrices, first formalized by Zwicky (Reference Zwicky1948), have long been used to improve conceptual generation phases in design. Similar to strategy tables, they specify functions (topics, in our language) and solutions (options) in columns and rows, and designers then compose alternatives by picking one solution from each function (one option for each topic; Pahl et al., Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007). Morphological matrices still do not explicitly note interactions between options, thus only achieving partial management of the flow view. Like strategy tables, morphological matrices do not manage the value view, because little to no information is conveyed, other than potential feasibility, about the performance of interactions and alternatives with respect to objectives.

Weas’ and Campbell's (Reference Weas and Campbell2004) analysis of interconnected decision areas (AIDA) provides another fundamental point of departure by visually and explicitly supporting breakdown structures through circles (representing topics) and dots within them (representing constituent options). Furthermore, these options are presented in a matrix similar to DSM, in which pair wise combinations are checked for interactions affecting feasibility. However, AIDA lacks notions of stakeholder objectives and value, both in its treatment of individual options and their interactions. AIDA thus exemplifies management of the conversion and flow views but lacks management of the value view. This is similarly true of the DeMAID process developed by Rogers (Reference Rogers1989). DeMAID's improves the DSM method by ordering these tasks, determining their interactions, grouping iterative processes, and displaying the entire process in a DSM matrix. However, DeMAID also lacks acknowledgement of the value view.

Kam's (Reference Kam2005) decision dashboard (DD) utilizes colored boxes and lines to represent breakdown of options into topics and aggregation into alternatives, as well as the evaluation of those elements to project objectives. In doing so, DD satisfies the conversion view. However, similar to controlled convergence and AIDA, DD does not explicitly represent the impact of interactions, nor does it provide for an overall calculation of project value from individual evaluations of options. DD also lacks explicit lines of communication that help facilitate the creation of its visual maps; thus, it fails to satisfy the flow view and only partially satisfies the value view. We hope to add rationale clarity to treatment of the interactions between options.

Quality function deployment (QFD; Akao, Reference Akao and Mazur2004) explicitly derives technical requirements from customer desires, using multiple matrices in a successively more detailed “house of quality” to systematically break down products and processes for design solutions, thus partially satisfying the conversion and value views. Because QFD focuses only on such a breakdown and not on aggregation of options into multiple alternatives, nor a comparison of those alternatives through explicit value calculations, it can only partially satisfy these views. QFD explicitly denotes interactions between organizational functions and technical assemblies in a matrix similar to DSM, thus both demonstrating their importance and partially satisfying the flow view.

We summarize our review of current literature and methods in the points of departure matrix (Table 2). A capital “X” signifies that a method fully satisfies a requirement; a lowercase “x” signifies that a method partially or incompletely satisfies a requirement, and a blank entry means that a method does not fulfill that requirement. We desire to create a column with a method that has an “X” in every row.

Table 2.

Points of departure matrix

Note: MACDADI, multiattribute, collaborative design, analysis, and decision integration; CC, controlled convergence; ST, strategy tables; DSM, design structure matrix; MM, morphological matrix; DD, decision dashboard; QFD, quality function deployment; AIDA, analysis of interconnected decision areas.

5. MAID

MAID builds on characterizations of conceptual design discussed earlier (Weber & Condoor, Reference Weber and Condoor1998; Pahl et al., Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007; Chachere & Haymaker, in press). To satisfy our proposed requirements within this context, we draw upon and combine the strengths of existing methodologies discussed earlier. A process and data flow view of MAID is presented in Figure 5. In subsequent figures, schema descriptions of each step are provided, as well as screen shots of a prototype tool, also called MAID, illustrating its use on the Stanford Green Dorm Project.

Fig. 5.

A process map of multiattribute interaction design. [A color version of this figure can be viewed online at journals.cambridge.org/aie]

For the “Project Team” and “Goals” steps, we defer to Haymaker and Chachere (Reference Haymaker, Chachere and Smith2006) and Green (Reference Green1994) to detail the development of stakeholder and objective models. For the purposes of MAID, data outputs from these steps include only the enumeration of project objectives, as illustrated by the four chosen in the charette (Fig. 6).

Fig. 6.

A schema description and example of the goals for multiattribute interaction design.

We note that stakeholder definition generally precedes objective identification, and we follow this convention here in ordering these first steps. Further refinement of the project team beyond stakeholders, especially including the selection of appropriate AEC professionals, will almost certainly occur in an iterative manner as objectives become clearer; this discussion is beyond the scope of this paper.

It is in the bottom section of the process map that MAID proposes a new and improved methodology. First, in the “topics” and “options” steps designers explicitly and hierarchically organize options within topics to manage the conversion view. Pahl et al. (Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007), Weber and Condoor (Reference Weber and Condoor1998), and Kunz and Fischer (Reference Kunz and Fischer2007) define specific methods for creating topics through functions, subfunctions, and breakdown structures. In the example charette, designers have chosen three topics: main sources of energy, supplementary sources, and water management (Fig. 7), within each of which can be defined a series of constituent options (Fig. 8).

Fig. 7.

A schema description and example of the topics for multiattribute interaction design.

Fig. 8.

A schema description and example of the options for multiattribute interaction design.

Second, as in MACDADI, design teams analyze options individually with respect to project objectives in the “option impacts” step. Option impacts are chosen as integers on a scale from −3 to 3, according to metrics that can be developed to varying levels of specificity (Fig. 9). Although such a ranking system may be arbitrary to some degree, Pahl et al. (2006) points out that quantifying all parameters during conceptual design is impossible, and thus qualitative judgments should be made on the basis of metrics and design intuition. Pahl adds, “Though the attribution of points raises problems, it is not advisable to evaluate too timidly during the design phase.” Rating options assigned to separate topics with respect to project objectives integrates conversion and value views. Notably, for some options, their “goodness” is determined in large part by other decisions made about other topics; this is the motivation behind the subsequent steps in the MAID methodology that analyze interactions between options.

Fig. 9.

A schema description and example of the option impacts for multiattribute interaction design. [A color version of this figure can be viewed online at journals.cambridge.org/aie]

In the “interactions” step, design teams identify and analyze potential Interactions between options. By using a matrix similar to DSM and AIDA, designers visually pick out combinations of options for which the consideration of integrated design is important (Fig. 10). Positive and negative interactions exist; both should be noted as a means of facilitating creative and value-based design. By explicitly noting interactions between options within topics, this step manages both conversion and flow views (Fig. 10, top).

Fig. 10.

A schema description and example of the interaction impacts for multiattribute interaction design. [A color version of this figure can be viewed online at journals.cambridge.org/aie]

To impart value management, MAID asks designers explicitly think about the specific objective(s) for which the relevant interaction has effects (“interaction impacts” step). Interactions are rated on a scale of −2 to 2, where 2 specifies a very positive interaction and −2 a very negative one. Such constructed scales are justified both in a conceptual design framework (Pahl et al., Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007) when much information is still unknown (as with interactions) and in the context of normalizing across variety of project objectives measured in very different units (Keeney & von Winterfeldt, Reference Keeney, von Winterfeldt, Edwards, Miles and von Winterfeldt2007). In this case, when designers recorded an interaction between photovoltaic cells and electric vehicles as elements of an energy system, they were prompted with the screen shown on the right in the bottom of Figure 10. Here, they noted that their research potential was more than simply the sum of their individual ratings, because research on the use of electric vehicles to store excess electricity produced by solar cells at hours of peak sun exposure was proposed by Stanford faculty in the Green Dorm Feasibility Study (EHDD Architects, 2006). This interaction affected the objective of reducing energy use as well, because if electric vehicles could serve an energy-storing purpose, then more of the energy production from photovoltaic cells could be used to help reduce energy consumption from other resources (presumably powered by fossil fuels). In short, evaluating interactions based on their impacts on specific project objectives provides more information on the potential impacts of those decisions in order to help find combinations of options that satisfy multiple requirements.

The cells in the matrix change color to reflect whether the total effect of that specific Interaction on all project objectives was positive or negative (Fig. 10, top). In effect, this provides an entirely new method of generating and explaining a DSM matrix, using information in the value view to generate Xs in the flow view. By executing these steps for matrices corresponding to different pairs of options, design teams can visualize which topics will require the most collaboration between designers during further stages of design. Explicit and documented rationale will be available to the team throughout the project, potentially helping to motivate and inspire creative configurations of options.

Using data entered up until this point, MAID uses strategy tables and morphological matrices to motivate the selection alternatives in the “alternatives” step. Design teams can choose combinations of options, as shown in Figure 11. Alternatives shown in columns are composed of options selected from each topic (one topic per row). As in strategy tables, design teams can title alternatives so as to remind themselves of the reasoning or “themes” behind their choices.

Fig. 11.

A schema description and example of the alternatives for multiattribute interaction design.

MAID explicitly calculates the value for each alternative using data already input in prior steps. As specified in Figure 12, value is calculated additively by summing individual option impacts and interaction impacts across all objectives, topics, and options. Note that stakeholder weighting of objectives was not included in this version of MAID software, but it could easily be integrated into the value calculation by multiplying each impact term by the weight of its respective objective, as is done in MACDADI. Although methods of calculating value will vary depending on the specific value model used, we have assumed that project objectives developed in MACDADI meet properties of fundamental objectives proposed by Keeney (Reference Keeney, Edwards, Miles and von Winterfeldt2007) and rationality (Edvardsson & Hansson, Reference Edvardsson and Hansson2005) that justify the use of additive value models (Keeney & von Wintefeldt, 2007). It is certainly possible to develop other, more complicated methods of aggregating metrics and differentiating between first- and second-order impacts, as well as to develop cardinal rating systems that allow relative ratings of options and alternatives against predefined benchmarks. However, doing so would require proof that the new method significantly improves upon simpler ones, and given previous justification and examples (e.g., Thurston & Carnahan, Reference Thurston and Carnahan1992) of ordinal methods in conceptual design, we leave extensions as an issue for further research.

Fig. 12.

A Schema description and example of the “value” step of multiattribute interaction design. [A color version of this figure can be viewed online at journals.cambridge.org/aie]

Thus, just like the generation of a flow view from a value view, MAID expresses the value of alternatives generated from the conversion view of Figure 11 through the charts shown in Figure 12. In doing so, MAID meets our aim of making explicit and more specific the implicit value proposition present in the “strategy themes,” lines, and boxes of strategy tables.

In these summary graphs, the left chart shows the total value of each alternative, as calculated by summing the individual and interactive effects of the constituent options on each of the project objectives (Fig. 12, the five bars represent the total value of each of the five alternatives). The chart on the right provides more detailed information on the performance of individual alternatives with respect to specific objectives. For instance, for the “car solar” option shown in Figure 12, the alternative was composed of both photovoltaic arrays and electric cars, and predictably, the data shows a high score for research potential, partly as a result of the interaction the design team denoted in previous steps. This view of project value helps designers to summarize the many levels of prior analysis. Because design, and especially conceptual design, is an iterative process, the explicit documentation of each step can help designers return to their original analyses and refine their initial intuitions as new information arises.

In sum, combining strengths of different design tools, MAID provides one framework for designers to explicitly and flexibly engage in integrated conceptual design by specifically noting interactions between options and their effect on project value.

6. VALIDATION

To this point, we have identified a problem in AEC practice through industry case studies, developed a theoretical framework of metrics and requirements capable of addressing the problem, and described a model that addresses these requirements. In order to begin to test MAID's power and generality, we created simple database software and user interface that allows repeatable use and storage of data. This software allowed us to test our validation strategy through charettes with university students. The results of these tests, in combination with our requirements framework, suggest that MAID may be adept at addressing our observed problem. As initial tests, these charettes are not meant to prove conclusive data about the consequences of MAID on final designs but are instead meant to provide evidence of the power and generality of the method to improve upon our proposed metrics as they relate to the three engineering views. However, rigorous comparison against baseline methods required to make stronger claims is left as an issue for further research.

The two motivating case studies described in Section 2 served as the basis for student charettes. Six groups of undergraduate and graduate students (one to three students) were provided with design scenarios based on these case studies. Because MAID focuses on managing information rather than creating it, students were provided objectives, topics, and options in advance, although they needed to input them into MAID software themselves as part of the charette. As students were not required to generate these on their own, we point to discussions of parallel concepts in other design literature for further insight into those processes. Objectives are easily related to ideas of customer and engineering requirements (Krishnan & Ulrich, Reference Krishnan and Ulrich2001) or to goals in the context of Thurston (Reference Thurston, Lewis, Chen and Schmidt2007), who also discusses options. Griffin and Hauser (2003) explore methods and consequences for generation of customer requirements in a variety of fields, as the previous reference to Green (Reference Green1994) does for the AEC industry. Topics derive from traditions of hierarchical decision making as discussed as part of organizational behavior and mathematical modeling literature by Chachere and Haymaker (in press). Prior references to Kunz and Fischer (Reference Kunz and Fischer2007) and Pahl et al. (Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007) are helpful for demonstrations of method, as is Hauser and Clausing's (Reference Hauser and Clausing1988) “house of quality” for building out engineering characteristics and specific option choices in product design and business contexts.

All students had at least moderate backgrounds in performance-based design and design theory, as well as familiarity with the case studies, although they had no prior knowledge of MAID itself. No time limit was explicitly given for the completion of the charette, although as will be described below, time needed averaged 75 min. For the purposes of the charette, students played all parts of the design team: stakeholders, experts, decision makers, and so forth.

6.1. Testing and results

For the high-speed rail project (Table 3), we presented users with a design scenario consisting of three topics, each with two options: location (Palo Alto or Redwood City), separation of grade (above-ground or below-ground train), and density of development (high or low density). In this design problem, eight potential alternatives exist, and we take as an ideal set of objectives the 15 already gathered as part of student projects at Stanford (Roedel et al., Reference Roedel, Talbot and Bian2009).

Table 3.

Results of Charettes and Green Dorm Charettes

Note: DSS, design space sampling; OSQ, objective space quality; IQ, interaction quotient; IGQ, interaction-goal quotient; MAID, multiattribute interaction design.

The data show evidence for three main points. First, MAID facilitates exploration of a large number of project alternatives, as shown by its DSS value. Second, although the OSQ value of the MAID process is relatively low, this is a direct result of constraints of time, not constraints or limitations of the process. By devoting more than 1 h to the charette, designers could have easily achieved greater exploration of the objective space in line with improvements in DSS.

Third, and perhaps most notably, values for IQ and IGQ suggest designers’ ability to note interactions proved very important. Designers noted not only the interactions for all possible combinations of options but also that each interaction impacted a very high number of project objectives. In all charettes, designers chose to note interaction effects for 100% of potential interactions, and for each of these interactions, they recorded consequences for an average of more than four out of five project objectives. These numbers give further credence to the idea that interactions between options are important in the minds of designers and that they can be explicitly expressed in the deign process. Furthermore, these interactions deserve attention in the flow and value view, exactly as provided in the MAID methodology.

In the case of the Stanford Green Dorm, we organized two charettes that tackled different topics and options (Table 3). The first dealt with energy production and site considerations, and the second considered mechanical systems, structural systems, and the inclusion of a living laboratory within the building.

In this data, we see very similar results to the high-speed rail charettes. Use of MAID shows high DSS levels, demonstrating appreciable exploration of alternatives. Furthermore, we continue to see high levels of IQ and IGQ, showing further evidence that this added ability to explore Interactions’ impact on a variety of project objectives proves important to designers.

In conclusion, charettes show increases on two different projects of five metrics that relate to the three engineering views. The data show that the MAID methodology facilitates a wider exploration of the solution space, both in terms of options and objectives. Although the professional design teams were not available for rigorous comparison, our discussion of motivating case studies suggests that MAID could improve these conceptual design processes. Most notably, MAID's ability to note interactions and to integrate their effect on project objectives into an explicit value calculation proves useful, at least in terms of the attention given to that step by designers. We now turn to its contribution to theory.

7. CLAIMED CONTRIBUTION

We claim as a contribution to AEC design theory the development and creation of a flexible and explicit conceptual design methodology that satisfies all the requirements in Table 1 by synthesizing existing methods that do not fully manage such views. MAID's contribution to design theory is relevant given Ballard's and Koskela's (1998) call for more conclusive testing of hypotheses surrounding the effect that management of three engineering views can have on design. Such testing requires methodologies that demonstrate such management. Thus, by providing one example and preliminary testing of such a synthetic methodology, MAID makes an important contribution to design theory.

8. PRACTICAL IMPACT

Integrated design is a complicated process. No methodology, and certainly not one performed in a few hours, will completely and accurately analyze building systems. Nonetheless, design theory holds that using practical value models for conceptual design, in conjunction with management of conversion and flow views, can increase design knowledge and eventually lead to better buildings. Further research is needed to prove this conclusively in our case, although current research suggests possibilities for potential impact.

First, use of MAID can potentially increase the number of alternatives explicitly considered by designers, aligning with current thinking that quality is in large part a function of quantity of potential designs considered, especially in light of increasing design complexity.

Second, use of MAID can potentially increase the number of objectives to be explicitly considered during conceptual design, and research by Green (Reference Green1994) suggests that such value management techniques result in large increases in project value. Furthermore, Green (Reference Green1994) holds that early project team coalescence around explicit objectives helps create a culture of cooperation and “buy in” that proves very useful in all stages of design. For purposes of overcoming obstacles presented by local task responsibility (Ballard & Koskela, Reference Ballard and Koskela1998) and weak cooperation (Clausing, Reference Clausing1994) that emerge from traditional conversion views of engineering, this provides immense value in its own right.

Third, MAID potentially increases the number and specificity of interactions explicitly identified by designers during conceptual design. These interactions where value may lie occur between options and topics for which responsibility falls to disparate design disciplines that may struggle to collaborate (Rechtin, Reference Rechtin1991).