1. INTRODUCTION
Developing a product or system architecture can be attained by a traditional (or point-based) design practice or by a set-based design practice. Firms following a traditional design practice usually develop a variety of design ideas on alternative product or system architectures first (Krishnan & Bhattacharya, Reference Krishnan and Bhattacharya2002).Footnote 1 Subsequently, these firms follow a design process that foresees a dedicated idea evaluation and selection stage that consists of the identification and careful selection of the best design idea (Kudrowitz et al., Reference Kudrowitz, Te and Wallace2012), which will then be developed into a system's design including a specification of components and interfaces, that is, the product's or system's architecture. For this purpose, firms provide financial commitment to this idea, which will be developed by a dedicated team to refine its initial component and interface specifications until it meets the design objectives (Clarkson et al., Reference Clarkson, Simons and Eckert2004; Cooper, Reference Cooper2008). This traditional (or point-based) design practice is effective if firms have the ability to discern the quality of the ideas and design alternatives and to pick the best one (Abernathy & Rosenbloom, Reference Abernathy and Rosenbloom1969; Girotra et al., Reference Girotra, Terwiesch and Ulrich2010). However, scholars found that recognizing the best idea in the concept phase is difficult and often suboptimal (Herstatt et al., Reference Herstatt, Verworn and Nagahira2004; Ozer, Reference Ozer2005; Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2010). The problem appears to be that neither idea generators nor experts from the respective technical field are able to predict which ideas or solutions will be effective or successful (Adler et al., Reference Adler, Goldoftas and Levine1999; Girotra et al., Reference Girotra, Terwiesch and Ulrich2010; Nelson, Reference Nelson1961).
Alternatively, set-based design, a method of lean product development (Liker et al., Reference Liker, Sobek, Ward and Cristiano1996; Sobek et al., Reference Sobek, Ward and Liker1999; Morgan & Liker, Reference Morgan and Liker2006; Kennedy et al., Reference Kennedy, Sobek and Kennedy2013; Ward & Sobek, Reference Ward and Sobek, II2014), follows a different design logic. With set-based design, firms gradually narrow the set of possibilities by eliminating unfeasible ideas and by converging on a final solution in the course of the design project, which makes finding the best design idea more likely. Thus, firms can focus on understanding trade-off curves and on the exploration of regions of the design space that contain an almost indefinite number of design alternatives. Alternatively, firms can deploy discrete sets of ideas and explore these conjointly, instead of just one idea at a time (Sobek et al., Reference Sobek, Ward and Liker1999; Kennedy et al., Reference Kennedy, Sobek and Kennedy2013). Our study pertains to this latter alternative. It studies set-based design cases that consider sets with discrete alternative designs. Moreover, set-based design builds on three principles: map the design space and represent initial requirements as sets or ranges rather than point values, integrate by intersection using the explicit characterization of trade-offs and design limits, and establish feasibility before commitment by developing sets of discrete design alternatives (Sobek et al., Reference Sobek, Ward and Liker1999; Kennedy et al., Reference Kennedy, Sobek and Kennedy2013). This study focuses on the third principle, because the “entire set based development process might be viewed as a system to fulfill the third and last principle: ensure that designs are feasible before committing to them” (Sobek et al., Reference Sobek, Ward and Liker1999, pp. 67–84).
Overall, the choice of the design practice by which system architecture design is performed requires an adequate design process and design organization. The literature on set-based design pays particular attention to how the product design evolves over time (Liker et al., Reference Liker, Sobek, Ward and Cristiano1996; Sobek et al., Reference Sobek, Ward and Liker1999; Morgan & Liker, Reference Morgan and Liker2006; Raudberget, Reference Raudberget2010; Ward & Sobek, Reference Ward and Sobek, II2014). However, we do not know much about how firms design the transition process from ideation and mapping the design space to identifying and committing to a feasible design alternative. Traditional design practice evaluates the feasibility of all design ideas by using the firm's extant knowledge base and by filling the knowledge gaps that prevent the firm from knowing which design will be best through extrapolating from its extant knowledge. Firms following set-based design, however, choose to very quickly close the knowledge gaps most critical for elimination and create the respective knowledge. Knowledge creation is not free of cost, however. Accordingly, these firms need to determine for which and how many ideas to initiate knowledge creation efforts.
Subsequently, firms need to organize for set-based design by allocating resources in the form of (team) labor to the pursuit of several design alternatives and the associated knowledge creation and gradual elimination task. Does one team develop the knowledge needed for all of the alternative ideas or is one team assigned per alternative? These questions are irrelevant in the context of traditional design practice that has only one system design to be developed. Further, the literature on set-based design has neglected to study how firms assign team resources to ideas. This research addresses these gaps by conducting exploratory case study research.
After a literature review and a description of this study's research methods, case study data, followed by data analysis and a discussion of the results, is presented. Findings show that in the five cases studied, firms did not formally or explicitly calculate an optimal number of ideas to be pursued. Instead, the number is determined by the number created during ideation. In addition, in the five cases studied, either firms engaged a single team to pursue all previously developed ideas or, alternatively, firms engaged one team per idea, where the teams pursued different ideas simultaneously and in parallel and where a team ceases its work as it discovers the unfeasibility of the design it pursues. Further, the firms of this case study either engaged internal teams or used external resources in cases where not enough skilled personnel was available within the firm. In all cases, teams collaborated and did not compete. Competitive teams could be observed only in the stage preceding selection and development, that is, the stage of idea development.
This study has limitations. It studies particular facets of set-based design: the set-based management of major alternative concepts; the third principle of set-based design (i.e., establishing feasibility before committing to a design); and the transition from mapping the design space to identifying a design alternative that has demonstrated some level of robustness. Thus, a number of set-based design issues are beyond the scope of our research and our case studies and remain to be studied in future research, such as the process of narrowing the idea portfolio or how teams learn the design limits and establish trade-off curves in order to optimize the system architecture design of the remaining alternatives.
2. LITERATURE REVIEW
The following sections build on the set-based design literature as well as on other literature strands that have been concerned with questions related to the research questions of this study.
2.1. Number of alternatives to be pursued
The literature on set-based design offers two studies that are concerned with the determination of the number of design alternatives to be pursued (Ford & Sobek, Reference Ford and Sobek2005; Schäfer & Sorensen, Reference Schäfer and Sorensen2010). They argue that the number of design alternatives to be pursued is based on economic trade-offs, and both are based on the real options’ approach. Similarly, the literature stream of parallel product development offers a number of econometric models that aim to answer the question of how many alternative solutions firms should pursue in parallel (Nelson, Reference Nelson1961; Abernathy & Rosenbloom, Reference Abernathy and Rosenbloom1969; Arditti & Levy, Reference Arditti and Levy1980; Dahan & Mendelson, Reference Dahan and Mendelson2001; Ding & Eliashberg, Reference Ding and Eliashberg2002; Krishnan & Bhattacharya, Reference Krishnan and Bhattacharya2002; Scherer, Reference Scherer2011; see Table 1). While parallelism and a decision on the number of design alternatives to pursue is inherently relevant to set-based and parallel product development, we acknowledge that the literature on parallel product development considers launching comprehensive development projects for each alternative. This differs from set-based design, which explores multiple designs in parallel to purposefully fill the knowledge gaps related to the feasibility of each design in order to gradually converge on a single one.
Research on parallel development
All of the models pertaining to parallel development projects state that to justify an additional alternative for parallel development, its differential economic benefits should be greater than the direct differential cost of the parallel approach. Thus, the benefit of the task outcome depends on the performance of the alternative and completion time, and cost comprises the cost to produce the outcome, which is determined by the number of choices. In addition, models consider the probability of success (i.e., uncertainty) and the opportunity costs of delays in project completion such as diminished competitive advantage and out-of-pocket costs incurred by an extension of the project duration (Abernathy & Rosenbloom, Reference Abernathy and Rosenbloom1969). Additional factors taken into account are the amount learned or the decrease in uncertainty (Abernathy & Rosenbloom, Reference Abernathy and Rosenbloom1969) and the difference between the cost and performance estimates as well as the design similarities and differences of the different alternatives (Nelson, Reference Nelson1961; Thomke et al., Reference Thomke, Von Hippel and Franke1998). Overall, the higher the potential value of a product (i.e., profit impact) and of time (i.e., speed-to-market), the higher the new product development uncertainty, and the lower the cost of running a project with parallel alternatives, the higher the optimal number of alternative solutions to be pursued. Expressed differently, the optimal number of parallel paths occurs when incremental value equals incremental cost and “There is always a correct number of parallel paths” (Reinertsen, Reference Reinertsen2009, p. 49).
Overall, economic models seek to facilitate the determination of the number of development efforts that should be run in parallel. To correctly evaluate parameters such as incremental cost and incremental value remains with the decision maker. Managers need to provide estimates of all these parameters such as cost, value of different alternatives, and uncertainty. Providing sound estimates on technical feasibility, the duration, and the cost of a particular design alternative or to capture the degree of uncertainty is often challenging. However, our understanding of how the calculation models firms use and how organizational actors attain values for the calculations’ parameters is incomplete (Dahan, Reference Dahan1998; Thomke et al., Reference Thomke, Von Hippel and Franke1998; Loch et al., Reference Loch, Terwiesch and Thomke2001). This research addresses this gap by the following research question: how do firms determine the number of design alternatives to pursue with set-based design?
2.2. Assignment of alternative design ideas to teams
Having determined the number of design alternatives to pursue, firms need to allocate the development tasks to teams. However, the literature on set-based design does not offer insights on this resource allocation. Hence, this research builds on insights of the cognate field of parallel product development. This research strand discusses the deployment of multiple teams and distinguishes between collaborative and competing ideas. Gold (Reference Gold1987) suggests a strategy that encourages innovative product development projects to define an advanced target, establish multiple competing teams to work on different alternatives to attain the target, and then eliminate the least successful alternatives at successive stages of development. This strategy, he argues, seems not only to offer the advantage of increasing the chances of success through exploring multiple approaches in parallel but also to help intensify motivation because of the competitive situation. In particular, Sundaresan and Zhang (Reference Sundaresan and Zhang2009) studied competitive and noncollaborative and collaborative parallel teams. Noncollaborative teams work independently without learning from or sharing knowledge with other teams, whereas collaborative teams work closely together so as to effectively increase the success rate of the overall project. Developing a mathematical model, the authors find that collaboration in parallel teams is vital to obtain maximal benefit.
Further, Zhang and Sundaresan (Reference Zhang and Sundaresan2012) investigate the design of incentives for effective concurrent team management. In their model, they consider that successful innovation teams can be rewarded individually or collectively. With an individual team reward policy, the incentive to motivate collaboration is weak because only successful teams receive the reward. In contrast, under the collaborative reward policy, teams will share the total reward as long as any team succeeds, so they are induced to collaborate voluntarily with other teams. This leads to a better overall project result. In support of this, Nelson (Reference Nelson1961) argues that when firms run parallel approaches and assign a different team for each approach, there is a smaller likelihood that the problem will be solved in the same way. There is also some evidence that companies undertake more imaginative development when there is a race for success than when there is no competition. Further, if information created while developing one idea turns out to be relevant to the work of a competing idea, it needs to be communicated and shared between teams to be beneficial. However, competition between teams may prevent knowledge sharing. In a different setting, Kreiner et al. (Reference Kreiner, Jacobsen and Jensen2011) analyze the introduction of legitimate dialogues between teams working in parallel in architectural competitions. There, design solutions existed independently at the beginning, became connected in the course of the project due to dialogue, and were eventually selected as the winner or not. The dialogue allowed architects to adapt, and in some cases, they adapted in ways that depreciated their outcome. The authors argue that the dialogue increased the number of ideas that made the choice harder.
Furthermore, Girotra et al. (Reference Girotra, Terwiesch and Ulrich2010) examine the effectiveness of two group structures: the team structure, in which the group works together in time and space, and the hybrid structure, in which individuals first work independently and then work together. A hybrid team structure was found to lead to a higher number of ideas with higher average quality and to the best idea overall. Transferring this insight to set-based design, teams are likely to be most successful when they start to work on their ideas independently first and then collaborate to exchange their experience.
Overall, the new product development literature suggests assigning one team to each idea, but it is inconclusive on the effects of competitive or collaborative teams. This research addresses this gap by addressing the following research question: how do firms assign teams to the development of design alternatives with set-based design?
3. METHOD
This research is based on an exploratory case study analysis that builds on data collected in three companies (A, B, and C), with Companies B and C figuring in two case studies. Although firms’ abilities to deploy set-based design are firm-level data, they become apparent in projects. Hence, new product development projects were selected for data collection and analysis. Moreover, a precondition was the presence of set-based design. This was assessed by means of three significant criteria (Sobek et al., Reference Sobek, Ward and Liker1999): establishing the feasibility of a design before committing to it; gradually eliminating designs that prove unfeasible and converging to a final design; and purposefully exchanging the knowledge that was newly created to fill the knowledge gaps (i.e., on feasibility of a design) among the parallel developments.
3.1. Research setting
Because set-based design has not been widely applied outside Toyota and its suppliers so far (Harkonen et al., Reference Harkonen, Mottonen, Belt and Haapasalo2009), the cases were selected based on theoretical sampling to study firms that deploy set-based design by considering multiple, architecturally different solutions and by establishing the feasibility of these solutions before committing to them, thus realizing the third principle of set-based design (Glaser & Strauss, Reference Glaser, Strauss and Denzin1970). With respect to establishing the feasibility of a design before committing to it, one out of the five cases in this paper shows signs of the traditional design practice by selecting a design based on best guesses and establishing feasibility later. All other cases establish feasibility first rather than picking a design. With respect to the latter two criteria, all cases gradually eliminate designs as the knowledge gaps on feasibility are filled and all cases pursue a purposeful knowledge exchange between the parallel design efforts within each project. The case that shows signs of the traditional design practice is Case 3. Its main activities are dedicated to the generation of knowledge for the purpose of understanding the feasibility (“does it work or not”) of 50 design alternatives, eliminating 46 designs. Out of the remaining 4, 1 design was selected. This corresponds to the traditional design practice as the firm commits to a design without having yet established feasibility. Despite showing signs of traditional design practice in this case, it was kept as a case in this study because the vast majority of design decisions (pursue or cull) were made in a set-based way. The case also provides relevant answers to the research questions, which both pertain to the first phase of the design process under consideration where decisions are made on the number of alternatives to pursue and on the team organization to be set up. This is largely independent of the latter phase of the design and, more specifically, the idea selection process.
Furthermore, case selection included multiple case companies differing in many ways, such as industry, size, or product characteristics, providing external validity (Yin, Reference Yin2008). In particular, the case companies operate in industries that differ in their clockspeed (Nadkarni & Narayanan, Reference Nadkarni and Narayanan2007). Company A operates in the electronics industry and has to innovate rapidly. Company B produces tooling equipment. In this industry, time-to-market is less important. Company C develops and manufactures machines for the chemical industry. While time-to-market is decisive, the product life times of Companies B and C are longer than at Company A. Further, all firms are of different sizes, ranging from a small (C) to a medium (B) to a large (A) company. Because set-based design requires substantial resources in the first phases of a project, firm size and resource availability are critical. Finally, the product type to be developed is related to the process of development. Hence, a firm was selected that develops products that are mainly software (A) as well as a firm with hardware products (B) and a firm (C) with products that are partly based on hardware and software. The companies are located in Germany and Switzerland, and all of the companies are original equipment manufacturers. Prior to deploying set-based design, company representatives were interested in potential methods for new product development efficiency. Hence, they learned about the concept of set-based design through the literature or seminars. An exception is Case 5, where set-based design was pursued based on intuition. In Cases 1, 2, and 5, it was the first time they had purposely pursued set-based design; in Cases 3 and 4, it was the second time. Although set-based design has not been deployed comprehensively in all cases, the third principle of set-based design (establishing feasibility by developing sets of alternatives), which is central to this study, was realized in all cases in this research. In sum, to enhance generalizability, the cases studied in this research were collected from firms from different industries (hence, clockspeed), of different sizes (hence, resources available for innovation), and with products of differing complexity (hence, resource need).
3.2. Data collection and analysis
Because the objective was to generate in-depth insights with regard to the stated “how” research questions, empirical data was collected in a way that allowed facts and opinions about, as well as insights into, phenomena to be obtained from firsthand sources (Yin, Reference Yin2008). In particular, case study data was collected longitudinally through observations of project meetings, semistructured interviews with engineers and managers who were actively involved in the projects, and collecting documents, such as project meeting minutes taken by the companies, or documents, such as technical result tables, that the teams created in the course of the project. Data was collected by these means as decision processes on team assignments as well as on the number of projects to pursue became apparent in meetings (which were observed) and meeting minutes and also by interviewing organizational actors who were involved in these processes. Data collection resulted in documentation, particularly protocols of project meeting observations, interview transcripts, and archival data. These served as a basis for data analysis.
In order to validate the case descriptions, each case was returned to the companies for verification. Necessary corrections either were made in writing or were discussed in supplementary interviews, partly on site and partly over the telephone. After several iterations, the case reports were finalized.
4. CASE STUDY DATA
4.1. Case 1: Company A
Company A is a large supplier of electronic products in a B2B market. On one occasion, customers asked for a product that was similar to others the firm had already sold to the market but with reduced functionality and at lower cost. A competitor of Company A was already on the market with such a product, but due to changes at the competitor company, the customers had started to look for an alternative. When Company A asked one requesting customer within what time frame they would need the product, the answer was 6 months. Despite this timeline, which was tighter than usual, Company A decided to develop the requested product and to rival the competitor. The objectives of the new product development project (i.e., timeline, product cost, and product specifications) were set by the competitive product and, hence, were very clear.
Company A started to review existing product concepts that could meet the objectives, and it also started to search for ideas on potentially new product concepts. A sister firm of Company A, which is located overseas, had a potential solution at hand and strongly put it forward. The solution was based on an existing product, and the sister company's opinion claimed that only slight adaptations would be needed. As a consequence, development cost and time would be exceptionally low. In addition, the sister company was in favor of its solution because introducing it to the market would have raised its sales volume significantly. However, because Company A's knowledge was low about the product and beliefs prevailed that the solution needed larger changes, Company A decided to look for further solutions. It identified two more possible designs. One design would be based on a downsized version of a current product of Company A. Downsizing implied the use of hardware with lower performance. However, the design functionalities would still exceed those of the competitor's product. However, it was risky because there was limited experience on the interaction of this new hardware with the firm's software packages. The second design would be based on components that Company A already offered to existing customers, who configure products according to their own needs. For the focal project, Company A wanted to configure a product based on these components. With the adaptation of some components, a product could be configured that would partially match the competitor's product but could match 95% of the customer requirements.
Product development was started by assigning three design teams, each team developing a design alternative for a complete product. One team was located overseas, the other two were colocated in Germany. In the course of development, the three teams communicated with each other on possible solutions and product family consistency; comparisons of timeline, effort, and risk for necessary adaptations; or additional markets for these new products. The communication with the overseas team was a challenge due to geographical distance and the time difference. During the investigation, the solution of the sister company overseas proved unfeasible, while unexpectedly, both solutions of Company A could be realized. The project even yielded two products that had such different solutions for the functionality needed and specialized in different sections that allowed differentiated positioning in the market; that is, Company A can now target different market segments. Therefore, it was decided that both solutions would be introduced into the market.
4.2. Case 2: Company B
Company B is a firm with 700 employees that develops and manufactures tooling equipment. In one case, it targeted a product that was similar to a competitor's. Differentiation to the competition would be higher quality and with a few more features. The design and technical solutions would be based on those that were incorporated in the competitor's product.
Company B launched a project and set up a design team that would dedicate about 60% of its time to this project. Soon after project start, the team had two ideas: an incrementally new one and a radical and very different idea for the design of a central product component. The idea seemed not only radical but also risky, and team resources were limited. Hence, Company B decided to spin three of the six team members off. They would work on the radical design alternative for 8 weeks and in parallel to the remaining team. The spin-off team consisted of a design engineer, an electronic specialist, and a test specialist. They were freed from all other responsibilities during that time and dedicated 100% of their working time to developing the radical design alternative. In this way, capacity was increased. The remaining team, however, pursued the incrementally new and low-risk solution, mainly as a fallback option. Even before the 8 weeks were over, the spin-off team understood that the radical solution was likely to work. After 8 weeks, all of the necessary testing had been carried out and the feasibility proven. The team and solution were reintegrated into the initial design team and product, and the conservative solution was dropped. All the team members were enthusiastic about the new solution, because it technically outmatched the conservative solution and offered the customer new features. However, requirements for production were more demanding, which posed further challenges.
Communication between the two parts of the team during the 8 weeks was informal and intense. Despite a temporary separation of work, the employees kept their workplace desks situated next to each other in an open-plan office. Curiosity and interest in the progress of the radical solution sparked chats in the hallway and over coffee breaks. Soon after the insight was gained that the radical solution was likely to work, the project leader of the initial team took part in the meetings of the spin-off team. This eased the reintegration of the team as a whole and the component after the 8-week timeline had passed.
4.3. Case 3: Company B
In another case, Company B wanted to significantly improve a component (i.e., an engine) of one of its products. The specifications to meet were very challenging; hence, the firm decided to start an advanced engineering project. It was staffed with an engine specialist, a design engineer, a test specialist, and a person responsible for product cost. When Company B assembled the team, it selected people who were competent in their field, proactive, open-minded, and with a can-do attitude. In the team, the division of tasks was unconventional. With the exception of the cost engineer, all of the others took and performed tasks regardless of their specialization, and all of the team members worked on all of the ideas. There was no assignment of people to ideas. The team had one project room, where the members did not work, but where they visualized the information and insights gained in the course of the project and where they met once a day to communicate with each other. All the team members were staffed 100% to the project, were very ambitious, and put in a lot of overtime. The project duration was about 6 months. Company B set the first milestone 8 weeks after project start. If at that point there had not been any promising design alternatives at hand, the company would have halted the project. Prior to project start, Company B planned how to design the project and also the resources to allocate. Requesting additional resources would have been possible if it helped to significantly advance the project, but this eventuality did not occur and the original budget was adhered to.
At project start, the team did not know of any engine that would meet the specifications set; that is, it was a challenging and risky project with a very tight timeline. Hence, the company wanted to try an unconventional way of new product development and selected the set-based design approach. It understood that it was less risky to test and, thus, to gain an understanding of the concepts in early development phases. To create design alternatives, the team conducted a number of TRIZFootnote 2 workshops, which yielded about 100 different design ideas. They were all visualized on a wall. For about 50 ideas, it was obvious that they were utopian. Because it was impossible to decide which solution of the approximately 50 remaining ideas would be the best and because it was impossible to further develop all 50, the team carried out some preliminary investigations and deselected stepwise 30 ideas that seemed to be the least promising. Then, the same team that had generated the ideas in the TRIZ workshops did more intense theoretical evaluations with models, calculations, simulations, and so on, in order to understand whether the remaining 20 ideas were feasible or not. Activities to test for feasibility were chosen dependent on the respective design idea, because each design idea posed different questions (had a different key parameter to be tested) that would reveal knowledge about the idea's feasibility. Company B generated all of the test reports, and in addition, there was extensive documentation on the experience gained during the project. To document project knowledge and experience, the team used OneNote, a Microsoft computer program for free-form information gathering and multiuser collaboration. In the course of the project, a number of solutions proved to be unfeasible and were dropped. Up to the point where the subsequent steps were accompanied by large resource investments (personnel hours, cost for making dies, etc.), about four designs were left. To further develop all of them would have been very resource intensive. Hence, the firm decided to select only one design for further development. The team selected the design members thought was the best solution, according to their knowledge at that point in time. At this point, the firm deviated from a pure set-based design by selecting from among the remaining four designs without establishing feasibility.
Finally, the team reached a solution that met the initial requirements by about 90%. In addition, the requirement specifications had changed during the project, and the component did not meet these new requirements. Hence, the component that was developed in the focal project was not included in the next product development project as had been foreseen at project start. However, a similar advanced engineering project started soon after the focal project was finished. The findings from the focal advanced engineering project were all incorporated and utilized in the new advanced engineering project, which eventually led to a product development project. The product of this project has now been introduced into the market.
4.4. Case 4: Company C
Company C is a small firm that develops and manufactures customized machines for the chemical industry. During a time of economic downturn, Company C was experiencing a low order intake, so the firm decided to use the situation as a chance to develop its core competencies and strengthen its innovativeness. Therefore, it launched an internal innovation contest open to people and teams from all parts of the company (not only research and development) with voluntary participation. Seven teams with four team members each signed up. A 1-day workshop kicked off the contest, followed by a 2-day workshop to refine the concepts and submit them to a jury that evaluated and selected the team with the best ideas. There were 8 weeks between the two workshops and participating teams could dedicate 20% of their work time to develop their ideas for the contest. While working on their ideas, communication within the teams was intense, but communication with other teams was rare. The innovation contest yielded a number of ideas that had previously existed only in the heads of employees but had never had a chance to be brought to the table.
The second workshop ended with a celebration and with the selection of the three most innovative teams, with the winning team awarded a financial prize. After the contest, all of the ideas were sorted and evaluated for further development. Company C's product consisted of three main components, and it turned out that among many other ideas, there were three alternative design ideas pertaining to Component A, three ideas to Component B, and four ideas to Component C. The company decided to discard the fourth idea for Component C, to assemble all other ideas in three product concepts, and to pursue these to develop three prototypes (see Fig. 1).
Components and their configuration of Case 4.
The firm decided to move ahead with all these ideas until feasibility was tested, although some experts expected that single solutions were not feasible. However, employees in favor of these solutions were persistent, and because the firm wanted them to test their solution and make their own experience, the concepts assessed as less feasible were not eliminated. The firm budgeted the project with all of the design alternatives to be developed until the end, that is, to be built as a prototype. Further, Company C assigned one team to each product concept with different team members and composition from the idea generation phase. All team leaders met monthly for formal communication on project progress and experiences. Because the firm thought it would be too expensive to leave interfaces open for different components to be mixed and matched, it assigned components to three product solutions. During the project, single component alternatives were dropped as they proved unfeasible. Components then had to be remixed and matched and, accordingly, interfaces adopted. Finally, two full product prototypes were built.
4.5. Case 5: Company C
The research and development department had ideas for three different technical principles and designs for a technical component. Due to limited resources, Company C decided to develop one idea with the research and development team and two ideas with students who were attracted by the offer to develop their master theses with these projects. The components were developed in parallel, and several prototypes were built. In the course of development, two concepts proved unfeasible from a physical point of view and were dropped. It was one of the master theses’ projects that proved functional and was implemented in the next product. Communication between the individuals involved was frequent and informal.
5. DATA ANALYSIS AND RESULTS
Based on case material and guided by this study's research questions, the data was analyzed (Miles & Huberman, Reference Miles and Huberman1994). A meta-matrix was developed and tabulated, thus condensing the data (see Table 2). To structure the data in the meta-matrix, criteria were developed along four categories: contextual data on industry and company, contextual data on the particular development project, data pertaining to Research Question 1, and data pertaining to Research Question 2. Subsequently, two other researchers as well as the author analyzed the data, identifying the similarities and differences of the different cases by comparing them.
Overview of case study data by industry and company size
5.1. Set-based design process: Establish feasibility for all design alternatives
First, we found in all five cases studied that the design process was characterized by an absence of a formal evaluation and selection stage as is found in traditional design processes. Contrary to economic reasoning, none of the cases studied displayed an explicit cost–benefit calculation to determine the optimal number of ideas. Instead, all of the projects incorporated all of the ideas previously created and eliminated them gradually. The firms’ underlying assumption was that organizational actors cannot reliably assign values to a design's parameters (e.g., judgments of cost, performance, or value).
Second, while case data revealed neither formal benefit analyses nor explicit costs analyses, this exploratory case study provides insights in a cognate field. In the cases studied, firms performed budget calculations for each case project. Partly, the number of ideas determined the project budget, accounting for the cost needed to develop several ideas. Partly, the observed firms calculated project budgets similarly to projects that develop comparable products and that select and pursue one product design only; that is, the calculations disregard the number of initial ideas to follow. While budgets were calculated for matters of resource allocation, a formal budget calculation was, contrary to the economic approach of determining the number of alternatives to follow at the outset, never part of the decision on how many alternatives to pursue (e.g., Dahan & Mendelson, Reference Dahan and Mendelson2001; Ding & Eliashberg, Reference Ding and Eliashberg2002).
With regard to budgeting, the data revealed two categories. First, the project budget was calculated as if only one idea was selected and pursued (Budgeting Category 1), based on the assumption that a higher investment at the beginning of a project due to the feasibility tests for a number of ideas will pay off because of significantly less rework later on, as in Case 2. Second, the firms studied allocated project budgets based on the number of alternatives to be pursued, as in Case 4 (Budgeting Category 2). In Case 4, the budget was multiplied by the number of ideas. In the course of the project, however, design alternatives were discarded before being fully developed. Thus, the total budget was not used, and the firm experienced and learned that a budget lower than a typical project budget times all ideas is sufficient. Overall, the difference to budgeting of the traditional design practice is that it cannot be assumed that all of the ideas will need development work until the end. Instead, a gradual weeding-out of alternatives due to proof of unfeasibility, convergence, or the merger of several alternatives is considered (Ding & Eliashberg, Reference Ding and Eliashberg2002).
5.2. Organizing for set-based design: one-on-one or one-for-all
Analyzing all five cases, three decision criteria on how to assign alternative designs to teams emerged (see Table 3). The first decision criterion refers to the number of alternatives to be developed per team, where two groups of cases emerged. In Cases 2, 3, and 5, firms assigned one team to pursue all of the alternatives and fill the knowledge gaps pertaining to the design alternatives’ feasibility. Only limited formal communication was needed to exchange test experience and results, as teams exchanged this knowledge naturally and as certain calculations or tests were all performed by the same specialized person. In Cases 1 and 4, firms assigned one team to each design alternative. In these cases, the cost of coordination was higher because formal communication between the teams was necessary to compare design alternatives and, foremost, to transfer beneficial features between design alternatives or possibly to merge design alternatives.
Team design options and parameter values for assigning alternative design ideas to product development teams
The second criterion pertains to the involvement of purely internal or a mixture of internal and external resources. While using internal resources was the default option, the case study firms’ engaged external resources to develop selected design alternatives if they faced tight resources, as can be seen in Case 5, where masters’ students were engaged. Potentially, firms can also contract external engineering offices. This is in line with Dahan and Mendelson (Reference Dahan and Mendelson2001), who describe a case where firms delegated development activities externally depending on the relative expertise and competence available internally. Engaging external sources also allows the loosening of organizational constraints, which Srinivasan et al. (Reference Srinivasan, Lovejoy and Beach1997) acknowledge. These authors argue that the constraints may limit the number of concepts carried forward, such as headcount constraints, to proceed on multiple fronts.
The third criterion for the assignment of design ideas to teams refers to the relationship among teams that can be collaborative or competitive. This study revealed that all of the firms observed followed the reasoning that development efforts are most efficient in a collaborative setting. The case study firms sought to integrate and to converge the knowledge that was created through the initial idea generation, and such conversion is only possible through collaboration. Competing teams had no incentive to share knowledge, an antecedent to knowledge integration. Moreover, an interesting insight is the exception of the ideation contest of Case 4, which targeted the creation of alternative design ideas. Subsequently, however, the development efforts and/or proof of feasibility were performed collectively in Case 4. This data supports Girotra et al. (Reference Girotra, Terwiesch and Ulrich2010) findings on idea creation, where the best results were attained when individuals first worked independently and then worked together, enhancing their ideas in a second stage. Hence, firms might attain the highest new product development performance if they frame the preceding ideation phase as a competition and initiate an ideation contest. Subsequent development and testing for feasibility can then be framed as collaboration, where teams communicate and act jointly to develop an excellent solution or product. As this case shows, firms can potentially deploy hybrid forms of collaboration beyond pursuing team assignments to either one or the other parameter value. This is also apparent for the location of resources where assigning external teams not only is possible but also may lead to extensive learning and knowledge creation outside but not inside the company. This is critical because product development capabilities are often part of firms’ core competencies and should be retained (Gray et al., Reference Gray, Runcie and Sleeman2015) and developed internally (Prahalad & Hamel, Reference Prahalad and Hamel1990).
In sum, the research questions can be answered as follows: first, in the cases studied, firms refrained from determining the number of alternatives to pursue in projects conducted with the set-based design practice. Instead, they pursued all of the ideas generated, and the larger the number of ideas generated, the faster the set of ideas was narrowed by testing for (un)feasibility. Second, the more important the time-to-market, the more likely the firms studied were to assign one design alternative per team, where teams worked in a collaborative manner. Alternatively, firms assigned all of the alternatives to one team.
6. DISCUSSION
This research contributes to a better understanding of design processes and design organizations when firms design new system architectures with the set-based design practice. For system architecture design, three elements are decisive, particularly in the early design stages, where the foundation for the design project's performance is built (Jankovic et al., Reference Jankovic, Holley and Yannou2012): the approach to product design, design process, and design organization. The literature has studied all three elements for the traditional design practice. The literature on set-based design, however, has left unanswered questions regarding design process and organization. Addressing this gap, this research sheds light on the following research questions: how do firms determine the number of design alternatives to pursue with set-based design; and how do firms assign teams to the development of design alternatives with set-based design? Thereby, the scope of this research pertains particularly to the third principle of the set-based design practice, that is, establish feasibility before commitment (see Fig. 2).
System architecture design by set-based and traditional design practice.
With regard to the first research question, this study reveals that in the cases studied, firms do not determine the number of alternatives by a cost–benefit calculation. Instead, the number of alternatives was determined by the number of initially created ideas, where the purposeful and directed creation of knowledge about the feasibility of each idea yielded a high rate of elimination, particularly at the beginning of the process. Overall, the cases studied display an absence of a formal, discrete idea evaluation and selection stage.
Addressing the second research question, this study identifies two approaches: either one team pursued all of the ideas or one team was assigned per idea dependent on the product's complexity, innovativeness, and the criticality of the time-to-market. If resources were constrained, external teams were engaged. Regardless of involving external resources or not, the engaged teams collaborated rather than competed in the development of design alternatives. However, following a competitive strategy in the preceding ideation phase seemed to yield higher performance.
Moreover, this exploratory case study provides insights not only on the design process and the design organization but also on relationships between product design, design process, and design organization (see Fig. 3).
Relationships between elements of set-based design practice (third principle: establish feasibility before commitment).
This study reveals no solid relationship between the initial number of design alternatives and the budgeting category (a relationship between two design process aspects). Instead, we observed, on the one hand, that cases with a broader set-based design scope (i.e., set-based design deployed to the whole product system) considered the number of design alternatives in their budgets (Budgeting Category 2) for designing a system architecture and assigned one team per design alternative. On the other hand, we observed in this study that in cases with a more narrow set-based design scope (i.e., set-based design for a product component) for designing a system architecture, firms calculated the respective budget as if a single design alternative was selected for further development (Budgeting Category 1), thus disregarding the number of design alternatives to be pursued.
Further research on system architecture design, however, is needed to better understand the details of this relationship. Here, we can only speculate. For example, to benefit from innovations, time-to-market is decisive. If the set-based design scope is broad, uncertaintyFootnote 3 is high, and gaining certainty by filling the respective knowledge gaps might take too long when assigning one team to all of the design alternatives. However, where the design alternative assigned to a team proved unfeasible, the teams had to be disbanded. To study leadership and motivational issues related to such project termination would be insightful. It was, however, beyond the scope of this research and could be addressed by future research. Overall, the narrowing process deserves further study because the literature to date has not explicitly discussed it.
In addition, the cases revealed that firms can be more innovative when designing system architectures by set-based design. While the literature refers to set-based design as a development approach that yields higher efficiency (e.g., Ward, Reference Ward2007; Raudberget, Reference Raudberget2010), this study shows that effectiveness (i.e., innovativeness) can also be increased. The cases in this study reveal that set-based design can also be used when targeting a product for radical advances. The engineering team developed multiple alternatives, often with a conservative (i.e., workable) solution (e.g., an existing or a similar design), which acted as the fallback design. This allows a company to be more innovative with low risk, as was observed in Case 2. This is in line with Ford and Sobek's (Reference Ford and Sobek2005) arguments and Raudberget's (Reference Raudberget2010) findings on an increase in firms’ innovative capability when applying set-based design.
Finally, empirical data on set-based design draws almost exclusively on the same single case study of Toyota (Liker et al., Reference Liker, Sobek, Ward and Cristiano1996; Morgan & Liker, Reference Morgan and Liker2006; Kennedy et al., Reference Kennedy, Sobek and Kennedy2013) or its close supplier Denso (Sobek et al., Reference Sobek, Ward and Liker1999; Meijer, Reference Meijer, ElMaraghy and ElMaraghy2006). Although there are no other references claiming Toyota's deployment of set-based design, there is evidence for the diffusion of set-based design in Europe. In particular, Raudberget (Reference Raudberget2010) has reported a case study on set-based design with four Swedish companies from the automotive, electronics, and heavy engineering industries. Overall, this research broadens the empirical base of set-based design applications to help the understanding of how set-based design is implemented in organizations other than Toyota or its supplier Denso and in settings that differ from these firms in terms of industry, firm size, or product complexity. This case study has limitations. The firms in this case study had just recently started to develop system architectures by deploying the set-based design practice, and they basically deployed the third principle of set-based design rather than the set-based design practice in its full scope. However, this case study does provide empirical data on set-based design practices deployed in firms much smaller than Toyota, firms from nonautomotive sectors, and products that are less complex than a car or a large car component or module.
Anja Schulze is a Professor of technology and innovation management at the University of Zurich. She also heads the Swiss Center for Automotive Research (Swiss CAR). Dr. Schulze previously worked as a Senior Researcher at ETH in Zurich. Her research interests include the creation of knowledge in the course of new product design, lean product development, and the efficiency of design processes. She has visited Japan several times and conducted several research visits to Toyota, Denso, and other firms practicing set-based design in Japan.
