1. INTRODUCTION
Large corporations are facing increased competition in the global market place, and their engineering departments have to constantly improve their work strategies to produce higher quality products in less time. Product developmentFootnote 1 teams are therefore pushed to enhance their practices to match current industrial trends in terms of multidisciplinary involvement, integration of tools and processes, and worldwide distribution of partners and stakeholders. To meet the implicit requirements imposed by “global” teamwork, meetings in the workplace have therefore multiplied in various forms over the years, but most stakeholders involved frequently have concerns as to their usefulness (Little, Reference Little2004). There have been a number of studies of face-to-face meetings spanning many research areas, but it is of interest that the topic has never, in itself, drawn much attention in the engineering design community. The work presented in this paper focuses on a specific type of meeting, namely, design reviews. These meetings are key elements of the design control process and are implemented across product development activities to assess progress and verify the quality of the work achieved. Design reviews in the aerospace sector are highly structured to follow precise company guidelines imposed by the international standard IEC 1160 (International Electrotechnical Commission, 1992) and adopted by national standards institutions, for example, BS 5760-14 (British Standards Institution, 1993) and CSA 1160-96 (Canadian Standards Association, 1996).
In practice, they are events in the product development process (PDP) in which, among other things, key collaborative decisions and their rationale are made explicit (Huet et al., Reference Huet, Culley and McMahon2004). Formal representations (or models) of the PDP can be found in abundance in engineering research literature (for a complete review see Wynn & Clarkson, Reference Wynn, Clarkson, Clarkson and Eckert2005). In the widely used stage–gate process defined by Cooper (Reference Cooper1993), a gate is a decision point that divides the PDP into discrete stages. The associated stage–gate model therefore explicitly places design reviews, also referred to as gates or milestones, across a stage-based view of the PDP. Of course, the number of milestones varies from one company to another. Phillips et al. (Reference Blessing and Chakrabarti1999) carried out a study with six different companies. These companies had adopted formal processes divided into phases or gates ranging from 4 to 10. According to the findings, it was established that a higher number of gates in the design and development stages improves the ability to review product cost and performance. In contrast, too many reviews can quickly become a counter productive issue for the project team, and there is therefore a need to balance and optimize the control process by operating with appropriate cross-functional teams, involving the correct variety of stakeholders over the life of a project (Phillips et al., Reference Phillips, Neailey and Broughton1999). Figure 1 is based on the stage–gate decomposition of the PDP and also illustrates the iterative nature of the informationFootnote 2 flowing between tasks in a concurrent engineering approach (Womack et al., Reference Womack, Jones and Roos1990; Clark & Fujimoto, Reference Clark and Fujimoto1991) to product development.
A stage–gate decomposition of the product development process (PDP) in which information flows between tasks are represented according to concurrent engineering practices according to Fortin and Huet (Reference Fortin and Huet2007).
The finalized information exchange flows depicted in Figure 1 are typically embodied in what engineers call “deliverables.” The effective management of the uncertain or incomplete information flows that occur between the preliminary information and the finalized information exchanges, illustrated in this figure, is a complicated issue for the improvement of information and knowledge services for engineers (Krishnan et al., Reference Krishnan, Eppinger and Whitney1997). Within this generic view of the PDP, design reviews provide a unique “information synchronization” point in the development of a product where the manufacturer and its suppliers can share information about the design and collaboratively evaluate the progress. The storage and archiving of the information and the subsequent knowledgeFootnote 3 generated during this type of event is increasingly important, and has to be considered as a major issue in the development of information and knowledge management tools for engineers (Bradley & Agogino, Reference Bradley and Agogino1990; Court et al., Reference Court, McMahon and Culley1996; Marsh, Reference Marsh1997; Boston, Reference Boston1998).
Although the design review process has been studied extensively, as discussed briefly above, little has been done to investigate the efficient capture of the contents of design reviews. The formal record of the discussions that have taken place during the meeting is normally embodied in the “minutes.” The literature associated with minute taking is very general and comes from the management sector; Weynton (Reference Weynton2002), Streibel (Reference Streibel2003), or Tropman (Reference Tropman2003) are good examples. Hence, to further develop a precise understanding of minute-taking practices in engineering design, a short questionnaire was distributed in 2005 to 5 industrial contacts in major aerospace companies who then forwarded the message to colleagues working in the same sector. The survey eventually reached 10 different aerospace companies and suppliers based in Canada and in Europe, with the aim of providing the authors with background on current practices and pitfalls of minute taking in the aerospace industry. Some 50 engineers replied to a set of 17 questions. Most of the respondents studied engineering in the United Kingdom, Canada, or France, and are now practicing engineers with an average of 15 years' experience in the aerospace sector. Overall, the primary activities in which the respondents are involved can be clustered as follows: 46% have a management role, 40% a design role, and 14% a manufacturing role. It is not the intent of the authors to detail the results of the survey; these are available in Huet et al. (Reference Huet, McMahon, Sellini, Culley and Fortin2006). However, the results are very relevant to the thrust of this paper and unequivocal; namely, engineers learn to take minutes by experience and only truly value the actions list, the practical side of minute taking. In addition, they respect the role of meeting records in the design process but are not given the right tools or training to take full advantage of the information richness of design reviews.
From these preliminary observations on design reviews, the essential issue that stands out in the improvement of information and knowledge management practices for engineers is “How is it possible to record aerospace design reviews to capture the important knowledge elements for further reuse?” This question has therefore guided the research reported in this paper. The following sections propose a fresh understanding of these formal meetings based on industrial and academic case studies in the aerospace domain, and outline a framework so that design reviews can efficiently support the product development environment described in this introductory section. The next section presents the research approach adopted to take advantage of the case studies. Section 3 reviews the relevant literature on the study of engineering meetings, which has led the authors to build a unified set of criteria for meeting analysis (Huet et al., Reference Huet, Culley and McMahon2004) and to propose a specific conceptual characterization of aerospace design reviews. Section 4 details the three meeting analysis tools developed for the purpose of this research: the transcript coding scheme (TCS), the meeting capture template (MCT), and the information mapping technique (IMT). Section 5 presents a selection of results obtained using these different analytical tools; the interpretations made are grouped according to three complementary perspectives on the design reviews monitored: communication processes, information processes, and knowledge loss. The final section will conclude this paper by outlining an overall strategy to record design reviews and suggesting possible enhancements to the analytical tools presented in Section 4.
2. RESEARCH APPROACH FOR THE DESIGN TRANSACTION MONITORING (DTM) PROJECT
Research in the field of mechanical engineering design has often focused on studying the act of designing (Hales, Reference Hales1987; Finger & Dixon, Reference Finger and Dixon1989; Minneman, Reference Minneman1991). Because of the empirical nature of the design research field and the way that the act of research may affect and influence the activity under study, it is of utmost importance for researchers to be clear about their methodology and the context from which the results have been drawn. This section therefore provides a retrospective description of the research methodology employed by the authors, which can be classified as a naturalistic observation approach (Minneman, Reference Minneman1991) integrating the “interaction analysis” method reported by Tang and Leifer (Reference Tang, Leifer, Waldron and Waldron1996) but also focusing on the various facets of in situ observations.
2.1. Rationale for the DTM research approach
Before detailing the research approach adopted, it is crucial to outline the context of the work that will be reported in the rest of this paper. The overall DTM project used three case studies to establish the approach and build up the core data. The work focused on the observation of design meetings and the case studies are briefly detailed in the following paragraphs.
2.1.1. Case study 1: Observation of a student design team at the University of Bath
The team chosen was composed of four undergraduate students who had the task of redesigning a portable Brinell hardness tester for a small company. Two academic supervisors supported the students. Ten meetings were monitored in total: 8 were recorded on audiotapes and 2 were simply observed by the author. This first case study was mainly an opportunity to organize a simple recording methodology for meetings and outline the foreseeable technical, organizational, and human issues linked to the monitoring of design meetings in situ.
2.1.2. Case study 2: Design reviews at Airbus UK
Two “real” design reviews were monitored on site at Airbus UK: a requirement review (RR) and a preliminary design review (PDR). Although the two meetings involved engineers from the same department, these were related to different aircraft programs. The detailed data collection taken from these two reviews provided a unique insight into the industrial realities of the aerospace design control process. The two Airbus UK design reviews were recorded on audio tapes and transcribed completely by the authors. Although a number of companies offer transcribing services, the level of specialization of the vocabulary employed and the number of abbreviations used in the discourse did not encourage the pursuit of this service.
2.1.3. Case study 3: The Centre for Aerospace Manpower Activities in Quebec (CAMAQ) Project at the École Polytechnique de Montréal
Fifteen graduate students participating in the CAMAQ Project, a large-scale aerospace design effort, were monitored during the whole length of the project in 2004–2005. This hands-on project was developed with CAMAQ, IBM, and three large aerospace companies based in the region of Montreal: Bell Helicopter Textron Canada, Bombardier Aerospace, and Pratt & Whitney Canada. This unique program is offered at the École Polytechnique to students enrolled in an aerospace engineering master's degree. The project involves the redesign of an aircraft engine pylon to enable the retrofit of a new engine and is controlled by a design review process, in which a team of industrial and academic experts review the design achievements presented by the graduate student team. To accomplish their task, the participants use a dedicated workspace, the CAMAQ Laboratory, which offers access to state-of-the-art digital mock-up and product life cycle management (PLM) technologies. The monitoring of this project resulted in the acquisition of a set of four design reviews: the RR, the concept review (CR), the PDR, and the critical design review (CDR). These were all videotaped and a complete archive of all the documentation generated during the project was also kept.
Figure 2 depicts the overall DTM research approach in which the three aforementioned case studies were involved. The methodology can be divided into three cyclic motions: “explore,” “tune,” and “interact.” The three DTM research cycles constitute a naturalistic observation approach and are, hence, mainly descriptive. They effectively integrate the interaction analysis method in the “interact” cycle (Tang & Leifer, Reference Tang, Leifer, Waldron and Waldron1996) but also focus on the various facets of in situ observations. Here, the observation step can take place either in a mockup environment (e.g., case studies 1 and 3) or the researcher can observe the design team in their work environment (e.g., case study 2), which is evidently a configuration more complicated to set up (Blessing & Chakrabarti, Reference Blessing and Chakrabarti1999). The exploration part of the research process is probably common to most disciplines; it is about positioning the work in accordance to past research (“research”), gaining an overall view of the domain of study (“synthesize”), and appropriately using past findings in a new context (“develop”). The tune cycle, in Figure 2, has been represented as a smaller gear than the two others. This illustrates a higher iteration speed of the process, where the theoretical techniques and approaches developed from the “explore cycle” (“develop”) need to be adjusted (“adjust”) to the observations (“observe”) made in the “interact cycle.”
The design transaction monitoring (DTM) research approach decomposed into three cycles.
2.2. Customizing the research methodology for each case study
In practice, the three case studies monitored during the DTM project fulfilled different objectives.
• Case study 1 helped to adjust the monitoring techniques using simple recording equipment. Here, the methodology was purely explorative and the naturalistic observation was essentially directed toward the understanding of participant behavior when monitoring equipment is introduced in the work environment.
• For case study 2 the data were collected on site with engineers working on real projects. Of course, the use of complete recording equipment was limited to audio only. In addition, the “intervene” element was forced out of the research approach to avoid disturbing the engineers. The research methodology was therefore once again very descriptive, but with a distinct objective: to collect data in situ from a real design review situation. The data collection taken from these two aerospace design reviews is quite unique, and the negotiation and management of such a feat was, as one can imagine, a long, difficult, but ultimately rewarding experience.
• Case study 3 provided valuable analytical data and a setting where the three DTM research cycles could take place. The monitoring equipment was complete with cameras to record the transactions. Here, the researcher was not only an observer but also a participant (fuel line specialist). This double role provided an ideal opportunity to gather all the necessary data and to gain a deep understanding of the engineering issues faced by the project team. In addition, some of the analytical tools developed during this research were trialed by other members of the project team to complete the “interact” cycle. In this case, the research approach can clearly be considered as “action research” (Reason & Bradbury, Reference Reason and Bradbury2001), where the researcher was part of the team of individuals under study.
Table 1 summarizes the role of each case study in the overall DTM research approach by outlining the details of the meetings observed (number of meetings involved per case study, average number of participants, and average duration), the research objectives, the analytical tools developed and used to acquire the data (TCS, MCT, IMT), and the research cycles of the DTM approach involved in each case study.
Summary of the three case studies
The three analytical tools (TCS, MCT, and IMT) developed during these case studies will be described in Section 4. In the interim, it is important to outline the theoretical understanding of design reviews based on the “explore” cycle of the DTM project. The next section will therefore present a review of literature on engineering meetings and the concepts used in an original framework to understand aerospace design reviews from an information process perspective.
3. DESIGN REVIEWS: A SPECIFIC TYPE OF ENGINEERING MEETING
Two main approaches have been used by researchers studying meetings: trying to understand what goes on during meetings, and creating tools to facilitate them. The survey results reported in the introductory section suggest that the “minutes” document, which constitutes the formal records of meetings, is often very limited in the extent to which it captures the information exchanged. New means are therefore needed to capture the design knowledge and experience generated during these collaborative situations. To further the understanding of the design review event, this section will develop a conceptual representation of the information processes typically involved in an aerospace design review. First, an overview of related research in the field of engineering meeting analysis will be presented in the following paragraphs.
3.1. The study of engineering meetings
Across the literature dealing with meeting analysis, six research teams the University of Michigan, Project Nick, Projet Eiffel, the Xerox Research Centre, the Knowledge Media Institute (KMi), and the International Computer Science Institute (ICSI) have been studied in detail based on the relevance, completeness, and rigor the work reported. Most of these research teams include experts from both engineering and human sciences, and have produced a number of interesting joint publications, for example, Reitmeier et al. (Reference Reitmeier, Chiu, Kapuskar and Wilcox1999), Morgan et al. (Reference Morgan, Baron, Edwards, Ellis, Gelbart, Janin, Pfau, Shriberg and Stolcke2001).
A detailed analysis of these research programs has been summarized in Table 2. Overall, certain similarities were noted. The common goals driving these projects can be summarized as follows:
• the creation of collaborative tools to enhance meeting facilities (ICSI, Xerox, KMi, Project Nick),
• understanding how engineers work/think/operate in a collaborative environment (University of Michigan, Projet Eiffel, Project Nick), and
• the facilitation of meetings to avoid failure (Project Nick, KMi, ICSI).
In Table 2, the “area of research” column shows the engineering discipline at the heart of the research observations; the “research objectives and outputs” reviews the overall aim of the research team and summarizes the achievements reported in the literature. The last two columns depict the research environment (“type of meeting”) and approach (“research approach”) taken by these research teams. The “meeting type” column shows that most of the teams have studied meetings in their domain of research, that is, computer science or software design. Only the KMi team used a different area of study (consulting company on governmental policymaking). A majority of the case studies (University of Michigan, Project Nick, Xerox) were directed toward meetings held in the initial stages of the design process (prior to the specification of the requirements) where exploration and brainstorming are key activities. Closer to the DTM case studies, Projet EIFFEL studied technical review meetings in the software design domain, with an emphasis on problem solving and decision making. However, these meetings differ from aerospace design reviews in many aspects, for example, they are not guided by international standards, they do not involve multidisciplinary teams (only software design engineers), they are relatively short, and so forth.
Presentation of the six key approaches to meeting analysis
The “research approach” column in Table 2 indicates the various methodologies used by the six research teams to complete their studies, based on the classification proposed by Minneman (Reference Minneman1991). The focus of the six research teams was on the development of software tools to support collaborative activities; thus, most have concentrated their efforts on a computational approach including variable levels of prescriptive research techniques to validate their prototypes. Nevertheless, most of the teams have spent some time observing meetings in a descriptive approach prior to the development of computer tools, especially in the case of Xerox and ICSI. Protocol analysis has also been a source of data for some of the teams, that is, University of Michigan and Projet EIFFEL. Because of the objectives of the research and the nature of the case studies, the DTM approach, as described in Section 2, differs from these approaches as it essentially focuses on a naturalistic observation methodology; here, the development of software tools to support design reviews was not the priority.
Based on these past research projects in the field of meeting analysis, one of the important practical aspects for an efficient study of spoken discourse is the use of verbatim transcripts. These enable the precise analysis of verbal transactions between participants based on a predetermined coding scheme. The TCS developed for the purpose of the DTM case studies will be the subject of Section 4, but its underlying coding criteria are the result of a comparative study of the terms used in the engineering domain for meeting analysis presented by the six projects described here. The comparative study, reported in Huet et al. (Reference Huet, Culley and McMahon2004), first exposed the lack of cohesion among the pool of concepts used by these research teams to describe and analyze meetings. The comparison of the concepts encountered in the literature was then undertaken to build a unified view on the topic of meeting analysis, also reported in Huet et al. (Reference Huet, Culley and McMahon2004).
Another view on design meetings, focused specifically on aerospace design review activities, was developed to represent the information processes that are expected to take place during the event. This information process-oriented model will be summarized in Section 3.2 to characterize aerospace design reviews.
3.2. Characterization of design reviews in the aerospace industry
A meeting can be seen as an activity where information elements are communicated, processed, and transformed (Kennedy et al., Reference Kennedy, Pinelli, Barclay, Pinelli, Barclay, Kennedy and Bishop1997). It therefore appears as an important step to build an overall model of the typical information processes that are expected to occur during aerospace design reviews. It is in this theoretical framework that the results from the DTM case studies can be analyzed. The following subsections will hence briefly present the communication and information processes expected to take place during a design review, an information process-oriented representation of an aerospace design review based on integration definition for function (IDEF0) modeling rules [National Institute of Standards and Technology (NIST), 1993], and a review of the key knowledge elements that can be generated during this type of meeting activity.
3.2.1. Communication and information processes in aerospace design reviews
The communication processes that take place during aerospace design reviews are typically synchronous and the essential communication channel, speech, is a proven knowledge production tool (Dong, Reference Dong2006), systematically augmented by a visual stimuli (three-dimensional models, sketches, documents, gestures, physical parts, etc.; Yen, Reference Yen2000). The event falls into the communication category of interface negotiation (Eckert et al., Reference Eckert, Maier, McMahon, Clarkson and Eckert2005) where engineers working on the same project are invited to share their opinions on predetermined issues. Participants are also required to report on their work as part of this formal problem handling situation (Eckert et al., Reference Eckert, Maier, McMahon, Clarkson and Eckert2005).
Using the information structure definitions outlined by Gardoni (Reference Gardoni1999), spoken information shared during meetings is typically of an unstructured nature, but in the case of design reviews the process is usually structured around formal textual and pictorial information inputs. Overall, a design review can be seen as an information process where the elements submitted, generated, and recorded can be clustered or categorized according to four product life cycle information types: the product, the process, the resources, and the external factors (PPRE; Labrousse, Reference Labrousse2004). Figure 3 proposes an IDEF0 parent diagram of the PPRE information elements related to the aerospace design review activity. This information blueprint is a first step toward a generic understanding of the information processes involved in aerospace design reviews. The overall representation is inspired from an analysis of procedures and guidelines for design review activities provided by the industrial partner for the DTM project and also from the feedback supplied by the industrial experts supervising the CAMAQ project (case study 3). The approach used to represent this functional model of an aerospace design review follows IDEF0 modeling rules (NIST, Reference Gero and Maher1993), taken from the perspective of the participants.
The integration definition for function modeling (IDEF0) parent diagram of an aerospace design review process.
The IDEF0 approach is well suited to organize information elements related to a process or activity according to the PPRE categorization scheme. Product and process information are embodied in the inputs and outputs of the activity box, resource information is placed in the mechanisms and controls, and the external factors information is usually represented as a control for a given process (or activity box). Figure 3 depicts the overall controls (C1–C4), mechanisms (M1–M3), and inputs (I1)/outputs (O1) involved in a formal aerospace design review activity [“review the design achievements (A0)”]. The following definitions are used:
• “Level of confidentiality” (C1) is a control for all design review activities with an impact on all the communication channels used (verbal, textual, or pictorial). The level can be set to personal, internal, or public, for example.
• “Synchronicity” (C3) is today an important factor that controls and dictates communication modes and technologies used during a design review. Formal design reviews are generally held in a synchronous manner and face to face. Nevertheless, the development of new information technologies (chat rooms, instant messaging, online forums, etc.) have enabled certain design review activities to be held in an asynchronous manner. A design review can sometimes be divided in submeetings to address specific issues or use videoconferencing technologies to connect remote teams; this has an impact on the resources involved and the location of the meeting can therefore be unique or multiple (British Standards Institution, Reference Gero and Maher1993).
• “Standards and company guidelines” (C4) are formal documents that control the overall activities involved in a design review.
• “Participants” (M1) can be viewed as mechanisms in the design review process. They influence the design review according to their role in the meeting, in the project team, or in the company. The individual expertise of the participants is also an important resource in meeting activities.
• “Resources” (M2) designates the various hardware and software used during the design review. These facilitation mechanisms include furniture, computers, videoconferencing software, and so forth.
• “Communication support” (M3) is the artifacts or behaviors used to support conversation during meetings. These can include presentation slides, sketches, drawings, objects, written notes, annotations, gesture types, and so forth.
3.2.2. A detailed IDEF0 information process-oriented model of aerospace design reviews
The IDEF0 model proposed in Figure 3 uses generic terms that enclose a variety of information elements: the “input information” (I1), “outputs” (O1), and “design control documents” (C2). These are described more precisely in the IDEF0 detail diagram of an aerospace design review process presented in Figure 4. Based on the general classification of design activities proposed by Sim and Duffy (Reference Sim and Duffy2003) and the theoretical understanding of the communication processes that can take place during the event (Section 3.2.1), the generic design activity represented by the box A0 can be further decomposed into three subactivities: “share information about the design” (A1), “evaluate the design” (A2), and “manage the design” (A3). The IDEF0 detail diagram in Figure 4 illustrates the relationships between the three main design activities (A1, A2, and A3) and their surrounding information elements, which constitute an aerospace design review. It is not the intent of this paper to fully describe the IDEF0 detail diagram; instead, Table 3 lists and describes six important elements of the model.
The integration definition for function modeling (IDEF0) detail diagram of an aerospace design review process.
Description of key elements of the IDEF0 detail diagram of an aerospace design review process
It is important to note that the O1 of the design review are not only the documents compiled in the review dossier (list of actions, minutes, deadlines, etc.) but also personal notes, and more generally the individual experience and knowledge retained by each participant. Although some of the rationale and lessons learned discussed during the meeting are recorded explicitly, most of this information, as shown in Figure 4, is part of an internal process and therefore only retained in the participants' memories.
3.2.3. Key knowledge elements generated during aerospace design review activities
Aerospace engineering design deals mainly with redesign activities (Ray, Reference Ray1985; Gero & Maher, Reference Gero and Maher1993) and has therefore recently focused on knowledge capture and reuse for an improved evaluation and control of their intellectual capital. Knowledge management (KM) can effectively be viewed as the management of the specific information elements that take part in the company knowledge process. Company knowledge conversion cycles (Nonaka & Takeushi, Reference Nonaka and Takeuchi1995), organizational knowing cycles (Choo, Reference Choo1998), and learning processes (Sim & Duffy, Reference Sim and Duffy2004) are some of the theories currently guiding KM practitioners. Based on these conceptualizations, design reviews are clearly events where knowledge externalization (making explicit knowledge that is hard to articulate, i.e., tacit knowledge) and combination (a combination of various explicit knowledge sources achieved through collaborative events) processes take place (Nonaka & Takeushi, Reference Nonaka and Takeuchi1995), and where participants have the opportunity to experience retrospective and in situ learning (Sim & Duffy, Reference Sim and Duffy2004).
From the analysis of the literature related to KM and the specificities of design review activities, these meetings are predisposed for substantial knowledge creating and decision making. Participants typically update their information about the design, discuss the rationale leading to a collaborative plan of action, and share past experiences. Four key elements (rationale, decisions, actions, and lessons learned) have therefore been singled out for the efficient knowledge-oriented recording of information exchanges during design reviews.
Rationale
Design rationale in its most general sense “is an explanation of why an artifact is designed the way it is” (Lee & Lai, Reference Lee, Lai, Moran and Carroll1996). However, this generic definition does not reveal the whole range of issues related to the topic. According to Moran and Carroll (Reference Moran, Carroll, Moran and Carroll1996), design rationale can be seen from many different perspectives: it could be the justifications for a designed artifact, a logical representation of the reasons for a designed artifact, a methodology whereby reasons are made explicit throughout the design process, or it could simply relate to the complete historical documentation of a design and its context. Shipman and McCall (Reference Shipman and McCall1997) argue that there are three distinctive approaches to design rationale: the argumentation perspective, the documentation perspective, and the communication perspective. Argumentation aims to relate the reasoning an individual or a group of designers use to solve a problem. Documentation of the information about the design decision-making process is another meaning commonly given to design rationale where descriptive accounts of decisions are captured. Finally, naturally occurring communication between designers, such as conversations, is also a source of design rationale, but its capture is more difficult because of its lack of structure and its unpredictability. Based on this classification, design reviews are clearly events where the communication perspective of design rationale needs to be applied. Design rationale research is not new in the engineering world, but the issues that revolve around its capture, representation, and use are still in working progress (Bracewell et al., Reference Bracewell, Ahmed and Wallace2004). The need for efficient methodologies to capture and reuse design rationale is a priority within current KM strategies (Karsenty, Reference Karsenty1996). Pragmatic approaches to represent engineering design rationale often distinguish between rationale relating to process knowledge or product knowledge (Regli et al., Reference Regli, Hu, Atwood and Sun2000; Wallace et al., Reference Wallace, Ahmed, Bracewell, Clarkson and Eckert2005); process-oriented solutions try to map out the history of the design process, whereas product-oriented solutions (also known as feature-oriented solutions) work on the design space trying to represent how a specific feature of a product can be ensured on the design (Regli et al., Reference Regli, Hu, Atwood and Sun2000).
Decisions and actions
The study of design reviews will give insights on the rationale and the decisions leading to courses of action taken by designers and project managers. All the decisions made during a review will therefore be explicitly or implicitly translated into design definition or design management actions (Sim & Duffy, Reference Sim and Duffy2003). An organization can effectively be viewed as a network of decision-making processes, where compromising often takes place despite well-defined standard procedures (Choo, Reference Choo1998). Closer to the practical act of decision making, Badke-Schaub and Gehrlicher (Reference Badke-Schaub and Gehrlicher2003) have outlined certain patterns that can be found in design teams: cycles, which include a reiteration of partial sequences of procedure steps; sequences, which strictly follow theoretical decision-making models (clarification, search, analysis, evaluation, decision, and control); and metaprocesses, where the decision process is guided by a moderator. It will therefore be interesting to validate and compare the occurrences of the decision patterns with the work reported by Badke-Schaub and Gehrlicher (Reference Badke-Schaub and Gehrlicher2003).
Lessons learned
For the purpose of this research, a lesson learned can be defined as a formal explanation of the solution to a problem that occurred in a specific context where new knowledge or an adaptation of existing knowledge was employed. In an environment where most of the designer's work involves adaptive design (redesign), information concerning past designed products and processes is of great importance. Lloyd (Reference Lloyd2000) distinguished three types of experiences used in engineering to transform a set of requirements into a reality: individual, social, and organizational experiences. Individual experience builds the designer's expertise. Social experiences, such as meetings, are usually constructed in a collaborative environment where experiences are communicated and shared. Organizational experience is formalized through company documents such as procedures, product histories, lessons learned, and so forth. Current industrial practices suggest that documenting lessons learned is essential to help design engineers constrain the design space based on past experiences (Ward et al., Reference Ward, Liker, Cristiano and Sobek1995).
The four knowledge elements or concepts reviewed previously are at the heart of the IMT, detailed in Section 4.3, and are also central to the knowledge-oriented strategy for the efficient capture of design review contents, presented in Section 6.
4. A SET OF CUSTOMIZED TOOLS FOR DESIGN REVIEW ANALYSIS
To improve the understanding of the content of design reviews, three approaches have been created. The three tools or techniques presented here (TCS, MCT, and IMT) at this stage do not employ any automatic or computerized mechanisms; they do, however, propose a detailed and systematic methodology validated and refined by representative data taken from the DTM case studies. The tools were developed as the result of the iterative research process presented in Section 2. This section will focus on describing the tools and on the practical implications of using them for the DTM case study data.
4.1. The TCS
The transcribing and coding process described in the following sections are part of a qualitative research approach to discourse analysis. Before attempting to analyze meeting transcripts it is of great importance to adopt a coherent methodology that can be applied in a systematic way to recordings of design meetings. The essential part of creating this methodology was to determine the criteria under which transcripts will be analyzed. It is now essential to present the TCS, necessary to an organized, structured, and systematic coding methodology for meeting transcripts, based on the review of past research presented in Section 3.1.
4.1.1. Description of the TCS
Transcribing is a task repeatedly used in social sciences and linguistics, but it has not warranted the development of official and standardized conventions. Robillard et al. (Reference Robillard, D'Astous, Détienne and Visser1998) detail clear and reproducible methods for coding interventions, and the TCS is largely formatted along the same guidelines. The TCS is built around a structured transcript that uses specific transcription conventions (Huet et al., Reference Huet, Culley and McMahon2004) and eight codification elements developed from the detailed analysis reported in Section 3.1. As shown in Figure 5, the meeting transcripts record the verbal discourse of the discussions that took place at the meetings, showing the interventions preceded by the initials of the speaker, and followed by the time at which the intervention ended.
An illustration and explanation of the transcript coding scheme (TCS). Note that the interventions used in the transcript are fictional.
The rest of the TCS is used for in-depth analysis purposes. The format of this analytical tool is essentially a table, where each row is an intervention made by one of the participants. Robillard et al. (Reference Robillard, D'Astous, Détienne and Visser1998) define an intervention as “a statement made by a single speaker . . . a series of interventions made by different speakers is called an exchange.” In the coding scheme, the interventions can then grouped by exchanges. Of course, in certain specific cases different interventions made by a single speaker can be considered as an exchange (e.g., in the case of a presentation). The size of exchanges, in number of interventions, varies according to the coding intent. There are different types of possible interventions and exchanges in a design meeting; derived from the literature reviewed in Section 3.1, and adapted to match the specific characteristics of aerospace design reviews, the authors chose to use an adaptive set of attributes, detailed in Figure 5, for both the “intervention type” and the “exchange role” coding criteria. The six other coding criteria are unique to the DTM project and answer specific objectives of the research program outlined in Table 4.
Summary of the research objectives for each coding element in the TCS
a See Figure 5 legend.
b See text Section 3.2.1.
4.1.2. Validation and development of the TCS
The TCS was applied to case study 2; both design reviews recorded at Airbus UK (RR and PDR) were therefore transcribed completely, generating some 1000 transcribed interventions. The TCS then went through an iterative validation process before the stabilized final version, presented in Figure 5, could be established. The evolution of the TCS was guided by a number of factors: the relevance and interpretability of the coding element, and importantly, the reproducibility of the coding intent. The main changes involved the refinement of the artifact coding and the coding of the topic. The initial “topic coding” proposed tried to include the type of knowledge conveyed, but this was a significant issue and led to the elaboration of a separate methodology to track key knowledge elements in the transcripts, the IMT, presented later in Section 4.3.
Moreover, the time necessary to complete the TCS, including the transcribing process, does not make this tool appealing for more practical meeting capture applications. A more manageable approach to analyze design meetings as they take place was therefore developed by the authors: the MCT. This tool, detailed next in Section 4.2, uses a simplified version of the TCS coding criteria and bypasses the time-consuming approach of transcribing. Nonetheless, the TCS remains a thorough and comprehensive research strategy to understand the design transactions that take place during a meeting based on the transcribed account of the event; this will be illustrated by the selected results discussed in Section 5.
4.2. The MCT
The TCS provides an in-depth analytical tool to analyze meeting transcripts. However, the process involved to produce the results is time consuming, and can therefore only readily be undertaken in a research environment. The need to develop a simplified approach, where the data could be collected as the meeting was taking place, emerged as an important issue for the authors and their industrial partners involved in the DTM project. This led to the creation of a MCT, which was used and developed during case study 3.
4.2.1. Description of the MCT
The MCT presents itself as a table, where each row is numbered and corresponds to an entry used by the minute taker during the meeting. Figure 6 shows an extract of the final MCT featuring the first three rows.
The final version of the meeting capture template (MCT) used during case study 3.
The second column, “topics and actions” provides space for the observer to make a few notes, using his own words to describe the conversation topic and the related actions. Each row in the MCT therefore corresponds to a new conversation topic. The “who” column helps the minute taker to quickly track the involved parties in the conversation. The “what” column is a simplification of the exchange roles found in the TCS; here, only six core exchange roles were kept (exploring, decision making, evaluating, clarifying, informing, and debating). These are essentially related to the “information sharing” and “evaluating” activities, whereas “management” activities are easily tracked by looking at the parties involved and the topic of conversation. Finally, the PPRE coding element in the TCS has been simplified here in the “impact” column, where each conversation topic can be tagged according to whether it has an impact on the product, the process, or the engineering tools used by the participants.
The columns are derived from the TCS but have been formulated with a more streamlined terminology. These changes occurred because the finalized MCT was the result of an intensive validation process involving the participants of the CAMAQ project (case study 3) as summarized in the next paragraphs.
4.2.2. Validation and development of the MCT
During each design review in case study 3, three to four participants were chosen to use the MCT to provide analytical data for the authors' research and, to a certain extent, help capture the minutes of their meetings. Their feedback helped to enhance the MCT and draw guidelines for the development of minute-taking templates to be used in the industry. The participants chosen varied from one design review to another according to their involvement in formal presentation activities, but all were given a training session to familiarize themselves with the template before the meeting took place. The results from the MCT were compiled by the authors, and each entry from each form handed back was double checked by the authors using the video recording of the design review monitored. This helped to maintain a high level of consistency in the results throughout the case study, even when the MCT was modified based on user feedback. Ultimately, the videotapes also provided an efficient means to evaluate, after the meeting, the approximate duration of each entry in the MCT.
The MCT presented previously went through three development stages or versions. The first version was a direct simplification of the TCS based on what the authors believed could be tracked on the fly by a trained minute taker. In the first version, the “exchange roles” coding, was simplified in the MCT based on the results obtained in the TCS analysis. Three exchange roles were omitted: “digressions,” as they occurred very rarely during the design reviews monitored; “resolving problems,” because they were often confused with “debates”; and “managing,” as this exchange role can easily be traced by other coding elements such as the topic and the participant's role. The second version, nearly identical to the final version presented in Figure 6, saw the addition of a more specific classification of the participants involved and a dedicated space to note the actions related to the topic of interest. The new participant classification was a direct consequence of the observations made on the usability of the previous template; users were effectively very comfortable with the multiple-choice boxes offered in the “who,” “what,” and “impact” columns, and the “who” column was therefore expanded to provide more information on the participants involved in the conversation. In addition, at the request of most of the participants involved in the development of the MCT, extra space was included in the “topic” column to provide means to capture the actions associated to each topic.
This last point can be seen as the key move from an analytical tool toward a content capture template. Indeed, two research and development objectives were hidden behind the MCT: the simplification of the TCS to enable the analysis of design reviews on the fly, and the progressive evaluation of templates to capture the content of meetings in a structured and reusable format. The results related to the use of the MCT in case study 3 are reported in Section 5.
4.3. The IMT
The IMT was developed to answer the following research question: What are the important information elements that are not currently captured during design reviews? Indeed, the TCS had a different purpose and was not designed to efficiently track the key knowledge concepts detailed in Section 3.2.3. The IMT was ultimately used to track them, while also providing means to qualitatively and quantitatively measure information loss between a meeting (based on its transcript) and its formal historical record (the minutes). Information loss has been shown to be very significant; up to 75% of content can be lost during minute taking (Huet et al., Reference Huet, McMahon, Sellini, Culley and Fortin2006). The implications of this study in terms of knowledge loss are further discussed in Section 5.
4.3.1. Description of the technique
The idea of an IMT to measure the related knowledge loss was effectively inspired from the work carried out by Hoffmann (Reference Hoffmann1980). His quest for a suitable definition of the term “information” led to a comparative study of textual documents and their abstracts. Analysis of the material was based on a graphical representation of the information content. It is not the intention of the tool developed for this research to evaluate the whole information content of the documents; thus, the IMT, developed by the authors, focuses on the occurrence of four specific knowledge concepts reviewed in Section 3.2.3: decisions, actions, rationale, and lessons learned. The IMT is therefore used to compare two types of documents: the transcripts and the minutes of meetings. In simple terms, each portion of the text corresponding to a knowledge concept (decision, action, rationale, or lesson learned) is represented by a symbol in an information map. The symbols are clustered around focal points: the main meeting topics. The result therefore presents itself as a succession of network graphs centered and sequenced following the different focuses of the event as shown in Figure 7.
An extract of the information map for the minutes of a design review. [A color version of this figure can be viewed online at www.journals.cambridge.org]
To produce these information maps, two stages must be followed: the encoding and the mapping of the document. To encode a document, each information element expressed by a statement or a group of statements must first be highlighted in the original text and summarized in a register table according to the knowledge concept it conveys (decision, action, rationale, or lesson learned). Figure 8 presents an extract of an “action elements” register to illustrate such tables. The main topics of the document are also listed in a distinct register table and the encoding therefore involves the creation of five separate registers. Other details such as the number of words and the subsequent coding size are also recorded in the tables.
An extract of a register table for the action elements.
Then, for the information mapping, each row in each table is represented in the graph by a symbol specific to its knowledge concept and a number that relates it directly to its register row number. Figure 9 illustrates the coding scheme for the symbols. The size of the symbol reflects the volume in number of words of each information item and is relative to the overall size of the document.
An example of the information mapping coding scheme used to map the minutes of the requirement review (RR) in case study 2. [A color version of this figure can be viewed online at www.journals.cambridge.org]
To finalize the mapping, the topics are used as focal points and the items are connected around them in “threads” using the conceptual relations inferred by the text, as previously illustrated in Figure 7. The links or arrows in the graph simply represent the order in which the nodes (information elements) were highlighted in the document.
4.3.2. Validation and development of the IMT
The information maps generated by the IMT offer a visual representation of the information contained in a document; comparing the transcript of a meeting and its formal record embodied in the minutes using these maps provided a useful and simple approach to characterize the transformation and loss of information that occurred during the minute-taking process. The IMT was only performed on the transcript of the RR (case study 2). The main reason for this was the high quality of this transcript; indeed, only 1% of the total meeting time could not be transcribed. The minutes of the meeting, provided by the secretary of the design review, were completely mapped with the IMT, whereas only the critical topics in the transcript were mapped.
The IMT is still being refined by the authors, but to ensure that the results shown in this paper are consistent and reproducible, the encoding and mapping processes were undertaken by the same person and verified by a number of peers. The encoding step of the IMT is the more difficult part of the approach; an automated version of the encoding steps, that is, defining the decision, action, rationale, and lesson learned elements in the text, would require semantic capabilities that are still not available in related software applications. The encoding of the documents must therefore be performed directly by the user, and the following definitions for each knowledge concept were used to provide a consistent framework to perform this task.
Rationale: An explanation or statement of reasons. In the specific context of engineering design, a more precise scope of the definition has been defined in Section 3.2.3 (Oxford English Dictionary, 1989).
Decision: Making up of one's mind on any point or on a course of action. In the context of a collaborative event such as a meeting, the definition needs to be widened to include “the making up of a group's mind” (Oxford English Dictionary, Reference Finger and Dixon1989). This definition also highlights the link between a decision and an action, which can be loosely defined as the necessary steps to complete a predetermined task. In the context of a meeting, an action is usually described as a task to be accomplished by a specific person and in a determined time frame.
The term lesson learned was defined in Section 3.2.3 as a formal explanation of the solution to a problem that occurred in a specific context where new knowledge or an adaptation of existing knowledge was employed. The formalization of a lesson learned usually requires the definition of a number of elements, such as the problem elicitation, the story (background), the recommendations related to a past company experience, and so forth. In the context of the IMT, only a partial description of a lesson learned in its formal sense was considered to be a lesson learned element.
In the next section, the IMT applied to the RR recorded during case study 2 will illustrate how this technique can be used to characterize the loss of information from design reviews, ultimately resulting in organizational knowledge loss.
5. SELECTED RESULTS FROM THE CASE STUDIES
This section presents selected results from the DTM case studies according to three analytical perspectives: the communication processes observed, the information processes detected, and the knowledge lost from the meeting records. The relevant data was extracted from the recorded case studies using the three meeting analysis tools presented in Section 4. The design reviews from case study 2 were analyzed with the TCS and the IMT, whereas the design reviews from case study 3 were studied using the MCT. The selected results illustrate the considerable range of analytical capabilities offered by the tools developed for the purpose of this research. It is important to note that the two case studies (case study 2 and case study 3) involved in this collection of data were not used in a comparative approach but rather in a complementary approach. Indeed, the design reviews in case study 3, which involved graduate students, were considered comparable to industry practices by all the industrial observers invited to the meetings (two to three colleagues, i.e., industrial experts, were invited by the industrial supervisors to observe the design reviews to promote the project in their respective companies).
5.1. Design reviews: A communication process perspective
The observed communication structure of the recorded design reviews has been analyzed at different levels. The study of the role of the participants in both case studies clearly illustrates specific communication patterns for the meeting as a whole; the results show the predominance of interface negotiation scenarios such as “justifications” and “information requests” scenarios (Eckert et al., Reference Eckert, Maier, McMahon, Clarkson and Eckert2005) during design reviews. When the detailed structure of speech in a design review situation is considered, the intervention type coding element of the TCS has helped to outline an interesting trend in the structure of spoken discourse verified in both design reviews of case study 2: questions are often hidden in a more global statement and even when explicit they are only occasionally answered directly by a straightforward answer. This aspect explains the failure of certain established design rationale capture techniques, such as gIBIS (Conklin & Begeman, Reference Conklin and Begeman1988), when applied to spoken discourse. Indeed, these techniques are focussed on “question and answer” sequences aiming at unveiling the rationale in the conversation.
In order to analyze the underlying communication intent, the results from the “exchange roles” coding element in the TCS and the MCT have been studied in detail. Overall, the striking aspect common to all the design reviews monitored is the importance of “informing” and “clarification” exchange roles (these roles occupied 60–70% of the conversations). These results suggest the “sharing information about the design” activity in the information process-oriented design review model proposed in Section 3.2.2 is essential in the overall design review process. Of course, decision making, exploring, and evaluating are also key exchange roles observed during design reviews. Their variation in percentage of conversation time across different design reviews, as observed during case study 3, can easily be related to specific objectives of each design review type. Moreover, this means that the “evaluate the design” and “manage the design” activities proposed in the process-oriented model (Section 3.2.2) will see their importance vary according to the position of the design review in the product development process; in case study 3, conversations related to “evaluating” peak around PDR, whereas “decision making” peaks in the early stages of the design process.
In the study of communication processes, a specific process has been chosen for detailed analysis: decision-making patterns. This observation was made for case study 2. The data generated from the exchange role criterion in the TCS enabled to outline typical sequences of exchange roles prior to decision making; six main sequences of decision making have been unveiled and illustrated in Figure 10. These sequence patterns ultimately reflect a rational course of decision making with few conflicts of interest between participants (Badke-Schaub & Gehrlicher, Reference Badke-Schaub and Gehrlicher2003).
The essential decision-making patterns observed during case study 2.
5.2. Design reviews: An information process perspective
The design review model detailed in Section 3.2.2 has outlined the expected conceptual information processes that might occur during the event. This section will now supplement this theoretical view through the characterization of the information processes observed during case studies 2 and 3, using the TCS and the MCT, respectively.
The “origin of the topic of conversation” coding criterion in the TCS has further supported the qualification of the level of structure of the information exchanged during design reviews. In effect, the measures resulting from case study 2 indicate that 60–70% of the conversation topics are predetermined by the meeting agenda and the remaining topics of discussion are directly derived from these. From this study, the authors would also have forecasted a higher percentage of totally unexpected conversation topics in the early stages of the product development process, but the influence of the artifacts used in the conversations seems to play an important role in the structure of the information process.
The content of the information shared between participants, in case study 3, were very much in line with concurrent engineering practices. Design issues were at the heart of most conversations throughout the four design reviews monitored, with a peak at PDR. Management issues were dealt with early in the project (peak at RR), whereas manufacturing issues were only the true concern of the participants at CDR (with a critical low point at CR). The “domain of competence” coding criterion in the TCS has provided useful insights into the specific topics discussed during the design reviews monitored in case study 2. The results show how in both cases “project management and business” and “certification and testing” were topics at the forefront of the discussions that took place.
The study of the types of information exchanged during the design reviews of case study 3 has provided a unique illustration of the shift in balance between process and product information that occurs during the evolution of a design project. Process information dominates the topics of conversation in the early stages of the project and then slowly diminishes, whereas product information gradually increases to dominate the topics of conversation at CDR as shown in Figure 11.
The evolution of product versus process information across case study 3 (percentage of conversation time).
This study is unique in the sense that it actually provides figures based on case studies to support claims on the shift between process knowledge and product knowledge across the life of a project (Regli et al., Reference Regli, Hu, Atwood and Sun2000). Nevertheless, the overall results from the DTM case studies (case studies 2 and 3) show that the balance between product and process information remains within a 40–60% bracket. This trend means that, overall, process and product information are shared in large amounts across the PDP, and systems aiming at capturing this design information should focus on a hybrid approach (feature oriented/process oriented; see Section 3.2.3).
The study of the types of artifacts used during both design reviews of case study 2 clearly suggests that they are important elements that structure and focus to a certain point both the communication intent and the type of information exchanged between the participants. Artifacts used during a design review have definitely a key role to play in the elaboration of improved techniques for the efficient capture of meeting contents.
5.3. Detailed knowledge loss study for two critical meeting topics
Section 4 detailed and illustrated the IMT developed for the assessment of the loss or modification of specific knowledge concepts in the minutes of design reviews. This section will interpret the findings from the RR monitored during case study 2 to highlight the implications in terms of knowledge loss.
In the map for the “minutes” document, the most important topic in terms of number of words involved and highlighted information elements was topic 5, and its map was therefore completely detailed both from the minutes document and from the transcript. In the transcript, however, the two most important topics based on the same criteria were topic 5 and topic 4. This was quite a surprise as the minutes map suggested that topic 4 was not of great importance. Figure 12 compares both maps (transcript map and minutes map) for topic 4 and topic 5.
A comparative table of the information maps for topics 4 and 5 of the requirement review (RR) of case study 2. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Based on the information maps presented in Figure 12, topic 4 appears to have been badly recorded by the secretary. In the transcript map for topic 4 a high number of threads, many rationale, decisions, and lessons learned elements appear, but with few actions associated with these. In contrast, when comparing the maps for topic 5 it seems that the minutes give an accurate account of the discussions that took place. Looking at topic 5, a different observation can be made immediately: it seems that when writing up the minutes, the secretary “transformed” some of the decisions into actions. This is perfectly understandable as decisions can always be interpreted as actions.
The single most important difference between topic 4 and topic 5 lays in the actions: in the discourse, most of the threads linked to topic 4 do not contain actions, whereas it is quite the opposite in the case of topic 5. The resulting difference in the minutes' maps suggests that it is easier for the minute taker to record the meeting when actions are set out following the decisions. A number of meeting management strategies, for example, Weynton (Reference Weynton2002), Streibel (Reference Streibel2003), or Tropman (Reference Tropman2003), suggest that certain meetings need to be action oriented to become effective.
Another factor that might have an influence on the difference in the way minutes were taken for topic 4 and 5 is the “distance” observed between the conversations and the artifacts under review. Although none of the meeting analysis tools presented in this thesis are capable of providing data for this type of measurement, the observations made by the authors suggest that the participants held discussions closely related to the document under review in the case of topic 5. This most definitely facilitated the secretary's work as he had an explicit reference to action any decision made.
Finally, four comparative criteria to evaluate knowledge loss, outlined in Huet et al. (Reference Huet, McMahon, Sellini, Culley and Fortin2006), can be further illustrated using the examples provided in Figure 12.
• Volume and length: These two criteria help to express the importance of a topic, knowledge concept, or thread relative to the rest of the document. They were used as visual indicators of critical topics in the maps that warranted further investigations.
• Variety: From the examples given in Figure 12, it can be immediately observed that the richness of the text based on the discourse is lost; actions and decisions on one side and rationale and lessons learned on the other are very often merged or transformed.
• Order/sequence: With more data, research for typical patterns could be conducted. One of the conclusions based on this criterion is that in the document relating the discourse, rationale is given before or after the decision or action, whereas in the minutes the sequence is invariably rationale then decision. Overall, the unstructured and unpredictable nature of speech is well reflected by the visual sequence provided by the IMT.
The essential finding that has emerged from the knowledge loss study, detailed in this section, is the importance of turning decisions into actions. Indeed, the secretary seems more capable of recoding the associated rationale, lessons learned, and decisions based on an explicit expression of the action to be taken. The detail IDEF0 process-oriented model described in Section 3.2.2 accounts for the transfer of rationale and lessons learned between design review activities. The model, however, does not show these knowledge types transferred as outputs of the design review process. Hence, the final section of this paper will briefly outline the action-oriented recording strategy under development, where rationale and lessons learned are “forced out” of design reviews.
6. CONCLUSION AND FUTURE WORK
Only a limited number of observational studies in engineering have focused on a clearly identifiable type of meeting. The DTM case studies, however, chose to use a very specific and widespread meeting event in the aerospace industry, namely design reviews. Companies, using the widely accepted stage–gate approach to control their product development activities, implement design reviews with similar guidelines. In particular, they are guided by a number of formalized constraints, they follow a clear set of predefined objectives, they are a unique “information synchronization” point for all stakeholders involved in the development of a product, they are visible activities in business planning tools and documents across projects and companies, and they are at the heart of the collaborative decision-making cycle inherent to any product development process.
The research reported in this paper has been based on a naturalistic observation of engineering teams in a design review situation. A specific research methodology composed of three research cycles (“explore,” “tune,” and “interact”) was developed and provided a flexible framework for the following:
• a transparent and reproducible qualitative research process;
• the collection of data from different types of case studies (in situ or simulated setting, academic or industrial participants, etc.); and
• the development and validation of three research tools focused on the analysis of verbal transactions during design reviews: TCS, MCT, and IMT.
Based on this successful research process, a number of contributions answering the overarching research question “how is it possible to record aerospace design reviews to capture the important knowledge elements for further reuse?” were made. Key findings detailed in this paper include the following:
• the survey of current minute-taking practices in the aerospace industry (introduction);
• a conceptual understanding of the activity under study (Section 3); and
• characterization of communication and information processes, and knowledge loss (Section 5) based on the results generated by the TCS, MCT, and IMT (Section 4).
6.1. Implications for the capture of the content of design reviews
A number of results from the analysis of the case studies and the survey on minute-taking practices in the aerospace industry have helped the authors to establish an action-oriented strategy to improve the capture of key knowledge elements from design reviews. The strategy outlined in the next paragraphs follows typical steps used in the “knowledge engineering” domain to develop knowledge-based systems or practices (Liebowitz, Reference Liebowitz2001). The strategy proposed here (for the capture of the contents of design reviews) can be summarized from the secretary's perspective using the following three phases:
1. Knowledge acquisition phase: During the meeting, the secretary should focus on keeping track of the actions with their associated rationale or lessons learned. This knowledge acquisition phase of the strategy would see the secretary turn decision points into actions whenever possible, and at the end of the meeting sufficient time should be allowed so that each action can be reviewed in detail and agreed by all participants.
2. Knowledge representation and encoding phases: At the end of the meeting, each action (noted on a customized form) needs to be detailed and tagged according to the type of information it contains (product or process). After the meeting, the secretary would finalize the formal meeting minutes by seeking the approval of the authorities responsible for the design review.
3. Knowledge implementation and reuse phases: Once all the “action forms” are approved, these can be linked to one of the two engineering tools typically used to manage product and process information in a PLM environment. Actions tagged as “product information” could be inserted in the product structure tree managed by product data management systems, whereas those tagged as “process information” could be included in workflow management systems.
To date, this strategy has not been completely tested. The authors have essentially focused on the knowledge acquisition stage of the process. TCS, MCT, and IMT analytical approaches have been described in detail in the paper along with their validation processes. This has enabled a number of high-level views to be taken and a pragmatic approach to minute taking to be developed. The solution is embodied in a “design review capture template” that uses a format based on the MCT described in Section 4.2. A preliminary version of the template is shown in Figure 13.
An extract of the preliminary version of the design review capture template.
The feedback from the participants who used the MCT encouraged the application of the same format for the capture of actions during meetings. This new template was trialed by the authors, and subsequent improvements and guidelines for its usage have been established. The template fulfills most of the requirements outlined in the strategy, and is currently being tested in industry.
6.2. Future work
Computer support for the action-oriented strategy has been investigated and a conceptual solution is also currently under investigation. The approach seeks to build a software solution based on the “design review capture template” illustrated in Figure 13, but with added digital functionalities such as hyperlinks, digital artifact annotation, and automatic summarization tables. The main advantage of this computer-based prototype over its paper-based counterpart is its integration to the digital environment in which design engineers work. Indeed, most of the artifacts discussed during a design review are nowadays available in a digital format. All the functionalities envisaged are already available in various software solutions (not necessarily for meeting capture), which adds credit to the scenario under development.
Finally, from the work presented in this paper, a number of issues also warrant further investigation:
• An automatization of the three meeting analysis tools, presented in Section 4, should be sought. This paper has described the necessary processes followed to deploy these paper-based tools, and these descriptions could be used to generate algorithms that would help automate certain steps in the processes.
• The DTM case studies have helped to illustrate the use of the meeting analysis tools, and more case studies could now be sought to deepen the characterization of other specific types of meetings.
• One of the interesting findings from the research is the importance of the role of artifacts. A specific investigation into the role of artifacts during meetings should be carried out to understand this influence. This type of study would enable a more efficient use of design artifacts during meetings and also help information capture systems integrate the information generated from the use of these artifacts.
• The IMT has been used in this research to measure organizational knowledge loss, but information mapping is thought to have much more to offer in the field of design research. A new form of design rationale representation could be developed and a further study of this technique could give practical insights into alternative information archiving strategies.
ACKNOWLEDGMENTS
The authors are grateful for the funding provided by the Engineering and Physical Sciences Research Council UK and Airbus UK under CASE Award 02300702. The authors also thank the engineers across Europe and North America who readily participated in the “meeting minutes” survey.
Greg Huet is a Research Associate at École Polytechnique de Montréal (Canada). He graduated as a mechanical engineer from the Université de Technologie de Compiègne (France) in 2002. He then completed his PhD at the University of Bath in 2006. Greg is currently working on a number of projects involving the use of new PLM tools to support collaborative engineering activities. Dr. Huet's interests include design control activities, information and knowledge management strategies, PLM software solutions, engineering change processes, and design research methodologies.
Steve Culley is Professor and Head of Design in the Department of Mechanical Engineering at the University of Bath. He pioneered work into the introduction and use of the electronic catalogue for standard engineering components, systems, and assemblies. He has over 150 publications and has recently coauthored a book on design and changeover. Professor Culley is a member of the Innovative Manufacturing Research Center, where he takes part in the Grand Challenge Project, immortal information, and through-life KM. His research interests are focused on the provision of information and knowledge to support engineering designers.
Chris McMahon worked initially as a Production Engineer, then as a Design Engineer before joining the University of Bristol in 1984. He is currently a Professor at the University of Bath and Director of its Innovative Manufacturing Research Centre. Professor McMahon's teaching and research interests are in the application of computers to engineering design, in particular, in assisting engineers in the organization and management of design information. This work has led to a number of articles on information and knowledge management in design and on the automation of the design process, in addition to a textbook on computer-aided design and manufacture.
Clément Fortin is a Professor at the École Polytechnique de Montréal and the Director of the Mechanical Engineering Department. Earlier in his career he was a pilot for the Canadian Armed Forces. Dr. Fortin is a founder of Polyplan Technologies Inc., a leading manufacturing process management package recently acquired by PTC to extend their current PLM solution offerings. His interests include product and manufacturing process development, engineering change management, virtual environments for product development, and computer-aided tolerancing.
