Introduction
Empathy, or the “reactions of one individual to the observed experiences of another” (Davis, Reference Davis1983, p. 113), has been viewed as an essential component of the design process. As such, engineering researchers have invested substantial time and attention to studying the impact of empathic design experiences (Tang, Reference Tang2018), such as simulating empathy-evoking scenarios (Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017), in the engineering design process. However, research on empathy has been primarily limited to individuals, meaning we do not know how empathy impacts team performance. This is problematic because teamwork is an essential component of engineering design (Carlson and Sullivan, Reference Carlson and Sullivan1999; Freuler et al., Reference Freuler, Fentiman, Demel, Gustafson and Merrill2001; Knight et al., Reference Knight, Carlson and Sullivan2007), due to its ability to promote problem solving (Felder and Silverman, Reference Felder and Silverman1988) and improve the exploration of the solution space (Ball et al., Reference Ball, Lambell, Reed and Reid2001; Stempfle and Badke-Schaub, Reference Stempfle and Badke-Schaub2002; Petre, Reference Petre2004). It is unknown how the empathic composition of teams, with respect to the average [i.e., elevation (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] and standard deviation [i.e., diversity (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] of team members’ empathy, would impact design outcomes during nominal group concept generation and early concept screening. The success and final cost of a product can be linked to the early conceptual stages of the idea's emergence (Goldenberg et al., Reference Goldenberg, Lehmann and Mazursky2001; Viswanathan and Linsey, Reference Viswanathan and Linsey2013b); being empathic in those stages could be the gateway to more creative solutions to a design problem (McGinley and Dong, Reference McGinley and Dong2011). However, it is unknown if, when, and how empathy is important in promoting creative design outcomes.
Currently, the engineering design literature provides conflicting interpretations on the role of empathy in the concept generation stage of the design process. For instance, Genco et al. (Reference Genco, Johnson, Holtta-Otto and Seepersad2011) and Johnson et al. (Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014) found that simulating empathy-evoking scenarios helped designers generate ideas that are of high quality (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011), novelty (Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014), and variety (Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014). However, other researchers have identified a dark side to empathy, empathic vampirism (Breithaupt, Reference Breithaupt2018, Reference Breithaupt2019), where designers’ empathy would allow them to over-identify with the end-users, resulting in designing for themselves (Breithaupt, Reference Breithaupt2018).
The concept generation and concept selection stages of the design process are generally recognized as important drivers in the development of creative outcomes (Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2006, Reference Rietzschel, Nijstad and Stroebe2009). For that reason, this work focuses explicitly on these stages. The reader is referred to Alzayed (Reference Alzayed2020) for a more detailed discussion of empathy development across the design process. While prior work provides conflicting interpretations on the role of empathy during those stages, it is critical to note that these studies mostly involve short-form workshops that do not accurately reflect the timescales of authentic design projects (Sandoval and Bell, Reference Sandoval and Bell2004). Notably, most of those studies do not explore the multifaced dimensions of creativity. Thus, this paper explores ideation creativity through the following dimensions: overall creativity, usefulness, uniqueness, and elegance (Besemer and O'Quin, Reference Besemer and O'Quin1999).
Additionally, most of the reported studies on empathy have reported designers’ perceptions of their empathy [e.g., empathic self-efficacy (Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017)] toward the end-user. Studying designers’ perceptions of their empathy might be misleading since designers may be biased to say they were empathic towards the end-user due to their self-serving bias (Gigliotti and Buchtel, Reference Gigliotti and Buchtel1990) or social desirability bias (Fisher, Reference Fisher1993). By studying an individual's trait empathy, a characteristic of an individual rather than a self-perception, these biases can be avoided.
When studying an individual's trait empathy, researchers have argued for the importance of encompassing both cognitive and affective components of an engineering designer's trait empathy (Surma-aho and Hölttä-Otto, Reference Surma-aho and Hölttä-Otto2022). Specifically, this paper defines trait empathy through four distinct components: perspective taking, fantasy, empathic concern, and personal distress. Specifically, perspective taking measures the ability “to adopt the perspectives of other people and see things from their point of view” (Davis, Reference Davis1980, p. 12); fantasy measures “the tendency to transpose themselves imaginatively into the feelings and actions of fictitious characters in books, movies, and plays” (Davis, Reference Davis1980, p. 12); empathic concern measures “the degree to which the respondent experiences feelings of warmth, compassion and concern for the observed individual” (Davis, Reference Davis1980, p. 12); and personal distress measures an “individual's own feelings of fear, apprehension and discomfort at witnessing the negative experiences of others” (Davis, Reference Davis1980, p. 12).
The objective of this paper is to explore the impact of team trait empathy on nominal group concept generation and early concept screening in an engineering design student project. Specifically, nominal group concept generation (see Section “Teams and simulation studies in engineering design”) is defined as an activity where designers first generate ideas individually and then the ideas are pooled together as a team (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980). This paper seeks to study the average [elevation (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] and standard deviation [diversity (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] of team trait empathy and its relation to designers’ generation and selection of ideas rated high in overall creativity, usefulness, uniqueness, and elegance (Besemer and O'Quin, Reference Besemer and O'Quin1999). To study team ideation, this study utilized nominal brainstorming teams.
Prior work by the authors examined the relationship between empathy and creativity in individual designers (Alzayed et al., Reference Alzayed, Miller and McComb2021), demonstrating that different design tasks benefit from unique dimensions of empathy. Specifically, empathic concern can aid ideation while personal distress harms ideation. In contrast, during concept selection, perspective-taking tendencies positively impacted participants’ propensity for selecting elegant ideas. The present work continues this line of investigation, seeking to better understand the dynamics of creativity and empathy in teams. We expect this work to support the development of AI assistants that are capable of interacting effectively within team settings. Taken as a whole, this research is one of the first to study empathy on a team level and provides one of the first evidence on the empathic composition of engineering design teams. The findings from this study can be used to guide team formation in design education and industry to promote effective design outcomes.
Related work
In order to establish the framework for the current investigation, this section highlights prior work in (1) the role of empathy in engineering design and (2) teams in engineering design that serve as the basis for the current study.
The role of empathy in engineering design
Over the past decade, the design community has become invested in studying empathy (Lin and Seepersad, Reference Lin and Seepersad2007; Strobel et al., Reference Strobel, Hess, Pan and Wachter Morris2013; Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018a, Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b; Tang, Reference Tang2018), due to the role of empathy in helping designers better understand the needs of users who are different from themselves (Gray et al., Reference Gray, Yilmaz, Daly, Seifert and Gonzalez2015; Schmitt and Morkos, Reference Schmitt and Morkos2016), as well as developing a deeper understanding of a design problem (Walther et al., Reference Walther, Miller and Kellam2012). Through semi-structured interviews with engineering students, Fila and Hess (Reference Fila and Hess2016) found empathy to be related to effective teamwork, problem contextualization, human-centered design, and individual design inspiration. While empathy has been established as an essential component of design (Lin and Seepersad, Reference Lin and Seepersad2007; Strobel et al., Reference Strobel, Hess, Pan and Wachter Morris2013; Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018a, Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b; Tang, Reference Tang2018), the effect of empathy on concept generation and selection is still unclear.
Research in engineering design provides contradictory explanations on the impact of empathy in the concept generation stage of the design process. Using design effectiveness measures, researchers have found that empathic design activities helped designers generate ideas that are of high quality (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011), novelty (Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014), and variety (Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014). When compared to briefing student designers on a scenario, simulating extraordinary user scenarios on visually impaired users improved designers’ empathic self-efficacy, their perceived ability to understand and design for the end-user (Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017). Similarly, van Rijn et al. (Reference van Rijn, Sleeswijk Visser, Stappers and Özakar2011) explored the influence of watching a video on the end-user to develop designers’ empathy for children with autism. By videotaping and transcribing the team conversations throughout a one-hour design challenge, they found that the time spent discussing facts about the users was related to coming up with designs that better fit the users’ needs, as assessed by five caregivers to autistic children.
Meanwhile, other research has found that empathy could impede designers from coming up with creative solutions to the design problem. For example, Mattelmäki et al. (Reference Mattelmäki, Vaajakallio and Koskinen2014) warn designers about the “empathy trap” (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014), where the designers’ “attempt to be empathic might articulate popular reflections instead of innovating more radical futures” (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014, p. 73). This can be somewhat analogous to design fixation, or the “blind and sometimes counter-productive adherence to a set of ideas or concepts limiting the output of conceptual design,” (Jansson and Smith, Reference Jansson and Smith1991, p. 4) that has been found to limit the solution space explored by designers (Viswanathan and Linsey, Reference Viswanathan and Linsey2013a). Similarly, Chung and Joo (Reference Chung and Joo2017) found that designers’ engagement with an empathic instruction task (watching a video clip about the end-user) harmed their concept evaluation scores. This impact of empathy has been framed as a “dark” side to empathy (Chung and Joo, Reference Chung and Joo2017; Breithaupt, Reference Breithaupt2019), or empathic vampirism (Breithaupt, Reference Breithaupt2018), where the empathizer would see the experiences of the end-user as a medium of their own experiences (Breithaupt, Reference Breithaupt2018).
While recent research explored the impact of empathy in concept generation, little is known about the impact of empathy on early concept selection. Ideas are typically filtered out during early concept selection in a rapid informal screening process where ideas are either considered or not considered for further development (e.g., see Cooper, Reference Cooper1990; Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016a, Reference Toh and Miller2016c, Reference Toh and Miller2019; Starkey et al., Reference Starkey, Toh and Miller2016). From the ideas selected for further development, design teams are advised to use formal concept selection techniques (e.g., Pugh matrix, quality function deployment, and the analytic hierarchy process) in order to provide a systemic structure to their decision-making. This early stage of concept selection (i.e., concept screening) is particularly important to study since it is believed to be a gatekeeper to creative ideas from either being selected, and hence have the potential to be prototyped and developed, or abandoned (Cooper, Reference Cooper1990).
Studying the relationship between empathy and creative design outcomes solely during concept generation is limiting, as generating creative ideas does not necessarily guarantee the final design's creativity (Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2009, Reference Rietzschel, Nijstad and Stroebe2010). Indeed, research has identified that decision-making biases can impact designers’ preferences for creativity during concept selection (Toh and Miller, Reference Toh and Miller2016c, Reference Toh and Miller2019), and thus research on the relationship between empathy and creativity during both concept generation and selection is warranted.
Additionally, research has found that a designer's creativity and empathy during the concept generation stage varies based on the designer's personal connection with the end-user (Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017) as well as the nature of the design task (Starkey et al., Reference Starkey, Toh and Miller2016). For example, Starkey et al. (Reference Starkey, Toh and Miller2016) reported that the design problem impacted the novelty and quality of the designs generated by student designers. Similarly, Hess and Fila (Reference Hess and Fila2016) found that the context of the design problem impacted the empathic techniques utilized by designers in the design process. This line of research highlighted the role of the design problem in impacting empathy and creativity, which this work controlled for.
Taken as a whole, prior research in engineering design provides a conflicting discussion on the role of empathy in design processes. Therefore, this study investigated the role of empathy in nominal group concept generation and early concept screening to understand if, when, and how empathy is important to teams in the design process.
Teams and simulation studies in engineering design
While the previous section discussed the role of empathy in engineering design, most of this prior work has focused on individual designers. This is problematic because design activities are typically deployed in teams (Carlson and Sullivan, Reference Carlson and Sullivan1999; Freuler et al., Reference Freuler, Fentiman, Demel, Gustafson and Merrill2001; Knight et al., Reference Knight, Carlson and Sullivan2007), as teamwork promotes promote peer learning (Springer et al., Reference Springer, Stanne and Donovan1999), problem solving (Felder and Silverman, Reference Felder and Silverman1988), and improves the exploration of the solution space (Ball et al., Reference Ball, Lambell, Reed and Reid2001; Stempfle and Badke-Schaub, Reference Stempfle and Badke-Schaub2002; Petre, Reference Petre2004). Studying teams is important in the context of empathy research because prior research has primarily related empathy with team performance in a business setting (Rapisarda, Reference Rapisarda2002). For example, in a study of senior multimedia students, empathy was found to be a critical component of successful teams as higher levels of empathy enabled students to deter conflict and establish team harmony (Luca and Tarricone, Reference Luca and Tarricone2001). Meanwhile, in a study on 97 organizational teams, high empathic concern was negatively related to productivity (Ayoko et al., Reference Ayoko, Callan and Härtel2008). Finally, in engineering design, empathy was perceived by engineering students as being a stimulant for team social harmony (Fila and Hess, Reference Fila and Hess2016). While the role of empathy has been studied in terms of influencing successful team outcomes (Luca and Tarricone, Reference Luca and Tarricone2001; Rapisarda, Reference Rapisarda2002; Ayoko et al., Reference Ayoko, Callan and Härtel2008; Fila and Hess, Reference Fila and Hess2016), the role of the empathic nature of team members in the concept generation and selection stages of the design process has been scarcely studied in engineering design research. Without this knowledge, it is not known if, when, or how team empathy is important in promoting design outcomes.
Here, we define a team's empathic composition as the combination of their empathy elevation, the average empathy level of the team, and empathy diversity (Neuman et al., Reference Neuman, Wagner and Christiansen1999), the standard deviation in teams’ empathy. Specifically, empathy diversity refers to the amount of empathic gap present in a team; for example, high empathy diversity would mean that there's a large empathy gap between team members. Meanwhile, empathy elevation is the average empathy present in a team – high empathy elevation would mean that the team is high in empathy on average while low empathy diversity means that a team is low in empathy on average. For example, Figure 1 illustrates the calculation of team empathy and diversity for two teams. The team to the left has a higher empathy elevation (average) and a lower empathy diversity (standard deviation) compared to the team to the right.
Sample calculation of empathy elevation and diversity for two simulated teams; the team to the left is high in empathy elevation but low in diversity while the team to the right is high in empathy diversity but low in empathy elevation.
Computing the average and standard deviation of individual attributes to represent team-level constructs is typical in personality research (e.g., Neuman et al., Reference Neuman, Wagner and Christiansen1999; Mohammed and Angell, Reference Mohammed and Angell2003; Reilly et al., Reference Reilly, Lynn and Aronson2001), and has been utilized in previous research in engineering design (Toh and Miller, Reference Toh and Miller2016b). While defining a team's empathic composition in this way may seem obvious, there is no research to date that has empirically assessed the relation of this construct to design outcomes.
One of the challenges of conducting research on teams is the potential cost and time associated with such experimental studies (Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019). Because of such challenges, engineering design researchers have turned their attention to utilizing computational simulations of problem-solving teams in lieu of user studies (McComb et al., Reference McComb, Cagan and Kotovsky2015, Reference McComb, Cagan and Kotovsky2017; Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019; Maier et al., Reference Maier, DeFranco and Mccomb2019). In these simulation-based studies, researchers sometimes model nominal groups (McComb et al., Reference McComb, Cagan and Kotovsky2017; Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2019; Maier et al., Reference Maier, DeFranco and Mccomb2019). In nominal groups, individuals work independently and pool their solutions together near task completion. In brainstorming, this means that individuals first come up with their own ideas rather than generate them as a group (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980). This approach is considered a best practice for concept generation and has been shown to help foster input from all team members (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980; Mullen et al., Reference Mullen, Johnson and Salas1991), and increase productivity (Mullen et al., Reference Mullen, Johnson and Salas1991; Paulus and Dzindolet, Reference Paulus and Dzindolet1993), particularly during the concept generation (Linsey et al., Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011). In engineering design, the nominal group technique has been shown to be more effective in producing novel ideas than traditional brainstorming methods (Lewis et al., Reference Lewis, Sadosky and Connolly1975). When compared to facilitated group brainstorming, Oxley et al. (Reference Oxley, Dzindolet and Paulus1996) found that student designers brainstorming in nominal groups can generate as many ideas as the students utilizing the facilitated group brainstorming technique while both techniques are superior to the interactive brainstorming technique. In support of nominal groups, Paulus and Dzindolet (Reference Paulus and Dzindolet1993) found that nominal groups, composed of four individuals, are four times as productive as interactive groups, also composed of four individuals.
Since nominal groups involve non-interacting individuals (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980), engineering design researchers have used datasets from human subject studies of individuals to computationally simulate a large set of nominal teams (McComb et al., Reference McComb, Cagan and Kotovsky2017; Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019; Maier et al., Reference Maier, DeFranco and Mccomb2019) which helps to mitigate the costs and efforts of large-scale data collection (Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019). In these simulations, researchers combine different combinations of all individuals to get every combination with replacement (McComb et al., Reference McComb, Cagan and Kotovsky2015, Reference McComb, Cagan and Kotovsky2017). This technique closely relates to the statistical bootstrapping technique, a technique that involves “re-sampling the data with replacement many times to get an empirical estimate of the entire sampling distribution” (Mooney et al., Reference Mooney, Duval and Duvall1993, p. 1). This method has been employed by Wright (Reference Wright2007) to create nominal groups in a prior study. These computational simulations have also been found to successfully emulate human team behavior (McComb et al., Reference McComb, Cagan and Kotovsky2017).
Some of the limitations of these simulation-based studies are that team members might not be aware of their team membership since the simulation does not account for social effects [e.g., social loafing (Mullen et al., Reference Mullen, Johnson and Salas1991) and free-riding (Hall and Buzwell, Reference Hall and Buzwell2013)] and thus may fail to capture the full context and dynamics of teams (Williges et al., Reference Williges, Johnston and Briggs1966). However, prior research successfully implemented the use of nominal teams of problem-solving individuals in engineering design and validated the simulation study results with a representative population (McComb et al., Reference McComb, Cagan and Kotovsky2017). While this type of simulation-based research on nominal teams provides a clear means to studying the performance of engineering design teams (McComb et al., Reference McComb, Cagan and Kotovsky2017; Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019; Maier et al., Reference Maier, DeFranco and Mccomb2019), it is still unknown how the empathic composition of design teams impacts concept generation and selection. Therefore, this paper studied the role of the empathic nature of teams on design outcomes during nominal group concept generation and early concept selection.
Research objectives
Based on this prior work, the objective of this paper is to explore the impact of team empathy on nominal group concept generation and early concept selection. This paper sought to study the elevation [average (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] and diversity [standard deviation (Neuman et al., Reference Neuman, Wagner and Christiansen1999)] of team trait empathy, see Figure 2. Specifically, the paper was developed to answer the following research questions (RQ):
• RQ1: Can the elevation and/or diversity of team trait empathy be used to predict the number of ideas generated by a team?
• RQ2: Can the elevation and/or diversity of team trait empathy be used to predict a team's ability to generate creative ideas?
• RQ3: Can the elevation and/or diversity of team empathy be used to predict a team's propensity for selecting creative ideas?
Research questions investigated in this study.
We hypothesized that team trait empathy elevation would be positively related to team creative concept generation and selection due to prior work with engineering graduate students which found that trait empathy was related to their innovative self-efficacy (Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). It was also hypothesized that team empathy diversity would be positively related to the number of ideas generated since prior research has reported that diversity could be a mediator to successful team outcomes (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013).
Methodology
The dataset used for the current study was derived from a design repository of 806 ideas generated by first-year engineering students that were involved in an 8-week design project (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, Miller, Menold, Huff and McComb2020b). Specifically, 103 first-year engineering design students (73 men and 30 women) from four different sections of a cornerstone engineering course participated in an 8-week design project that focused on “ensuring healthy lives and promoting well-being for all at all ages” (Edition, Reference Edition2019, p. 8). Teams were asked to select between the following challenges: (1) lack of safe water, sanitation, and hygiene services, (2) access to vaccinations, (3) indoor and ambient air pollution, and (4) road traffic injuries (Alzayed, Reference Alzayed2019; Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021), see “Problem Statements – Sustainable Development Goal 3” (2020) for the complete problem statements. While teams were allowed to select from these four design challenges, the design context of these challenges was different across the course sections; two of the sections focused on designing for the developed world (n = 50 participants) while the remaining two sections focused on designing for the developing world (n = 53 participants), see Alzayed (Reference Alzayed2019) and Alzayed et al. (Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021) for more details on the experimental setup (Figure 3).
Timeline of the project.
Prior to starting the project, informed consent was obtained according to the Institutional Review Board guidelines, and participants’ demographics were collected. In weeks 1–2, the participants were asked to conduct extreme user research, formulate a problem statement, and create an empathy map. During week 4, participants were involved in an individual brainstorming session where they were asked to individually generate ideas for 20 min. During week 5, participants were involved in a concept selection activity where they were individually asked to select from the ideas they generated during week 4 using a concept screening matrix (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021). After the concept generation and selection activities, in weeks 4 and 5, respectively, participants completed a 28-item survey that measured their trait empathy (see Section “Data collection instruments and metrics”).
The dataset from the previous study (Alzayed, Reference Alzayed2019; Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021) was used to run a computational simulation of 13,482 nominal brainstorming teams, where individuals first generate ideas individually and then the ideas are pooled together as a team (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980). Specifically, the aim of the simulation was to study the role of team empathy on concept generation and selection. In this work, we arrange team members in computationally simulated nominal groups, meaning that there is no direct interaction between the simulated team members. This is not intended to emulate nominal group technique (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980), but to study nominal groups in a more general sense. Our simulation closely aligns with many nominal brainstorming sessions, in which participants ideate separately and then pool their solutions. Our simulation of concept selection departs somewhat from common practice, and best represents early concept selection, during which members of the team conduct individual screening prior to more collaborative concept selection. The remainder of this section highlights the collected data and the simulation procedure.
Data collection instruments and metrics
In order to explore the factors critical to achieving the research objectives, the following instruments were used:
Trait empathy
Participants’ trait empathy was measured using the interpersonal reactivity index (IRI; Davis, Reference Davis1980), a 28-item survey answered on a 5-point Likert scale ranging from “does not describe me well” to “describes me very well.” This instrument was utilized in prior research in assessing the empathic tendencies of engineering students (Hess et al., Reference Hess, Fila, Purzer and Strobel2015; Hess et al., Reference Hess, Fila and Purzer2016; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). The IRI includes the following four subscales, each made up of seven different items: perspective taking, fantasy, empathic concern, and personal distress. For example, a survey item in empathic concern is “Other people's misfortunes do not usually disturb me a great deal.” Davis’ approach to measuring trait empathy is one of the few measures in the literature that encompasses both cognitive and affective components of empathy (Duan and Hill, Reference Duan and Hill1996). In engineering design, Hess and Fila (Reference Hess and Fila2016) argue that both components are needed to help designers better understand the end-user needs.
The four subscales of the IRI have been validated in previous work to be distinct from each other (Davis, Reference Davis1983). The instrument has been implemented to assess individuals’ empathic tendencies (Péloquin and Lafontaine, Reference Péloquin and Lafontaine2010; Gilet et al., Reference Gilet, Mella, Studer, Grühn and Labouvie-Vief2013), including engineering students (Hess et al., Reference Hess, Fila and Purzer2016; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). For internal reliability purposes, Cronbach alpha values were calculated for each subscale for the two time points (concept generation and concept selection) of the data collection. A high Cronbach's α (Cronbach, Reference Cronbach1951) was observed for fantasy (concept generation, α = 0.83, concept selection α = 0.91), perspective-taking (concept generation, α = 0.78, concept selection α = 0.82), empathic concern (concept generation, α = 0.80, concept selection α = 0.80), and personal distress (concept generation, α = 0.83, concept selection α = 0.85).
To measure trait empathy for each of the 13,482 simulated teams, both team empathy elevation and team empathy diversity were considered. Team empathy elevation takes the average across all team members’ trait empathy scores for each subscale (fantasy, personal distress, perspective-taking, and empathic concern) while team empathy diversity takes the standard deviation in team members’ trait empathy scores for each subscale. Studying both the team's elevation and diversity is important since previous research has found that both of those metrics predicted team performance in engineering (Devlin et al., Reference Devlin, Flynn and Riggs2018).
Consensual assessment technique (CAT)
The consensual assessment technique (Amabile, Reference Amabile1989) was used to assess the effectiveness of the ideas generated by the 103 participants. This technique has been used in prior research in engineering design (Christiaans and Venselaar, Reference Christiaans and Venselaar2005; Nikander et al., Reference Nikander, Liikkanen and Laakso2014) and has been identified as a global measure of creativity (Fischer, Reference Fischer, Sannino and Ellis2013; Cseh and Jeffries, Reference Cseh and Jeffries2019). The CAT defines that an idea is creative when judges independently agree that it as creative (Amabile, Reference Amabile1982). Using a 6-point Likert Scale, the ideas were rated on the following criteria: overall creativity, usefulness, uniqueness, and elegance (Besemer and O'Quin, Reference Besemer and O'Quin1999). Specifically, (1) overall creativity relates to experts’ judgement of the overall creativity of an idea, (2) uniqueness relates to overall perceptions of how original and surprising the idea was (Besemer and O'Quin, Reference Besemer and O'Quin1999), (3) usefulness relates to the overall perceptions of value, logic, and how understandable the ideas were, while (4) elegance refers to the idea's “simplicity, insight shown, and conciseness of [the idea's] presentation” (Besemer and O'Quin, Reference Besemer and O'Quin1999, p. 288). The four metrics have been previously used in design research to assess ideation effectiveness (Klein et al., Reference Klein, DeRouin and Salas2006; Buelin-Biesecker and Wiebe, Reference Buelin-Biesecker and Wiebe2013; Sinha et al., Reference Sinha, Chen, Meisel and Miller2017; Cseh and Jeffries, Reference Cseh and Jeffries2019; Prabhu et al., Reference Prabhu, Miller, Simpson and Meisel2018; Zheng and Miller, Reference Zheng and Miller2019). Additionally, we asked the raters to rate the drawing abilities possessed by each idea to control for that factor, since the drawing abilities have been found to influence ratings of creativity (Chan and Chan, Reference Chan and Chan2007).
The CAT method uses experts to rate 20% of the complete idea set to provide a training set for quasi-experts to rate the remaining set based on the experts’ mindset in rating the ideas (Kaufman and Baer, Reference Kaufman and Baer2012; Cseh and Jeffries, Reference Cseh and Jeffries2019). Two faculty members experienced in engineering design research independently rated 20% of the ideas. The two experts have more than 3 years of experience in facilitating user-centered engineering design projects and have conducted similar humanitarian engineering projects for at least two semesters. Additionally, two quasi-experts (PhD candidate and third-year undergraduate student, both studying Industrial Engineering) independently rated the 20% overlap of ideas to ensure agreement with the expert judges (Landis and Koch, Reference Landis and Koch1977). The average of the quasi-experts’ ratings had high agreement (α > 0.75) (Koo and Li, Reference Koo and Li2016) on each of the five metrics. Once inter-rater reliability was achieved, the two quasi-experts rated the remaining 80% of the ideas independently and high inter-rater reliability (α > 0.75) (Koo and Li, Reference Koo and Li2016) was achieved between the two quasi-expert raters for each of the five metrics. An average of the scores from the two quasi-expert raters was calculated for each metric (overall creativity, elegance, usefulness, uniqueness, and drawing abilities), as per recommendations by Silvia (Reference Silvia2011). To obtain a score on overall creativity, uniqueness, usefulness, elegance, and drawing abilities for each simulated team, an average score across all team members is taken for each of the three metrics, see Figures 4–6 for examples of ratings.
An example of an idea from participant 22 that scored 4 on overall creativity, 1 on usefulness, 5.5 on uniqueness, 1 on elegance, and 4 on drawing abilities.
An example of an idea from participant 91 that scored 4 on overall creativity, 4 on usefulness, 5 on uniqueness, 5 on elegance, and 4 on drawing abilities.
An example of an idea from participant 8 that scored 1 on overall creativity, 1 on usefulness, 1 on uniqueness, 1 on elegance, and 1 on drawing abilities.
Propensity for selecting creative ideas
To assess simulated teams’ propensity for selecting creative concepts, we used the propensity toward creative concept selection metric, P C (Toh and Miller, Reference Toh and Miller2015), a metric that has been used in engineering design research (Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016b; Zheng et al., Reference Zheng, Ritter and Miller2018). Specifically, P C measures the “ … tendency towards selecting (or filtering) creative concepts during the concept selection process” (Toh and Miller, Reference Toh and Miller2015, p. 118). For instance, the formula to calculate participants’ propensity toward selecting unique concepts (P Uniqueness) can be summarized as the following:
$$P_{{\rm Uniqueness}} = \displaystyle{{{\rm average}\;{\rm uniqueness\;}{\rm of}\;{\rm selected}\;{\rm concepts}} \over {{\rm average}\;{\rm uniqueness\;}{\rm of}\;{\rm generated}\;{\rm concepts}}}.$$Similarly, an individual's propensity toward concept selection of ideas rated high in (1) overall creativity, (2) usefulness, (3) elegance, and (4) drawing abilities was also assessed in the same manner. An individual can receive a value (P Uniqueness) greater than 1 if the average uniqueness of the selected ideas is higher than the average uniqueness of the available ideas, indicating a propensity for selecting unique ideas. Meanwhile, a value on P Uniqueness that is less than 1 indicated an aversion for selecting unique concepts (Toh and Miller, Reference Toh and Miller2015). Toh and Miller's paper (Toh and Miller, Reference Toh and Miller2015) provides further details on the scoring methodology. To obtain P Overall Creativity, P Uniqueness, P Usefulness, P Elegance, and P Drawing abilities scores for each simulated team, an average score across all team members is taken for each of the four metrics.
It is important to note this work did not directly simulate team decision-making behavior. Rather, the average concept selection characteristics of composed teams were assessed (i.e., scores from the propensity for selection of creative ideas of individual team members were averaged to achieve a team assessment). This aligns with early stages of team concept selection, during which individuals may conduct independent downselection on a large set of ideas before coming together as a team to perform more detailed and rigorous concept selection.
Simulation procedure
Since user studies are time-consuming and potentially costly, a computational simulation was used in the current study. Prior work has shown that computational simulations are an effective means of exploring characteristics of human teams (McComb et al., Reference McComb, Cagan and Kotovsky2015, Reference McComb, Cagan and Kotovsky2017). For instance, computational simulations have been used to successfully generate nominal problem-solving teams (McComb et al., Reference McComb, Cagan and Kotovsky2017). The simulation model used in this study involved the use of nominal brainstorming teams, where individuals first generate ideas individually and then the ideas are pooled together as a team (Van De Van and Delbecq, Reference Van De Van and Delbecq1974; Delbecq et al., Reference Delbecq, Van de Ven and Gustafson1975; Delp et al., Reference Delp, Thesen, Motiwalla and Seshardi1977; Horton, Reference Horton1980).
During concept generation, our simulation methodology accurately simulates the results of a nominal group that brainstorms separately and then pools ideas. The CAT was applied independently to each idea, thus enabling average characteristics to be computed for the combined idea sets of individual teams. During concept selection, our simulation methodology simulates “early concept selection” during which team members individually perform some concept selection (after which they would work together on a combined pool of down-selected ideas). Since we average P Overall Creativity scores within simulated teams, these aggregate values are indicative of the team's overall propensity for selecting creative ideas that would make it to the later collaborative session.
The dataset for the simulation was derived from a design repository of 806 ideas generated by first-year engineering students that were involved in an 8-week design project (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021). Specifically, from the 103 participants, four-person teams were simulated. The simulation setup controlled for instructor, design context, and design problem. In other words, participants from different design problems, contexts, or instructors were not mixed in the same team, see Table 1 for a summary of the 13,482 possible team combinations included in the simulation. Each of the nine team types in Table 1 included different combinations of all participants to get every combination with replacement; a similar methodology has been implemented by engineering design researchers to generate nominal problem-solving teams in previous studies (McComb et al., Reference McComb, Cagan and Kotovsky2017; Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2018, Reference Alzayed, McComb, Hunter and Miller2019; Maier et al., Reference Maier, DeFranco and Mccomb2019). This technique closely relates to the statistical bootstrapping technique (Mooney et al., Reference Mooney, Duval and Duvall1993) which has been employed by Wright (Reference Wright2007) to create nominal groups, see Figure 7 for an example of a computational simulation.
An example of all possible computationally simulated teams from six individuals.
Number of possible simulated teams
Data analysis and results
In order to answer the research questions, statistical analyses were computed using SPSS 25.0, and a significance level of 0.05 was used in all analyses. The results are presented as mean ± one standard deviation (SD) unless otherwise denoted. In addition, effect sizes were calculated and reported according to Cohen (Reference Cohen1988). Finally, as a reminder, elevation relates to the average scores of the team (Neuman et al., Reference Neuman, Wagner and Christiansen1999) while diversity relates to the standard deviation of the team (Neuman et al., Reference Neuman, Wagner and Christiansen1999) for each attribute. Notably, IRI scores from week 4 – survey taken after concept generation – were used for any statistical models involving concept generation (RQ1 and RQ2) – and IRI scores from week 5 – survey taken after concept selection – were used for any statistical models involving concept selection (RQ3).
Prior to the analysis, statistical assumptions were checked. The results showed linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. Visual inspection of a plot of studentized residuals revealed that the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1 (Fox, Reference Fox1991). As assessed by the studentized deleted residuals greater than ±3 standard deviations, there were outliers in all of the dependent variables. The outliers were found to have no significant impact on the significance of the results, and therefore, the full analyses is presented in all three RQs. Additionally, there were no leverage values greater than 0.2 (Huber, Reference Huber1981), and no values for Cook's distance above 1 (Cook and Weisberg, Reference Cook and Weisberg1982). Finally, normality was confirmed by visually inspecting the histograms and Q–Q plots. Based on these results, the analyses proceeded as planned for all RQs.
RQ1: Can the elevation and/or diversity of team trait empathy be used to predict the number of ideas generated by a team?
The first research question was devised to assess the impact of trait empathy elevation and diversity on the number of ideas generated by simulated teams. Based on prior research (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b), we hypothesized that trait empathy would be positively related to the number of ideas generated by a team. To answer this research question, a hierarchical regression models was computed with the dependent variables being the number of ideas generated by each team. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. To account for this, the independent variables were entered in two blocks: (i) design context (developing, developed), course instructor, and design problem and (ii) team empathy elevation and team empathy diversity. A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 8.
Schematic representation of the two-step hierarchical regression model for RQ1.
The results from the hierarchical regression model showed that the design context and problem, and the course instructor, significantly predicted number of ideas, R 2 = 0.287, F (4,13481) = 1804.75, P < 0.01, which is considered a medium effect. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) elevation and diversity to this model also led to a statistically significant model F (12, 13481) = 731.82, P < 0.01, with an R 2 change of 0.087. Specifically, simulated teams’ elevation in personal distress and fantasy positively impacted the number of the ideas generated, while the elevation in empathic concern and perspective-taking negatively impacted the number of ideas generated. Meanwhile, the diversity in personal distress had a positive impact on the number of ideas generated while the diversity in perspective-taking negatively impacted the number of generated ideas; the diversity in fantasy and empathic concern had no significant impact.
The findings from this research question partially support our hypothesis that trait empathy positively impacted the number of ideas generated during concept generation. This finding is congruent to the discussion in the literature that note the varying points of view on the role of empathy (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014; Breithaupt, Reference Breithaupt2019), whereby we find evidence that supports the notion that the influence of the elevation and diversity of empathy differs among the different empathic tendencies (perspective-taking, fantasy, empathic concern, and personal distress).
RQ2: Can the elevation and/or diversity of team trait empathy be used to predict a team's ability to generate creative ideas?
The second research question was devised to assess whether the elevation and diversity of simulated teams’ trait empathy predicted their ability to generate overall creative, elegant, useful, or unique ideas. Based on prior research (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b), we hypothesized that team trait empathy elevation and diversity would be positively related to the generation of overall creative, elegant, useful, or unique ideas. To answer this research question, four hierarchical regression models were computed with the dependent variables being the average overall creativity, average elegance, average usefulness, and average uniqueness of the teams’ generated ideas. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. Additionally, we controlled for the drawing abilities of each simulated team as the drawing abilities have been found to influence ratings of creativity. To account for this, the independent variables were entered in two blocks: (i) teams’ average drawing abilities, design context (developing, developed), course instructor, and design problem and (ii) team empathy elevation and team empathy diversity. A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 9.
Schematic representation of the two-step hierarchical regression model for RQ2 and RQ3.
The results from the first hierarchical regression model showed that team drawing abilities, design context and problem, as well as the course instructor, significantly predicted the average overall creativity of the ideas generated by simulated teams, R 2 = 0.346, F (4, 13481) = 1781.482, P < 0.01, which is considered a medium effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13481) = 675.714, P < 0.01, with an R 2 change of 0.030, see Figure 10 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in fantasy, perspective-taking, personal distress, and empathic concern positively impacted the average overall creativity of the ideas generated. Meanwhile, the diversity in personal distress and perspective-taking negatively impacted the overall creativity of teams’ ideas, while the diversity in fantasy and empathic concern had no significant impact.
Standardized beta coefficients of the predictors from the four regression models displaying the relationship between the average overall creativity, elegance, usefulness, uniqueness of generated ideas and fantasy, personal distress, perspective-taking, and empathic concern (1) elevation and (2) diversity.
While the first regression model investigated the role of team empathy elevation and diversity on the average overall creativity of ideas, the second hierarchical regression model investigated the role of trait empathy elevation and diversity on the average elegance of generated ideas. The results from the second hierarchical regression model showed that the team drawing abilities, design context and problem, as well as the course instructor, significantly predicted the average elegance of the ideas generated by simulated teams, R 2 = 0.169, F (4, 13481) = 686.290, P < 0.01, which is considered a small effect. The addition of the team empathy elevation and diversity to this model also led to a statistically significant model, F (12, 13481) = 292.083, P < 0.01, with an R 2 change of 0.037, see Figure 10 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in empathic concern positively impacted the average elegance of the ideas generated while fantasy, perspective-taking, and personal distress had a negative impact. Meanwhile, the diversity in personal distress and perspective-taking positively impacted the elegance of teams’ ideas, while the diversity in fantasy and empathic concern had no significant impact.
The third hierarchical regression model investigated the role of trait empathy elevation and diversity on the average usefulness of generated ideas. The results from the third hierarchical regression model showed that the design context and problem, as well as the course instructor, significantly predicted the average usefulness of the ideas generated by simulated teams, R 2 = 0.281, F (4, 13481) = 1315.951, P < 0.01, which is considered a medium effect. The addition of team empathy elevation and diversity to this model also led to a statistically significant model, F (12, 13481) = 576.999, P < 0.01, with an R 2 change of 0.059, see Figure 10 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in empathic concern positively impacted the average usefulness of the ideas generated while fantasy, perspective-taking, and personal distress had a negative impact. Meanwhile, the diversity in personal distress positively impacted the usefulness of teams’ ideas, while the diversity in empathic concern had a negative impact; fantasy and perspective-taking had no impact on the usefulness of teams’ ideas.
Finally, the results from the fourth hierarchical regression model showed that team drawing abilities, design context and problem, as well as the course instructor, significantly predicted the average uniqueness of the ideas generated by simulated teams, R 2 = 0.522, F (4, 13481) = 3680.821, P < 0.01, which is considered a large effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13481) = 1346.902, P < 0.01, with an R 2 change of 0.023, see Figure 10 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in fantasy, perspective-taking, and personal distress positively impacted the average uniqueness generated by teams, while empathic concern had no significant impact. Meanwhile, the diversity in fantasy and perspective-taking positively impacted the uniqueness of teams’ ideas, while the diversity in personal distress and empathic concern had an opposite effect. The effect size of the overall model was considered large.
The results from this research question partially support our hypothesis that team empathy elevation positively predicted simulated teams’ ability to generate overall creative, elegant, useful, or unique ideas. These results are in similar in nature to the results from RQ1, whereby the influence of the elevation and diversity of empathy on creative concept generations differs amongst the different empathic tendencies (perspective-taking, fantasy, empathic concern, and personal distress). These results highlight the complicated nature of empathy in design and call against a one-size fits all view of empathy in design.
RQ3: Can the elevation and/or diversity of team empathy be used to predict a team's propensity for selecting creative ideas?
The third research question was devised to assess whether the elevation and diversity of simulated teams’ trait empathy predicted their selection of overall creative, elegant, useful, or unique ideas. Based on prior research (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b), we hypothesized that team trait empathy elevation and diversity would be positively related to the selection of overall creative, elegant, useful, and unique ideas. To answer this research question, four hierarchical regression models were computed with the dependent variables being the teams’ propensity for selecting (1) overall creative, elegant, (2) useful, and (3) unique ideas. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alzayed et al., Reference Alzayed, Miller and McComb2020a, Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. Additionally, we controlled for teams’ propensity for selecting ideas that are rated high in drawing abilities since a preliminary analysis found that it had an impact on teams’ propensity for selecting overall creative, elegant, useful, or unique ideas. To account for this, the independent variables were entered in two blocks: (i) teams’ propensity for selecting ideas rated high in drawing abilities, design context (developing, developed), course instructor, and design problem and (ii) team empathy elevation and team empathy diversity. A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 9.
The results from the first hierarchical regression model showed that team's propensity for selecting ideas rated high in drawing abilities, design context and problem, as well as the course instructor, significantly predicted simulated teams’ propensity for selecting overall creative ideas, R 2 = 0.154, F (4, 13480) = 614.180, P < 0.01, which is considered a medium effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13480) = 259.460, P < 0.01, with an R 2 change of 0.034, see Figure 11 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in empathic concern positively impacted the propensity for selecting overall creative ideas while fantasy, perspective-taking, and personal distress had a negative impact. Meanwhile, the diversity in fantasy positively impacted the propensity for selecting overall creative ideas, while the diversity in personal distress and empathic concern had a negative impact; the diversity in perspective-taking had no significant impact.
Standardized beta coefficients of the predictors from the four regression models displaying the relationship between the propensity for selecting overall creative, elegant, useful, and unique ideas and fantasy, personal distress, perspective-taking, and empathic concern (1) elevation and (2) diversity.
While the first regression model investigated the role of team empathy elevation and diversity on the propensity for selecting overall creative ideas, the second hierarchical regression model investigated the role of team empathy elevation and diversity on the propensity for selecting elegant ideas. The results from the second hierarchical regression model showed that team's propensity for selecting ideas rated high in drawing abilities, design context and problem, as well as the course instructor, significantly predicted simulated teams’ propensity for selecting elegant ideas, R 2 = 0.193, F (4, 13480) = 803.391, P < 0.01, which is considered a small effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13480) = 443.401, P < 0.01, with an R 2 change of 0.090, see Figure 11 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in personal distress and perspective-taking positively impacted the propensity for selecting elegant ideas while the elevation in fantasy and empathic concern had a negative impact. Meanwhile, the diversity in fantasy and empathic concern positively impacted the propensity for selecting elegant ideas, while the diversity in personal distress and perspective-taking had a negative impact. The effect size of the overall model was considered medium.
The third hierarchical regression model investigated the role of team empathy elevation and diversity on the propensity for selecting useful ideas. The results from the third hierarchical regression model showed that team's propensity for selecting ideas rated high in drawing abilities, design context and problem, as well as the course instructor, significantly predicted simulated teams’ propensity for selecting useful ideas, R 2 = 0.163, F (4, 13480) = 654.098, P < 0.01, which is considered a small effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13480) = 395.137, P < 0.01, with an R 2 change of 0.097, see Figure 11 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in personal distress and perspective-taking positively impacted the propensity for selecting useful ideas while the elevation in fantasy and empathic concern had a negative impact. Meanwhile, the diversity in fantasy and empathic concern positively impacted the propensity for selecting useful ideas, while the diversity in personal distress and personal distress had a negative impact. The effect size of the overall model was considered medium.
Finally, the fourth hierarchical regression model investigated the role of team empathy elevation and diversity on the propensity for selecting unique ideas. The results from the fourth hierarchical regression model showed that team's propensity for selecting ideas rated high in drawing abilities, design context and problem, as well as the course instructor, significantly predicted simulated teams’ propensity for selecting unique ideas, R 2 = 0.198, F (4, 13480) = 829.690, P < 0.01, which is considered a small effect. The addition of team empathy elevation and team empathy diversity to this model also led to a statistically significant model F (12, 13480) = 519.387, P < 0.01, with an R 2 change of 0.118, see Figure 11 for a summary of the contributing factors and the Appendix for summary statistics of the regression model. Specifically, simulated teams’ elevation in empathic concern positively impacted the propensity for selecting unique ideas while fantasy, perspective-taking, and personal distress had a negative impact. Meanwhile, the diversity in personal distress positively impacted the propensity for selecting unique ideas, while the diversity in perspective-taking and empathic concern had a negative impact; the diversity in fantasy had no significant impact.
Notably, the addition of team empathy elevation and team empathy diversity to the linear regression models in all RQs mostly led to R 2 improvements of no more than 0.2, which are considered small improvements (Cohen, Reference Cohen1988). However, these results demonstrated that there is a measurable impact to team empathy elevation and diversity which were quantified in this study.
Discussion
The main goal of this paper was to explore the role of team empathy in nominal group concept generation and early concept screening. The main findings from this study indicated that the utility of the elevation and diversity of different types of empathy varied depending upon the specific design outcome (elegance, usefulness, uniqueness) and design stage (concept generation and selection), see Table 2 for a summary of the results. These results highlight the complicated nature of empathy in design and call against a one-size fits all view of empathy in design. The remainder of this section highlights the results with respect to the research questions.
Summary of simulation results (+ indicates positive impact of the empathy component, − indicates negative impact of the empathy component)
The role of empathy in nominal group concept generation and early concept screening
The findings from concept generation corroborate with prior research that displays varying points of view on the role of empathy in concept generation (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014; Breithaupt, Reference Breithaupt2019). For example, in terms of the number of ideas generated by design teams, teams’ elevation in personal distress and fantasy positively impacted the number of the ideas generated, while the elevation in empathic concern and perspective-taking negatively impacted the number of ideas generated.
The results from concept selection are similar in nature to the concept generation stage, whereby we find evidence that supports the notion that the elevation and diversity of different types of team empathic tendencies varied depending upon the specific design outcome (overall creativity, elegance, usefulness, uniqueness). These results confirm prior work that discussed that empathy could have positive (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014) and negative (Strobel et al., Reference Strobel, Hess, Pan and Wachter Morris2013; Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014; Fila and Hess, Reference Fila and Hess2016) impacts in design.
These mixed results could be due, in part, to other mediator factors taking a role in those relationships. These factors include domain expertise (Cross, Reference Cross2004), task focus (Matthews and Campbell, Reference Matthews and Campbell1998), cognitive load (Chen et al., Reference Chen, Chang and Chuang2022), and task motivation (Carberry et al., Reference Carberry, Lee and Ohland2010). For example, task focus could be an important factor to consider since individuals with high trait perspective taking may think of idea generation tasks in terms of solving others’ problems, instead of, for example, a focus on the engineering point of view. As such, these individuals may tend toward simple solutions rather than creative solutions to the problem. These mixed results also call for further qualitative investigations to deepen our understanding of how designers utilize their empathic tendencies during concept generation and selection activities. For instance, future studies could qualitatively analyze designers working during a think-aloud brainstorming session; this analysis can allow us to infer on which empathic tendencies they utilize in ideation. During concept selection, similarly, the analysis of team conversations can allow for a better understanding of how designers use their empathic tendencies in making their design decisions.
Notably, this study is an extension of prior work by the co-authors investigating the relationship between empathic tendencies and creativity in individual designers. The results from this simulation study identified several departures from trends observed in that previous work with individual designers. For instance, elevation of personal distress was correlated to more ideas generated, and elevation of empathic concern was correlated to fewer ideas – this is in fact opposite of the individual results (Alzayed et al., Reference Alzayed, Miller and McComb2021). These results indicate that there might be a “sweet spot” to the level of each empathic tendency prevalent in a team for positive design outcomes. For example, from the results on individual designers, empathic concern could be helpful for individual designers to come up with more ideas. However, after a certain plateau of empathic concern prevalent in a team, we might see negative impacts (i.e., less ideas generated) of empathic concern in concept generation. This interpretation resonates with prior research in psychology that discussed that while empathy is positively associated with prosocial behavior, high levels of empathy could have negative impacts on individuals (Stern and Divecha, Reference Stern and Divecha2015). These impacts include empathic over-arousal (Eisenberg et al., Reference Eisenberg, Fabes, Murphy, Karbon, Maszk, Smith and Suh1994), emotional vulnerability (Davis, Reference Davis1996), anxiety (Davis, Reference Davis1996), and negative affect (Batson et al., Reference Batson, Fultz and Schoenrade1987). In design, researchers have named this phenomena “empathy trap” (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014) or “empathic vampirism” where the designer will end up designing for themselves if they over empathize (see Section “The role of empathy in engineering design”).
This research adds to the existing body of knowledge by suggesting that while empathy may be useful throughout the design, the utility of specific types of empathy might vary depending upon the design stage and design outcome. This is in line with prior work in psychology that suggests the importance of triggering interventions targeted toward specific components of empathy (Cameron, Reference Cameron2018). Taken as a whole, the findings from this paper call for the importance of identifying which outcome (overall creativity, elegance, usefulness, uniqueness) is desired, and thus compose teams based on the empathic tendency (fantasy, personal distress, perspective-taking, and empathic concern) these outcomes are impacted by.
Implications for design science and education
The implications of these relationships are twofold. First, for other researchers, these relationships provide hypothetical nominal team behaviors (computationally predicted but not yet experimentally validated) which can inform future research in the field. Second, for educators, this set of relationships can be used to reason on the empathic antecedents to specific team outcomes. For instance, if a team experiences low ideation output, one possible explanation would be high empathic concern elevation. The educator could then choose to discuss this relationship with the team as a way of cueing metacognitive discourse, which has been tied to beneficial learning outcomes.
Specifically, depending on the learning outcomes of the course and the nature of the design project, educators need to identify which design stage is more relevant and what design outcome is desired, and thus target interventions and compose teams based on that specific design stage (e.g., concept generation or concept selection). In first-year design courses, both the concept generation and selection stages are critical and educators typically spend an equal amount of time on both of these stages (Meisel et al., Reference Meisel, Ritter, McComb and Menold2019; Ritter and Bilen, Reference Ritter and Bilen2019). However, these stages are inherently different in terms of the educational outcomes and cognitive skillsets that are used in these stages. For example, Toh and Miller (Reference Toh and Miller2016a, Reference Toh and Miller2016b) identified that the cognitive skills used in concept selection are very different from the skills used during concept generation. In the same line of research, Hay et al. (Reference Hay, Duffy, McTeague, Pidgeon, Vuletic and Grealy2017) found that different design activities might require different working memory operators and reasoning processes based on the specific design goals (Stauffer and Ullman, Reference Stauffer and Ullman1991). In terms of the design outcomes, all of the four design outcomes (overall creativity, usefulness, uniqueness, and elegance) are ideally important since they constitute creative design outcomes – an essential component of design thinking in engineering design education (Dym et al., Reference Dym, Agogino, Eris, Frey and Leifer2005; Brown, Reference Brown2008; Melles et al., Reference Melles, Howard and Thompson-Whiteside2012). However, on a team level, the team composition recommendations were found to be different between concept generation and concept selection. Thus, it would not be ideal or feasible to re-form teams between concept generation and selection. Therefore, educators might be required to select between those two design stages depending on the nature of the design project.
In addition to driving design outcomes, these results advance design science by identifying team empathic composition recommendations. Since empathy and its related concepts are important outcomes in cornerstone design courses (Ritter and Bilen, Reference Ritter and Bilen2019), the insights from this paper could be used to add to that body of knowledge that could be delivered to engineering students through lectures and other empathy-building activities. This is in congruence with current insights in design education that stress the need to provide theoretical insights to design students as part of developing students’ academic competence (Tempelman and Pilot, Reference Tempelman and Pilot2011).
Conclusions, limitations, and future work
The main goal of this paper was to explore the role of team empathy on nominal group concept generation and early concept selection in an engineering design student project. In order to achieve this goal, a computational simulation study examined the empathic composition of 13,482 nominal brainstorming teams composed of four members. The simulated teams were generated from a design repository of 806 ideas generated by 103 first-year engineering design students. The main findings from the study indicated that the utility of the elevation and diversity of different types of team empathy varied depending upon the specific design outcome (overall creativity, elegance, usefulness, uniqueness) and design stage (concept generation and selection). These results highlight the complicated nature of the role of empathy in design and confirms prior research on the varying roles of empathy in the design process. The results from this study provide guidelines for the composition of teams in design education and industry to promote effective design outcomes.
However, there are several limitations that need to be identified that could lead to interesting avenues for future research. First, due to the nature of the simulation study, simulated team members were not aware of their team membership. Therefore, this study does not account for social effects [e.g., social loafing (Hall and Buzwell, Reference Hall and Buzwell2013)], and the results should thus be interpreted conservatively. Second, this simulation study did not take into account the potential impact of team dynamics and individual's empathy during problem formulation and the needs assessment stage on team members’ creativity during concept generation and selection. This warrants future research that should control for those factors by having student designers complete all of the design activities individually and not as a team. Third, while this work studied the relationship between empathy and the team's ability to generate and select elegant, useful, or unique ideas, future work should investigate other design outcomes, such as the quality of the final design. Fourth, while prior research found that the ideation patterns of first-year and senior-level students differ (Alzayed et al., Reference Alzayed, McComb, Hunter and Miller2019), this work only studied first-year students. Thus, future research is warranted to extend those findings beyond first-year student design teams. Additionally, this study fails to address the full degree of heterogeneity present in the design teams, including gender, ethnicity, and cognitive attributes. This should be a subject of future investigation, as purposeful and holistic design of a team across multiple dimensions of heterogeneity is likely to yield significant benefits. Finally, while prior work depicts the utility of nominal brainstorming teams in aggregating individual ideation, future work should explicitly compare nominal brainstorming teams with collaborative brainstorming teams to assess whether team communication during the ideation and concept selection activities might impact teams’ creativity during those activities.
Taken as a whole, this research is one of the first to formalize the role of empathy on a team level. The pursuit of such a formalization is critical, given the need for engineering designers to engage with a broad range of stakeholders. Without understanding when or how to prepare engineering designers to be empathic, they could fail to understand the needs of diverse users and subsequently fail in solving those users’ problems.
Acknowledgments
An earlier version of this work has been published as part of the Proceedings of the 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Alzayed et al., Reference Alzayed, Miller, Menold, Huff and McComb2020b). We thank Kuwait University for funding the doctoral fellowship of Mohammad Alsager Alzayed. The authors are also grateful to Dr. Daryl Cameron and undergraduate research assistants Abby O'Connell and Lois Jung for help on this project.Mohammad Alsager Alzayed is an Assistant Professor of Industrial and Management Systems Engineering at Kuwait University.
Mohammad Alsager Alzayed is an Assistant Professor of Industrial and Management Systems Engineering at Kuwait University. He completed his PhD, MS and BS degrees in Industrial Engineering all from the Pennsylvania State University. His research interests include human factors, engineering design, and engineering education.
Scarlett Miller is an expert in innovation, human-computer interaction, engineering design, and human factors product design. She is a professor in the School of Engineering Design, Technology and Professional Programs (SEDTAPP) and the Hal and Inge Marcus Department of Industrial and Manufacturing Engineering at Penn State University. She also holds affiliate appointments in Mechanical Engineering, the College of Information Sciences and Technology, and in Surgery.
Jessica Menold is an Assistant Professor in the School of Engineering Design and Innovation and the Department of Mechanical Engineering at Penn State. She is the director of the Technology and Human Research in Engineering Design lab and conducts research at the intersection of engineering design, manufacturing, and new product development. Her current work focuses on improving the efficiency and effectiveness of new product development processes, integrating design thinking into engineering education, identifying the factors that contribute to or detract from the success of high-tech startups, and Design for Inspection in advanced manufacturing environments. Her work is dedicated to improving the design of engineered products and systems through evidence-based design methods, rapid prototyping, and performance analysis. Her work is supported by the United States National Science Foundation and the United States Air Force Office of Scientific Research. Dr. Menold is also the Associate Director for Outreach and Inclusion at the Bernard M. Gordon Learning Factory, Penn State's Makerspace.
Jacquelyn Huff is an Assistant Teaching Professor of Engineering Design at Penn State University. She received her Bachelors and Masters degree in Electrical Engineering from the University of Illinois at Urbana-Champaign.
Christopher McComb is a faculty member in Carnegie Mellon University's Department of Mechanical Engineering. Previously, he was an assistant professor in the School of Engineering Design, Technology, and Professional Programs at Penn State. He also served as director of Penn State's Center for Research in Design and Innovation and led its Technology and Human Research in Engineering Design Group. His research interests include human social systems in design and engineering; machine learning for engineering design; human-AI collaboration and teaming; and STEM education, with funding from NSF, DARPA, and private corporations.
Appendix
Summary statistics of the regression model on the relationship between the number of ideas generated by simulated teams and team trait empathy elevation and diversity
Summary statistics of the regression model on the relationship between the average usefulness and uniqueness of simulated teams’ ideas and team trait empathy elevation and diversity
Summary statistics of the regression model on the relationship between the average overall creativity and elegance of simulated teams’ ideas and team trait empathy elevation and diversity
Summary statistics of the regression model on the relationship between the propensity for selecting useful and unique ideas and team trait empathy elevation and diversity
Summary statistics of the regression model on the relationship between the propensity for selecting overall creative and elegant ideas and team trait empathy elevation and diversity
