Jobs are becoming increasingly complex due to the proliferation of knowledge-based economies, where knowledge industries have gained prominence over traditional manufacturing-type trades (Becton, Carr, Mossholder, & Walker, Reference Becton, Carr, Mossholder and Walker2017). Such complex jobs are multidimensional (Shalley, Gilson, & Blum, Reference Shalley, Gilson and Blum2009), difficult to execute, and require many high-level skills (Morgeson & Humphrey, Reference Morgeson and Humphrey2006). Importantly, most of these jobs are also ill-structured and ambiguous (Chung-Yan, Reference Chung-Yan2010), which often makes it difficult to determine the exact criteria for job success. Consequently, a significant challenge faced by organizations is how to select individuals that have a high probability of being successful employees in these complex jobs.
Research suggests that analytical methods of employee selection, such as utilizing highly structured interviews, are generally an effective way to make hiring decisions (Conway, Jako, & Goodman, Reference Conway, Jako and Goodman1995; Huffcutt & Arthur, Reference Huffcutt and Arthur1994; Wiesner & Cronshaw, Reference Wiesner and Cronshaw1988). However, establishing a robust analytical hiring process can take significant resources (Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014). In addition, there is still a high level of uncertainty in this context as, based on meta-analytic studies, various predictors of future job performance only explain, at the most, about 30% of the variance (Luan, Reb, & Gigerenzer, Reference Luan, Reb and Gigerenzer2019). This limitation may be amplified for complex jobs, where it is difficult to identify the predictors of good job performance (Chen, Tsai, & Hu, Reference Chen, Tsai and Hu2008). Specifically relating to the aforementioned structured interviews, some studies have found inconsistent results pertaining to its validity for complex jobs (Levashina, Hartwell, Morgeson, & Campion, Reference Levashina, Hartwell, Morgeson and Campion2014).
Perhaps due to perceived limitations of such analytical methods, hiring managers often rely on their intuition for employee selection (Colarelli & Thompson, Reference Colarelli and Thompson2008; Diab, Pui, Yankelevich, & Highhouse, Reference Diab, Pui, Yankelevich and Highhouse2011; Nowicki & Rosse, Reference Nowicki and Rosse2002), and they tend to believe that the ability to make effective intuitive hiring decisions increases with recruiting experience (Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014). Intuition, as used in this paper, is the result of a cognitive process that automatically and unconsciously processes information to provide a holistic solution (Dane & Pratt, Reference Dane and Pratt2007). In practice, intuitive judgments are often referred to as decisions based on ‘gut instinct’ or ‘gut feeling’. Research related to intuition has found intuitive decisions of experts to be effective in uncertain decision environments (Agor, Reference Agor1986; Burke & Miller, Reference Burke and Miller1999). Therefore, considering the uncertainties involved in selecting employees for complex jobs, perhaps the intuition of expert interviewers may be beneficial in such jobs.
Some scholars, however, discourage the use of intuition in employee selection by pointing to biases of the intuitive process, such as the tendency to gravitate toward candidates who are similar to oneself (Grove, Zald, Lebow, Snitz, & Nelson, Reference Grove, Zald, Lebow, Snitz and Nelson2000; Kausel, Culbertson, & Madrid, Reference Kausel, Culbertson and Madrid2016). Some go as far as to say that even expert interviewers should not rely on their intuition because the intuitive ability to accurately predict the job performance of an applicant does not increase with experience (Highhouse, Reference Highhouse2008; Highhouse & Kostek, Reference Highhouse and Kostek2013). This disparity between academic and practitioner approaches to employee selection, especially as it relates to the perceived value of the interviewer's expert intuition, brings forth a research question that is particularly important given the increased complexity of jobs today. That is, for complex jobs, can the intuition of expert decision-makers enhance the effectiveness of hiring decisions?
Using an experimental design with expert and nonexpert interviewers, this paper examines this research question by assessing the impact of interviewer expertise on the effectiveness of intuition when recruiting for complex jobs. Although intuition is effective in many decision-making domains, there are few empirical examinations of its effect on real-world organizational decision-making environments, including employee selection (Highhouse & Kostek, Reference Highhouse and Kostek2013; Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014). Therefore, the results of our study expand our knowledge of when intuition can be useful in such contexts. Relatedly, this approach underscores the importance of contextual factors, such as interviewer expertise and job complexity, in determining the effectiveness of intuition in managerial decision-making. In so doing, it lays the foundation for future research to explore organizational factors that may impact the effectiveness of intuition. Notably, the purpose of this paper is not to undermine and devalue the importance of analytical and structured methods of employee selection. As evidenced by decades of research, there is great value in using a systematic approach to selecting employees. Rather, the goal is to stimulate the conversation on if and how the valuable intuitive judgment of expert decision-makers can be integrated into employee selection procedures while, at the same time, minimizing the inherent biases and legal implications of using intuition in this context.
Theory and hypotheses development
Conditions for effective intuitive decision-making
When assessing the impact of intuition on employee selection, it is first important to understand what intuition is and when it is useful in decision-making. Intuition and analysis represent two distinct cognitive processes of information processing and decision-making (Kahneman & Klein, Reference Kahneman and Klein2009). On the one hand, intuitive processing is a nonconscious, affectively charged, holistic, and rapid cognitive operation, while on the other hand, analysis is conscious, rational, and comparatively slower (Dane & Pratt, Reference Dane and Pratt2007; Epstein, Reference Epstein1994, Reference Epstein2010; Hammond, Reference Hammond2010). Individuals tend to use either an intuitive or analytical approach to make decisions depending on their predisposition (i.e., natural tendency) or the properties of the task environment.
Two conditions that appear to impact the effectiveness of intuitive decision-making relative to analytical decisions are domain expertise and task structure (Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012). Domain expertise is the extent to which an individual has experience in the decision-making environment. For example, due to their extensive knowledge and experience with the game, chess grandmasters can be considered to be domain experts in the game of chess. In an organizational context, a human resource professional with extensive experience in a particular industry (e.g., healthcare or information technology) can be considered to be a domain expert. Such experts possess highly complex, domain-relevant mental schemas (Dane & Pratt, Reference Dane and Pratt2007). As a result, domain experts can make effective intuitive decisions (e.g., Chase & Simon, Reference Chase and Simon1973; Dijkstra, Pligt, & Kleef, Reference Dijkstra, Pligt and Kleef2013; Hammond, Hamm, Grassia, & Pearson, Reference Hammond, Hamm, Grassia and Pearson1987; Klein, Reference Klein1993; Klein, Calderwood, & Clinton-Cirocco, Reference Klein, Calderwood and Clinton-Cirocco1986).
Task structure is the other condition that influences the effectiveness of intuitive versus analytical decisions. When dealing with problems that are conducive to analytical solutions, such as mathematical problems, analytical decision-making may be best. These types of problems are decomposable (Hammond et al., Reference Hammond, Hamm, Grassia and Pearson1987) and can be solved using logic or mathematical formulas. In contrast, when dealing with complex problems that are ambiguous, ill-structured, and that do not have pre-established decision criteria, intuitive decision-making may well be a better option (Denhardt & Dugan, Reference Denhardt and Dugan1978; Friedman, Howell, & Jensen, Reference Friedman, Howell and Jensen1985; Hammond et al., Reference Hammond, Hamm, Grassia and Pearson1987). For example, through five experimental studies, Dijksterhuis (Reference Dijksterhuis2004) found that unconscious thought processing (i.e., intuition) outperformed conscious thought (i.e., analysis) when making complex decisions related to selecting an apartment or roommate. In that study, participants were assigned to either an intuitive or analytical decision-making condition and were asked to select the best option, out of numerous alternatives, where each alternative had multiple attributes. The ineffectiveness of analytical processing may be due to the inability of the conscious mind to absorb and synthesize a large amount of information since individuals who use analytical processing pay too much attention to a limited number of attributes of the problem (Dijksterhuis, Reference Dijksterhuis2004). On the other hand, the effectiveness of intuition to solve complex problems may be due to the remarkable ability of the human mind to unconsciously, automatically, and rapidly process a large number of disparate pieces of information.
In general, intuition is thought to result in better decisions in decision-making environments where there is greater uncertainty, complexity, time pressure, insufficient data, and multiple solution possibilities (Agor, Reference Agor1986; Baldacchino, Ucbasaran, Cabantous, & Lockett, Reference Baldacchino, Ucbasaran, Cabantous and Lockett2015; Burke & Miller, Reference Burke and Miller1999). An employee selection environment typically consists of these characteristics. This is especially true for complex positions where there is often a high level of complexity and uncertainty in determining the best candidate due to the limited relationship between a candidate's qualifications and future job performance (e.g., managerial positions).
Impact of job complexity on intuitive hiring decisions
Complex jobs are ambiguous and ill-structured (Chung-Yan, Reference Chung-Yan2010). Consistent with Campbell's (Reference Campbell1988) definition of complexity, a complex job may have many tacit elements that lead to successful job performance. Tacit knowledge involves the development of mental models that shape an individual's perspective and their understanding of how best to proceed in a given situation and is the result of extensive experience in a specific domain (Nonaka, Reference Nonaka1994). Unlike explicit knowledge, which involves codifiable facts and theories, tacit knowledge involves knowing ‘how’ (Grant, Reference Grant1996) in such a way that the knowledge cannot easily be codified (Nonaka, Reference Nonaka1994).
Since there are only a few tacit elements that lead to job success when job complexity is low, it is relatively easier to ascertain the knowledge, skills, and abilities that are required for successful job performance. This is because the job requirements for a low complexity job are fairly straightforward. For example, the qualifications of stenographers can be adequately assessed by testing their shorthand writing skills, typing speed and accuracy, and transcription skills. Thus, the lower the job complexity, the clearer the prescriptive causes of good performance, and the easier it is to standardize selection criteria (Dipboye, Reference Dipboye1994). Consequently, since the objective data clearly establishes the qualifications of the candidate, the interviewer does not need to use their intuitive judgment to make a hiring decision. This argument is reinforced in employee selection research as highly structured interview methods that leave little room for the interviewer's intuitive judgment have typically been found to be more reliable than unstructured interviews (i.e., a purely intuitive process) for low complexity jobs (e.g., Conway, Jako, & Goodman, Reference Conway, Jako and Goodman1995; Huffcutt & Arthur, Reference Huffcutt and Arthur1994; Levashina et al., Reference Levashina, Hartwell, Morgeson and Campion2014; Wiesner & Cronshaw, Reference Wiesner and Cronshaw1988).
In contrast, when job complexity is high, it is much more difficult to specify evaluation standards or to identify factors that contribute to good job performance due to the ambiguity surrounding the correct formula for successful job performance (Chen, Tsai, & Hu, Reference Chen, Tsai and Hu2008). For example, through a review of employment interview literature, Levashina et al. (Reference Levashina, Hartwell, Morgeson and Campion2014) noted there have been mixed findings regarding the validity of the structured interview for high complexity jobs. In some studies reviewed by the authors, the validity of structured interviews, especially the situational-interview (where job candidates are given hypothetical situations and asked to explain how they would respond to the event), decreased for high complexity jobs.
Huffcutt, Allen, Conway, Roth, and Klehe's (Reference Huffcutt, Allen, Conway, Roth and Klehe2004) meta-analysis found job complexity to decrease the validity of the situational-interview but not the patterned-behavior-description-interview (where job candidates are asked to recall a past experience and describe how they responded to that event). Although both of these interview types are considered structured interviews, the patterned-behavior-description-interview can be considered less structured as it does not always require standardization (Conway & Peneno, Reference Conway and Peneno1999) since, for example, the interviewer can ask probing questions. This approach would not occur in a true situational-interview. As such, compared to the situational-interview, the patterned-behavior-description-interview may provide an opportunity for intuitive assessment.
Huffcutt et al. (Reference Huffcutt, Allen, Conway, Roth and Klehe2004) discussed two potential reasons for the moderating effect of job complexity on the situational-interview. First is the inadequacy of the scoring system. Although the standard situational scoring system may work well for low to medium complexity jobs, for high complexity jobs, the scoring system may not be detailed enough to capture the more complex answers provided by the applicants. Second, because a complex job will have more complicated facets, it will be difficult to come up with hypothetical situational questions that accurately measure the applicant's ability to perform complex tasks. Therefore, the quality of the situational questions may not be sufficient to accurately assess a candidate for a complex job.
Given these findings, the intuition of expert interviewers may be a supplemental way to assess candidates for complex jobs where the antecedents for effective job performance are not easily identifiable or measurable (Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014). Supporting this argument, Highhouse and Kostek (Reference Highhouse and Kostek2013) note that milder forms of holistic belief systems (i.e., an intuitive approach) are held by organizational psychologists who conduct assessments for managerial and executive-level positions. Since these two types of positions can be categorized as complex jobs, the authors' statement implies that intuitive assessments, in addition to standardized selection techniques, may be useful for filling such positions. This can especially be true for interviewers that have expertise recruiting for similar positions.
We posit that job complexity is a necessary condition for our hypotheses because, given that structured interview methods have been repeatedly found to adequately predict the job performance of candidates for low complexity jobs, it seems a futile exercise to measure the impact of intuition for low complexity jobs. However, as described earlier, the effectiveness of some structured interviews may be lower for complex jobs, which opens the possibility that the intuition of expert interviewers may have some value over and above structured techniques. For these reasons, our study solely focuses on complex jobs.
Impact of interviewer expertise on intuitive hiring decisions for complex jobs
The level of expertise in a particular field depends on the nature of the mental schemas established in the mind of the decision-maker. These mental schemas can either be (1) simple heuristics with minimal domain-relevant knowledge, or (2) complex cognitive maps with a high level of domain-specific information (Dane & Pratt, Reference Dane and Pratt2007). Those with simple heuristics are individuals who have little to no experience and knowledge in the decision-making environment. Consequently, nonexpert's mental schemas lack the capacity to process information presented in a complex problem. Thus, an intuitive decision of a nonexpert may not be optimal.
However, and as previously noted, due to extensive experience and knowledge in the decision-making environment, an expert possesses highly complex domain-relevant mental schemas. This complex mental schema allows the expert to quickly and automatically process a large amount of disparate information and make an intuitive decision. Thus, compared to a nonexpert, an expert's intuitive judgment is much more effective. Supporting this assertion, Dane, Rockmann, and Pratt (Reference Dane, Rockmann and Pratt2012) conducted two experimental studies to examine the impact of expertise on the effectiveness of intuition. In their first study, which asked participants to assess the difficulty of a basketball shot, expertise was determined by the number of years of experience playing competitive basketball. In their second study, which asked participants to identify real and fake designer handbags, expertise was determined by the total number of designer handbags owned by each participant. In both studies, those with high expertise made significantly better intuitive decisions than those with low expertise.
In employee selection, we argue that expertise will have a similarly positive effect on intuition. This is because, compared to nonexpert interviewers, expert interviewers have a higher capacity to identify the idiosyncrasies of a candidate and to interpret configurations of traits that lead to job success (Highhouse, Reference Highhouse2008). This is especially true for complex jobs where, as previously noted, it is difficult to identify the criteria that lead to job success. Therefore, we hypothesize:
Hypothesis 1: For complex jobs, expert interviewers will make better intuitive hiring decisions than nonexpert interviewers
Impact of expert intuition versus expert analysis on hiring decisions for complex jobs
Although expertise increases the effectiveness of intuitive decisions, it may not have a similar impact on analytical decisions. For example, Dane, Rockmann, and Pratt (Reference Dane, Rockmann and Pratt2012) found that, among participants who used analytical thought processes to solve a complex task, there was not a significant difference in task performance between those with high expertise and those with low expertise. In fact, there is some evidence that prompting an expert to use analysis may negatively impact decision quality (e.g., Melcher & Schooler, Reference Melcher and Schooler2004; Wimmers, Schmidt, Verkoeijen, & Van De Wiel, Reference Wimmers, Schmidt, Verkoeijen and Van De Wiel2005) because forcing experts to use analytical thinking disrupts their highly developed and effective intuitive thought process.
In line with the argument above, when recruiting for a complex job, if the interviewer has extensive experience recruiting for a similar job, intuition may be more useful than analytical thinking to make a hiring decision. As previously noted, since complex jobs are ill-structured and ambiguous, it is difficult to set accurate selection standards. As a result, if expert interviewers are required to use analytical thinking processes to make a hiring decision for a complex job, they will likely forgo their intuition and focus on the criteria that do not adequately measure a candidate's ability to be successful on the job. A decision based on the wrong criteria will predictably lead to a sub-optimal decision. Thus, compared to an intuitive decision that holistically combines explicit as well as implicit elements for job success, decisions made by experts using analytical thought processes will be less effective. Therefore, we hypothesize:
Hypothesis 2: For complex jobs, expert interviewers who use intuition will make better hiring decisions than experts who use analysis
Methods
Participants and setting
The healthcare staffing industry in the United States was selected as the job setting in which to test the hypotheses as it amply demonstrates the previously described criteria for job complexity. Due to the complex nature of healthcare, the role of a healthcare professional typically involves a high level of ambiguity and complexity. Healthcare staffing companies hire healthcare professionals (e.g., nurses and therapists) who are then placed on assignments at various client sites (i.e., healthcare facilities such as hospitals). Therefore, in addition to the inherent complexity of healthcare, these positions have an added dimension of complexity as healthcare staffing firms must select employees who not only are qualified for the healthcare position but also meet the business needs of the staffing firm.
A total of 184 participants completed the study, out of which 13 were eliminated for failing the manipulation check discussed below. The final participants for the expert sample were 88 recruiters employed by healthcare staffing companies operating within the United States (45 female). These recruiters are responsible for recruiting healthcare professionals and typically go through extensive training on recruiting in this field. Not only do the recruiters have to ensure that the candidates sufficiently meet the job requirements, but they also must evaluate other factors such as the candidate's past job performance, availability, flexibility, cultural fit, customer service skills, seriousness about taking a new position, and monetary expectations. Through this complex recruiting process, recruiters are expected to identify any irregularities in the candidate's profile.
The nonexpert sample was composed of 83 undergraduate students from several southeastern universities in the United States (41 female). This method of using an expert and nonexpert sample is consistent with prior studies that explored the effect of expertise on intuition (Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012) and the employment interview (Maurer, Reference Maurer2002). Although the undergraduate nonexpert sample likely included some participants with limited work experience, none of these participants had experience recruiting healthcare professionals. All participants were offered the opportunity to participate in a raffle for four $50 Starbucks gift cards to encourage their participation.
Task and procedure
Participants were asked to assume the role of a recruiter working for a healthcare staffing company and charged with the task of employee selection for a complex job (i.e., healthcare professionals). They were exposed to 10 interview scenarios, and in each scenario, they were asked to select the best response out of two candidate responses to the same interview question (see Appendix A for the 10 interview questions and responses). As most candidate interviews by healthcare staffing companies are conducted by phone, candidate responses were audio recordings to create a realistic interview environment. To control for cueing effects due to voice differences, the candidate responses were recorded using a single female voice.
We used a forced-choice approach in this study by asking participants to select the best candidate response out of two options. While an alternate method could have been to ask the participants to rate the candidate responses to the interview questions without having to make a selection, in a typical employment selection context, interviewers are required to make a selection. Therefore, a forced choice between the candidates represented a more realistic employment selection situation. In addition, there is some debate as to the usefulness of the standard situational interview scoring system for complex jobs (Huffcutt, Weekley, Wiesner, Groot, & Jones, Reference Huffcutt, Weekley, Wiesner, Groot and Jones2001). Since our study included situational questions, not providing a rating scale eliminated any valid concerns that may have been attributable to the scoring system.
Although we did not provide a rating scale as a decision aid, all participants were made aware of the job dimension to be assessed along with the definition of that job dimension for each interview question (see Appendix B for a sample interview scenario as presented to the participants). The experiment was administered electronically using Qualtrics survey software. For the expert sample, a link to the online experiment was distributed using email and LinkedIn messages. For the nonexperts, the link was provided by the student's class instructor via email. The participants completed the task on their computers, and in the case of most of the expert sample, at their work desks. This method induced a natural work environment as the phone interviews are generally conducted at the recruiters' desks. After the 10 interview scenarios, all participants completed a questionnaire that assessed the quality of the experimental manipulation, gender, and healthcare recruiting experience.
Experimental conditions
Similar to prior studies that explored the effect of intuition in decision-making (e.g., Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012; Pretz, Reference Pretz2008), both the experts and nonexperts were randomly assigned to either an intuitive or an analysis condition, thereby creating four experimental groups (i.e., expert-intuition [N = 64]; expert-analysis [N = 24]; nonexpert-intuition [N = 42]; nonexpert-analysis [N = 41]) (see Figure 1).
Experimental conditions.
Intuition condition
Participants in the intuition condition were asked to make their decision solely based on their intuition, first impression, and gut feeling. Similar to Dane, Baer, Pratt, and Oldham (Reference Dane, Baer, Pratt and Oldham2011) and Dane, Rockmann, and Pratt (Reference Dane, Rockmann and Pratt2012), they were asked not to think too hard and were encouraged to select the first decision that came to their mind (see Appendix C for specific instructions).
Analysis condition
To induce analytical reasoning, participants in the analytical condition were given explicit instructions to use deliberation, logic, and analysis. This method of inducing analytical cognitive processing is similar to the methods used in the studies cited above as well as Pretz (Reference Pretz2008). Participants were encouraged to ignore any first impressions and gut feelings and instead to carefully consider all available information before deciding (see Appendix D for specific instructions).
It is important to note that the purpose of our study was to assess the impact of expertise on intuitive versus analytical cognitive information processing and decision-making. Our goal was not to compare types of interview techniques (e.g., structured vs. unstructured interview). Therefore, participants in both the intuition and analysis conditions were exposed to the same standardized and structured interview scenarios that were developed through the process described below.
Development of interview scenarios
The preliminary step of the experimental design was to create the interview questions that were used for the study. Interview questions were developed following methods similar to prior employment interview-related studies (e.g., Campion, Campion, & Hudson, Reference Campion, Campion and Hudson1994; Campion, Pursell, & Brown, Reference Campion, Pursell and Brown1988; Day & Carroll, Reference Day and Carroll2003; DeGroot & Kluemper, Reference DeGroot and Kluemper2007; Maurer, Reference Maurer2002; Maurer & Lee, Reference Maurer and Lee2000; Weekley & Gier, Reference Weekley and Gier1987). First, the critical incident technique was used to identify behaviors that affect job performance. The critical incident technique involves the systematic process of collecting direct observations of behavior to assist in solving practical problems and developing broad psychological principles (Flanagan, Reference Flanagan1954). Specifically, 210 behaviors of healthcare professionals that lead to a successful hire were identified by interviewing a sample of healthcare recruiters. Once the critical incidents were gathered, the first author, who has over 11 years of experience in healthcare recruiting, reviewed and sorted the incidents into groups of similar incidents to form underlying job dimensions. Through this process, 10 job dimensions were identified (see Appendix A for job dimensions). To assess the accuracy of the categorization, an expert sample of three healthcare recruiters (average healthcare recruiting experience was 7 years) were asked to review and match a sample of the critical incidents to its corresponding job dimension. There was 100% agreement among the expert sample that the behaviors were appropriately categorized into job dimensions.
Next, situational interview questions were written for each job dimension. The expert sample was asked to match the interview questions to the correct job dimension to assess how well the interview questions reflect the job dimensions. There was a 90% average agreement that the interview questions accurately represent the job dimension. Then, for each question, two candidate responses were scripted, where one response was intentionally written to be better than the other. To assess the accuracy of the ranking order of the scripted responses, the expert sample was given the candidate responses for each question in random order and was asked to rank them based on the quality of the response to the interview question. The experts were also asked to determine how well the responses represent realistic candidate responses. Adjustments to the interview questions and responses were made until 80% average agreement was achieved.
As outlined in Figure 2, through this rigorous validation process, 10 interview questions and two corresponding scripted answers for each of those questions were developed. Using these interview questions and answers, 10 candidate interview scenarios were developed. As noted, the candidate responses were audio recordings.
Interview questions and responses development process. Note: Expert sample (N = 3, average healthcare recruiting experience = 7 years).
Only one question per interview scenario, as opposed to several questions, was used for two reasons. The first reason was to provide a reasonable number of decision scenarios so that the participants' score was based on multiple hiring decisions (i.e., 10) and not based on a single decision. The second reason was to control experiment length. Since the target participants included working professionals, a lengthy experiment may have resulted in decreased participation and task completion. Prior interview-related studies have used a single interview question format (e.g., Brtek & Motowidlo, Reference Brtek and Motowidlo2002).
Measures
Dependent variable
A score for each participant was assigned by calculating the number of times they selected the best candidate response in each of the 10 interview scenarios. Thus, the score could range from 0 (did not select the best candidate response in any of the scenarios) to 10 (selected the best candidate response in all of the scenarios). As depicted in Figure 3, a rigorous three-step process was followed to ensure the dependent variable was a valid measure. Prior employment interview studies have also used interview scenarios using scripted candidate responses to measure the dependent variable (e.g., Maurer & Lee, Reference Maurer and Lee2000). For example, Maurer (Reference Maurer2002) developed six situational interview videos using scripted candidate responses as a means to measure the impact of the interviewer's job expertise and the behaviorally anchored rating scale on the interrater agreement and the accuracy of ratings of situational questions.
Dependent variable validation process. Note: The experts for step 2 and step 3 were different samples.
Manipulation check
The manipulation check evaluated whether participants in each condition complied with expected cognitive manipulations (i.e., those in the intuition condition used intuition more than analysis to make decisions while those in the analysis condition used analysis more than intuition to make decisions). For this purpose, a four-item measure adapted from Dane et al. (Reference Dane, Baer, Pratt and Oldham2011) was used (see Table 1). Two questions measured the use of intuition and two questions measured the use of analysis.
Manipulation check – intuition and analysis conditions
Control
Prior studies have found gender affected how individuals process information (e.g., Allinson & Hayes, Reference Allinson and Hayes1996; Epstein, Pacini, Denes-Raj, & Heier, Reference Epstein, Pacini, Denes-Raj and Heier1996). Thus, consistent with Dane et al. (Reference Dane, Baer, Pratt and Oldham2011, Reference Dane, Rockmann and Pratt2012) and Norris and Epstein (Reference Norris and Epstein2011), this study also controlled for gender.
Results
Manipulations
Participants' responses to the manipulation check statements were analyzed to eliminate those that did not adhere to the instructions. In total, 13 participants were removed as their responses to the statements did not clearly indicate that they used the intended cognitive decision-making process. For the remaining participants, as depicted in Table 2, univariate analysis of variance (ANOVA) revealed a significant difference between the conditions on how they responded to each of the four manipulation check statements. Based on these results, it was concluded that the desired conditions were satisfactorily induced.
ANOVA results for manipulation check statements
**p < .01.
Hypotheses tests
Descriptive statistics and correlations among variables are provided in Table 3. Hypothesis 1 proposed that for complex jobs, expert interviewers will make better intuitive hiring decisions than nonexpert interviewers. To test this hypothesis, the expert-intuition group (N = 64, M = 7.66, SD = 1.32) was compared to the nonexpert-intuition group (N = 42, M = 6.40, SD = 1.93) shown in Figure 1. An ANOVA showed a significant difference between the two groups [F(1, 102) = 15.42, p < .01, η2 = .13], thus confirming the hypothesis that experts in the sample perform better using intuition than the nonexperts.
Means, standard deviations, and correlations
N = 171; expertise (0 = nonexpert, 1 = expert); gender (0 = male, 1 = female).
**p < .01 (two-tailed).
In contrast, when the expert-analysis (N = 24, M = 7.38, SD = 1.56) and the nonexpert analysis (N = 41, M = 7.17, SD = 1.43) groups were compared, there was no significant difference in decision accuracy [F(1, 61) = .25, p = .62]. These findings suggest that even though expertise may increase the effectiveness of intuitive decision-making, expertise may not make a difference when it comes to analytical decision-making. This conclusion is consistent with Dane, Rockmann, and Pratt (Reference Dane, Rockmann and Pratt2012) who found expertise to amplify the effectiveness of intuitive decisions and not analytical decisions.
Hypothesis 2 proposed that for complex jobs, expert interviewers who use intuitive processes will make better hiring decisions than those experts who use analysis. This hypothesis was tested by comparing the expert-intuition group (N = 64, M = 7.66, SD = 1.32) with the expert-analysis group (N = 24, M = 7.38, SD = 1.56) shown in Figure 1. Although at an absolute level the expert-intuition group was more accurate than the expert-analysis group, results showed the difference between the two groups was not significant [F(1, 84) = .745, p = .39]. Thus, hypothesis 2 was not supported.
Post-hoc analysis
Hypothesis 1 found that experts (i.e., healthcare recruiters) make better intuitive hiring decisions than nonexperts (i.e., undergraduate students with no healthcare recruiting experience). Building on this result and to further examine the impact of expertise on intuitive decision quality, a post-hoc analysis was conducted where two groups of experts, with low and high expertise, were compared. For this purpose, the expert intuition group (N = 64) shown in Figure 1 was subsequently split into two groups based on years of healthcare recruiting experience. Individuals with up to 3 years of healthcare recruiting experience were categorized as low-expertise (N = 36) and those with more than 3 years of experience were categorized as high-expertise (N = 28). This breakpoint was used because prior research has found the effectiveness of intuitive decision-making to increase when individuals have at least 3 years of experience in the field (Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012).
Results from the post-hoc analysis showed that those with high expertise (N = 28, M = 8.07, SD = 1.12) performed significantly better [F(1, 60) = 5.19, p < .05, η2 = .08] than those with low-expertise (N = 36, M = 7.33, SD = 1.39) in healthcare recruiting. Consequently, our study found that not only do experts make better intuitive decisions than nonexperts (Hypothesis 1), but also that their ability to make good intuitive decisions increased with experience (post-hoc analysis). Combined, these important findings strengthen the argument that expertise does increase the effectiveness of intuitive hiring decisions when recruiting for complex jobs.
The findings of Dane, Rockmann, and Pratt (Reference Dane, Rockmann and Pratt2012) suggest that nonexperts may perform better using analysis than intuition. Although there was no strong basis to hypothesize such a relationship in the present study, a post-hoc analysis was conducted to consider whether these findings hold true in this sample. Comparing the decision quality of nonexperts in the analysis condition (N = 41, M = 7.17, SD = 1.43) to the nonexperts in the intuition condition (N = 42, M = 6.40, SD = 1.93) shown in Figure 1, ANOVA revealed a significant difference in performance between the two groups [F(1, 79) = 4.15, p < .05, η2 = .05]. This finding suggests that when interviewers are nonexperts, they are more accurate using an analytical approach as oppose to an intuitive approach.
Our post-hoc analysis also found that healthcare recruiters with high-expertise (more than 3 years of experience) in the intuition condition performed significantly better [F(1, 65) = 7.72, p < .01, η2 = .11] than nonexperts in the analysis condition. Consequently, when recruiting for complex jobs, even though nonexperts may make better hiring decisions using analysis over their intuition, their decisions may not meet the level of accuracy of intuitive hiring decisions made by highly experienced healthcare recruiters.
Discussion
Although scholars have been attempting to delineate what intuition is and when it can be used effectively, there has been a scarcity of empirical research that explores the role of intuition in making real-world organizational decisions such as employee selection. In fact, many studies that explore the role of intuition do so in contexts that are not closely related to organizational decision-making environments (e.g., Dane et al., Reference Dane, Baer, Pratt and Oldham2011; Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012; Pretz, Reference Pretz2008). Specifically relating to employee selection, although many managers use intuition to make hiring decisions (Nowicki & Rosse, Reference Nowicki and Rosse2002), scholars discourage the use of intuition by highlighting the biases of the intuitive process (e.g., Highhouse, Reference Highhouse2008). However, to the authors' knowledge, there are no prior studies that empirically examine contextual factors that may inform us as to the conditions when the actual use of intuition in employee selection might be beneficial.
In addition, the conceptual development of intuition has been limited due to the difficulty in directly examining the intuitive process (Baylor, Reference Baylor2001). Since intuition is an unconscious, automatic, and rapid process, it is challenging to assess the actual use of intuition. Due to these complications, most studies measure one's preference for intuitive decision-making (Blume & Covin, Reference Blume and Covin2011) or use self-reported measures that rely on retroactive accounts (e.g., Busenitz & Barney, Reference Busenitz and Barney1997; Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014; Nowicki & Rosse, Reference Nowicki and Rosse2002, Pretz & Totz, Reference Pretz and Totz2007). There are two issues with this methodology. One is that self-reported measures are susceptible to recollection bias and the other is that people tend to glorify their successes while minimizing failures (Dimov, Reference Dimov2007). Especially because intuition is a nonconscious and automatic process, it is difficult to assume that people will be able to accurately recollect the cognitive process they used during a past event. In response, scholars have called for the use of experimental methods to better capture the cognitive process during the point of action (Baldacchino et al., Reference Baldacchino, Ucbasaran, Cabantous and Lockett2015; Fisher, Reference Fisher2008; Hodgkinson & Clarke, Reference Hodgkinson and Clarke2007). Through a rigorous experimental design that captured intuitive and analytical decision-making at the point of action, this paper attempted to address these limitations by examining the role of interviewer expertise and job complexity on the effectiveness of intuitive hiring decisions.
The current study found that expertise does increase the effectiveness of intuitive hiring decisions when recruiting for complex jobs. Importantly, not only did experts make better intuitive decisions than nonexperts, but also among experts, the amount of experience had a positive impact on decision quality. There are at least two important implications of this finding. First, it extends prior intuition research that suggests intuitive decision-making is effective only when the decision-maker is a domain expert, to the arena of employee selection. This finding also suggests that not only does domain expertise increase one's intuitive ability to solve a complex task; it may also increase an individual's intuitive aptitude to judge another person's capacity to be successful in a particularly complex task or job. Second, the result challenges the notion that the ability to make good intuitive hiring decisions does not increase with experience. As found in this paper, for complex jobs, prior experience recruiting for similar positions did increase the interviewer's ability to make effective intuitive decisions. Therefore, the finding underscores the importance of considering the role of job complexity when determining the effectiveness of expert intuition in employee selection.
It is important to reiterate that the argument here is not that we should abandon structured and standardized hiring methods in favor of intuition. In fact, the experimental design of the present study included elements that are consistent with structured interview techniques (e.g., interview scenarios were standardized where both candidate responses were for the same interview question). It is simply argued that when the interviewer is an expert and the job is complex, it may be prudent to give some weight to the expert interviewer's intuitive judgment. How this expert intuition can be integrated into the employee selection process while minimizing the inherent biases and legal implications of using such a method is a question to ponder for researchers and practitioners alike.
Contrary to what was predicted, the study also found no significant difference in performance between the experts in the intuition and analytical conditions. This finding suggests that, perhaps due to their expertise in recruiting healthcare professionals, the method of decision-making had less impact on the quality of the expert's decision. Given the fast-paced nature of many work environments, if there is a limited difference between intuitive and analytical decision-making for complex tasks, then the relative speed of intuition may be what makes it a more appealing option for experts. Stated differently, experienced organizational managers often find themselves in situations where perfect information relating to a particular business decision is not available or takes too much time and money to obtain. In such situations, an intuitive decision may be the most practical and effective option.
Through a post-hoc analysis, the study found that nonexperts performed significantly better when they used an analytical approach compared to an intuitive approach. While not originally hypothesized, this finding provides useful insight as to what type of decision-making approach may be better suited for nonexperts (e.g., new managers and recent college graduates). Because nonexperts do not have the complex cognitive schemas that enable experts to make effective intuitive decisions, their intuitive judgments are often nothing more than a guess with an equal probability of being correct or incorrect. However, when nonexperts use an analytical approach, even though they may not know precisely what information is critical to solving the problem (Pretz, Reference Pretz2008), the deliberate thinking process may unearth elements that lead them toward, or at least increase their chances of, making a more effective decision. Therefore, when the decision-maker is a nonexpert, an analytical decision-making approach may be more effective than relying on their intuition.
It is, however, important to note that even though the nonexperts in this study performed better using analysis than intuition, their performance could not compete with the accuracy of intuitive hiring decisions made by highly experienced healthcare recruiters. This finding further accentuates the value of expert intuition in complex decision-making environments. Even though decision aids and analytical processes are extremely helpful in organizational decision-making, expertise in the decision-making domain is still a critical element that should not be overlooked. Therefore, successful organizations will not only harness the intuitive power of their experts, but also find ways to effectively and quickly transfer this expert intuition to their novices.
From a practitioner perspective, the findings of the present study provide some insight as to when it may be useful to consider the role of intuition in organizational decision-making. The study found that, in the uncertain decision-making environment of recruiting for complex jobs, expertise increased the ability to make effective intuitive hiring decisions. Furthermore, when the interviewer is an expert, their intuitive decisions are as good as their analytical decisions. Therefore, in today's dynamic and uncertain organizational decision-making environments, perhaps it might be prudent to give some weight to an expert's intuitive judgment in conjunction with objective and standardized techniques. To reduce individual biases, perhaps the expert intuition of multiple decision-makers could be combined (Miles & Sadler-Smith, Reference Miles and Sadler-Smith2014).
On the other hand, this study found that nonexpert interviewers perform significantly better when using analytical decision-making instead of intuition. Accordingly, in situations where the decision-makers are nonexperts (e.g., a new manager with no prior hiring experience), it seems imperative that the opportunity for intuitive judgment is minimized and they must be provided with the necessary training, tools, and processes that will allow them to make analytical decisions. Interestingly, the study also found that there was no significant difference in decision quality between experts and nonexperts when using an analytical decision-making approach. This finding suggests that when nonexperts take an analytical approach, their decisions may, in fact, be as good as experts who take a similar approach (all else being equal).
Future directions and limitations
The findings of the present study increase our understanding of when intuition might be useful in organizational decision-making environments such as employee selection. However, for us to fully comprehend the boundary conditions related to the efficacy of intuition, future research should further explore other contextual factors that impact intuition in organizational decision-making environments. In addition, to more precisely measure the effect of intuition, future research might also use neurological (Akinci & Sadler-Smith, Reference Akinci and Sadler-Smith2012; Lieberman, Reference Lieberman2000) and biometric techniques such as eye-tracking and facial expression analysis (McLain & Kefallonitis, Reference McLain, Kefallonitis, Kavoura, Kefallonitis and Giovanis2019).
This paper explored the effect of intuition in a realistic organizational decision-making context using a sample of industry experienced decision-makers. This stands in contrast to most prior empirical studies that examined the role of intuition focused on tasks less closely related to actual organizational decision-making situations (e.g., Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012; Dijksterhuis, Reference Dijksterhuis2004; Pretz, Reference Pretz2008). To fully understand the implications of these findings, future research should continue to explore the effects of intuition in solving organizational problems in real-world settings.
As with any research, the findings of the present study should be considered in light of its limitations. First, the dependent variable was not a predictive measure of job performance (i.e., a measure that assesses the actual job performance of the interviewees). As noted earlier, the dependent variable was validated through a rigorous three-step process using experts and was measured by calculating the number of times the participants selected the best candidate response in each of the 10 scripted interview scenarios. Although similar scripted interview techniques have been used in prior research (Maurer, Reference Maurer2002; Maurer & Lee, Reference Maurer and Lee2000), this method may be less accurate than using actual interviewees and measuring the effectiveness of hiring decisions through a subsequent evaluation of their job performance. Consequently, future research in this area should consider a longitudinal design.
Second, since intuition is a nonconscious process, it is difficult to determine if the participants actually used intuition in making their decisions. Although the method and instructions to prompt intuitive decision-making was consistent with prior research (Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012; Pretz, Reference Pretz2008), and the final analysis only included the participants who successfully passed the manipulation checks, it is still possible that some participants may not have entirely relied on their intuition to make decisions.
Third, participants completed the experiment remotely (i.e., on their computers at their desks) rather than in a more controlled lab environment. This method was used as it induced a natural work environment since the candidate interviews are generally conducted at the recruiters' desks. Furthermore, the method adheres to the call from scholars to conduct intuition research related to organizational decision-making in field settings (Dane, Rockmann, & Pratt, Reference Dane, Rockmann and Pratt2012). However, compared to a lab environment, the method used in this study does make the participants more susceptible to environmental factors that may disrupt their task performance and/or their attention to study details.
Finally, even though this study used a sample of real-world expert decision-makers in their natural work setting, it is susceptible to a drawback of most experimental research to the extent that it may limit the generalizability of these findings outside of the setting of the healthcare staffing industry (McGrath, Reference McGrath1981). Therefore, replication studies in similarly complex employment settings outside of that industry are necessary to strengthen the arguments presented here.
Conclusion
Despite years of study on the role of intuition in decision-making contexts, we still have a limited understanding of what intuition is, how it works, and when it can be useful. The purpose of this paper was to address the latter – to expand our knowledge of when intuition may be beneficial in an organizational decision-making environment such as employee selection. By using a sample of real-world decision-makers in a natural setting, the study found that when the interviewer is an expert and the position of focus is complex; intuition was an effective way to make decisions. More specifically, this study found that experts made better intuitive hiring decisions than nonexperts and, importantly, the amount of expertise mattered. Additionally, even though nonexperts using analysis outperformed those using intuition, the quality of their decisions was no match to the intuitive decisions of highly experienced healthcare recruiters. Collectively, these findings are significant to academic research as it extends our understanding of when intuition can be useful to a domain (i.e., employee selection) where scholars have often discouraged the use of intuition. For practitioners, these findings suggest that when conditions for effective intuitive decision-making are sufficiently met, it may be prudent to consider expert intuitive judgments in the decision process.
Appendix A
Interview questions and responses
Appendix B
Sample interview scenario as seen by participants
The interview question below is designed to assess a candidate's ‘Attention to Detail’. Attention to detail is defined as:
“being detailed and thorough in completing work tasks.”
Complete the following steps
• Read the interview question
• Listen to each of the two candidate responses
• Select the best response
For instructions on how to make your decision, please click on the link below:
Link to instructions
Interview Question: In most of our positions, you often have a large patient load and you're in a fast-paced environment. What would you do to avoid medical errors?
Candidate Responses: Use the “▶” icon to play each response
Response X: 
Response Y: 
Select the Best Response: Click on the shaded box below that represents your choice
Response X
Response Y
Appendix C
Instructions for the intuition condition
Appendix D
Instructions for the analysis condition
Vinod U. Vincent, DBA, SPHR, SHRM-SCP is an Assistant Professor of Management at the Clayton State University's College of Business. Dr. Vincent's research interests include managerial cognition, intuition, decision-making, and HR topics such as employee selection. His research focuses on expanding our understanding of managerial decision-making in organizational environments. Dr. Vincent has over 12 years of experience in the US healthcare staffing industry where his expertise include new business ventures, strategic management, business operations management, employee selection, performance management, and employee training and development. He is certified as a Senior Professional in Human Resources (SPHR) and SHRM Senior Certified Professional (SHRM-SCP).
Rebecca M. Guidice: My initial research interest in organizational control has evolved into a research stream investigating the issues and complexities surrounding corporate governance, particularly as it relates to accountability, job performance, decision-making, innovation, and strategic behavior in organizations. As reflected in my research, I have taken a keen interest in cross-disciplinary research. Most recently, I have initiated two collaborative projects that apply my areas of expertise on governance and accountability to family business and entrepreneurship research. Lastly, I continue to expand on my competitive dynamics research, which in recent years has led to two studies on competitive bluffing behavior – on macro and the other micro in outlook.
Neal P. Mero is the Dean of the School of Business Administration at Stetson University. Mero has extensive administrative and leadership experience in higher education, business, and the military. He served as the founding director of the Kennesaw D.B.A. Program since 2008. Previously, Mero was the vice president and chief advocacy officer for 3 years at AACSB International, the premier professional accreditor for business education, providing leadership for AACSB's global communication, membership services, and business development staff, as well as support for the AACSB's accreditation, through leadership and other global initiatives. Mero also taught management, primarily in the fields of organizational behavior and human resources management, at the University of Central Florida, the University of Mississippi, Washington State University, and the United States Air Force Academy. Mero's business experience includes extensive consulting services in various industries with special emphasis on the healthcare industry. In the military, he served as the director of human resources and faculty supervisor for the United States Air Force Academy, systems development program manager at Norton Air Force Base in California, and director of training for the US Air Force in Great Falls, MT.
