INTRODUCTION
The research aims to discover and disseminate facts about the world. Researchers postulate hypotheses, collect data, and test whether or not the hypotheses are consistent with the data. While researchers aspire rigor, errors are inevitable (Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011). Rigor is essential for advancing research and is a major factor in defining the right problem and suggesting the appropriate solution (Spradlin, Reference Spradlin2012; Hsieh, Reference Hsieh2013). Notwithstanding its importance, measurement in business research is often one of the main weaknesses (Mersman & Donaldson, Reference Mersman and Donaldson2000). There is increasing concern that ‘most current published research findings are false’ (Ioannidis, Reference Ioannidis2005: 0696). Further, the notion that our published research literature is lacking and untrustworthy has gained prominence (Ioannidis, Reference Ioannidis2005; Schooler, Reference Schooler2011; John, Loewenstein, & Prelec, Reference John, Loewenstein and Prelec2012; Brutus, Gill, & Duniewicz, Reference Brutus, Gill and Duniewicz2013; Cumming, Reference Cumming2014; Sinha & Hassan, Reference Sinha and Hassan2014; Yüksel, Reference Yüksel2017). According to Simmons, Nelson, and Simonsohn ‘Ambiguity is rampant in empirical research’ (Reference Simmons, Nelson and Simonsohn2011: 1360). This tendency, however, is not driven by researchers’ willingness to deceive but by biased data, which convince researchers that the outcomes are the most appropriate. Numerous misperceptions in business literature result from researchers’ myopic reliance on partial data, which is exacerbated by a lack of verification. For example, a widely held notion that ‘high rewards necessarily enhance employee productivity’ lived long to misinform the business practice. Facts came to a head more recently when a high profile study discovered that under certain circumstances, rewards and financial inducements adversely affect the performance, and may not be relied upon to yield optimal behavior, particularly for jobs that involve cognitive skills (Ariely, Gneezy, Loewenstein, & Mazar, Reference Ariely, Gneezy, Loewenstein and Mazar2009). The incongruity between ‘what science knows and what business does’ (Pink, Reference Pink2009) may be due to several factors including incorrect data and picking and choosing inappropriate methods (Cumming, Reference Cumming2014). Simmons, Nelson, and Simonsohn argue that ‘undisclosed flexibility in data collection and analysis allows presenting anything as significant’ (Reference Simmons, Nelson and Simonsohn2011: 1359).
Studies that do not heed accuracy in data gathering fail to extract the real message and focus the right problem (Bendle & Wang, Reference Bendle and Wang2016). Without exactitude, they explore blind alleys synonymous with the prose ‘scratching a place that is not itching.’ They waste resources and generate unrealistic results (Morse, Barrett, Mayan, Olson, & Spiers, Reference Morse, Barrett, Mayan, Olson and Spiers2008; Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011; Hsieh, Reference Hsieh2013). In the similar vein, Yüksel (Reference Yüksel2017) documents that research findings in the organization studies based on questionnaires (i.e., semantic differential and Likert items) are doubtful and such growing trend can potentially damage the bottom lines of business. Sackett and Larson (Reference Sackett and Larson1990) analyzed the studies published during 1977–1987 in the renowned organization behavior journals and found that more than one-third of the published work was questionnaire-based; 83% adopted a cross-sectional data design, and a majority of them relied on self-report measures. Espousing self-reports for assessing behavior have many response biases (Huber, Reference Huber1985; Schwartz, Reference Schwartz1999; Stone, Bachrach, Jobe, Kurtzman, & Cain, Reference Stone, Bachrach, Jobe, Kurtzman and Cain1999; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Scholderer, Grunert, & Brunsø, Reference Scholderer, Grunert and Brunsø2005). It inflates causal and correlational relationships (Borman, Reference Borman1991; Spector, Reference Spector1994; Yüksel, Reference Yüksel2017). Researchers (e.g., Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Sinha & Hassan, Reference Sinha and Hassan2014; Peticca-Harris & Elias, Reference Peticca-Harris and Elias2016), suggest ‘informant approach’ for data collection, but it also suffers from many disadvantages (Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993). There is a body of literature (Fisher, Reference Fisher1993; Wade & Tavris, Reference Wade and Tavris2000) suggesting that reports of peers are the projection of one’s own behavior – since there’s a lower cost to pointing the finger at peers than there is to report on one’s own conduct.
Organization theorists (Wagenmakers, Wetzels, Borsboom, & van der Maas, Reference Wagenmakers, Wetzels, Borsboom and van der Maas2011; Cumming, Reference Cumming2014; Yüksel, Reference Yüksel2017) increasingly recommend adopting new techniques to ensure research integrity and suggest making substantial changes to how we collect data and carry out analysis. These include avoidance of data bias, selection of appropriate data analytic practices, fact finding, reassurance of replication, and complete reporting. Our aim here is to clarify why the changes are needed and to recommend how, practically, business research should proceed. We believe that identifying what is false is as essential to science as knowing what is true. We, therefore, fist elucidate the falsity of business research for its reliance on either respondent-based approach (RBA) or informant-based approach (IBA) in measuring a particular behavior. We explain it through an analysis of data collected from 200 respondents and 200 informants on workplace deviance (WD). Our primary goal is to revisit the data collection approaches – examining whether IBA and RBA accurately measure the actual behavior. Our second goal is to investigate the extent to which the conventional techniques for assessing the goodness of measure such as common-method bias and construct validity eliminate the response bias and assure the validity of realistic measurement. We separately evaluate the construct validity, common-method bias, and reliability of each method. Given the pervading biases in both the data collection methods, we propose a hybrid approach by drawing on the theory of psychological projection (Freud, Reference Freud1890; Wade & Tavris, Reference Wade and Tavris2000). It espouses an indirect questioning technique (Fisher, Reference Fisher1993). The hybrid approach recommends data collection from an equaled split sample of respondents and informants, assuming that taking their mean values for every item can epitomize the actual behavior that would otherwise be confounded by using any method alone. The study validates the efficacy of hybrid approach through qualitative confirmation by informal discussions with senior managers in the investigated firms and review of the literature.
LITERATURE REVIEW
WD
First, we highlight the salient features of WD before methodically probing the data collection approaches that are commonly used in the management research. The construct of WD conceptually draws on the viewpoint of Robinson and Bennett (Reference Robinson and Bennett1995). It states that WD is a voluntary behavior that defies instructions, and in so doing it threatens the interest and well-being of the organization and/or its members (Sulaiman & Bhatti, Reference Sulaiman and Bhatti2013). The term ‘voluntary behavior’ indicates the deviant behavior (Kaplan, Reference Kaplan1975) and covers a range of conducts that may cause damage or harm (Spector & Fox, Reference Spector and Fox2005). Prior studies use different terms for WD such as workplace violence (Bentley, Catley, Forsyth, & Tappin, Reference Bentley, Catley, Forsyth and Tappin2013), antisocial behavior, retaliation, workplace incivility, aggression, organizational misbehavior, employee theft and revenge, dysfunctional behavior, deliberately working slower, and unconventional practices (Spector & Fox, Reference Spector and Fox2005). Robinson and Bennett (Reference Robinson and Bennett1995) identify two main classes of deviant behavior at the workplace: (a) interpersonal deviance, which insinuates abnormal behavior toward other individuals within an organization, and (b) organizational deviance, which indicates nonstandard behavior toward the organization. Based on the target and severity, Robinson and Bennett (Reference Robinson and Bennett1995) categorize WD into four types: (1) property deviance; (2) production deviance; (3) political deviance, and (4) personal aggression (see Figure 1).
Robinson and Bennett (Reference Robinson and Bennett1995) typology of deviant workplace behavior
Interpersonal deviance comprises being hurtful or unkind in dealings, stealing from other colleagues, acting rudely, threatening workmates, and quarreling and gossiping about others (Robinson & Bennett, Reference Robinson and Bennett1995). Whereas, organizational deviance implies unexpected behavior toward the organization that comprises leaving early, deliberately working slower, wasting the business resources, embezzlement and stealing, and damaging office equipment (Robinson & Bennett, Reference Robinson and Bennett1995; Bennett & Robinson, Reference Bennett and Robinson2000).
Respondents and informants
Respondents describe their own opinion, feeling, and behavior, whereas informants summarize observed or expected relations, and generalize about the pattern of behaviors (Seidler, Reference Seidler1974). ‘Individual’ as the unit of analysis was coined and popularized by Allport brothers. They believe that the psychology of group is essentially and entirely the psychology of an individual (i.e., Allport, 1924; Allport, 1968). They theorize group as an aggregate of individuals, and thus, examine the group phenomenon from individual’s perspective, presuming that people are mindful of their cognition and rational behavior (Sinha & Hassan, Reference Sinha and Hassan2014). Similarly, Edwards (Reference Edwards1957) also speculates that social reality depicts the totality of how individuals perceive it, and advocates getting direct responses from individuals about their feelings regarding a particular psychological aspect. However, the reliability and validity of self-reporting method have been the focus of scholarly discourse. Even the proponents of RBA (e.g., Edwards, Reference Edwards1957; Triandis, Reference Triandis1980) indicate their disbelief in the data collected through this approach and admit that people are hesitant to express their sincere feelings on controversial matters.
Self-reports have two main problems: first, they are prone to many response biases (Schwartz, Reference Schwartz1999; Stone et al., Reference Stone, Bachrach, Jobe, Kurtzman and Cain1999; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Scholderer, Grunert, & Brunsø, Reference Scholderer, Grunert and Brunsø2005), and second, they lead to inflated causal and correlational relationships (Borman, Reference Borman1991; Spector, Reference Spector1994). When the subjects respond to a stimulus that is presented by the researcher, they distort their answer due to various confounding factors such as social desirability, idealism, peer pressure, and cultural authority. Notably, the effects are more pronounced in the collectivist culture where respondents’ reports are more influenced by the social norms than their self-belief and personal view (Triandis, Reference Triandis1995; Sinha & Hassan, Reference Sinha and Hassan2014). Researchers concerned with quantitative investigations in organizational behavior, particularly in the business context, frequently confront the lack of archival data for measuring the constructs of interest. Thus, they often rely on reports of informants (Peticca-Harris & Elias, Reference Peticca-Harris and Elias2016). IBA is useful when an in-depth information about a particular behavior cannot be expected from the survey respondents.
IBA has various methodological and theoretical benefits (Seidler, Reference Seidler1974; Phillips, Reference Phillips1981; Golden, Reference Golden1992; Chen, Farh, & MacMillan, Reference Chen, Farh and MacMillan1993; Sinha, Reference Sinha2010). For example, in the informant’s role, people are free from their personal view of social reality and make observations objectively. The IBA is also beneficial in evading the stringent sampling representativeness, which is obligatory in case of RBA. A convenience sampling technique can empirically provide an outsider’s opinion about a particular behavior. However, like other methods, IBA is not without some significant drawbacks (Provan & Skinner, Reference Provan and Skinner1989; Heide & John, Reference Heide and John1990; Chen, Farh, & MacMillan, Reference Chen, Farh and MacMillan1993; Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993; Triandis, Reference Triandis1994). Random error and informant bias can considerably taint the informant’s report. Random error is mainly the fallout of attributional bias, hindsight bias, subliminal attempts of keeping one’s self-esteem and impression management (Salancik & Meindl, Reference Salancik and Meindl1984; Huber, Reference Huber1985). Data bias also results from variations related to the changing role of informants in an organization (Seidler, Reference Seidler1974; Phillips, Reference Phillips1981). Mostly, there is little similarity between actual event and informant’s report. For instance, the role of a CEO’s varies from that of a second-level executive because their understanding of events is influenced by their respective positions in the organization (Hambrick, Reference Hambrick1981; Golden, Reference Golden1992). Errors also tend to be more prominent for informants whose roles are not intimately connected to the issue under study. Moreover, other idiosyncratic and retrospective sources of errors may contaminate the informant’s view (Huber, Reference Huber1985; Golden, Reference Golden1992). It suffers from mistakes in recalling the past events or retention problem, and distortion of informant’s memory (Nutt, Reference Nutt1986; Golden, Reference Golden1992). Likewise, IBA suffers from selection difficulty – identifying informants who are capable enough to comment on a particular behavior, and perceptual agreement problem – concerning variation in the multiple respondents’ reports (Heide & John, Reference Heide and John1990; Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993).
To evaluate the informant’s competency, researchers rely on the length of informant’s tenure with the investigated firm (Phillips, Reference Phillips1981), the time-span that informant has observed or interacted with the companies (Phillips, Reference Phillips1982), and the level of their knowledge (Cusumano & Takeishi, Reference Cusumano and Takeishi1991). Many researchers (e.g., Provan & Skinner, Reference Provan and Skinner1989; Heide & John, Reference Heide and John1990) suggest not using IBA due to lack of qualified informants. Consequently, researchers recommend the consensual approach for multiple informants (Glick, Huber, Miller, Doty, & Sutcliffe, Reference Glick, Huber, Miller, Doty and Sutcliffe1990), and consensus-averaging methods for interorganizational research (Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993), but they have scarcely been adopted in the organizational research.
METHOD
Study setting
We assess respondent and informant data and espouse a hybrid tactic for resolving the discrepancies between the deviant behaviors measured through the two approaches. The primary data were collected in a self-administered manner from employees of 12 business groups comprising manufacturing and services industry in Pakistan. Four facets of WD described by Robinson and Bennett (Reference Robinson and Bennett1995) were employed in a questionnaire. Questionnaires were distributed to 800 participants (400 using RBA and 400 IBA), and a total of 592 were returned. After data screening, it was found that only 416 (200 RBA and 216 IBA) were fully attempted. To ensure an equal representation, 16 IBA reports were excluded randomly. The final sample comprised 200 surveys from RBA and 200 from IBA. Data normality were assessed through Skewness and Kurtosis values, and they were found within limits, that is, between ±1.0. The study followed commonly recommended techniques separately for both the approaches such as evaluating the construct validity, reliability, and common-method bias. The correlations between the four facets of WD and the participant’s demographic factors were analyzed, and the model fit was assessed through renowned model fit indices.
Hybrid approach
Given the perennial biases in respondent and informant reports (discussed earlier), the issue of disparity was deliberated with the academicians. After discussion and having it honed by the experts, we propose a hybrid approach that involves data collection through both methods. Following a convenient sampling method, this study randomly distributed an equal number of adapted (respondent and informant) version of the same questionnaire to the subjects. Data collection procedure following this technique would adequately cater for various response biases (Triandis, Reference Triandis1994; Schwartz, Reference Schwartz1999; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Sinha & Hassan, Reference Sinha and Hassan2014). This method also caters for selection and the perceptual problems of informants due to their role, competency, qualification, knowledge, and perception indicated by previous researchers (e.g., Seidler, Reference Seidler1974; Phillips, Reference Phillips1981; Huber, Reference Huber1985; Chen, Farh, & MacMillan, Reference Chen, Farh and MacMillan1993; Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993; Spector, Reference Spector1994; Stone et al., Reference Stone, Bachrach, Jobe, Kurtzman and Cain1999). The hybrid approach considers the difference between respondents and informants scores as the confidence interval, believing that the real parameter estimate is somewhere between the two limits (Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002). Thus, computing mean values for all the items’ scores from respondents and informants formed the hybrid data consistent with the statistical and consensual method specified by Kumar, Stern, and Anderson (Reference Kumar, Stern and Anderson1993).
Principally, the hybrid approach adopts an indirect (i.e., structured projective) questioning technique that is frequently applied in social sciences to reduce social desirability bias, which results from the respondents’ desire to avoiding embarrassment and projecting a favorable image to others (Fisher, Reference Fisher1993). It draws on Sigmund Freud’s (Reference Freud1890) theory of psychological projection. Psychological projection is a defense mechanism in which people unconsciously rejects their undesirable behaviors and ascribe them to others around them. For instance, an individual who is rude accuses others of being rude. According to research, such projection of unpleasant qualities onto others is common in everyday life (Wade & Tavris, Reference Wade and Tavris2000). Hybrid approach complements the RBA and IBA offering a more inclusive approach to measuring organizational/group behavior.
Measures
The construct (WD) was measured using RBA and IBA through 21-items developed by Bennett and Robinson (Reference Bennett and Robinson2000) using a 7-point Likert-type scale (1=‘never,’ to 7=‘always’), with the Cronbach’s α reliability of 0.84 (RBA), 0.88 (IBA), and 0.86 (hybrid). Example items for respondents comprised ‘Taken office property without permission’ and ‘Come in late to work without permission,’ etc. Items were slightly adapted to match the informant mode (Harkness, Reference Harkness2010), example items included ‘Employees in my organization take office property without permission’ and ‘Employees in my organization come in late to work without permission,’ etc. The gist of the items is provided in Table 1.
Gist of items with informant’s and respondent’s scores
Note. The scores represents retorts from 200 respondents and 200 informants.
Common-method bias
Since all the data were collected using a common technique (questionnaire), the study assessed the likelihood of common-method variance in both the approaches independently. We used Herman one-factor test (Podsakoff, MacKenzie, Lee, & Podsakoff, Reference Podsakoff, MacKenzie, Lee and Podsakoff2003), as well as common latent factor method (Gaskin, Reference Gaskin2012). In Herman one-factor test, the lone extracted factor in IBA (15.32% explained variance) and RBA (10.69% explained variance) accounted for much lower than 50% explained variance in WD. Similarly, the results of common latent factor method (IBA 14% and RBA 10% explained variance in WD) indicate that both approaches are free from common-method bias.
Construct validity
We assessed the convergent and discriminant validity of measures. Factor loading of all items and their significance to the construct in both approaches was evaluated separately. The standardized loadings above 0.4 and p≤.001 for all items indicated the acceptable convergent validity of both approaches (Hair, Black, Babin, Anderson, & Tatham, Reference Hair, Black, Babin, Anderson and Tatham2006). The results for both methods had χ2/df≤5.0 and root mean square error of approximation≤0.05 that confirmed the discriminant validity of the constructs (Hair et al., Reference Hair, Black, Babin, Anderson and Tatham2006).
RESULTS
Respondent and informant approach
The scores on 21 items of WD from 200 informants and 200 respondents are provided in Table 1. The data from both approaches exhibited a normal distribution; however, IBA was more Kurtotic and presented higher mean values. Given the two methods, it is evident from the results that significant discrepancy exists across all items. The mean values, standard deviation, and t scores depict an altogether different pattern of WD. The informants reported that employees in the investigated firms have a higher level of deviant behavior. On the contrary, respondents reported significantly lower on all indicators of WD. The general impression that can be inferred from the scores presented in Table 1 is that informants tend to overreport the observed pattern of undesirable behavior (Salancik & Meindl, Reference Salancik and Meindl1984; Huber, Reference Huber1985), and indicate a high level of WD. Whereas, respondents, in contrast, tend to distort their answer by underreporting their own behavior on similar items (Schwartz, Reference Schwartz1999; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Sinha & Hassan, Reference Sinha and Hassan2014) and report a lower level of WD.
Overall then, the outcomes from these alternate approaches suggest that while both the methods are valid, reliable, and free from common-method bias (method’s assessment techniques in vogue), there are significant discrepancies between their results across all four dimensions of WD. Further, we conducted the independent sample analysis of variance to examine whether the mean scores on the four WD dimension vary across respondents and informants, and to determine the dimension on which the two approaches differ significantly. Results show a significant group effect indicating that the RBA and IBA deviate significantly on all four attributes. Table 2 presents the means, standard deviations, standard error, and results of the analysis of variances on the WD dimensions. Results indicate that the two approaches vary on all four dimensions of WD, that is, property deviance (F=691.4, p<.01), personal aggression (F=578.5, p<.01), production deviance (F=502.7, p<.01), and political deviance (F=814, p<.01). The overall results of RBA show that respondents report production deviance as the most prominent deviant behavior followed by personal aggression, property deviance, and political deviance. The IBA, on the contrary, say property deviance as the most prominent deviant behavior followed by production deviance, personal aggression, and political deviance.
Comparison between respondent and informant data on workplace deviance
Note. **p≤.01.
Posthoc analysis
With the results provided in Table 2, a posthoc analysis, that is, the descriptive discriminant study was conducted to calculate discriminant functions (uncorrelated linear functions) that further elucidate the disparities between two approaches for better interpretation (Stevens, Reference Stevens2002; Warne, Reference Warne2014). First, the group statistics reveal that informants had significantly higher scores on all four facades of WD (production deviance being the highest). The test of equality of group means shows that the two approaches are significantly different across all four facades of WD (political deviance had lowest Wilk’s λ=0.246, p<.001). Second, as projected the Box’s M (40.49, p<.001) exhibits unequal group variance. Third, the statistical significance of the prediction model (Wilk’s λ=0.245, p<.001) indicate that higher and lower scores on the WD facets significantly predict a particular approach (i.e., RBA or IBA). Also, the standardized canonical discriminant function coefficient shows that production deviance has the highest predictive ability of group membership to RBA and IBA. With relatively high disagreements between the two approaches, the present study model appears reasonably accurate (Warne, Reference Warne2014).
Hybrid approach
With the hybrid approach (Table 2) the mean scores of all four facades that were low from RBA’s perspective, and high from IBA’s view, attain a somewhat balanced estimate. The hybrid data offer a normal distribution with significant t values for upper and lower bound limits of all four dimensions of WD.
The significant disparities in correlation coefficients between WD and the demographic features among RBA, IBA, and hybrid approach can be witnessed in Table 3. For example, unlike RBA, the IBA shows a negative correlation between WD and the employee position suggesting that employees at the higher position have less deviant behavior compared with, the lower level. Likewise, IBA indicates negative correlations between education level and WD stating that with the increase in employee level of education the deviant behavior decreases, whereas RBA shows a positive relationship. Similar disparities exist for the type of industry, that is, IBA indicates less WD in the service sector as compared to manufacturing, but RBA shows different results altogether. The RBA illustrates significantly negative correlations between employee age and WD, whereas, IBA provides opposite results. Likewise, RBA suggests that married employees have lower WD, but IBA indicates married employees are more inclined toward WD. As alluded above, the two approaches also provide varying results regarding the relationship between WD and other attributes, that is, gender, education, job experience, the size of an organization, position in the organization, and the position’s time-span. The hybrid approach, however, does not find a significant relationship between WD and employee gender, age, job experience, and time-span of the position held.
Comparison between respondent, informant, and hybrid approach based on correlations
Note. *p≤.05; **p≤.01(two tailed).
DISCUSSION
This research has two main findings. First, the conventional approaches for assessing construct’s validity and common-method bias do not guarantee that the measure is correctly evaluating the parameter, and is free from the response bias. Cronbach and Meehl (1955) argue that construct validity indicates the degree to which particular test measures what it purports, or claims to be measuring. Likewise, in the applied statistics (i.e., applied to the social sciences), common-method bias or common-method variance is ‘the spurious variance, which is attributable to the measurement method rather than to the construct that the measures are assumed to represent’ (Podsakoff et al., Reference Podsakoff, MacKenzie, Lee and Podsakoff2003). Employing the commonly used assessment techniques for assessing the goodness of test, construct validity and common-method variance, this study found that both approaches are statistically reliable, valid, and free from common-method bias (Cronbach & Meehl, 1955; Podsakoff et al., Reference Podsakoff, MacKenzie, Lee and Podsakoff2003; Hair et al., Reference Hair, Black, Babin, Anderson and Tatham2006). Nevertheless, they generate significantly different results that point to dissimilar patterns of deviant behavior. Thus, as shown by previous researchers (e.g., Becker & Vance, Reference Becker and Vance1992; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002), we argue that the currently most advanced methodical procedures available for identifying the response bias, they are less likely to illustrate the actual behavior; therefore some new methods are needed.
Second, as do prior researchers (Sinha, Reference Sinha2010; Sinha & Hassan, Reference Sinha and Hassan2014), this study finds significant variation across all items between respondents and informants. The results (Table 3) suggest that RBA and IBA are not uniform and perceptual disagreement exists across all four facets of WD between respondents and informants. The respondents report a significantly lower level of WD as compared to informants. These results support the assertions of prior researchers that respondents distort their answers due to numerous biases and confounding factors such as, peer pressure, idealism, social desirability, and cultural authority (Campbell & Fiske, 1959; Huber, Reference Huber1985; Schwartz, Reference Schwartz1999; Stone et al., Reference Stone, Bachrach, Jobe, Kurtzman and Cain1999; Donaldson & Grant-Vallone, Reference Donaldson and Grant-Vallone2002; Scholderer, Grunert, & Brunsø, Reference Scholderer, Grunert and Brunsø2005). The effects may be more pronounced in this study because it is conducted in the collectivist culture where social norms get more preference than self-belief and personal attitudes (Triandis, Reference Triandis1995; Sinha & Hassan, Reference Sinha and Hassan2014). The informants, on the other hand, report significantly higher WD. These results sustain the idea that random error (owing to attributional bias, hindsight bias, and subliminal attempts of keeping one’s self-esteem and impression management) potentially taints the informant reports (Huber, Reference Huber1985). The overrated reports on all facades of WD can also be attributed to variations in the informant’s role (Seidler, Reference Seidler1974; Phillips, Reference Phillips1981). The correlational analyses of the four dimensions of WD with the employees’ and organizational characteristics (Table 3) support the notion of data partiality; spurious correlations miscalculated claims, and the flawed research posited by prior researchers (Borman, Reference Borman1991; Spector, Reference Spector1994; Morse et al., Reference Morse, Barrett, Mayan, Olson and Spiers2008; Cumming, Reference Cumming2014). Moreover, the significant disparities between RBA and IBA (discussed earlier) rise suspicions about the validity of outcomes (Scholderer, Grunert, & Brunsø, Reference Scholderer, Grunert and Brunsø2005).
A body of literature documents that researchers’ degree of freedom in sampling, data collection and analysis is a main reason behind misinterpreting results and reaching incorrect conclusions (Ioannidis, Reference Ioannidis2005; Schooler, Reference Schooler2011; Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011; Wagenmakers et al., Reference Wagenmakers, Wetzels, Borsboom and van der Maas2011; John, Loewenstein, & Prelec, Reference John, Loewenstein and Prelec2012; Cumming, Reference Cumming2014; Yüksel, Reference Yüksel2017). This stream of literature suggests that when a researcher faces unclear analytic decisions, tends to conclude that the right decision is the one that is statistically significant (p≤.05). This tendency lead to one of the costliest errors, the menace of false positives – incorrectly rejecting a null hypothesis (Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011). False positives can potentially lead to ineffectual policy changes thereby decreasing the credibility of research (Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011; Cumming, Reference Cumming2014). As can be seen in the case of IBA and RBA, the results from two approaches pinpoint a different set of problems tempting researcher to suggest altogether different solutions. The proposed hybrid approach, however, reveals that among the nine demographic factors (Table 3), the problematic is the deviant behavior of unmarried employees at higher positions in the large manufacturing firms regardless of their gender, age, job experience, and position. To assess the cogency of these results, we conducted informal interviews with senior managers in the investigated firms.
Validity of hybrid approach
We compared the present results (hybrid approach) with the findings of prior empirical research and views of the senior executives (General Managers and senior HR managers) in the investigated firms to assess the validity and practical application of the proposed approach. First, the managers had different views regarding the relationship between deviant behavior and employees’ age, education, experience, and gender. Regarding employee education, they had mixed feelings. Some were of the opinion that less educated employees are more deviant, whereas, others asserted that the frequency of WD is higher among more educated employees. These findings are in line with the results of hybrid approach as well as prior literature (e.g., Nor Hayati, Reference Nor Hayati2006; Sulaiman & Bhatti, Reference Robinson and Bennett2013). Second, nine out of the ten GMs, which we interviewed were unequivocal that they had maximum complaints of deviant behavior about unmarried employees. The similar results were obtained through the hybrid approach. Third, prior studies (e.g., Bell & Hughes-Jones, Reference Bell and Hughes-Jones2007; Hershcovis, Reich, Parker, & Bozeman, Reference Hershcovis, Reich, Parker and Bozeman2012) point that employee behavior varies among manufacturing and services industry. Results of hybrid approach also show that employees in services industry have higher WD. Likewise, all HR managers concurred that their services strategic business units had reported more instances of WD as compared to their production units. Fourth, they also said in close congruence with the present result that larger units had more cases of WD. Not only because, they have a large number of employees, but mostly because of weaknesses in the organizational defense mechanism. Finally, as do present results, seven out of ten participants indicated that employees at the higher position have more deviant behavior as compared to employees at the lower level regardless of how long they had held the position. Prior research (Haidt, Koller, & Dias, Reference Haidt, Koller and Dias1993; Brewer, Mitchell, & Weber, Reference Brewer, Mitchell and Weber2002; Hershcovis et al., Reference Hershcovis, Reich, Parker and Bozeman2012) also suggests that hierarchal position influences the behavior of employees.
The illustrated variations between RBA and IBA results (discussed earlier) exhibit how easily an incorrect data can mislead researchers, and how data collection poses potential threats to the validity of management research. Misinformed decisions based on biased data can result in increased costs, higher operational inefficiencies, and more risks (Simmons, Nelson, & Simonsohn, Reference Simmons, Nelson and Simonsohn2011; Yüksel, Reference Yüksel2017). The example presented in this study is like many others that people in the organizations see frequently. Researchers try to fix a particular problem; they speed toward a quick fix, fearing that in-depth data gathering may take long to get to the starting line. Since they do not employ a rigorous method for data compilation that helps understand the right problem and its implications for the firm, they fail to tackle the fundamental issues (Spradlin, Reference Spradlin2012). This study advocate using two data sources to help rule out the validity threats of mono-method biases in organizational research. Employing multiple lines of evidence, in contrast to either one (respondent or informant), is more desirable for avoiding the biases (Shadish, Reference Shadish1993).
Limitations and prospects
Management researchers must address some critical issues in future studies. First, in this paper, the empirical analyses are based on cross-sectional data collected for WD. The findings may differ for other constructs of concern to business psychologists, and it limits the present study to sort out causal relationships. For instance, the fear of reprisal and social desirability are open to reverse causality with alternative explanations. Therefore, unequivocal interpretation of present results must be treated cautiously. Second, it should not be automatically assumed that hybrid is always the best approach, that self-reports are dubious in all settings, or that informants are necessarily superior to respondents (Howard, Reference Howard1994). The proposed hybrid method is just an improvement to IBA and RBA and is more appropriate for collectivist culture. It has its downsides that are open for further development. Further research is required for developing some inclusive framework that can guide which behavioral and psychological measures are likely to be valid under particular circumstances (Kumar, Stern, & Anderson, Reference Kumar, Stern and Anderson1993). Third, it is indeed more demanding to assure that research is focusing the right problem (Spradlin, Reference Spradlin2012), than suggesting the solution. Future research should explore the additional validity and reliability measures ensuring that research is addressing the right issue. Fourth, research also needs to examine the effect of editorial preferences in publishing studies with a statistically significant or favorable outcome more than studies that show unfavorably different results. Finally, WD is among the variables that are frequently analyzed by organizational researchers. Though the proposed (hybrid) approach seems to apply across a wide range of variables in business research, the external validity and robustness of this technique need to be established in future studies.
Suggestions and conclusions
Response biases are more prevalent than thought. Yüksel (Reference Yüksel2017) shows that due to response bias even a correctly established research would produce unsatisfying results. In social pressures, the respondents more likely give meaningless replies. Biased data spuriously increase the internal consistency, reduces the validity and provides unreliable frequency distributions (Dolnicar & Grun, Reference Dolnicar and Grun2009; Peer & Gamliel, Reference Peer and Gamliel2011). Response bias also significantly affect statistical investigations, and hence research outcomes. Mainly, it can lead to biased results in analyses that are based on correlations (Yüksel, Reference Yüksel2017). When the standard deviations rise, and correlations fall, it mainly affects regression analysis or factor analysis because they primarily use correlations as their basis (Peer & Gamliel, Reference Peer and Gamliel2011; Yüksel, Reference Yüksel2017). Further, boredom and fatigue of respondents can also increase the chances of dropout (study abandonment), and it also results in sampling biases posing severe complications to studies relying on the representativeness of sample (such as experiments, public opinion surveys).
Numerous procedural remedies are recommended to protect against data bias. For example, changing the question formats (as done in this study), more careful development of survey items, balancing the positive and negative worded items, cautious wording, employing different methods for collecting data for every construct (Yüksel, Reference Yüksel2017). Some statistical remedies may include latent factor method, marker variable method through correlation-based and regression-based technique, etc. (Podsakof, MacKenzie, & Podsakoff, Reference Podsakof, MacKenzie and Podsakoff2012). However, the effectiveness of all such remedies is still debatable owing to perceptual blindness, motivational factors, and memory limitations that may cause biased responses (Yüksel, Reference Yüksel2017). Despite deliberate research efforts, the results based on biased data can potentially mislead decisions (Cumming, Reference Cumming2014). Evidence in this study suggests that collecting data through both the IBA and RBA (adopting the hybrid approach) can possibly decrease data bias to an optimum level.
It is important to note that social desirability would be higher in a collectivist culture where individuals delineate themselves regarding collectives, where they follow social customs rather than personal attitudes and prefer relationships over rationality (Triandis, Reference Triandis1995). This cultural idyllic implies that respondents present themselves in a manner that is socially desirable. Sinha (Reference Sinha2010) argues that studying social reality in collectivist culture would entail placing respondents in the role of informants. Because in the informant’s role they are likely to distance themselves from their personal views and make impersonal observations of the social reality. Also, collectives, not individuals, are the core of social reality in a collectivist culture, it is more appropriate to ask respondents about the way they view collectives rather than they see themselves. The field of anthropology and sociology (that studies the social reality) have indeed established a popular tradition of employing informants (Bernard, Killworth, Kronenfeld, & Sailer, Reference Bernard, Killworth, Kronenfeld and Sailer1984; Campbell, Reference Campbell1955).
We exposed the falsity of business research in relying on either respondents or informants for data collection, and argue that with the partial data it cannot focus the right problem or suggest a practical solution. This study also provide empirical evidence that renowned techniques for eliminating the common-method bias and establishing construct validity do not serve the purpose. We proposed a hybrid approach for data forming, particularly for studies that are intended to infer causal relationship in the context of organizational behavior. In offering the hybrid approach, we do not suggest that researchers should abandon the methods outlined by previous researchers. Our goal here is not to provide a recipe for deciding the ideal approach for measuring a particular behavior. Instead, with little change in sampling technique and added data collection, our recommended method curtails some of the primary biases and problems related to data cogency in the organizational research. The purpose is to advance the measurement techniques by integrating the strengths of existing approaches to seek some substantial grounds for further work. It is in this spirit that we urge researchers to use the hybrid methods in future research and look for new improvements.
Acknowledgements
The authors express their profound gratitude to the editor and two reviewers for their insightful comments.
