PROSOCIALITY, PERSONALITY, AND POLITICS

The concept of prosociality is of fundamental importance to Political Science, as it underlies motivations for political participation, social cooperation, charitable giving, voluntarism, and redistributive preferences. Many studies focus on the question of prosociality and political behavior, without reference to personality traits. Fowler (Reference Flynn2006) demonstrates that altruism predicts voter turnout. Further investigations (Bekkers Reference Bekkers2005; Fowler and Kam Reference Fowler and Dawes2007; Sautter, Littvay and Bearnes Reference Pothos, Perry, Corr, Matthew and Busemeyer2007) differentiate between general altruism and in-group favoritism in political participation. Similarly, Loewen (Reference Lee and Ashton2010) examines the roles of affinity and antipathy in the turnout decision while Edlin, Gelman and Kaplan (Reference Druckman, Green, Kuklinski and Lupia2007) incorporate altruistic preferences into utility functions to predict significant voter turnout.

“Prosociality” has many definitions (see Beilin and Eisenberg Reference Beilin and Eisenberg2013 for at least half a dozen) and often includes both altruistic and cooperative components. We will not wade into the debate about how exactly to define prosocial behavior, but a common thread that runs through most of the definitions is that “prosociality” is the opposite of “selfishness.” Therefore, we define prosociality for the purposes of this study as exactly that: we define behavior as prosocial to the extent that it goes against one’s selfish interests and as a result potentially increases the payoffs to another person. In the context of the standard game forms that we consider here, this means either cooperation in social dilemmas such as the prisoner’s dilemma and the public goods game, or generosity (i.e., altruism) in distributional games such as the dictator game or the trustee’s role in the trust game (all of the games we consider will be described in more detail below). In each of these cases, we will consider behavior to be prosocial to the extent that it deviates from the selfish choice prescribed by the Nash equilibrium strategy—in other words, prosociality is characterized as cooperation in the cooperative dilemmas and as generosity in the distributive games.

There is some evidence that prosocial tendencies are related to a person’s underlying personality traits. Personality should predict prosocial tendencies and prosocial political behaviors. Alford and Hibbing (Reference Alford, Funk and Hibbing2007) and Denny and Doyle (Reference Denny and Doyle2008) find only a weak relationship between personality traits and general prosocial behavior, though they conclude that personality traits are much better predictors of prosocial political behavior. We will address the literature on how personality relates to prosocial tendencies, as measured by economic games, after we introduce our hypotheses.

Political scientists have long recognized the importance of personality in politics. Since Lasswell (Reference Kurzban and Houser1930) examined the role of personality in politics by applying psychoanalytic theory, personality research has been a persistent domain of Political Science research. Researchers have examined the role of the authoritarian personality type (Adorno et al. Reference Adorno, Frenkel-Brunswik, Levinson and Sanford1950; Altemeyer Reference Altemeyer1988), the role of Big Five personality traits, and more recently the role of personality in mediating genetic influences in understanding politics (Alford, Funk and Hibbing Reference Alford and Hibbing2005; Fowler and Dawes Reference Fowler2008). The Big Five personality traits in particular have helped improve our understanding of partisanship, ideology, political sophistication, and political participation (see Gerber et al. Reference Goldstein, Yang, Omar, Turner and Thompson2010, for a review, and Mondak Reference McCrae and Costa2010 for detailed discussion).

The most common measures of Big Five personality traits are variations on Costa and McCrae’s (Reference Costa and McCrae1992) NEO measure. There are, however, at least two other scales that measure traits that are very similar to the Big Five, and in our MLMA we include studies that use these two measures as well. These scales are typically employed by asking respondents to agree or disagree with a number of self-referential statements. The Big Five traits were developed from a five-factor analysis of these responses. These factors have subsequently been labeled Agreeableness, Conscientiousness, Extraversion, Neuroticism, and Openness to Experience. We include two scales which are similar to this canonical Big Five. The HEXACO measure (Lee and Ashton Reference Lau, Sigelman, Heldman and Babbitt2004), is substantively very similar, but arises from a six-factor solution, rather than a five-factor one. The significant degree of similarity between these scales is underscored by the high correlationFootnote ¹ between the Big Five factors with their relevant HEXACO factors (Lee and Ashton Reference Lau, Sigelman, Heldman and Babbitt2004).

We also examine the Myers–Briggs Type Indicator (MBTI) (Myers, McCaulley and Most Reference Peysakhovich, Nowak and Rand1985), whose types of Extraversion/Introversion and Sensing/Intuition correlate very strongly with the Big Five traits of Extraversion and Openness to Experience, and whose types of Thinking/Feeling and Judging/Perceiving show moderately strong correlations with Agreeableness and Conscientiousness, respectively (McCrae and Costa Reference Lönnqvist, Verkasalo and Walkowitz1989). Given the evidence on the correspondence between the HEXACO scale, the Big Five traits and the MBTI, we feel comfortable including them all together in a single analysis, at least initially. The ability to do this is a function of the flexibility of MLMA in that it allows us to combine studies using each of the three scales, while also controlling for any effects that may arise from the measures themselves. Combining disparately measured concepts in this way would be much more problematic in a conventional meta-analysis because it would be difficult to control for these additional factors in an efficient way, especially with a small number of studies.

MLMA

Though common in psychology and epidemiology, meta-analyses are quite rare in Political Science (Morton and Williams Reference Mondak2010). Those that have been conducted and published in leading journals (e.g., Lau et al. Reference Lasswell1999; Doucouliagos and Ulubaşoğlu 2008) employ the conventional approach which relies on statistical techniques to standardize and then synthesize published effect sizes.

MLMA is a more flexible approach to meta-analysis that employs multilevel regression techniques to individual-level data from multiple studies. The reliance on individual observations rather than published effects as a unit of analysis allows us to synthesize results across any number of studies using any of the following types of data:

1. 1. individual characteristics (e.g., attitudes or personality traits);

2. 2. study-level variables (e.g., common investigator, common methodology, etc.);Footnote ²

3. 3. aggregated data (i.e., study-level summary statistics such as treatment effects and their variability); and

4. 4. a combination of (1) and (2).

MLMA, is a powerful technique that is capable of, among other things, examining differences between studies, conducting subgroup analyses, and uncovering heterogeneous treatment effects (Thompson and Higgins Reference Thompson and Higgins2002). What ties together the items on our list above is their reliance on modeling the data’s underlying multilevel structure. The multilevel structure allows us to treat the studies themselves as distinct sources of random error. Given the wide range of MLMA’s potential uses, we cannot apply them all in a single study. In our application to personality and prosociality, we focus on (1), (2), and (4).

Multilevel modeling conceptualizes individual data points—called “level 1” units—as being nested within larger groups—called “level 2” units. Often these level 2 groups are countries, states, or some other well-defined grouping, in MLMA the level 2 units are the studies themselves. The level 1 units are the individual participants or subjects in a study. Depending on the application, the level 2 units themselves may be nested in larger groups—level 3 units—and so on. While we only examine two-level multilevel structures in the present study, MLMA could, mutatis mutandis, be conducted with three or more levels.

Here, we focus on the promise that MLMA holds in terms of its capacity to synthesize individual-level data from many studies, while simultaneously accounting for study-level variance. Multilevel models can account for such variation in two distinct but complementary ways: through “random intercept” parameters, which model the intercept of each study as a random draw from a given distribution, and through “random slopes” parameters, which model one or more slope parameters in a similar way. With MLMA it is also possible to model interactions between individual and study-level variables (“cross-level” interactions). In our empirical application we only consider random intercepts and random slopes models, but cross-level interactions can be incorporated into MLMA in a manner identical to that in multilevel models more generally. For a discussion of such issues see Gelman and Hill (Reference Fowler and Kam2007) and Hox (Reference Hirsh and Peterson2010).

To employ a MLMA that can model individual data, study-level variables and a combination of both (cases 1, 2, and 4 above), we first specify a basic multilevel model with varying intercepts:

(1) $$\matrix{ {y_{i} \:\sim\:N\left( {\alpha _{{j\left[ i \right]}} {\plus}\bibeta {\mib x}_{i} ,\sigma _{y}^{2} } \right),} & {{\rm for}\;i\in\left\{ {1, \cdots ,N} \right\},} \cr } $$

(2) $$\matrix{ {\alpha _{j} \:\sim\:N\left( {\gamma {\plus}\bidelta {\mib z}_{j} ,\sigma _{\alpha }^{2} } \right),} & {{\rm for}\;j\in\left\{ {1, \cdots ,J} \right\},} \cr } $$

where y denotes the outcome of interest for observation i (nested in study j), x _i a vector of individual-level variables, and z _j a vector of study-level covariates. The remainder are parameters to be estimated in the model. In the context of MLMA, the parameter vector β in Equation 1 denotes the effects of individual-level predictors on the outcome of interest for individual observations in each study. The parameter α _j denotes the varying intercepts which are modeled as a function of study-level characteristics in Equation 2.

The model specification described above assumes that the effects of individual-level characteristics, such as specific treatments, are constant across studies. We can relax this assumption by allowing the coefficients in β to vary by study as well:

(3) $$\matrix{ {y_{i} \:\sim\:N\left( {\alpha _{{j\left[ i \right]}} {\plus}{\bi \bibeta } _{{j\left[ i \right]}} {\mib{x}} _{i} ,\sigma _{y}^{2} } \right),} & {{\rm for}\;i\in\left\{ {1, \cdots ,N} \right\},} \cr } $$

(4) $$\matrix{ {\alpha _{j} \:\sim\:N\left( {\gamma {\plus}\bidelta {\mib z}_{j} ,\sigma _{\alpha }^{2} } \right),} & {{\rm for}\;j\in\left\{ {1, \cdots ,J} \right\},} \cr } $$

(5) $$\matrix{ {{\mib \bibeta } _{j} \:\sim\:N\left( {{\mib \bimu } ,\Sigma _{\beta } } \right),} & {{\rm for}\;j\in\left\{ {1, \cdots ,J} \right\},} \cr } $$

Using such a specification, we allow treatment effects to vary by study but follow a common distribution with mean vector μ . Note that in Equation 5, Σ _β is diagonal covariance matrix with a unique variance parameter for each element in β .

We recommend the MLMA approach to scholars who plan meta-analyses that involve looking at published summary statistics, such as effect sizes. MLMA using summary data, like all multilevel approaches, is more efficient than its fixed- or random-effects counterparts. Here, we focus on the promise that MLMA holds for leveraging individual-level data nested within studies. An important advantage of MLMA is that it can leverage the statistical power of all the individual data points in the constituent studies, and therefore make efficient estimates of level 1 and level 2 effects with only a small number of studies.

Still, despite its efficiency, too few studies may limit the inferences MLMA can yield with respect to study-level factors. Nonetheless, there is no consensus on the minimum number of level 2 units required to properly estimate their coefficients in a multilevel framework. Rules of thumb vary from 8 at the low end to more than 100 at the high end (Stegmueller Reference Stegmueller2013). With standard frequentist techniques, having a small number of studies in MLMA could result in biased estimates of coefficients and incorrect standard errors at the study level. To avoid this problem, we employ Bayesian techniques that have been shown in Stegmueller (Reference Stegmueller2013) to yield largely unbiased point estimates and more reasonable credible intervals (compared to frequentist confidence intervals) even when the number of level 2 units is as low as 5. For most applications of MLMA, we would recommend that researchers use the Bayesian approach, and we would insist on it if one’s goal is to estimate the effects of study-level factors. MLMA is particularly useful in literatures in which diverse research designs are used, and in cases where there are thought to be important individual-level variables that are subject to significant individual variation.

HYPOTHESES

We hold clear hypotheses for two of the Big Five traits. The trait of Openness to Experience (sometimes referred to as “intellect”) is aptly named, and reflects a general orientation toward learning and experiencing new things. This trait has also been found to relate to self-reported social Openness and willing to trust others, and is negatively related to prejudice (Flynn Reference Fischbacher, Gächter and Fehr2005). We would therefore expect that individuals higher in Openness to Experience will exhibit more prosocial behavior. The results of previous literature in this area in no case show a significant negative relation between Openness and prosocial tendencies, but there are many more null findings than significantly positive ones. It is therefore appropriate to question whether these Openness findings are part of a pattern, or merely statistical anomalies. We anticipate, based on the literature on Openness, that a meta-analysis will detect a significant positive relation between Openness and prosocial behavior in experimental games.

The trait of Agreeableness carries with it a sense of likeability, but also care for others, and sometimes a tendency toward compliance. In short, people high in this trait are “nice” (McCrae and Costa Reference McCrae and Costa1995). This, too, would sensibly generate more prosocial goals and behavior. Again, prior findings are not so clear cut. While no study we have examined detects a significantly negative relation between Agreeableness and prosocial game behavior, only four detect a significant positive one. Again, based on the conceptual underpinnings of the trait, we anticipate a significant positive relation between Agreeableness and prosocial tendencies in experimental games.

For the remaining traits, we have no clear theoretically driven hypotheses. Extraversion is perhaps the most well-known Big Five trait. Some characterize it primarily as an orientation toward enjoying social interactions (Ashton and Lee Reference Ashton and Lee2007; Denissen and Penke Reference Denissen and Penke2008), whereas others emphasize assertiveness and activity seeking, in addition to social warmth (McCrae and Costa Reference McCrae and Costa1995). It is not clear that Extraversion has direct implications for prosocial behavior in games, in that the sociable aspect of Extraversion might lead one to expect greater prosocial behavior, but the tendency among extroverts to assert their own views and interests might suggest the reverse pattern. Indeed, the papers in our sample have contradictory findings with respect to Extraversion. Those who are Conscientious demonstrate a tendency to strive and plan. They are diligent and dependable (Ashton and Lee Reference Ashton and Lee2007; Denissen and Penke Reference Denissen and Penke2008), as well as competent, orderly, and self-disciplined (McCrae and Costa Reference McCrae and Costa1995). It is unclear that this has any implications for prosociality, in that we would expect those who are conscientious to diligently and competently follow a strategy, but the end goal of that strategy (the common good or personal gain) would be best determined by other traits and factors. Indeed, only one paper in our sample shows a significant impact of Conscientiousness upon prosocial tendencies. Those who are high in the final Big Five trait, Neuroticism (sometimes called “emotionality”; McCrae and Costa Reference McCrae and Costa1995), are anxious and sensitive, particularly when confronted with threats (Ashton and Lee Reference Ashton and Lee2007; Denissen and Penke Reference Denissen and Penke2008), but it is unclear how Neuroticism might relate to prosocial behavior or experimental games. Again, we find mixed results in the studies we identify examining Neuroticism and prosocial tendencies.

For the study-level variables, we lack sufficient prior evidence to form any theoretically driven hypotheses. It is more intuitive to think that monetary payments might induce less prosocial behavior when compared with non-incentivized studies in which prosocial behavior does not imply monetary costs. Indeed in the only study we are aware of that directly tests the effects of incentives on prosociality, Lönnqvist, Verkasalo and Walkowitz (Reference Loewen2011) find evidence that participants are less generous in incentivized prisoner’s dilemma games rather than hypothetical ones. Hence, we take an exploratory approach to the expected role of payments on prosocial behavior while we simply use the other study-level variables to better control for study-level variance.

MLMA: DATA AND ANALYSIS

Evidence on the relationship between personality and prosocial behavior is decidedly mixed. Even when limiting our focus to the Big Five type measures, there is a tremendous amount of heterogeneity in results across studies. Table 1 lists each of the studies we use in our MLMA along with the significant findings (as reported in the study’s manuscript) and their direction for each study. A number of studies do show that Openness and Agreeableness are positively associated with prosocial behavior, though others show no such relationship. No two studies, however, have the same set of findings with respect to all factors: each factor is significant in at least one study, but never in all of them, or even a majority of them. In some studies most of the Big Five traits predict behavior (e.g., Ben-Ner and Kramer Reference Ben-Ner and Kramer2011; Brocklebank, Lewis and Bates Reference Denissen and Penke2011), whereas in others, no trait does (e.g., Kurzban and Houser Reference Koole, Jager, van den Berg, Vlek and Hofstee2001). In some cases, depending on the study, a given trait can be both positively and negatively correlated with prosociality. For example, Koole et al. (Reference Hox, Moerbeek and van de Schoot2001) and Brocklebank, Lewis and Bates (Reference Denissen and Penke2011) find that Extraversion and Neuroticism are negatively and significantly associated with prosociality. In other studies, Extraversion (Swope et al. Reference Thompson, Turner and Warn2008) and Neuroticism (Hirsh and Peterson Reference Hilbig, Zettler and Heydasch2009) are found to have a positive and significant association. Sometimes, even negative and positive correlations appear in different studies in the same paper (e.g., Extraversion in Ben-Ner and Kramer Reference Ben-Ner and Kramer2011). This variety of mixed and contradictory findings regarding how personality traits predict prosocial tendencies is precisely the situation in which MLMA is most useful. Meta-analysis can clarify these mixed findings and tell us whether sporadic significant findings are likely to be a result of Type I errors, or part of a systematic pattern of results. Table 1 summarizes the findings from the studies in our sample.

Table 1

Studies Included in Meta-Analysis

Note: DG=dictator game; PGG=public goods game; PD=prisoner’s dilemma; CPRD=common pool resource dilemma; UG=ultimatum game; TG=trust game; MBTI=Myers–Briggs Type Indicator; N=Neuroticism; E=Extraversion; C=Conscientiousness; A=Agreeableness; O=Openness; (−)=statistically significant, negative association; (+)=statistically significant, positive association.

Coding of the Variables

Because the studies in our sample use different game forms and parameterizations, we must make our dependent variable—prosociality—commensurable across studies. To do so, we determine the upper and lower bounds of prosocial behavior which were available to participants, and then re-scaled the values from 0 to 1. As discussed above, we operationalize prosocial behavior as the extent to which behavior does not coincide with the equilibrium prediction of defection in cooperative games or retaining all rewards for oneself in distributional games. For example, in a dictator game with an endowment of $10, if a participant allocates $3 to their counterpart, then our measure of prosociality would be 0.3. For studies which involved a prisoner’s dilemma, “cooperate” was coded as 1 and “defect” was coded as 0. In total, our studies include five distinct games: prisoner’s dilemma (PD), public goods game (PGG), common pool resource dilemma (CPRD), ultimatum game (UG), and dictator game (DG) and trust game (TG). These are summarized in Table 2, along with the normalization rule for each type. In both UG and TG, there are two distinct roles. In order to simplify our analysis of these games, we analyze only the “distributive” roles in these two games: in UG, the role of the proposer—who proposes a split of a fixed sum between herself and the responder—and in TG the role of the trustee—who receives a transfer from the trustor, and then must decide how to split this fixed sum between them. We define “distributional” games as those involving the allocation of a pot of money that is fixed at the time of the decision---e.g., DG, UG (proposer) and TG (trustee)—and “cooperative” as those that involve a choice between cooperation and defection (PD, CPRD), or contribution and non-contribution (PGG). Strategically, these cooperative games are equivalent, with defection/non-contribution/full-extraction the unique equilibrium in the one-shot context. In these cases, we code cooperation and full contribution as 1 and defection and zero contribution as 0. There are of course alternative methods of scaling these outcome measures, but we feel that this approach is the most straightforward and generally better preserves the original study’s scaling. Further details about the specific normalization procedure that was used for each study can be found in the Online Appendix Table A.1.

Table 2

Game Types and Normalization of Outcome Measures

Proscociality, like personality, is now thought to be a stable and domain general individual trait (Peysakhovich, Nowak and Rand Reference Pothos, Perry, Corr, Matthew and Busemeyer2014), justifying the pooling of these different games in our analysis. In addition to Peysakhovich, Nowak and Rand (Reference Pothos, Perry, Corr, Matthew and Busemeyer2014), who find behavioral stability across four games (DG, PGG, and both roles in TG); Yamagishi et al. (Reference Yamagishi, Mifune, Li, Shinada, Hashimoto, Horita, Miura, Inukai, Tanida, Kiyonari, Takagishi and Simunovic.2013) also find that prosocial behavioral tendencies are consistent across PD, TG, and DG. To the extent that there is additional variance as a function of the different types of games, it will be incorporated in the multilevel structure of the MLMA estimations. We also include a dummy variable indicating whether the game type is cooperative or distributional in nature.

Finally, a note on our Payment variable. Psychologists rely primarily on participant pools composed of students who are given course credit in exchange for their participation in the experiment. On the other hand, the norms of experimental Economics require that subjects be compensated based on an incentivized payment scheme designed to induce preferences over experimental alternatives (Smith Reference Schmitt, Shupp, Swope and Mayer1976). The sample of studies included in this analysis spans both the Psychology and Economics literature. A study is coded as offering incentivized payments if it offers monetary payments and the amount of these payments depends on the choices of the participants. Otherwise, studies are coded as unincentivized. All 15 studies in our sample were conducted with student populations.

Multilevel Models and Results

Since we have access to very few common covariates in the data sets, the models we present here have to be parsimonious. As such, in the context of MLMA, there is a trade-off between complex model specifications and the total number of studies which can be included in the analysis. However, the multilevel nature of MLMA allows us to capture any remaining study-level effects as sources of variation in the random intercepts and random slopes, which reduces the chance that parsimony in variable selection results in underspecification.

As a first step, we specify three varying intercept models, as described in Equation 2. Model 1 estimates the effects of personality on prosocial behavior across the 12 studies for which we have prosociality and personality data. Two of our studies (Schmitt et al. Reference Sautter, Littvay and Bearnes2004; Swope et al. Reference Thompson, Turner and Warn2008) employ MBTI. The four dimensions of MBTI have been shown to be well correlated with four of the five Big Five factors, but, there is no MBTI factor that is analogous to Neuroticism so we exclude that factor in Model 1. Model 1 includes the individual-level variables of Extraversion, Openness, Agreeableness, and Conscientiousness, as well as the following study-level variables: a dummy for whether the study was incentivized (Payment), a dummy indicating the use of MBTI (MBTI), dummies for experimental game type—“cooperative”Footnote ³ —and dummy variables for common authorship of studies (Hilbig and Ben-Ner). Additionally, to the extent that HEXACO differs from the traditional measurement of the Big Five, our Hilbig author dummy would pick up this difference as well, as all three Hilbig et al. studies, and no others, employ the HEXACO measure. MLMA easily allows us to determine whether study-level variables that capture crucial conceptual distinctions have any significant effect on our outcomes.

If we include MBTI-based studies we cannot investigate the effect of Neuroticism. So, to estimate the effect of Neuroticism, and to test the robustness of our results, Model 2 includes only the ten studies that employ the Big Five/HEXACO measurements of personality. In Model 2, we include the five personality dimensions as well as study-level variables relating to shared authorship and game type. Because the MBTI studies are dropped from this sample, so is the variable in the specification of Model 2.

Finally, in Model 3, we include prosociality and incentivization data from an additional three studies that did not have personality data. This allows us to further examine the potential effect of incentivization in a larger set of studies.

Following the notation laid out above in Equations 1 and 2, our multilevel specifications are as follows, for individuals i∈{1, … , N} within studies j∈{1, … , J}:Footnote ^{4 ,} Footnote ⁵

(6) $$\eqalignno{ {\rm Level}\;1\!\colon\,y_{i} \:\sim\, & N\big( {\alpha _{{j\left[ i \right]}} {\plus}\beta _{1} {\times}Openness_{i} {\plus}\beta _{2} {\times}Agreeableness_{i} {\plus}\beta _{3} {\times}Extraversion_{i} }\cr & {\plus}\beta _{4} {\times}Conscientiousness_{i} {\plus}\beta _{5} {\times}Neuroticism_{i} ,\sigma _{y}^{2} ),$$

(7) $$\eqalignno{ {\rm Level}\;2\!\colon\,\alpha _{{j\left[ i \right]}} \:\sim\, & N ( {\gamma {\plus}\delta _{1} {\times}Payment_{j} {\plus}\delta _{2} {\times}Hilbig_{j} {\plus}\delta _{3} {\times}BenNer_{j} {\plus}\delta _{4} {\times}MBTI_{j} }\cr & {\plus}\delta _{5}{\times}Cooperative_{j} ,\sigma _{\alpha }^{2} \big).$$

Throughout the models, we specify the following weakly informative prior distributions for our parameters of interest:

(8) $$\alpha _{j} \:\sim\:N\left( {0,1} \right),$$

(9) $${\mib \bibeta } \:\sim\:N\left( {0,1} \right),$$

(10) $$\gamma _{j} \:\sim\:N\left( {0,1} \right),$$

(11) $${\mib \bidelta } _{j} \:\sim\:N\left( {0,1} \right).$$

Note that the prior distributions are vague in relation to the scaling of the dependent and independent variables included in the model. The priors for the remaining variance parameters were specified as uniform. Alternatively, employing Cauchy(0,5) or non-informative uniform prior distributions for all parameters does not affect the substantive results (see e.g., Gelman et al. Reference Gelman and Hill2006, for a discussion of prior choice on variance parameters in multilevel models). All models were estimated via Hamiltonian Monte Carlo implemented in Stan Development Team (Reference Smith2016). Individual chain lengths were set at 10,000 or 20,000 in order to assure convergence. Trace plots of the chains as well as $$\hat{R}$$ statistics for each model (not shown) indicate that the chains mixed well (cf. Gelman et al. Reference Gelman2014).

Table 3 displays the results from our three models, consisting of the posterior mean and its 95 percent credible interval.Footnote ⁶ The credible interval describes the range of parameter values that encompasses 95 percent of the posterior distribution’s probability mass. As such, there is a 95 percent probability that a given parameter lies in the respective credible interval. Figure 1 graphically displays this same information with each dot representing the mean prediction of the parameter value and the bar representing the 95 percent credible interval.

Fig. 1

Multilevel meta-analysis results: predicting prosocial behavior Note: For each model and each parameter, the dot represents the posterior mean and the bar represents the 95 percent credible interval.

Table 3

Prosociality, Personality and Incentivization: Multilevel Regression Results

Note: MBTI=Myers–Briggs Type Indicator.

At the individual level, higher levels of Agreeableness and Openness increased prosociality in both Models 1 and 2. Though the magnitude of the effect is relatively small, the results for these two traits are quite robust. Irrespective of the specification of the model, these two factors are consistently associated with increased prosociality. For the remaining three traits, the effects are smaller and there is much larger uncertainty about their direction. Given the inconsistency of the findings when looking at each of the studies in our sample in isolation, it is remarkable that the results with respect to Agreeableness and Openness are so robust. These results demonstrate the power of MLMA in that it can extract aggregate relationships from a large number of studies that would not be apparent when looking at any of the studies on their own.

Moving on to the study-level variables, we do not find that any study-level factors systematically affect prosociality. This may simply be because of a lack of statistical power at the study level. Additional analyses (not shown) controlling for each type of game (summarized in Table 2) yield the same conclusion of no study-level effects, but such a model leaves us with very few degrees of freedom at the study-level. Thus, from our analysis we can conclude that incentivization, at least in the set of studies we have, does not play a significant role in prosocial behavior.

In order to ensure that the effects are not driven solely by a single study, we test whether the results are robust to individual studies being removed from the analysis. Using a jack-knife method, we find that the one-by-one exclusion of each of the 12 studies included in Model 1 does not change the estimates substantially. These results are included in the Online Appendix Figure A.3. Both Agreeableness and Openness remain significant in each of the 12 estimations. As such, it does not appear to be the case that the results are driven by a single study included in the meta-analysis. A more systematic way to address this question, however, is to directly investigate the variation in individual-level effects across the studies. To do so, we specify a model which adds a random slopes component to the model outlined in Equations 3 through 5. The model takes the following form:

(12) $$\eqalignno{ {\rm Level}\,1\!\colon\!\,y_{i} \:\sim\,& N\big( {\alpha _{{j\left[ i \right]}} {\plus}\beta _{{j\left[ i \right],1}} {\times}Openness_{i} {\plus}\beta _{{j\left[ i \right],2}} {\times}Agreeableness_{i} } \cr & {\plus}\beta _{{j\left[ i \right],3}} {\times} Extraversion_{i} {\plus}\beta _{{j\left[ i \right],4}} {\times}Conscientiousness_{i} ,\sigma _{y}^{2} \big),$$

(13) $$\eqalignno{ {\rm Level}\;2\:\colon\,\alpha _{j} \:\sim\,& N\big( {\gamma {\plus}\delta _{1} {\times}Payment_{j} {\plus}\delta _{2} {\times}Hilbig_{j} {\plus}\delta _{3} {\times}BenNer_{j} {\plus}\delta _{4} {\times}MBTI_{j} } \cr &{\plus}\delta _{5} {\times}Cooperative_{j} ,\sigma _{\alpha }^{2} \big)$$

(14) $${\hskip -160pt {\mib \bibeta } _{j} \:\sim\:N\left( {{\mib \bimu } ,\Sigma _{\beta } } \right)},$$

where prior specifications are equivalent to the previous models. The results are presented in Figure 2. It displays the individual-level effects for each of the 12 studies included in the analysis, the aggregate effects of individual-level variables across all studies, as well as study-level effects.

Fig. 2

Multilevel meta-analysis results: predicting prosocial behavior (varying slopes) Note: For each model and each parameter, the dot represents the posterior mean and the bar represents the 95 percent credible interval.

We can see that there is some substantial variation in individual-level effects between studies. While the posterior mean for the coefficients of Agreeableness and Openness is positive in most of the studies, the effects appear to be substantially smaller in some of the experiments than in the others—that is to say there is cross-study variation in the strength of the effect of these personality factors on prosociality. Still, the aggregated individual-level effects at the bottom of the figure (in the “Overall” panel) show the same pattern as the previous analysis. As such, even though there is some variation in effects between experiments, we recover a substantial positive influence of Agreeableness and Openness on prosocial behavior when combining the individual studies in a common framework.