1. Introduction

Over the course of a decade, Scott S. Reuben, an influential American anesthesiologist, published a series of articles examining the role of cyclooxygenase-2 specific inhibitors in controlling postoperative pain following orthopedic surgery (Buckwalter, Tolo, and O’Keefe Reference Buckwalter, Tolo and O’Keefe2015). In 2010, though, Reuben was convicted of fraud for accepting grants from drug companies to perform clinical trials and then fabricating the data and publishing ‘results’ without having conducted trials.Footnote ¹ An investigation determined that at least 21 of Reuben’s articles contained fabricated data; all of these articles were subsequently retracted (Shafer Reference Shafer2015).

Before retraction, the articles had collectively obtained nearly 1,200 citations. By 2014, however, a case study found that roughly half of these articles continued to be cited consistently, and only one-quarter of the citing articles clearly stated that Reuben’s work had been retracted (Bornemann-Cimenti, Szilagyi, and Sandner-Kiesling Reference Bornemann-Cimenti, Szilagyi and Sandner-Kiesling2016). It seems remarkable that scientific findings of this sort would be so widely used in the literature after being withdrawn as fraudulent, but this is by no means an isolated case. Studies of retracted results have found that they are often widely cited after retraction. Moreover, Cor and Sood (Reference Cor and Sood2021) find that 91% of these postretraction citations are approving of the original research (see also Budd, Sievert, and Schultz Reference Budd, Sievert and Schultz1998; Neale, Dailey, and Abrams Reference Neale, Dailey and Abrams2010).

In this article, we examine how false information perpetuates, even in the light of correction. To explore the dynamics of retraction, we develop agent-based models in which actors on networks share and spread beliefs. Our models make the following fundamental assumption: there is an asymmetry in the way a new finding spreads versus a retracted one. While individuals are apt to share novel information, they only tend to share retractions when the topic of conversation already centers on the false information. This assumption is consistent with Grice’s (Reference Grice, Cole and Morgan1975) maxim of relation that, in conversation, one should be relevant.

In our models, false beliefs can take a long time to eliminate after retraction. Additionally, we find that whenever information gets old, in the sense that individuals stop sharing information after some time frame, false beliefs can persist indefinitely even when a retraction is issued. This occurs without any biased reasoning—we assume that any individuals exposed to a retraction will change their minds. The persistence of false beliefs is a direct result of the fact that some individuals who received false beliefs from their neighbors happen never to receive a retraction.

Additionally, we consider the conditions under which retractions are more or less successful. We find there can be unexpected interactions between how long a retraction is delayed and how effective it is. In particular, a retraction that is introduced later (i.e., once a false belief is held widely) may be more efficacious, because it is relevant to a greater number of individuals. We also explore how network structure influences these processes. We examine small-world networks and preferential-attachment networks to see what effect the location of a retraction has on its success and find that retractions are more successful when issued from the original source. Additionally, we show that homophily (i.e., disproportionate in-group communication) can slow the spread of a retraction, especially if it is introduced in a subgroup where the false belief is not widely held.

Our article proceeds as follows. In section 2, we give an overview of relevant literature and introduce the modeling framework on which this article draws. In section 3, we present the simplest model explored here and describe its fundamental behavior. Sections 3.2, 3.3, and 3.4 extend these results to several different scenarios intended to tease out the dynamics of retraction. Section 4 concludes.

2. Retraction and Contagion

As we have seen, retraction of a scientific paper does not always work the way it should. There are two things we should distinguish here. First is whether articles are cited after retraction, and second is whether scientists continue to hold beliefs now known to be false. Our models actually consider the second question: How might false beliefs persist in the face of retraction/refutation of a result? However, the empirical literature focuses on citations, rather than underlying beliefs. Given scientific norms against citing known falsehoods, we take this literature to provide evidence (albeit imperfect) about false beliefs in the face of retraction.

Papers typically are retracted because of error, fraud, or failure to replicate (Steen Reference Steen2011; Wager and Williams Reference Wager and Williams2011; Fang, Steen, and Casadevall Reference Fang, Steen and Casadevall2012), and retractions have become increasingly common (Cokol, Ozbay, and Rodriguez-Esteban Reference Cokol, Ozbay and Rodriguez-Esteban2008; Grieneisen and Zhang Reference Grieneisen and Zhang2012; Steen, Casadevall, and Fang Reference Steen, Casadevall and Fang2013; Madlock-Brown and Eichmann Reference Madlock-Brown and Eichmann2015). Importantly, study after study has found that, even after retraction, papers continue to be cited, sometimes for years (Pfeifer and Snodgrass Reference Pfeifer and Snodgrass1990; Budd et al. Reference Budd, Sievert and Schultz1998; Madlock-Brown and Eichmann Reference Madlock-Brown and Eichmann2015; van der Vet and Nijveen Reference van der Vet and Nijveen2016; Cor and Sood Reference Cor and Sood2021). Some studies find declines in citation rate after retraction; others, no change or even an increase in citation rate (Madlock-Brown and Eichmann Reference Madlock-Brown and Eichmann2015; van der Vet and Nijveen Reference van der Vet and Nijveen2016; Cor and Sood Reference Cor and Sood2021).

There is another relevant kind of retraction that occurs in the news media. This often involves issuing an erratum or apology concerning a specific claim instead of retracting an entire article or report. There are few data about the effects of news media retraction on belief. However, in many cases claims continue to spread widely after media retraction. During the COVID-19 pandemic, for instance, claims that the virus strips hemoglobin of iron were tweeted thousands of times after the Medium article originating them was removed. We do not claim here that retraction in science and retraction in media are just the same, but we think that many cases from both arenas can be captured by the models we develop.

Our models draw on the network-contagion modeling paradigm.Footnote ² These models are widely used to represent the spread of both disease and belief (Hayhoe, Alajaji, and Gharesifard Reference Hayhoe, Alajaji and Gharesifard2018). The idea is that just as infections spread to susceptible individuals, in some cases beliefs spread in social networks from those who hold them to those who do not.

There is an extensive literature on social-contagion models to which it would be impossible to do justice here. Nonetheless, the basic idea is simple. We use network models where the nodes are individuals (scientists, journalists, members of the public), and the vertices are communication channels between them. False information can spread from individual to individual. Information about a retraction can also spread; although, as we outline below, we assume there is an asymmetry in how false reports and retractions spread.Footnote ³

Of course, not all beliefs are well modeled as contagions. These models are best tuned to beliefs that spread unit like from person to person and are easily adopted. In many cases, individuals depend on evidence to form beliefs in a rational or semirational way and can hold graded degrees of belief. This is especially true in scientific communities, where beliefs are expected to be evidence based. (For this reason, philosophers of science have more often used the network-epistemology framework to represent the spread of scientific beliefs [Zollman Reference Zollman2013; Goldman and O’Connor Reference Goldman, O’Connor and Zalta2019].)

What cases, then, are appropriate targets of the investigation here? The models apply to cases in which beliefs are adopted relatively unreflectively. For instance, scientists may trust freestanding claims published by peers (e.g., that a particular hormone causes hunger or that a drug effectively reduces pain). Media consumers may trust journalists who claim there is a fire in some city, that Moonstruck is a great movie, or that SARS-CoV-2 strips iron from hemoglobin. In cases of beliefs that are controversial or that depend on background theory, so that individuals engage in calculation or reasoning before adopting them, the models will be less applicable. Additionally, the models will apply best in cases in which retractions/refutations are apparent (i.e., when it becomes uncontroversial that the original claim is, in fact, false).

3. Model and Results

In this section, we present a base model and results. We also examine variations on this foundation to obtain a clearer picture of the dynamics of false and retracted information.

Suppose we have a population of N individuals. Each has a belief about the world, contingent on information he or she has received. In particular, we consider false information, representing an erroneous or fraudulent scientific finding or a misleading news item, and also retracted (or corrected) information. Individuals can thus have three belief states: neutral (before receiving any information), false (having received false information), and true or retracted (having received the retraction).

We start simulations with the majority of the population holding neutral beliefs. One individual holds a false belief. A retraction is introduced to one individual either at the start of the simulation or after some number of rounds. At each time step, we pair two network neighbors at random to interact. This is done by randomly selecting a focal individual and then randomly selecting a partner from the individual’s neighbors.Footnote ⁴

Beliefs spread as follows. If one individual holds the false belief and the other a neutral belief, we assume the false belief always spreads. The idea is that the individuals pass on new information per the contagion framework discussed in section 2. Retracted beliefs also spread in this way but only to those who already hold the false belief. As mentioned in the introduction, this captures the idea that sharing some novel, albeit false, information is more apt than sharing a retraction of that same information, because a retraction is parasitic on the context. It is only interesting or relevant to those who already hold false beliefs. We might alternatively interpret the model in the following way: if one individual mentions a false belief to a partner who knows it has been retracted, the partner will then share the retraction. (To be clear, there are cases not well captured by these models in which retractions are of interest independent of initial findings.)

3.1. Complete Network

We begin with a population connected in a complete network, as in figure 1. This means every agent is eligible for pairing with the focal individual on every given round. (We might also describe this as a population without a network, where individuals meet randomly for interaction.) When, after some time step, no further belief revision can occur, we say that the population is in a stable state. For a model of this sort, of any size, several population states are stable. For all of these, it is the case that no individual in the population holds a false belief. In other words, for the model to be in a stable state, every individual must hold either neutral or retracted information. This fact actually holds of a broader set of networks—those that are connected (meaning there is a path between every pair of nodes). Intuitively, this occurs because the retraction can spread to all individuals with the false belief but not to individuals with a neutral belief. Since all individuals are connected, eventually each individual with a false belief will pair with someone holding the retraction and end up with the true belief. Either all members of the network are exposed to the false belief, and end up eventually with the true belief, or else some of them never see the false belief and at the end will have neutral beliefs that are stable because there are no false beliefs left in the network.

Proposition 1. For any connected network, a population configuration is stable if and only if no individual holds a false belief.

Proof. Assume that no individual in the population holds a false belief. Then every individual in the population either has neutral information or retracted information.

On a particular trial, we pair two individuals, A and B. Either A and B have the same information, or they have different information. If they have the same information—both neutral or both retracted—then they do not update, so the state configuration remains unchanged. This leaves the case in which they have different information. Without loss of generality, assume that A holds a neutral belief and B holds a retracted belief. In this case, ex hypothesi, they do not share information. Therefore, the state configuration remains unchanged.

We proceed via the contrapositive.

Assume at least one individual in the population holds false information. By assumption, at least one individual in the network holds a retracted belief. Because the network is connected, there is at least one path of vertices connecting these individuals. On every round, there is a positive probability that any two neighbors meet. Thus, from any starting time step, in the limit, all neighbors on this path will be selected to meet with probability 1. Consider moving down the path from the individual with false belief. If the individual with false belief has a neighbor with neutral or retracted belief, the configuration is thus not stable. If the individual’s neighbor has a false belief, but if the next neighbor has a neutral or retracted belief, the configuration is not stable, and so on. Since the final node on the path holds the retracted belief, the subnetwork consisting in this path is not stable, and thus the entire network is not in a stable configuration. QED

Furthermore, the population will always converge toward one of these stable states given enough time. This fact is stated in corollary 1.

Figure 1.

Complete network, with a population of $N=10$ agents.

Corollary 1. For a connected network, in the limit, the population configuration always reaches a stable state.

Proof. This follows from the proof of proposition 1. QED

Since the long-term fate of false belief is reasonably predictable, in the sense just outlined, for now, we are interested in the ‘medium-run’ results. How long does false information persist in this setup? How widely does it spread? How do alterations influence this process? To answer these questions, we run simulations.Footnote ⁵

We examine populations of $N=10$, 50, 100, and 1,000 individuals. In each case, one individual from the population is given false information, and one the retraction, at the outset. Simulations proceed round by round. Reported results are averages across simulations.Footnote ⁶ In each case, the population converges to a stable configuration (as expected). The typical behavior of the model involves first the spread of the false belief, sometimes saturating the population (or coming close), and subsequently the spread of the retraction until all individuals hold true or neutral beliefs. In figure 2, we can see this process for three population sizes. The X-axis tracks time (note the different time scales), and the Y-axis tracks the proportion of the population in the three possible belief states.

Figure 2.

Simulation results for a population on a complete network: (a) N = 10; (b) N = 50; (c) N = 1,000.

There are a few things to notice. A group of 10 individuals sees, on average, 40% of the population holding false beliefs at some point. When we increase the population to 1,000 individuals, this becomes 90%. In other words, for a larger population, on average, false beliefs spread further. This is because, for smaller networks, the false belief is easier to nip in the bud. Although the proportion of individuals holding retracted and false beliefs at the outset is equivalent for a given population size (10% of the population for $N=10$ and 0.1% of the population for $N=\mathrm{1,000}$), for smaller networks, it is more likely that early pairings bring all those with the false belief in contact with the retraction.Footnote ⁷ For similar reasons, there appears to be a strong, positive relationship between the size of the population and the average length of time (number of time steps) for which an agent holds her false belief, as is evident in figure 3.

Figure 3.

Length of time false beliefs are held for populations of different sizes. For larger populations, false beliefs are held longer on average.

These observations could be outlined analytically, given that pairings are based solely on probability distributions for the complete network (i.e., they depend entirely on how many individuals hold each type of belief at a given time step and how large the population is). While the results here are perhaps unsurprising, we now extend this model by looking at some variations.

3.1.1. Delayed Retraction

We assume in the base model that the false information and the retraction enter the network at the outset of an episode. This assumption is perhaps accurate with something like real-time fact-checking during a political debate; however, in the case of replication and retraction of scientific studies, there is usually a (potentially significant) delay before the retraction enters into the population. For instance, Fang et al. (Reference Fang, Steen and Casadevall2012) find that it takes an average of 3 years for a finding to be retracted. In some cases, the discovery of fraud can trigger retraction of an author’s older articles, including those published many years previously.

We delay the introduction of retracted information by a parameter, delay. Otherwise, the simulations are run as before. The delay generally tends to increase the number of individuals who ever hold the false belief. It also increases the average amount of time individuals hold false beliefs. However, for relatively short delays, there is little impact on the average time false beliefs are held. This is shown in figure 4. As is apparent, for $N=100$ there is virtually no change for delay ≤ 200. This is because if a retraction is issued when relatively few individuals hold the false belief, it spreads very slowly, since few individuals will take it up. If it is issued once false beliefs have saturated the community, each interaction is one in which the retraction will spread, so it catches on more quickly. As we will see later in the article, this means that, perhaps surprisingly, under some conditions, later retractions are successful.

Figure 4.

Delayed retraction in complete networks. As retractions are delayed, false beliefs are held for a longer time on average. For short delays, there is relatively little effect. $N=100$.

3.1.2. Timed Novelty

We also previously assumed that agents share information indefinitely. However, in many cases new information is more readily shared than old. We model this as follows. Each agent—upon receiving novel (false or retracted) information—only shares it for a specified time frame. This small, realistic change means that the analytic results in proposition 1 and corollary 1 no longer hold. Now false information may stably persist. This happens when enough time has passed that no one shares the retraction, even though some individuals in the network hold false beliefs.

As before, in each case we give one individual from the population false information and one individual the correction. Figure 5 shows results for population size $N=100$.Footnote ⁸ The X-axis tracks time, and the Y-axis tracks the average belief state of the population over simulations.

Figure 5.

Simulation results for the model with timed novelty on a complete network with a population of $N=100$ individuals: (a) time frame = 400; (b) time frame = 300; (c) time frame = 200; (d) time frame = 100; (e) time frame = 75; (f) time frame = 25.

As is apparent, when the window for which individuals share beliefs is long, simulations are much like the previous models (fig. 5a). The false belief spreads and then is supplanted by the true belief. As the sharing window gets shorter, though, false beliefs start to persist alongside retracted beliefs because the retraction is no longer shared (figs. 5b and 5c). As the window grows shorter still, the retraction does not spread at all, and only false and neutral beliefs persist (figs. 5d and 5e). For the shortest time windows, neither belief spreads (fig. 5f). In other words, there is a regime of moderate sharing in which false beliefs are common at the end of a simulation.

Why do we see these effects? For the false information to begin to spread in the first place, the individuals holding the false belief must be chosen within the early rounds before they stop sharing. Since there are 100 individuals in the population, each equally likely to be picked on a given round, the agent holding the false information has a reasonably low probability of being chosen within a short frame of time. It is harder still for the retraction to spread since this requires that the individual with the retracted belief be selected in the first rounds and also meet a neighbor with the false belief.

Since false beliefs are stable in these models, we can ask: Under different regimes, what proportion of the population ends up with a false belief? Figure 6 shows the qualitative trend visible in figure 5. Each color band represents the average proportion of individuals at that end state for some time frame of belief sharing. As the time frame grows, neutral beliefs decrease, the proportion of stable false beliefs grows and then shrinks, and the proportion of retracted beliefs grows.Footnote ⁹

Figure 6.

Timed novelty and average end state of belief. As sharing time increases, false beliefs become more prevalent and then less prevalent. $N=100$; delay$\mathrm{delay}=0$.

We have been discussing independent variations to the base model, but there are interaction effects between these parameters. In particular, a delayed retraction can interact with timed novelty in the following way. When there is only one individual with false information and one individual with retracted information, on any given round, there is a minuscule probability that the retracted information can spread. However, if almost everyone in the population already holds a false belief, this probability is much higher. We have already seen that a delay in retraction does not necessarily lead to a significant increase in average length of false belief, for this reason. However, in some cases, a delay in retraction can actually improve belief because a population where the false belief is widely held will be more receptive to the retraction. It can catch on like a contagion. Figure 7 shows this. As the delay increases, we see a decrease in the final number of false beliefs because the delay makes the retraction relevant during the time frame when individuals are sharing it.

Figure 7.

Timed novelty, delayed retraction, and average end state of belief. For a fixed sharing time frame of 200, as the delay in retraction increases, final false beliefs decrease, and retracted beliefs increase. $N=100$.

3.2. Small-World Networks

To this point, we have considered a trivial network structure that assumes every individual meets every other randomly. But some scientists are regular communicators, and others do not interact at all. We can vary network structure to restrict the individuals with whom any particular agent can share information—two agents are eligible for pairing just in case they are connected.

We consider small-world networks because many real-world social networks exhibit small-world properties (Telesford et al. Reference Telesford, Joyce, Hayasaka, Burdette and Laurienti2011). Small-world networks are defined by short average path length (i.e., the distance between any pair of nodes is relatively short) and high clustering coefficients. If your friends are all friends with one another, then you have a high clustering coefficient. Small-world networks also tend to have hubs, which are nodes with higher-than-average connections (e.g., a popular individual at the center of a clique is a ‘hub’ for social interactions).Footnote ¹⁰

We generate networks according to the Watts-Strogatz model (Watts and Strogatz Reference Watts and Strogatz1998). The algorithm begins with a network with N nodes, each of which connects to its K nearest neighbors. Then, for every node, n_i, it takes every edge connecting n_i to its K/2 rightmost neighbors and rewires it with probability p. Rewiring is done such that the new link connects (n_i, n_k), where k is chosen at random, subject to the constraints that there are no loops and no duplications. We examine models with $(k,p)=(8,0.1)$, (16, 0.07), and (32, 0.016).Footnote ¹¹ Figure 8 shows an example of a regular lattice, a small-world network, and a random network. As the figure makes clear, the small-world network is related to both of these. The more rewiring in the algorithm, the further the network is from a lattice, and the closer to the random network. Small worlds exist for intermediate values between these extremes.

Figure 8.

Particular instances of (a) a lattice network, (b) a small-world network, and (c) a random network. In each case, the population consists of 50 individuals, and the average degree for each node is 8.

With a nontrivial network structure, we must now specify the relationship between the source of the false belief and the source of the retraction. They might originate at the same node. This captures a situation in which, for example, a specific individual (lab, journal, institute) publishes a result and then subsequently retracts it. Alternatively, a retraction might originate at a different node if another journal or lab publishes a failure to replicate or if one news outlet invalidates another’s claim.

The qualitative results from our base model hold for small worlds. We discuss simulations for $N=100$. When individuals never stop sharing information, the only stable states are ones in which there are no false beliefs. Delaying the introduction of retracted information again tends to increase the average amount of time individuals hold false beliefs. As before, when we introduce timed novelty to our model, stable false beliefs are possible. And as before, delay can create a surprising benefit in such cases.

Does the location of a retraction influence its success? We looked at simulations in which the retraction was either introduced (1) by the same agent who introduced the false belief or (2) by another random agent in the network. We find that introducing the retraction in a different spot leads to more stable false belief (assuming there is some time frame for sharing). Figure 9 shows this for a particular model. This happens because the retraction, when introduced in the same location, can chase and overtake false beliefs.Footnote ¹² When introduced in another location, it takes longer for the retraction and false beliefs to come into contact, and thus the false belief is harder to eradicate. As we will discuss in the conclusion, this may have important policy implications for journals.

Figure 9.

Location of retraction in small-world networks. False beliefs are less prevalent when the same source issues a retraction. $N=100$; $K=8$; time frame for sharing = 200.

3.3. Preferential-Attachment Networks

As noted, small-world networks accord with some empirically observed properties of social networks. However, unlike small-world networks, many real-world networks have been observed to have scale-free properties such that some small number of individuals are very highly connected while most are not.Footnote ¹³ In particular, scale-free properties are common in citation networks and scientific collaboration networks and on the internet (Barabási and Albert Reference Barabási and Albert1999; Albert and Barabási Reference Albert and Barabási2002; Barabási et al. Reference Barabási, Jeong, Néda, Ravasz, Schubert and Vicsek2002; Steyvers and Tenenbaum Reference Steyvers and Tenenbaum2005).Footnote ¹⁴ This might correspond, for instance, to scientific communities where some individuals are highly connected and others are marginally so.

We now look at networks generated according to the Barabási-Albert preferential-attachment model (Barabási and Albert Reference Barabási and Albert1999). The algorithm begins with a connected network of m individuals. New nodes (up to N) are added one at a time. Each connects to m existing nodes with probability directly proportional to the number of links the nodes already have. Therefore, existing nodes with many links tend to receive more new attachments; for example, new articles are proportionally more likely to cite ‘famous’ (already heavily cited) articles rather than newer articles with fewer citations. Figure 10 shows an example of networks for $m=1$, 3, and 5. We examine networks with $m=1$, 2, 3, 4, and 5.

Figure 10.

Example preferential-attachment networks: (a) m = 1; (b) m = 3; (c) m = 5.

Our results are generally robust in these new networks. Timed novelty can lead to the indefinite persistence of false beliefs, and delayed retraction can sometimes improve the saturation of a retraction. As with small-world networks, for preferential-attachment networks, we can ask: How does the location of retraction influence outcomes? We were particularly interested in cases in which a founding, central node introduced a false result. We then looked at treatments where either (1) the retraction was introduced by this same node or (2) the last node introduced the retraction. This might correspond to situations in which either well-established scientists retract their own findings or a relative newcomer to the community refutes them.

We found that, across parameter values, retraction is more effective when introduced by the original, founding node. It is not entirely surprising. There are two advantages to a central node issuing a retraction. First, as with the small-world networks, there is an advantage of a retraction coming from the same place because it can chase the same paths the false belief took. Second, there is a benefit when the retraction is issued by a highly connected node, with relatively short paths to the rest of the network. Figure 11 shows these results for different values of m. Note that when m is low, there is relatively little false belief because the sparse network structure impairs its spread. In all cases, there is a clear benefit to retraction from the original node.

Figure 11.

Location of retraction in preferential-attachment networks. False beliefs are less prevalent when a central node issues a retraction of its own falsehood, rather than when a relatively less central node does. $N=100$; delay$\mathrm{delay}=100$; time frame for sharing = 200.

3.4. Homophilic Networks

Sometimes populations fracture into political camps or research communities that take different approaches. Such networks are especially relevant to thinking about failures of retraction, since retractions accepted by one subgroup may be ignored by another.

To consider this sort of case, we look at how homophily affects the persistence of retracted information. Network homophily describes the tendency for nodes that are ‘similar’ in some respect (i.e., in-groups) to be more likely to attach to one another than ‘dissimilar’ (i.e., out-group) nodes. Because of this tendency, in homophilic networks, we see highly connected subgroups with relatively fewer connections between. Figure 12 shows an example.

Figure 12.

Example of a network exhibiting homophily. Each subpopulation has $N=50$ agents. The probability that a particular individual has a connection with a member of her in-group is significantly higher than the probability that she has a connection with a member of her out-group. In this case, $p_{\mathrm{in}}=0.25$, and $p_{\mathrm{out}}=0.10$.

There are two reasons that we examine homophilic networks. First, we see subgroups of this sort in many real-world networks, including scientific communities.Footnote ¹⁵ Additionally, social networks surrounding political orientation are often homophilic (Himelboim et al. Reference Himelboim, Sweetser, Tinkham, Cameron, Danelo and West2016). Second, by splitting the population into subgroups, we can examine the effect that distinct sources of information have on the persistence of false information in the population as a whole and within each subpopulation.

We always assume that for N individuals, each subpopulation consists of N/2 individuals. For each individual, n_i, in the network, there is some probability, p _in, that n_i is connected with a given member of her in-group, and there is some other probability, p _out, that n_i is paired with a member of her out-group. In figure 12, for example, $p_{\mathrm{in}}=0.25$ and $p_{\mathrm{out}}=0.10$.

What happens in models with homophily? General results are similar to those reported. Compared to the model with a complete network, retracted beliefs tend to spread less quickly in homophilic networks, meaning that on average neutral and false beliefs persist longer, and in models with timed novelty fewer individuals ever reach retracted beliefs.

In cases with a high level of homophily between the groups, we find that retractions are less successful when introduced in the group that did not generate the original false belief. This is unsurprising since homophily means that it takes longer for false beliefs to reach the other group, making retraction less relevant and more likely to stop spreading.Footnote ¹⁶ Once the retraction does manage to spread, there are fewer links by which it can travel to the group that originated the false belief. Figure 13 shows this.

Figure 13.

Location of retraction in homophilic networks. Homophilic networks may have more persistent false belief when retraction is introduced to a partition different from where it originated: $p_{\mathrm{in}}=0.4$, $p_{\mathrm{out}}=0.004$, $N=100$, and timed novelty stops after 200 rounds.

Additionally, both the location of the original false belief and the location of the retraction can influence the relative levels of false belief in the two subgroups. For simplicity, let us call the groups 1 and 2. We assume the false belief is always introduced in group 1, and the retraction can be introduced in either. In most cases, group 1 tends to hold false beliefs for more extended periods than group 2, since the false belief originated in its area of the network. Although in cases in which both the false belief and the retraction show up in group 1, it will sometimes be the case that the false belief, but not the retraction, manages to spread effectively to group 2. In such cases, more members of group 1 will hold the false belief at some time or another, but they will also learn the retraction faster. When the retraction is introduced in group 2, though, group 1 holds false beliefs for longer in all models. If the individuals with the incorrect belief do not receive a correction in their own subgroup, they are left in a state of false belief as the retraction slowly trickles back to them. Figure 14 shows data supporting these claims. We report average end beliefs for groups 1 and 2, for cases in which the retraction is introduced in group 1 (same) and 2 (other). When the retraction is in the same subgroup, that group ends up with many more retracted beliefs and, for these parameter values, shorter average false beliefs. When the retraction is introduced in group 2, group 1 has dramatically higher levels of false belief and lower levels of retracted beliefs.

Figure 14.

Average length of belief for two subgroups in a homophilic network. Results are for groups 1 and 2, for cases in which the retraction is introduced in the same subgroup (1) and the other subgroup (2); $p_{\mathrm{in}}=0.4$, $p_{\mathrm{out}}=0.004$, $N=100$, $\mathrm{DELAY}=400$, and timed novelty stops after 200 rounds.

4. Conclusion

We find that contagion-type models are a useful tool in exploring the dynamics of retraction. They show why false beliefs can persist indefinitely, even in light of a retraction, when agents stop sharing new beliefs. They illustrate how delays might influence the success of a retraction, including the surprising finding that in some cases delay might improve the saturation of a retraction. They show how network structure can affect the success of retraction and, in particular, how retractions are less successful when they do not come from the original source. In homophilic networks, retractions are more successful when they originate in the same group as the false claim.

Simplified models like the ones we present here must always be treated carefully when applied to real-world cases. In particular, there are mismatches between model and target in our work that may attenuate the relevance of the results. We will now talk about a few of these mismatches before outlining in greater detail what we think these models can do.

One of these gaps is that we do not model prominent or central communicating agents such as a journal, or academic search engine, that spread ideas to large portions of a population at once. One might worry that in science authors will always check with these sorts of central agents before adopting a new belief or citing a source, thus invalidating our models. Empirically, though, we know that this does not always happen. Instead, authors often pull citations directly from citing papers, rather than looking to the original source (Broadus Reference Broadus1983). Moreover, as we made clear earlier in the article, the existence of these central agents does not seem to stop the widespread citation of retracted work. Still, this kind of structure should affect the flow of information in scientific communities, and below we will discuss this further.

Another issue relates to the representation of agents’ cognition. Our agents have only three belief states—neutral, false, or retracted. However, in many cases, results of scientific studies are not easily deemed to be true or false but are controversial. For this reason, as noted above, the models will not apply to a wide range of cases in which beliefs vary in degree. Additionally, the agents in our populations have none of the special reasoning biases humans exhibit. Thus, there are no instances of, for example, confirmation bias, where individuals seek out all and only ‘facts’ that support their beliefs. They do not yield to conformist bias (i.e., imitating the beliefs of neighbors for social reasons). Further work might look at whether the results here hold up under more complex representations of agents.

Our models also assume that agents are not motivated to shape the beliefs of those around them. Whenever agents learn that a piece of information is false, they stop sharing it and start sharing the retraction. However, it seems that scientists often continue to cite and share their own retracted findings. Madlock-Brown and Eichmann (Reference Madlock-Brown and Eichmann2015) show this is a common practice and that scientists who do it boost their citation count. One reason for this may be that scientists often seek credit—attention and good reputation from those in the community (Merton Reference Merton1973).

Scientific journals and research institutions are also incentivized to maintain both readership and reputation, which may make them both reluctant to retract papers and to communicate these retractions to readers (Unger and Couzin Reference Unger and Couzin2006; Wager and Williams Reference Wager and Williams2011; Madlock-Brown and Eichmann Reference Madlock-Brown and Eichmann2015). For instance, there seem to be many cases in which journal retractions are overly vague or imply an error when, in fact, they were the result of discovered fraud (Wager and Williams Reference Wager and Williams2011; Fang et al. Reference Fang, Steen and Casadevall2012).Footnote ¹⁷ Many philosophers of science have used Merton’s framework to model scientists as taking part in a ‘credit economy’ (Kitcher Reference Kitcher1990; Bright Reference Bright2017; Heesen Reference Heesen2018). A next version of our model might include credit motivations as well as epistemic motivations for our agents. In other cases, retracted information is shared by those with political or economic motivations. We might also consider agents who seek to shape the landscape of belief.Footnote ¹⁸

Despite the limitations of these models, there are several appropriate takeaways. First, they are useful for hypothesis generation and directing further empirical research. This is especially true for cases in which empirical work is lacking, such as in the case of uptake of media retractions. To give an example: our results suggest that moderate delays in retraction may sometimes make them more effective. Although this is not an obvious hypothesis to test before this modeling work, it is worth examining given the theoretical support generated here. It is also worth considering how the location of a retraction in real networks influences outcomes. Does the source matter, especially in homophilic groups, as we suggest?

Second, the models here provide tools for thinking about how to improve current systems to make retractions more effective. (Improvements that can be empirically tested.) What solutions do they suggest? We cannot easily alter the network connections between human individuals, laboratories, or research institutes. Additionally, we probably cannot convince agents to keep talking about retractions for a longer period than they usually would. However, we might be able to institute changes for central communicators like those mentioned above—journals and academic search engines. Imagine the addition of a node into our network models that communicates with a large proportion of the population and continues to share information about the retraction actively and indefinitely. Such a node would move the population toward something like the networks we discussed first, where falsehood is always eventually stamped out. If individuals are connected to a node that continues to share the retraction, they should eventually get that information.

This also implies that real organizations should be more active about communicating retractions. For instance, in searching Google scholar, it is easy to yield retracted research papers as results without also seeing the retraction. A better practice would involve tying these search results together. Journals should implement editorial policies that check to see whether cited sources have been retracted and then ask authors to remove these sources when appropriate. This would, in effect, be a policy designed to promote continued and widespread sharing of the retraction.

Our results also suggest that it is important for retractions to be spread by the same sources that originated a false belief. If a refutation is published in another field or subfield, it might not be effective. The journal that originally published the false result should publicize this refutation to its own readers. Likewise, if a news source proves that another is wrong, it is important that the original news source share this information as well. Journalists who, for instance, publicize false claims should try to use the same venues to clarify matters.

In sum, while these models are highly simplified, they are useful in directing further research into retraction. In particular, they help yield hypotheses regarding network structure and community design that may help improve retraction and the elimination of false beliefs.