As Leibovich et al. point out, the dominant view is that mechanisms involved in performing “number sense” or approximate number system (ANS) tasks underlie the basis of symbolic mathematical skill. This view is based on findings from two main experimental paradigms: In comparison tasks, participants select which of two arrays contains more dots; in addition tasks, they assess whether two sequentially displayed arrays contain more dots than a third array. Researchers have assumed that such ANS tasks harness an innate sense of number, but Leibovich et al. argue that this assumption is not warranted.

In our view, there are three main sources of evidence for the view critiqued by Leibovich et al.:

1. 1. Face validity. Tasks in which children or adults compare, for example, the number of yellow and blue dots do, on the face of it, seem to be about number.

2. 2. Correlational evidence. Recent meta-analyses have reported that performance on standardized mathematics tests and ANS tasks correlate at r = 0.2 to 0.3 (Chen & Li Reference Chen and Li2014; Fazio et al. Reference Fazio, Bailey, Thompson and Siegler2014; Schneider et al., Reference Schneider, Beeres, Coban, Merz, Schmidt, Stricker and De Smedt2017).

3. 3. Causal evidence. Some recent experimental studies have claimed that improving performance on ANS tasks causes higher mathematics achievement and faster mathematics performance.

Leibovich et al. argue compellingly that evidence from comparison tasks is insufficient to conclude that ANS tasks involve numerical processing, at least not as currently conceived by proponents of the “number sense” theory (cf. Gebuis et al. Reference Gebuis, Cohen Kadosh and Gevers2016). Further, accounting for the ANS/mathematics achievement correlation does not require the assumption of an innate sense of number, because of the inhibitory control demands of incongruent trials on ANS tasks (e.g., Fuhs & McNeil Reference Fuhs and McNeil2013; Gilmore et al. Reference Gilmore, Attridge, Clayton, Cragg, Johnson, Marlow, Simms and Inglis2013). However, this inhibition confound does not account for the third source of evidence, which Leibovich et al. do not address.

The causal evidence comes from two sources. One line of research has found training on ANS tasks leads to improved performance and faster responses on mathematics tests (Hyde et al. Reference Hyde, Khanum and Spelke2014; Khanum et al. Reference Khanum, Hanif, Spelke, Berteletti and Hyde2016; Park & Brannon, Reference Park and Brannon2013; Reference Park and Brannon2014; Park et al. Reference Park, Bermudez, Roberts and Brannon2016a). Another has found that manipulating the order in which ANS trials are presented (easy-to-difficult or difficult-to-easy) improves mathematics performance (Wang et al. Reference Wang, Odic, Halberda and Feigenson2016). These findings present a problem for the inhibition account. Earlier research found that inhibition training does not transfer to non-trained tasks (Thorell et al. Reference Thorell, Lindqvist, Bergman Nutley, Bohlin and Klingberg2009), so potential inhibitory control demands of ANS tasks cannot explain these findings.

Although there has been a debate about the quality of this evidence (e.g., Lindskog & Winman Reference Lindskog and Winman2016; Merkley et al. Reference Merkley, Matejko and Ansari2017; Park & Brannon Reference Park and Brannon2016; Wang et al. Reference Wang, Odic, Halberda and Feigenson2017), here we ask whether, taken together at face value, current experimental studies provide sufficient evidence to conclude that there is a causal link between the ANS and mathematics performance. To this end, we performed a p-curve analysis on the set of all studies we are aware of that report a causal link between the ANS and mathematics performance (Hyde et al. Reference Hyde, Khanum and Spelke2014; Khanum et al. Reference Khanum, Hanif, Spelke, Berteletti and Hyde2016; Park & Brannon Reference Park and Brannon2013; Reference Park and Brannon2014; Park et al. Reference Park, Bermudez, Roberts and Brannon2016a; Wang et al. Reference Wang, Odic, Halberda and Feigenson2016).

P-Curve analyses (Simonsohn et al. Reference Simonsohn, Nelson and Simmons2014; Reference Simonsohn, Simmons and Nelson2015) rely on the fact that p-values follow a uniform distribution under the null hypothesis. In contrast, when the null is false, p-values are right skewed (i.e., there are more low values than high values). This is true not only for the full 0-to-1 interval, but also for the interval from 0 to 0.05. Simonsohn et al. proposed that the shape of the distribution of significant p-values in a set of studies can be used to assess if they collectively contain evidential value. If the significant p-values follow a roughly uniform distribution, publication bias might explain the results.

We followed Simonsohn et al.'s (2015) procedure and, for each reported study, extracted the test statistic associated with the hypothesis of interest (whether the experimental manipulation influenced mathematics performance). If there was doubt about which statistic to select (e.g., the study contained two control groups), we conservatively selected the comparison with the smaller p-value (retaining the other for a robustness check). Details are given in our p-curve disclosure table at https://figshare.com/s/e6188771f048ecc6f881.

We analyzed the test statistics using the p-curve app v4.05 (http://www.p-curve.com/app4/). The p-value distribution is shown in Figure 1. Of nine p-values, five were below 0.025, a frequency not significantly different to the 4.5 expected under the null hypothesis, one-tailed binomial test, p = 0.5. Stouffer's method (Simonsohn et al. Reference Simonsohn, Simmons and Nelson2015) also indicated that these studies do not contain evidential value (p-values = 0.206, 0.299). The p-curve method also provides an estimate of the power of the studies. Here this was 13% (90% confidence interval: 5%, 54%), indicating insufficient evidence to reject the null of 33% power (which Simonsohn et al. would take to indicate that evidential value was absent and that replications would not be expected to succeed).

Figure 1.

Distribution of p-values for studies finding a causal connection between the ANS and mathematics performance. The observed p-curve includes nine statistically significant (p < .05) results, five of which are p < .025, No nonsignificant values were entered.

Our findings indicate that the published literature to date does not contain evidence of a causal link between performance on ANS tasks and standardized mathematics tests. To be clear, we have not demonstrated there is no causal connection between the ANS and mathematics performance, only that the existing literature does not provide evidence for one. However, we can definitively conclude that existing studies are substantially underpowered, rendering their interpretation difficult. In the future, researchers should address this limitation through preregistration and larger samples.

To conclude, we endorse Leibovich et al.'s suggestion that the assumption that ANS tasks involve number sense is not justified. Although Leibovich et al. did not address it, we believe that existing evidence of a causal link between the ANS and mathematics performance is insufficient to challenge their argument.