FINDINGS

Complete data were available for seventy projects. The median study duration was 16 months with a median level of funding per project of £44,361. The median length of follow-up after project completion was 35 months, by which time 40 percent had published in a peer-reviewed journal and 70 percent had published in some form. A total of 12 percent of research projects were responsive (researcher-led or training fellowships), and 78 percent were commissioned according to national or locally set R&D priorities. Ninety-three percent of projects were classified as research and the remaining 7 percent as development projects. A total of 47.1 percent included some quantitative analysis, whereas the remaining 52.9 percent did not.

Analyses of the two-by-two tables undertaken to explore possible associations between a range of independent variables and the proportion that resulted in publication are summarized in Table 2. These tests did not find evidence of an association between publication and whether a study (i) showed an effect, (ii) used statistical tests for significance, or (iii) contained a quantitative element. Two interesting associations between characteristics of studies were found. First, there was a positive association between whether a study showed an effect, and whether it used statistical tests of significance (χ² (1) 6.275, p=.01), indicating that studies which used statistical tests of significance are more likely to have reported an effect in the study (bottom of Table 2). Second, we also found a strong linear correlation between the length of the project (in months) and the amount of funding the project received (r=.6; p=.01). Thus, when developing the multivariate model, we excluded “duration of project” and “use of statistical tests,” including instead “size of funding” and “showing an effect,” respectively.

The variables were explored one by one (univariate Cox regression). Several variables showed a statistically significant association with publication in peer-reviewed journals (Table 3). Backward stepwise Cox regression was used to explore which of the variables were the most important influencers of publication pattern. All the factors explored so far were entered, and the best model selected by progressively removing the variable showing the least evidence of an independent association (Table 3).

Beginning with the eight funding programs, variation was noted between the curves, indicating a difference in publication patterns and suggesting that there is a relationship between funding program and publication outcome.

The output showed that, overall, funding program was marginally significantly related (p=.057) to eventual publication in a peer-reviewed journal; however, there appeared to be little difference between individual funding programs. Therefore, the five Commissioned programs were grouped together so that four categories of program remained: Responsive; All Commissioned programs, including Research Implementation; Commissioned—Organization & Management (O&M); Education and Training (ET)/Fellowships (PROG2).

The survival curve indicated a natural break between responsive and ET/Fellowships; and Commissioned programs including O&M. The final best Cox regression model is shown at Table 4, and in Figures 1 and 2. It indicates that the size of the funding (£TWOCAT, which differentiates projects that received up to £22,713 from those that received over this amount) and type of research program under which the project was funded (PROG4, which differentiates Responsive and ET/Fellowship projects from those in Commissioned programs) are both associated with publication rate. Other variables were not shown to be associated with the rate of publication, but a larger sample is needed to explore this.

“1 minus survival function” for patterns 1–2. PROG4: 1, Responsive and Education and Training/Fellowships; 2, Commissioned programs and Organization & Management

“1 minus survival function” for patterns 1–2. £TWOCAT: 1, Up to £22,713 (minimum £1000); 2, Over £22,713 (maximum $141,124)

COMMENT

Statistically significant independent relationships were found between the rate of publication in peer-reviewed journals and both the type of program (Responsive/Commissioned) and the cost of the research. Whether studies showed an effect or the direction of this effect was not statistically significantly associated with rate of publication, although this could be due to type II error.

This is one of the first studies exploring publication of outputs from the NHS R&D program. Two factors clearly affected the rate of publication over a median follow-up period of 35 months after study completion. Researcher-initiated projects were more likely to be published. This finding may be important, given the gradual replacement of responsive funding streams in the NHS in favor of research commissioned in areas of nationally determined priorities. In addition, better-funded projects had a higher rate of publication; however, it is not clear whether this finding reflects a tendency for larger projects to result in peer-reviewed publications or whether researchers capable of attracting greater funds are more successful at publishing.

Importantly, there was no evidence of publication bias, although the numbers were small. This study should be repeated in other parts of the national and regional NHS R&D program and pooled.