In one form or another, peer review has always been regarded as crucial to the reputation and reliability of scientific research.
UK House of Commons committee on Science and Technology, 2011Footnote 1Public discourse on scientific and medical research places significant emphasis on the process known as ‘peer review’: it is seen as crucial to building the reputation both of individual scientists and of the scientific enterprise at large, and it is believed to certify the quality and reliability of research findings. It promises supposedly impartial evaluation of research, through close scrutiny by subject-specialists, and is widely used by journal editors, grant-making bodies, and government. In recent decades, the effectiveness of peer review in both of these roles has been attacked, by those – particularly in the bio-medical sciences – who point to failures to detect error and fraud, and by those who identify inappropriate bias due to the social dynamics of the process.Footnote 2 The term ‘peer review’ was itself coined only in the early 1970s, but it ‘has been elevated to a “principle” – a unifying principle’ for widely diverse spheres of research.Footnote 3 In all fields of academia, reputations and careers are now expected to be built on peer-reviewed publication; concerns with its efficacy and appropriateness thus seem to strike at the heart of scholarship.
In both public and scholarly discourse, peer review is routinely taken to be as old as the scientific enterprise, and its origins usually located at the Royal Society of London, with the creation of the Philosophical Transactions in 1665.Footnote 4 One of the most influential early studies of research evaluation was that published by Harriet Zuckerman and Robert Merton in 1971. They recognized that ‘the referee system did not appear all at once’ but ‘evolved’; however, their discussion of the early Royal Society (reflecting Merton's earlier work on science in seventeenth-century England) was followed by a leap to the twentieth century, thus resulting in their paper being widely cited to support the invention of peer review in 1665.Footnote 5 We argue that this ahistorical treatment of peer review misunderstands both the nature of early modern editorial practice, and the significant ways in which editorial practice evolved in the three centuries after 1665, before ‘refereeing’ was rebranded ‘peer review’.
Those historians who have more closely examined editorial practices locate the origins of refereeing in learned societies in the first half of the eighteenth century.Footnote 6 They suggest that refereeing then came to be used at (a few) independent scientific journals in the late nineteenth century, with widespread adoption occurring only in the later twentieth.Footnote 7 Historians of science have recently begun to investigate surviving referees’ reports, but so far, they have sought to uncover the hidden dynamics of intellectual communities at particular times and places, rather than to investigate long-term development.Footnote 8
In this article, we consider how and why learned societies should have felt it necessary to develop distinctive forms of editorial practice, including the use of referees and committees. By historicizing the development of peer review, we show that the processes of evaluation prior to publication in scientific periodicals have been startlingly various, and only gradually accrued the functions now routinely attributed to peer review. We hope thereby to demonstrate that peer review was not (and was historically not intended to be) a unitary phenomenon, good for all places and times. Scholarship on contemporary peer review already acknowledges how practice varies between disciplines and journals.Footnote 9 Our work extends this by pointing out the antecedents of that variety of practice; but we also seek to show that it emerges from a wide historical variety of purpose.
We investigate these issues through examination of the rich archives relating to the Philosophical Transactions. We do not claim that the Royal Society was the sole origin of modern peer review. But as the organization with responsibility for the world's longest-running scholarly journal – and, importantly, its archive – a study of the Royal Society offers a unique insight into the evolution of learned society editorial practices between the establishment of the earliest scientific periodicals and the late twentieth century.
We open with an analysis of the evidence for something like peer review at the early Royal Society. We structure the rest of the article around three episodes when changes of editorial practice at the Royal Society were formalized in response to criticism of current practice: the move away from sole editorship (formalized in 1752); the use of expert referees (formalized in 1832); and changes to the broader gatekeeping processes (formalized in 1896). This last change left the Society with a system that was accused of being anachronistic and out-of-step with modern science; and yet, by the 1960s and 1970s, elements of that system – specifically, refereeing – had been widely adopted by scientific journals, and transformed into ‘the imprimatur of scientific authenticity’.Footnote 10
Our analysis reveals that refereeing was one element within a wider set of practices which shaped the selection and evaluation of papers for publication; and we argue that the distinctive editorial practices of learned societies arose from the desire to create forms of collective editorial responsibility for publications which appeared under institutional auspices. We show how the Royal Society transformed the Philosophical Transactions from a periodical in the charge of a single editor into one run by a committee. We then show how that committee came to ‘refer’ papers to particular individuals for closer scrutiny, and how a practice that was informal (and oral) in the late eighteenth century turned into something routine, documented and written, justified by a need for expertise, in the nineteenth century.
We argue that refereeing and the associated editorial practices of the Royal Society were intended, initially, to disarm specific attacks upon the eighteenth-century Society; sometimes, to protect the Society's finances; and, by the later nineteenth century, to award prestige to members of the nascent profession of natural scientists. The growing professionalization and internationalization of scientific research in the early twentieth century changed the dynamics and function of editorial processes that had developed in the context of a gentlemanly learned society. Yet, contrary to some modern claims for peer review, the committees and referees of the Royal Society, throughout our period, were only intermittently concerned with anything that might be termed the ‘reliability of scientific research’.
I
The first durable scientific societies emerged in the later seventeenth century: the Academia Naturae Curiosorum in Schweinfurt (1652), the Royal Society in London (1660), and the Académie Royale des Sciences in Paris (1666). These new and privileged spaces afforded (to varying degrees) official recognition and reward for inquiry into natural phenomena and processes, and new opportunities for collective discussion, comment, and critique. The Royal Society's motto, ‘nullius in verba’, usually rendered as ‘take no man's word for it’, implied a promise that its fellows would turn their critical gaze as ruthlessly upon each other as upon the rest of the learned world.Footnote 11 The chief manifestation of the Royal Society's collective basis was its weekly London meeting. In its early years, meetings involved both the devising and witnessing of experiments, and the critical discussion of experiments and observations reported by members and the natural philosophical community at large.
The Society's role as a proving ground for early modern claims to natural knowledge is not in dispute, nor its significance as a space for free and open discussion. But it has been too rarely appreciated how distinct the practices of early Royal Society meetings were from the editorial practices of the Philosophical Transactions. The Transactions was run by the Society's secretary, Henry Oldenburg, as a private venture. Letters from Oldenburg to Boyle in 1664 have been cited as proof that Oldenburg was already envisioning the so-called ‘four key functions’ of the modern academic journal, yet those letters actually concerned the role of the Society and the function of its manuscript records, not his as-yet-unlaunched periodical.Footnote 12
Close examination of Oldenburg's practices as editor of the Transactions, from 1665 until his death in 1677, reveals how different his role was from that of the modern scholarly journal editor. He did not receive submissions from authors and choose among them on the basis of intellectual merit, let alone engage in systematic consultation about those merits. Rather, Oldenburg worked hard to secure copy, drawing on his very wide correspondence with the learned men of Europe and his exceptional command of languages.Footnote 13 This is not to say that Oldenburg was unconcerned with the quality of what was published; only that he articulated no clear set of standards, and only occasionally referred to any judgement other than his own. Most of what appeared in Oldenburg's periodical does not conform to a standard.Footnote 14 Nor can it be said to represent knowledge sanctioned by the Royal Society, since the Transactions was published under Oldenburg's independent control, and he was careful to distinguish between its contents and the Society's activity – a point missed by many of his contemporaries and some modern historians. Despite the Society's palpable approval of Oldenburg's project, in that early period, the research sponsored by the Society was published, not in the Transactions, but in separate books and treatises.Footnote 15
Some historians have pointed to the Society's right – under its founding charter – to license books for publication on its own authority as evidence of collective scrutiny and sanction.Footnote 16 The Society's licensing practice involved the perusal of a work prior to printing by at least two members of the Council and the approval of the Council as a whole, and was part of a wider mechanism of state censorship intended to ensure the proscription of politically seditious or religiously heterodox material.Footnote 17 Mario Biagioli has suggested that the responsibility of licensing in both the English and French contexts simultaneously made the new scientific societies instruments of government, and thus made them communities of ‘peers’ in a legal sense. The associated burden, of policing works for seditious or heterodox material they were unlikely to contain in the first place, was largely notional, but created space, according to Biagioli, for an institution to turn its imprimatur into a means of defining what kind of science it approved of.Footnote 18
Between 1665 and 1708, the Royal Society licensed the publication of all issues of the Transactions, and about fifty books.Footnote 19 Pre-publication scrutiny was usually casual, and in the case of the Transactions, there are rarely traces of any at all. Furthermore, any simple conflation of early modern book censorship with the endorsement of intellectual claims is undermined by the Royal Society's own uncertainty about the extent and implications of its privilege and whether it was truly a licensing privilege: it sought legal advice before using it for the first time in 1663.Footnote 20 On that occasion, the newly chartered Society was eager to associate itself with John Evelyn's Sylva (1664), a practical treatise responding to a crown commission on the best way to secure the kingdom's supply of shipbuilding timber. But within a year, this precedent had become problematic: when the Society tried and failed to persuade Robert Hooke to omit some of the more speculative flights in Micrographia (1665), the Council insisted he include a disclaimer absolving the Society of responsibility for them. At the early Royal Society, licensing represented less an endorsement of particular research claims, and more a judgement of how far association with a given work would redound to the Society's credit.Footnote 21 Similarly, and despite Biagioli's plausible argument that licensing at the Royal Society was better at excluding than at selecting for specific, positive, criteria, there is only one recorded instance of any work being denied the imprimatur, and no unambiguous evidence of any intended contribution to the Transactions being rejected on the Council's say-so.Footnote 22 The Royal Society's scrutiny for licensing purposes was, according to the best available evidence, neither rigorous nor systematic nor (strictly speaking) collective, since works were often licensed on the word of the presiding officer, apparently without debate.Footnote 23 It is, therefore, difficult to argue that the editorial and licensing mechanisms of the seventeenth-century Philosophical Transactions can legitimately be seen either as a positively articulated protocol for choosing among particular knowledge-claims, or as a seal of collective approval establishing standards for natural-philosophical print.
II
Following Oldenburg's death in 1677, the Transactions was edited for seventy-five years by the secretaries to the Society.Footnote 24 Few observers recognized that the editors were acting in a private capacity, not least because the content of the Transactions became increasingly identified with the activity of Society meetings.Footnote 25 This left the Society vulnerable to the imputation of failing to enforce adequate standards in the Transactions, yet with no obvious means of exercising control, and little hope of being believed when it tried to deny responsibility.Footnote 26 This difficulty lay at the root of crucial statutory changes to the periodical's management in 1752. The coincidence of a new series of attacks on the Society and the Transactions with a time of difficult personal circumstances in the Society's leadership resulted in a new model of collective editorship (although it tacitly incorporated a good deal of existing practice).
In the early 1750s, a failed candidate for the fellowship, the botanist, actor, and apothecary John Hill, launched a series of public attacks upon the Society, criticizing the conduct of its meetings; the intelligence and character of its members in general (and of the president, Martin Folkes, in particular); and, most damagingly, the Philosophical Transactions. Hill took advantage of the perceived association between the Society and the periodical to dredge up, and mock, weak papers dating all the way back to 1665. Hill's critique was satirical as much as it was philosophical and he had particular fun excoriating the self-evidently absurd or trivial. He solemnly proposed, for instance, a string of escalatingly ludicrous improvements to a 1703 paper on a Ceylonese technique of hunting waterfowl that involved the hunter wading into the water up to his neck with a clay pot over his head, and pulling the birds under by the feet.Footnote 27 In other cases, Hill objected to the space and precedence granted to minor natural-historical observations by people he despised as cronies of the president, and in still others he raised more substantive criticisms. In each instance, however, the basic force of the critique came from his ability to exploit the assumption that everything published in the Transactions had in some way passed the Society's scrutiny, and that the Society was therefore intellectually responsible for the contents. Hill cemented the perceived link by calling his critique A review of the works of the Royal Society (1751).
Shortly after Hill's attack, Cromwell Mortimer, the editor (and secretary of the Society), died suddenly. Combined with the long-term incapacity of the president, this afforded the Council an opportunity to reform the existing system without making scapegoats of its official leadership. Their response was to enact precisely the kind of collective editorial responsibility that Hill had insinuated. In January 1752, the Royal Society assumed both financial and editorial management of the Transactions. From this point until 1990, the Transactions officially had no editor. The members of the Council acted as a Committee of Papers, charged with deciding collectively which of the papers communicated to the Society should be published.
This was not a necessary or obvious step. Individual decision-making by editors, or even groups of editors, was a widespread and successful model for editing a periodical, and by assuming financial responsibility, the Society's Council acquired a means to control any editor it might have appointed.Footnote 28 By involving more people in the editorial process, the Society was protecting itself from the incompetence or idleness of individual editors; and by making decisions through committee voting, it protected the president and secretaries from ad hominem attacks on their judgement, and deflected many of Hill's criticisms without ever publicly acknowledging them.
The new statutes of 1752 laid down that the committee should consider all papers communicated to the Society, in the order in which they had been read at meetings.Footnote 29 Committee members met roughly every six weeks, were furnished with abstracts of the papers on which to base their judgements, and were supposed to reach their decision by secret ballot without discussion.Footnote 30 This contrasts with the practices of both the Paris Académie Royale, where rapporteurs produced jointly-authored reports on submissions by outsiders, and of the Royal Society of Edinburgh (f. 1783), whose statutes would explicitly allow committee members to discuss the merits of the papers.Footnote 31 The London system did not seek consensus, but created collective judgement from a group of equally weighted individual judgements. The ‘no discussion’ rule was avowedly intended to prevent the committee decision from being unduly swayed by any particular individual. How this worked in practice remains obscure, but the written procedures for decision-making after 1752 gave the appearance of probity, and produced judgements that were hard to contest.
The practice of the Committee of Papers seems on balance to have been more concerned to weed out unsuitable papers than proactively to select the best for publication. In the decades around 1800, around 65 per cent of papers read to the Society were later published in some form in the Transactions.Footnote 32 This inclusive practice was governed by several factors: first, the question of whom the periodical should most benefit; second, the relationship between the Society's meetings and papers; and third, the Society's established reluctance to adjudicate claims to knowledge.
First, the post-1752 Transactions was officially to be run ‘for the sole use and benefit of this Society’, a statement with a range of possible meanings covering reputational or financial benefit to the institution as a whole, or utility to the fellows. It is clear that the Society did not benefit financially from the takeover.Footnote 33 We have already seen how the Society aimed to protect its reputation and to shield individuals from external criticism, by imbuing its publishing decisions with collective authority. For individual fellows, many of whom could not or did not attend meetings in London, the key benefit of the Transactions lay in better access to the matters communicated at meetings. For such an audience, the value of the periodical lay in being broadly representative rather than in showcasing the very best papers.
Second, although the procedures of the Committee of Papers provided a semi-public justification for the Society's publication decisions, they masked the fact that the main filtering of papers had occurred silently and much earlier.Footnote 34 Papers would only be presented at a meeting of the Society if ‘communicated’ (in effect, vouched for) by a fellow. This early gate-keeping enabled the weeding out of obvious nonsense, such as proposals for perpetual motion machines and squaring the circle. But its existence introduced a degree of social delicacy to the subsequent selection of papers for publication: to refuse a paper was to imply a criticism of the judgement of the communicating fellow. More broadly, if the committee routinely declined to publish many papers, it ran the risk of implying that the meetings were filled with material too dull or too weak to appear in print.
The Society was not obliged to grant time at a meeting to every paper submitted to it, and decisions were in the gift of the president and officers. The protocols for deciding what would feature at meetings remained, to outsiders, dauntingly opaque; much depended on the interests and prejudices of the individuals concerned, especially during the presidency of Joseph Banks (1778–1820). Banks sometimes informally sought a second opinion on the intellectual merits of a paper, but was under no obligation to follow the advice he received. Surviving correspondence and diaries from the late eighteenth century demonstrate that such unofficial consultations were common, both before and after a paper was formally read to a meeting.Footnote 35
The third significant factor governing the broadly inclusive tendency of the Society's editorial practice was its habitual reluctance to appear to be endorsing the truth of what was contained in the Transactions. Thus, while reputational control demanded that trivial papers not be published, anything else of interest might. In an ‘advertisement’ printed at the front of every part of Transactions from 1752 until 1957, the Society explicitly distanced itself from the types of judgements contained in the official reports on patents and discoveries produced by the Paris Académie. It insisted that ‘it is an established rule of the Society, to which they will always adhere, never to give their opinion, as a body, upon any subject, either of Nature or Art, that comes before them’. Appearing in the Transactions should signify only the committee's collective recognition of ‘the importance and singularity of the subjects, or the advantageous manner of treating them’, and should in no way be taken to imply that the Society answered ‘for the certainty of the facts, or propriety of the reasonings…, which must still rest on the credit or judgment of their respective authors’.Footnote 36 By denying that it made public epistemic judgements, the Society avoided tying its reputation to any particular knowledge-claim, but also sought to prevent unscrupulous authors and projectors from using the Society's name for their own advantage.
This helps explain why a committee-based editorial system, which could have been used (as in Paris) as a way of expressing the collective, corporate opinion of the fellowship as a whole, actually sought to prevent its judgements from being read that way. The Transactions was not supposed to be a repository of officially sanctioned knowledge, but of interesting or intriguing phenomena that were worthy of further consideration. This remained the official understanding of the meaning of the editorial process until the mid-twentieth century, although, in practice, it would shift significantly with the introduction of referees, and of a second Society periodical.
III
In 1830, the Royal Society came under published attack from two of its own fellows: mathematician Charles Babbage and physician Augustus Bozzi Granville. Both men argued for reforms of the Royal Society, though they would take opposing sides in that year's contested election of a new president; Granville supported the candidacy of the duke of Sussex, the younger brother of George IV and William IV, who ultimately defeated the astronomer John Herschel. Despite their differences, both Granville and Babbage highlighted the role of the Society's publications, and the significance of the editorial decisions behind them. It was in this context that refereeing became a standard element of the Society's editorial practice.
Babbage's Reflections on the decline of science in England is well known for its pessimistic view of the state of science in Britain, compared to France and Germany, and for its place in the debates leading to the formation of the British Association for the Advancement of Science in 1831.Footnote 37 Babbage had served on a Royal Society committee in 1827 that had proposed various internal reforms, and he published the committee's report in Reflections. The reformers sought initially to turn the Society into a smaller, more elite organization, made up of members with active research interests, somewhat like a voluntary version of the Paris Académie. Publication in the Transactions could thus be seen as an indication of the author's suitability for membership of such an organization. This approach potentially changed the meaning of publication decisions, which would no longer merely imply that a published paper would be of some interest to readers, but would be a positive recommendation of its author as a man of science. This motivated a critique of the editorial practices of the Committee of Papers. The 1827 reformers emphasized the need for the committee to have ‘sufficient time…to examine [papers] carefully’ and to communicate directly with the authors when necessary, implicitly critiquing the habit of relying merely on abstracts of papers, and voting with no discussion or opportunity for revision.Footnote 38
Writing anonymously in Science without a head, Granville disagreed with Babbage about the state of British science in general, but agreed that the leadership provided by the Royal Society was lamentably bad. Granville substantiated his concerns by examining the Society's archive in forensic detail, and he too saw problems with the way decisions were reached by the Committee of Papers. He argued that the increasing specialization of scientific research meant that the committee, limited by statute to twenty-one members and whose meetings were seldom fully attended, was not qualified to decide the fate of the wide variety of papers received by the Society.Footnote 39
By November 1832, the duke of Sussex was able, in his anniversary address, to announce an apparent change of practice. He reported that henceforward a paper would be approved for Transactions only if ‘a written report of its fitness shall have been previously made by one or more members of the Council, to whom it shall have been especially referred for examination’, adding that the new system had already been in operation for almost a year.Footnote 40 This insistence on the close examination of the full paper by someone who (presumably) had relevant expertise could be seen as a direct response to Babbage's and Granville's concerns, and the written reports – which survive in the Society's archive in a continuous run from 1832 – certainly turned refereeing into a very visible element of the Society's editorial practice. The Royal Society nowadays proudly cites 1832 as the invention of refereeing.
However, refereeing was not actually new in 1832. The 1752 statutes enabled the Committee of Papers to summon any other fellow, who was ‘knowing and well-skilled in the particular branch of Science’, to deliver an opinion of a paper on whose merits the committee felt unqualified to decide.Footnote 41 There are only a few records of such referrals in the surviving minute books: just five between 1780 and 1815, and not many more thereafter. Yet Granville, who used his fellow's privilege to examine these same records in 1830, nonetheless asserted that ‘every communication is supposed to have been previously…referred to the judgment of some competent member who reports his opinion’.Footnote 42 Granville's confident assertion suggests that there was assumed to be, and may perhaps have been, far more use of oral reporting at the Committee of Papers prior to 1832 than either the statutes required, or the minute books recorded.
According to the duke of Sussex, the Royal Society's 1832 move to (mostly) written reports was in emulation of ‘many Foreign Societies’, but particularly the Paris Académie, which required ‘written Reports…from a Committee of their Members’. He claimed the key virtues of the French reports were, first, that they expressed the judgement of ‘veterans… who have earned by their labours an European reputation’, and second, that they were made public. Those sitting in judgement on submissions had ‘an authority sufficient to establish at once the full importance of a discovery, to fix its relation to the existing mass of knowledge, and to define its probable effect upon the future progress of science’, and their public reports were ‘often more valuable than the original communications upon which they are founded’.Footnote 43 Sussex's justification of refereeing was predicated on a claim to precisely the kind of authority that the Académie Royale had always assumed as part of its function as the head of French science and from which the Royal Society always demurred. Nonetheless, by (initially) seeking reports only from members of Council, the Society imitated this top-down model of evaluation.
The Royal Society further imitated the French by making some of the written reports (those ‘of a favourable nature’) public at Society meetings and in print. As in France, some of these early reports were collaborative, with referees expected to reach consensus and issue a joint report. Sussex acknowledged that this would call for ‘the occasional sacrifice both of time and labour’ by referees;Footnote 44 and as Alex Csiszar has shown, collaborative refereeing quickly proved problematic, especially when referees disagreed about both the paper's precise merits and the purpose of their report.Footnote 45 Moreover, codes of politeness meant that reports were only ever published when the referees felt able to offer ringing endorsements. It was potentially an excellent way of adding value to outstanding papers, but a significant waste of ‘time and labour’ if the paper were bad or merely mediocre. Within a year, the Society abandoned both the requirement of a joint verdict and the publishing of reports. Written refereeing continued, but the referees henceforth reported independently and their reports (and names) were treated as confidential.Footnote 46
One way to lessen the new burden of refereeing was to spread it more widely among the fellowship. From 1833, various ad hoc subject committees were established to adjudicate the award of the Society's Royal Medals, and these committees rapidly assumed an editorial function. From 1838, they were formally established as permanent Scientific Committees and charged with delivering recommendations to the Committee of Papers about what to publish and what not. For the next decade, these committees sometimes came to a collective decision amongst themselves, and sometimes referred papers to one or two individual members. The committee members thus became a pool of subject-specialist referees, involving a wider circle of fellows in decision-making, and potentially deflecting criticism aimed at a Council clique.
It is clear that, during the 1830s and 1840s, the way refereeing fitted into editorial practices had not yet standardized. The number of referees varied, reports were not necessarily delivered in writing, and they varied from single sentences to twenty closely written pages. Referees were unsure whether they were to offer criticism and suggestions, or just a recommendation.Footnote 47 Recommendations were not necessarily dogmatic: in June 1833, one referee sent a letter full of criticisms of David Brewster's paper on the structure of the eye, but was happy to leave it to his fellow referee to ‘draw up such a report as you think necessary for the occasion, and on your better judgement I shall most willingly rely’. (The paper was published.)Footnote 48 In some cases we have only one surviving report for a paper, in others two; in some cases the two referees agreed on a joint decision, and in others they submitted their reports separately. It was up to the Scientific Committees or the Committee of Papers to make sense of the form in which the reports happened to be received.
In early 1831, the Royal Society had also created a new periodical, and this changed the perceived role of the Transactions and the refereeing process associated with it. The Proceedings was issued monthly during the Society's session, in contrast to the twice-yearly parts of Transactions. It reported on each meeting of the Society, including lists of gifts received, elections of new fellows, and annual reports, as well as summaries of the papers read.Footnote 49 By 1833, the initial Proceedings print run of 750 copies (enough for the fellowship, plus a hundred more) had been doubled.Footnote 50 Proceedings thus assumed the function of representing the Society's meetings to the fellowship and to the wider public.
The post-1831 Transactions became correspondingly more selective: by the 1850s, Transactions published only around 30 per cent of papers submitted to the Society.Footnote 51 The more systematic use of referees, introduced shortly after the launch of Proceedings, was specifically for the Transactions. Only around half of the papers communicated to the Society were sent to referees for possible consideration for the Transactions, indicating that some pre-selection was being done by the Committee of Papers. Reports advising publication in the Transactions frequently commended scope, originality, and significance, much the same evaluation criteria as those advocated by the duke of Sussex in his 1832 address.
The greater attention paid to publication decisions for Transactions – as evidenced by the use of refereeing – suggests that they carried greater consequences for the Society. With the 1752 advertisement still in place, there was no endorsement of the knowledge-claims put forward in either periodical. But a Transactions paper represented a financial commitment from the Society (because these papers were lengthy and well illustrated), and a mark of prestige for both the Society (because of the glory potentially reflected on the Society for having published important research) and the author (from 1840, authorship of a paper in Transactions, but not Proceedings, was seen as sufficient evidence of scientific merit to justify a discount on the life membership fee for fellows).Footnote 52 Given that the pool of papers deemed worthy of reading at a meeting could now be seen in Proceedings, the publication decisions for Transactions could potentially be scrutinized as never before. The refereeing process could be seen (internally) as protecting the Society's reputation and finances and (externally) as a mechanism for generating expert evaluation of research.
Following the variety of the 1830s and 1840s, the Society's refereeing practices stabilized. After the Scientific Committees were disbanded in 1849, amidst a scandal over the award of a Royal Medal, referees were drawn from the entire fellowship. Papers for Transactions were usually sent to two referees, one after the other, to save the labour of recopying what might be a very substantial manuscript. Acting as referee permitted fellows to respond to papers at more considered length than was possible at a meeting, as well as enabling distant fellows to engage with the research presented at the London meetings. For instance, William Thomson in Glasgow was one of the most active referees in the 1860s and 1870s. (His colleague W. J. M. Rankine was also active, as was Henry Roscoe in Manchester, and many fellows based in London, Oxford, and Cambridge.) Although papers as published in Transactions were supposed to be substantively the same as when read to the Society, referees often recommended stylistic changes: flabby introductions and overly speculative conclusions were vigorously targeted for cutting.Footnote 53 This improving-and-mentoring function for refereeing was cultivated by long-serving secretary George Gabriel Stokes (1854–85). Stokes mediated between author (or communicator) and referees, passing on the official decision and usually sharing some of the referees’ remarks.Footnote 54 By 1894, a guidance letter codified the dual role now expected of referees, advising that ‘the guidance [for] the Committee of Papers’ be kept ‘separate from any detailed criticisms, or suggestions intended to be communicated to the author’.Footnote 55
Referees’ identities and reports were once again kept confidential, just as Joseph Banks had always done with the informal advice he received.Footnote 56 Thus, the 1894 guidance allowed referees to request that their comments be transcribed before forwarding to the author.Footnote 57 The secrecy of this process occasionally led to complaints, and in 1871, one rejected author had railed against ‘accursed…Secret Committees, secret members, [and] secret judgements’. Yet he admitted that he had been told the gist of the referees’ complaints, and although the secretary refused to reveal the referees’ names, the fact that referees were de facto fellows of the Society meant that their credentials were to some extent known.Footnote 58 Authors, on the other hand, were not permitted to be anonymous because the Society wished to be able to evaluate the credentials (social and intellectual) of its contributors.
By the late nineteenth century, the Royal Society had a well-established set of editorial practices, with referees consulted specifically for expensive and high-prestige publication. The fact that refereeing was not deemed necessary for selecting papers to be read at meetings, or for short-form publication in the Proceedings, suggests that the long-standing, tacit, and social processes for winnowing papers ahead of meetings – which relied on the judgement of the fellows acting as ‘communicators’ and of the secretaries – were still felt to be working adequately well. By the 1890s, however, these gate-keeping practices were under pressure from finances and from the shifting demographic of what had become the scientific profession.
IV
In his anniversary address in November 1896, Joseph Lister, then president of the Royal Society, introduced a major overhaul of the Society's procedures. The changes were intended to ‘increase the interest of the meetings’ and to achieve a ‘greater rapidity in the publication’. The first aim would be achieved by reading only a limited subset of the papers received, thus freeing up time at meetings for commentary and discussion. Second, new ‘Sectional Committees’ were to be ‘entrusted’ with ‘reviewing the communications’ received by the Society. By delegating the initial editorial evaluation to men versed in the various sections of knowledge, Lister hoped the committees would produce ‘a more secure, and, at the same time, more rapid judgment as to the value of communications’.Footnote 59 These restored Scientific Committees and their chairmen became the de facto guardians of the editorial process, though the secretaries and Council retained ultimate responsibility. The committees organized referees for papers being considered for the Transactions and provided input into decisions about publication in Proceedings and selection for discussion meetings.
Despite the changes in management, the practice of refereeing continued largely unaffected through the 1890s. The new 1894 letter of guidance for referees had codified the intellectual distinction between Proceedings and Transactions that referees had been working with for decades, stating that Transactions papers should ‘mark a distinct step in the advancement of Natural Knowledge’.Footnote 60 Publication in the Proceedings was still seen as more routine: ‘short’ papers (of less than twelve pages) and abstracts could be printed there on the authority of the secretary and the chair of the relevant Sectional Committee, without necessarily consulting the other committee members.
There was, however, one newly prominent aspect to the refereeing process: money. In spring 1894, John Evans, the Society's treasurer, had reported to Council on ‘the difficulties in which we are placed’ due to the soaring cost of the Society's publications.Footnote 61 He therefore made a series of recommendations to Council, including limits on the length of individual papers and the cost of the accompanying illustrations, and greater scrutiny of all submissions at an earlier point in the process. The Council's response was lukewarm, but it eventually set limits on pages and illustrations for Transactions, with loopholes that would be regularly exploited.Footnote 62 The Sectional Committees were a response to the desire for scrutiny earlier in the process.
The financial concerns were clear in the new guidance to referees, who were now asked specifically about length and illustrations. Should papers ‘be published in full or in an abridged form’? Could ‘any portions be omitted as being unnecessary’ (or as ‘liable to give offence’)? And, most explicitly, could the illustrations ‘be reduced in number or extent without actual injury to the paper, with a view to economy?’Footnote 63 Referees had, from time to time, suggested possible cuts for economic reasons, but the scale of the underlying problem was new and not resolved by the Treasury grant-in-aid of publications, first awarded in 1895.Footnote 64 Thus, new procedures for the Committee of Papers in 1896 specified that it was to consider estimated costs alongside the referees’ reports, and from 1907, referees were also informed of the estimated costs.Footnote 65 Evans's memorandum of 1894 had thus inaugurated a practice of weighing financial implications against intellectual merit, though how referees were expected to do this remained unclear.Footnote 66
The understanding that refereeing was not simply about judging merit – whether a paper was ‘fit and proper’ for a Royal Society periodical – is also apparent from its role in the editorial process for the Proceedings. In the nineteenth century, Proceedings had been regarded as secondary to Transactions, and its decisions were usually made without input from referees. By the early twentieth century, around 75 per cent of papers submitted to the Society appeared in Proceedings, with only 12 per cent in Transactions.Footnote 67 A large element of the Society's public reputation thus rested on the Proceedings and those who controlled access to its pages. Involving the chairmen of the Sectional Committees suggests some concern that the Society's long-established gate-keeping procedures – dependent on the communicators and the secretaries – were not completely adequate for Proceedings.
The requirement that papers be ‘communicated’ by a fellow acted as a filter, both social and intellectual, on submissions, and helps to explain the low overall rejection rate for papers received by the Society. A fellow acting as a communicator had long been expected to ‘satisfy himself that the paper is a fit and proper one to be communicated to the Society, and has not been previously published elsewhere’, but already in 1894, John Evans had been worried that this did not lead to adequate scrutiny.Footnote 68 It was becoming a pressing concern because the growing number of scientific researchers, combined with the more restrictive admissions policy that the Society had been operating since 1847, had resulted in an increase in the number of papers communicated on behalf of non-fellows. Such papers had accounted for barely 40 per cent of submissions in the 1860s but had risen to over 60 per cent by the early twentieth century.Footnote 69 Evans proposed that all papers by outsiders – even for Proceedings – should be examined by referees; but Council rejected the idea.
The secretaries (and other officers) had always acted as a check on what communicators submitted; but by the late nineteenth century, two men could not hope to be knowledgeable on all possible subjects. Moreover, they also had responsibility for an increasing range of Royal Society activities.Footnote 70 One of Lister's rationales for the new Sectional Committees was to free the Council and officers from the minutiae of publications, so they could devote more attention to ‘matters of larger policy’.Footnote 71 Thus, having the chairmen of the new Sectional Committees assist the secretaries in making decisions for Proceedings was a compromise which ensured someone with knowledge of the general field was involved, without slowing things down as much as refereeing would.
The role referees played as stewards of the Society's finances helps us to understand an otherwise puzzling element of the editorial history of Proceedings: from 1914, Proceedings papers were granted equivalent intellectual status to those in Transactions, but this did not lead to the institution of refereeing.Footnote 72 One key difference between the journals was that the page limit for Proceedings papers was more rigorously enforced. Even newly increased to twenty-four pages, that limit constrained the financial implications. The forty-page limit for Transactions, on the other hand, was routinely breached, with the Committee of Papers sometimes approving papers of more than a hundred pages plus illustrations.Footnote 73 Thus, the financial implications of approving a paper for Transactions were more variable (and potentially far higher) than for Proceedings, and merited greater scrutiny. A second issue was that the Society was keen for Proceedings papers to be published more rapidly, and seeking referee reports slowed things down. Although refereeing did gradually come to be used for Proceedings papers over the next few decades, one referee rather than two (or three) became the norm. The Society appears to have been content to retain a lighter-touch editorial regime for Proceedings.
V
The decision to continue refereeing – amidst the many changes of the 1890s – is significant. As the chemist (and member of Council) Henry Armstrong pointed out in 1902, Lister's reforms had done little to streamline the editorial process: by adding committees as well as referees, ‘the machinery of publication has…been complicated rather than simplified’. Armstrong argued that the Society should reconsider ‘the appointment of an Editor’, after 150 years without one; and he described the continued use of refereeing as ‘the old plan’ and ‘an anachronism’.Footnote 74 One of the routine criticisms of refereeing was its one-sided confidentiality: Armstrong repeated the concern that it ‘too frequently’ led to ‘ill-feeling’; and in 1922, an early trade union for scientists would claim that Society referees were ‘anonymous and irresponsible’.Footnote 75 Another criticism was the time taken by referees, with authors feeling that referees delayed publication, and referees (according to Armstrong) worrying that much of their ‘valuable time’ was being ‘practically wasted on such work’.Footnote 76
A very different concern had been publicly admitted by Lord Rayleigh in 1892, when he arranged for the belated publication in the Transactions of a paper by John Waterston that had pre-empted Maxwell's work on the kinetic theory of gases. Waterston's paper had been rejected by Royal Society referees in 1845, and thereafter languished in the Society's archive. Its history demonstrated, said Rayleigh, the conservativism inherent in the refereeing process, since a representative of a learned society ‘naturally hesitates to admit into its printed records matter of uncertain value’. Rayleigh read this as an indication that learned societies were not the best channels for bringing ‘highly speculative investigations, especially by an unknown author’ before the world.Footnote 77 Such an admission, by the serving secretary of the Royal Society, was a striking indictment of the refereeing process. With such criticisms of refereeing, from both within and without the Society in the decades around 1900, the Society's on-going commitment to its slow, convoluted editorial processes could be seen as out of step with the needs of professional, international science in the twentieth century.
Certainly, as others have shown, it is clear that few proprietors of independent scientific journals in the late nineteenth or early twentieth centuries felt any need to adopt similarly complex processes for editorial scrutiny.Footnote 78 What ‘refereeing’ there was tended to take the form of informal consultations with trusted acquaintances, and editors relied strongly on their own instincts, and on the reputations of the individuals and institutions concerned – much as Joseph Banks had done.Footnote 79 The lack of enthusiasm for systematic refereeing at the independent journals is further confirmation that refereeing was originally part of an editorial system distinct to the learned societies.Footnote 80
Compared with the long, labour-intensive, and comparatively inaccessible publishing processes at learned societies, the swift editorial decision-making and more rapid publishing frequency of the independent journals made them attractive to authors looking to publish quickly, especially in fast-moving fields like physics.Footnote 81 Independent journal editors could follow their own instincts and interests, with no need to represent or protect the corporate reputation of a sponsoring organization through mechanisms for collective responsibility. Their desire for speedy publication was better served by making executive decisions than by seeking referees’ reports. Thus, in the early twentieth century, the practice of refereeing could be seen, in some quarters, as an obsolete holdover from an age of amateur dominance, out of touch with the needs of the new professional scientist – a remarkable transformation from the 1830s, when refereeing had been one of the chief demands of a reform movement that championed the expansion of professional science and the imposition of more stringent qualifications upon men of science.
The Royal Society – and other learned societies – continued to use referees (and communicators and committees) through the twentieth century. However, just as they had done in the eighteenth and nineteenth centuries, the societies’ editorial practices were compelled to adapt to changing circumstances. The editorial system developed by the Royal Society to protect the prestige of a very old organization, much of whose conduct was still rooted in the idea of gentlemanly civility, responded – gradually – to the needs of professional, international scientific research, during a period in which its responsibilities increased yet the share of British scientific activity it represented and its role in the life of most of its members diminished.
It was not until the late 1960s that any major reforms were discussed: in 1967, the system of editorial management was (again) described as ‘outdated and cumbersome’,Footnote 82 and in the reforms which followed – as in subsequent reforms in 1990 – the aim was to make the Society's procedures more effective and streamlined. From 1969, the editorial work done by the chairmen of Sectional Committees was transferred to a new (larger) group of fellows designated as associate editors. Those associate editors were still nominally under the authority of the secretaries and Committee of Papers, but positive recommendations were to be ‘automatically endorsed by the appropriate secretary’;Footnote 83 and from 1990, fellows were appointed as editors with full responsibility for each of the Society's journals. After 238 years, the Committee of Papers was disbanded, and the secretaries relinquished their role in managing the Society's publications. The Society's corporate interests are now represented by the fellow acting as editor, and by the fellows who serve (alongside non-fellows) on the advisory Editorial Boards.
While both sets of twentieth-century reforms were principally about management, they incorporated some changes to the procedures of communication and refereeing. The end result was the removal of the privileged role of fellows in the editorial process. The duke of Sussex in 1832 had felt it entirely appropriate that publication decisions be made by those who ‘have earned by their labours an European reputation’, but by the 1960s and 1970s questions might have been raised about the fairness of a self-selecting group of senior scientists, mostly male and mostly British, sitting in judgement on the work of researchers of all genders, ages, and nationalities.Footnote 84 However, the rationale behind the Society's reforms appears to have been practical effectiveness, rather than an attempt to dispel any accusations of unfairness.Footnote 85
By the 1960s, refereeing had become standard practice for both the Society's journals, and it was clarified that this meant ‘at least one independent referee other than the communicator’.Footnote 86 Other than drafting new guidance, and from time to time revising the printed report form, the Society made few changes to the actual practice of refereeing during the twentieth century. For instance, it continued to keep referees’ names confidential but to share the identity of the authors, even though, from the mid-1950s, some journals began anonymizing authors as a means to protect them from the perceived biases of referees.Footnote 87 The ongoing use of ‘single-blind’ refereeing at the Society – and in the sciences more generally – illustrates the enduring legacy of nineteenth-century learned society practices. However, the rules about the involvement of non-fellows were relaxed.Footnote 88 The guidelines drawn up for the new associate editors in 1969 included explicit provision for dealing with referees who were not fellows, and even those who were resident overseas.Footnote 89 Like so many earlier reforms, widening the pool of potential referees could be presented as a means of ensuring appropriate expertise, but it also helped to spread the load on busy fellows: in 1950, one had complained that ‘if I get much more heavy refereeing like this, it is goodbye to any chance of doing real scientific work myself’.Footnote 90
Refereeing retained a financial dimension, with referees in the 1960s receiving an estimate of the number of pages and images (though no longer an actual cost). For public consumption, however, refereeing was increasingly presented as a valuable process for enhancing the quality of the publication of primary research. The Society's fellows were ‘so critical and helpful a body’ of referees, that their voluntary work, it was argued, was an asset that could be matched by ‘no Journal in the world’.Footnote 91 In 1957, the Society's assistant secretary had linked refereeing to quality, telling members of the Association of Special Libraries and Information Bureaux that ‘the quality of scientific content’ published by learned societies was ‘maintained by high-class refereeing’ carried out by their members.Footnote 92 A 1967 suggestion to publish un-refereed papers in Proceedings (as had been done prior to the 1930s) was dismissed as risking ‘a degeneration of standards for presenting new scientific knowledge’, despite its advantages for speedy publication.Footnote 93 This positive articulation of the value added by referees was the more necessary in a context where the professional advantages of rapid publication weighed increasingly heavily with authors.
If referees were claimed to be an asset to the Society's periodicals, communicators were a more ambiguous legacy. The 1890s worries about whether they were screening submissions carefully enough continued. In 1936, for instance, one fellow blamed the ‘increasing bulk’ of submissions of ‘routine research’ – which he deemed inappropriate for the Society – on Ph.D. supervisors who were too keen to push their students forward. He claimed that these fellows had forgotten that communication involved ‘a duty as well as a privilege’.Footnote 94 And if communicators could not be relied upon, the referees had more to do. Hence, one referee wished that fellows sending in weak papers by their students ‘would only take the trouble, exercise their undoubted critical powers and have the papers put into proper shape, or in some cases stopped, before sending them in’.Footnote 95 Such complaints hint at the challenge for senior scientists in balancing loyalties to their universities and their students as well as to the Royal Society.
VI
When David Davies became editor of Nature in 1973, he made refereeing a standard practice, seeing it as a way to raise the journal above accusations of cronyism and elitism, and during the cold fusion episode in 1989, his successor John Maddox would trumpet peer review as an essential process for scrutinizing scientific research before announcing it.Footnote 96 Thus, by the 1990s, a process which was once an oddity of learned societies had come to be seen as a normal and essential practice for all scholarly journals (and in other research evaluation contexts).Footnote 97 Even though John Burnham's 1990 survey of editorial peer review acknowledged that it had ‘been essential to science and medicine’ only for ‘at least two generations’, Zuckerman and Merton's discussion of the early Royal Society has enabled many subsequent commentators to project ‘peer review’ back onto the 1660s.Footnote 98 By glossing over the intervening three centuries, scholars have ignored the period in which refereeing and collective decision-making actually developed, and whose legacy is still apparent in current practice.
Peer review has become conceptually inseparable from professional science in Britain. Even though recent scholarship has suggested that the professionalization of science was far from complete by the late nineteenth century, it was clearly well advanced at least fifty or sixty years before peer review began to acquire the indispensable status it now enjoys.Footnote 99 The point goes beyond the fact that widespread peer review was not apparently a necessary condition for the rise of professional science. Over longer perspectives of the kind opened up in this article, we see that the relationship fluctuates. For instance, refereeing at the Royal Society when it was first instituted was strongly championed by fellows such as Babbage and Herschel who, if they were not precisely advocates of professionalization, certainly favoured stronger commitments to the advancement of science among the fellows, and their impulse to reform the Society is now widely understood as a precursor to it. By the turn of the century, however, refereeing had come to be thought of in some quarters as a holdover from the age of amateur dominance, and an impediment to the Society's efforts and those of its members to engage with the modern scientific world. Some of the first scientific trade unions in Britain spoke out against it in the 1920s, as if to clarify the tension between the Society's practice and emergent professional norms, while the criticisms of the 1930s read the communicator's privilege as a means of subverting professional standards. As late as the 1950s, refereeing was still in need of defence, as a practice underpinning the learned societies’ unique role in publishing high-quality original research. The epistemic purpose of refereeing also underwent a transformation, from a public foil to set off and amplify the very best of the research received by the Society in the early nineteenth century to an instrument for ensuring the application of minimum thresholds of quality across the board while allocating space (and therefore resources and prestige) on the basis of expert assessment. At the same time, the implementation of refereeing could be modified locally, creating space to compromise and reconcile the Society's desire to publish high-quality research at fully developed length with the mounting, discipline-wide pressure towards shorter, more rapid communication in the sciences.
This is our central point: that the relative durability of refereeing as a practice should not be mistaken for simple continuity of purpose or of meaning. What it was meant to accomplish, whom it was intended to benefit, and the perception of its virtues and defects varied considerably with time and place.
It has, of course, also varied with discipline. There has been less research into the historical development of refereeing and peer review in the humanities and social sciences.Footnote 100 This is partly because scholarly journals in these fields developed later than in the natural sciences (English Historical Review, 1886; American Historical Review, 1895; Annales, 1929; Past and Present, 1952); and their editors' adoption of refereeing or peer review appears to have been even later. As Mark Goldie has described, this journal's predecessor, the Cambridge Historical Journal (f. 1923), was originally edited by a society of university historians who selected papers for publication from the talks at their meetings, and double-blind peer review at the Historical Journal dates only from the late 1990s.Footnote 101 The adoption of peer review by a wide variety of humanistic and social science disciplines reveals both the long-standing (if contested) envy of the epistemic rigour apparently associated with the natural sciences, and the professionalizing desire to adopt what has come to be seen as ‘proper’ academic practice.Footnote 102 It is also important to consider how the ongoing importance of monograph publishing in the humanities creates a different history of practices of editorial evaluation and selection. The evaluative practices of book publishing include the issuing of advance contracts before the manuscript has been completed; as well as the vibrant practice of public, post-publication reviewing (i.e. book reviews). The actors are also different: (non-academic) commissioning editors wield significant power; and publishers' readers have traditionally evaluated both intellectual merit and market potential.Footnote 103 This brings us back to our central point: that the nature and purpose of refereeing, and of peer review, vary importantly with context.
For our formative example of the Royal Society in the late eighteenth and nineteenth centuries, refereeing was a luxury – a possibility afforded to an organization with a unique position in the history of science and in British scientific organization, one strongly aware of that position, and possessing both a captive population of scholars obligated to serve the Society's ends and sufficient financial resources to promote scholarship (mostly) for its own sake. The mismatch between the context of the gentlemanly learned society (in a national context) and modern, professional, international science helps to explain some of the accusations now being levelled against peer review as not being ‘fit for purpose’. If our aim, therefore, has been to show the complexity, contingency, and historical specificity of peer review's origins, our ambition is to start a scholarly conversation about which of its attributes still seem desirable, whether it remains good for all disciplines, whose interests it serves, and what the realistic limits of its pretensions might be.
