INTRODUCTION
This article examines a selection of online information resources used by criminologists and compares their particular features and deficiencies. Despite extensive research into the information needs and behaviour of many different groups of users, there seems to be no recent examination of the needs of criminology and criminal justice researchers. David Nicholas and his colleagues, in their 2010 researchFootnote 1 into the e-journal use of and information-seeking behaviour of UK researchers, surveyed ten major institutions and looked at Life Sciences, Chemistry, Earth and Environmental Sciences, and Physics from STEM subjects, but only Economics from the social sciences. In the early days of digital integration into the academic environment, efforts were made in a number of disciplines to address changing information literacy needs through the curricula of courses of study. Bouloukos, Benmati and Newman's 1995 articleFootnote 2 describes the information literacy course they devised and taught for criminal justice students at Albany, based on the then-prevalent idea of cultural literacy. It included an introduction to email and TELNET, online bulletin boards and newsgroups, using e-journals, searching library catalogues around the world, critical thinking and evaluating authority, basic legal research skills, construction of databases, and managing retrieved information. The course was successful and, allowing for intervening advances in technologies, the structure looks serviceable today. Between then and now, though, no more research into the needs of criminology students seems to have taken place and the teaching of those skills has moved out of the core curriculum to the optional training available through library and similar services.
The present article is much more restricted in scope and does not seek to replicate or update any of the work mentioned above. Instead, it brings to bear the experience of the author as the Librarian of a post-graduate research and teaching institute, training and working with early career researchers (ECRs), in the realm of research resources. The resources examined herein are not exclusively criminological because criminology itself is such an interdisciplinary subject. There are a number of well-established sub-specialisms, including policing, prisons, probation, sentencing, forensic psychiatry and psychology, and socio-legal studies. Different and occasionally antagonistic perspectives also thrive: sociological, psychological, biological and neurological criminology are examples. Accordingly, criminological research involves fishing in a number of ponds: politics, medicine, physiology, philosophy, theology, and media studies, among others, all contribute to the subject as, of course, do law and jurisprudence.
Although there is no formal methodology behind this article, as a librarian training ECRs in the use of the resources reviewed in this article, it has been possible for the author to draw on users’ responses to the resources. Those experiences, as recorded over a period of some years, explicitly inform this review. The ECRs involved were all members of the Institute of Criminology at the University of Cambridge and include MPhil and MSt students, PhD students, postdoctoral researchers and junior research associates. All are defined here as early career researchers. Although it is perhaps not usual to include Masters-level students as ECRs, the Cambridge MPhil programme has a markedly research-led culture, including a research stream, and the MSt programmes are research degrees.
Users’ expressed experiences of the resources largely align with Tsakonas and Papatheodorou's components of usefulness and usability (Tsakonas and Papatheodorou, 2006Footnote 3 ), which are used here as assessment criteria: strong preferences for relevance, currency, reliability, and learnability are all expressed by users. Currency is often a stronger concern in ECRs engaged in project work and/or working towards publication. Masters students writing essays value currency but are understandably more directed towards finding the material that fits most closely with the question under consideration. They do, however, turn their attention more to currency when working on dissertations, especially as many of those dissertations involve original research. Ease of use and navigation preferences are rolled into learnability, with the concept of similarity or familiarity between different interfaces being the most common way of expressing these preferences. These latter concerns comprise the usability content of the responses and reflect the most widely acknowledged guidelines for user-interface design (Johnson, 2014Footnote 4 ). Not listed by Tsakonas and Papatheodorou but vital for the ECRs is the issue of coverage. This relates to both the breadth of information available and the temporal depth: comparisons of e.g. crime rates or prisons populations over time are an important dimension in the subject.
For ease, the resources examined here are grouped into three categories: academic databases, social networks, and government information services. These categories are necessarily broad, with academic databases in particular covering everything from commercially provided subscription services to search engines and repositories.
ACADEMIC DATABASES
Criminal Justice Abstracts
Criminal Justice Abstracts (CJA) is an online database provided through the EBSCOHost platform on a subscription basis. Originally published in print form, it began life in 1968 as Information Review on Crime and Delinquency then, in 1969, became Crime and Delinquency Literature until 1976, then Criminal Justice Abstracts, all published by the National Council on Crime and Delinquency and latterly in cooperation with the Criminal Justice Collection of Rutgers University Libraries. Print publication ceased with the December 2010 issue.
As a constituent part of the EBSCOHost platform, CJA benefits from combined searching with other databases relevant to the subject: PsychARTICLES, PsychINFO, Child Development and Adolescent Studies, Education Abstracts, EconLit, and, for a few, ALTA Religion Database. Breadth of coverage is greatly expanded by this and the need for depth is sufficiently met with content going back to 1910. Coverage is spread over academic journals, books, trade publications, magazines and reviews, with the useful ability to filter results by these publication types. Trade publications can be advantageous in a subject that deals extensively with difficult-to-access professions like policing, security, or the prison service, and in helping to bridge the gap between theory and practice. Introductory material for the databases promises cover-to-cover indexing and abstracts for over 600 journals, searchable cited references for over 220 journals, and over 560,000 records. Also noted as helpful by a number of users was the inclusion of delimiters for full text access, available references, and peer reviewed journals. It was noted by one user that a previous design incarnation of CJA had results from books, periodicals, and peer reviewed journals on separate tabs and that the loss of this was regretted. Nevertheless, it is generally felt that filtering for peer reviewed material was a strong and visible guarantee of reliability. The full text access delimiter is particularly welcome for students and researchers working to deadlines. Currency of material is excellent, with results returned for articles in current journal issues. The flexible interface in the Advanced Search option allows for very exact searches that specify title words, keywords, author, companies, publication names, ISSN and ISBN, geographical terms, date range, document and publication type, language, and other criteria. It also supports wildcard searching. Starting from a search page presenting multiple configurable search fields, this was the type of interface that users tend to characterise as ‘standard’ or ‘familiar’, making any resource that features it quick and easy to learn.
Web of Science
Web of Science (WoS) is an online database provided by Clarivate Analytics as a subscription service. It covers disciplines in the physical sciences, social sciences, and arts and humanities. Describing its Core Collection, the publishers claim coverage of over 18,000 journals with 1.3 billion cited references, and 100% fully indexed cited references, authors, and author affiliations. In the social sciences, they claim over 3,000 journals across 55 social science disciplines. As well as accessing the WoS Core Collection, searches include MEDLINE content by default.
On the face of it, WoS would appear to be less usable than Criminal Justice Abstracts. It is less specialised and, whilst results can be refined by domain (e.g. social sciences), research area (e.g. psychology, sociology), document type, authors/editors, language, source title, and even funding agency, there is no option to refine by full text access or by peer review status. Nevertheless, the results retrieved by WoS tend to feature a high percentage of journals that are generally recognised as leading titles in their sub-specialism and users find this reassuring. Regarding user selection for full text access, WoS presents a ‘Find Full Text’ button for each result, linked to the user's home institution, and ‘Full Text from Publisher’ button for each result where it can find a full text subscription at that institution. It also allows the user to refine results by Open Access status. Some users notice that relevance is markedly less than Criminal Justice Abstracts although the subject coverage is very much wider, of course. Several ECRs noted that the same search run in both databases would produce similar but by no means identical results, estimated as 80–90% matching. Whilst WoS produced plenty of less relevant results than CJA, it also retrieved a number of useful results that the specialist database did not. Again, currency was excellent with the database returning results for current journal issues. The interface also matches the ‘familiar’, ‘standard’ design and wildcard searching is supported. Web of Science supports a number of usability features that were very positively regarded by ECRs. The first is its integration with bibliographic management software. Whereas other databases (e.g. CJA) allow users to export lists to bibliographic software, WoS interfaces directly with EndNote Online, allowing the user to add references to their EndNote account with a single click. EndNote Online is a free, cloud-based utility, making it simple for users to access their references from anywhere and impossible to lose them, both points warmly welcomed by researchers. The Citation Network in individual WoS results is another valued feature. From this sidebar, users can follow the life of an article backwards in time, via links to the references cited in it, and forwards via links to other works that cite the current article. Citation alerts, to notify the user when the article is referenced in future, are also available, as is the ability to save searches.
Repositories
As open access publishing and depositing work in institutional repositories become expected components of the academic workflow, so those repositories become an increasingly valuable research resource. Whilst coverage is still far from total in many subjects, users value the open access dimension of repositories and especially the access to new doctoral theses, which can otherwise by expensive and difficult to procure. At the moment, though, cross-searching repositories can be difficult, with a number of utilities existing, all with different levels of reach and transparency. The Sherpa Partnership, run by the Centre for Research Communications at the University of Nottingham, provides a full-text search facilityFootnote 5 , of all UK open access repositories based on the Google custom search engine. It presents a single non-configurable search box and, though the search is fast and can retrieve large numbers of results, there are no tools to refine those results and no indication of the ranking criteria. The Directory of Open Access Repositories, OpenDOARFootnote 6 , is another Sherpa service, a project to list and categorise academic open access research repositories across the world, a sister project of the Directory of Open Access Journals (DOAJFootnote 7 ). OpenDOAR searches for repositories themselves, rather than for their content, and has a configurable but slightly idiosyncratic interface. It has controls for repository type (aggregating, disciplinary, governmental, institutional) and content type (books, articles, datasets, theses, etc.) and for subject area and country. The subject menu has no listing for criminology, although it does have social sciences as a general category, with Law and Politics listed as a subset of that. The country selection menu is initially puzzling as it ignores the standard web practice of listing countries alphabetically by name, instead choosing to list them as subdivisions of the different continents.
The Digital Commons NetworkFootnote 8 is curated by university librarians and claims to list 2.4 million works from 530 institutions. Again, it has no specific criminology listing but it does have both law and social/behavioural Sciences sections with a number of useful subdivisions – law and society, law enforcement and corrections, juvenile law, psychology, and sociology all feature, along with many others. The search interface is a single non-configurable box but results can be refined by discipline, institution, keyword, date, publication type, and file type. Subjects have their own sections and can be searched individually or together from the top level of the site. Whilst Boolean searching clearly works, there is no guidance provided on search capabilities. Testing wildcard searches produced unpredictable results that often contained none of the search terms.
The most user-friendly and configurable utility for cross-searching repositories is BASE, the Bielefeld Academic Search EngineFootnote 9 . Operated by Bielefeld University Library, it claims more than 100 million documents from more than 5,000 sources, with full-text open access to about 60% of the indexed documents. The basic search is a single box, configurable by title, author, subject, or the entire document, and offering verbatim search, additional word forms, and multilingual synonyms via the Eurovoc thesaurus. The advanced search is much more configurable, adding options for DOI, (part of) URL, along with options for date range, country, terms of re-use/licences, and 23 different document types. These include books and their chapters, articles, audio, video, still images, maps, lectures, course materials, datasets, software, and dissertations divided into Bachelor, Master, and Doctoral and Postdoctoral. Results can be sorted by relevance, author, title, and date, and can be refined by author, subject, Dewey classification, year, content provider, language, document type, access, and terms of re-use. Searches can also be saved and results emailed. Both the basic and advanced search allow the user to boost open access results. Of a number of repository cross-search facilities, BASE is by far the most impressive.
Google Scholar
Google Scholar claims to provide ‘a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites.Footnote 10 ’ Like Web of Science, Google Scholar is a general academic search resource, not a specifically criminological one. Also like WoS, some ECRs report comparing identical searches with WoS and CJA and retrieving many results that were not relevant but some fruitful ones that neither of the other databases found. The unique results are seen as a partial trade-off against the lack of specialisation, ameliorating it somewhat but not entirely. The search interface, with its single non-configurable entry field, is a significant drawback, reducing the accuracy of the initial search and lowering the perceived relevance of the results. This is despite the interface being familiar from the standard Google web search tool: users tend to experience it as ‘not academic enough’. This lack of perceived relevance undermined one of the features that could otherwise have been seen as an advantage - the ‘related articles’ link below each result. As each original result is less than optimally relevant, and each related article only partly relevant to the first, any sense of the user's original intention or need can be quickly lost. Reliability is also a concern. Whereas Web of Science is trusted at least partly on the reputation of the journals it retrieves in results, Google Scholar consistently finds much more obscure titles about which users have no trustworthy information: peer review status, reputation, and perceived value within the field are all unknown. Although Google, and by extension Google Scholar, is recognised to have exceptionally broad coverage, this lack of contextual information undermines the value of that coverage. Broadly, a huge list of results is not too helpful if users don't know whether they can rely on them or not. Commenting on this problem, one ECR said that “Google Scholar is handy for ‘quick and dirty’ searching but it gives you quick and dirty answers.” Reliability is also brought into question by inconsistent results. A quick test of the search on a small sample of titles known to be in academic repositories, for example, produced positive results for only half. Currency, on the other hand, is excellent and another positive feature is that Google Scholar allows users to set up alerts by saving queries, although the interface for this is again unsophisticated – another single, non-configurable entry field.
Zetoc
Zetoc is a research database ‘giving access to over 30,000 journals and more than 52 million article citations and conference papers through the British Library's electronic table of contents’ (Zetoc, 2018).Footnote 11 It is funded by JISC, the Joint Information Systems Committee, described as ‘the UK higher, further education and skills sectors’ not-for-profit organisation for digital services and solutions’ (JISC, 2018Footnote 12 ). Although Zetoc is a searchable academic database, its main value to ECRs is an alerting service. Users can add journal titles or keyword searches (including subject terms and author names) to lists and be alerted when a new publication matches their query. For journal titles, the user receives an emailed table of contents, with hyperlinks to the online journal, for each new issue: the ‘etoc’ part of the name stands for ‘electronic table of contents’. Clearly, currency is Zetoc's major asset, coupled with the same ‘push’ factor that ECRs value in social networks: the user identifies and formulates ongoing information needs and leaves Zetoc to retain and respond to those needs in the future. Again, like social networks, the user builds relevance and reliability into the query from the beginning by selecting trusted and useful journal titles and authors. Coverage is extremely broad but spread over the whole spectrum of academic disciplines. As with Google Scholar, the problem of contextual information about unknown journal titles arises: Zetoc present long lists of titles to choose from but no information from which a user could deduce their academic value. The fact that Zetoc derives all its data from the British Library is seen as a mark of reliability, although it is not necessarily clear to researchers that the BL's involvement is more an imprimatur of the accurate presentation of the information, rather than the quality. The Zetoc interface is very simple, with clearly labelled buttons to initiate straightforward tasks. There is, however, no clear explanation of what an alert list is or does, or why users would want one. ECRs have said that, without an initial grasp of that concept, they felt they were ‘poking around in the dark.’ Once the idea was explained, though, both the process and the value became immediately transparent.
ACADEMIC SOCIAL NETWORKS
ResearchGate and Academia.edu are probably the two best known academic social networking websites, whose ‘mission is to connect the world of science and make research open to allFootnote 13 ’, in the words of ResearchGate. They allow members to share publications, connect and collaborate with colleagues, ask and answer questions, and search for jobs. ResearchGate claims more than 14 million members and Academia more than 59 million. The most valuable feature for ECRs is that they push information towards the user, in the same way as any other social network. One need only ‘follow’ another member to have all that person's additions to the site appear in one's email inbox. To many respondents, this holds the potential of being the most current service possible because information is available and notified to users as soon as it is uploaded. Similarly, users find the information highly reliable and relevant because they have elected to follow authors whose work is consonant with their own interests and of high academic standing. The integrity of the work is here guaranteed by the combination of both the professional status and the accessibility of the author, more than by the title of the journal in which it appears. Coverage is naturally limited by the membership of the network and, whilst it is not expected to reach the level of the major commercial databases, it will certainly improve over time. Members are also currently adding their own back catalogues of work, deepening the available content. As a result of this structuring principle, the level of availability (full text, abstract only) depends on what the author chooses to upload and is therefore less predictable than it would be in a subscription database. The default search interface for both is a single-entry field (Academia has an advanced search facility as a premium feature) and results are grouped into useful categories: researchers, projects, publications, institutions, departments, questions, and jobs. The fact that the results present people before publications is telling: the search function is not the primary interface for this site; the social network timeline is. To most younger researchers, this makes these sites easily learnable and immediately comprehensible.
The familiarity of the site model, though, is reflected in a familiar controversy about social media sites. Users of Academia may not be aware that they are freely providing their work and data to a venture-capital backed, for-profit enterprise. In 2016, Academia was heavily criticised for emailing members asking if they would pay a fee to have their work recommended on the site. The criticism put an end to that initiative but Academia has found other ways to monetize its content by adding premium features. ResearchGate has been criticised for copyright infringements, for spamming co-authors of users with unsolicited invitations that appear to be written by those users, and for automatically creating profiles for people who are not registered by scraping the web for people's affiliation, publications records, and PDFs. Campaigns exist asking researchers to delete their Academia and ResearchGate accounts and to migrate their work to open access platforms instead.
GOVERNMENT INFORMATION SOURCES
Official Publications Online
Provided by The Stationery Office (TSO), Official Publications Online (OPO) is an online collection of the UK government's publications, described as a ‘web based subscription service that offers instant, searchable access to official publications, including legislation, Command and House papers, Hansard, bills, votes and proceedings, Select Committee reports, Public Bill Committee Debates.Footnote 14 ’ It includes ‘PDF availability on the day of publication, customisable email alerting service and RSS feeds, and a single, comprehensive search engine across all official publications.’ OPO is a subscription service and, as such, is restricted in accessibility, although a free alternative covering some of the same material, exists at www.gov.uk. That website allows any user to search 25 UK ministerial departments and 385 other agencies and public bodies, whilst providing separate links to policies, announcements, publications, statistics and consultations. Statistics, for example, can be searched by keyword, policy area, department, and published before/after dates. Publications can be searched by those criteria plus official document status and world location. The site is intended for general public use but its search functions are sufficiently configurable for an informed searcher to quickly find material for academic use.
Returning to OPO, the default search option is a single non-configurable entry field but an advanced search page is available. This does, however, present its own problems. A generous number of pre-search filters allows the user to select for publication type, subject classification, year or date range, or ‘publication specifics’: ISBN, title, author, or corporate author. For a known publication, these can make a search exact and instantaneous. Subject searching lacks granularity, though, with the Subject Classification menu listing only Law to cover subjects such as crime, policing, prisons, probation, etc. By contrast, the Publication Type menu is very long but often incomprehensible: options include CAA, CM, HLCR, HMG-Other, LCCD, and WAA-EN, among many others. The resource has a Help section but there is no list of what all these abbreviations denote. Along with these, there are comprehensible terms (e.g., House of Commons Paper, Act of the Northern Ireland Assembly) but the opacity of many means that users lack the opportunity to use these filters to their full effect. They may miss something that hides behind an acronym they don't recognise. The presentation of search results also lacks any tools to further refine those results and, given the complexity of the material on offer, this is an important omission. Wildcard searching is supported but this is not mentioned in the Help section on searching. Whilst learnability is often seen by users as low, currency, relevance, reliability and coverage are all experienced as high. All publications are available in full text on the day of release and all originate from the machinery of UK government. As a specialist resource for official publications, total coverage is achieved with the added convenience of bringing all together behind a single interface. Some users also noted as helpful that print copies of the publications retrieved can be bought from TSO, the provider of OPO, by click-through sale.
National Criminal Justice Reference Service
The National Criminal Justice Reference Service (NCJRSFootnote 15 ) is the USA federal government's online centre for crime and justice-related research and data, and policy and practice publications. Established in 1972, it is ‘a federally funded resource offering justice and drug-related information to support research, policy, and program development worldwide.’Footnote 16 It is the ‘research arm of the National Institute of Justice’ (Nelson, Killoran and Dunham, 1995Footnote 17 ), sponsored by a partnership of Federal agencies from the Department of Justice. Along with freely available publications, NCJRS offers an alerting service including email alerts, listserv invitations, mailing of publications, and JUSTINFO, ‘a bi-weekly electronic newsletter that includes links to full text publications, notices of upcoming trainings and conferences, funding announcements, and other resources.’ Again, print versions of publication are for sale through the site. American criminal justice systems and data are an important source of comparison for UK criminology in general; in particular, American research students and post-docs working in the UK rely on NCJRS as a significant resource.
Like OPO, NCJRS presents a simple non-configurable search option, which researchers usually ignore in favour of the advanced search. Even this, though, is seen as ‘primitive’, with a single search box refined by content type and options to search by Any, All or Phrase. The content types on offer include federal justice websites, events, Q&A, and full-text or abstract-only availability. Deselecting any of these options does not, however, prevent them being searched and presented in the results, although the selected types do appear at the top of the list. Wildcard searching is supported but not mentioned in either the separate basic or advanced search help sections, only in the search hints – again, presented separately. The results list is presented with a sidebar enabling refinement by the same content types as the search interface so these filters can be successfully applied after the fact. Still, the sophistication of the interface has clearly fallen far behind the complexity of material to be searched. Like OPO, the perceived value of NCJRS lies in the relevance, currency, reliability and coverage. More specialised than OPO, the coverage of American criminal justice information is complete. As a research organisation, NCJRS presents current contextualised data as accessible reports, mostly 12 pages long, with pull-out highlights, tables, and crisply written text. This style of easily digestible presentation works well for researchers under time pressure. Also appreciated is the fact that these reports often appear in long-running series with stable titles and methodologies, enabling like-for-like comparisons over time. These annual series include subjects such as drug use and dependence in state and federal prisoners; HIV in prisons; compendia of federal justice statistics; censuses of law enforcement officers; gangs; uniform national crime reports; capital punishment; juvenile arrests, and a great many others. Many of these series date back for decades and longer-term digests are published at other intervals, e.g. ten, twenty or twenty-five years. The granularity of this information enables high relevance to the needs of researchers and NCJRS's position as the US government's criminal justice research agency guarantees its reliability.
UNAFEI
Finally, an often-overlooked resource that is little known among younger researchers. UNAFEI is the United Nations Asia and Far East Institute for the Prevention of Crime and Treatment of Offenders, based in Tokyo. As an organisation, it is something of a hidden gem, being relatively low profile and known mainly to more established academics who come across it in the course of their work. Its websiteFootnote 18 is unsophisticated, although it does support an advanced search facility that allows basic selection of target resources, Boolean searching, phrase searching, and wildcard searching with prefix, mid-word, and suffix matching. The richest resource featured, though, is the Resource Materials Series in the site's Publications section. These are an ongoing series of congresses and seminars, each on a particular subject, published three times a year. Number 102, for instance, is entitled Juvenile Justice and the United Nations Standards and Norms, and includes papers by invited experts from Finland, Italy, South Africa, Thailand, and the United States, as well as participants’ papers from Brazil, Kenya, and the Maldives, outcome reports from the seminar itself, and supplemental materials. Other congresses in the series, all publishing material on the same model, cover a wide range of criminological specialisms and subjects, with titles including The State of Cybercrime; Assessment and Treatment of Special Needs Offenders; Public Participation in Community Corrections; Community Involvement in Offender Treatment; Measures to Combat Economic Crime; and Effective Administration of the Police, among a long list of others.
The Resource Material Series is one of the selectable options for the site's advanced search facility though, as the results cannot be refined and their ranking criteria remain mysterious, it is probably more productive to browse the series onscreen. With the impressive coverage of subjects and geographical comparisons available, especially for countries where academic criminological research is rarely available, this resource repays careful exploration.
