TERMINOLOGY

Broadly speaking two types of events are recorded by the police: offences and incidents. Offences are in two categories: those that are notifiable to the Home Office and include all offences triable by jury or either-way, and those less serious offences that are dealt with exclusively by the Magistrates' Courts. Offences are categorised as a crime type according to National Crime Recording Standards (NCRS) and Home Office Counting Rules (HOCR). The Notifiable Offences List (NOL)Footnote ⁵ which forms part of the HOCR is revised from time to time to reflect changing legislation and forms the basis for the compilation of the ONS statistics on police-recorded crime. Some crimes, such as domestic abuse, are not statutory offences and are recorded according to the appropriate notifiable offences category (e.g. assault with injury) and are marked in the police database using a flag. Incidents are recorded according to the National Standards for Incident Recording (NSIR) and associated counting rules. Incidents are all manner of events reported to the police where the public has cause for concern (e.g. traffic accidents, missing persons, anti-social behaviour). Only three categories of incident are notifiable to the Home Office and these deal with reports of rape which, for specific reasons, are not crimed. There are two practical points to note: there are many more incidents recorded than offences and that a full picture of victimisation needs to include data on both offences and non-crime incidents. Because there are these two categories of events – offences and incidents – and because PRC refers generally to offences only, the more general term of police-recorded data is used here.

ADVANTAGES OF POLICE-RECORDED DATA

Police databases record the facts and evidence pertinent to an offence or incident. This includes details of the location where the event occurred, victim(s), informants and witnesses, perpetrator(s) (either as suspects or accused) and the modus operandi (MO). Because there can, for example, be more than one victim and/or perpetrator to an offence, the database has many-to-many relations. This adds complexity in organising the data for analysis, but also adds richness and extraordinary granularity given that each event is a separate entry. A researcher can expect to be working with tens to hundreds of thousands of records. In general, the objective of research using police-recorded data is not to focus on specific individuals per se but to identify aggregate patterns that provide new understanding and suggest interventions or solutions to problems. Two different documented examples of this type of approach are: a) an analysis of the upsurge in SatNav theft from the beginning of 2006 given in Brimicombe (Reference Brimicombe2012), and b) an analysis of offender histories of those arrested in the London riots of 2011 given in Stanko and Dawson (Reference Stanko and Dawson2012).

Police-recorded data allows research into all crime and incident types rather than being restricted to individual and household victims as is the case with CSEW or restricted to the NOL. For a start, a face-to-face survey can't include experience of homicide and there are practical and ethical issues of including questions such as on child abuse. Then there are the so-called victimless crimes that are classed as crimes against the state (e.g. possession of drugs, perjury). In the NOL there are currently 1069 categories of crime against the state versus 275 categories of victim-based crimeFootnote ⁶. But as already observed there are many more incidents than offences, and the categories of offences dealt with by Magistrates' Courts are not included in the NOL. The CSEW, important as it is, only covers a minority of offence categories whereas police-recorded data covers the totality of what is reported to or detected by the police.

The granularity of police-recorded data is the highest available. Events are recorded with the time and date at which they are reported to the police, and where the actual occurrence of an event is uncertain (e.g. a burglary while away on holiday) a beginning and ending time and date are also recorded. Events reported retrospectively (such as child abuse reported later in life) are similarly handled. This uncertainty in the actual time and date of occurrence for some crime types can be offset to some extent by aggregation, such as by the day or week, using the mid-point of the date range or it can be handled probabilistically (Ratcliffe, Reference Ratcliffe2002). Police-recorded data also has a high level of geographical granularity in that many events are recorded against an address, either where the event happened (e.g. a burglary), to the nearest place where it happened (e.g. a violent altercation outside a pub) or, say, at a road junction. This forms the basis of ‘hotspot’ mapping allowing police to focus their resources on those locations that have highest intensities of crime (Harries, Reference Harries1999; Brimicombe, Reference Brimicombe2012). The data on victims and perpetrators can be further filtered by gender, age, ethnicity and offence/incident category in order to study specific patterns of activity. The volume of data records and the number of variables make these data particularly amenable to predictive analytics and techniques of machine learning (see for example Berk, Reference Berk2013).

Notwithstanding some of the difficulties of working with police-recorded data discussed in the next section, it is feasible to identify and analyse patterns of repeat victimisation and repeat offending (prolific offenders). Whilst it is technically quite difficult to reliably identify and profile repeats (see Brimicombe, Reference Brimicombeforthcoming), these are at the heart of informed crime reduction (Pease & Tseloni, Reference Pease and Tseloni2014), yet for example, one third of police forces were found to have no data on repeat victims of domestic abuse (HMIC, 2014a). Whilst CSEW includes data on repeat victimisation (though obviously not repeat offenders), the number of victimisations is capped for statistical reasons at five events in the past twelve months (ONS, 2015). Police-recorded data holds out the possibility of longer victimisation chronologies (not just the last twelve months) and the full number of repeat occurrences in chronic cases.

Finally, of growing importance is data linkage. This is where individuals are matched across data systems in order to look at patterns of multiple service use. An example would be victims who report violent crimes to the police and report to accident and emergency (A&E). There may be occasions when violence is reported without attendance at A&E and other occasions when A&E is attended as a result of an assault but not reported to the police. Linking the two up can reveal informative patterns of activity. Some government initiated programmes such as Troubled Families Phase 2Footnote ⁷, require local authorities to link social services, housing, education, benefits, mental health and police data in order to identify families classed as ‘troubled’ and thus qualifying for interventions on a payment by results model.

ISSUES IN USING POLICE-RECORDED DATA

Having discussed the advantages of using police-recorded data for a range of analyses that can inform policy, strategy and operations, the use of such data is not without its problems.

Historically, the 43 police forces in England and Wales have independently developed their IT systems and designed their own database schemas for recording events reported to them. There are at least 88 data centres (PASC, 2011) and some 2,000 IT systems (CPA, 2012). There can be separate databases for call and dispatch (999 calls), incidents and offences, details of victims and details of perpetrators/accused and so on. There may not be unique keys that connect these databases because of the many-to-many relationships that occur in crime events and keys meant to achieve greater integration may not be assiduously copied across due to time and effort. As already mentioned, some offences are not statutory crimes (such as domestic abuse) and may be identified by a flag (or several different flags) in police databases, marked in a separate register, or be mirrored in a separate database specific for that type of offence. This all adds complexity when trying to retrieve all the relevant data for analysis.

The personal details present in police-recorded data means that they are subject to the provisions of the 1998 Data Protection Act and disclosure must be prevented. Breaches reported to the Information Commissioner's Office (ICO), such as loss or theft of sensitive personal data will attract a financial penalty of up to £500,000Footnote ⁸. Not surprisingly high security IT systems operating within a well-founded information security framework are required for the storage, analysis and eventual archiving or destruction of such data in order to prevent breaches. Individuals with access to the data also usually need to be vetted by the relevant police force(s) and may be asked to sign the Official Secrets Act. These conditions are not so severe for anonymised or aggregated police data, but then granularity and analytical depth are lost.

The public's trust in crime statistics is fundamental in a transparent democracy but had declined in the late 1990s and 2000s due in large part to political ‘spin’. Despite subsequent work to repair the trust (UKSA, 2010), the quality and reliability of PRC has come in for heavy criticism at Parliamentary Committee (PASC, 2014) and the UK Statistics Authority subsequently withdrew its National Statistics designation (UKSA, 2014). At the same time a divergence was noted between the amount of crime as measured through CSEW and that given for comparable police-recorded crime (ONS, 2013). Whilst the Audit Commission had carried out regular checks of police data quality from 2003/04 following the introduction of the NCRS, they were discontinued after 2006/07. By 2011, quality concerns lead Her Majesty's Inspectorate of Constabulary (HMIC) to carry out a series of reviews of police crime and incident reports in England and Wales. As might be expected, the inspections have resulted in changes in police-recorded crime which can introduce marked discontinuities in the data series. By way of illustration, HMIC criticised the lack of accuracy in recording sexual offences (HMIC, 2014b); Figure 1 shows for London how monthly counts (indexed to 100 at the start of the data series) for all victim-based crimes and sexual offences which roughly track each other for five years and then markedly diverge with a steep increase in recorded sexual offences once HMIC begins their review. The 80% increase in recorded sexual offences between April 2013 and mid-2014 is more likely to be the result of better recording practices than an actual increase in the amount of sexual offences in London. The National Statistics designation of quality is unlikely to be restored until changes in PRC consistently reflects real changes in the amount of crime.

Figure 1

Marked divergence in indexed monthly counts of all victim-based crime and sexual offences from April 2013 coterminous with HMIC inspections which led to criticisms of the accuracy in recording sexual offences (data available from: http://data.london.gov.uk/dataset/metropolitan-police-service-recorded-crime-figures-and-associated-data).

Whilst much of the quality debate has focused on adherence to the NCRS and HOCR in determining if an event is an offence or incident and the correct classification of offences by crime type, quality problems also concern the accuracy, consistency and completeness with which fields in the databases are populated with data. Whilst it needs to be recognised that there is no such thing as the perfectly correct database, the recording of names, addresses and other particulars of events, often in difficult, tense situations, are subject to inadvertent errors, gaps and lack of consistency. Victims do not always give accurate responses. Extensive data cleaning is required to maximise the analytical use of the data (Brimicombe et al., Reference Brimicombe, Brimicombe and Li2007). Furthermore, it is well understood from CSEW that 20–60% of crime, depending on crime type, is not reported to the police and that police-recorded data are always an undercount of the true level of activity. Whilst CSEW is accepted as giving a consistently reliable trend of personal and household victimisation at a national scale, and notwithstanding the issues of data quality discussed above, police-recorded data are the only means of understanding sub-national activity down to police command unit and Community Safety Partnership (CSP) level – and even down to neighbourhood geographies - as a means to compare the performance of command units and CSP and what works in preventing crime.

EXAMPLES OF ANALYSING POLICE-RECORDED DATA

Presented in this section are a few examples of the types of analyses that can be carried out from police-recorded data. This represents only a fraction of what can be extracted from this type of transactions database. The data for these examples are for domestic violence offences over a five and a half year period from 2007 to mid-2012 (and thus predates the methodological difficulty introduced by the HMIC inspections as illustrated in Figure 1) for an entire police force (county level) with a mixture of urban and rural areas.

Underlying most approaches to analyses that inform problem-solving or evidence-based approached to crime reduction is the understanding that crimes tend to form patterns (Brantingham & Brantingham, Reference Brantingham and Brantingham1984). Such patterns are the product of processes (actions) which, once the drivers or risk factors are understood, can be modified or stopped through an appropriate set of interventions (legal, social, or situational) that disrupts the patterns of crime. The patterning of crime occurs in one or more key dimensions: spatially (location), temporally, the victims, the offenders and their modus operandi. The patterns of most interest are those that cause clustering (as in a crime hotspot) or a degree of regularity and thus have a degree of predictability. The results of analyses that cause surprise are more likely to change thinking about crime and its prevention.

Figure 2 plots the daily occurrence for domestic violence for the case study area. This appears to have chaotic oscillations from day to day. Two sets of smoothing have been applied to aid interpretation: one is a 28-day Gaussian smoothing and the other is a best fit linear trend line. Apart from the fact that police-recorded domestic violence is rising over the period and that on no single day is there no record of domestic violence, the most eye-catching elements of this graph are the spikes that occur once a year, every year – this is new year's day, signalling for a significant number of households the end of the season of peace and goodwill. Some of the other spikes may have a discernable cause, but some are difficult to determine. For example Brimicombe and Cafe (Reference Brimicombe and Cafe2012) were able to conclusively attribute spikes in the national rate of domestic violence when the England team either lost or won (but not drew) their matches in the 2010 FIFA World Cup.

Figure 2

Daily counts of domestic violence with 28-day Gaussian smoothing and trend line 2007 to mid-2012.

The apparent chaotic oscillations in Figure 2 become clearer when aggregated into weekly patterns whereby it becomes evident that that counts of domestic violence are much higher at the weekend. Figure 3 illustrates this effect by plotting percentage change on the previous day in a spider diagram showing the whole week in a cyclical arrangement. By way of contrast Figure 3(a) is for events suffered by female victims who have only reported to the police once, while Figure 3(b) is for female repeat victims. Whereas in Figure 3(a) reporting of domestic violence starts to increase on Fridays, increases by a further 35% on Saturdays, increases a further 10% on Sundays to then decline dramatically on a Monday, the weekend effect in Figure 3(b) is less pronounced and is indeed statistically significantly different. This would suggest that the process of, or events leading up to victimisation is subtly different between the two and merits more detailed analysis of the MO.

Figure 3

Comparison of aggregated weekly cycle of domestic violence (percentage change on previous day) for females having reported only one event to the police and for female repeat victims.

A geographical view of the data is given in Figure 4. This is by victim home address (not necessarily the same as the location at which the crime was perpetrated as it may have been carried out at the perpetrator's address if the perpetrator and victim are not currently co-habiting; domestic violence also takes place in public locations such as shopping centres and pubs), aggregated to lower super output areas (LSOA)Footnote ⁹. For each full year of data (2007–11), the count of domestic violence victims by LSOA has been tested for statistical significanceFootnote ¹⁰ and if the count of victims is above the 95% confidence interval (i.e. this level of activity is unlikely to have arisen by chance), then the LSOA has been deemed to be a ‘hotspot’. How often in each of the 5 years an LSOA is a statistical hotspot then becomes a measure of how persistent these hotspots are, with the distribution of these persistences given in Figure 4. The map shows the majority of LSOA are not statistical hotspots even though domestic violence is reported to the police everywhere. This is contrasted with those areas that are hotspots and these are geographically clustered with a number of areas that are hotspots every or nearly every year. These tend to be urban areas with multiple deprivation, have a high level of predictability regarding the number of domestic violence victims, and should thus be the focus of police and partnership intervention.

Figure 4

Persistence of statistically significant hotspots of domestic violence by victim home address.