Introduction
In June 2010, at the annual conference in Brighton, BIALL made its first ever award of Supplier of the Year. At the time, the Association's focus was rightly on the company that had won the award and on communicating both to the membership and to the other suppliers why that company had won. There is another side to any award and that is the process which the people making the award have gone through in deciding who should win. That process is all the more complicated when an award is being made for the first time. In this article I will describe the creation of the BIALL Supplier of the Year award.
Reviewing the situation – the Code of Good Practice
The original idea that eventually led to the Supplier of the Year award goes back to when I became Chair of BIALL's Legal Information Group in February 2009. At that time, LIG had managed to persuade most of the main legal publishers to sign a document called the BIALL Code of Good Practice for Law Publishers. A lot of work had gone into devising this document to provide a commitment that publishers would make to provide a certain level of service to their customers and as something that BIALL members could refer to during a complaint, should they feel that the publisher concerned had not lived up to that commitment. The Code was reviewed regularly and changes discussed with, and approved by, representatives of the publishers at an annual meeting. It was an admirable effort to regulate BIALL's relations with the legal publishers and to inculcate standards across the sector.
The Code had been launched at the Harrogate BIALL conference in June 2005. By 2009, the people who had been involved in designing and implementing it had moved on from the Legal Information Group and the new membership of the committee had certain misgivings about it. Despite the prominent launch at Conference and the document's availability on the BIALL website, it was not clear how many BIALL members knew of the Code's existence. We also did not know what proportion of the account reps and sales staff working for those publishers that had signed up knew of the Code, or what effect a BIALL member mentioning it during a negotiation or complaint could have.
This seems like a fairly standard promotional problem for LIG, but it was complicated by the fact that most of the publishers who had signed up to the Code had also sent a letter to BIALL stipulating which parts of the Code they would and would not adhere to. So, even if a member knew of the Code and wanted to use it, they would have to know how the relevant company interpreted or applied it.
There was also the question of enforcement. As far as we could see, BIALL had little recourse if a company that had signed up to the Code decided to repudiate it. LIG had refused to let one legal publisher sign up, and the negotiation process soured relations with that company for some considerable time, greatly affecting LIG's ability to represent properly the interests of BIALL members. Any strong action against an offending publisher, while potentially achieving some short-term publicity, might have greater negative long-term effects for LIG and BIALL members.
Coming up with a plan – LIG goes to the pub
In April 2009 LIG took the unusual step of organising a meeting solely for discussion of the future of the Code of Good Practice. There seemed to be two possible approaches. The first was to try to develop some method of enforcing the principles of practice embodied by the Code. Some of the smaller publishers had signed up to the Code without appending any protocols, but also tended not to apply it in full, so we could have tried to make an example of one. However, that appeared to be a risky strategy for the reasons outlined above.
Alternatively, we could have gone back to the Code and developed something, preferably with input from one or more of the publishers, that they would have been willing to endorse without reservation. However, this would almost certainly have involved jettisoning principles that we felt were important and that deserved support from LIG.
The alternative to changing the content of the Code was to change the approach. It was useful to have a statement of how BIALL believed publishers should treat their customers. It seemed more productive to use the document as an encouragement than as an instruction, to try to promote good behaviour among publishers rather than to try to punish bad behaviour.
This approach meant that we could include anything that BIALL members wanted in the document, without having to seek the approval of the publishers who had signed up to the Code. It also meant that it could be made more general in application and therefore used by LIG to represent the interests of BIALL members to suppliers beyond the traditional legal publishers and database providers. This aspirational approach received unanimous support at the LIG meeting.
It was at this point that the concept of an award was mentioned. It was considered helpful for there to be some form of incentive for the suppliers to follow the principles outlined in the new document. An award made at the BIALL conference would be a way of reinforcing preferred behaviour and encouraging it among other suppliers. At the same time, the process of deciding which supplier should be given the award would be an opportunity to assess the opinion of BIALL members about their suppliers.
The Legal Information Group only hears from a small proportion of BIALL members and normally in circumstances where they have a specific complaint about an individual supplier. Asking the membership as a whole for their opinions, at a time of our choosing, would be likely to produce both more and more general views, which LIG could use to inform its relations with suppliers.
The basic plan agreed in April 2009 was to design a survey that would ask BIALL members to rate their suppliers according to how they felt that they applied the Principles of Good Practice. The suppliers could then be graded according to their scores, with the supplier receiving the highest overall score also receiving the award. LIG had in fact surveyed the BIALL membership in previous years, but as far as I know each survey had only been about one supplier, and had only asked for comments rather than numerical performance ratings.
Designing the survey – consultation, revision and persuasion
LIG was very conscious that a significant policy change had been worked out by just six people and even that small a group had disagreed about the best method to implement it. Nominations for awards to be made at the 2009 conference were already being requested by BIALL's Awards and Bursaries Committee, so it was clear that it was not possible to design and run a survey to fit in with that timetable. That provided a year before the next nominations process and the first three months of that was devoted to raising the idea with representatives of suppliers, the BIALL membership and BIALL Council.
Many issues were raised that influenced the design of the survey. Around twenty suppliers would be included, meaning that it was necessary to ensure that respondents would not have to take a long time rating individual suppliers. Related questions were how to refer to the companies involved so that respondents recognised them as their suppliers and how to order them within the survey. Also, should respondents be required to identify themselves, or would anonymous responses be allowed, increasing the number of responses but potentially allowing non-members to submit surveys and members to submit more than one.
Two possible designs were submitted to BIALL Council for approval in October 2009, and the one chosen was shown to supplier representatives in January 2010. The survey ran in the last two weeks of March 2010, allowing some time for LIG to go through the results before making a nomination for the prize in April. The final form of the survey is shown below.
BIALL Supplier Survey 2010
The suppliers assessed in this survey are those who have an established relationship with BIALL through communications with the Legal Information Group.
The areas of business listed on the next page are, broadly speaking, those described in BIALL's Principles of Good Practice for Suppliers of Legal Information, available on the LIG page of the BIALL website.
When we ask about advertising and marketing, we want to know whether suppliers are clear and accurate in their descriptions of their products and services and whether they communicate with you in the ways you want. Pricing includes negotiation and renewal increases as well as raw figures. Customer service includes all communications with existing customers, such as technical support, complaints handling and credit control. Products/services asks about the usefulness and reliability of the products and services supplied to you, ignoring cost.
Please indicate how your experience with each of your suppliers compares to your expectations for suppliers generally. All responses will be treated confidentially, with the names of individuals and organisations not being passed on to suppliers without the permission of the BIALL member(s) concerned.
1. Please rate the performance of the following suppliers in the indicated areas of business. Only answer for the suppliers whose products and services you use, leaving the remainder blank.
2. Who is your favourite supplier?
3. Why?
4. Do you have any issues that you would like the Legal Information Group to raise with suppliers? If so, provide details here, or email david.percik@lawsociety.org.uk.
5. If you would like to be entered into the prize draw for £50, please provide your name and email address. All responses to the survey will be treated confidentially. No names of individuals or organisations will be passed on to suppliers without the permission of the individual concerned.
Name:
Email:
Collating the results – lots of cutting and pasting
The survey was designed using SurveyMonkey, which BIALL had recently subscribed to. The columns were areas of performance distilled from the Principles of Good Practice, with an overall rating added. The companies could have been ordered in any way and even differently for each respondent, but in the end it was decided that alphabetical order was the most sensible when the intention was that few if any respondents would rate all suppliers, as it made it easier for respondents to locate those suppliers that they did want to rate.
It will be obvious that not all suppliers used by BIALL members were included. It was felt that, as the award was being made by BIALL, it was appropriate that only suppliers who were willing to provide channels of communication for BIALL and LIG would be eligible. Each of the spaces contained a drop-down menu, offering a choice of five options, ranging from excellent to very poor, with a neutral option in the middle. The survey's software converted these ratings into a point score from 1 to 5, with 1 being very poor and 5 being excellent. This created an average score across all respondents rating that supplier.
The questions that asked respondents to name their favourite supplier, and to give reasons, as well as for any issues that they wanted LIG to raise with the suppliers, provided LIG with feedback, both positive and negative, that could be given to the suppliers along with their scores. It also told LIG what issues were of particular concern to BIALL members at that time. Finally, respondents were asked to provide their name and contact details. This provided some measure of protection against misuse of the survey and also meant that LIG would be able to follow up any of the comments received if desired. Identification was optional, but encouraged by the offer of a £50 prize for a randomly-selected respondent. It was also felt that the prize would increase the likely response rate.
At any given time BIALL has roughly eight hundred members. The precise membership constantly changes as individuals join or leave and, as a result, it is difficult to ensure that the entire membership receives any given communication. During the survey period of two weeks a large variety of promotional methods were used, including the mailing list for the BIALL newsletter, the BIALL mailtalk forum and the BIALL blog. Promotional efforts were spread out across the two weeks, on different days and at different times of day, in part to avoid any member feeling bombarded by communications, but also to act as a reminder to members who had considered responding but had not yet done so.
In the end, 144 responses were received, a rate of about 18%, fairly high for a questionnaire, even one targeted to a fairly small group of potential respondents. The lowest number of respondents rating an individual supplier was six, the highest was 131. As respondents were asked only to rate the suppliers they used, these figures may give some indication to LIG of the relative market penetration of the suppliers. Some trends were identifiable from the data, the most obvious being that in general suppliers were rated significantly better on the quality of their products and services than on their pricing. LIG agreed that the detailed results for each supplier would be shown only to that supplier, so I am not at liberty to go into any detail here. There has been demand both from BIALL members and from suppliers for more detailed results, so LIG hopes to be able to publish the 2011 survey results more fully.
Making the award – the involvement of ABC
The survey was designed so that several different measures could be used to decide that a particular supplier deserved the award. This was a deliberate policy to allow LIG some flexibility in the nominations process. Most obviously, suppliers received a mean overall rating and also votes as favourite supplier. While some suppliers did better on one of those measures than the other, Wildy & Sons were the best on both. Their average rating in all categories was above 4, and nearly a third of the respondents who nominated a favourite supplier chose them. While LIG made a point of mentioning other significant pieces of data in the results when making a nomination, there was really only one possible recipient of the award.
The Awards and Bursaries Committee has been referred to already in this article and they have traditionally managed BIALL's awards. When the idea of the award was first developed, they were consulted in order to benefit from their expertise and also to make sure that the Supplier of the Year award fitted in with the rest of their programme for 2010. There was already an award given at the BIALL Conference to a supplier, the Customer Relations Initiative award. However, this award was not made at the 2009 conference and it had for some years been difficult to gain nominations for it. As a result, ABC were pleased to receive a proposal from LIG that the Customer Relations Initiative award be replaced with a new award and that LIG had devised a new way of obtaining nominations.
The two committees worked in tandem in the course of 2009, LIG consulting ABC about the design of the survey, and ABC consulting LIG about the design of the award. Then chair of ABC, Susan Scorey, attended a meeting of LIG in August 2009, at which the timetable for the survey and nomination process was finalised. Once the survey had been run, the data analysed and the nomination made, ABC proceeded with the build-up to the conference and the presentation of the award in the same way as with all the others. The process stands as a good example of collaboration between BIALL committees on a project overlapping between the two remits.
Finally, at the Brighton conference, more than a year after the first planning meeting, the members of the Legal Information Group and the Awards and Bursaries Committee sat in the audience and watched John Sinkins accept the first ever BIALL Supplier of the Year award on behalf of Wildy & Sons. I am sure that he was proud that he and his company had been voted the favourite supplier of BIALL members in 2010; the two committees could be just as proud of the work that they had put into developing the award. I hope that the survey and the award will continue to be an important part of BIALL's relations with suppliers of legal information for many years to come.
