Introduction
Innovation is regarded as critical to improving the performance of public services in the UK and internationally. Research on public sector innovation has contributed to the mapping of multiple potential sources of innovation (Cinar, Trott, & Simms, Reference Cinar, Trott and Simms2019, Reference Cinar, Trott and Simms2021; Clausen, Demircioglu, & Alsos, Reference Clausen, Demircioglu and Alsos2020; De Vries, Bekkers, & Tummers, Reference De Vries, Bekkers and Tummers2016). However, there is less consensus about where managerial attention should be placed in the public sector to encourage innovation. This paper contributes to debate in the research literature concerning how to encourage public sector innovation; for example, should emphasis be placed on improving strategic management in the public sector, recognizing distributed or ‘bottom-up’ learning by staff members at the professional level, or should policymakers seek to stimulate innovation by introducing mandatory changes at the regulatory level? A systematic review of public sector innovation (De Vries, Bekkers, & Tummers, Reference De Vries, Bekkers and Tummers2016) provides a useful map for unpacking the broad concept of innovation by distinguishing between types of innovation (e.g., process, product, conceptual), and antecedent factors (barriers or enablers) that influence its introduction at different levels: environmental (e.g., external political mandates), organizational (e.g., slack resources), and individual employees (e.g., their empowerment). While the review highlights the multiple influences on innovation, it also points to a need for multi-level approaches, including studies that use participant observation methods, that can analyse processes across levels which are theorized to influence public sector innovation. It is important to address this gap to give a more complete account of innovation processes in public sector organizations (i.e., how innovation is shaped by interplay between processes at the professional, organizational, and environmental level). Public sector organizations need to respond to multiple stakeholder interests in the external environment signalling the importance of including this level and responses at the professional and organizational level to regulatory and industrial change (Burton & Rycroft-Malone, Reference Burton and Rycroft-Malone2014).
One limitation of De Vries, Bekkers, and Tummers' (Reference De Vries, Bekkers and Tummers2016) review is that it neglects processes at the professional group level, despite its relevance in theoretical frameworks utilized in their review, e.g. the social evaluation of innovations by professional groups (Rogers, Reference Rogers2003), and their relevance to studies of how professional groups shape innovation processes in other areas of the public sector, including healthcare (Ferlie, Fitzgerald, Wood, & Hawkins, Reference Ferlie, Fitzgerald, Wood and Hawkins2005; Turner, Allen, Bartlett, & Pérotin, Reference Turner, Allen, Bartlett and Pérotin2011), local government (Laffin & Entwistle, Reference Laffin and Entwistle2000; Snyder, Wenger, & de Sousa Briggs, Reference Snyder, Wenger and de Sousa Briggs2004), and broadcasting (Turner & Lourenço, Reference Turner and Lourenço2011). A multi-level approach to the study of innovation, that includes the professional group level, is critical because innovation is shaped by processes within, and interactions across, the internal and external context of organizations (Greenhalgh, Robert, & Macfarlane, Reference Greenhalgh, Robert, Macfarlane, Bate and Kyriakidou2004; Pettigrew, Ferlie, & McKee, Reference Pettigrew, Ferlie and McKee1992).
This paper addresses this gap concerning multi-level studies of innovation in the public sector by examining empirically the influence of processes at the environmental, organizational, and professional levels on the implementation of innovation by the privatized UK postal provider, Royal Mail. It addresses the following research question: how do professional and organizational processes, and interactions between the two, influence learning and innovation by a public sector organization in the context of environmental change?
Definition of levels
Professional
Professions are occupations that can vary along several dimensions: member or peer control, specialized knowledge and skill, training and education, internal organization, and their relations with other occupations and stakeholders (Freidson, Reference Freidson1989). Studying processes at this level means examining how professions’ internal dynamics and external relationships influence practices like learning and innovation within an organizational setting.
Organizational
Organizations are purposeful, coordinated, and durable associations among individuals, that is, ‘stable associations of persons engaged in concerted activities directed to the attainment of specific objectives’ (Bittner, Reference Bittner1965: 239). Organizational-level processes refer to the components used by organizations (with a varying degree of formality) to implement their strategy or purpose, which include task definition, coordination structures, cultural values, and people and processes.
Professional and organizational-level processes interact because the definition of professions includes their external relationships (which might be a corporate setting), while components of organizations like human resource management (including authority structures and working conditions) will influence professional practices within organizations.
Environmental
Environmental change, such as changes in industrial regulation, is likely to have a knock-on effect at the organizational level of public sector organizations (e.g., a shift in strategy to respond to regulatory change) which, in turn, is likely to affect processes at the professional level as staff members respond to new requirements set at the organizational level. The presence of, and interactions between, all three levels matter in public sector organizations. For example, the corporatization of public organizations in health care (an environmental change) highlights how the organizational level has been affected by changes in governance (i.e., the managerial consequences of legal independence), while the professional level has been affected by increasing managerialism or control of medical work (Turner & Wright, Reference Turner and Wright2022).
Use of concepts to study professional and organizational-level processes
Professional level: communities of practice
At the professional group level, we explore the role of social interactions in creating new knowledge using communities of practice (CoPs) theory (Lave & Wenger, Reference Lave and Wenger1991). CoPs is a situated or micro-level theory which emphasizes the role of repeated interaction within social groups as a source of learning. Ongoing interactions within groups produce social and material resources which fuel the group's collective learning and competencies. For Wenger (Reference Wenger1998), learning within CoPs has three dimensions: joint enterprise (alignment of interests and internal accountability), shared repertoire (local resources for learning produced or co-opted by the group), and mutual engagement (learning through participation in shared activities).
A classic example of CoPs operating in an organizational context comes from Orr's (Reference Orr1996) ethnography of roaming photocopier service technicians. Orr found that the community of Xerox technicians he observed in their daily work would regularly find that the ‘institutional knowledge’ they had been given – the formal training and documentation – was inadequate for the majority of photocopier problems that they faced. Interactions characteristic of CoPs supported the negotiation of their daily working activities. When technicians were not engaged on service visits, they would spend time together, engage in meetings, talk about work, and exchange experiences (mutual engagement). Technicians’ shared expertise informally by telling ‘war stores’; communal resources were also traded, including otherwise unobtainable photocopier components (shared repertoire). Competence was defined in relation to the community as technicians’ reputations were derived from getting photocopiers back online no matter how unusual the problem faced (joint enterprise).
Pointing to these types of informal interaction within social groups, CoPs theory argues that learning and innovation within organizations can be traced to the social practices of durable communities, irrespective of whether these coincide with formal business units or teams (Wenger, Reference Wenger1998). In this paper, CoPs theory is used to examine the social practices through which professional groups negotiate their everyday working activities and engage in learning processes within a large public sector organization.
However, CoPs theory needs further critical development. As a micro-level theory of learning, CoPs has been criticized for neglecting processes at the wider organizational level in which CoPs are situated (Lave, Reference Lave, Amin and Roberts2008). The ways in which strategic management influences the activities of CoPs within organizations need to be acknowledged (Roberts, Reference Roberts2006). This could include examining: the influence of the formal hierarchy, including task design, differentiation, and integration, on the formation of CoPs that develop around those structures; disruption of existing CoPs’ interactions through organizational restructuring, downsizing, or merger; managerial recognition or sponsorship of CoPs which may influence their internal dynamics and access to external organizational resources; and the ways in which wider professional and organizational cultures interact with, inform, or conflict with the internal social norms of CoPs developed through repeated interaction. To take account of one aspect of the organizational context, the concept of absorptive capacity – which refers to an organization's innovation ability – is used in this paper to complement the professional-level focus of CoPs theory.
Organizational level: absorptive capacity
The concept of absorptive capacity is employed to capture organizational factors that influence an organization's ability to innovate. Innovation is often linked to an organization's absorptive capacity which is the ‘ability to recognize the value of new information, assimilate it, and apply it to commercial ends’ (Cohen & Levinthal, Reference Cohen and Levinthal1990: 128). This organizational-level concept focusses attention on how organizations acquire, communicate, and apply new information to support learning and innovation. Drawing on cognitive theories of individual learning which treat knowledge as a ‘possession’ (e.g., Simon, Reference Simon and Kuhn1985), the concept of absorptive capacity suggests that organizational learning depends on the entity possessing prior related knowledge which enables new information to be recognized and applied. The role of cognitive mechanisms, i.e. possession of prior related knowledge and R&D investment, in acquiring knowledge has received the most attention from management scholars. While the related experience needed to acquire knowledge is held by individual staff members, exploiting this knowledge to pursue innovation at the organizational level depends on the organization's structure of communication.
Through ‘process’-based conceptions of absorptive capacity (Easterby-Smith, Graca, Antonacopoulou, & Ferdinand, Reference Easterby-Smith, Graca, Antonacopoulou and Ferdinand2008), interest has grown in understanding the role of social processes in sharing and exploiting knowledge within organizations (Lane, Koka, & Pathak, Reference Lane, Koka and Pathak2006; Todorova & Durisin, Reference Todorova and Durisin2007; Zahra & George, Reference Zahra and George2002). To aid knowledge sharing, Cohen and Levinthal (Reference Cohen and Levinthal1990) referred to the role of a shared language and symbols, and close links between R&D and other functions, enabled by interventions such as employee rotation and cross-functional teams. Process perspectives represent renewed interest in unpacking Cohen and Levinthal's references to ‘communication systems’ in facilitating organizational knowledge use. For instance, drawing a distinction between ‘potential’ and ‘realized’ absorptive capacity, Zahra and George (Reference Zahra and George2002) claim that ‘social integration mechanisms’ allow firms to make commercial use of knowledge acquired in different parts of the firm by ‘facilitating the sharing and eventual exploitation of knowledge’ (p.194). Todorova and Durisin (Reference Todorova and Durisin2007) go further in arguing that particular social arrangements are appropriate for acquiring specific types of knowledge, e.g., use of formal integration mechanisms for managing complex knowledge within the firm, while weaker ties can be used to access novel information beyond the firm's boundaries. In the absorptive capacity literature, further research is needed on the ways in which antecedent organizational factors, e.g. strategy, human resource management, staff turnover, and training programmes (Volberda, Foss, & Lyles, Reference Volberda, Foss and Lyles2010), influence the development of these social processes in particular organizations and, in turn, their ability to acquire and share knowledge.
The concept of absorptive capacity, with its focus on organizational antecedents that influence the use of knowledge for innovation, can complement CoPs perspectives that focus on internal interactions within professional communities, by pointing to wider organizational mechanisms that influence the acquisition, sharing, and exploitation of knowledge for innovation. Recognising that processes of learning within CoPs may be influenced by their organizational context, combining the professional-level concept of CoPs with the organizational-level concept of absorptive capacity promises to offer a more complete account of innovation processes within organizations. This paper analyses how CoPs' embedding in specific organizational contexts may influence their development and knowledge processes. The construct of absorptive capacity is of potential value in this regard in highlighting the influence of organizational factors on learning processes (Volberda, Foss, & Lyles, Reference Volberda, Foss and Lyles2010).
One critique of the absorptive capacity literature, including work advocating a ‘process’ perspective, is its focus on the transfer of existing knowledge, rather than practices of knowledge creation. The role of social processes is limited to supporting the sharing or transfer of existing knowledge. As Marabelli and Newell (Reference Marabelli and Newell2014) argue, much of the literature on absorptive capacity that considers social processes still conceives of knowledge as an entity or ‘possession’ that can be shared progressively between different units (e.g., between the individual, group, and organizational level) if appropriate social mechanisms to enable the transfer of knowledge exist. For instance, while Easterby-Smith et al. (Reference Easterby-Smith, Graca, Antonacopoulou and Ferdinand2008) advocate a ‘process’-based approach, knowledge is still treated as an ‘entity’ that may be transferred in supportive organizational contexts, e.g. through leadership and other managerial initiatives to support knowledge sharing. The concept that social practice is not just a vehicle for sharing existing knowledge, but may contribute to the formation of new knowledge, has received less attention. Consequently, the ways in which social interactions might enable the creation of new knowledge, and not only its transfer, have been neglected in theories of absorptive capacity. With its emphasis on practices of learning through interaction, CoPs theory holds the potential to address this limitation within the absorptive capacity literature by providing a concept of how organizations construct new knowledge through social practices (and not merely acquire and transfer existing knowledge using organizational learning mechanisms, as absorptive capacity suggests).
This paper brings together the professional-level construct of CoPs with the organizational-level construct of absorptive capacity to (a) offer a more complete account of multi-level innovation processes within public sector organizations and (b) examine the potential for each theory to address limitations of the other (i.e., absorptive capacity's neglect of practices of knowledge creation, and CoPs’ neglect of the organizational context of interaction). Finally, both theories have been criticized for neglecting the role of power in shaping learning and innovation practices. CoPs approaches have tended to neglect the role of power in social interaction, despite Lave and Wenger's (Reference Lave and Wenger1991) early acknowledgement that ‘unequal relations of power must be included more systematically in our analysis’ (p. 42). This could stem from a ‘managerialist agenda’ that reads CoPs as consensual groups eager to learn and contribute to corporate objectives (Contu & Willmott, Reference Contu and Willmott2003). Lacking an adequate theorization of power, the early literature on CoPs neglected the ‘forceful’ processes through which change and innovation are often negotiated within and between communities (Fox, Reference Fox2000), including potential resistance across professional boundaries (Ferlie et al., Reference Ferlie, Fitzgerald, Wood and Hawkins2005). Power has also been neglected in constructs of absorptive capacity, prompting a call for organizational sources of power, e.g. access to internal resources, to be given further consideration in influencing the exploitation of new knowledge (Todorova & Durisin, Reference Todorova and Durisin2007). This paper considers how power – defined as the ability to influence others’ behaviour in organizations (Pfeffer, Reference Pfeffer1992) – both internal and external to an organization's CoPs shapes processes of learning and innovation.
Using the concepts of CoPs and absorptive capacity, and acknowledging potential limitations of both concepts, the relevance of multi-level processes to the study of public sector innovation is analysed empirically through vignettes from an ethnography of two innovation-seeking projects led within Royal Mail's marketing department.
Materials and methods
Research setting
A 9-month ethnographic study of learning and innovation was conducted within the marketing department of Royal Mail, that was contracting with external organizations as a strategy for innovation in the mid-2000s (Turner, Reference Turner2006). Royal Mail provides a universal postal service to the UK's 30 million addresses, operates in 44 countries internationally, employs 162,000 staff, and has an annual income of £10.5 billion (Royal Mail plc, 2019). In 2001, Royal Mail was transformed into a public limited company (still owned by the government) and postal competition was introduced progressively as licences were granted to new providers. The company responded to competition by restructuring (involving the voluntary redundancy of 30,000 staff) and investing in new mail-related capabilities to sustain client loyalty, including data and media services. In response to competition, the marketing department was central to developing new capabilities predominantly through contracting with external organizations. In 2013, the Conservative Liberal Democrat coalition government oversaw the privatization of Royal Mail and its initial public offering on the London stock exchange.
The findings presented here describe examples of the marketing department's response to the introduction of competition to the UK postal market in the mid-2000s, which preceded privatization. During the period of the ethnography, Royal Mail was subjected to external pressures linked to commercialization that influenced the organization in several ways. First, introducing a formal strategy for promoting an ‘innovative culture’ across the organization (Whitehead, Reference Whitehead1999), which included improving knowledge management, establishing an innovation fund to support new technology projects (Radzin, Reference Radzin2001), and introducing a dislocated innovation laboratory (Lewis & Moultrie, Reference Lewis and Moultrie2005). Second, the organization was positioned as a ‘media owner’ in the advertising industry. A new open-plan office space in London's West End was opened to act as the organization's ‘shop window’ in the advertising industry (this was ‘Media House’, where the ethnography was conducted). To increase the credibility of Royal Mail in the advertising industry, reliance was placed on accessing external sources of knowledge. Sales consultants with expertise in direct marketing were recruited to work at Media House and partnership arrangements were established with leading advertising agencies to allow Royal Mail to offer a range of direct mail-related marketing tools to clients. Third, Royal Mail's structure was affected by an organization-wide restructuring exercise that was designed to reduce costs as part of a 3-year ‘renewal plan’.
The ethnography of Royal Mail began in January 2004 coinciding with the second year of the renewal plan, when the company was in the midst of the largest restructuring exercise in its history, and on the brink of the liberalization of the organization's postal markets. The findings reported in this paper are based on the study of two types of potential public sector innovation (De Vries, Bekkers, & Tummers, Reference De Vries, Bekkers and Tummers2016): firstly, a process innovation is represented by the restructuring of Royal Mail to extend the core competence in mail collection and delivery to include data and media services through external contracting; and, secondly, product and service innovations that issued from the changes in organizational process, including digital projects ‘Wed DM’ and ‘Help Online’ (described below).
Data collection
The data were collected using participant observation, interviews, and documentary analysis. Using ethnographic methods resonates with a recent call from a systematic review of literature on public sector innovation to understand the relative impact of different organizational characteristics and antecedents of innovation (De Vries, Bekkers, & Tummers, Reference De Vries, Bekkers and Tummers2016). Participant observation involved securing a working role with different project teams within the marketing department, allowing the author to follow the teams' everyday activities, transcribe fly-on-the-wall discussions, and participate in team meetings, creative sessions, and other formal and informal contexts for learning. Such an approach afforded access to ‘a variety of unguarded behaviour’ (Latour & Woolgar, Reference Latour and Woolgar1986: 153).
The perceptions of project work gathered through participant observation are described through vignettes of two contrasting projects, both of which were undertaken in conjunction with external creative agencies. The first project was the development of an advertising service that enabled small organizations to create direct-mail campaigns online. A creative agency was involved to provide technical expertise, including knowledge of small and medium sized enterprises and website development skills. In order to become immersed in the typical role of a marketer, I asked the product manager to treat me as if I were a new graduate starting work at the company.
The second project involved the redesign of Royal Mail's website to reduce unnecessary telephone and email enquiries to customer services. A creative agency was employed to provide specialist knowledge, including expertise in creative design and project management support. As this project also involved an external organization, the social practices of this team could be compared and contrasted with those examined through the first project. In relation to both projects, the use of participant observation proved effective, as I was able to attend project meetings, undertake pieces of project work, collect organizational and project-related documents, and use impressions gathered through observation to inform interviews with employees from Royal Mail and related organizations.
Interviews were conducted with internal and external members of both project teams (some were interviewed twice) and middle and senior managers from Royal Mail. The impact of the wider managerial and organizational context in which project work took place was assessed through the managerial interviews, as these actors were asked to discuss formal organizational processes for encouraging innovation, notably partnering strategy and R&D activity. All interviews were audio-recorded. A summary of data collected during the ethnographic study is provided in Table 1.
Summary of research conducted during the ethnographic study
Data analysis
The analysis was conducted in two stages, first, vignettes of the two projects were developed using a ‘narrative strategy’ in which contextual details of the projects were arranged into stories of events and activities (Langley, Reference Langley1999). The inclusion of contrasting projects, one deemed more successful and one less, allowed comparisons to be drawn across the cases thereby facilitating analysis of the processes that influenced the dynamics of interactions between internal and external members of both project teams. Second, the projects were subjected to thematic analysis (Fereday & Muir-Cochrane, Reference Fereday and Muir-Cochrane2006) which focused on identifying the key processes that shaped each project's development and outcomes. The analysis was both inductive and deductive, as coding was led by the data but also informed by insights from previous literature on absorptive capacity and CoPs. Informed by these concepts and the empirical data, the initial codes were organized into three overarching themes that were pertinent to the primary data and conceptual concerns (social processes, organizational context, and power relations) which were used to evaluate learning and innovation associated with the projects studied. The analytic questions associated with each theme, and divided by concept, are presented in Table 2. The discussion teases out implications for further theoretical development of the concepts applied to the empirical case, future research, and managerial practice.
Themes emerging from thematic analysis
Results
Key social practices associated with the navigation of the two innovation-seeking projects are set out in vignettes. The first describes the introduction and management of a new product, ‘Web DM’, that aimed to support customers with producing direct marketing campaigns online (Figure 1). The second describes early interactions across a multi-disciplinary project, ‘Help Online’, that aimed to improve the customer service aspect of Royal Mail's website (Figure 2). Following the vignettes, learning processes associated with the two projects are analysed along three dimensions: social interaction, organizational context, and power relations.
Vignette of the development of a new advertising product.
Vignette of embedding a new project management process.
Interpretation: influences on project-based learning
In this section, three prominent themes from the vignettes which appeared to influence learning in both projects are described: patterns of social interaction, organizational context, and power relations.
Social interaction
The nature of social interaction in the two projects differed. In the Web DM project, tasks were allocated to either Royal Mail or the agency meaning that project work could be managed largely at a distance using Information and Communications Technology (ICTs), while in the Help Online project tasks were often shared and accomplished through collaboration between Royal Mail and the agency. In the Web DM project, there was a lack of joint work involving Royal Mail and the agency. The only time when the group did meet face-to-face was for monthly ‘status or confirmation meetings’ as the product manager described. These meetings tended to be organized around specific activities, e.g. coordinating market research or advertising campaigns. In the meeting that was described in the vignette, spreadsheet charts depicting customer visits to the website were presented by the agency but the ensuing discussion only related trends in website usage to the impact of advertising campaigns or website design changes. The team did not debate the product's design or discuss differing interpretations of the reasons for the product's low sales volume. Instead, both parties acted in accordance with a rigid division of roles. Royal Mail did not get involved in website development by the agency (‘the agency need to come up with a time-plan and costs’), while the agency would not be drawn on Royal Mail's methods for achieving sales revenue (‘we are now waiting for the marketing’).
The Help Online project involved more intensive interaction between Royal Mail and the agency. Employees from the agency were regularly at Media House, either working in the touchdown area or engaged in formal or informal project meetings. Two desks at Create's offices were also made available to the digital team. The team themselves, led by Chris, defined explicitly how they would work together on projects, and with other business functions, by developing a ‘new process’ for delivering digital projects:
Previously, the way of delivering projects was to work through Technology, whereby they provided the technical elements and we provided the commercial input, you know, we gave them a project brief. We needed to find a new process because there wasn't a process in place for delivering projects in the digital space where many different parties needed to be involved.
The new process, which was formalized in a structure for cross-functional working (Figure 3), and predetermined stages for managing each project (Figure 4), was developed through collaboration between the project team and the agency.
The hub structure for delivering digital projects.
Document tree process for digital projects.
Awareness of the project management process, which was an integral part of the agency's way of working, grew among members of the team from Royal Mail through interaction with Create employees during marketing projects in the year before the department's restructuring. The team engaged in extended debates about influencing the trajectory of digital projects and facilitated project-related activities, e.g. the brainstorming workshop and reviews of customer requirements. For a recent graduate in the digital team, Ben, observing the way in which a talented employee from the agency approached project work was inspiring:
I would watch him at a meeting and he was fantastic. He would facilitate everything; he had great ideas but he didn't always impose them. He always threw everything at people and got people to think and talk. That is something I am trying to learn from. I had to chair a meeting the other day for the first time and I tried to facilitate it in the same manner.
By participating in such activities, members of the digital team appeared to recognize skilful behaviour in others and attempted to comport with, or act ‘in the same manner’, as the agency's employees. In summary, while the Help Online project was characterized by intensive interaction between the contracting parties which appeared to support learning, there was a lack of joint work between Royal Mail and the creative agency in the Web DM project.
Influence of organizational context upon learning
The organizational context in which the projects took place shaped processes of learning related to each project team. To prepare for competition in postal services, Royal Mail initiated both an internal restructuring programme and developed new products and services by contracting with external organizations. The company's expectation was that innovation could be secured more readily through contracting relationships than by developing new capabilities in-house:
Often we want to work with these partners because they are extremely different to us, and if they can bring a technical capability, which we can secure to our satisfaction through a partnership agreement or an alliance or something, I see it as almost counter-productive [to develop capability in-house]. (Senior manager, marketing department)
Royal Mail's strategy of developing relationships with organizations that were ‘extremely different to us’ rather than invest in internal capabilities affected project-based learning in a number of ways. First, some marketers perceived that internal training and development was being overlooked. Web DM's project manager said that the lack of investment in developing skills internally had informed his decision to leave the company: ‘Training and development is totally invisible at the moment and this is one of the main reasons why I am leaving’.
By contrast, in the Help Online project, members of the digital team highlighted the importance of learning driven by themselves rather than the company. When discussing the new project management process, Chris indicated that senior management would have little knowledge or awareness of the changing way in which his team was managing cross-functional projects that involve external agencies: ‘The business is pretty relaxed about it; I mean it's not that savvy, it wouldn't have recognition of it’. The newest member of the team, Ben, singled out the dynamism of his own team relative to the rest of the department: ‘The team I work in, maybe this is slightly blowing our own trumpets or arrogant, but we all believe that we're quite a bit ahead of a lot of other people in Marketing’.
Second, the restructuring affected the management of products as a swathe of marketers either left Royal Mail or moved into new roles. Mark reflected on the changes in product management over the 2-year life of the Web DM project:
From a project management perspective, there's been several changes from Royal Mail's perspective and, actually, the agency, although one of the original representatives is still involved. So, the expertise and the background learning, while some was passed over, you never are going to get everything.
Due to restructuring, ‘expertise and background learning’ were lost as existing relationships among project team members broke down and incoming staff adjusted to new roles. In the Help Online project, by contrast, the digital team used the restructuring as an opportunity to draw others' attention to an emerging ‘digital space’. Prior to the restructuring, a functional team responsible for products and services within a digital domain did not exist. When a digital team was created, ambiguity existed with respect to the role and remit of the team. In the hands of Chris, the team's namesake became ‘any way we relate to our customers using mobile phones, website […] there's a whole range of stuff there’. So defined, Chris's team had a stake in a wide range of projects within Royal Mail.
Third, the contractual agreements between Royal Mail and partnering organizations encouraged particular types of interaction in each project team. In the Web DM project, observation of face-to-face meetings suggested that Royal Mail and the agency acted in accordance with a rigid division of roles. The nature of the contractual relationship between both parties meant that the fees the agency received did not vary with sales revenue generated by the product. The agency did not, therefore, share the same innovation imperative as Royal Mail when the product failed to achieve sales targets. In the Help Online project, the contractual agreement between Royal Mail and the agency differed. Although Create had traditionally been employed from project to project by Royal Mail, the shift to a ‘retainer’ contract following the restructuring facilitated a more durable relationship. Consequently, members of the digital team and the agency interacted on a daily basis through joint involvement in ongoing projects.
In summary, the patterns of social interaction observed in each project appeared to be shaped by formal contracting in the Web DM project (i.e., tasks were delegated to the agency), while stronger collaboration between the digital team and the agency in the Help Online project was enabled by a longer contract. A key aspect of the organizational context, the restructuring of the department, affected the projects in different ways; while it was used by the digital team to expand the scope of their activities, it appeared to have a negative impact on continuity in management of the Web DM project.
Power relations
Use of external agencies appeared to mediate how new information was both interpreted and applied in relation to the two projects. For instance, in relation to the Web DM project, any market research was coordinated by a separate market research team within the department, who drew up briefs for external marketing agencies who then conducted the research. The marketers commissioning the research did not interact with ‘the market’ directly but experienced it in codified form through research reports or, more commonly, as an audience member in the delivery of PowerPoint presentations by marketing agencies. Hence, marketers were not routinely involved in acquiring or interpreting feedback from the market themselves; what they appeared to learn was how to make sense of the market as depicted in the reports and presentations of the market research agencies. Rather than enable knowledge to be produced interactively, the employment of agencies appeared to encourage a relationship of dependency in which the agencies would ‘speak for the market’ and thereby mediate the acquisition of external knowledge.
The creative agencies also influenced how knowledge was applied and used. In the Help Online project, collaboration between the digital team and the creative agency helped to change the trajectory of the cross-functional project with the Technology function. As described in the second vignette, the beginning of the initial workshop indicates how the project may have been managed had the agency not been involved. Led by Technology, it was an opportunity to present the problem to all of the stakeholders, to secure funding for the project, and to begin to make website design changes. In Alex's view, the role of the marketing department was limited to acting as a ‘usability forum’ once the design changes had been made. The digital team held an alternative view about how the project should be managed. The key difference between their method and Technology's proposed way forward was the marketers' desire to proceed by responding to well defined ‘business requirements’. James undermined Alex's proposed methodology by venturing forth an alternative way of proceeding: one driven by Customer Services' requirements. At the second meeting between the two parties, Chris brought a diagram with him that depicted the document tree process for digital projects (Figure 4) and, with James' support, he described again how they envisaged the project being managed. Although the project manager could ‘see two different approaches’, he assigned authority to the digital team at this meeting: ‘Chris is master’.
Adoption of the digital team's project delivery method, a version of which had been used successfully by the creative agency for many years, was aided by the perception that the employees within agencies were ‘experts’ in their field, as Ben, the graduate stated: ‘The reason I see we employ agencies is because they are experts. Ideally Royal Mail would be full of those people but I think agencies headhunt the best and offer them very lucrative salaries’. The perception that Create was a successful expert organization was reinforced by the style of their offices: the soaring glass cylindrical building; the advertising campaigns of top-name brands – their work – displayed proudly on the walls; the leather seats and couches; and the pots of cafetière coffee and freshly made sandwich platters. When the group of attendees arrived at Create's offices for the second workshop they are likely to have been impressed by the new venue. During the lunch break of the workshop, one of the attendees from Customer Services was so impressed with the general ambience that she asked whether it would be possible for a relative of hers to get a work placement with the agency.
In summary, the roles of the creative agencies that worked with the project teams were not limited to providing external knowledge, as the agencies also influenced how knowledge was developed and applied more broadly in both projects, shaping the interpretation of new information in the Web DM project and, in the Help Online project, steering the project towards using the project management process that the agency helped to develop.
Discussion
The projects studied during the organizational ethnography aimed to produce innovation through cross-functional teams, including external design agencies. The study's findings raise questions about the managerial expectation that innovation could be secured through transactional or arm's length relationships, especially where Royal Mail saw a benefit in working with organizations that were ‘extremely different to us’. Implementation of this strategy was mediated by learning antecedents at the professional and organizational levels as well as interactions between those levels. This study contributes to a line of research which suggests the importance of the relationships or connections between antecedents in influencing public sector innovation (Cinar, Trott, & Simms, Reference Cinar, Trott and Simms2021; Clausen, Demircioglu, & Alsos, Reference Clausen, Demircioglu and Alsos2020; De Vries, Bekkers, & Tummers, Reference De Vries, Bekkers and Tummers2016). Innovation antecedents were taken into account at the organizational and professional level for each project using themes (social processes, organizational context, power relations) informed by CoPs and absorptive capacity theory (Table 3).
Summary of project-based observations and antecedents by dimension
Summary of findings by theme
Social interaction
At the professional level, activities characteristic of CoPs appeared to underpin learning through social interaction within the cross-functional teams studied. Akin to apprenticeship learning, the graduate on the Help Online team sought to emulate the ‘experts’ from the agency, suggesting that an intrinsic incentive could be striving for capable engagement in a practice. In Web DM, the team met to monitor, rather than collectively participate in project tasks (which had been delegated outside the core team). This appeared to encourage a ‘hollowed out’ (Storey, Reference Storey, Little, Quintas and Ray2002) project, rather than resembling a CoP which coalesces around learning. The variation in social interaction across the projects observed suggests that managerial expectations that absorptive capacity alone (i.e., possessing prior related knowledge) will allow the exploitation of external knowledge are likely to be blunted. This study suggests instead that absorptive capacity needs to be complemented with professional-level mechanisms that promote mutual engagement among project teams, with CoPs theory pointing to some dimensions (e.g., opportunities for building shared resources that support learning, as evident in the Help Online project) that influence the quality of social interactions from a learning perspective.
Organizational context
The emergence of activities characteristic of CoPs is also mediated by the organizational context in which they are situated. The departmental restructuring affected the two projects differently. One consequence of restructuring was to engender a perception among some Royal Mail staff that resources were being redistributed away from internal teams and towards external agents causing discontent. Deficiencies in the organizational context were apparent to the Web DM team. As the project manager stated, training and development had become ‘invisible’, he felt the ‘absence of any guidance’, causing ‘frustration’, which informed his decision to leave the company. By contrast, the digital team received a mandate to expand their activities as digital projects became a priority in the restructuring. The digital team appeared to identify more with their own team than the wider organizational environment in which they worked; the perceived deficiencies of a company undergoing restructuring in order to prepare for competition in postal services (‘it's not that savvy’) appeared to be a spur for the team's learning: ‘we're quite a bit ahead of a lot of other people in Marketing’. Thus, organizational antecedents for building absorptive capacity, for example, training and development, resources to support project activities, and staff turnover (Volberda, Foss, & Lyles, Reference Volberda, Foss and Lyles2010), also influenced how fertile the organizational context was for CoPs-like behaviour to emerge.
Power relations
Interactions associated with the two projects were not undertaken uncritically by the participants in service of learning; the shape that interactions took was influenced by politics and power. In the Wed DM project, market research knowledge was not merely shared through interaction between the marketing agency and Royal Mail's marketers; being involved in interpreting and sharing new information conferred power on marketing agencies as ‘mediators’ (Latour, Reference Latour2005) able to maintain a legitimate role in marketing projects and make the case for further client work. In the Help Online project, while the digital team participated in co-developing knowledge with the creative agency (i.e., the new project management process), the adjacent Technology function was resistant to the peripheral role they had been cast in the cross-functional project, favouring the alternative approach of the ‘implementation of quick wins’. These examples highlight that absorbing new knowledge (e.g., market research data or new project management techniques) does not just depend on possessing prior related knowledge, but the attitudes that different group participants have towards the knowledge under consideration which is likely, in turn, to be informed by the perceived implications its use would have for each project stakeholder's legitimacy.
Implications for research on public sector innovation
This study contributes to current research on public sector innovation by analysing, and examining the interplay between, antecedents of innovation at the professional and organizational levels. It does so by constructing a new conceptual relationship between CoPs theory (professional level) and the absorptive capacity literature (organizational level). First, insights from CoPs theory can complement absorptive capacity by specifying the social processes through which new knowledge is created. Rather than assume the possession of ‘prior’ knowledge, as found in absorptive capacity, CoPs theory attends to the micro practices through which such knowledge is developed (and can point to social barriers to the formation of new knowledge). Using evidence from UK public service organizations, Butler and Ferlie (Reference Butler and Ferlie2020) argue that utilising absorptive capacity starts with ‘co-creating’ rather than acquiring knowledge, as early proponents of absorptive capacity have claimed (Zahra & George, Reference Zahra and George2002). CoPs theory is well placed to examine ‘co-creation’ processes as communities develop common resources for learning (e.g., shared repertoire). For example, the Help Online project highlighted the critical role developing the new project management process (and associated tools) for promoting learning opportunities (e.g., graduate observing the agency's employee at meetings) and shared purpose among the team (e.g., encouraging use of the new process on cross-functional projects). Analysing CoPs within the UK National Health Service, Pyrko, Dörfler, and Eden (Reference Pyrko, Dörfler and Eden2017) develop the concept of ‘thinking together’ to signify how ‘knowledge is redeveloped rather than literally transferred from one person to another’ (p. 403). This resonates with Marabelli and Newell's (Reference Marabelli and Newell2014) critique of absorptive capacity based on the proposition that knowledge is created through social practice and not just shared or transferred. At the professional level, social practices of learning are an important antecedent to the development of absorptive capacity, making CoPs a complementary micro-level theory to organizational antecedents for analysing absorptive capacity.
Second, insights from absorptive capacity can complement CoPs theory by situating their knowledge and learning activities in a commercial context. CoPs are often represented as sites of interactive knowledge formation. For example, interaction through mutual engagement of CoP members could be seen as a dialogical practice, stimulating collective forms of ‘knowing’ within groups that go beyond the knowledge brought to the interaction by any one individual (Shotter, Reference Shotter1993). However, this study shows that interactions within CoPs are shaped by the organizational context in which they are situated. When introducing the term absorptive capacity, Cohen and Levinthal referred both to the acquisition of knowledge and to its use for meeting ‘commercial ends’. When placed in an organizational context, this suggests that the formation, knowledge practices, and durability of CoPs are also influenced by wider organizational goals. Where public sector organizations undergo commercialization, as was the case at Royal Mail, the dynamics of CoPs can be read as contingent upon the shifting commercial context and imperatives in which their activities are situated. A lack of absorptive capacity at the organizational level was shown to inhibit learning practices within CoPs (e.g., the ‘hollowed out’ structure of the Web DM team encouraged by the organization's strategy for innovation failed to encourage the co-creation of ideas and products among the group). This deficiency was composed of components that were both motivational (i.e., the group lacked shared incentives to improve the product linked to the form of contract) and knowledge-based (i.e., pronounced task differentiation within and beyond the organization inhibited learning through mutual engagement within the Web DM team). This study shows that the internal activities of CoPs need to be understood in relation to the wider organizational context in which they are situated, including learning imperatives and activities associated with the wider organization's absorptive capacity.
Third, it addresses the neglected role of power in learning and innovation processes from the perspective of CoPs and absorptive capacity. With regard to CoPs theory, Wenger remarked in a published interview that ‘learning from a social perspective entails the power to define competence’ (Farnsworth, Kleanthous, & Wenger-Trayner, Reference Farnsworth, Kleanthous and Wenger-Trayner2016: 13). In this study, organizational sources of power that influenced perceptions of competence were found to be influential within CoPs. Organizational-level restructuring signalled shifting forms of competence to staff members at the professional level (i.e., the valuing of external knowledge provided by partnering organizations). At Royal Mail, the strategy of working with external organizations to pursue innovations created a sense among some marketers that senior management felt the capabilities for generating innovation lay increasingly outside the organization, while giving legitimacy to external agents' skills and competences for steering innovation projects.
At the professional level, this shift was interpreted differently by members of the two project groups observed. While the Web DM team was exposed to the negative effects of the restructuring (e.g., an absence of training), members of the digital team used the organizational restructuring to legitimize their own activities in the digital domain, regarding the creative agency's involvement key to their own interests. In the Help Online study, the competences of the external creative agency (represented by James' skills) were positioned as key to implementing the new project management process, while the Technology function's role was marginalized. As well as perceived competences, resource and budget availability also influenced the adoption of the new project management process (the project manager proclaimed that ‘Chris is master’ because he was the internal ‘customer’, holding the project budget). The findings underline the importance of power in shaping social interactions concerning knowledge and innovation, contributing to CoPs theory by suggesting the need to consider how the external organizational context – shifting definitions of competences and access to external resources – influence the internal dynamics of CoPs. Equally, some CoPs appear to be able to generate shared resources that allow them to signal competencies in relation to, and thereby take advantage of, organizational-level change (this ability to organize internally to produce power through coordinated practice is similar to ‘co-optation’ of system-level change found among senior clinicians in the health care sector, who draw on pre-existing power related to professional status) (Waring & Currie, Reference Waring and Currie2009). Further research should examine the characteristics (pre-existing and emergent) that influence the ability of CoPs in different areas of the public sector to be insulated from, and change their status in relation to, organizational and environmental-level change.
With regard to organizational-level absorptive capacity, the study shows how politics and power are often entangled in the development and application of knowledge in organizations, meaning that attitudes of different groups towards new knowledge, as well as cognition or understanding as typically emphasized in absorptive capacity theory, can help to explain why some types of knowledge are absorbed more readily into routine practice than others. Royal Mail's organizational strategy of partnering with creative agencies signalled receptivity to external knowledge, but attitudes towards such knowledge differed at the professional level (in the Help Online project, the agency's role was embraced by the digital marketing team, but resistance was apparent among the technology function due to concern that the agency's approach undermined their own role in the project). Going beyond analysing the cognitive capacity of organizations to understand new knowledge, future studies of absorptive capacity should acknowledge political factors that influence receptivity within an organization to the emerging knowledge under consideration (e.g., potential resistance where the knowledge could signal shifts in professional expertise, status, and training, as found in studies of technological innovation in health care) (Turner et al., Reference Turner, Vasilakis, Utley, Foster, Kotecha and Fulop2018); Mørk, Hoholm, Ellingsen, & Edwin, Reference Mørk, Hoholm, Ellingsen, Edwin and Aanestad2010).
Implications for practice
Management implications emerge from recognition of social practice as an antecedent of absorptive capacity and the interplay between CoPs and the organizational and managerial context in which they are situated. This study suggests that investment in organizational-level absorptive capacity is critical to not disrupt, and indeed potentially strengthen, the contribution of CoPs to professional-level learning. Paying attention to the dynamics of internal CoPs, even when pursuing an innovation strategy involving external partners, is likely to improve the formation of new knowledge and sustained learning. Attending to the structure of learning within project teams is not only important for supporting the absorption of new knowledge, but also to regulate power and politics emergent in relations between contracting organizations that shape the production and use of knowledge. Managerial actions should centre on supporting social practices that underpin absorptive capacity by investing in the development of durable teams, promoting mutual engagement in project teams, and ensuring internal knowledge and skills are maintained to avoid cognitive dependency on external partners.
Limitations
A strength of this ethnographic study is the depth of analysis of the fine-grained social practices of two innovation-seeking projects within a large organization. A potential limitation is the wider applicability of the findings to other settings, including other departments within Royal Mail, or other organizational settings, due to the specificity of the actors involved, the inter-functional and inter-organizational relationships, and circumstances of innovation. However, the findings are relevant to other areas of the public sector that have been subjected to ‘business-like’ reforms, including corporatization and privatization, such as health and social care, broadcasting, public utilities, and local government services. In particular, the findings illustrate the interplay of, and potential tensions between, ‘internal’ and ‘external’ sources of learning, as once publicly managed organizations experience commercial pressures in relation to their processes, products, and services. Further research could compare ethnographic findings on learning and innovation processes across departments within commercialising organizations, and across organizational settings, to identify how CoPs are shaped by the organizational and policy context.
Acknowledgements
The research was funded by an Economic and Social Research Council (ESRC) Collaborative Awards in Science and Engineering (CASE) studentship, award number PTA-033-2002-00073.
