10.4.1 Design Paradigms to Examine the Legitimacy of Privacy Rules-in-Use

As discussed in the previous section, the alignment of PBD with privacy engineering could make the former an important enactor of privacy-preserving capabilities of a sociotechnical system. Wong and Mulligan (Reference Wong and MulliganWong and Mulligan 2019) outline the important place PBD has come to occupy in the privacy policy sphere, owing to its inclusion in the EU’s General Data Protection Regulation, the United States Federal Trade Commission’s privacy policy recommendations, and other privacy advisory and regulatory institutions. They argue that PBD is currently, largely, dominated by engineering approaches that assume that privacy is pre-defined and exogenous to the design process, whereas HCI has a rich collection of design methodologies and tools that are capable of identifying relevant conceptualizations of privacy and related values within the design process. Such approaches, they further argue, are largely absent from policy-making and practice of PBD. Furthermore, even within HCI, they find that most PBD approaches use design and associated principles “to solve a privacy problem” or “to support or inform privacy decision making”, and that “design to explore people and situations and to critique, speculate, or present critical alternatives” – design approaches available from the field of HCI – are largely absent from both the policy-making and the practice dimensions of PBD. They argue that the latter are particularly pertinent when the “conception of privacy that ought to guide design is unknown or contested” (Reference Wong and MulliganWong and Mulligan 2019). This resonates with the GKC framework:

Both approaches emphasize the contested nature of privacy and the procedural aspects of exploring and uncovering these contestations. An important question that a synthesis of this shared emphasis raises is: what kinds of design paradigms in computer science, generally, but HCI and adjoining disciplines, specifically, provide a way for questions of governance and procedural legitimacy to enter into the design and implementation of technology that mediates or enables information flows? How can the tools and methodologies of design be employed to explore questions of governance and procedural legitimacy both to assess the appropriateness of entrenched norms or rules-in-use, and to handle previously unresolved, hidden, un-surfaced ethical disagreements?

Gurses and van Hoboken argue that contextual integrity while not tied down to concepts of time and location requires “looking back in time” to identify entrenched social norms that govern the “appropriate” information flows, in order to enable an informed and reflective design of novel socio-technical systems. Utilizing such a lens on norms, and considering the GKC framework, what can the tools and methodologies of design reveal about the procedural legitimacy of entrenched privacy norms and values?

One way forward toward exploring this question further is contained in the approaches outlined by Reference Wong and MulliganWong and Mulligan (2019), who map out the purposes for which design is employed in relation to privacy in the existing HCI literature. On examining 64 scholarly publications in HCI venues that use design in relation to privacy, they find that 56 percent use design “to solve a privacy problem,” where “privacy is a problem that has already been well-defined outside of the design process,” and 52 percent use design “to inform and support decision-making,” which foregrounds the individualized framing of privacy by focusing on providing information to users to enable them to make privacy-preserving decisions, or on the creation of tools and processes so that designers can incorporate privacy more easily in their practice. Only 22 percent used design “to explore people and situations” where design and other methodologies are used to explore what conceptualizations of privacy in varying social and cultural contexts are “at play” – an approach that has “implications for design”. Finally, only 11 percent use design to “to critique, speculate or present critical alternatives,” where questions such as “what should be considered as privacy?,” “privacy for whom?,” and “how does privacy emerge from technical, social, and legal entanglements” are considered. The latter two orientations are particularly well suited to the surfacing of privacy conceptualizations in relation to surrounding social, cultural, and political factors, yet are under-explored in the literature. These design approaches have the potential to provide tools to bring procedural legitimacy “into play in assessing whether the rules-in-use for personal information are normatively appropriate” (Reference Sanfilippo and StrandburgSanfilippo, Frischmann, and Strandburg 2018). Furthermore, these approaches directly relate to the three distinct ways identified by Sanfilippo et al. in which procedural legitimacy is in play the GKC framework: first, whether the procedures that construct the rules-in-use are deemed to be legitimate by diverse community members, and aid them in achieving their objectives; second, whether the governance practices account for the interests and needs of “impacted outsiders”; and third, whether the “exogenous rules and norms” to which a community is subject are responsive to member needs and interests.

In particular, three design methodologies are well positioned to explore these orientations: (a) speculative design, where design is undertaken to present critical alternatives (Reference Wong, Khovanskaya, Filimowicz and TzankovaWong and Khovanskaya 2018; Reference AugerAuger 2013; Reference DiSalvo, Jenkins and LodatoDiSalvo, Jenkins, and Lodato 2016); (b) value centered design, where design is used to achieve certain human values (Reference FriedmanFriedman 1997; Reference ShiltonShilton 2018); and (c) participatory design (Reference MullerMuller 2009), where design is undertaken not only for, but by impacted stakeholders.

In this section, I outline one possible direction that directly opens up points of engagement between privacy as governance of privacy rules and speculative design methodologies. Reference DiSalvo, Jenkins and LodatoDiSalvo et al. (2016) use speculative design in the context of “civic tech” as “a way to explore potential, alternative, and future conditions by articulating their existence in generative forms, with a particular focus on the complications of governance and politics disposed by computational technologies.” The tools of speculative design can speak directly to aspects of governance that the commons approach focuses on.

To summarize, design paradigms in HCI provide potent tools to explore questions of procedural legitimacy of rules-in-use in the commons governance framework. In addition to achieving, what Reference Wong and MulliganWong and Mulligan (2019) consider important, namely, broadening the notion of design in PBD, these orientations could build important bridges between the PBD framework and the GKC framework.

10.4.2 Formulation and Governance of Privacy Norms via Participatory Design

In this subsection, I explore the framework of Participatory Design (PD) in detail to consider the opportunities it presents for democratic governance of privacy norms. PD as a design methodology has historically had clear political commitments to democratic ideals. Reference PilemalmPilemalm (2018) notes that PD developed in the late 60s and early 70s (as cooperative design) with the intention of involving citizens in urban areas in Scandinavia in the planning and design of their living environments. Soon, PD entered workplaces in Scandinavia with the intention of making workplaces more democratic, and empowering workers to participate in and influence their working conditions and workplace technology through the use of collaborative design processes between the workers and the designers (Reference Bjerknes, Ehn, Kyng and NygaardBjerknes et al. 1987; Reference EhnEhn 1988; Reference Simonsen and RobertsonSimonsen and Robertson 2012). Often, this occurred by assisting workplace unions in devising technological “control activities and policies” (Asaro 2000). Subsequent “generations” of PD, particularly its variants in the United Kingdom and North America were more focused on involving users and other stakeholders in the process of design of technologies to create better systems, an adoption that largely found resonance in HCI (Reference MullerMuller 2009). Several studies since then have argued to actively re-introduce the political and ideological dimensions of PD, highlighting the importance of democracy as a core political ideal to PD (Reference BeckBeck 2002; Reference KanstrupKanstrup 2003).

Regan’s argument (Reference ReganRegan 1986; Reference Regan, Roessler and Mokrosinska2015) that privacy is both a collective and a democratic value lends credence to the idea of using democratic processes to determine which norms or rules regarding privacy should be in use, how they should be governed, how the appropriateness of specific privacy rules should be evaluated, and by whom. As Sanfilippo et al. articulate:

Scholarly and political commitments to democratic decision-making in the governance of privacy takes us down the route of exploring connections to PD, and its democratic and political ideals, in particular. Some preliminary attempts in this direction are due to Reference Mir, Shvartzshnaider and LatoneroMir et al. (2018) and Reference Shilton, Burke, Estrin, Hansen and SrivastavaShilton et al. (2008). Yet, at the time of writing this chapter, there is almost no work on operationalizing PD to conceptualize privacy. There is much important work to be done in this direction, such as determining which privacy rules-in-use in specific contexts are normatively appropriate, what the characteristics of the community are that determine these rules-in-use, how communities and other stakeholders, particularly dynamic ones, can negotiate around conflicting values such as privacy. In this section, I examine the affordances of PD to speak to such concerns.

While PD processes have largely been absent both in the shaping of privacy policy and in exploring contested aspects of privacy, privacy scholarship can learn and adapt from the vast body of literature that does envision using participatory, democratic processes in shaping and determining aspects of public policy. Such adaptations are especially pertinent in cases where technology (including potentially privacy-invasive technology) is employed within contexts that are democratic by their very nature, such as several decision-making processes employed by states, cities, municipalities, and public services, a context that is often dubbed as “civic tech.” In such contexts, participants’ relationship to the technology in question is more appropriately framed as that of a citizen rather than a consumer. For example, Reference PilemalmPilemalm (2018) studies the role of PD in public sector contexts, including civic engagement and “we-government” initiatives. He presents case studies showing that after addressing the challenges and practical difficulties of involving civil citizens, PD can be employed in the design of technologies in the public sector and lead to empowerment of citizens involved by both including them in designing the products that impact them and enhancing their understanding and skills.

In particular, the democratic framing of PD harkening back to its historical roots had led several PD researchers and practitioners to view PD as a process that interrogates issues of power and politics with the ultimate aim of enhancing democratic ideals, mutual learning and empowerment of the participants (Reference EhnEhn 1988). While PD flourished as a practice and value-based design system (Reference ShiltonShilton 2018) in the context of unionized workers in the Scandinavian workplace, the changing nature of work organizations and the adoption of PD outside Scandinavia led to the adoption of PD beyond the workplace. In particular Reference Teli, Lyle and SciannambloTeli et al. ( 2018) remark that the adoption of PD in the early 2000s extended beyond the “renewed workplace” – workplaces they term as arising out of “transformations in the mode of production toward post-Fordism” – to domains considered to be constituting the “public realm” (Reference Huybrechts, Benesch and GeibHuybrechts, Benesch, and Geib 2017). This expression continues in what Reference DiSalvo, Clement and PipekDiSalvo et al. (2012) call community-based PD, where the participants are not workers, but rather citizens interested in community-related issues, and the context involves negotiations among multiple parties with heterogeneous, and often conflicting values (Reference Grönvall, Malmborg and MesseterGrönvall, Malmborg, and Messeter 2016). As Grönvall and coauthors remark, in such settings:

Grönvall et al. present three case studies to demonstrate how design interventions enable the participants to become aware of other participant’s attitudes toward the collaboration at hand as well as their values. The case studies illustrate how even as PD as a process can enable a consensus and an understanding, the dynamic nature of the participant population leads to a continuously changing landscape of values as each participant brings in their own roles, stances, and values into these collaborations. They remark that:

Reference Lodato and CarlLodato and DiSalvo (2018) consider PD in the context of institutions operating in the public realm, examining the constraints produced through employing PD in working with or through these institutions – what they call “institutional constraints,” and are ultimately interested in understanding such institutions through the lens of PD.

PD, when employed in the so-called public realm, raises questions about who the participants are, who is considered to be part of the community, how those boundaries are drawn, and who is left out of the “participation.” For example, Lodato and DiSalvo claim that:

Since PD aims to enhance democratic decision-making, mutual learning between designers and participants, and empowerment of participants, Reference Bossen, Dindler and IversenBossen et al. (2016) consider the question of evaluating whether PD processes indeed achieve these goals. They present a framework to systematically evaluate PD projects for these goals paying attention to the purpose of the evaluation, who conducts and leads the evaluation, who participates, the methods used, and the audience for the evaluation. These criteria help understand questions of participation, legitimacy, and empowerment in PD.

There is some literature on the commonalities between Commons Design and Participatory Design; here I briefly review that literature to explore ideas pertinent to the design of a privacy commons. Reference Marttila, Botero and Saad-SulonenMarttila et al. (2014) examine the connections between the literature on commons (for example, using Ostrom’s framework (Reference OstromOstrom 1990)) and PD, with the aim of developing design strategies and approaches to designing the commons. They argue that both PD and the commons literatures “build upon stakeholders and communities’ capabilities and right to act and decide upon their future.” They point out how while Ostrom’s “design principles”(Reference OstromOstrom 1990) for long-enduring commons were not intended to provide a framework to design a commons, nevertheless, they can be integrated in the PD process “to develop a nuanced understanding of design agency and its interplay with multiple mechanisms of collective action” (Reference Marttila, Botero and Saad-SulonenMarttila, Botero, and Saad-Sulonen 2014).

Such orientations are also available (and arguably, direly needed) for the conceptualizations and implementations of privacy. However, such engagements open up questions about efficiency of processes, and scalability of solutions, two framings that technologists are particularly attuned to.

In his book titled the “Smart Enough City” (Reference GreenGreen 2019), Ben Green presents an example that instead works with an alternative concept: “meaningful inefficiencies” that he borrows from civic media scholars (Reference Gordon and WalterGordon and Walter 2016). Green cites work by Gordon and coauthors (Reference Gordon and Baldwin-PhilippiGordon and Baldwin-Philippi 2014) to create Community PlanIt (CPI),Footnote ¹ an online, multiplayer game to promote engagement, deliberation, and decision-making within communities. The game is focused not on making the process of deliberation and engagement efficient, but rather to recognize that these are necessarily inefficient processes, and to design such platforms for “meaningful inefficiencies” that highlight aspects of community member engagement, coordination, and reflection:

Could community engagement platforms that are designed to enhance civic engagement and are embedded appropriately in the civic, social, and cultural contexts of communities, such as Community PlanIt, be deployed to develop models of participatory governance of information norms? This question is inextricably linked to the larger goals of PD – that of enhancing democratic ideals, mutual learning and empowerment of the participants. The next section will delve into some of the literature on “civic learning” and reflective decision-making that enables participants to negotiate around and make collective decisions about issues impacting them.

10.4.3 Privacy Literacy, Civic Leaning, and Participatory Governance

Questions of participation in mechanisms of governance lead to underlying questions about people’s understanding of the information flow landscape, their perception of their roles in it, and what kinds of coordination and deliberation mechanisms enable people to engage meaningfully in such participatory frameworks. In relation to the GKC framework, “adequate” privacy literacy may be viewed as “attributes of the community members” (Reference Strandburg, Frischmann, Madison, Strandburg, Frischmann and MadisonStrandburg, Frischmann, and Madison 2017). Community members can effectively govern the privacy commons only when they understand the underlying information flows and consequences of appropriate and inappropriate flows.

An important question that such considerations raise is: What kinds of (pedagogical) tools can be used to enhance people’s understanding of the data ecosystem and its implications? As Regan outlines, “the goal here would be to make visible the privacy implications which to date have effectively remained invisible to those affected” (Reference ReganP. Regan 2016). Here, Reference KumarKumar (2018) offers some preliminary research directions by outlining the possibility of using CI as an educational tool. This stems from an earlier study Kumar conducted with her co-authors (Reference Kumar, Naik, Devkar, Chetty, Clegg and VitakKumar et al. 2017), where CI was used as an analytical tool to understand how children used digital devices and how they both understood and navigated privacy concerns online. The study provided evidence that children (especially over ten) largely understand how the parameters of CI affect norms of information flow, and in particular, they had an understanding of actors and attributes, even as they don’t use the same terminology. Based on this, Kumar suggests exploring CI as a tool for privacy education (Reference KumarKumar 2018). In related studies, Reference Martin and NissenbaumMartin and Nissenbaum (2015) use survey-based methods to show that people typically understand the parameters of an informational norm, and frame their privacy expectations in view of the context in which the information flow occurs, as well as how the information is transmitted and used, and who the senders and receivers of this information are (Reference MartinMartin 2012).

While Kumar is largely interested in privacy literacy for children, with the objective of equipping children to make better decisions about their privacy, a larger additional question worth examining would be to understand whether and how CI can be used as an educational tool to equip adults (and, potentially, children) to better understand information flows within a larger governance context.

Much work in the privacy literacy space has focused on the understanding and empowerment of individual actors with respect to their privacy – another place where individualistic, subject-centered notions of privacy have gained traction. As Park notes:

However, as Reference CohenCohen (2019) argues, to consider effective responses to the erosion of privacy, scholarship and practice needs to shift from “subject-centered” to “condition-centered” frameworks. In this vein, literacy can also be broadly conceptualized as the building of capacity for an individual to act in a deliberative democratic system, a direction that remains under-explored in studies of privacy literacy. Reference Gordon and Baldwin-PhilippiGordon and Baldwin-Phillipi (2014) call this “civic learning”. They present two case studies, in which the online game Community PlanIt (CPI) was deployed in a community to enhance civic-engagement with support from local community organizations. One was part of a district wide planning process in the Boston Public Schools and the second as part of a master planning process in Detroit, Michigan. On assessing the impact of CPI in both case studies, they concluded that the gaming platform allowed what they term as “civic learning” to occur. This has important implications for privacy governance and privacy literacy: what kinds of tools and systems can help build individuals’ capacities as engaged, informed, and empowered citizens in the governance of privacy rules?

10.4.4 Empirical Studies of Privacy Norms and Their Relation to Individuals’ Expectations and Preferences

A focus on procedural legitimacy of informational norms raises another related important question: how can community members’ expectations and preferences of privacy be used to assess the legitimacy of contextual informational norms?

This calls for ways of empirically studying such expectations and preferences, not merely at an individual level, but at a group level. In prior work (Reference Shvartzshnaider, Tong, Wies, Kift, Nissenbaum, Subramanian and MittalShvartzshnaider et al. 2016) survey-based methods were used to measure users’ expectations and preferences of privacy to determine whether or not specific information flows are appropriate. However, as Benthall at al. outline:

Since the GKC approach seeks to further direct attention from the individual, an important research direction is to explore how individuals’ understanding, expectations, and preferences regarding privacy change in a group setting, and how such changes reflect on the larger governance procedures, particularly when these processes are democratic and participatory in nature?

In her articulation of privacy as a Common Good (P. M. Reference ReganRegan 2002; Reference Regan, Roessler and Mokrosinska2015), Regan raises an important and nuanced point to differentiate between “groups” and “individuals in a group” as a unit of analysis. She also poses the question of probing how individuals in groups differ from individuals acting individually in regards to privacy, highlighting that focusing on individuals who act and are aware of their actions and experiences as members of a group rather than merely as individuals acting in isolated capacities will aid our understanding of privacy behaviors and consequent “privacy actions and inactions.” A consequent key problem Regan identifies is to create avenues to help individuals realize that they are not merely individuals but members of a group both being impacted by the actions of others in the privacy dimension and affecting other people’s privacy. This has close connections to the idea of civic learning explored in the previous section. She recommends drawing on the work of sociologists, social psychologists, and communication scholars who study individual behavior in groups. This line of investigation is also open and available to computer science researchers, particularly those in HCI.

10.4.5 Calibrating Norm Evaluation and Enforcement Engines for Dynamic Sources of Norms

Technical systems that implement CI usually express informational norms in formal systems, and operationalize these norms on information flows that act on specific data exchange between actors in a particular context. Such systems typically rely on norm evaluation and enforcement engines that check whether the information flows are consistent with the supplied norms (Reference Barth, Datta, Mitchell and NissenbaumBarth et al. 2006; Reference Chowdhury, Gampe, Niu, von Ronne, Bennatt, Datta, Jia and WinsboroughChowdhury et al. 2013). An important research consideration that the governance perspective raises is related to the design and architecture of CI norm evaluation and enforcement engines (along with accompanying human–computer interfaces) that are more suited for dynamic deliberative sources of these norms rather than static sources such as laws and policies, as has been the case in prior work (Reference Barth, Datta, Mitchell and NissenbaumBarth et al. 2006).

Reference Shvartzshanider, Balashankar, Wies and SubramanianShvartzshanider et al. (2018) provide important directions here – they use natural language processing techniques such as dependency parsing to automatically extract the parameters of CI from individual sentences. Their approach extracts the CI norm parameters based on the syntactic structure of a single sentence, and uses an accompanying reading comprehension model to incorporate a semantic understanding of the larger scope in order to incorporate it into the CI parameters. They apply their techniques on a corpus that contains website privacy policies in natural text alongside annotations by law students. By supplementing this process with crowdsourcing, they demonstrate that information flows can be automatically extracted from natural text and can be made more precise by appropriate crowdsourcing techniques. While they use a corpus of website privacy policies for this purpose, an open direction is to use natural language processing to infer the parameters of privacy norms from privacy policies generated in a more participatory setting.

10.4.6 Normative Considerations in Differential Privacy

Contextual Integrity could provide a normative framework to embed technical notions such as differential privacy within it (Reference DworkDwork 2006). To the best of the author’s knowledge, there is no existing work that considers the appropriateness (or not) of releasing specific functions of a database from the perspective of CI. The GKC framework could further engage with these questions of appropriateness by considering aspects of governance of these rules of appropriateness.

Differential privacy (DP) is primarily suitable for settings where there is interest in releasing an aggregate function of a dataset consisting of data from individuals. This could include simple functions such as averages or more complex machine learning predictors. As Dwork and Roth state:

This is a more intuitive explanation of an underlying mathematical guarantee of plausible deniability, modulated by a privacy parameter, that has been called epsilon in the literature (Reference DworkDwork 2006; Reference Dwork and RothDwork and Roth 2013). For a detailed non-technical discussion of differential privacy consult Reference Wood, Altman, Bembenek, Bun, Gaboardi, Honaker, Nissim, O’Brien, Steinke and VadhanWood et al.’s (2018) primer.

Even though the DP guarantee targets individuals, functions that could be potentially publicly released or shared are computed over a dataset consisting of several individuals. Such a guarantee might, therefore, be meaningful to examine within the context of community governance and deliberation about sharing of data or functions of data more widely. For example, access to information that furthers understanding of medical ailments has a different normative valence than that of aggregation and prediction for commercial purposes such as online advertising and applications that might intentionally or unintentionally enact discrimination. Communities are likely to evaluate the appropriateness of sharing aggregate functions for these two purposes in different ways. For example, many polls indicate that the public views sharing of personal health data with researchers to be different from sharing such data with other more commercializing applications, indicating the need for context-specific attention to such details. On surveying personally controlled health records (PCHRs) users, Weitzman et al. found that 91 percent were willing to share medical information for health research with such willingness “conditioned by anonymity, research use, engagement with a trusted intermediary, transparency around PCHR access and use, and payment” (Reference Weitzman, Kaci and MandlWeitzman, Kaci, and Mandl 2010). In survey-based research conducted at the Pew Center, Reference Madden and RainieMadden and Rainie (2015) found that only 76 percent of respondents say they are “not too confident” or “not at all confident” that data on their online activity held by the online advertisers who place ads on the websites they visit will remain private and secure.

If sharing data at an aggregate level for, say, medical research purposes is deemed to be appropriate, DP can be employed within a governance framework to achieve the guarantee of plausible deniability for individual community members, and to consider questions about what are appropriate aggregate functions that should be shared with people outside the community. By paying attention to the larger normative elements of the use, purpose, and politics of aggregation, DP can be a powerful and effective tool to disrupt what Cohen terms “semantic continuity” (Reference CohenCohen 2019).

Several other research directions open up when we consider embedding DP within the larger normative elements of the commons framework: what kinds of interfaces will enable citizens (without a deep mathematical background) to understand the larger guarantees of DP, and make good governance decisions? Reference Bullek, Garboski, Mir and PeckBullek et al.’s (2017) preliminary work on making the core guarantees of DP understandable and accessible to the larger public provides one step in this direction. Further research that examines groups as units of analysis, rather than only individuals, along with considering contextual dimensions of the settings in which communities might want to share aggregate data, is needed here.