THE POTENTIAL OF QUALITATIVE DATA ARCHIVING

Qualitative data archiving empowers scholars to store, access, and download electronic qualitative data of all types. These include, for example, interview tapes, text files (transcripts from interviews, focus groups, and oral histories; case notes; meeting minutes; research diaries), scans of newspaper articles, images of official documents, and photographic, audio, and video materials. Registered users of the contemplated repository will be able to find and access potentially useful qualitative data sets through searching metadata elements as well as content.Footnote ² User-access controls will regulate who is able to search and retrieve the data sets housed in the repository, with more sensitive materials being restricted. The “public face” of such a repository is likely to be a Web-based portal that makes visible portions of the metadata associated with the projects in the repository (within the bounds of confidentiality and human-subjects concerns).

Qualitative data archiving will have several crucial benefits. At present, if a qualitative scholar wishes to check the findings of another, they typically have to do research on the same topic. By making data available and increasing the transparency and visibility of research processes, qualitative data archiving can dramatically reduce the costs of assessing empirically based analysis through replication (Swan and Brown Reference Swan and Brown2008, 7). This is likely to be especially significant for multi-method studies (see e.g., Lieberman Reference Lieberman2005), where the unavailability of the data underpinning the qualitative portion of the research has complicated evaluation and replication, rendering the qualitative component the presumed weaker dimension of the research.

Second, qualitative data archiving will allow for the vertical integration of primary data, secondary analysis, and scholarly output.Footnote ³ This integration will generate transparency—researchers will be able to show more clearly how they arrived at their inferences and interpretations—with multiple positive effects on practice, pedagogy, and collaboration. In terms of practice, increased visibility will lead to increased rigor. Researchers who anticipate transparency will be motivated to carry out data collection and analysis in a systematic, replicable way. Having access to clear descriptions of how primary research and secondary data analysis were carried out will also allow scholars to learn from others' experiences and avoid reproducing mistakes—as well as to discuss, critique, and improve qualitative methods more readily. Data archiving will also provide valuable pedagogical and coordination tools. Scholars teaching qualitative methods courses can ask their students to identify and discuss a range of empirical research techniques employed in studies housed in such repositories, and practice the analytic techniques they are learning on “real” empirical data. Also, the publication of data will vastly increase the visibility of scholars working on particular topics. This greater visibility, in addition to the tools for collaborative research that such repositories can provide, will facilitate team research and the formation of epistemic communities around research areas and questions (Swan and Brown Reference Swan and Brown2008, 26).

Finally, archiving qualitative data will allow data to accumulate, and will facilitate comparative research. Archiving qualitative data reduces the risks of data loss that personal storage poses, and minimizes the transaction costs associated with data sharing. Usually, qualitative data are researcher-centric, gathered in connection with a specific inquiry, and used just once (Swan and Brown Reference Swan and Brown2008, 7).Footnote ⁴ Depositing data in a repository and making them available to other analysts will encourage efficiency by preempting the unnecessary repetition of research. Further, a repository will allow scholars to undertake research in the context of a much larger universe of available data. Hence, qualitative data archiving will help individual scholars make comparisons of many types—across space (countries, cities, etc.), across time, across policy areas, across groups, and so on—that could otherwise require additional research resources or assembling a research team (Corti Reference Corti2000a, 6.2).

Data archiving raises the interesting and seemingly disconcerting possibility that qualitative research could be undertaken without the scholar directly engaging with any of the cases from which the data under analysis were drawn. While this is entirely unexceptional for quantitative scholars (who routinely start their analysis with another scholar's data set), it is a departure for qualitative scholars for whom close engagement with cases is typical, and perhaps obligatory. We do not anticipate that the introduction of qualitative social science data archiving in the United States will lead to either an immediate or eventual disconnect from cases for qualitative researchers. Nevertheless, beginning to archive qualitative data will doubtless provoke a spirited, healthy (and perhaps overdue) debate on the potential value of and risks involved in secondary data analysis.

ADDRESSING THE CHALLENGES OF QUALITATIVE DATA ARCHIVING

Qualitative data sharing presents a range of challenges that differ both in kind and degree from those involved in quantitative data archiving. Qualitative data are more heterogeneous than are quantitative data, their collection and sharing raise more human-subjects concerns, and they pose different metadata and indexing challenges (Heaton Reference Heaton2004, 29; Parry and Mauthner Reference Parry and Mauthner2004, 144). In this section, we discuss some concerns regarding the effects that qualitative data archiving might have on data-collection practices, the “shareability” of certain types of information, and the evolution of disciplinary norms. We also highlight some possible solutions.

A first set of issues concerns data-collection practices. As we noted above, by making research procedures transparent, qualitative data archiving can encourage scholars to be more self-conscious and rigorous in how they undertake data collection. However, this transparency may also have the unintended consequence of encouraging researchers to engage in self-censorship, omitting data collected using a technique they fear will not be considered rigorous, or failing to record relevant observations that they worry may reflect badly upon themselves or a subject.Footnote ⁵ A related problem is that if “shareability” becomes an overriding virtue by which data are judged, scholars may be influenced to focus solely on contexts where they can collect data that can be easily deposited. Important information that could have been collected and used with discretion will go unsolicited. The possibility of unintended collateral outcomes such as these highlights the importance of a sustained disciplinary conversation about the merits and limitations of data sharing.

Another series of concerns relates to limits on sharing qualitative data. Much field research involves interaction with human subjects—interviews and focus groups, oral histories, and ethnographic studies. Institutional review boards (IRBs) are likely to require that researchers who wish to store and share via the repository information gained through these interactions solicit subjects' permission (Mauthner, Parry, and Backett-Milburn Reference Mauthner, Parry and Backett-Milburn1998, 743; Heaton Reference Heaton2004, 79).Footnote ⁶ Inevitably, some subjects will be unable or unwilling to have their identity revealed and/or information they provide made available to other scholars (Corti, Foster, and Thompson Reference Corti, Foster and Thompson1995, 3). Further, interviews are sometimes given off-the-record and for background purposes only, and subjects are sometimes promised anonymity (Heaton Reference Heaton2004, 81; Parry and Mauthner Reference Parry and Mauthner2004, 146). This conditional provision of data may also affect the form data take: unproblematic conversations may appear as audio and/or video recordings, whereas more difficult exchanges may appear as edited formal transcripts, redacted summaries, contemporaneous notes, or near contemporaneous recollections that may be less faithful renditions of scholar-respondent exchanges. In short, the conditions under which subjects offer information may require a resource-intensive process of contextualizing data (or the storage of partial or incomplete accounts and data sets), and can affect the type and form of data scholars collect. These outcomes can have significant consequences for analysis, interpretation, and inference.

Without minimizing these concerns, it is important to note that a range of measures can be taken to protect human subjects while sharing data collected from and about them. For example, washing qualitative data can help preserve anonymity, and information thereby removed can be explicitly noted and the impact of its deletion assessed. Differential user access can also help to address human-subjects concerns.Footnote ⁷ In sum, while protecting human subjects and conforming fully to standards set by IRBs may prevent some data from being shared, these concerns do not stand in the way of making a great deal of material available to a broad range of users.

Non-interactive forms of data collection, such as archival research, may also produce data that cannot be posted to qualitative data repositories, or that can only be posted with some time lag, or in attenuated form. For example, documents accessed in archives may have copyright restrictions that prevent them from being reproduced or made publicly available for secondary reproduction. Further, the legal frameworks guiding such restrictions differ by context, and may change over time (Heaton Reference Heaton2004, 83–85). While these challenges require specialized solutions, the sustained success of institutions such as the National Security Archive suggest that, with careful attention, they can be addressed.

In sum, while we recognize that not all data collected via qualitative and multi-method research can be deposited, a considerable amount likely can. Sharing a significant subset of the qualitative data that social science researchers collect and produce is vastly preferable to the status quo, in which practically no qualitative data are publicly available in the United States.

Finally, as anticipated above, disciplinary norms will need to change if sharing and reusing qualitative data—and producing scholarship relying (wholly or partly) on secondary data analysis—are to become accepted practices. There are several ways in which disciplinary norms can be influenced. For instance, the discipline's professional organization (APSA) could clarify the guidance it offers on data release and sharing for qualitative research.Footnote ⁸ Likewise, funding agencies could advise grantees to make qualitative data available,Footnote ⁹ and more journals could require authors to do so.Footnote ¹⁰ Training opportunities, outreach, and guidance on qualitative data sharing (see Corti Reference Corti2000a, 6.3) could also be offered through IQMR or the short courses that are taught at the annual APSA meetings. In addition, professional associations and academic departments could consider revisiting career-based incentives for scholars such that, for instance, the publication of a qualitative data set receives due credit.

With respect to persuading qualitative researchers to deposit and use shared data, the most effective solution is likely establishing a dedicated qualitative repository, rather than building on a pre-existing quantitative archive. A specialized repository will be more attuned to the particular challenges posed by archiving qualitative data and will have a larger demonstration effect. A dedicated repository can also be epistemically neutral. While a considerable amount of qualitative and multi-method research is situated in epistemes that value replication, this is by no means a universal value. A repository viewed broadly as a means of increasing the transparency of the evidentiary basis for interpretive, descriptive, or inferential arguments based on qualitative data will be better positioned to attract all types of qualitative researchers. Finally, while a dedicated repository will focus on developing expertise in storing, searching, and sharing qualitative data, it will not be a “silo.” Rather, it will be designed to be visible to, and open to communication and interaction with, a wide audience, as well as with existing archives.

EXPERIENCES IN QUALITATIVE DATA SHARING

Efforts to increase the use of digital platforms to facilitate sharing qualitative data and encourage collaboration among researchers have had positive results elsewhere in the world. For instance, recognizing that qualitative data pose challenges that cannot be adequately addressed by archives designed to hold numerical data sets, and aiming to promote greater transparency and reduce duplication of research, funding agencies in many OECD countries initiated the push to close the infrastructure gap in qualitative research beginning in the 1990s. Several funding organizations adopted a mandatory sharing policy for grant holders,Footnote ¹¹ and the resulting repositories receive regular deposits on a national scale and hold a wide range of qualitative materials. While there is variation in the degree of development across these institutions, their archives play an important role in training and support for researchers and students throughout the process of data deposit and reanalysis.Footnote ¹²

The Qualitative Data Archival Resource Center (QUALIDATA), associated with the University of Essex, UK, is at the forefront of innovation. QUALIDATA was created in 1993, and is funded by the Economic and Social Science Research Council (ESRC)—the United Kingdom's largest sponsor of social science research. QUALIDATA was created to save “endangered” qualitative social research that was held, at the time, in the National Social Policy and Social Change Archive (NSPSCA). In the process of digitizing the NSPSCA collection and building its electronic infrastructure, QUALIDATA established procedures for cataloguing and extracting metadata, depositing data, ensuring respondent confidentiality and abiding by agreements made at the time data were collected, and limiting user access (Corti Reference Corti2000b). QUALIDATA has served as a model for other European countries, and its development offers important insights for the introduction of a repository in the United States.