The four earlier articles in this themed section are based on presentations at a workshop convened to explore challenges in using qualitative longitudinal methods for policy-related research, with particular focus on data analysis and interpretation. Together, they provide useful pointers to the scope of qualitative longitudinal approaches, and extensive discussion of management and analysis of the particular and complex data produced. Readers who are interested in pursuing themselves a qualitative longitudinal approach to policy-related social research will want also to look to a wider literature to enhance their understanding of issues to be taken into consideration, and to explore the range of methods and options. The aim of this review article is to provide some ideas about the kind of literature likely to prove helpful. The bibliographic references for this review article are supplemented by the guide to sources which completes this themed section.

The first section of the article looks at selected publications which address concepts of time and change in relation to human experience. This is, of course, a topic of considerable theoretical complexity and one that has been approached in different ways in different academic disciplines. We have not set ourselves the impossible task of providing a systematic review of the literature. What we offer is a selective review, staying close to social policy, and focusing on the specific issues of analysing time and change in longitudinal qualitative data sets. The second and third sections discuss definitions and design issues, and ethical issues. The final section reviews some overviews and edited collections in which readers will find discussion of theoretical and methodological issues about longitudinal approaches along with examples of empirical research.

Time and change

Looking for evidence and understanding of change by studying process and experience over time requires careful thought about what we understand by using the language of ‘time’ and ‘change’. The first chapters of Saldana's book, discussing methodologies for design and analysis of longitudinal qualitative research, are useful here. Saldana (2003: 5) warns readers against assuming obvious meaning in these terms and urges them to search for the constructs and definitions relevant for the task in hand. He ponders on the idea of time as a physically contextual construct, but moves quickly to time as a cultural and gendered construct, and time as an individually and subjectively interpreted construct. He finds also many different concepts and definitions of change, emphasising the importance to the researcher of understanding that change is contextual and multi-faceted.

Brannen (Reference Brannen2002: 2) identifies three sorts of time: ‘time as in present time; time as in the life course; and time as framed by historical events and historical period’ and explores some of the ways in which these three time frames have been used in the analysis of family life. Time in the present is the focus of much research into how people manage their everyday lives, for example, in combining paid employment and care. Lifecourse time puts attention on the timing and sequence of life course phases and transitions. This lifecourse perspective has become increasingly important in social policy research and we discuss this further below. Historical time provides a context for understanding both present time and lifecourse time, and placing people in their generation draws attention to what values as well as material experiences people might share as a consequence of ‘collective exposure to the same historical set of cultural and political events and exposures’ (ibid.: 14). The historical dimension can be applied at a much more microor short-term level; for example, the qualitative longitudinal research designs described by Lewis and by Corden and Nice in their articles in this volume both identify different ‘cohorts’ of entrants to employment programmes in order to explore whether and how changing programme design (and to some extent changing economic context) have an impact on the experience of participants. Similarly Pettigrew, from the perspective of organisation science, argues for the importance of studying any particular change within the context of changes at other levels of analysis; of locating the change within past, present and future time; of understanding that context itself helps to shape process and that changes have multiple causes and are explained ‘more by loops than by lines’ (Reference Pettigrew, Huber and Van de Ven1995: 96).

Past and future time influence present time; as McKie et al. (Reference McKie, Gregory and Bowlby2002: 904) put it, ‘everyday realities of managing spatial and temporal frameworks are informed by past experiences and future anticipations’. They draw on the concept of ‘timescape’ developed by Adam (Reference Adam2000) in their analysis of work and care. The concept of timescape encompasses quantitative time, the connection between time and space, and time as a multidimensional experience operating at different levels. They point out that employed mothers manage work and care in the context of time at three levels – by anticipating contingency (such as child illness), by monitoring the everyday (such as different patterns of school and childcare for each child) and by planning for the long term (such as the changing needs of family members over the lifecourse). These different types of time horizons can provide a framework for the analysis of longitudinal qualitative data. Brannen and Neilson (Reference Brannen and Neilson2007) also discuss time horizons and concepts of ‘planning’, and argue that present context is essential to understanding how people think about the future. Their article is in response to a discussion of their research on how young people plan for the future by Anderson et al. (Reference Anderson, Bechhofer, McCrone, Jamieson, Li and Stewart2005) and the debate between these two sets of authors raises some important points about the challenges of combining quantitative and qualitative longitudinal approaches.

There is also a rich stream of ideas about time and change in sociological literature about the dynamism of modern society. Leisering and Walker (Reference Leisering, Walker, Leisering and Walker1998) provide a useful short review of writing about the distinctive nature and rate of change affecting individuals and social institutions in modern societies, and the emergence of the ‘life course’ as a new social entity. They explore some of these ideas with specific reference to understanding of poverty, as do Dewilde (Reference Dewilde2003) and Alcock (Reference Alcock2004), see also Millar's article in this volume. Walker and Leisering (Reference Walker, Leisering, Leisering and Walker1998) discuss how these new ideas about change are to be explored, and what particular forms of enquiry may be used. They suggest that concepts of states, trajectories and domains of life for individuals will be useful, with an additional component of time, operationalised as duration. They emphasise that the duration of units of analysis is often greater than the period over which the social researcher can observe. The beginning of a social state may be unknown, and the end unknown or unobserved. Among the different kinds of longitudinal enquiry and analysis of data which make dynamic research possible, Walker and Leisering identify qualitative panels and the biographical method as potentially valuable.

Lifecourse approaches are of particular interest to many policy questions about how interventions at particular times may affect future outcomes. Heinz and Kruger discuss the development of and key concepts in the contemporary life-course approach which explores interactions over time between structural constraints, institutional rules, subjective meanings and decision making (Reference Heinz and Kruger2001: 33). They suggest that this way of studying social structure and individual agency can help find answers to some current social policy challenges, citing examples in fields of education, unemployment, poverty and family breakdown.

Definitions and design issues

Here we discuss definitions of longitudinal qualitative research and consider some design issues that arise from these definitions. Holland et al. (Reference Holland, Thomson and Henderson2006) note that approaches to qualitative longitudinal research tend to vary across academic disciplines, including for example continuous research in the same community over time, follow-up studies or returns to sites of previous research, repeated interviews with the same people at regular intervals, and lifecourse research involving data collection across several generations. Farrall (Reference Farrall2006: 2) focuses on repeat interview studies and defines qualitative longitudinal research as ‘returning to interviewees to measure and explore changes which occur over time and the processes associated with these changes’. Thomson et al. (2003: 185) argue that not all qualitative research over time should necessarily be considered to be longitudinal: ‘many qualitative research studies have employed longitudinal components, re-interviewing informants or returning to original study sites. What distinguishes longitudinal qualitative research is the deliberate way in which temporality is designed into the research process making change a central focus of analytical attention’. Similarly Vallance (Reference Vallance2005: 4) argues that ‘longitudinal qualitative analysis is that qualitative analysis which is conducted in order to examine developmental or causal relationships’ and is thus distinguished by three elements: a longitudinal research question (i.e. about developmental or causal relationships), a sample that includes data collection over time, and a means of analysis that ‘explicitly addresses change over time for individuals in such a manner as to describe meaningful relationships between the changes and the maturation or change in time itself’.

There are many texts available on the design of qualitative studies, and Saldana (2003: 16) provides a useful list of publications which cover research design for short- and long-term fieldwork. Here we briefly consider two particular design issues that arise out of the above definitions: the focus on change in the initial study design and the iterative nature of data collection. In terms of the initial design, these definitions imply that the longitudinal focus must be part of the study from the outset. In practice, however, longitudinal qualitative studies may come about in more diverse ways. The study by Millar and Ridge (discussed by Millar in this volume and also by Millar, Reference Millar, Glendinning and Kemp2006; Ridge, Reference Ridge, Glendinning and Kemp2006, Reference Ridge2007) was first funded as a single round of interviews to examine issues of transition into work. The second and upcoming third rounds of interviews have each been funded separately. However, in this case, the longitudinal aspect was envisaged right from the start, with the expectation that further funding would be successfully raised. The costs of longitudinal research can be very expensive and getting funders to commit to long time-frames may be difficult, so such mixed models of funding do happen and present challenges to the researchers in creating a design that includes the potential for longitudinal research. It also raises ethical issues in terms of ensuring that participants understand that future contacts and follow-up are contingent rather than certain.

The study by Graham et al. (Reference Graham, Tennant, Huxley and O'Connor2005) is another example of diverse routes to longitudinal qualitative research. They drew their longitudinal sample for their study of employment sustainability from two previous separate studies of people at the point of the transition to work. This meant that they had to design their interviews from two different sets of baseline interviews. MacDonald et al. (Reference MacDonald, Shildrick, Webster and Simpson2005) similarly drew their sample of young people growing up in poor neighbourhoods from two previous qualitative studies carried out in the late 1990s. It could be argued that these sorts of examples are not longitudinal qualitative research in the sense set out above, with temporality built into the design, but rather that they are follow-up studies from previous research. On the other hand, however, these are all examples of research which is concerned with understanding development and change over time. The longitudinal design in these cases is retrospective (linking back to the earlier situation) rather than prospective (acting as a baseline for the future).

Qualitative longitudinal research must include the capacity to analyse change over time, but the samples for this can be generated in various ways. Barnes et al. (Reference Barnes, Hudson, Parry, Sahin-Dikmen, Taylor and Wilkinson2005) provide another example, in which the qualitative longitudinal component is part of a wider programme of evaluation (see also the articles by Corden and Nice and by Lewis in this volume). Their research was examining ethnic minority outreach programmes and they started with case studies of 20 projects in five geographical areas, followed by in-depth interviews with 148 people who used the service, from which they included 65 people in their longitudinal study. These people were selected as ‘representing a continuum of distance from the labour market’, thus seeking to incorporate a dynamic variable into the sample selection.

A second important design issue concerns the relationship between data collection at the various points in time. It is generally agreed in the literature that decisions about the overall time period for the study and the frequency of the repeat contacts will depend very much on the aims and purposes of the research and that there can be no set rules to determine these. There is no clear definition of the meaning of ‘long’ in longitudinal research. Decisions must also be made about how the content of the interviews at each stage relate to one another. Smith (Reference Smith2003: 275), for example, discusses the process of ‘allowing findings from one wave to inform the next’ through an iterative process of analysis and design (see also discussions of this issue in the four articles in this volume). With four interviews over the three years, this approach allowed the researchers to explore emerging and changing ideas of citizenship and in particular the ‘fluid and dynamic’ perceptions of citizenship identity as a ‘contingent, life-long project’ (Smith et al., Reference Smith, Lister, Middleton and Cox2005: 441).

Ethical issues

A number of the authors mentioned refer to ethical issues which arise in their qualitative longitudinal studies. As Farrall points out (Reference Farrall2006: 11), qualitative longitudinal research does not raise ethical issues different from those already relevant to both quantitative and qualitative social research, but it heightens them. There is increasing recognition that participation in research on the basis of so-called ‘informed consent’ should be a continuous process, rather than a one-off agreement at the start of an interview (Crow et al., Reference Crow, Wiles, Heath and Charles2006; Holland et al., Reference Holland, Thomson and Henderson2006). This idea has particular salience within qualitative longitudinal research, where there should be repeated consultation with people about whether they want to continue to take part, discussed by France, Bendelow and Williams (Reference France, Bendelow, Williams, Lewis and Lindsay2000) and Saldana (Reference Saldana2003) in the context of longitudinal research with children and young people.

The attrition which is a feature of most qualitative longitudinal approaches is usually discussed in terms of practicalities, and what might be done to minimise attrition. There are ethical challenges here, however. Researchers must recognise tensions that may be inherent between minimising attrition and encouraging people to make preferences about whether and how they take part (Graham et al., Reference Graham, Grewal and Lewis2006). In their earlier article in this themed section Corden and Nice mention ethical issues in deciding how to deal with information provided by participants who later withdraw from further rounds of interviews.

Issues of data protection and confidentiality are sharpened when repeated interactions with participants give researchers large amounts of detailed information about people's lives. The impact of repeated interviews, involving reflection or looking to the future, is also likely to be greater for some people than would be the impact of a single research interaction. Thomson and Holland (Reference Thomson and Holland2003) discuss the responsibility to maintain privacy and integrity for young people who gained self-awareness as they reflected on their lives with researchers. In Thomson's paper in this themed section, she discusses the ethical challenges encountered in sharing the researchers’ far-reaching insights into people's lives, both with participants and in dissemination.

Overviews of longitudinal approaches

Understanding change over time at individual and societal levels draws variously on historical, psychological and sociological perspectives, underpinned by different philosophical approaches to what can be known about the social world and how it is possible to find out. These can seem relatively complex ideas, but readers who would like to know more about the philosophical foundations of qualitative research, generally, will find useful reading in Snape and Spencer (Reference Snape, Spencer, Ritchie and Lewis2003). Research that is useful for policy making may draw on a range of such theoretical perspectives, across disciplines and traditions. A pragmatic approach may be adopted, using different techniques to find answers to questions which are relevant and timely within policy planning, and which can feed into policy implementation. What counts as the evidence base for policy and practice is discussed in Becker and Bryman (Reference Becker and Bryman2004).

For this part of the review, we looked for published overviews and collections of qualitative longitudinal approaches. We found three major collections of articles drawing on longitudinal studies, each with an introductory overview, discussing theoretical and methodological issues, illustrated with references to the empirical research presented in the articles in the collections. Two of the overviews and the collections following (Ruspini, Reference Ruspini1999; White and Arzi, 2005) included both quantitative and qualitative approaches, while the third, edited by Thomson et al. (Reference Thomson, Plumridge and Holland2003), focused on qualitative approaches. In addition there are two recent reports discussing qualitative longitudinal methodology. The report by Molloy et al. (Reference Molloy, Woodfield and Bacon2002) was commissioned by the Department for Work and Pensions to consider qualitative longitudinal research for policy-related evaluation. The discussion paper by Holland et al. (Reference Holland, Thomson and Henderson2006) is based on their review for the Economic and Social Research Council of the feasibility and desirability of carrying out a large-scale qualitative longitudinal study. We go on to explain briefly what readers will find in this literature.

Ruspini (Reference Ruspini1999) brings together papers originally presented at an international workshop in Padua on longitudinal analysis, with the aim of exploring the potential of longitudinal data as a powerful tool for analysis of social change. In her editorial introduction, Ruspini offers a broad definition of longitudinal research as that in which: data are collected for each item or variable for two or more distinct periods; the subjects or cases analysed are the same or broadly comparable; and the analysis involves some comparison of data between or among periods. She goes on to discuss different ways of constructing longitudinal evidence: using repeated cross-sectional studies; prospective studies such as panel surveys; and retrospective studies, such as oral histories, each with particular advantages and disadvantages. While prospective panel studies have great potential to provide valuable information about human behaviour, methodological problems include panel attrition and possible effects on participants such as ‘conditioned responses’ or changes in thoughts or behaviour.

Ruspini's introductory article cites many references to other publications about longitudinal research, in general. In the collection of essays which she presents, each author focuses on one particular type (or more) of data to explore the potential of longitudinal analysis. Included are event history data; household panel surveys; repeated cross-sectional data and administrative records. Much of what is discussed would fall within what is traditionally understood as ‘quantitative research’. However, Ruspini herself argues that the development of research using longitudinal data provides new possibilities for building bridges between quantitative and qualitative traditions, and re-shaping these concepts. Other authors have argued for less emphasis on the differences between quantitative and qualitative research (Bryman, Reference Bryman2004; Hammersley, Reference Hammersley and Hammersley1992). Used in combination, quantitative and qualitative studies can provide the kind of powerful complementary data that is particularly useful to government policy makers.

A second collection of longitudinal studies was brought together by White and Arzi (Reference White and Arzi2005), with the aim of encouraging longitudinal research in science education. Introducing the collection, White and Arzi clarify conceptual and methodological aspects using examples from the authors of articles following. They make a broad distinction between experimental studies, for example introducing and evaluating an innovative teaching programme, and descriptive studies, such as following students’ growth in understanding of specific topics. For both types of studies, they discuss the validity of insights and conclusions and some of the practicalities which recur throughout discussions about longitudinal approaches generally, and qualitative longitudinal approaches in particular – resources, data management and the possibility of attrition.

In their collection, Thomson et al. (Reference Thomson, Plumridge and Holland2003) were concerned specifically with qualitative longitudinal research, arguing that this was an exciting, developing methodology but without much in the way of relevant specific literature to guide thinking and practice. Their edited collection draws on discussion and presentations at a seminar in London in 2002. Although policy relevance was not a specific objective for these editors in bringing the collection together, Thomson et al. comment on the increasing interest among policy makers in the value of having a ‘holistic’ understanding of the way behaviour is influenced by diverse factors. Many of the contributions address young people's transitions, which might suggest that a qualitative longitudinal approach is particularly appropriate for exploring change which is continuous but has an ill-defined time period.

Taken together, the Thomson et al. papers present a variety of research designs and methods including repeat interviews, life history and biography, diary keeping and scrapbooks, observation and case study. The collection begins with a set of papers which explore the relationship of qualitative longitudinal methods to theory, and provide different insights into relationships between agency and structure, and between sociological and psychological perspectives. Another set of papers focuses on methods and different ways of managing and exploring complex data sets. The studies included in the collection span different time periods, and several papers discuss how time enters the analytic process. Another focus of discussion is the relationship between the researcher and research subject, and how this counts as data. Practical issues discussed in several of the papers include maintaining contact with research subjects, and some of the authors draw attention to particular ethical issues raised. The overall collection provides a rich source of ideas and challenges in this developing methodology.

The one report found with specific focus on policy-related research was a report on longitudinal qualitative research approaches in evaluation studies (Molloy et al., Reference Molloy, Woodfield and Bacon2002). This was commissioned by the Department for Work and Pensions as part of a series to provide information on methodological issues in research. The context was an aim towards increasing awareness of what constitutes ‘quality’ in research, and thus to increase generation of good quality research evidence for policy decisions (see also the general discussion of ‘quality in qualitative research’ by Spencer et al., Reference Spencer, Ritchie, Lewis and Dillon2003).

Molloy et al. adopted Ruspini's broad definition of the common characteristics of longitudinal research (above) but emphasised that longitudinal research means that data collected earlier always forms an integral part of the research, involving going back to the same people, and addressing the original research questions. Their report discusses the potential of longitudinal qualitative research for understanding some of the processes and causes of changes that occur with policy implementation or programme delivery. They use recent examples of empirical studies to discuss the use of qualitative longitudinal research in both process and outcome evaluation. They show the kind of information that can be provided to answer the questions on which policy makers require key information.

The second part of this report is concerned with methodological issues. Molloy et al. again draw on specific examples of recent research to address the three considerations they see as most important in designing longitudinal approaches:

• • identifying appropriate research objectives for longitudinal components

• • identifying appropriate samples, and

• • selecting appropriate time frames.

They go on to discuss selection of appropriate ‘tools’ (interviews, group discussions, case studies, documentary analysis or observation) and the practicalities of conducting the work. They explain one method of analysis using ‘Framework’, which has been discussed earlier in this themed section by Lewis. The report concludes with a short section directed towards research managers in the policy arena. This discusses the implications of study design for project management, cost, timing and dissemination of results.

In this report, most of the studies were concerned with policy initiatives directed towards encouraging and enabling transitions in the fields of employment and education, reflecting key issues in current government welfare policy. The issues and principles discussed within the report, however, and the technical detail addressed, are widely relevant across other fields of social policy.

Holland et al. (Reference Holland, Thomson and Henderson2006) were commissioned by the ESRC to explore the possible contribution and feasibility of a large-scale and multi-purpose qualitative longitudinal panel, in the context of the ESRC's ongoing commitment to the development of longitudinal research. Their literature review groups publications according to discipline, including anthropology, education, psychology, health studies, sociology (lifecourse, childhood and youth studies, and criminology) and policy studies. They discuss the issues of sample type and size, duration, structure and organisation that would need to be addressed in the setting up of a large-scale qualitative panel, as well as issues of ethics (privacy, confidentiality and access). They concluded that there is significant support for such a panel.

The literature – theoretical and methodological – on qualitative longitudinal research for social policy is growing rapidly, as are the number of empirical studies that are using this methodology. This review will, we hope, provide a starting point for further reflections on the possibilities, but also the challenges, of this methodology, both in its own right and alongside other types of data. Mixed method studies are used in much policy research and the combination of quantitative and qualitative longitudinal data is potentially very powerful in providing links between causation, processes and outcomes. This must be an important part of the future social policy research agenda.