1. Introduction

While the viva voce (oral) examination has always been used in content-based educational assessment (Latham Reference Latham1877: 132), the assessment of second language (L2) speaking in performance tests is relatively recent. The impetus for the growth in testing speaking during the 19th and 20th centuries is twofold. Firstly, in educational settings the development of rating scales was driven by the need to improve achievement in public schools, and to communicate that improvement to the outside world. Chadwick (1864, see timeline) implies that the rating scales first devised in the 1830s served two purposes: providing information to the classroom teacher on learner progress for formative use, and generating data for school accountability. From the earliest days, such data was used for parents to select schools for their children in order to ‘maximize the benefit of their investment’ (Chadwick Reference Chadwick1858). Secondly, in military settings it was imperative to be able to predict which soldiers were able to undertake tasks in the field without risk to themselves or other personnel (Kaulfers 1944, see timeline). Many of the key developments in speaking test design and rating scales are linked to military needs.

The speaking assessment project is therefore primarily a practical one. The need for speaking tests has expanded from the educational and military domain to decision making for international mobility, entrance to higher education, and employment. But investigating how we make sound decisions based on inferences from speaking test scores remains the central concern of research. A model of speaking test performance is essential in this context, as it helps focus attention on facets of the testing context under investigation. The first such model developed by Kenyon (Reference Kenyon1992) was subsequently extended by McNamara (Reference McNamara1995), Milanovic & Saville (Reference Milanovic, Saville and Milanovic1996), Skehan (Reference Skehan, Bygate, Skehan and Swain2001), Bachman (Reference Bachman2001), and most recently by Fulcher (Reference Fulcher2003: 115), providing a framework within which research might be structured. The latter is reproduced at Figure 1 to indicate the extensive range of factors that have been and continue to be investigated in speaking assessment research, and these are reflected in my selection of themes and associated papers for this timeline.

Figure 1

An expanded model of speaking test performance (Fulcher Reference Fulcher2003: 115)

Overviews of the issues illustrated in Figure 1 are discussed in a number of texts devoted to assessing speaking that I have not included in the timeline (Lazaraton Reference Lazaraton2002; Fulcher Reference Fulcher2003; Luoma Reference Luoma2004; Taylor (Reference Taylor2011). Rather, I have selected publications based on 12 themes that arise from these texts, from Figure 1, and from my analysis of the literature.

Themes that pervade the research literature are rating scale development, construct definition, operationalization, and validation. Scale development and construct definition are inextricably bound together because it is the rating scale descriptors that define the construct. Yet, rating scales are developed in a number of different ways. The data-based approach requires detailed analysis of performance. Others are informed by the views of expert judges using performance samples to describe levels. Some scales are a patchwork quilt created by bundling descriptors from other scales together based on scaled teacher judgments. How we define the speaking construct and how we design the rating scale descriptors are therefore interconnected. Design decisions therefore need to be informed by testing purpose and relevant theoretical frameworks.

Underlying design decisions are research issues that are extremely contentious. Perhaps these can be presented in a series of binary alternatives to show stark contrasts, although in reality there are clines at work.

Specific purposes tests vs. Generalizability. Should the construct definition and task design be related to specific communicative purposes and domains? Or is it possible to produce test scores that are relevant to any and every type of real-world decision that we may wish to make? This is critical not least because the more generalizable we wish scores to be, the more difficult it becomes to select test content.

Psycholinguistic criteria vs. Sociolinguistic criteria. Closely related to the specific purpose issue is the selection of scoring criteria. Usually, the more abstract or psycholinguistic the criteria used, the greater the claims made for generalizability. These criteria or ‘facilities’ are said to be part of the construct of speaking that is not context dependent. These may be the more traditional constructs of ‘fluency’ or ‘accuracy’, or more basic observable variables related to automaticity of language processing, such as response latency or speed of delivery. The latter are required for the automated assessment of speaking. Yet, as the generalizability claim grows, the relationship between score and any specific language use context is eroded. This particular antithesis is not only a research issue, but one that impacts upon the commercial viability of tests; it is therefore not surprising that from time to time the arguments flare up, and research is called into the service of confirmatory defence (Chun Reference Chun2006; Downey et al. Reference Downey, Farhady, Present-Thomas, Suzuki and Van Moere2008).

Normal conversation vs. Domain specific interaction. It is widely claimed that the ‘gold standard’ of spoken language is ‘normal’ conversation, loosely defined as interactions in which there are no power differentials, so that all participants have equal speaking rights. Other types of interaction are compared to this ‘norm’ and the validity of test formats such as the interview is brought into question (e.g. Johnson Reference Johnson2001). But we must question whether ‘friends chatting’ is indeed the ‘norm’ in most spoken interaction. In higher education, for example, this kind of talk is very rare, and scores from simulated ‘normal’ conversations are unlikely to be relevant to communication with a professor, accommodation staff, or library assistants. Research that describes the language used in specific communicative contexts to support test design is becoming more common, such as that in academic contexts to underpin task design (Biber Reference Biber2006).

Rater cognition vs. Performance analysis. It has become increasingly common to look at ‘what raters pay attention to’. When we discover what is going on in their heads, should it be treated as construct irrelevant if it is at odds with the rating scale descriptors and/or an analysis of performance on test tasks? Or should it be used to define the construct and populate the rating scale descriptors? Do all raters bring the same analysis of performance to the task? Or are we merely incorporating variable degrees of perverseness that dilute the construct? The most challenging question is perhaps: Are rater perceptions at odds with reality?

Freedom vs. Control. Left to their own devices, raters tend to vary in how they score the same performance. The variability decreases if they are trained; and it decreases over time through the process of social moderation. With repeated practice raters start to interpret performances in the same way as their peers. But when severed from the collective for a period of time, judges begin to reassert their own individuality, and disagreement rises. How do we identify and control this variability? This question now extends to interlocutor behaviour, as we know that interlocutors provide differing levels of scaffolding and support to test takers. This variability may lead to different scores for the same test taker depending on which interlocutor they work with. Much work has been done in the co-construction of speech in test contexts. And here comes the crunch. For some, this variation is part of a richer speaking construct and should therefore be built into the test. For others, the variation removes the principle of equality of experience and opportunity at the moment of testing, and therefore the interlocutors should be controlled in what they say. In face-to-face speaking tests we have seen the growth of the interlocutor frame to control speakers, and proponents of indirect speaking tests claim that the removal of an interlocutor eliminates subjective variation.

Publications selected to illustrate a timeline are inevitably subjective to some degree, and the list cannot be exhaustive. My selection avoids clustering in particular years or decades, and attempts to show how the contrasts and themes identified play out historically. You will notice that themes H and I are different from the others in that they are about particular methodologies. I have included these because of their pervasiveness in speaking assessment research, and may help others to identify key discourse or multi-faceted Rasch measurement studies (MFRM). What I have not been able to cover is the assessment of pronunciation and intonation, or the detailed issues surrounding semi-direct (or simulated) tests of speaking, both of which require separate timelines. Finally, I am very much aware that the assessment of speaking was common in the United Kingdom from the early 20th century. Yet, there is sparse reference to research outside the United States in the early part of the timeline. The reason for this is that apart from Roach (see timeline, reprinted as an appendix in Weir, Vidaković & Galaczi (Reference Weir, Vidaković and Galaczi2013) (eds.) there is very little published research from Europe (Fulcher Reference Fulcher2003: 1). The requirement that research is in the public domain for independent inspection and critique was a criterion for selection in this timeline. For a retrospective interpretation of the early period in the United Kingdom with reference to unpublished material and confidential internal examination board reports to which we do not have access, see Weir & Milanovic (Reference Weir and Milanovic2003) and Vidaković & Galaczi (Reference Vidaković, Galaczi, Weir, Vidaković and Galaczi2013).