1. Introduction
Awareness of the importance of testing consequences is not new; it has existed since the birth of modern testing (Latham Reference Latham1877). The prevalence of large-scale, high-stakes testing worldwide (Ungerleider Reference Ungerleider2003; Cheng Reference Cheng, Shohamy and Hornberger2008; Klinger, DeLuca & Miller Reference Klinger, DeLuca and Miller2008) has given rise to growing attention and concerns over its impact on stakeholders, the educational system, and society at large. The result has been a fast growing body of research in education investigating intended and unintended relationships between testing, teaching, and learning. Given the range and extent of testing consequences reported worldwide, it is critical that testing practices yield valid data about student achievement and performance. However, the core issue of test consequences resides in the (mis)use of test scores, the values and the stakes attached to a test within society, and the teaching and learning context within which a particular test exists. Therefore, to achieve the intended positive consequences of testing, and to mitigate its negative consequences, it is critical that test results are accurately interpreted and appropriately used.
Considering the nature of testing consequences, Messick (Reference Messick1996: 243) regards washback as ‘only one form of testing consequence that needs to be weighted in evaluating validity, and testing consequences are only one aspect of construct validity needing to be addressed’. Messick (Reference Messick1996) also points out that the consequences of tests are likely to be a function of factors both within the test itself and within the setting where the test is implemented. He recommends, from the test design perspective, minimizing the two threats to construct validity – construct under-representation and construct irrelevant variance – in order to enhance the quality of the test and thus promote positive washback and reduce negative washback. Bailey (Reference Bailey1996: 268), however, argues that any test, whether good or bad, can have either positive or negative washback depending on whether ‘it impedes or promotes the accomplishment of educational goals held by learners and/or program personnel’. She focuses on the specificity of this phenomenon, which could induce differential impact on test stakeholders within a range of teaching and learning contexts – a view increasingly shared by many researchers in language testing (e.g., Cheng Reference Cheng, Shohamy and Hornberger2008). In a sense, positive or negative washback is likely defined by test stakeholders, possibly differentially, as they see how a test serves its purpose from their points of view. More recently, Bachman (Reference Bachman2005) proposes a validity framework with a set of principles and procedures for linking test scores and score-based inferences to test use and the consequences of test use, an area in which, he argues, more research needs to be conducted.
In addition, testing consequences have increasingly been discussed from perspectives such as critical language testing (Shohamy Reference Shohamy2001), which focuses on ethics and fairness (Hamp-Lyons Reference Hamp-Lyons1997; Kunnan Reference Kunnan, Milanovic, Weir and Bolton2004). Shohamy (Reference Shohamy2001) points out the power embedded in political uses and abuses of language tests, and calls for examining the hidden agendas of the testing industry regarding high-stakes tests. Kunnan (Reference Kunnan, Milanovic, Weir and Bolton2004) discusses the role of tests as instruments of social policy and control, drawing on research in ethics to link test validity and test consequences in order to create a test fairness framework. Hamp-Lyons (Reference Hamp-Lyons1997) argues for an encompassing ethics framework to examine the consequences of testing on language learning at the classroom level as well as the educational, social and political levels. These perspectives reveal the complexity of consequences, and highlight the importance of contextual factors in studying washback.
The term ‘consequences’ is a general term in the field of educational assessment, and is associated with the results of the (mis)use of test scores. Research into consequences of large-scale testing tends to focus on questions related to consequential validity (Messick Reference Messick and Linn1989): the after-effects of test interpretations and their use on various stakeholders including value implications and social consequences. The terms ‘impact’ and ‘washback’ – both now commonly employed in applied linguistics – are specific research terms used to refer to the influence of testing on teaching and learning. When researchers use the term washback, they tend to ask questions about the influence of the introduction of a new test, or an existing test, on classroom teaching and learning. Washback (‘backwash’ in early literature) is the term that originated, and has been used specifically, in applied linguistics since the publication of the oft-cited article ‘Does washback exist?’ (Alderson & Wall Reference Alderson and Wall1993). Researchers tend to use the term impact to investigate the extended influence of testing beyond classroom teaching and learning to other stakeholders in education. McNamara (Reference McNamara2000) uses two terms to distinguish between two levels of this phenomenon: ‘impact’, the effects of tests on the macro-levels of education and society; and ‘washback’, the effects of language tests on micro-levels of language teaching and learning inside the classroom. In this sense, the difference between impact and washback resides in the scope of the effects of testing – a more narrow one of washback and the all-encompassing one of impact (Hamp-Lyons Reference Hamp-Lyons1997). These two specific terms are used in this review as individual researchers use them in their studies.
Figure 1 illustrates the differences in terms of research focus, research questions asked, and on-going debates. Hereafter in this review, when we use the term ‘consequence’, we frame our discussion to the direct results of (mis)use of test scores, e.g., if a student fails a particular test, one of the consequences could be not being able to graduate. When we use the terms ‘washback’ (‘backwash’) and ‘impact’, we regard them as bearing the same meaning, i.e., the influence of large scale testing on teaching and learning, yet taking into account the mediation roles of the contextual variables in this relationship. Not all empirical studies we reviewed here examined all three areas, i.e., the consequences of the testing, the influence of testing on teaching and learning (washback), or the influence on society (impact). There are currently debates in each of the three areas over questions such as these: Should consequences be studied as a part of the validity or separately? How should evidence about consequences be weighed in test validation? Consequences of a test are largely beyond the control of the test developer and cannot necessarily be traced to the quality of a test either. Does the tail wag the dog? The premise of this analogy (Fullilove Reference Fullilove1992) is that testing tends to happen at the end of the teaching and learning, yet testing drives teaching and learning. Can we design new, alternative, or innovative testing to drive teaching and learning? Do contextual variables mediate the effect? And finally, how do other stakeholders interact with these variables? These are the areas that have been dealt with in the empirical studies and will be discussed in this state-of-the art review.
Consequences, washback (backwash) and impact
In addition, ‘testing’ (‘tests’) is used consistently, which bears similar meaning to examinations as cited by individual researchers. Of note, the testing and examination practices reviewed here are mostly large-scale and high-stakes in nature. Studies on the consequences, impact, and washback of classroom-based teacher-led formative assessments are just starting to emerge. They are reviewed in the later part of this paper and are not its major focus because such issues require different sets of considerations (Brookhart Reference Brookhart2004). We present this argument at the end of our review and point to this omission as a potential area for future research.
2. Educational background
Large-scale high-stakes examinations have long been used for selection purposes in education and employment. In many parts of the world, these examinations have been valued as ways to encourage the development of talent and to upgrade the performance of schools and colleges. They have also been employed as a means to counter, to some degree, nepotism, favoritism, and even outright corruption in the allocation of scarce opportunities, as in the case of the creation and use of the Imperial Examinations in China thousands of years ago (Eckstein & Noah Reference Eckstein and Noah1992).
If the initial spread of examinations can be traced to the above, the very same reasons appear to be as powerful today as ever. Linn (Reference Linn2000) classified the uses of tests and assessments as key elements in relation to five waves of educational reform over the past 50 years in the USA: their tracking and selecting role in the 1950s; their program accountability role in the 1960s; minimum competency testing in the 1970s; school and district accountability in the 1980s; and the standards-based accountability systems in the 1990s. Clearly, tests and assessments have played a crucial and critical role in both education and society in general. Testing consequences are influenced by the ideological, social, and political milieu surrounding particular educational systems.
Due to the extensive uses of large-scale testing and the high stakes associated with their outcomes, testing consequences have been an issue of long-standing concern in education. The earliest literature can be traced back to Latham (Reference Latham1877), who referred to an examination system as an ‘encroaching power,’ and pointed out ‘how it influences the prevalent view of life and work among young men, and how it affects parents, teachers, the writers of educational books, and the notion of the public about education’ in the UK (p. 2). In spite of its long and well-established place in educational history, the use of tests has, constantly, been subject to criticism. Nevertheless, tests continue to occupy a leading place in educational policies and practices in most countries around the world.
Aware of the power of tests, policy-makers in many parts of the world continue to use them to manipulate the local educational systems, to control curricula and to impose (or promote) new textbooks and new teaching methods. Testing is ‘the darling of the policymakers’ (Madaus Reference Madaus1985: 5) and it would not be too much of an exaggeration to say that they have become the engine for implementing educational policy, despite the fact that tests have been the focus of controversy for as long as they have existed. One reason for the longevity in the face of such criticism is that tests are viewed as the primary tools through which changes in the educational system can be introduced, without having to change other educational components such as teacher education and curricula – a naïve and simplistic view of the power of testing practices.
This systematic review of washback research in language teaching stands on three foundational groundings. First, there is a need to set the premise of the background of this educational phenomenon within the contemporary research context. This is to illustrate that washback is not a new concept in education and, more importantly, that this phenomenon is developing and evolving within the current educational climate. Second, since the seminal publication of Alderson & Wall's (Reference Alderson and Wall1993) 15 washback hypotheses, there have been a large, and steadily growing number of empirical studies conducted in language education, many of which are large-scale, multiple-phased, and doctoral-level works. This body of knowledge is significant at the interface of language testing, teaching and learning, and demonstrates important developments in language teaching and learning. It is therefore worthy of systematic documentation as a review article at this point in educational research. Third, washback research has become a dominant area of investigation in language assessment, which has significant implications regarding test validation and test fairness. This body of work has employed both psychometric-driven test validation methods and philosophically argued methodologies and thus offers both theoretical and methodological guidance to language assessment research.
3. Search methods
We searched studies on washback that were conducted between 1993 and 2013. We chose this time period because it coincides with the aforementioned seminal publication by Alderson & Wall in 1993, which ignited the growing interest in investigating washback in applied linguistics, and in language testing in particular. We searched three types of washback studies according to their publication formats: published studies (including journal articles, books/book chapters, and research monographs), unpublished doctoral dissertations, and review articles discussing this educational phenomenon. We examined both academic journals and research databases.
We started the search for published studies in two major journals: Language Testing and Language Assessment Quarterly. These journals were selected because they are highly regarded by scholars in the language assessment field. We then expanded our search to applied linguistics and language education journals, in order to locate studies on washback. The expanded search included journals such as TESOL Quarterly, Applied Linguistics, and System. We used the keywords ‘washback’, ‘backwash’, ‘impact’, and ‘consequences’ in the advanced search engine on the webpages of these journals to identify all studies that contained these keywords in their titles and/or abstracts.
After exploring these journals, we next searched two major education databases and testing agency databases to widen our search: ERIC and PsycINFO. The same keywords used in searching the journal articles were entered into the advanced search engines of these databases. To ensure the accurate presentation of empirical studies, and thorough conceptualization of washback review studies, we limited our search to academic journal articles and books/book chapters to which we had full-text access. We excluded reports, documents and conference proceedings. We also focused on the research databases of two testing agencies – Cambridge English (Cambridge ESOL), and Educational Testing Service (ETS) – for their published washback studies such as Research Report, Studies in Language Testing by Cambridge ESOL, and research monographs by ETS.
In addition to published studies, doctoral dissertations constituted a substantial body of empirical studies on washback. The international theses and dissertations database was used to find unpublished doctoral dissertations on this topic. The terms used for searching were the same as those employed in previous searches in journals and databases, and only dissertations with full text were included in this review.
Our search generated a total of 123 publications consisting of 36 review articles and 87 empirical studies. However, the latter were not inclusive of all studies on washback within the period 1993 to 2013. Due to the search limitations through our library portal, we were unable to get access to some empirical studies mentioned in reference lists and/or bibliographies (e.g., the selected references of washback in language assessment by The International Research Foundation for English Language Education (TIRF)). For example, we found some of the doctoral dissertation works (e.g., Ben Nkosana Reference Ben Nkosana2008, Froehlich Reference Froehlich2010, Wei Reference Wei2011), yet we could not gain access to the full dissertations. We did not have access to that of Nemati (Reference Nemati2003) in Indian Journal of Applied Linguistics either. Therefore, we have focused on the empirical studies on washback for which we had access to the full-text for purposes of this review.
In order to support future researchers on this educational phenomenon, we have included a chronologically ordered master list of all the references cited. We have focused primarily on the 87 empirical studies. Of these, 40 are journal articles, 27 are doctoral dissertations, 11 are books and monographs, and 9 are book chapters. Table 1 gives the date of publication and the author(s) of these washback studies. Table 2 shows these studies in terms of the tests and the contexts/countries in which these studies were conducted. Both tables were created to present to our research community the range of these studies, and are designed to guide future research.
Washback studies 1993–2013
Note: Total studies = 87. b= book (4); c = chapter in book (9); d = doctoral dissertation (27); j = journal article (40); m = monograph (7).
Numbers of washback studies 1993–2013 by test type and countries
Note: The number in each cell concerning the test studied and country in which the empirical study was conducted is based on the information obtained from the main text of these studies. Due to the unavailability of the information in some studies, we were unable to categorize each of 87 empirical studies into tabulations by test and country.
4. Theoretical framework
Kane's (Reference Kane and Brennan2006, Reference Kane2013) argument-based validation framework is used in this literature review to categorize the empirical washback studies mentioned above. According to this framework, validation involves evaluation of the interpretation/use argument (IUA). Based on different testing contexts, the IUA can be specified as a network of inferences and assumptions leading from the test performances to the score interpretations, and to any decisions of test uses. The decision inference links interpretations of the test score to decisions of score uses and is evaluated in terms of consequences, i.e., expected success in achieving goals at a reasonable cost and with acceptable consequences (Kane Reference Kane2013). Citing Bachman & Palmer (Reference Bachman and Palmer2010) and Messick (Reference Messick and Linn1989), Kane (Reference Kane2013) noted that the kinds of consequences to be considered under validity have tended to expand over time, as our understanding of how testing programs function in the world has grown. Thus, the main concerns are what kinds of consequences should be focused on in test evaluation, how these consequences are investigated, and how to understand findings concerning the consequences. Kane (Reference Kane2013) delimits three broad categories of consequences relevant to the evaluation of score-based decision rules: (1) the extent to which intended outcomes are realized; (2) the differential impact on groups, particularly adverse impact on legally protected groups as well as racial and ethnic minorities; and (3) positive and negative systemic effects, particularly in education.
Washback studies conducted in the language-testing field over the past few decades have indeed covered all these three categories. First, language-testing programs have been increasingly developed in different parts of the world as a policy instrument to promote language education. This has blurred the distinction between intended outcomes and washback. The extent to which these programs achieved their intended purposes in terms of improving teaching and learning has been a focus of washback research. Second, the effects of large-scale high-stakes testing programs on linguistic minority students have received increasing attention in recent years, particularly in the international education context. Third, the systemic effects of testing on different aspects of teaching and learning have continued to be the focus of washback studies.
We have thus adopted Kane's categorization of consequences in test validation for this review as a guideline for categorizing and synthesizing washback studies. Studies in Category 1 investigated the extent to which language testing programs achieved the purposes they were designed for. Studies in Category 2 focused on the effects of large-scale high-stakes testing programs on linguistic minority students, particularly in educational contexts where there are increasing numbers of English language learners. The systemic effects of testing on different aspects of teaching and learning were the focus of washback studies in Category 3. These categories are neither mutually exclusive nor exhaustive. A study may also cover more than one category, which occurs mostly for studies that can be categorized into either Category 1 or Category 3. For example, we chose to put the TOEFL iBT studies (Wall & Horák Reference Wall and Horák2006, Reference Wall and Horák2008) into Category 1, but the studies on the systemic effects of testing on different aspects of teaching and learning (e.g., the College English Test) were put into Category 3 even if certain studies (Qi Reference Qi2007; Wang Reference Wang2011) investigated the effects of a reform of the existing test. When categorizing the washback studies, we used the role of testing in the context as our guide rather than the particular research questions asked or methods employed in the study.
5. Major washback research
Using Kane's framework, we analyzed the 87 empirical studies and categorized them according to Kane's three categories: 32 in the first category, 7 in the second, and 44 in the third. We found four studies that could not be categorized using Kane's framework. They were studies looking at the issues of washback within the classroom assessment setting. As mentioned earlier, studies of this nature require a different consideration; therefore, we reviewed them separately as the fourth category. The theoretical categorization of the studies provided us with the overall picture of what has been conducted so far in our attempt to understand consequences, washback, and impact. We three authors conducted the analysis independently at first, and then re-examined, discussed, and reached consensus over those studies that were not straightforward to classify and which fit a combination of categories. The review below presents these studies in more detail, and illustrates how researchers conducted them.
5.1 Category 1: The extent to which intended outcomes are realized
There has been growing interest among policy-makers worldwide in using large-scale high-stakes language assessment for accountability and educational reform purposes. Such educational reform is based on the premise that introduction of new assessment regimes will result in improved outcomes for all students (see Brindley Reference Brindley and Shohamy2008 for a review, Adair-Hauck et al. Reference Adair-Hauck, Glisan, Koda, Swender and Sandrock2006 for an example). However, as noted by Wall (Reference Wall1996), attempts to use high-stakes tests to ‘introduce change in the classroom are often not as effective as their designers hoped they would be’ (p. 334). Moreover, studies of curriculum and assessment reform in language teaching contexts show that such centrally driven reform is likely to bring changes in some areas and not others (Cheng Reference Cheng1997, Reference Cheng1999, Reference Cheng and Savignon2002, Reference Cheng2003; Muñoz & Álvarez Reference Muñoz and Álvarez2010), with some learners and not others (Cheng, Andrews & Yu Reference Cheng, Andrews and Yu2011), and even produce a range of unintended consequences (Qi Reference Qi2007).
A large number of washback studies in language testing that were published in the last decade (32 out of 87) fall into this area of research. Among them are major empirical studies published in the Studies in Language Testing (SiLT) series book volumes: (Cheng Reference Cheng2005; Wall Reference Wall2005);Footnote 1 in research reports: (Wall & Horák Reference Wall and Horák2006, Reference Wall and Horák2008); and in doctoral dissertations (e.g., Pan Reference Pan2010, Yu Reference Yu2010, Schissel Reference Schissel2012), as well as Cheng, Watanabe & Curtis (Reference Cheng, Watanabe and Curtis2004) – the first systematic attempt to capture the essence of the washback phenomenon through its collection of washback studies from around the world. With respect to theory, some researchers turned to classroom teaching and learning theories to guide their studies (e.g., Cheng Reference Cheng2005), others to curriculum innovation theories (e.g., Wall Reference Wall2005), and increasingly, researchers have turned to a validity framework to guide their studies (Saif Reference Saif2006; Cheng, Klinger & Zheng Reference Cheng, Klinger and Zheng2007; Abdul Kadir Reference Abdul Kadir2008; Wang Reference Wang2010; Xie Reference Xie2010).
Kane (Reference Kane and Brennan2006) argues that testing programs used to implement policies are necessarily evaluated in terms of consequences, positive and negative, intended and unintended. In language assessment, researchers have increasingly investigated the consequences of language assessment programs from the perspective of validity. Each of these studies investigates the influence of testing on various aspects of teaching and, increasingly, on various aspects of learning. These explorations have also expanded our understanding of test impact on other stakeholders such as parents (Shih Reference Shih2006; Cheng, Andrews & Yu Reference Yu2010), employers (Pan Reference Pan2010), and publishers (Hawkey Reference Hawkey2006; Mishan Reference Mishan, Tomlinson and Masuhara2010).
Wall (Reference Wall2005) was one of the first empirical studies examining washback within the context of teaching English in Sri Lanka (see also Wall & Alderson Reference Wall and Alderson1993). Through a detailed analysis of the impact of examination reform in one specific educational setting, it considered the effects of a test that was meant to serve as a lever for change in teaching and learning. This study also described how the intended outcome was shaped by factors in the test itself, as well as by features of the context, the teachers, and the learners. This study provided an operationalized model for researching washback and impact, as well as practical guidelines for the planning and management of change within an educational context. It is of particular relevance to all those who are involved in the process of curriculum and examination reform, in order to understand what can or cannot be achieved.
Cheng (Reference Cheng2005) presented a study of how the introduction in 1996 of a high-stakes public examination impacted classroom teaching and learning in Hong Kong secondary schools. Using a balanced combination of both quantitative and qualitative methods, Cheng observed the washback effect among large samples of different stakeholder groups within the local educational context – in terms of teachers’ attitudes (Cheng Reference Cheng, Cheng, Watanabe and Curtis2004), teaching content and classroom interactions (Cheng Reference Cheng1999), and student perceptions (Cheng Reference Cheng1998). The study documented in detail the process of how an intended examination change brought about modifications to aspects of teaching and learning. Furthermore, this study revealed how using an examination can change perceptions, and even some instructional activities in the classroom, but made no fundamental changes to how teachers teach. This study pointed out the limited function of an examination (washback), i.e., how dog wags the tail (not the other way around). It also directed researchers’ attention to what drives teachers in their pedagogical choices and classroom assessment practices. This study is of particular relevance to language test developers and researchers interested in the consequential validity of tests, as well as to teachers, curriculum designers, policy makers, and others concerned with the interface between language testing and teaching practices.
Similarly, Andrews, Fullilove & Wong (Reference Andrews, Fullilove and Wong2002) studied a major examination change in Hong Kong that was intended to bring about positive washback on teaching and learning. They found that the introduction of the Use of English oral examination as a requirement for university admission in Hong Kong appeared to lead to general improvements in students’ spoken performance. However, some students’ inappropriate use of transitional words and discourse markers seemed to indicate a rote-learning of exam-targeted strategies, and formulaic phrases rather than meaningful internalization. What needs further research, in this context, is why students think memorization can help them to cope with a speaking examination. Though not discussed in the study, these findings, together with Cheng (Reference Cheng2005) and similarly Qi (Reference Qi2007) in the mainland Chinese context, seem to suggest that students may not always focus on developing the construct of a test. This may happen especially in the public examination context, where test-related information may not be directly accessible to students. In addition, students’ perceptions of tests are likely to be shaped by the school context, for example, by their teachers and peers. Therefore, it is important to examine not only what students understand about a test, but also how they obtain such knowledge.
High-stakes language tests are being used as an important part of the educational reform movement worldwide. Liauh's (Reference Liauh2011) study relates to this movement in Taiwan. It compared teachers’ and students’ perceptions of the implementation of the Exit English Examination (EEE), an English language test employed in Taiwanese higher education institutions. Liauh's (Reference Liauh2011) mixed-method study found that various factors, concerning teachers and students, played significant roles in their attitudes toward EEE implementation. Furthermore, Liauh found that the teachers and students had conflicting opinions with respect to teaching to the test, the curriculum, and teaching effectiveness. Hirai & Koizumi (Reference Hirai and Koizumi2009) studied the development of a practical speaking test with the intention to enhance and specifically ease the process of test preparation and administration and bring about positive washback effects on learning. Their questionnaire results, from a group of Japanese university students, demonstrated positive impact on their learning and motivation to enhance speaking ability, especially oral paraphrase and summary abilities. Testing processes in both studies were clearly designed to achieve positive washback.
Wall & Horák's (Reference Wall and Horák2006, Reference Wall and Horák2008) studies are built on a solid theoretical foundation and framework, specifically Henrichsen's hybrid model of Diffusion/Implementation Process (1989), which is also used in Wall (Reference Wall2005). This research was funded by ETS to investigate the impact of the two-phase introduction of the new TOEFL, within the context of Central and Eastern Europe (see also another study on the new TOEFL by Hamp-Lyons & Brown Reference Hamp-Lyons and Brown2007). Wall & Horák (Reference Wall and Horák2006) provided a baseline description of the teaching and learning that took place in TOEFL preparation classes in Central and Eastern Europe. They mapped out three stages of the impact study and conducted classroom observations in listening, reading, writing, or speaking. Phase 2 addressed the question of how teachers (n = 6, the same groups of teachers as in Phase 1) coped with the new TOEFL examination and perceived aspects of this test in relation to their teaching. The researchers collected a large volume of data via the internet over a period of five months. The implications of this study provided valuable information on how an educational system, in terms of teachers, reacts when a major change to an examination takes place. Such information may, in turn, help the test designers, constructors, and administrators understand how the test constructs operationalize in the teaching process and how teachers and students conduct test-preparation activities.
Stoneman's (Reference Stoneman2006) study provided further evidence that students faced with a high-stakes examination tended to choose activities mainly intended for test-specific coaching. The contribution of her study to our knowledge about washback on learners is that students’ past learning and test-taking experiences have a major influence on their choice of the types of test preparation activities. These studies, conducted in Europe (Wall & Horák Reference Wall and Horák2006, Reference Wall and Horák2008) and Asia (Stoneman Reference Stoneman2006), provided evidence of the targeted test preparation activities as a phenomenon of washback and test impact.
The doctoral dissertations (a total of 13 in Category 1) demonstrated the strong impetus in language education around the world to use large-scale testing to drive teaching and learning. Apart from Schissel's (Reference Schissel2012) investigation into the policy-enforced washback of test accommodations for emergent bilingual students in two urban public schools in Pennsylvania, USA, the doctoral studies were conducted in the context of teaching English as a foreign language. This is due, among other reasons, to the existence of large-scale high-stakes testing and the drive to introduce ‘innovative’ teaching pedagogy to existing teaching practices, which tend to be teacher-centred and lack quality teachers of English, primarily in the Asian context. These studies were conducted in different contexts around the world, e.g., mainland China (Zhao Reference Zhao2005), Hong Kong (Lumley & Stoneman Reference Lumley and Stoneman2000; Stoneman Reference Stoneman2006; Yu Reference Yu2010), Korea (O Reference K-M.2001), Malaysia (Abdul Kadir Reference Abdul Kadir2008; Tan Reference Tan2009), Taiwan (Pan Reference Pan2010; Shih Reference Shih2006, Reference Shih2010), Vietnam (Nguyen Reference Nguyen1997), Greece (Tsagari Reference Tsagari2007), and Costa Rica (Quesada Reference Quesada2001).
To illustrate a few of these studies, Shih (Reference Shih2006), Tan (Reference Tan2009) and Zhao (Reference Zhao2005) examined the washback of testing programs on teaching and learning. Shih (Reference Shih2006) used a case-study approach to investigate a range of stakeholders’ perceptions of the Taiwanese General English Proficiency Test (GEPT) and its washback on schools’ policies, teaching, and English learning. The GEPT is an English language proficiency test for Chinese learners of English at all levels and has gained societal acceptance for educational and employment purposes in Taiwan. Shih's (Reference Shih2006) study was conducted at the applied foreign language departments of two technological (vocational) institutes. The primary research methods used were interviews with the departmental chair, teachers, students, and their immediate family members, coupled with observations of English language courses. Such research methods are often used in washback studies. However, Shih also included family members as stakeholders, a group rarely considered in studies of this nature. His findings revealed varying aspects of the GEPT washback on teaching and learning within the research context. The dissertation concluded with a proposed washback model of students’ learning, which could guide future research into key components of students’ learning.
Tan's (Reference Tan2009) study, which took place in a secondary education setting in Malaysia, examined a change in the language of instruction for mathematics and science subjects from Bahasa Malaysia to English in Malaysia. This policy had two objectives: to promote student learning of mathematics and science, and to increase student proficiency in English. The Education Ministry also chose to create a washback effect by introducing a bilingual high-stakes secondary exit examination. The new test (a revised testing system) was introduced to bring about change. The study drew on insights and perspectives offered by the literature on educational change, content-based instruction, and washback in language testing, in order to examine the impact of the policy. This was a mixed-methods study involving both a longitudinal and a cross sectional research design. The results pointed to the complexity of educational change processes and indicated that classroom implementation of this policy was affected by multiple factors. This finding echoed that of Shohamy, Donitsa-Schmidt & Ferman (Reference Shohamy, Donitsa-Schmidt and Ferman1996), who found that the degree of impact of a test is often influenced by several other contextual factors: the status of the subject matter tested, the nature of the test (low or high stakes), and the uses to which the test scores are put.
Abdul Kadir (Reference Abdul Kadir2008), among others (e.g., Cheng et al. Reference Cheng, Klinger and Zheng2007; Wang Reference Wang2010; Xie Reference Xie2010), adopted the framework that testing consequences are part of test validity, and responded to the call by Bachman (Reference Bachman2005), when he proposed a framework with a set of principles and procedures for linking test scores and score-based inferences to test uses and their consequences. This study evaluated the usefulness and impact of an occupational language proficiency assessment that was developed for one specific test group but extended to a wider population following a policy shift within the context of the study. In utilizing Kane's approach to validation, Abdul Kadir (Reference Abdul Kadir2008) examined the claims for the use of the test, employing a network of inferences to form the basis for the validity argument. This study is one of the few that examined the evidence from scoring procedures, the generalization of observed scores, and the extrapolation of observed scores to non-test behavior, and investigated the impact of the test.
5.2 Category 2: Differential impact on racial and ethnical minorities groups
This section reviews the limited number of empirical studies (seven in total) on the washback effects of large-scale high-stakes testing programs on linguistic minority students, which have started to receive increasing attention in recent years, particularly in the context of international education. They focus on how linguistic minority students (e.g., English language learners, migrant students, and international visa students) and other stakeholders at school- and classroom-levels (e.g., principals, teachers) experienced the influence of large-scale tests on their practices. Studies by Burrows (Reference Burrows, Cheng, Watanabe and Curtis2004) and by Stecher, Chun & Barron (Reference Stecher, Chun and Barron2004) provide evidence regarding what and how testing influences teachers’ instruction, and also how washback influences teachers differently. Studies focusing on the washback on different groups of test-takers, including research by Cheng et al. (Reference Cheng, Klinger and Zheng2007), and by Fox & Cheng (Reference Fox and Cheng2007), have investigated the washback effects on their language learning, while Menken (Reference Menken2006) and Scott (Reference Scott2005) have explored the ways in which policy-driven assessments and testing programs influence English language learners.
Burrows’ (Reference Burrows, Cheng, Watanabe and Curtis2004) study examined the washback effects of the Certificate in Spoken and Written English (CSWE) at the classroom level, and focused on teachers’ pedagogical perceptions and practices. As a newly introduced assessment in Australia for the adult migrant English program, CSWE was intended to explicitly connect curriculum with assessment tasks (e.g., teaching objectives are the assessment outcomes) to direct teachers’ pedagogy focus onto competency-based English teaching. Burrows collected multiple sources of data, i.e., survey, interview, and classroom observations, to understand the influence of this assessment on the content (what), pedagogy (how) and extent (to what degree) of teachers’ classroom practices.
The interview data revealed two distinct groups of teachers, divided according to their perceived reasons for changing their teaching practice. One group related the changes to the implementation of the assessment (CSWE); the other associated them with the implementation of the curriculum (competence-based curriculum). However, the changes in teaching practices reported by these two groups had great similarity, indicating the blur between the implementation of the assessment and the introduction of the curriculum framework, as well as the uncertainty surrounding the causes underlying the teachers’ pedagogical shifts.
The observation data further revealed the varying extent of influence that this assessment had on different teachers, such as the aspects of content they covered and the types of materials they used. The results of this study supported the washback hypothesis that ‘tests will have washback effects for some teachers and some learners, not for others’ (Alderson & Hamp-Lyons Reference Alderson and Hamp-Lyons1996: 282). Furthermore, Burrows proposed a new washback model that allows for the analysis of teachers’ responses to the introduction of a new test or assessment. This model is a way of examining washback from the perspectives of teachers’ response patterns to tests, that is, to understand how washback effects were differential among teachers.
Stecher et al. (Reference Stecher, Chun and Barron2004) examined how, and to what extent, school and classroom practices were influenced by the Washington education reform, which included both assessment components (the Washington Assessment of Student Learning, or WASL) and standards (the Essential Academic Learning Requirements, or EALRs). Focusing on the writing subject (WASL), they surveyed teachers on the changes in their practices which were related to the reform, such as teaching content, teaching strategies, and instruction time allocation. The teachers’ responses showed a clear influence by both the assessment component (WASL) and the standards (EARLs) on many aspects of classroom practices. A greater percentage of teachers thought the WASL was influential in organizing their classroom instruction, and in making changes in writing programs at school. Teachers increased instruction time on test subjects and decreased it on non-tested subjects. Additionally, their writing instruction covered most of the writing specifications in the standards, but more frequently covered the writing requirements (e.g., writing conventions, writing process), and increased the use of the rubric-based approach to model WASL rubrics. Despite these reported changes in teachers’ classroom practices, students’ higher WASL scores were found not to be associated with the practices aligned to the assessment, but rather with the alignment of the curriculum standards and with the teachers’ understanding of the reform. The survey evidence showed that the intended washback effects of the educational reform – promotion of students’ writing practices and increased effectiveness in their writing communication – were both achieved. However, spending less instruction time on non-tested subjects was clearly a sacrifice in order to achieve positive effects on writing, indicating that the assessments produced more washback effects than did the standards. This was one of the limited studies in this category using students’ test scores as one form of empirical evidence to examine washback effects. This study provided direct evidence regarding whether, and what kind of (e.g., positive or negative), washback effects on teachers’ pedagogy and beliefs are related to students’ test performance.
Cheng et al. (Reference Cheng, Klinger and Zheng2007) and Fox & Cheng (Reference Fox and Cheng2007) examined a provincial literacy test – the Ontario Secondary School Literacy Test (OSSLT) in Canada. Using students’ test scores for the quantitative data, Cheng et al. (Reference Cheng, Klinger and Zheng2007) investigated the test performance patterns of English as second language (ESL) and English Literacy Development (ELD) students and compared their outcomes to those of their native-speaking counterparts. The results revealed consistent and clear low performance across test formats and literacy skills (i.e., reading and writing) for ESL/ELD students. Factors of text type (e.g., narrative), skill (e.g., indirect understanding in reading), and strategy (vocabulary reading strategy) contributed to the distinction in test performances between ESL/ELD and their counterparts. These factors also explained the higher failure rate of the linguistic minority students compared to their English-native counterparts.
The study by Cheng et al. (Reference Cheng, Klinger and Zheng2007) raised concerns about the intended purpose of this test: to ensure that students have acquired essential reading and writing skills. It also concerns the validity of this test, whose blueprint was modelled on English as it is used by first language (L1) students, but which ESL students were required to take. Taking the qualitative approach of using accounts by both English-native and ESL test-takers in the OSSLT, Fox & Cheng (Reference Fox and Cheng2007) provided a close lens through which to examine and understand test-takers’ differing perceptions regarding what was assessed during the test, and what they were taught in class. L1 students thought the reading test tasks did not reflect the degree of reading ability that they used in the classroom; however, ESL students reported difficulty in undertaking reading tasks because they needed to employ various sources of information to understand the meaning. The difference in the intensity and extensiveness of test preparation between these two groups also indicated the different degrees of washback on students’ learning. These two studies, Cheng et al. (Reference Cheng, Klinger and Zheng2007) with quantitative evidence of language minority students’ test performance, and Fox & Cheng (Reference Fox and Cheng2007) with qualitative accounts of their perceptions and interpretations of test task and skill requirements, clearly presented the disadvantaged position the test design had given the language minority students. This test was intended to ensure that students, including language minority students, reached the required literacy level. However, the difference in test performance between native English-speaking students and language minority students, as well as discrepancies in their understanding of the test tasks, raised questions regarding this test's intended washback effects.
Menken (Reference Menken2006) focused on the washback effects of policy-driven high-stakes assessment on English language learners. Under the pressure of high-stakes consequences of English language assessments (e.g., the criteria for high school graduation), teachers and students interviewed and observed in this study stated there was much emphasis on ‘teaching to the test’. It was found that the curriculum and the instruction for English language learners (ELL) were driven by the tests, rather than aligned to the students’ specific needs. In turn, it was doubtful that the instruction that these students received and that was directed towards the requirements of the test not their learning needs had a positive effect on their English language learning. Similarly, Scott's (Reference Scott2005) study also found a potential tension between preparing English as an additional language (EAL) learners to perform their best on high-stakes, statutory national curriculum tests, and focusing classroom practices on the promotion and support of their language development. Similarly to Fox & Cheng (Reference Fox and Cheng2007) and Cheng et al. (Reference Cheng, Klinger and Zheng2007), this study raised the concern that English language minority students (e.g., ESL learners and ELL) were disadvantaged when taking high-stakes, mandatory assessments that are designed for native English-speaking students.
Winke's (Reference Winke2011) survey study evaluated educators’ (teachers, principals, and school administrators) perceived effectiveness of the English Language Proficiency Assessment (ELPA) as evidence of its validity. The ELPA is a test used in the state of Michigan in the USA to fulfill the government-mandated No Child Left Behind (NCLB) requirements. The test is used to promote and monitor achievement in English language learning in schools which receive federal NCLB funding. This study found that the ELPA both positively and negatively influenced the NCLB instruction context and the allocation of services and funds. On the one hand, the ELPA can increase funding and education opportunities for ELL by focusing attention on them. On the other hand, administering the test kept experienced educators out of the classroom for as long as several weeks.
In summary, the empirical studies reviewed in Category 2 showed how large-scale assessments influenced language minority students and those working with them (e.g., teachers and school administrators). It has been revealed in these studies that large-scale assessments affected both how language minority students learned the tested language (e.g., Fox & Cheng Reference Fox and Cheng2007) and how teachers organised and implemented their instruction to language minority students (e.g., Stecher et al. Reference Stecher, Chun and Barron2004), even though some large-scale assessments in these studies were intended as an educational tool to monitor and improve minority students’ learning.
5.3 Category 3: Systemic effects of testing programs on educational institutions
This section reviews empirical washback studies focusing on how washback operates within the educational system, and in relation to society at large. According to Cheng (2008), serious discussions on the impact of testing in language education began in the early 1990s. The catalyst for these was Alderson & Wall's (Reference Alderson and Wall1993) article ‘Does washback exist?’ The authors suggested, through a series of ‘washback hypotheses’, which areas could be influenced by tests, and called for empirical studies in language testing to investigate washback in these areas. Over the past two decades, empirical studies investigating the systemic effects of language testing in education have proliferated. The largest number of studies in this literature review is on the washback phenomenon (44 of 87 empirical studies). Findings provide evidence regarding how and to what extent tests affect the participants, processes, and products in the washback mechanism as illustrated in Bailey's (Reference Bailey1996) washback model. These findings shed light on the role that language tests play with respect to factors related to the test, the participants, and the context. They highlight the complexity of the washback phenomenon in the complicated and dynamic educational context, as well as the methodological challenges encountered when doing washback research. Due to the large number, we have divided this section of our current review into two sub-categories: washback on teaching and washback on learning.
5.3.1 Washback on teaching
A response to Alderson & Wall's (Reference Alderson and Wall1993) call was a special issue on washback in the journal Language Testing (edited by Alderson & Wall Reference Alderson and Wall1996). It opened with Messick's (Reference Messick1996) discussion of washback within the broader framework of construct validity and Bailey's (Reference Bailey1996) review of the washback literature. Messick (Reference Messick1996) argued that, given the complexity of testing consequences in general and washback in particular, neither of them can stand alone as a standard of validity. He suggested from the test designers’ perspective that positive washback can be facilitated, and negative washback can be reduced, through improvements in test design, particularly by minimizing construct under-representation and construct-irrelevant variance. Bailey (Reference Bailey1996) made the distinction between washback to the learner and washback to the program, based on her review of the washback literature and referring to Hughes’ (Reference Hughes1993) trichotomy (i.e., participants, process, and product) and Alderson & Wall's (Reference Alderson and Wall1993) 15 washback hypotheses. Washback to the learner involves the effects of testing on various aspects of students’ learning. Washback to the program is concerned with different aspects of teaching and involves various participants (e.g., teachers, educational administrators, course designers, and material developers), whose work is influenced by the test, which ultimately affects the courses, programs and materials students receive.
Three articles in the special issue of Language Testing reported empirical washback studies conducted in different parts of the world. These studies all focused on washback to the program, especially washback on teaching. Two qualitative studies (Alderson & Hamp-Lyons Reference Alderson and Hamp-Lyons1996; Watanabe Reference Watanabe1996, which was also reported in Watanabe Reference Watanabe2004) investigated the role of teacher factors in washback by collecting interview and classroom observation data. Alderson & Hamp-Lyons’ case study focused on the effects of the TOEFL test by comparing TOEFL preparation classes and parallel non-TOEFL ‘normal’ language proficiency classes in the same institution in the USA. Watanabe (Reference Watanabe1996, Reference Watanabe2004) focused on the English language component of Japanese university entrance examinations, which were claimed to dominate the whole educational process in Japan, including the practice of English as a foreign language (EFL) education. Despite the difference in educational context, both studies concluded that it is at least as much the teacher who brings about washback, be it positive or negative, as it is the test.
Shohamy, Donitsa-Schmidt & Ferman's (Reference Shohamy, Donitsa-Schmidt and Ferman1996) study was situated in the broader educational and social contexts. They investigated the impact over time of two national tests in Israel – one on Arabic as a second language (ASL) and the other on EFL – from the perspectives of teachers, students, and language inspectors. Their study highlighted different washback patterns for these tests. Modifications in the ASL test produced no washback whereas modifications in the EFL test created major impact in terms of teaching activities, time devoted for test preparation, and production of new teaching materials. The researchers challenged the bureaucrats’ belief that both tests had reached their objectives with no need for teacher training or curricular revision. The study concludes that washback changes with time because of factors such as language status and test uses in society (see also Ferman Reference Ferman, Cheng, Watanabe and Curtis2004).
The three studies described highlight the complexity of washback by demonstrating how it takes place within complex dynamic educational systems and social contexts. The complicated interactions between the factors that make up the micro-level teaching/learning context and the macro-level factors beyond the classroom make it difficult to attribute impact to any one factor alone, such as the test. As noted by Messick (Reference Messick1996: 242): ‘A poor test may be associated with positive effects and a good test with negative effects because of other things that are done or not done in the education system’. Considering this complexity, it is necessary for washback researchers to expand their research scope and methodologies and to draw insights from research in other fields, such as innovation and motivation, to better understand the role of test participants as stakeholders. A significant implication that can be drawn from these three studies is that in order to induce beneficial washback from the examination, different types of problems need to be solved at the individual, school, and societal levels – a point also voiced by Wall (Reference Wall1999). The collection of washback research in this Language Testing special issue represents an important milestone in discussions of consequential validity and test impact (Alderson & Wall Reference Alderson and Wall1996), and its influences on washback research are still felt today.
Washback on language teaching continues to receive attention from language testing researchers. Empirical washback studies have investigated the effects of testing on different aspects of teaching, such as content of teaching, teaching methods, and assessment methods used by teachers, as well as overall teaching styles. These studies focused on both national and international large-scale high-stakes tests. Their general purpose was to describe and explain the effects of particular tests and testing policies on teaching in relation to test-related and teacher factors, as well as micro- and macro-level contextual factors (Shih Reference Shih2009).
Washback of large-scale high-stakes tests draws attention from both test developers and language testing researchers due to the impact of such tests on education and society. For example, research into washback from the International English Language Testing System (IELTS) (e.g., Read & Hayes Reference Read and Hayes2003; Hayes & Read Reference Hayes, Read, Cheng, Watanabe and Curtis2004; Saville & Hawkey Reference Saville and Hawkey2004; Hawkey Reference Hawkey2006; Mickan & Motteram Reference Mickan and Motteram2008, Reference Mickan and Motteram2009) has provided insight into the complexity of the educational system and test use, especially in relation to the role of the teacher (see also Gibson & Swan Reference Gibson and Swan2008).
Read & Hayes reported a two-phased investigation of the impact of the IELTS on test preparation classes in New Zealand (see Hayes Reference Hayes2003; Read & Hayes Reference Read and Hayes2003; Hayes & Read Reference Hayes, Read, Cheng, Watanabe and Curtis2004). In the first phase of the survey, they found that IELTS preparation was structured in three main ways across the country: (1) as an independent, part-time course that was relatively short; (2) as an optional component of a full-time general English program; and (3) as an integrated component of an extended full-time course in English for academic purposes (EAP).
The second phase of the study, using classroom observation, focused specifically on two IELTS preparation courses offered through the language schools of two public institutions. Both courses were aimed at students preparing for the academic module of the IELTS. The course offered at School A was an independent IELTS preparation course, which focused almost entirely on skills needed in the examination, and provided practice on the test's various components. The course offered at School B was part of the general English course aimed to develop general language and academic English skills, as well as to familiarize students with the test.
The study revealed a number of substantial differences between the two courses. The most obvious was ‘who had control of the lessons’ (Hayes & Read Reference Hayes, Read, Cheng, Watanabe and Curtis2004: 104). The teacher at School A was the predominant focus of the class and spent much more class time teaching about effective strategies to use in the test. The teacher at School B spent more class time assisting students both individually and in pairs or groups. In terms of content, the activities at School A were almost exclusively IELTS-like tasks, whereas a wider range of activities was introduced at School B, aiming at building up English language skills. Despite these differences, the pre- and post-test score analyses did not reveal a significant variation between the students in these two courses. Based on these findings, the study concluded that negative washback effects were evidenced in the IELTS preparation course at School A. However, the differences between the two courses may be attributable to their specific aims and structures rather than to the test per se. Therefore, to better understand and explain its effects on teaching, a comprehensive evaluation of the test preparation program is necessary.
In another observational study on the washback of IELTS on teaching, Mickan & Motteram (Reference Mickan and Motteram2008) went a step further by investigating closely the instructional tasks and the language of instruction in a test preparation class in Australia. Consistent with the findings of Hayes & Read (Reference Hayes, Read, Cheng, Watanabe and Curtis2004), this study found that pedagogy in the IELTS preparation class was teacher directed and test focused. Dominant activities were test practice, test skills-focused exercises, and explanations of the format and content of the IELTS modules and test-taking procedures. The instruction also focused on language awareness, particularly on the discourses and linguistic elements relevant to actually doing the test. The focus on language was integrated into the modelling of responses, reflections on test items and giving feedback on practice tests. However, Mickan & Motteram did not simply categorize these observations as negative washback of IELTS. Rather, they argued from the sociocultural perspective that the instruction in the IELTS preparation class constituted a process of socialization into test-taking behaviors and into the values or priorities embodied in the test.
A limitation of the IELTS-related observational studies reviewed here is that they did not incorporate a comparison with courses that were not directly related to the test. Without this comparison, it would be hard to evidentially link the observed practices to the test. To address this issue, Green (Reference Green2006b) conducted a series of classroom observations in the UK to explore how instruction varied between IELTS preparation classes and general EAP classes in academic writing. By comparing the test preparation classes and the non-test EAP classes, ‘evidence was found for substantial areas of common practice between IELTS and other forms of EAP, but also for some narrowing of focus in IELTS preparation classes that could be traced to test design features’ (Green Reference Green2006b: 333). Consistent with the findings of Alderson & Hamp-Lyons’ (Reference Alderson and Wall1996) study in the TOEFL context, Green's study also provided evidence that teacher variables may encourage practices that cannot be traced to test design features of IELTS. Therefore, Green (Reference Green2006b: 363‒364) noted ‘many of the differences observed between classes might be linked rather to teacher or institutional variables, such as levels of professional training and beliefs about effective learning, than to the influence of the test’. This conclusion is also supported by the findings of Huang's (Reference Huang2009) study on the washback of the English oral test component of the Senior Secondary School Entrance Examination in China.
Studies of curriculum and assessment reform in language teaching contexts show that centrally driven reform of the existing large-scale testing is likely to have a range of unintended consequences. For example, the Ministry of Education of China introduced the National Matriculation English Test (NMET) in order to bring about a shift in language pedagogy from knowledge-based instruction to more communicative practices in the existing NMET. The NMET is a high-stakes test that is used for university entry. It is also designed specifically to promote change in English language teaching in schools.
Qi (Reference Qi, Cheng, Watanabe and Curtis2004, Reference Qi2005) investigated NMET's washback effects in the Chinese secondary education context with a large-scale study, using questionnaires and interviews with teachers, students, test constructors and school inspectors. She found that the selection function of the test ‘compels teachers and students to work for the immediate goal of raising scores’ (Qi Reference Qi2005: 163). Instead of focusing on language use, teachers tended to focus on linguistic knowledge, which many of them interpreted as the test construct, contrary to the test designers’ intentions. Teachers also spent a large amount of time on explicit test preparation activities such as teaching to the test content and administering mock tests. Qi (Reference Qi2005) attributed the failure of the NMET to bring about the intended changes or washback effects to its dual functions: selection and promoting change. She argued that these two functions are in conflict with each other, making the test a powerful trigger for teaching to the test, but an ineffective agent for changing teaching and learning in the way intended by its designers and the policy-makers.
Qi's (Reference Qi2005) argument is consistent with Shohamy et al.'s (Reference Shohamy, Donitsa-Schmidt and Ferman1996) conclusion that washback is related to the use and power of the test. While Qi (Reference Qi2005) and Shohamy et al. (Reference Shohamy, Donitsa-Schmidt and Ferman1996) explained impact in terms of educational and societal contexts, other researchers investigated consequences of language assessment used as a policy instrument, drawing on insights from educational innovation theory, with a particular focus on teacher factors.
Two studies (Turner Reference Turner2006; Wang Reference Wang2011) focused on the perspectives of teachers as they experienced curriculum innovations introduced into the educational system via large-scale, high-stakes tests. Turner's (Reference Turner2006) study was a survey with secondary school ESL teachers in Quebec, Canada. It attempted to explore the perspectives or beliefs of teachers when a change in the educational system was introduced via a speaking test. Unlike most other studies, which generally reported negative washback of external tests from teachers’ perspectives, Turner (Reference Turner2006) revealed that teachers integrated changes into their teaching or assessment practice according to their own beliefs and professional stances. This study provides insight into the nature of how curriculum, teaching, and assessment could move toward synchronization when changes are introduced through a high-stakes examination. In the process, teachers play a significant role in moving the system into a position where curriculum, their teaching and assessment, and the system's high-stakes exam correspond.
Wang's (Reference Wang2011) study was situated in the Chinese tertiary educational context. It investigated the extent to which educational change carried out in the College English curriculum and the College English Test (CET) resulted in teacher change in terms of pedagogical strategies. In contrast with Turner's (Reference Turner2006) findings, Wang (Reference Wang2011) provided ample evidence of teachers’ resistance to the intended change by the policy-makers, as well as the teachers’ negative perceptions of the CET. It also revealed that the CET neither constitutes a facilitator nor poses a constraint on instructional innovation. Based on these findings, the researcher argued that in order for fundamental changes in teaching practice to occur, changes in teachers’ knowledge, beliefs, attitudes, and thinking that inform their teaching practices are essential. To achieve this purpose, enhanced teacher training is necessary to convince EFL teachers of the purpose and benefits of tests, and to help them maintain a positive attitude toward their role in the educational system. This necessity was also highlighted by Chen's (Reference Chen2002) study investigating the washback of a reformed public examination called the Basic Competency Test (BCT) in the high school context in Taiwan. This study found that the BCT influenced what the teachers taught, but not how they taught, because of the lack of extensive professional development opportunities.
5.3.2 Washback on learning
Cheng (Reference Cheng, Shohamy and Hornberger2008) noted that compared with the wealth of washback studies on teachers and teaching, it was not until recently that empirical research studies into washback on learning and learners began to appear in large numbers. Gosa's (Reference Gosa2004) research into IELTS washback in the Romanian context was an early study that focused on students’ expectations of assessment. Her research revealed that students’ expectation of assessment was the single most important factor in explaining their perspectives on teaching and learning activities. It influenced not only their attitudes towards teaching but also their own learning.
Several IELTS impact studies have since compared learners’ perspectives and teachers’ perspectives with the focus on the interaction between washback on teaching and washback on learning. For example, Hawkey (Reference Hawkey2006) conducted a classroom observation study to investigate IELTS impact in the test preparation classes. The study collected questionnaire data regarding test preparation activities reported by both teachers and students. Results of the classroom observations showed that teachers were willing to employ a variety of teaching methods and to use material both within and beyond the textbook. However, the teachers reported that their teaching activities were constrained, both by the institutions providing the courses and by the students who were motivated to succeed in the test, such that the teachers focused their instruction on IELTS. Consequently, teachers showed a preference for task-based activities in class, targeting micro-skills they believed to be relevant to the test. Comparisons of the questionnaire data between students and teachers revealed some interesting differences. The teachers were generally more likely to indicate that activities (whether test related or not) occurred in their classes than did the learners. Such differences raise questions about the extent to which washback to teachers can be assumed to be the same as washback to learners.
Latimer's (Reference Latimer2009) ethnographic study was situated in an English language program at one Argentinean bilingual school. At the school, the broad-based K-6 bilingual curriculum was highly effective in terms of promoting students’ communicative abilities, comfort in the second language (L2), and eagerness to use the language for academic and social purposes. Beginning in grade 7, when the Cambridge ESOL examination preparation became the program goal and curriculum, most students failed to progress in the sorts of communicative and analytical language abilities they would need in future higher education and professional settings. Latimer (Reference Latimer2009) explained the negative effects of the Cambridge exams in terms of the interactions between curriculum, teaching, and learning, showing that the exams provided a structure for curriculum organization, and motivated both teachers and students. The preparatory texts used at the school, however, restricted classroom interaction and limited extended output from students.
To better understand the interaction between washback to students and washback to teachers, Green (Reference Green2006a) used questionnaires to explore the influence of teachers’ priorities on learners preparing for the IELTS Academic Writing Module. Participants in this study included students and teachers in both IELTS preparation courses and non-IELTS EAP courses. Data were collected regarding expectations students brought to their courses on entry, students’ retrospective perceptions of course focus at exit, and teachers’ perceptions of the focus of their courses.
Results were divided into three groups and compared: among IELTS preparation students, non-IELTS students and teachers; between the students at course entry and at course exit; and between the students and the teachers. Consistent with the findings from other IELTS washback studies (e.g., Read & Hayes Reference Read and Hayes2003; Hayes & Read Reference Hayes, Read, Cheng, Watanabe and Curtis2004; Hawkey Reference Hawkey2006) the comparisons here indicated that test preparation courses covered a relatively narrow range of skills. However, inconsistent with previous results, Green (Reference Green2006a) found that narrow preparation strategies were not driven primarily by students’ expectations. Students did not directly shape washback on the teachers. In contrast, teachers and courses were found to be influential in shaping washback on students, i.e., students’ perceptions of course outcomes were affected by the course focus reported by teachers.
The ultimate purpose of language education is to promote students’ language proficiency. Therefore, the effects of language testing on learning outcomes have begun to receive growing attention from language assessment researchers. Two studies focus on the influence of the test preparation program on students’ IELTS performance, but draw different conclusions. In an earlier study in Fiji, where students’ IELTS performance was consistently poor, Rao, McPherson, Chan & Khan (Reference Rao, McPherson, Chand and Khan2003) compared students’ performance before and after a 30-hour IELTS preparation program and found significant gains in their IELTS writing module scores, but no significant gains were found in their performance on the IELTS reading module. The researchers highlighted the influence of several factors such as time, motivation, anxiety, and the nature of the skill itself on the students’ performance.
In a more recent study, Green (Reference Green2007b) focused on the effects of IELTS preparation courses on assisting students in improving their test scores. Participants in this study included three sub-groups: students attending the IELTS preparation courses, students in the pre-sessional EAP courses (with no IELTS component), and students in combination courses (pre-sessional EAP courses with IELTS preparation strands). All participants were asked to take the IELTS grammar/vocabulary tests at the beginning and again at the end of their 4-to-14-week courses. In addition to the test scores, data relating to differences in participants and learning practices were also collected. Analysis of the data revealed that score gains were primarily between two groups of learners: those who planned to take the test again, and those who had low initial writing test scores. Narrow test preparation did not necessarily enhance test scores more effectively than did wider-ranging alternative preparations. Green (Reference Green2007b) concluded that ‘washback to the learner (possibly in the form of motivation to succeed) had more to do with the improvement than did washback to the programme. Individual learners’ goals and their understanding of the test's demand influenced their learning outcomes to a greater extent than did their choice of course and its content’.
Green's (Reference Green2007b) conclusion that test preparation does not contribute much to improving performance is supported by Perrone's (reported in Perrone Reference Perrone2010, Reference Perrone2011) investigation into the notions of washback and classroom-based assessment within the context of the First Certificate of English (FCE) examination in the UK. This study found that students’ mean scores on the FCE in the FCE preparation course and the general EFL course were not significantly different, suggesting that classroom methodology may have had a limited impact on performance. However, results from the study suggested that the classroom assessment task had a differential impact on the accuracy and speed with which the individual learners processed the new target language forms. Therefore, Perrone concluded, while there are limited differences in student test performance between the test preparation and the general EFL classes, there is a significant difference in student learning/processing at the individual level, attributable to the classroom assessment practices they were engaged in.
More recent washback studies that focused on students’ or test-takers’/candidates’ perspectives have attempted to explain washback in terms of various factors related to the student and to the context. For example, Murray, Riazi & Cross (Reference Murray, Riazi and Cross2012) focused on the role test-taker attitudes (operationalized in terms of opinions, beliefs, and emotions) played in washback. The participants in this study were a group of overseas-trained teachers preparing for a professional gate-keeping test in Australia. This study linked the candidates’ attitudes to the test to their demographic and experiential backgrounds. Results of this study indicated negative attitudes towards the test, which were associated with personal experiences, particularly the experience of failure in the test, as well as the impact of other people: notably teachers and other test-takers. Cheng & Deluca (Reference Cheng and DeLuca2011) examined 59 test-takers’ perspectives in writing large-scale English language tests in Asia. Both negative and positive experiences were collected through these test-takers’ previous test-taking experiences. Their findings offer test-takers’ voices on fundamental aspects of language assessment and demonstrated the need for obtaining additional evidence from test-takers for validating large-scale language tests. Virkiru (Reference Virkiru2011) carried out a study in Kenya on strengthening the development of literacy in English among primary school children. This study found that learners performed poorly on skills that were not directly tested in the conventional examinations.
Cheng, Andrews & Yu’s (Reference Cheng, Andrews and Yu2011) study on the impact of school-based assessment in Hong Kong reported that students with higher self-rated English proficiency performed more examination-related activities inside and outside the classroom, and that the intensity of their preparation increased as the examination date drew nearer. These findings are supported by two studies on the washback of the CET on students and their learning in the Chinese tertiary educational context (one reported in Xie Reference Xie2010, and Xie & Andrews Reference Xie and Andrews2012; the other in Zhan & Wan Reference Zhan and Wan2013).
Drawing on expectancy-value motivation theory (Wigfield Reference Wigfield1994; Wigfield & Eccles Reference Wigfield and Eccles2000), Xie & Andrews (Reference Xie and Andrews2012) proposed a model of CET washback on test preparation to explain the paths of influences, from perceptions of test design and uses to test preparation. This model hypothesizes that these perceptions influence test preparation both directly and via two motivation factors: expectancy and value. The results of this study indicate that both perceptions of test design and high-stakes test uses influenced preparation simultaneously. Evaluation of test importance and value explains the influence observed by the perception of test uses to preparation. Consistent with findings by Cheng, Andrews & Yu (Reference Cheng, Andrews and Yu2011), this study demonstrated that students with higher self-efficacy were engaged in more examination preparation activities. Xie (Reference Xie2010) included comparison of pre- and post-test performance data in the model and found that substantial improvement was achieved; however, test preparation contributed only a little to this improvement.
Zhan & Wan (Reference Zhan and Wan2013) examined how the revised CET influenced Chinese non-English major undergraduates’ out-of-class English learning practices by following two informants in one university over time. Consistent with Cheng et al.'s (Reference Cheng, Andrews and Yu2011) finding, the qualitative data analysis of this study showed that the washback effects from the CET on the two informants’ out-of-class English learning practices seemed to vary across time, and tended to increase as the test approached.
Drawing on insights from sociocultural theories, Booth (Reference Booth2012) conducted a longitudinal study in South Korea to explore the washback on learning as a socially situated process, focusing on the TOEIC, a high-stakes test for Korean university students. Triangulating data collected through semi-structured journals and multiple interviews with students, the study highlighted the important contribution of situated learner goals and actions to the effects of the TOEIC on learning. This research provided evidence for the significant roles of learner factors in directing the washback effect of the test. Thus, Booth's (Reference Booth2012) findings support the view that washback is a complex system, where a range of factors may mediate the influence that a test has on learning (Alderson & Wall Reference Alderson and Wall1993; Messick Reference Messick1996; Cheng & Curtis 2004; Shih Reference Shih2007; Cheng Reference Cheng, Shohamy and Hornberger2008), therefore he suggests a shift in the direction of washback research towards a more socially situated view.
To sum up, students and learning are at the heart of educational assessment. However, it was not until recently that researchers’ attention began to turn to washback on students and their learning. Recent studies have focused on the direct influence of testing on students (e.g., their perceptions, their strategy use, motivation, anxiety, and affect) and on their learning (e.g., what and how they learn or how they perform on a test). The findings of these studies highlight the necessity for investigations of washback on students and their learning in relation to the test construct and the broader sociocultural context where the construct is interpreted and used. For example, in a recent study on the impact of commercial international language tests on young learners in Hong Kong, Chik & Besser (Reference Chik and Besser2011) included perspectives of stakeholders from the education and commercial sectors, These stakeholders’ interpretations and uses of the tests contribute to the impact of international language testing among children in Hong Kong. Linking findings concerning washback on students and their learning – learning outcomes in particular – to implications regarding construct validity as well as context will shed light on what brings about washback and how positive washback should be induced and negative washback reduced for the benefit of students.
5.4 Washback within the classroom assessment context
In addition to the washback of external high-stakes tests, the influence of classroom-based assessments (including internal testing) on students and their learning has also begun to receive attention from washback researchers. We found only four related research studies within the search period of the current review. One such study by Fournier-Kowalesk (Reference Fournier-Kowaleski2005) focused on the washback of a low-stakes intermediate Spanish as a foreign language testing program within a large university program in the USA. The study found that teachers changed their instructional habits because they wanted their students to perform well on a test even though the test was not associated with high-stakes decisions. Teacher beliefs, as well as their previous learning and testing experiences, influenced their behaviors deeply no matter how the curriculum, course objectives, and the testing program worked together. Fournier-Kowaleski therefore supported the notion that in order to affect the instruction that occurs in the classroom and to create positive washback, the relevant direct guidance and training must be part of a teacher's professional development. This echoes the conclusion arrived at by Cheng (Reference Cheng2005) in that bringing about changes to teaching has to start with teacher education.
Weigle (Reference Weigle2004) examined the washback effect of a locally developed English competence test for non-native English speakers. The students who failed this test were required to take a remedial writing course until they passed it. The purpose of the course was to prepare students for the test. Weigle interviewed the course instructors, and found they refocused their instruction towards building skills that were required in broader English academic setting. The methodology changes were concluded to be the positive washback effects of this locally developed test intended to improve non-English speakers’ learning.
Hung (Reference Hung2012) implemented an e-portfolio project as an alternative assessment technique in an EFL teacher preparation course, and explored the positive and negative washback effects this assessment type produced on learning. Analyses of the interviews, observations, documents, and reflective journals suggested that e-portfolio assessment generated both positive and negative washback effects on learning. The positive washback effects included building a community of practice, facilitating peer learning, enhancing learning of content knowledge, promoting professional development, and cultivating critical thinking. The negative washback effects included learning anxiety and resistance to technology.
S. Huang (Reference Huang2011) compared two types of classroom assessment – convergent assessment (CA) and divergent assessment (DA) – in terms of their influence on learners’ motivation and learning strategies in a Taiwanese university. According to Torrance & Pryor's (Reference Torrance and Pryor2001) distinction, the CA, with its behaviorist theoretical origin, is an assessment of the learner by the teacher (assessor) with the purpose of finding out if the learner knows, understands or can perform a predetermined concept specified in the curriculum. CA is characterized by detailed planning and generally accomplished by closed or pseudo-open questioning and tasks. The DA, with its constructivist theoretical origin, is an assessment accomplished jointly by the teacher and the student with the purpose of discovering what the learner knows, understands, and can do. It is characterized by less detailed planning, where open questions and tasks are of greater relevance. S. Huang (Reference Huang2011) examined students’ reactions to the CA and DA, as well as to their task-specific motivation and learning strategies, after they experienced these two types of assessment in their EFL listening and speaking classes. Results of this study indicated that the CA was better received by students with higher self-efficacy and the DA was better received by lower-achievement students. After controlling for self-efficacy, it was found that students’ reactions to these types of assessment were different in the listening and speaking classes. In the speaking class, students’ motivation and strategy were higher for the DA, whereas in the listening class their motivations and strategies were higher for the CA.
These studies on low-stakes classroom-based assessment, albeit limited in number, demonstrate the existence and functionality of washback beyond the context of large-scale high-stakes testing. Researchers (e.g., Brookhart Reference Brookhart2004; McMillan, Hellsten & Klinger Reference McMillan, Hellsten and Klinger2010) suggest that more quality, classroom-based formative assessments be used in order to lessen the unintended consequences of large-scale high-stakes testing, However, the aforementioned reviews indicate that the effects of classroom-based assessment seem to function in a similar way to those of large-scale testing. This is an area that deserves further research. It is beyond the scope of this review article to draw any tangible conclusions, given the limited number of studies found.
6. Future directions
Reviewing this large body of washback studies over a period of 20 years between 1993 and 2013 was a daunting task. Using Kane's (Reference Kane and Brennan2006, Reference Kane2013) framework to put these studies into three analytic categories has been fruitful because this process sharpened our understanding of the nature of the tests, the testing contexts, and the stakeholders being studied. Kane (Reference Kane2013) asserts that decisions based on accurate test score interpretations generally should induce intended positive outcomes while decisions based on inaccurate interpretations would be unacceptable ethically, legally, and socially, and tend to bring about adverse consequences. The exploration of these contributing factors (the test, the testing contexts, and the stakeholders) will help understand how these ‘building blocks’ (e.g., test constructs, test uses) interplay with one another to bring washback to different groups of stakeholders.
Evidence concerning washback needs to be linked to other evidence of factors related to test validity. According to Kane (Reference Kane2013), the decision inference is the capstone of the IUA, with the other inferences developed to support it. Therefore, in order to investigate washback in light of various factors influencing test validity, future studies need to identify and/or associate these factors that bring about certain washback effects. For example, the investigation could trace the association between positive washback effects and test developers’ efforts as support of other inferences in the IUA. Meanwhile, unintended negative washback should trigger investigations into sources of invalidity, in terms of construct underrepresentation and construct-irrelevant variances (Messick Reference Messick1996). The synthesis of this review of empirical washback studies provides the following directions for future research: (1) inclusiveness of stakeholders, (2) mechanism of interplaying factors, and (3) insights from related fields and theories.
6.1 Inclusiveness of stakeholders
Studies on using testing as a lever for change to teaching and learning, as well as those on the systemic effects of testing on educational institutions, have mostly examined washback on teaching, predominantly focusing on the process of teachers implementing and/or adjusting pedagogy. To be specific, the large body of literature has presented multiple layers of washback effects related to teacher factors (e.g., beliefs, professional training, and sociocultural and linguistic backgrounds). These studies took teachers’ perspectives in order to unpack how test factors interacted with a variety of teacher factors to bring about washback on teaching.
Washback on students and their learning has recently received attention from washback researchers for two reasons. First, as shown by studies in Category 2 of this review, large-scale high-stakes assessments have been increasingly used worldwide and students are the most direct recipients of the consequences of these assessments. Particularly, unintended negative consequences on linguistic minority students have received increasing attention in recent washback studies. Second, the ultimate goal of educational assessment is to promote learning. Washback on learners’ learning and particularly learning outcomes was studied less often in early research.
Although Bailey (Reference Bailey1996) included multiple agents (e.g., teachers, learners, test material publishers) as participants of washback mechanisms in her washback model, a large number of the empirical studies reviewed in this article still focused on teachers only. Washback, particularly on student learning outcomes, should receive attention in the future. Developments in research on learning in light of social cognitive, sociocultural and self-regulated leaning theories would also provide new insights. Unpacking the complexity of washback on students’ learning outcomes by investigating learning, including test preparation, as a mediated activity situated in context where different perspectives, perceptions and practices interact would be particularly useful. For this purpose, the scope of washback research needs to be expanded to include more groups of participants/stakeholders such as parents and principals (e.g., Babcock Reference Babcock2006), and by linking student data and data from other sources together (e.g., Cheng et al. Reference Cheng, Andrews and Yu2011). Bringing together multiple groups of stakeholders in washback can add dimensionality to the interwoven washback effects, such as the discrepancy between teachers and students in terms of their perceptions of test preparation revealed in Alderson & Hamp-Lyons’ (Reference Alderson and Hamp-Lyons1996) study in the TOEFL context. It is high time for more empirical studies involving all stakeholders in washback to explore how washback operates and functions as a comprehensive and interwoven entity (e.g. Qi Reference Qi, Cheng, Watanabe and Curtis2004, Reference Qi2005; Cheng Reference Cheng2005; Green Reference Green2007a, Reference Green2007b).
6.2 Mechanism of interplaying factors
Washback studies over the past few decades have revealed the complexity of the notion. This lies in the function of numerous factors related to the nature of the test (e.g., test constructs, test design, and test formats), educational policy concerning test uses (e.g., retention, graduation, and admission), and social context (e.g., the country, immigration policy) where the test is implemented. Future research should continue to unpack the complexity of washback by exploring the roles of different contributing factors within the large-scale and classroom-based assessment contexts. Future studies might carefully examine these factors, such as the constructs built into a test, and the intended and unintended test uses, because consequences are associated with test use decisions, which are, in turn, based on test score interpretations.
Further, future studies will need to understand the social and educational setting for which the test is designed and administered. This is particularly applicable to studies in Category 2, where the research focus is on the consequences of testing related to language minority students. With the global pattern of increasing immigration and internationalization, the consequences of testing on language minority students are becoming an increasingly critical issue in many countries and many levels of educational institutions. Only two studies (Cheng et al. Reference Cheng, Klinger and Zheng2007; Fox & Cheng Reference Fox and Cheng2007) that we have reviewed took the perspectives of these language minority students and collected data directly from their test performance (e.g., their test scores on OSSLT) or their test-taking experience (e.g., their account of what was assessed in the provincial literacy test). The results provided a direct source and inner context to clearly understand how a large-scale assessment with high-stakes consequences influences language minority students. More evidence, directly from the perspective of such students, might serve two purposes: (1) to further clarify whether the effectiveness of large-scale assessment in determining whether or not language minority students are meeting the required levels of academic skills; and (2) to expand the understanding of washback effects related to these students and embedded in the corresponding sociocultural environment. For these purposes, future research needs to investigate washback as a dimension of impact, and explore the relationship between washback and other forces operative in the educational context. This entails value judgment about the local educational context as well as the greater social, political, and economic factors governing teaching and learning in relation to a testing program.
6.3 Insights from related fields and theories
Washback studies generally focus on unintended and indirect consequences of testing. Indirect consequences of testing involve the influences of testing on stakeholders’ actions, attitudes, and outcomes. They are usually the function of multiple factors related to the test, context and participants. It is true that investigations on washback are never straightforward because of the involvement of multiple factors. For example, while there is evidence that, when faced with an examination, students tend to practice for that examination, it cannot be taken for granted that where there is a test, there is a direct effect on learning. In light of the socio-cognitive view of learning, students’ beliefs and expectations of the test, as well as the micro- and macro-level contextual factors, all play important roles in bringing about both positive and negative washback effects. Therefore, insights and collaborations from related fields (e.g., educational policy, educational psychology) and theories (e.g., self-regulated learning) have become very necessary for future studies in order to map out a structure of how washback functions in particular contexts. And in a practical way, stakeholders such as test developers, educators, and policy-makers need to work collaboratively to bring about the benefits of assessment to students and their learning, and to minimize its potentially negative consequences. Unintended consequences are varied, unplanned, and often harder to anticipate. They may unfold over time, but before they can be investigated and weighed up, they must first be noticed or discovered. For example, studies in Category 2 in this review focused on the unintended negative effects of large-scale high-stakes testing programs on linguistic minority students, which unfolded particularly in educational contexts where there are increasing numbers of English language learners.
Consequences are associated with the results of test uses, which are, in turn, based on test core interpretations. Both test score interpretations and test uses are value laden and situated in political, social, cultural, and educational contexts. Future studies therefore need to link washback with the macro-level context where the research is conducted, and require washback researchers to collaborate with researchers in other fields such as policy analysis, innovation reforms, and socio-cognitive studies. Data from the same testing program may be interpreted and used by different stakeholders for different purposes, so future washback studies need to account for the interactions of the multiple interpretations and uses. To achieve this purpose, insights from program evaluation can be drawn to make value judgments about the testing effects from different perspectives.
7. Conclusion
This review of a large number of empirical studies reveals that teachers play a vital role in bringing about washback effects, be they positive or negative. Therefore, classroom teachers should be a fundamental component of any collaborative empirical investigations on washback. They play an important role in terms of pedagogical implementations of bringing about positive washback intended by test developers and policy-makers, and expected by students and parents (Lopez Reference Lopez2005; X. Huang Reference Huang2011). To make the collaborations feasible, classroom teachers could learn from these washback studies and (1) increase awareness of the existence of washback, (2) understand the factors that possibly influence washback, and (3) build up their knowledge of educational theories related to washback. First, teachers need to realize that assessment (either large scale, classroom based or policy driven) shapes the curriculum and pedagogy in certain ways, such as teaching-towards-test demands and practicing test-like items in class. In addition, understanding the conceptualization of washback (e.g., Bailey's 1996 model of participants, process and products of washback) will facilitate teachers’ further recognition of various factors that also contribute to washback. Second, teachers ought to be aware of those active agents that can influence their teaching and students’ learning. These could be how teachers understand and interpret policy and curriculum on which assessments are built and designed, whether and how teachers address students’ learning characteristics and learning needs, how students respond to teachers’ instructional activities, whether and how parents are involved in students’ learning practices, and equally important, which kind of sociocultural and educational context students are situated in. Knowing who the agents are and how they work in washback will enable teachers to seek targeted and specific resources to shape washback effects in a positive direction. Third, teachers need to build up and enrich their knowledge of educational theories related to an array of factors of washback. For example,the understanding of the affective domain of students’ learning styles (e.g., motivation, anxiety) in educational psychology can help teachers identify students’ learning characteristics and then adjust instructional activities to better match their characteristics; patterns of students’ responses to formative assessment or learning-oriented assessment in educational assessment may give teachers information about how to design classroom assessment activity to maximize students’ learning. All such incremental knowledge will increase classroom teachers’ capacity to induce positive washback effects.
Finally, washback and impact studies face methodological challenges as these studies are primarily about unintended and indirect consequences. To address these challenges, future studies will need collaboration across research fields and professions to determine how to bring about positive washback and how to reduce negative washback effects.
Liying Cheng Ph.D. is Professor at the Faculty of Education, Queen's University, Kingston, Ontario, Canada. Her research interests are the impact of large-scale testing on instruction, the relationships between assessment and instruction, and the academic and professional acculturation of international and immigrant students, workers and professionals to Canada. Her recent books are Language classroom assessment (single-authored, TESOL, 2013); English language assessment and the Chinese learner (co-edited with A. Curtis, Taylor & Francis, 2010); Language testing reconsidered (co-edited with J. Fox et. al., University of Ottawa Press, 2007); Changing language teaching through language testing (single-authored, Cambridge University Press, 2005); and Washback in language testing: Research contexts and methods (co-edited with Y. Watanabe and A. Curtis, Lawrence Erlbaum, 2004).
Youyi Sun is assessment consultant at Instruction, Curriculum, and Assessment Branch, Manitoba Education and Advanced Learning, Winnipeg, Canada, and a Ph.D. candidate in the Assessment and Evaluation Group, Faculty of Education, Queen's University, Canada. He has a Master's degree in applied linguistics (Carleton University, Ottawa, Canada). His primary research interests include validation and consequences (impact and washback) of large-scale testing and classroom-based assessment. He has recently published in The Canadian Journal of Applied Linguistics, Assessment in Education: Principles, Policy & Practice, and Language Assessment Quarterly.
Jia Ma is a Ph.D. student in the Assessment and Evaluation Group at the Faculty of Education, Queen's University, Canada. She received her M.Ed. from Queen's University. Her primary research interests are consequences of large-scale assessment and the interface between large-scale assessment and classroom-based assessment. Her Ph.D. dissertation examines washback on teaching and learning in the particular context of Chinese students’ test preparation for high-stakes English language tests. She has been involved in one three-year funded research project and has led a funded research project on washback effects of large-scale assessment. She recently co-authored a chapter with Dr Liying Cheng in TESOL Encyclopedia of English Language Teaching: Assessment and Evaluation.
