The Growth of Testing and Assessment Worldwide

There can be no doubt that testing and assessment are growing in importance and influence around the world today. Countries with established educational systems that already include public examinations are considering how these can be reengineered to reflect new approaches to teaching and learning as well as to meet the changing needs of contemporary society. Other nations are seeking to establish a national education system for the first time in their history, or to upgrade an existing system to extend educational opportunities to larger sections of their population. In such contexts there is often a policy imperative to design new assessment components into the educational infrastructure at local, regional, or national level in order to promote teaching and learning and to provide systems for accountability. In today's globalized environment there is sometimes an additional aspiration to align local or indigenous assessment frameworks with internationally accepted standards or recognized benchmarks. A further dimension of globalization over the past decade has been the unprecedented movement of vast numbers of people to and from various parts of the world. Though the reasons for this mass migration are complex and varied (including travel for tourism, access to educational or professional opportunities, economic or environmental migration, and conflict or political unrest), assessment tools such as skills-based competency tests and citizenship tests, as well as language tests, are being integrated into the sociopolitical and economic infrastructures that support and regulate these trends.

Against this wider backdrop, the testing and assessment of language skills (where the language is a second, third, or additional language) have undoubtedly moved from periphery to centre stage in recent years. Language tests, and the scores they generate, are increasingly used across contemporary society worldwide: within education, from primary age to higher education; in employment contexts, from the professional registration of health professionals to health and safety issues in the catering or construction industry; and, more controversially, in migration and citizenship policy and practice around the world. As Spolsky observed, “testing has become big business” (Spolsky, Reference Spolsky2008, p. 297), and not simply in commercial terms. Shohamy (Reference Shohamy, Shohamy and Hornberger2008) also noted “the societal role that language tests play, the power that they hold, and their central functions in education, politics and society” (p. xiv). The No Child Left Behind act in the United States (see Menken & Shohamy, Reference Menken and Shohamy2008), the Every Child Matters legislation in the United Kingdom, and the recent educational reform program in Hong Kong all testify to the central role that language assessment practices now occupy in society.

The Growing Numbers Involved in Language Assessment

These trends mean that there are not simply more people taking tests; in addition, growing numbers are directly involved in selecting or developing tests, and/or in using test scores for decision-making purposes. In practice, they often find themselves doing this without much background or training in assessment, or indeed in linguistics and language education, to equip them for the role. They include classroom teachers tasked with designing and/or delivering standardized tests to evaluate their pupils' progress, tests that are sometimes then used to hold teachers and schools accountable for that progress, or the apparent lack of it. A similar burden of expectation is laid on staff in university admissions, in professional registration bodies, and in immigration agencies to know what tests measure and what test scores mean, and to understand how to integrate them into their complex and often high-stakes decision-making processes. Public and policymaker expectations can be unrealistic about what a single score represents, as well as what information it does or does not provide, and this is exacerbated by a general lack of understanding about the imprecision or error inherent in any measurement. Spolsky (Reference Spolsky2008) complained that an “oversimplified view of the ease of producing meaningful measurement is held by the general public and educational administrators alike” (p. 300).

Applied linguists are among those regularly asked to help shape policy and practice in language planning, curriculum and syllabus design, and pedagogy and teacher education, and they are increasingly called upon to advise on developing or revising language assessment systems. This consultancy role tends to assume that they (along with classroom teachers, university admissions staff, and immigration officials) possess a sound understanding of the relevant theoretical and practical issues. This may be true in relation to language teaching and learning, but it is much less so for matters of testing and assessment. Language education programs at graduate level typically devote little time or attention to assessment theory and practice, perhaps just a short (often optional) module; and although there is no shortage of books on language testing and assessment available today, many of these are perceived to be (and often are) highly technical or too specialized for language educators seeking to understand basic principles and practice in assessment.

The Professionalization of Language Testing and Assessment

The field of language testing and assessment has itself undergone a process of increasing professionalization over recent years, in response to calls from both within and outside the field (Bachman, Reference Bachman1990). A growing raft of quality standards, ethical codes, and guidelines for good testing practice has been embraced or generated by language testers, and in many cases these touch upon matters that extend well beyond a test's purely technical qualities. Examples include the Code of Fair Testing Practices in Education (2004) and the AERA/APA/NCME Standards (1999), both of which concern assessment in general. There also exist language testing specific codes such as the ALTE Code of Practice (1994), the ILTA Code of Ethics (2001), the EALTA Guidelines for Good Practice (2006) and the ILTA Guidelines for Practice (2007). Professional associations of language testers and testing organizations were established during the 1990s, at national, regional, and international level, including the Japan Language Testing Association (JLTA), the Association of Language Testers in Europe (ALTE) and the International Language Testing Association (ILTA). At the same time, there has been growing awareness of the consequences of testing, intended and unintended, positive and negative, as illustrated by the burgeoning research literature on the theory and practice of language testing washback and impact (see, among others, Alderson & Wall, Reference Alderson and Wall1996; Cheng, Reference Cheng2005; Cheng, Watanabe, & Curtis, Reference Cheng, Watanabe and Curtis2004; Green, Reference Green2007; Hawkey, Reference Hawkey2006; Kunnan, Reference Kunnan and Kunnan2000; Wall, Reference Wall2005).

Kunnan (Reference Kunnan, Milanovic and Weir2004, Reference Kunnan, Taylor and Weir2008) reflected on the ethical milieu that has emerged for language testers, prompted by various factors in the professional field such as demands for accountability and responsiveness to clients, increased use of language tests and types of delivery methods, use of new measurement and other analytical techniques, and expanded concepts of validity (i.e., Messick, Reference Messick and Linn1989, Reference Messick1996). He suggested the language testing community has responded by developing explicit standards and codes for its community and practice, and that its professional ethic continues to evolve informed by the wider literature in ethics and moral philosophy.

One might imagine that the articulation of standards, ethical codes, and guidelines for good practice makes the business of designing and developing language tests more transparent, accessible, and manageable to anyone charged with this task, including those nonspecialist testing stakeholders mentioned above. To some extent this may be true, but it also risks promoting a view of language testing as rather technical and specialized–and best left to professional experts, the only people properly equipped and qualified to attain industry standards. This has clear policy and practice implications among those test stakeholders who find themselves increasingly involved, either directly (e.g., classroom teachers and university lecturers) or indirectly (e.g., college admissions officers, immigration officials, and parents), in producing and using tests and test scores. Although such stakeholders may be keen to improve their assessment knowledge and skills, they may feel discouraged from doing so by perceptions of the field's growing professionalization. Both Davies (Reference Davies2008a) and Spolsky (Reference Spolsky2008) expressed a concern that this process of professionalization over the last quarter century has led to the isolation of the language testing field from other academic disciplines. This may be equally true for its relationship with the wider world of education and society. We might even speculate that the language testing community's preoccupation with technical and measurement qualities in testing from the 1960s to 1980s, followed in turn by a similarly inward-looking focus on developing its own professional ethic during the 1990s, means that it has singularly failed to encourage wider public understanding of assessment principles and practices, that is, it has neglected to engage in the critical task of developing assessment literacy.

The Concept of Assessment Literacy

The term assessment literacy has been coined by several writers in recent years to describe what the constituency of language teachers and instructors needs to know about assessment matters (Inbar-Lourie, Reference Inbar-Lourie2008; Malone, Reference Malone, Shohamy and Hornberger2008; Stiggins, Reference Stiggins1997; Stoynoff & Chapelle, Reference Stoynoff and Chapelle2005). Clearly, language teachers and instructors need some measure of assessment training if they are engaged in selecting, administering, interpreting, and sharing results of large-scale tests produced by professional testing organizations, or in developing, scoring, interpreting, and improving classroom-based assessments. The requirement for adequate assessment training among language test developers and researchers is also self-evident, whether they are working in academically oriented contexts such as university departments of language or education, or in more commercially oriented contexts such as examination boards or testing agencies, or in the hybrid professional contexts of organizations such as the Educational Testing Service (ETS) or University of Cambridge ESOL Examinations (Cambridge ESOL). But the concept of assessment literacy could be expanded to describe the level of knowledge, skills, and understanding of assessment principles and practice that is increasingly required by other test stakeholder groups, depending on their needs and context (e.g., among educational advisors or government officials, policy planners and decision makers, the media, and the general public).

Since the 1960s there has been a steady stream of textbooks and course materials offering general training in testing theory and practice, from basic to very advanced. Examples include, in order of their publication date, Lado (Reference Lado1961), Carroll (Reference Carroll1961), Vallette (Reference Vallette1967), Harris (Reference Harris1969), Heaton (Reference Heaton1975), Allen and Davies (Reference Allen and Davies1977), Oller (Reference Oller1979), Carroll (Reference Carroll1980), Madsen (Reference Madsen1983), Carroll and Hall (Reference Carroll and Hall1985), Henning (Reference Henning1987), Hughes (Reference Hughes1989, Reference Hughes2003), Weir (Reference Weir1990), Davies (Reference Davies1990), Bachman (Reference Bachman1990), Weir (Reference Weir1993), Cohen (Reference Cohen1994), Alderson, Clapham, and Wall (Reference Alderson, Clapham and Wall1995), Bachman and Palmer (Reference Bachman and Palmer1996), McNamara (Reference McNamara2000), Davidson and Lynch (Reference Davidson and Lynch2002), Brown and Hudson (Reference Brown and Hudson2002), Weir (Reference Weir2005), McNamara and Roever (Reference McNamara and Roever2006), Downing and Haladyna (Reference Downing and Haladyna2006), Fulcher and Davidson (Reference Fulcher and Davidson2007). Other titles have focused more narrowly on testing specific language skills, subskills, or domains: the testing of reading (Alderson, Reference Alderson2000), listening (Buck, Reference Buck2001), writing (Shaw & Weir, Reference Shaw and Weir2007; Weigle, Reference Weigle2002), and speaking (Fulcher, Reference Fulcher2003; Luoma, Reference Luoma2004); the testing of vocabulary (Read, Reference Read2000) and grammar (Purpura, Reference Purpura2004); performance testing (McNamara, Reference McNamara1996); specific purpose testing (Douglas, Reference Douglas2000); computer-based testing (Chapelle & Douglas, Reference Chapelle and Douglas2006); and the testing of young learners (McKay, Reference McKay2006). Most of these publications were written primarily for an academic or specialized readership, and though some were specifically aimed at language teachers, few could be described as accessible to a general, nonspecialist audience, which might include parents, politicians, or policymakers. The textbooks listed above typically set out to offer practical advice in technical aspects of test production and score interpretation, or instruction in theoretical aspects of language testing and assessment, or a combination of the two. A few notable exceptions (e.g., McNamara, Reference McNamara2000) offer a general introduction to the sociopolitical and educational nature of assessment, together with practical guidance on the appropriate selection and use of tests in these contexts.

The increasingly central role of testing and assessment worldwide, however, and the concerns raised from within the language testing community recently about the potential misuse or abuse of tests, suggest that an appropriate level of assessment literacy needs to be nurtured not just among engineers and technicians who are actively involved in test development or research activities, or even among applied linguists and language teachers involved in delivering language education, but much more broadly in the public domain if a better understanding of the function and values of assessment tools and their outcomes is to be realized throughout society. Assessment literacy needs to be developed, therefore, across an increasingly diverse set of test stakeholder groups or constituencies including: personnel in both existing and newly established and emerging national examination boards, academics and students engaged in language testing research, language teachers or instructors, advisors and decision makers in language planning and education policy, parents, politicians, and the general public.

This is perhaps not dissimilar to the notion of computer literacy that emerged during the 1990s when computer knowledge, skills, and expertise, previously restricted to a fairly narrow, specialist sector of the population, gradually extended throughout the wider population as computers themselves became easier to access and more widely used in daily life. Higher levels of assessment literacy, that is, a sound knowledge and understanding of the principles and practice of assessment, are essential if applied linguists and other language specialists (e.g., teachers) are to engage effectively in matters of language assessment policy and practice, and if the benefits and the limitations of assessment tools are to be communicated to those in wider society (e.g., policymakers, parents).

The Components of Assessment Literacy

Views on the types of knowledge and understanding underpinning assessment literacy have changed over time. Davies (Reference Davies2008a), for example, reviewed the teaching of language testing over the past 50 years and observed two main trends: (1) a steady expansion in the volume of assessment training materials required due to the growing professionalism of the field (as we have seen from the sample selection above) and (2) an expansion in the profession's own view of the knowledge and skills needed by its members. He noted a steady curricular expansion from basic technical testing expertise (the how-to), to knowledge about language and measurement (the what), and, most recently, to understanding of key principles with regard, for example, to validity and ethics (also reflected in the list of titles). Davies characterized the present paradigm as that of Skills + Knowledge + Principles:

Davies's article reviews several seminal textbooks used to teach language testing between 1960 and the present day (including some of those referenced above), showing how they exemplify changing trends and perceptions in training focus. The focus in his article is clearly on the training needed by those within the field of language testing, a mainly academic community that occasionally extends to embrace language teachers and classroom based assessment, but rarely concerns itself with the need to educate other test stakeholder groups in assessment matters.

In the mid-1990s Bailey and Brown (Reference Bailey, Brown, Cumming and Berwick1995) reported on a survey of language testing courses in the context of language teacher training programs around the world. They examined various course characteristics, including topics covered and core textbooks used. A decade later, when they replicated their study, they found “the presence of a stable knowledge base that is evolving and expanding rather than shifting radically” (Bailey & Brown, Reference Bailey and Brown2008, p. 371). They also found evidence of other types of advanced language testing courses being provided to prospective language teachers in some graduate programs. It remains unclear, however, how far the content of such courses is likely to equip teachers to manage the complex sociopolitical and educational issues that are implicated in assessment today.

McNamara and Roever (Reference McNamara and Roever2006) cautioned against a potentially narrow view of the training needed by test developers and, presumably, other test users: “In terms of academic training, we stress the importance of a well-rounded training for language testers that goes beyond applied psychometrics . . . a training that includes a critical view of testing and social consequences” (p. 255). Their view parallels the earlier concern raised by Davies (Reference Davies2008a) and Spolsky (Reference Spolsky2008) that language testing, if viewed in purely professional or technical terms, risks being isolated from other key areas of life, and that this carries with it an inherent danger given the central role assessment now occupies in education and society.

Like McNamara and Roever, Inbar-Lourie (Reference Inbar-Lourie2008) took a broader view of how assessment literacy can be conceptualized and how it might be promoted more widely. She described an attempt to articulate the possible components of a language assessment literacy knowledge base as the foundation for establishing a core knowledge framework for assessment training. What seems innovative in this approach is that it was initiated from largely outside the professional testing culture and community, and it is informed by social-constructivist perspectives that encourage a more holistic, integrated, and dynamic understanding of assessment in context than the language testing profession has been used to in recent decades. Thus Inbar-Lourie argued that the core competencies to be acquired through assessment training will “reflect current views about the social role of assessment in general and language assessment in particular, contemporary views about the nature of language knowledge, and give due emphases to both classroom and external assessment practices” (pp. 396–397). Her perspective chimes with other initiatives that seek to reconceptualize assessment paradigms and the competencies needed by those involved, including, among others, work on assessment for learning (Assessment Reform Group, 2002), classroom-based assessment (Rea-Dickins, Reference Rea-Dickins, Shohamy and Hornberger2008), alternative assessment (Fox, Reference Fox, Shohamy and Hornberger2008), and dynamic assessment (Lantolf & Poehner, Reference Lantolf, Poehner, Shohamy and Hornberger2008).

This current trend in thinking seems to be that training for assessment literacy entails an appropriate balance of technical know-how, practical skills, theoretical knowledge, and understanding of principles, but all firmly contextualized within a sound understanding of the role and function of assessment within education and society. The exact nature of this balance and the level of literacy to be acquired may vary according to context, and according to whether test stakeholder constituencies are more or less directly involved in matters of test selection, test production, test score interpretation, or test evaluation.

Promoting Assessment Literacy Both Within and Beyond the Profession

As we have already seen, there is today no shortage of textbooks offering instruction in the principles and practice of assessment. Much of this material is specifically designed for those about to enter or already inducted into the language testing profession, that is, relative novices as well more established members committed to ongoing professional development. This constituency is relatively well-served in terms of access to assessment training, but if such materials are too specialized or technical for those outside the professional language testing community, the question remains how to encourage improved levels of assessment literacy more widely.

Malone (Reference Malone, Shohamy and Hornberger2008) reminded us that training in language assessment can be delivered not only through text-based materials, such as textbooks, but also via other channels. Malone highlighted the contribution made by face-to-face workshops and self-instructional options, which often make use of new technologies and permit a wide range of materials to be made easily available (e.g., free downloading from the Internet). She cited the examples of the Council of Europe materials associated with the Common European Framework of Reference for Languages (CEFR) (http://www.coe.int/t/dg4/Linguistic), as well as those provided by the Centre for Canadian Language Benchmarks (CLB) (www.language.ca).

Reference frameworks such as the CEFR and CLB can themselves serve as communication tools and heuristic devices, aiding transparency and supporting the coherence of language learning, teaching, and assessment activities across communities and regions. Professor John Trim underlined this point when he described the original development of the CEFR:

North (Reference North, Taylor and Weir2008) described the three main aims of the CEFR: “to establish a metalanguage common across educational sectors, national and linguistic boundaries that could be used to talk about objectives and language levels; to encourage practitioners in the language field to reflect on their current practice . . .; [and] to agree common reference points.” (p. 22). Not surprisingly, perhaps, views differ as to how far the aims of transparency and coherence have been achieved, and to what extent the CEFR risks being used instead as an instrument for harmonization, stifling diversity in relation to language teaching, learning, and assessment.

Other innovative approaches to help develop assessment literacy have included video projects such as Mark My Words (Melbourne LTC, Reference Melbourne1997) and the ILTA Video FAQs (Fulcher & Thrasher, Reference Fulcher and Thrasher1999/2000), though these types of resources need regular updating if they are to remain relevant and keep up with new developments in the field.

Professional language testing associations have taken on an increasingly active role in promoting assessment literacy more widely among language test developers, test users, and other stakeholders, especially teachers involved in classroom language assessment and other educators involved in developing language tests in national centers or institutional units. Training is now delivered, for example, through the preconference workshops of professional bodies such as the International Language Testing Association (ILTA) and the European Association of Language Testing and Assessment (EALTA), whose 2008 annual conference in Athens took assessment literacy as its theme. The Association of Language Testers in Europe (ALTE), a consortium of European language testing agencies, provides a range of resources designed as communication tools for developing understanding and expertise, from publications such as item writing guidelines and test content analysis tools (freely downloadable from the ALTE Web site), to conference opportunities and a training seminar program for ALTE members and interested others, based around ALTE's Principles of Good Practice for ALTE Examinations, Minimum Quality Standards, and Quality Management System for test developers (www.alte.org). The publication in the late 1990s of the Multilingual Glossary of Language Testing Terms (ALTE Members, 1998) and the Dictionary of Language Testing (Davies et al., Reference Davies, Brown, Elder, Hill, Lumley and McNamara1999) made available valuable practical resources for training specialists and nonspecialists alike in the terminology and discourse of language testing, a prerequisite for promoting assessment literacy. Language test providers such as examination boards and testing agencies (e.g., Cambridge ESOL) now provide a range of publications specially written to be accessible to a wide range of test stakeholder groups, from item writers and examiners, to learners and their parents, to teachers and publishers. In addition, they often offer conferences, seminar programs, and workshops for teachers, educational policymakers, and other test stakeholders.

An additional resource for developing assessment literacy is the series of narrative accounts that have recently emerged chronicling testing developments over the decades. The first of these by Spolsky (Reference Spolsky1995) provides an historical account of two influential and contrasting testing traditions on either side of the Atlantic. More recently published volumes in this historical narrative genre include the accounts of various Cambridge ESOL tests (Hawkey, Reference Hawkey2009; Weir & Milanovic, Reference Weir and Milanovic2003), the story of the communicative language testing movement (Hawkey, Reference Hawkey2004), the development of testing English for business purposes (O'Sullivan, Reference O'Sullivan2006), and the evolution of academic language proficiency assessment (Davies, Reference Davies2008b; Taylor & Angelis, Reference Taylor, Angelis, Chapelle, Enright and Jamieson2008). All these publications are helpful in describing and analyzing particular tests, as well as the more general business of testing and assessment, within the educational and sociopolitical contexts of their time. They contextualize the practice of language testing as a socially constructed and interpreted phenomenon, rather than treating it primarily as a pseudoscientific endeavor that is removed and isolated from human individuals and social values. It may well be that popular adaptations of this narrative, storytelling approach will prove a more effective means of developing assessment literacy among the wider stakeholder community in the future.

The overarching aim is not simply to train well-qualified engineers and technicians for quality test development, but to educate and build a much broader and inclusive community of good practice for the longer term, a community that is concerned with the impact of testing on society and with fairness issues in the broadest sense. Bachman commented as follows: “The advocacy for fairness by language testers and language test developers is thus, in my view, a critical element not only in the design and implementation of language assessment systems, but also, and perhaps more importantly, in the education and mentoring of those who will carry the field forward in the years to come” (Bachman, Reference Bachman and Kunnan2000, p. 40).

Conclusion: Reconceptualizing Test Wiseness

Traditionally, the term test wiseness has been used by the testing community to refer to the “amount and type of preparation or prior experience that a test taker has” (Davies et al., Reference Davies, Brown, Elder, Hill, Lumley and McNamara1999, p. 210). Familiarity or lack of familiarity with the content and format of a test is acknowledged to impact on a test taker's performance, potentially leading to an over- or underestimation of the ability of interest. Perhaps we can borrow the term test wiseness and reconceptualize it to reflect an understanding of testing skills, knowledge and principles–a different sort of test wiseness on the part of test developers and test users, and one that helps encourage the positive impact of testing on people and society and enhances the value of test use.

Improving levels of assessment literacy, both within the language testing profession and in wider society, is about more than simply offering opportunities for individual professional development (Hamp-Lyons, Reference Hamp-Lyons, Taylor and Weir2008) or for further professionalization of the field. It may help to mitigate the potentially negative effects that risk resulting from the growing influence of testing and assessment in education and society worldwide.

Finally, in the specific context of this annual review, it is perhaps worth reminding ourselves that applied linguists are often at the same time academic staff members who find themselves using test scores for decisions about student admissions; or they may be tasked with creating assessment tools for end-of-year student evaluations. Thus, applied linguists too are among those who need to develop a good understanding of what test instruments can and cannot do, along with a correct understanding of how to interpret the scores generated. A sound understanding of measurement principles and practice is also essential for anyone who designs or uses tests for the purposes of applied linguistics research. Sadly, it is not uncommon to see the findings of otherwise interesting applied linguistics research studies undermined because of the poor quality of the research tools (i.e., testing instruments) that were used or the inappropriate interpretation of their outcomes (i.e., test scores). A better grasp of key measurement concepts such as validity and reliability would undoubtedly help to address this problem. This last point highlights the potential interface between language testing researchers and those from other fields of language study. As McNamara (Reference McNamara2000) observed, language testing is not just a technical activity but “a site for research activity of a fundamental nature in applied linguistics” (p. 54). A stronger interface with other disciplines might also address the concerns raised by both Davies (Reference Davies2008a) and Spolsky (Reference Spolsky2008) over the apparent isolation of the language testing field, which in turn risks the marginalization of language testers themselves and any potential contribution for good they have to make to the wider academic and social community.

As more and more new test development projects are undertaken at national and regional level, so the need to understand the principles of sound assessment to implement these as good practice can only continue to grow among language teachers, applied linguists and a wide diversity of other test stakeholders. In light of this, there is an important and urgent role for the language testing and assessment community to play in helping to ensure that the core knowledge, skills, and experience needed to underpin good quality and ethically sound assessment policy and practice are shared as widely as possible for the benefit of all. As Stansfield astutely observed: “This will allow us to reach our full potential, both as professionals and as contributors to an organized civil society” (Stansfield, Reference Stansfield2008, p. 323).