Impact of the Standards Movement on L2 Learners and Teachers

Standards, also called outcomes-based curricula, attainment levels, and bandscales, are descriptions of curriculum outcomes, usually described in stages of progress. They may be content standards (describing what students should know and be able to do) or performance standards (describing how much, or at what level students need to perform to demonstrate achievement of the content standard). Many standards combine both purposes in the one document. Achievement on standards is often measured through external tests, although data are sometimes also collected on achievement through teachers' reports based on classroom assessment.

Use of large-scale standardized tests can bring a number of problems for second language learners and their teachers and schools, as I will outline next, especially when (as is usually the case) the reference group for the standards is mother-tongue speakers of the majority language. However, the results of assessment against standards can provide feedback to teachers, students, and parents about individual student learning in the established curriculum, and can give a common reference point for discussion of required pathways and whether they are being achieved.

The construction and implementation of standards by governments is driven by neo-liberal ideology, not simply by principles of educational philosophy. An understanding of this fact helps to make clear why governments employ standards, and why they are reluctant to address negative impact of those standards on minority groups. Briefly, governments are following business-world “managerialist” principles that seek to raise standards by (1) establishing competition between individuals, teachers, schools, and states; (2) commodifying the curriculum (making it measurable through standards); and (3) measuring, publishing, and then rewarding or punishing achievement. Standards are central to neo-liberal management of education and are closely tied to accountability. As noted by Katz, Low, Stack, and Tasang, “It is because of this accountability function that assessment serves a key role in the standards-reform effort” (2004, p. 5).

Researchers are investigating the impact of the standards-based movement on school-age minority learners, their parents, their teachers and their schools. The extent of concern is directly related to how high the stakes are; when children and subsequently their teachers and schools are labelled as failing, and suffer consequences for this, then the stakes are very high, for the children, their teachers, their school, their parents, and their community. Many studies in the United States have examined the equity of assessing ELL (English Language Learners) against states' standards through large-scale tests written with the expected language proficiency of mother-tongue speakers in mind (Katz et al., 2004; Liu, Anderson, & Thurlow, 1999; Swierzbin, Liu, & Thurlow, 2000). These studies are described further in the next section. In Australia, a national literacy standards test is administered to all elementary-age students regardless of English language proficiency (though new arrivals are exempt). Through their knowledge of second language development in the school context and analysis of students' language proficiency, Australian researchers have illustrated that the benchmarks fail to take sufficient account of minority learners who are in the process of developing their English language proficiency, and are bound not to show their progress in English in these tests, because the tests are focused on mother-tongue literacy (Davison & McKay, 2002; Davison & Williams, 2001a, 2001b; Hammond & Derewianka, 1999; McKay, 1998, 2000). A more recently discovered impact of literacy benchmarks testing in Australia is that, even as early as kindergarten, the more competitive elementary schools are checking their children for readiness for the benchmarks test in year 3. The result is a significant increase in children, especially ESL learners, being expected to repeat classes before year 3, as well as stress for teacher and children who are aware of this early monitoring process (McKay, 2004). As teachers point out to researchers in frustration, these procedures contrast with most principles of early childhood education that advocate individualized instruction and assessment to meet individual needs through the early years (e.g., Jalongo, 2000).

Many second language educators advocate the development and implementation of second language specific standards for second language learners to overcome these difficulties (Butler & Stevens, 2001; Davison & McKay, 2002; TESOL, 1997). The development of ESL-specific standards is discussed next. Breen et al. (1997) and Davison and Williams (2001a; 2001b) have investigated how teachers use ESL-specific standards (a number of which are available in Australia) in the classroom. Breen (1997) has observed that teachers go through stages of accommodation in the use of standards in their classrooms. Breen (1997) observed that genuine “accommodation” or take-up by teachers of external curriculum frameworks entails three phases: first, teachers need to recognize both conceptually and affectively the ultimate benefit to their own pedagogic priorities; second, they need to trial and adapt the framework to their established assessment procedures; third, they fully integrate the procedures into their practice. Davison and Williams (2001a; 2001b) also found that teachers differed in their use of the standards, depending on their experience and confidence and attitudes to the documents. In the United Kingdom, Leung and Teasdale (1997) investigated teacher assessment of speaking and listening of ESL learners at Key Stage 1 of the National Curriculum. They asked teachers to rate children's performance, and found that teachers did have some shared understanding about the general criteria for speaking and listening, but that there were also criteria used that were not specified in the National Curriculum. They suggest that some criteria were based on an understanding of an idealized native speaker norm rather than a model that reflects the language of ESL children.

All these researchers have noted the variability of interpretation of standards by teachers. Variability of interpretation is often because of the different experiences, understandings and attitudes of teachers, but not always. Arkoudis and O'Loughlin (2004) report on a secondary ESL situation where teachers came to understand that their inconsistent results were due to problems with the validity of the ESL standards themselves. When they tried to rewrite the standards to suit how they saw ESL development, the education authority was displeased and discounted their changes. One education department commentator presents his response:

In an insight into teachers' roles in a standards-based education system, Arkoudis and O'Loughlin concluded “that teachers' knowledge and experience, which have informed the working of the [standards document], have been positioned as not really relevant to the bureaucracy” (p. 13). This study illustrates that teachers are not always the “culprits” in unreliable interpretations. It also demonstrates the negative impact on and disempowerment of teachers when standards descriptions are not valid.

The writing of performance standards (describing how much, or at what level, students need to perform to demonstrate achievement) can be a type of research in itself, depending on how this is done. Performance standards may be based on expected progress through curriculum content standards, often closely reflecting the outcomes described in the content standards; see, for example, the Illinois Foreign Language Learning Standards (Illinois State Board of Education, 2003) or they may be stand-alone descriptions of proficiency stages broadly situated in the known curriculum context, such as the National Languages and Literacy Institute of Australia (NLLIA) ESL Bandscales (McKay, Hudson, & Sapuppo, 1994); or the Common European Framework of Reference (Council of Europe, 2001). The latter type of performance standard usually involves research to ensure a degree of validity in the descriptions. Validity of proficiency-based performance standards can be attended to primarily as the standards are being developed (a priori) or after they have been developed (a posteriori; Sadler, 1987). North (1995) describes how the Common European Framework of Reference was developed in an a priori process. The Australian NLLIA ESL Bandscales, on the other hand, were written out of teachers' experiences, and are currently undergoing an a posteriori validation process, involving analyses of work samples and student performance, focus groups involving moderation, teacher interviews, and classroom observations of teacher assessment processes. Importantly, it is the final assessment decision on the quality of learners' performance that requires validation, therefore teachers' assessment processes are an important element in validation of standards. Brindley's (1998b; Brindley, 2001) articles on outcomes-based assessment are important references for researchers who require on overview of the issues in the development and implementation of standards.

To strengthen validity and reliability in standards, we need to know as much as possible about the proficiency development of language learners. Davison and Williams (2001a; 2001b) in Australia, and Cameron and Besser (2004) in the United Kingdom are researchers who have closely examined the nature of learners' developing proficiency in writing in English. Liddicoat (1997) has worked with teachers in a foreign language context to provide teachers with samples of work to help ensure reliable assessment of standards. Similarly, McKay (1999) provides an analysis of the effectiveness of language samples as aides to teacher assessment against standards. This kind of research is necessary to raise the likelihood that standards descriptors are valid, and that resulting assessment decisions are valid and reliable.

There is a need for continuing research into the validity of standards themselves, and into the validity of the tests, tasks, and decision-making processes used by teachers and assessors to identify children's levels on the standards. Much more research into the impact of standards is also needed, including research into the impact of standards on teacher assessment practices, discussed further in the following section.

Countering the Negative Impact of Large-scale Assessment on L2 Learners

Many standards are designed to assess students' content knowledge, that is, their knowledge and skills in a curriculum area, for example, in literacy, mathematics, and science. In these cases minority language learners in schools are being assessed, usually in large-scale tests, on their content knowledge in a language in which they are not yet proficient:

Butler and Stevens (2001) provide a valuable overview of issues in the assessment of English language learners' (ELL) content knowledge in large-scale tests in the United States. Recent federal government acts have decreed that all students should be included in state and district assessment programs in order that comparable information about student progress can be obtained. The immediate questions for educators (including assessors) are the following:

1. When is it appropriate to give standardized content assessments to ELLs? That is, when are the inferences made about the performance of ELLs on standardized content assessments valid?
2. Until it is appropriate to give these assessments to ELLs, how do we provide accountability and assure equity?

Butler and Stevens list the following as strategies to include ELLs in assessment processes:

• [bull ] Testing in the first language
• [bull ] Using accommodations; there can be modifications of the test (such as assessment in the native language, modification of linguistic complexity, addition of visual supports) or modifications of the test procedures (extra assessment time, breaks during testing, oral directions in the native language)
• [bull ] Measuring growth in English (through tests in English) as an alternative to inclusion in content-based tests

Results of research on whether accommodations make any significant difference in the performance of second language learners on content tests is mixed (Butler & Stevens, 1997; Gottlieb, 2003; Koenig & Bachman, 2004), and research is needed into the idea of tailoring accommodations to the nature of students' language proficiency and knowledge. Butler and Stevens (2001) also suggest that opportunity to learn (OTL) is an inherent problem with large-scale content assessments with ELLs. If ELLs are not receiving the content instruction that is covered on large-scale assessments, “then accountability data based on these assessments are neither valid nor reliable” (p. 421). Butler and Stevens, along with Abedi, Courtney, Mirocha, Leon, and Goldberg (2000), recommend that classroom-based research is needed to determine whether ELLs are receiving the appropriate content instruction. They argue for the use of a multiple-assessment approach to evaluate school programs, with large-scale assessment being one approach. Gottlieb (2003) also advocates a multiassessment approach in which teacher-based assessment can be used “as a form of large-scale testing” when ELL students are at the beginning stages of language proficiency and when the following three conditions are met:

• [bull ] Standard prompts (blueprints) appropriate for students' age and development are made available to teachers
• [bull ] Content-related language samples are collected (a) in the fall to establish an initial baseline, (b) at midyear to monitor progress, and (c) at the end of the year to measure growth
• [bull ] Samples of performance are collected and held in the students' records

Large-scale teacher-based assessment needs further research, but may provide some alternatives to facilitate inclusion for beginning ELL students before they can validly enter the testing system.

Analysis of the test results of ELL learners in state-wide content tests is quite commonly carried out by education departments. Liu, Anderson, and Thurlow (1999), for example, examine data trends in Minnesota's Basic Standards Tests (BST), statewide tests of reading and math, for the years 1996–1999 for LEP students. They made several observations, including the following:

• [bull ] Participation for LEP students in the test remains high
• [bull ] Performance of LEP students remains low, with a persistent although narrowing achievement gap of 20–25% between LEP and non-LEP students taking reading texts than math tests, which suggests ESL teachers may be successful in teaching the reading skills needed on the BST.
• [bull ] The relative number of Minnesota LEP students taking advantage of test accommodations (i.e., extended testing times, translation of directions, special test settings etc) is low. (Liu et al., 1999, p. 1)

Another report of this type is by Gonzalez for the Austin Independent School District (Gonzalez, 1999). These reports help researchers to gather layers of data about ELL performance and about the impact of testing on ELLs in content tests. Katz et al. (2004) analyzed student assessment data compiled by the San Francisco Unified School District to explore the relationship between content area testing and students' English language proficiency. They also reported on the results of classroom observations of ELLS to describe their academic performances, and conducted an auxiliary study of California's English Language Proficiency test. The researchers were particularly interested in determining at what point along the second language acquisition continuum educators “can regain confidence in the results of standardized tests conducted in English with ELLs” (p. 5). The researchers found that “more situated notions of English language proficiency are needed to enable educators to make reasoned decisions as to when students can move into English-only instruction and English-only assessment” (p. 69). They also found that testing results do not accurately reflect how ELL students function in classroom settings. They support the provision in the current No Child Left Behind legislation that allows ELLs to be exempted from testing for at least three years while being provided with appropriate language support.

Without research into the validity, reliability, and impact of large-scale content-based tests for ELL learners, we do not know the extent to which these tests limit or expand students' avenues for demonstrating competence and thus their life choices (Darling-Hammond, 1994). The use of large-scale tests is, after all, not necessarily to be accepted without question. Many educators have written about the advantages of alternative assessment over standardized assessment (Brown, 1998; Genishi & Brainard, 1995; Herman, Aschbacher, & Winters, 1992). Hasselgren (2000) describes how the alternative assessment conducted in Norway by the Norwegian Ministry of Education, in conjunction with the University of Bergen, yields valuable results for the Ministry, the children, and the teachers:

Academic Language Proficiency Assessment in L2 Contexts

To assess the academic language proficiency needed for minority language learners, especially those in upper elementary and secondary schools, to succeed in content-based tests and mainstream classroom learning, a clear definition of the construct “academic language proficiency” is needed. A group of researchers at CRESST (National Center for Research on Evaluation, Standards, and Student Testing) in the United States, have been researching the nature of academic language proficiency for test development purposes (Bailey, in press; Bailey & Butler, 2003; Bailey, Butler, LaFramenta, & Ong, 2004; Butler & Bailey, 2002; Butler, Lord, Stevens, Borrego, & Bailey, 2004; Stevens, Butler, & Castellon-Wellington, 2000). In a recent report, Butler and colleagues (2004) describe how they have examined standards documents, textbooks, and classroom video samples to collect data about the type of language use required in upper elementary science and math classrooms. Their work has identified the grammatical features of language functions, the structural features of academic texts, the types of texts found in textbooks and the textual and linguistic features of the texts. Features of classroom discourse have also been identified. The researchers then used this information to write draft academic language proficiency test specifications and prototype tasks. This work is ongoing. Investigation into academic language proficiency is a high priority to ensure understanding of (a) what language is needed for successful participation in mainstream classes, (b) how test tasks should be constructed, and (c) what ELL students should be taught to be successful at school and in tests.

Classroom Assessment in School-Level Second and Foreign Language Learning

A cutting edge area of assessment research activity at present is research into classroom assessment. In recent years, a growing body of research has been conducted into formative assessment in generalist classrooms, that is, classrooms described without specific reference to language learners (Black & Wiliam, 1998; Hall, Webber, Varley, Young, & Dorman, 1997; McMillan, 2003; Smith, 2003). McMillan writes “What is needed is an understanding of how assessment and instruction are interwoven, with new conceptions about what assessment is and how it affects learning” (2003, p. 7).

McMillan's research has revealed, for example, that teachers make decisions about their classroom assessment based on achieving goals for students that include noncognitive outcomes (such as confidence and a sense of achievement) as well as those stated in the curriculum. Teachers “pull” for their students, in that they try to find ways that help their students succeed. They put great emphasis on promoting students' understanding and on accommodating individual differences, and they vary assessments to accommodate these differences. Teachers believe it is imperative for students to be actively engaged in learning, and for them to be motivated to do their best work.

In the context of formative assessment, then, McMillan suggests that there is validity in formative classroom assessment when an assessment decision has resulted in more student engagement, and when progress can be identified. Gipps (1994) had claimed earlier that teacher assessment is valid and reliable if carried out in a structured, repeated, and collegial way. Smith (2003) suggests that reliability should be deemphasized in formative assessment, because teachers are typically interested in how well the student does the task, rather than finding out how well the student has performed in relation to others (Smith, 2003, p. 5). Reliability, Smith suggests, can be checked through the collection of sufficient observation data over many tasks, and the impact of classroom assessment can be evaluated through a consideration of the intended and unintended consequences of teachers' decisions (2003, p. 9). These new ways of looking at validity, reliability, and impact demonstrate that new ways of thinking are emerging with regard to formative assessment. Leung (2004) and Leung and Teasdale (1997) have contemplated the implications of these new perspectives for second language assessment. Leung (forthcoming) has suggested that three kinds of questions require some immediate attention in the second language field:

• [bull ] What do teachers do when they carry out formative assessment?
• [bull ] What do teachers look for when they are assessing?
• [bull ] What theory or “standards” do teachers use when they make judgments and decisions?

He questions whether teachers can assess

Language researchers have only recently undertaken investigations of teachers' formative assessment. Rea-Dickins and her colleagues (Gardner & Rea-Dickins, 2001; Rea-Dickins, 2001; Rea-Dickins & Gardner, 2000; Rea-Dickins & Rixon, 1999) have explored the nature of formative assessment in elementary classrooms, examining the range and quality of teacher assessment, the issues, and the assessment processes. Like Leung, Rea-Dickins has asked several questions about formative assessment:

• [bull ] What constitutes ‘quality’ in formative assessment?
• [bull ] Are these assessments creating opportunities for learning?
• [bull ] What constitutes evidence of language learning?
• [bull ] Are teachers in the EAL [English as additional language] context able to distinguish between a language learning need, a special education need, a curriculum content need?

A current issue investigated by Rea-Dickins (2001) is the assessment processes followed by mainstream teachers in United Kingdom classrooms that include ESL students when teachers have to both formatively assess and also report on children's progress against externally-developed criteria or standards. She refers to the cycles of assessment described by Hall, Webber, Varley, Young, and Dorman (1997) and from her observations maps a cycle of assessment that involves planning (Stage 1); implementation, where scaffolding, self and peer assessment, and feedback to learners takes place (Stage 2); and monitoring, recording and dissemination (Stage 3). The fourth stage is where the teacher undertakes recording and dissemination work for accountability purposes.

Processes and strategies in instruction-embedded classroom assessment. (from Rea-Dickins, 2001, p. 435)

(© Hodder Arnold. Permission to reproduce gratefully acknowledged.)

Because it is often the case that teachers' formative assessment and monitoring (in Stages 2 and 3) are translated into reporting (in Stage 4), Rea-Dickins warns that we need to remember that formative assessment may be more high-stakes than we usually acknowledge.

In the English as a foreign language (EFL) situation of Hong Kong, Morris, Lo, Chik, and Chan (2000) found that elementary schools high stakes assessment is a stark reality; such assessment is seen as a preparation for survival in a highly competitive society. After a new system of formative assessment against Bands of Performance had been introduced through the Hong Kong Target Oriented Curriculum, it was found that teachers were responding by formally assessing each outcome in the Bands, and noting each assessment result in detailed records. However, they said that they had little time to do follow-up work, and also that their workload was greatly increased. The lesson to be learned from this, according to the researchers, is that governments need to consider long-term and coherent strategies that address both the structural features of schooling and the prevailing beliefs about assessment. They suggest that the following conditions are needed if classroom, criterion-referenced assessment is to succeed:

• [bull ] A clear linkage between external and school-based assessments
• [bull ] The development of a system of recording and reporting assessment which stresses the role of teacher collaboration, the exercise of professional judgment and the provision of feedback designed to support learning
• [bull ] Ongoing support for teacher professional development designed to promote their understanding of the roles and processes of assessment

Also in Hong Kong, Davison is undertaking two projects into formative and summative school-based assessment (to be completed in 2006). One project is investigating the transition from norm-referenced to criterion-referenced, school-based assessment, and the other, alongside improvement of practices through action research, is aimed at “identifying and describing the factors which may facilitate and/or hinder the connection of formative assessment and feedback with learning and teaching in a range of different English language situations in Hong Kong” (Davison, personal communication). Gatullo (2000), in Italy, has reported on a two-year pilot study in which she found that teachers tended not to make productive use of information they had collected for formative purposes, and they made little or no use of some types of questioning and negotiations that could be fed into formative assessment and enhance the learning process.

To return to standards, there appears to be recognition, in both mainstream and language assessment research, of the tensions that exist for teachers required to operate in an accountability regime (preparing students for external tests, reporting on achievement to stakeholders) and at the same time conduct formative assessment (Black & Wiliam, 1998; Brindley, 1998a; McKay, 2004; McMillan, 2003; Rea-Dickins, 2001). The “managerialist” approach to education stands in direct opposition to the professionalism of teachers whose natural inclination is to work collaboratively to improve learning (Broadfoot, 1996; Clair, Adger, Short, & Millen, 1998; Hartley, 1997; McKay, 2004; Wiliam, 2001). Clair et al. (1998) have also mapped the extent of tensions between managerialist and professional perspectives in the introduction of standards and large-scale testing in California. McKay (2004) interviewed teachers of young learners in Australia who indicated changes in practice and stress for teachers with the introduction of performance indicators as early as grade 1. One teacher described the situation this way: “To get these indicators, instead of observing what children can do during activities, and planning for observing those, they're just teaching teaching teaching and testing, creating a test that tests each one of these” (Griffiths-Chandran interview in McKay, 2004). This teacher continued:

Breen and his colleagues (1997) also found these tensions exist for ESL specialists working with ESL learners in the mainstream:

It is imperative to investigate ways in which teachers can meet the accountability requirements but at the same time successfully assess for formative purposes in the classroom. We also need to know more about the ways that ESL specialists coassess with mainstream teachers to monitor ESL students' progress over time, and through this, to coteach with mainstream teachers to improve learning of ESL students. Breen et al.'s (1997) study provides information of this kind.

Assessment of Young Language Learners

Much of what has gone before in this chapter relates to the assessment of both elementary and secondary age learners. Young learners are defined here as elementary age learners from around 5 to 12 years of age. Because there is relatively little in the literature on the assessment of young learners (Rea-Dickins & Rixon, 1997) it is worth highlighting existing research into this area to motivate further related efforts. A chapter summarizing research into the assessment of young language learners can be found in McKay (2005). There are several challenges for researchers in young learner assessment, not least because of the variability across programs, especially foreign language programs, and a lack of consensus about proficiency (Johnstone, 2000). There is general reporting, too, of variable teacher expertise in assessment. Teachers tend not to have assessment as a top priority (Hill, 2000; Jantscher & Landsiedler, 2000; Low, Brown, Johnstone, & Pirrie, 1995). In a survey in which British elementary teachers were asked about ways in which teaching English could be improved, teachers' responses did not mention assessment (Teasdale & Leung, 2000). In Hong Kong, too, teacher trainers' understanding of assessment was found to be somewhat “impoverished” (Morris et al., 2000).

The characteristics of young learners are highly relevant in the assessment process. Young learners are going through a period of social, emotional, and cognitive growth, they are developing literacy; and they are highly vulnerable (McKay, 2005). Researchers require specialist knowledge of young learners to investigate assessment issues. Such knowledge includes, for example, the characteristics of young learners and the characteristics of tasks (the setting, the input, the nature of the expected response, the relationship between the input and the response) likely to affect performance in assessment procedures. Carpenter, Fujii, and Kataoka (1995) present one of the few accounts of assessment task development and evaluation for young learners. In the Cambridge Young Learners English Test (University of Cambridge ESOL Examinations, 2003), we see a test development and implementation process that takes into account the characteristics of young learners, and follows implementation with ongoing research (Ball & Wilson, 2002; Marshall & Gutteridge, 2002; Taylor & Saville, 2002). Young learner assessment deserves to be established as a highly expert field of endeavor requiring, for example, knowledge of the social and cognitive development of young learners, knowledge of second language literacy development, and understanding of assessment principles and practices.