1. INTRODUCTION: IDENTITY, AGENCY, AND LANGUAGE LEARNING
This article offers some reflections on language learning and the assessment of language proficiency from the linked perspectives of learner identity and learner agency, with particular reference to the Common European Framework of Reference for Languages: Learning, teaching, assessment (CEFR; Council of Europe, 2001).
We assume that each of us has multiple subjective identities (ways in which we think of ourselves) that are shaped by and interact with the different contexts in which we lead our lives; that those identities are rooted in our selfhood, which derives from our sense of being human and gendered; and that they are fluid and subject to change, which may be intentional and predictable or accidental and unpredictable. We further assume that our subjective identities are complemented by identities that are assigned or attributed to us by others, individuals but also institutions. The extent to which we accept these assigned identities and incorporate them into our subjective identities is infinitely variable (see, for example, Reference JenkinsJenkins, 2014).
Agency, defined by Ahearn (Reference Ahearn2001, p. 112) as “the socioculturally mediated capacity to act,” is central to selfhood and fundamental to the assertion of identity. Van Lier (Reference van Lier, O'Rourke and Carson2010, p. xii) has cautioned against the assumption that there is sociocultural mediation in everything we do; but in many contexts, including education, the exercise of agency clearly is socioculturally mediated. What is more, as Wertsch, Tulviste, and Hagström (Reference Wertsch, Tulviste, Hagström, Forman, Minick and Stone1993, p. 337) have pointed out, agency is “frequently a property of dyads and other small groups rather than individuals,” and mental functioning “extends beyond the skin” by virtue of the mediational means employed (p. 343). The shared agency that arises from “mutual engagement” (Wenger, Reference Wenger1998, p. 73) is a defining feature of communities of practice (Lave & Wenger, Reference Lave and Wenger1991; Wenger, Reference Wenger1998).
Education is a process of “people shaping” designed to extend and perhaps in some ways modify learners’ identity while exploiting and developing their agency. Curricula specify the competences (knowledge and skills) that learners should acquire; to the extent that their acquisition is successful, learners’ identities are expanded and/or modified. A major concern of pedagogical theory has been to find ways of empowering learners such that their agency is positively engaged in the learning process—hence the importance attached to exploratory talk as a means of collaboratively constructing knowledge (e.g., Mercer, Reference Mercer1995, Reference Mercer2000; Mercer & Littleton, Reference Mercer and Littleton2007; Wells, Reference Wells2009). Agency and identity are inextricably linked in Barnes's view that exploratory talk enables learners to incorporate “school knowledge” into what he called their “action knowledge”—“that view of the world on which our actions are based” (Barnes, Reference Barnes1976, p. 81). The same concern for learner empowerment is fundamental to the notion of learner autonomy (Holec, Reference Holec1979; Little, Reference Little1991), which entails that learners accept responsibility for their learning and are proactive in pursuit of personal and collaborative learning agendas. Self-assessment—a habit of mind as well as a recursive process—helps to make learners consciously aware of the ways in which formal learning of any kind extends their capacities and expands their identity. Assessment by peers, teachers, and examination boards may confirm or alter the self-concept generated by self-assessment. All forms of assessment help to shape the individual's identity—subjective and assigned—as a language learner.
For the past 20 years or so the key reference for identity in language learning has been the work of Bonny Norton (most recently, Norton, Reference Norton2013), whose poststructural conception of identity is underpinned by her notion of “investment” (what she terms a sociological alternative to the psychological notion of motivation) and Anderson's (Reference Anderson1983) concept of “imagined communities.” Kramsch (Reference Kramsch and Norton2013) has summarized Norton's position thus: “Unlike motivation, investment carries connotations of hopes of returns and benefits; it accentuates the role of human agency and identity in engaging with the task at hand, in accumulating economic and symbolic capital, in having stakes in the endeavor and in persevering in that endeavor” (p. 155). Norton's position has much in common with earlier work, especially in Europe, that likewise assumed a close link between learner identity, learner agency, learning purpose, and motivation. This work is closely associated with the concept of learner autonomy, which for more than four decades has been central to the Council of Europe's projects in education generally and language education in particular.
Since its publication in 2001, the CEFR (Council of Europe, 2001) has had a major impact on language testing in Europe and beyond. Its proficiency levels have been widely adopted, and manuals have been published to guide the linking of existing tests to the CEFR (Council of Europe, 2009) and the development of new tests based on its descriptive scheme and levels (Council of Europe, 2011b). But the CEFR is a great deal more than a collection of proficiency levels and scales (for a comprehensive overview, see North, Reference North2014). Its action-oriented approach to the description of language proficiency as language use clearly implicates the user–learner's identity and agency, both of which are central to the CEFR's companion piece, the European Language Portfolio (ELP), in which self-assessment plays a key role. Taken together, the CEFR and the ELP imply a rather different assessment culture from the one that is often associated with the CEFR's reference levels. Section 2 elaborates this argument, while Section 3 considers current trends in language assessment from this perspective.
2. COMMON EUROPEAN FRAMEWORK OF REFERENCE FOR LANGUAGES
2.1. Some historical background
It is no accident that in the second half of the CEFR's title, Learning, teaching, assessment, learning comes before teaching and teaching comes before assessment. The order of the three nouns implies a learner- and learning-centered view; it also reflects the chronological development of the Council of Europe's involvement in language education. Its earliest modern languages projects were carried out in the 1970s under the aegis of the Committee for Out-of-School Education, whose principal concern was the organization, content, and methods of adult education. The concept of “self-learning” was central to its thinking. Whereas “self-instruction” meant dispensing with the help of a teacher, self-learning was understood as a process guided and supported by a teacher. It “generally refers to the practice of working in groups, and to the choice by participants of objectives, curriculum content and working methods and pace” (Janne, Reference Janne1977, p. 31). Self-learning, in other words, is a concept that acknowledges the contribution adult learners themselves can make to the educational process: the knowledge, skills, and experience they bring with them (identity) and their potential capacity to co-manage their learning (agency). It also assumes that learning in formal educational contexts is socially situated and collaborative as well as individual.
The commitment to self-learning explains the interest of the Council of Europe's early modern languages projects in self-assessment (Oskarsson, Reference Oskarsson1978) and autonomous learning (Holec, Reference Holec1979), and their concern with the “democratization” of language education based on needs analysis, learner involvement, and the belief that decisions should be taken as close as possible to the point of teaching/learning (Trim, Reference Trim1978). Trim (Reference Trim, van Ek and Trim1984) explained why the modern languages project group decided to prioritize the specification of objectives over issues of testing, assessment, and evaluation:
There seemed to be a distinct danger that the premature involvement of the formidable expertise of the institutions engaged in language testing and examining might lead to definitions of language learning objectives being determined, not by the communicative needs of learners, but rather by what the professionals believed to be testable by their existing techniques and in accordance with their received criteria. (Trim, Reference Trim, van Ek and Trim1984, p. 161)
Assessment and certification came into their own, however, with the Rüschlikon Symposium, hosted by the Federal Swiss authorities in 1991, as the title of the symposium indicates: Transparency and Coherence in Language Learning in Europe: Objectives, Evaluation, Certification. The symposium made two recommendations: that the Council of Europe should develop “a comprehensive, coherent and transparent framework for the description of language proficiency” (Council of Europe, 1992, p. 39) and “set up a working party to consider possible forms and functions of a ‘European Language Portfolio’ to be issued under its aegis” (Council of Europe, 1992, p. 40). The report on the symposium shows that the need for a framework was most urgently felt in respect of assessment and certification, and much of the discussion of a possible ELP focused on its function as a supplement to formal certification. Nevertheless, the detail of the two recommendations continues to reflect the Council of Europe's commitment to the autonomy of the individual learner: the proposed framework “will enable learners to find their place and assess their progress with reference to a set of defined reference points” (Council of Europe 1992, p. 39), while the ELP will be “held by individuals” and used to “record their cumulative experience and qualifications in modern languages” (Council of Europe, 1992, p. 40).
2.2. The CEFR's learner-centered approach
The CEFR is explicitly aimed at learners as well as teachers, course designers, examining bodies, and educational administrators (Council of Europe, 2001, p. 6). Perhaps more importantly, the CEFR defines language proficiency in terms of what the individual user-learner can do as an autonomous social agent. It summarizes its action-oriented approach to the description of language use as follows:
Language use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents develop a range of competences, both general and in particular communicative language competences. They draw on the competences at their disposal in various contexts under various conditions and under various constraints to engage in language activities involving language processes to produce and/or receive texts in relation to themes in specific domains, activating those strategies which seem most appropriate for carrying out the tasks to be accomplished. The monitoring of these actions by the participants leads to the reinforcement or modification of their competences. (Council of Europe, 2001, p. 9; emphasis in original—the words and phrases set in boldface, to which “contexts” in the second sentence should probably be added, refer to the principal components of the CEFR's descriptive scheme.)
According to this summary, language use entails that we draw on our competences—knowledge, skills, experience, and characteristics—in order to engage in communicative activities. Language learning is a variety of language use in the sense that proficiency develops from sustained interaction between the learner's gradually developing competences and the communicative tasks whose performance requires him or her to use the target language. If learners are to develop a proficiency that allows them to act as individuals and social agents, it seems clear that the target language should be the principal medium of their learning, which should be organized so as to give them unlimited access to initiating as well as responding discourse roles. By exercising their agency in the target language, learners gradually develop a proficiency that adds a new dimension to their identity.
The last sentence of the summary deserves special attention: “The monitoring of these actions by the participants leads to the reinforcement or modification of their competences.” The CEFR includes monitoring among the communicative language processes that it considers relevant to the development of language proficiency, defining it as the strategic component that “deals with updating of mental activities and competences in the course of communication” (Council of Europe, 2001, p. 92). Monitoring as the updating of mental activities recalls Levelt's (Reference Levelt1989) model of the first language (L1) speech production mechanism; while monitoring as the updating of competences is reminiscent of Schmidt's (Reference Schmidt1990, Reference Schmidt, Hulstijn and Schmidt1994) “noticing hypothesis,” according to which second language (L2) development requires that learners pay conscious attention to the form of linguistic input in order to address gaps in their knowledge.Footnote 1 In both these senses, and also when it focuses on “the communicative process as it proceeds, and with ways of managing the process accordingly” (Council of Europe, 2001, p. 92), monitoring is a conscious phenomenon. Bearing in mind the CEFR's view of language learning as a variety of language use and its commitment to promoting the agency of the learner, monitoring is an essential part of self-regulation not only in language use but also in the exercise of strategic control over the learning process (Little, Reference Little, Holec, Little and Richterich1996). By extension, self-assessment plays an essential role in the learning process. According to this interpretation, the CEFR's action-oriented approach encourages the development of a proficiency that is both communicative and metacognitive, and licenses a pedagogical approach in which the target language is the principal channel of the learners’ agency—the communicative and metacognitive medium through which, individually and collaboratively, they plan, execute, monitor, and evaluate their own learning.
2.3. The European Language Portfolio, identity, and agency
Conceived as a way of mediating the CEFR's ethos and action-oriented approach to learners, the ELP has three obligatory components: a language passport that summarizes the owner's experience of learning and using second and foreign languages and records his or her cumulative self-assessment in terms of the communicative activities and proficiency levels of the CEFR; a language biography that provides a reflective accompaniment to language learning and language use; and a dossier in which the owner may keep work in progress and/or collect evidence of his or her achieved proficiency. The ELP has three pedagogical focuses: the development of learner autonomy, intercultural awareness/competence, and plurilingualism; and two functions, pedagogical and reporting (Council of Europe, 2011a). It is linked to the CEFR by its learner-centered ethos and by the goal-setting and self-assessment checklists of “I can” descriptors arranged by CEFR activities and levels.
The ELP supports the exercise of learner agency in the self-management of learning. Self-assessment—“monitoring” in the broad sense we have given the term—plays a pivotal role in its effective use (Little, Reference Little2006, Reference Little2009, Reference Little2011), and if checklists are provided in the learners’ target language, the regular “deconstruction” of descriptors can play a central role in the development of learners’ metacognitive proficiency. (It should be noted, however, that in many versions of the ELP, checklists are provided in the language of schooling, assumed to be the learners’ L1, which sits awkwardly with the CEFR's description of language learning as a variety of language use.) The ELP is a means of managing and documenting the learning process that links learning to assessment, especially if learners have access to the rating criteria used in external tests and exams. When the ELP is central to the learning process, it provides a dynamic reflection of the owner's developing identity as a language learner and user.
2.4. The CEFR: towards a new assessment culture
The CEFR challenges us to develop a pedagogical culture in which learning, teaching, and assessment are naturally integrated with each other. This challenge derives from the fact that each of the CEFR's “can do” descriptors can be used simultaneously to specify a learning outcome, provide a learning focus, and imply an assessment task. The CEFR is, in other words, a tool for constructive alignment (Biggs & Tang, Reference Biggs and Tang2011). The ELP enables learners themselves to participate in this new culture, provided there are clear links between checklist descriptors and curriculum goals and steps are taken to bring self-assessment into interaction with peer and teacher assessment and external examinations.
To date, the CEFR's underlying ethos has largely gone unrecognized or been ignored. Especially in school education, there have been few attempts to assert the interdependence of curriculum, learning/teaching, and assessment. Many ministries of education associate intended learning outcomes with CEFR proficiency levels, but without exploring the implications of such an association. The CEFR has had little impact on most external school exams, while the ELP has usually been developed as an optional extra, and checklist descriptors have rarely been explicitly related to curriculum goals. As a result, the CEFR is widely assumed to be of concern mainly, or exclusively, to language-testing agencies, and the ELP has failed to gain significant purchase in any of the Council of Europe's member states. One implication of the ethos that underlies the CEFR is that curriculum and pedagogy have little meaning unless they are respectively committed to and grounded in processes of assessment that are concerned with the valorization of learning and learners. Section 3 considers current trends in various forms of assessment from this perspective.
3. SOME CURRENT TRENDS IN THE ASSESSMENT OF LANGUAGE PROFICIENCY
3.1. The concept of assessment
Assessment and testing are not synonyms. Typical dictionary definitions of assessment include appraisal, estimate, and judgment; definitions of testing include examination, measurement, and trial. As these definitions suggest, assessment is broader than and superordinate to testing, which tends to emphasize quantification and accuracy of measurement. In other words, tests are likely to form part of any assessment system, which may also include other, less formal instruments and procedures. The challenge that education systems face is to ensure (as far as possible) that all modes of assessment work together to produce observations and judgments that are as accurate and equitable as possible, and thus lead to appropriate and adequate conclusions and actions. Such an assessment culture should contribute beneficially to the learning of individual language learners and to their developing identity as users of the language in question. Besides promoting learning, however, assessment has a second function, to promote fairness and equity at personal and systemic levels. According to this understanding, assessment is equally important in its formative and summative functions—as Taras (Reference Taras2005) has reminded us, all assessment is summative in the sense that it makes a judgment; formative assessment provides learners with feedback based on that judgment.
The basic principles that should govern all types of assessment are set out in a number of readily available codes of good practice that derive from the Standards for Educational and Psychological Testing developed jointly by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education (AERA/APA/NCME, 2014). Examples of such codes are those published by ILTA (International Language Testing Association), ALTE (Association of Language Testers in Europe), and EALTA (European Association for Language Testing and Assessment). The Guidelines for Good Practice in Language Testing and Assessment developed by EALTA and available in 35 languages (www.ealta.eu.org/guidelines.htm) identify three main stakeholder groups: teacher educators, teachers in schools, and institutions and agencies involved in large-scale national or international testing. The same general principles apply to all three groups: “respect for the students/examinees, responsibility, fairness, reliability, validity and collaboration among the parties involved” (EALTA, 2006, p. 1). Whatever their professional role, EALTA members are expected to “adhere to the principles of transparency, accountability and quality appropriate to their particular contexts and spheres of professional involvement” (EALTA, 2006, p. 1). In short, all types of assessment should be guided by five fundamental concepts: transparency, validity, reliability, respect, and collaboration.
When planning, analyzing, and evaluating any type of assessment, it is necessary to answer five questions with these principles and concepts in mind:
1. Why is the assessment being undertaken?
2. What is being assessed; what is the construct?
3. How is it being assessed, evaluated, and fed back or reported?
4. Who are the agents in the assessment processes?
5. What are the intended and envisaged uses and consequences of the assessment?
In the remainder of this article, we are especially concerned with the last two issues, agency and use/consequences, including the impact of assessment on individuals, especially their self-perception, self-esteem, and identity. If teaching/learning and assessment are reciprocally integrated, it follows that assessment involves four categories of agent: the individual learner, the individual learner's peers, the teacher, and external bodies. Bearing in mind the “people-shaping” function of education and the desirability of complementarity between subjective and assigned identities, our ultimate goal should be to establish multiple constellations of assessment that work together harmoniously.
3.2. Agents in assessment: the language classroom
3.2.1. Self-assessment
In section 2.2 we pointed out that the CEFR uses monitoring in three distinct senses: monitoring of the speech production process, monitoring in order to notice gaps in linguistic knowledge, and monitoring communication with a view to managing it strategically. We argued that because the CEFR views language learning as a variety of language use and is committed to promoting the agency of the individual learner, monitoring in these senses sits comfortably with monitoring as an essential feature of the self-regulated or autonomous learning that is commended by the CEFR and supported by the ELP. A teaching approach that seeks to engage learners’ agency and identity by involving them fully in planning, monitoring, and evaluating their learning necessarily entails metacognitive activity that brings into play each learner's inborn but often unwitting capacity for self-assessment. The dynamic of one such approach is shaped by “underlying,” “operational” and “procedural” principles (Legenhausen, Reference Legenhausen, Little, Ridley and Ushioda2003, pp. 67–68). The underlying principle claims that a process of creative construction lies at the heart of effective language learning; the operational principle assumes that creative construction arises from authentic social interaction in the target language (authentic in the sense that it acknowledges and seeks to exploit the individual learner's agency and identity); and the procedural principle entails strict management of a regular and predictable work cycle: planning followed by implementation followed by evaluation, which leads to further planning, and so on (see also Dam, Reference Dam1995).
According to Dam, this work cycle is driven by repeatedly posing and seeking to answer the following questions:
• What am I/are we doing?
• Why am I/are we doing it?
• How am I/are we doing it?
• Good experiences?
• Bad experiences?
• Ideas for change?
• What can it be used for?
• What next? (Dam, Reference Dam1995, p. 49)
There is obvious common ground between these questions and the five questions we used in section 3.1 to summarize good practice in the design, implementation, and validation of assessment instruments. Procedures designed to support autonomous learning insist on explicitness and transparency, elicit knowledge that cannot be generated externally, and are reliable and valid to the extent that they lead to effective learning and provide a sound basis for further learning. The successful operationalization of the procedures requires that classroom interaction is based on an ethic of responsibility and respect. Learners in interaction with their teacher generate their own feedback and, crucially, their own “feedforward”; from the first, they contribute to the elaboration of evaluative criteria. Reflective processes begin as a whole-class activity managed by the teacher and carried out as far as possible in the target language. At regular intervals, learners record their own judgments in the logbooks that they use to manage their learning.
Dam and Legenhausen (Reference Dam, Legenhausen, Paran and Sercu2010, Reference Dam and Legenhausen2011) explore the impact of these procedures on learners’ awareness of linguistic competence, how they should proceed, and preferred/dis-preferred activities (Dam & Legenhausen, Reference Dam, Legenhausen, Paran and Sercu2010, pp. 127–130); and on their awareness of the setting and organization of the classroom, social interaction, and language and communication (Dam & Legenhausen, Reference Dam and Legenhausen2011, pp. 185–186). They also consider learners’ reflections on their capacity to set their own learning goals, form working groups, evaluate the effectiveness of working procedures, evaluate one another, and assess their own learning outcomes and needs in relation to future learning (Dam & Legenhausen, Reference Dam, Legenhausen, Paran and Sercu2010, pp. 130–136). Within this reflective-evaluative framework, learners regularly rated themselves, sometimes using criteria that they negotiated with their teacher, sometimes using the scales and criteria that were applied in national exams. Numerous quotations from learners’ logbooks confirm the development of a proficiency in L2 English that is metacognitive as well as communicative. This version of self-assessment is an essential foundation of teaching and learning rather than an activity that is added to existing classroom practice.
The same approach to self-assessment has been advocated for use with the ELP (Little, Reference Little2009) and has occasionally been implemented (Little, Reference Little2005), but perhaps because the ELP has failed to gain significant purchase in Europe's education systems, there appear to be no empirical studies of this dimension of ELP use to complement the research reported by Moeller, Theiler, and Wu (Reference Moeller, Theiler and Wu2012). They undertook a 5-year quasi-experimental study that introduced LinguaFolio (an American cousin of the ELP) into 23 high schools and found a statistically significant relationship between goal setting based on checklists of “I can” descriptors and learning achievement in L2 Spanish.
Butler and Lee (Reference Butler and Lee2010) undertook a study of the impact of self-assessment on young learners of English in South Korea. A summative self-assessment instrument was administered at the beginning and end of the study, and unit-based instruments were used at the end of each unit of learning. Both types of instrument mostly required learners to indicate on a 5-point scale the extent to which a series of first-person statements applied to them. Statistical analysis of the data revealed only marginal effects for learners’ proficiency in English and their confidence (Butler & Lee, Reference Butler and Lee2010, p. 18). This is hardly surprising, however. Although Butler and Lee's (Reference Butler and Lee2010) literature review discusses at some length the theoretical importance of self-assessment for learning, its practice was evidently not deeply rooted in the teaching/learning culture of the classrooms that participated in the study. In any case, Butler and Lee's conceptualization of self-assessment seems to leave little room for the exercise of learner agency. And although their use of first-person statements recalls the checklists of “I can” descriptors on which the ELP bases self-assessment, the descriptors they devised use the past rather than the present tense, and most of them focus on classroom activities (e.g., “I could understand sentences in the textbook”, “I could write the sentences that we learned in English classes in the 5th grade”; Butler & Lee, Reference Butler and Lee2010, p. 30) rather than communicative tasks. The past tense encourages learners to look back rather than forward, and the classroom focus can do little for the learner's sense that the ability to communicate in the target language is somehow enlarging her or his identity.
Self-assessment that is limited to self-rating and is not embedded in more general evaluative reflection can hardly be expected to yield pedagogically informative results. Another potential limitation has to do with teacher–learner power relations: Does self-assessment really empower learners and enhance their autonomy—grant them greater agency—if the teacher has the right to correct or veto their judgments? This question was posed by Tan (Reference Tan2004), who argued that students’ judgments of themselves yield information that would not otherwise be available to the teacher, and this makes them “subject to greater control and surveillance as a result of exercising more autonomy in their assessment” (Tan, Reference Tan2004, p. 659). The solution that Tan (Reference Tan2004) proposed is to focus on how self-assessment can benefit student learning rather than on the extent to which it enhances their autonomy. This coincides with the approach to language teaching/learning that we have argued is implied by the CEFR and the discussion of self-assessment with which we began this section. The dangers that Tan discusses are nevertheless real and must be addressed wherever self-assessment is on the teaching/learning agenda.
3.2.2. Peer assessment
The reflective procedures that Dam and Legenhausen (Reference Dam, Legenhausen, Paran and Sercu2010, Reference Dam and Legenhausen2011) describe are grounded in evaluative reflection that starts life as oral interaction between teacher and learners and among learners. The self-evaluations that learners write in their logbooks are shared with the teacher and other learners, which exposes them to challenge and/or modification; that is how peer assessment begins. As Dam and Legenhausen (Reference Dam and Legenhausen2011) point out, “it is the ‘expertise’ gained from constantly evaluating oneself that qualifies the learners to also comment on their peers’ performance” (p. 182). Such comments are potentially face-threatening and require sensitivity, tact, and empathy, all qualities that are worth developing in learners. When peer assessment is first introduced, learners are instructed always to mention positive as well as negative features and to suggest possible improvements, and the teacher intervenes if the process becomes excessively critical.
Most published research on peer assessment is concerned with university students. Matsuno (Reference Matsuno2009), for example, set out to explore the relationship between self-assessment and peer assessment on the one hand and teacher assessment on the other in Japanese university English as a foreign language writing classrooms. The aim was to discover whether self-assessment and peer assessment in this context were sufficiently reliable to include as a part of formal assessment. The study focused on 300-word essays written by each member of the two participating classes. The students were given an analytic scoring grid that they used together in class to evaluate three essays. They were then asked to evaluate their own essay and the essays of five of their peers at home. One teacher evaluated all of the essays, while three other teachers each evaluated a third of the essays; there were thus two independent teacher evaluations for each essay. Analysis showed that self-assessment was somewhat idiosyncratic, whereas peer assessment was consistent and not influenced by students’ own writing performance. As with Butler and Lee's (Reference Butler and Lee2010) research, there is no indication that the participants in Matsuno's study were being taught in ways that encouraged reflective learning. It is thus perhaps not surprising that they performed the external task of evaluating their peers’ essays more accurately than the subjective task of evaluating their own.
Saito (Reference Saito2008) explored the effect of training on students’ rating of and comments on their peers’ work, finding that there were no significant differences between treatment and control groups with regard to rating, but that the treatment groups were superior when it came to commenting on their peers’ work. Zhao's (Reference Zhao2010) investigation of learners’ use and understanding of peer and teacher feedback on writing similarly involved university students learning English. The study revealed that learners made more use of teacher than of peer feedback, but without understanding its significance or value. In neither of these studies, however, was peer assessment grounded in reflective learning, so they cannot be said to illustrate the integration of learning and assessment, and it is difficult to know what value to attach to their findings.
3.2.3. Teacher assessment
The literature on teacher assessment is extensive, and much of it stresses the importance of good assessment practice to effective learning. For reasons of space we shall limit ourselves to a consideration of the extent to which three current approaches appear to engage learner agency.
The first approach, assessment for learning (AfL), is concerned with assessment across the curriculum and was established in England at the end of the 1990s, partly in response to the government's attempt to raise educational standards by increasing the number and frequency of external tests. In 1999 the Assessment Reform Group (ARG) proposed that “the focus needs to be on helping teachers use assessment, as part of teaching and learning, in ways that will raise pupils’ achievement” (ARG, 1999, p. 2). According to the ARG, improving learning through assessment depends on “five, deceptively simple, key factors”:
• the provision of effective feedback to pupils
• the active involvement of pupils in their own learning
• adjusting teaching to take account of the results of assessment
• a recognition of the profound influence assessment has on the motivation and self-esteem of pupils, both of which are crucial influences on learning
• the need for pupils to be able to assess themselves and understand how to improve (ARG, 1999, pp. 4–5)
Despite the reference to the involvement of pupils in their own learning and their need to be able to assess themselves, the teacher seems to be firmly in control. This is further confirmed by some of the stated characteristics of AfL (ARG, 1999): “It involves sharing learning goals with pupils” (rather than helping pupils to set their own goals on the basis of the official curriculum); “it aims to help pupils to know and to recognize the standards they are aiming for” (rather than engaging them in setting appropriate standards for the classroom, again with reference to the official curriculum); and “it provides feedback which leads to pupils recognizing their next steps and how to take them” (rather than involving pupils in creating the informational structures to generate their own feedback) (p. 7).
Proponents of AfL argue that self-assessment is “essential to learning because students can only achieve a learning goal if they understand that goal and can assess what they need to do to reach it” (Black & Wiliam, Reference Black, Wiliam and Gardner2012, p. 18). But the teacher remains firmly in control: “the criteria for evaluating any learning achievements must be made transparent to students to enable them to have a clear overview both of the aims of their work and of what it means to complete it successfully” (Black & Wiliam, Reference Black, Wiliam and Gardner2012, p. 18). James and Pedder (Reference James, Pedder and Gardner2006) argue that when AfL is fully implemented, “it gives explicit roles to learners, not just to teachers, for instigating teaching and learning”; “learners are brought into the heart of teaching and learning processes and decision making as they adopt pedagogical practices to further their own learning and that of their peers. It gives the old expression of being ‘self-taught’ a new meaning” (p. 28). Learner agency is clearly in play here. But whereas for James and Pedder, teacher-led AfL stimulates peer assessment and self-assessment, which generate a clearer understanding of the learning process, which leads to learner autonomy, the approach to learning and teaching we described in sections 2.2 and 3.2.1 moves in the opposite direction. Learners begin by evaluating their own learning; this gives them the experience and knowledge necessary to evaluate the learning of their peers; and the evaluative discourse in which these procedures are embedded helps to inform and shape teacher assessment.
Like AfL, learning-oriented assessment (LOA) is concerned with assessment in general, not just the assessment of language proficiency. According to one of its early proponents (Carless, Reference Carless2007), LOA is governed by three principles: “assessment tasks should be designed to stimulate sound learning practices among students”; “assessment should involve students actively in engaging with criteria, quality, their own and/or peers’ performance”; and “feedback should be timely and forward-looking so as to support current and future student learning” (pp. 59–60). The second of these principles assigns an essential role to learner agency, though it should be pointed out that Carless's (Reference Carless2007) focus is higher education, where (whatever the reality) students are traditionally expected to be proactive and autonomous. In a later article, Carless (Reference Carless, Meyer, Davidson, Rees, Fletcher, Johnston and Anderson2009, p. 4) describes LOA as an attempt to reconcile formative and summative assessment. An essential role is assigned to self-monitoring, self-assessment, and peer feedback (Carless, Reference Carless, Meyer, Davidson, Rees, Fletcher, Johnston and Anderson2009, p. 4), but it is unclear whether they arise quasi-naturally from reflective learning processes or are added to existing teaching practices—unclear, in other words, whether or not learning and assessment are fully integrated. LOA has been applied to the assessment of language proficiency and has attracted the attention of language-testing agencies. Cambridge English, for example, is currently promoting LOA as “a systematic approach to language learning that uses formal and informal assessment to help teachers and learners to plan learning more effectively; measure progress; identify areas for improvement; deliver measurable improvements” (Cambridge English, n.d.). It is unclear, however, whether learner agency has any role to play, and only time will tell whether LOA can have a significant impact on language learning.
Dynamic assessment (DA) derives from Vygotsky's sociocultural psychology, specifically his concepts of the zone of proximal development (ZPD; Vygotsky, Reference Vygotsky1978, p. 86) and mediation. The application of DA to the assessment of language proficiency has been a particular concern of the Center for Advanced Language Proficiency Education and Research (CALPER) at Pennsylvania State University, which has produced a large proportion of recently published work (e.g., Lantolf, Reference Lantolf2009; Lantolf & Poehner, Reference Lantolf, Poehner, Shohamy and Hornberger2008, Reference Lantolf and Poehner2011; Reference PoehnerPoehner, 2009). The center's website offers this definition:
Contemporary approaches to assessment—including large-scale standardized testing, performance assessments, and classroom-based assessments—require the examinee to function in relative isolation. Dynamic Assessment, in contrast, prescribes mediation of the examinee's performance as an integral part of the assessment process. Dynamic Assessment calls for the unity of assessment and instruction with the goal of learner development. (CALPER, n.d.)
As Lantolf and Poehner (Reference Lantolf and Poehner2011) have pointed out, the ZPD proposes that development moves from other-regulation to self-regulation (p. 17). This perhaps accounts for the fact that DA is teacher-led and seems to offer little room for the exercise of learner agency. According to Lantolf (Reference Lantolf2009), DA rests on the dialectical integration of instruction and assessment (not learning and assessment), and elsewhere Lantolf (Reference Lantolf, Benson and Cooker2013, p. 20) agrees with Kozulin that individuals become independent agents only “towards the very end of the formal learning experience” (Kozulin, Reference Kozulin1998, p. 3). In that case, how does agency develop? How do learners become independent agents if not through the exercise of agency? And if they pass through a succession of ZPDs, is the capacity for independent task performance achieved at the end of one ZPD irrelevant to progress through the next? Whatever the answers to these questions, Lantolf (Reference Lantolf, Benson and Cooker2013) seems not to believe that learner agency has much to contribute to the learning process, arguing that “teacher–learner dialogue is preferable to peer mediation because it increases the chances of pushing learners to be as explicit as possible” (p. 22).
3.3. Aspects of learner identity and agency in large-scale testing and assessment
As previously emphasized, all types of assessment rest on the same basic principles, though these are operationalized in different ways depending on contextual factors. Transparency is a fundamentally important concept that is closely related to validity and reliability, but also to ethics and hence to respect (in a broad sense) for the uses and users of tests, in particular students and test takers. Tasks and items should be specified as clearly as possible, both in relation to the construct and with regard to more technical parameters concerning, for example, target domains, functions, proportions, and formats. This is essential in order to create stability and to facilitate empirical studies of proficiency development over time. Specifications of tasks and items also need to be available in versions that are comprehensible and meaningful to practitioners, including students, relating different tasks to what students are supposed to learn and contributing in this way to the clarification of goals and expected learning outcomes. Openness and clarity may also be considered in relation to the concepts of power and empowerment—transparency can be assumed to have an important function in relation to students’ sense of agency and confidence. To meet this requirement, information materials and sample tasks are often provided, both by national authorities and testing agencies. Helpful though they can be, however, such materials are not without their dangers; in particular, they can encourage instrumental teaching and learning to a test rather than to the intended construct in all its complexity.
Validity is a fundamental concept in all types of assessment, requiring optimal quality of instruments and procedures to ensure optimal use of the observations made. Moss, Girard, and Haniford (Reference Moss, Girard and Haniford2006) provide the following characterization of the concept:
Professionals working in different contexts have different decisions to make, different sources of evidence, different resources for interpreting the available evidence, and different administrative constraints on their practice. Educational assessment should be able to support these professionals in developing interpretations, decisions, and actions that enhance students’ learning. Validity refers to the soundness of those interpretations, decisions, or actions. (p. 109)
Whether regarded as integrated in the concept of validity or as a separate phenomenon, reliability is an inevitable prerequisite for validity; nothing can be claimed to be truly valid, if it cannot be shown that all possible measures have been taken to make it as trustworthy as possible. This is an obvious concern in large-scale assessment, with degree and type of standardization as an important aspect, but the phenomenon has to be addressed in small-scale circumstances as well, one example being whether students have actually understood instructions correctly.
One way, among many others, of handling the fundamental quality demands in large-scale assessment is to actively involve different categories of stakeholders in the development and composition of instruments and procedures. An obvious example of such collaboration—a participatory approach, as it is sometimes called—is to work with teachers; another is to engage researchers from different disciplines. However, students are undoubtedly the largest group affected by testing and assessment. The value of actively involving students has been pointed out by many researchers in the language assessment field (e.g., Alderson, Clapham, & Wall, Reference Alderson, Clapham and Wall1995; Bachman & Palmer, Reference Bachman and Palmer2010; Shohamy, Reference Shohamy2001), and the decision to do so can easily be justified in terms of validity in a wide sense that embraces democratic principles, ethics, and potential pedagogical impact. An example of the use of test-taker feedback in the development of national, high-stakes tests can be found in the Swedish educational system. Since the middle of the 1990s, students’ opinions have been systematically collected in large-scale (n ≈ 400) pretesting of all materials, for all age groups from young learners to adults, and analyzed and used to inform the final stages of test composition.
Among the more salient results are the following: Students mostly respond quite positively to the materials offered (this is especially true of younger learners; they seem to prefer tasks that they perceive as authentic (“real”) and/or that require language production and interaction); they are usually accurate in their estimation of task difficulty; and girls tend to underestimate themselves when asked to assess their achievements retrospectively (Erickson & Åberg-Bengtsson, Reference Erickson, Åberg-Bengtsson, Tsagari and Csépes2012). Analysis of open responses given by Swedish students and participants in a European survey of students’ and teachers’ views on language testing and assessment (Erickson & Gustafsson, Reference Erickson and Gustafsson2005), has shown that students’ comments quite often coincide with basic definitions of validity. Compare Messick's (Reference Messick and Linn1989, p. 34) definition of construct underrepresentation—“the test is too narrow and fails to include important dimensions or facets of the construct”—with what a 15-year-old student wrote: “A bad test/assessment is the one which is only about grammar, because if a person knows grammar well, it doesn't mean he/she can speak the language as well and communication is the most important thing in language study”; or Messick's definition of construct-irrelevant variance—“the test contains excess reliable variance that is irrelevant to the interpreted construct”—with what two European teenagers had to say: “if you don't understand the questions, you can't show what you know”; “a bad test is when we just have a little time to do it on. Because then you have to stress thru the test and maybe you cannot show how much you can” (Erickson, Reference Erickson, Kao and Lin2010, pp. 251–252).
In conclusion, it is worth remembering that large-scale assessments and examinations are rare phenomena in the life of learners. This, however, further emphasizes the need to adhere to basic principles of good practice, because large-scale assessment most often generates results that are crucial for students’ future choices and chances, and thus for their self-esteem, sense of identity, and agency.
4. CONCLUSION
We began this article by proposing that education is a process of people shaping that seeks to engage and exploit the individual's agency while extending and/or modifying his or her identity. We argued that the CEFR's action-oriented approach to the description of language proficiency as language use is harmonious with this view; according to the CEFR, proficiency develops as a result of the individual's engagement in and monitoring of communicative activity. The CEFR's companion piece, the ELP, is explicitly designed to support learner agency while providing a dynamic reflection of his or her developing identity as language learner and user. We further argued that according to this interpretation, the CEFR implies that teaching/learning and assessment should be reciprocally integrated. Our review of current trends in language assessment has found support for this kind of assessment culture in some approaches to self-assessment and peer assessment and in the use of test-taker feedback in large-scale, high-stakes assessment. Other approaches to self-assessment, peer assessment, and teacher assessment that aims to improve learning, on the other hand, seem to leave rather little room for the exercise of learner agency. There is no doubt that all forms of language assessment show a greater concern for the learner now than they did 20 or 30 years ago. But in order to meet the challenge posed by the broad and only partially recognized perspectives of the CEFR, more work needs to be done to develop approaches to teaching and learning that assign a central role to learner agency and identity, recognizing that the efficient exercise of agency depends on accurate assessment, and effective language learning requires the full engagement of the learner's identity.
