Language Policy and the CEFR

Language policy at a European level, as opposed to the national policies of particular European states, is shaped by two organizations: the Council of Europe and the European Union. The former is the larger organization (47 member countries) and the older. It does not make economic policy and is most influential in the field of human rights. The statutory institutions of the Council of Europe include the Committee of Ministers, comprising the foreign ministers of each member state; the Parliamentary Assembly, composed of members of Parliament (MPs) from the Parliament of each member state; and the secretary general, heading the secretariat of the Council of Europe. The European Cultural Convention states that “the aim of the Council of Europe is to achieve a greater unity between its members for the purpose . . . of safeguarding and realising the ideals and principles which are their common heritage” (Council of Europe, 1954). The Council of Europe's activities to promote linguistic diversity and language learning are carried out within the framework of this convention. Its Language Policy Division implements agreed intergovernmental programs with a special emphasis on policy development. This work has emphasized communication, exchange, and intercultural awareness. The influential Waystage and Threshold learning objectives, first published in the 1970s (Van Ek & Trim, 1990a, 1990b), marked the beginning of efforts to identify a set of European levels of language proficiency, work that culminated in the publication of the Common European Framework of References for Language (CEFR) (Council of Europe, 2001).

The European Union (EU) is the political grouping of 27 countries. Political leadership and the basis for policymaking in the EU comes from the European Council, which is composed of one representative per member state—the head of state or head of government—plus the president of the Commission. The European Commission (EC) acts as the EU's executive arm and is responsible for initiating the legislation and for the day-to-day running of the EU. The European Parliament, with 785 directly elected members, forms half of the EU's legislature; the Council of the European Union forms the other half.

Although the Council of Europe offers advice or advocacy, policy guidelines at EU level tend to be concrete and binding. Education remains outside the EU's direct control, according to the “principle of subsidiarity.” However, EU decisions may strongly impact on education policies (Bonnet, Reference Bonnet2007, p. 670).

Current language education policy themes are, of course, heterogeneous: teacher training, lifelong and early learning, migrants and languages of schooling, the “language friendly school,” CLIL (Content and Language Integrated Learning), the use of Information and communication technologies (ICTs), and so on. However, the theme that brings these strands together at a European level is that of multilingualism. The key concepts of multilingualism and plurilingualism are presented in Council of Europe (2007). Multilingualism being a fundamental feature of European societies, language education is an important aspect of social policy, particularly regarding notions of social inclusion and shared democratic citizenship. The term plurilingualism, which focuses on the individual rather than the collective, is a specifically Council of Europe concept, not used in EU discourse.

The Language Policy Division section of the Council of Europe web site illustrates the range of current areas of work. A recent focus is on the languages of education (Fleming, Reference Fleming2008). The Council also assists member states in producing self-evaluative language education policy profiles, as an aid to defining possible future policy developments.

The EU's language policy is best seen as part of the so-called Lisbon Strategy, the goal set in 2000 for Europe to become by 2010 “the most competitive and dynamic knowledge-based economy in the world” (Council of the European Union, 2000, p. 2). In 2002 member countries agreed action to improve language learning, in particular by teaching at least two foreign languages to all from a very early age. A European Indicator of Language Competences was first mooted at this time as one measure of progress toward the Lisbon goals. An action plan to promote language learning and linguistic diversity (European Commission, 2003) stressed among other things that “English is not enough” (p. 7); it reflected the concerns of some member countries to counteract the increasingly dominant role of English as a lingua franca seen to threaten other traditionally important languages such as French and German. Multilingualism now has its own commissioner. The EU Multilingualism Web site provides access to information about a wide range of activities relevant to shaping language policy.

The CEFR is a key element in both organizations' approach to framing language policy. It sets out to provide “a common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe” (p. 1). Developed by the Council of Europe and adopted by the EU as an instrument of policy, it has rapidly become ubiquitous. Studies conducted by the Council of Europe in 2005 and 2006 (Council of Europe 2006; Martyniuk & Noijons, Reference Martyniuk and Noijons2007) concluded that in all countries the CEFR has had a major impact at the level of policy. It was found to be most useful for the planning and the development of curricula and syllabi, and equally for the development of tests and certification, though somewhat less so for teacher education and training. However, at the level of the classroom the impact was found to be less. The framework was widely accepted, but its full potential not yet realized, because it is found difficult to understand and teachers are not equipped to make use of it.

Thus some writers have concluded that the CEFR's major influence to date has been on assessment (Coste, Reference Coste2007; Fulcher, Reference Fulcher2008; Little, Reference Little2007). Certainly the publication of the CEFR and the subsequent preliminary pilot version of a manual for relating language examinations to it (Council of Europe, 2003) has stimulated a great deal of work by language testers. It has led to increasing insistence by governments and other test users on proof of alignment, and a rush on the part of testers to provide it. This influence is set to increase. A recent recommendation on the use of the CEFR by the Council of Ministers (Council of Europe, 2008) includes the call for countries to

The European Survey on Language Competences, now scheduled for delivery in 2011, will deliver the long-awaited Indicator, and will use the CEFR as the benchmark for reporting on the language competences of secondary-school pupils across Europe, lending further visibility to the CEFR as an assessment framework.

This somewhat one-sided adoption of the CEFR may be linked to a one-sided reading of its text, as noted by Coste (Reference Coste2007): “In various settings and on various levels of discourse . . . people who talk about the Framework are actually referring only to its scales of proficiency and their descriptors.” Whatever the reason, one can identify a disjunction between the current widespread adoption of the CEFR in assessment and the framing of targets, and its generally less developed application to classroom practice and to specific learning contexts.

For some commentators this situation illustrates the CEFR's lack of construct validity as the framework for learning, teaching, and assessment that its title lays claim to. For example, Fulcher (Reference Fulcher2004, Reference Fulcher2008) comprehensively rejected the CEFR, on the grounds that its descriptive scales, being empirically derived on the basis of teacher judgments, are atheoretical, have no basis in second language acquisition (SLA) research, and describe progression in a circular and reductive manner (Fulcher, Reference Fulcher2008, p. 22).

Others interpret the situation described above as evidence that the CEFR is being exploited as a tool of authority and control—“manipulated unthinkingly by juggernaut-like centralizing institutions” (Davies, Reference Davies and Shohamy2008, p. 438, cited by Fulcher Reference Fulcher2008, p. 21). As Trim, one of the principal framers of the framework, conceded, “there will always be people who are trying to use it as an instrument of power” (Saville, Reference Saville2005, p. 282).

But the impact of the CEFR's rise to prominence as an instrument of policy is far more complex than is suggested by any such simple interpretations, and this raises issues that language testers cannot but engage with. Coste (Reference Coste2007) identified several trends and tensions. It has stimulated a new market in language testing, throwing doubt on test quality and on comparability of standards. Tests are tending to simplify reporting of outcomes, neglecting the CEFR's potential for profiling abilities, and limiting possible differentiation of contexts. Doubts are raised about the adequacy of the current CEFR as an assessment framework. In this context some would see the Manual for Relating Examinations to the Common European Framework of Reference for Languages as an obligatory prescriptive methodology. Alternatively, some would replace the CEFR with a different, more precisely graduated framework. From many quarters come demands for some centralized authority to be appointed: a “CEFR police” to enforce standards. Let us take this last proposal as the starting point for presenting our own viewpoint on these issues.

The notion of policing the CEFR assumes some authorization to use it properly. The metaphor seems faulty: people speak of applying the CEFR to some context, as a hammer gets applied to a nail. We should speak rather of referring a context to the CEFR. The transitivity is the other way round. The argument for an alignment is to be constructed, the basis of comparison to be established. It is the specific context that determines the final meaning of the claim. By engaging with the process in this way, we put the CEFR in its correct place as a point of reference and also contribute to its future evolution. A policed CEFR would almost by definition be a closed-down, fossilized system, whereas we should certainly be thinking in terms of its further development, as Coste himself foresees.

We can assert that the notion of policing implies elevating a quantitative, reliability-driven conception of the framework over a qualitative, validity-driven one. Also, readiness expressed to take on such a policing role (Alderson, Reference Alderson2007, p. 662) must raise the question quis custodiet ipsos custodes?

But although policing the use of the CEFR is certainly not the way to go, there is certainly a great deal of work to be done to guarantee its sensible use and ensure that its impact on the learning and teaching of languages turns out on balance positive. This work involves actors at every level of language education, and in every field, and there is a large role for assessment and measurement expertise. The use of the CEFR for assessment carries risks, as is true of any large-scale assessment initiative. It is reasonable that language testers should work to understand these, and even to mitigate or overcome them. The CEFR is a fit object for the study of impact, as that concept has been developed within language testing (Alderson & Wall, Reference Alderson and Wall1993; Hawkey, Reference Hawkey2006).

Within language testing, the concept of impact has emerged as a fundamental consideration for test developers. The notion of washback, that is, the effects of examinations on learning and teaching, has been expanded in recent years to cover the interaction of examinations with societal processes more generally. Researchers now realize that to understand the consequences and effects of assessment procedures more effectively, they need to gain better understandings of the ways in which their tests and examinations fit within complex educational systems; these systems are themselves influenced by sociopolitical and economic factors and the relevant policy decisions made at different levels within society (local, national, regional).

Saville (Reference Saville2009) included the concept of impact by design in his model of impact; he suggested that test developers, in responding to policy initiatives and requirements for new tests, should anticipate scenarios resulting from the uses of their tests and plan to take action of various kinds—to find out what happens and to mitigate risks of negative impacts. This planning needs to be built in at the start of the design and development process and should be an aspect of ongoing validation over the long term.

In a similar vein, Chalhoub-Deville (Reference Chalhoub-Deville2008) referring to the case of the No Child Left Behind legislation in the United States, argued that those engaged with large-scale educational initiatives have a duty to anticipate and attempt to influence the effect that they exert. In this respect she suggested that language planners and policy advisers on assessment should look to the literature on “social impact assessment” (SIA) for guidance (see her article in this volume).

This concept of impact by design acknowledges that it is not possible to predict impact based on the implementation of policy-related initiatives and recognizes the complexity of the contexts within which assessments operate. Much of it lies beyond the control of the assessment provider, or possibly anyone else and therefore causal models and expectations of linearity are not realistic. Nonetheless, a couple of case studies may illuminate some of the factors that bear on the impact of assessment initiatives, and show where effort and attention may need to focus in order to secure intended benefits or mitigate predictable problems.

Case Studies: Asset Languages and the European Language Portfolio

Asset Languages, a project coordinated by Cambridge ESOL (English for speakers of other languages) during its development phase, is a multilingual assessment framework developed as part of the United Kingdom's national languages strategy. It represents a particularly explicit national language policy initiative that aligns closely with European concerns: specifically, with the use of a proficiency framework to organize learning and interpret learning outcomes. The national languages strategy was launched in 2002 to counteract a serious downward trend in language learning. A key element of this strategy was to provide a complementary assessment framework as an alternative to current schools assessments. Called the Languages Ladder, this framework was presented as “a new voluntary recognition system to complement existing national qualifications frameworks and the Common European Framework” (DfES, 2002).

The study that proposed this new recognition system (Nuffield Languages Programme, 2002) found existing qualifications inadequate as mechanisms for promoting successful language learning. They could not function formatively, given their summative role at the end of an extended period of study. Consequently, the new framework should embody a set of graded, accessible learning targets. Also, many qualifications in languages were found to be “confusing and uninformative about the levels of competence they represented” (p. 8); thus the new framework should embody meaningful proficiency levels. For both these purposes the CEFR was identified as a model to be followed.

Other shortcomings of current qualifications included their lack of availability for a wide range of languages, including those spoken as community languages. Neither did they provide a suitable framework for lifelong learning.

Despite these criticisms, the framers of the original recommendations did not see the new framework entering the field of formal qualifications, but somehow operating alongside it and offering informal certification. However, the scheme as launched envisaged twin strands of accreditation: an informal, teacher-led one of micro-levels, and an externally administered assessment scheme at six major levels having equal formal status to other language qualifications within the National Qualifications Framework. This was the scheme that Cambridge Assessment was awarded by tender and developed under the brand name Asset Languages. It encompasses 25 languages; the skills of reading, writing, listening and speaking separately accredited; and three contexts (primary, secondary, adult), with differentiation of test content for each context. The external assessments are offered frequently throughout the year, enabling a test-when-ready approach.

The Department for Education and Skills (DfES) provided “approximate” equivalences between the Languages Ladder and the CEFR. In practice the two could be treated as broadly equivalent, and Cambridge ESOL made close reference to the CEFR and to reference test material in English at different CEFR levels, in developing and scaling the tests.

The system is now fully operational, if not complete in terms of the levels covered by each language; however, it is still too early to begin to evaluate its success. But working on the project has been instructive in a number of ways. Beyond the merely technical challenge of bringing multiple languages onto a common proficiency framework, it reveals challenges of a different order related to introducing a new assessment, having its rationale and values, into an education system with its own rather different drivers. Simply accommodating individualized testing-when-ready is difficult within a system used to a cohort-based lockstep approach; likewise, a proficiency-testing approach based on minimal exam preparation, when the norm is achievement testing of closely specified and heavily prepared objectives. The value of clear criterion-referenced proficiency levels may count for little to school heads focused on the value of performance points attached to achieving exam grades. Moreover, the end users of exam results may not value evidence of useful language skills, however acquired, if they are accustomed to valuing them simply as evidence of transferable academic ability. Some of these may be specifically UK issues, and others may have more general resonance.

Asset Languages offers the two options of external assessment and informal teacher-led certification. The latter, intended to be used formatively, takes the form of short tasks that can be used off-the-shelf or adapted for particular contexts. Feedback from teachers suggests that incorporating the scheme into their teaching is seen as introducing a considerable amount of extra work, which they may feel too busy or insufficiently confident to deal with. This is one reason why they tend to use the supposedly formative tests unchanged, as mini-summative events unconnected to their teaching. Teachers also want more detailed information on the content of the external exams, to prepare pupils better. Generally, they want more guidance in relating their classroom work to the functional, can-do framework of the Languages Ladder.

This parallels the findings of surveys into the use of the CEFR mentioned above, and also of another Council of Europe initiative, the European Language Portfolio (ELP), which is closely linked to the CEFR and has focused the most coherent and principled attempts to integrate it into classroom learning. The ELP serves the pedagogical function of encouraging reflection on the language learning process and the development of learner autonomy; at the same time it documents the learner's developing proficiency (Little, Reference Little2007, p. 649). Little is an advocate for the ELP and has produced extensive support material for teachers using it (Little & Perclová, Reference Little and Perclova2001), but admits himself that it is impossible to say how widely it is actually used and refers to “a wealth of anecdotal evidence to suggest that because most models have been developed independently of curricula, teachers and learners see the ELP as an optional extra whose use will involve them in extra work” (Little, Reference Little2007, p. 652). He concludes that wide uptake of the ELP in the longer term depends on establishing an integral relationship between ELP models and curriculum, as has successfully been done with the primary level ELP in Ireland.

Thus the issues faced by both the ELP and Asset Languages may to some extent result from their voluntary status. Compare this with the situation in France, where the CEFR has been integrated more completely into the national curriculum (Bonnet, Reference Bonnet2007, p. 670). This began with the establishment of performance indicators and targets in CEFR terms, and continued with fundamental reforms of foreign language teaching explicitly organized around the CEFR. Targets are set for three stages of education, and foreign language (FL) curricula have been rewritten. Professional development of language teachers is accordingly being refocused on teaching and assessing in relation to the CEFR. The French example is likely to be followed elsewhere. Certainly the previously mentioned recommendation on the use of the CEFR by the Council of Ministers (Council of Europe, 2008) points strongly in this direction.

The European Survey on Language Competences

The European Survey on Language Competences, which will deliver the European Language Indicator, is currently under development by SurveyLang, an international consortium of eight partners. These are the Centre International d'Études Pédagogiques (CIEP) of France, Gallup Europe, Goethe Institut of Germany, Instituto Cervantes of Spain, National Institute for Educational Measurement (Cito) of The Netherlands, Universidad de Salamanca of Spain, Università per Stranieri di Perugia of Italy, University of Cambridge ESOL Examinations of the United Kingdom.

As members of this team, we will introduce the project and relate it briefly to the context discussed earlier. The survey is intended to provide information on the general level of foreign language knowledge of the pupils in the participating countries and provide strategic information to policymakers, teachers, and learners with the aim of improving teaching methods and outcomes. The language tests and interpretation of results must relate to the CEFR and cover the levels A1 to B2.

SurveyLang was awarded the contract in February 2008, the European Language Indicator having been first proposed in 2002. This long gestation reflects difficulty in agreeing on terms of reference. In the structure of the Commission the project is being managed by a unit dealing with several educational indicators, within a directorate concerned with lifelong learning and the pursuit of Lisbon policy objectives, within the Directorate-General for Education and Culture. The terms of reference have been agreed in consultation with an advisory board comprising nominated experts for each country. Finally, considerations of practicality have kept the scope of this first run of the survey quite narrow: pupils at the end of lower secondary education (or higher secondary if that is where a second foreign language is first taught), and only the two most taught foreign languages in a given country, out of English, French, German, Italian, and Spanish. Reading, listening, and writing are included in the first iteration, with speaking deferred for practical reasons. Only individuals currently learning a language will be sampled, and nobody is to be tested in more than one language. The Commission has stated that future iterations of the survey will include all European languages, and the skill of speaking, something which will entail a rather different approach.

Clearly, to measure and compare language proficiency levels in school settings across Europe is quite a challenge. Languages are introduced at very different ages, taught with differing duration and intensity, and as compulsory or optional subjects. Exposure to languages outside school varies, as does the impact of the culture that the language represents. The range of achievement within a grade-based cohort will be very wide, which is a challenge for efficient test design. Given the terms of reference, and the very different profiles of language study across countries, making meaningful comparisons will not be straightforward. Reporting a “league table” of outcomes by country, though it will doubtless be done, is to be discouraged.

These comparisons will be based on two kinds of data: from the language tests and from questionnaires administered to students, teachers, and school heads. The questionnaires will investigate, at a number of levels from the individual learner to the country, how educational outcomes relate to malleable processes (such as teaching methods) and antecedent or contextual conditions. The CEFR has a dual status in the survey: It is both the explicit point of reference and one of the variables whose impact may emerge in the analysis of outcomes.

At the time of writing, detailed proposals on conducting the survey have been presented to, and accepted by, the Commission and its advisory board of participating countries. From a language assessment point of view, key issues are the design of the language tests themselves, the alignment of tests across languages, and the assignment of testees to CEFR levels. Space precludes discussion here. Suffice it to say that the design of the tests aims to reflect the CEFR's action-oriented, functional model of language use while ensuring relevance for 15-year-olds in a school setting. Cross-language alignment and standard setting against the CEFR levels are current active areas of research in the language testing community that the survey aims to move forward. It is hoped that the survey will provide as one of its outcomes useful resources that can be made available to others engaged in this area.

The developers' remit is, of course, limited: the task is to construct valid assessment tools, collect good response data, and provide interpretable results. In the process we will, we believe, be able to contribute to the empirical construction of the CEFR as a practical multilingual framework. Beyond that, however, the survey's developers have no remit to promote the CEFR, or develop the links between pedagogy and assessment which this article has suggested are desirable and necessary. However, we see the survey, and the subsequent discussion of outcomes, as a favorable context for progress in this area. What is needed is thoughtful and positive engagement, including that of the language assessment community. The impact of the CEFR on language education is complex and heterogeneous. Coste (Reference Coste2007) insisted on careful analysis of each context of use. He merits quoting at length:

There is much in this for the language assessment community to take to heart. The program of work indicated here by Coste offers them substantive issues to engage with, and real opportunities to impact positively on language education in the European context.