The invited colloquium on New Directions in Language Assessment held at the American Association for Applied Linguistics (AAAL) annual meeting in Portland, Oregon on March 22, 2014 brought together an international panel of scholars to together explore the possibilities and challenges of translanguaging and bi/multilingual approaches in assessment. Together, the presenters of this colloquium argued that allowing bi/multilinguals to demonstrate their knowledge and skills using their entire linguistic repertoire offers a promising new approach for teaching and assessment.
Globalization and increasing language contact have galvanized a paradigm shift in applied linguistics, whereby a new line of theory and research has emerged that seeks to break away from static language constructs and instead offers more complex and fluid understandings about language (described in research as translanguaging, hybrid language practices, codemeshing, fluid lects, etc.; Canagarajah Reference Canagarajah2011; García & Li Wei Reference García and Wei2014). While recent approaches to assessment have introduced powerful high-stakes standardized tests, resulting in ever more rigid, narrow, and uniform conceptualizations of language correctness, these approaches overlook the actual language practices of those who are proficient in more than one language and/or are in the process of acquiring a new language. Several studies (e.g., Rea-Dickens, Khamis & Olivero Reference Rea-Dickins, Khamis, Olivero, Erling and Seargeant2011; Shohamy Reference Shohamy2011) suggest that the use of multilingual assessments significantly contributes to higher scores on academic tasks and more accurately reflects the knowledge of multilingual test takers. However, very little research has been conducted on the topic, and multiple challenges and questions remain.
This colloquium comprised scholars who accept translanguaging as a reality in the daily language practices of bi/multilinguals, and who have been engaged in this line of research in the U.S., the Basque Country, Israel, and South Africa, to move beyond the simplistic notion of ‘accommodations’ and account for multilingualism from the outset. The presenters were asked to respond to the following questions in their presentations:
a. What are specific cases of translanguaging practices in assessment within your context?
b. Based on data you have gathered, what are the major findings and the specific challenges that translanguaging poses for assessment of bi/multilingual children or adults?
c. How are these challenges being negotiated by educators, policymakers, and/or testers within your context?
d. What do you see as the agenda for further research on the topic?
Kate Menken and Elana Shohamy began the colloquium by defining translanguaging and reviewing empirical research conducted to date on translanguaging in assessment as outlined above (e.g., Rea-Dickens et al. Reference Rea-Dickins, Khamis, Olivero, Erling and Seargeant2011; Shohamy Reference Shohamy2011)
Durk Gorter (University of the Basque Country UPV/EHU – IKERBASQUE) followed with a presentation entitled ‘Approaches to multilingual assessment in education in the Basque Country.’ His paper, based on research he conducted with Jasone Cenoz, explored different approaches to multilingual assessment in education in the Basque Country (Spain), where Basque and Spanish are generally taught as school languages with English learned as the third language. The study investigates students who speak Basque and/or Spanish as their home language(s) and who are taught through the medium of Basque or through both Basque and Spanish in schools. Their schools aim at multilingualism as an outcome for their students, but the research showed how their ideology is to separate languages in teaching and assessment, and how the aims for English are usually somewhat lower. In this context, testing of language proficiency takes place for each of the three languages separately.
The paper suggests that multilingual assessment should consider students as multilingual speakers in their own right instead of using monolingual speakers as the yardstick. To that end, they collected and analyzed data examining different ways to assess language and content, among other elements, using a multilingual vocabulary availability test and analyzing the results of large scale Programme for International Student Assessment (PISA) testing in the Basque Country. There the testers experimented with different tests for home and school languages, which provided additional opportunities for evaluating the effects of bilingual testing. Gorter concluded by positing that a multilingual approach to language assessment is more valid and just, because it better resembles how languages are actually used in multilingual contexts.
Alexis Lopez, Danielle Guzman-Orth, and Sultan Turkan of the Educational Testing Service (ETS) presented the second paper, ‘A study on the use of translanguaging to assess the content knowledge of emergent bilingual students.’ They suggest using flexible bilingual assessments with emergent bilinguals to provide students with the opportunity to demonstrate both their content knowledge and skills using their entire linguistic repertoire. Their presentation reported findings from a study of the use of translanguaging to assess the mathematics content knowledge of emergent bilingual students.
They designed ten mathematics test items that were administered on a computer, incorporating translanguaging tools at the stem and response option levels (i.e., translation at the word and sentence level, read-alouds in English and Spanish, responding orally or in writing with English or Spanish). In their presentation, they showed screen shots of how this works in actual practice, in the first computerized test to incorporate translanguaging practices in actual testing. Twenty 9th grade students participated in the study. Target participants had the following characteristics: (1) low English proficiency, (2) attained literacy in Spanish, (3) high math abilities and (4) newly arrived. The presenters also used cognitive laboratories and usability protocols to elicit information regarding the students’ perceptions and use of the translanguaging tools. Their combination of empirical data in the form of test results and student perceptions showed that translanguaging tools made the assessment tasks more accessible, and they identified which translanguaging tools were more effective in supporting students in efforts to assess their knowledge of math content.
The third paper presented was by Kathleen Heugh (University of South Australia), entitled ‘Multilingual assessment: From bilinguality to multilinguality in South Africa.’ Heugh explains how, despite overt promotion of multilingualism, high-stakes assessments in South Africa have been administered in multiple versions of monolingual instruments in the recent past, thus preventing or limiting students’ translanguaging. In 2006 the Human Sciences Research Council developed two trilingual versions of a mathematics instrument to be used with students who speak Xhosa, Afrikaans and/or English, and who learn at least two of these languages at school. The intention was to reduce linguistic inequity, especially for speakers of Xhosa, who are expected to learn through the medium of their second language (L2) from the fourth grade of school onwards. It was also intended as an attempt to extend the boundaries of what was believed possible within the constraints of an administrative bureaucracy resistant to change.
Heugh describes how it was expected that speakers of Xhosa, the most widely used African language in the Western Cape, and possibly also speakers of Afrikaans, would be most likely to make use of translations. However, an unexpected result was that English speakers also found the use of translated items helpful. Surprisingly, students from each linguistic background made use of the trilingual translations, even speakers of English. The students from each linguistic cohort who made use of the translations indicated that they found these helpful, and achievement on translated items was higher than on those without translation. An attempt to achieve linguistic equity in the administration of system-wide assessment unexpectedly uncovered a surprising degree of multilinguality and translanguaging amongst all students.
Nelson Flores (University of Pennsylvania) and Jamie L. Schissel (University of North Carolina at Greensboro) presented the next paper, entitled ‘A water cycle-based approach to bilingual assessment in the era of the Common Core.’ The presenters argue that while the Common Core State Standards (CCSS) propose to create global citizens, they remain silent on issues related to bi/multilingualism. Flores and Schissel first overviewed the more harmful interpretations of No Child Left Behind in the testing of emergent bilinguals, and argue that CCSS policy and testing practices are monoglossic, treating bi/multilingualism as abnormal, and pressure emergent bilinguals to assimilate to an idealized monolingual standard American English norm.
Drawing from the New York State Common Core Initiative and fieldnotes, interviews, and classroom artifacts collected from ethnographic classroom case studies, they then focus on a different interpretation of the CCSS that promotes dynamic assessment with bilingual scaffolds rather than test accommodations. They introduce the water cycle as a metaphor to develop an ecological approach to conceptualize a role for bi/multilingualism within the CCSS. Their vision of a water cycle-based approach to assessment includes: (1) the use of ethnographic methods for connecting assessments with classroom practices; (2) the standardization of assessments in ways that treat bilingualism as the norm; and (3) the development of dynamic assessment approaches to content testing that account for various levels of new language proficiency.
Guillermo Solano-Flores (University of Colorado at Boulder) was the final presenter, whose work was briefly presented in absentia, in a talk entitled ‘Translanguaging-based approaches to linguistic variation in the assessment of linguistically diverse populations.’ His presentation examined the contribution of translanguaging-based approaches in testing as key to valid, fair testing for linguistically diverse populations in the assessment of academic achievement, and the technical challenges of developing sound tests with these approaches. The paper explored the relationship between linguistic variation and score variation and the need for addressing the fluidity of language from a probabilistic perspective. He argued that translanguaging-based testing approaches must address the tension between the following two goals: the linguistic features of test items should be accessible to all students on the one hand, yet also sensitive to the tremendous linguistic heterogeneity of bilingual students and bilingual classrooms on the other.
Several strategies were offered to address this tension. They include: (1) test localization, (2) the inclusion of representative samples of students and teachers who are users of the target languages in the process of test development, (3) the use of generalizability theory—a psychometric theory of measurement error—as a tool for examining linguistic heterogeneity in terms of sampling, and (4) the development of alternative conceptual frameworks of standardization that allow examination of the relationship between language and the constructs measured by tests.
Tim McNamara (The University of Melbourne) served as discussant for the session. After providing an overview of prior research on translanguaging practices in assessment, he shared a table in which he compared the approaches taken by the different presenters to answer the questions Menken and Shohamy posed at the outset (see Table 1). The discussant pointed out the theoretical nature of the papers by Flores & Schissel and by Solano-Flores, and how each provides a framework for approaching translanguaging practices in assessment. He noted how the papers by Gorter, Lopez et al., and Heugh each examine translanguaging in school-based assessments, with the research by Lopez et al. and Heugh also addressing standards-based assessment issues. Gorter and Heugh take on the topic of multilingual assessments, with Heugh looking at trilingual versions and Gorter at the evaluation of assessments, and institutional resistance to these efforts. McNamara then highlighted a number of the challenges posed by translanguaging and assessment, including teacher multilingual competence, resource intensiveness in developing, administering, and scoring dynamic assessments, and overcoming resistance.
McNamara's comparison
Taken together, the papers in this panel make a strong case for a broader conceptualization of assessment, which accounts for and incorporates translanguaging practices as a means to garner more accurate, valid and fair information about bi/multilingual test takers. The papers also point to the need for further empirical research on the development, implementation, results and consequences of multilingual assessments.
