1. Introduction
In this summary, we present the results of an online survey of educators who have experience working with heritage language learners (HLLs). The purpose of this survey was to better understand current trends in heritage language (HL) assessment in the U.S. context; to better understand the language assessment literacy needs of educators working with HLLs; and to gather information to inform the development of an online assessment tutorial for HL educators.
2. Background
Although interest in the field of HL education has grown significantly in recent years, more research is needed on educators’ understanding and use of assessment for HLLs so that assessment tools and resources can meet the needs of this population. There is an increasing number of HLLs in the U.S. school system (Kagan & Dillon, Reference Kagan, Dillon, Deusen-Scholl and Hornberger2008), making it important for educators to consider the unique linguistic and cultural backgrounds that these students bring to the language classroom (Beaudrie, Reference Beaudrie and y Cabo2016; Carreira & Chik, Reference Carreira, Chik and Potowski2018; Lacorte, Reference Lacorte, Fairclough and Beaudrie2016). While there is a growing body of research on the characteristics of HLLs and the resulting implications for instruction, there are a limited number of resources and training opportunities related to HL assessment.
To address this need, researchers at the Center for Applied Linguistics (CAL) and the Assessment and Evaluation Language Resource Center (AELRC) conducted research on teachers’ knowledge, practices, and needs related to HL assessment. The purpose of the survey discussed in this poster was to investigate two research questions: (1) what do HL educators know about their students and their assessment needs; and (2) what are current assessment practices, challenges, and needs when working with HLLs.
For the purposes of this research, we define a HLL as someone with a cultural or familial relationship to a language. HLLs may have different degrees of exposure to the language, and different areas where they demonstrate more knowledge or familiarity with the language or culture.
3. Methods
We administered a 28-question online survey via SurveyMonkey, a web-based survey development platform. The survey was developed using information gathered from a literature review examining current trends in and research related to HL assessment, as well as one-on-one interviews with eight HL educators investigating testing uses, needs, and challenges in HL programs. We recruited survey respondents via listservs and email contacts. The survey remained open for approximately one month, from 5 May 2020 to 1 June 2020.
Of the total 294 responses received from educators working across 34 U.S. states, we included 215 (73%) complete or partially complete responses in the analysis. Some question items allowed for multiple responses and not all questions were required, which accounts for any differences in the total amount of responses reported in the findings. Survey data were exported, organized, and cleaned via Microsoft Excel. Open-ended questions were coded using emergent thematic categories, and responses to these questions were included as a qualitative complement to the analyses below.
4. Participants
First, we asked survey respondents about their experiences working with HLLs. Most respondents reported that they were highly experienced language educators, and 70.7% of respondents had 11 or more years of language teaching experience. Most respondents (41.4%) also reported having 11 or more years of experience working specifically with HLLs, followed by 1–5 years of experience (34.4%), 6–10 years of experience (20.5%), and less than 1 year of experience (3.7%).
Survey respondents included educators working with kindergarten through post-secondary students. Most reported working with HLLs in the high school (49.8%), post-secondary (42.3%), and middle school (38.1%) contexts. When asked to select the response that best describes HL instruction in their teaching context, the majority of respondents indicated that HLLs take school, college, or university-based classes specifically designed for HLLs (36.5%), followed by foreign language classes with non-heritage students (27.6%), community-based HL classes (19.16%), and partial or full immersion classes with non-heritage students (13.6%). The survey results reflect the diverse teaching contexts within HL education.
We asked respondents which languages they work with in the HL context, and although 42 HLs were represented in the survey data, 60% of respondents reported working with Spanish, followed by Mandarin (10.2%), French (7.9%), and Russian (7.4%). Commonly taught languages, less commonly taught languages, and Indigenous languages were all represented in the survey responses.
5. Results
To answer our first research question, we asked respondents to rate their agreement with statements about working with and understanding the needs of HLLs. While more than half of respondents strongly agreed that (1) they have a clear understanding of the definition of a HLL (61.2%), and (2) they understand the ways that HLLs differ from other language learners (55.5%), nearly 20% of respondents disagreed with the statements ‘I feel confident addressing the assessment needs of HLLs’ and ‘I have received professional training or professional development on working with HLLs’. Respondents were also asked to provide their own definition of a HLL. From 184 responses, nearly all respondents described HLLs as language learners with a linguistic or cultural connection to the language. Many described HLLs as individuals who have been exposed to or speak the target language at home and in their communities. Others defined HLLs in terms of their membership in ‘minority’ and ‘majority’ language communities, or as members of a speech community other than that of the societally dominant language.
To address our second research question, we asked respondents about their current assessment practices, challenges, and needs. Results show that 73.9% of respondents use formative assessment for HLLs, followed by summative assessment (72.5%) and placement tests (64.3%). In response to an open-ended question about how they use each type of assessment, most respondents indicated that they use placement testing to place or move students into classes (49.5%), understand students’ language backgrounds (33.3%), and determine instructional needs (16.2%). Frequent approaches to placement testing included oral interviews (30.9%), evaluation of discrete skills such as writing, listening, and speaking (26.5%), and written assignments (25.0%). For formative assessment, most frequently described purposes included measuring progress (34.2%), assessing comprehension (33.3%), and/or addressing discrete skills (21.9%), and frequent approaches included traditional exams or quizzes (38.8%), written assignments (37.5%), and/or informal discussions (30.0%) to monitor student progress. Most frequently mentioned reasons for using summative assessment included using it as a formal checkpoint in the curriculum (37.8%), to measure specific skills (35.3%), and/or to measure general progress (26.9%), and frequent approaches included exams (56.7%), projects (27.9%), and written assignments (16.3%).
We asked respondents if they develop their own assessments, modify or adapt existing assessments, or directly use existing assessments when working with HLLs. Most respondents reported developing their own assessments for their students (68.1%), followed by modification or adaptation of existing assessments (50.5%), and using existing assessments directly (26.5%). When we asked respondents why they develop their own assessments or modify existing assessments, many indicated that they find existing assessments inappropriate for HLLs, their teaching context, or their students’ proficiency levels.
In the following section of the survey, respondents indicated their agreement with statements about using assessment in HL programs. Respondents agreed most strongly that it is important to use assessments designed specifically for HLLs (50.0%) and that assessment is a valuable part of their teaching practice (46.2%). Most respondents disagreed or strongly disagreed that it is easy to find assessments for HLLs (77.2%) or to develop assessments for HLLs (55.0%). Respondents indicated that the biggest challenge in assessing HLLs is learner diversity in the classroom (36%), followed by finding appropriate or relevant materials (24.8%) and a lack of professional development (15.2%). Finally, when asked to rate the helpfulness of resources, most respondents indicated that test development resources (52.8%), test selection resources (45.3%), and language-specific resources and guidelines (40.1%) would be extremely helpful in understanding and selecting assessments for HLLs.
6. Discussion and implications
Survey results indicate that while most HL educators reported having a foundational understanding of their students’ needs, they only feel somewhat comfortable or confident in addressing the assessment needs of HLLs. This may be due to a lack of professional development related to HL assessment, as well as the reported difficulties respondents have encountered when planning to assess HLLs, such as finding appropriate or accessible assessment materials. Responses show that although respondents find it difficult to develop assessments for this population, they often need to develop or modify assessments because existing assessments are not appropriate for HLLs.
Overall, responses indicate that while HL educators are currently using a variety of assessments and believe that assessment is a valuable part of teaching, they have limited resources. These findings are in line with other research findings indicating a need for heritage-specific professional development, including that which is focused on HL assessment (Ilieva & Clark-Gareca, Reference Ilieva, Clark-Gareca, Fairclough and Beaudrie2016; Llosa, Reference Llosa and Kunnan2013; Magaña, Reference Magaña2015). HL educators need targeted professional development resources about assessment to support them in creating and adapting tools for their contexts.
Based on these findings and using information gathered from the survey, we developed an online module designed to support HL educators in making assessment decisions. The module supplements an existing resource about world language assessment with information that is specifically designed to support educators working in U.S.-based kindergarten to 12th grade (K–12) HL classes, community-based HL schools, and university-level HL courses. It includes an overview of the linguistic and cultural profiles of HLLs, special considerations for working with this population, and the resulting implications for assessment. For more information and to view the module, visit https://www.cal.org/flad/heritage-language-assessment-module.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0261444821000288
Jamie Morgan is an Associate Project Manager in the Language Assessment Division at the Center for Applied Linguistics, where she manages projects related to research, assessment, and professional development.
Valentina Michelotti is a graduate research assistant at the Assessment and Evaluation Language Resource Center at Georgetown University, where she is finishing her M.A. in Language & Communication.
Meg Montee is the Director of Performance-based Language Assessment in the Language Assessment Division at the Center for Applied Linguistics, where she leads projects related to test development, test validation, and scoring for performance-based language assessments.
