Language proficiency in the context of migration

One stream of LTRC research has considered issues in assessing the language proficiency of foreign-trained health professionals seeking work in English-speaking contexts. These migrants play an important part in meeting healthcare workforce shortages in, for example, Australia and the UK (Australian Government Department of Health 2008; General Medical Council 2016). Two recent inter-disciplinary projects involving the collaboration of language testers, applied linguists and health professionals in medicine, nursing and physiotherapy (Elder et al. Reference Elder, McNamara, Woodward-Kron, Manias, McColl, Webb, Pill and O’Hagan2013; Elder Reference Elder2016; Knoch et al. Reference Knoch, McNamara, Woodward-Kron, Elder, Manias, Flynn and Zhang2015) focussed on the written and spoken communication in healthcare with the aim of proposing revisions to the Occupational English Test (OET), an English language screening test used in the professional registration process for health professionals from non-English-speaking countries intending to work in Australia. The impetus for this study came from concerns about the written and spoken skills of some overseas-trained health professionals and the associated impact on patient care. The aim of the projects, which were funded through Australian Research Council Linkage grants (LP0991153 & LP130100171) and co-funded by Cambridge Boxhill Language Assessments, was to ensure that the tasks, rating criteria and passing standards on the speaking and writing test are professionally-relevant and adequately represent the construct of this type of communication in the healthcare work domain. Both studies employed mixed methods designs with both qualitative and quantitative approaches. There were several project phases: (1) identifying the indigenous criteria (Jacoby Reference Jacoby1998; Jacoby & McNamara Reference Jacoby and McNamara1999) doctors and nurses use in establishing effective written and spoken communication; (2) applying these indigenous criteria to candidate performances on the OET writing and speaking tasks; and (3) developing new stakeholder-informed passing standards against these criteria. The studies were able to show that while some criteria valued by health professionals when commenting on authentic communication from their work domain can be incorporated into specific purpose language tests such as the OET, others do not lend themselves to inclusion in the more restricted domain of a language test. The projects were also able to explore some of the methodological challenges when eliciting indigenous criteria from domain experts (Elder & McNamara Reference Elder and McNamara2016; McNamara et al. under review).

A further recent LTRC research project considered the language proficiency requirements in relation to the migration of overseas students studying accounting and engineering in Australia (Knoch et al. Reference Knoch, May, Macqueen, Pill and Storch2016). Many of these students aim to enter the workforce in Australia after finishing their degrees. To be able to gain permanent residency (which is often a pre-requisite for paid employment), applicants need to take a language proficiency test, such as the International English Language Testing System (IELTS) as one of the requirements for skilled migration. The motivation for the study came from concerns about the perceived inadequacy of English skills of international students exiting tertiary institutions and entering the workforce, with writing skills being seen as particularly problematic (Healy & Trounson Reference Healy and Trounson2010; Rowbotham Reference Rowbotham2011) and a large-scale study (Blackmore et al. Reference Blackmore, Gribble, Farrell, Rahimi, Arber and Devlin2014) showing that international students often face a variety of barriers to entry to the labour market, including visa status and poor communication skills. The study therefore set out to investigate the relevance of the IELTS test in the progression from tertiary education to the workforce for accountants and engineers, with a particular focus on the writing sub-test. One aspect of the project focussed on the relevance of the IELTS writing tasks to the characteristics of written professional genres required by new graduates entering the workforce. The findings showed that the most frequent text types encountered by newcomers to the two professions were emails and reports. These text types were themselves found to be very diverse, including very technical sections as well as explanatory, descriptive and interpretive text. They were also written for very different audiences, from readers with technical (intra- and inter-professional) backgrounds to non-specialists. Writing often involved teamwork, collaboration and a great deal of supervision for new graduates, following the apprenticeship model of workplace learning. The degree of independence granted to new staff members as writers was closely related to the importance and intended readership of the document they were working on. The two IELTS writing tasks were perceived by employers to be relevant to some extent, although the omission of a specified audience in task instructions was noted as a concern. Employers also commented on the test's restricted sampling of tasks required in the workplace domain: this limitation is unsurprising considering that IELTS was originally developed to assess language proficiency for entry to higher education.

The use of language tests by governments in the process of immigration raises a number of issues around test use and test impact for test takers. Tests, in this case, are not only used to ensure minimum language standards but often also have a role in controlling migration numbers. Frost (Reference Frost2016) conducted a detailed study of four test takers in the context of Australia's skilled migration policy as they were taking language tests to gain permanent residency in Australia. She showed how the language test score requirements frequently changed over a period of a number of years and how participants negotiated this ever changing situation. Through detailed interviews with the participants over an 18-month period, Frost examined how the participants interpreted the purposes of test taking as well as the meaning of their scores. She was able to show that test impact, rather than being a fixed construct, was co-constructed by the participants in the study. It was dynamic and unpredictable according to decisions test takers made based on their perceptions of test purposes and score meanings and how they could best negotiate the policy demands. The study has important implications for our theoretical understanding of test consequences.

Language proficiency in schools

A study conducted by the LTRC in collaboration with the Research Unit for Indigenous Language at the University of Melbourne (Macqueen et al. Reference Macqueen, Knoch, Wigglesworth, Nordlinger, McNamara, Green, Ginger and Brickle2015) set out to investigate the impact of the National Assessment Program – Literacy and Numeracy (NAPLAN) on children and schools in remote indigenous communities in Australia. NAPLAN is a large-scale standardised test administered annually to all Australian school children in years 3, 5, 7 and 9. It is used to make important policy and funding decisions. The test assumes proficiency in Standard Australian English (SAE) and is mostly designed with populations of children living in metropolitan areas in mind. Children from remote indigenous communities, however, face a very different linguistic landscape. They often grow up hearing and using a variety of codes, including traditional languages, non-standard varieties of English and Aboriginal English. Most children only encounter SAE when they enter formal schooling. Additionally, their community relies largely on oral traditions of knowledge transmission and their parents may also lack basic literacy skills. This situation makes it difficult for these children to access the NAPLAN materials, which also are often culturally remote from their own upbringing. The research employed a qualitative methodology, interviewing teachers and principals working in school in a variety of remote indigenous communities, in several states and territories in Australia and from different school sectors. The findings showed an impact of the test on students and educational staff. In terms of students’ engagement with the test, it was found that the children were not able to fully engage with the test materials because of cultural barriers and their limited proficiency in SAE. Many therefore resorted to coping strategies, such as filling in multiple-choice bubbles or copying question content onto their answer papers. Teachers also reported some mild to strong negative impacts on the students such as undue stress or disengagement. Teachers felt pressure from their principals because of the pressure for schools to demonstrate improvements in test results. Finally, most participants agreed that the students’ results at both the individual and school level were mostly irrelevant because the test was not designed to be informative about students with very limited proficiency in SAE. The paper discusses these issues of test fairness (Kunnan Reference Kunnan2000) and their implications for test designers.

The LTRC is also regularly engaged in program evaluation studies. One study (Macqueen & Knoch Reference Macqueen and Knoch2015) commissioned by the Victorian Department of Education and Training required the Centre to evaluate the pilot of a new English as an additional language (EAL) online learning program designed in response to challenges associated with the provision of adequate language support to recently-arrived students in geographically isolated areas in Victoria, Australia. The program is designed to deliver individualised intensive EAL programs to students in these remote areas using online technologies. The content was taught through a combination of flexible curriculum-related content and individualised content using a range of video-conferencing and other technologies. The LTRC evaluated the effectiveness of the new program by examining gains in student assessed work samples and through stakeholder interviews. The findings showed that the EAL online program as a whole was an effective means of addressing EAL learner needs at participating schools. All students showed some improvement on the Australian EAL standards and most participating students developed confidence. There was also evidence of skill-building in the area of technology and an increase in awareness around the needs of newly arriving EAL learners by the participating schools.