Introduction
The movement of professional services and multinational back office functions from the United States, United Kingdom, and Australia to developing countries in South America, Eastern Europe, and North and Southeast Asia makes great economic sense to both parties. However, the political, social, and educational repercussions on the market economy of outsourcing to these destinations are less clear. In some countries where the offshore and outsourcing (O&O) industry is expanding, such as the Philippines, language policy in the schools and in vocational and other tertiary sectors is being mandated to underpin the success of this multibillion-dollar industry. Social change is also occurring: Social roles and family structures are changing to accommodate night shifts; the incomes of new graduates, especially women, employed in the O&O sector are lifting families out of poverty; the increasing number of women earning a substantial income is expanding the domestic retail sector. The continuing success and expansion of the O&O industry in countries such as the Philippines and India is dependent on the effective English language proficiency and performance of customer service representatives (CSRs), who must keep native English-speaking customers satisfied through smooth and efficient communication around their problems.
We first describe the O&O industry and discuss the central importance of English language capability to this rapidly expanding industry, taking the Philippines and its social, political, and linguistic context as our example. We review the limited research literature on language in call centers and on workplace language assessment. We outline key features of call center communication breakdown and discuss the resulting key issues and challenges for language assessment research and development for this specific context. We look at how this emerging industry has influenced governmental and industry language policy and responses within higher or further education. We conclude with a summary of why high-quality English language assessment tools and processes are central to solving problems in this industry sector, and we set an agenda for ongoing research in this area.
What Is Offshoring and Outsourcing?
Offshoring and outsourcing (O&O) is the movement of work to a country other than that where goods or services are provided to customers. Offshoring refers to companies who own the business but want to domicile it in a cheaper location; outsourcing refers to companies carrying out the business for other companies overseas also in cheaper locations. This practice has grown with the rise of the Internet and the decreasing cost of telecommunications, both of which allow data to be accessed and worked on anywhere in the world. The major growth area in O&O has been business process outsourcing (BPO), including back-office work, such as insurance claims processing, and customer service work, especially call centers. Multinational companies constantly search for more efficient and cost effective options of running their businesses, and many business leaders claim that in addition to cost saving, offshore locations such as India and Philippines offer improved quality, given that jobs that are carried out by high school graduates in the home country are typically done by college graduates offshore. It is estimated that by 2015, a total of 15 million jobs will be generated by this offshored and outsourced industry, turning over in excess of US$ 300 billion (NASSCOM, 2005). Although the call center industry overall is a relatively small segment of this market, in the Philippines it employs more than 150,000 young graduates, and this number is set to grow rapidly (75% in the next 5 years, according to Tuchman, Reference Tuchman2004).
Why Is Offshoring and Outsourcing Important to Countries Such As the Philippines?
O&O represents a large potential market and draws on local human resources rather than nonrenewable resources such as oil, timber, and minerals. For this reason O&O has a significant role to play in the economies of a growing number of “underdeveloped” countries. Countries such as India and the Philippines, which have a large number of high school and college graduates with a solid foundation in English, offer a particularly attractive workforce. In the Philippines, where companies such as AOL, AIG, HSBC, Deutsche Bank, and Manulife have set up BPOs and call centers, and where third-party service providers such as Accenture, IBM, and Logica have set up in the last 5- to 10-year period (BPAP, 2007), the growth of the O&O industry has had important repercussions on Filipino society, the education system, government policy, and of course the Filipino economy.
For many years, the Filipino economy has been supported by the inflow of overseas worker earnings (14 billion dollars in 2007). The Middle East, Singapore, and Hong Kong continue to be favored destinations for Filipinos wanting employment. Much of the work overseas is unskilled (e.g., domestic workers in Hong Kong and Singapore; construction workers in the Middle East), although this work is frequently carried out by skilled Filipinos: A Filipino trained teacher or accountant can earn more being a domestic worker in Hong Kong than working in their own profession in the Philippines; doctors can earn more as nurses in the United States and United Kingdom. All this has resulted in a “brain drain” from the Philippines overseas and has caused a growing discrepancy between the very rich and those living on the verge of poverty.
However, the opening up of the O&O industry in the Philippines has allowed fresh graduates to gain well-paid employment at home. According to the U.S. State Department Profile (http://www.state.gov/r/pa/ei/bgn/2794.htm: retrieved 24‒9‒08), “the business process outsourcing (BPO) sector was the fastest-growing segment of the Philippine economy in 2006 and 2007, totaling an estimated 10% of the global outsourcing market and generating an estimated $5 billion in revenues during 2007 (equivalent to about 3.5% of Philippine gross domestic product). BPO revenue grew 40% during 2006 and 2007, and [was] expected to repeat this growth over the next two years.” The typical call center agent wage is more than double the minimum wage in the Philippines. As a result, a highly educated middle class sector of the population is now deciding to stay at home rather than go overseas to seek employment. But the rapid development of the O&O industry over the last few years has led to questions about language policy in the Philippines.
The Philippines was a U.S. colony and commonwealth between 1898 and 1946, and American style education and English medium instruction were common. A period of U.S.-assisted postwar reconstruction, marked by political unrest and change, followed until 1972, when President Marcos declared martial law, which was maintained until 1981. During this period there was a strong movement to “decolonize” the education system, and a bilingual education policy passed in 1974 meant that English and Tagalog were used to teach different subjects at school. For the last 30 years, most Filipinos have received their education in both Tagalog and English (Gonzalez Reference Gonzalez and Bautista1997, Reference Gonzalez2004).The policy of bilingual education in the schools and universities continues to be controversial today. Private education in the urban areas is producing good language speakers, while the rural schools lag badly behind (Nical, Smolicz, & Secombe Reference Nical, Smolicz, Secombe, Tollefson and Tsui2004). Two views prevail about this issue to the current day. One view reflects the business interests supported by the government that aspires to “native-speaker-like” proficiency to staff the new O&O business; while the other, supported by the academe, promotes a Filipino variety of English that is globally intelligible and requires intellectualization to become a medium of instruction in schools and universities (Friginal, Reference Friginal2007). The current government recently announced a presidential decree that made the medium of instruction in schools to be English.
Two major economic drivers have continued to underpin the ongoing motivation for the Filipino people to become proficient in English. First, the Overseas Filipino Worker (OFW) scheme, which gained momentum in the Marcos era, has been in place for over 3 decades and provides overseas employment opportunities to Filipinos with “internationally comprehensible” English, in a range of regions including Singapore, Hong Kong, the Middle East, and some European countries. Second, and of increasing importance as already discussed, the O&O industry is providing lucrative employment and career development for young Filipino graduates with nativelike fluency in English.
The tremendous contribution of the O&O industry to the Philippine economy makes it essential for government, local business, and the education sector to facilitate its continued health and expansion. The supply of young graduates with high levels of English has traditionally been associated with the tier one universities, which conduct almost all instruction in English and produce graduates with near-native competence. However, in other tertiary institutions new graduates typically have reasonable levels of English proficiency, but not the near-native levels required in large numbers by the call centers, so that, as Friginal (Reference Friginal2007) points out:
Given that the schooled bilingualism has achieved only minimal proficiency, within very limited register (Kaplan & Baldauf, 1997), the outcome could still be a disappointment for sectors demanding fluency in English from Filipino workforce in international business. (p. 334)
As this market sector grows, the O&O industry complains there is a lack of applicants with high language ability, good pronunciation (sometimes meaning an American accent) and intercultural awareness among the newly graduating groups. The O&O industry provides a significant flow of investment money from international firms (Uy, Reference Uy2004), and the Philippine government appears very responsive in its support. However, governmental pronouncements about the importance of English, and the efforts of the quasi-governmental Technical Education and Skills Development Agency (TESDA) are, in the short term, frustrated by a lack of expertise, lack of time, and lack of infrastructure and resources (Lockwood, Reference Lockwood2008).
The Context of the Call Center
The language used in call centers is complex, multifaceted, and produced in high-stakes situations repeatedly over many hours each day; and those who use it are subject to frequent oversight and evaluation by any manner of different stakeholders in more senior positions in the organization, as well as their coparticipants in the discourse. CSRs also work in contexts where hiring and firing decisions are made quickly on the basis of limited evidence, and being fired is in no way unusual.
From the point of view of language researchers and test developers, one major advantage of call-center-related work is that almost all the work is audiorecorded at all times for quality control. This makes data for language descriptions based on analyses of authentic call center interaction relatively easy to obtain, and serious linguistic work has been done in this area (e.g., Cowie, Reference Cowie2007; Clark, Rogers, Murfett, & Ang, 2008; Forey & Hood, Reference Forey and Hood2008; Forey & Lockwood, Reference Forey and Lockwood2007; Friginal, Reference Friginal2007; Lockwood, Reference Lockwood2008). This research can provide the basis of language for specific purposes (LSP) needs analysis and materials development and of empirically grounded call center tests. However, it is not just the data descriptions and analyses—the products if you like—that inform call center assessment solutions, but also the business requirement processes, for example, recruitment and quality assurance, in which such language assessment events occur. These are described in detail later in the article.
If workplace assessments are to meet specific business requirements, logically the place to start looking for existing research literature is at the level of English language business needs analysis. Prince (Reference Prince1992) and Joyce (Reference Joyce1992) each took a situational analysis and looked at the network and processes of language needs in an Australian workplace setting. Prince specifically explored the workplace trainer role in an industry context that is constantly changing, while Joyce looked at issues of literacy and provided frameworks for planning and delivery of specific language programs in an industry setting. Reeves and Wright (Reference Reeves and Wright1996) took a broad-angled ethnographic view of needs analysis in the workplace and encouraged language providers to tailor courses specifically to the needs of the workplace through what they termed “a language audit.” Language auditing (Reeves & Wright Reference Reeves and Wright1996) assists the organization to understand its own language needs across different business requirements, accounts, and postholders: Language needs are defined in a gap analysis of what a post ideally demands of language level (e.g., an office supervisor) and what the existing language level of that postholder may be. Language scales and descriptors are used to establish this picture. Large-scale language assessments are conducted on every employee within the company, and a simultaneous language consultancy is conducted through interviews and observations, to establish desirable language levels for each post. A similar “auditing” approach is suitable for the call center industry where the different client accounts, although located in the same call center, may be very different in terms of linguistic demand; for example, a financial services account dealing with, and counseling, rich investors, will be different from directory enquiries.
The Role of Workplace English Language Assessment
Language and intercultural issues in communication are important elements of business success. Now that English is the de facto global language of international business, multinational corporations need ways of ensuring that the workplace English demands of their operations in non-English-speaking destinations are being met.
There is a clear role in this context for assessment systems that focus on professional language needs. Tests of English for more, or less, specific workplace contexts do exist. Large testing organizations such as Cambridge ESOL (United Kingdom) and the Educational Testing Service (United States) have produced generic business English tests (BULATS, BEC, TOEIC) to meet the needs of this sector. These generic business English skills tests appeal to a broad sector of the working population as they are offered by large well-known examinations body and provide useful certification for entry into a variety of workplaces. For example, TOEIC certification is important for gaining a job in many business workplaces in Korea and Japan.
As well, governments of countries with high levels of immigration, for example, Canada and Australia, have variously funded the development of workplace language benchmarks for a range of industries to ensure a steady supply of English-speaking professionals into identified unfilled jobs. For example, the Canadian government sponsors the generic Centre for Canadian Language Benchmarks (CCLB) to provide assured levels of language competency for employment in general, and the specific Canadian English Language Benchmark Assessment for Nurses (CELBAN) to provide assured levels of English language competency for employment in nursing and allied professions. Similarly, the Australian government has for many years sponsored the Adult Migrant English Programme, including a separate program for industry, to ensure quality language training programs and to assess participants prior to employment.
Furthermore, specific workplace needs sometimes have implications well beyond national boundaries: for example, Strevens (Reference Strevens1984) developed “Seaspeak” as a language for maritime communication between ships “whose captains' native tongues differ”; this may have been the earliest such specific purpose (SP) development with international applications, and in the past 25 years the continuing need for a stable language for maritime communication has led to much materials development and teaching in English for maritime purposes (Benton, Reference Benton2005; Trenkner & Cole, Reference Trenkner and Cole2003), although forms of assessment are much less developed and unstable (Ivasyuk, Reference Ivasyuk and Zhukov2008). More recently, language tests for aviation English have been developed to meet the International Civil Aviation Organization (ICAO) (revised) deadline for certification of March 2011. A recent validation study (Alderson, Reference Alderson2007, Reference Alderson2008) has been carried out on the English Language Proficiency for Aeronautical Communications (ELPAC) test, which is intended to assess the language proficiency of air traffic controllers. Alderson concluded that ELPAC is a sound test but criticized many other competing tests for lack of evidence of reliability or validity (see next article).
The Literature on Workplace Language Assessment
Douglas (2000) looked at specific language use situations to derive test content and test methods for highly specific language for specific purposes (LSP), citing the Temporary Registration Assessment Board (TRAB) examination introduced in 1975 by the British General Medical Council for the purpose of assessing the professional and language abilities of doctors trained outside the United Kingdom as an early example of a “true LSP test” (Douglas, p. 3) Among specific purpose professional and workplace language tests, the Occupational English Test (OET) is perhaps one of the best-known. Introduced to screen the English language skills of medical workers immigrating to Australia, the OET was funded by the National Office of Overseas Skills Recognition (NOOSR), and the research, development, and management were carried out by the Language Testing and Research Centre (LTRC) at the University of Melbourne under contract to the Australian federal government. This project has fuelled several studies into specific language assessment (Lumley & Brown Reference Lumley and Brown1996, Lumley, Reference Lumley1998; McNamara, Reference McNamara1996). Other significant language assessments for specific purposes include the Language Proficiency Assessment for Teachers of English (LPATE) administered annually by the Hong Kong Examinations and Assessment Authority (HKEAA), the ELSA test for benchmarking the language of recently arrived overseas subject teachers to Australia (Elder Reference Elder2001; McDowell Reference McDowell and Brindley1995) and TOLES for testing the English of lawyers (www.toles.tricycle-media.com). These tests have appeared as critical needs have arisen for professional groups to set benchmarks for professional entry and practice.
However, workplace assessment is neither easy nor cheap in development, implementation, or validation. For example, Hamp-Lyons, Lumley, Hamilton, & Lockwood (Reference Hamp-Lyons, Lumley, Hamilton and Lockwood2003) conducted research and development for a test of the English of Accountants in Hong Kong, but concluded that the client base for the test was too small to make it market-viable from the Hong Kong Society of Accountants (HKSA) point of view. In what follows we look at the contexts, needs, and assessment issues relating to the call center industry and consider what is feasible as well as effective.
The Language Assessment Needs of the Call Centers
A recent survey of business English tests carried out on behalf of a large education group concluded that the O&O industry sector requires a specific approach (restricted report from Lexicon Consulting Group, 2007). First, the test content and construct need to be specific to the O&O sector, with voice assessment encompassing interactive capability with Western native speaker customers. Early descriptions of the customer service interaction between nonnative CSRs and native speaker customers on the phones in O&O destinations are revealing the need for highly specific language to complete successful calls (Clark et al., 2008; Forey & Hood, Reference Forey and Hood2008; Forey & Lockwood, Reference Forey and Lockwood2007). Second, because of the need for a competitive advantage, the large number of applicants, the large number of recruits, and the low hiring success rate, test delivery in this context needs to be fast and flexible in order to meet the immediate business needs of expansion and the problem of attrition. Such quick turnaround times would suggest a computerized solution. Third, the testing needs to be integrated with the analysis of needs for training and coaching, quality assurance, and promotion. Last, testing solutions need to be embedded in the whole language development and provision process: They must be transparent to key industry employees such as recruiters, trainers, and quality assurance personnel as well as to management. The growing frustration with conventional business English tests and approaches was recently voiced:
The only time language testers come and talk to us is when they want to sell their business English tests off the shelf. They are not interested in finding out what we do and whilst we know what we want in terms of the business requirement, we do not know what we need nor do we know how to articulate all this. You are the experts. We've wasted a lot of money on business tests that promise the earth and deliver nothing. . . . We need good assessment consultants helping us develop our own. (Extracted from a June 2008 conference call with a regional South East Asian Bank communications manager, cited in Lockwood, Reference Lockwood2008)
Call centers need to be able to language assess the CSRs when they apply for a job, when they exit prehire language support training, when they talk to customers on the phones, when they apply for migration to a different account, and when they apply for promotion. They also require the ability to audit or tag their different accounts. We use the word audit to mean gauge the linguistic demands of the various client accounts. For example, one account may require extremely high levels of English because of the nature of the service (financial counseling or advice) and the audience (speaking to financial advisors in the United States), whereas another account may be very routine in nature with a less demanding audience (e.g., directory enquiries) and speaking to the general population. In some of the Philippines call centers, this has become known as tagging the account. During a number of consultancies in the O&O workplace carried out by one of the authors, it appeared that a crucial piece of information required by the call centers was to establish how easy or difficult the client accounts were from a language point of view. With this information, new recruits could be hired for specific accounts based on their language proficiency levels and similarly could be moved to more complex accounts as proficiency improved. Call centers in Manila have generally responded to these complex business requirements for communications assessments in different ways. First, some call centers have developed their own systems for assessments. While there has, to date, been little research on such assessments, given that they have been developed by HRO departments with no language assessment expertise, these are likely to lack validity and reliability. A second option has been to use a combination of commercial business tests for entry and then homegrown solutions for the internal business needs. It has been reported (Lockwood, Reference Lockwood2008) that such commercial tests are limited in what they can predict for CSRs' speaking skills when talking to native speaker customers on the phones. As well, such solutions have been expensive given the numbers of CSRs that need testing each month for entry to the workplace. Another solution to this problem of call center language assessment has been to commission outside providers to develop a tailored solution to this need. One such company has developed the Business Processing Outsourcing Language Assessment Scales (BUPLAS) for speaking and writing. In Manila, many call centers have found these scales to be very useful in providing the possibility of a systematic reporting system from entry to quality assurance. However, research and development in this area remains scarce (Lockwood, Reference Lockwood2008).
It seems, then, that the O&O industry is poorly equipped to make its own language assessment decisions, and (like many specific purpose stakeholder groups) is being let down and limited by traditional testing paradigms of test development and practice; a general malaise that has, in recent times, been articulated by McNamara and Roever (Reference McNamara and Roever2006), who described this as a “difficult juncture” in language testing. They wrote:
A discipline whose intellectual sources are fundamentally asocial, in measurement, psychology and a linguistics that focused on individual cognition, has found itself lacking effective tools to conceptualize, acknowledge, and confront the social dimension of assessment either at the micro level (of face -to-face interaction) or at the macro level, the social context in which test use occurs. (p. 247)
While it is common in the testing and assessment literature to find references to the need for context and purpose sensitivity (Douglas, Reference Douglas2000; Elder Reference Elder2001; Lumley & Brown Reference Lumley and Brown1996; McNamara, Reference McNamara1990), little seems to have been written on how this might be achieved in such a dynamic workplace as the call center. An urgent need remains for a multidimensional model for call center language assessment, enabling a consistent strategy for identifying diverse participant groups' needs, incorporating different stakeholders' views, identifying key contextual constraints such as budgets, technical access, and so forth. Such a model could then be applied in individual O&Os to construct a case-specific framework of requirements for English language assessment provision and reporting that can meet the needs identified within the operating constraints.
Lockwood (Reference Lockwood2002) argued that an interdisciplinary approach to program evaluation of workplace training that incorporates stakeholders' varied concerns has practical value. However, in 2008 there was still little sign of systematic inclusion of various workplace stakeholders into the curriculum and assessment planning process for workplace English. Kirkpatrick (Reference Kirkpatrick1994) reported that in one large Hong Kong bank, the training manager had implemented regular line manager observational reports detailing language assessment improvements in the front line of banking service, that is, of the bank tellers. In the business management evaluation literature, this framework for evaluation is called a Level 3 assessment and answers the question about whether training impacts job performance on the floor. Frameworks for workplace program evaluation should incorporate a systematic needs analysis for training that encompasses all stakeholders' concerns (Bramley & Pahl, Reference Bramley and Pahl1996; Goldstein Reference Goldstein1993; Kirkpatrick Reference Kirkpatrick1994). Such frameworks are highly relevant in the call center industry where the multiple stakeholders' needs and businesses' requirements for language assessments are complex.
Commercial language testing agencies, for whom scalability of their products and profit margins are key drivers, understandably do not want to investigate the complexity of the business requirements of language assessment in call centers. Commercial tests at present are geared around individual tests with relatively high prices and long turnaround. It would take a massive investment in infrastructure to address this, or assessment would need to be delegated to company staff, thus impacting reliability.
Focus: Assessing the Language in Call Centers
In satisfying the business requirements for language assessment in the O&O industry, and in call centers in particular, it is first necessary to ensure validity by reflecting research findings describing call center communication breakdown into the test design. Further, it is necessary to ensure practicality by reflecting the different business requirements within the call centers themselves in a flexible model of assessment.
Recent Studies into Communication Breakdown in the Call Centers
The following authentic transcript is an excerpt from an insurance company transaction in a Filipino call center. The customer service representative (CSR) is Filipino, and the customer is an American native English speaker:
D: CSR/C: Customer (call is in progress)
D: Actually, Ma'am, before that there was a check that was sent out for $842.
C: To who? I didn't get no check for $800.
D: I'm seeing here, Ma'am, that a check sent over for Barbara Anne F. Jones at 452 Meadow Lane. . . .
C: I did not get a check for $800—all I have received, which I have not cashed yet, is a check for $9.92. I have never received an $800 check.
D: Well, we actually have a copy of. . . .
C: OK. We need to find out where it went . . . I never cashed a check for $800. You need to put a tracer on it and find out whose signature is on it 'cos I never cashed it, I never got one.
D: OK, I'm seeing here, Ma'am, that it was sent over to [withheld] Meadow Lane 87. . . .
C: OK. I live at 452 Meadow Lane.. . . I never received a check.
D: OK, well, the most I can do is to have this one sent over for further research because we are seeing here the cancelled check with a signature on it.
C: OK, give me a copy of that check back so I can send it to some lawyer or somebody because somebody cashed it and I never got it.
D: OK, may I have your daytime phone number please, Ma'am?
C: Area code [withheld]—that's a cell number.
D: OK, I'll have this sent over to the proper department for research. Is there anything else I can help you with today?
C: Why have you been calling me this week? Do we know that answer?
This is a small snippet taken from a lengthy conversation illustrating the mounting frustration felt by customers who feel their problems are not being adequately addressed. The breakdown here relates not to discrete points of grammar inaccuracy or even phonological difficulties, but more to misunderstanding of the nuance used by the customer and the poor interactive response of the CSR. This customer, an American woman, had been called by the insurance company in the previous week, but she could not, at that time, take the call. She was merely ringing back to see why she had been contacted. The CSR is not responding to the obvious anxiety of the customer who doesn't know why she was contacted in the first place and then hears about a check that has been cashed by someone else. A good CSR would empathize and defuse this situation through good functional language choices made for the interaction. Because the CSR does not have this language capability, this call escalates and results in customer frustration and sarcasm.
In such language descriptions the cause of communication breakdown rarely appears to be related to incorrect grammar and poor pronunciation, but relates more to the interactive and discourse capabilities of the CSR when speaking as demonstrated in the excerpt above, where the customer becomes very frustrated and angry at the CSR's inability to answer her questions and reassure her. This is exacerbated by the CSR's problem in understanding the nuances of what the customer is saying. For example, the customer very deliberately “uncontracts” her message early on: “I did not get a check. . . . I have not cashed. . . . I have never received.” making it clear to the CSR that these things have not happened, and she is worried about this situation. Normally the CSR would acknowledge this nuance with an appropriate responses like: “You didn't get a check? OK. Let me deal with this urgently as. . . .,” whereas this CSR simply insists that she has a copy of the check, checks the address, and says she will send it on for “further research.” These are unhelpful responses and distancing solutions that further anger the customer. Responding to the linguistic research to date showing that O&O communication problems have to do more with the interactive capability of the CSR than with grammatical or phonological inaccuracies is a critical first step in language assessment design for the call centers.
Despite these early findings, many call centers in the Philippines administer language screening tests based on discrete item grammar tests. In one call center, we observed the human resources (HR) training department using a 50-item grammar test, of which over 30 of the items related to verb–subject agreement (e.g., every group of politicians has/have money in the bank). When asked about the preponderance of this kind of item, it was justified as a common Filipino mistake and therefore undesirable. There appeared to be little thought about whether this kind of error would cause communication breakdown when talking to customers in America. We also observed, in a number of call center recruitment interviews, HR screeners simply counting the number of pronunciation mistakes. If the prospective agent had in excess of six mistakes in a 15-minute interview they were automatically put back into training or rejected for work. Again, when probed for a justification of such practices, the response related to the fact that the agent was using “Taglish” (combination of Tagalog and English), and this was undesirable because it might cause communication breakdown.
The Business Requirements for Language Assessment in the O&O Industry
In the context of the complex linguistic and pragmatic demands on CSRs, requests for a single test solution to the English language communication problems being experienced by the call centers are unrealistic. Some companies realize that a more complex solution is required and talk about the desirability of an end-to-end solution. An end-to-end assessment solution requires assessment tools and processes that reflect the business requirements of the call centers at different stages and processes at work; it must be embedded into those processes; it must be appropriate to each of the stages and stakeholder needs; and it must be accountable to the participants in the work and the assessment—the assessed, the assessor, and the oversight body. An end-to-end solution would provide the right measure at the right level and the most suitable moment; and it would report scores or decisions rapidly and in targeted detail, so that effective decisions can be made, and validated, within the pacing and demands of this industry.
For the call centers, the key stakeholders in language assessment outcomes are typically HR personnel, communications trainers, client account managers, team leaders, and quality assurance personnel (see Figure 1). Between them, this group decides who gets hired and which accounts they go to; what they get trained in and how they are measured for training success; and who gets coached and how judgments are arrived at for quality success. At all stages, of course, the CSRs and applicants for CSR positions are also stakeholders.
Who are the stakeholders? Business stakeholder requirements for language assessment in call centers (Lockwood, Reference Lockwood2008)
As Figure 1 shows, the first step toward a language assessment is a language account audit: This will tell us what each call center account in that company expects in language level(s) and interactive capability. A good language audit requires valid and reliable language proficiency scales and descriptors, and the assessors trained to understand and use them.
For recruitment, because of the huge number of people applying for these positions, most companies do an initial screening test. The aim of this prerecruitment screening assessment is to choose the best language communicators from a very large pool of applicants. At present, such screening is very haphazard and unreliable. This initial assessment is normally done by phone screeners, who are often employed as casual members of staff, may or may not have tertiary qualifications, are not screened themselves for language level, and do not receive language assessment training or calibration. It is not surprising that this important initial gatekeeping assessment process is not having the desired result of screening in only the stronger 50% of all applicants. Such inefficiency of assessment is very costly for companies, which may need to take on hundreds of CSRs every month to meet business expansion and CSR attrition. It is also surprising and disappointing to companies that have been lured to the Philippines because of supposedly good levels of English, and they are beginning to question the usefulness of their in-house communications assessment processes and look for improved validity and reliability.
Once CSRs have begun work, quality assurance needs become important. Call centers have service level agreements (SLAs) with either their client companies or their own onshore headquarters, which stipulate elaborate quality assurance (QA) processes and the requirement for high levels of success to be evidenced in these processes. If the call centers cannot demonstrate the high communication quality they promised in their SLA, they are in breach of contract and may not get paid. Unfortunately, the QA tools and processes provided by the client companies have been designed for NS agents rather than NNS agents. In transferring the call center business accounts to destinations like the Philippines, the language and intercultural implications of assessing NNS agents have been largely ignored. Typically the QA monitors who only work on the floor in each call center (whose own English may also be problematic) work off a quality scorecard designed for use in the United States or United Kingdom. The scorecard is completed by the QA monitors: They simply tick for yes, cross for no, and N/A for not applicable. Quality outcomes as indicated by scorecard scores are usually much higher than the customer satisfaction (CSAT) scores generated by third parties in the home country. Multinationals are therefore getting two contradictory reports of the quality of communication with the customer and are beginning to look much more closely at what is going wrong in the communications assessments carried out locally. Figure 2 is adapted from a scorecard typical of those used by most call centers in the Philippines and India. (For a deconstruction of this scorecard against language assessment principles and practices see Lockwood, Forey, & Elias, Reference Lockwood, Forey, Price, Bautista and Bolton2008.)
The Scorecard
However, quality assurance in the call centers is not just for the purposes of SLA compliance, but also serves an important developmental function. If a CSR is noted to be particularly capable in their current post, they may be targeted for promotion to a more demanding administrative position or for migration over to a more difficult and complex account. Similarly, if the CSR is experiencing difficulty on the phone, a proper diagnosis of the problem can lead to support for the representative through further training and coaching. Currently, however, both remediation and advancement are being frustrated by poor levels of English and a lack of TESOL knowledge within the QA teams, by inadequate homegrown solutions to specialized language assessment and support problems, and by needs that require tailored assessment tools. These deficiencies pose grave threats to the ongoing viability and success of the O&O industry in the Philippines, and our work with call centers in India suggests that there are very similar problems there too.
Policy: The Response of the Philippines Government, Quasi-Governmental Organizations, and Chambers of Commerce
Faced with the needs discussed above, and the economic importance of the O&O industry to the country, the Philippine government has made several moves to strengthen the supply of good English language skills in the workforce. First, it has recently mandated English as the Medium of Instruction (MOI) in schools and universities, although it does little in the way of planning, implementing, or resourcing this mandate. The capability for following through on major language reforms and other initiatives requires the alignment of key education players (e.g., the University of the Philippines remains committed to bilingual approaches), which requires expert educational management and the alignment of policy with resourcing. A recent voucher training scheme aimed at providing support to prospective call center agents to learn English was rushed in by government, but it was poorly implemented, and the acceptance criteria were unclear. The result was that this money was diverted to other kinds of training. The allocation of money to such schemes will not be effective if those responsible for implementation lack the expertise to deliver the program. Top-down interventions and policies such as these need time to develop, commitment to embed long term, and expertise to implement, while government wants speedy fixes. The Chambers of Commerce in Manila have been active in promoting English with “English is COOL” advertisements enticing young people to learn English. However, the Chambers too have been bedeviled by a lack of expertise behind their efforts and consequently have sponsored groups eager to promote into this lucrative market reductive, inappropriate testing “solutions,” comprising merely, for example, a multiple-choice test of listening and speaking.
Nationally, the Technical Education and Skills Development Agency (TESDA; the government vocational training instrumentality), continues to look for a one-size-fits-all solution to the training and assessment needs of call centers. Over the last few years TESDA has endorsed a number of testing solutions including the Ordinate phone assessment, known in the Philippines under its franchised name Prospeak. However, recently the use of Prospeak was called into question when a group of call center potential recruits identified by TESDA all failed the test, but subsequently managed to secure jobs in the call centers and appear to be doing well. Since TESDA has very little funding and does not have staff with the necessary expertise to develop solutions, at best its efforts will be piecemeal and amateurish, adding no real value to underpinning O&O industry training and assessment needs with principled approaches, materials, and assessments.
Practical Imperatives: The Response of the Industry
The O&O industry and the multinationals themselves are spending significant money, time, and effort in devising their own language assessment and training solutions. The urgency of these assessment solutions cannot be underestimated, given that O&O companies are taking on CSRs continuously to meet demand and attrition. However, the quality of these solutions is currently coming under close scrutiny by the companies' own senior management, given that many of the “solutions” seem to be providing little value. Companies may have developed their own screening tests without expert help; or else they use private language centers which offer speaking tests via telephone to the United States—note that specific tests, appropriate or not, are often mandated by the account customer. Practice in this area is fragmented and not informed by linguistic theory or by theory and best practice in language testing.
The Response of the Academic Community in the Philippines
The academic tertiary sector in the Philippines initially did not welcome the O&O industry with great enthusiasm, partly because the nascent industry seemed to offer few career development opportunities and partly because of the perceived low skill levels required. In early 2000, the industry recruited its first cohorts of call center agents from the top-tier universities such as University of the Philippines, De la Salle, and Ateneo University. The O&O industry enthusiastically welcomed these top-tier graduates because of their very good English language skills and good education. There was, however, a mixed response from the Filipino academic community, who felt that their graduates should not be working in the call center industry. These negative feelings stalled collaborative partnerships, and until recently few of the best universities have offered specific O&O tertiary programs to prepare graduates to join this workforce, although a large third-party call center has recently developed collaborative links with one of them in providing a pipeline of graduates. Other universities are also responding to the business need of training middle and senior managers in leadership and management skills. Second- and third-tier tertiary institutions are also securing jobs for their graduates through collaborative links and through generally upgrading their curricula to reflect the needs of the O&O sector.
In the early days of the O&O industry, companies sought solutions to their language training and assessment problems by contacting the best universities for O&O TESOL consultancies and training. These solutions have been frustrated by a lack of knowledge on the part of the academic community about the business requirements (Lockwood, Reference Lockwood2004). One well-known university was seen to be providing training programs focusing on pronunciation and grammar drills, with some of the materials based on commercial business English course books developed for European businesses. The company that had commissioned them, at a high price, found that they had not carried out any kind of needs analysis within the company before deciding what to teach. Not surprisingly, the company felt little impact from the training; as a result, they terminated the contract early and proceeded to build their own training team internally (Lockwood, Reference Lockwood2004).
Commercial training colleges have set up in response to the O&O industry's need for improved levels of English. Many teach “accent neutralization” and grammar; some charge high costs for long assessment interviews on the phone (45 minutes); and most are staffed by trainers with no TESOL qualifications. With few local academics taking an interest in this industry, research-based solutions have been slow. Interestingly, it has been overseas academics who have become interested in the new discourse of the NNS agent interacting with the NS customer (e.g., Forey & Hood, 2007; Lockwood, Forey & Price, 2008).
Conclusion
Language assessment is an ongoing and major concern as the O&O industry struggles to get valid and reliable tools to recruit at the right levels for its client accounts, to target its training at the best candidates, to ensure that staff once hired do perform up to the expectations set for them, to identify those who are not meeting expectations and provide further targeted training or to remove them quickly from the customer-handling situation, and above all, to find convincing evidence of communication effectiveness as key quality measures. Unfortunately, with little expertise within the companies themselves about how to assess for what they need, and the reluctance of commercial testing agencies to get involved in developing new, carefully targeted products and approaches that meet the needs of this industry, homegrown and uninformed solutions are proliferating. This leads to repeat assessments each time a CSR moves from one O&O to another; another concern for the industry is the lack of an O&O shared information system of benchmark levels for different kinds of accounts. The professional body BPAP is keen to facilitate more consistency across the O&O sector for communications assessment, but still lacks the resources and expertise.
Our interactions with senior management in BPOs in the Philippines and with academics similarly interested in this very specific ESP have enabled us to summarize what the industry sees as the key areas for assessment: Unsurprisingly, these relate to the real problems in their businesses that threaten effectiveness and therefore profitability (see Figure 1). Tests are needed for screening and recruitment: These need to be quick and easy to administer, yet they must assess areas of lexicogrammar and pragmatics as well as micro-linguistic elements. Measures are needed for prehire and near-hire training, to ensure that appropriate kinds of language support are provided for the right period. A high-quality diagnostic language assessment tool is needed for QA personnel to use once the agent is on the phone. In the same context, a progress assessment is needed to identify whether the CSR is improving her or his language proficiency through the daily round of workplace discourse, and to predict the levels of improvements in language proficiency that can be expected, for example, in the first 3, 6, or 9 months at work in a call center. Call center managers also need to know what level of language is needed for each client's account. Perhaps most importantly, the BPO industry needs to understand, and be able to measure, the level of English proficiency their recruiters, trainers, coaches, and QA specialists need to carry out these language assessments in our call center. A fundamental specific language testing question underlies this last: Can you train nonlanguage specialists to be language assessors? A full research agenda is contained in these questions.
Recently, in the third Talking Across the World (TAW) Conference, which took place in Bangalore in 2008, there were, for the first time at this annual forum, keynote presentations on language assessment (Hamp-Lyons Reference Hamp-Lyons2008, Lockwood Reference Lockwood2008). These presentations argued for increased validity, reliability and practicality in the O&O assessment solutions for industry. Cases were cited in current practice that result in unfairness to the recruit being interviewed for call center work, and in unnecessary expense to the industry itself because of poor language assessment practices. There was also a call for specificity in the language assessment tools and processes and for stakeholder involvement in addressing the business requirements for language assessment.
It is hard to know where language and language assessment researchers should begin. Clearly, a multidimensional model to flexibly meet the different stakeholder and business requirements is important, and the tools and processes developed need validity, reliability, and practicality. The concern for language assessment research in this O&O industry context is perhaps not dissimilar to the concerns of language programme evaluation articulated by Lynch (Reference Lynch1996) over a decade ago:
An important part of the new approaches to evaluation is the increasingly felt need to incorporate stakeholders. . . . such ideas have been raised in ELT but little has been applied yet. What are the social and political factors that affect language and teaching? How can we best define and measure language abilities? What are the best research designs for our inquiry? How do learners acquire a second language? These questions, which cut a wide swath across the applied linguistics terrain, are critically related to the enterprise of programme evaluation. (p. 10)
Clearly, to solve the language training and assessment problems the O&O industry is facing will require specialist language expertise and specialist language testing and assessment expertise, in addition to adequate time and money.
