Collectivism and Testing

Collectivist societies are those in which the identity, life, and value of the individual is determined by membership of the state and its institutions. Decisions are made to benefit the collective and its survival rather than its individual members. Collectivism has been the predominant political philosophy throughout human history, allowing individualism to take front stage only in the coming together of Enlightenment ideas, late protestant individualism in Europe, and the beginnings of the industrial revolution. In the late 18th century individuals in Europe and the United States were no longer willing to play their part in collectives that ignored their rights and brutalized their lives. Locke (1690/Reference Locke and Crawford2002, p. 22) had already set the tone: “Lay then the axe to the root, and teach governments humanity.” The Enlightenment was subversive. Divine rights were no longer tenable. The security of an elite, unrepresentative, and unelected political class was undermined. But long before the Enlightenment project could be finished, neocollectivist movements appeared in the form of extreme political ideologies in the 19th and early 20th centuries. Collectivisms prior to the Enlightenment were untroubled by notions of freedom, democracy, reason, and individual liberty. Neocollectivisms had to survive in a world that had experienced the American and French revolutions, as well as the popular antigovernment uprisings across Europe in the mid-19th century (Sperber, Reference Sperber1994). Tillich argued that these new collectivisms, which we still face today, had to adapt in order to control populations in more subtle ways:

The most obvious example of extreme neocollectivism is that of Germany from 1933 to 1945, which also used education to achieve control over its people. “The regime endeavored to assert its control over the education system through reorganization and centralization” (Noakes & Pridham, Reference Noakes and Pridham1983, p. 433) through the nationalization of tests and the introduction of a strict standards-based system, in which curriculum and content was set by the Ministry of Education. Teachers were not allowed to innovate, and the state controlled both teacher education and certification (Cecil, Reference Cecil1971). Personal intellectual development was abandoned as a goal of learning, and “it was preferred that people should not have a will of their own and should totally subordinate themselves” (Cecil, p. 428). The authorities created a new myth of the state based on a biological reinterpretation of history (Cassirer, Reference Cassirer1946) that had begun in the 1850s (Gobineau, Reference Gobineau, de1999), and in which:

Education and testing, along with a mythological reading of history, are organized to create subservience to the collective. Individuality is ignored, and educational policies are designed to protect and intensify the identity and future of the collective in terms of its past. Popper (Reference Popper2002) called this “historicism,” and argued that equating “true happiness” and “justice” with the well-being of the collective links Plato's view of education and society to that of National Socialism and other forms of neocollectivist government (Popper, p. 186). Thus, centralization of educational systems and testing is but a symptom of a deeper political philosophy that can be traced directly to Plato.

Nor are established democracies immune from neocollectivism, for collectivist tendencies thrive wherever there is a perceived external threat. In the United Kingdom, for example, at the turn of the 20th century there was a fear of the loss of empire after the disastrous Boer War (1899–1902) and the loss of economic supremacy to the emerging American and German superpowers. Policymakers in the United Kingdom drew upon American eugenics (Black, Reference Black2003) when they began to see the degeneration in national intelligence as the cause of state decline. For four decades, the solution to this perceived problem was testing for allocation to suitable education and employment (Evans & Waites, Reference Evans and Waites1981, pp. 48–49). Spearman (Reference Spearman1904, p. 271) argued that language tests (particularly Greek) were more saturated with “g” (general intelligence) than other tests, and therefore especially valuable. Educational practitioners such as Cattell (Reference Cattell1937), working in the United Kingdom during the 1930s, drew on evidence from mass testing in the First World War (Yerkes, Reference Yerkes1921) to argue that structuring an efficient society through testing was essential to national survival. Along with many others he saw an opportunity to regiment society, even to the extent of following Terman's (Reference Terman1919, p. 17) advice on “determining vocational fitness” and placing the least able in the lowest ranks of the military—a policy for which he praised Mussolini (Cattell, Reference Cattell1937, p. 51). Cattell's work shows that by the 1930s educational and psychological testing had become part of the technology of a neocollectivist democracy, with a tendency to become less and less democratic:

Although such sentiments could not be expressed after the Second World War, the selective and elitist education advocated by Cattell and his colleagues (such as Burt) were embedded into the British Education system until the advent of Comprehensive Education in the 1970s.

Weber (Reference Weber, Henderson and Parsons1947) understood the educational policies of Germany and the United Kingdom, different as they were in scale, in terms of the drive for modern technical collectives to maximize production and efficiency. This involves the application of science to both industry and war, the specialization of labor, and bureaucratization. Testing is the technology that allows a collective to select the most appropriate person for a task: “Aptitude, regardless of whether it is the product of hereditary or environmental and educational influences, can only be determined by testing” (Weber, p. 261).

Since the war, collectivist governments have been benign, for the most part. But they embody many features that make Foucault's (Reference Foucault1975) social critique of their use of tests highly relevant. Indeed, testing has become ubiquitous as all skills, including language learning, are interpreted in terms of their economic value (Fulcher, Reference Fulcher2008). Even universities are subject to “the new regimes of accountability, and strict adherence to the economic imperative” (Harris, Reference Harris2008, p. 347). Examples of centrally controlled standards-based education systems, with a high level of control over teacher training and school learning, are not hard to find (Brindley, Reference Brindley, Shohamy and Hornberger2008). The clearest example is that of the United Kingdom, which has systematically introduced standards-based testing in an accountability framework that ensures total state control over the national curriculum and national tests, as well as teacher training; even educational staff are rewarded or disciplined based on national league tables (Mansell, Reference Mansell2007). Collectivist authorities operating within modern democracies have not learned the lessons from totalitarian collectives, that “whatever the exact character of the built-in safeguards, the best Ministry of Education is that which interferes least in the operation of the system” (Cecil, Reference Cecil1971, p. 4). The reason why such collectivist authorities continue to pursue hyperaccountability policies is because there is once again a sense that the state is failing to perform well in the global market place, and the educational system is reengineered to deliver the kinds of people who will serve the perceived needs of the economy (Fulcher, Reference Fulcher2008).

At the supranational level one example of the use of a system that is increasingly being used to harmonize and control language learning, to deal with perceived threats such as a weakened position in global markets, is the Common European Framework of Reference (CEFR) (Council of Europe, 2001; Fulcher, Reference Fulcher2004). Its primary use is emerging as a tool for designing curricula, reporting both standards and outcomes on its scales, and for the recognition of language qualifications through linking test scores to levels on the CEFR scales. We now see stronger evidence for more intrusive collectivist policy emerging in calls for claims of linkage to the CEFR to be approved by a central body (Alderson, Reference Alderson2007), and the removal of the principle of subsidiarity from language education in Europe (Bonnet, Reference Bonnet2007). If realized, these changes would lead to unaccountable centralized control of education and qualification recognition across the continent. However, many language educators have been convinced that without implementing European language policy in full and without question, language education will not serve the European economy well (Kelly, Reference Kelly2004). This is closely tied to the implementation of the Bologna declaration (1999) to harmonize and standardize higher education, including the structure and content of programs. With this package come “new systems of quality assurance” and a “restrictive view of education” (Harris, Reference Harris2008, p. 349) that can only be understood as the action of a neocollective mobilizing its human resources to combat perceived economic threats through standardization. The language of Bologna and other documentation surrounding standardization fabricates a set of specifically “European values” that a European citizen will hold; simultaneously “such ways of speaking also create ‘the other,’ the one who does not have this European essence” (Fejes, Reference Fejes2008, p. 525). References to “toleration” and “appreciation of diversity” create illusory symbols to act as “a technique which fosters nations to desire to participate in standardizing their higher education system at the same time as it fosters students to desire to become a specific European citizen” (Fejes, p. 526).Footnote ¹ The contradictory rhetoric that diversity can only be achieved through standardization to a common model is all too often left unchallenged.Footnote ²

Using tests to impose standardization and achieve political goals in this way is not new. Charlemagne is also known to have imposed a centrally controlled curriculum upon the monastic schools of Europe as he developed an educational system to control his empire: “In textbooks, as in fundamental practice, the Carolingians pressed for uniformity, and as such an examination was one form of pressure” (Jones, Reference Jones1963, p. 23). The earliest example of this is a report of an examination board to an unknown central authority in 809 ce. The test was a group oral examination for “computists” who taught principles of calculating the time relevant to the celebration of moveable festivals. There are no objectively “correct” answers to the questions set, but as they are all based on Bede's On the Reckoning of Time, the calculations of which were being imposed on the Western liturgical calendar, “the examiners had as one of the several aims the enforcing of a strict party line” (Jones, p. 23). This example should also make us wary of the current tendency to appeal to a so-called golden age of European unity in the construction of a new myth for the foundation of a superstate.Footnote ³

In this section, I have explored the nature of collectivist societies and their use of tests. Modern collectives use testing as a “scientific and technical tool” to control the educational system with the intention of (1) creating or reinforcing the identity of the state, (2) increasing the sense of belonging to the state, (3) selecting and allocating individuals to roles or tasks that benefit the collective, and (4) introducing hyperaccountability to ensure uniformity and standardization. This may not imply centralization where the state does not have the power to centralize, but in such cases it may create social and economic structures that would lead to anyone who does not comply being at a significant disadvantage. This may be the case with the financial and other penalties available under the No Child Left Behind legislation in the United States (Chalhoub-Deville & Deville, Reference Chalhoub-Deville, Deville, Spolsky and Hult2008, p. 512). It is certainly the case with the CEFR, where recognition of curricula or tests increasingly requires linkage to the CEFR, and “the main goal of this linking exercise is to demonstrate compliance with a mandate” (Kaftandjieva (Reference Kaftandjieva, Carlsen and Moe2007, p. 35). Validity is conceptualized as test recognition by institutions, and validity evidence becomes the extent to which linkage is demonstrated by institutionally approved procedures (North, Figueras, Takala, Van Avermaet, & Verhelst, Reference North, Figueras, Takala, Van Avermaet and Verhelst2003). Whether centralization has been achieved or not, test use to achieve collectivist goals usually occurs where there is a sense of external threat, which can co-occur with an “insider–outsider” mentality.

Individualism and Testing

Modern individualism has its origins in the writings of Locke (1690/Reference Locke and Crawford2002), Paine (1791/Reference Paine1999), the Declaration of Independence (Maier, Reference Maier1998), and the liberalism of the late 18th and 19th centuries. The starting point is radically different: “Men being, as has been said, by nature all free, equal, and independent, no one can be put out of this estate, and subjected to the political power of another, without his own consent” (Locke, 1690/Reference Locke and Crawford2002, p. 44) and that should he consent, there are limits upon the authority of the state, such that laws apply to all equally, that they protect the rights of individuals, that taxation should only be raised by consent, and that the legislative “neither must nor can transfer the power of making laws to anybody else, or place it anywhere but where the people have” (Locke, p. 66). Individualists also use history in support of their position, and it is also sometimes romanticized, but the sources are different: Magna Carta 1215, Henry III and the first parliaments from 1237 (Carpenter, Reference Carpenter2004), and the Levellers in Cromwell's army who demanded equality and universal suffrage (Benn, Reference Benn1976; Sharp, Reference Sharp1998). The central pillar of Enlightenment individualism is the right of each person to be free from control or oppression from a state that acquires too much power and begins to control the lives of citizens. Mill (Reference Mill and Gray1859) argued that individuals must even be protected from the tendency of liberal democracies to control them, for “whatever crushes individuality is despotism, by whatever name it may be called” (Mill, p. 71). This is not a right-wing position, as attested to by the fact that Tony BennFootnote ⁴ is found among its most outspoken advocates. Rather, it is a nonparty political struggle to escape oppression, currently benign or otherwise. Attempts to summarily dismiss individualistic critiques of test use as right-wing reactionism by labeling them “Eurosceptic” (Alderson, Reference Alderson2007, p. 660) therefore fail to engage with the social consequences of test use and misuse.

The purpose of education and language learning within a liberal individualistic tradition is the growth of the individual. The classic position was first put by von Humboldt:

Mill (Reference Mill and Gray1859, p. 81) took two principles from von Humboldt as necessary for human development, and the advancement of society: freedom and variety. For von Humboldt and Mill it is difference and divergence that create the conditions for a healthy, questioning society. For where the individual is limited in thought or choice of how to live by “rule of conduct, there is wanting one of the principal ingredients of human happiness, and quite the chief ingredient of individual and social progress” (Mill, Reference Mill and Gray1859, p. 63). Mill (Reference Mill1873, p. 257) believed that difference and divergence could be achieved through “a systematic National Education” that he believed would provide a stepping stone to universal suffrage. The link between individualism, education, and democracy with severe limits to the power of the state, particularly with regard to noninterference in education and testing, is not incidental. Dewey (Reference Dewey and Menand1888, p. 191) argued that the purpose of education is to develop individual personality, that “personality is the one thing of permanent and abiding worth, and that in every human individual there lies personality”; and for Dewey “democracy is not just a form of social life among other workable forms of social life, it is the precondition for the full application of intelligence to the solution of social problems” (Putnam, 1990, p. 185).

In this tradition, it is argued that a move away from individualism creates societies that do not experiment, lack tolerance, and tend to stagnate. Individuals grow through experiment, and experimentation is the most efficient way of changing and developing society. Similarly, a variety of school types with the freedom to experiment with curriculum and methodology is necessary “for the purpose of example and stimulus, to keep others up to the standard of excellence” (Mill, Reference Mill and Gray1859, p. 118). This happens through freedom, not control. Thus, states should have little say in what is taught, or how. Within this paradigm tests should only be used in ways that promote personal growth or provide individuals with new learning opportunities. They should only be used in gate-keeping mode when they are essential for the protection of individuals. Mill argued that public tests should be given to children at set ages to ensure the acquisition of key abilities that are essential for individuals to play an active role in society, but that “the knowledge for passing an examination (beyond the merely instrumental parts of knowledge, such as languages and their use) should, even in the higher classes of examinations, be confined to facts and positive science exclusively,” for the examination of anything upon which there could be disagreement requires a political decision about what views an individual should hold, and “all attempts by the state to bias the conclusions of its citizens on disputed subjects, are evil” (Mill, Reference Mill and Gray1859, p. 119). When it comes to gate keeping, the state should once again play no role in test use; rather “public certificates of scientific or professional acquirements, should be given to all who present themselves for examination, and stand the test; but that such certificates should confer no advantage over competitors, other than the weight which may be attached to their testimony by public opinion” (Mill, Reference Mill and Gray1859, p. 119). Tests can therefore play an important positive role in society, but not as Foucault's “normalizing gaze” (Fulcher & Davidson, Reference Fulcher and Davidson2008).

Examples of tests being used in keeping with individualism include the original Binet tests, designed for the sole purpose of identifying children in need of additional help. Work on diagnostic and classroom testing, loosely defined as “low-stakes formative assessment” would also fall into this category. Language assessment in the classroom is an underresearched area, but what has been done shows that its purpose is to act as a way of providing individual learners with feedback that helps them to improve in an ongoing cycle of teaching and learning (Rea-Dickens, Reference Rea-Dickens2001). In such a context Dewey's notion of personal growth as a validity criterion is echoed by current researchers, such as Moss (Reference Moss2003). The recent interest in “dynamic assessment” in language testing is not different in principle from this position (Lantolf & Poehner, Reference Lantolf, Poehner, Shohamy and Hornberger2008a, Reference Lantolf and Poehner2008b). However, it draws upon Vygotsky's zone of proximal development and defines validity as the extent to which testing improves learner performance; it therefore differs from other approaches in its conceptualization of the educational intervention: “In DA [dynamic assessment], assessment and instruction are a single activity that seeks to simultaneously diagnose and promote learner development by offering learners mediation, a qualitatively different form of support from feedback” (Lantolf & Poehner, Reference Lantolf and Poehner2008a, p. 273).

When tests become high stakes and their primary purpose is still to provide access to education or employment for nonelite individuals, even though what is “valued knowledge” is still decided upon by society, they still serve the individual. This is arguably the case with the Indian civil service examinations, which some see as having introduced liberty and equality to selection (Spolsky, Reference Spolsky1995, p. 17–19). That this was termed “the Chinese principle” (Spolsky, p. 18) indicates that the Chinese Civil Service Examinations fulfilled a similar purpose, even though they are further away from the individualistic end of the cline. Miyazaki (Reference Miyazaki1981, p. 111) stated that “the purpose of instituting the examinations . . . was to strike a blow against government by the hereditary aristocracy . . . and to establish in its place an imperial autocracy.” This was no democratic system, but the tests, as Miyazaki pointed out (pp. 118–121) were open to any candidate, there was no entry fee, the tests were conducted under standardized conditions, and scripts were blind marked.

Many modern language tests for academic and other specialist purposes may be categorized in this way when they are used for their original explicitly stated purposes (Fulcher & Davidson, Reference Fulcher and Davidson2009), especially those built on careful analyses of how individuals need to use language successfully in a specific context (e.g., Chapelle, Enright, & Jamieson, Reference Chapelle, Enright and Jamieson2008), and avoid bias against individuals or subgroups of the test taking population (Elder, Reference Elder1997).

In this section I have explored the nature of individualism and the use of tests that is likely to be made within this paradigm. Specifically, tests are used to (1) help individuals to develop their own potential without reference to others, (2) ensure that individuals acquire the key knowledge and skills they will need to innovate in their own lives and participate in democratic societies, and (3) provide access to employment through the assessment of critical skills where practicing without those skills would be detrimental to others. This may or may not imply centralization, but if it does, the state should not specify the content of the tests; this should be left to professional educators and stakeholders. Validity is assessed in terms of the success in helping individuals to achieve their goals and develop necessary skills, and validity evidence will be collected in relation to the claims of intended test effect. External systems are never imposed upon teachers. Teachers are involved in the process of defining the knowledge and skills to be taught and assessed, or design their own assessments as part of the learning process. One of the criteria for success is therefore the empowerment of professional educators to make their own judgments and decisions in their own contexts of work (Fulcher & Davidson, Reference Fulcher and Davidson2007, p. 34–35).

A Way Forward

Although a Foucaultian critique provides insights into testing in collectivist states, it offers few solutions; the marginalized cannot do anything to ameliorate their condition. One way forward is to rediscover the unfinished business of the Enlightenment.

Such an agenda would place the needs of the individual first, but recognize that the individual is only able to grow within a community. This is the essential insight of Dewey: Individual growth is about personal experiment and the implementation of intelligent learning through inference; society is strengthened when individuals within communities are allowed to inference, and communities can “try out” innovative solutions to social problems. Educational systems that value standardization of outcomes produced according to a mould created by “juggernaut-like centralizing institutions” (Davies, Reference Davies, Shohamy and Hornberger2008, p. 438) have tended to stifle the variety and divergence that are prerequisites for innovation, experimentation, and growth.

Shohamy's (Reference Shohamy2001b) view of “democratic assessment” highlights a way forward, calling as it does for participatory democracy, the responsibility of language testers for the use of powerful tests, the consideration of the views of stakeholders, and the protection of individuals from powerful institutions. Building upon Shohamy (Reference Shohamy2001a, Reference Shohamy2001b), Fulcher and Davidson (Reference Fulcher and Davidson2007) introduced the notion of effect-driven testing, which holds that

This position explicitly links the consequences of test use to test design, and it requires the statement of intended effects as drivers of design decisions. Decisions with regard to intended effects and how these are implemented in design are arrived at through open, democratic, and inclusive discussion (Fulcher & Davidson, Reference Fulcher and Davidson2007, p. 147). The practical implications are spelled out in Fulcher and Davidson (Reference Fulcher and Davidson2009). Test design is not dissimilar from architecture, in which the design of a building is directly related to its intended effect on a defined user group. The specification of the user group and the intended effect dictates the shape, size, and layout of the building or test. The explicit intended effect will produce a very different architecture in each case. This is not only an important principle of test design (see Mislevy & Riconscente, Reference Mislevy and Riconscente2005, p. 3), but also imposes restrictions on the post-hoc use of tests for purposes other than those for which they were originally designed.

Aligning explicit statements of intended test-effect to test design decisions through explicit validity arguments and test architectures will have a number of corollaries. First, any collectivist use of a test will either be made explicit, or will be ruled out, at the design stage. This introduces the potential for accountability and denunciation of illegitimate test use. Second, as the use of a test for an originally unintended purpose would require an architectural retrofit with a new validity argument, the misuse of existing tests for collectivist purposes would be reduced.

Mill (Reference Mill1873, pp. 253–254) predicted that his work on “the liberty” would be rediscovered whenever “this noxious power” of collectivism is exercised. As Foucault (Reference Foucault1975) correctly observed, the operations of neocollectivist authorities in the postwar world are more subtle (even subliminal) and manipulative than their predecessors. Prophetically, Mill urged us to be sensitive to the impact of collectivist policies and practices on individual liberties, ways of thinking and behaving. All growth, including economic development, comes from variety and innovation, not from uniformity and standardization. Effect-driven testing is a way of revitalizing the Enlightenment project in the understanding of test use by making explicit the intended effects of test use, thus making it possible to more easily identify the misuse of these powerful tools for collectivist ends.