There is nothing new about testing in schools. Teachers need to know how their students are performing to tailor their teaching to the needs of students. Students need to know how they are performing to help them recognise their strengths and understand where their learning challenges lie. Schools also need to know how students perform. Without some measure of student achievement, there are fewer opportunities to assess how the school as a whole is meeting the needs of its key users. Testing under the National Assessment Program—Literacy and Numeracy (NAPLAN) began in 2008. This testing regime brought together various state and territory tests that were in place before that time. Since then it has become embedded in the annual cycle of school activities. Beyond the reasons listed above, NAPLAN is an instrument that is used for accountability purposes. It is used to help determine school funding formulae. It is touted as an instrument that will help assess teaching quality. It also reflects curriculum. In Australia, it has become part of the rhetoric of schooling. NAPLAN scores are also used in conjunction with measures of disadvantage. In remote Australia, ‘disadvantage’ is a term often associated with Aboriginal and Torres Strait Islander student performance. It is used to explain poor performance, as measured by NAPLAN. Coupled with other information provided on the My School website, these measures have become a powerful tool to describe and analyse education systems across Australia. This paper uses that information to draw attention to analysis conducted by the Cooperative Research Centre for Remote Economic Participation for its Remote Education Systems project about education in very remote schools.
The purpose of the article is to challenge assumptions about the statistical association between attendance rates, socio-economic advantage and student outcomes as measured by NAPLAN. Those assumptions have to a large extent driven a policy response under the banner of ‘Closing the Gap’ and ‘Smarter Schools National Partnership’ programs, among others. They have become so embedded in the discourse of education — with a different nuance in remote Australia — that they have become axiomatic. The analysis may suggest reasons for the very small amount of progress (and in some cases regress) being made under those national initiatives. The ultimate result of the huge effort being put into potentially futile initiatives is that we as educators will lose focus of the main purpose of education, to make it count in the lives of students. Instead, in the end we may just be counting education.
The Discourse of Remote Education
Before presenting the analysis, it is important to gain an understanding of the discourse of remote education. While much of the discourse is couched in similar terms to that of the mainstream, it has particular nuances in the remote context. This article is particularly concerned with the very remote context of education, where the bulk of schools are populated with Aboriginal and Torres Strait Islander students, many of whom — if not most — have grown up with English as another language. Schools that are identified in this way are often described and defined in stereotypical ways. At the top of the list of stereotypes is that of disadvantage.
The Rhetoric of ‘Disadvantage’
One of the predominant themes that pervades much of the literature on remote education is that of Aboriginal and Torres Strait Islander ‘disadvantage’. The intent of the word is perhaps to convey a sense of the ‘disparity’ (Bath, Reference Bath2011) between Aboriginal and Torres Strait Islander and non-Indigenous people on a range of indicators (see, e.g., Steering Committee for the Review of Government Service Provision, 2011). It has been defined specifically as ‘The difference (or gap) in outcomes for Indigenous Australians when compared with non-Indigenous Australians’ (Steering Committee for the Review of Government Service Provision, 2012, p. xiv). The concept then extends to ‘closing the gap’ (Council of Australian Governments, 2009) in a general sense and in a more specific educational context (What Works. The Work Program, 2012).
There can and should be no denial of the data and their practical implications that are behind these labels, but there are problems with the pervasive rhetoric of disadvantage. First, there is a real risk that being Aboriginal or Torres Strait Islander is the disadvantage, in effect ‘cultural dysfunction’ (Cowlishaw, Reference Cowlishaw2012, p. 412). Second, the deficit discourse is most frequently based on non-Indigenous understandings of advantage, and developing a sense of the ‘Aboriginal problem’ (Gorringe, Reference Gorringe2011). Third, the racial nature of disadvantage may lead to a promulgation of responses that lead to ‘exceptionalism’ of Aboriginal and Torres Strait Islander people on the basis of race (Langton, Reference Langton2012) — that is, an exceptionalist view that comes with race categorisations segregates and therefore discriminates against Aboriginal and Torres Strait Islander peoples. Fourth, the disadvantage discourse may idealise the interests of the privileged, reinforcing a hegemony that in turn reinforces existing power dynamics in society and results in ‘self-fulfilling prophecies’ of the disadvantaged (Orlowski, Reference Orlowski2011, p. 43).
But what about the data on Aboriginal and Torres Strait Islander disadvantage in education? The Overcoming Indigenous Disadvantage Report (Steering Committee for the Review of Government Service Provision, 2011) points to a number of key indicators that represent the ‘gap’. These are:
• lower school attendance and enrolment rates;
• poorer teacher quality;
• a lack of Indigenous Cultural Studies in school curricula;
• low levels of Year 9 attainment;
• low levels of Year 10 attainment; and
• difficulties in the transition from school to work.
The Overcoming Indigenous Disadvantage Report, while not singularly focused on remote disadvantage, highlights the larger gap for remote Aboriginal and Torres Strait Islander people. Further, it makes links from education to other areas of disadvantage: health, employment, early childhood development, and the home environment. The report paints what could be described as a very sad picture of the Aboriginal and Torres Strait Islander population. A picture that on the whole (with the notable exceptions of mortality rates, home ownership, post-secondary outcomes, employment and income) does not appear to be getting much better. Again, the data should not be dismissed. It does have great utility. However, repeated releases of the same sad stories denies a view of Aboriginal and Torres Strait Islander culture that ought to be celebrated, honoured and valued. Seldom is the richness and diversity of life in remote communities discussed in the media, let alone the literature. Nor are the learning journeys of many remote Aboriginal and Torres Strait Islanders celebrated. It is, however, heartening to see an alternative rhetoric emerging from Aboriginal and Torres Strait Islander authors, who allow those of us who are non-Indigenous to take a step back from our otherwise uncontested philosophical positions and reflect on difference in terms of epistemologies, axiologies, ontologies and cosmologies (see, e.g., Arbon, Reference Arbon2008; Ford, Reference Ford2010; Nakata, Reference Nakata2008), rather than deficits.
The Rhetoric of Poor Outcomes
More specifically, in the realm of the national ‘education system’ there is another rhetoric associated with the rhetoric of disadvantage: the rhetoric of poor outcomes. The 2009 Closing the Gap on Indigenous Disadvantage report exemplifies this rhetoric:
Results for Indigenous students in very remote Australia are extremely poor. The majority of Indigenous students in very remote Australia currently do not meet the national minimum standard in reading, writing and numeracy. (Department of Families Housing Community Services and Indigenous Affairs, 2009, p. 15)
It is also exemplified in the comparative presentation of NAPLAN data, which highlights the stark difference between results for Aboriginal and Torres Strait Islander students and their non-Indigenous counterparts. The 2012 NAPLAN National Report (ACARA, 2012) compares average scores by geolocation, Indigenous status and state. For example, for the Year 3 Persuasive Writing test in the Northern Territory (see p. 18 and p. 19), the average score for Indigenous students in provincial regions was 337.5 while in remote areas it was 171.1. By contrast, for non-Indigenous students, the respective results were 393.8 and 392.8. At first glance, the data suggests that Aboriginal and Torres Strait Islander students are failing compared to non-Indigenous students. Next, it could be deduced that they are failing more in very remote areas. The logical question that follows from these conclusions, is ‘why?’.
There have been many studies that have explored why this apparent failure is so ‘bad’ in very remote locations. Inevitably, many studies come to the conclusion that a key factor for the reasons of failure is related to (poor) attendance and enrolment rates (Purdie & Buckley, Reference Purdie and Buckley2010; Wright, Arnold, & Dandie, Reference Wright, Arnold and Dandie2012), based on a strong (presumably causal) association between attendance and outcomes in a variety of contexts (Ehrich et al., Reference Ehrich, Wolgemuth, Helmer, Oteng, Lea, Bartlett and Emmett2010; Gottfried, Reference Gottfried2010; Zubrick et al., Reference Zubrick, Silburn, De Maio, Shepherd, Griffin, Dalby and Cox2006). However, other studies suggest that the biggest single predictor of outcomes is the Index of Community Socio-Educational Advantage (ICSEA), which draws together a number of education related variables (Ainley & Rothman, Reference Ainley and Rothman2010; Miller & Voon, Reference Miller and Voon2011). Other commentators couple poor attendance with unemployment, ‘underperforming schools’ and poor teaching in remote schools (Hughes & Hughes, Reference Hughes and Hughes2012). Some talk of a ‘crisis’ in Aboriginal and Torres Strait Islander education (Rigney, Reference Rigney, Purdie, Milgate and Bell2011) and more specifically in remote education, because ‘young people are not being well educated’ (Ma Rhea, Reference Ma Rhea2011, p. 61).
Some studies point out that the problem of poor test results might not be about the student but about the test itself, which favours mainstream cultural values and proficiency in English rather than proficiency in another language (Wigglesworth, Simpson, & Loake, Reference Wigglesworth, Simpson and Loakes2011). Schwab (Reference Schwab2012, p. 13), commenting on the validity of NAPLAN testing regimes, suggest that ‘High-stakes tests are dull tools of assessment’, which in themselves do nothing to increase student or school performance. Kral (Reference Kral2012), speaking from the Ngaanyatjarra Lands context, suggests that knowledge is socially negotiated rather than individually acquired. If that is the case, NAPLAN is not only a dull tool but it generates false readings. Griffiths (Reference Griffiths2011), in an international comparative study of best-practice indigenous education, suggests the need for both culturally sensitive assessment tools, as well as standardised mainstream testing regimes. The point of the former is to acknowledge that ‘success’ may well be defined differently by indigenous peoples, and therefore the measures of that success ought to be captured. Others see the ‘problem’ lying with the system itself. Ford (Reference Ford2012), for example, argues that: ‘It is hard to imagine an education system that has failed a cohort of students so badly’ (p. 19).
Regardless of the reasons for apparent ‘failure’, what NAPLAN — perhaps unintentionally —does is to normalise the ‘success’ associated with whiteness. Lingard, Creagh, and Vass (Reference Lingard, Creagh and Vass2011) conclude that:
the coupling of ‘closing the gap’ with NAPLAN establishes the achievements of (mostly) ‘white’ students as the benchmark, with the flip side being that failure to reach these standards enables ‘blame’ to be directed towards Indigenous students. (p. 328)
This is an important observation. If NAPLAN was seen as an instrument that measures ‘whiteness’ it would probably be abandoned immediately. As such, it would be seen at best as a tool that promotes western ontologies and epistemologies, and at worst as a tool that promulgates racist views. There is no doubt about the need for a tool that assesses student learning. But that tool ought to reflect the student and learning context — it must be culturally fair (Klenowski, Reference Klenowski2009). It should be applied to a better understanding of how to better facilitate good pedagogy. Unfortunately, NAPLAN in remote Aboriginal and Torres Strait Islander communities generally falls short of these ideals — it is commonly used as an accountability instrument (Klenowski, Reference Klenowski2011). Could it be that the problem with NAPLAN is that it is an artificial measure of outcomes in remote communities, which really does not measure the real learning that goes on? In fact, is it a measure of schooling rather than a measure of learning?
The Rhetoric of Remote Schooling
While the discourse of schooling is both national and international in its presentation, it has its own nuances in remote contexts. The language is by and large consistent with the language of schools in urban, regional and rural settings. The talk in remote education is about attendance, teachers, teaching, pedagogy, curriculum, school-community partnerships and accountability. While these fundamentals are the same, the way they are discussed is different. The reason is that there is something quite different about remote education.
School Attendance
There is an apparent assumption that schooling (and all that goes with it) is the same thing as learning. The argument for attendance goes something like this. Attendance is a fundamental of schooling. Therefore if children do not attend school how can they be taught, and how can they learn? And if they do not learn, how can they achieve educational outcomes? On the surface, this sounds like a reasonable argument. As the Aboriginal and Torres Strait Islander Education Action Plan 2010–2014 states: ‘Successful learning cannot be built on irregular attendance. There is evidence to suggest that the more regularly students attend school the greater their success in learning’ (Ministerial Council for Education Early Childhood Development and Youth Affairs, 2011, p. 16). These assertions can be challenged at a number of levels. First, success is seldom defined and often assumed to mean certain things (Feinstein, Duckworth, & Sabates, Reference Feinstein, Duckworth and Sabates2008). But what does success look like in remote schools? Second, there is an assumption that learning will automatically happen because a child attends. Engagement in learning is not the same as attendance at school (Appleton, Christenson, & Furlong, Reference Appleton, Christenson and Furlong2008). For example, what happens when a young person's disruptive behaviour (perhaps because they do not want to be at school) prevents others from learning? Third, there is a tacit assumption that schooling is the primary vehicle through which children and young people learn. But is not learning much broader than what happens at school? Does not learning occur at home, on the football field, in church, through technology and indeed through the course of life? (Collins & Halverson, Reference Collins and Halverson2009). There is no question that schooling and learning are related. If students are to learn from their teachers, they must be taught by their teachers and they must therefore spend time with their teachers. But this is not the same thing as attendance. Attendance is of course easy to measure, but it is not a proxy for student learning. However, the assumption here is that improved attendance will improve student outcomes.
Teacher Quality
As stated earlier, if students want to learn from their teachers, they have to spend time with their teachers. This could be in person or it could be virtually. However, the rhetoric of schooling has shifted well beyond this. Teachers need to be ‘quality teachers’ because quality teachers make a difference. This argument has been brought out in recent times in reports about the apparent successes of East Asian and Finnish education systems, which place a high value on teachers (ABC, 2012; Jensen, Reference Jensen2012). Hattie (Reference Hattie2009) comments that the ‘mantra, that teachers make the difference, is misleading’ (p. 108). He suggests that the power of the teacher is in their teaching methods, their high expectations of all students and their creation of positive student-teacher relationships, that makes ‘above average effects on student achievement’ (p. 126). To this end, the recently developed National Professional Standards for Teachers (Australian Institute for Teaching and School Leadership, 2011) codify what makes a quality teacher. They provide a practical framework for assessing the professional standard of a teacher, and therefore potentially provide benchmarks for teachers wanting to progress from graduate through to lead teacher status. But questions remain about the appropriateness of these standards in remote contexts.
If the teacher is important, then in remote communities, the challenge may well be not only one of attracting teachers who have the right qualities for teaching, but also of being suited to the remote cross-cultural context in which they are expected to work. Then there is the ongoing challenge of professional development. Efforts in this regard supported by the National Alliance for Remote Indigenous Schools (NARIS) are designed to address these challenges: NARIS ‘aims to attract, retain and develop exceptional teachers and leaders in these schools’ (NARIS, n.d.). NARIS, along with other initiatives, aims to increase the number of Aboriginal and Torres Strait Islander teachers generally as well as specifically in remote contexts (see also More Aboriginal and Torres Strait Islander Teachers Initiative, 2012). In recognition of the inadequacies of the Standards, the work of the Tertiary Educators Rural, Regional and Remote Network, which has produced a supplementary set of standards for remote contexts, is noted (Society for the Provision of Education for Rural Australia, 2012). But the increased attention on professional standards, may make it even more difficult for remote Aboriginal and Torres Strait Islander educators to be recognised as fully qualified ‘teachers’. The underpinning assumption that sits behind these initiatives is that better teaching will produce better outcomes. There are no quantitative studies to show that in remote contexts, this relationship actually exists.
Teaching and Learning (Pedagogical) Quality
If the teacher is important to schooling, so too is the teacher's teaching. Hattie's (Reference Hattie2009) Visible Learning devotes two chapters to teaching approaches. He suggests that four of the top ten influences on student learning are related to teaching approaches. In the context of remote Aboriginal and Torres Strait Islander communities, a range of other teaching and learning strategies are also suggested. These include a culture of ‘high expectations’ (Sarra, Reference Sarra2011; What Works. The Work Program, 2012), whole-school approaches to evidence-based literacy and numeracy teaching (What Works. The Work Program, 2012), bilingual approaches (Devlin, Reference Devlin, Purdie, Milgate and Bell2011), embedding racial identity into pedagogy (Kickett-Tucker & Coffin, Reference Kickett-Tucker, Coffin, Purdie, Milgate and Bell2011; Sarra, Reference Sarra2011), use of ‘direct instruction’ techniques (Pearson, Reference Pearson2011), and the imperative of incorporating Aboriginal and Torres Strait Islander epistemologies, ontologies and axiologies in teaching and learning strategies (Arbon, Reference Arbon2008; Ford, Reference Ford2010; Nakata, Reference Nakata2008). Interestingly, despite the breadth of what works literature, the clear links between any of these strategies in remote contexts and student outcomes is less than clear. For example, in the recent discussion of eleven remote schools in the What Works program (2012), of the nine very remote schools, only one was able to show consistent improvement in both attendance and NAPLAN outcomes over 4 years. Similarly, a review of the Stronger Smarter Learning Communities project showed that while teachers’ perceptions had changed, student outcomes were no different to control schools (Luke et al., Reference Luke, Cazden, Coopes, Klenowski, Ladwig, Lester and Woods2011). The long list of pedagogical imperatives suggests that schools are ‘looking for superman’ in order to achieve ever-elusive outcomes.
Curriculum
A further element in the rhetoric of schooling centres on curriculum. In the international literature (Hattie, Reference Hattie2009), curriculum has been shown to be a factor that contributes to learning outcomes — though not as significant as teaching or teacher quality. In Australia, this has been amplified by the development of an Australian Curriculum Framework: ‘The rationale for introducing an Australian Curriculum centres on improving the quality, equity and transparency of Australia's education system’ (ACARA, 2012b, p. 5). For Aboriginal and Torres Strait Islanders the Curriculum ‘promotes the importance of pursuing excellence within education settings that respect and promote their cultural identity’ (p. 7).
In remote contexts, curriculum content is considered to be an important factor contributing to school success. The What Works program's (2012) analysis of what makes successful remote schools indicates that making ‘learning content engaging, accessible and culturally responsive’ (p. 69) is important. The Overcoming Indigenous Disadvantage report (Steering Committee for the Review of Government Service Provision, 2011) concurs: ‘culturally appropriate education for Indigenous students can contribute to good ‘mainstream’ academic outcomes, as well as consolidating community teachings and knowledge’ (p. 62). The assumption underlying these assessments is that better outcomes will flow from a more culturally responsive curriculum. However, there are no studies that show the direct (let alone causal) relationship between curriculum content and learning outcomes for remote Aboriginal and Torres Strait Islander students. If teachers in remote schools are required to report to standards prescribed in the Australian Curriculum (ACARA, 2009), it is likely that many (if not most) students in remote students will receive an ‘E’, on the A-to-E scale. Notwithstanding the allowances made for reporting for English as Another Language/Dialect students, this has the potential to reinforce the perceptions that Aboriginal and Torres Strait Islander students are failures and are in deficit.
School-Community Partnerships
Among the many factors that are reported to improve educational outcomes, is ‘school-community partnership’ (Silburn, Nutton, McKenzie, & Landrigan, Reference Silburn, Nutton, McKenzie and Landrigan2011; What Works. The Work Program, 2012). Hence, the discourse relating to successful schools has developed to incorporate the language of partnerships and strategies designed to address ‘disadvantage’ frequently incorporate reference to school-community partnerships (Productivity Commission, 2012). The Smarter Schools National Partnership for Low Socio-economic Status School Communities specifically incorporates outputs that ‘build external partnerships with parents, schools, businesses and local communities’ (Council of Australian Governments, 2008). This then feeds into the reporting frameworks for the states and territories. For example, the 2011 Northern Territory Annual Report for Smarter Schools reports as follows:
Identified as key to the improvement of Indigenous student outcomes, School-Community Partnership Agreements have been led by the Community Engagement team to support the consultation process between schools, communities and families in creating strategic frameworks for 55 Aboriginal and Torres Strait Islander Education Action Plan focus schools. As at 31 December 2011, 33 School-Community Partnership Agreements across all regions in the Northern Territory were finalised and signed, with five nearing completion and 14 in progress. (Department Education Employment and Workplace Relations, 2012, p. 16)
Indeed, the term ‘school-community partnership’ is almost part of the vernacular of school improvement agendas, so much so that it is rarely contested, and almost universally accepted as inherently ‘good’. There are some important critiques, however. Lowe (Reference Lowe, Purdie, Milgate and Bell2011), for example, argues that the ‘concept of “partnership” has largely been suborned by a deliberate neoliberal government agenda contrary to the spirit and practice of social relationships as Aboriginal families and communities have understood them’ (p. 28). Lea, Thompson, McRae-Williams, and Wegner (Reference Lea, Thompson, McRae-Williams and Wegner2011) suggest that the processes of building engagement may ironically erode the trust that parents have in their children's schools to do the job they have to do — to provide a good education, to teach literacy and numeracy and to prepare them for a future. No one would deny the importance of good relationships between schools and communities. However, there are some inherent unstated assumptions that go with the rhetoric of school-community partnerships. One is that the focus of partnership is the school — what would happen if the language were changed to community-school partnerships? There is another assumption that partnership agreements can be measured in such a way to demonstrate improved learning outcomes. Nowhere in the literature is this assumption explored in any detail.
Accountability
The final element of the rhetoric of remote schooling relates to accountability. The national discourse couples accountability with choice and school autonomy. However, the reality in remote contexts is that choice often does not exist. The choice, if any, could be to go to school (which may well be boarding school) or not. This is reflected in the fact that while in Australia as a whole, one-third of all students attend non-government schools, in remote areas just 17% of all students attend non-government schools (Gonski et al., Reference Gonski, Boston, Greiner, Lawrence, Scales and Tannock2012, p. 10). The argument for improved accountability of schools is that it is related to better student outcomes (Schültz, West, & Wößmann, Reference Schütz, West and Wößmann2007). The Schools Workforce report (Productivity Commission, 2012) discusses this issue in relation to addressing educational disadvantage:
In the context of educational disadvantage, [systems of accountability] would imply that school leaders need to set goals for their school, measure and assess their progress, and be held accountable for outcomes. (p. 277)
The report suggests that accountability measures could include student outcomes (presumably NAPLAN could be used here), attendance rates, students’ attitudes to school, parental involvement in school, and students’ feelings of connection to the school. Collecting this data in remote cross-cultural, multilingual contexts has its own challenges. Attempts to contextualise standard community or student perception surveys, for example, are fraught with difficulty. Recent work carried out by Osborne (Reference Osborne2012) for a remote independent school in the Northern Territory showed that many of the concepts associated with schooling do not necessarily translate across languages, and the fundamental assumptions about what school is for in mainstream schools do not necessarily apply in Aboriginal cultures. It would appear that for many, it is assumed that the language of schooling translates seamlessly across cultures in such a way that the discourse is universally understood by all stakeholders. This is in itself an important issue that deserves a detailed critique, beyond what has been presented here.
Analysis of NAPLAN Results in Very Remote Schools
It should be evident that the rhetoric discussed above is underpinned by a number of measurement assumptions that can be captured using information collected by schools for annual NAPLAN tests. Information provided on the My School website includes a range of school level data, including NAPLAN test results, ICSEA scores, VET in schools data, attendance rates, student gender, Aboriginal and Torres Strait Islander status, student enrolments, school geolocation, school finances, staffing levels, languages spoken other than English, along with some descriptive information about the school itself. The data used for this article was drawn from NAPLAN numeracy and reading data, school attendance data and ICSEA scores for the school.
The presentation that follows is divided into two sections. The first set of three charts compares school attendance figures with Year 3 reading scores for very remote schools. The second set of three charts compares ICSEA values with Year 5 numeracy scores for very remote schools. Similar patterns could be replicated for other year levels. For ease of interpretation, the analysis focuses only on two NAPLAN variables. Other variables (Spelling, Persuasive Writing, Grammar and Punctuation) have not been tested in the same way for this analysis.
The analysis was conducted using Microsoft Excel scatter plots with trend lines, associated equations and r 2 coefficients of determination displayed. Only those schools with corresponding test scores were included in the analysis. In 2012 there were 276 schools that were classified as ‘very remote’ (using the Australian Bureau of Statistics remoteness classifications). Offshore and Bass Strait islands were not included in this analysis.
Year 3 Reading in Very Remote Schools
Figure 1 is a scatter plot showing the relationship between the Year 3 Reading score in 2012 for very remote schools that reported greater than 80% Aboriginal and Torres Strait Islander student population. The trend line does suggest a positive relationship between the two, but the r 2 value of .1067 suggests there is little correspondence between the variables of attendance and NAPLAN score. In other words, the low correlation suggests that school attendance is not a good predictor of test results (or vice versa). The equation of the line (produced by Microsoft Excel), with a slope at 1.97 suggest that for every 10% increase in school attendance there will be a 19.7-point increase in scores. Based on the line of best fit, attendance would need to be about 160% to achieve the national average score of 420.
NAPLAN Year 3 Reading scores for very remote schools with greater than 80% Aboriginal and Torres Strait Islander students, by per cent school attendance, 2012 (n = 78).
Figure 2 is similar to the previous chart except the data shown is for schools with less than or equal to 80% of Aboriginal and Torres Strait Islander students. The r 2 value in this instance is much higher, which could be described as a moderate correlation. For this group of schools there is a reasonable likelihood that as attendance increases, so too will reading scores. The equation of the trend line suggests that at an attendance rate of 99%, the national average reading score of 420 will be achieved.
NAPLAN Year 3 Reading scores for very remote schools with less than or equal to 80% Aboriginal and Torres Strait Islander students, by per cent school attendance, 2012 (n = 55).
Figure 3 combines the previous two figures. Now the r 2 value of .45 suggests a moderate correlation between the variables of school attendance and reading score. The slope of the trend line suggests that an increase of 10% in attendance will yield a corresponding 42-point increase in NAPLAN scores. At this rate the national average is achieved with an attendance rate of 106%.
NAPLAN Year 3 Reading scores for all very remote schools, by per cent school attendance, 2012 (n = 133).
Year 5 Numeracy in Very Remote Schools
Figure 4 takes the 2012 Year 5 Numeracy test results for very remote schools with greater than 80% Aboriginal and Torres Strait Islander student population, and plots them against the Index of Community Socio-Educational Advantage (ICSEA). The r 2 value suggests that there is a weak relationship between the two variables. In other words, a change in ICSEA does not reliably predict a corresponding change in test results (or vice versa).
NAPLAN Year 5 Numeracy scores for very remote schools with greater than 80% Aboriginal and Torres Strait Islander students, by ICSEA, 2012 (n = 74).
Figure 5, as above, shows the relationship between ICSEA and Year 5 NAPLAN Numeracy test scores, this time for very remote schools with Aboriginal and Torres Strait Islander student populations less than or equal to 80%. An r 2 value of .29 suggests a moderate relationship between the two variables. The equation of the trend line suggests that for each 100-point increase in ICSEA value there will be a 21.6-point increase in test scores. This is largely consistent with national analyses of NAPLAN scores versus ICSEA (Miller & Voon, Reference Miller and Voon2011).
NAPLAN Year 5 Numeracy scores for very remote schools with less than or equal to 80% Aboriginal and Torres Strait Islander students, by ICSEA, 2012 (n = 53).
Figure 6 combines the data from the previous two charts. In this instance, an even stronger relationship is suggested (r 2 = .68), indicating that a change in one variable will be associated with a change in the other — for each 100-point increase in ICSEA value there will be a 32-point increase in test scores.
NAPLAN Year 5 Numeracy scores for all very remote schools by ICSEA, 2012 (n = 127).
Implications for the Discourse
Before considering how the above findings affect and influence an understanding of the discourse of remote education, it is important to acknowledge the limitations of this analysis. It is only based on two outcome areas for one year. Deeper analysis may reveal different trends and relationships. However, even with its limitations, the data does raise some important questions. The second limitation of this analysis is that it is based on school-level data. School populations change with time and in remote communities particularly, movement in and out of schools and communities can affect outcomes from year to year.
It is important to note, based on Figure 3 and Figure 6, that the assumptions about the associations between attendance, disadvantage and NAPLAN based outcomes hold true for very remote schools generally, as they do for all schools in Australia. That is, as socio-economic advantage increases, and as attendance increases, so too does student achievement (see, e.g., Hancock, Carrington, Shepherd, & Zubrick, Reference Hancock, Carrington, Shepherd and Zubrick2013).
However, when those schools with Aboriginal and Torres Strait Islander populations above 80% are set aside and analysed separately, as shown in Figure 1 and Figure 4, the results are quite different. The relationship between ICSEA and outcomes becomes weak and the relationship between attendance and outcomes is effectively non-existent. However, for the remainder of schools with fewer than 80% Aboriginal and Torres Strait Islander students, as shown in Figure 2 and Figure 5, the relationships remain largely intact.
What this then means is that strategies designed to increase attendance or reduce ‘disadvantage’ will not necessarily work to produce better learning outcomes. This is reflected to some extent in the findings of the Smarter Schools National Partnerships where the amount of effort put into ‘Closing the Gap’ schools and ‘Low SES’ schools in the Northern Territory is not matched by a commensurate improvement in outcomes. For example, the 2011 report provides details of 15 separate initiatives that are designed to support the Closing the Gap National Partnership. Table 1 compares the outcomes for Closing the Gap National Partnership Schools (Department Education Employment and Workplace Relations, 2012) with the outcomes for all schools, based on time series data presented in the 2011 NAPLAN National Report (ACARA, 2011). Without making allowances for standard deviations and statistical error, which are not provided in the Northern Territory Smarter Schools 2011 Annual Report, it is evident that the ‘gap’ has only been closed in four of the six reporting groups shown: Year 3 Reading, Year 5 Reading, Year 7 Reading and Year 3 Numeracy. For the other two groups, the ‘gap’ widened. The picture is similar for the low SES schools, where there was an improvement in the raw scores for four of the six groups shown.
These data confirm the proposition that for very remote schools with mainly Aboriginal and Torres Strait Islander students, that the link — be it causal or not — between educational outcomes, attendance and socio-economic advantage is not strong enough to base interventions on these assumptions. This finding leads to more questions than it answers.
The first obvious question then is: ‘Why is it so?’. Importantly, the question is not why there is a gap. The answer(s) to that is (are) well-documented in the literature, albeit predominantly from a western frame of reference. Rather, the question is about why the statistical relationships that work for the whole population do not work for this group of schools. Here are a few as yet untested propositions:
• What is measured (either as attendance, outcomes or advantage) may not reflect the actual reality in remote communities.
• The culturally laden concepts associated with NAPLAN testing do not reflect the learning that is actually going on in schools.
• The individuated way of administering NAPLAN tests does not reflect the socially negotiated way that way that remote learners may tackle learning challenges.
• The expectations of ‘progress’ of young learners up the NAPLAN ‘ladder’ are unrealistic for remote communities.
• The supply side drivers of ‘improvement’ do not necessarily match the demand side motivators for ‘improvement’.
• The definitions of ‘improvement’ may be different in both the supply and demand side of remote education.
• The socio-cultural factors, language and related ontologies, epistemologies, axiologies and cosmologies present in remote communities have a greater impact on student outcomes than the corresponding philosophical positions of the supply side of education.
• Coercive (or voluntary) interventions designed to lift attendance or address disadvantage, result in resistance and therefore are not adopted by people in remote communities.
Regardless of the truth or otherwise of these propositions, something different is happening in remote Aboriginal and Torres Strait Islander communities.
The second obvious question is ‘what does this then mean?’. If the current interventions designed to reduce the ‘gap’ are not working as well as might be hoped, what would then work? What should be abandoned? Again, here are a few propositions:
• The positioning of remote students as ‘disadvantaged’ — disadvantage implies deficit which in turn needs rectifying — should be reconsidered in favour of a recognition of (and perhaps celebration of) diversity.
• Forcing parents to send their children to school via punitive instruments is not working and should be abandoned.
• While recognising that attendance measures have intrinsic values, the measure as a proxy for school performance in remote schools should be abandoned.
• Alternative measures of school performance that reflect the attributes of a truly successful remote school need to be developed.
• Definitions of success that reflect local aspirations, ontologies, epistemologies, axiologies and cosmologies should be developed.
• Redefinitions of success may in turn result in a redefinition of what it means to be a quality teacher and what it means to teach effectively.
• The premise behind the argument for a standardised Australian curriculum should to be questioned — particularly if it consistently produces ‘E’ grade students in remote communities.
• Assumptions about the outcomes of school-community partnerships should be challenged — do they really result in better learning outcomes and should their forms be prescribed as they currently are?
• The field of remote education is ripe for radical innovation — innovation that goes beyond school improvement and is transformative.
• Instruments of accountability designed to produce better outcomes for remote learners should be reconfigured to reflect true measures of success in remote schools.
None of the above should be taken to suggest that testing as such should be abandoned. Nor should they be taken to suggest that remote schools should not or could not improve. Nor should they suggest that measures of performance should be ignored. However, as noted earlier, something is markedly different about the context of remote schools which suggests that the assumptions of a good education — and how it is measured — need to be rethought.
Conclusions
The analysis of NAPLAN data presented here challenges the traditional assumptions about the importance of attendance and socio-economic advantage for students’ educational outcomes for very remote schools. It shows quite clearly that, at least for two variables (Reading and Numeracy) the expectation that outcomes will improve with improved attendance and reducing disadvantage, is flawed.
These findings should then bring into question other assumptions about the nature of ‘good schooling’ in very remote Australia. They also may help explain why it is that so little progress has been made in ‘closing the gap of educational disadvantage’. Further, they draw into question whether many initiatives (whether punitive or incentive) designed to improve attendance are really worthwhile.
It should not be concluded that NAPLAN should necessarily be abandoned. Rather, the assumptions behind the administration of testing instruments, the language used in them and the values embedded within the tests, should be questioned. As stakeholders in the education ‘system’ we should be asking whether or not we are merely counting irrelevant elements of education, or whether we are indeed using the measures of successful teaching and learning to better inform an education system that really counts in making a difference for Aboriginal and Torres Strait Islander students in remote communities. While the propositions presented here are not designed to be prescriptive — they form a foundation for a significant research program being undertaken by the Cooperative Research Centre for Remote Economic Participation — there is a strong case to be made for considering further what defines ‘success’ in remote education. This in turn could lead to changes in the way we as educators present our discourse of remote education. It may also have significant implications for the fundamentals of schooling in remote Aboriginal and Torres Strait Islander communities.
Acknowledgments
The work reported in this publication was supported by funding from the Australian Government Cooperative Research Centres Program through the Cooperative Research Centre for Remote Economic Participation (CRC-REP). The views expressed herein do not necessarily represent the views of the CRC-REP or Ninti One Limited or its participants. Errors or omissions remain with the author.
About the Author
John Guenther (Guest Co-Editor of this issue) is the Principal Research Leader for the Remote Education Systems project with the Cooperative Research Centre for Remote Economic Participation and Flinders University. John has worked as a researcher and evaluator in remote Australian contexts — particularly the Northern Territory — for the past 10 years on issues related to education, training, families and children, justice, child protection and domestic violence. His current role is focused on understanding how education systems can better respond to the needs of students and families living in very remote Aboriginal and Torres Strait Islander communities. Beyond this, the Remote Education Systems project is looking for innovative models of schooling, teaching and learning that improve livelihood opportunities for students from remote communities.
