Research into composing in the curriculum

Composing as a curricular activity has received increasing attention in recent academic research, and a number of facets of it have been investigated. Burnard has researched a variety of approaches to, and outcomes from, the composing process (Burnard, Reference BURNARD2000b, Reference BURNARD and Mcpherson2006; Burnard & Younker, Reference BURNARD and YOUNKER2002, Reference BURNARD and YOUNKER2008), including how learners make meaning from composing and improvising, and the cognitive and practical mechanisms in place whilst undertaking it (Burnard, Reference BURNARD2000a, Reference BURNARD2002). In the context of the UK National Curriculum (NC), composing often takes place as a collaborative process, and this has formed the locus of investigation for a number of studies (Burland & Davidson, Reference BURLAND and DAVIDSON2001; Fautley, Reference FAUTLEY2004, Reference FAUTLEY2005). Social interaction plays a large part in group composing, and this has also been investigated (Miell & MacDonald, Reference MIELL and MACDONALD2000; Burland & Davidson, Reference BURLAND and DAVIDSON2001; MacDonald et al., Reference MACDONALD, MIELL and MITCHELL2002; Major, Reference MAJOR2007, Reference MAJOR2008). More recently, what Activity Theory can tell us about group composing has been studied (Burnard & Younker, Reference BURNARD and YOUNKER2008). Pedagogic attitudes to composing have sometimes been an issue, and this area has been investigated too (Odam, Reference ODAM2000; Paynter, Reference PAYNTER2000; Berkley, Reference BERKLEY2001; Byrne & Sheridan, Reference BYRNE and SHERIDAN2001).

Assessment research

Assessment is a key area of interest in contemporary educational discourse. Key distinctions are drawn between summative and formative assessment. Summative assessment is where marking, grading and certification of learning takes place, usually at the conclusion of a learning episode (Harlen, Reference HARLEN2005, Reference HARLEN2007). Formative assessment, frequently referred to as assessment for learning, has been shown to play a key role in the development of learning, and has been well documented (Black, Reference BLACK1995; James, Reference JAMES1998; Black & Wiliam, Reference BLACK and WILIAM1998, Reference BLACK, WILIAM and Gardner2006; Assessment Reform Group, 1999, 2002; Black et al., Reference BLACK, HARRISON, LEE, MARSHALL and WILIAM2003a, Reference BLACK, HARRISON, LEE, MARSHALL and WILIAM2003b, Reference BLACK, HARRISON, LEE, MARSHALL and WILIAM2004).

Assessment of composing

Assessment of composing, particularly in the English situation has received less attention, however. There are some general overviews (Stephens, Reference STEPHENS and Hickey2003; MacDonald et al., Reference MACDONALD, BYRNE and CARLTON2006) and some researchers have investigated teacher practice in this area (Byrne & Sheridan, Reference BYRNE and SHERIDAN2001; Byrne et al., Reference BYRNE, MACDONALD and CARLTON2003). In their discussions concerning the composing pathways undertaken by individuals, Burnard and Younker (Reference BURNARD and YOUNKER2004) mention assessment. Mills (Reference MILLS1991) investigated the musical nature of assessment, whilst Brophy (Reference BROPHY2000) and Colwell (Reference COLWELL, Colwell and Richardson2002, Reference COLWELL and Brophy2007) have both considered a range of other issues. From a classroom perspective, Bray (Reference BRAY2000, Reference BRAY and Spruce2002) and Adams (Reference ADAMS and Philpott2000) discuss ways in which teachers can undertake assessment in the classroom, whilst Fautley (Reference FAUTLEY2008, Reference FAUTLEY, Evans and Philpott2009) has provided guidance for both serving and trainee teachers, as well as considering assessment issues more widely (Fautley, Reference FAUTLEY2010). From a more general perspective, issues concerned with assessment and the arts have been discussed (Murphy & Espeland, Reference MURPHY, ESPELAND and Bresler2007), as have the more specific issues of assessment in music education (Murphy, Reference MURPHY and Bresler2007). Kaschub and Smith, writing from the perspective of the USA, address the issue of grading composing, observing that ‘In many cases there are no compelling reasons why grades must be given in composition classes’ (Kaschub & Smith, Reference KASCHUB and SMITH2009, p. 97). This is a situation that many older music teachers in the UK will recognise from pre-NC days.

The context of schools

In England, considerable investment has been made into developing teacher proficiency in assessment, in term of both resources (DfES, 2002, 2003, 2004a, 2004b) and in-service training. Formal examination-based assessment of composing in schooling happens after the period of investigation of this current study, at Key Stage 4. Assessment of composing at KS3 does take place, as it forms part of the teacher assessment which gives an overall level grade to all pupils at the end of the key stage. It is important to note that teacher assessments of pupils in terms of the NC levels they are awarded are not subject to any form of external verification or moderation. There are number of issues associated with NC assessment practice, both from an academic perspective (Ruthven, Reference RUTHVEN1995; Wiliam, Reference WILIAM2003) and from music teachers themselves (Fowler, Reference FOWLER2008). Whilst there have been moves to facilitate a common understanding of what might constitute attainment at each specific level, previous research in this area has shown differences in understanding amongst teachers (Harlen, Reference HARLEN2005).

National curriculum levels in music

Statutory assessment of attainment in music occurs, as we have observed, using National Curriculum levels, and takes place at the end of the period of study of compulsory music lessons, normally at the age of 14. In common with all NC subjects there are eight levels, plus one for ‘exceptional performance’. The levels holistically delineate the processes of musical learning, including performing, composing and listening. It is important to note that there are not separate levels for each of these activities, rather, the levels are predicated on the conception of integrated practice. There are no assessment tests in music, the judgements for levels are decided by teachers alone, and there is no statutory requirement for external moderation or verification of an individual teacher's NC levels.

The level expected of all pupils at the end of their compulsory period of study is level 5. The level statement for which reads thus:

The composing component is contained within this level statement, but is not delineated in attainment terms, such as ‘pupils will be able to . . .’. This enables holistic judgements to be made concerning pupil progress, without having unwieldy assessments to undertake on classes of 30 pupils at a time. NC level statements can be viewed as having a constructivist perspective, rather than a behaviour-driven series of outcome statements. This has implications, as we shall see.

The original intention of the NC levels was that they should only be used once, at the end of a key stage, in this instance at age 14+, therefore they were not intended to be used for individual pieces of work. Anecdotally we understood that this was no longer the case, and teachers were required to provide schools with regular updates of all their pupils using the NC levels. We wanted to find out how much truth there was in these stories, and how the NC levels were used by teachers.

Research methodology

This research involved combining both qualitative and quantitative methodologies (Gorard & Taylor, Reference GORARD and TAYLOR2004). The assessment of composing is a complex issue, and employing a mixed methodology enabled us to grapple with the issues raised. As Creswell observes, ‘the problems addressed . . . are complex, and the use of either quantitative or qualitative approaches by themselves is inadequate to address this complexity’ (Creswell, Reference CRESWELL2009, p. 203). Another factor in the employment of a mixed methods approach was that we wished to combine quantitative responses with qualitative judgements, and were cognisant of the notion that ‘. . . research approaches should be mixed in ways that offer the best opportunities for answering important research questions’ (Johnson & Onwuegbuzie, Reference JOHNSON and ONWUEGBUZIE2004, p. 16), and that employing a mixed methodology is ‘. . . more than simply collecting and analysing both kinds of data; it also involves the use of both approaches in tandem . . .’ (Creswell, Reference CRESWELL2009, p. 4).

With these factors in mind, the research was designed to take place in three phases. The first phase was an on-line survey. The second phase was a follow-up on-line survey, which developed themes from the first survey in greater depth. Materials from both of these surveys were employed to inform the final phase, which was a series of individual semi-structured interviews with teachers. In the initial on-line survey questions were asked which involved attitudinal responses, Likert scale prioritisations, pedagogical procedural questions and free text responses. The methodological justification for this modality was that we were keen to establish an ontological perspective with regard to what was taking place with regard to assessment of composing, before moving on to explore reasons why this was the case in the survey phase, what Punch (Reference PUNCH2009, p. 25) refers to as ‘question-method connections’. Bearing in mind the observation that ‘Rating scales . . . are limited in their usefulness to researchers by their fixity of response caused by the need to select from a given choice’ (Cohen et al., Reference COHEN, MANION and MORRISON2007, p. 328), we wanted to explore using semi-structured interviews the reasons for answers to the surveys.

The initial on-line survey was made available to all secondary school music teachers in England, and was widely advertised in subject association material, and by direct e-mail contact. The survey received a considerable number of views (n = 866) and was started by 176 respondents, of whom 94 went on to complete it. There was a range of respondent contexts, from rural to inner city. In this initial survey, respondents were asked if they would be prepared to be involved in a more in-depth follow-up survey, also to be undertaken on-line. This considerably more complex survey received 339 initial views, was started by 34 respondents, and completed by 18. In reporting the data from the various stages of the research, we have included in the analysis details as to the phase of the research the data in question arose from, and the number of respondents. Although not all respondents completed the surveys, the on-line survey tool (QuestionPro) allowed completed questions to be analysed, even if the survey itself was unfinished. On completion of the second questionnaire, respondents were asked if they would be prepared to be interviewed, and 11 teachers were selected for this. Geographically, the spread of teachers was from the North-East of England, via the North-West and the Midlands, to London and the South East. All but one of the interviews was conducted on a face-to-face basis, the other being done by telephone. Interviews were semi-structured, based on a pre-determined interview schedule, which allowed the possibility for supplementary questioning to take place (Cohen et al., Reference COHEN, MANION and MORRISON2007; Denscombe, Reference DENSCOMBE2007). In reporting speech from teachers, we have transcribed directly what they said, and used conventionally represented punctuation to aid meaning (Marshall & Rossman, Reference MARSHALL and ROSSMAN2006).

Data analysis was undertaken in a number of ways. Qualitative analysis of free text responses was coded using a grounded theory approach (Glaser & Strauss, Reference GLASER and STRAUSS1967) where codings for the data arise from analysis thereof. This was undertaken as an iterative process, with increasingly fine-scaled unique codings arising as a result, in a developed form of axial coding, followed by coding for process (Strauss & Corbin, Reference STRAUSS and CORBIN1990, p. 163). We assumed items to be significant when mentioned by numbers of respondents, and repeatedly revisited the free-text responses to ensure we were capturing the importance of what was being said.

Quantitative data were analysed in a number of ways. Nominal scales of denotation were employed in order to establish categories of responses, such as composing pedagogies. Likert scales for attitudinal responses were used, and had inferences drawn from them. Ordinal scales were also employed to find how much use was made of assessment techniques. In undertaking this analysis we were mindful of Wright's assertion that ‘. . . the level of measurement is not an inherent characteristic of a particular variable, but a characteristic that we, as researchers, bestow on it based on our theories of that variable. It is a belief we hold about the variable’ (Wright, Reference WRIGHT2003, p. 127).

For the purposes of our research we did not qualify or define the term ‘classroom composing’. We used this consistently, and respondents and interviewees did not question this usage.

NC levels

The results shown in Table 1 clearly demonstrate that NC levels were the main modality for teacher assessment at KS3, with 49% of respondents mentioning them directly. The text from which this analysis is derived reveals that, for some teachers, use of the NC levels was felt to be sufficient in and of itself, with short answers such as these being not atypical: ‘Use NC Level descriptors’; ‘National Curriculum criteria’; ‘KS3 levels’; ‘NC orders’. This seems to show that in teacher thinking there exists an unproblematic connection between NC levels and assessment of composing.

From a methodological perspective, an issue here is that our analysis of teacher responses only counted when teachers mentioned NC levels directly in their answers. Reading the text of respondents it is possible that what teachers were doing was explaining how they use NC levels, but not mentioning them. For example, none of these three (different) teacher respondents mention NC levels directly:

To investigate this in more detail, a later question in the first survey asked to what extent teachers used NC levels to make judgements about their pupils. One hundred and four teachers answered this question. Despite the original intentionality concerning the use of NC levels as outlined above, 25% of teachers responded that they use levels to assess individual pieces of work. Only about 9% of teachers use levels in the way which they were originally intended, to report at the end of the Key Stage, with 49% of teachers giving levels to all pupils at least once per term, with about 15% using them on an annual basis in order to review progress. This is shown in Table 2.

Table 2

Frequency of NC level use

What this means is that some 74% of teachers are employing NC levels at least once a term. This frequency of utilisation is an issue for music teachers. We then asked whether teachers were assessing frequently because they had to, or because they wanted to. Again, 104 teachers answered this question. Results from this are shown in Table 3.

Table 3

Reasons for frequency of NC level use

Table 4

Question: Do you find the National Curriculum levels helpful?

Only about 11% of respondents answered that they assess with this frequency because they wanted to; 36% of respondents were using the NC levels this often because the school required them to. However, 45% of teachers were using the NC levels as a way of monitoring progress, using them for benchmarking so they could track progress over time, and it is to this use of the NC levels that we now turn.

Using levels to show progression

In order to develop our understanding of the ways in which teachers were using the NC levels to show progression in composing, in interviews we asked teachers about this. The NC levels, as was noted above, were not intended to show differentiated development of key processes in music, so in many ways this can be considered to be an artificial question. However, it was designed to uncover teacher thinking in this area.

Answers were broad and varied. For some teachers the levels were not used in this way:

One teacher had clearly considered the intended nature of the NC levels, and observed that they were not meant to be stand-alone assessment criteria, and so this was not possible:

Some teachers used the NC levels for internal recording purposes, but did not share them with the pupils:

A number of teachers had broken down the level statements from the NC, and used these to show progression:

In terms of deconstructing level statements, many teachers talked about the school requiring them to break down the existing level statements into sub-levels, often three. This breaking down is not an officially sanctioned activity, and sub-levels only exist where individual teachers or music departments have re-written them. What this means is that each set of school-specific sub-levels is unique. For example, some schools in our research sub-divided levels into three, in other schools we found sub-division into tenths of a level. The net result of this on a nationwide basis is that lots of different systems are operating simultaneously, with limited or no transferability.

Related to sub-levelling, one teacher we interviewed was having trouble with a school-imposed linear improvement requirement:

Another teacher was also facing similar issues:

These two teachers, and many others besides, talked of how attainment could only be upwards, and only in a linear fashion, ‘every child should increase by two sub levels per year’, the teacher above noted. One teacher, ‘off the record’, talked of how he had written a spreadsheet routine that did this automatically at the touch of a button! Another teacher was even more disconnected from the process:

Fautley writes of how assessment in music education is geared to the requirements of three groups of users, teachers, pupils and systems, ‘. . . there is a clear difference between assessment for classroom purposes which helps with music-making, and assessment for auditing purposes’ (Fautley, Reference FAUTLEY2010, p. 69). Many teachers here spoke of assessment being undertaken to fulfil a systemic need for audit, rather than a learning one.

Do teachers find NC levels helpful?

We then asked the simple and stark question: Do you find the National Curriculum levels helpful?

Despite their having been in existence since 2000, and many training courses having been run concerning their use, 30% of teachers reported that they found them unhelpful. Clearly this means that 70% of teachers do find them helpful, but it does seem that the use of NC levels raises a number of questions, not least concerning that of ways of meeting the concerns of this significant minority of teachers.

Translation of levels into ‘pupil-speak’

From answers to survey questions, and from anecdotal evidence amongst teachers, we wondered how prevalent the practice of re-writing the wording of the published NC levels into pupil-friendly vocabulary might be. To investigate this we asked interviewees whether they did this. There was a range of responses, from the definite and unequivocal, to those who had not and would not.

This teacher was very definite:

Whereas this teacher was having second thoughts about using the ones he had rewritten:

This teacher had done so to a limited extent, but was concerned about the effect that doing this might have:

For some respondents, the prevailing assessment culture in schools meant that teachers found themselves having to do this in order to meet pupil expectations of what was required:

Criterion referencing

Rewriting level statements into pupil-speak takes us into the territory of criterion referencing. In free-text responses which did not mention NC levels directly, what many teachers did discuss was criterion referencing. Taking this as a unit for analysis, Table 1 showed that 44% of respondents mentioned criterion-referencing of some sort, even if they did not employ the terminology directly. This seems to play an important part in the way teachers think about KS3 assessment.

We were keen to find about the sorts of criteria which were being employed, and so in interviews we asked teachers about this, and found that many of the criteria used were the rewritten pupil-speak NC levels discussed above. This point was made by one teacher:

The point made by this teacher is key. The rewritten NC level statements may well be criteria, but they are not the NC levels. Stand-alone criteria of this sort derived from NC levels are possibly a helpful way for teachers to address key aspects of teaching and learning, but as grading criteria they may be problematic. This is because a fundamental issue here is that many of these rewritten pupil-speak criteria were transposed into what might be termed behavioural objectives, such as ‘can use an ostinato’ or ‘plays in time’. This is problematic, as examples of assessment criteria provided seldom addressed issues of composing-ness. The shift from constructivist levels to their behavioural outworking in classrooms is clearly an issue of more than philosophical concern. This is a point which has been raised by Ofsted:

What our investigation also reveals is that there is a question as to whether there might be a conflation in teachers’ minds between criterion assessment and NC levels. The free-text responses indicate that for a number of teachers, the NC levels themselves are unproblematic examples of criterion-referenced assessments. This runs counter to observation of NC levels generally, that ‘. . . the level descriptions contain, in themselves, collections of varied attainments that have no necessary unity or coherence. It might be argued that this is a collection of descriptions, not of linked performances’ (Sainsbury & Sizmur, Reference SAINSBURY and SIZMUR1998, p. 190).

In criterion-referenced assessment we would normally expect a construct to be linked to an assessment criterion. What we found was that teachers were moving readily between atomistic task criteria based outcome assessment and the NC level statements. In one instance, the assessment criterion observed was: ‘can play the keyboard melody with more than one finger’. This was equated with performance at NC level 4, the text of which is:

There is no mention here of keyboard playing, nor of how keyboard playing equates to composing. In the school in question, the outworking of the composing process was enacted in assessment terms by a performance-related criterion, which in itself has marginal relevance to the NC level statement to which it is linked in the teacher's assessment. This has implications for the validity and reliability of NC assessments nationally.

Summative assessment

Summative assessment was the assessment modality which was mentioned least by respondents. When it was, the way summative assessment was described was that for many respondents a composing unit would end with an assessment lesson. In the assessment lesson pupils performed their composed pieces to the rest of the class, and the teacher graded it. Criteria for this grading remained unclear. Typical comments include, from different teachers:

Interestingly what happens in many cases is that these assessment lessons seem to assess performance of the finished composition, rather than of the process of composing itself. The resultant grade seems to be based on performance of the composition, rather than the process of composing. We noted that the process of composing was not normally subject to summative assessment, but rather to formative assessment, and so it is to that we now turn.

Formative assessment

Table 1 showed formative assessment was only mentioned by 16% of respondents. In analysing responses involving it, many teachers did not feel the need to expand on what formative assessment involved. Responses such as this were unusual in that they provide examples of formative assessment in action:

The formative assessment of composing was more often described as ‘informal’ and ‘regular’, but we need more research to find out what is really going on in the formative assessment of composing. A number of teachers described things which could more properly be called the formative use of summative assessment (Black et al., Reference BLACK, HARRISON, LEE, MARSHALL and WILIAM2003a), rather than formative assessment per se. Our suspicion, therefore, for which more research is needed, is that what many music teachers are calling formative assessment is in fact the formative use of summative assessment, and this, whilst useful, is not true formative assessment.