Context
This paper reports on the outcomes of a study undertaken at a university-based Conservatorium of Music in Australia. The study was funded by the host university with the intention of examining assessment processes in music ensembles within its Bachelor of Music programme. The stimulus for the project stemmed from discussions at the institution concerning equity, transparency, lack of feedback to students about their progress within ensembles and perceived misunderstandings about assessment tasks. Four representative music ensembles were selected and the rationale and processes for their assessment was investigated. The project is innovative in learning and teaching in the conservatorium context: while ensembles have constituted a major component of learning at this site, the use of peer, self and/or group assessment has been restricted in its application beyond courses in the institution in that incorporate jazz, music technology and popular idioms. This project sought to:
(1) establish an evidence base to inform ensemble assessment decisions;
(2) develop clearer, more transparent processes for the assessment of ensembles within the Conservatorium thereby improving the provision of feedback to students.
The provision of an evidence base was intended to unpack the processes directors used to arrive at their marks, and the rationale for their decision making. The funding was provided to the Conservatorium due to a perceived lack of transparency in the assessment process, and the absence of constructive feedback.
Background
Functions and dimensions of assessment
Eisner (Reference Eisner1998) refers to five functions of assessment. He terms the first as a ‘temperature taking’ (p. 139) or diagnostic function in which an initial assessment about capabilities is made. The second is a gate-keeping one in which the results of testing determine entry to courses and institutions, somewhat similar to audition processes in tertiary music. The third is the determination of whether or not course objectives have been attained and therefore relates, in part, to the content of this study. The fourth and fifth, respectively, are the functions of helping a teacher (and student) become more reflective about his or her own performance, and the assessment of programme quality. These last two are activities that ‘contribute to the enhancement of practice’ (p. 135). The last three of these functions underpin this project as it pertains to ensemble directors and students considering their practices in the light of course objectives. As such, the ensemble becomes a site for cooperative learning and display of performance-based outcomes.
In a similar vein, Lebler (Reference Lebler2008) lists three dimensions of assessment and asserts that a combination of all three will produce optimal learning benefits for students:
Assessment can be broadly grouped into three types. Assessment of learning occurs when a student's understanding of curriculum content is measured and this is the traditional role of assessment. Assessment for learningFootnote 1 occurs when the goal is to identify areas in which more work may be needed. Assessment as learning involves students in the act of assessment as active participants and this involvement is intended to produce learning in itself (p. 194).
In the context of this study, the intention was to assist students and directors in moving towards assessment as learning.
Ensembles as a site for learning and feedback
Hitchcock (Reference Hitchcock2008) suggests that the complexity inherent in ensemble-based multi-various practices more closely represents the complexity found in professional creative work places. Such artistic practice by its nature is a collaborative endeavour, requiring intricate and multiple forms of communication and feedback (Hitchcock, Reference Hitchcock2009). This form of feedback produces a shared ability to shape social and community structures that more closely resemble the sorts of passion-based and intrinsically motivated interactions found in professional communities of practice. Eisner (Reference Eisner1998) comments that what is important in life requires group effort, and suggests that curricula and assessment procedures must be redesigned to gauge both the success of the group effort, and the contribution of the individual towards that effort.
Therefore, a developmental ensemble culture can play an important role in relation to students ‘being’ or ‘becoming’ someone within the group (Brown, Reference Brown2006), where ‘mastering a field of knowledge involves not only “learning about” the subject matter but also “learning to be” a full participant in the field’ (Brown & Adler, Reference Brown and Adler2008, p. 4). In this context, learning to ‘be’ is about knowing how to learn, negotiate and appropriate the ‘ways’ of different professions (Wenger et al., Reference Wenger, McDermott and Snyder2002), including musical performance. Consequently, attention to student–student and staff–student interactions, particularly in relation to complex and multiple forms of feedback between all participants, should be seen as integral to the formation and growth of any ensemble where learning is considered one of the primary goals, as it is in the tertiary music setting. As such, the project's design interrogated the staff–student and student–student interactions as a way of ensuring feedback assisted students in becoming adept at creating a community of practice within the ensemble as ‘here and now’ experience and as preparation for the profession.
Assessment in the ensemble setting
Although there is an extensive body of research on the benefits of cooperative learning, including in higher education (Johnson et al., Reference Johnson, Johnson and Smith1991; Kagan, Reference Kagan1992; Millis & Cottell, Reference Millis and Cottell1998), little research has been undertaken into the unique nature of ensemble assessment in the tertiary musical context in Australia. The centrality of assessment (both formative and summative) to the learning process is apparent in the work of Boud et al. (Reference Boud, Cohen and Sampson1999): ‘Assessment is the single most powerful influence on learning in formal courses and, if not designed well, can easily undermine the positive features of an important strategy in the repertoire of teaching and learning approaches’ (p. 413).
Prosser and Trigwell (Reference Prosser and Trigwell1999) and Struyven et al. (Reference Struyven, Dochy and Janssens2005) also agree that assessment practices influence the nature of student learning. In relation to music and the other arts, Elliott (Reference Elliott1987) commented that: ‘. . . there have been unique problems relating to questions of assessment. In subjects which are practically based and which by their very nature are designed to provoke divergent responses, there has been considerable argument over appropriate methods of assessment’ (p. 92).
Developing this theme of possible incommensurability further, Swanwick (Reference Swanwick1998) reflected that in music:
Our most significant learning experiences are often tacit, unspeakable, in-communicable, out of reach of formal testing and examinations. But this is not to say that musical knowledge is an absolute mystery and that teachers can have nothing helpful to say about what students are doing . . . Teachers are not only accountable to institutions, to parents, to their employers: they are accountable above all to their students. (p. 2)
As teachers of music tend to teach in the way they have been taught (Karlsson et al., Reference Karlsson, Juslin and Olsson2008) there is a danger that stagnation of learning, teaching and assessment practices can be generationally perpetuated. Curruthers (Reference Carruthers2008) proposes that university performance programmes still intend a kind of learning that has long been dismissed as limiting, restrictive and ultimately inappropriate for public school curricula. While universities have been slow to catch up with their school colleagues in this regard, Lebler (Reference Lebler2007) suggests that a way forward is to consider that ‘teaching practices that have dominated in the past will need to be rethought, and alternatives considered that are likely to produce graduates with the abilities and attributes necessary to adapt readily to a rapidly changing environment’ (p. 206).
A balance between formative and summative assessment is central to the formation and development of the music ensemble in higher education contexts. Scriven's (Reference Scriven, Tyler, Gagné and Scriven1967) definition of these provides some illumination for this context: Formative assessment activities are used to provide feedback and evaluate learning progress in order to motivate students to higher levels, while summative assessment activities are used to judge final products for completion, competency and/or demonstrated improvement. The latter, with the exception of ‘improvement’, would appear to align with institutional imperatives, while the former emphasizes more authentic, profession-related industry skills. Improvement would, of course, be expected as an outcome of undertaking a degree programme. However, with the exception of the jazz ensemble, these are multi-year-level groups and the progression of the individual within the larger ensemble is difficult to assess.
More recently, Denson and Nulty (Reference Denson and Nulty2008) claim that all the activities students undertake in ensembles need to be authentic to the profession of being a musician: ‘The tasks students undertake do not simply mimic what a musician does, they are what a musician does’ (p. 4). According to Denson and Nulty, there is no possibility that students will have difficulty with the relevance of these musical learning activities. Conversely, Corkhill (Reference Corkhill2005) found that senior conservatoire students demonstrated ‘unpreparedness and a lack of awareness of the challenges and difficulties of being an orchestral musician’ (p. 270). Pontious (Reference Pontious2001) contends that teaching and assessment focus should be shifted to mastery of the standards so students can focus less on the completion of grades (summative) and more on intrinsic motivation and achievement of music (formative). This relates strongly to Eisner's (Reference Eisner1998) notion that assessment tasks resemble the challenges of ordinary living, requiring an ‘entirely different framework of reference’ for their construction (p. 141).
Suggestions for addressing this dilemma can be found in the recent work of Orr (Reference Orr2010) who interrogated the nature of group learning and assessment in the performing arts, noting: ‘Group work is central to pedagogy in the performing arts. It is the norm, not the exception’ (p. 302). While Orr's study focused on dance, theatre and film, her process of eliciting the views of students and lecturers in relation to the assessment process and the concept of fairness has much to offer the current project.
McCoy (Reference McCoy1991) notes that, while suggestions for grading systems for performing arts groups have been put forward by several authors (Boyle & Radocy, Reference Boyle and Radocy1987; Branum et al., Reference Branum, Fusco, Haag, Richmond and Russo1988; McPherson & Thompson, Reference McPherson and Thompson1998), little research has been conducted to examine the actual grading practices of ensemble directors. Barratt and Moore (Reference Barratt and Moore2005) maintain, ‘a balance of musical and extra musical factors may be justified on the premise that student contributions to an ensemble are integral to its overall success, and therefore require regular, active participation’ (p. 14). Lehman (Reference Lehman1997) adds that ‘the only justifiable basis for grades is student progress in learning specific skills and knowledge in music, as outlined in applicable standards and made explicit in a written curriculum guide’ (p. 58). More recently, Wrigley (Reference Wrigley2005) measured a range of experience variables involving solo and ensemble performance. He found that students underestimated their ratings in performance and suggested that it was imperative to improve accountability and so improve teaching and learning outcomes in tertiary music education. Lebler (Reference Lebler2010) has found that involving students as assessors in a peer assessment process has benefits including enhancement of students’
abilities to conduct systematic assessment of music and also their ability to provide feedback in positive ways even when they might be drawing attention to flaws. . . . The assessing of peers can enhance not only content-related learning and the ability of students to conduct assessments of other people, but can also produce improved self-reflection skills resulting in increased confidence and better awareness of the quality of the students’ own work. (p. 71)
Similarly, Daniel (Reference Daniel2004) and Blom and Poole (Reference Blom and Poole2004) have referred to the efficacy of peer learning (and assessment) in the Australian tertiary setting. It should be noted, however, that this writing largely pertains to the instrumental tuition in one-to-one and small group settings, rather than the ensemble settings under investigation here.
The preceding summary of literature on assessment provides clear evidence of the importance of assessment as a key driver of student learning, the characteristics of that learning in addition to shaping the content. If assessment is to have a positive influence on learning in ensemble settings, students must have confidence in its efficacy; as is demonstrated below, this was not the case for ensemble studies in this Conservatorium.
Setting and participants
The site of this research was established as an independent Conservatorium in 1957, and achieved autonomous status as a College of Advanced Education in 1971. As part of a national move to amalgamate higher education institutions, it became a College of its host university in 1991. Bachelor of Music (including jazz and classical streams), Bachelor of Popular Music and Bachelor of Music Technology programmes are currently on offer at the undergraduate level. This project focuses specifically on ensemble activities in the Bachelor of Music programme across a range of music styles and genres, including classical, jazz and contemporary ensembles.
The Bachelor of Music programme is the largest of the programmes offered at the Conservatorium. It aims to produce graduates who are highly skilled, musically adaptable and equipped to enter professional life as creative and flexible 21st century musicians. For the approximately 400 students enrolled in the programme, high-quality focused performance training is provided for the classical and contemporary instrumentalist, vocalist or composer along with exposure to diverse musical styles and idioms. The programme is designed to develop a clear sense of direction and a repertoire of relevant professional skills built up through practical experience and rigorous workplace training.
A vital part of this practical experience is participation in instrumental or vocal ensemble studies. This is embedded in the programme to assist in preparing students for the rigours and demands of the music profession and, at the time of the inquiry, formed up to 25% of students’ course activities in each semester. There are a total of 22 large, medium and small ensembles2 ranging from choral groups to a single type or family of instruments and, while all enrolled students are offered the opportunity to participate, for the majority, enrolment in at least one of these ensembles per semester is compulsory.
The ensemble courses offer unique opportunities for students to develop musical ensemble techniques and interpersonal and organisational skills, while familiarising them with the existing and evolving repertoire. They also provide an ideal environment for students to initiate musical partnerships that may continue into their professional careers. Students are required to adapt to the expectations made of professional ensemble members, including attendance and individual preparation before each rehearsal period.
At the time of the study, assessment tasks varied from ensemble to ensemble. Core elements included participation in rehearsals and performances, a self-assessment reflective essay of 500 words for some ensembles, and, for the large instrumental ensembles, an individual, screened, mock audition judged by a small panel of experts. The director of each ensemble evaluated student performances and rehearsals, while the self-assessed reflective essay was assessed by the convenors of the ensemble courses for which the self-assessment activity was required. There was no peer assessment requirement in most ensembles. Assessment task percentages between ensemble courses varied considerably. Rehearsal and performance marks varied between ensemble courses, depending on the music genre or ensemble size. Thus, there was considerable disparity between ensemble assessment tasks and criteria in a number of the courses on offer.
Methods
Collective, illuminative case-study methods were used to evaluate ensemble interactions. This method was deemed to be well suited to the project because it drew on many of the representative participant perspectives within each of the ensembles chosen for the project. The framework for this approach can be found in the work of Stake (Reference Stake1995) where a case study catches the complexity and particularity in order to understand it. Data were generated through three means: rehearsal observations, artefact analysis and semi-structured interviews.
Directors of four ensembles (Symphony Orchestra, Big Band, Saxophone Ensemble and a first year Jazz Ensemble) were invited, via email, to take part in the study, and active informed consent was obtained from all interviewees. Three males and one female, ranging in age from 37–65 participated. Ensemble directors took part in individual, semi-structured interviews lasting between 30 and 45 minutes, to elicit their perceptions about ensemble practices and assessment protocols. Questions fell into the following broad themes: what do you know of the assessment requirements for your ensemble; how do you assess your ensemble; how do you run your ensemble; what are the enduring outcomes you see as important for your ensemble members; what do you think of various assessment practices available to you in this institution?
Hour-long observations of each of the ensemble rehearsals were video-recorded and field notes taken to illuminate the practices and processes of each ensemble director's approaches to rehearsal. These observations were used to enrich the analysis of the interview data.
Semi-structured 15–20-minute group interviews were held with four to six students from each ensemble. Students from the ensembles were recruited through a blanket email sent to each member of the ensemble, inviting them to participate in the project. Students then volunteered to be interviewed and each interview was recorded. Students have been de-identified. There were equal numbers of male and female students taking part, ranging from first year to final year, and interviews were held after their ensemble rehearsal had taken place. Questions asked were intended to elicit perceptions of their ensemble and assessment practices of the ensemble, and fell into the following broad themes: what do you know of the assessment requirements for the ensemble; how is the ensemble assessed; how is the ensemble run; what are the enduring outcomes you see as important for ensemble members; what do you think of various assessment practices in this institution, including feedback and peer assessment?
In addition to observations and interviews, course outlines and assessment requirements available on the university website were analysed to provide triangulation with the perceptions of staff and students. Using a standard approach to the analysis of documents, the research focused primarily on ‘what is contained within them’ (Prior, Reference Prior2003, p. 23). Through content analysis typically applied to policies of an organisation (Krippendorf, Reference Krippendorf2004) words and phrases were identified to determine what was stated explicitly as well as what was not present.
A thematic analysis of ensemble directors’ and ensemble participants’ perceptions was undertaken, largely based on the interviews, but also using data gathered from the observations, and using the artefacts from course guides. Using an inductive approach (Thomas, Reference Thomas2006), emergent themes were documented and sorted into similar categories for each cohort (conductors and students). Codes were assigned to each conductor (JR, MR, TT, TG) and for each student focus group (SFG 1,2,3,4). The relative incidence of each theme was considered prior to presentation in the discussion below.
Findings
Findings were organised from two perspectives: Directors’ perceptions of learning, teaching and assessing ensembles and students’ perceptions of learning, teaching and assessing ensembles. While the findings generally related to assessment, grading and feedback practices within the institution, participants also commented on related topics of being and becoming a musician and on the culture of the profession.
Assessment and grading practices
The awarding of grades was the subject of considerable discussion in the interviews with the ensemble directors. One director considered the process as contractual:
I give them a lot and I expect them to give me a lot. It's a contract: I give them a good grade because they're giving me a lot, and working hard for me, and playing to the best of their ability. They don't have to be emotionally enthusiastic but they have to be committed. (JR, Interviewed 6.6.10)
One director commented that ticking institutional compliance boxes was a way of meeting a top-down imposed demand. The director conceived of his role, rather, as ‘leading students to the music’. He commented:
I don't think about grading them so much, I think about my responsibility to them and theirs to me. You try to make them all responsible to the music. They have to learn about the tradition of the music. You always learn the dance, whatever's going on in the school, you learn the dance and you learn how to get around things and make things work in spite of the constraints of academia. (JR, 6.6.10)
Some directors questioned the need to give students an average mark when they were dedicated, elite musicians and noted ways to overcome this:
I cannot make a bell curve out of the ensemble. I don't think the university expects it but it just kind of looks funny when you hand in all the grades and look at all those high distinctions and distinctions. I think all my sections leaders should have high distinctions . . . I rarely give them just a distinction. Some people really have made some extraordinary progress over the years. And I think that needs to be rewarded as well. (MR, 4.6.10)
Given the institutional emphasis on summative assessment at the time of the project (the university assessment policy is currently under revision), students and staff involved in formative assessment practices may have seen themselves as marginalised. In addition, the institution does not ask for a bell-curve but rather that grades awarded are defensible and comparable across cohorts and years. However, this project was funded through a university-wide scheme and as such, troubling and critiquing of established knowledge and conventional practices within the academy would seem a worthwhile and institutionally supported goal.
Individual marking within group assessment
The difficulty of marking an individual within a large group setting was also identified as an issue. As one (perhaps conscience-constrained) director explained:
For the majority of them they take pride in what they are doing already, it will be hard to assess them individually and give them a mark as they are all so committed already. Everyone's working, trying their best, producing a good product, how can I individually assess for that? (MR, 4.6.10)
Students were vocal about perceptions of favouritism by directors towards particular students in the ensembles and the impact of this on overall marks. It was not uncommon for students to perceive unfairness in the assessment processes of some directors, commonly in favour of section leaders and soloists:
Assessment has been problematic in the past; in the case of personal preference it really affects the marks of some players. The ensemble director likes particular players and they get much better marks than the others. (SFG 2, 1.6.10)
In addition, some students questioned the ability of an ensemble director to clearly identify and separate out the individual contributions of each student in an ensemble, and to therefore be able to appropriately assign a justifiable mark. One section leader typified these concerns:
I'm a principal player this year, so obviously you can hear what I'm doing, but I think it's probably worse for stringed instruments. I often wonder how you actually can hear what they're doing. I think the subject should be made so that everyone's capable of getting the highest possible mark, but I don't know how everyone is capable of getting the highest possible mark if the person marking you isn't really aware of how you're really playing. (SFG 3, 1.6.10)
Directors and students both perceived potential issues in a proposed student peer-assessment. One director noted:
I reckon it would have worked 20 years ago, but these days a student's questions are ‘what do I have to do to pass’? I wonder if they would feel comfortable leaving it up to a peer. (TG, 3.6.10)
They further commented:
I think it actually depends on the work ethic and mentality of the group at the time, and if you have a good group they could be quite tough on each other, certainly even tougher on the person that wasn't coping as well. (TG, 3.6.10)
Students also noted issues with peer-assessment, mainly concerned with personal discrimination and fairness of the process:
It may cause a bit of tension between the students. What if you didn't like someone – you might give them a bad mark, and vice versa? (SFG 3, 1.6.10)
Another commented:
I wouldn't rubbish someone in the band – they might know me! (SFG 1, 1.6.10)
Despite concern shown by some, other students welcomed the opportunity to assess their peers:
Everyone cares what one's peers think. The construction of the assessment would have to be very clear, not naff like the self-assessment is now! The principal players need to be assessed as well. (SFG 3, 1.6.10)
As one director asserted:
It's not formal, but peer reputation occurs outside the rehearsal room. Peer dynamics will help determine attitudes about groups. (JR, 6.6.10)
Communication and feedback
Many students were concerned about feedback for formative assessment items:
Do we get feedback? . . . I didn't get any feedback . . . we just get the mark at the end of semester. (SFG 3, 1.6.10)
This was particularly evident in the mock auditions conducted as part of the ensemble activity. Students appreciated the opportunity to participate in these activities, but disliked the perceived lack of feedback, inadequate planning, and lack of transparency in the process. One student commented that:
I think they're a good idea, I like doing them but we don't even get feedback from the mock auditions, we don't receive the forms that are filled out at our auditions and we also don't get marks back. (SFG 3, 1.6.10)
Students felt that the lack of planning potentially diminished the quality of the student experience:
They are very poorly organised, I mean I'm playing excerpts that we played in first year. There's a book in the library that has been there about 10 years and the excerpts come from that. There are about five the panel can choose from, but the same ones are chosen every year. I think the principal teachers should choose the excerpts. (SFG 2, 1.6.10)
Similarly, students commented on a lack of feedback and procedural clarity in relation to the written reflection portion of the assessment:
One student asked for her assignment back in first year and there were no comments on it apart from a mark. She asked for feedback. They said they'll get back to her but never did. (SFG 2, 1.6.10)
One student summarised students’ perceptions of the feedback process for the written assignment:
You actually have to collect it if you want to see how they marked you. You have to seek out the assignment, which I find is a little bit pointless, if you've marked yourself, and them some ensemble administrator has either verified that mark or has said something different. (SGF 2, 1.6.10)
This confusion and frustration about feedback was echoed in comments made by some directors about communication and transparency within the institution generally. Some directors felt that there is not enough transparency within the institution in relation to compliance with institutional imperatives. Planning, in some instances was perceived as ad hoc and informal, as noted by one director:
Planning for assessment is not as formal as some would like . . . a couple of chats over the course of the year. We aren't always consulted in the finalising of the feedback, that there is no formal feedback, or actually, ensuring that everyone is clear on what's occurring. (TT, 5.6.10)
I think making the cycle known a little bit more, whether it be the review process, formulation of course reviews for ensemble, or any other subject, making it a little bit more transparent and at the foreground of people's knowledge. (TT, 5.6.10)
Some confusion as to who assessed which aspects of large ensembles was also apparent. The self-assessment task proved particularly problematic for casual staff. While there was no actual dismissal of the written assignment, the lack of clarity about this item was evident in comments such as:
The written assessments go to convenor. I've never seen one in my entire life. I wouldn't know what one looks like. (TT, 5.6.10)
Students seemed on the whole motivated to participate in ensembles. In the main, directors felt that the students were intrinsically motivated to play in the groups and that this outweighed perceived benefits from marks. Most directors therefore welcomed the notion of a pass/fail assessment for large ensemble. One director noted:
If it were pass or fail it would be easier. When I went through it was pass or fail and it didn't add to your GPA, just a requirement. (SFG 1, 1.6.10)
Another director claimed of pass/fail assessment:
It wouldn't make much difference. It wouldn't change the product. It wouldn't change their commitment to the product. They're not in it for a grade. They're in it to be part of it and it's my responsibility to make them feel privileged to be in that band. And other student bodies have created that privilege. They've created the reputation, other bands, so now I'm maintaining that and trying to keep it where it is. (JR, 6.6.10)
What became apparent from the interviews and focus groups was the confusion surrounding assessment processes. A lack of alignment between policy and practice emerged, as well as considerable variation between ensembles in the enactment of assessment processes. In addition, a sense of subversion of assessment activities in order to produce performance outcomes was evident in some cases.
Discussion
All participants in the study welcomed the opportunity to have a voice in relation to learning, teaching and assessment of large ensembles within the Conservatorium. A particularly strong theme to emerge from the study was the need for greater clarity and transparency regarding ensemble learning, with a particular emphasis on practices and procedures surrounding assessment. Disturbingly, both fulltime and casual staff were unfamiliar with curriculum documents and procedures relating to the course profile, assessment weightings within courses; and processes for providing feedback to students. Perhaps unsurprisingly, different assessment processes for different ensembles contributed to the confusion. Further, and as noted above, while casual staff welcomed the opportunity to be consulted about learning processes through the study, there was a common perception that they were largely ignored when it came to decision making about course content and assessment procedures, which is in part related to the increasing casualisation of the workforce in higher education (Robinson, Reference Robinson2005). The effect was reduced ownership of the assessment by casual staff, and a lack of confidence in describing some assessment items to their students.
Both casual and full-time staff found ways of subverting the system to meet their own needs as educators and musicians. For some, the core business of educating professional musicians remained their focus. In these cases, assessment was about ‘dancing the dance’ and ‘ticking the boxes’ to meet the compliance requirements of the institution. For others, adding extra-curricular, non-assessed collaborative activities to the ensemble tasks was their way of providing educational learning opportunities for students that extended beyond the narrowness of the assessment tasks. These activities included the marketing and presentation of concerts outside the institution, and the development of management and organisational skills beyond pure playing activity. This subversion is emblematic of the current perception of Conservatoria amongst disenfranchised and frustrated musicians and academics, many of whom see the institution as either irrelevant to the current crop of media-savvy students (see Schippers, Reference Schippers and Musumeci2004), or conversely, disrupting hundreds of years of conservatorium tradition through the addition of courses in popular music or business. For ensemble directors, at any rate, the perception of powerlessness and disenfranchisement impacted differently on each, producing a variety of fundamentally individualistic and highly non-conformist approaches to ensemble assessment, each director committed to the principle of education of the musicians in their charge by whatever means necessary. Nevertheless, a perception among staff was that by denying the staff a legitimate voice and input into the creation of their courses, the institution loses the opportunity to foster meaningful learning experiences for its students, informed by those who, arguably, have the most experience in the music profession: the teachers.
There was a great deal of confusion amongst students about the purpose of ensembles and particularly the way in which they were assessed. In relation to specific assessment items, written self-assessment tasks were not seen as a valid way of assessing ensemble involvement, though mock auditions were regarded highly by students who participated in them. This would appear to corroborate the claim earlier made by Hitchcock (Reference Hitchcock2008) and Denson and Nulty (Reference Denson and Nulty2008) that ensemble-based practices can mirror those of the profession. However, the level of feedback on the mock auditions and ensemble activities failed, in the students’ eyes, to reveal what they had achieved and what they could do to improve in future, consequently limiting learning outcomes.
Overwhelmingly, both students and staff noted a need to be clearer about assessment processes and practices in ensembles and the need to provide timely feedback to students about their progress. This would appear to align with the views of Pitts (Reference Pitts2005, p. 218) who notes ‘the value of feedback as a communicative learning tool’.
Ultimately, all participants preferred an emphasis on formative assessment leading to a high-quality performance outcome. While staff and students espoused the desirability of ensembles developing professional musicians with a high degree of autonomy and professionalism, the actual reported assessment procedures were seen as dedicated to satisfying institutional demands instead. Staff felt constrained by these demands and comments about criteria and summative assessment practices by both students and staff highlight the complexity of the existing assessment processes for ensembles.
Perhaps most relevantly, staff perceived that current assessment practices were of little use to learning and teaching approaches when their stated aim was to firstly develop professional musicians and secondly to develop a love for and respect of music. Many participants in this study expressed the view that summative assessment of individual students in ensembles did not allow for collaborative and cooperative learning processes to take place; as both staff and students noted, learning was the intrinsic reason for ensemble activities in music institutions. Without collaboration, collegiality and cooperation a music ensemble simply cannot function, as one student saliently noted:
The way we get on with each other is integral. With an assignment or an essay you're on your own until you bounce your ideas off people, but how well you do in this particular course [Ensemble] is quite dependent on the relationship you have with everyone else in the group. (SFG 3, 1.6.10)
The conflict between assessment for compliance and preparation for the profession was evident. In cases where the boundaries were tested, the ‘real-life’ outcomes were more readily apparent. In other cases, students and staff were frustrated that assessment items did not always result in feedback that assisted in vocational learning outcomes.
Conclusions and future directions
We have investigated an area in ensemble assessment rarely explored in music education research in Australia. The findings demonstrate that, while the staff goals of developing autonomy and professionalism in students were embraced in theory, in practice the widespread lack of understanding, transparency and feedback was seen as a serious shortcoming for a professional programme. The institution has since undergone substantial curriculum review (see Carey & Lebler, Reference Carey and Lebler2012), including broad stakeholder consultation in an attempt to ensure a stronger alignment of theory and practice.
The first aim of the larger project was to develop the collaborative learning and assessment attributes of the students. The documentation of the assessment process enhanced students’ awareness of (and dissatisfaction with) ensemble assessment and the more open and collaborative experience of consultation was appreciated. The second aim focused on the development of clearer, more transparent processes for the assessment of ensembles within the Conservatorium and to provide students with improved feedback about their progress. The findings from the project on feedback, communication and grading practices have, in part, changed the ways in which ensembles are assessed at the site under investigation. Students are now assessed on a pass–fail basis, and imbuing students with the culture of the profession has, as a result, come to the forefront of learning and teaching practices. Ensemble directors now sense that there is less of a need to artificially provide fine-grained results for students, particularly in the larger ensembles.
Feedback is provided in a variety of ways, principally verbally at rehearsals, as was previously the case. When ensembles require written submissions and/or such activities as mock auditions, feedback is still provided in the same manner as was previously the case, though the marking regime is somewhat more transparent. Some ensemble directors have expressed the view that graded assessment can be conducted in an appropriate manner for at least some ensembles, particularly smaller ensembles. These directors have been encouraged to engage in a collaborative development of an assessment protocol that may be initially applied to small ensembles.
Another approach under consideration is drawn from cooperative learning (Gross Davis, Reference Gross Davis1993). In this process, students’ concerns can be addressed through adding self- and peer- (group, team, ensemble) reviews. After the rehearsal/performance is completed, each student can be invited to submit to the instructor a confidential assessment of the participation of the other group members: who performed well, did extra and those who did not. The peer assessment process could be used to influence final results. When players (staff and students) come together and align their intentions, synergy (playing together) can result which brings benefits far greater than the sum of its parts. Self and peer assessment practices are already in place in other degree programmes at the host institution (see Lebler, Reference Lebler2008, Reference Lebler2010) and there is potential for the application of these practices to the Bachelor of Music. In the interim, students now have greater access to feedback and there is some hope that permanent and casual staff, as well as the student body, will gain clearer understandings of ensemble assessment.
While this study focused on an Australian Conservatorium setting, there may be implications for ensemble experiences in other settings. Given that this study was restricted to instrumental ensembles, further comparative research into vocal ensembles may be warranted. Perhaps most significantly, however, the project revealed previously unacknowledged power relations on a number of levels: a perception of conflict between broader university imperatives and local requirements in practice-based domains was exposed, along with tensions between casual and full-time staff. Similarly, the voice of the students was acknowledged in this project, bringing with it renewed reflection on the role of this major stakeholder in the learning, teaching and assessment regimes of the Conservatorium.
