Background
Throughout this paper Indigenous health is used as a global term to describe issues and information pertaining to both Aboriginal and Torres Strait Islander people in Australia. The focus of this paper and questionnaire is specific to Aboriginal people and Aboriginal health education. This concentration stems from the research team's expertise in these areas, but also relates to the high population of Aboriginal people over the Flinders University teachings sites in South Australia (Adelaide, Mount Gambier, and Renmark) and the Northern Territory (Darwin, Alice Springs, and Katherine), which connects to this questionnaire and its development.
Australian medical schools and health professional programmes are increasingly required to demonstrate social accountability to Australian communities that experience health inequities. Social accountability, in medical schools, has been defined by the World Health Organisation as ‘the obligation to direct their education, research and school service activities towards addressing priority health concerns of the community, region and/or nation which they have a mandate to serve’ (Strasser et al., Reference Strasser, Hogenbirk, Minore, March, Berry, Mccready and Graves2013). Indigenous health is a national health and social priority due to Indigenous disadvantage on every social indicator and poor health outcomes. This has led to the establishment of core accreditation requirements for cultural safety training to be delivered to health professionals in order for these professionals to meet the needs of Indigenous communities, such as those outlined by: Australian Medical Council, Australian Nursing and Midwifery Accreditation Council, Occupational Therapy Council (Australia and New Zealand) and Dietitians Association of Australia (AMC, 2010; ANMAC, 2012; DAA, 2009; OTC, 2013).
The approach taken by health professional programmes to meet these core accreditation requirements can vary significantly from one-off workshops to semester long topics, fully integrated curriculums or a combination of these approaches. While the aims and objectives underlying theoretical frameworks (e.g. cultural safety, cultural awareness or cultural sensitivity) may differ, generally such programmes are expected to facilitate a shift in student attitudes’ and to foster skills and knowledge to better equip future health professionals to work with Indigenous people and communities to achieve better health outcomes. While some research indicates that as little as one hour of cultural/diversity training can make a difference to cultural understandings, others suggest that this ‘sheep-dip approach’ has little meaningful impact (Walton, Reference Walton2011; Westwood & Westwood, Reference Westwood and Westwood2010). There is limited evidence in the literature about what length of time, delivery and content of programmes will shift students’, health professionals’ and academic staff member's attitudes in order to achieve positive outcomes in their future practice.
The research team for this project conceptualised Indigenous health education into three thematic areas; cultural safety, critical thinking and transformative unlearning. These three terms are considered multidimensional, involving or marked by several dimensions, describing the learning processes that health professional students undertake in the journey to becoming culturally safe health professionals. It starts through a process of critically reflecting and challenging personal implicit attitudes and core beliefs, then the facilitation of a transformation of these personal core beliefs through a process of unlearning. This process is followed by learning about the cultural safety framework, and how the principles of the framework may translate into future practice. This study is the first step in developing a questionnaire to measure how well the thematic areas are successfully achieved through exposure to an Indigenous health curriculum. The questionnaire will be applicable to learners in Indigenous health at any level, including undergraduate and postgraduate health students, qualified health professionals and tertiary health education professionals who are involved in student education and training.
Attitudes in Health Professional Training
In Australia, the education system (primary–tertiary) has been informed through a colonial construct, this central mindset has allowed a colonial mindset to ensue, informing the visions and perceptions of Australians, hence informing their actions overall (Almeida, Reference Almeida2016). More specifically, the Australian health care system is a western biomedical ‘one size fits all’ approach, which assimilates all cultures into a standard monocultural framework of cultural superiority: white, privileged, colonised and ethnocentric. This overall health care framework has significant influence on the health care education model and pedagogical approach taken to educate future health care professionals in Australia, which is informed through this dominant monocultural framework (McCleland, Reference McCleland2011). This monocultural framework discourages questioning or critical engagement and examination of this framework, and as such future health professionals, albeit unknowingly, risk becoming agents of colonialism (Beavis et al., Reference Beavis, Hojjati, Kassam, Choudhury, Fraser, Masching and Nixon2015).
For these reasons the attitudes that health professionals may bring to clinical encounters with Indigenous people are sometimes underpinned, consciously or unconsciously, by interpersonal racism and white privilege. Interpersonal racism has been described as racism that occurs through particular modes of interaction between people which may result in prolonged inequitable outcomes (Paradies, Harris, & Anderson, Reference Paradies, Harris and Anderson2008). A recent study in Victorian Aboriginal communities found that 227 of the 755 Aboriginal participants reported experiencing incidents of interpersonal racism in health settings in the last 12 months, causing psychological distress (Kelaher, Ferdinand, & Paradies, Reference Kelaher, Ferdinand and Paradies2014). In clinical settings, health practices underpinned by interpersonal racism may lead to assumptions, and stereotypical views of an Aboriginal patient's culture, family and needs. These assumptions may contribute to professionals altering their diagnostic reasoning processes which then may impact on health outcomes (Durey & Thompson, Reference Durey and Thompson2012; Larson, Gillies, Howard, & Coffin, Reference Larson, Gillies, Howard and Coffin2007; Paradies, Harris, & Anderson, Reference Paradies, Harris and Anderson2008). The majority of health professionals may, in fact be unaware that their attitudes, implicit biases and assumptions affect their diagnostic reasoning processes and may further contribute to individual health inequity (Blair, Steiner, & Havranek, Reference Blair, Steiner and Havranek2011). Furthermore interpersonal racism can contribute to institutional racism where policies, practices and processes of organisations contribute to inequitable outcomes. Both forms of racism influence the health disparities that are observed between Indigenous and non-Indigenous Australians (Larson, Gillies, Howard, & Coffin, Reference Larson, Gillies, Howard and Coffin2007; Paradies, Harris, & Anderson, Reference Paradies, Harris and Anderson2008).
Implicit bias and assumptive attitudes may be evident in the early training of health professionals in the areas of cultural safety and Indigenous health, whereby students often respond to such material negatively and sometimes with considerable hostility (Phillips, Reference Phillips2015). These reactions may stem from discomfort as students are challenged to ‘unlearn’ assumptions and stereotypes that they may have held their entire lives, without necessarily being conscious of these beliefs. For some it could be the first time they realise they may be a part of the problem rather than the solution (Durey, Reference Durey2010; Macdonald, Reference Macdonald2002; McDermott, Reference McDermott2012). Similarly, for practicing health professionals and academics (who may possess unconscious biases in relation to Indigenous health) there may be a manifestation of these attitudes through the concept of ‘bio-power’ that results in the deprioritising of Indigenous health education as a core ‘western bio-medical’ curriculum requirement (Phillips, Reference Phillips2015).
Thematic Areas
Cultural Safety
In Australia, it has been commonly reported that Aboriginal patients find health services culturally unsafe—they are unwelcoming, and alienating due mainly to the attitudes staff in these centres bring to their care (McGough, Wynaden, & Wright, Reference McGough, Wynaden and Wright2017). Similar experiences have been documented abroad where in New Zealand non-Māori nurses have been unable to see beyond their own whiteness and assumptive behaviour when caring for Māori patients (Benham, Reference Benham2001). Cultural safety originated from Māori nurses and midwives, as a framework for Indigenous patient care and is prominently theorised and articulated by Irihapeti Ramsden in her 2002 doctoral thesis. The framework was founded in the context of colonisation in a state of neo-colonialism; recognising the impact that colonisation has and continues to have on contemporary health outcomes for Indigenous peoples in colonised countries, sometimes called ‘setter’ countries, such as Australia, New Zealand, Canada and the United States (Brascoupé & Waters, Reference Brascoupé and Waters2009; Nursing Council of New Zealand, 2005; Ramsden, Reference Ramsden2002). It is important to articulate that cultural safety is not embedded in postcolonial scholarship, despite claims by Anderson et al. (Reference Anderson, Perry, Blue, Browne, Henderson, Khan, Kirkham, Lynam, Semeniuk and Smye2003), Beavis et al. (Reference Beavis, Hojjati, Kassam, Choudhury, Fraser, Masching and Nixon2015) and Browne, Smye, and Varcoe (Reference Browne, Smye and Varcoe2005). Ramsden (Reference Ramsden2002), like other scholars (Moreton-Robinson, 2004; Trees & Nyoongah, Reference Trees and Nyoongah1993), argues that use of the theory of postcolonialism is not appropriate, according to the philosophy of this theory, in colonised countries, such as New Zealand and Australia. Instead these countries are in a state of neo-colonialism, although this is not widely articulated, and furthermore that postcolonialism theory is a construct of whiteness which has been criticised widely by non-Western scholars.
One of the major differences of cultural safety as compared to other cultural frameworks in health (transcultural nursing, cultural sensitivity, etc.), is that is has been created from an Indigenous space of health and wellbeing, and an overall decolonised stand point aimed at addressing the inherent colonial oppression in health care systems (Ramsden, Reference Ramsden2002). It creates a shift in paradigm away from a western clinical academic thought process, to a reflexive approach, providing Indigenous patients, rather than health professional, with the power throughout the consultation process. However, in recent years, in the literature and health professional accreditation bodies, there has been an attempt to ‘redefine’ and ‘broaden’ cultural safety. This has been described by Ramsden (Reference Ramsden2002) as ‘naïve often romantic reconstructions’, to be inclusive of all cultures including intercultural (youth, sexual orientation), crosscultural (migrant), religion and disability (Gerlach, Reference Gerlach2012; Jungersen, Reference Jungersen2002). These inclusive movements are well intentioned as these cultural groups face tremendous prejudice and require a health care framework to meet the needs of their health inequalities. It could also be argued, nevertheless, that this reconstruction of cultural safety has served to colonise the Indigenous paradigm of cultural safety through a shift away from its original intent. These broader definitions have the potential to dilute or impact on the integrity and uniqueness of an Indigenous cultural safety framework founded on the sovereign health rights of Indigenous peoples.
Cultural safety allows students to engage in critical analysis through a decolonising lens, allowing students to recognise the impact of colonisation on health and wellbeing, but also to be aware of the new patterns and forms of colonisation that are ongoing and reemerging in the health care system today. The overall cultural safety framework focuses on five key principles (Best, Reference Best, Best and Fredericks2014; Nursing Council of New Zealand, 2005):
1. Reflective practice — health professionals have the ability to undertake and acknowledge true personal reflection of oneself
2. Power differential minimisation — health professionals have the ability to recognise and minimise the power differentials that may occur in the clinical setting
3. Engagement and discourse — health professionals undertake appropriate engagement with patients, as defined by the patient, to understand and assist in meeting the patient's unique needs
4. Decolonisation — health professionals are able to identify the ongoing impact of colonisation on health equity and the need to be regardful of history and culture, and to decolonise practice in relation to Indigenous peoples
5. Regardful care — health professionals deliver patient-centred care, which does not diminish, demean or disempower the patient, and which incorporates the patient's unique cultural background.
Understanding cultural safety is a learning journey for learners, such as students, health professionals and academics. The framework builds upon cultural awareness, which is the initial step where health professionals notice cultural differences among their patients. The next step is cultural sensitivity where learners seek to actively explore these differences in terms of their own life experiences, and how these experiences could impact on patient interactions. The next step in this journey is cultural safety, whereby health professionals are able to reflect on their own positions of power, belief systems, and decolonise their practice, to into improve patient care (Ramsden, Reference Ramsden2002). Unlike other frameworks, cultural safety does not require learners to become experts on Indigenous cultures, but rather the focus is on their own ability to facilitate patient autonomy and regardful care encompassing respect, trust and sharing (Brascoupé & Waters, Reference Brascoupé and Waters2009; Paul, Ewen, & Jones, Reference Paul, Ewen and Jones2014; Ramsden, Reference Ramsden2002).
Cultural safety challenges a health professional's implicit and explicit exercised power, allowing a paradigm shift in this power from health professional to Indigenous patient (Ramsden, Reference Ramsden2002). This shift allows the beliefs, needs, and voice of the Indigenous patient, their family and community to be the predominant focus, allowing safe patient care (as defined by those receiving care) to ensue (Best, Reference Best, Best and Fredericks2014; Brascoupé & Waters, Reference Brascoupé and Waters2009; Ramsden, Reference Ramsden2002).
A critical principle of culturally safe practice is that health care professionals must be able to critically reflect on their own practice and that this critical reflection extends beyond a clinical skills focus (Best, Reference Best, Best and Fredericks2014). This requires learners to engage with critical thinking theory whereby they undertake a true reflection on their own identity, a reflexive approach. They should understand, their unique position of privilege and power not only in the context of their own lives, but also when considering Indigenous health; particularly the impact of colonisation. This, for many learners, is a confronting and discomforting experience because it is probably the first time they have considered the, largely unacknowledged and unearned, benefits of their societal status.
Critical Thinking
The ability to undertake true personal reflection depends on one's capacity to think critically, which is a crucial step in identifying a learner's capability to engage with, interpret and implement cultural safety principles into their practice. Critical thinking encompasses how a student thinks, rather than what they think. It is a responsive process where learners examine their implicit beliefs and values against academic literature, allowing them to arrive at a conclusion that may indicate that their personal beliefs and values are flawed (Mulnix, Reference Mulnix2012). The critical thinking process involved in reflection can take learners to the edge of their knowledge base and, challenge deeply held implicit attitudes. For some people this is an unpleasant and frightening experience, but for others can be exciting and energising (Berger, Reference Berger2004).
As unpleasant or exciting as it maybe, the practice of reflection using a critical thinking approach is vital to the cultural safety learning journey, it builds essential reflexivity skills in learners (Wilson, Reference Wilson2014). The body of knowledge and framework of cultural safety provides the required foundation for learners to engage in critical thinking (i.e. decolonisation). These critical thinking skills in the context of cultural safety are imperative, as it is at this stage that the transformative unlearning process is initiated, which when followed by targeted and relevant learning, provokes lasting change in a learner, while at the same time being a deeply emotional and challenging process personally (Hislop, Bosley, Coombs, & Holland, Reference Hislop, Bosley, Coombs and Holland2013). Hence, the critical thinking process is an important aspect of the cultural safety framework and transformative unlearning process.
Transformative Unlearning
Transformative unlearning is guided personal growth that takes learners to the edge of their understanding ‘through grief and mourning while maintaining hope in the possibilities for a new vision’ that ‘challenges deeply held cultural beliefs in the western world, including deeply held individual/family attitudes and beliefs’ (MacDonald, Reference Macdonald2002, p. 172). It is an education process informed by a de-colonial epistemology, and requires an approach in which learning to unlearn must occur in order to relearn (Tlostanova & Mignolo, Reference Tlostanova and Mignolo2012). During the unlearning process learners have reported feeling insecure, under attack, a loss of control over a situation, or even threatened and blamed. For these learners changes in understanding may be hindered if the appropriate amount of time is not allocated in order for the unlearning process to occur (Durey, Reference Durey2010; Macdonald, Reference Macdonald2002).
MacDonald (Reference Macdonald2002) has conceptualised three stages to transformative unlearning:
• Receptiveness — whereby learners are willing to consider and accept information that challenges their own perspective,
• Recognition — whereby learners accept the truth of this information and the limitations of their own perspective, and
• Grieving — whereby learners accept the loss of their past perspectives and assumptions, which they may have justified on a long-term basis. Learners can feel defeated even devastated by their prior perspectives and assumptions.
The transformative unlearning process in relation to Indigenous health and cultural safety is critical as it challenges learners’ personal and social responsibilities, as well as ethical accountability, producing a deep conscious shift in their attitude and assumption base. The intention in initiating this process is to facilitate a shift in consciousness that will inform and transform future practice as culturally safe health professionals.
An obstacle to this process arises when students, who after having undertaken Indigenous health and cultural safety training in the early years, then undertake clinical placements in the latter parts of their degree or after graduation, and subsequently enter a workplace which is rife with negative attitudes and racism. Research has shown that medical students, for example, model their behaviour on residents and clinicians who have mentoring roles (Branch, Reference Branch2010). Such learners find themselves in a moral dilemma where medical education has instilled an understanding of cultural values along with the need for patient empathy, but this understanding becomes unravelled when value is not given to these principles in the clinical environment (Branch, Reference Branch2010).
It is therefore important to extend cultural safety training to be included in all years of health professional training programmes (including clinical), as part of a whole of school responsibility. It should include health professionals and academic staff who are influential in shaping the values of health professional students, who have undertaken this transformative unlearning journey themselves, and can model and advocate for culturally safe patient interactions with Indigenous patients. Teaching staff, who have close student contact in the early years of studies, also need to be included in this education process to ensure students’ learning outcomes are appropriate for the different stages of the learning journey.
While the authors’ teaching experiences suggest that transformative unlearning does occur amongst learners to produce lasting change, evidence is needed to determine the quantity and quality of teaching required to bring about this change and whether the timing and spread across a course, and the mode of delivery are significant factors in this process. In order to build evidence in relation to the effectiveness of cultural safety education in health at undergraduate, postgraduate and academic staff training levels, a validated tool to measure the effectiveness of the transformative unlearning process, particularly in relation to cultural safety training is needed.
This paper describes the process of developing and piloting a questionnaire designed to measure the impact and effectiveness in relation to student and staff engagement with Aboriginal health training through cultural safety training and transformative unlearning.
Method
Ethics approval for this pilot study to develop and validate a questionnaire for future education programmes in Indigenous health and cultural safety specific to Australia was obtained from Flinders University Social and Behavioural Research Ethics Committee (Project number 6038).
Instrument Development
A literature search was performed by the lead author to define operational definitions in the key thematic areas of cultural safety and transformative unlearning. Education, health and Indigenous collections in the Informit database were searched using the search terms: Cultural Safety, Assessment, Indigenous Health (including Aboriginal Health), Health Professional Education and Transformative Unlearning. A total of 16 papers were returned from which the following definitions were selected:
Cultural safety (Nursing Council of New Zealand, 2005):
The effective practice of a health professional, team or organisation from another culture, which is determined by the patient. The individual, team or service has reflected on their culture and recognised the impact it has on their practice, resulting in adjustment of action.
Transformative unlearning (MacDonald, Reference Macdonald2002):
Unlearning is conceptualized within a transformative education paradigm, one whose primary orientation is discernment, a personal growth process involving the activities of receptivity, recognition and grieving.
The research team felt the principles of critical thinking: personal reflection, self-examination, self-comparison to academic evidence, formation of new or changed beliefs, was captured in the two definitions of cultural safety and transformative unlearning, thus not requiring a separate definition.
Face and Content Validity
To construct the questionnaire a two-step content validity and face validity approach was utilised for selection of relevant questions. The initial step for the content validity process involved reviewing current surveys used to measure the defined thematic areas (cultural safety and transformative unlearning) by the lead author (Lynn, Reference Lynn1985). The content validity process resulted in selection of four papers that used survey evaluation tools to evaluate individual attitudes, perceptions and stereotypes of Indigenous people, along with cultural awareness and safety in the health sector (Carr, Paul, & Bazen, Reference Carr, Paul and Bazen2011; Pedersen, Beven, Walker, & Griffiths, Reference Pedersen, Beven, Walker and Griffiths2004; Rew, Becker, Cookston, Khosropour, & Martinez, Reference Rew, Becker, Cookston, Khosropour and Martinez2003; Woloschuk, Harasym, & Temple, Reference Woloschuk, Harasym and Temple2004).
All questions from the four papers were included for face validity, resulting in a total of 122 items for consideration by content experts. The content experts consisted of five clinical and academic staff members from the Poche Centres of Indigenous Health and Well-being, at Flinders University, Adelaide and Alice Springs. The staff members have extensive experience working in Aboriginal community contexts, cultural safety education, Indigenous health curriculum design and delivery. Content experts, were asked to independently consider each of the 122 items and whether they could be used to measure a change in student attitudes to demonstrate understanding of cultural safety principles, the experience of transformative unlearning, with critical thinking being captured in these two. A three-point Likert scale similar to Lawshe's was utilised for content experts to report on each question (DeVon et al., 2009). A score of −1 to +1 was used as follows: Question does not meet thematic areas and should not be included in tool (−1), Question meets at least one of the thematic areas and could be included (0), Question meets both thematic areas so should be included in the tool (+1). Panel members were asked to support their rankings with references from the literature.
Responses were then collated by the lead author and reviewed by the research team. The final questions were agreed upon and reassembled into a useable format for the questionnaire (Lynn, Reference Lynn1985). This process required the research team to consider all 122 items with arguments for inclusion and exclusion, allowing the research team to arrive at a consensus on each item. Where items received a −1 from at least two or more content experts or received a 0 from three experts, it was agreed that these did not fit at least one thematic area and were subsequently excluded (n = 107). The remaining 15 questions were found by all content expert panel members to meet at least one of the thematic definitions of cultural safety, transformative unlearning and critical thinking. These questions formed the questionnaire content to be rated on a five-point Likert scale ranging from strongly disagree (1) to strongly agree (5) (see Table A.1). Questionnaire reliability and agreement between the content expert panel members for item classification was calculated by Fleiss's kappa coefficient. Fleiss’ kappa allows the degree of agreement to be calculated for any number of assessors (expert panel) giving ratings to a set number of items (Fleiss, Reference Fleiss1971).
Test–Retest Reliability
Test–retest methodology with a 2 week interval period was considered the best practice method to determine stability and reliability of the questionnaire. This research method is consistent with similar research published in survey reliability and validity, for example, Carr, Paul, Bazen (Reference Carr, Paul and Bazen2011), Miville et al. (Reference Miville, Gelso, Pannu, Liu, Touradji, Holloway and Fuertes1999), Nichols-English and Gunion (Reference Nichols-English and Guion2008). Data was entered onto a Microsoft Excel spreadsheet, checked and edited before being transferred to data analysis and statistical software package STATA version 13.0 for analyses of all test–retest questionnaire statistics (StataCorp, 2011).
The reliability of the test–retest questionnaire was calculated using a weighted kappa (kw2) statistic with quadratic weights for ordinal items (i.e. questionnaire questions) so that different levels of agreement could influence the kappa output (Cronbach, Reference Cronbach1951; Efron, Reference Efron1986; Hume, Ball, & Salmon, Reference Hume, Ball and Salmon2006). Given the questionnaire contained 15 items a total of 15 weighted kappa values with 95% confidence intervals (CI) were calculated, where the following kappa parameters ensued; no agreement (<0.00), slight agreement (0.01–0.20), fair agreement (0.21–0.40), moderate agreement (0.41–0.60), substantial agreement (0.61–0.80), and almost perfect agreement (0.81–0.99) (Cohen, Reference Cohen1968). The test–retest reliability of sum scores of the 15 items in the pre and posttest was calculated using the intraclass correlation coefficient (ICC) that measures the level of agreement between the two tests, where a coefficient of 0 represents a totally unreliable measurement and 1 indicates a perfect reliability. However, levels of ICC above 0.7 are recommended as reliable (Streiner & Norman, Reference Streiner and Norman2008).
Factor Validity and Scale Reliability
Exploratory factor analysis (EFA) was applied to identify questionnaire item loadings to factors. In this investigation the content expert panel assumed that a two factor model would appear capturing the thematic areas of transformative unlearning and critical thinking, where the thematic area of cultural safety could be apparent in all of the questionnaire items.
Prior to EFA analysis it was determined that, five of the 15 items were negatively worded, these items were reversed coded (i.e. a score of 5 became 1) to ensure all items were loaded in a consistent direction. Questionnaire data suitability was assessed using Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett's test of sphericity, where KMO and Bartlett's values above 0.5 were considered suitable for EFA (Tabachnick & Fidell, Reference Tabachnick and Fidell2007; Williams, Brown, & Onsman, Reference Williams, Brown and Onsman2012). A multiple approach model for factor extraction was used consisting of principal component analysis, Kaiser's criterion (Eigenvalues >1.0), scree test and parallel analysis to simplify and reduce the factor structure of items into factors and determine the number of factors retained (Tabachnick & Fidell, Reference Tabachnick and Fidell2007).
To simplify the EFA structure for analyses, varimax rotation was utilised, where adjusting the factor loadings, maximising items with high correlations and minimising low correlations simplified the factor structure (Williams, Brown, & Onsman, Reference Williams, Brown and Onsman2012). On factor discovery the internal reliability was calculated for each through Cronbach's alpha, where statistical significance was set to p < 0.05 and values between 0.7 and 0.9 indicated acceptable homogeneity between items with limited redundancy (Tavakol & Dennick, Reference Tavakol and Dennick2011).
The Recruitment of Questionnaire Participants
All undergraduate and masters courses in the Faculty of Medicine, Nursing and Health Sciences (now known as, College of Medicine and Public Health and College of Nursing and Health Sciences) were reviewed for participant suitability. A course was selected as being suitable if topics (subjects/units) of the course did not include the teaching of Indigenous health in first semester of the first or second year of the course. This was because the questionnaire was to be administered in second semester of the university year for these students. Students enrolled in topics in such courses were deemed suitable for recruitment to the study. Topic coordinators were approached by the lead author to see if they were willing for the questionnaire to be distributed at the commencement of a lecture or tutorial, on two occasions, for one to two undergraduate groups and one postgraduate group.
Target samples sizes of 20 undergraduate students, 5 postgraduate students and 5 staff members were established for the initial validation. Undergraduate students in the Bachelor of Medical Science programme, who thus far in their degree had undertaken no prior education in Indigenous health or cultural safety, were invited, in person, by the research assistant (RA) to participate in the survey. Postgraduate students who were studying in the Master of Physiotherapy and Master of Occupational Therapy, again with no prior education in Indigenous health or cultural safety in their current degree, were invited, in person, using the same method as undergraduate students to participate in the survey. It was considered important that students had no prior tertiary education in the content area, so that students would be representative of the future target population with whom this tool might be administered. Table 1 indicates the results from this recruitment where undergraduate n = 22 and postgraduate n = 9. Student volunteers were enlisted who were able to come to the class 20 min prior to the class (or a nearby venue), on two occasions, for 20 min. Given the busy schedules of staff members, and the assumption that the majority of staff have access to the University intranet, an invitation was sent, by internal email, to all Faculty of Medicine, Nursing and Health Sciences inviting staff to participate in the survey online. Faculty staff members were included in the validation process to broaden the target population with whom this tool may be used in the future, as few such staff members have undertaken Indigenous health and cultural safety training in their prior study. The invitation email was sent to 696 staff, and after administration of both surveys and coding through computer internet protocol (IP) addresses of survey one to survey two the resultant number analysed was n = 9 (Table 1).
Participant Recruitment Numbers for Survey Participation
* Results after survey two coding to survey one.
Questionnaire Administration
The RA met each group of students (undergraduate and postgraduate) at the appointed time and place and, after recording each student's Flinders Authentication Name (FAN) administered the questionnaire. Information was also provided on why it was important that students had the ability to attend and complete the questionnaire on both occasions. A participant information sheet was provided with contact details in case participants were unable to participate or if they had any questions. The RA met with the student groups on the second occasion at the same time, 2 weeks later, and administered the same questionnaire, noting student's FANs and recording student absences or those who were not present on the first occasion. Students who could not attend the retest were asked to contact the RA to ask for an alternative time to sit the retest, either with another group, or as an individual. This process needed to be undertaken within 48 h of the original appointed time, to address being as close to a 14 day period of difference as possible. The questionnaire was prenumbered for participants, and the RA matched the prenumber to each student FAN so that responses could be cross-checked while preserving anonymity.
All Faculty of Medicine, Nursing and Health Science staff at Flinders University were targeted through direct email. Participating staff were asked to complete the tool online, and were required to do so within 48 h after the release of the questionnaire (responses after this time were excluded from the study). Participating staff were then reminded 2 weeks later, through email, to participate in the retest and having again to do so within 48 h of the retest date time to allow for fortnight reliability requirements. Computer IP addresses, logged dates and times were used to adhere to time requirements, track and code participation of staff members by the RA.
Results
Questionnaire Design Results
Face and content validation enabled a selection of 4 papers from the literature, which yielded 122 questions for review by the content expert panel, along with the creation of a 15 item questionnaire with a 5 point Likert scale. The Fleiss’ kappa coefficient was calculated at 0.27 demonstrating fair agreement of the questions across the 5 different content experts, allowing all questions to be included into a 15 item 5 point-Likert scale questionnaire. The questions in the survey asked participants to rate their agreement (between 1 = strongly disagree and 5 = strongly agree) in three main areas focussed on cultural safety, transformative unlearning and critical thinking.
Test–Retest Results
Test–retest reliability results (Table 2) indicates that substantial overall agreement was reached for 6 of the 15 items (40%), with a kappa range of 0.66–0.78. For the remaining 9 items, reliability was moderate for 5 items (33%) with a kappa range from 0.42–0.58, and 4 items (27%) reached only slight agreement with the lowest kappa rating being 0.28. Table B.1 contains subgroup results for reliability; the results in this area differ slightly to the overall population and have not been included due to the small sample size in two of the subgroups.
Test–Retest Reliability of Questionnaire
Test–retest reliability (weighted kappa, SE) of a questionnaire to measure attitudinal change in health science students and staff who complete a course in Indigenous health/cultural safety (cultural safety and training questions); SE = standard error.
†If kappa is less than 0, ‘no agreement’, if 0–0.2, ‘slight agreement’, if 0.2–0.4, ‘fair agreement’, if 0.4–0.6, ‘moderate agreement’, if 0.6–0.8, ‘substantial agreement’, if 0.8–1.0, ‘almost perfect agreement’.
The ICC for the questionnaire between baseline and retest 2 weeks later, reached a satisfactory level, with ICC = 0.72, and a 95% CI of 0.55–0.85.
The correlation matrix that provides output on the item interrelations for the EFA (Figure 1) was demonstrated graphically by using the image function to emphasise the factor structure of the 15 items of cultural safety from the questionnaire. Low correlation outputs for items CSTQ1 to CSTQ7 were obtained suggesting a low association with factors. Items CSTQ8–CSTQ10 and CSTQ13 were structured together with correlations ranging from 0.45 to 0.61, and items CSTQ11, CSTQ12, CSTQ14 and CSTQ15 together produced correlations from 0.48 to 0.58.
Correlation structure of 15 items for factors of critical thinking and transformative unlearning.
Parallel analysis and scree plots (Figure 2) confirmed a two factor model existed in the cultural safety questionnaire, where a review of the academic literature provided by the content expert panel connected factor 1 to critical thinking and factor 2 to transformative unlearning. Internal reliabilities for critical thinking and transformative unlearning were acceptable (α ≥ .70) for the questionnaire (Table 3).
Parallel analysis plots for two factor model.
Exploratory Factor Analysis and Internal Consistency of Questionnaire
Factor loadings from exploratory factor analysis (EFA) and internal consistency for transformative unlearning and critical thinking (n = 40).
A total of six items were loaded to critical thinking, where four of these items had a range from 0.60 to 0.76, suggesting a strong association with this factor. Two of these items had below 0.25. Nine items were loaded to transformative unlearning where four items had a range from 0.55 to 0.71 providing a sound level of association with this factor. However five items provided lower levels of associations with a range of 0.22–0.38.
Discussion
Good reliability of the tool overall for undergraduate, postgraduate students and staff members was obtained for 11 of the 15 items (73%). There were four items (27%) that met only fair reliability, where three of these four items focus on the thematic area of transformative unlearning, and item 2 is the only item which relates to critical thinking. The four items that only met fair agreement were the following:
CSTQ2: Health professionals’ own cultural beliefs influence their health care decision
CSTQ3: I need to think beyond the individual when considering Aboriginal health issues
CSTQ7: As a health professional if I needed more information about a person's culture to provide a service, I would feel comfortable asking the person or one of their family members
CSTQ14: Time in the health professional curriculum devoted to the promotion of student self-awareness and well-being is time well spent.
The lower kappa values for these items indicate systemic disagreement in the overall population. This systemic disagreement could be related to differences between undergraduate, postgraduate and staff groupings and also the demographic profiles of each of the subgroups. For instance students entering into postgraduate programmes enter with an undergraduate degree that can be anything from a 3-year health sciences bachelor's degree with no work experience to a qualified paramedic with many years’ experience. Generally students with extensive past working experience bring a difference in maturity in comparison with their student counterparts who lack this experience. This maturity shapes their responses to new academic knowledge and is apparent in their self-awareness and problem solving skills (Cooper, Reference Cooper2011; Newman & Peile, Reference Newman and Peile2002). Such diversity in a small population of students may have contributed to disagreement in students, resulting in four items in the questionnaire not reaching a desired level of reliability. Furthermore for items CSTQ3, CSTQ7 and CSTQ14 it has been suggested that students often feel topics on Indigenous health are ‘soft’ and ‘easy’, and should not be given the same priority as topics focussed on human homeostasis, physiology or anatomy (McDermott & Sjoberg, Reference McDermott and Sjoberg2012; Thackrah & Thompson, Reference Thackrah and Thompson2013). Such potentially varied beliefs held by students are perhaps being reinforced by some academic staff, could account for systematic disagreement in each of the subgroups. It is important, however, to consider that these three items have been specifically constructed by the content expert team to capture student and staff development in the thematic area of transformative unlearning, which can be challenging for some individuals. Furthermore, it may be anticipated that a well-designed curriculum, with focussed pedagogical approaches in Indigenous health, covering the thematic areas, should have a profound impact on the interpretation of these questions by learners on completion of their training and that such attitudinal change would be captured by the questionnaire. For this reason, it is suggested that a larger cohort study with greater diversity in each of the subgroups, should be undertaken, where a small amount of demographic information is collected on staff and students current and past degree(s), along with occupational experience in relation to Indigenous health and cultural safety.
The measurement of internal reliability was considered satisfactory (α ≥ .70) for the factors of transformative unlearning and critical thinking. High loadings were outputted for over half of the 15 items in the questionnaire, with four items for critical thinking and four items for transformative unlearning. These high loadings demonstrate strong interconnectedness between these items, thus being representative and capturing their factor areas of critical thinking or transformative unlearning in the questionnaire. There were, however, items with low loadings in the questionnaire; two items for critical thinking and five items for transformative unlearning. The items with low loadings for both these factors, suggest that these items have a low connection with their prescribed factor. This could be because these items were not loaded correctly, as a result of unrealised theoretical consideration, such that these items did not fit into their particular hypothesised dimensions.
For items with low loadings, improvements could be facilitated by reviewing the wording and intent of each item, altering their structure to enhance relatedness, or adding additional items to create clearer factor connection thus enhancing their loading output (Tavakol & Dennick, Reference Tavakol and Dennick2011). However while altering the language, or adding new items, may assist in improving factor item loading in the questionnaire for the study population, pedagogical approaches and teaching environments used to educate staff and students on Indigenous health and cultural safety can have a profound impact on the interpretation of these items. As such, item alterations and additions should be approached with care. Too much alteration may diminish the intended outcomes of the process, and also impact negatively on the questionnaire overall (Carr, Paul, & Bazen, Reference Carr, Paul and Bazen2011).
This was a pilot study, the next stage for this research will be a larger cohort study, using a revised tool with undergraduate and postgraduate students, and staff, involved in these courses from a range of tertiary education institutes across Australia. For this planned study, the collection of demographic information for postgraduate students (i.e. prior degree, working experience) and for staff (i.e. clinical academic, academic or professional) will assist in analysing the difference in internal consistency between these sample groups, and further assist analysis of the reliability and validity of this tool. Undertaking a larger study will decrease the impact of systematic difference, and obtain a result of higher statistical significance (Weir, Reference Weir2005).
Another limitation of this study is that the tool was developed by the content expert team with a focus on Aboriginal health, as all members of the content expert team had significant experience working in Aboriginal community settings. This tool has not been developed to capture the unique training required specifically for Torres Strait Islander health, and therefore usability in this setting is unknown. At this stage, the research team will not be adjusting the tool to expand its utility to include this domain.
Conclusion
The study discussed in this paper presents the first stage in validating a new questionnaire to measure the effectiveness of Indigenous health and cultural safety curriculum through its impact on health professional learners, including undergraduate and postgraduate students and university staff responsible for health professional education. The initial validation process has revealed some concepts that have insufficient internal validity. The researchers will revisit these concepts and make adjustments to the tool before we conduct a further large scale trial with additional tertiary institutes to allow further development and to strengthen the validity and reliability measures of this questionnaire. The validated and improved questionnaire will assist educators to evaluate their approaches to cultural safety pedagogy, and will potentially contribute to improved outcomes from Indigenous health and cultural safety training in Australia.
Acknowledgments
We would like to acknowledge the Faculty of Medicine, Nursing and Health Sciences for the seeding grant provision to carry out the design and validation of this questionnaire. We would also like to thank members of the content expert team Dr Kerry Taylor from the Poche Centre of Indigenous Health in Alice Springs, Mr David Sjoberg from the Poche Centre of Indigenous Health and Well-Being in Adelaide for their contribution to the tool development.
Appendix A: A tool to measure attitudinal change in health professional staff and students after completion of an Indigenous health and cultural safety training programme
Questionnaire Items
Appendix B: Subgroup (undergraduate, postgraduate and staff) statistical results
Test–Retest Reliability of Questionnaire
About the Authors
Courtney Ryder is an Indigenous Health Lecturer in Southgate at Flinders University, and Nunga woman presently undertaking her PhD. Her prior work has focused on topic coordination, curriculum and pedagogical development for Indigenous health in the Doctor of Medicine program.
Tamara Mackean is a Senior Research Fellow at the Southgate Institute for Health, Society and Equity at Flinders University. A descendent of the Waljen Peoples, she is the Chairperson for the RACP's Aboriginal and Torres Strait Islander Health Committee and board member for the Lowitja Institute.
Shahid Ullah is a Senior Biostatistician in the South Australian Health and Medical Research Institute. He has an excellent broad-based research background and expertise in the field of applied statistics. His research currently focuses on application of functional data analysis to the area of health sciences.
Heather Burton is a Doctor of Public Health student with a focus on education as a social determinant of health for young Aboriginal people. Prior to this, she was a Lecturer in the Flinders University Rural Clinical School, with teaching and research interests in primary health care and health equity.
Heather Halls is a Research and Teaching Coordinator at the International Centre for Point-of-Care Testing. She teaches a variety of undergraduate and graduate programmes with a focus on Global Point-of-Care Testing and her role in research and training includes the Quality Assurance for Aboriginal & Torres Strait Islander Medical Services.
Professor Dennis McDermott is the Director of the Poche Centre for Indigenous Health and Well-Being Adelaide, at Flinders University. A Koori man, psychologist, academic and poet, his teaching and research encompass social determinants of Indigenous health, pedagogy, and wellbeing.
Associate Professor Wendy Edmondson is a health education consultant and Badimia and Amungu Aboriginal woman. She has over 37 years' experience in Aboriginal education and Aboriginal health, she has worked across state and federal governments, universities, and the Aboriginal community controlled health sector.
