Introduction
Diabetes mellitus is rising in prevalence around the world (Danaei et al., Reference Danaei, Finucane, Lu, Singh, Cowan, Paciorek, Lin, Farzadfar, Khang, Stevens, Rao, Ali, Riley, Robinson and Ezzati2011) and in Australia (Australian Institute of Health and Welfare, 2010). The prediction is that if diabetes continues to rise at the current rates, up to 3 million Australians over the age of 25 years will have diabetes by the year 2025 (Diabetes Australia, 2012). Projections based on current expenditure indicate that by 2025 the annual cost for Australians with type 2 diabetes will be up to $6 billion including healthcare costs, the cost of carers and Commonwealth government subsidies (Diabetes Australia, 2012). For type 2 diabetes, this is likely driven by rising obesity, the ageing population, dietary changes, and sedentary lifestyles (Diabetes Australia, 2012). Hence, it is necessary to focus on diabetes prevention programmes as well as strategies that promote effective use of healthcare resources. In November 2010, the Queensland State Government allocated $7.5 million to primary health care organisations to support a range of health initiatives focusing on chronic disease prevention, early intervention and management (Queensland Health, 2005). Within this initiative an emphasis was placed on supporting general practitioners (GPs) to enhance patient care by raising awareness of roles within the multidisciplinary team and increasing accessibility to best practice clinical education.
Literature available on improving quality of diabetes care in primary care settings shows that professional practices benefit with educational interventions (Forsetlund et al., Reference Forsetlund, Bjorndal, Rashidian, Jamtvedt, O’Brien, Wolf, Davis, Odgaard-Jensen and Oxman2009) and feedback that focuses on quality of care (Guldberg, Reference Guldberg, Lauritzen, Kristensen and Vedsted2009). As the number of patients with diabetes presenting to general practice for treatment is increasing, available state funding led to the development of a unique training opportunity, Diabetes Connect, with the key aim of increasing the capacity of general practice to meet the complex and diverse needs of patients with type 2 diabetes (CheckUP Australia, 2012). The Diabetes Connect programme was developed to enhance communication across primary care networks, support best practice in clinical interventions and advance multidisciplinary team work to benefit patient care. The programme invited and trained general practitioners, practice nurses, allied health and other professionals involved in the care of patients with type 2 diabetes. It did so through two specific but interrelated initiatives: the provision of online clinical education and interprofessional face-to-face workshops across the state. The Diabetes Connect programme eventually aims at providing training using the most effective method of delivery. The evaluation of the Diabetes Connect programme, the focus of this paper, investigated the current knowledge of participants, their perceived ability to adopt this knowledge at work and willingness to change professional behaviour when treating patients with type 2 diabetes. The specific aim of this evaluation study is limited to investigating the extent to which two methods of delivering the programme, workshop and online modules, improve knowledge and confidence of the health professionals in the assessment and management of patients with diabetes.
Methods
Recruitment of the participants
By August 2012, the administrative records provided to the evaluators showed 553 health professionals had registered for online clinical education modules and/or workshops. There was some overlap between those attending workshops and those undertaking online clinical education modules. Hence, the participants can be divided into three groups (Figure 1):
-
1. Participants who attended workshops only
-
2. Participants who completed online modules only
-
3. Participants who attended workshops and also completed online modules (between one and nine modules).
Flow chart for the ‘Diabetes Connect Evaluation Project’ 2011–2012.
The number of participants completing baseline and one week follow-up surveys is set out in Figure 1. At baseline and one week post training the self-reported information on individual competencies was collected via an online survey.
Training details
Online clinical modules
The online clinical education modules are designed to improve participants’ knowledge and confidence in assessment and management techniques, as well as their recognition about when referring to other services is beneficial to patients. Participants are strongly encouraged to complete the online clinical education modules prior to attending a face to face workshop. There were nine modules available for the participants with each focusing on a specific aspect of diabetes management.
Workshops
The Diabetes Connect workshop sessions were designed to be interactive and involve predominantly small group discussion followed by reporting back to the plenary group as the methodology. The small groups were to include a mix of health care professionals to work together on case studies while discussing patient management from a multidisciplinary perspective. The sessions encourage an examination of one’s own professional scope of practice, and an awareness of roles and responsibilities of other members of the health care team. Participants were also exposed to guest presentations from a number of Allied Health professionals (psychologists, podiatrists, and dietitian) providing further awareness and understanding of the role allied health play in management of type 2 diabetes. The workshop programme was designed to deliberately omit didactic sessions, thus encouraging interaction, networking and collaborative problem solving. Participants were strongly encouraged to complete the online learning modules before attending the one day workshop to ensure that they had sufficient clinical knowledge to get the most from workshop discussions. The full day (8 h) workshops held at six places across Queensland including urban and regional areas.
Evaluation design
The evaluation design included completion of surveys at two time points: first upon registering for the online modules or workshops; second, one week after attending a workshop. Both surveys were offered online. Participants that did not attend workshop were not offered to complete the survey at one week as the aim was to test the impact of undertaking online modules over and above the impact of attending the workshops. This survey included the role performance questionnaire that was developed from previous work on professional performance in related fields for general health practitioners (Skinner et al., Reference Skinner, Roche, Freeman and Addy2005).
Role performance individual competencies questionnaire
The individual and workplace role questionnaire was developed from earlier questionnaires designed to measure changes in the practices of health professionals when challenged to adopt new ways of working with patients (Skinner et al., Reference Skinner, Roche, Freeman and Addy2005). The questionnaire consists of three scales that measure: (a) individual competencies, (b) inter-professional practice and (c) aspects of the work environment. This evaluation focuses solely on changes that can be expected within the individual competencies scale immediately after attending the workshop and/or undertaking online modules. Each item was grouped into four performance measures (see Appendix 1) and was scored using a six-point Likert: disagree a lot, disagree, slightly disagree, slightly agree, agree, and agree a lot. The minimum score was 1 (disagree a lot) and maximum score was 6 (agree a lot).
Data analysis
All the items were scored from 1 to 6 and some items (indicated as ‘reverse’ in Appendix 1) were scored in reverse manner due to negative responses. The total score for each performance measure was calculated by adding scores for the items in that scale. The internal consistency of the scales was measured using Cronbach α and was above 0.8 for each of the combined scale. Mean scores were calculated by dividing total score of that performance measure by number of items under that measure.
For the ease of analysis, the primary care system was split into four related sub-systems.
-
1. Medical Primary Care System (n=67) consisting of GPs, pharmacists, medical specialists and pathology services. Included in this group are practice nurses who usually work with GPs.
-
2. Allied Health Clinical Primary Care System (n=14): This consists of a range of allied health professionals as well as dentists and their related professionals.
-
3. Patient Education System (n=4) that includes diabetes educators.
-
4. Wellbeing and Equity System (n=8) that includes professionals and services that enhance prevention and are designed to provide better access and support for patients such as indigenous health workers.
The statistical software SPSS (version19) was used to analyse the data. To assess the differences between demographics at baseline and at one week follow-up, χ 2 test was used for categorical variables. To measure the differences in the scores on a continuous scale paired t-test was used and P-value of less than 0.05 was set to confirm the statistical significance.
The ethics approval was sought through the Behavioural and Social Sciences Ethics Committee at the University of Queensland.
Results
The number of participants completing baseline and one week follow-up is set out in Figure 1. To establish the representativeness of the follow-up participants’ the socio-demographic information at two different time points is compared (Table 1). Apart from the significant increase (P<0.05) in the responses from participants between the age group 45–54 years and a higher percentage of participating allied health professionals at follow-up, there was no apparent difference between the demographic distributions of the participants at baseline and at follow-up. The number of participants from urban (n=45) or other areas (n=48) that responded at one-week was very similar.
Distribution of study participants, 2011–2012
*P>0.05 Significant change from baseline based on χ 2 test.
From the 93 participants, 33.3% (n=31) had previous diabetes training experience and 66.7% (n=62) did not. Of these two groups, 74.2% (n=23) who had previous diabetes and 67.7% (n=42) of the participants who did not have previous diabetes training completed four or more online modules. The difference between the two groups was not significant (χ 2=0.428, P=0.80). The mean of completed online modules was 4.84 (SD=3.38) and higher percentage of females (23.4%) completed all nine modules compared to male participants (18.8%). While only 16% of participating general practitioners and 10% of participating dietitians completed seven or more online modules; almost 65% of the participating practice nurses did so. The trend for the number of completed modules did not differ by the age group (χ 2=3.98, P=0.67). Table 2 summarises the change in mean scores for four performance measures related to professional practice in diabetes management. There was a significant increase in the participant’s perceived level of knowledge (Role adequacy scale), perceived ability to adopt this knowledge (Role legitimacy scale) and willingness to change professional behaviours to enhance patient management (Individual motivation and personal value scale). There was no significant change in participants’ career motivation for working in type 2 diabetes management which was already high.
Within the group differences from baseline to one week post intervention for participants that completed the training (n=93)
*Statistically significant differences.
Participants who completed both baseline and one week follow-up were a mix of those who had completed a workshop only and those who had completed a workshop and some online modules. To evaluate the impact of undertaking online modules in addition to the workshop participation, three groups of participants were created. From 93 participants that completed the one week post training questionnaire, 28 (30%) completed only workshop or workshop plus up to three online modules, 29 (31%) completed workshop plus four to six online modules while 36 (39%) completed workshop plus seven to nine modules. Figure 2 and Table 3 shows the difference score (one week score minus baseline score) for four measures of individual role performance. Participants that completed seven to nine online modules along with a workshop showed the greatest change in current perceived level of knowledge and skill in relation to management of type 2 diabetes (Role adequacy). In other words, online didactic learning (the online modules) does impact on knowledge acquisition over and above workshops. However, it did not impact on other areas of performance. Although there was a significant change from baseline in perceived ability to adopt new knowledge and skill in the work setting (Role legitimacy) this was not affected by undertaking increasing numbers of online modules. Participants that completed less than three online modules along with the workshops had slightly lower scores for career motivation than the participants that completed four to six modules. This trend continued for the participants that completed seven to nine online modules along with the workshop. This trend was, however, rather minimal. There was no specific advantage of completing online modules over and above attending the workshop on participants’ willingness to change professional behaviours to enhance patient management (Individual motivation).
Mean difference scores pre-post training for four individual role performance measures summarised for all participants and three workshop groups separately
Within the group differences for participants that completed workshops and online modules
Overall, online modules added value to knowledge acquisition as would be expected, but not to the motivation to change practice.
Discussion
The current study analysed the impact of an educational intervention that attempted to measure changes in the practices of health professionals when challenged to adopt new ways of working with patients. Overall, significant increases in participants’ current knowledge, perceived ability to adopt this knowledge at work and willingness to change professional behaviour in the short term were observed.
Our results are similar to previous studies that have shown benefits from training programmes such as workshops (Bloom, Reference Bloom2005; Forsetlund et al., Reference Forsetlund, Bjorndal, Rashidian, Jamtvedt, O’Brien, Wolf, Davis, Odgaard-Jensen and Oxman2009). It was apparent from participant feedback that they valued the opportunity for networking and the availability of flexible online modules. This appears similar to previous research suggesting that didactic training alone has minimal interest and impact on professional practice (Bloom, Reference Bloom2005). It is worth noting that the participants in the programme were volunteers who had strong professional motivation to work with diabetes patients. A wider impact on service improvements over time might be possible if strategies that encourage primary care professionals to take up training beyond those who are motivated to do so are established. Furthermore, it might be possible to optimise the effect of training programmes by encouraging participants to complete online modules before the workshops. This will then allow the workshops to focus on strategies that make the practice more ready to encourage change in professional practice.
This evaluation was limited by the small number of responses at one week after workshop and greater response would have possibly made the results more generalisable. However, the analysis shows that the response at one week was mostly representative of the professionals that participated in the study. A significant number of practitioners reported at baseline that their workplace was not particularly encouraging to the application of new forms of professional patient management (Bush et al., Reference Bush and Parekh2012). Addressing barriers in the work setting as an addition to the training programme may improve the uptake of knowledge and skills with patients. Evaluations with longer term follow-up would more readily address the question of cost-value of similar programmes particularly the impact of changes in professional practice in local primary care networks. The study cohort is heavily weighted towards participation of primary care practitioners and future studies might benefit from efforts to recruit and retain professionals from backgrounds other than primary care. Randomised trials of training delivery methods based on this evaluation will help to determine the cost benefit of face-to-face workshops and online training on professional role performance with diabetes patients.
Conclusion
Based on the outcomes of this evaluation; a programme that incorporates face-to-face training seems to have the potential to increase multidisciplinary practice while online learning modules seems to improve clinical knowledge but not necessarily its application in the work setting. The study suggests that for maximum benefit both, workshop and online training, should be combined and made available widely. Future programmes should use a randomised trial design to test the delivery model.
Acknowledgement
Diabetes Connect is a collaborative initiative between CheckUP (formerly General Practice Queensland), Medicare Locals and the Queensland General Practice Network funded by Queensland Health. S.P. planned and undertook data cleaning, coding and analysis and drafted the manuscript. R.B. conceptualised the evaluation project. S.C. participated in undertaking literature search. S.C. and P.G. implemented the ‘Diabetes Connect’ initiative as well as the evaluation project. All the authors approved the final manuscript.
Appendix 1
Survey items measuring individual competencies:
-
1. Role adequacy: Current level of knowledge and skill in relation to management of type 2 diabetes; which is measured using responses to five questions.
-
∙ I have the necessary experience to respond to a diversity of patients with complex diabetes problems
-
∙ In my work I have responded to patients with a wide range of complex diabetes problems
-
∙ I am confident in my ability to respond to a diversity of patients with complex diabetes problems
-
∙ I have the necessary knowledge to assist patients with complex diabetes problems
-
∙ I have the necessary skills to assist a diversity of patients with complex diabetes problems.
-
-
2. Role legitimacy: Perceived ability to adopt this knowledge and skill in the current work setting; that is measured using responses to seven questions.
-
∙ At work I have a legitimate role to play in responding to a diversity of patients with complex diabetes problems
-
∙ I am reluctant to take responsibility of all patients with complex diabetes problems (reverse)
-
∙ It is more appropriate for other health professions rather than me to respond to complex diabetes problems (reverse)
-
∙ I am uncertain of my role in responding to patients with complex diabetes problems (reverse)
-
∙ I am clear about my responsibilities when responding to patients with complex diabetes problems
-
∙ I have a responsibility to undertake interventions with those with complex diabetes problems
-
∙ Most diabetes patients seen by me believe I have a legitimate role to play in their treatment
-
-
3. Individual motivation: Willingness to change professional behaviours to enhance patient management that is measured using 10 questions.
-
∙ I would prefer not to work with patients with complex diabetes needs because it is too difficult (reverse)
-
∙ I refer people with complex diabetes problems because there is not much I can do for them (reverse)
-
∙ I believe people in my profession should be involved in the treatment of complex diabetes problems
-
∙ I get personal satisfaction from working with a diversity of patients with complex diabetes problems
-
∙ My experience of working with a diversity of patients with complex diabetes problems has been very rewarding
-
∙ On the whole I am satisfied with the way I work with a diversity of patients that have complex diabetes problems
-
∙ I like to respond to a diversity of patients with complex diabetes problems
-
∙ Many people with complex diabetes problems are not really interested in the interventions I can offer (reverse)
-
∙ Most patients with complex diabetes problems can manage reasonably well without my intervention (reverse)
-
∙ I try to avoid responding to diabetes patients who have complex problems because they don’t respond well to my interventions (reverse)
-
-
4. Career motivation: Working in type 2 diabetes management seen as a career advantage that is measured using three questions.
-
∙ There are clear professional advantages to me if I can demonstrate I work effectively with diabetes patients with complex problems
-
∙ Openly demonstrating my ability work in a multidisciplinary team approach to diabetes management is highly regarded by my professional colleagues
-
∙ In career terms, there are definite advantages in improving my expertise to take a multidisciplinary team approach to diabetes management
-
