Introduction
Concern about the decision-making abilities of child protection practitioners, for example, in assessing risk in relation to complex cases, has emerged as a strong theme in child death reviews (Jones, Reference Jones2014; Reder, Duncan & Gray, Reference Reder, Duncan and Gray1993) and inquiries into child protection services (for example, CMC, 2004). Decision making has also become a significant focus for research, as researchers have striven to gain insights into how decisions are made and the factors that influence the decision-making process (Saltiel, Reference Saltiel2015). The role of context in affecting decision making has been found to be important. Significant differences between child protection practitioners in different countries in decisions made have been found in comparative international studies (Benbenishty et al., Reference Benbenishty, Davidson-Arad, Lopez, Devaney, Spratt and Hayes2015). At an organisational level, contextual factors such as high caseloads, a lack of professional support (Saltiel, Reference Saltiel2015) and weakness in organisational practice procedures (Broadhurst et al., Reference Broadhurst, Wastell, White, Hall, Peckover and Davey2010) have been found to increase errors in decision making by practitioners. Decision making may also be affected by the availability of alternatives, such as supportive services (Font & McGuire, 2015).
Individual factors about practitioners, such as their attitude (Davidson-Arad & Benbenishty, Reference Davidson-Arad and Benbenishty2010), length of experience (Drury-Hudson, Reference Drury-Hudson1999) and the particular form of cognitive processes they use (De Bortolli & Dolan, Reference De Bortolli and Dolan2015), have been found to affect decision making. For example, practitioners exhibit confirmation bias in their rationales for decisions, selectively interpreting information to confirm their underlying hypotheses. Child and family characteristics, such as socio-economic background and ethnicity, are also thought to generate a bias in the minds of decision-makers (Bradt et al., Reference Bradt, Roets, Roose, Rosseel and Bouverne-De Bie2015; Chang, Rhee, & Weaver, Reference Chang, Rhee and Weaver2006; Enosh & Bayer-Topilsky, Reference Enosh and Bayer-Topilsky2015). However, interpreting research findings is not straightforward in this complex area. Wittenstrom, Baumann, Fluke, Graham, and James (Reference Wittenstrom, Baumann, Fluke, Graham and James2015) found that ethnicity was an important factor but only under certain conditions that related to specific combinations of both case-level and external factors. Indeed, case factors rather than practitioner factors have been found to be more influential (Graham, Detlaff, Bauman, & Fluke, Reference Graham, Detlaff, Bauman and Fluke2015). Though much of the research about decision making has been conducted in statutory child protection services, it is highly relevant to this research with non-government agency practitioners, as explained below.
One response to concerns about both the process and outcomes of decision making has been the development of tools designed to address ‘inconsistency across decision-makers and the weak ability of human services professionals to predict important outcomes of interest’ (Schwalbe, Reference Schwalbe2004, p. 563). Since the 1980s, tools have been developed based on different forms of risk assessment (Gillingham, Reference Gillingham2006; Rycus & Hughes, Reference Rycus and Hughes2003; Schwalbe, Reference Schwalbe2004). More recently, practice frameworks that promote particular approaches to working with children and families have been developed and implemented, such as Signs of Safety (Munro, Turnell, & Murphy, Reference Munro, Turnell and Murphy2016; Stanley, Reference Stanley and Mills2014). There has, however, been little research that has focused on how child protection practitioners use and regard decision-making tools and practice frameworks in their practice, though research so far suggests that practitioners may not use them as their designers intended (Gillingham, Reference Gillingham2009a). An independent study of how practitioners in child protection services used the Structured Decision-Making (SDM) tools found that practitioners did not use the tools to support decision making (Gillingham & Humphreys, Reference Gillingham and Humphreys2010) and were concerned that they might undermine the development of expertise in new practitioners (Gillingham, Reference Gillingham2009b). Decision-making tools may be one of a ‘range of rationalities’ (Keddell, Reference Keddell2013, p. ii) that influence decision making by practitioners in an NGO and may limit the discretion they can exercise (Høybye-Mortensen, Reference Høybye-Mortensen2015). Quite how and to what extent tools affect decision making is still open for debate (Høybye-Mortensen, Reference Høybye-Mortensen2015). The findings reported in this article, which focus on the decision-making tools that were used by participants, make a contribution to this under-researched area and assist with clarifying areas for future research about the use of tools and practice frameworks.
Considering who is making decisions about the risk of harm to children and in what agency was an important part of the research as there has recently been significant change in service provision for children and families in Queensland. In response to the recommendations of the Carmody Inquiry (Reference Carmody2013), and particularly to growing numbers of children involved with statutory child protection services in Queensland, the Department of Communities, Child Safety and Disability Services has, through the Family and Child Connect initiative, provided funding to non-government agencies to provide a range of services to support families, with the aim of intervening early in children's lives to prevent involvement with child protection services. Consequently, decision making about potentially vulnerable children and families who previously may have been dealt with by child protection services has been shifted to the non-government sector. A key aim, therefore, was to examine the support for decision making provided by non-government agencies to practitioners. In this article, the findings of one part of a research project about the decision making of frontline practitioners working to support children at risk of harm and their families are reported, namely, how participants described their use of decision-making tools and practice frameworks. In the research, 30 practitioners from three agencies, in both metropolitan and regional offices in Queensland, were interviewed.
The Research
In keeping with exploratory research, a qualitative research design was developed to address the research questions (Alston & Bowles, Reference Alston and Bowles2012). The aim was to explore how practitioners made decisions and, as reported in this article, how they used tools and practice frameworks, with the emphasis on capturing meaning rather than measuring usage (Scott, Reference Scott2002). The research was supported by the University of Queensland from a fund for new research initiatives. Ethical clearance for the study was obtained from the University of Queensland Behavioural and Social Sciences Ethical Review Committee.
Recruitment
Access to the recruitment sites was facilitated through existing networks of the project team. These included agencies in metropolitan Brisbane, and the regional areas of Toowoomba (to the west of Brisbane) and Far North Queensland (Cairns and Atherton tablelands). Metropolitan and regional areas were selected to maximise opportunities to examine the influence of geographical, cultural and local service system variations on practitioner judgment and decision making. Agencies A and B provided participants from both regional and metropolitan areas.
The Sample
A purposive sample of 30 frontline practitioners (with more than 12 months experience) in non-government family support agencies in both metropolitan and regional areas of Queensland was recruited. Participants had worked in the industry for an average of 8.4 years with a range of 1–20+ years. The mean age of the participants was 41.59 with a range of 22–62 years. Twenty of the participants worked in a metropolitan area with 10 participants in a regional location. For highest tertiary qualification, 22 participants held a Bachelor's degree (psychology (11), social work (4), arts (1), human services (1), community welfare (1), community development (1), teaching (1), social science (1) and early childhood (1)). Six of these participants (for whom the Bachelor's level qualification was not recorded) had qualifications at Master's level (social work (2), international relations (1), mental health (1), applied linguistics (1) and educational psychology (1)). Three participants held diploma level qualifications (community service (1), welfare (1), counselling (1)) and three participants had no qualifications. Data were missing for one participant.
Interviews
A semi-structured interview was conducted with each participant that involved the collection of demographic information including age, qualification type and length of professional experience in working with vulnerable families. Participants were then provided with two case vignettes involving a medium risk child protection matter, one of which was identified as an Aboriginal family and one in which ethnicity was not identified. Based on a ‘situational awareness’ framework (Flin, O'Connor, & Crichton, Reference Flin, O'Connor and Crichton2008) participants were asked to ‘think aloud’ about the cues they identified in each case vignette, the level of risk they assigned to the cases and the decisions and actions they would take based on their judgment of risk. Specific decisions concerned whether they considered the case suitable for a family support service, how they might begin to assess and intervene in the family and whether they would refer the family to the statutory child protection service.
After participants had completed their reflections on the case vignette, more general questions were asked about organisational support for judgment and decision making in their practice context and how they decide what might be important information to think about when interpreting a referral. In this article, participant responses are reported in relation to the following prompt during this part of the interview: ‘tell me about any assessment or decision-making tools that might be used when assessing a referral?’
Data Analysis
Data were analysed using Braun and Clarke's (Reference Braun and Clarke2006) staged approach to qualitative data analysis. Transcripts from the interviews were read through and responses from the participants to the above prompt were copied and pasted into a single document. The range of tools that participants used was identified according to which agency they worked at (see Table 1 below). The responses were then analysed to identify the key themes about how they regarded and used tools. Care was taken to ensure that within the themes, the range of responses was noted and that contradictory responses were included to illustrate the range (Alston & Bowles, Reference Alston and Bowles2012). Representative quotes from the participants were chosen to illustrate the themes. The analysis was initially conducted by one member of the research team and then checked and amended as necessary by the other research team members.
Agencies and the range of tools used by participants.
Practice Frameworks or Decision-Making Tools?
It is important to note that in the range of tools identified by participants, practice frameworks were included. Practice frameworks can be very different to decision-making tools. Practice frameworks may be considered as guides to practice, to be invoked when thinking about or discussing decisions about action (Gillingham, Reference Gillingham2009a; Stanley & Mills, Reference Stanley and Mills2014). Decision-making tools, such as the SDM tools, are more prescriptive in that they require specific data about a case to be entered and then they provide a clear recommendation (Gillingham, Reference Gillingham2009a). However, practice frameworks can also be prescriptive when they are embedded in an electronic information system and serve to direct and control the types of information recorded about a case and the means for doing so (Cheers, Fernandez, Morwitzer, & Tregeagle, Reference Cheers, Fernandez, Morwitzer, Tregeagle, Kufeldt and McKenzie2011; Gillingham, Reference Gillingham2013). Given that the participants referred to both practice frameworks and tools in their responses, and our interest was in what they used and how, no distinction between them has been made in the reporting of the findings. The differences between different forms of tools and frameworks and their impacts on decision making are topics for future research.
Findings
What Tools?
Our analysis revealed that a wide range of tools were used by practitioners to support decision making. We also found that some practitioners used no formal tools to support decision making. The range of responses to the types of tools used to support decision-making is outlined in Table 1.
As Table 1 shows, excluding one participant from Agency C and three from Agency A (metro), participants said that they used a range of tools to support their decision making about risk to children and when to make a notification about a child. Five main tools were mentioned: the SDM tools, the online Child Protection Guide (CPG), Signs of Safety, the Resilience Practice Framework and the Collaborative Assessment and Planning Framework (CAP). Two-thirds of the participants said that they used the SDM tools to assist in their decision making when assessing levels of risk to children. One-third said that they would also use the online CPG, a relatively new SDM tool that was, according to participants, made available to them some time in 2014. Within Agency A, participants in both the metro and regional sites reported that they would also use the Resilience Practice Framework, with one participant in each site also mentioning the Signs of Safety Framework. Within Agency B, SDM was mentioned at both sites, with the participants at the regional site also mentioning CAP.
The Role of Tools in Decision Making
The following key themes emerged from the analysis of how participants described the role of tools in their decision making when assessing a referral. Selected quotes from the interview transcripts are presented to illustrate these themes and, in places, to illustrate the range of responses.
Focusing practice
Participants described using the tools to focus their attention on matters that needed to be considered when making decisions:
So I think having these tools, it does help you keep getting back on track and asking some of those more difficult questions. (Participant 11)
Focusing attention included using tools to exclude what was considered as less important information about a family:
The structured decision-making stuff is about – it's a guide, I guess, to filter out a lot of the irrelevant stuff, where you actually look at what pieces of information are going to impact on the outcome that you want to get to. . . . rather than – you know, knowing everything about the family isn't necessarily helpful. It's only critical pieces of information that are going to impact on what I'm trying to achieve. (Participant 1)
Objective tools and subjective expertise
There was a range of opinion expressed about whether decisions should be made with reference to tools and/or with practitioner expertise. At one end of the continuum of participant perspectives, tools were used to replace subjective expertise:
Great, because we've all got different expectations and thoughts in our heads about what's safe and unsafe, so those tools are a great guide to us. You can't just leave it up to people's opinions as to what needs to be referred, and what's unsafe and what's not. (Participant 20)
I think having lots of evidence-based tools is really good because again you don't want your own subjectivity to get in the road. . .(Participant 11)
Tools were used to check impressions, change decisions and to challenge subjectivity:
Look it's often usually the other way around. You often think, “Wow, that's really bad,” because I guess you've also got to be careful of your own emotional investment in the case and . . . . not letting emotions cloud it. . . . and it either confirms what you're already thinking about, “Yes, there is risk there. I do need to report it.” Or sometimes you're thinking, “Oh, I probably need to report that.” And it comes out actually it probably is safe and it is just better with trying to get a support service to work with the family. (Participant 3)
So when I've come to this side of working in [family support services], one of those first few cases I got through, I was like, “Oh my God! This family, they need to be reported! I can't believe this is happening.” It was – thank God that I could go and consult the Child Protection practitioner. She'd go, “No, let's put it through the Guide. You'll find that it doesn't need to be reported. It's fine.” So it was that learning curve in the beginning. Thought I knew stuff from working already on the other [side of family support services] – but no. (Participant 20)
Although Participant 20 describes a reliance on tools at an early stage of her career, others referred to the importance of practitioner expertise and the limitations of tools:
Participant: It's the old guide again. But it can't replace your practice wisdom and it can't replace your experience.
Interviewer: So it assists you but it doesn't define the outcome.
Participant: No. I mean, look, tools are tools. They can be manipulated. (Participant 9)
The same participant went further in describing decision-making tools as a guide:
Yes. But obviously with tools that's what they are – they're just tools. They can guide your practice. (Participant 9)
We then – say the family risk evaluation, for example. It goes through different areas and has questions such as “Has the family been involved with the Department of Child Safety? If yes, on how many occasions?” That teases it out – teases information out and the tool guides you to yes, it's safe, unsafe and should be transferred to intensive family support. It just guides our practice. (Participant 9)
Participants also stressed the importance of using professional expertise in combination with using tools, as a guide in making decisions:
And it's usually about the level of risk to the child if we were to delay and monitor the child, all that sort of thing. So, we don't always take the outcome [of using the tool] as the gospel. It's to be used with discretion and professional judgement as well. But, yeah, we're finding them a very helpful tool to have. (Participant 10)
In much the same vein, the following quote illustrates how tools may still be a useful guide, even if expertise is relied on more heavily in some situations:
I think experience, and we also use tools. When we do notifications, if we have concerns we go onto the online Child Protection Guide and it'll direct us as to whether we need to do notifications or we don't. But it's also about using your own common sense, because at the end of the day the online thing is just a tool to guide you. If you feel that there is a real need for that notification to go through, you can override that and send a notification in, just to make the Department aware of your concerns and stuff like that. I guess for me, having tools that you can use to guide you in what you should be doing is really good. (Participant 6)
In addition to professional expertise, intuition was still an important part of the process:
But also really trusting your gut feeling about it as well, and if your gut doesn't feel right, even though those tools mightn't come out with an answer, you know, really exploring why is that not sitting right with you and why do you feel that it could be something else. (Participant 3)
Limitations of the tools
Though participants were mostly positive about decision-making tools, some pointed out their limitations. The potential for inaccuracy in assigning levels of risk in two real cases was described by one participant:
Yeah. That one with the mum sleeping on the floor with all the kids so, to protect them, that, you know, even when I overrid it, it came up as moderate, when it should have come up as high. (Participant 29)
And the other one that the risk is really minimal, came up as so high, like, you know, it doesn't really reflect. (Participant 29)
Another was concerned that the need to categorise people according to pre-determined definitions tended to oversimplify the complexity of the situations thay had to deal with:
It is that they want us to do it but if it [the assessment process] takes six months it takes six months. You can't force people into boxes. (Participant 11)
Another explained how the SDM tools could not always be applied to situations, with reference to their focus on parental behaviour as the key factors in protection or potential harm:
I don't know if I 100% agree with the SDM tools and all of that, because when you're talking about okay, that I guess like the child is extremely violent, that's not actually reflected in the SDM tools, it's all about the parents, so if parents are acting protectively, and one I worked with, I did the SDM, it came back as I can – and I override it as well, overrid it to make it higher and it's still only came up as medium, and it should have been high, because even though mum isn't really – at the end of the day I guess she is part of the problem, but if mum is sleeping in the lounge room with kids to protect them, like that's not reflected in the SDM. So if the child's violent, it's not reflected, so I don't agree with it. (Participant 29)
Single or multiple tools
Only one participant described using a single tool as being one way to ensure consistency in practice, specifically in relation to the language used within and between agencies to articulate problems and solutions:
So that when two services are involved with the one family, we're using the same language so the families then become familiar with those words. For example, worries. You know, we don't say – try not to say “concerns” because then, you know, try and deter from that by using the word “worry”, for example. “We're worried about this. You know, what are your worries?” So those kind of documents or framework around particular questions around protective factors and behaviours and goals and things like that kind of helps us get the right information and – and making sure that – excuse me – that we stay on track, you know, and keeping it focussed on what we're there for and addressing those – those issues or those worries that's been referred to us. So that we put the right supports in place. (Participant 30).
Although consistency in practice could not be described as a strong theme in the data, this comment is pertinent to debate about the use of tools (see below). Consistency in decision making was a key aim of the implementation of the SDM tools in the Department of Child Safety in Queensland in 2006 (see Gillingham, Reference Gillingham2009a), tools which have, as this research has shown, been made available to practitioners in non-government agencies.
In contrast, another participant articulated the need to use multiple tools, which is reflected in Table 1.
Yes. We do have a lot of different tools, and there's no one right way. Our organisation doesn't really force anything in particular. (Participant 19)
Discussion
In summary, for the participants in this research, using tools and practice frameworks was an important part of their practice in making decisions about levels of risk to children and modes of intervention with families. Tools were not mandated for use in any of the agencies and were positioned by most practitioners as a guide to their practice, to be used to supplement their own expertise. Mostly, practitioners were positive about using tools, in many cases, a range of tools, to assist their decision making. Such conclusions remain tentative though due to the limited nature of this part of the research. More significantly, areas for further research and broad discussion have emerged from the findings.
As mentioned in the introduction, there has been little research about how practitioners use tools and practice frameworks in their decision making. There is a need for more in depth research about how tools are being used generally, but also, now, as the context for their use has changed in Queensland in non-government agencies. The use of practice frameworks in non-government agencies is not new, of course, but they are now dealing with children and families who, before the Family and Child Connect initiative, would have been more likely to be provided services at the tertiary rather than voluntary end of the service continuum. Practitioners in non-government agencies are using the same risk assessment tools as child protection practitioners and are making decisions about the risk of harm to children and whether a notification to statutory services is necessary. Further research is required to explore how shifting the responsibility for these decisions to the non-government sector might change patterns of notifications and, more generally, how it might affect the safety of children.
Demographic information about the participants was collected. Further analysis of the findings, though beyond the scope of this article, will reveal whether differences in how tools and practice frameworks are used exist between participants according to their professional qualifications and length of experience. Previous research suggests that more experienced practitioners are less dependent on tools and practice frameworks than those with less experience.
A further rationale and direction for research about the use of tools in non-government agencies is the finding that different tools are being used in different, and the same, agencies. However, as mentioned in the introduction, all practitioners in agencies that receive funding from the Family and Child Connect initiative will be required to use the Strengthening Families Protecting Children Framework (SFPCF). The SFCPF is based on key elements of the Signs of Safety Approach and contains a detailed Collaborative assessment and planning framework. According to the department's website, this framework is intended to be used in ‘partnership with children, young people, their family and networks to undertake a balanced and comprehensive assessment of harm, risk and safety, and to collaboratively identify goals and action steps to build future safety, belonging and wellbeing for a child’ (DCCSDS, 2016). Being restricted to using only one practice framework could create a dilemma for some practitioners, who, as found in this research, might prefer to use a range of tools to support their decision making. A key question for further research in this area is whether practitioners might also choose to use other tools, in addition to the SFPCF, and whether one facet of expertise is the ability to be able to choose the right tool for a specific situation. Within such research, there is also the more general question of how practitioners regard and use the SFPCF. The Department of Communities, Child Safety and Disability Services plans to use a form of appreciative inquiry (DCCSDS, 2016) to support the implementation of the SFPCF that will address this question to some extent, but will be limited by the focus within an appreciative inquiry approach on the success of implementation, which, for example, seeks to address rather than explore and evaluate matters such as non-compliance with using the framework (see Whitney & Trosten-Bloom, Reference Whitney and Trosten-Bloom2010). As the Department has made clear, it aims to continue to work with the developers of the SFPCF for a three-year period to ‘identify opportunities to challenge and align all departmental practices with the framework. . .’ (DCCSDS, 2016). There is a need for scholarly research that is independent from this agenda and that can then contribute more broadly to debates about whether implementing single practice frameworks are the best way to support decision making. Such research would be pertinent to developments in services for children and families nationally and internationally, such as the implementation of the Signs of Safety approach in child and family support services across England (see Munro et al., Reference Munro, Turnell and Murphy2016) and New Zealand (Keddell, Reference Keddell2014) and the Looking after Children framework in Australia (Cheers et al., Reference Cheers, Fernandez, Morwitzer, Tregeagle, Kufeldt and McKenzie2011; Gillingham, Reference Gillingham2016).
Conclusion
A limitation of the research presented in this article is that, with hindsight, the role that tools and practice frameworks played in supporting participants in their decision making about children at risk of harm was underestimated as participants did find a range of tools and frameworks useful to support their expertise. Further research is required to investigate in more depth how practitioners use tools, the relationship between tools and expertise and how and why some tools may be more or less useful than others when assessing risk and making decisions about whether referral to statutory child protection services is required. Such research might consider both the process of implementation (Mildon & Shlonsky, Reference Mildon and Shlonsky2011) and outcomes, for example, in terms of how decision-making tools and practice frameworks might affect referral rates, the types of intervention being offered to families and the outcomes for children and families. Further research in this area is pertinent to both Queensland and internationally as jurisdictions choose to implement single practice frameworks to improve practice with children and families.
Acknowledgements
This research was financially supported by a University of Queensland Faculty of Health and Behavioural Science Collaborative Research Seeding Grant. The authors would like to thank the staff in the agencies for their participation in the research.
