Background
Literacy is an important skill for children because it helps them cope with the demands of life, school and work. A high level of literacy is associated with completion of Year 12, post-school study and improved likelihood of employment (Kutner et al., Reference Kutner, Greenberg, Jin, Boyle, Hsu and Dunleavy2007; Nguyen, Reference Nguyen2010). Literacy is also an essential pre-requisite for active and equitable participation by citizens in social, cultural, political and economic life (OECD, 2001).
Given the importance of literacy, it is concerning that one in five Australians do not have the necessary literacy skills to effectively participate in daily life, and are at risk of social exclusion and economic disadvantage (OECD, 2000). Children living in foster care are at particular risk in this context. There is growing evidence that a considerable proportion of the more than 35,000 children living in foster care in Australia are not meeting the national benchmarks for literacy and numeracy (Australian Institute of Health and Welfare, 2011). Compared with their peers, these children are more likely to score lower on standardised tests and report higher levels of absenteeism, tardiness, truancy, and school dropout rates (Christian, Reference Christian2003). Research suggests that the poor educational outcomes of youth in foster care can jeopardise their future by lowering their educational aspirations and increasing the risk of substance abuse and long-term unemployment (Australian Institute of Health and Welfare, 2011; Harvey & Testro, Reference Harvey and Testro2006; Jones, Reference Jones2010; McDowall, Reference McDowall2009; Shin, Reference Shin2003).
The most effective way to improve literacy has been the subject of much debate for many decades and there is still no simple answer about what constitutes effective literacy teaching and learning strategies (Smarter Schools National Partnership, 2010). However, there is ample research demonstrating that reading aloud to young children promotes the development of language and other emergent literacy skills (Duursma, Augustyn, & Zuckerman, Reference Duursma, Augustyn and Zuckerman2008).
The importance of reading aloud to children is promoted by children's author and literacy expert, Mem Fox. Fox is an advocate for reading books aloud to children, but she stresses that it is not only reading books that is important, but also the relationship between the child and the person reading the book to them. She states that it is the loving interaction occurring when reading aloud to children that teaches them, gives them security and makes literacy fun rather than a challenge or a chore (Fox, Reference Fox2006).
Further research confirms that reading aloud to children does not by itself appear to be the magic formula to increase children's literacy skills (Meyer & Wardrop, Reference Meyer and Wardrop1994). The positive nurturing relationship between the adult and child and how well the adult engages the child in the process of reading and understanding what they are reading are equally important. Ideally, reading to a child entails a mentoring relationship, where a healthy, close and trusting relationship is developed to support a child to learn and mature. Research suggests that enduring mentoring relationships are associated with a range of benefits to youth (Grossman & Rhodes, Reference Grossman and Rhodes2002). Mentoring can create a positive relationship between an adult and a child that can improve academic functioning (Zimmerman, Bingenheimer, & Notaro, Reference Zimmerman, Bingenheimer and Notaro2002), relationships, life skills, and rule-governed conduct in addition to overall self-worth (Rhodes, Grossman, & Resch, Reference Rhodes, Grossman and Resch2000). Importantly, studies have revealed that when a child perceives their mentor to be an authentic, empathic companion, a high-quality mentoring relationship can develop which provides psychological benefit to the child being mentored (Thomson & Zand, Reference Thomson and Zand2010).
While at-risk youth are quite often participants in mentoring programmes and accompanying research, only a few studies (e.g. Mech, Pryde, & Rycraft, Reference Mech, Pryde and Rycraft1995; Rhodes, Haight, & Briggs, 1999; Traussig & Culhane, Reference Taussig and Culhane2010) focus specifically on children in foster care. This might be because of the limited number of mentoring programmes available to foster children, or because developing warm and positive mentoring relationships with children in foster care is not easy to achieve as these children often have a history of abuse, rejection and loss (Rhodes et al., 1999). They have often experienced physical and emotional trauma, exposure to violence and frequent changes in foster placement. These experiences place children at greater risk of developing physical, emotional and behavioural disorders that interfere with learning (Christian, Reference Christian2003). Healthy adult attachments are essential for any child, but the stability and reassurance of caring adults is particularly important for children in foster care to ensure their self-efficacy and self-worth and promote strong social and emotional functioning (American Academy of Pediatrics, 2000).
The challenges of mentoring children in foster care confront volunteers who become Pyjama Foundation Love of Learning mentors. They are given the job of engaging with a child in foster care, and then reading, playing and learning with them to help increase their literacy skills. The programme is unique in Australia in its use of volunteer mentors and focus on developing the literacy skills of this particular disadvantaged group. The present study is the first step towards a comprehensive evaluation of the effectiveness of the programme. It focuses specifically on perceptions of mentors about their relationships with the children during the programme, and is concerned with the effects of the programme on the children. Further research will need to combine this work on the mentors’ perspectives with objective measures of changes in the literacy outcomes for children participating in the programme in order to gain a full picture of the impact of the programme.
The study
The Love of Learning Programme is an early intervention programme initiated in 2004 by The Pyjama Foundation aimed at improving the literacy outcomes of children who live in foster care. The programme involves more than 1000 volunteer mentors, called Pyjama Angels, visiting children in foster care to develop their literacy, numeracy and learning skills through weekly visits to read books, play games and engage in other learning-based activities.
The primary goal of this study is to assess the effectiveness of the programme and analyse the major factors that impact on the child's literacy improvement and engagement in the learning process. In particular, the study explores the mentors’ perceptions of the impact that a warm and positive mentoring relationship with the children (including engaging a child through reading and educational play) has on the children's learning progress and improvement of their literacy skills. Given that previous research suggests mentoring is more effective the longer the relationship has been established (Rhodes, Reddy, Roffman, & Grossman, Reference Rhodes, Reddy, Roffman and Grossman2005; Slicker & Palmer, Reference Slicker and Palmer1993), survey data in this study were also analysed to determine whether the outcomes are more favourable for children who were mentored for longer. The gender and age of mentors was also measured as an indicator of performance. The study is based on reports from Love of Learning mentors (‘Pyjama Angels’) gathered via responses to a questionnaire. The study did not include objective measures of the children's literacy outcomes, so its results are limited to the mentors’ perceptions. It should, therefore, be viewed as a preliminary assessment of the Love of Learning programme, which will need to be supplemented by further research that compares the mentors’ reporting with standardised literacy assessments and data on the perceptions of the children, foster carers, and case managers.
Method
Participants
The study included a random sample of 121 mentors in the Love of Learning programme who completed a questionnaire that was sent to 180 mentors. This represents a response rate of 67%. Of the group participating in the study, 91.7% were female (n = 111) and 8.3% were male (n = 10). The participants were aged from 18 years to over 75 years, with the largest number of angels (n = 33) being between 56 and 65 years old.
The mentors in this study were all mentoring a child between 2 and 17 years of age. In the case of mentors who were mentoring more than one child, data used was that relating to the child they first mentored. The mean age of the children within the programme was 7.6 years with a standard deviation SD of 3.5 years. 43.8% of the mentors had been working with the children between two and four years, and 33.9% of the mentors had been working with the children for between six months and one year.
The majority of the mentors (97.5%) said they met with their child at least twice a month (66% visit once a week) and 82.5% reported that they spent most of the time or all of the time undertaking educational activities (such as reading, puzzles and educational games) with the children. Most mentors (58.5%) spent between one and two hours with their child each meeting.
Measures
The questionnaire included questions about mentors’ gender, age, length of volunteering experience, motivation to volunteer, role clarity and confidence to conduct their mentoring role. The questionnaires were sent to the mentors by e-mail in November 2010. To investigate the strength and effects of the mentoring relationship, three measures were designed and incorporated into the questionnaire, which were based on recent mentoring literature and analysis of measures used to quantify youth mentoring relationships (e.g. Big Brothers Big Sisters of America, 2011; Jucovy, Reference Jucovy2002; Rhodes et al., Reference Rhodes, Reddy, Roffman and Grossman2005).
The first measure (three items in the questionnaire) was used to assess ‘relationship’, that is, the mentors’ perception of how supportive and positive the relationship was between the mentor and the child. The resultant scores for the relationship variable used in the analysis below were derived as average scores over the three related items in the questionnaire. The second measure (four items in the questionnaire) was used to assess ‘engagement’, that is, the mentor's perception about how engaged and interested the child was when working and talking with the mentor. The third measure (four items in the questionnaire) was used to assess ‘literacy improvement’, that is, the mentor's perception about whether the child had demonstrated any improvement to literacy, speech, comprehension and confidence since the mentor had been visiting and mentoring the child. The respective scores for literacy improvement and engagement were obtained by averaging the scores over the four items in the questionnaire relevant to each of these measures.
The designed measures asked the mentors to rate the statements in the respective questionnaire items on the five-point Likert scales from 1 (strongly agree) to 5 (strongly disagree). For the purposes of analysis, the scores were inversed to a new scale from 1 (strongly disagree) to 5 (strongly agree). Cronbach's alpha reliability coefficients of the three measures were 0.76, 0.77 and 0.83, respectively, which is an indication of good internal agreement between the raters (participating mentors).
A total of 95.8% of mentors reported that they had a supportive and positive relationship with the child they mentored (with the overall mean relationship score of 4.23 and SD = 0.61). According to the mentors’ reports, 64.1% of the children being mentored were actively engaged with the programme (with the overall mean engagement score of 3.43 and SD = 0.82); and 74% of mentors agreed that, overall, they had seen the literacy skills of the foster children improve while they had been mentoring them (with the overall mean literacy improvement score of 3.67 and SD = 0.71).
Statistical methodology
Statistical analysis was conducted using the R statistical software package to determine and understand the dependences between the following variables: the three considered measures (i.e., the relationship, engagement, and literacy improvement variables), tenure in the programme (tenure), frequency of meetings, mentors’ age and gender, and children's age. The preliminary analysis of the data revealed significant mutual correlations between relationship, engagement, literacy improvement, tenure and frequency of meetings. Therefore, to understand the complex mutual relationships between these variables, structural equation model (SEM) was used (Kline, Reference Kline2011). This model is particularly useful in the case of multiple variables where there is insufficient initial knowledge about which of these variables is capable of influencing the other. SEM allowed identification and quantification of possible pathways for mutual influences of the involved variables. For example, using this approach, we were able to find that relationship influences literacy improvement directly and indirectly through its influence on the engagement variable.
Subsequent multiple regression analysis was undertaken for the variables that had significant mutual impacts on each other. As a result, two different models were proposed and investigated. In the first model, the impact of the relationship, engagement and tenure variables on literacy improvement was analysed. The second model investigated relationship, tenure, and frequency of meetings as the factors influencing engagement. This statistical approach enabled simultaneous consideration and determination of the impact of several variables on the two major variables – engagement and literacy improvement – and also determined the relative importance of the contributions of different variables (factors). Important relationships leading to practical recommendations for more effective literacy improvement were identified and studied by this methodology.
Results
The preliminary investigation and analysis of the available data (using the Shapiro-Wilk test) revealed that the literacy improvement, relationship and engagement variables were distributed normally; therefore t-tests and anova-tests were used to evaluate any possible relationships between the mean values of these variables. For example, using the t-test, it was demonstrated that the average scores for the relationship, engagement and literacy improvement variables did not display statistically significant differences associated with the gender variable. Similarly, anova-tests showed the absence of statistically significant dependences of these variables on mentor age. No significant correlations were found between child age and any of the three variables – relationship, engagement and literacy improvement.
At the same time, all the major four variables – relationship, engagement, literacy improvement, and tenure in the programme – displayed statistically significant mutual correlations (Table 1). The presence of a large number of mutually correlated pairs of variables presents a problem for a simple regression analysis, so SEM was used to identify and investigate possible pathways for the mutual influences between the variables in the set. Figure 1 shows the calculated diagram with the determined structural relationships between the variables. The values of the standardised regression coefficients between the pairs of variables are indicated next to the arrows connecting the variables. All regression coefficients shown in Figure 1 are statistically significant. The goodness of fit test showed excellent fit of overall model (the standardised residuals root mean square is equal to 0.012 < 0.05, the conventional cut-off, and the comparative fit index and Tucker-Lewis index are both equal to 1). The value of the coefficient of determination for this model is about 40% (i.e., ~ 40% of the variance of the literacy improvement variable can be explained by the considered model). This shows that there are other factors (socio-economic, environmental, psychological, physiological, etc.) that could have an impact on the literacy ability of a child and his/her engagement in the process of learning. Though it is very important that the considered major factors (variables) have a significant effect on literacy improvement (as they are responsible for 40% of the improvement), future analysis of other factors is essential.
Correlations between pairs of the four major variables.
Structural equation model.
Note: The asterisks indicate the levels of significance for the respective regression coefficients: (*) p < 0.05; (**) p < 0.01; (***) p < 0.001.
The structural equation model diagram in Figure 1 identifies (by the solid arrows) direct dependences between the variables (shown in the boxes). The numbers next to the solid arrows show the corresponding regression coefficients between the respective variables in the boxes. Direction of the solid arrows indicates the direction of the mutual influence of the variables, and thickness of the solid arrows is approximately proportional to the values of the regression coefficients. The dashed arrow shows the statistically significant (p = 0.03) indirect effect of the relationship variable on literacy improvement through engagement.
Two different types of processes, or mutual effects between the variables, are shown in Figure 1. First, there are direct effects between pairs of variables shown by solid arrows in Figure 1. For example, the relationship variable has direct impacts on literacy improvement and engagement (see the solid arrows with the regression coefficients 0.41 and 0.45 in Figure 1). Second, there are indirect effects between the variables where one variable (relationship) influences another variable (literacy improvement) through a third variable (engagement), shown in Figure 1 by the dashed arrow. It has been calculated that the indirect effect of the relationship variable on literacy improvement through engagement was statistically significant (p = 0.03) and constituted about 17% of the total effect of relationship on literacy improvement. It has also been shown that, although the direct effects of the frequency of meetings and tenure variables were statistically significant, their indirect effects on literacy improvement through the engagement variable were not significant (p = 0.1 > 0.05 for frequency, and p = 0.06 > 0.05 for tenure).
The diagram shown in Figure 1 demonstrates that there are two identifiable sets of variables affecting other variables in a statistically significant way: (1) relationship, tenure and engagement affecting literacy improvement, and (2) relationship, tenure and frequency of meetings affecting engagement. As a result, two multiple regression models were considered to investigate the mutual influence of these sets of predictor variables on the respective dependent variables of literacy improvement and engagement. The results of this analysis are presented in Table 2. Standardised predictor variables were used in the considered multiple regression models. This means that the values of the regression coefficients in Table 2 are proportional to the contributions of the respective predictor variables to the variation of the dependent variable (which is called size effect for the predictor variable). In particular, it can be seen that in the first model the greatest size effect on literacy improvement was the relationship variable, which demonstrates the dominant impact of relationships between the mentor and the child for the fast improvement of literacy skills. The corresponding p-values demonstrate the very high statistical significance of this outcome. Similarly, relationship has a major and significant impact on the engagement of the child in the learning process (see the first row for model 2 in Table 2). In this second model, frequency of meetings also seems to have a significant effect on engagement, although the statistical significance of this result is somewhat lower than for relationship. Tenure in the programme also seems to be important for the development of active engagement of children in the learning process.
Regression coefficients in the two considered multiple regression models with standardized variables corresponding to the effect size, p-values, and 95% confidence intervals (CIs) for the regression coefficients.
Figure 2(a) shows the dependence of the literacy improvement score on relationship score (thick solid line) together with the 95% prediction interval (shown by the thin curves). Figure 2(b) shows the dependences of the engagement score on relationship adjusted for tenure (lines 1, 2 and 3) in the programme and frequency of meetings: (1) 4 year tenure; (2) 2.5 year tenure (mean tenure); and (3) 1 year tenure. All of these curves are for the frequency of at least one visit per week. Line 4 is the same as line 2, but for frequency less than one visit per week.
(a) The dependence of the literacy improvement score on relationship score (b) The dependences of the engagement score on relationship adjusted for tenure.
Figure 2 provides graphic illustrations of the effects of the variables on literacy improvement and engagement. For example, it can be seen that literacy improvement significantly and strongly increases with increasing relationship score (Figure 2a). This, once again, demonstrates the important role of relationship for the perceived fast progress of children's learning. Figure 2(b) demonstrates that not only the child's engagement in learning strongly depends on relationship, but also that increasing tenure causes a significant increase in engagement (compare curves 1, 2 and 3). Similarly, increasing frequency of meetings results in a substantial increase in the child's engagement (particularly at low levels of relationship – compare curves 2 and 4). The relative impact of frequency or tenure on engagement reduces with increasing relationship score; that is, from the viewpoint of the child's engagement, good relationship tends to compensate for the rarity of the meetings and short time in the programme (i.e., relationship appears to be dominant). Nevertheless, these outcomes show additional options for enhancing learning experiences, engagement, and thus efficiency of learning by increasing tenure in the programme and frequency of meetings. Future studies in this direction will be important to further investigate the possibility of the indirect effects of tenure and frequency on literacy improvements (which did not appear statistically significant in this study as show in Figure 1).
Table 3 presents the predicted (from the calculated dependences in Figure 2) levels of perceived literacy improvement as a function of each of the three major predictor variables, adjusted for the effects of the mean values of the other two variables. The effect of each of the considered three variables was adjusted for the effects of the indicated mean values of the other two variables. The frequency of meetings was assumed here to be at least once per week. Once again, an increase in the relationship score had the greatest impact on literacy improvement, which increased almost two times with an increase of the relationship score from 1 to 5 (Table 3; see also Figure 2a). According to the calculated 95% CIs, all the obtained results in Table 3 are statistically significant.
The modelled (predicted) mean values for the literacy improvement scores corresponding to the main considered variables.
Discussion
The key aim of The Pyjama Foundation Love of Learning Programme is to improve the literacy skills of children who live in foster care. The current study focused on the perceptions of volunteer mentors participating in the programme about their mentoring relationships and the influence of these relationships on improving the children's literacy skills. The results identified several important factors for faster and more efficient improvement of children's literacy skills. In particular, a good and positive mentoring relationship between the mentor and the child appears to be the dominant factor for faster and more efficient learning. A good relationship with a child has been found to positively influence the learning process directly and indirectly through enhancing the child's engagement in the process.
Other important parameters affecting perceived literacy improvement include the child's engagement in the learning process, tenure in the programme, and frequency of meetings. In particular, it has been demonstrated that increases in all or some of these factors result in significant improvements in perceived literacy skills. However, the analysis of the impact of frequency of meetings on literacy skills should only be regarded as preliminary because of the rather crude subdivision into only two categorical options: at least one meeting per week and less than one meeting per week. Nevertheless, even such an approximate approach has revealed the strong influence of the frequency factor on the literacy progress of the children. Although the considered model, based on the SEM analysis, did not identify statistically significant indirect effects of the tenure and frequency variables on literacy improvement, the existence of such effects could still be determined in future research aimed at clarifying these issues (e.g., by means of specially designed survey tools).
In addition, an obvious limitation of the current study is that the assessments of mentor relationships and the child's literacy skills were based solely on the mentors’ perceptions at the time they completed the survey. Future research could compare the mentors’ reporting with standardised literacy assessments and perceptions of the children, foster carers, and case managers. This additional data would reduce the risk of response bias and other types of subjectivity in the analyses. Additionally, the current study exclusively focused on the mentoring programme and did not explore or allow for other factors that could have contributed to the children's improvement in literacy. For example, some of the perceived improvement in the children's literacy levels may be attributable to natural progression of reading skills over time rather than to the mentoring programme. Furthermore, some level of reporter bias could be expected as mentors who are emotionally and intellectually engaged with the programme might be expected to report improvement in literacy levels as a response to that engagement as much as to actual evidence of improvement.
The study has important implications for both research and practice. The Pyjama Foundation is making a serious commitment to developing a quality mentoring programme for children in foster care and can be encouraged by the results of the study which indicate that foster children, who often have a history of abuse and difficulty establishing close and supportive relationships with adults (Carlson, Cicchetti, Barnett, & Braunwald, Reference Carlson, Cicchetti, Barnett and Braunwald1989), can benefit from having an adult in their life who cares about their personal and educational development and achievements. The results also suggest that mentoring programmes should recruit volunteers who are committed to mentoring a child for the long term, and that these programmes need to train and support volunteers to develop trust and rapport with children who have poor attachment and social skills. Other improvements to the programme could be introduced, including increased frequency of meetings in the programme, and other factors that enhance the child's engagement in the programme. Further research could help determine whether the Love of Learning Programme is having any other direct or indirect outcomes, such as improving the children's mental and social health and improving their attitude towards adults and relationships.
