INTRODUCTION
The study of faith-based initiatives continues to be an important research topic in the field of religion and politics. The term “faith-based initiatives” refers to a new way of approaching religiously-based organizations that are providing social services, by means of strengthening such services through the use of partnerships between government agencies and the religiously-based organizations (usually referred to as faith-based organizations, or FBOs). These initiatives seek to lower the barriers between religion and government, making it possible for faith-based organization to maintain their religious nature while using public funds to deliver social services.
Scholars of religion and politics studying these initiatives have focused on three different facets of the issue. Some analysts have approached the topic primarily in terms of issues related to the separation of church and state (Monsma Reference Monsma1996; Lupu and Tuttle Reference Lupu and Tuttle2007). Other analysts have examined the topic in terms of how advancing such programs may relate to generating public support for election to public office (Black, Koopman, and Ryden Reference Black, Koopman and Ryden2004, 11–14; Kuo Reference Kuo2006), and still others have examined the issue in terms of the relative effectiveness of faith-based versus secular programs in terms of their delivery of social services (Johnson Reference Johnson2003; Monsma and Soper Reference Monsma and Soper2006; Fischer Reference Fischer2008).
This study addresses this last issue. It seeks to test empirically the comparative effectiveness — defined in terms of positive client outcomes — of FBOs and their secular counterparts. Ever since President George W. Bush's faith-based initiative focused attention on faith-based organizations as providers of social services, proponents and critics of faith-based initiatives alike have made claims related to the relative effectiveness of these different programs, with proponents of faith-based programs touting the greater effectiveness and critics claiming the opposite (Berrien, McRoberts, and Winship Reference Berrien, McRoberts, Winshop, Bane, Coffin and Thiemann2000; Bush Reference Bush2001; Wineburg Reference Wineburg2001; Towey Reference Towey2002; Kennedy and Bielefeld Reference Kennedy and Bielefeld2006).
Still, even as late as 2001 when Bush launched his faith-based initiative, there were almost no empirical studies of the outcome effectiveness of different social service programs — whether faith-based or secular in nature. As a result, most claims of faith-based outcome successes (or failures) were based simply on anecdotal accounts usually made by advocates of one policy position or another. However, prompted by the debates that have raged over the use of FBOs to provide important social services, some empirically-based, comparative studies of service program outcomes have begun to emerge.
This study therefore engages in a comparative analysis of the positive outcomes of faith-based and secular endeavors related to a federally-funded program of intervention for adjudicated and at-risk Latino youths. It not only seeks to contribute to the relatively meager body of literature related to such comparative analyses, but it also endeavors to break new ground in terms of identifying potentially important causal factors that may serve to shape the programmatic success of faith-based and secular social service agencies.
COMPARATIVE ASSESSMENTS OF PROGRAM OUTCOMES
Research related to analyses of public policy programs comparing faith-based and secular programs is still largely in its infancy. Not surprisingly, therefore, there has been considerable diversity in the methods utilized, the policy areas analyzed, the nature of the faith-based human service programs examined, and the findings obtained. Moreover, these initial comparative empirical studies are marked by limitations inherent in any research endeavor that seeks to address very difficult-to-answer questions. As a result, there are few, if any, established generalizations with regard to differences or similarities in client outcomes between faith-based and secular delivery systems.
Nevertheless, based on a review of these initial comparative analyses, there seems to be enough evidence to put forward the following tentative generalization: namely, that faith-based programs, when compared to secular programs, tend either to achieve slightly more successful outcomes overall or to demonstrate mixed results (by exhibiting more successful outcomes by some measures and less successful outcomes by other measures). For example, a study of Los Angeles welfare-to-work programs found that clients in faith-based programs had about as much success as clients in secular programs in finding employment, but did somewhat better at retaining their jobs (Monsma and Soper Reference Monsma and Soper2006, 158). A study of cocaine-dependent, African-American women, some of whom were randomly assigned to a faith-based mentoring and group activity program and some to a similar, but secular, program, found no significant difference in abstinence after three months, though the FBO program participants had significantly higher rates of abstinence after six months (Stahler, Kirby, and Kerwin Reference Stahler, Kirby and Kerwin2007).Footnote 1 Similarly, a recent study of employment programs in Los Angeles County concluded: “Clients of FBOs are between 5.3 to 16.2 percentage points more likely to successfully secure a job after exit compared to those of non-FBOs” (Briggs Reference Briggs2007, 19); on the other hand, having a faith-based provider did not increase one's chances of earning more money, retaining employment, or achieving employment-related credentials (Briggs Reference Briggs2007, 25). Likewise, a study of recidivism rates among released prisoners compared the recidivism rates of those who had taken part in a prerelease, in-prison, faith-based program with a non-participating group of prisoners exhibiting similar characteristics as those who took part in the program (Johnson Reference Johnson2003). Two years after release from prison, those prisoners who had completed the faith-based program had lower recidivism rates; the non-participating, matched group of prisoners had more than twice the percentage that had been rearrested and nearly three times the percentage had been re-incarcerated (Johnson Reference Johnson2003, 17, 19).Footnote 2 Finally, Robert Fischer (Reference Fischer2008) has analyzed and summarized the findings of 18 existing empirical studies that utilized a comparative approach to study program outcomes, and he concluded that “the overall effect of Faith-Based and Community Organization (FBCO) programs, although modest in size, demonstrates that these programs tend to produce somewhat better outcomes compared with usual services, secular services, or no special programming” (Fischer Reference Fischer2008, 190). Thus, the spectacularly more positive outcomes of FBOs in comparison to government-run programs that are often cited in anecdotal accounts largely evaporate when comparative, more systematic and empirical, analyses are done.
To say that this field is in need of more research is an understatement. It is plagued not only by the limited number of studies that have been conducted, but also by methodological problems inherent in a field that is difficult to research. These challenges have been described elsewhere (Monsma and Soper Reference Monsma and Soper2006, 7–13; Grettenberger, Bartkowski, and Smith Reference Grettenberger, Bartkowski, Smith, Boddie and Cnaan2006). In particular, the problem inherent with almost all policy studies in the field is that it is either very difficult or simply impossible to impose the kinds of controls needed to assess accurately the causal properties of the particular programs under study (e.g., the ability to make random assignments of program participants to faith-based and secular programs). Moreover, when considering the performance of FBOs, the size and nature of the FBOs vary greatly. Some are multi-million dollar budget, professionalized organizations; others are small, struggling organizations operating largely with volunteers. Some FBOs are faith-based in an explicit, integral manner, while the faith-based nature of other FBOs is largely implicit in nature and less direct in presentation (Unruh and Sider Reference Unruh and Sider2005). And the human service programs provided by FBOs are extremely wide and diverse — from prisoner release programs to welfare-to-work efforts, from weight-loss programs to addiction-treatment programs. These enormous differences in the organizational nature and programs provided under the rubric of “faith-based” programs make the development of theory and the advancement of generalizations related to faith-based programs difficult.
Nevertheless, this study seeks to advance the study of the comparative effectiveness of FBOs and their secular counterparts by reporting on the outcome effectiveness of faith-based and secular intervention programs for adjudicated and at-risk Latino youths in five western cities.Footnote 3 More specifically, it seeks to test the following hypothesis: Faith-based intervention programs will have more positive — but only slightly more positive — outcomes than their community-based counterparts.
This study has several qualities that distinguish it from earlier studies and make its findings more robust. First, our study focuses on the programs and clients involved only in faith-based and community-based organizations; in other words, it does not include large government-run programs or programs administered by large, highly professionalized, secular agencies. As a result, different outcomes between the faith-based and secular programs can more readily be attributed to their declared faith-based or secular nature, since all the organizations under study are similar in size and professionalism as well as being similarly rooted in the communities they served.Footnote 4 In most other studies, the comparison of faith-based and secular program outcomes are marred by many organizational and program differences beyond their faith-based or secular natures.
Second, as we will explain shortly, our study also had the advantage of having personal background data on each of the 2,748 youths who participated in the program, their individual levels of involvement in the programs in which they participated, and their individual levels of accomplishment. Thus, not only does this study analyze a far greater number of cases than is typically evident in most previous outcome studies, but the data are organized at both the individual, and the aggregate level.
And, finally, our study suggests that a variable not previously considered in outcome studies may explain outcome successes and failures better than any particular differentiation between the faith-based versus secular programs may provide. We identify this variable in our analysis below and offer evidence in support of its importance.
Before we begin the analysis of the data, however, we will first provide a description of the program and how we engaged in our study of it. After presenting this contextual information, we then present our findings. Then in the final section of the article we draw several conclusions from our findings and advance several public policy implications that derive from our study.
THE CONTEXT
In June 2004, the Latino Coalition for Faith and Community Initiatives of Bakersfield, California (hereafter called simply the Latino Coalition) was awarded a three-year, $10 million grant by the Department of Labor (DOL) to fund its Reclamando Nuestro Futuro (Reclaiming our Future or RNF) program. This programmatic effort was designed to assist at-risk and adjudicated Latino youths to obtain needed education or training, find employment, and avoid involvement with the juvenile justice system. The Latino Coalition was founded in 2003 as a 501 (c) 3 nonprofit organization, with Richard Ramos, an ordained pastor, as president (Richard Ramos and Richard P. Morales, September 20, 2007, Personal interview by authors Baylor Institute for Studies of Religion 2008). Its basic goal was “to enhance and strengthen the capacity of FBCOs to transform Latino youth, families, and communities” (Baylor Institute for Studies of Religion 2008, 5). For the most part, the Latino Coalition was not a direct provider of services, but rather an intermediary organization working with, and empowering, direct service organizations. Thus, its role in the RNF program was to serve as an intermediary organization, charged with recruiting, training, and funding sub-grantee organizations working with Latino youth.
Programmatic Stipulations
The DOL requirements stipulated that at least 80 percent of the youths served by each RNF sub-grantee had to be Latino and that all youths enrolled in the various programs had to be either at-risk or adjudicated youths between the ages of 14 and 21. Adjudicated youths were those who had either served time in a juvenile detention facility or had been placed on probation by the juvenile justice system. At-risk youths were identified as those who exhibited one or more of the following six characteristics: deficient in basic reading and writing skills, a high school dropout, a sibling of an adjudicated youth, gang involvement, a foster child, or single and pregnant or a single parent. Other factors could also, on an individual basis, lead a youth to be classified as at-risk. However, the DOL stipulated that at least 60 percent of the youths involved in the program had to be adjudicated.
The sub-grantees could choose from among eleven possible services what particular services they wished to provide such at-risk and adjudicated youths. The eleven possible services that could be offered were: skill training, community service, subsidized and unsubsidized work experience, internships, job preparation, occupational training, GED preparation, basic and remedial education, substance abuse services, mentoring, and case management. In the end, however, the four services most commonly provided by these sub-grantees were case management (or individual counseling), community service, job preparation (often including anger management), and substance abuse services.
In RNF's first year, the Latino Coalition funded 16 programs in four cities (Los Angeles, Denver, Phoenix, and Houston). During its second year, it funded 13 of the original 16 programs and added 15 more, including several in San Diego, for a total of 28 programs in five cities. Based on the organizations' formal declarations,Footnote 5 the Latino Coalition considered 22 of these programs to be faith-based and six community-based (that is, secular in nature). In the third year of the program the Latino Coalition continued funding for 19 of the 28 programs.
Data and Measurement
The RNF program provided an unusual opportunity to explore the outcomes of social service programs in that the Latino Coalition contracted with Social Solutions of Baltimore, Maryland, to develop the Efforts-to-Outcome (ETO) case management system; this was a highly developed, computer-based, system that enabled the sub-grantees to track the individual participants and the various outcomes they experienced. The data gathered by the ETO system were not merely aggregate data, but included data related to each individual's participation for all 2,748 youths who participated in one of the 28 sub-grantee programs. The Latino Coalition gave us access to these data, and, as a result, we are able to use them to relate the individual participant's background characteristics and his/her level of program participation with the particular outcomes each participant achieved related to their involvement in the program.
In addition to analyzing these ETO data, one or the other of us visited each of the 19 sub-grantee program sites that were receiving funding in 2007 and conducted 19 focus groups with program participants at 11 program sites. The site visits were conducted in November and December, 2007, lasted at least a half day, and consisted of interviews with both program administrators and line staff and, when possible, observation of program activities. Then, in the spring of 2008, we conducted focus groups with selected participants in the programs.Footnote 6 In this study we concentrate primarily on the outcomes experienced by the 2,748 program participants who were included in the ETO system and report the key findings that emerged from these data.Footnote 7
In our analysis, we examined five basic positive outcomes that could be associated with participation in the RNF program. The first positive outcome was success in the employment field, as one goal of the program was to enable youths to obtain gainful employment. We sought to measure the extent to which the participants had achieved this goal by combining (1) those who had obtained unsubsidized employment for the first time, (2) those who had obtained a new or additional job, and (3) those who had been accepted into the military. If a program participant achieved any one of these three outcomes, we considered him/her to have achieved a positive employment outcome.
A second goal of the program was to prepare youths for gainful employment. As a result, we considered a youth to have had a positive employment preparation outcome if that youth had either (1) entered an occupational training course of study or (2) entered full-time post-secondary education.
Our third positive program outcome measure was the achievement of certain educational milestones. We deemed four different outcomes as meeting this criterion: (1) completing some long-term occupational training, (2) obtaining a high school diploma, (3) obtaining a GED, or (4) obtaining some other kind of educational certification. If a participant achieved any one of these four educational outcomes, we considered the youth to have had a positive outcome by this measure.
A fourth possible positive program outcome was simply the absence of recidivism. In order to be considered to have had a positive outcome by this standard, the program participant had to achieve all three of the following outcomes: (1) not being convicted of a crime without incarceration, (2) not being convicted of a crime with incarceration, or (3) not having one's parole or probation revoked. Only if the youth had achieved all three of these outcomes would the program participant be considered to have achieved a positive outcome on recidivism.
Our final positive outcome related to continued participation in the program. Because three of the first four outcome measures were not fully applicable to the 14 to 17 year old youths who were still in school (62.6 percent of the participants), we also viewed those youths who completed at least 25 hours in a RNF program as having achieved a positive outcome. Many of the qualities needed to succeed in school or the workplace — an ability to set goals and stick with them, to overcome discouragement and counter pressures, to get along with others, to accept guidance from authority figures — are also needed simply to persevere with a program such as the RNF. Consequently, we treated program participants who stayed in the program for at least 25 hours as having achieved, by that very act, a positive outcome. The criterion of 25 hours or more in the program as the cut-off point was adopted for two reasons: (1) it was the median number of hours participants spent in the RNF program, and (2) it meant that a program participant who had spent at least 25 hours in the program would have completed at least one class or some other type of program segment (the shortest RNF classes usually met for about three hours, once a week for six weeks).
THE FINDINGS
The RNF Participants
In relationship to the previously noted DOL requirements, the ETO data revealed that 40.5 percent of the RNF program participants were classified as at-risk and 59.5 percent were adjudicated youths, with 81.7 percent of the participants being Latino (and another 7.6 percent being African-American and 5.1 percent being white). Thus, the RNF program participants met the specified DOL requirements related to proportions of Hispanics and the more-difficult-to-work-with adjudicated youth. Overall, the RNF participants were also generally young (76.0 percent fell into the 14 to 17 age group), male (65.7 percent), still in school (70.3 percent), and living with at least one parent (85.3 percent).
These latter two characteristics, however, must be put into context. In our focus groups, we found that a significant number of those listed as being in school were attending some type of alternative school, either having been expelled from, or having chosen to leave, a standard high school. Moreover, many reported that they were attending poorly performing schools, with high dropout rates, endemic gang activity, and the threat of violence. Many claimed their teachers were uncaring and distant. And among those participants who were designated as living in a stable home environment were many who were living in home situations marked by one or both parents exhibiting marginal parenting skills or parents who themselves were struggling with drug or alcohol addictions. Moreover, the neighborhoods in which the program participants lived typically were marked by violence and by gangs whose members had served as role models for many of the program participants while they were growing up. In our focus groups with program participants, these points were repeatedly made. In short, the youths with whom the Latino Coalition and its sub-grantees were working consisted of minority youths who came from very challenging backgrounds.
Overall Outcomes
Table 1 presents the percentage of RNF program participants who experienced/achieved each of the five positive program outcomes previously described. First, nearly all program participants (91.0 percent) achieved the desired program outcome of having avoided recidivism during their involvement in the program. Almost half of the youths completed at least 25 hours in their programs (49.0 percent) and one in five (20.0 percent) had found new or additional employment (the employment outcome). Overall, the data reveal that the RNF program had positive outcomes for most, even if not all, the youths who took part in it. Even if one eliminates the very highly positive recidivism outcome, almost 70 percent of the youths (69.4 percent) experienced at least one of the remaining four positive outcomes and one in 5 (19.8 percent) experienced two positive outcomes (data not shown).
Participant Outcomes
1The percentages of program participants who achieved one or more of the following employment outcomes: first time unsubsidized employment, found an additional job, or entered the military.
2The percentages of program participants who either entered long-term occupational training or entered full-time post-secondary education.
3The percentages of program participants who achieved one or more of the following education attainments: Completed long-term occupational training, obtained a high school degree, obtained a GED, obtained an educational certificate.
4The percentage of program participants who were not convicted of a crime (with or without incarceration) or did not have their parole or probation revoked. The N for this outcome is slightly lower than that for the other four outcomes because the recidivism data were not available for one program due to its having made an error in recording the recidivism outcome in the ETO system.
Outcomes by Participants' Characteristics
The question then becomes what factors help to account for differences in success rates of the program participants. Some such differences may be a function of differences in personal characteristics of the program participants themselves. Table 2 presents the percentages of participants who had at least one positive outcome (other than that of the recidivism outcome) by several socio-demographic characteristics of the program participants — namely, in terms of differences according to gender, age, enrollment in school, and whether the participant was an at-risk or adjudicated participant.
Percent with at least One Positive Outcome by Participant Characteristics
1The percentage of program participates who had at least one positive outcome, not taking into account the outcome of avoiding recidivism.
2The program participants who were either 18–21 years of age OR were 14 to 17 years of age and out-of-school.
As noted above, nearly 70 percent (69.4 percent) achieved at least one positive outcome — even when not considering the outcome of avoiding recidivism. But, when broken down by the various social characteristics of the program participants, the data reveal that some differences in success rates among the participants were not related to differences in their socio-demographic characteristics, while others were. For example, with regard to gender, differences in success rates were hardly evident on the basis of whether the program participant was a male or female — though female participants had a slightly higher rate of exhibiting at least one positive program outcome than did the male participants (71.5 percent versus 68.3 percent, respectively).
But, in relationship to some other variables, greater differences in success rates were evident across the categories of analysis. For example, at-risk youth were somewhat more likely to exhibit success than adjudicated youth (73.7 percent versus 66.4 percent, respectively). This finding is hardly surprising given that adjudicated youth had already experienced legal problems, and, as a result, they would be expected to have greater difficulty than non-adjudicated youth in following the stipulated rules or in meeting the proscribed expectations associated with the programs offered (and thereby achieving positive outcomes).
Likewise, the out-of-school and older youths were more likely to experience a positive outcome than the in-school and younger participants. Among the 18 to 21 year old youths along with those 14 to 17 year old youths who were out of school, 88.4 percent experienced at least one positive outcome, while a much lower percentage of 66.6 percent of the 14 to 17 year olds who were still in school did so. But, this finding is largely a function of the fact that several of the success measures are clearly tied to age and school status. For example, those outcome measures that deal with employment or employment training achievements are not fully applicable to the younger, still-in-school, program participants.
Outcomes by Organizational Characteristics
If the individual characteristics of the participants do not provide much help in accounting for differences in rates of programmatic success, are such differences then more a function of the organizational characteristics and the types of programs offered? At the outset it is important to note that the percentage of program participants with positive outcomes varied considerably among the 28 organizations involved in the RNF program. One organization, for example, had 97 percent of their participants achieving at least one of the four positive outcomes other than recidivism. In contrast, another organization had only 30 percent of its participants achieving at least one such positive outcome. Thus, the sub-grantee programs were not mirror images of each other in terms of outcomes.
This raises the possibility that either the faith-based or community-based programs may have been significantly more effective in terms of achieving positive outcomes. And this leads directly to the question that this study specifically addresses: Were the participants in the 22 faith-based programs more likely to exhibit positive outcomes than the participants in the six community-based programs?
If one considers the number of programs whose participants had at least one positive outcome other than the recidivism outcome, the faith-based and community-based programs had almost the same outcomes: 69.7 percent of the participants in the faith-based programs had at least one positive outcome and 68.2 percent of the participants in the community-based programs had at least one positive outcome. The faith-based participants also fared slightly better than the community-based participants among those with three or four positive outcomes. Here 5.3 percent of the faith-based participants achieved three or four of the possible four positive outcomes, while only 3.2 percent of the community-based participants did so. In short, in terms of examining differences in positive outcomes in this fashion, the faith-based programs did very slightly better than the secular, community-based programs.
It may be, however, that faith-based programs did better — or worse — with regard to the different kinds of positive outcomes than did community-based programs. Moreover, given possible differences in the program participants enrolled in the two different types of programs, it is unclear to what extent differences between the two may be a function of differences in the participants themselves. It is possible that differences between the two types of program would either disappear completely or become more attenuated once controls for other related variables are taken into account.
Table 3 addresses these issues and presents the percentage of positive outcomes among participants for the faith-based and the community-based programs, as well as the results of a multivariate analysisFootnote 8 that reveals the relative importance of this faith-based vs. community-based program distinction in accounting for differences in each of the four possible positive outcome results (other than recidivism). The first percentage given is the actual difference evident between the faith-based and community-based programs related to the particular positive outcome under examination. The multivariate analysis then controls for differences in the participants' age, gender, school status, home situation, and participation in a program the Latino Coalition central office had discontinued due to poor performance,Footnote 9 and presents the adjusted or expected percentage Footnote 10 obtained once the covariates have been taken into account. Though these covariates do not appear in the table, their effects are controlled for statistically in the multivariate analysis presented in Table 3.
Participant Outcomes by Type of Subgrantee
1The beta value assesses the relative importance of the variable after taking into account the influence of the participants' age, gender, school status, and home situation, and participation in a program the Latino Coalition central office had discontinued (due to poor performance). The score marked by a single asterisk is statistically significant at the .01 level.
With regard to the faith-based and the community-based (that is, secular) programs there were almost no differences in terms of outcomes — even after one controls for the effects of the various covariates. Prior to the advent of controls, faith-based programs did slightly better than the secular programs on two of the four outcome measures presented in Table 3 (employment outcomes and completing educational milestones), while the community-based programs did slightly better on the other two outcome measures (beginning employment training programs and completing 25 plus hours in the program).
However, once the effects of the various covariates have been taken into account, community-based programming exhibits a success rate that slightly exceeds that of the faith-based programs. However, these differences in positive outcomes for the two types of programs are so small that they fail to achieve statistical significance — except for the case of completing 25 hours or more in program participation where the beta coefficient is statistically significant at the .01 level.
Therefore, given these findings, we conclude that our hypothesis that faith-based intervention programs will have more positive — but only slightly more positive — outcomes than their community-based counterparts cannot be affirmed. Faith-based and secular programs exhibited very similar outcomes. In terms of simply comparing the outcome results of faith-based and secular programs, faith-based program exhibited more positive outcomes by some measures and the secular programs by others. But, once controls were introduced, the secular programs slightly outperformed the faith-based programs — but, for the most part, only marginally. Thus, we conclude that with the variable of the faith-based or secular nature of programs isolated, the data reveal that faith-based and secular programs have very similar outcomes. Thus, neither those who advance faith-based programs as having much better outcomes than secular programs nor those who argue the contrary can find support for their arguments in our findings.
There are at least two potential explanations for the fact that our study found somewhat fewer differences between faith-based and secular programs than that revealed by earlier studies. One possible explanation is that our study only included faith-based and community-based programs and did not include any government-run or other large, professionalized, secular programs. The entire RNF program run by the Latino Coalition emphasized locating and empowering small, grass-roots level programs that operated in the communities they served. As noted earlier, from a research point of view, this had an advantage, as most of the anecdotal claims for greater FBO effectiveness, as well as a number of empirical comparative studies, have compared large, professionalized government-run and nonprofit programs with small, faith-based organizations located in the neighborhoods they are serving. Our findings suggest to the extent that localized, faith-based programs are producing positive outcomes, it may be more the result of their localized, community-based nature than their faith component.
A second explanation for why we found so few differences between faith-based and community-based program outcomes lies in the fact that we found the faith-based versus community-based distinction hardly represented a clear, bright line between the religiously-based programming and secularly-based programming provided by these sub-grantees. Other researchers have found the same. For example, Steven Rathgeb Smith concluded, based on a series of case studies of faith-based and secular service programs in three different areas of the country: “One particularly noteworthy finding was the extensive use of religious discourse and symbols in secular and faith-based agencies . . . Further, religious symbols and discourses were even evident in secular programs receiving public funds” (Smith Reference Smith, Smith, Bartkowski and Grettenberger2006, 6–7. Italics added.).
The same pattern was evident in this study. In our site visits and focus groups we found that the faith-based programs were not isolated from their communities nor were they operating in closed, integrally religious, settings.Footnote 11 Nor were the community-based programs necessarily devoid of religious aspects or overtones. Instead, the faith-based programs worked closely with juvenile court systems, public school districts, and large, professionalized secular agencies. The services they offered under the RNF program did not differ greatly from those we observed at the community-based programs. Meanwhile, at least three out of the six community-based programs possessed religious overtones; one had the word “Christian” in its name even though it was officially classified as non-religious, while another met in a church. A third community-based program had been started by a church and continued to have loose connections with a couple neighboring churches. Some of the youths in the focus groups we conducted mentioned its church connections. And, when we asked the agency head about a large cross that was in one of the classrooms and how this squared with its nonreligious nature, he explained that a church rents that room for services on Sundays, so they just leave the cross up during the week. Even its name, which refers to changing personal attitudes and values, can be seen to have religious overtones. Given the blurring of the religious-secular distinction in real life, the lack of clear-cut differences between the outcomes of faith-based and secularly-based programs becomes more understandable.
However, there is one additional programming distinction we observed that proved to be highly revealing. During our site visits and while conducting our focus groups, we noted a very basic difference in the character of the programs. Some of the sub-grantees offered many activities and services in addition to the RNF program, and these sub-grantees worked to integrate their participants into these other activities and services. Their goal was the establishment of an on-going relationship with the youths, one that would last for years, with no set ending point to the youths' involvement with the organization and its various programs and activities. These sub-grantees offered a variety of classes, and even before a youth completed one, they would urge the participant to become involved in another. They offered trips to sporting events or concerts; they had recreational activities, such as Ping-Pong, video games, and basketball available; some would invite program participants to youth groups at an affiliated church; and they would urge the youths just to stop by and simply hang out. One of the programs bought each of the participants a membership in a nearby gym, or sports center, where they could go to take part in aerobics classes or play basketball and other sports. When we visited these program sites, there always seemed to be youths present — visiting with each other and staff members, playing games, or just hanging out. The youths taking part in these programs often described their relationship to the program and its staff members in family terms. For example, in one Phoenix faith-based program we asked the participants in a focus group if the program acted as a sort of second family for them. There were general assents, and then one youth replied, “I consider it my first. I feel more comfortable here than with my real, regular family.” Another said, “That's true.” A third said, “[Name of a staff member] is my mama.” A fourth said, “Yeah.”
In contrast, other sub-grantee programs focused simply on offering certain classes or providing counseling and mentoring sessions, with such opportunities lasting for a set number of weeks after which they simply would end the relationship with the participants of the program. In these situations, the participating youth would usually come for the class or counseling session and then leave when the class or session was completed.
Because of this distinction, we divided the sub-grantee programs into those that took a more comprehensive, relationship-building approach and those that took a more limited, non-comprehensive, approach. More specifically, the sub-grantees we classified as comprehensive in nature were marked by two overlapping characteristics: (1) they worked hard to integrate the RNF youths into a variety of activities other than those that were a part of the RNF program itself, and (2) they sought to keep the youths engaged in those activities on a continuing, on-going basis. Admittedly this involved our making some subjective judgments, but based on site visits and focus groups with participants, we are confident that this division reflects a real distinction among the sub-grantees' programs. Among the program sites we visited, we categorized eight as comprehensive in nature and 11 as non-comprehensive. Of the eight comprehensive programs, seven were faith-based and one was community-based, although the one community-based program was the program referred to earlier that had been started by a church and that still maintained connections to a couple churches in the community. It should also be noted that we decided to classify all nine of the programs we did not visit as being non-comprehensive in nature, since there was compelling evidence that these nine programs were indeed non-comprehensive in nature.Footnote 12
Table 4 presents a similar analysis to that presented in Table 3, but this time it examines differences in positive outcomes between comprehensive and non-comprehensive sub-grantee programs. The data reveal that, with regard to each of the four outcome measures considered, the comprehensive programs exhibited statistically significant more positive outcomes than did the non-comprehensive programs. With regard to employment, nearly one-third (32.6 percent) of participants in the comprehensive programs attained a positive employment outcome after controlling for the various covariates noted, while the non-comprehensive programs had less than one in seven (15.0 percent) attaining this positive outcome. Similarly, with regard to achieving an educational milestone, the comprehensive program participants were more than twice as likely as non-comprehensive participants to have had a positive outcome (19.2 percent versus 7.5 percent). And, when success is examined in terms of completing at least 25 hours in the program, nearly 70 percent of the of the youth involved in a comprehensive program (67.7 percent) completed this level of participation, while just slightly more than 40 percent of those in a non-comprehensive program (41.6 percent) did so. This last finding is made more impressive when one keeps in mind that this measure of 25-plus hours were based on participation in the RNF program alone, not in all activities that were a part of the comprehensive-type programs. In short, the differences in outcomes between the two types of programs were consistent, substantial, and statistically significant.
Participant Outcomes by Type of Subgrantee
1The beta value assesses the relative importance of the variable after taking into account the influence of the participants' age, gender, school status, and home situation, participation a program the Latino Coalition central office had discontinued (due to poor performance). The scores marked by a single asterisk are statistically significant at the .01 level and those marked by a double asterisk are significant at the .001 level.
CONCLUSIONS
This comprehensive versus non-comprehensive distinction among programs suggests two tentative generalizations related to public policy. The most direct generalization is simply that, when participants involved in social services come out of a highly negative environment, such social service programs in which participants are involved in a comprehensive fashion are more likely to have positive outcomes than when they are involved in programs that end upon completion of a particular class or counseling session. That was the case here. The at-risk and adjudicated youths in the RNF program came from backgrounds often marked by gang-infested neighborhoods, dysfunctional families, and failing schools. In such situations, it is quite unrealistic to expect that such youth will turn their lives around simply as a result of six weeks of two-hour classes meeting twice a week, after which they return back into an environment (or more likely never have left an environment) filled with negative influences and pressures. What is needed is a new environment — for them a counter-culture — with new values, contacts, and role models. “A new family” is needed, as many of our focus group members from comprehensive programs put it.
A similar point can be made concerning many other social service programs, such as prison release, drug and alcohol treatment, and homeless assistance programs. The findings of this study suggest that, in such cases, a comprehensive program that creates new, positive relationships and a constructive environment, or culture, is more likely to lead to positive outcomes than a program that merely teaches certain skills or the importance of certain attitudes or values, but does not provide along with it a supportive network of activities and relationships.
This comprehensive versus non-comprehensive distinction also suggests a tentative generalization that explains why most empirical studies have found that faith-based groups tend to have better, but only slightly better, outcomes than their secular counterparts. As was just noted, our study found that the comprehensive programs, whether faith-based on not, tended to have more positive results than did the non-comprehensive programs. This fact needs to be combined with the tentative (though admittedly far from conclusive) evidence that faith-based organizations are more likely to provide comprehensive services than community-based or other secular organizations. In our study, for example, though the numbers are small, seven of the eight RNF sub-grantees that offered comprehensive programs were faith-based in nature, and the eighth, while technically secular in nature, had been started by a church and still had some ties to two area churches.
Other studies also suggest that faith-based organizations may provide more comprehensive programs and offer more life skills training (in distinction from technical skills) than do secular organizations. For example, Malcolm Goggin and Deborah Orth studied one government program and six faith-based programs that provide homeless families with services aimed at helping them to transition to employment and stable housing. A key conclusion of the researchers — based on in-depth interviews, site visits, and client focus groups — is that “FBOs' values extend beyond the goal of permanent housing for the homeless, and they are more likely to address clients' needs in a holistic way . . . FBOs tend to promote spiritual health, church involvement, hope and dignity for clients, and social justice” (Goggin and Orth Reference Goggin and Orth2002, 45). In a similar fashion Steven Rathgeb Smith concluded based on three case studies: “In brief, services provided by FBOs are more likely to be longer in duration than comparable secular organizations. Religious or religiously-affiliated groups are more likely to approach the specific services they provide as elements of a moral endeavor as opposed to services that provides [sic] technical skills” (Smith Reference Smith, Smith, Bartkowski and Grettenberger2006, 9). And, finally, in a study of welfare-to-work programs in four large cities, Monsma (Reference Monsma2004, 104) found that the faith-based programs offered a higher proportion of life-skills services than did secular programs, whether governmental or private. But, as we noted earlier, the evidence that faith-based organizations tend to offer more comprehensive services — while suggestive — is currently less than conclusive.
If faith-based social service programs in fact do offer more comprehensive services for longer periods of time than do secular programs, this may help explain why many previous studies have found that faith-based programs exhibit somewhat higher levels of success — though only slightly more so — than their secular counterparts. They may exhibit this greater success by means of a two-step process: (1) faith-based organizations offer programs with more comprehensive services than do secular organizations, and (2) programs with more comprehensive services have more positive outcomes. However, since faith-based programs only tend to be comprehensive in nature, with other faith-based programs being non-comprehensive, and since secular programs only tend to be non-comprehensive in nature, with other secular programs being comprehensive, the net result has been that faith-based programs have only slightly more positive outcomes than secular programs.
This possible explanation needs further testing, but should it prove to be accurate, it would have important public policy implications. Since comprehensive programs are labor intensive and difficult to provide on a large-scale basis, it would be hard to duplicate the success of small, comprehensive faith-based and community social service programs in governmental as well as in large, professionalized nonprofit or for-profit programs. These findings do argue for putting more resources — including tax dollars and foundation grants — into small-scale, comprehensive programs that work to build on-going relationships with their program participants.
In summary, our study supports two key public policy conclusions. First, in terms of positive outcomes that benefit society, the public policy consequences of the lack of a clear effective-ineffective distinction between faith-based and secular programs argues that there is no basis for government policy to favor either secularly-based or religiously-based programs that offer prevention and intervention programs for minority youths. In other words, the goal of “leveling the playing field” that was often invoked during the Bush administration (Bush Reference Bush2001) and the “all hands on deck” position of the Obama administration (Obama, Reference Obama2008) make good public policy sense. Second, human service programs seeking to serve persons coming out of negative social backgrounds appear to generate far more positive outcomes to the extent they can create new environments that supports participants on a continuing basis. Thus it makes good public policy sense to support and encourage such continuing programs over those that only offer certain technical training of a limited duration.
