Introduction
Evaluation of the success of conservation interventions lags behind that of other fields (Kleiman et al. Reference Kleiman, Reading, Miller, Clark, Scott, Robinson, Wallace, Cabin and Felleman2000; Saterson et al. Reference Saterson, Christensen, Jackson, Kramer, Pimm, Smith and Wiener2004; Bernhardt et al. Reference Bernhardt, Palmer, Allan, Alexander, Barnas, Brooks, Carr, Clayton, Dahm and Follstad-Shah2005; Ferraro & Pattanayak Reference Ferraro and Pattanayak2006). As conservation funding is very limited, few international programmes set aside budgets for detailed evaluation of their efforts, and pleas for consistent collection of empirical data and better data management systems recur in assessments of the overall state of the environment (Anon. 2002; Alcamo et al. Reference Alcamo, Ash, Butler, Callicott, Capistrano, Carpenter, Castilla, Chambers, Chopra and Cropper2003; Redford et al. Reference Redford, Coppolillo, Sanderson, Fonseca, Dinerstein, Groves, Mace, Maginnis, Mittermeier, Noss, Olson, Robinson, Vedder and Wright2003; Stem et al. Reference Stem, Margolius, Salafsky and Brown2005). There is, however, a growing consensus among large conservation groups that systems enabling conservation success to be measured will enhance ability to maximize the ecological effects of limited conservation funds.
Larger conservation organizations are now following in the steps of macro-economists and are beginning to standardize environmental indicators that can be used to assess the state of the environment at a global level. For example, one of the United Nations’ millennium goals will be assessed by measuring the percentage change in each nation's forested and protected land (Anon. 2006a). The Heinz Center's State of the Nations Ecosystems survey systematically analyses a wide range of environmental data comprising 113 indicators of environmental health collected by federal agencies and conservation groups in the USA (Anon. 2002). Several collaborative efforts are also bringing conservation practitioners together to tackle environmental evaluation problems, such as the Conservation Measures Partnership (see URL http://www.conservationmeasures.org) and the Environmental Evaluators Network (see URL http://www.nfwf.org/environmentalevaluators).
While most people in the tiger conservation community can agree on desirable conservation outcomes such as saving wild tigers, the methods to achieve those outcomes are hotly debated. One reason for this is that conservation values and outcomes are nested within complex social, political and biological landscapes that affect conservation actions, making them ‘wicked problems’ (Rittel & Webber Reference Rittel and Webber1973; Rauscher Reference Rauscher1999).
Conservation donors also have a responsibility to their own grantees to foster critical thinking about the real effects of their conservation actions, and to initiate efforts that will achieve real ecological outcomes. Conservation outcomes should go beyond the immediate project outputs such as the number of uniforms supplied and consider how to measure the desired effects on the plants, animals or habitats of particular interest, while taking precautions not to create new unanticipated problems.
Donors are also faced with managing the incongruence between available funding for conservation and desired outcomes articulated in their mission. A study by the Zoological Society of London showed that between 1998 and 2003 just US 31 million had been spent by non-governmental organizations (NGOs) and international donors on tiger conservation projects. This constitutes an average of c. US 5 million yr−1 spent on 13 tiger-range countries, with c. US $1.25 million coming from Save The Tiger Fund (STF) (Sanderson et al. Reference Sanderson, Forrest, Loucks, Ginsberg, Dinerstein, Seidensticker, Leimgruber, Songer, Heydlauff, O'Brien, Bryja, Klenzendorf and Wikramanayake2006). To put these numbers into context, consider an analogous species recovery programme in the USA, the Pacific Coastal Salmon Recovery Fund, which spends c. US $60 million yr−1 on salmon conservation and c. US $30 million yr−1 in the state of Washington alone (Anon. 2006b). Conservation investments in tigers are inadequate to achieve the desired outcome of saving wild tigers across their range, no matter how well-intentioned the investors or how wisely they allocate their resources. Tiger habitats declined in area by 40% between 1995 and 2005; tiger conservation efforts have made notable gains in some landscapes, but the global situation for tigers has continued to deteriorate (Dinerstein et al. Reference Dinerstein, Loucks, Heydlauff, Wikramanayake, Bryja, Forrest, Ginsberg, Klenzendorf, Leimgruber, O'Brien, Sanderson, Seidensticker and Songer2006).
Conservation organizations and foundations often spare themselves from self-analysis. Individuals in the organizations typically select successful projects from their portfolios in order to communicate their worth to their own constituents, but this does not facilitate the process of improvement by learning (Jepson & Canney Reference Jepson and Canney2003). Foundations are in a unique position to provide thought leadership in this field. They have access to a range of different grantee organizations, each of which has a slightly different approach to conservation, depending on its mission. Because grant-makers have a close relationship with their grantees, they are better positioned than many conservation practitioners or academics to capture a truly global snapshot of conservation methods used by different practitioners tackling similar problems, such as combating poaching, and to provide incentives to improve conservation practices in that area.
The National Fish and Wildlife Foundation has a strong institutional commitment to evaluation and is beginning to develop the tools to better manage the risks and uncertainties of conservation grant making. There are two ways in which an evaluation may be conducted: an ‘implementation evaluation’ of projects essentially tries to measure grant outputs; did the grantees do what they said they would do? (Elmore Reference Elmore, Williams, Elmore, Hall, Jung, Kirst, MacManus, Narver, Nathan and Yin1982). An ‘impact evaluation’ defines success by conservation outcomes; examples of ‘outcomes’ including effects on land use or tiger populations. Outcome evaluations are especially difficult in the landscapes where STF works because they are shared by multiple actors all conducting interventions focused on a range of impacts, many of which are only indirectly related to tiger populations. It is therefore very difficult to determine a link between intervention outputs and observed changes such as increased tiger populations (White Reference White, Black and White2003; Vaessen & Todd Reference Vaessen and Todd2007). After conducting an initial evaluability assessment (Rossi et al. Reference Rossi, Lipsey and Freeman2004) and drawing up logic models for the programme, we considered an implementation evaluation based on data contained in the final reports to be the most feasible first step to assembling the 10 years of project implementation lessons learned from the STF programme. Other conservation organizations that find themselves in similar situations may find the methods presented here useful.
The aim of this paper is to assess the conservation investments made by STF between 1995 and 2004, and propose a simple method to conduct an implementation evaluation that examines the outputs of a complex grant portfolio in a scientifically disciplined manner. This paper shares lessons learned with the wider conservation community and helps refine future STF investment strategies.
Methods
We conducted an archival evaluation of the US $12.6 million invested in 254 grants between 1995 and 2004 in the 13 tiger-range countries. The Russian Far East, the Terai Arc Landscape of India and Nepal and Sumatra were the landscapes that received the highest levels of focused investment, getting 21%, 12% and 8% of the total investments, respectively. All proposals to Save The Tiger Fund were developed by grantees in response to a request for proposals to save tigers, excluding lobbying and litigation activities and genetic studies. After 1997, the request for proposals was refined to ‘saving wild tigers’ and investments were guided by a tiger conservation priorities assessment (Dinerstein et al. Reference Dinerstein, Wikramanayake, Robinson, Karanth, Rabinowitz, Olson, Mathew, Hedao, Connor, Hemley and Bolze1997).
Classification
We classified all of the grant activities in the portfolio and weighted them by the amount of money invested in each activity type, then scored performance by comparing the deliverables in the final report with the initial proposals. Detailed notes were kept to justify scores awarded in each category. Proposed activities were classified into the following categories:
(1) Understanding: monitoring and research on tigers and their prey and habitats, dissemination of findings and building local research and monitoring capacity.
(2) Education: building schools, developing teaching capacity, developing conservation curricula in schools and outreach to the general public using awareness materials, events and the media.
(3) Anti-poaching: monitoring poaching incidents, outreach to hunters, enforcement activities and increasing anti-poaching capacity of reserve staff through training and provision of equipment.
(4) Sustainable development: improving human well-being through development of community-based natural resource management, alternative livelihoods, community health programmes, resettlement assistance, alternative energy sources and formation of village resource committees.
(5) Habitat: acquiring, restoring and consolidating tiger habitats for conservation.
(6) Leadership: grooming future generations of tiger conservation leaders through specific leadership training or post-graduate degree programmes.
(7) Trafficking: increasing capacity of enforcement officials and customs agents, monitoring trade, conducting enforcement activities and targeted education of consumer groups.
(8) Zoo breeding: improving breeding facilities or management of tiger subspecies held in zoos.
(9) Human-tiger conflict: providing human-tiger conflict response units, monitoring human-tiger conflict, conducting outreach and depredation compensation schemes in tiger landscapes and relocating problem tigers.
Each deliverable outlined in the proposal was recorded in a database using this classification scheme and weighted to the nearest 10% according to the amount of money invested in each activity. This allowed us to define and quantify what was promised prior to the start of the grant in monetary terms.
Performance measures
After classification, the final reports (available online at URL http://www.savethetigerfund.org/reports) were read and the performance in relation to each deliverable was assessed based on what the grantees claimed to have delivered in their final report. A five-point ordinal scale was used to rate performance: 1 = unsatisfactory, 2 = less than satisfactory, 3 = satisfactory, 4 = very satisfactory, 5 = exceeded expectations. In each case, a brief qualitative description of the promise, the outputs and the lessons learned was also recorded to justify the score. Additional indicators that the grantees used to quantify the success of their own efforts were also recorded, such as tiger density, number of arrests, number of people taught and hectares of tiger habitat acquired.
The variability in performance was calculated in two distinct ways. The average performance of each dollar spent on activity types across the whole portfolio was calculated as:
where n is one of the following activities: understanding, education, anti-poaching, sustainable development, habitat, leadership, trafficking, zoo breeding or human-tiger conflict.
Project performance per dollar spent was calculated for each project as:
In order to provide context for the meta-analysis, a qualitative assessment of tangible achievements and lessons learned was also compiled from the qualitative description given to justify each score.
Attributes of success and failure
In order to capture the main factors associated with success and failure, the highest and lowest scoring 50 projects (20%) were identified and the qualitative notes were scrutinized to identify recurring factors associated with successful and unsuccessful projects. A list of all factors was compiled and each grant was then tagged accordingly and a tally conducted to identify the 10 most common factors associated with success and failure in the portfolio.
Assumptions
An archival evaluation of this sort makes three important assumptions. It assumes grantees reported their results honestly and had similar abilities to communicate their results back to STF. Periodic site visits made by STF staff found that final reports were usually consistent with on the ground observations, so this is a reasonable assumption, although some embellishment of accomplishments by grantees was inevitable. It was assumed that projects selected for support yielded positive outcomes for tiger conservation if implemented in line with the proposals. As with any grant-making programme there was some risk that even well-designed projects might not always lead to the intended outcomes. STF minimized the risk of design failures through a stringent peer-review process to weed out the weakest proposals. With a competitive grant slate, where only about a quarter of proposals was successful, the incentive for grantees to set their own performance bar high was intense. Thirdly, the assumption was that grantees were able to tailor their final reports to specifically address STF's expectations, even if the overall project involved multiple donors and implementation organizations. This potential difficulty was minimized by STF's grant administration system, which required that final reports be submitted specifically addressing the objectives outlined in proposals.
Results
On average, grantees achieved what they had originally proposed. The average performance per US$ invested in each completed project was 3.36; this was slightly above the ‘satisfactory’ mark.
When broken down by activity type, substantial funds (30%) were invested in research and monitoring activities to improve understanding of tiger conservation needs (Fig. 1). This was closely followed by education and outreach activities (28%), then by anti-poaching activities (13%). The best-performing suites of activities had average performance scores > 3.5 (Fig. 1). They improved and protected tiger habitats, reduced tiger trafficking, and mitigated human-tiger conflict, each of these receiving c. 5% of the total investments.
Breakdown of the US$ 12.6 million STF investments between 1995 and 2004. (a) The average performance of activities per US$ invested on a scale of 1–5, where 1 = unsatisfactory, 2 = less than satisfactory, 3 = satisfactory, 4 = very satisfactory and 5 = exceeded expectations. (b) Total US invested in each activity type.
Projects focusing on zoo breeding performed satisfactorily and helped to secure a well-managed captive population of tigers and stud books for Amur, Sumatran and South China tigers in zoos around the world. This was an initial focus of STF, but after 1997, awards focused exclusively on wild tiger conservation issues.
Both success and failure are expected in any portfolio. From an evaluation perspective, these tails of the distribution are particularly valuable as they contain a wealth of lessons learned about what constitutes a successful or unsuccessful project. When plotted graphically, the performance per US$ invested and project performance graphs appear very similar; they both have a mode of ‘satisfactory,’ but the number of ‘more than satisfactory’ projects strongly outweighed the ‘less than satisfactory’ projects (Fig. 2). Also, the two ‘unsatisfactory columns’ were unevenly matched. While some STF projects truly did not perform as expected, the US$ amounts invested in ‘risky’ projects was significantly lower than the average (Kruskal-Wallis test, df = 4, p = 0.006). Although STF was willing to invest in high-risk grantees, these investments were made cautiously, with lower than average US$ amounts.
Variation in STF project performance per US$ invested by (a) number of projects and (b) total US$ invested.
The 50 lowest-scoring projects had performances per US$ invested in the range from 1.0–2.9. They included projects by both local and international NGOs, with grant awards ranging from US$ 1000 to US$ 150 000. Most of the factors associated with poorly performing projects were related to poor project management and administration, rather than external problems beyond the grantees’ control (Table 1).
Ten common factors associated with poorly performing projects, based on a qualitative assessment of the 50 lowest-scoring projects (frequency = number of projects scored for each factor), which had overall project performance per US$ invested scores ranging from 1–2.9.
The 50 highest-scoring projects (4.0–5.0) were conducted by both local and international NGOs, with grant awards ranging from US$ 4000 to US$ 250 000. In general, successful projects worked collaboratively, their results were disseminated effectively, had clearly defined goals, were informed by good science, had buy-in from local communities, informed government policy, employed adaptive management principles and attracted other donors (Table 2). Grantees were even able to implement successful programmes in the face of significant external problems such as unstable political situations. Factors associated with success or failure are not necessarily the causes of success or failure, but they can provide insight to grantees working to improve their project management.
Ten common factors associated with successful projects, based on a qualitative assessment of the 50 highest-scoring projects (frequency = number of projects scored for each factor) that had overall project performance per US invested scores ranging from 4–5.
An attempt was made to conduct a broad, quantitative meta-analysis of project accomplishments, but it was found that there was no suitable way to track the overall conservation outcomes because of the diverse array of conservation activities and indicators used to track progress. We did, however, make qualitative observations on best conservation practices and lessons learned for conservation practice.
Understanding
Best practices
STF monitoring programmes yielded important baseline information about tiger populations across their range, but few STF studies were conducted beyond the scale of an individual protected area (PA); many used different designs that did not allow effective comparisons at the landscape level. The Russian Far East is the only tiger landscape for which there are long-term tiger population data. The most recent census indicated that the Amur tiger population has remained stable over the last 10 years at about 450 individuals and that collective actions have been sufficient to prevent further declines (D. Miquelle, personal communication 2005).
Many PAs staffed by government employees with limited human and financial resources did not view research and monitoring as priority activities. This constrains their ability to design effective adaptive management programmes informed by science. Some successful NGOs identified this need and work with government representatives to conduct monitoring that provides information to guide management actions and influence public policy (for example Johnsingh et al. Reference Johnsingh, Qureshi, Goyal, Rawat, Ramesh, Ashish, Rajapandian and Prasad2004).
Some STF research projects, which at the outset failed to generate the necessary management responses to save imperiled populations, eventually lead to positive outcomes. In 2002, an Indian grantee demonstrated that tigers in Sariska Tiger Reserve were declining and on the brink of extirpation (Avindan Reference Avindan2003). The grantee pointed out the disparity between inflated official tiger census results and the declining number of tiger sightings noted by Reserve staff. In 2005, when the news broke that tigers were extinct in Sariska, these observations were used to expose the weaknesses of past Reserve management and tiger census methods throughout India (Narain et al. Reference Narain, Panwar, Gadgil and Singh2005). This information appears to have led to a response involving the highest levels of government that will hopefully result in improved management and census methods country-wide.
Lessons learned
Poor dissemination of results was the primary weakness of STF projects designed to improve understanding of tiger conservation. Grantees had an excellent track record and had published over 100 peer-reviewed tiger conservation papers, including some seminal work (for example Seidensticker et al. Reference Seidensticker, Christie and Jackson1999). However, many other studies were never published, nor were meaningful data presented in the final reports.
Reasons for not publishing varied. Most straightforward tiger density surveys measuring tiger population size would be of little interest to academic journals, but the information was crucial to management officials and donors. Hypothesis-driven research may not have been published due to difficulties at peer review, language barriers, poor research design, constraints of time, lack of incentives to publish, time conflicts by over-worked investigators or a reluctance to expose issues that may be politically sensitive.
Considerable amounts of time and effort were spent on gathering data, but they were not always analysed. For example, one grantee reported how many tiger photos were taken within a certain area, but did not make the small, but crucial step to calculate the tiger densities using mark-recapture theory. Yet all properly reported and analysed data should have been retained on record and disseminated as grey literature via the STF website for use by third parties.
Education
Best practices
Many education campaigns targeted teachers, providing them with information, ideas and materials to use in their classes. However, few education programmes followed through with an assessment of their impact on people's attitudes and behaviour, which must largely be taken on good faith. One exception, a Chinese ‘model school’ programme clearly demonstrates the potential value of education. Between 2000 and 2004, the ‘model school’ programme had grown to incorporate 55 schools, and attitudinal surveys demonstrated that ‘model school’ children had significantly improved attitudes towards wildlife and conservation over comparison groups (Zhang & Li Reference Zhang and Li2004).
Lessons learned
A few education projects had only tangential ties to tiger conservation, for example building a school or providing scholarships to individuals, which were in themselves good education activities, but it was impossible to relate these back to increased tiger and prey populations, or improved habitats. Most education projects were targeted at improving conservation knowledge within communities but they were not sufficiently targeted on changing human attitudes and behaviour patterns towards tigers. In some cases, grantees reported that they conducted education campaigns focusing on species other than tigers. Thus the message developed by grantees was not always entirely focused on tigers or of consistently high quality.
Sometimes grantees tackled problems at localized scales that could not possibly have had much lasting regional effect. For example, one project was designed to educate judges in India about tiger conservation, with the hope that sentences would increasingly reflect the gravity of wildlife-related crimes, and hence deter people from poaching. The proposal did not state how many judges were going to be educated, but the work all appeared to have been done quite satisfactorily, educating 19 judges about the complexity of wildlife crime and CITES issues. These 19 judges represented about 0.2% of a population of 10 000 judges in India (Debroy Reference Debroy and Dethier2000); this project may have been appropriate if it were a pilot for a wider campaign aiming to educate significantly more judges in states with large tiger populations, or if it targeted a smaller population of judges that routinely handled wildlife-crime related cases, but it is difficult to see how it could have made any meaningful impact in relation to the need identified in the proposal.
Anti-poaching
Best practices
There are always at least two possible reasons for declining poaching incidents: one is the increased deterrent effects of strengthened anti-poaching operations, and the other is decreased availability of species targeted by poachers, leading to diminishing returns and reduced subsequent effort. Clearly, anti-poaching operations need to take account of both the animal populations and poaching incidents. The best examples of anti-poaching work have either incorporated scientifically-sound wildlife monitoring programmes into their methods, or are the result of a collaboration between two groups, one focusing on wildlife monitoring and the other focusing on anti-poaching. For example, a project in Sumatra identified declining tiger and prey populations in the southern part of Bukit Barisan Selatan National Park and this was used to inform and strengthen anti-poaching activities carried out by another organization also funded by STF.
Lessons learned
About two-thirds of the projects providing equipment or infrastructure (such as anti-poaching outposts) did not report on how they improved their performance. However, the remaining grantees provided good anecdotal or quantitative evidence demonstrating how the equipment enabled anti-poaching patrols to cover larger areas, improved communications and/or reduced numbers of poaching incidents.
There were several different examples of ineffective community ranger-type projects, where locals were employed to patrol and assist law enforcement officers in a particular area. Even if the work conducted by these groups had the approval of PA authorities, the enforcement units could not carry weapons or make arrests and therefore could not tackle armed poachers unless accompanied by PA officials. In some cases, grantees had the authority to make arrests but they were out-ranked when it became clear that the poaching rings were run by military officers. In Cambodia, STF invested US$ 335 000 in anti-poaching activities. Despite these investments, if the military was involved in poaching rings, little could be done to stop the poaching. Thus enforcement proved to be very difficult, and several monitoring reports indicated that tiger numbers declined sharply over the last 10 years.
In some areas anti-poaching work has been very effective, but grantees noted that once enforcement activities began, they had to be funded consistently in order to remain effective. Even one year of reduced funding could jeopardize years of effective enforcement work. Conservationists in general sometimes argue that the enforcement of laws is the responsibility of the state and that funding NGOs to perform any law-enforcement work other than training or capacity-building is creating a dependency cycle. Nonetheless, NGO anti-poaching work has led to some significant benefits for tigers. Tiger patrol teams run by NGOs have proven to be very responsive and effective in certain areas and have even uncovered the involvement of corrupt state officials and led to their prosecution.
Certain anti-poaching activities were only indirectly associated with tiger conservation; for instance, it is unclear what effects the morale-improvement projects had on PA staff in any STF grants because no baseline indicators of work performance or morale were maintained. It is recognized that PA rangers have dangerous and demoralizing jobs, but there is uncertainty about what ecological outcomes be achieved using incentives like rewards, life insurance, scholarships or field kits. In contrast, some grantees noted that the high turnover of staff in low-paying and dangerous law-enforcement work undermined investment in training. While it was not quantified in any projects, staff turnover would be a critical indicator to examine in future projects.
Sustainable development
Best practices
Some activities had more potential to generate ecological outcomes than others. For example, the use of biogas plants could theoretically reduce fuelwood collecting pressure on nearby forests. In order to collect dung, cattle are stall-fed and therefore reduce grazing pressure in forest habitats and the associated risk of cattle being killed by tigers, therefore reducing human-tiger conflict. Tree nurseries supplying seedlings for habitat restoration had yielded income for communities. The most successful of these schemes is found in the Terai Arc Landscape, but the projects provided only anecdotal evidence that they had led to direct conservation outcomes, such as reduced human pressure on tiger habitats (Dinerstein et al. Reference Dinerstein, Loucks, Wikramanayake, Ginsberg, Sanderson, Seidensticker, Forrest, Bryja, Heydlauff, Klenzendorf, Leimgruber, Mills, O'Brien, Shrestha, Simons and Songer2007).
Lessons learned
STF's sustainable development-type grants have very indirect effects on conservation. For example, human health and micro-credit projects may have improved human well-being in the targeted communities, and proponents argue that improved human livelihoods will ultimately lead to improved conservation of tigers, but they often do not consider a host of other confounding factors such as increased growth of settlements around service centres, making this claim debatable. Thus, it is vital to decide how closely the grant effort is tied to a specific tiger conservation outcome.
Habitat
Best practices
Habitat-related activities such as acquisition and restoration had the most easily defined and measurable indicators, such as hectares acquired or restored, representing clear outcomes for conservation, and the habitat-related projects often exceeded their goals, making them high-scoring projects in this evaluation.
Grantees in the Terai Arc Landscape and the Russian Far East had achieved notable successes by developing detailed landscape-level visions and implementation strategies that, in both cases, had protected tigers in core areas and allowed tigers to move between core areas through carefully managed forests. This model has helped maintain a secure, genetically viable tiger population at a landscape-level.
Lessons learned
The support from PA management in habitat acquisition and restoration schemes was a key ingredient to this conservation work. Without good management and clearly demarcated land use plans that had buy-in from local people (often obtained by implementing good complementary sustainable development and education programmes), the PAs would have existed only on paper. Several indices have been used to measure the effectiveness of management in PAs (Hockings Reference Hockings2003; Parrish et al. Reference Parrish, Braun and Unnasch2003), but these approaches have not yet been adopted by grantees to evaluate the performance of their management approaches in tiger landscapes.
Leadership
Best practices
Students trained at MSc and PhD levels had to do on-the-ground research related to tiger conservation and ecology, giving them good field experience. The thesis work often provided valuable insights into their chosen research topic, giving a high degree of confidence in the quality of training and skills acquired. While receiving a post-graduate degree does not necessarily imply that the individual has good leadership skills, the relationships and skills acquired during this period were valuable assets that students use for the rest of their lives and to train and mentor the next generation of conservationists.
Lessons learned
Most of the short training courses did not focus on leadership skills. Instead, conservation experts trained small groups of people on some aspect of conservation. The impact of short courses is very difficult to ascertain, other than the immediate benefits probably derived from informal networking.
Trafficking
Best practices
This suite of projects performed remarkably well, and many of the trafficking educational projects had a strong focus on changing the behaviour of people who consume tiger parts. Several projects in China and the USA were awareness campaigns that were accompanied by pre- and post-project attitudinal or market surveys. The surveys demonstrated significant changes in availability of tiger products in target markets and an increased willingness from traditional Chinese medicine practitioners to use alternative medicines that are not derived from endangered species. The projects demonstrated that demand for tiger parts could be significantly reduced over time by clearly defining target practitioner populations for education work, accompanied by surveys to monitor the resulting behavioural changes (Henry Reference Henry2004).
Lessons learned
STF projects for tiger trafficking appear to have made an excellent impact in the areas where they have been carried out. Yet it is clear that understanding of the tiger trade across Asia remains very limited, and that demand and illegal smuggling of tiger parts continues to threaten wild tiger populations. A more systematic, coordinated approach is required.
Zoo breeding
The zoo community has endeavoured to prevent tigers from becoming extinct and has secured well-managed, genetically diverse captive populations of the Sumatran, Amur and Malayan tiger subspecies through carefully designed animal husbandry and breeding programmes known as species survival plans. STF subsequently shifted its focus away from these relatively secure zoo breeding populations to focus on mitigating the threats faced by endangered wild tiger populations.
Human-tiger conflict
Best practices
The approaches to human-tiger conflict mitigation vary widely depending on the landscape. Grantees in the Russian Far East have employed a model that uses dedicated human-tiger conflict response teams to scare tigers away from human dwellings, relocate problem tigers if necessary, educate locals about how to avoid conflict, rescue injured tigers and investigate causes of tiger deaths. In the Russian Far East, STF grantees reported 60 human-tiger conflicts each year, but the number of tigers killed as a result of human-tiger conflict appears to have declined. This serves as a potential best practice model because it operates at a landscape level, is well known to the public and can demonstrate real reductions in mortality from human-tiger conflict.
Other promising approaches independently developed include that in the Terai Arc Landscape, where grantees working on human-tiger conflict have focused on compensating individuals for livestock killed by tigers. Conflict mitigation in Sumatra has focused on research and counselling individuals affected by human-tiger conflict, or taking problem tigers into captivity, which is not congruent with STF's mission to protect wild tigers.
Lessons learned
Given the diversity of approaches and potential outcomes, best practices are difficult to ascertain. There is a need for better communication between the different groups working on human-tiger conflict issues so that the experts themselves can share lessons learned and come up with a set of best practices that would be applicable in each landscape. This issue is likely to be one of the greatest challenges in increasingly human dominated Asian landscapes.
Discussion
The big picture
STF grantees have performed adequately at the project level, however success at this level does not necessarily result in sustained landscape-level conservation outcomes, such as increased or stabilized tiger and prey populations or improved habitat. Given the diversity of other grant-makers spending money on tiger conservation, the range of institutional strengths and weaknesses of grantees and the array of conservation activities being implemented, it is apparent that landscape-level outcomes are only realized if they are guided by a clearly defined landscape-level plan. Examples of such plans include the Russian Far East (Miquelle et al. Reference Miquelle, Merrill, Dunishenko, Smirnov, Quigley, Pikunov, Hornocker, Seidensticker, Christie and Jackson1999) and tiger country action plans, which are government policy documents devised by tiger-range countries, as drafted for Bhutan, Malaysia, Myanmar, Nepal, Russia and Thailand with direct or indirect support from STF.
These larger plans are also tools for the diverse donor community, enabling them to coordinate actions in such a way that larger landscape-level outcomes can be achieved. Two landscape-level tiger conservation plans funded by STF (the Russian Far East and the Terai Arc Landscape in Nepal) have achieved notable landscape-level outcomes, attracted a disproportionately high share of all the available funding for tiger conservation and they each won an additional US$ 750 000 in implementation grants from the Global Environment Facility. The conservation actions taken in these two landscapes have also managed to secure tiger populations there (D. Miquelle, personal communication 2005; M. Shrestha, personal communication 2006) but there is a strong need for improved transboundary coordination.
The secret is in the conservation method
One striking factor about the STF project portfolio was that every conservation group used a slightly different conservation method, even though they were tackling the same threats. Apart from monitoring methodology, which has been extensively published and disseminated (Karanth Reference Karanth1995; Carbone et al. Reference Carbone, Christie, Conforti, Coulson, Franklin, Ginsberg, Griffiths, Holden, Kawanishi, Kinnaird, Laidlaw, Lynam, Macdonald, Martyr, McDougal, Nath, O'Brien, Seidensticker, Smith, Sunquist, Tilson and Shahruddin2001; Hayward et al. Reference Hayward, Miquelle, Smirnov and Nations2002; Karanth et al. Reference Karanth, Nichols, Seidensticker, Dinerstein, Smith, McDougal, Johnsingh, Chundawat and Thapar2003, Reference Karanth, Chundawat, Nichols and Kumar2004; Kawanishi & Sunquist Reference Kawanishi and Sunquist2004), there was very little evidence of learning between conservation groups. Monitoring methods were also adapted from place to place, so that results were often not comparable. One extreme example of the lack of peer learning involves human-tiger conflict activities. Here, completely different solutions have been tried in every different landscape. However, there is no evidence of common best practices emerging, probably because the diverse groups are working in far-flung reaches of the globe. There has been no platform for grantees to learn from each other's experiences, giving the impression that every different group devise solutions independently of other groups’ experiences.
It was impossible to perform a post hoc quantitative meta-evaluation of similar conservation activities because conservation methods and definitions of indicators were inconsistent. For example, the number of arrests was a common indicator used to report on the success of anti-poaching efforts, however definitions of the term ‘arrest’ varied significantly from place to place. Other site-specific factors such as the recent political instability caused by Maoist insurgents in Nepal add new social dimensions to already complex problems and make cost comparisons challenging (Baral & Heinen Reference Baral and Heinen2006).
STF is committed to its grantees and gives them the flexibility to work on threat mitigation within their own geographically-specific contexts, but this approach means that grantees have not learned well from each others’ experiences and few groups measured results using comparable indicators. It is clear from this evaluation that grant-makers themselves need to play an active role in facilitating learning by making final reports accessible among groups and by identifying a suite of indicators that grantees should track so that more quantitative meta-analyses of conservation outcomes can be performed in future evaluations.
Conclusions
In summary, implications for donors and grantees are as follows:
(1) Grantees need to collaborate to develop a long-term landscape-level conservation vision for their regions that can be used to coordinate their individual actions and guide donor investments.
(2) Grantees and donors should be encouraged to work towards long-term goals that will result in meaningful landscape-level outcomes. Donors need to be prepared to make longer-term commitments to these projects, and track grantees’ progress using meaningful indicators.
(3) Grantees need to make far better use of conservation methods and practices that have been successfully established by others. Donors should provide a platform to facilitate such peer learning.
(4) Donors need to find additional financing mechanisms for tiger conservation or narrow the geographical scope of their existing investments to ensure congruence between available funds and their organizational goals. Donors should prioritize activities that directly lead to the desired conservation outcomes and cut funding to actions that cannot be tied to outcomes, even though they may be socially desirable.
(5) Donors urgently need to devise adequate tracking mechanisms to periodically evaluate progress towards their own portfolio goals using meaningful indicators.
The challenges ahead
The evaluation method that we have presented involves discipline that requires organizations to think carefully and methodically about what they have achieved and to plan ahead. There are many more sophisticated methods to tackle evaluation problems, but this method was effective and valuable, allowing STF to take stock and share important lessons learned. However, many different confounding variables operating at a landscape-level made it difficult to attribute conservation outcomes, such as a stable tiger population, directly to the actions of our grantees. Landscape-level conservation should thus be viewed as a ‘wicked problem’ that cannot be tackled as a set of isolated simple problems to be solved in linear ways (Rittel & Webber Reference Rittel and Webber1973). Instead, currently the best tools to tackle such conservation problems are transparent collaborative solutions that engage in multiple types of focused conservation activities with sustained buy-in from governments, conservation groups, funding agencies and stakeholders whose individual contributions and goals are framed by a larger landscape-level vision.
ACKNOWLEDGEMENTS
We are very grateful to Ed Ahnert, Syd Butler, Lorie Jackson, Dan Martin, Jorgen Thomsen, Jean McConville, Mandy Chesnutt, Susan Lumpkin and three anonymous reviewers for their insights. We thank the ExxonMobil Foundation, the National Fish and Wildlife Foundation and the Critical Ecosystem Partnership Fund for resources, and all our donors for their continued financial support for wild tiger conservation.
