INTRODUCTION
There is a considerable body of literature available evaluating the uptake of natural resource management innovations (e.g. Barrett et al., Reference Barrett, Place and Aboud2002). Participatory monitoring and evaluation methods are well established; these involve locally relevant, stakeholder-based processes for gathering, analysing and using information (Abbot and Guijt, Reference Abbot and Guijt1998; Estrella et al., Reference Estrella, Blauert, Campilan, Gaventa, Gonsalves, Guijt, Johnson and Ricafort2000). Conventional monitoring and evaluation, in contrast, involves extractive, externally controlled data-seeking procedures focusing on whether a project has met its objectives.
Practitioners of participatory approaches focus on how beneficiaries, usually poor farmers, can play a more active role and benefit from monitoring and evaluation. However, little has been written about collaborative monitoring and evaluation, that is, how a group of diverse organizations promoting similar innovations in a particular location can work together to monitor and evaluate their efforts. The two approaches, participatory and collaborative, are not at all conflicting: participatory monitoring and evaluation focuses on engaging beneficiaries in the process, whereas collaborative monitoring and evaluation focuses on how a group of diverse organizations implement the process.
Monitoring and evaluating uptake is particularly critical for natural resource management innovations, because they are complex, that is, they involve a number of diverse components (e.g. trees and crops), the length of the cycle of the practice is usually long (often three or more seasons) and many of the benefits (e.g. soil erosion control) are not easily quantifiable (Franzel et al., Reference Franzel, Coe, Cooper, Place and Scherr2001). Accurate and timely feedback on uptake and farmers' views and modifications of innovations have proved critical for modifying research recommendations so that practices are better suited to farmers (Franzel and Scherr, Reference Franzel and Scherr2002). Obtaining and integrating such feedback into research and extension programmes is thus critical for success in developing and promoting natural resource management innovations.
The objective of this paper is to assess the experience of 30 organizations working together in western Kenya, from 1999 to 2003, to assess their impact in helping farmers to develop, adapt and adopt two soil fertility practices aimed at improving household livelihoods. Our objectives in developing a collaborative monitoring and evaluation approach were to (a) enhance mutual learning among stakeholders, (b) increase efficiency, and (c) build capacity so as to use the information to change organizations' actions and policies. Therefore, we assessed the effect of the collaborative exercise on participating organizations' mutual learning, on the efficiency of working together as compared to working individually, and on the actions and policies of the participating organizations. The two practices that the organizations were promoting were biomass transfer (cutting leaves and applying them as green manure) and improved fallows (enriching or replacing natural fallows with planted, nitrogen-fixing shrubs).
METHODS
Description of the study area
The highlands of western Kenya cover an area of about 85 000 km2 and have about 12 million inhabitants, about one-third of the country's population. Altitude ranges from 1250 m to 1600 m asl and the topography is undulating with moderate slopes. Soils are of generally good physical structure, but are nutrient depleted – N, P and K, are all deficient. Infestation of the parasitic weed striga (Striga hermonthica) is common and increases with declining soil fertility.
The population pressure is high and, in the most densely populated areas, ranges from about 500 to 1200 persons km−2. Annual crops are grown during two rainy seasons (about 1500 mm y−1): the major cropping season is from February to June and the minor one from August to November. Maize is the major food and occupies the largest proportion of land under crops. Other crops grown include sorghum, beans, kale and tomato. The main types of livestock found are zebu (local-breed cattle) and goats.
Due to the land pressure, average farm size has fallen to below 1 ha. Nevertheless, fallowing is common – over half of the farmers fallow some of their land for at least one season (DeWolf et al., Reference DeWolf, Rommelse and Pisanelli2000). Land holdings consist mainly of a single piece of land and land tenure is relatively secure. Land pressure has resulted in high rates of urban migration and about one-third of households are female-headed. Crop yields are low and most farmers are not able to produce more than 1 t ha−1 maize. Poverty rates are among the highest in Kenya, exceeding over 50 % of the population in many areas (Place et al., Reference Place, Franzel, DeWolf, Rommelse, Kwesiga, Niang, Jama, Barrett, Place and Aboud2002).
Research and dissemination of soil fertility practices
Diagnostic surveys conducted in the late 1980s and early 1990s indicated that poor soil fertility severely limited farm productivity and that farmers lacked cash for buying mineral fertilizers (Ohlsson et al., Reference Ohlsson, Shepherd and David1998). In the early- and mid-1990s, researchers from ICRAF, the Kenya Agricultural Research Institute (KARI) and the Kenya Forestry Research Institute (KEFRI), in partnership with farmers, developed and promoted two practices, biomass transfer and improved fallows, to address these problems.
Biomass transfer involves collecting leaves from shrubs grown on the farm (usually along boundaries) or off the farm (such as along roads and paths) and applying them to fields as green manure to improve soil fertility. The most common species used by farmers for biomass transfer is Tithonia diversifolia, which is plentiful in the area. Although it is not a nitrogen-fixing plant, tithonia contains relatively high levels of nitrogen, potassium and phosphorus. On-station and on-farm trials demonstrated the high yield responses of maize and vegetable crops (kale and tomato) to applications of tithonia. Moreover, the residual effect of the biomass lasts through 2–3 cropping seasons (Jama et al., Reference Jama, Palm, Buresh, Niang, Gachengo, Nziguheba and Amadalo2000).
Improved fallows involve the enrichment or replacement of natural fallows with planted nitrogen-fixing shrubs. In western Kenya, farmers plant the shrubs into an existing crop, usually by broadcasting the seed or planting in rows at weeding time. Following the harvest of the crop, the shrubs are allowed to grow for a second season. Just before the third season begins, the shrubs are cut and the leaves are incorporated into the soil during land preparation. The field is then planted to crops. During the fourth season, crops are again planted and the shrubs are planted into the standing crop, and the rotation begins again (Amadalo et al., Reference Amadalo, Jama, Nang, Noordin, Nyasimi, Place, Franzel and Beniest2003).
The tree fallow species contribute high levels of nutrients, both through incorporation of leafy biomass and underground root biomass. Farmers have experimented with six species; Crotalaria grahamiana and Tephrosia vogelii are the most popular. Maize yields increase significantly and the system provides higher returns to land and labour than continuous cultivation or natural fallows. Establishing, maintaining and cutting the fallows require minimal extra labour and, over a four-season cycle, the improved fallow system requires 17 % less labour than continuous cultivation. In addition, many of the species have important by-products, such as firewood and stakes (DeWolf et al., Reference DeWolf, Rommelse and Pisanelli2000; Place et al., Reference Place, Franzel, DeWolf, Rommelse, Kwesiga, Niang, Jama, Barrett, Place and Aboud2002).
Collaborative monitoring and evaluation exercises
Stakeholder planning workshop
In 1999, the ICRAF/KEFRI/KARI project (hereafter called ‘the project’) invited representatives of about 30 organizationsFootnote 1 promoting biomass transfer and improved fallows to a workshop on collaborative monitoring and evaluation. Workshop objectives were to (a) share experiences in monitoring and evaluation, (b) generate common indicators to monitor, focusing on those identified as important by farmers, and (c) determine who should collect which information, when and how (Aluoch et al., Reference Aluoch, Noordin and Franzel2000).
Organization-level surveys
The organizations promoting biomass transfer and improved fallows assembled farmer lists and collected other information in 2000 and repeated the exercise in 2002. Information collected included the organizations' extension messages and management practices being promoted, the numbers of farmers they had trained and that were using the practices, and the farmer innovations identified and being promoted. Other topics included farmers' views on the advantages and disadvantages of each practice, the organizations' information sources, extension methods and messages, and problems limiting adoption. Data collection, analysis and reporting were led by an ICRAF staff member for the 2000 survey and by an MSc student, using the information for his thesis, for the 2002 survey.
Special studies
Four studies led by ICRAF researchers or graduate students examined the adoption and profitability of biomass transfer and improved fallows. The study of the adoption of biomass transfer involved monitoring the uptake of the practice among farmers who had participated in on-farm trials, a second group who had learned about the practice from extension services and a third group in a pilot zone comprising 17 villages where researchers and extension staff were promoting the practice (Obonyo, Reference Obonyo2002). Two studies on improved fallows examined uptake among farmers in the pilot zone (DeWolf et al., Reference DeWolf, Rommelse and Pisanelli2000; Pisanelli et al., Reference Pisanelli, Franzel, DeWolf, Rommelse and Poole2002). A fourth study examined the profitability of improved fallows and biomass transfer among farmers participating in on-farm trials (Rommelse, Reference Rommelse2001). All of the studies involved informal focus group discussions, participatory appraisal methods and formal questionnaire surveys.
Stakeholder workshops and evaluation of the process
Stakeholder workshops were conducted in 2000 and 2002 to share results and to plan further studies. Results of the organizational surveys and special studies were presented at these workshops and summaries of the studies were circulated. The participating organizations also evaluated the impact assessment tools, and the monitoring and evaluation process being followed. The MSc student mentioned above visited each of the participating organizations to solicit their views.
RESULTS
Stakeholder planning workshop
A range of different types of organizations attended – government extension services, national and international non-governmental organizations (NGOs), community-based organizations (CBOs), and international and national research organizations (Table 1). All of the organizations present were promoting the use of biomass transfer and improved fallows, and many were monitoring farmers' uptake of the practices. But none, except the project, had written reports on their monitoring or impact assessment activities. ICRAF had conducted research on farmers' experiences in testing the practices and these were shared with participants. In addition, ICRAF shared the results of village workshops on farmers' expectations about impacts from using the practices (Kristjanson et al., Reference Kristjanson, Place, Franzel and Thornton2002).
Numbers and types of different organizations participating in the monitoring and evaluation planning workshop and surveys.
†Numbers of extension services refer to the numbers of district offices of the Ministry of Agriculture involved in the exercise.
All agreed that a collaborative monitoring of impact was preferable to uncoordinated, individual endeavours. Besides, each organization was free to continue monitoring in the way it saw fit, but still participate in the collaborative monitoring exercise. Advantages of the joint effort cited by participants included:
• CBOs, NGOs and extension services lacked the staff, resources and expertise needed to do technical studies on profitability, adoption and impact studies. Yet they needed the results of these studies, to know whether they should continue promoting biomass transfer and improved fallows, which management practices should be promoted, and to show to their donors the impact of their efforts.
• Research organizations lacked the day-to-day contact with farmers using biomass transfer and improved fallows that CBOs, NGOs and extension services had. These latter organizations could provide valuable information about their experiences, even without conducting surveys of the farmers they worked with.
• Joint efforts would be more efficient, as organizations could divide up tasks instead of repeating the same studies.
• Representatives of several of the smaller CBOs and NGOs expressed their lack of knowledge about monitoring and evaluation, and their strong interest in learning about it from more experienced organizations.
Participants also cited several constraints they felt might limit their participation in the exercise. First, some said that their managers did not understand the need for collaborative monitoring of impact and might not be willing to allocate staff time and resources for the exercise. Others noted their lack of staff and resources, particularly transportation, for collecting and assembling the needed data. Finally, many staff were already overextended and worried that they would not find the time to participate.
Table 2 shows the breakdown of tasks agreed upon by participating organizations. Representatives agreed that each organization would be responsible for assembling lists of farmers who had used the practices during the last year. These lists would be useful for finding out how many farmers were using the practice and, over the years, would provide information on the spread of the practices.
Workshop participants' master list of information required for monitoring and evaluation and how it would be collected (1999 planning workshop).
In addition, organizations would collect information on numbers of farmers trained, farmers' problems, assessments, management practices, and innovations. Researchers would conduct special studies on numbers of farmers testing, adopting and discontinuing, farmer-to-farmer spread of the options, farm and household characteristics associated with adoption, especially wealth and gender, and assessment of economic benefits of using the practices. An eight-person committee was established to develop the forms for the organizations to use in the ‘organization-level survey’, that is, assembling lists of farmers and collecting other data about the practices. The committee also assisted the organizations in completing the forms.
Organization-level surveys
In 2001, some 2533 farmers were using improved fallows, an increase of 35 % over two years earlier. Over the same period, the numbers using biomass transfer remained constant, at just over 2000 (Table 3). The proportions of women users appeared to decline, from 52–59 % to 42–48 %, but this change was probably due to a change in the number of organizations reporting a gender breakdown of users, rather than a real change in the proportion of women users. Place et al. (Reference Place, Adato, Hebinck and Omosa2004) found that women accounted for just over 50 % of users of improved fallows and biomass transfer, but only 42 % of fertilizer users.
Numbers of farmers using biomass transfer and improved fallows.
†Data on gender were available for about half of the farmers in 1999 and about one-third of the farmers in 2000.
Biomass transfer's main advantages to farmers, as reported by the organizations, were that it improved soil fertility and crop yields with no cash cost (Table 4). But its main disadvantage was that it required a lot of labour to implement. The proportion of organizations citing lack of information about biomass transfer as a problem declined from 31 % in 2000 to 13 % in 2002. The decline reflected a better flow of information about the practice due, in part, to the collaborative monitoring system.
Advantages and problems of using biomass transfer and improved fallows, from perspective of farmers as reported by organizations supporting them (% of organizations reporting).
Improved fallows' main advantages were increased soil fertility, firewood production and weed control. Only one-quarter of respondents mentioned increased crop yields, in contrast to the frequent adage of researchers that improved fallows double maize yields (Amadalo et al., Reference Amadalo, Jama, Nang, Noordin, Nyasimi, Place, Franzel and Beniest2003). The difference in perception was because researchers were only comparing the crop yields after the fallow with the adjacent control plot, whereas farmers were aware that they had missed a cropping season while the improved fallow was being established. The proportion of organizations claiming that lack of information about improved fallows was a problem remained constant at about 20 % between 2000 and 2002, probably reflecting confusion because of the large number of shrub species (five) available and that two had similar names (Tephrosia vogelii and Tephrosia candida).
The organizations' awareness of farmer innovations increased: the number reporting such innovations increased from 14 to 29 between 1999 and 2001 (Table 5). More importantly, the organizations substantially increased their promotion of these innovations. The most widely promoted innovations included using tithonia biomass in compost, using improved fallows with crops other than maize, using tithonia to make liquid manure, and using tithonia to control pests. The number of times an organization promoted a specific farmer innovation increased from 11 in 1999 to 26 in 2001. It is likely that the strong emphasis given to identifying and promoting farmer innovations in the collaborative monitoring exercise played an important role in the increased awareness and promotion of innovations. In fact, the description of these innovations, their merits and demerits was one of the most time consuming and interesting parts of the stakeholder workshops. Participants were eager to test new innovations and share ideas about them with their farmers.
Farmer innovations reported and promoted by surveyed organizations (no. of organizations reporting and promoting).
†Includes applying as a top-dress and mixing with manure or fertilizer.
‡Includes planting on untilled land, raising seedlings in a nursery and transplanting, mixing fallow species, and incorporating farmyard manure.
Special studies
The findings were presented to the participants at the stakeholder workshops and were summarized in 3- to 4-page extension briefs, written in non-technical language. It is unlikely that most of the findings would have been available to the representatives had there been no collaborative monitoring process. Representatives appreciated hearing and discussing the results and highlighted two important reasons. First, by learning about others' use of the practices, they could speak with more authority about them to their clients (farmers), colleagues and donors. This was especially important for participants who were just starting to use the practices. Second, as mentioned above, participants enjoyed hearing about and discussing the farmer innovations that the researchers identified in their studies. Full-length reports were also made available to those who requested them.
Adoption and farmer assessments of biomass transfer
Obonyo (Reference Obonyo2002) found that 15–23 % of farmers who learned about biomass transfer from researchers and extension staff were ‘strong’ adopters, that is, they used the practice every season. About one-fifth of the farmers who had learned about the practice from extension staff had planted tithonia on their own farm. They had shifted from using biomass mainly on maize–beans to using it primarily on kale and other vegetables. This shift is logical, as labour requirements are high, vegetable plots are much smaller than maize plots and the value of yield increases on vegetables is much higher than on maize. Moreover, farmers reported that applications on vegetables improved the quality of their produce and extended the harvest season, so that they could take advantage of higher prices. Farmers' main problem in using the practice was its high labour demands.
The average size of field on which biomass transfer was used increased from 196 m2 to 252 m2 over five seasons for extension farmers, and from 79 m2 to 344 m2 for research farmers. In the pilot zone, most users of the practice were in middle-wealth categories: only 15 % of farmers in the two lowest-wealth categories had tried the practice, while about 45 % of farmers in the three highest-wealth categories had tried it. The labour demands for the poorest and female farmers were probably constraining them from adopting the practice. Two key farmer innovations were mixing tithonia with water to prepare liquid fertilizer and mixing tithonia leaves into compost. The main motivation in both cases was to reduce labour requirements (Obonyo, Reference Obonyo2002).
Adoption and farmer assessments of improved fallows
Seventy-nine per cent of farmers planting improved fallows reported increased crop yields after the fallow. Farmer-reported main benefits included improved soil fertility, reduced weeds (especially striga), increased crop production, and firewood. Areas planted to fallows among the initial testers increased from 363 m2 to 511 m2 between 1998 and 2000. Among the testers, the poor were adopting at similar rates to the other wealth groups. Women had smaller plots than men but, over time, they increased their area planted at a more rapid rate than men. Main farmer innovations included mixing shrub species, leaving the fallow plots for less time than recommended before cutting the shrubs and early planting of the shrubs relative to the maize crop (DeWolf et al., Reference DeWolf, Rommelse and Pisanelli2000; Pisanelli et al., Reference Pisanelli, Franzel, DeWolf, Rommelse and Poole2002).
Profitability of biomass transfer and improved fallows
Applications of tithonia leaves to maize had mixed results. In one study, the leaves increased yields by 60 %, but the benefits were not sufficient to compensate for the labour used to cut, carry and apply the leaves. In a second study, application of the leaves increased yields and profits substantially, especially when combined with phosphate fertilizer. Applications of tithonia biomass to kale and tomato were much more profitable for farmers than applications to maize (Rommelse, Reference Rommelse2001).
Crotalaria- and tephrosia-improved fallows generally gave higher returns than continuous cropping, but not in all cases. Risks were relatively low and overall labour requirements of the improved fallow systems were lower than for continuously cropped maize (Rommelse, Reference Rommelse2001).
It should be noted that during these years, no comprehensive impact assessment was undertaken of either practice, because it was considered to be too early. Plot sizes were still small and it was unlikely that any impact on household welfare could be ascertained. The first comprehensive impact assessment of biomass transfer and improved fallows in western Kenya was reported in Place et al. (Reference Place, Adato, Hebinck and Omosa2004). They found that while the practices were used and appreciated by many farmers, no impact on such variables as household assets, welfare and food security could be ascertained, due primarily to the small size of the plots where the practices were applied.
Stakeholders' evaluation
Stakeholders appreciated the monitoring and evaluation (M&E) exercises, as demonstrated by their increasing involvement between 2000 and 2002 (Table 1), the high participation and level of discussion at stakeholder meetings, and the high scores they gave when evaluating the process (Table 6). Participants did not receive any allowances for contributing to M&E exercises or participating in meetings. Their overall satisfaction with the process was therefore not due to increased material benefits from participating. Sixty-three per cent of the respondents thought the process was highly or sufficiently participatory and only 11 % thought that it was insufficiently participatory. The main benefits of the process, according to the participants, were that it documented the spread of the practices, farmers' views and innovations, and that it brought stakeholders together to share experiences. Many also appreciated the skills they learned in designing monitoring forms, keeping records and identifying farmer innovations. Participants also cited two weaknesses of the process: the time and resources required to participate and the need for more leadership from the project to facilitate the process.
Respondents' scoring (1 is low and 5 is high) of the collaborative monitoring and evaluation process, 2002.
DISCUSSION
The collaborative monitoring and evaluation process proved highly popular, as evidenced by the high level of attendance at meetings and the high scores participants gave it in their evaluations. We had expected that staff of large national and international NGOs would be more committed to the process than representatives of CBOs and local NGOs, since staff of national and international NGOs were likely to be more educated and better trained in monitoring and evaluation. However, the organizations most involved and committed to the process were the CBOs, government extension services and local NGOs. Their staff often had little training in monitoring and evaluation, but they had a strong interest in learning such skills. Several of the larger national and international NGOs declined to participate or to share data; they had their own monitoring and evaluation procedures and apparently saw little to gain from collaborating with others.
The collaborative monitoring and evaluation process had several important impacts.
Effects on mutual learning
Many of the participants, especially those from farmer organizations and CBOs, reported that they learned valuable skills in monitoring, evaluation and impact assessment. Awareness of farmer innovations also increased: the number of organizations reporting such innovations rose from 14 to 29 between 1999 and 2001. Over the same period, the proportion of participants reporting that they lacked information about biomass transfer declined from 31 % to 13 %.
Researchers from national and international organizations were obliged to produce simple, easy-to-understand summaries of their studies, called ‘extension briefs’, which they would not have done had they not been involved in the collaborative process. These contributed to increased information flows, not only from researchers to other stakeholders, but also in terms of feedback given by these stakeholders to research and extension organizations.
Efficiency of working together as compared to working individually
Measured in terms of resources used, the collaborative monitoring process did not result in any savings to the project, as compared to if the monitoring had been done by the project alone, through a staff member or student conducting a monitoring survey. Rather, the collaborative process involved higher costs. The main costs of the process were the salary of the monitoring and evaluation specialist, the three collaborative meetings held to plan the two data collection exercises and review results, and field expenses (vehicle, fuel and daily allowances). Salary and field expenses would likely have been about the same had the project conducted the monitoring exercise itself. The extra costs of the collaborative process, as compared to working alone, were the three collaborative meetings, which cost about US$3000 each. Moreover, unrecorded ‘transaction costs’, making contacts and arranging meetings, were much higher than they would have been if the project had monitored on its own. The extra savings of the process were that the staff of the organizations themselves, not the staff of the project, collected data from their farmers. But the organizational staff needed training to do this and the extra costs of providing training was probably about the same as the extra savings from having organization staff collect data. Over time, the training costs would have gone down, but as the data collection exercise was infrequent (once every two years) some training would have always been needed.
Effects on the actions and policies of the participating organizations
Survey data provided clear evidence of the increased degree to which participating organizations promoted the practices and the innovations they learned about in the monitoring and evaluation workshops. Extension organizations became more interested in:
• identifying and reporting on innovations
• finding out from farmers, colleagues and researchers about their value
• if valuable, promoting them.
The number of organizations promoting specific innovations increased from 11 in 1999 to 26 in 2001. That these innovations were then adopted by farmers is confirmed by data from other studies conducted later, after the collaborative M&E exercises ended. For example, Kiptot et al. (Reference Kiptot, Franzel, Hebinck and Richards2006) assessed farmers' use of tithonia in compost, the most common innovation promoted by organizations attending the monitoring workshops (Table 5), among farmers who had tested improved fallows or biomass transfer in the 17-village pilot zone referred to above. They found that the proportion of farmers using tithonia in compost increased from 0 in 2001, to 3 % of farmers in 2002, to 20 % of farmers in 2003. There were certainly other factors encouraging organizations to promote innovations (e.g. informal contacts with other organizations), but the M&E process played a leading role.
The farmer innovations identified in the M&E process also led to changes in researchers' priorities. Several trials were initiated on mixing shrub species in improved fallow trials, in collaboration with farmers who had started their own investigations. These trials led to recommendations for mixed fallows that increased farmers' returns and reduced the risk of a single species failing because of pests or disease (Gathumbi et al., Reference Gathumbi, Cadisch and Giller2004).
Effect on enhancing social capital
The collaborative M&E exercise served to improve partnerships among the organizations across a range of research and development activities, not just in M&E. Stakeholders realized the benefits of working together on M&E and a range of other project-financed activities, such as field days and training workshops. In 2001, the organizations joined in the formation of the Consortium for Scaling Up Options for Increased Agricultural Productivity (COSOFAP), a network of partners seeking to improve and better coordinate their assistance to small-scale farmers (Njui and Wambwile, Reference Njui and Wambwile2003). The 2001 M&E exercise, while funded by the project, was one of COSOFAP's first activities. By 2003, the consortium numbered over 70 organizations and had secured donor funding for information sharing, capacity building and advocacy. Facilitating M&E was one of the consortium's 10 objectives and 18 strategic elements, and the collaborative M&E exercises reported in this paper were listed among COSOFAP's achievements (Njui and Wambwile, Reference Njui and Wambwile2003). The collaborative M&E process was thus one of the main elements leading to the formation of COSOFAP, which has made major contributions to smallholder development in western Kenya. The process was quickly institutionalized in COSOFAP and appeared to be set to continue in its new institutional home, using funds from its donor-financed project.
But, as successful as stakeholders thought the M&E process was, and as effective and as institutionalized as it appeared to be, the process proved to be unsustainable. In the end, COSOFAP decided not to finance the process. Rather, stakeholders at the 2003 annual meeting decided to give priority to activities that they felt would more directly benefit their members, such as development of a product marketing service, staff training and expansion of field school learning sites (Njui and Wambwile, Reference Njui and Wambwile2003).
COSOFAP's unwillingness to pay for a collaborative M&E process should not be viewed as an indicator of the process's worth, just as their unwillingness to construct rural roads should not be viewed as an indicator of roads’ value. Several measures could have been taken to save the process, and are offered as possibilities for other projects and agencies to help sustain collaborative M&E efforts.
• Some COSOFAP members considered the M&E process to be a public good to be paid for by government-financed research and extension, as part of the technology development process that government pays for. In some places, research services may be able to cover the costs of such M&E processes.
• Alternatively, researchers and donors could have lobbied more for including the collaborative M&E process in COSOFAP's budget. Donors could have insisted that COSOFAP conduct the process as a condition for receiving funds and researchers could have insisted that they give assistance in developing technology on condition that they receive feedback from a collaborative M&E process.
• The costs of the process could have been greatly reduced by combining the stakeholder meetings with meetings of COSOFAP members on other issues. One-day meetings could have been held, instead of the 2- to 3-day meetings which required covering the costs of room and board. The use of an MSc student in the 2001 exercise, in place of the ICRAF staff member used in the 1999 exercise, greatly reduced costs.
• The larger organizations participating in the consortium could have been solicited to take turns paying for the monitoring meetings, as was successfully done in the Adaptive Research and Development Network for Agroforestry in eastern Zambia (Katanga et al., Reference Katanga, Phiri, Böhringer, Mafongoya, Kwesiga, Ayuk and Agumya2002).
Our experiences showed that collaborative M&E promoted mutual learning among organizations, enhanced partnerships, social capital and M&E skills, and helped promote practices and innovations learned in the M&E workshops. It is not likely that any of these developments would have resulted from a conventional M&E exercise. But, at the same time, it must be recognized that collaborative M&E involves more costs and coordination than conventional M&E, especially for participant workshops and capacity building. In our case, the extra costs roughly doubled the cost of M&E. We feel that the benefits were far greater than the costs and strongly recommend that organizations promoting similar innovations in similar environments conduct collaborative M&E. Most of the stakeholders who were involved in the collaborative M&E process, though benefiting from and appreciating it, were unable or unwilling to contribute financially to supporting the process. This highlights the difficulty in sustaining such processes unless they are financed by the public sector.
The results suggest several avenues for future research. A weakness of the exercise was that there was no control group of practices being promoted, that is, practices that the organizations were promoting, but that were not included in the collaborative M&E. Including such a control group would have strengthened the impact assessment, as researchers would have been able to assess, for example, whether innovations increased over time because of the collaborative approach or because of other factors. Such control groups should thus be included in future assessments. Another important research idea would be to compare a collaborative approach with a consultative approach, in which an organization conducts M&E in an area where many organizations are promoting a practice and consults periodically with them. A key research question would be whether exchanging experiences about promoting a practice in consultations and meetings would lead to as many benefits, e.g. initiatives to identify and promote innovations, as would conducting M&E jointly.
