INTRODUCTION
Participatory plant breeding (PPB), participatory varietal selection (PVS) and participatory natural resource management (PNRM) emerged in the early 1980s as potential solutions to the problem of limited adoption of improved varieties and natural resource management technologies by farmers in developing countries (Ashby, Reference Ashby, Pound, Snapp, McDougall and Braun2003; Farrington, Reference Farrington1988). The empirical studies show that participatory methods are being applied in strategic, applied and adaptive stages of research (Johnson et al., Reference Johnson, Lilja, Ashby and Garcia2004; Weltzein et al., Reference Weltzein, Smith, Meitzner and Sperling2000) and there is empirical evidence in support of their effectiveness in terms of improved farmer acceptance of crop varieties and resource management techniques (e.g. Bellon et al., Reference Bellon, Berthaud, Smale, Aguirre, Taba, Aragon, Diaz and Castro2003; Ceccarelli et al., Reference Ceccarelli, Grando, Singh, Michael, Shikho, Al Issam, Al Saleh, Kaleonjy, Al Ghanem, Al Hasam, Dalla, Basha and Basha2000, Reference Ceccarelli, Grando, Bailey, Amri, El-Felah, Nassif, Rezqui and Yahyaoui2001, Reference Ceccarelli, Grando, Tutwiler, Baha, Martini, Salahieh, Goodchild and Michael2003; De Jager et al., Reference de Jager, Onduru and Walaga2004; Dorward et al., Reference Dorward, Galpin and Shepherd2003; Ortiz et al., Reference Ortiz, Garrett, Heath, Orrego and Nelson2004; Witcombe et al., Reference Witcombe, Joshi, Joshi and Sthapit1996) and impact (e.g. Dalton et al., Reference Dalton, Lilja, Johnson and Howeler2005; Johnson et al., Reference Johnson, Lilja and Ashby2003; Joshi and Witcombe, Reference Joshi and Witcombe2003; Onduru et al., Reference Onduru, de Jager, Van Der Werf and Diop2002; Smale et al., Reference Smale, Bellon, Rosas, Mendoza, Solano, Martinez, Ramirez and Berthaud2003; Witcombe et al., Reference Witcombe, Petre, Jones and Joshi1999, Reference Witcombe, Joshi, Rana and Virk2001). There are a number of explanations for the observed effectiveness of participatory methods in accelerating adoption and impact, which are explored in the papers that follow in this special issue of Experimental Agriculture.
The integration of gender analysis into agricultural research processes began over 30 years ago, when Boserup (Reference Boserup1970) initiated inquiry into the role of women in economic development. She reported empirical analysis of women's participation in agriculture and its links to the evolution of farming systems, to population pressure, technological change in agriculture and the participation of women in the labour force. As a result of the increased understanding of gender issues in agricultural systems research and development, women, and later broader gender considerations, were integrated into agricultural projects – aspects of the experience are reported in several empirical studies (e.g. Adesina and Djato, Reference Adesina and Djato1997; Doss, Reference Doss1999; Jewett, Reference Jewett2000).
However, there is growing evidence of limited impact of agricultural research impact assessment on project identification, approval and implementation management, or on poverty reduction itself Footnote 1 (e.g. Appleton and Hill, Reference Appleton and Hill1994; Richards, Reference Richards1995); and within the context of the Consultative Group on International Agricultural Research (CGIAR) this is perhaps associated with the evolving role of social sciences (Cernea and Kassam, Reference Cernea and Kassam2006; Kassam, Reference Kassam2006; Kassam et al., Reference Kassam, Gregersen, Ferreres, Javier, Harwood, De Janvry and Cernea2004). A variety of innovations, which potentially complement existing methods of impact assessment, offer the potential of increasing the value of the results of impact assessment. Some of these methods use participatory approaches, which are being used more widely in agricultural research itself. Greater use of participatory and gender analysis approaches in agricultural research has significant conceptual and methodological implications for impact assessment. First, impact assessors must document a much broader range of project impacts, especially in the areas of poverty alleviation, gender, social capital and environmental sustainability. Second, the number of stakeholders in impact assessment has grown dramatically, and now includes research management, researchers, donors, partner institutions, beneficiaries and civil society organizations – different stakeholders demand different types of information, in different formats. Third, in the past impact assessment results have served primarily as a management tool and an accountability measure to evaluate past investments in agricultural research and to set research priorities for future investment, but with growing acceptance and mainstreaming of participatory and multi-stakeholder paradigms, impact assessment is increasingly seen as a tool for institutional learning and change that has close links to ongoing project monitoring and evaluation processes. Nevertheless, a recent empirical study based on donor survey data (Raitzer et al., Reference Raitzer, Kelley and Gregersen2005) highlights a paradox between strong demand from the donor community for (especially poverty-related) impact assessment evidence, and yet limited direct influence of impact evidence on funding decisions – perhaps in part because of the time lag between research and measurable people-level impacts.Footnote 2
These changes in agricultural research practices and the policy and institutional environments, as well as in how impact assessment is viewed and practised, have led many scientists to find impact assessment increasingly challenging. Some observers express dissatisfaction with the scope and focus of past impact assessment approaches and practices, partly in terms of understanding how effective and relevant agricultural research is in addressing the needs of the world's poor, but especially its contribution to institutional learning and change processes. In the light of the changing environment for impact assessment of international agricultural research, the CGIAR Systemwide Program on Participatory Research and Gender Analysis for Technology Development and Institutional Innovation (PRGA Program) and the International Maize and Wheat Improvement Center (CIMMYT) co-hosted a workshop in October 2006, at which some 30 empirical studies were presented, six of which were selected for this special issue of Experimental Agriculture.
This overview article is organized as follows: after discussing some of the conceptual and methodological issues in impact assessment of participatory research and gender analysis, we briefly introduce each of the six papers included in this special issue, and highlight how to respond to some of these challenges. We conclude by offering some final remarks about the overall contribution of this set of papers.
conceptual challenges
Organizational transformation
Poverty is diverse, complex and dynamic (Ellis, Reference Ellis2000); it is not a mechanistic process for agricultural research to generate improved germplasm and resource management technologies with widespread application in the highly variable conditions typical of the livelihood systems of the poor. The poor not only have multiple and complex sets of livelihoods, but also suffer from high levels of vulnerability to agro-ecological and socio-economic shocks and, in general, they posses few means of managing such risk (Dixon et al., Reference Dixon, Gulliver and Gibbon2001). Public sector agricultural research can be organized in different ways to cope with the complexity and risk faced by its clients. The training-and-visit (T&V) system of extension, previously promoted as a ‘transfer-of-technology’ or ‘pipeline’ model, can still be found. Although the T&V model has been harshly criticized, it can be effective when the probabilities for controlling variability in the physical and socio-economic environment are good (irrigated environments, good input supply, fixed product prices) (Dalton and Guei, Reference Dalton and Guei2003; Evenson and Gollin, Reference Evenson and Gollin2002), as was the case with the spread of many of the Green Revolution plant varieties in Asia.
The world today is different from the one that existed when the Green Revolution began in the late 1960s. Since the mid-1990s, the locus of agricultural research and development has shifted dramatically from the public to the private multinational sector. According to Pingali (Reference Pingali2007), three interrelated forces are responsible: the first is a stronger and evolving environment for protecting intellectual property in plant innovations; the second is the rapid pace of discovery and growth in importance of molecular biology and genetic engineering (potential of ‘gene revolution’); the third is that agricultural input and output trade is becoming more open in nearly all countries. Pingali (Reference Pingali2007) argues that these developments have created a powerful new set of incentives for private research investment, altering the structure of the public–private agricultural research endeavour, particularly with respect to crop improvement. Studies show that the crops that are dominating private sector interests are mostly soya bean, maize, cotton and rapeseed (canola) (James, Reference James2004, Reference James2005). Private sector research has focused primarily on two production traits: borer resistance and herbicide tolerance. Absent from private sector research, not surprisingly, are many crops cultivated by the world's poor, including small-grain cereals, tubers and legumes. This sets the current agricultural research scene in a very simplistic way: private agricultural research is narrowly focused geographically, on a narrow crop basis and on production and not consumption traits. Some researchers argue that the current trend of amalgamation in global food systems that is being driven by the process of globalization is accelerating a two-tiered production system in which small-scale food producers will be neglected in future benefits from agricultural research (Pingali, Reference Pingali2007). It also provides an additional justification of why targeted research on the distributional benefits of agricultural research is important.
In order to target their research better to serve the needs of the poor and to reach the Millennium Development Goals (MDGs), many public agricultural research institutions have now adopted the use of participatory research models as a way of generating and sustaining a rapid rate of innovation, adoption and adaptation, especially in highly uncertain and variable environments, and in some cases focusing on the crops neglected by private sector research. Reviews have shown that projects that allow resource-poor farmers to choose, design and adopt/adapt appropriate technologies that will help them survive the harsh drought- and pest-prone conditions quickly report successes (Lilja and Bellon, Reference Lilja and Bellon2006). Impacts of these projects are often significant (Scoones and Thompson, Reference Scoones and Thompson1994), and are often achieved more rapidly than with the T&V model. Success is often not found in use of the agricultural technology alone, but rather in its grounding in and building of human and social capital – confidence, knowledge, networks and capacity – which then allow technologies to have a full effect on livelihoods. In this case, the challenge is how to scale up these individual successes.
It follows that one challenging institutional question is whether the T&V and participatory models can effectively co-exist in a public agricultural research institution (Morris and Bellon, Reference Morris and Bellon2004). It has been argued that scaling up and out the impact of the agricultural research that benefits the poor requires change in the way the current agricultural innovation system is organized (Rhoades, Reference Rhoades, Cernea and Kassam2005). A shift from a T&V agricultural research model to participatory research and extension does not just require the adoption of a simple set of techniques such as participatory rural appraisal, farmer visits, farmer evaluations, farmer experiments – these are necessary, but not sufficient, conditions. It will also require an organizational change process to occur (Biggs and Smith, Reference Biggs and Smith2003). Most importantly, a complementary process must occur in partner and users’ organizations, that is, in national agricultural research systems (NARS) and local farmer organizations (Scoones and Thompson, Reference Scoones and Thompson1994).
From the economics perspective, the increasing use of participatory and gender analysis methods provides a new challenge in measuring the sources of change. Appropriate analyses are very different from the standard techniques that are used in many rate of return studies that are the mainstay of economics professionals. New challenges are posed for research evaluators, because these research approaches are responsive to changing community needs. Consequently, one of the most pressing challenges is to develop flexible participatory and systems-based evaluative processes, which allow for ongoing learning, correction and adjustment of the processes themselves by all parties concerned. Traditionally, impact assessment is conducted by external persons to provide objectivity and unbiased results, but this view is beginning to change, and researchers and end-users are becoming key participants in the impact assessment process in order to feed the results into improvements in practice (Hall et al., Reference Hall, Yoganand, Sulaiman and Clark2004).
As the number of stakeholders in impact assessment increases, the challenge is what should be evaluated and how. Mitchell et al. (Reference Mitchell, Agle and Wood1997) characterize stakeholders according to the attributes of power, legitimacy and urgency. Stakeholder salience, or importance, depends on the level of expression of each of these three attributes. Different stakeholders have different needs from impact assessment, and the evaluation design needs to take into account the specific needs of different stakeholders, their dynamics and, most importantly, the salience of each stakeholder. These may influence not only the assessment design, but also the methods – for example, affecting the choice of indicators and the measures of success or failure (Kristjanson et al., Reference Kristjanson, Place, Franzel and Thornton2002). Unfortunately, empirical examples of fully collaborative impact assessment are rare.
Multiple goals of participatory research
There are two major sets of goals associated with participatory research. One is to improve the efficiency of the research process by involving its intended beneficiaries in various stages of the process to generate more relevant and appropriate research products – that is, a functional goal or objective (Ashby, Reference Ashby1996; Pretty, Reference Pretty1994), and common examples of this are PVS (e.g. Ceccarelli et al., Reference Ceccarelli, Grando, Bailey, Amri, El-Felah, Nassif, Rezqui and Yahyaoui2001, Reference Ceccarelli, Grando, Tutwiler, Baha, Martini, Salahieh, Goodchild and Michael2003; WARDA, 1999, Reference Webb2000, 2001, Reference Witcombe and Virk2002; Witcombe and Virk, 2001; Witcombe et al., Reference Witcombe, Joshi, Joshi and Sthapit1996, Reference Witcombe, Petre, Jones and Joshi2001) and mother–baby trials (e.g. Snapp et al., Reference Snapp, Freeman, Simtowe and Rohrbach2002). The second goal is to empower intended beneficiaries by supporting the formation of groups capable of assessing their own needs and addressing them, whether directly or indirectly (Ashby, Reference Ashby1996; Okali et al., Reference Okali, Sumberg and Farrington1994). This approach is claimed to be particularly relevant where markets, social and biophysical contexts are often complex and where the technical solutions require adaptation to the specific situations (Weltzien et al., 2000).
The use of functional participation in agricultural innovation itself does not necessarily call for changes in impact assessment methodology. Whether or not there is farmer participation in the technology development process, agronomic outcomes at the farm level can be assessed in terms of yield changes, loss of soil or soil nutrients, or changes in pest or weed pressures. Economically, the sustainability of cropping systems can be assessed at the farm level by looking at net income over time, amenity gains, increased positive externalities or mitigated negative externalities (such as soil erosion or nutrient loading). Involving farmers may increase or decrease costs during the technology development process (Johnson et al., Reference Johnson, Lilja, Ashby and Garcia2004; Lilja and Aw-Hassan, Reference Lilja and Aw-Hassan2003) and much depends on whether or not the participatory activities replace activities in the conventional research process or if they are included as additional activities, therefore generating additional costs. Once developed, however, the technologies are typically diffused through conventional channels.
Empowering participation, however, does have significant implications for how impacts are generated and measured (Dalton et al., Reference Dalton, Lilja, Johnson and Howeler2005). As with conventional technologies, benefits can still be quantified in terms of increased agricultural productivity or reduced environmental damage; however, the sources of the benefits are of two types. Part of any observed increase in productivity can be attributed directly to the superiority of the new technology or practice. These are often referred to as ‘embodied’ effects, since they are part of the technology itself (Chambers, Reference Chambers1988). The second source of improved productivity is the increased knowledge or capacity that the farmer obtains by participating in the research process. These are often referred to as ‘disembodied’ effects because they are not part of the technology. These two types of impacts are not independent, since a more knowledgeable farmer can make better use of a new technology. Therefore, it is important to be able to separate the embodied and disembodied effects in order to evaluate accurately the impact of both the participatory research process and the technology.
In addition to farmer-level impacts, participatory research can increase researchers’ technical knowledge of cropping systems as well as their appreciation of farmer knowledge, priorities and constraints (Johnson et al., Reference Johnson, Lilja and Ashby2003; Weltzein et al., Reference Weltzein, Smith, Meitzner and Sperling2000). These kinds of impacts can have important long-term effects on the productivity of national and international research systems and the expansion of available technologies, by creating an institutional capacity to develop improved techniques of production and improved varieties (Maredia et al., Reference Maredia, Byerlee and Pee1998). It may be difficult to quantify the benefits associated with institutional changes; however, impact assessment processes should try to document and characterize these intermediate impacts and identify factors that constrain change, and analyse their potential implications for institutional performance, and ultimately the impact on end-users.
Gender analysis in agricultural research and development
Nearly three decades of research on the role of women in economic development have concluded that households in many developing countries are diverse – they pursue numerous agricultural and non-agricultural enterprises. In addition, households include people with competing goals and objectives, cooperating fully in some issues and less on others (e.g. Doss, Reference Doss1999). Studies have highlighted the ‘double burden’ of work already carried out by women, and the low status of women that limits their access to education and extension (Due et al., Reference Due, Sikaponde and Magayane1991; Saito and Weidermann, Reference Saito and Weidermann1990). In addition, some studies have found that female access to land, credit, machinery, technology and markets for their products (Baser, Reference Baser1988; Buvinić and Mehra, Reference Buvinić, Mehra, Eicher and Staats1990) and control over outputs or any earned income (Jones, Reference Jones1983; Lilja and Sanders, Reference Lilja and Sanders1998) are limited. Empirical studies have provided evidence that gender affects farmer preferences for characteristics of agricultural technologies as well (Lilja and Dalton, Reference Lilja and Dalton1997). Gender relations are dynamic and respond to economic incentives and opportunities, and several empirical studies have shown that individuals with more power and access to resources are in a better position to take advantage of new agricultural opportunities than those without (Carney, Reference Carney1998; Dey, Reference Dey1985; Jones, Reference Jones1983; Webb, Reference Webb1989).
Agricultural research project goals and objectives related to social policy often refer to desires to improve the income of specific groups (poor, women) or geographic regions, and to make incomes more stable. Drawing boundaries for an agricultural research project's realistic impact on women, as well as poverty can be challenging.
The recent interest among public research institutes in ‘innovation processes’ tends to accept uncritically that technology is the main driver of change, and that motivation is economic self-interest. It thereby ignores the extensive ethnographic, sociological and economic literature that demonstrates more complex, multi-source change processes. Varietal development indeed contributes to growth in agricultural production, which is important for improved welfare and overall economic development policy in many countries. Yet, plant breeding and natural resource management practices are very blunt tools for social change and, although outputs from such research can have significant distributional implications (Jones, Reference Jones1983; Lilja and Sanders, Reference Lilja and Sanders1998), food security and long-term sustainability of the production systems are more obtainable objectives. The implications of social and institutional complexity are that any strategy for research and development intervention always needs to investigate the context in which it can work effectively.
Social change goals may be more effectively addressed by other available policy tools – for example, efforts targeted to removing gender bias in education or access to credit (Baser, Reference Baser1988; Buvinić and Mehra, Reference Buvinić, Mehra, Eicher and Staats1990; Due et al., Reference Due, Sikaponde and Magayane1991; Saito and Weidermann, Reference Saito and Weidermann1990). Given the socio-cultural constraints women face, simplistic participatory approaches alone are unlikely to empower them (Jewett, Reference Jewett2000).
Time lags and attribution
Agricultural research, and especially international agricultural research, is a very long-cycle activity; therefore, the measurement of actual impacts on fields and farm households necessarily occurs many years after the initial investment in agricultural research. Taking the example of international wheat breeding, the release of varieties often occurs 5–10 years after breeding, and ‘full’ adoption in suitable areas may require a further 5–10 years (Dixon et al., Reference Dixon, Nally, Aquino, Kosina, Rovere and Hellin2006, drawing on Brennan and Byerlee, Reference Brennan and Byerlee1991) depending on a wide variety of scale, farming system and institutional factors. Whilst some large-scale impact assessments are conducted after full adoption (e.g. Lantican et al., Reference Lantican, Dubin and Morris2005), a large proportion of ex-post impact assessments are conducted in the early or mid-stages of adoption and thus directly observe only a portion of the ultimate benefits.
The long time to impact is often associated with multiple pathways – for example, seed multiplication and distribution by various agencies and development initiatives, farmers may acquire the knowledge of production practices from a variety of sources including neighbours, agricultural policies may change and relative input prices may shift – all of which aggravate the specification of plausible models of attribution. These amount to a complex and dynamic institutional and economic context for adoption and impact, which also increases the degree of difficulty in the specification of a credible counter-factual.
However, the increasing use of participatory methods in technology development may help to alleviate this issue for impact assessment. As is noted in some of the ensuing papers, PPB may accelerate the development of suitable improved lines and varieties which meet more of farmers’ expectations. Similarly, PVS can speed up the selection and testing of varieties (Ceccarelli et al., Reference Ceccarelli, Grando, Singh, Michael, Shikho, Al Issam, Al Saleh, Kaleonjy, Al Ghanem, Al Hasam, Dalla, Basha and Basha2000, Reference Ceccarelli, Grando, Bailey, Amri, El-Felah, Nassif, Rezqui and Yahyaoui2003). Both processes can, in principle, be augmented by gender analysis. In this way, participatory research and gender analysis can shorten the time from breeding to adoption and impact of improved varieties, and therefore alleviate (but not eliminate) the issue of time-lag from research to impact.
The close engagement with stakeholders during PPB and PVS may deepen the knowledge of breeders and supporting disciplines of impact pathways, attribution and the specification of a plausible counter-factual.
methodological challenges
Methods selection and integration to promote learning
The conceptual challenges described above bring about some methodological challenges. Impact assessment and project monitoring and evaluation procedures have been formalized for different types of agricultural research. For example, in order to capture the impact of plant breeding on farm productivity and consumer welfare, we typically calculate estimates of variety adoption and rates of return. These will provide a measure of profitability of investment made in plant breeding activities in a given project. However, given our expanded understanding of the concepts of poverty, participation and gender analysis, these methods alone are not sufficient to capture the impacts on poverty, differential impacts based on gender or the impacts of user participation in agricultural research per se – in terms of, for example, empowerment or increase in human capital due to participation. This has implications for the selection of methods. Even more challenging, in order to understand better these types of impacts, there is an increasing interest in more integrated impact assessment – that is, the use of various combinations of quantitative, qualitative and participatory methods in impact assessment and evaluation (Baur and Kradi, Reference Baur and Kradi2001; Campilan et al., Reference Campilan, Prain and Bagalanon1999; Cromwell et al., Reference Cromwell, Kambewa, Mwanza and Chirwa2001; Douthwaite, Reference Douthwaite2002; Farrington et al., Reference Farrington, Thirtle and Henderson1997). As a practical strategy, many impact assessors rely on the well-known process of triangulation, and accumulate a wide range of data from conventional and non-conventional sources.
Measuring impact of participation vs technology
A farmer's decision to adopt a technology is separate from the decision to participate in the project activities, since participants can choose not to adopt and non-participants can choose to adopt the varieties. However, the decisions are not independent, because some of the same factors that influence the decision to participate are likely also to influence the decision to adopt. The methodological challenge to distinguish the impact of participation from the impact of adoption of technology itself is not trivial, and requires specification of a model, often econometric, that separates the technology effects from the presumed knowledge effects. Integrated production–consumption–household theory can serve as the general conceptual framework to measure impacts of technology adoption and knowledge. Therefore a household utility function with multi-product consumption and production functions including commodity and non-commodity outputs needs to be formulated, and knowledge can be included as a stock resource to be enhanced by project participation (Dalton, Reference Dalton2004).
When measuring the adoption, both behavioural (e.g. changes in area planted, changes in area under a new variety) and productivity (e.g. change in yields) outcomes need to be considered. Since some of the same farm and farmer characteristics that affect participation and adoption are also likely to influence land allocation and production, the empirical analysis must be done via estimations of sets of simultaneous equations.
Capturing gender outcomes
Many agricultural research and development projects, even when designed to benefit women, have failed because they did not recognize the complexity of women's roles and responsibilities. In her review of past research on gender issues and adoption of maize technologies in Africa, Doss (Reference Doss1999) points out that it is difficult to determine ex ante whether the positive or the negative effect will be the strongest. Therefore, analysis of the benefits arising from the participation alone, and from the technology itself (as discussed above), is not sufficient to understand benefits to women.
The contribution of years of empirical results from gender studies (especially intra-household analyses) guides estimation of gender-disaggregated impacts of agricultural research, and using a dummy variable to capture the gender-differentiated impact is hardly sufficient. Theory of intra-household allocation of resources states that the benefits from the new agricultural technologies are influenced by characteristics of the agricultural technology itself, the opportunity cost of an individual's time and relative bargaining power of each household member. Empirical measurement of these factors is documented in several studies (e.g. Quisembing, 1995). The opportunity cost of women's time (shadow wage rate) is typically difficult to measure in a developing country context. A woman's earning opportunities cannot be measured directly from earned income since participation in other income-earning activities is self-determined. In order to obtain the shadow wage rate estimate, one must estimate a model that predicts potential earnings as a function of individual female circumstances (Heckman, Reference Heckman1979).
responding to challenges
This section briefly introduces the six papers in this special issue and highlights how they respond to some of the challenges discussed above.
Watts et al.note that scores of impact assessments have been carried out in the CGIAR since the 1970s. These have demonstrated high rates of return to agricultural research investments, but their contribution to decision making on resource allocation has been disappointing (Pingali, Reference Pingali2001). Researchers in CGIAR Centres are experimenting with methods for innovative use of impact assessment and evaluation in improving agricultural research programmes. Emphasizing learning seeks to add value to conventional impact assessment, and can be accomplished by encouraging the complementary use of a variety of new methods that capture dimensions and complexities inherent in impact pathways, innovation and institutional histories and partnerships in international agricultural research. Research impacts are the result of many actors and factors, and they should rarely be fully attributed to any small group of actors in the innovation system. Within an integrated context, the authors propose a different set of criteria, such as the following:
• Does the assessment focus on key research questions relevant to target audiences, stakeholders and managers?
• Does it go beyond measuring the magnitude of impacts to assess the processes and pathways by which impact is achieved in the context of an innovation system? Similarly, does it assess the roles that different actors and ‘institutions’ play in achieving impact? Does it use collaborative approaches to interpret findings and develop recommendations?
• Does it take into consideration changes in capacity, institutions and policies?
Biggs argues that opportunities for reducing poverty, improving social inclusion, and influencing policy and institutional changes are being missed as a result of not learning sufficiently from development situations where there have been positive changes in the past. He distinguishes the processes of technical and institutional innovation, and illustrates through case study examples how institutional innovations came about and gave rise to positive development outcomes. Implications for innovations theory and rural development are discussed. From these cases, he derives some general lessons for innovation theory and for rural development practice, including: institutional innovation processes are different from technical innovation processes; institutional innovation is location- and time-specific; there are multiple sources of innovation; the contending coalitions of actors who create the innovation are, surprisingly, often temporary; transparency on the ground is critical; the central role of agency and personal behaviour; attitudes towards development interventions are important; flexibility to encourage and support building on the positive; broadening monitoring to include positive deviance analysis; production and circulation of ethnographies of positive case studies.
Bantilan and Padmaja explore the importance of social capital creation for agricultural research for development in general, and for the adoption of groundnut technology in particular, and the different processes and outcomes for women and men. The focus of the paper is on social capital as a mechanism to stimulate gender-equitable change processes. They systematically document the process of empowerment, whereby the marginalized groups (including women and tribals) gained better access to resources, information, knowledge and some opportunities for political participation. Collective action was enhanced with the increased involvement and participation of women. Kinship ties were more common among women's networks than among men's. The study shows that technology uptake improved as rural Indian farmers were empowered through building social capital, in particular, where social networks crossed caste and class barriers. The process of build-up of social capital played an important role in influencing the distribution of benefits from the technology, because of the ways in which social networks and social relationships facilitated technology dissemination. Thus, social networks played a crucial mediating role in the process of technology uptake.
Hellinet al. discuss functional and empowering objectives of the use of farmer participatory methods in crop improvement research. Participatory crop research can contribute to increased research efficiency through improved understanding of farmers’ crop genetic resources management, and lead to better targeting of research and policy, as well as practical recommendations for development interventions. Although empowerment may be a by-product of farmer participatory research, it has a greater role in scaling up and development than in a majority of crop improvement research per se. The majority of participatory agricultural research projects, including the illustrations of farmer participatory projects discussed in the paper (notably a CIMMYT project in Oaxaca), focus primarily on understanding the challenges farmers face, their practices and priorities, and finding sustainable and viable solutions; secondary foci are on empowering the farmers directly engaged in the research process (in contrast to empowerment as part of a scaling up process). The authors argue that scaling up the benefits of this research is more cost-effectively carried out by organizations engaged in longer-term development work. In terms of the empowerment of large numbers of farmers, the strength of research organizations may be in ‘research’ (i.e. research on improved ways to empower farmers and communities) rather than in empowerment per se. When the objectives of participatory research are primarily functional, impact assessment should focus primarily on the impacts of the participatory research on other research-for-development outcomes in a broader sense. This would include impacts in terms of policy development, technology development, extension efficiency and partner organizations’ capacity. In the infrequent cases where empowerment is an explicit objective of the participatory research process, albeit often secondary, impact assessment should direct attention to the impacts of the participatory research process on the skills, organizing capabilities, initiatives and livelihoods of the participating individuals and communities.
Pariset al. assess the impact of gender analysis on the research process and outcomes, particularly on women's empowerment. They demonstrate that integrating farmer participatory breeding and gender/stakeholder analysis, and paying greater attention to women in relation to men, increased women's confidence in making decisions related to varietal choice, acquisition and disposal. Participation of both men and women in the early evaluation of the performance of the rice lines/genotypes on their own farms led to the further development of varieties that are suited to their fragile environments. However, involving women in PVS is only the first step towards reducing gender disparity in access to new seeds and new techniques in crop resource management. Ongoing efforts to provide women farmers with new knowledge and skills in conserving and managing seed, as well in crop resource management, show that adoption is accelerated by a farmer/community participatory approach which involves ‘learning by doing’ and ‘farmer to farmer’ training or exchange of information. With the increasing emigration of men to the cities, the roles of women will shift from unpaid labour to de-facto farm managers. Thus, women should be actively involved in farmer participatory research.
Franzelet al. document the operational experience – the ground-level realities – of a group of diverse stakeholder organizations working on participatory monitoring and evaluation. Although increase in use of participatory research methods requires formation of partnerships, this type of multi-stakeholder collaborative impact assessment requires management and incurs transaction costs. There is little literature on collaborative monitoring and evaluation, that is, how diverse stakeholder organizations promoting similar innovations can together assess their efforts. However, there are many published examples of ‘participative evaluation’ approaches in the adult education development intervention literature (Stern, Reference Stern1993). Franzel et al. examine the experience of 30 organizations in western Kenya, from 1999 to 2003, collectively assessing their impact in helping farmers to adapt and adopt two soil fertility practices. While the collaborative process improved the flow of information among organizations, it did not reduce monitoring costs – rather, it increased them. The process increased participating organizations’ awareness of farmer innovations and, more importantly, the number of farmer innovations being promoted. The process also contributed to the formation of a consortium among the participating organizations for increasing agricultural productivity. However, the decision of the consortium not to fund collaborative monitoring and evaluation highlights the difficulty in sustaining the process unless it is financed by the public sector.
concluding remarks
At the global level, slow progress on the MDGs challenges impact assessors to map out effective impact pathways to achieve the Goals quickly. Similar challenges exist for a multitude of projects and micro-initiatives. However, evidence is growing of the limited use of the knowledge derived from ex-post impact assessment in project identification, approval or management. If we find ways to investigate and understand how different factors have contributed to a wide range of impacts and outcomes of research and development interventions, we are opening up ‘the black box’ and the results of all old economic rates of return and impact assessment studies will need to be reassessed and recalculated. Such assessments might help to shed light on why past economic impact studies have had such mixed effects on agricultural and natural resources policy and development processes. This set of papers moves ahead the debate on some key aspects of impact assessment – reflecting on innovative approaches and methods from several regions – and its role in adding value to agricultural research.
The authors included in this special issue have responded to various aspects of the conceptual and methodological challenges described in this introduction. Numerous experiences and innovative uses of methods are presented in these papers, in addition to discussion on the institutional issues of impact assessment related to moving from project-level learning to institutional learning. The papers presented here will no doubt provide evidence that, when used well, participatory approaches and methods have their own rigour in research and technology development and can generate not only qualitative insights, but also quantitative data that can accurately document project and programme impact. In summary, the papers underscore a number of lessons, such as: the importance of augmenting accountability with internal learning and lessons for scaling out; the overlap of monitoring and evaluation and impact assessment; the intrinsic nature of functional and empowering farmer participation; the persistent lack of widespread attention to gender; and the operational and political complexity of multi-stakeholder impact assessment.
The papers also suggest a number of key implications for the research-for-development community. Impact assessors have a whole range of formal and informal methods from which to choose those most suited to the needs of the project and community, taking into consideration the cultural, social, political and environmental realities of the project, the stakeholders and the communities. Both implementation and interpretation are complex, with multiple ‘layers’ of stakeholders and alternative attribution and counter-factuals. An important role for the public agricultural research community is refining methods for challenging application areas (such as Waibel and Zilberman, Reference Waibel and Zilberman2007) and building capacity and quality for impact assessment. Given that donors often make limited direct use of findings of impact assessment, more effort is required in communicating results and implications, perhaps in innovative ways. Despite decades of work, gender analysis is still viewed by many as a sideline activity within agricultural research; there is a continuing need to raise awareness among biophysical researchers of the value of gender-sensitive and participatory approaches to research.
Noting that the primary objective of most international public agricultural research institutions is the production of international public goods, a shift of focus is in order: from the historical focus on technologies alone, to an international public goods portfolio including knowledge and methods. This can only be achieved if the participatory and gender analysis research is broadened and better understood, and the implications for science and technology mainstreamed in various ways in those same institutions.
Acknowledgements
We gratefully acknowledge the editorial assistance from Guy Manners in revising the set of papers presented in this special issue.
