INTRODUCTION
Over the past decades, the context of Agricultural Research for Development (AR4D) has changed considerably. The rise of the participation agenda, increasing private sector influence, philantrocapitalism, and a growing donor focus on generating large-scale farmer impact have significantly influenced the nature of research funding (Sumberg and Thompson, Reference Sumberg and Thompson2012). AR4D organisations are encouraged to demonstrate impact at scale, preferably as quickly and efficiently as possible (Glover et al., Reference Glover, Sumberg and Andersson2016). This new funding environment has come with increased use of development indicators and metrics that are quantified outside direct research efforts, such as ‘farmer adoption’, ‘training and participation rates’, among others. In this context, many funding agencies increasingly base their investments on how effectively research results contribute to metrics of use, as evidenced by funders’ use of terms like ‘outcome investing’ (BMGF, 2016) and ‘value for money’ (OECD, 2016). Overt focus on such metrics with limited nuanced interpretation risks a reductionist perspective of technological change, potentially disregarding farmer technology adaptation, dis-adoption and the external drivers of technological change (Glover et al., Reference Glover, Sumberg and Andersson2016; Kiptot et al., Reference Kiptot, Hebinck, Franzel and Richards2007). Consequently, contemporary development-oriented agronomy appears to prioritise the demonstration of positive results, rather than critically reflect on the relationship between applied research methods and development outcomes, i.e. increased agricultural productivity, income and/or food security (Andersson and D'Souza, Reference Andersson and D'Souza2014; Glover et al., Reference Glover, Sumberg and Andersson2016).
For instance, the second phase of the Consortium of international agricultural research centres’ CGIAR Research Programs (CRP) targets that 350 million farmers will adopt improved agronomic practices and varieties by 2030, with the ultimate goal of lifting 100 million people – half of whom should be women – out of poverty (CGIAR, 2016).
Research station trials have long been complemented by on-farm and participatory research. The pressure to move quickly from piloting and refining technologies to demonstrating their impact at scale has, however, led on-farm experiments to be increasingly utilised as a method to both understand and spur farmer technology adoption (Andersson and D'Souza, Reference Andersson and D'Souza2014). In other words, new funding priorities appear to have shifted agronomy from a focus on technology development and evaluation, to the practice of development itself. The way researchers deal with this new situation they are faced with – in terms of their choice of methods and working modalities – has important ramifications for both the research process and outcomes.
In this paper, we aim to understand the challenges that development-oriented agronomists are faced with, and how they are coming to grips with this new context for applied research. More specifically, we ask ‘what are the implications of the increased pressure to demonstrate rapid farmer impact at scale for the use of on-farm trials in scaling agricultural technologies?’ In response, we draw upon insights from Science, Technology and Society (STS) studies and political agronomy. STS is an interdisciplinary field that examines how science and technology evolve in society over time. A fundamental theoretical contribution of STS is the perception of science as a socially embedded practice, characterised by different cultures of practice (Crane, Reference Crane2014). STS can be applied to many scientific disciplines and socio-cultural settings; most STS literature, however, is limited to science and technology in Western societies (Bijker et al., Reference Bijker, Hughes and Pinch2012), or experimentation in general (Maat and Glover, Reference Maat, Glover, Sumberg and Thompson2012). This paper applies an STS perspective to the context of contemporary development-oriented agronomy, while building on Bruno Latour's key insight that there is an inherent tension in experiments in the natural sciences, as researchers are caught between providing evidence while simultaneously having to persuade a selected audience (Latour and Woolgar, Reference Latour and Woolgar1979; Latour, Reference Latour1988).
We also draw upon recent insights from political agronomy that views agronomic research as politically situated practice (Sumberg and Thompson, Reference Sumberg and Thompson2012). This facilitates the analysis of the consequences of a changing donor landscape and the use of on-farm experiments in agricultural development. Combining these two analytical frames, we evaluate the premise that on-farm trials can be used as a means to understand and foster widespread technology adoption by smallholder farmers. We further reflect on whether combining evidence generation and persuasion in on-farm trials results in desirable development outcomes. This paper builds on three case studies of the organisational practices of research organisations conducting on-farm experiments for scaling agricultural technologies in southern Africa, Ethiopia and Bangladesh.
METHODOLOGICAL APPROACH
The methodological approach differs from more conventional agronomic studies. Rather than based on a priori outlined research procedures geared towards hypothesis testing, the research was empirically driven and inductive in nature. Reflecting on field observations and experiences, we developed three qualitative case studies of AR4D projects that employ on-farm trials as a central part of their strategy to encourage farmer technology adoption at a large scale. Each author is or was involved in one case study, respectively, as social scientists (southern Africa case and Ethiopia case), and systems agronomist (Bangladesh case). Participant observation, a research method most commonly used in social anthropology but also in development studies, was used as the primary mode of data collection (van Donge, Reference van Donge, Desai and Potter2006). With participant observation, researchers minimally disturb the social situations studied, in order to capture unbiased and primary information (Spradley, Reference Spradley1980). In addition to participant observation, project documents, on-farm trial results, interviews and focus group discussions with farmers, project colleagues and partners, were used to collect data.
We present three independent case studies. The analysis of the these studies is not intended to be representative of the large diversity of AR4D projects that employ on-farm trails to collect agronomic data as well as to scale-out agricultural technologies. Rather, our use of the case study method is to illuminate principles of social organisation in AR4D practice by examining specific situations (van Donge, Reference van Donge, Desai and Potter2006). The three case studies show how issues relating to the use of trials for scaling are not unique, incidental or purely context specific. Many of the organisational practices analysed, such as the selection of research sites and trial-hosting farmers, seem to be typical for contemporary AR4D projects as they manifest themselves in rather different institutional contexts. The analysis of three cases thus provides a perspective with wider applicability; one that can be used to assess the use of on-farm trials in other research for development projects.
Case study descriptions
Southern Africa
Since 2004, successive AR4D projectsFootnote 1 across southern Africa have focused on scaling conservation agricultureFootnote 2 (CA) practices with the aim to improve smallholder farmers’ livelihoods, food security and resilience to climate variability. Organised as collaborative projects between international agricultural research institutes, national research and extension programmes and NGOs, these projects bring together researchers, farmers and development actors around so-called demonstration-trials or demo-trials that aim to simultaneously study and scale CA practices. These demo-trials constitute the central project activity around which additional activities such as trainings, field days and innovation platforms are organised. The author of this case study, who was involved in innovation platform related-project activities and technology adoption research, collected data for this study through participant observation during agronomists’ trial inspection visits in selected project sites in southern Africa in 2013 and 2014 (Figure 1).
On-farm demo-trials sites (each dot represents 4–8 replicates) of conservation agriculture focused AR4D projects in southern Africa, between 2004 and 2015 (visited field sites in red).
Ethiopia
The second case study analyses the organisation of on-farm trials during the first three years (2012–2014) of the Capacity building for scaling-up of evidence-based best practices in agriculture in Ethiopia (CASCAPE) project in Ethiopia. This Dutch-financed project was designed to support the Ethiopian government's Agricultural Growth Programme, mainly through the generation of evidence on best practices in smallholder agriculture. Six implementing Ethiopian universities introduced agricultural technologies that increase food crop productivity in 30 so-called high-potential districts (CASCAPE, 2012; Figure 2). The author of this case study is employed as a social scientist by Wageningen University and Research, which provides project management and technical backstopping in the project. Between 2012 and 2014, project work was combined with participant observation and interviews with project staff, farmers and extension agents. Before the start of each interview, respondents were always asked for their consent to use the interviews as qualitative data for research purposes. In addition, project reports and interviews were analysed qualitatively, while scoring methods were employed for quantitative data extracted from project publications. Of the six partner universities, only two had submitted sufficiently reliable data from their on-farm trials to justify detailed study (which took place in 2014). These universities were therefore selected for further examination. The data used for this case study, and the subsequent picture that emerges from it, covers the period 2012–2014 and may not reflect practices in the second phase of the CASCAPE project (2016–2019).
Intervention districts of the CASCAPE project in Ethiopia (2011–2015). Each district has an average of 378 sites (i.e. farmers’ fields and/or farmer training centres’ demonstration plots) where on-farm experiments were carried out throughout the project.
Southern Bangladesh
The third case study addresses farmer participation in projects in Bangladesh. The Cereal Systems Initiative for South Asia in Bangladesh (CSISA-BD) was an AR4D initiative funded by the United States Agency for International Agricultural Development (USAID). Implemented by three CGIAR centres, CSISA-BD sought to increase smallholder farmers’ incomes through agronomic and aquacultural management interventions. While CSISA-BD was also operational in northern Bangladesh, this case focuses on AR4D activities in USAID's 33,750 km2 ‘Feed the Future (FtF) Zone’ in the country's south (Figure 3).
Bangladesh case study locations within the ‘Feed the Future’ zone. (A) The Kaliganj Upazila, Satkhira, (B) Babogonj Upazilla and (C) Lalerhat char (see case study). Dots represent seed drill machinery ‘step’ demonstration-trial locations in the 2013–14 dry rabi season. Black lines indicate District boundaries.
More than 30 development projects implemented by over 40 organisations are active in Bangladesh's FtF zone, which was chosen by USAID due to the region's high poverty incidence and climate change disaster risks. This case study's author was involved in CSISA-BD as an agronomist who led the implementation of farmer participatory experiments and extension activities in parts of the FtF zone, in addition to project management. The case study builds on participant observation and focus group discussions (FGDs) from 2011 to 2015 in this capacity. Prior to FGDs, participants were requested for formal consent and assured of anonymity (USAID, 2016a). Additional quantitative data from particular field experiments conducted by the International Maize and Wheat Improvement Center (CIMMYT), selected to illustrate challenges encountered in the implementation of on-farm trials, were analysed using classical statistics as detailed in the case study.
CASE STUDIES
Conservation agriculture demo-trials in Southern Africa
Building on one of the oldest methods in agricultural extension – demonstration – the demo-trial model addresses a heterogeneous audience. Used in scaling-focused CA projects in southern Africa, demo-trials not only produce evidence for scientists (Cheesman et al., Reference Cheesman, Thierfelder, Eash, Kassie and Frossard2016; Ngwira et al., Reference Ngwira, Johnsen, Aune, Mekuria and Thierfelder2014; Thierfelder et al., Reference Thierfelder, Rusinamhodzi, Setimela, Walker and Eash2015a, Reference Thierfelder, Bunderson and Mupangwa2015b, Reference Thierfelder, Matemba-Mutasa, Bunderson, Mutenje, Nyagumbo and Mupangwa2016), as all scientific experiments do (cf. Latour and Woolgar, Reference Latour and Woolgar1979). They also explicitly target an audience of farmers, who are expected to replicate the practices of the trials on their own fields. Enhancing the capacity of increasing numbers of farmers to replicate demo-trial results, constitutes the core mechanism of trial-based scaling, as a project document suggests
These [demo-trial] plots serve both as a demonstration vehicle for farmers and the project encourages the host farmers who manage the plots and participate in all activities to be the protagonists in extending the technologies based on their experiences. At the same time, because the plots are organised in such a way that they can be statistically analysed, researchers use the results to validate the effectiveness and efficiency of the technologies (Project Completion Report, 2011).
This case study highlights the inherent tension in the demo-trial model by discussing the organisational practices of agronomic researchers and their collaborating partners, such as site and farmer selection, demo-trial supervision and management. Revolving around the re-creation of controlled experimental situations (replications) in farmer fields, the studied demo-trials unintentionally undermine farmer adoption at scale.
Demo-trial location, management protocol and implementation modalities
Demo-trials are researcher-designed agronomic experiments, implemented by farmers. In order for their successful implementation as on-farm experiments, project officials requested selected farmers to commit themselves and a plot on their farm for several seasons. This simultaneously enables repeated measurements for the trials’ scientific audience, and farmer learning of new technologies and practices. Trial-hosting farmers received inputs (fertiliser, seeds, herbicides and pesticides) for the trial plots at reduced cost (and/or on credit), as well as technical support from technicians and/or government extension or NGO staff.
Agronomists and their extension partners jointly selected demo-trial sites and trial-hosting farmers, often based on the collaborative partners’ history in an area and existing relations within farming communities. For instance, one project used criteria, such as the presence of partners in the area, adequate infrastructure for trial management and existing farmer collaborations that could be built upon. These were weighed against potential failure risks. Primarily concerned with agronomic data collection, researchers thus aimed to implement well-managed demo-trials. They did so by seeking synergies with existing farmer support infrastructure, which also reduces transaction costs and allows farmers to gain hands-on technical experience.
This organisational set-up has two important consequences. First, by partially devolving control over the site and farmer selection process to partners, selection bias may be introduced, which can undermine researchers’ goals to evaluate the consequences of CA under farmer management. Aiming to be good project collaborators, partners are likely to select farmers capable of re-creating well-managed trial plots reminiscent of those found on research stations. They are also likely to select farmers with fields in locations that are easy to access and monitor. These farmers are not necessarily representative of the larger, socio-economically differentiated, population of farmers to which the CA technologies were to be scaled.
Second, project partners’ intervention histories, and operational modalities also appear to influence the organisational set-up of the demo-trials. For example, in most locations individual farmers were selected to implement a single demo-trial replicate on their farm. In at least one location (in Zimbabwe), a group approach was followed with 6 to 10 farmers jointly managing a single replicate on a field of one of the group members. This collective action approach was apparently analogous to a project partner's intervention modality in that area. Both trial management modalities were nonetheless analysed as independent replicate blocks. Group-managed plots nonetheless proved less suitable for replicating experimental conditions. For example, farmers were encouraged to share trial management tasks. Equal but small individual shares in the demo-trials’ harvests and the possibility of the field owner demanding a larger share of the yield, seem to have created free rider problems that negatively impacted trial management and consequent data. Comparisons between farmer fields and demo-trial fields revealed that – unlike the demo-trials managed by individual farmers – trial-hosting farmers achieved higher yields on their own fields than on the group-managed demo-trials, even when they applied less fertiliser (Cheesman et al, Reference Cheesman, Andersson and Frossard2017).
On-farm experiments: Generalizable results?
As the above quote of from the project document illustrates, agronomic researchers employing the demo-trial model seek an arrangement that yields statistically analysable data.
This requires that the experimental situation is standardised as much as possible, and differences between blocks controlled or monitored so these can be used as explanatory variables. Such standardisation may, however, have unintended consequences for scaling.
First, agronomists preference for establishing demo-trials in close proximity to ensure similar biophysical (e.g., soil and microclimate) conditions, limits their outreach (Figure 4). Researchers’ concern with clustering trial replicates under similar circumstances is reminiscent of on-station randomised block agronomic experiments established on one field. When demo-trials require the use of new farming equipment – such as CA ripper-tines and direct seeders – geographical concentration of trial replicates also enables equipment sharing, which lowers costs. Living near one another, receiving instructions together, sharing equipment, being visited frequently by technicians; all this enhances demo-trial-hosting farmers’ capacity to replicate, that is, to implement trials according to researchers’ guidelines. Yet, the thus formed group of farmers implementing the same practices may also obscure the potential variability in farmer practices and technology performance across different biophysical and socio-economic environments. In addition, the geographical concentration of demo-trials – sometimes in areas that are distinct or secluded from a wider farming community – hampers their function as ‘demonstrations’ and hence, their replicability at scale. Demo-trials established on farms in resettlement areas in Zimbabwe exemplify this. These farms are often not only geographically isolated, they are usually also located on better soils, larger and often owned by wealthier farmers (Dekker and Kinsey, Reference Dekker and Kinsey2011). This limits their value as demonstrations for the wider farmer community (Figure 5).
Example of demo-trial replications (green boxes) in close proximity in a geographically isolated location in a resettlement area, northern Zimbabwe.
Example of a demo-trial site (red rectangles are trial replications) in an area with rather distinct farm and soil types: Large-scale farms in a resettlement area (white area) and small-scale farms in a Communal Area (yellow area), northern Zimbabwe.
A second concern is the standardisation and control of the experimental situation. In order to ease statistical comparison of the demo-trial replicates, farmers were provided a standardised package of seeds, fertiliser, herbicide and pesticide inputs. Input costs were to be repaid by hosting farmers following harvest, either in cash or grain. Such use of standardised input packages and prescribed management regimes serves to (re-)create controlled experimental situations on farmers’ fields. Yet, it can also obscure socio-economic differences among farmers that are likely to influence CA management under real-world circumstances (Giller et al., Reference Giller, Witter, Corbeels and Tittonell2009). In other words, farmers’ management of CA fields outside the experimental setting – where different crops, fields, livestock and off-farm livelihood activities compete for farmers’ attention – may differ substantially from demo-trial management.
Farmer selection and trial evaluation in Ethiopia
Building on the paradigm of farmer participatory research as developed in the 1980s, CASCAPE is premised on the idea that the likelihood of technology adoption increases when farmers participate in technology selection and testing (CASCAPE, 2012). CASCAPE also focused on ‘integrated technology evaluation’, referring to a protocol indicating that technologies should be evaluated based on multiple criteria, including farmers’ preferences, yield, costs and benefits, gender equity and environmental criteria (CASCAPE, 2012). By focusing on two aspects of on-farm trials, namely the selection of trial-host farmers and trial evaluations, this case study contrasts CASCAPE's organisational principles with its implementation practice.
Participatory technology development
Based on extensive participatory rural appraisal exercises in 30 intervention districts, CASCAPE's implementing universities proposed improved agronomic practices or technologies thought to fit the agroecological zone and farming systems of respective project districts (CASCAPE, 2013a). Most entailed ‘off the shelf’ technologies developed by regional research institutes. If no existing technologies were available, researchers proposed an innovation for design and testing. Subsequently, roughly two types of trials were implemented: (1) testing innovations and (2) demonstration-trials for known technologies. Farmers were to host, manage and jointly evaluate results from both trial types. CASCAPE's management and its university teams jointly established the criteria for technology evaluation (Table 1). Trials were iterative, and after a negative evaluation, unpromising technologies or practices would be either dropped or modified. Positive evaluations would advance technologies to the next testing phase (Figure 6). When pre-scaling trials produced what CASCAPE's researchers felt to be sufficiently and consistently positive results, the positively evaluated agronomic technologies were ‘handed over’ to the government for scaling.
Criteria used for evaluating agricultural technologies in CASCAPE (2011–2015).
(Source: Interviews with project staff, 2014).
CASCAPE's technology development and scaling strategy (2011–2015).
The rationale behind the ‘chain’ of on-farm trials (Figure 6) was that while a technology or crop management practice moves from the development and adaption to scaling stages, the conditions of testing and evaluation are supposed to gradually change from a research setting towards farming reality (CASCAPE, 2015). The Ethiopian agricultural research and extension system was also expected to gradually assume leadership of the scaling process. These arrangements were intended to improve the potential for rapid farmer adoption. Between 2011 and 2014, 7869 farmers participated in on-farm trials and field days, 14% of whom directly hosted on-farm trials (CASCAPE, 2015).
Farmer selection
In CASCAPE's first phase, trial-host selection criteria were not very clear. Without explicit guidelines, university coordinators had relative freedom in farmer selection and trial organisation. Selection practices introduced a number of biases in trial-host selection. In most cases, potential trial-hosting farmers were first identified in collaboration with the district extension personnel. While such partnership at first appears advantageous, CASCAPE quickly encountered problems. Government extension policy mandates the introduction of new technologies first to ‘model’ farmers thought of as innovative and capable to adopt technologies, after which other farmers are expected to follow (MoA, 2014). Extension workers could therefore be quite persistent in their preference:
During a gender activity analysis conducted in 2014, we (CASCAPE) were supposed to select 50 farmers, 2/3 being ‘non-model’ and/or female. It seemed almost impossible to convince the extension agent to include so many ‘non-model’ female farmers in the sample. The extension agent continued to bring male and ‘model’ farmers for us, telling us that they were the best examples to be included. (Interview with project staff, May 2014).
CASCAPE's agronomists and livestock fodder experts, working in their respective university teams, continuously indicated their concern that the technology evaluation trials must also be successful as demonstrations. Poor or female farmers, they argued, may not have sufficient land, oxen or labour, and may lack the knowledge, experience and time required to adequately manage experiments, thereby risking trial ‘failure’:
It is difficult to select female headed households, because they often do not have enough labour to plough and sow in time. We experienced this several times resulting in delayed planting . . . Female farmers are generally not seen as a good example for other farmers. So selecting a women farmer is also not helpful to convince other farmers to adopt the technology. (Interview with project agronomist, August 2015)
Creating ideal conditions for technology performance, rather than technology evaluation, emerged as a major concern:
The farmer needs to be knowledgeable and needs to have the necessary resources and interest to manage the trial well. If not, the trial might fail. Our job is to show that the technology works, it is the job of extension to make sure that other farmers will adopt it. (Interview with project agronomist, February 2014)
Agronomists therefore tended to select farmers they described as more ‘serious’, who could serve as positive examples in their village. Preference for educated and better-off farmers with prior trial experience, and who were expected to follow agronomists’ recommendations, was common.
Franzel et al. (Reference Franzel, Coe, Cooper, Place and Scherr2001) pledge to separate trials according to their goal: type I trials (researcher designed and managed) to evaluate biophysical parameters, type II trials (researcher designed but farmer managed) for evaluating economic parameters and type III trials (farmer designed and managed) for evaluating farmers’ preference. In CASCAPE, this distinction was not made. Despite CASCAPE's emphasis on iterative and participatory technology vetting under farmer conditions, agronomists struggled to leave the management to farmers (only), because they feared trial failure. They thus appeared to favour proving new technology performance over farmer management. Two observations illustrate this point. First, data from farmers who did not manage trials adequately were occasionally excluded from analysis, without accounting for the reasons for poor management. Where poor management may have resulted from farmers’ prioritising other livelihood activities, for example, a more remunerative off-farm activity, exclusion of the trial results constitutes a missed opportunity for researchers to learn about technology performance outside the experimental setting. Second, the labour needed for land preparation, planting, weeding and harvesting in the trials was in some cases paid for by the project to ensure that the plot was well managed from an agronomic perspective. As a result, researchers not only missed an opportunity to learn why farmers may be unable to implement a new crop management practice or technology in their own fields. Such practices also undermined technology evaluation under ‘real’ farmer management.
These approaches to experimental implementation and management resulted in selection biases favouring male and ‘model’ farmers as trial-hosting farmers in the early phases of the project (Figure 7).
On-farm trial evaluation
Although CASCAPE established criteria for multidisciplinary trial evaluation, most technologies were promoted from adaptation to pre-scaling mainly based only on reported yield performance. Figure 8 shows that while in almost all annual reports agronomic data were presented (i.e. yield data), only less than half of the reports presented a farmer preference analysis and only in one out of five reports an economic analysis was reported (CASCAPE, 2013b, 2014).
Frequency of reporting on three broad categories of data types, including agronomic, economic and farmer preference analysis data from on-farm trials in 2013 and 2014 (n = 101). Source: Project annual reports of six participating universities, 2013 and 2014.
Problems and pitfalls of trials for scaling in Bangladesh
Bangladesh is the globe's most populated agrarian nation. Forty-nine million people, many of whom are farmers, subsist below the poverty line (MOA and FAO, 2013). In addition to its strategic geopolitical importance, Bangladesh attracts considerable development assistance. Over USD 17.4 billion were awarded by various donors between 2000 and 2009 (World Bank, 2015). Approximately 230 international NGOs and six CGIAR institutes work in Bangladesh (Gauri and Galef, Reference Gauri and Galef2005). Coordination among projects is complicated by differing donors and development organisations’ agendas. This case study provides examples of difficulties encountered in managing on-farm trials implemented by CIMMYT, a partner in the CSISA-BD project. We then describe how the project matured and shifted from a focus on agronomic trials for scaling to a more systems oriented research and extension agenda.
Scale or reach? Farmers’ motivations for participation
In Bangladesh, AR4D projects increasingly compete with NGOs and private-sector development organisations for project funding. Mandates to deliver impact at scale among the rural poor are now common. This new funding environment has ramifications for how AR4D projects are implemented and how farmer participation is recorded and reported to donors.
USAID's FtF indicators, for example, require projects to count farmers including “. . . the total number of farmers, ranchers and other primary sector producers, etc. that applied new technologies anywhere within the food and fibre system as a result of US government assistance. . .” (USAID, 2016b). Emphasis is thus placed on use of technologies and crop management practices rather than on sustained adoption (or disadoption). Similar approaches are utilised by other major donors. Development organisations can therefore achieve farmer-beneficiary targets by assuring that many farmers simply try out new technologies, including participation in demonstrations and/or on-farm trials. The broader political economy of development therefore appears to be at least partially driven by the need for projects and donors to demonstrate large numbers of project beneficiaries, in order to justify continued bilateral investment.
Different organisations also have different approaches to engaging farmers in development activities. Where coordination among organisations is poor, AR4D efforts may suffer. For instance, when CIMMYT researchers asked farmers to participate in demonstration-trials during the CSISA-BD's inception phase, they were told
Save the Children was here last week. They will give education for our children and we have asked them for a deep tube well. What are you able to give us for planting your crops?
(Farmer commentary, Kaliganj Upazila, Satkhira, September 2011, Figure 3)
This example demonstrates how farmers may approach participation pragmatically, valuing the potential economic benefit from engaging with researchers over the research process itself. CSISA-BD conversely discouraged direct payment and material transfer to farmers, although this often made it difficult for researchers to implement fieldwork.
Over half the farmers in USAID's FtF zone are landless tenants (MOA and FAO, 2013). They may therefore approach the sharing of land for trials and demonstrations from the perspective of tenant-landlord relationships. Farmers in In Babugonj Upazilla, for example, commonly requested additional financial compensation for the land used for participatory wheat varietal trials in the early phases of the CSISA-BD project, despite agreements that any yield loss caused by the experiments would be compensated (Figure 3). Problems were also experienced with implementing experimental protocols. Some farmers, for example, avoided labour-intensive weeding practices despite consistent requests by researchers. In order to reduce the risk of demonstration-trial failure, and to assure quality data, research technicians frequently responded by hiring labourers to manually weed. By doing so, participating farmers in adjacent villages reasoned that the project would support additional labour and also withdrew from weeding the following season.
Emerging dependency syndrome
In CSISA-BD, CIMMYT demonstrated small-scale agricultural machinery to farmers (Figure 3). Many demonstration-trials emphasised agronomic methods for the conversion of fallow land to wheat cropping. On Lalerhat char in Barisal District, 13 farmers were engaged on 10 hectares of their own land to implement large-scale demonstration-trials of mechanised row-sown wheat. With CIMMYT input support, farmers implemented researcher-designed, farmer-managed multiple-plot ‘step’ demonstration-trials in a dispersed randomised block design. Treatments included farmers’ own nutrient and weed management practices with manual broadcast seeding (T1), farmers’ own nutrient and weed management practices but with machine sown wheat (T2), and extension service recommended nutrient and weed management practices with machine sown wheat (T3) (Table 2). Moisture corrected yields were measured from the sum of five randomly placed 2 m2 quadrats plot–1.
On-farm ‘step’ experimental design for Lalerhat char in southern Bangladesh (2013–14 season).
a Significant (p < 0.001) yield differences detected following ANOVA. Data in column not sharing the same letter are significantly different according to Tukey's HSD test at α = 0.05.
Trial results indicated significant (p < 0.001) 0.4 t ha–1 yield increases from T2 compared to T1. Further significant yield increases of 0.63 t ha–1 were also observed when comparing T1 to T3.
After the season, farmers, however, commented that they would be unable to afford or find sufficient fertiliser for the elevated rates (61, 2.8 and 65 kg more N, P and K ha–1) required for the T3 (recommended practice) treatment. In the following 2014–15 dry season, CIMMYT withdrew from demonstration-trials but continued to observe farmers’ practices and performance. Only 8 of the original 13 farmers opted to grow wheat. The remainder opted for less input intensive yet profitable legume crops, or reverted to fallowing. One of the eight farmers nonetheless purchased the mechanised seed drill as used in the previous year's demonstration-trials, and planted the other farmers’ fields on a fee-for-service basis. Despite the agronomic performance of T3 in the previous year, farmers who grew wheat used methods similar to T2, by employing their own management practices with machine sowing, rather than adopting recommended management techniques.
Growing conditions in the 2014–15 dry season were similar to the previous year. Yield data were collected as in the 2013–14 season. Comparing data collected from these farmers, yields were, however, significantly (p < 0.01) and 1.27 t ha–1 less than T2 in the previous season (Figure 9). Follow-up discussions revealed that the poorer wheat performance probably resulted from farmers’ reduced willingness to actively manage weeds in the absence of regular researcher visits. Although small in number, this case illustrates the problems associated with scaling metrics and indicators that measure farmers’ application of a technology alone, without further examination of the quality of continued technology use outside of demonstration-trial and project-mediated settings.
Machine drilled wheat yield decline (under farmers’ own management practices, or FP) on Lalerhat char comparing the 2013–14 and 2014–15 dry season following the withdrawal of researcher trials. ** indicates p < 0.01, n = 8.
New approaches to AR4D
Subsequent projects implemented by CSISA-BD partners continued to focus on small-scale machinery. While still AR4D efforts, these initiatives placed more attention on interventions aimed at creating a favourable policy and market environment to enable adoption and scaling. Built on theories of change and using value-chain approaches, these new efforts aimed to overcome bottlenecks in the supply of machinery by linking farmers to machinery service providers, dealers and manufacturers. In other words, linear technology-transfer approaches associated with demonstration-trials were downplayed in favour of innovation systems development AR4D. Research agendas also shifted to emphasise systems analysis and studies of adoption constraints to better guide efforts to increase farmer-to-farmer technology diffusion.
DISCUSSION
In the 1980 and 1990s, STS scholars made detailed analyses of how natural scientists conduct and write about experiments (Gooding, et al., Reference Gooding, Pinch and Schaffer1989; Latour, Reference Latour1988). These studies provide insights that are highly relevant to the understanding of on-farm experimentation in contemporary AR4D. Paramount is the problematisation of the dual role of experiments: providing evidence while also persuading selected audiences that the technology in question ‘works’. Each case study has revealed how this dual role applies to on-farm trials: they are employed to evaluate if a given technology ‘works’ under smallholder farming circumstances, while simultaneously being used to convince other scientists and farmers of the success of the given technology. While this has long been the case for AR4D, the three case studies suggest on-farm trials now have a third audience – donors interested in impact at scale.
The donor audience is interested in somewhat different evidence than scientific or farmer audiences: metrics of adoption and farmer participation have become necessary to prove that technologies ‘work’ and create impact at scale (BMGF, 2016; CGIAR, 2016; OECD, 2016). Pushed by this metrics agenda, the nature of agronomic evidence production has shifted from technology assessment (i.e. yield and economic performance, environmental impacts, etc.) typically presented in journal papers, to include ‘head counting’ (i.e. number of farmers participating in trials, number of farmers applying a technology), widely reported in project documents and donor reports.
The case study from Bangladesh is particularly explicit about the consequences of this donor focus, stressing the problematic emphasis on application rather than adoption (or disadoption) dynamics, while indicating that projects can achieve beneficiary targets simply by assuring that many farmers try out new technologies. This situation becomes even more problematic when farmers are given incentives to participate in technology testing or demonstrations to meet the metrics required by donors. Without independent evidence substantiating farmers’ interest in sustained use of improved technologies, such metrics can provide the misleading indication that a given technology ‘works’, while the produced metrics are merely an artefact of the AR4D process itself. Moreover, farmers may position projects or NGOs against each other for increased social, material or monetary gain.
Besides a perspective on the audiences of experiments, STS studies have drawn attention to the tension between providing evidence and persuasion. Latour (Reference Latour1988) argued that the interest to persuade the audience(s) always dominates the interest to provide evidence. This is most obvious in contemporary donor-funded AR4D projects; for sustained funding, trial designers and implementers are under pressure to develop experimental treatments that outperform existing technologies and practices. As the southern Africa and Ethiopian case studies have shown, to increase the likelihood of a successful experiment, agronomists and their partners tend to (consciously or unconsciously) introduce biases when (re)creating the controlled experimental environments. They tend to select areas they already know, farmers that are ‘more serious’, more receptive to new technologies, live in close proximity to all-weather roads, etc. Such selection biases contribute to the creation of a receptive farmer audience, but risk compromising the external validity of the trials, and hence, their scalability. This is evidenced by the demo-trial case study from southern Africa, where replications are often located close to one another, which limits the potential for scaling-out to other areas with different biophysical conditions. In Ethiopia, biased farmer selection and a preference for yield data (over other parameters) resulted in on-farm trial results that may not be representative or relevant for the wider farming community. For instance, poor and female farmers are generally less likely to adopt labour or capital intensive agricultural technologies (Phiri et al., Reference Phiri, Franzel, Mafongoya, Jere, Katanga and Phiri2004). The Ethiopia case study demonstrates how these groups may be purposely excluded from agronomic technology testing and evaluation. This is pertinent since increased scale is typically accompanied by increased heterogeneity in biophysical and socio-economic conditions, in addition to farmers’ abilities, indicating that results achieved by so-called ‘model’ farmers may not be fully replicable outside the experimental setting.
Next to problems of farmer selection biases reducing the external validity of on-farm experiments, there is an inherent tension within on-farm experiments. The need to control the experimental situation makes on-farm trials ill-disposed towards replication at scale – that is, by many different farmers in diverse agricultural landscapes – as increased scale is typically accompanied by increased heterogeneity in conditions. This argument was presented by Latour (Reference Latour1988) in his study of the microbiologist Louis Pasteur, who developed rabies and anthrax vaccines. Pasteur's vaccines only worked when the experimental conditions were transferred to other settings; extending the vaccines’ use thus necessitated the ‘Pasteurization of France’. The selection biases and standardisation of trial inputs and management regimes in the organisation of demo-trials can have a similar effect; they mould circumstances and farmers in such a way so that the technologies can work. Whether this is a feasible and appropriate scaling strategy is a moot point.
CONCLUSIONS
In addition to evaluating the performance of new technologies in different environments, the case studies presented in this paper demonstrate how on-farm trials are conceptualised by AR4D organisations as a tool to demonstrate new technologies and crop management practices to farmers. Participating and neighbouring farmers are often thought of by these organisations as potential adopters, and as such, agronomists – as well as other research project staff – may assume that on-farm trials can be used to understand the scaling potential of new technologies under different socio-economic and institutional environments. The perspective on on-farm trials developed in this paper problematizes this assumption. An understanding of scalability requires first and foremost an assessment of the drivers of technology adoption. Our case studies indicate that on-farm trials are in many circumstances ill-suited to understand these drivers. An understanding of the scalability of technologies requires the use of methods that can elucidate barriers to and opportunities for scaling. This requires methods that increase the insights on farmers’, asset base, in- and output markets, and insights in how trends such as urbanisation and climate change affect farmers’ livelihood options. On-farm trials are poorly suited to acquire these insights since drivers operating at higher levels are difficult to recreate under agronomic trial settings. Hence, a focus on scaling requires a re-orientation of both the research process and scaling activities themselves. This has important ramifications for the division of labour within AR4D organisations (what kind of scientific disciplines should be involved?), the farmers who participate (what farmers are representative?), research methods used (are manipulative trials better suited than surveys, choice experiments, or randomised control trials?), research locations (which farms to include?) and research protocols deployed (what data to collect, and how to analyse them?).
Our case studies provide evidence that the emphasis placed by funders of international agricultural research on benefitting farmers at scale can unintentionally result in a ‘donor dependency syndrome’, in addition to the misalignment of researcher and farmer interests. The drive to demonstrate impact at scale can thus undermine the potential of on-farm trials to serve as sites for experimentation and adaptive research. Review of the case studies also problematizes the replicability of trial results when experiments are carefully staged such as experiments designed and highly controlled by researchers. When experiments are analogous to ‘performances’ conducted under ideal circumstances (rather than under realistic circumstances and variable management), experimental results are less likely to be reproducible at a larger scale.
In order to both improve the potential for farmer adoption of promising technologies as well as the quality and rigor of agronomic science itself, this dilemma needs an urgent solution.
While on-farm experiments role in scaling may be limited, we see their continued importance, as a space for joint experimentation and adaptive research. A future direction would be to employ an interdisciplinary approach to technology introduction and testing, by embracing the environmental and social heterogeneity of farming communities, and perhaps by using randomised and representative samples of trial implementing farmers, rather than those chosen because of their ‘advanced’ abilities to manage experiments. In this scenario, on-farm trials should be complemented by assessments at other levels and including other domains, to increase the understanding of scalability of the technologies at stake.
Acknowledgement
Research for this paper was partially funded by the Royal Netherlands Embassy in Addis Ababa (CASCAPE), the CGIAR Research Programs on MAIZE and WHEAT and by USAID/Bangladesh (CSISA-BD). The contents and opinions expressed herein are those of the authors and do not necessarily reflect the views of CASCAPE, USAID or the United States Government. We thank Faisal Washiq for data collection in Bangladesh.
