The purpose and role of the ‘demo-trials’

In the projects that were the basis for the southern African case study of de Roo et al. (Reference de Roo, Andersson and Krupnik2019), the authors failed to acknowledge one of the key project components – farmer experimentation, where farmers, both project collaborators and others, test the technologies on their own fields. These farmer tests where the farmer adapts the technology to his/her own system and conditions, are seen as a crucial activity that is monitored by the project to evaluate how farmers’ adapt and manage CA systems outside the experimental setting, just as ‘proposed’ by de Roo et al. (Reference de Roo, Andersson and Krupnik2019). These farmer-designed and managed adaptations of CA are the basis for farmer-to-farmer information exchange, and the key to the scaling strategy in the projects. The main tenet of the de Roo et al. paper that AR4D projects ‘employ (demonstration plots) as a central part of their strategy to encourage farmer adoption at a large scale’ is incorrect, at least in the case of the southern African case study, nor did these projects ‘conduct. . . on-farm experiments for scaling agricultural technologies’. The primary objective of the demonstration plots is not, as supposed by de Roo et al. (Reference de Roo, Andersson and Krupnik2019), to convince farmers ‘to replicate the practices of the trials on their own fields’ and thereby scale out CA, but rather to work with a small group of collaborating farmers who host the demonstrations on their fields, to develop and adapt functional CA systems. The demonstration plots gave these farmers experience with the management of the system, and some of the component technologies, and aimed to interest them in testing the system themselves on their own fields with their own modifications and adaptations.

Even though the large plots comparing one or two CA options to the common farmer practice were called ‘demonstration plots’, often referred to as ‘demo-trials’, they addressed several objectives. Initially, the plots were used by researchers, development partners and farmers to observe the performance of ‘best bet’ options for conducting CA under local conditions, suggest modifications which could then be incorporated into the plots in succeeding seasons and, in so doing, adapt a CA option to local conditions. To this end other research trials, both on research stations and on-farm were conducted to evaluate solutions to problems observed in the demonstration plots, and select options, which could then be incorporated into the on-farm demonstrations. Other activities within the projects included trials to evaluate the relative importance of different CA technological components.

Failure to recognise and acknowledge the role of farmer experimentation in the southern African projects lead de Roo et al. (Reference de Roo, Andersson and Krupnik2019) to criticize many aspects of the design and management of the on-farm demo-trials, based on their assumption that these plots were intended for the scaling-out of CA technologies. We are unsure whether they would make these same criticisms if they understood the real role of the demonstration plots – that of working with host farmers, and local partners, to develop locally-adapted CA options. We should note that, had the objective of the demonstration plots been the scaling-out of CA technologies we would have shared many of the criticisms of de Roo et al. (Reference de Roo, Andersson and Krupnik2019) and this is precisely why the projects were designed to avoid these problems. In the following paragraphs we address some of the concerns of de Roo et al. (Reference de Roo, Andersson and Krupnik2019), acknowledging that these concerns may not be relevant in the light of the clarification of the purpose of the demonstration plots.

Site selection

The projects consisted of multiple, interlinked activities and were, and are, far more complex than the single-activity projects suggested by the description in de Roo et al. (Reference de Roo, Andersson and Krupnik2019). The goal of these projects was not to help develop and promote CA systems in one area or country, but rather, with the active participation of local scientists, extension personnel and farmers, to develop examples of CA adaptation and adoption in a series of relatively distant and distinct environments, and refine and promote a methodology that would allow the scaling-up of project methodologies (replication of the activities and methodologies by national partners – Rossing et al., Reference Rossing, Otzelberger, Girot, Lisa, Schipper, Ayers, Reid, Huq and Rahman2014). The intended outscaling and widespread adoption of CA would be achieved by the NARES through the institutionalization of the concepts and methodologies developed through the projects, including, especially, the use of farmer-designed and managed field tests as the basis for farmer-to-farmer information exchange. This was not expected to be a rapid process – experience elsewhere has shown that initial growth in the area of CA systems is slow, as it takes time for innovative farmers to become convinced, adapt, begin to adopt and extend CA practices to other farmers. Institutional change in research and extension systems may take even longer.

In defining where to invest the limited project funds, we opted for dispersed but accessible sites representing a wide range of biophysical and socioeconomic circumstances – i.e., widely divergent recommendation domains (Perrin et al., Reference Perrin, Winkelmann, Moscardi and Anderson1976), each comprising farmers with similar circumstances who were likely to benefit from similar technologies. Biophysical conditions at the sites ranged from very marginal (e.g., Zimuto Communal Area, Zimbabwe) to high potential sites (e.g., Songani Village, Zomba, Malawi). Through this we aimed to develop ‘infection points’ and training grounds for the scaling of CA, as well as exploring the performance of CA under contrasting conditions, and carefully worded project titles to reflect this ‘facilitation of the widespread adoption of CA’. Obviously, there are farmers in inaccessible sites, but it would be expensive, and therefore inefficient, to focus on these farmers initially. It is expected that partners with more local representation will reach these areas later. We are surprised by the lack of understanding of this strategy of working within recommendation domains evident in de Roo et al. (Reference de Roo, Andersson and Krupnik2019), together with the apparent belief that CA systems will be the same for farmers with divergent conditions. This led the authors to suggest that concentrating demonstration plots within a community of resettled farmers on good soils in Zimbabwe ‘limits their value as demonstrations for the wider farming community’. Obviously, the value of these demonstrations was limited to adapting CA technologies for the recommendation domain in which they were installed, and it is highly unlikely that the CA options developed would be relevant to nearby farmers on communal lands with poorer, and more degraded, soils.

Within each of the project ‘pilot’ sites, shown graphically on a map in de Roo et al. (Reference de Roo, Andersson and Krupnik2019), community meetings were held to analyse farmers’ problems, describe CA and its benefits, and if the community was interested, select participating farmers, design ‘best-bet’ CA options to compare with local farmer practices and initiate field studies. We did not have any cases of disinterested communities although we did have to withdraw from one because a local politician insisted that he select the participating farmers. Within the community we endeavoured to confine activities to a small area, not to facilitate equipment sharing, technical field visits, etc. as suggested by de Roo et al., although it certainly helped with these aspects, but rather to ensure that participating farmers were able to give each other mutual support and overcome derisive peer pressure (Wall, Reference Wall2007).

Farmer selection

After discussing problems and possible solutions at the community meetings, farmers in the community were asked to volunteer to host the best-bet comparisons on one of their fields. This tended to ensure that we worked with interested and progressive farmers. The community then selected the 8–10 farmers who would host the demonstration plots from among these volunteers. As far as we know, in no case was farmer selection done by the agronomists and their extension partners as stated by de Roo et al. (Reference de Roo, Andersson and Krupnik2019), but rather by the community itself, in an effort to ensure that interested and representative farmers hosted the trials – the community was unlikely to select farmers they thought were outliers or who did not represent their conditions. Thus, the bias suggested by de Roo et al. (Reference de Roo, Andersson and Krupnik2019) in that ‘partners are likely to select farmers capable of re-creating well-managed trial plots reminiscent of those found on research stations’ was obviated. However, we did find out in retrospect that in some communities’ farmers had picked some influential farmers rather than only representative farmers, and we are unsure how to overcome this problem. Participating farmers were asked to commit to hosting the field plots for at least 3 years so that the longer-term effects of CA could be observed.

Management of the ‘demonstration plots’

Initially, the various demonstration plots within a community followed the same protocols. Evidence from elsewhere suggested that crop variety was unlikely to affect the behaviour of the CA options, and so farmers were asked which variety they preferred to use and this was generally standardized across all plots in the community. Based on our understanding that CA cannot be managed as a low-input system we established a fertilizer application rate to be used, aiming at an economically viable level under local conditions. This level was adjusted over time, but was constant across the plots and comparisons within a community to facilitate farmer comparisons of the crops in the different plots. However, fertility still varied among fields as a result of previous management and cropping history. Importantly, farmers knew about fertilizer and its benefits and were able to participate in the discussions to define the best level to be used in the plots.

Use of the demonstration plots as on-farm experiments

Added value was obtained from the on-farm demonstration plots by statistical analysis of the results from each community. This does not require, as de Roo et al. state, that the experimental situation be standardised as much as possible, simply that researchers understand that the effect of all factors that vary between replications are included in the statistical error term. However, as the case study author states, individual demonstration plots (replications) are monitored, allowing for the analysis of subsets of replications within the community to take into account and separate factors such as soil type, variety, etc. The dual misunderstandings that demonstration plots are standardized to use them as trials, and that the main objective of the demonstration plots is the scaling of CA, lead the authors to state that this standardization ‘may. . . obscure the potential variability in farmer practices and technology performance across different biophysical and socio-economic environments’. This important variation and adaptation of CA technologies is observed on farmer experiments or test plots, not on the demonstration plots.

One issue which the authors do not address, but which often results in less than optimal results and lower yields than the farmers’ own crops, is the choice of fields on which the demonstration plots are situated. Farmers, especially smallholder farmers, are averse to risk, and dedicating part of their prime land to what they perceive as a risky new idea is not palatable. In some cases, therefore, farmers provided their worst (i.e., least fertile and most degraded) fields as demonstration sites. This is a challenge as it compromises the results, but we find that farmers are well aware of this problem and do not expect outstanding results from these fields. However, it did highlight the marked degree of soil degradation in the communal areas of Zimbabwe.

Have we been able to achieve the objectives of the CA projects?

Farmer adoption of CA in most of the project areas has been slow, and farmer testing of CA technologies on their own fields has been limited. Adoption studies on CA in sub-Saharan Africa have generally shown that poorly developed input, service and output markets were serious impediments to CA adoption (Affholder et al., Reference Affholder, Jourdain, Quang, Tuong, Morize and Ricome2010; Bolliger et al., Reference Bolliger, Damgaard Hansen and Fowler2005; Corbeels et al., Reference Corbeels, de Graaff, Ndah, Penot, Baudron, Naudin, Andrieu, Chirat, Schuler, Nyagumbo, Rusinamhodzi, Traore, Mzoba and Adolwa2013) as they are for agricultural technologies in general (Ehui and Pender, Reference Ehui and Pender2005). Analysis of CA adoption in the region using the Qualitative expert-based Assessment Tool of CA Adoption in Africa (Ndah et al., Reference Ndah, Schuler, Uthes, Zander, Traore, Gama, Nyagumbo, Triomphe, Sieber and Corbeels2014) showed that input and output market conditions were the major impediment to CA adoption in five of six cases analysed (Corbeels et al., Reference Corbeels, de Graaff, Ndah, Penot, Baudron, Naudin, Andrieu, Chirat, Schuler, Nyagumbo, Rusinamhodzi, Traore, Mzoba and Adolwa2013). Farmers in the project region state that restricted access to the necessary inputs is one of the principal causes of failure to adopt CA. This is a common problem in the region, not only for CA but for the adoption of even simple technologies such as crop varieties (Wall et al., Reference Wall, Thierfelder, Ngwira, Govaerts, Nyagumbo, Baudron, Jat, Sahrawat and Kassam2014) and the incorporation of legume species into the cropping system (Tripp and Rohrbach, Reference Tripp and Rohrbach2001). However, there are exceptions, such as in the area around Lake Malawi (the only site where markets were not an impediment to adoption in the assessment of Corbeels et al. (Reference Corbeels, de Graaff, Ndah, Penot, Baudron, Naudin, Andrieu, Chirat, Schuler, Nyagumbo, Rusinamhodzi, Traore, Mzoba and Adolwa2013)), where adoption and scaling of CA has been very successful. This was largely due to the institutional support provided by one of our project partners: Total LandCare, an NGO focussed not only on agriculture, but on community development, was able to provide unsubsidized credit and input supply channels, which facilitated the adoption of CA technologies. Arslan et al. (Reference Arslan2014) found that in-season rainfall and extension worker contact were important determinants of both adoption and the intensity of CA adoption in Zambia. However, the effects of extension contact in this study were confounded with free or subsidized input supply, effectively influencing the analysis of the importance of input markets. While markets and other institutional arrangements were a concern (Wall, Reference Wall2007) in the projects reviewed by de Roo et al. (Reference de Roo, Andersson and Krupnik2019), initially the focus was to provide a proof of concept that CA could function and provide benefits to farmers and the environment under the conditions of smallholder farmers in southern Africa. We were unable to dedicate enough effort to institutional aspects from the outset, but are now in the process of increasing the focus on institutional support for CA. In new projects we endeavour to address the contextual issues, or ‘prerequisite conditions’ (Sumberg, Reference Sumberg2005), affecting the farming system described by Corbeels et al., (Reference Corbeels, de Graaff, Ndah, Penot, Baudron, Naudin, Andrieu, Chirat, Schuler, Nyagumbo, Rusinamhodzi, Traore, Mzoba and Adolwa2013), through the development of local, regional and national innovation platforms incorporating representatives of all major components of the principal agricultural value chains (Ekboir Reference Ekboir and Ekboir2002; Schut et al., Reference Schut, Klerkx, Sartas, Lamers, Campbell, Ogbonna, Kaushik, Atta-Krah and Leeuwis2016). Corbeels et al. (Reference Corbeels, de Graaff, Ndah, Penot, Baudron, Naudin, Andrieu, Chirat, Schuler, Nyagumbo, Rusinamhodzi, Traore, Mzoba and Adolwa2013) state that ‘markets and policy are often outside the control or influence of the CA development-dissemination process’ but through the development of innovation platforms incorporating market agents and policy makers (at different scales) these factors can indeed be addressed (Schut et al., Reference Schut, Klerkx, Sartas, Lamers, Campbell, Ogbonna, Kaushik, Atta-Krah and Leeuwis2016). Understanding the effects of institutional factors on CA adoption will have important implications for agricultural policy-makers, but will require improved analyses of smallholder farmer resource allocation strategies as well as studies on the wider market, institutional and policy factors affecting CA adoption (Andersson and D'Souza, Reference Andersson and D'Souza2014).

In their paper de Roo et al. refer to project ‘donor dependency’ and a new focus of AR4D projects that has come about because ‘donors are interested in impact at scale’. We argue that investors in agricultural research for development, whether from the public or private sectors, have always been, and will always be, interested in impact at scale. After many bad experiences in the past, monitoring and evaluation of research and development projects has become, understandably, more and more institutionalised over the past few decades (Crawford and Bryce, Reference Crawford and Bryce2003). In preparing frameworks for agricultural development project monitoring, both implementers and investors wrangle with the issue of finding meaningful indicators. Technology adoption is a long-term process, and discussions over when one can consider a technology adopted are complicated (Mwangi and Kariuki, Reference Mwangi and Kariuki2015). The duration of most projects is considerably shorter than the time-frame for meaningful adoption and impact of agricultural technologies, and so, to ensure some level of accountability, investors necessarily need to establish intermediate metrics to evaluate the completion of project activities. In the design of all projects in which we have been involved we have negotiated these indicators with the proposed investor, and have been prepared to walk away from the project if we felt we could not achieve the milestones and complete the indicators required by the investor. Where an investor insists on a metric that we feel is not worthwhile we will try to negotiate a change in indicator or simply collect the information if it does not require an inordinate amount of effort. This does not imply that we change our objective of achieving an efficient and effective research for development project, and we certainly do not denigrate our project because we feel that investors’ metrics are not adequate.