Introduction
Access to technologies suitable for smallholder farmers still remains a major barrier in developing countries, primarily due to limited agricultural research capacity and investment (Emerick et al., Reference Emerick, de Janvry, Sadoulet and Dar2016). Developing countries can potentially capitalize on the commercially-available agricultural technologies of other countries to prioritize investment in adaptive research, dissemination, and scaling (Piesse and Thirtle, Reference Piesse and Thirtle2010). However, in many developing countries, support services such as extension, credit, and market access that are indispensable for acceptance and wide-scale adoption of technologies is also lacking (Christoplos, Reference Christoplos2010; Gebremedhin et al., Reference Gebremedhin, Jaleta and Hoekstra2009; Rajalahti and Swanson, Reference Rajalahti and Swanson2010). These challenges can be lessened when farmers organize in groups or cooperatives to adopt new technologies and management practices and collectively bargain for output marketing, especially when HHs are resource-poor and scattered in rural areas (Rivera and Qamar, Reference Rivera and Qamar2003; Wanyama et al., Reference Wanyama, Develtere and Pollet2009; Woldu et al., Reference Woldu, Tadesse and Waller2013).
Most of the rural farm HHs in developing countries are relatively poor and illiterate, which impedes technology adoption. Extension agents are important to introduce, educate, and persuade farmers to adopt new technologies or management practices. However, the traditional ‘top–down’ approach, where extension agents impose their solutions to all the farmers’ problems, is also not ideal. Farmer’s group could help solve some technology adoption challenges when programs encourage farmer’s groups or cooperative members in participatory technology evaluation together with the researchers, extension agents, and service providers. This can help ensure more realistic assessments of farmer needs and technology suitability with support from extension agents (Pretty, Reference Pretty1995; Rivera and Qamar, Reference Rivera and Qamar2003). Group-based agricultural extension activities are becoming increasingly popular for different types of agricultural interventions like integrated pest management (Ravnborg, Reference Ravnborg2004), user-managed irrigation systems (Chaudhry, Reference Chaudhry2018), and natural resource management (Gautam and Shivakoti, Reference Gautam and Shivakoti2005; McCarthy et al., Reference McCarthy, Dutilly-Diané and Drabo2004) including in Nepal. They are closely-aligned with the collective action concept under the New Institutional Economics theory, in which individual members of farmers’ groups together make decisions that benefit the collective interests (Olson, Reference Olson1971; Ostrom, Reference Ostrom1990).
The literature shows that impacts of farmers’ group or cooperative membership seem to vary by country and type of technology. African countries are disproportionately represented in these studies. Some of the examples of such studies are from countries like Ethiopia (Abebaw and Haile, Reference Abebaw and Haile2013; Ahmed and Mesfin, Reference Ahmed and Mesfin2017; Bernard et al., Reference Bernard, Taffesse and Gabre-Madhin2008), Kenya (Fischer and Qaim, Reference Fischer and Qaim2012; Shumeta and D’Haese, Reference Shumeta and D’Haese2016), Uganda (Mwaura, Reference Mwaura2014), and Nigeria (Wossen et al., Reference Wossen, Abdoulaye, Alene, Haile, Feleke, Olanrewaju and Manyong2017). In Ethiopia, cooperative membership played an important role in accelerating the adoption of fertilizer but had a limited impact on the adoption of improved seed and pesticides (Abebaw and Haile, Reference Abebaw and Haile2013). This study also shows that cooperative membership has a heterogeneous impact on fertilizer adoption among its members. In Uganda, membership in farmers’ groups increased the yield of banana and cassava, but not of sweet potatoes, beans, and maize (Mwaura, Reference Mwaura2014). The same study also reported that the members were less likely to adopt inorganic fertilizer and improved seed compared to non-members.
In Nigeria, Wossen et al. (Reference Wossen, Abdoulaye, Alene, Haile, Feleke, Olanrewaju and Manyong2017) found that cooperative membership along with access to extension services increased technology adoption and household welfare, but the impact was heterogeneous with significantly stronger impact on technology adoption for smallholders with access to formal credit than for those without access. Some of these examples from African countries indicate that the impact of agricultural group or cooperative membership varies across membership status, crops, technologies, and countries. The impact also depends on the extension modalities employed for the transfer of technology as suggested by Anderson and Feder (Reference Anderson, Feder, Evenson and Pingali2007). These frequently cited papers focus on the direct impact of programs on group or cooperative members, but they did not discuss the spillover effects of the program.
A limited number of studies focus on analyzing the indirect or spillover effects of agricultural technology adoption in comparison to those that focus on direct impacts (Aramburu et al., Reference Aramburu, Garone, Maffioli, Salazar and López2019). It is hard to find direct and spillover effects on adoption, income, and household welfare indicators of a program that involves farmers’ groups in technology testing and validation, which is the focus of this study. Development programs or projects often focus their interventions on a specific target population from certain geographies, institutions, or social/economic groups. Intervention benefits are not only limited to the target population via direct impact but also spread to the non-target population, known as the spillover effect (Angelucci and Di Maro, Reference Angelucci and Di Maro2015). Different authors have reported the spillover effect in agriculture, though the spillover impact is not always significant and positive for all outcomes. For instance, Foster and Rosenzweig (Reference Foster and Rosenzweig1995) and Holloway et al. (Reference Holloway, Shankar and Rahmanb2002) report the positive impact of high-yielding varieties of rice and wheat in rural India and rice in Bangladesh, respectively, while Aramburu et al. (Reference Aramburu, Garone, Maffioli, Salazar and López2019) report the absence of the spillover effect of improved pasture and irrigation technology in the Dominican Republic. Researchers point to the role of social network in the positive spillover effect in the adoption of sunflower seed in Mozambique (Bandiera and Rasul, Reference Bandiera and Rasul2006) and BT cotton in India (Maertens, Reference Maertens2010).
Here we focus on one particular set of farmers’ groups: Community-Based Seed Producer (CBSP) groups that were formed by the Hill Maize Research Project (HMRP) in the hills of Nepal. These groups were mobilized for the participatory evaluation and production of improved maize seed and their release for scaling-up adoption (Tiwari et al., Reference Tiwari, Vir and Sinclair2009a; Tiwari et al., Reference Tiwari, Ortiz-Ferrara, Urrea, Katuwal, Koirala, Prasad, Gurung, Sharma, Hamal, Bhandari and Thapa2009b). This paper aims to measure the direct effects of household CBSP group membership on income, household food security, and social welfare related to female workload and female leadership opportunities, and the spillover effects on improved variety adoption, yield, and household food security. In Asia, particularly South Asia, studies on the impact of group/cooperative membership on farm HHs’ agricultural technology adoption and household welfare are rather limited, and none were found for Nepal. Specifically, for maize-related technologies, the impact literature focuses more on sub-Saharan Africa, with limited evidence on the technology impacts in maize production systems of South or Southeast Asia (Krishna et al., Reference Krishna, Feleke, Marenya, Abdoulaye and Erenstein2019). In this regard, this paper will generate evidence with respect to direct and spillover effects of a group-based intervention in maize seed adoption and subsequent outcome indicators to contribute to the existing gap in the literature.
Background
Agriculture in Nepal remains largely subsistence. This is particularly true in the hills, where staple crops like maize dominate a farming system that generally lacks innovation and efficiency. Private sector extension has very limited reach and is mostly available in Terai districtsFootnote 1 or nearby cities. Public sector agricultural extension is traditional, often ineffective, and hampered by political instability and lack of clear direction. Other support services like agricultural subsidies or credit are poorly targeted, mismanaged, and outdated, which contributes to stagnant productivity.
Maize is the second-largest staple crop in Nepal after paddy and comprises about 26% of the total cereal production. Maize is mostly grown by HHs as a staple food crop, and average maize consumption in Nepal is around 100 g per day (Ranum et al., Reference Ranum, Peña-Rosas and Garcia-Casal2014). Maize is also sold for food, feed, fodder, and fuel, and the use of maize grain differs across agro-ecological domains in Nepal. In the mid-hills (600 to 1800 m altitude) and high hills (>1800 m altitude), maize is primarily used for human consumption, whereas in the Terai region, it is primarily used for industrial/feed purposes (CBS, 2015; KC et al., Reference KC, Karki, Shrestha and Achhami2015). The hills of Nepal are the major maize-growing area in the country and dominated by maize-based farming systems, which are often rainfed. With the increasing demand in recent years for maize as feed for the poultry industry and the sale of green cob, maize has become a cash crop in some districts, especially those with better access to market (Dhakal et al., Reference Dhakal, Regmi, Thapa, Sah and Khatri-Chhetri2015; Ghimire et al., Reference Ghimire, Timsina, Devkota, Gautam, Choudhary and Poudel2018). Maize productivity in Nepal is low (2.4 t/ha) compared to the global average (5.5 t/ha) and lower than most of its South Asian neighbors (FAO, 2019b; MoAD, 2016). Productivity is constrained by many factors, including poor access to and adoption of improved agricultural technologies like improved seed. Only around 31% of farmers reported purchasing seed in past three years, and hybrid maize adoption is low in the hills (CIMMYT 2017a; Tiwari et al., Reference Tiwari, Brook and Sinclair2004). Yield in the hills of Nepal is even lower than in the Terai region, primarily due to low adoption of hybrid seed or improved open-pollinated maize varieties (OPVs) and poor agronomical literacy around best management practices compared to farmers in the Terai region.
The adoption of improved maize seed is limited, even though there are 88 improved maize varieties registered or released in the country and almost one-third come from the national research system, including seven hybrids (AICC 2016). Per the National Seed Vision: 2013–2025 (NSV), the seed replacement rate (SRR), which is an indicator of improved seed adoption, was only 9% in 2010 and lower than rice and wheat at 11%. That rate is reported to have improved to 13% in 2017 (SQCC, 2018), still well below the target of 25% by 2025 (GoN, 2013). Around 84% of the seed used in Nepal comes from an informal seed system, which includes seed with no truthful labeling or certification, farmer-saved seed, and non-registered varieties. The other 16% comes from the formal seed sector (SQCC, 2018). For many resource-poor farmers, the availability and affordability of good quality seed are important for agricultural intensification and generating surplus. Local seed production can alleviate supply constraints, as well as provide income to seed-producing farmers. With this realization, several District Agricultural Development Offices (DADO) started community-level seed production programs focused on cereal crops like rice, wheat, and maize (mainly in the hills). The CBSPs gave rise to farmers’ seed systems, which are sometimes referred to as a semi-formal seed system. The systems produce and market affordable seed available within the district (LI-BIRD & TDF, 2017).
Methodology
Community-based seed producer groups
The HMRP in Nepal was implemented in four phases (1999–2014) by the International Maize and Wheat Improvement Center (CIMMYT) in collaboration with national partners, including the Nepal Agricultural Research Council (NARC). The first three phases were funded solely by the Swiss Agency for Development and Cooperation (SDC). The fourth phase was funded by SDC and the U.S. Agency for International Development (USAID). The first phase (1999–2002) was mostly a research phase. The second (2003–2007) and third phases (2008–2010) included both research [validation and participatory varietal selection (PVS)] and extension (agronomical literacy, seed production training, strengthening partnerships with local stakeholders, and promoting the maize varieties for adoption). The fourth stage (2010–2014) focused on improving productivity, farm income, and economic and social outcomes of resource-poor farmers dependent on maize-based cropping systems, including women and disadvantaged farmers.
The project, in collaboration with the NARC, released 10 OPVs. One was released in 2002 and two were released in each of 2006, 2008, and 2009. Three others were in the pipeline when the project completed its fourth and final phase in 2014 and were released in 2015. Improved maize seed production increased from 7 tons in 2000 to 1460 tons in 2014. At the end of the project, 223 CBSP groups in the project districts were producing improved maize seed. The CBSP groups bring together 15 to 25 community members who express interest in producing maize seed and register them in the DADO to formalize them as seed producers. The CBSP group produces improved maize seed of the varieties best-suited to the locality as identified through the PVS process (Tiwari et al., Reference Tiwari, Vir and Sinclair2009a, Tiwari et al., Reference Tiwari, Ortiz-Ferrara, Urrea, Katuwal, Koirala, Prasad, Gurung, Sharma, Hamal, Bhandari and Thapa2009b). The groups receive source seed for the chosen variety and training on seed production, postharvest, seed testing (to improve their agronomical literacy and skills), and seed marketing (connecting them with seed and grain buyers).
An earlier assessment suggests adoption of the seed produced by these CBSPs to have improved maize productivity and household food security. The earlier study reported that HHs in food deficit and those from underprivileged castes benefitted more than HHs with more resources (Tiwari et al., Reference Tiwari, Ortiz-Ferrara, Gurung, Dhakal, Katuwal, Hamal, Gadal and Virk2010); however, this was based on limited data from 2008 to 2009 (nearly halfway through the project) and used analysis of variance. Tiwari et al. (Reference Tiwari, Ortiz-Ferrara, Gurung, Dhakal, Katuwal, Hamal, Gadal and Virk2010) is likely to have over or underestimated the true impact of the project, especially because the project was still running and did not consider a comparable control group (La Rovere et al., Reference La Rovere, Mathema, Dixon, Mercado and Gurung2009). Our aim in this paper is to assess both direct and indirect (spillover) impact of the CBSP group program using a quasi-experimental method to estimate both direct and indirect (spillover) impact of the CBSP group program. For this, we use project end-line survey conducted in 2014 which captures data from 1260 HHs from 60 Village Development Committees (VDCs) in 20 districts to produce more accurate and robust impact assessment of the project that is generalizable to the entire project districts. To the best of our knowledge, no impact literature related to community-based groups looks at both direct and spillover impacts for groups that specialize in seed production in general, particularly in South Asia.
Conceptual framework and impact pathway
For the impact effects, we compare three types of HHs. For convenience, hereafter, we will call HHs who are a member of a CBSP group as the treatment group, HHs who are not a member of the CBSP group but are within the project district as a control-in group, and the HHs who are not a member of the CBSP group and are outside of the project district as a control-out group. These names are also reflected in the conceptual framework in Figure 1.
Conceptual framework and the impact pathway of improved germplasm and Community-Based Seed Producer (CBSP) groups.
Seed producers are expected to differ from grain producers when it comes to income from maize, workload, and leadership opportunities including that for female members of the HHs. Membership in CBSP groups is especially attractive to resource-poor farmers with smaller landholdings. Seed producers (treatment group) are expected to get lower yields but possibly higher prices and higher income than grain producers per unit area. The seed producers are likely to receive more competitive price when they collectively sell seed through their CBSP group than when they sell on their own. Unlike maize grain producers (the control-in group and control-out group) who can keep all the grain or what they need for household consumption, the seed producers (treatment group) use the income from the sale of maize seed to purchase food, thus spending more on food than control groups. In addition, involvement in group activities and seed production is expected to have some social impacts on the treatment group. Maize seed production and seed postharvest are more labor intensive than maize grain production, and thus seed production is likely to increase workload. Given that a disproportionate share of household workload falls on women in Nepal, it is critical that the gain in income does not come at the expense of increased workloads for women. Additionally, the involvement of family members in group meetings and decision-making increases leadership opportunities for the treatment group HHs, including women. In this regard, it is expected that the women in the treatment group will participate more in group activities (any group or community-level institution) than women from the two control groups.
Based on the conceptual framework discussed above, we compare the outcome indicators between treatment and the control-in group (direct impact) and the control-in and control-out groups (spillover impact) as shown in Table 1.
List of outcome indicators considered for direct and spillover impacts with the expected impact direction
† The variety adoption and yield are not relevant for CBSP members because they focus on improved seed production rather than use improved seed for grain production. The yield of grain is higher than yield of seed, and comparison between treatment and control-in groups is not reasonable. For further detail, see the text.
‡ Comparing the control-in and control-out groups for female workload and female leadership opportunity is not relevant because spillover is only assumed through improved seed adoption. For further detail, see the text.
We consider five outcome variables for direct impact assessment and four variables for spillover impact assessment. The treatment group uses source seed to produce improved seed, which they sell for income. ‘Adoption’ of improved seed, therefore, is not a relevant indicator for the treatment group. The treatment group produces seed which typically has lower yields (but potentially higher prices and more income) than when maize is produced for grain, making ‘yield’ a less relevant indicator. We use maize sufficiency and per-capita food expenditure on cereal crops to assess the impact of CBSP program on food security. We expect the treatment group that focused on the sale of seed to have lower maize grain self-sufficiency from their own production. However, with their higher income from the sale of seed, we expect them to spend more on food purchases. Similarly, we expect the control-in group to have a higher maize yield (thus more production) which they primarily use for their own consumption. Some HHs, as most are smallholders may take surplus to market and use the income to purchase food, mostly rice. On the other hand, the control-out group is expected to have lower access to improved seed, and thus will have lower yields (production). They therefore will spend more money on the purchase of food using agricultural or non-agricultural income.
We consider female workload and female leadership opportunities as direct effects but do not find them relevant for spillover effect. The Women’s Empowerment Index in Agriculture (WEIA) is comprehensive and composed of five dimensions (Alkire and Meinzen-Dick, Reference Alkire and Meinzen-Dick2013). These are: access and decision-making with respect to input in production decisions (Production), access to or ownership of assets and decisions on credit (Resources), control over the use of income (Income), group membership or public speaking (Leadership), and workload and leisure (Time). It was not feasible in this research to conduct an elaborate survey for WEIA calculation. Depending on the type of intervention, not all dimensions were of interest to us (at least in the short-term). We capture some information related to the role of women in the HHs surveyed with a particular focus on workload and participation in groups or cooperative or community-level organizations. Our interest here was to see if women members in the treatment group were more involved in group activities/meetings (as a measure of leadership) and if women household membership in the treatment group had higher workloads than the control-in group.
Sample selection
In 2014, there were 75 districts in Nepal (20 Terai, 39 Hill, and 16 Mountain districts). The HMRP covered 20 of 39 hilly districts in Nepal (14 mid-hill and 6 high-hill). This study covers 20 mid-hill districts, 10 in project area, and 10 outside project area. The selected districts are distributed from east to west and represent the country’s mid-hill HHs as shown in Figure S1 (see the supplementary material). Multistage random sampling was employed to draw 1260 HHs comprised of three types of HHs. A total of 60 VDCs were surveyed, three from each of the selected 20 districts. However, only 1213 HHsFootnote 2 were retained for the data analysis (Table 2). The data allow for two types of comparisons to estimate the direct and spillover impacts of CBSP membership. Direct impacts are estimated in the project intervention districts by comparing HHs who are members of the treatment group with a control-in group. The spillover impacts are estimated by comparing the control-in group from the project district and the control-out group from non-project districts.
Multistage random sampling for the selection of the districts, Village Development Committees (VDCs), and households (HHs) for the survey
Statistical matching and heterogeneity
An ex-ante random assignment of the household to the treatment and control group would be an ideal way to assess the impact of CBSP group membership (treatment) on outcome variables, but such random assignment is not always possible. In a non-randomized study, like the one here, HHs’ probability of receiving a treatment as well as the magnitude of the outcome variables will depend on HHs’ observable and unobserved characteristics (Heckman et al., Reference Heckman, Ichimura and Todd1997). When using observational data for impact evaluation, there is a problem of selection bias, which limits the validity of the estimates of the causal effect. It is possible to minimize such bias using statistical methods of matching (Cochran and Rubin, Reference Cochran and Rubin1973; Rosenbaum and Rubin, Reference Rosenbaum and Rubin1983). There is growing interest in the use of quasi-experimental matching approaches for impact evaluations. In this study, we employ the propensity score matching (PSM) method commonly used in impact assessment literature in different fields, including agriculture (Gautam et al., Reference Gautam, Schreinemachers, Uddin and Ramasamy2017; Mendola, Reference Mendola2007; Schreinemachers et al., Reference Schreinemachers, Wu, Uddin, Ahmad and Hanson2016), and specifically to look at the impact of group membership on outcome variables (Abebaw and Haile, Reference Abebaw and Haile2013; Ahmed and Mesfin, Reference Ahmed and Mesfin2017; Mwaura, Reference Mwaura2014; Priscilla and Chauhan, Reference Priscilla and Chauhan2019; Wossen et al., Reference Wossen, Abdoulaye, Alene, Haile, Feleke, Olanrewaju and Manyong2017). PSM allows matching each treatment household with similar control households and compares the average difference in the outcome variables between these two groups using the average treatment effect (ATE) estimates. The PSM method can produce biased results due to misspecification (Robins et al., Reference Robins, Sued, Lei-Gomez and Rotnitzky2007; Wooldridge, Reference Wooldridge2007). Use of the inverse probability weighting (IPW) method in addition to PSM to estimate the ATE is a potential remedy for such misspecification (Wooldridge, Reference Wooldridge2010). The IPW method is similar to PSM, but it estimates the ATE in two steps (Imbens and Wooldridge, Reference Imbens and Wooldridge2009). The first step is the estimation of propensity scores. The second stage runs a linear regression to estimate the ATE using an inverse probability weighted least square where the weights are the inverse of the propensity scores (i.e., gives greater weightage to HHs with a higher predicted probability of being selected for the treatment group, which is CBSP membership in our case) rather than directly using propensity scores to compare treatment and control HHs. The ATE is then calculated as the difference between the weighted averages of treatment and control HHs. This paper employs both PSM and IPW to control for selection bias to evaluate the impact of CBSP group membership. We employ the nearest neighbor approach to calculate the propensity score.
One of the major criticisms of the PSM method is the omitted variable bias. To address this, we use large number of covariates (17 covariates) in PSM estimation, use observations within the region of common support, test the matching quality, and examine the sensitivity of the estimated results to minimize hidden bias as suggested in PSM literature (Heckman et al., Reference Heckman, Ichimura and Todd1997; Rosenbaum Reference Rosenbaum2000; Smith and Todd Reference Smith and Todd2005). To check the sensitivity of the treatment effect, we evaluate the result using the propensity scores from four different approaches (one and five nearest neighbors, radius, and kernel density). More recently, it is advised to use PSM with placebo regression (Imbens and Wooldridge, Reference Imbens and Wooldridge2009). Following Abebaw and Haile (Reference Abebaw and Haile2013) and Cunguara and Darnhofer (Reference Cunguara and Darnhofer2011), this paper uses this regression to ensure the robustness of the treatment effect results. To measure this, an Ordinary Least Squares (OLS) regression is estimated using a priori-determined dependent variable on treatment variable and the same covariates as used for PSM estimation. The coefficient for this priori-determined dependent variable, which in our case is the number of years the spouse of the household head has been involved in agriculture, is not expected to be significantly different from zero to indicate there are no omitted variables correlated with the outcome variables.
The extent of impact based on the full sample of HHs can underestimate the true impact (Rubin and Thomas Reference Rubin and Thomas2000), but the PSM allows estimation of the heterogeneous treatment effect based on socio-economic household types. Following the approach used by multiple researchers (Abedaw et al., Reference Abedaw, Fentie and Kassa2010; Abebaw and Haile, Reference Abebaw and Haile2013; Cunguara and Darnhofer, Reference Cunguara and Darnhofer2011; Wossen et al., Reference Wossen, Abdoulaye, Alene, Haile, Feleke, Olanrewaju and Manyong2017), we check for heterogeneous effect by regressing household treatment effect from PSM on different household characteristics of interest. In our case, these are: (i) social class of the household (1 = Dalit caste household, 0 = other), and (ii) the household agricultural holding area (1 = less than 0.2 ha, 0 = 0.2 ha or more). In Nepal, Dalit HHs are considered socially and economically deprived, and caste/ethnic identity affects the degree to which rural HHs benefit from development activities (Bennett Reference Bennett2005). We choose 0.2 ha as the area cut-off because it reflects the lowest quartile of landholding in our sample HHs. In rural Nepal, landholding size is highly correlated with household’s wealth and income, as agriculture remains the dominant source of income. With this in mind, we assess whether the project interventions showed heterogeneity by social grouping and level of wealth based on landholding.
While we choose to use PSM rather than other quasi-experimental methods, other methods like the instrumental variable approach could have been used. However, since we have considered multiple outcome variables (five for direct impact and four for spillover impact), it was hard to find suitable instruments. Another option could have been to use simultaneous equations (where there is some sort of sequential relationship among the outcomes: adoption to yield to food security); however, we have different direct impact pathways and five different outcome indicators considered for direct impact (see Figure 1). To make our analysis and approach consistent and comparable across different outcome variables (attained though different impact pathways) for both direct and spillover impact assessment, we chose to use PSM over other methods.
Result and Discussion
Estimation of the propensity score matching
A face-value comparison of observable characteristics between the treatment and control-in groups and between the control-in and control-out groups showed several variables to be significantly different as presented in Table S1 (see supplementary material). The treatment group has significantly larger landholdings, lower percentage increase in maize area in five years, a higher percentage of HHs using fertilizer, and more HHs using more than two varieties of maize compared to the control-in group. On the other hand, the control-in group has limited access to market and extension offices, smaller landholdings, lower percentage increase in maize area in the past five years, and a higher percentage area in the upland (unirrigated) compared to the control-out group. In addition, the control-in group had a higher percentage of HHs growing rice and using fertilizer but a lower percentage of HHs owning a television and more female-headed HHs.
However, these differences between the treatment group and the control-in group and the control-in group and control-out group need to be interpreted with care due to selection biases. For any meaningful treatment effect calculation for direct (treatment vs. control-in group) and spillover impact (control-in vs. control-out group), we check for the matching quality of the mean (mean before matching vs. mean after matching) of the 10 continuous and 7 binary variables that are used for estimation of the propensity scores based on logistic regression models.
The use of propensity score estimators eliminated selection bias in observable characteristics of the treatment group and control-in HHs in the project intervention districts as shown in Table S2 (see supplementary material). The explanatory variables jointly explained 6% of the variation in program placement (CBSP membership) based on logistic regression. After matching, this decreased to 1%. The overall standard percentage bias was 4%, while none of the covariates individually had a bias greater than 10%. This meets the benchmarks for matching suggested by Rosenbaum and Rubin (Reference Rosenbaum and Rubin1983). An unpaired t-test of the mean of the covariates after matching also shows no significant differences (p < 0.1) between the two groups after matching. Of the 807 HHs in the project intervention district, 355 (44%) were treatment HHs, while the other 452 (56%) HHs were from the control-in group. A total 797 HHs (out of 807 HHs) found at least one match (treatment: 349, control: 448). Six treatments and four control HHs did not find a match. The balancing property was satisfied with four blocks, and the propensity score distribution of the treatment group and control-in group ranges from 0.2 to 0.9. The common support plot of the two groups shows considerable overlap (Figure 2a). In addition, we tested the matching quality to check the reduction in bias and equality of the mean of the covariates after matching. Table S2 (see supplementary material) shows the mean of the treatment and control-in group after matching where the mean bias reduced from 10 to 4% with all of the covariates with percentage bias less than 10% after matching, and the mean of the covariates between the treatment and control-in group are statistically similar after matching.
Overlap between treatment group (household (HH) that are member of Community-based Seed Producer (CBSP) group) and control-in group (HH from project districts that are not members of CBSP group) in Kernel density distribution.
Similarly, the use of propensity score estimators also eliminated selection bias in observable characteristics of the control-in group and control-out group HHs. The explanatory variables jointly explained 11% of the variation, which decreased to 1% after matching. The overall standard percentage bias was 4.4%, and an unpaired t-test of the covariates after matching also showed no significant differences (p < 0.1) between the two groups after matching. Of the 858 HHs who were not members of CBSP groups, 452 (53%) were from the control-in group and 406 (47%) were from the control-out group. Of the 452 control-in HHs, 17 HHs did not find a match from the HHs in the control-out group. The balancing property was satisfied with nine blocks, and the propensity score distribution of the control-in group and control-out group ranged from 0.16 to 0.94. The common support plot of the two groups shows considerable overlap (Figure 2b).
Overlap between control-in group (household (HH) from project districts that are not members of Community-based Seed Producer CBSP group) and control-out group (HH outside project districts) in Kernel density distribution.
Impact of CBSP groups on member HHs
At face value, the treatment group compared to the control-in group has statistically higher mean maize income (1% level of significance), maize sufficiency (10% level of significance), and female leadership opportunity (1% level of significance), though there is no significant difference in food expenditure and female workload (see Table S3 in supplementary material). However, these descriptive statistics cannot be used to conclude that the differences between the treatment and control-in groups are due to CBSP group membership. To account for the confounding factors and to attribute any differences in the mean of the outcome variables to CBSP group membership, we use matching methods.
The results based on the PSM confirm that the treatment group had significantly higher maize income (around 40% higher) and higher female leadership opportunities (around 65% higher) compared to the control-in group (Table 3). On the other hand, the result did not show the treatment effect to be significant for maize sufficiency, food expenditure, and female workload. The results for all five impact indicators are consistent for both PSM and IPW methods.
Average treatment effect (ATE) of Community-based Seed Producer (CBSP) group membership on maize income, maize sufficiency, food expenditure, female workload, and female leadership opportunity calculated using PSM and IPW methods
PSM, propensity score matching; IPW, inverse probability weighting; ATE, average treatment effect; PO mean, potential outcome mean (the mean for the control group); SE, standard error; *** denote significance of mean difference at the 1% levels; ns, not significant at 10%.
It is important to check for sensitivity of the result and omitted variable bias to assess the robustness of the result based on the PSM or IPW method that we have used (we used one nearest neighbor). We re-estimated the treatment effects using three additional methods of matching (nearest neighbor with five neighbors, Kernel matching, and radium matching), and these results conform to the result from the PSM and IPW method in Table 3. The result of the sensitivity of treatment effect based on these additional matching methods is in Table S4 (see supplementary material). In addition, we follow Cunguara and Darnhofer (Reference Cunguara and Darnhofer2011) and Abebaw and Haile (Reference Abebaw and Haile2013) and run a placebo regression. Our result shows that the coefficient on the treatment variable (i.e., CBSP membership status where 1 = yes, 0 = no) is not significant on the a priori dependent variable (see Table S5 in supplementary material) which further shows that the estimated treatment effects using the PSM or IPW approach are not likely to be affected by omitted variable bias.
Two points need to be discussed based on these results. First, the difference in maize sufficiency in Table S3 (see supplementary material) cannot be attributed to CBSP group membership because that result cannot be confirmed based on matching. Second, seed production is more labor intensive than grain production, and we suspected it would increase the workload of female members in the treatment group; however, the results show no effect. The treatment effect on female leadership opportunities, though, is positive and significant, which is probably due to their participation in the CBSP groups or other groups in their communities. In addition, we expected a significant treatment effect on food expenditure, assuming the treatment group would focus on seed production (which means lower grain for household use), and the income from seed sales would be used to purchase food. The result did not support this assumption. Exploring further, we found that around 31% of the HHs in the treatment group did not produce seed but produced grain only in the study year, while many treatment group HHs also produced maize grain and reported higher yield compared to the control-in group (2.5 vs. 2.2 t/ha). This is likely due to the improved seed they have from their own seed production and their slightly higher total cultivable area.
Heterogeneous effect on the direct impact
In addition to treatment effect, we also tested for heterogeneity on impact across two major types of HHs targeted by the project: (i) Dalit HHs (disadvantaged social caste) and (ii) marginal HHs (landholding size < 0.2 ha). For both Dalit HHs and marginal HHs, the impact on maize income is greater than their counterparts but not for food expenditure and female workload (Table 4). In comparison to their counterparts, the Dalit HHs show a greater impact on female leadership opportunities, and the marginal HHs show a greater impact on maize sufficiency. The higher impact on Dalit and marginal HHs indicates that the project was successful in targeting these disadvantaged HHs to some extent. The higher impact on marginal HHs for maize self-sufficiency is counter-intuitive but may be due to marginal HHs’ small household size (five vs. six people), lower percent of maize sold (5 vs. 11%), percentage of area under maize (90 vs. 50%), and higher yield (2.4 vs. 2.1 t/ha) compared to their counterparts.
Heterogeneous average treatment effect on different outcome indicators based on caste and size of landholding
The dependent variable is the ATT of each respective outcome indicator. Standard errors are reported in parentheses. * Significance at the 10% level. ** Significance at the 5% level. *** Significance at the 1% level.
The difference in mean treatment effect is estimated using diff = [mean (1)–mean (0)]. Thus, positive signs indicate that the treatment effect is larger for Dalit households compared to other social groups (in Column 2) and small-holders (<0.2 ha) compared to larger-holders (≥0.2 ha) (in Column 4).
Spillover effect of improved seed produced by CBSP groups
At face value, the control-in group has a statistically higher rate of improved seed adoption, higher maize yield, and better maize sufficiency, while the control-out group spends more on food (see Table S6 in supplementary material). However, this difference cannot be attributed to CBSP groups without controlling for confounding factors. Therefore, we present the results from the PSM and IPW methods (Table 5) and discuss the result in this section.
Average treatment effect (spillover) for control-in group compared to control-out group based on PSM and IPW methods
*, **, *** denote significance of ATE at the 10, 5, and 1% level, respectively; ns, not significant at 10%.
The study sample allows us to contrast control-in and control-out groups to estimate the spillover effects of improved seed (and variety information), as discussed in the conceptual framework earlier. The improved seed adoption and maize yield are both significantly higher for the control-in group than the control-out group. On the other hand, the food expenditure is significantly lower and maize sufficiency significantly higher for the control-in group compared to the control-out group. These results are consistent for both the PSM and IPW methods, even though the magnitude of such significance varies slightly for these two methods. As discussed earlier, the major objective of the CBSPs is to increase the adoption of improved seed for higher productivity and household food security. Based on these results for spillover impact, there is evidence that engaging the treatment HHs in seed producer groups to produce seed not only increased the income of the seed producers but also led to improved seed adoption, maize yield, and food security of the non-seed producer HHs in the project districts.
We tested for heterogeneous effect on caste and size of landholding for spillover effect and found no evidence of heterogeneity both in adoption and crop yield. Thus, it did not make sense to explore for heterogeneity in other higher-order impact indicators like maize sufficiency and food expenditure because any spillover heterogeneity may not be attributable to the spillover of improved seed or variety information. We tested for the sensitivity of the spillover effect reported in Table 5 using three additional methods of matching and results are presented in Table S7 (see supplementary material). We found the ATE for the spillover impact variables calculated based on these three different matching methods presented (see Table S7) and results based on PSM with one nearest neighbor and IPW method (see Table 5) to be similar. This indicates that the results are not sensitive to ATE calculation method.
Conclusions and Policy Implications
Maize is strategically important to rural livelihoods in the hills of Nepal, yet access to improved technology, including high-yielding maize varieties, has been a challenge. To enhance timely access to affordable improved maize seed, CIMMYT, together with national partners, initiated a CBSP program coupled with activities to promote new and promising varieties within the country. CBSP groups were formed and mobilized in the hilly region, which is the major maize growing region in the country. This study evaluated the impact of the CBSP program using the PSM method to assess direct and spillover impacts based on a set of outcome indicators. Our results suggest a significant positive direct impact of CBSP membership on maize income and female leadership opportunities but not on maize sufficiency, food expenditure, and female workload. The direct effects were heterogeneous with more maize income (for both Dalit and <0.2 ha landholding HHs), more female leadership opportunities (for Dalit HHs only), and greater maize sufficiency (for <0.2 ha landholding HHs only). We also found spillover impacts: a positive effect on improved seed adoption, maize yield, maize sufficiency, and a negative effect on food expenditure (HHs spent less on food), which aligns with our expectations. There was no heterogeneous spillover effect based on caste and size of holdings.
Farmer seed systems are by far the most important source of seed in most developing countries (Almekinders and Louwaars, Reference Almekinders and Louwaars2008). The same is true in Nepal, with the informal seed system contributing more than 80% of the seed used by the farmers. Our results further prove the important role played by farmer seed systems like CBSP groups because there are benefits for both seed producers (direct effect) and the users of the seed produced by CBSP groups (spillover effect). The managerial capacity of the CBSPs (especially in business and marketing) is essential for CBSP groups to be sustainable and successful after donor support is withdrawn (Witcombe et al., Reference Witcombe, Devkota and Joshi2010). We suggest that policies and programs make sure seed producers like CBSP groups are sustainable to ensure the supply of improved seed in the country. We recommend four interrelated policies and priorities for the sustainability and growth of CBSP groups.
First, the Agriculture Development Strategy (ADS, 2015–2035) envisions a self-reliant, sustainable, competitive, and inclusive agricultural sector that drives economic growth and contributes to improved livelihoods and food and nutrition security (MoAD, 2014). Nepal is a net importer of maize, and the major demand for maize grain comes from the country’s poultry sector (Timsina et al., Reference Timsina, Ghimire and Lamichhane2016). For self-reliance, gradual import substitution is necessary, which can be achieved with a gradual increase in production and/or productivity improvement. This requires more domestic production, which can happen with more area or more productivity which makes the role of farmer seed systems like CBSP groups vital in the hills of Nepal. In this regard, government policies should focus on building strong linkages across the poultry-feed maize grain and maize seed value chain. This must benefit all value chain actors, including seed producers like CBSP groups as a strategy for import substitution and self-reliance.
Second, the country in the recent years has transitioned to a federal governance structure from centralized governance that grants authority over agriculture development and extension to the provincial and local level. This makes it important that seed production, extension, and regulations align with national policies (like ADS and NSV) and value the role of seed producers to ensure the supply of quality seed at the provincial and local levels. We suggest provincial and local authorities to identify maize seed production zones in their area because maize is a cross-pollinated crop and maintaining isolation distance is important to produce quality seed. Authorities should also focus on reducing risk for seed producers through seed financing and insurance programs and link these provincial informal farmer seed systems with the formal seed system.
Third, authorities should heed the slowly increasing demand for hybrid maize seed in the country (especially in the Terai region, though the hybrid use is gradually increasing in the hills as well), which is currently exclusively met by import. In this regard, it is time to consider preparing these CBSP groups for diversifying their seed production to hybrid maize but continuing OPV seed production as well. This will require investment in hybrid research capacity and strong public-private partnerships, including the community-level seed producer groups. Given the limited investment and capacity for crop breeding in the country, continued engagement with CIMMYT (for germplasm, market-ready lines from international hubs, and capacity development) or other organizations with institutional knowledge and competency in maize research and development, at least for the short- and medium-term, is necessary.
Fourth, the success of the agriculture sector is not reflected merely by yield, income, and food security; nutritional security is also critically important. Nutritional insecurity among women and children is high in Nepal. Disadvantaged castes are disproportionately affected (MoH, 2017; Pandey and Fusaro, Reference Pandey and Fusaro2020), especially in rural areas. We suggest taking advantage of the spillover effect of the CBSP-produced seed and diversify their seed production to Quality Protein Maize (QPM)/biofortified maize seed in the hills (where maize is a staple food). Seed production can start with two already-released QPM varieties and uptake biofortified varieties currently under testing once they are released. This could be a potentially cost-effective way to improve nutritional security. As an incentive to switch to hybrid and QPM/biofortified seed production, the government could channel current untargeted seed subsidy programs to these value-added varieties. Again, given the cost advantage of using biofortified maize in the poultry industry (Thapa et al., Reference Thapa, Gautam, Rahut and Choudhary2020), programs and policies to encourage these seed producers to focus on biofortified maize seed production and developing strong links between the feed industry and the maize seed and grain value chains are recommended.
The impact of this project provides more insight into developing new projects for improving capacity (breeding, research, and development) in Nepal. CIMMYT, as a research-for-development organization, strives to strengthen global partnerships and to contribute to meeting the Sustainable Development Goals (SDGs) with its maize and wheat research and development work (CIMMYT 2017b). This study adds to the documentation of approaches, impacts, and lessons learned – both for institutional memory (La Rovere, Reference La Rovere, Mathema, Dixon, Mercado and Gurung2009) and the wider agricultural R&D community. Our findings on the direct impact of CBSP membership are consistent with some similar studies on cooperatives, especially in Africa; but we also document important spillover impacts which are rarely documented. In this regard, this study will add to the impact literature related to farmers’ group specifically for maize technology in South and Southeast Asia region for which there is limited evidence compared to sub-Saharan Africa (Krishna et al., Reference Krishna, Feleke, Marenya, Abdoulaye and Erenstein2019). This impact assessment is based on data collected immediately after the HMRP completion in 2014. A follow-up may provide more insight into how the impacts have persisted and become institutionalized over the years.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0014479720000381
Acknowledgements
This study is an ex post evaluation of components originally implemented under the Hill Maize Research Project (HMRP), a multi-donor and multi-phase project in Nepal. This study was supported by the CGIAR Research Program on maize agri-food systems (CRP MAIZE). The contents and opinions expressed herein are those of the authors and do not necessarily reflect the views of the associated and/or supporting institutions. The usual disclaimer applies. We thank past CIMMYT colleagues, Dr. Christian Böber (Agriculture and Market Economist) and Subash S.P. (Research Associate), for leading the survey design and data collection work. We also acknowledge Mr. Uttam Khanal (Program Officer, Agricultural Economist) from Local Initiative for Biodiversity, Research and Development (LI-BIRD) and Mr. Ram Krishna Neupane (Program Director) from Forum for Rural Welfare and Agricultural Reform for Development (FORWARD), for their collaboration with CIMMYT for implementing the survey for collecting the data for this study. We also extend our thanks Mr. Shashish Maharjan (Geo-spatial Analyst, CIMMYT Nepal) and Mr. Prabin Dongol (Assistant Research Associate) for their help in generating maps and data cleaning respectively. We are grateful to four anonymous reviewers for their valuable comments and suggestions.
