THE ARGUMENT: HOW RESEARCH IMPROVES TEACHING AND VISA VERSA

Professors are at their best when they are teaching their students how to do research by practicing it together. Also, arguably, collaborating with undergraduates as full coauthors can produce superior research products. This argument draws on some of the key lessons from the last few decades of findings in psychology and neuroscience about how humans acquire and retain learning. It also uses insights about how scientific breakthroughs occur and applies one of the most powerful ideas from social science to the domain of the research group.

First, designing and executing research makes learning tangible and concrete in a way that students’ highlighting textbooks, committing facts to short-term memory, and practicing stenography during lectures cannot. Second, research reinforces the key lessons from conceptual learning through repeated, deliberate practice. And third, research motivates learning by engaging the core human drive of curiosity while enabling study to be fully self-directed as students proceed at their own pace on topics of their own choosing.

Moreover, we argue that research collaborations with undergraduates can produce superior social science. Undergraduates, as the consummate outsiders, are able to make outside-the-box connections among disparate ideas in ways that often defy the abilities of seasoned veterans long socialized in received wisdom. In addition, research collaborations with undergraduates capitalize on specialization and exchange of intellectual labor and exploit skill complementarities that can more efficiently produce more high-quality research outputs.

We elaborate on these points and provide illustrations in the following text, beginning with a discussion of how research improves learning. In the examples, we emphasize field experiments in international development because that is where our experience has been concentrated. Because experiments are simple in conception (even if sometimes complicated in execution), very early in the process students can begin to brainstorm ideas for interventions and outcomes of interest even without much methodological training—as long as they receive attentive coaching from faculty. Most of the student projects described in this article were initiated within the first month of learning about experimental political science. Experiments may thus make undergraduate engagement in research especially welcoming. However, as we describe, we believe that some of the insights about learning through research collaboration will likely generalize to other areas of political science as well.

RESEARCH MAKES LEARNING CONCRETE

Findings in psychology and neuroscience have made it clear that audiences retain ideas best when they have concrete handles to grasp (Heath and Heath Reference Heath and Heath2007; Rubin Reference Rubin1995; Sadoski Reference Sadoski2001; Sadoski, Goetz, and Rodriquez Reference Sadoski, Goetz and Rodriguez2000 ). Heath and Heath (Reference Heath and Heath2007) note that human memory does not resemble a large filing cabinet, rather it is more like Velcro: “the more [tangible, concrete] hooks an idea has, the better it will cling to memory” (111). Students may be able to commit to short-term recall the fact that variance in independent variables is expected to be statistically associated with variance in dependent variables. However, when students design an experimental intervention, such as a threat that a needed development project might go to Rwandan foreigners, and actually measure and statistically analyze the intervention’s effects compared to control on whether non-co-ethnics cooperate with each other, the concepts become tangible, practical, and meaningful. Thus, the lesson about statistical inference is much more likely to stick over the long term.

A collaboration between coauthors Matthias and Nielson and two other undergraduates further illustrates this point. In Nielson’s course on experiments in political science, team-taught with Joshua Gubler, Matthias’ team designed an e-mail experiment testing confirmation bias among microfinance institutions (MFIs). Confirmation bias has been widely studied at the individual level, however, this experiment would be the first study of confirmation bias in nonprofit organizations. By semester’s end, the team had contacted more than 2,000 MFIs and invited them sincerely to learn more about potential research partnerships with Brigham Young University’s (BYU) Political and Economic Development Labs (PEDL). We randomly assigned and sent three different versions of the invitation: (1) a basic invitation as control; (2) the basic invitation plus a new paragraph added near the beginning that cited prominent scientific evidence suggesting that microfinance is successful in alleviating poverty; and (3) the invitation plus a different new paragraph citing high-profile scientific evidence (by the same two well-known authors as the study cited in second invitation) suggesting that microfinance fails to meet key goals in relieving poverty.

The results prompted a second, follow-up study in a subsequent semester. (We also provided more information, as promised, to all MFIs that expressed interest in the potential partnership.) Both experiments produced evidence of confirmation bias: microfinance institutions were significantly less likely to express interest in research collaborations if they had been prompted that the results of the study might be negative, even when reassured that MFIs vary and that they might expect a more positive outcome (Brigham et al. Reference Brigham, Findley, Matthias, Petrey and Nielson2014). The project is in a drafting stage for submission to a peer-reviewed journal in international relations. The study made all aspects of research design and statistical analysis more tangible. Treatment, control, outcomes, measurement, descriptive statistics, difference-in-means tests, and regression analysis all became not merely abstract concepts but concrete tools necessary for getting answers to interesting research questions.

RESEARCH AS DELIBERATE PRACTICE

Perhaps even more important than committing ideas to memory is practicing the ideas deliberately, which reinforces the learning. With his runaway best-seller, Outliers, journalist Malcolm Gladwell (2011) popularized K. Anders Ericsson’s finding that mastery of a complex skill is achieved only after an average of 10,000 practice hours (see Ericsson Reference Ericsson, Krampe and Tesch-Römer1993).^{Footnote 1} Subsequent research has made clear that a vast range exists in the number of deliberate-practice hours required (Hambrick et al. Reference Hambrick, Oswald, Altmann, Meinz, Gobet and Campitelli2014), suggesting that innate talent is at least equally important. Still, few in the debate deny that mastery of complex cognitive skills requires hundreds of hours of deliberate practice.

Practicing the acquisition of new knowledge in the academic sphere prepares students for careers in the information economy. After all, even if most students are not interested in advancing to doctoral programs in political science, the production of legal briefs, business plans, and public policy proposals (to name three of the most frequently delivered professional outputs by political science graduates who go on to careers in law, business, or public administration) bear strong resemblances to novel research projects, and the skills acquired in undergraduate research might therefore transfer to those professional domains.

Over the course of two years, the student coauthors (Herrick and Matthias) spent hundreds of hours learning research skills and practicing them. For example, the three coauthors, along with University of Texas (UT)-Austin professor Michael Findley and other students from BYU and UT-Austin, jointly designed a project to measure and understand the role of media bias in what we term “poverty porn” portrayals of underdevelopment through the sensationalizing of despair, disasters, and conflict.

In the project, we hired and paid dozens of professional Ugandan photographers, randomly assigning them to one of four experimental conditions in a fully crossed design. The assignments required the photographers to take pictures of either “typical Ugandan life” or “typical Ugandan poverty” for either “educational” or “fundraising” purposes. We later loaded the photographs onto tablet computers, and Ugandan enumerators then randomly selected and surveyed hundreds of Ugandan citizens to learn which set of photographs more closely portrayed the lives of normal Ugandans. Not surprisingly, we found that the photos targeting poverty for fundraising presented a significantly biased portrait compared to self reports of a large random sample of Ugandans and compared to control photographs shot according to a set protocol intended to produce an unbiased baseline.

With coaching from Findley and Nielson, the undergraduates planned all of the logistics including the budget, the timeline in country, and project enumeration. After the team arrived in Uganda, the “perfect” design on paper met the challenging realities of the developing world, and the team flexed accordingly. Student researchers executed all aspects of the experiment: they recruited photographers, trained them on their assignments, set up photo collection areas, loaded photos onto tablets, set up randomization schema, hired and trained enumerators on survey distribution, and analyzed results.

The hands-on experience in the field allowed students to synthesize acquired knowledge, practice it in a real-world setting, and reinforce the learning by designing a project capable of producing new knowledge. This also illustrates that collaborative research with undergraduates has the additional advantage of motivating the learning in the first place.

RESEARCH FEEDS CURIOSITY

Psychology and neuroscience have elevated curiosity to the status of human drive, along with the needs for security, food, and sex (Loewenstein Reference Loewenstein1994). Moreover, several field experimental studies by social psychologists show that students are much more likely both to sustain their engagement and to learn more completely when the learning takes place in a context where autonomy is emphasized (Deci, Ryan, and Williams Reference Deci, Ryan and Williams1996; Grolnick and Ryan Reference Grolnick and Ryan1987; Niemiec and Ryan Reference Niemiec and Ryan2009; Vansteenkiste et al. Reference Vansteenkiste, Simons, Lens, Sheldon and Deci2004).

In the collaborative research that we describe in this article, students first select topics that intensely interest them. Faculty then help the students design studies that both can answer the questions and that are connected to important research programs in the discipline in order that the findings speak to a broader audience. The learning thus is self-directed in terms of the topic chosen, the core elements of the design, and the pieces each member of the team elects to undertake.

For example, in a subsequent semester of Gubler’s and Nielson’s experiments in a political science course, one of the coauthors (Herrick) jumped into his experimental design based on a hunch rather than after a thorough literature review. The omission of the standard step of reviewing the literature was intentional—it was scheduled to take place after the basic design was conceived to engender greater creativity. Subsequently, the passion Herrick developed for the subject drove him to master the literature in the subfield and to refine and precisely situate his study.

But this scrambled assignment scheduling does entail some risk. For Herrick and many other undergraduates, one of the most difficult aspects of research writing is performing a comprehensive literature review. Many times promising undergraduate research endeavors falter because they fail to “stand on the shoulders of giants” in developing a compelling question and argument. In the course, Herrick initially was entirely focused on hitting the targets on the assignment schedule, but as he began studying a topic for which he felt passion he started to assume fuller intellectual ownership.

The project, a survey experiment on moderating aggression toward outgroup members, became a self-directed and self-motivated learning experience. During the next two years, the experiment progressed from a course pilot study, to a full experiment, to an academic poster presentation at the Midwest Political Science Association annual meeting, and finally to an honors thesis. With Herrick’s newfound passion for the subject matter came a desire to master the received wisdom related to aggression. As a result, Herrick eventually developed a strong sense of the relevant literature and a solid understanding of where his study fit in the broader research program. The typical slog of developing a thorough literature review, instead, became a compelling process of discovery.

UNDERGRADUATE CREATIVITY CAN IMPROVE PROFESSIONAL RESEARCH

Perhaps more controversially, we also argue that research collaborations with undergraduates are more likely to produce scientific breakthroughs. Experimental political science in particular places a premium on outside-the-box thinking. The best experiments in any scientific field require a novel idea for either the intervention or the outcome measure, or both. While the value of innovation for experiments is perhaps especially obvious, novel questions, arguments, and measures are also highly prized in observational research, so it is likely that collaboration with undergraduates there might also spur new insights.

Whether their research is observational or experimental, seasoned scholars, steeped for many years in given literatures, may have difficulty conceiving new ideas. One of the more interesting findings about creativity to emerge from psychology holds that innovation tends to emerge from minds that are in key senses naive to the fields they are revolutionizing (Csikszentmihalyi Reference Csikszentmihalyi1996; Simonton Reference Simonton1999). Undergraduates, who are by definition outsiders, can sometimes make those new connections with extraordinary speed, which can lead to fresh insights that might evade a professor thoroughly socialized in conventional wisdom.

An illustrative project involving coauthor Herrick explored how violent media consumption affects an individual’s ethical behavior in a business setting. Herrick joined the project very late, after multiple experiments had already been run—with each round generating contradictory results. One of the first meetings Herrick had with the principal investigators was a strategy session where the research team brainstormed ideas for project improvement. Herrick felt like he was treated as an equal colleague with important insight and ideas to contribute to a study in which two academics had already been invested for some time.

In the midst of this conversation Herrick suggested that they employ a more robust form of media—videos and movie trailers—compared to the prior exclusive use of text and also use the amount of time a participant watched the video as a key outcome measure. After weeks of redesign implementing Herrick’s recommendations, the team developed a final product and ran a successful round of the experiment.

COLLABORATION WITH UNDERGRADUATES VALIDATES RICARDO’S LAW

When professors and students work together, the collaborative nature of the enterprise, as in the example discussed previously, often sparks creativity and productivity in a way that scholars toiling alone struggle to reproduce. Collaboration enables one of the truly revolutionary ideas from social science to work inside the research group: David Ricardo’s Law of Comparative Advantage (Ricardo 1817/1891).

Even if a given faculty member is absolutely better at performing all aspects of a research project, Ricardo’s logic suggests that the scholar is still better off specializing in the research tasks at which he or she relatively excels and then exchanging with undergraduates for the skills for which they in turn have a comparative advantage. Professors can thus produce a greater volume of high-quality research more efficiently, and students can take away valuable preprofessional experience and perhaps even coauthorship on a publication or two. Beyond Ricardo, compelling evidence from neuroscience suggests that multiple people working together can perform mental feats that consistently outshine even the most gifted person in the group working alone (Bahrami Reference Bahrami, Olsen, Latham, Roepstorff, Rees and Frith2010; Dehaene Reference Dehaene2014). Indeed, such collaborations approach the theoretical optimum in their outputs (Bahrami Reference Bahrami, Olsen, Latham, Roepstorff, Rees and Frith2010, 1084).

A set of experiments exploring how nongovernmental organizations (NGOs) might behave opportunistically illustrates this idea and also reinforces the prior point about creativity. In 2011, during the first year of Nielson’s Uganda mentored research abroad program, a team of three undergraduates became frustrated by the many requests for cash they received from new acquaintances.

Turning frustration into research, the undergraduates went on to acquire the official list of all registered Ugandan NGOs and e-mailed 1,400 of the organizations with a tentative, but sincere, invitation to consider a potential research partnership with BYU’s PEDL. As part of the initial assessment of compatibility, all contacted NGOs were invited to estimate personnel costs for the study that PEDL might reimburse if funds later became available. These “bids” became the key outcome measure.

Five different invitations were randomly assigned and subjects were sent a control (a basic invitation) or treatment, which included one of four added notes to the basic invitation. These included statements that (1) all projects are audited; (2) that invitations were going to many NGOs; (3) that PEDL had previous experience with NGO partnerships and personnel costs; and (4) that past PEDL projects had been supported by $3 million from named foundations and international organizations.

The results from the Uganda experiment suggested that the mention of meaningful resources caused a significant increase in opportunism. Bids in the “Big Money” condition were on average double the NGOs’ estimates of personnel costs in the control condition. But ambiguous results in other conditions prompted a follow-up study on 14,000 larger, global NGOs. That experiment yielded results that contradicted the Uganda findings: average estimates for personnel costs did not change among conditions, but the Audit, Competition, and Big-Money conditions did significantly raise the response rate of contacted NGOs, suggesting that, for these more professionalized organizations, those three invitations sent signals of PEDL’s credibility to potential partners. These contradictory findings prompted yet a third experiment on 52,000 NGOs in India. Results suggest that responding Indian NGOs behave more professionally than the Ugandan NGOs: Indian NGOs do not raise their bids at the allure of Big Money and they also drop their bid prices in the face of competition (Bakow et al. Reference Bakow, Brigham, Egbert, Findley, Matthias, Nielson and Reed2014).

The first two experiments necessitated individualized e-mail replies to all responding NGOs. The second experiment involved the extraction of NGO contact information from an online list of more than 25,000 development organizations. One student collaborator was skilled in programing and data management. In just a few hours, he was able to webscrape and populate a database with more than one million individual data points. The task would have been gargantuan and entirely impractical for even a large team of research assistants coding by hand, but it proved relatively simple for an undergraduate using his comparative advantage.

The third experiment on Indian NGOs, executed by Matthias as an additional coauthor, required elaborate programming of the Qualtrics platform involving complicated piping and looping logics, much of which Matthias learned specifically for the project. It also demanded the management of the hundreds of individual replies and requests received from the pool of 52,000 original contacts. In the end, the three experiments involved seven coauthors and required hundreds of pooled hours of the undergraduates’ time. The two professor coauthors on the project, Findley and Nielson, could not have executed the project on their own given their other commitments—or at least not within any reasonable timeframe. Their research grants would not extend to such a novel undertaking, and paid research assistants working on the clock likely would not have felt the investment that the undergraduate coauthors did nor been motivated to take ownership and solve the key research challenges.