There is always a well-known solution to every human problem—neat, plausible, and wrong.
(Mencken Reference Mencken1920, 158)1. Introduction
The influence of industry funding on the reliability of scientific research has reached a crisis point (Greenhalgh, Howick, and Maskrey Reference Greenhalgh, Howick and Maskrey2014). Analyses have traced hundreds of thousands of deaths to industry manipulation (Biddle Reference Biddle2007; Holman Reference Holman2017), medical journal editors fear they “have devolved into information laundering operations for the pharmaceutical industry” (Horton Reference Horton2004, 8), and even proponents of evidence-based medicine (EBM) worry that although “EBM and its major tools, randomized trials and meta-analyses, have become highly respected, the EBM movement has been hijacked [by the pharmaceutical industry]” (Ioannidis Reference Ioannidis2016, 83). The front lines of research have moved to cataloging industry tactics, their subsequent distortion of the literature, and proposed remedies.
There is especially no shortage of proposed remedies.Footnote 1 It is a standard editorial demand for authors who have spent hundreds of pages identifying the means and extent of industry bias to finish their work with a chapter on proposed solutions. However, few authors invest as much research and careful thinking in proposing solutions as they do to exposing problems. Even for those who do, it is not clear what separates the potentially effective from the neat, plausible, and wrong.
The current article aims to advance the discussion via a framework for policy evaluation in medical epistemology. Specifically, we proceed from the supposition that medical epistemology is best seen as an asymmetric arms race, a dynamic typified by a series of moves and countermoves between competing parties who are adjusting to one another’s behavior (Holman Reference Holman2015). Here we draw on this view of medical epistemology as a tool to generate and evaluate proposed solutions to manage the influence of industry funding.
We consider as an example of such solutions the Food and Drug Administration’s (FDA’s) patient-focused drug development meetings (Patient Meetings, hereafter), a recent initiative to ensure that approval requirements do not grow detached from patient needs. It is also the type of collaborative approach philosophers have characterized as the “sort of win-win solution [that] deserves much more attention in the future” (Elliott Reference Elliott2014, 935; see also Douglas Reference Douglas, Maasen and Weingart2005; Elliott Reference Elliott2016). Similarly, the format of an open forum might be thought of as an instantiation of Longino’s (Reference Longino2002) ideal epistemic community. Indeed, Jukola (Reference Jukola2017) has argued that Longino’s account is particularly useful in evaluating industry-funded medical research.
In section 2, we provide the background of a failed drug for female sexual dysfunction (FSD) that ultimately gained FDA approval as a result of patient involvement in regulatory decisions. In section 3, we analyze the Patient Meeting. By analyzing meeting transcripts, we show that industry-affiliated participants presented a unified message almost completely distinct from (and often in direct opposition to) participants without conflicts of interest. In section 4, we argue that because industry-funded participants dominated the discussion, the official FDA report was distorted in ways that ultimately contributed to drug approval.
While corruption of this particular process is important, there are more general morals for epistemology. We use this example and the arms race framework to draw three. First, policy solutions should incorporate ongoing assessments of reliability. Second, reformers typically search for policies that reliably address current problems; however, policies must also be evaluated on their robustness to manipulation. Finally, the best solutions are often not policies that merely counter an opponent’s measure but rather general solutions that restructure the incentives driving the interaction.
2. “Evening the Score,” or Putting a Thumb on the Scale
In August 2015, after two failed applications, Addyi (flibanserin) became the first drug approved by the FDA for hypoactive sexual desire disorder in women. While numerous methodological and conceptual issues are at play, of central interest here is the role that patient input had in reversing earlier decisions. In this section, we provide a brief regulatory history to contextualize the Patient Meeting.
In 2010, the first application was rejected shortly after a unanimous vote by the Bone, Reproductive and Urologic Drugs Advisory Committee that flibanserin failed to demonstrate a positive risk/benefit ratio in two separate trials (FDA 2010, 8). In particular, flibanserin failed to increase desire as recorded in an eDiary, a measure of efficacy agreed on before the trials (7). However, the application also included an alternative measure of sexual desire from the Female Sexual Function Index (FSFI). The FSFI showed a small but statistically significant improvement and was, the manufacturer claimed, a better measure of desire. By 9–2 votes, the advisory committee concluded both that it was inappropriate to deviate from the prespecified measurement and that, even if it were not, the eDiary was a superior measure of desire (7).
In consultation with the FDA, Sprout Pharmaceuticals conducted a third trial prespecifying the FSFI measure. Despite a statistically significant increase in desire (FSFI), the review from FDA statisticians was equivocal: “From a statistical perspective, the efficacy of flibanserin has been demonstrated, but the clinical meaningfulness of these results should be considered with respect to [the] clinical utility of such a small treatment benefit over the safety profile of this product” (FDA Reference Elliott2013b, 15). Ultimately, the Office of Drug Evaluation was “not convinced that the treatment benefits observed with flibanserin outweigh the identified substantial safety concerns” (FDA Reference Elliott2013a, 1).
The third attempt for FDA approval included only a minimal amount of additional safety data. According to the medical review, what little additional information was submitted “fails to resolve the concern” regarding the validity of the FSFI, and its use “continues to raise questions concerning the magnitude of the treatment effect observed with flibanserin” (FDA Reference Elliott2015a, 22–23). The medical reviewers, statistical reviewers, clinical pharmacology reviewers, and the team leader in charge of the application all concluded that there was an overall negative risk/benefit balance and recommended against approving flibanserin. Despite the general agreement among reviewers that flibanserin was unsafe, the external advisory committee recommended approval.
The division director noted that there was “internal agreement on the facts, but not on whether the demonstrated benefits outweigh the known risks” (FDA Reference Elliott2015b, 23); ultimately, he sided with the advisory committee, overruled the reviewers, and issued approval.Footnote 2 What primarily drove the FDA’s reversal was a reevaluation of the relative importance of the risks and benefits. The next section discusses the primary source of information for this reevaluation—the Patient Meeting.
3. The Patient’s Voice
As part of the renewal of the Prescription Drug User Fee Act, the FDA committed to hold 20 Patient Meetings to improve risk-benefit assessment. The FDA saw these meetings as especially important “when the impact of a disease on patients is not well understood or endpoints for studying drugs for a disease are not clearly defined or established” (78 Fed. Reg. 21614 [April 11, 2013]). The schedule, purpose, and scope of meetings (e.g., background material and discussion topics) were published in the Federal Register. Patients were asked to preregister for the meeting and indicate whether they would be interested in sitting on a panel that would initiate discussion. Volunteers submitted written responses to discussion questions, and the FDA chose panelists in order to accommodate various stakeholders. An open discussion followed among those in attendance, moderated by the FDA.
3.1. Method
To analyze the Patient Meeting transcripts (FDA Reference Elliott2014), one author (B.H.) identified participants with conflicts of interest based on verbal declarations and further investigation. Many participants disclosed that their travel was paid for by the pharmaceutical company or declared that they had no such conflicts. The exact nature of the compensation is unstated, although a degree of organization is suggested by the fact that roughly 40 industry-affiliated women arrived together in a chartered bus sporting teal scarfs and buttons that promoted the “women deserve” campaign (Hicks Reference Hicks2014). In some cases, women made no declaration yet stated their full name. In two of these cases (Sue* and Sheryl*) we were able to identify significant ongoing funding relationships with Sprout Pharmaceuticals by examining conflict of interest declarations on published articles.Footnote 3 Another participant indicated no conflicts, but we established that she owned a company involved in Sprout’s public relations campaign. Accordingly, all three were included in the industry-affiliated group (19 of 33 speakers). Of the remaining 14, half declared no conflicts of interest, and half were undeclared and no further determination could be made.
We then used grounded theory methods to analyze transcripts from the Patient Meeting. One author (B.H.) conducted open and focused coding and developed several themes based on these codes. To ensure the results were not the product of unconscious bias, he then redacted all attendees’ declarations of conflicts of interest, and the other author (S.G.) conducted blinded qualitative coding with no information regarding the larger context of the meeting or industry affiliations. After completing coding, she was unblinded, and we compared theme expression by industry-affiliated and unaffiliated participants. Disagreements were rare, but when they occurred, deference was given to the second author (S.G.).
3.2. The Shared Experience
Our analysis shows that all women shared a set of core experiences. However, there emerged two fundamentally different ways of understanding causes, implications, and reaction to the shared experience of a loss of sexual desire. Moreover, these differences in understanding are perfectly correlated with whether a participant had an affiliation with Sprout Pharmaceuticals.Footnote 4
The shared core experience was often a “total lack of sexual desire” (Karen). For many, the physical ability to have sex was not a problem, in part because as some participants suggested, this could be facilitated by over-the-counter lubricants (Amanda* and Louanne*). But the subjective experience of desire was missing before and even during sex. “It’s not an issue of not being able to have sex, what I want is to want it. … I want to always desire my husband” (Amanda*). Several participants expressed secondary psychological impacts. Judith* stated she felt like “less of a woman as I no longer had sexual appetite.”
Lack of sexual desire precipitated situations in which women obligingly acquiesced to their partner’s advances. For Barbara*, “on the rare occasion that we did have sex it was done out of obligation rather than desire.” However, “duty sex” was unsatisfying for both parties. When asked to describe a pleasurable sexual experience, Kelly replied “that when it ends your husband doesn’t automatically think it was out of obligation.” The desire to initiate sex “is a wonderful blessing [for my husband] because he knows that it is not duty sex but that I actually want him” (Carmon*).
The impacts on romantic relationships were just part of the picture. “It affects the male [partner] and their mental state which affects your relationship, how you deal with your kids and how you deal with everybody else and how you think of yourself” (Kelly). For Meg*, “it affects all areas of my life, including my work, how well I can perform. I get the brain fog, I start crying.” Another shared that “I run a small business. I have to get up every day and go out and interact with people at high levels in the community. I almost lost the ability to do that” (Beverly*).
Some women experienced gradual loss of desire and arousal: “It began with subtle physical changes. The ability to be stimulated by being touched slowly disappeared. Sexual arousal and response time kept taking longer and longer until it became nonexistent” (Carol*). For others, like Karen, an immediate change occurred after a catalytic event such as a hysterectomy. Both types of onset were described by women independent of industry affiliation. However, numerous industry-affiliated women described a sudden loss of sexual desire without a catalyzing event, all using the same phrase “like a switch flipped” (Sue*, Katherine*, Natalie*).
3.3. Divergent Narratives
Despite these similarities, deep divisions emerged between women with and without industry affiliation. Most significantly, the former understood their symptoms as a biological disorder, while the latter considered it a relational one. Participants with industry affiliation discussed their experience as one of being “betrayed” (Sue*) by their body, that their “body has let [them] down” (Katherine*). “This disorder is real” Barbara* asserted. Beverly* agreed: “This is a severe medical condition … and no amount of talk therapy is going to fix it.” These participants described detecting drug effects, “for sure there is a direct correlation” (Beverly*). For Natalie* the change was as immediate as the onset; “literally a week or two after I was treated, everything changed and I was fine again.”
By contrast, participants without industry affiliation understood sexual problems as relationship problems. “Our sex problems were a co-created problem. … I wasn’t the problem” (Susan). They discussed their body, not as having betrayed them but in terms of “accepting the reality of my age and past challenges” (Karen) or in terms of physical differences with their partner “that are no longer threatening … accepting our sexual differences has been part of the whole change” (Susan). Moreover, when treatments were discussed, talk therapy dominated. “Instead of looking for something outside the relationship for help, we go to what is going on in our life right now that could be affecting us” (Susan). Others emphasized communication with partners “about what pleasure they can have and how they can give it to each other” (Karen) as a preventative measure. What experiences they had with drugs were negative—“I didn’t see any effect whatsoever” (Kelly)—whereas focusing on the relationship was transformative: “our kiss became connected and deep. My orgasm came back with a quality I hadn’t experienced before” (Susan).
Interestingly, both groups of women expressed a feminist critique of FSD, but its expression again varied depending on industry affiliation. While several women described the influence of “unrealistic [cultural] expectations” (Karen) of female sexuality, not a single one had industry affiliation. Karen elaborated, stating that “I, and many other women, young or old are not ever going to achieve the mind blowing nirvana of orgasmic ecstasy that saturates our popular culture.” Some women explicitly linked these expectations with their symptoms. “We would bring these [stereotypical] ideas to the sexual relationship without verbalizing them, thus creating distance and disconnection” (Susan).
In contrast, women affiliated with the industry accused the FDA of sexism: “The thing that makes me most angry and most disappointed is that if I went to my doctor and I was a man they would be able to write me a prescription that is insurance-covered and FDA-approved” (Beverly*). Many explained that if they were a man they “would have many options for treatment” (Victoria*) and implored “the FDA to approve a treatment so women, like men, can have a solution to their most common form of sexual dysfunction” (Barbara*).
Ultimately, our analysis indicates that industry affiliates characterized the issue as biological and, consistent with this interpretation, described their success with pharmaceutical solutions. Furthermore, affiliates were silent on the issue of cultural pressures and were alone in their insistence that the FDA “even the score” by approving flibanserin. By contrast, the unaffiliated participants understood their symptoms as a natural part of aging or as a reflection of relationship or life stressors and described success with talk therapy. The critiques of both groups centered on fairness; however, while unaffiliated participants criticized unrealistic cultural standards for women, industry affiliates demanded pharmaceutical equality.
4. Which Voices the FDA Heard
Although tracking industry affiliation reveals deep divisions in women’s reports, these differences were not noted by the FDA. In this section we explore how the Patient Meeting was summarized by the FDA and how it figured into the advisory committee decision. At each step, our primary interest is which voices remain as a full day of discussion is progressively distilled into a few sentences.
4.1. Patient Meeting Summary
The first condensing of the day was the 24-page summary document produced by the FDA entitled “The Voice of the Patient” (FDA Reference Elliott2015e). Overall, industry affiliates dominated the meeting, in terms of both proportion of participants (58%) and speaking time (73%). While this disparity likely influenced the conclusions the FDA drew from the meeting, it was not exacerbated by the immediate summary process. The FDA used quotes from industry-affiliated women proportional (76%) to their speaking time. However, the only hint that conflicts of interest existed was a brief statement that “some participants voluntarily disclosed that their travel to the meeting was funded” (4). There is no mention of who funded their travel or the fact that most women made such disclosures. No further attempt was made to distinguish women by affiliation.
Given their interest in regulatory issues, the FDA provides a fuller treatment of patient’s thoughts on clinical endpoints and various treatments than we have, and rightly so. With respect to the themes described above, it identifies that women find their lack of desire distressing (FDA Reference Elliott2015e, 5, 7), that onset varies (8), that women often had sex out of obligation (9), and that low desire affected their relationship (9). It also acknowledges disagreement over whether the problem was biological or relational (8), and although it did not explicitly contrast them, discussed both women who felt that their body had betrayed them and women who accepted the change (12). Indeed, the only major themes not centrally addressed were the two feminist critiques of FSD. Nevertheless, we claim that the FDA’s summary mischaracterizes the discourse at the meeting in consequential ways.
Ignoring industry affiliations obscures a simple fact: there are two coherent and largely dissociable sets of experiences reported at the meeting, one by women affiliated with the pharmaceutical industry and another by women without such ties. The effect of ignoring industry affiliation is to frame women’s experience as consisting of a predominant expression with some minor variation. For example, the FDA summary claims that most women want a treatment that consistently increases desire, although some participants cautioned that it was important to pay attention to “the cultural aspects of sex” (FDA Reference Elliott2015e13). As we attempt to illustrate in table 1, the framing of what is essentially the same fact can fundamentally alter its implication.
Comparative Analysis
| FDA Analysis | Our Analysis |
|---|---|
| “Some participants believe that given the significant cultural component of sex and sexuality, interest and arousal effects can and should be primarily addressed through counseling and other behavioral therapies. However, many participants indicated that they see a need to determine and address underlying physiological causes or contributors to their interest and arousal symptoms” (FDA Reference Elliott2015e, 6). | Women without conflicts of interest believed that given the significant cultural component of sex and sexuality, interest and arousal effects can and should be primarily addressed through counselling and other behavioral therapies. However, participants who received funding from the pharmaceutical industry indicated that they see a need to determine and address underlying physiological causes or contributors to their interest and arousal symptoms. |
4.2. Advisory Committee
The advisory committee is the first stage at which the Patient Meeting affected regulatory decision makers. Here we look at the information provided to the advisory committee. The first is a preparatory briefing document (FDA Reference Elliott2015c). It contains an executive summary (16 pages) and a full report supported by a series of appendixes (289 pages). Afterward, we consider the presentations to the committee (FDA Reference Elliott2015d).
In the executive summary, the only mention of the Patient Meeting is a report that women conveyed “a willingness to risk serious (and often unknown) adverse effects and even to undergo periodic minor surgery with its related risk of serious infection in order to obtain [FSD] relief” (FDA Reference Elliott2015c, xvi).Footnote 5 In the full report, the meeting is first summarized in two sentences relaying that women experience FSD as a condition that significantly affects their self-esteem, personal relationships, and overall quality of life (13). The ensuing paragraph notes that several unproven therapies are being employed, including psychotherapy, which underscores the need for a safe and effective medication.
At subsequent points, the Patient Meeting is referred to in ways that clearly incline toward approval. For example, the brief later claims that women “seek movement toward more satisfying sexual health that is more in line with their previous personal experience of unimpaired sexual function” (FDA Reference Elliott2015c, 48). Similarly, the brief suggests that comments of participants were consistent with the claim that “the effects of flibanserin [are] clinically meaningful and important to their condition” (92). The voice of women without conflicts of interest has now been silenced, even as a dissenting minority.Footnote 6
This entire process climaxes in the live presentations to the advisory committee in which first industry, then the FDA reviewers present their case. We suggest that portions of the former can best be described as “corporate ventriloquism” (Bsumek et al. Reference Bsumek, Schneider, Schwarze, Peeples, Depoe and Peeples2014). Now, rather than speaking for themselves, industry representatives deferred to the voice of the patient. It was patients who demanded a treatment to regain the sexual desire they used to have and patients who said the benefits they experienced with flibanserin were meaningful: “They told us that … in this very room at the October 2014 patient-focused drug development meeting hosted by the FDA” (FDA Reference Elliott2015d, 97). Indeed they did. And many of them were there again to retell their story during the “public comment” phase.
5. Lessons Learned
The episode above is not just an illustration of how public participation in industry-funded science can fail to be a win-win solution for managing bias. In closing, we briefly describe dynamics of asymmetric arms races and identify broader epistemological implications. In so doing, we show how the arms race account can serve as a framework for thinking through policies in medical epistemology and other domains where the profit motive threatens the production of reliable knowledge.
As described in previous work (Holman Reference Holman2015), asymmetric arms races occur when two groups have mutually exclusive and conflicting goals but are confined to use different strategy sets to achieve their aims (e.g., the evolution of castle defenses and siege weapons). This dynamic is marked by a series of measures and countermeasures as each side innovates or responds to the innovations of the other side.
Holman (Reference Holman2015) argues that asymmetric arms races are a general class of strategic situations that exhibit the following features: “(1) the reliability of any strategy (once it is employed) typically decreases over time; this is because both (2) opponent responses often attenuate the efficacy of one’s strategy, and (3) opponents engage in a search process to identify and exploit weaknesses; however, (4) because measures are costly it is often disadvantageous to adopt new strategies until they are necessitated by an opponent; and (5) the process results in the gradual accumulation of costly measures” (60). He also establishes that medical epistemology exhibits these five features through an examination of the history of medical regulation. In particular, Holman argues that the rise of sophisticated methodology (e.g., randomized controlled trials) implemented by medical reformers has been in response to pharmaceutical manufacturers’ practices intended to increase profits. In addition to resisting such reforms, the primary strategies of manufacturers have been increasingly sophisticated marketing techniques and “attempts to capture and subvert influential levers of power if reform is implemented” (171).
Although the episode described above is only a snapshot, it illustrates a portion of this dynamic. The ability of Sprout Pharmaceuticals to stack the Patient Meeting with preselected participants represents a clear example of attenuating the efficacy of reform by capturing and subverting an influential lever of power. In closing we argue that adopting the arms race framework suggests a reorientation for thinking about science policy; in particular, we draw three morals that are especially salient in the case discussed above.
Philosophers who focus on the effects of industry funding have suggested on the basis of a handful of successful examples that citizen collaboration in research is a promising corrective to the problems of industry-funded science (Douglas Reference Douglas, Maasen and Weingart2005; Elliott Reference Elliott2014, Reference Elliott2016). The first moral of the asymmetric arms race framework is that solutions are rarely durable. Even if it could be shown that such interventions had a reliable track record in the present environment, arms races are necessarily dynamic and changing.
In the case of Patient Meetings, the FDA’s goal was to ensure that measures of drug efficacy and risk remained relevant to the patient population. While previous meetings may have been successful, it is clear that Sprout Pharmaceuticals successfully captured this one. Moreover, it is reasonable to expect that given this success, future meetings will be similarly manipulated. Such a situation underscores the need to regard policies as situated responses to achieve certain aims. Given that one is engaged in an asymmetric arms race, attempts to undermine successful strategies should be expected. Accordingly, policies should incorporate ongoing reliability assessments that signal when current measures must be abandoned or supplemented. In this case, if public participation is to serve its intended purpose, then the FDA must employ further countermeasures to prevent industry manipulation.
The second moral is that policies must do more than address current problems. For example, upon recognition that pharmaceutical manufacturers manipulate meetings, it might be suggested that the FDA prohibit participants with conflicts of interest or at least examine whether patients’ experiences differ on the basis of conflicts of interest as we have done here. One might suggest that such analysis is a reliable way to sort out industry influence and an effective countermeasure to stacking the meeting with industry-friendly participants. We certainly think that with a fair degree of reliability we have disentangled the voice of industry from other participants in the meeting. However, the arms race framework suggests that assessment of a measure’s reliability (i.e., policy efficacy given the present situation) is a necessary but not sufficient component of effective policy evaluation.
The arms race framework situates each action in a dynamic and adversarial context. Accordingly, not only must a solution be reliable, it must be robust to foreseeable countermeasures. In military arms races, such as the asymmetric war in Iraq between the United States and insurgents, the army employs a “red team” trained to think like insurgents. Before any technology gets fielded, the red team first ensures it cannot be easily circumvented. As a result, numerous effective (but fragile) technologies are abandoned before they fail on the battlefield (Holman Reference Holman2015).
Policy solutions are rarely evaluated for robustness, and authors typically do not anticipate what actions might counter their policy proposals. The arms race framework suggests that such analysis would dramatically increase the likelihood of implementing robustly successful policies. Patient Meetings (in their current form) may not have been implemented had the FDA red teamed the proposal as the potential for capture by pharmaceutical companies was foreseeable in light of their increased interactions with patient groups (Buttle and Boldrini Reference Buttle and Boldrini2001; Herxheimer Reference Herxheimer2003; Mintzes Reference Mintzes2007). Similarly, both solutions proposed above (i.e., barring participants with conflicts of interest or coding their input separately) could be quickly circumvented by pharmaceutical companies obscuring the source of funding.Footnote 7
The final moral is that the best solutions do not merely counter an opponent’s measure; instead they align the incentives driving the interaction. As long as epistemic and financial goals run at cross-purposes, there will be continued pressure to undermine reform. For example, one of the central dividing lines in the debates about FSD has been the extent to which it is properly classified as a disease. This dividing line was reflected in the reports of women in the Patient Meeting. All women were in agreement that a lack of sexual desire was a seriously distressing situation, but they disagreed about whether it should be conceptualized as a medical disease.
Again, a full exploration of this issue is beyond the scope of the article, but a fruitful place to start would involve considering current regulatory policy that requires that for a drug to be approved it must meet a recognized medical need—merely improving one’s life is insufficient for a substance to be approved by the FDA. As a result, if there is an FDA approved treatment, there must be an FDA approved disease. This has been one of the driving forces of medicalization generally (Moynihan and Cassels Reference Moynihan and Cassels2006) and of the medicalization of female sexual problems in particular (Fishman Reference Fishman2003, Reference Fishman2004).
These are but three morals that can be drawn from the episode; nevertheless, they suffice for our central claim: the arms race model provides a framework for beginning to think through science policy solutions in the context of industry funding. Douglas (Reference Douglas, Maasen and Weingart2005) and Elliott (Reference Elliott2014, Reference Elliott2016) may be right that involving citizen stakeholders offers a promising way to address issues related to industry bias. Longino (Reference Longino2002) supposes that in the ideal community an egalitarian ethos is crucial for producing objective knowledge. However, at a minimum, given the ability of industry to marshal resources, it is clear that giving each participant an equal weight allows industry to crowd out the opposition (cf. Fernandez Pinto Reference Fernandez Pinto2014). As the above example shows, if such policies create venues that have substantive effects, they become prime targets for corruption. One sure lesson of the arms race framework is that it is best not to fashion a new lever of power unless you are relatively confident it will be the right hand at the controls.
