What Is It Like to Be Kept in the Decisional Loop?

There is currently a lack of evidence concerning the various impacts of closed-loop system BCIs on patients’ decisionmaking processes, especially how being in the decisional loop impacts patients’ sense of autonomy.^{Footnote 16,Footnote 17} To address the lack of relevant evidence, we conducted in-depth, semistructured interviews^{Footnote 18} using open-ended questions directed at patients who had volunteered to be implanted with the first-in-human experimental advisory brain devices capable of predicting epilepsy seizures.^{Footnote 19} Description of the trial’s details can be found in the 2013 study by Cook et al.^{Footnote 20} As this was a qualitative study based on first-person narrative interviews, the results are presented as excerpts.

Interviews were analyzed by regrouping patients’ subjective experiences into five main phenomenological clusters that reflect patients’ autonomy and decision-making processes: (1) insecurities and risks attached to living with epilepsy; (2) how patients integrate device predictions into their deliberative processes leading up to their relevant decisions; (3) patients’ distrust of the device; (4) device-induced sense of control and empowerment; and (5) Device-induced lack of confidence and sense of control.

In the following sections, these clusters are populated by patients’ key answers and comments.

Analysis of the Data

To understand how being in the decisional loop can affect patients’ decision-making processes and sense of autonomy,^{Footnote 22} it is fundamental to note that individuals with chronic epilepsy, as indicated in Cluster 1, live in a constant state of insecurity because of the possibility of having unpredictable seizures. Many daily and basic decisions by these patients are experienced as challenging because they are impacted by the insecurities and risks attached to living with epilepsy.

Because they are living in a permanent state of uncertainty, it is easy to understand that all patients, pre- and postimplantation, reported being skeptical regarding whether the predictive and advisory functionalities would work. Still, when implanted individuals realized that the advisory functionalities were helping—albeit with varying degrees of perceived effectiveness^{Footnote 23}—they began integrating the device predictions into the deliberative processes leading up to their relevant decisions. For example, Patient 02 declared: “I had very little confidence that it was going to be of any assistance. But then over time, I got more and more confident and so, yeah, I trusted it.” In another instance, Patient 05 started to trust his own biological phenomenology and the device forecast because he realized that specific mental images of Kermit the Frog (auras) associated with device prediction indicated an upcoming epileptic seizure. In the latter case, Patient 05 used the device signal when synchronized with auras as an informational basis on which to initiate therapeutic decisions and choices.

In contrast, in Cluster 3, patients did not experience trust while being in the loop because they had seizures without warning and surmised that their devices were not reliable, leading them to ignore signals. For example, Patient 04 said: “No I wasn’t trusting it.…I just ignore it anyway”; or, as Patient 05 indicated: “Because there were so many false warnings that you never knew what to believe at the time.” From these reports, it seems that for patients to be comfortable in the decisional loop and to integrate the device’s predictions into their decisionmaking, they require a certain amount of trust in the device. It seems that the trust is built on accurate cumulative interactions; the hypothesis being that trust took the place of what these patients lacked in knowledge.

It is difficult to ascribe decisional outcomes to any single cause; however, factoring trust into the prediction while being in the loop appears to substantially affect patients’ deliberations. In some cases, the trust induced a level of certainty that influenced their decisions. Patient 06 said: “With the device I felt like I could do anything—I can do this—I can do everything I want to do.” In some instances, being in the loop allowed patients to diminish decisional uncertainty by notifying them of the signs indicative of a potential upcoming seizure. In the words of Patient 06: “I was more capable of making good decisions—not bad decisions—because there’s been times [without the device] where I’ve made bad decisions.” As a result, “[w]hen the red light came on was when I took the pill, depending on the severity of my symptoms or after I took one or two.” From this perspective, it would seem that Patient 06 experienced an augmented sense of autonomy. In opposition, Patient 03 describes her experience as follows: “[the device] made me feel like I had no control. So I didn’t have control over what I was going to do.” Feelings associated with having no control would seem to indicate a perceived loss of autonomy. As the device “was always beeping and always red,” Patient 03 experienced being in the decisional loop as a malaise, as evidenced by her self-description of feeling “really depressed.”^{Footnote 24,Footnote 25 ,Footnote 26}

It seems that the reliability of the implanted device translated into a reason for adopting these predictions as trustworthy evidence. As evidence accumulated, most patients not experiencing false warning signals gradually stopped doubting the accuracy of the device and instead just followed the machine’s predictions and advice. The net effect of this seems to have allowed them to confidently enjoy their daily life activities without the uncertainties of when they might become symptomatic and have a seizure. These effects directly impacted patients’ sense of their own autonomy. According to these accounts, being in the decisional loop seems to have enabled some patients to take their decisions beyond the preimplanted repertoire. As Patient 2 testifies: “[the device allow me to] do things that I wouldn’t necessarily and normally do”; or, as Patient 1 put it, “I felt more in control when I used the device. I could push on” and “do what I wanted to do.”.

The abovementioned data displayed in our cluster as obtained from our interviews are evidence that being in the loop can have a radical and profound influence on how patients retain some sense of autonomy over decisionmaking. Our analysis of the data leads us to advance the following conclusions.

1. 1) Being in the loop may partly increase a sense of autonomy over decisionmaking.

2. 2) Being in the loop may partly decrease a sense of autonomy over decisionmaking.

3. 3) Being in the loop may not impact a sense of autonomy over decision-making.

Conclusions 1 and 2 teach us that being in the loop may partly impact a patient’s sense of autonomy over decisionmaking, which raises many ethical concerns. These ethical concerns translate into asking what are the risks of iatrogenic harms involved with being in the loop? For example, Conclusion 2 is primarily explained by a malaise stemming from an experience of loss of control over decisionmaking. For Patient 03, being in the loop made her feel as if she did not have control over what she was going to do. In that respect, the procedure impaired this patient’s postoperative sense of autonomy, which translated into her experience of iatrogenic harms related to feelings of depression. In some contexts, the risk of having an agent lose control raises questions about how a patient can give a genuine informed consent to an intervention that may not offer the prospect of choosing to control oneself in the future.^{Footnote 27}

However, for Conclusion 1, although on first approximation boosting a sense of autonomy seems to confer a direct benefit, augmented sense of autonomy may also contain risk of harms that are not as explicit as appear in Conclusion 2. How can retaining some autonomy over decisionmaking become harmful for a patient’s decisional autonomy (even if it does not strictly speaking remove choice from the agent)? The rest of the article addresses risks and ethical concerns related to Conclusion 1.

Being in the Loop: Ethical Issues Around Retaining Some Autonomy Over Decisionmaking?

While patients are being kept in the decisional loop, and facing uncertainties about seizures, if the device can provide a rare glimpse of accurate information with relevant predictions, the device (over time) will provide the only relevant information that is seen as trustworthy (Patient 06: “The device took all of that insecurity away because now I’ve got to trust myself with that…I was more capable of making good decisions.…When the red light came on was when I took the pill, depending on the severity of my symptoms or after I took one or two”). It appears the patients are willing to rely on device prediction as long as interactions are phenomenally experienced as trustworthy. If we look at how decisionmaking is influenced by the device, then we understand that being in the loop plays a critical role in patients’ deliberative psychology. The effect of this trust increases the influence of, and dependence on, the device. Most often this means that the implanted individual will act and decide with an increased sense of autonomy and control (Patient 01 explained: “I felt more in control when I used the device. I could push on and do what I wanted to do”). Here, we will put further attention on the point that experiencing an augmented sense of autonomy may lead to harm.

By providing patients with greater autonomy over relevant decision-making, the experience of being in the loop also includes the choice not to act according to device prediction. For example, when the device offers options, the patient appears to be an autonomous contributor to the causal pathway that leads to a decision to take, or not to take, the antiseizure medication. It would be different if the advisory functions shifted to automatic medication delivery, taking the subject out of the decisional loop entirely, leaving no choice or opportunity for the patient to autonomously contribute to therapeutic responses. By being kept in the decisional loop, not only are patients able to retain some degree of autonomy, but also they report being “more capable of making good decisions”(Patient 06). Giving control to patients over therapeutic interventions by allowing them into the decisional loop suggests that patients may appreciate what is to their benefit. This is precisely where an important ethical concern appears: by providing assistive guidance (in the agent’s choice to act or not) a closed-loop advisory device may become a device that decreases decisional autonomy (even if it does not strictly speaking remove choice from the agent).

In 2016, Eran Klein et al, 2016, while discussing closed-loop BCIs, suggest that “if a patient is given control over device settings, the temptation to increase stimulation settings to feel increasingly may be difficult to resist.”^{Footnote 28} Klein and colleagues’ concerns about addiction with respect to closed-loop devices further support results that we obtained in 2015.^{Footnote 29} In our preliminary data—obtained after interviewing the first patients to be implanted with advisory DBS devices—we highlighted that some implanted patients may be at risk of over-reliance on advisory devices. We argued that this translates into decisional vulnerability when patients are faced with forming a decision to follow the device information.^{Footnote 30} Decisional vulnerability occurs in a context of epistemic dependence, in particular when patients outsource their deliberative capacities to device instructions despite the absence of immediate evidence. “Technological outsourcing” can be defined as the practice whereby people rely on their computers, smart phones, or other electronic devices to perform certain day-to-day tasks that they would otherwise have to perform themselves. ^{Footnote 31} Here, however, it is not the case that patients could otherwise decide for themselves: they rely entirely on the BCI device as their sole source of information to forecast their seizures.

All things being considered, if implanted individuals are not in a position to base their decision on any other relevant and reliable available evidence, then they may not have any other choice but to outsource that decision to an advisory system. This is precisely a context in which the patient may be said to be facing epistemic dependency. The concern here is that such a case of decisional vulnerability compounded by epistemic dependency puts the patient in a precarious deliberative position. In practical terms, this makes it very hard for the patient to resist undue external influences,^{Footnote 32} and that patient will not be in a position to make decisions free of the control of some confounding influences.^{Footnote 33,Footnote 34} By predicting upcoming neurological events, devices may have substantial epistemic influence that propels patients to initiate a particular decision.

This situation is not at all unusual. People frequently become over-reliant on sources of information that make and keep them dependent. Doing so makes them vulnerable and unable to resist certain influences. The potential medical consequences are what makes this situation ethically alarming to us: not following the instructions as provided by the BCI device increases the risk of the patient having epileptic seizures, and the fear of experiencing such consequences may increase dosage intake (Patient 06).

We have identified this risk of decisional vulnerability and the risk of over-reliance on the device to highlight the risk of decreasing relevant patients’ capacities to make freely informed choices on how to proceed with the advice received. This hypothesis is in line with our earlier observations that implanted individuals may start overtrusting, then progressively over-relying on the advisory system while being in the decisional loop. Consequently, over-relying on device information simply means that in some cases the patient will no longer be sovereign in the decisional loop. The ethical problem with over-reliance is that the device ends up supplanting agency rather than supplementing it.

Over-reliance on advisory implantable brain devices may entail the risk of a false sense of security for some patients. There is not yet any published research on this topic, but, to take a related example, studies have observed that devices that can continuously inform and guide an individual, such as global positioning system (GPS) devices, may lull users into a false sense of security, with the effect being that individuals neglect other stimuli that may guide them just as well.^{Footnote 35} Our findings as taken from patients’ reports show that individuals sometimes push their limits when they trust DBS advisory devices, as evidenced by the following patients’ comments:

If some patients are over-reliant on these devices, then the central concern likely is not whether over-reliance is ethically wrong, but rather whether over-reliance is justified or helpful. Over-reliance is particularly problematic when market forces might be influencing treatment.^{Footnote 36} For example, if a company offers patients free neuronal drug delivery systems, but asks those patients to pay for medication, and then suddenly increase the price of the medication, what then? Here the question would be whether over-reliance on this particular drug is necessary.

Applied to the case of epilepsy, over-reliance on DBS advisory devices may mean that an implanted patient may stop trying to look at other sources of available information to guide responsive decisionmaking. In saying this, we need to ask what are the other sources of information that would be relevant, and how could implanted individuals confirm (or disconfirm) the reliability of such information? For example, should individuals pay more attention to their own subjective experiences, their instincts of a seizure about to happen? How reliable can such instincts be? The reason why patients are implanted in the first place is because they cannot reliably detect or control epileptic symptoms leading up to a seizure. Given the lack of other relevant and reliable available evidence on which to base responsive decisions, there is a strong chance that affected individuals at risk for seizures will come to the conclusion that they should follow the instructions of the advisory device. In that respect, being over-reliant on the device might be justified for some patients when facing some circumstances. Adequate predictive and advisory settings might be a matter of individual preference.^{Footnote 37}