Doris builds his framework largely on the foundation of social psychology. He generally does not address the cognitive underpinnings of social psychology, on which the collaborative dialog is built and through which it can easily be exploited. Several well-established cognitive principles enhance and extend his idea, but are also notoriously used by advertisers, politicians, and other hostile actors. His framework is largely correct, but it is wrong to assume that everyone in the “dialog of self” is playing nice. In fact, many interlocutors are using cognitive principles to shape narratives and change values (and even selves) in surreptitious ways that may not result in the clarity that he envisions. The dialog itself will not naturally favor truth over falsehood, but it can be saved.
We would propose that the highest ethical standard in his system ought to be the process of increasing metacognition – the ability to self-regulate (to beat defeaters) and to avoid biases and hostile narratives (or meta-defeaters, if you will). Because metacognition is a skill and a tool, it is insufficient to simply say that “when defeaters obtain, the exercise of agency does not obtain” (p. 10). It is through metacognition that one can become better at recognizing and defeating defeaters. And it is doubly insufficient because by intentionally playing on human decision-making biases (e.g., heuristics, selective attention, hindsight bias, the first-instinct fallacy, etc.), hostile interlocutors can introduce a whole class of meta-defeaters that are engineered to surreptitiously change narratives, and thus values, and thus selves.
For example, suppose Candidate Creepy reads Simons and Chabris (Reference Simons and Chabris1999) and learns about selective attention. He would discover that while focused intently on one event, humans ignore or do not notice things that would normally be salient. (In the original experiment, many participants counting the number of basketball passes in a busy environment did not notice a man in a gorilla suit walk into the middle of the scene, beat his chest, and walk off.) Thus, when journalists publish an in-depth investigative article about his indebtedness to foreign entities and business conflicts around the world, the Candidate might falsely but loudly declare that millions of people voted illegally in the recent election. Now the “collaborative” conversation focuses entirely on whether that claim is true, how he could possibly believe that, how others might believe it, etc. Temporarily interesting but ultimately meaningless content has monopolized the conversation. This will be internalized and go into forming the beliefs and values and selves of society, not the objectively more important and true information.
Suppose the Candidate claims he will build a giant dome over the country to protect us from aliens, and that the aliens will pay for it. The conversation ought to be about the merits of a dome, but instead we naturally anchor (Tversky & Kahneman Reference Tversky and Kahneman1974) to the outrageous idea of aliens paying for our dome, and by comparison the actual building of the dome seems relatively sane and doable. Also, the only way to selectively attend to the payment discussion forces us to accept, as a premise, that the dome will be built. Even attempting to move the collaborative negotiation (p. 35) back to the dome itself is impossible because the main objection – the cost – is subverted by the baseless claim that it will be covered by aliens. You can't talk about the cost without including that aliens will pay for it, and you can't talk about aliens paying for it without accepting that it will be built. Again, the narrative is hijacked and the beliefs, values, and selves that will form out of it will be based on misinformation.
Of course, dialog is even more vulnerable when we enter the fragile realm of human memory. Candidate Creepy might use hindsight bias (Fischhoff Reference Fischhoff2007), anecdotal evidence, confirmation bias (Wason Reference Wason1960), or confuse correlation and causation. All of these effects just happen to play into the false claim that vaccines cause autism. How do you combat this falsehood? The natural reaction might be dialogic. Educate people to correct the myth! Unfortunately, Schwarz et al. (Reference Schwarz, Sanna, Skurnik and Yoon2007) describe how this leads to the backfire effect. Even if people are initially converted to the truth, many will end up continuing to accept the myth and will even believe it more strongly after several weeks have passed, not because of any surreptitious motives on their part, but because that is just how memory works. And the end result is that people believe vaccines cause autism, believe evidence supports this “fact,” and form values and selves to match this belief. They might even go further and believe, again as core values and expressions of their selves, that scientists and journalists are misguided or evil for contradicting this “fact.”
These three examples (there are hundreds more) might be called meta-defeaters, or processes by which defeaters are created. They all prey on implicit processing, in which the people experiencing these effects would not be able to say to themselves “I am suffering from this psychological phenomenon engineered by Candidate Creepy.” Instead they would believe, as a reflection of their values, that the Candidate was correct. But those beliefs were engineered explicitly and inserted into the dialog in order to change people into selves with values counter to the truth. Doris (sect. 2, para. 7) contends that we can rule out defeaters, and agency is exercised, if the action reflects the values of the actor. But what if your values are the result of meta-defeaters? Can a person be blamed for any of the “mistakes” above, given that each one would be a reflection of their unknowingly hijacked values?
This is not to say that Doris's dialog always leads to falsehood, but rather that it could equally lead to clarity or muddiness or even strengthened belief in falsehood unless some process is in place to guide it. Two brief examples will help illustrate this. Consider the primacy and recency effects (Ebbinghaus Reference Ebbinghaus1913), where people tend to remember the beginning and end of a dialog better than the middle. Managers use this effectively by providing positive feedback to employees at the beginning and end of a meeting and placing criticism in the middle. The employee will leave happy but will have a list of things to improve. Wonderful collaboration. More surreptitiously, advertisers place positive aspects of a drug at the beginning and end of a commercial, “hiding” the negative effects in the middle where they are more likely to be deemphasized or forgotten.
What is the difference between these two situations? Metacognition, also known as “thinking about thinking” or the ability to monitor mental states and use that information to control behaviors (Nelson & Narens Reference Nelson, Narens and Bower1990). Knowing that they will not remember the full content of any meeting – that is to say, being aware of their own memory limits – employees will take (or be given) written notes to aid memory. They'll still be subject to primacy and recency, but their preemptive information-preserving behavior will improve their understanding of the job. Commercial viewers will probably not take notes or be aware of their memory in that moment, and their understanding of the drug will actually get worse. Dialog plus a metacognitive component (recognizing your own mental abilities and incorporating that information into decisions) leads to clarity and a constructive self. Without metacognition there is nothing to protect against ignorance, misinformation, and values that are out of line with reality.
Note that this is not just a passive phenomenon, but a skill that can be improved. This is why we suggest that it is insufficient to merely accept defeaters; they must be actively defended against. Consider one final example of a cognitive bias called the first-instinct fallacy, which is the false belief that first instincts are special or more likely to be correct. It is well known that taking an exam is a form of dialog that can be constructive and lead to better memory/learning/clarity (Roediger & Karpicke Reference Roediger and Karpicke2006). But many people have learned, and even been told by teachers, that their first instincts ought to be trusted. Despite overwhelming research to the contrary, the belief persists (Kruger et al. Reference Kruger, Wirtz and Miller2005). If someone trusted their first instinct and got a question wrong, Doris (sect. 2, para. 7) would likely contend that defeaters obtained and the act was not agentic. But clearly this is insufficient, because there is a method to overcome the defeater and the person would be held responsible for their choice whether they overcame the defeater or not.
Couchman et al. (Reference Couchman, Miller, Zmuda, Feather and Schwartzmeyer2016) used a simple method of having exam-takers keep track of their level of confidence in each answer, specifically to guard against the transience of memory and several of Tversky and Kahneman's (Reference Tversky and Kahneman1974) heuristics. Confidence ratings are a common form of metacognitive assessment, and they found that these in-the-moment self-assessments were predictive of objective accuracy. In fact, when deciding whether to change an answer or not, confidence ratings were a better guide than first instincts. Contrastingly, assessments made before and after the exam – explicit judgments subject to the fragility of memory and the cognitive biases described above – were less accurate. Overall, they found that defeaters could be overcome, misconceptions avoided and biases circumvented, by tracking confidence to increase the availability of metacognitive information and aid the decision-making process.
Thus, implicit self-monitoring, a reflective process in humans and even some animals (Smith et al. Reference Smith, Couchman and Beran2012) that uses working memory and can be trained (Coutinho et al. Reference Coutinho, Redford, Church, Zakrzewski, Couchman and Smith2015), can correct explicit biases that can result from dialogic collaboration. The same process could extend to any “collaboration,” to ensure that values are represented and formed in ways that are not based on known errors (even if they are unnoticed defeaters).
Finally, we would note that metacognition is closely related to self-agency (Couchman Reference Couchman2012), self-awareness, and importantly to theory of mind, the process of understanding others' thoughts and taking their perspective (Carruthers Reference Carruthers2009; Couchman et al. Reference Couchman, Coutinho, Beran and Smith2009). Metacognition is the primary tool we use to overcome self-ignorance. As Doris (sect. 9, para. 5) states, “agency-impairing self-ignorance is considerable, but people are collectively possessed of epistemic assets fit to ameliorate it.” This is partially correct, but without working to improve our uncertainty (or ignorance) via our monitoring abilities, we would have no way to judge when defeaters have obtained and no way to know which collaborations increase agency and which might ameliorate it. Similarly, theory of mind is the primary tool through which meta-defeaters and surreptitious dialogic tactics are generated. By taking the perspective of others, and understanding well-established cognitive techniques, a hostile interlocutor can change beliefs, values, and even selves to be out of line with the truth.
To use a metaphor: Doris essentially describes how journalism (collaborative dialog) could build a great society (the self). We fully agree, but at the same time we welcome him to the war. Propaganda (via meta-defeaters) is already being used to harm the great society, through the very channels he proposes will save it. There is no check whatsoever to ensure that dialog alone will lead to truth, and in fact we know it can easily be used to create selves with values out of line with reality. Thus, we are compelled to improve our fact-checking and editing process (metacognition) to regulate journalism and make sure it steers us toward a clarifying truthful dialog.
The self is indeed a function of collaborative rationalization, and Doris (sect. 8, para. 7) correctly points out that agency (and by extension the self) does not require freedom from influence – an impossible criterion – but rather mutual influence. But with collaboration comes conflict, and it is a grave mistake not to recognize this. Collaboration could lead to Utopia, but hostile actors could also steer us into the Wild Wild West, or worse. And as such, the highest ethical standard ought to be to increase metacognition. To increase self-monitoring and fend off untruthful narratives that intentionally play on human cognitive biases. This holds true in all human activities, even in extreme cases like addiction (Hajloo et al. Reference Hajloo, Sadeghi, Nadinleoi and Habibi2014) that Doris discusses.
You have
Access
Doris builds his framework largely on the foundation of social psychology. He generally does not address the cognitive underpinnings of social psychology, on which the collaborative dialog is built and through which it can easily be exploited. Several well-established cognitive principles enhance and extend his idea, but are also notoriously used by advertisers, politicians, and other hostile actors. His framework is largely correct, but it is wrong to assume that everyone in the “dialog of self” is playing nice. In fact, many interlocutors are using cognitive principles to shape narratives and change values (and even selves) in surreptitious ways that may not result in the clarity that he envisions. The dialog itself will not naturally favor truth over falsehood, but it can be saved.
We would propose that the highest ethical standard in his system ought to be the process of increasing metacognition – the ability to self-regulate (to beat defeaters) and to avoid biases and hostile narratives (or meta-defeaters, if you will). Because metacognition is a skill and a tool, it is insufficient to simply say that “when defeaters obtain, the exercise of agency does not obtain” (p. 10). It is through metacognition that one can become better at recognizing and defeating defeaters. And it is doubly insufficient because by intentionally playing on human decision-making biases (e.g., heuristics, selective attention, hindsight bias, the first-instinct fallacy, etc.), hostile interlocutors can introduce a whole class of meta-defeaters that are engineered to surreptitiously change narratives, and thus values, and thus selves.
For example, suppose Candidate Creepy reads Simons and Chabris (Reference Simons and Chabris1999) and learns about selective attention. He would discover that while focused intently on one event, humans ignore or do not notice things that would normally be salient. (In the original experiment, many participants counting the number of basketball passes in a busy environment did not notice a man in a gorilla suit walk into the middle of the scene, beat his chest, and walk off.) Thus, when journalists publish an in-depth investigative article about his indebtedness to foreign entities and business conflicts around the world, the Candidate might falsely but loudly declare that millions of people voted illegally in the recent election. Now the “collaborative” conversation focuses entirely on whether that claim is true, how he could possibly believe that, how others might believe it, etc. Temporarily interesting but ultimately meaningless content has monopolized the conversation. This will be internalized and go into forming the beliefs and values and selves of society, not the objectively more important and true information.
Suppose the Candidate claims he will build a giant dome over the country to protect us from aliens, and that the aliens will pay for it. The conversation ought to be about the merits of a dome, but instead we naturally anchor (Tversky & Kahneman Reference Tversky and Kahneman1974) to the outrageous idea of aliens paying for our dome, and by comparison the actual building of the dome seems relatively sane and doable. Also, the only way to selectively attend to the payment discussion forces us to accept, as a premise, that the dome will be built. Even attempting to move the collaborative negotiation (p. 35) back to the dome itself is impossible because the main objection – the cost – is subverted by the baseless claim that it will be covered by aliens. You can't talk about the cost without including that aliens will pay for it, and you can't talk about aliens paying for it without accepting that it will be built. Again, the narrative is hijacked and the beliefs, values, and selves that will form out of it will be based on misinformation.
Of course, dialog is even more vulnerable when we enter the fragile realm of human memory. Candidate Creepy might use hindsight bias (Fischhoff Reference Fischhoff2007), anecdotal evidence, confirmation bias (Wason Reference Wason1960), or confuse correlation and causation. All of these effects just happen to play into the false claim that vaccines cause autism. How do you combat this falsehood? The natural reaction might be dialogic. Educate people to correct the myth! Unfortunately, Schwarz et al. (Reference Schwarz, Sanna, Skurnik and Yoon2007) describe how this leads to the backfire effect. Even if people are initially converted to the truth, many will end up continuing to accept the myth and will even believe it more strongly after several weeks have passed, not because of any surreptitious motives on their part, but because that is just how memory works. And the end result is that people believe vaccines cause autism, believe evidence supports this “fact,” and form values and selves to match this belief. They might even go further and believe, again as core values and expressions of their selves, that scientists and journalists are misguided or evil for contradicting this “fact.”
These three examples (there are hundreds more) might be called meta-defeaters, or processes by which defeaters are created. They all prey on implicit processing, in which the people experiencing these effects would not be able to say to themselves “I am suffering from this psychological phenomenon engineered by Candidate Creepy.” Instead they would believe, as a reflection of their values, that the Candidate was correct. But those beliefs were engineered explicitly and inserted into the dialog in order to change people into selves with values counter to the truth. Doris (sect. 2, para. 7) contends that we can rule out defeaters, and agency is exercised, if the action reflects the values of the actor. But what if your values are the result of meta-defeaters? Can a person be blamed for any of the “mistakes” above, given that each one would be a reflection of their unknowingly hijacked values?
This is not to say that Doris's dialog always leads to falsehood, but rather that it could equally lead to clarity or muddiness or even strengthened belief in falsehood unless some process is in place to guide it. Two brief examples will help illustrate this. Consider the primacy and recency effects (Ebbinghaus Reference Ebbinghaus1913), where people tend to remember the beginning and end of a dialog better than the middle. Managers use this effectively by providing positive feedback to employees at the beginning and end of a meeting and placing criticism in the middle. The employee will leave happy but will have a list of things to improve. Wonderful collaboration. More surreptitiously, advertisers place positive aspects of a drug at the beginning and end of a commercial, “hiding” the negative effects in the middle where they are more likely to be deemphasized or forgotten.
What is the difference between these two situations? Metacognition, also known as “thinking about thinking” or the ability to monitor mental states and use that information to control behaviors (Nelson & Narens Reference Nelson, Narens and Bower1990). Knowing that they will not remember the full content of any meeting – that is to say, being aware of their own memory limits – employees will take (or be given) written notes to aid memory. They'll still be subject to primacy and recency, but their preemptive information-preserving behavior will improve their understanding of the job. Commercial viewers will probably not take notes or be aware of their memory in that moment, and their understanding of the drug will actually get worse. Dialog plus a metacognitive component (recognizing your own mental abilities and incorporating that information into decisions) leads to clarity and a constructive self. Without metacognition there is nothing to protect against ignorance, misinformation, and values that are out of line with reality.
Note that this is not just a passive phenomenon, but a skill that can be improved. This is why we suggest that it is insufficient to merely accept defeaters; they must be actively defended against. Consider one final example of a cognitive bias called the first-instinct fallacy, which is the false belief that first instincts are special or more likely to be correct. It is well known that taking an exam is a form of dialog that can be constructive and lead to better memory/learning/clarity (Roediger & Karpicke Reference Roediger and Karpicke2006). But many people have learned, and even been told by teachers, that their first instincts ought to be trusted. Despite overwhelming research to the contrary, the belief persists (Kruger et al. Reference Kruger, Wirtz and Miller2005). If someone trusted their first instinct and got a question wrong, Doris (sect. 2, para. 7) would likely contend that defeaters obtained and the act was not agentic. But clearly this is insufficient, because there is a method to overcome the defeater and the person would be held responsible for their choice whether they overcame the defeater or not.
Couchman et al. (Reference Couchman, Miller, Zmuda, Feather and Schwartzmeyer2016) used a simple method of having exam-takers keep track of their level of confidence in each answer, specifically to guard against the transience of memory and several of Tversky and Kahneman's (Reference Tversky and Kahneman1974) heuristics. Confidence ratings are a common form of metacognitive assessment, and they found that these in-the-moment self-assessments were predictive of objective accuracy. In fact, when deciding whether to change an answer or not, confidence ratings were a better guide than first instincts. Contrastingly, assessments made before and after the exam – explicit judgments subject to the fragility of memory and the cognitive biases described above – were less accurate. Overall, they found that defeaters could be overcome, misconceptions avoided and biases circumvented, by tracking confidence to increase the availability of metacognitive information and aid the decision-making process.
Thus, implicit self-monitoring, a reflective process in humans and even some animals (Smith et al. Reference Smith, Couchman and Beran2012) that uses working memory and can be trained (Coutinho et al. Reference Coutinho, Redford, Church, Zakrzewski, Couchman and Smith2015), can correct explicit biases that can result from dialogic collaboration. The same process could extend to any “collaboration,” to ensure that values are represented and formed in ways that are not based on known errors (even if they are unnoticed defeaters).
Finally, we would note that metacognition is closely related to self-agency (Couchman Reference Couchman2012), self-awareness, and importantly to theory of mind, the process of understanding others' thoughts and taking their perspective (Carruthers Reference Carruthers2009; Couchman et al. Reference Couchman, Coutinho, Beran and Smith2009). Metacognition is the primary tool we use to overcome self-ignorance. As Doris (sect. 9, para. 5) states, “agency-impairing self-ignorance is considerable, but people are collectively possessed of epistemic assets fit to ameliorate it.” This is partially correct, but without working to improve our uncertainty (or ignorance) via our monitoring abilities, we would have no way to judge when defeaters have obtained and no way to know which collaborations increase agency and which might ameliorate it. Similarly, theory of mind is the primary tool through which meta-defeaters and surreptitious dialogic tactics are generated. By taking the perspective of others, and understanding well-established cognitive techniques, a hostile interlocutor can change beliefs, values, and even selves to be out of line with the truth.
To use a metaphor: Doris essentially describes how journalism (collaborative dialog) could build a great society (the self). We fully agree, but at the same time we welcome him to the war. Propaganda (via meta-defeaters) is already being used to harm the great society, through the very channels he proposes will save it. There is no check whatsoever to ensure that dialog alone will lead to truth, and in fact we know it can easily be used to create selves with values out of line with reality. Thus, we are compelled to improve our fact-checking and editing process (metacognition) to regulate journalism and make sure it steers us toward a clarifying truthful dialog.
The self is indeed a function of collaborative rationalization, and Doris (sect. 8, para. 7) correctly points out that agency (and by extension the self) does not require freedom from influence – an impossible criterion – but rather mutual influence. But with collaboration comes conflict, and it is a grave mistake not to recognize this. Collaboration could lead to Utopia, but hostile actors could also steer us into the Wild Wild West, or worse. And as such, the highest ethical standard ought to be to increase metacognition. To increase self-monitoring and fend off untruthful narratives that intentionally play on human cognitive biases. This holds true in all human activities, even in extreme cases like addiction (Hajloo et al. Reference Hajloo, Sadeghi, Nadinleoi and Habibi2014) that Doris discusses.