Objectivity and the Rio Scale

As noted, one of the primary goals of the Rio Scale has been to bring a degree of objectivity into the discussion of the significance for humans of a valid signal indicating the existence of extraterrestrial intelligence. Significance is a complex term, but in general it is reasonable to think of significance in terms of whether or not an action, event, or result is meaningful. When it comes to detection of a signal from extraterrestrial intelligence, the term typically is used in two different ways to indicate: (1) whether or not the findings related to analysis of a signal suggest extraterrestrial intelligence and (2) the extent to which a valid signal should be viewed as important for humans. In many cases, these two usages are somewhat conflated in literature that discusses the importance and relevance of SETI research.

As is common in the SETI community, the authors of the Rio Scale argue that either discovery of extraterrestrial intelligence or confirmation of its absence is necessarily significant, because it ‘would clearly have enormous consequences for humanity’ (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018). This is a widespread trope used in the SETI community to explain and justify the importance of their work, and, indeed, the authors of the paper on Rio 2.0 thought it significant enough that this assumption is presented in the first sentence. When any idea becomes uncritically assumed to be true, it is important as a social scientist to interrogate the validity of that assumption. Thus, I want to begin by asking if it is actually the case that discovery of either the presence or absence of ETI would have significant consequences for humanity?

The answer to this seems ambiguous. Throughout all of human history there has been no evidence of life anywhere other than on Earth; however, humans do have a long history of imagining and accepting the reality of non-human intelligences, particularly in the forms of demons and deities. As Partridge notes, ‘[p]ersonifications of evil in the form of demons, devils, spirits and malign entities can be found across the religious spectrum’ (Partridge, Reference Partridge2004). The idea of intelligent non-humans is virtually universal among human societies and thus it would seem reasonable to assume that the news of intelligent beings living on other worlds would not necessarily be ground-breaking, but may actually fit fairly neatly into beliefs humans have had about other forms of intelligence throughout our history, even if the basis shifts from religious belief to scientific evidence. In other words, the notion of non-human intelligence is not alien to humans and has not been throughout much of our history.

Humans also have experience with the idea that intelligent aliens might be living in our immediate neighbourhood. During the late 19th century Percival Lowell's claims that there was an advanced civilization on Mars gained considerable press in the USA and in Europe and the possibility of a Martian civilization was debated in the public discourse (Lane, Reference Lane2011; Traphagan, Reference Traphagan2014, Reference Traphagan2016). That was a long time ago, and the social impact, despite the assumed civilization being right next door, was minimal and brief. We might chalk this up to the fact that it eventually became clear that there was no civilization on Mars, but for several decades after Lowell's claims many people, at least in the USA and Europe, continued to believe that there might be a civilization on Mars and, in fact, one that was technologically superior to our own (as Lowell thought). Interestingly enough, as late as 1958 Morrison and Cocconi in their influential work on SETI claimed that Mars ‘very probably’ hosted life, even if it was not intelligent life (Morrison and Cocconi, Reference Morrison and Cocconi1959). Furthermore, in 1996 NASA announced that research on a meteorite found on Earth showed ‘evidence that strongly suggests primitive life may have existed on Mars’ (Savage et al., Reference Savage, Hartsfield and Salisbury1996). This was considered significant enough that President Clinton made comments specifically on the report in a brief announcement to the press.

Of course, this finding was fairly quickly called into question, so there remains no uncontested evidence that life exists anywhere other than on Earth. However, given that significant public events related to finding life, even if unconfirmed or untrue, have occurred in recent history, we have some empirical basis for thinking about the potential influence of actual discovery of life elsewhere; and analysis of those historical events does not obviously suggest that they generated a dramatic change in how humans see themselves or their place in the Universe. The discourse related to a great civilization on Mars that Lowell generated may have opened our thinking to the possibility of life elsewhere, and certainly provided the fodder for some great science fiction stories, but it also fit fairly neatly into a preexisting frontier mythology that had shaped American thinking for many decades and, as such, did not represent any form of sea change in human thought (Eisfeld, Reference Eisfeld2018). When it comes to humanity being alone, since throughout all of human history there has been no clear evidence of life anywhere else, it is difficult to see how confirmation that we are alone would necessarily cause dramatic changes in anything related to our thinking about our place in the Universe. In fact, because some religions such as Christianity have long presented humans as being unique in an intentionally created order, it might simply confirm beliefs already held by a large segment of the human population. Such knowledge might influence the thinking of some scientists and other intellectual elites to think in new ways, but it is hard to imagine the presence or absence of alien life being of great importance to the vast number of humans who live in extreme poverty and for whom finding a meal or shelter represents the most pressing of concerns (Traphagan, Reference Traphagan2014). The key point here is that the meaningfulness of a contact event, or of confirmation that we are alone, is neither simple nor incontrovertibly significant.

The issue when it comes to the Rio Scale, of course, is not whether contact or lack thereof would change humanity; the problem lies in the fact that it is a large assumption that awareness of the absence or presence of alien life necessarily would represent a watershed moment in human history that would have significant impact. Many thought the Apollo 11 moon landing would have a profound influence on humanity, but there is little data to indicate that it did. Humans felt a brief moment of unity as Armstrong stepped on the lunar surface and then promptly went back to their daily lives and continued polluting, killing, loving, etc. Even Americans lost interest in going to the Moon rather rapidly after the initial excitement had waned (Dick, Reference Dick2007). In short, the idea that the Rio Scale is a necessary tool for the management of an event that would represent a major moment in human history is itself something of a major assumption. It is unclear that such management would be necessary, because it is unclear that a valid contact or confirmation that we are alone would have any long-term impact on humans. Perhaps there would be significant short-term excitement and public discourse, but our historical experience (with Apollo in particular) suggests that it would likely be fairly short-lived.

Although questions related to the broad assumption behind the creation of the Rio Scale are important, they do not necessarily indicate that the scale itself is unhelpful, at least for SETI scientists and perhaps for the media, in determining the relative importance of different types of contact events. The original version of the Rio Scale was constructed around the following equation:

$$R = Q\delta $$

where Q is the significance of the consequence of the signal and δ is the credibility of that signal, describing the likelihood that the event was of non-terrestrial origin as opposed to the result of an instrumentation error, interpretation error, a terrestrial signal or a hoax. Certainly, the variable δ is an important element in any attempt to assess the significance of a signal event; the problem that I want to focus on here actually lies in the variable Q, the interpretation of which is necessarily subjective.

In their construction of the variable Q, the authors of Rio 2.0 fail to take into account the fact that perceptions about those consequences are culturally shaped and, thus, are likely to vary significantly from one group to another. As noted above, the Rio Scale uses three variables to assess the consequences of signal receipt: type of phenomenon, method of discovery and distance from Earth. Thus, a signal that appears to be directed at Earth is given a higher rating in terms of significance than leakage radiation or detection of traces of astroengineering along the lines of ideas raised about a possible artificial structure surrounding Boyajian's Star (Wright and Sigurd̵sson, Reference Wright and Sigurd̵sson2016). Similarly, a transient signal, such as a serendipitous signal observation, is given a lower significance level than a steady signal, because it is less likely to have been intentionally aimed at humans or at anyone in particular. And the distance from Earth is also rated in terms of whether or not it is within the Solar System, within 50 light-years of our Solar System, within the Milky Way (a fairly large jump from within 50 light-years of Earth), or extragalactic. Each of these factors is given a score from 1 through 6, which leads to a significance level calculated to be between 3 and 15 for any signal observed. In the original Rio 1.0 approach, the δ parameter was allowed to vary between 0 and 1, giving the range of R between 0 and 15, and the later Rio 1.1 version modified the parameter so that R would range between 0 and 10.

For my purposes here, the mathematics of arriving at a score is less important than the way in which numbers are assigned by the scientists who developed this as a way of representing relative significance of a signal. In creating scales that measure ‘significance’, or more specifically as the designers of the scale write, that ‘help calibrate public expectations about tentative signals’, the designers of Rio 2.0 (as with earlier versions) have made a significant error in methodology, because they use an ordinal scale to measure data the authors treat as interval.

Ordinal variables are mutually exclusive, exhaustive and can be ordered by rank. Thus, measures such as low, medium and high are ordinal and when we place a particular data point, such as a person being classified in terms of social class, within one of these ranks, we are excluding that data point from all other ranks. However, ordinal variables, although creating a rank hierarchy, do not tell us anything about the distance between the ranks. In other words, we do not know how much more being in the high rank represents as opposed to being in the medium rank when we assign these ranks (or ask research participants to self-assign them) to social class. If we think about this in terms of something more concrete like human social organization, we can categorize human groups as more or less complex based on the types of agricultural practices they employ and this would represent ordinal categories, but it would not tell us the extent to which one group is more advanced as opposed to another (Bernard, Reference Bernard1995). In short, while ordinal variables represent rank order, the distances between the ranks are not meaningful.

Interval variables have the same properties as ordinal variables, but the distances between the attributes associated with the variables are meaningful. For example, the difference between 80°C and 90°C does not vary in any way from the difference between 10°C and 20°C. It is always 10°C. It is distinct, however, from the difference between 10°F and 20°F, because the scales themselves are not calibrated in the same way, thus the difference in ranking systems has meaning and that meaning can be understood clearly through comparison – such as noting that the freezing point of water occurs at different locations on the numerical scales for the two systems.

Meaningfulness in understanding rank orders becomes quite difficult to deal with when involving human perceptions about relative ranks of ideas, objects or events. Even if one creates a clearly defined ranking scale, it is difficult to determine what people actually mean when they rank attributes. For example, if I give several people a group of ice cream flavours and ask them to rank them in terms of flavour, I might get something like this as average rankings across the group:

1. 1. Black Raspberry

2. 2. Mint chocolate chip

3. 3. Chocolate

4. 4. Strawberry

5. 5. Squid Ink

While I can see that there is an order, I cannot determine the relationships between the items beyond the fact of the rank order. Imagine that all of our participants ranked the flavours as listed above. This would tell us that they agree on the rank order, but it would not tell us anything about what that order means to them. The issue here is that one person might like one and two about the same amount, and three a little less. Perhaps she doesn't like Strawberry much, and she hates squid ink flavoured ice cream (yes, it does exist). In fact, if this participant were to quantify the relationships here, the scale might look more like 1, 1.1, 1.2, 10 and 5000. Another participant might see the distances as being basically equal. Although the scale makes it look like the distances between variables are equal, this is not the case between these two research participants. In other words, the meaning of the ranking is different for these two people, despite the fact that the order of the ranking is the same. However, the fact of the matter is that the relationships here are not actually quantifiable, because the numerical values assigned to each flavour are arbitrary and are likely to vary considerably form one person to another and also are likely to change for individuals over time. I like strawberry ice cream more than I did in the past; I can eat it now, when in the past it made me gag. Thus, the distance between strawberry ice cream and other flavours has changed over time for me.

In social science research, we normally measure concepts like subjective attitudes about ice cream flavours – or the significance of contact with extraterrestrial beings – at the ordinal level and when concepts such as perceived and subjectively interpreted levels of significance or risk are measured at the interval level, questions arise about the validity of the measuring instrument being used (Bernard, Reference Bernard1995). Things that can be observed objectively, such as the readings of temperature on a thermometer, are measured at the interval level. If a social scientist wants to know about household income, an interval measure is the way to go – we can ask the exact amount and the differences in amounts are meaningful when compared with other households. However, if we want to know about perceptions of wealth based on the kind of car someone drives, we are now looking at rankings that are ordinal – the fact that my neighbour drives a BMW as opposed to my other neighbour who drives a Buick does not necessarily tell us anything about their relative wealth. If we have data that rank BMW above Buick in terms of status markers, which seems fairly common in the USA, we might draw that conclusion (although it would be a mistake to do so without other data with which to triangulate the data we have), but it could be a significant mistake. If my Buick-driving neighbour were Chinese it might mean quite the opposite because Buicks are popular cars in China, unlike in the USA. The point here is that the ranking of one car over another is arbitrary – and the distances between ranks are subjective and, therefore, typically idiosyncratic notions held by individuals. Therefore, they have no inherent meaning – meaning is a property associated with the individuals participating in the study.

When analysing data, it is acceptable to divide interval-level data into ordinal categories, so we can, for example, take ages of individuals and group them into young, middle-aged and old; however, it is not acceptable to do this in reverse. If you collect data that groups people by age into one of these categories, it is not acceptable to go back and assign numbers, in part because there is no way to know what the numbers would be.

In terms of the Rio Scale, the variable Q represents an example of confusing ordinal data with interval data. The goal of the scale is to rank the potential consequences of a contact scenario and through that ranking provide a means to calibrate ‘the expectations of the world at large when the signals are discussed in the media’ (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018). The scale is intended to generate a number between 1 and 10, with 10 representing the most significant possible contact scenario, which the designers of the scale define as indicating ‘how excited we should be’ about a valid signal (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018).

It should be immediately clear where this approach fails methodologically – it is an attempt to take ordinal data and treat it as interval data – because the distances between the numbers produced by the scale are subjective. They only have unambiguous meaning to those who created the scale. While some astronomers might see the level of significance as being related to variables such as proximity and directionality of the signal, the general public might place more emphasis on whether or not the aliens have menacing physical features (if we were to get a picture) or showed significant technological superiority even if very far away from Earth. Another option is that the idea of a scale might not itself be seen as meaningful with the public, or parts of the public, who instead might interpret the event as a binary: it's important because it happened, but the proximity and directionality of the signal are not particularly important. In fact, the authors of Rio 2.0 do precisely this when they open their paper by indicating that contact or lack thereof would represent ‘enormous consequences for humanity’ (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018).

Perhaps the most telling problem with Q is that all interpretations for the variable are presented as being highly significant. In detailing how to interpret the numbers generated by operating the scale, the authors indicate the following schema for interpreting the results:

• Q = 10: Revolutionary. Everyday life on Earth will change forever.

• Q = 8–9: The making of an epoch; the future direction of humanity is changed.

• Q = 6–7: SETI becomes the ‘study of ETI’. There are good prospects for near-future, limited understanding of ETI.

• Q = 4–5: Scientifically revolutionary, but of no everyday consequence. Prospects for understanding ETIs remain decades in the future.

• Q = 0–3: Philosophically ground-breaking, but of limited immediate social or scientific impact. The prospects for understanding ETIs remain unclear (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018).

Note that all interpretations of Q work from the assumption that a signal will be a powerfully significant event for humanity. The lowest level of significance is ‘ground-breaking’ while the highest is ‘revolutionary’. There are no options for interpreting a signal as being of minimal significance, which is certainly one possible interpretation. Interestingly, when the authors indicate their three desired properties of the Rio Scale, the third is that the score should be ‘as objective as possible – the subjectivity of the user must not be allowed to dominate the final score’. However, the options offered for interpreting the score are entirely subjective – in fact, they make use of highly biased and subjective terms such as ‘revolutionary’, ‘the making of an epoch’ and ‘ground-breaking’. Interestingly, the explanations for Q = 10 and Q = 8–9 are essentially the same thing. Everyday life on Earth will change forever and the future direction of humanity is changed, which implies that everyday life will change. Basically, from 0–10, Q has the same significance – a signal will be big, very big. But we have no scientific evidence to support this claim – which means we have no scientific evidence to support these interpretations of the significance of Q. We do, however, as noted above, have historical evidence that might suggest receipt of a signal would not be as important an event for humanity as SETI scientists would like to believe (Traphagan, Reference Traphagan2016).

The point I want to emphasize here is not that the scientists who created this interpretive schema are necessarily wrong, but that the research has not been done to determine if this is a reasonable way to interpret the Q scores. Therefore, as it stands the scale is entirely the product of the subjective experiences of the scientists who created it.

Homogenizing cultures

This brings me to what I see as another problem with the Rio Scale – it generalizes the meaningfulness of contact for ‘humanity’ and thus homogenizes human experience into a single cultural form that is represented as being shared by people living in all social, economic, educational and political conditions (Anderson, Reference Anderson2009). If one thinks about the conditions of human life, it is difficult to imagine that the significance of any contact event, regardless of where it ranks on the Rio Scale, would be interpreted in the same way by intellectual elites such as American and European astronomers or philosophers, Native American farmers living on reservations, hunter-gatherers such as the Saan People in Africa, or people in developing countries trying to survive on three dollars per day. It also is unlikely that people living in the United States would interpret the significance in precisely the same way as people living in China, and even within a particular society such as the USA, it seems a highly questionable to conclude that inner city poor, conservative Christians, or members of UFO cults, will interpret the significance of contact in the same way as SETI astronomers. In fact, we already have considerable data on how variable the reactions might be. Zeller argues that the Heaven's Gate cult drew on Christian ideas to claim that Earth had already been visited by aliens using an extraterrestrial biblical hermeneutics that placed Jesus as an extraterrestrial visitor. This represents an excellent example of how belief structures can influence and shape the ways in which people react to the idea that extraterrestrials exist and have made contact with humans. In the end, the members of the cult committed mass suicide not out of a concern about the significance of contact with aliens, but precisely because they thought that shedding of their corporeal existence was the best way to hasten that contact – their ideology worked from the assumption that contact was inherently of high significance and good (Zeller, Reference Zeller2010).

The peculiar interpretation generated by the Heaven's Gate cult is not particularly important here; rather, the issue is that when we take into account variables such as age, religious affiliation, race, ethnicity and socio-economic status, we are bound to have a very diverse set of interpretations about the significance of a contact event. And the Rio Scale has no way to capture this diversity. However, it is not clear that the authors of the scale are actually concerned about this diversity. As noted by the authors of the paper on Rio 2.0, a primary goal of the scale is to create a tool to communicate to the general public the significance of a signal and to convey the level of excitement people, meaning all people on Earth, ‘should’ have as a result of a valid signal (Forgan et al., Reference Forgan, Wright, Tarter, Korpela, Siemion, Almár and Piotelat2018).

The question, of course, that arises here is: on what and for whom are the criteria about significance and excitement based? Although the authors are careful to note that different scientists may interpret a contact event in different ways, leading to multiple scores, they also argue that repeated calculation by researchers from multiple disciplines as more data are collected should lead to a better barometer that would allow the general public to assess the level of consensus about a signal among members of the scientific community.

Of course, one is left wondering why the scientific community should be deciding on what is significant or what the basis for assessing significance should be. Since, as the designers of the Rio Scale point out, a valid signal intercepted by humans has the potential to affect all of humanity, why are SETI scientists the ones determining the level of significance? The paper does raise the idea of science writers also getting in on the act, but we are left wondering about how politicians, policy experts, social scientists, artists, etc. would be part of this process. And, as noted above, these categories remain only those of intellectual elites. They exclude the vast majority of the stakeholders in a signal event. In fact, the aim of creating a scale to convey the significance of a valid signal to the general public assumes a position of paternalistic authority in relation to that public – it works from the perspective that astronomers and astrobiologists know how the public should interpret the significance of a signal and, therefore, that astronomers and astrobiologists are in a position to instruct the public about how they should react. However, there is no evidence that astronomers are in any way more qualified to make risk analyses or to judge the meaningfulness and significance of a signal from extraterrestrials than any other portion of the human population. In fact, I would argue that while astronomers and other SETI scientists certainly are an important part of the chorus, there need to be multiple voices involved, including (but not limited to) social scientists, artists, policy analysts and members of the general public from multiple cultural contexts throughout the world for any valid assessment of significance to occur.