2. The mirage of similarity

In the literature on depiction, both in aesthetics and the theory of images, there is a longstanding tradition that focuses on similarity to explain how pictures represent the real world (Wollheim Reference Wollheim1987; Hyman Reference Hyman2006, Reference Hyman2012; Peacocke Reference Peacocke1987; Gombrich Reference Gombrich1960, Reference Gombrich1982).Footnote ¹ However, applying the similarity view to scientific pictures has been largely unexplored by its proponents in the context of scientific models (e.g., Giere Reference Giere2004, Reference Giere2010; Weisberg Reference Weisberg2013, ch. 8).Footnote ² An important exception is Meynell (Reference Meynell2013), who explicitly clarifies the role of similarity in scientific pictures and visual representations. I take her view as a reference for critically analysing the similarity account in this context.

Meynell’s account combines a critique of Perini’s (Reference Perini2005) Goodmanian conventionalism with a constructive proposal inspired by Willats’s (Reference Willats1997) work in psychology. While Meynell accepts that Perini’s approach may work for linguistic or schematic visual representations, she argues it is insufficient for ‘dense’ pictures, like photographs, scans, and astronomic images. She proposes instead a similarity-based approach, informed by psychology, perception theory, and geometry.

Following Willats, Meynell (Reference Meynell2013, 338) argues for a two-step relationship between a picture and its target. First, picture primitives (e.g., lines, points, and coloured areas) are associated with scene primitives, which are the basic shape elements in the scene—3D (lumps, sticks, slabs), 2D (surfaces), 1D (edges), or 0D (corners). Second, these scene elements are connected to the target system in the real world.

Meynell then characterises both the relation between picture primitives and scene primitives, and the relation between scene primitives and the target. The aim is to show that both steps are based on (objective) similarity and how our perception works, in contrast with the position defended by Perini and Goodman, who focus on interpretation and convention. Both steps, she claims, are based on objective similarity and perceptual mechanisms. The first step relates picture primitives to scene primitives via geometrical projection, while the second step ties the scene to the target system through visual or perceptual similarities: the scene represents the target because it visually resembles what we would see if we directly observed the target system.

Assume Meynell’s account succeeds for photographs, realistic paintings, and simple geometrical figures. The point I want to make here is that even if the similarity view were successful for more mundane cases, the account is inadequate when applied to more complex examples of visual representations used in science. Let us then consider the picture of the M87* black hole produced by the Event Horizon Telescope (EHT) as shown in figure 1.

Figure 1.

Picture of the black hole M87* (Event Horizon Telescope Collaboration 2019a, 5).

First, according to Meynell, we should be able to reconstruct the scene represented by the picture by geometrically projecting the picture primitives (coloured regions) onto the scene’s basic elements. Yet even knowing the geometrical projection (e.g., perspectival), it is unclear what the scene itself would be, as projections are neutral with respect to what is projected. Second, if similarity between the scene and target is defined in terms of visual appearance, we would infer that the black hole is a reddish-yellow doughnut in the middle of nothing—an obviously false depiction.

In contrast, what we need is a function that associates colours with what they represent—levels of radiation intensity. This requires interpreting the picture, translating visual properties into non-visual ones. Thus, on closer inspection, similarity is not the core concept for understanding how such pictures epistemically function; the critical role is played by interpretation. Once colours are translated into radiation intensity levels, there is no meaningful sense in which the picture and target are visually similar.Footnote ³

3. One step back: Imaging a black hole

There is already an emerging literature on the epistemology of the picture of M87* and black holes in general (Skulberg Reference Skulberg2021; Muhr Reference Muhr2023; Doboszewski and Lehmkuhl Reference Doboszewski, Lehmkuhl, Mills Boyd, Baerdemaeker, Heng and Matarese2023; Doboszewski and Elder Reference Doboszewski and Elder2024). Curiously, however, philosophers of science have not studied this picture as a picture. Namely, they have not focused on the features of this picture as a representation – as an object allowing surrogative reasoning about its target system. I want to suggest that, while the epistemological analyses conducted so far are crucial, they remain incomplete. For a study of the picture as evidence presupposes an analysis of how the picture is supposed to be “read” as a representation of a black hole. In this sense, my analysis will also be a useful contribution to the epistemological enquiries about black hole pictures more generally.

To analyse the picture of M87* as a representation, some details on black holes and their imaging are needed. Imagine an object, 54.8 million light years away, with a mass around 6.5 billion times the Sun’s, compressed to the size of our Solar System. Scientists think such a supermassiveFootnote ⁴ black hole lies at the centre of the Messier 87 galaxy (Gebhardt et al. Reference Gebhardt, Adams, Richstone, Lauer, Faber, Gültekin, Murphy and Tremaine2011; Walsh et al. Reference Walsh, Barth, Ho and Sarzi2013). Black holes, predicted by general relativity (GR; Einstein Reference Einstein1915; Penrose Reference Penrose1965), remain central to unifying GR with quantum physics (Hawking Reference Hawking1976; Giddings Reference Giddings2017).Footnote ⁵ The gravitational pull is so strong that even light cannot escape (Schwarzschild Reference Schwarzschild1916), marking a boundary called the event horizon. Beyond it, no observationFootnote ⁶ is possible. However, black holes attract surrounding matter.Footnote ⁷ Near the event horizon, matter orbits at incredible speeds, forming the accretion disk—ionised gases burning at 1 to 10 billion Kelvin—emitting measurable radiation.

The fundamental theory for such measurements is interferometry.Footnote ⁸ By decomposing and measuring phase differences in two light beams, interferometry reveals the distribution of radiation emitted by a radiating source—astronomical objects included. This measurement of the distribution of radiation offers crucial insights into the dimensions and shape of the black hole’s shadow. In April 2017, seven telescopes across the globe synchronised to measure radio signals from M87* for four days. Together, they formed a composite ‘lens’ the size of Earth, gathering 1.5 petabytes of data each night—the largest dataset ever for a single experimental measurement. The data was integrated by a supercomputer and calibrated (EHTC 2019b).

Four independent research teams processed the data into images without interacting with each other. Two teams used the so-called CLEAN algorithm, while the other two adopted the SMILI and EHT-imaging algorithms, both from the common family of algorithms known as regularised maximum likelihood (RML; EHTC 2019c). The resulting images from the independent pipelines all shared two key features: a ring shape with brighter southern regions, and a ring diameter of about 40 μas (9). Each team produced one image per night, which were further blurred to achieve a “common, conservative resolution” (20). To further emphasise the common features of the images, the scientists then produced an average picture for each of the four days (21).Footnote ⁹

The resulting image is a heatmap—a visual representation where colours and spatial coordinates translate into properties of the target system. These properties can also be colours and spatial or geometrical properties, but do not need to be. What we need to be able to read a heatmap, then, is a systematic way to translate the visual properties of the heatmap into the relevant properties of the target system that the heatmap is meant to represent. In section 4, I outline the basic elements of such an account.

4. An interpretation-based account of scientific pictures

The picture of M87*, I argue, interpreted as such, refers to M87*. Following Goodman (Reference Goodman1976), I take this referential relation from a symbol to an object to be denotation, namely, the referential relation between a name and its bearer. In order to denote M87*, though, we need to look at the picture as not, say, mere pixels coloured in a specific way: we need to interpret the material vehicle, or carrier, of the picture, as a radiation-heatmap of the electromagnetic field surrounding M87*. Interpretation here can be understood as a function I that maps colours of the picture into levels of radiation intensity of an electromagnetic field, where black is interpreted as lower levels and yellow as higher. So, we have three elements: the carrier, the heatmap, and the target. The picture, once interpreted, represents M87* as a radiation-heatmap—or, alternatively, it is radiation-heatmap-representation of M87*.

The heatmap resulting from our interpretation of the carrier is not supposed to be read as a truthful description of the target, as it may involve idealisations and distortions with respect to the target system. Again, I use Goodman’s (Reference Goodman1976) terminology and call the radiation-heatmap a Z-representation. Distinguishing these three elements (carrier, Z-representation, and target system) is crucial to appreciate the type of reasoning involved when we make inferences about the actual black hole on the basis of the picture. This reasoning is surrogative in nature (Swoyer Reference Swoyer1991): reading the picture as a representation requires us to study a system in order to make inferences about another system.

It is nevertheless useful to study the Z-representation in question because it highlights certain properties of the target that we would have not been able to detect if we had just looked at the raw interferometric data collected by our telescope. Using the terminology of Goodman (Reference Goodman1976) and Elgin (Reference Elgin1983, Reference Elgin1996) we can call this function exemplification. Technically, an object exemplifies a property A if it instantiates A and refers to A by instantiating it. A typical example is a swatch in a tailor shop. The swatch possesses many properties (say, being rectangular, being produced in Thailand, weighing 1.5 g, and so on) but, in the context of the shop where people decide how their clothes should be, the swatch refers only to a certain set of properties: colour, texture, material, and so on. By referring to them, it makes them salient: these properties become, or should become, more epistemically accessible for a user or observer.

The same happens with the radiation-heatmap: by abstracting away informational noise and resulting from an interpolation of the original sparse data, this picture summarises and highlights certain prominent features of M87*, particularly the remarkable shadow of the black hole, its dimensions, and an asymmetry in radiation intensity between the southern and northern areas of the accretion disk. For example, from the scaled dimensions of the pictures, astronomers can infer that the dark object at the centre cannot be a naked singularity or a wormhole, but it is more likely to actually be a supermassive black hole as it is theoretically predicted by the general theory of relativity (Bouman Reference Bouman2020).

We can go further and, coupling the picture properties with our models of black holes and “information on the inclination angle, […] derive the sense of rotation of the black hole to be in the clockwise direction, i.e., the spin of the black hole points away from us” (EHTC 2019a, 9). Also, the EHTC authors explain the brightness asymmetry in the south region of the black hole “as relativistic beaming of material rotating in the clockwise direction as […] moving toward the observer”. The south–north asymmetry in radiation intensity, then, is explained as a case of the so-called Doppler effect.

The point of all this is of course to eventually impute some of these exemplified properties to the target system, the actual black hole M87*. When I talk about imputation, I simply mean property attribution, with no assumptions about whether this attribution is correct or not.

Sometimes, the properties exemplified by a representation can be imputed to its target system unchanged, but other times this imputation will require some form of de-idealisation. To distinguish this further interpretive activity from the interpretation function I, let us call the function that translates idealised properties of the representation into non-idealised properties imputed to the target a key.

In the case of the picture of M87*, there are many keys at work. One is simply a scale factor, multiplying the dimensions of the object in the picture into the actual dimensions that we expect M87* has, based on the picture together with our knowledge of the distance between us and the centre of the Messier galaxy. Another important key is a geometrical projection that translates the two-dimensional spatial properties of the heatmap into a set of three-dimensional ones.Footnote ¹⁰ A final interesting type of key seems in place in the process of blurring the final visual outputs of the algorithms in order to decrease precision but increase reliability. Here, a key should be used to de-blur the picture if our aim is to attribute more fine-grained, precise properties to the actual black hole.Footnote ¹¹

From this reconstruction, one can see that no appeal to similarity, perception, or psychology has been made: given the presence of the I-function and the key, neither the picture as a carrier nor the picture as heatmap need to be similar to the target to represent it. At the same time, the account still retains Meynell’s basic intuition about the role of geometrical projections in visual representation. This role, though, is inserted in a more general framework entirely based on interpretation (in the various fashions of the I-function, denotation, the selective process of exemplification, and the key). Interpretation here is arbitrary but not random: while heavily theory-laden, our interpretation of the picture strictly depends on the way in which the projection was in fact produced.

Once the picture is interpreted as a heatmap, four basic elements have been highlighted in my analysis: denotation, exemplification, keying-up, and imputation. These are the basic ingredients of the so-called DEKI account of scientific representation (Frigg and Nguyen Reference Frigg and Nguyen2020), which takes its name exactly from those ingredients and has been applied to many cases of scientific models. More formally, according to DEKI, a model system M (that is, a carrier C endowed with an interpretation I) is an epistemic representation of a designated target system T iff four conditions apply:

1. (i) M denotes T;

2. (ii) M exemplifies properties P ₁, …, P _n ;

3. (iii) P ₁, …, P _n are associated with a second set of properties Q ₁, …, Q _n via a key;

4. (iv) Q ₁, …, Q _n are imputed to T.Footnote ¹²

At this level of analysis, then, it seems correct to say that a mechanically produced picture like the picture of M87* functions as an epistemic representation in the same way as scientific models represent their target systems.

5. From semantics to epistemology: Measurement vs. model

So far, I have focused on how to interpret the picture as a representation and characterised this representational function in the terms of the DEKI account. However, the account has two important shortcomings. First, it is skeletal by design: it needs to be completed with the specifics of each case study. The analysis of the black hole picture just offered provides the relevant details on how to apply DEKI to this specific case study. Second, the account remains silent on the justification of our inferences from the picture to the target system. This is because the account correctly acknowledges that the justificatory roots of our inferences lie outside the single representational system. In this respect, pictures are exactly like models, because even for inferences drawn from a model about a target system, the only way to justify our inferences is to support them with reasons extrinsic to the single model system (direct observations of the target system, theories, observations, and other models).

The similarity between pictures and models, I want to suggest, ends here. In the case of pictures like the picture of M87*, the root of the justification of both our interpretation of the picture and of our inferences about its target is the history of production that causally connects a given picture to its designated target system. This is not usually the case with models.

Assume that it is impossible to observe the target system directly. If one wants to convince me that the picture of M87* is giving me (approximately) correct results about the actual black hole, they will have to explain how the picture was produced and how the visual output is causally dependent on the target system.

For example, Doboszewski and Elder (Reference Doboszewski and Elder2024) analyse the picture in terms of robustness analysis, showing that multiple algorithms employed for the imaging converged on similar results despite taking different procedures and assumptions. The algorithms also exhibited reasonable sensitivity to different data. This robustness was tested against synthetic images showing different geometrical shapes, which the algorithms reconstructed as they did with the black hole picture. By doing this, researchers obtained two results. First, they identified the fiducial parameters that allowed a more faithful reconstruction of the original image. Second, they showed robustness of the algorithms, as outputs were different for each synthetic image and the black hole picture, demonstrating strong counterfactual dependence of the visual output on the original source.

This two-face robustness analysis was necessary to secure a reliable causal relation between the data and the visual output obtained by applying the algorithm, and consequently, the accuracy of the latter with respect to the former.

This causal relation, I suggest, lies at the root of the inferential stabilityFootnote ¹³ from measurement-like representations to targets more generally. The more numerous and complex the steps in the causal chain of producing the picture from the target, the more difficult it will be to justify the inferences we draw from the former to the latter. Here, I am not arguing that the picture of M87* is, in fact, epistemically reliable. My point is that to assess such reliability, one must examine its production and how it causally relates the target with the interpreted visual output.

Again, the notions of reliability, accuracy, and success I employ here do well without any appeal to similarity between the picture and the target system: what counts is the counterfactual stability between the target and the visual representation endowed with an interpretation, which depends on the causal mechanisms connecting them.

These considerations about the justificatory role played by the causal relation between target and representation can be extended to many other scientific pictures. The justification of the inferences drawn from photographs, astronomical pictures, maps, and X-ray and MRI scans all require assessment of the causal, counterfactual stability between the target and the visual output in question. Furthermore, this can be generalised to other examples of representations that we do not normally call pictures. The colour displayed on a litmus paper can be reliably interpreted as the indication of the level of acidity of a chemical solution only if one can give a story about the counterfactual stability between the chemical features of the liquid under measurement and the resulting colours appearing on the paper. The same applies to clock dials representing time, thermometers reading the surrounding temperatures, and even model organisms when, instead of being used to represent properties of other organisms (Sartori Reference Sartori2023a), they are used to represent toxicity levels in the surrounding environment.Footnote ¹⁴ That is why I find it useful to label all these instances of representations as ‘measurement representations’, and not focus too much on the pictorial or imagistic features.

Nothing of the sort of what I have said about justification in the case of the picture of M87* applies to other forms of representations, like models. A model system is usually constituted by a set of assumptions on that system (an abstract object or a material one), often in interaction with each other. Let us take the simple case of an assumption that is expressed by a certain functional relation between two quantities. There are many ways in which we can justify this assumption. It may directly derive from more general theory in the relevant discipline. Or our assumption may be a simplification of a more general functional relation that however is intractable in its current form (e.g., an equation with no analytic solutions). Here, the justification follows from our reason to hold the original formula, plus some further reason to consider the simplification acceptable. Alternatively, our assumption may boil down to a hypothesis abstracted away from data, perhaps via an abductive inference.

In all these cases, the justification of the assumption will be more or less provided on the basis of previously acquired knowledge. However, the assumption could also be something completely new, detached from theory and experiments. The justification of that assumption will then solely depend on the success of the model as a whole. Success can take many forms: empirical adequacy, unification, explanation by providing an underlying mechanism. The more the model proves itself successful, the more we can justify its further application as an epistemic surrogate system. However, as it should be evident, there is no appeal to causal relations between the target and the model in our justification of the inferences from the latter to the former.

What I have said about models applies to other sorts of representations. Schematic diagrams, for example, seem to be justified in a similar way as models, even if they are visual representations like pictures. Elsewhere, I have also argued that scientific thought experiments (Sartori Reference Sartori2023b) and model organisms (Sartori Reference Sartori2023a) also function as representations of phenomena in the sense expressed by the DEKI account. For both thought experiments and model organisms, I want to suggest that their style of justification is model-like.Footnote ¹⁵

Nevertheless, it is important to remind the reader that the characterisation of the causal relation holding between target and representation in the instances of measurement-like representations is still based on theoretical assumptions and previously acquired empirical knowledge. I do not want thus to undermine the theory-ladenness of our interpretations of images: the assessment of the hypotheses about the causal relations in play in our production of images will also strictly depend on the theoretical framework we are assuming in the first place. Similarly, I do not want my focus on causation to foster the idea that pictures are somewhat more ‘objective’ representations than models: whether an image is causally linked to its target and how accurately can be, and is, a matter of dispute, even among experts.