Save

On Visual Representations in Science

In: Grazer Philosophische Studien
Author:
Sebastian De Haro Trinity College, University of Cambridge, Cambridge, UK
Amsterdam University College, Amsterdam, Netherlands s.deharo@uva.nl

Search for other papers by Sebastian De Haro in
Current site
Google Scholar
PubMed
Close
Open Access

Abstract

This is a review of Nicola Mößner’s Visual Representations in Science. Concept and Epistemology.

Nicola Mößner. Visual Representations in Science: Concept and Epistemology. Series: History and Philosophy of Technoscience. London / New York, NY: Routledge, 2018, 372 pp.; hardback isbn: 9781138089938.

On 10 April 2019, the Event Horizon Telescope collaboration announced that it had made “paradigm-shifting observations of the gargantuan black hole at the heart of distant galaxy Messier 87”. 1 The Event Horizon Telescope (eht) is an array of eight ground-based radio telescopes, bringing together 13 research institutes around the globe.

While astronomy has always been an observational science, it is striking that the eht collaboration, in its public presentations and press releases, describes its achievements in terms of the production of a single image, albeit of an unprecedented sensitivity and resolution, rather than in terms of its ability to test the theories that describe physics at the event horizon. eht’s public communications clearly present the image of the black hole as the collaboration’s primary result. The more detailed physical questions that the image makes it possible to answer, and which constitute eht’s scientific aim, get, in such presentations, a second place. 2

This primacy of the image in scientific communication raises several important questions: not only about the cognitive content of images in science, but also about the high value that scientists ostensibly ascribe to imaging.

Nicola Mößner’s book Visual Representations in Science: Concept and Epistemology is a systematic analysis of the nature of visualisation in science and of its epistemic import. Its contribution to these discussions is highly relevant and well-informed.

I would like to summarise the book by four questions, each of which contains multiple possible ramifications:

  1. (1) What are scientific visualisations?
  2. (2) What justifies scientists’ reliance on visual information, in particular contexts, i.e. not only in the context of research but also in the context of communication with scientists in other fields and with the general public?
  3. (3) What kind of knowledge can we gain from visual representations? Does this include conceptual and/or propositional knowledge?
  4. (4) Can the knowledge that we obtain from visual representations be reduced to linguistically expressed knowledge?

This is a thoroughly argued and researched monograph. No topic required to address these questions is left undiscussed. Thus the book treats the reader to a wide journey across a vast philosophical landscape. Despite its density, I found the reading both rewarding and informative—as I hope to show in the next section. Section 2 offers a critique.

Mößner’s monograph is a timely call to philosophers to give thought to the (in her opinion, neglected) topic of visual representations and their epistemic import. She stresses that philosophers have traditionally regarded argumentation as a logical and linguistic matter. On this view, visual representations can only function as arguments in so far as they are, or at least can be, expressed linguistically. Already Plato (1997, 597d–598c) called the painter an imitator of appearances rather than truth, lacking the craftsman’s knowledge; and Frege (1956, 291–292) famously wrote that “what is improperly called the truth of pictures is reduced to the truth of sentences”, thus reducing image to language.

Mößner carefully navigates a number of recent philosophical debates that bear on these questions, thereby arguing for and developing her own balanced position. The exercise demonstrates how the apparently narrow topic of scientific visualisation can be an invitation to, at least, question some basic presuppositions in contemporary epistemology.

1 Content of the Monograph

After a brief introduction, Chapter 2 focusses on question (1) above, i.e. on the nature of scientific visualisations.

First, Mößner describes the extension of the term “scientific visualisation”. She describes four paradigmatic cases of visualisation in science, namely, photographs, imaging techniques (e.g. X-rays and functional magnetic resonance imaging or fMRI), data visualisations (e.g. visualisations of the Higgs particle), and diagrams (including graphs and drawings).

To adequately describe the characteristics of these types of visualisation, it is important to regard them as artefacts: in particular, one should take into account that these visualisations have been purposefully brought about to store and/or transmit information. She returns to this theme in Chapter 3.

She discusses this “informativeness” of visual representations about their object of depiction using Woodward’s interventionist account of causation. The use of this account has two main problems: (1) Observability is a matter of degree: and so, visual representations cannot always be used to make direct inferences about an object. In many cases, they are mere indicators of the properties of the object. (2) The context of usage plays a major role in ascribing meaning to an image—and her agreeing with this thesis leads in to her distinction of different contexts in Chapters 3 and 4.

Second, Mößner describes the intension of scientific representation: in particular, the nature of depiction. She does this by reviewing five types of picture theories, where pictures are defined as entities that depict (i.e. represent visually or pictorially) one or more objects or states of affairs. The question that these theories attempt to answer is: in virtue of what do pictures depict?

Thus Mößner summarises five theories: (a) resemblance theories (i.e. pictures depict in virtue of their resembling the subject matter, as e.g. in Plato and Peirce); (b) conventionalism (developed by Nelson Goodman: here, conventions are the result of negotiations, within a group, about how to visually represent particular entities); (c) experience-based theories (i.e. visual representations occasion a particular kind of visual experience); (d) recognition theories (i.e. a picture has the capacity to engage appropriate visual recognitional abilities); and (e) mixed theories. In mixed theories, pictures have a certain “two-foldness”: the pictorial “seeing-in” involves the veridical experience of seeing the picture surface and the non-veridical experience of seeing the depicted subject.

Mößner discusses the constraints that such picture theories must satisfy in order to explain how information is encoded and decoded. Mixed theories fit the bill best.

Thus, understanding an image—a specific type of artefact—implies grasping two meanings: the image’s content and the user’s intentions when using the image for communicative purposes.

Chapter 3 is about the functional roles of visual representations, and it answers question (2) above. Since visual representations are artefacts, the starting point is an analysis of the use of visual representations in science as communicative acts in which scientists bear in mind both the purpose of the visualisation and their target audience. With respect to the audience, there are two uses of visual representations (which Mößner also calls the “contexts” of the communicative act):

  1. (i) an exploratory use: in the genuine research activities; and
  2. (ii) an explanatory use: in the communication to peers, laymen, and in education.

In these two contexts, the visualisations in question do not relate the same relata: namely, in the first context, the visualisation makes the object of research epistemically accessible to scientists, while in the second it communicates the scientist’s intended interpretation of that research object.

The natural question is then (2)—“What justifies scientists’ reliance on visual information?”. Here, two sorts of reasons can be used:

  1. (a) Causality and informativeness: i.e. the scientists’ background knowledge about the instrument’s causal connection to the object of research, about its calibration, and about the relevant mapping function that explains how the image resembles the depicted entity.
  2. (b) Trust and reputation: in particular, the use of visual representations is embedded in the epistemic practice of testimony. However, the reasons that justify taking visual representations as evidence of course go beyond mere trust. The speaker’s intention, reputation, and professional ethics are also important.

The second point above leads in to considerations of social epistemology. Following Fleck, Mößner distinguishes between a thought collective (i.e. the people involved in a conversation) and a thought style (the collective’s unifying element). Fleck’s jargon can be used to describe the diversity of scientific images. Mößner distinguishes different types of communication, namely, intra-collective and inter-collective communication, which is mirrored by a variety of kinds of science: popular, textbook, handbook, and journal science. All of these have different aims and modes of scientific communication, which Mößner discusses.

Chapter 4 addresses questions (3) and (4) above, i.e. about the kind of knowledge that can be gained from visual representations, and whether this can be reduced to linguistically expressed knowledge.

The possibility of visual information is based on the human capacity of visual perception, as our primary sense. However, while perception is thought to be an admissible source of knowledge in scientific observations and experiments, it is questioned in the context of scientific arguments. One problem is that, in many instances, visualisations need to be interpreted, in order to correctly understand their content.

The bone of contention in this debate is whether the information transmitted by visual representations is propositional. For epistemologists understand “knowledge” primarily as “knowing-that”, i.e. as propositional knowledge. Mößner argues that it seems unlikely that scientists would lay such emphasis on visual representations if the latter were useless in the epistemic context of argumentation, i.e. if visual representations were unable to transmit propositional information.

But from the fact that visual content can be, at least in part, translated from one form to another, e.g. from a visual to a linguistic representation, it follows that visual representations can contain cognitively accessible content.

Furthermore, by embedding visual representations in communication, they can also be directly regarded as (parts of) assertions, without the need for a translation.

Debate then ensues about whether visual representations can themselves have truth values. Some authors argue that visual representations are truth makers, and not truth bearers. Others point out that e.g. micrographs represent states of affairs: and, since truth values are connected to states of affairs, there is a criterion of evaluation—hence truth values. Furthermore, at least some visualisations function as models, and so they make claims that can be true or false: they are epistemically relevant. Mößner points to the example of abortion debates, where the contending parties use visual representations as arguments: and so, images can refute each other.

Although there is among philosophers no consensus about the nature of perceptual content—whether it is propositional or non-propositional—, perception is generally seen to be an epistemic source: thus, to the question “How do you know that p?”, it is admissible to reply: “Because I saw that p.”

But if at least some visual information is translatable into language, perhaps all visual information is reducible to linguistically expressed information?

Mößner develops arguments that should give one some pause here. First, human observers are able to receive perceptual information even in cases in which they lack the concepts required to explain and describe that information. Furthermore, images can be used to transmit information about phenomena during the process of conceptualisation. Since scientific images are usually causally related to the object under investigation, they can contain non-conceptual information about the entity they represent. Thus we grasp such non-conceptual information through perception, and human observers can learn about those entities from images. Since non-conceptual information is acquired via perception, it also follows that at least some perceptual contents are not bound to concepts.

Mößner gives the example of a bird guide with drawings devised to learn to recognise birds in the wild. The resemblance relations are detected in the experiences of the drawing and the object (the particular bird) in the visual field of the perceiver. This comparison, i.e. considering the similarities and dissimilarities between the appearance of the bird that I remember and the image printed in the book, makes a classification possible—in a way that words could not teach us. Thus by using images as teaching tools, resemblance relations help us to conceptualise phenomena. This non-conceptual content of perception also explains how we are able to acquire observational concepts in the first place.

The above argument implies that our visual capabilities are not restricted to already conceptualised domains: our perceptual apparatus allows us to access non-conceptual, and thus non-propositional, content. Acknowledging the capacity of images to transmit non-conceptual content enables scientists to draw on the further merits of visual representations. Namely, scholars can communicate about phenomena that have not yet been completely conceptualised. Indeed, visual representations allow the elaboration of the initial concept itself.

Mößner uses the theory of dual coding to strengthen her case, according to which the human brain stores information in two distinct ways: visually and propositionally. Educational psychology suggests that presenting information visually is particularly helpful to students with low prior knowledge of the particular domain of science involved.

The thesis is further supported by the contrast between “knowing-that” and “knowing-how”, i.e. theoretical vs. practical knowledge. These two types of knowledge also have different aims: truth vs. success (at some activity). This puts—what Mößner calls—the “reductionist” account under further pressure, since a subject might know all the rules (e.g. about how to play the piano) and still be unable to carry out the activity in question (viz. playing the piano): there is a “knowledge-of-rules-to-action-gap”. Also, some of the actions we perform to reach goals are not guided by rules.

2 Critique

Mößner argues (in my opinion, successfully) that visual representations in science, given an appropriate context, can store and transmit both propositional and non-propositional, conceptual and non-conceptual knowledge. Her argument—framed as an extensive compilation of arguments from the literature, adding arguments of her own—is important and casts light on current scientific practice: see, for example, the discussion of the image of the black hole with which I began. But I disagree with some of Mößner’s arguments; I will here mention four (minor) points of content and three of structure and presentation.

First, concerning her main thesis I just mentioned: the abundance of quotes from the literature, which advocate very disparate positions, does not facilitate the clarity of Mößner’s own theses. The book’s discourse meanders between arguments and counter-arguments, in a way that sometimes makes it hard to (a) distinguish Mößner’s own thoughts from those of the literature and (b) discern the exact thesis she is arguing for. The text is erudite and largely written as a commentary to other philosophers’ work.

This leaves us with a number of questions that do not seem to have been clearly addressed: What, apart from the particular examples, distinguishes the cases where visual representations store and transmit propositional, as against non-propositional knowledge? How exactly do visual representations store and transmit propositional information, and what does the use of e.g. captions imply for the transmission of information in visual representations? A fuller theoretical discussion of Mößner’s own theses, independent of the literature, would have helped clarify the discussion.

The second point concerns the literature discussed. While the book successfully brings together various strands of literature (some of it only available in German), there are obvious omissions. For example, Mößner emphasises that philosophers (of science) have neglected visual representations. But two important examples that come to mind, and which Mößner does not mention at all, are Galison and Daston (2007) and Kaiser (2005).

Something else that seems lacking is a more pointed discussion of some of the semantic aspects of images, as against the epistemic aspects that Mößner emphasises. When speaking about translating from one representation to another, Mößner never specifies whether this translation is assumed to be merely syntactic or also respects the semantics. Without a discussion of this point, it is hard to follow claims, such as those in Chapter 3, that numeric representations of digital images are ontologically more basic than the images themselves. Also, one would have expected at least some discussion of recent work on representation in scientific theories and models, now only mentioned in a single footnote (see e.g. Frigg and Nguyen (2016)).

Third, her claim, towards the end of Chapter 4, that visualisation is the only way to acquire certain types of knowledge (e.g. that the only way to learn to discern the different birds is using a visual field guide or similar visual aid) does not seem to follow. 3 Although visual information, as in the case of the bird guide, can indeed be non-propositional, there may be other ways to cognize the same non-propositional knowledge, i.e. to establish the same biological classification (for example: by auditive means, smell, or by employing artificial devices). Mößner is convincing when she argues that there is visual information that cannot be transmitted linguistically, but she has not argued that visualisation is the only means and that there are no other non-linguistic means to transmit such information.

Finally, the concluding chapter (“Outlook. New responsibilities?”) first summarises the findings of the book and then discusses the responsibilities of the agents involved (usually, the scientists). The chapter, as an attempt to draw some normative conclusions, strikes me as a weak and as an unnecessary addition to an otherwise excellent monograph. To illustrate, we read things like: “producers and distributors of […] images should take into account the limited abilities of their lay audience to assess visually presented information”, “visual representations are a powerful means to influence public opinion”, “scientists should handle […] representational means thoughtfully […]. They should avoid the distribution of misleading images”. Assertions like these strike me as so common-sensical that they hardly require reading the rest of the monograph—the author does not make clear what her analysis adds to these platitudes.

This “outlook” section might have been more successful if instead it had pointed to the philosophical road ahead (about which one could indeed expect Mößner to have interesting things to say).

My three remaining qualms about the structure and presentation of the book are as follows.

First, it would have been desirable to increase the number of chapters and sections. Apart from brief introductory and conclusive chapters, the book consists of just three main chapters, each averaging 108 pages (!). Each chapter contains only two, and in one case three, main sections (setting aside the section’s summary). This makes the chapters and sections forbiddingly long.

Second, a book about visual representations might have made better use of illustrations and visual cues in the text. There are a good number of black-and-white illustrations, but they are not all of good quality. The text sometimes refers to illustrations in other texts that are not included in the book. And more efficient use of labels, numberings, and visual cues in the (already very long!) text would have increased its readability.

This leads in to my third point, about the use of summaries: each chapter indeed contains a “summary”, and each section contains a “discussion of interim results”. However, most of these are not, in fact, summaries at all, but rather further developments of the argument. They introduce new concepts and quote or cite new authors, so that the “summary” aspect is often lost. Combined with the dialectic style where the author’s own voice is usually hidden among the voices of other authors, this reduces the accessibility of the book.

My main qualms have been about structure and presentation not content—indeed Mößner’s line of argumentation is, in my opinion, solid. The monograph is an important contribution to this timely topic: a must-read for authors who wish to write, or simply learn, about visualisation in science.

References

  • Daston, Lorraine and Galison, Peter 2007. Objectivity. Cambridge, MA: MIT Press.

  • eht Collaboration 2019. “First M87 Event Horizon Telescope Results. i. The Shadow of the Supermassive Black Hole”. The Astrophysical Journal Letters 875: L1, 128.

    • Search Google Scholar
    • Export Citation
  • eht Collaboration 2019a. “First M87 Event Horizon Telescope Results. ii. Array and Instrumentation”. The Astrophysical Journal Letters 875: L2, 128.

    • Search Google Scholar
    • Export Citation
  • eht Collaboration 2019b. “First M87 Event Horizon Telescope Results. iii. Data Processing and Calibration”. The Astrophysical Journal Letters 875: L3, 132.

    • Search Google Scholar
    • Export Citation
  • eht Collaboration 2019c. “First M87 Event Horizon Telescope Results. iv. Imaging the Central Supermassive Black Hole”. The Astrophysical Journal Letters 875: L4, 152.

    • Search Google Scholar
    • Export Citation
  • eht Collaboration 2019d. “First M87 Event Horizon Telescope Results. v. Physical Origin of the Asymmetric Ring”. The Astrophysical Journal Letters 875: L5, 131.

    • Search Google Scholar
    • Export Citation
  • eht Collaboration 2019e. “First M87 Event Horizon Telescope Results. vi. The Shadow and Mass of the Central Black Hole”. The Astrophysical Journal Letters 875: L6, 144.

    • Search Google Scholar
    • Export Citation
  • Frege, Gottlob 1956. “The Thought. A Logical Inquiry”. Mind 65 (259), 289311.

  • Frigg, Roman and Nguyen, James , “Scientific Representation”, The Stanford Encyclopedia of Philosophy (2016 Edition), Edward N. Zalta (ed.), url = <https://plato.stanford.edu/archives/spr2020/entries/scientific-representation/>.

    • Search Google Scholar
    • Export Citation
  • Kaiser, David 2005. Drawing Theories Apart. Chicago: The University of Chicago Press.

  • Plato 1997. Complete Works. Edited by John M. Cooper . Indianapolis/Cambridge: Hackett Publishing Company.

2

The scientific papers published by the collaboration do mention the testing of various general relativity effects at horizon scales as the main scientific goal of the eht array ( eht Collaboration, 2019a, 2). Nevertheless, out of the six papers published by the collaboration around the April 2019 announcement, four papers discuss, respectively, the image, the instrument, the data reduction, and the M87 shadow ( eht Collaboration, 2019–2019c). Only two papers discuss the theoretical models and the black hole mass estimate (2019d–e).

3

Mößner mentions here an analogy with the Mary-argument in the philosophy of mind, but in my opinion the two cases are very different, since that example argues for the existence of individually perceived qualia, while the bird guide aims to establish an objective biological classification.

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 408 59 8
PDF Views & Downloads 467 72 4