The failure of e-books to take over from the traditional print format, as was so confidently predicted would happen only a few years ago, highlights how there is more to reading than merely the content of what we see. In fact, like any other object, the experience of interacting with a book, especially an old or historic volume, offers the reader the potential for a multisensory encounter. One that involves not only what the book looks and feels like, both the weight of the volume and the feel of the pages, but also the distinctive smell. In fact, one might also want to consider the particular sound made by the pages as they are turned over. However, it is the smell of older, and seemingly more olfactorily-redolent, works that appears to be especially effective at triggering nostalgic associations amongst readers. It is therefore only by understanding the multisensory nature of handling books, as stressed by this review, that one can really hope to fully appreciate the enduring appeal of the traditional format in the modern digital era. Several recent exhibitions that have attempted to engage their visitors by means of exploring the multisensory appeal of historic books or manuscripts in their collections are briefly discussed. While the multisensory mental imagery that is typically evoked by reading is unlikely to differ much between the print and e-book formats, there is nevertheless still some evidence to suggest that physical books can occasionally convey information more effectively than their digital counterparts.
We often estimate, or perceive, the quality of materials, surfaces, and objects, what the Japanese refer to as ‘shitsukan’, by means of several of our senses. The majority of the literature on shitsukan perception has, though, tended to focus on the unimodal visual evaluation of stimulus properties. In part, this presumably reflects the widespread hegemony of the visual in the modern era and, in part, is a result of the growing interest, not to mention the impressive advances, in digital rendering amongst the computer graphics community. Nevertheless, regardless of such an oculocentric bias in so much of the empirical literature, it is important to note that several other senses often do contribute to the impression of the material quality of surfaces, materials, and objects as experienced in the real world, rather than just in virtual reality. Understanding the multisensory contributions to the perception of material quality, especially when combined with computational and neural data, is likely to have implications for a number of fields of basic research as well as being applicable to emerging domains such as, for example, multisensory augmented retail, not to mention multisensory packaging design.
The last few years have seen an explosive growth of research interest in the crossmodal correspondences, the sometimes surprising associations that people experience between stimuli, attributes, or perceptual dimensions, such as between auditory pitch and visual size, or elevation. To date, the majority of this research has tended to focus on audiovisual correspondences. However, a variety of crossmodal correspondences have also been demonstrated with tactile stimuli, involving everything from felt shape to texture, and from weight through to temperature. In this review, I take a closer look at temperature-based correspondences. The empirical research not only supports the existence of robust crossmodal correspondences between temperature and colour (as captured by everyday phrases such as ‘red hot’) but also between temperature and auditory pitch. Importantly, such correspondences have (on occasion) been shown to influence everything from our thermal comfort in coloured environments through to our response to the thermal and chemical warmth associated with stimulation of the chemical senses, as when eating, drinking, and sniffing olfactory stimuli. Temperature-based correspondences are considered in terms of the four main classes of correspondence that have been identified to date, namely statistical, structural, semantic, and affective. The hope is that gaining a better understanding of temperature-based crossmodal correspondences may one day also potentially help in the design of more intuitive sensory-substitution devices, and support the delivery of immersive virtual and augmented reality experiences.
This review deals with the question of the relative vs absolute nature of crossmodal correspondences, with a specific focus on those correspondences involving the auditory dimension of pitch. Crossmodal correspondences have been defined as the often-surprising crossmodal associations that people experience between features, attributes, or dimensions of experience in different sensory modalities, when either physically present, or else merely imagined. In the literature, crossmodal correspondences have often been contrasted with synaesthesia in that the former are frequently said to be relative phenomena (e.g., it is the higher-pitched of two sounds that is matched with the smaller of two visual stimuli, say, rather than there being a specific one-to-one crossmodal mapping between a particular pitch of sound and size of object). By contrast, in the case of synaesthesia, the idiosyncratic mapping between inducer and concurrent tends to be absolute (e.g., it is a particular sonic inducer that elicits a specific colour concurrent). However, a closer analysis of the literature soon reveals that the distinction between relative and absolute in the case of crossmodal correspondences may not be as clear-cut as some commentators would have us believe. Furthermore, it is important to note that the relative vs absolute question may receive different answers depending on the particular (class of) correspondence under empirical investigation.
Theorizing around the topic of attention and its role in human information processing largely emerged out of research on the so-called spatial senses: vision, audition, and to a lesser extent, touch. Thus far, the chemical senses have received far less research interest (or should that be attention) from those experimental psychologists and cognitive neuroscientists interested in the topic. Nevertheless, this review highlights the key role that attentional selection also plays in chemosensory information processing and awareness. Indeed, many of the same theoretical approaches/experimental paradigms that were originally developed in the context of the spatial senses, can be (and in some cases already have been) extended to provide a useful framework for thinking about the perception of taste/flavour. Furthermore, a number of those creative individuals interested in modifying the perception of taste/flavour by manipulating product-extrinsic cues (such as, for example, music in the case of sonic seasoning) are increasingly looking to attentional accounts in order to help explain the empirical phenomena that they are starting to uncover. However, separate from its role in explaining sonic seasoning, gaining a better understanding of the role of attentional distraction in modulating our eating/drinking behaviours really ought to be a topic of growing societal concern. This is because distracted diners (e.g., those who eat while watching TV, fiddling with a mobile device or smartphone, or even while driving) consume significantly more than those who mindfully pay attention to the sensations associated with eating and drinking.
A wide variety of crossmodal correspondences, defined as the often surprising connections that people appear to experience between simple features, attributes, or dimensions of experience, either physically present or else merely imagined, in different sensory modalities, have been demonstrated in recent years. However, a number of crossmodal correspondences have also been documented between more complex (i.e., multi-component) stimuli, such as, for example, pieces of music and paintings. In this review, the extensive evidence supporting the emotional mediation account of the crossmodal correspondences between musical stimuli (mostly pre-recorded short classical music excerpts) and visual stimuli, including colour patches through to, on occasion, paintings, is critically evaluated. According to the emotional mediation account, it is the emotional associations that people have with stimuli that constitutes one of the fundamental bases on which crossmodal associations are established. Taken together, the literature that has been published to date supports emotional mediation as one of the key factors underlying the crossmodal correspondences involving emotionally-valenced stimuli, both simple and complex.