Search Results
Abstract
There has been a rapid growth of interest amongst researchers in the cross-modal correspondences in recent years. In part, this has resulted from the emerging realization of the important role that the correspondences can sometimes play in multisensory integration. In turn, this has led to an interest in the nature of any differences between individuals, or rather, between groups of individuals, in the strength and/or consensuality of cross-modal correspondences that may be observed in both neurotypically normal groups cross-culturally, developmentally, and across various special populations (including those who have lost a sense, as well as those with autistic tendencies). The hope is that our emerging understanding of such group differences may one day provide grounds for supporting the reality of the various different types of correspondence that have so far been proposed, namely structural, statistical, semantic, and hedonic (or emotionally mediated).
The last 50 years or so has seen great optimism concerning the potential of sensory substitution and augmentation devices to enhance the lives of those with (or even those without) some form of sensory loss (in practice, this has typically meant those who are blind or suffering from low vision). One commonly discussed solution for those individuals who are blind has been to use one of a range of tactile–visual sensory substitution systems that represent objects captured by a camera as outline images on the skin surface in real-time (what Loomis, Klatzky and Giudice, , term general-purpose sensory substitution devices). However, despite the fact that touch, like vision, initially codes information spatiotopically, I would like to argue that a number of fundamental perceptual, attentional, and cognitive limitations constraining the processing of tactile information mean that the skin surface is unlikely ever to provide such general-purpose sensory substitution capabilities. At present, there is little evidence to suggest that the extensive cortical plasticity that has been demonstrated in those who have lost (or never had) a sense can do much to overcome the limitations associated with trying to perceive high rates of spatiotemporally varying information presented via the skin surface (no matter whether that surface be the back, stomach, forehead, or tongue). Instead, the use of the skin will likely be restricted to various special-purpose devices that enable specific activities such as navigation, the control of locomotion, pattern perception, etc.
Abstract
This review deals with the question of the relative vs absolute nature of crossmodal correspondences, with a specific focus on those correspondences involving the auditory dimension of pitch. Crossmodal correspondences have been defined as the often-surprising crossmodal associations that people experience between features, attributes, or dimensions of experience in different sensory modalities, when either physically present, or else merely imagined. In the literature, crossmodal correspondences have often been contrasted with synaesthesia in that the former are frequently said to be relative phenomena (e.g., it is the higher-pitched of two sounds that is matched with the smaller of two visual stimuli, say, rather than there being a specific one-to-one crossmodal mapping between a particular pitch of sound and size of object). By contrast, in the case of synaesthesia, the idiosyncratic mapping between inducer and concurrent tends to be absolute (e.g., it is a particular sonic inducer that elicits a specific colour concurrent). However, a closer analysis of the literature soon reveals that the distinction between relative and absolute in the case of crossmodal correspondences may not be as clear-cut as some commentators would have us believe. Furthermore, it is important to note that the relative vs absolute question may receive different answers depending on the particular (class of) correspondence under empirical investigation.
Abstract
A wide variety of crossmodal correspondences, defined as the often surprising connections that people appear to experience between simple features, attributes, or dimensions of experience, either physically present or else merely imagined, in different sensory modalities, have been demonstrated in recent years. However, a number of crossmodal correspondences have also been documented between more complex (i.e., multi-component) stimuli, such as, for example, pieces of music and paintings. In this review, the extensive evidence supporting the emotional mediation account of the crossmodal correspondences between musical stimuli (mostly pre-recorded short classical music excerpts) and visual stimuli, including colour patches through to, on occasion, paintings, is critically evaluated. According to the emotional mediation account, it is the emotional associations that people have with stimuli that constitutes one of the fundamental bases on which crossmodal associations are established. Taken together, the literature that has been published to date supports emotional mediation as one of the key factors underlying the crossmodal correspondences involving emotionally-valenced stimuli, both simple and complex.
Abstract
We often estimate, or perceive, the quality of materials, surfaces, and objects, what the Japanese refer to as ‘shitsukan’, by means of several of our senses. The majority of the literature on shitsukan perception has, though, tended to focus on the unimodal visual evaluation of stimulus properties. In part, this presumably reflects the widespread hegemony of the visual in the modern era and, in part, is a result of the growing interest, not to mention the impressive advances, in digital rendering amongst the computer graphics community. Nevertheless, regardless of such an oculocentric bias in so much of the empirical literature, it is important to note that several other senses often do contribute to the impression of the material quality of surfaces, materials, and objects as experienced in the real world, rather than just in virtual reality. Understanding the multisensory contributions to the perception of material quality, especially when combined with computational and neural data, is likely to have implications for a number of fields of basic research as well as being applicable to emerging domains such as, for example, multisensory augmented retail, not to mention multisensory packaging design.
Abstract
The failure of e-books to take over from the traditional print format, as was so confidently predicted would happen only a few years ago, highlights how there is more to reading than merely the content of what we see. In fact, like any other object, the experience of interacting with a book, especially an old or historic volume, offers the reader the potential for a multisensory encounter. One that involves not only what the book looks and feels like, both the weight of the volume and the feel of the pages, but also the distinctive smell. In fact, one might also want to consider the particular sound made by the pages as they are turned over. However, it is the smell of older, and seemingly more olfactorily-redolent, works that appears to be especially effective at triggering nostalgic associations amongst readers. It is therefore only by understanding the multisensory nature of handling books, as stressed by this review, that one can really hope to fully appreciate the enduring appeal of the traditional format in the modern digital era. Several recent exhibitions that have attempted to engage their visitors by means of exploring the multisensory appeal of historic books or manuscripts in their collections are briefly discussed. While the multisensory mental imagery that is typically evoked by reading is unlikely to differ much between the print and e-book formats, there is nevertheless still some evidence to suggest that physical books can occasionally convey information more effectively than their digital counterparts.
Abstract
The last few years have seen an explosive growth of research interest in the crossmodal correspondences, the sometimes surprising associations that people experience between stimuli, attributes, or perceptual dimensions, such as between auditory pitch and visual size, or elevation. To date, the majority of this research has tended to focus on audiovisual correspondences. However, a variety of crossmodal correspondences have also been demonstrated with tactile stimuli, involving everything from felt shape to texture, and from weight through to temperature. In this review, I take a closer look at temperature-based correspondences. The empirical research not only supports the existence of robust crossmodal correspondences between temperature and colour (as captured by everyday phrases such as ‘red hot’) but also between temperature and auditory pitch. Importantly, such correspondences have (on occasion) been shown to influence everything from our thermal comfort in coloured environments through to our response to the thermal and chemical warmth associated with stimulation of the chemical senses, as when eating, drinking, and sniffing olfactory stimuli. Temperature-based correspondences are considered in terms of the four main classes of correspondence that have been identified to date, namely statistical, structural, semantic, and affective. The hope is that gaining a better understanding of temperature-based crossmodal correspondences may one day also potentially help in the design of more intuitive sensory-substitution devices, and support the delivery of immersive virtual and augmented reality experiences.