Emotional and Semantic Associations Between Cinematographic Aesthetics and Haptic Perception

In: Multisensory Research
View More View Less
  • 1 Moscow State University, Faculty of Psychology, Mokhovaya st. 11/9 125009 Moscow, Russia
  • 2 Center for Experimental Psychology, Moscow State University of Psychology and Education, 2a Shelepikhinskaya Quay, 123290 Moscow, Russia
  • 3 VTB Capital, 12, Presnenskaya emb. 123100 Moscow, Russia

This study investigates systematic links between haptic perception and multimodal cinema perception. It differs from previous research conducted on cross-modal associations as it focuses on a complex intermodal stimulus, close to one people experience in reality: cinema. Participants chose materials that are most/least consistent with three-minute samples of films with elements of beauty and ugliness. We found that specific materials are associated with certain films significantly different from chance. Silk was associated with films including elements of beauty, while sandpaper was associated with films including elements of ugliness. To investigate the nature of this phenomenon, we tested the mediation effect of emotional/semantic representations on cinema–haptic associations. We found that affective representations at least partly explain the cross-modal associations between films and materials.

Abstract

This study investigates systematic links between haptic perception and multimodal cinema perception. It differs from previous research conducted on cross-modal associations as it focuses on a complex intermodal stimulus, close to one people experience in reality: cinema. Participants chose materials that are most/least consistent with three-minute samples of films with elements of beauty and ugliness. We found that specific materials are associated with certain films significantly different from chance. Silk was associated with films including elements of beauty, while sandpaper was associated with films including elements of ugliness. To investigate the nature of this phenomenon, we tested the mediation effect of emotional/semantic representations on cinema–haptic associations. We found that affective representations at least partly explain the cross-modal associations between films and materials.

1. Introduction

Cross-modal associations appear when sensations from one modality (e.g., high-pitched sounds) are associated with sensations from another modality (e.g., bright colors). Investigating the nature of these associations is important, since it contributes to our understanding of multisensory perception (ability to integrate information from multiple sensory modalities).

Four mechanisms may explain cross-modal associations (see Spence, 2011). Firstly, cross-modal associations may appear because of structural correspondences of stimuli (e.g., bright colors and loud voices have similar magnitude and thus are associated). Secondly, cross-modal associations can be determined by statistical correspondences of stimuli (e.g., big shapes are associated with loud sounds because big objects often produce loud sounds). Thirdly, cross-modal associations may appear due to semantic correspondences — a similar descriptive terminology of sensations from different modalities (e.g., the word ‘warm’ can be attributed to a tactile sensation as well as to a color). Finally, emotional processing itself can be a mechanism that underlies cross-modal associations.

The idea that sensations from different modalities are associated with each other due to emotional processing was developed in the second half of the nineteenth century. Flournoy supposed that sensations from different modalities, colors and sounds, having a similar emotional content, are strongly associated with each other (Flournoy, 1893, pp. 20–37). Today numerous empirical studies show that emotional processing at least partly mediates associations between stimuli of different modalities.

Emotional representations of happiness, sadness, anger, calmness, etc. mediate associations between music and colors (Palmer et al., 2013), odors and music (Levitan et al., 2015), odors and colors (Guerdoux et al., 2014; Levitan et al., 2014; Schifferstein and Tanudjaja, 2004), and tastes and shapes (Velasco et al., 2016). In the above-mentioned studies, participants were asked to rate sensations from different modalities (e.g., colors and sounds) on different semantic/emotional scales (e.g., happy, sad etc.). They were also asked to directly associate the sensations from different modalities with each other (e.g., colors with sounds). It turned out that if two sensations from different modalities are scaled similarly on these semantic/emotional scales, they are more strongly associated with each other.

In the present study the primary goal was to investigate cross-modal associations between multimodal cinema perception and unimodal haptic perception. Cross-modal associations between haptic perception and visual/auditory modalities have already been established. However, synesthesia involving haptic perception and auditory/visual modalities is quite uncommon; such cases have been documented in the literature (Beauchamp and Ro, 2008; Day, 2005; Simner and Ludwig, 2012; Steven et al., 2006). We also know about cross-modal associations between touch and other simple stimuli from different modalities among non-synesthetes: touch and color (softness and smoothness are associated with luminance, Ludwig and Simner, 2013; Slobodenyuk et al., 2015), touch and non-words/words (smooth materials are associated with round-shaped sounds, rougher materials with sharp-transient sounds — Etzi et al., 2016). Beyond psychological research, futurists supposed that cross-modal associations between haptic perception and taste evoke specific aesthetic experience (‘tactile dinner parties’, Marinetti, 1989). However, systematic links between haptic perception and multimodal cinema perception have not been confirmed yet.

In our opinion, a study of cross-modal associations with cinema is important, since cinema is very close to one of those complex and intermodal stimuli people usually experience during their life. It is known that we face complex stimuli much more often than simple ones. For example, we perceive not just a color, but different colors and shapes which may contrast with each other. In this context art is a particularly interesting object which can be used as a complex stimulus in such studies (e.g., painting as complex visual stimulus, music as complex auditory stimulus). As a synthetic art, cinema combines different stimuli of auditory and visual modalities: sounds, music, colors, shapes, movement. The effects of cinema on human cognition and brain activity are investigated in such a multidisciplinary field of study as neurocinematics (Hasson et al., 2008).

The synchrony of visual and auditory montage is often used to accentuate moments important for movie understanding. For example, Eisenstein explained that filmmakers do use music-color correlation to evoke specific emotions among the viewers (Afra, 2015; Eisenstein, 1940). The interaction of different modalities in cinema is important, since it is related to the sense of presence. Philosophers of mind and arts have developed principles of this multisensory film experience, including more than just visual and auditory modalities (Astrinaki, 2012). For example, vestibular processing is used by the filmmakers to determine what the spectator sees and hears in the movie and what he/she understands about the film characters’ identity (Antunes, 2012).

The secondary target of the present study was to investigate the nature of cross-modal associations due to emotional/semantic associations between movie scenes with different aesthetic content and haptic stimuli. We therefore tested whether cross-modal associations between haptic and cinema perception are mediated by a semantic/emotional association. Answering this question, we may contribute not only to understanding the nature of cross-modal associations and multisensory perception, but also to the development of an implicit measure of the aesthetic experience of movies.

Among other tests performed we analyzed whether particular haptic sensations are associated with elements of aesthetics of beauty and ugliness in cinema. Beauty and ugliness are among the fundamental notions in aesthetics. Beauty is related to perfection and harmony. Ugliness stands in opposition to beauty; however, that does not mean that ugliness is not aesthetically valuable. Some philosophers define ugliness as a disharmony, which captures attention and imagination (Kant, 2008; Wenzel, 1999) or as ‘complicated beauty’ in contrast to ‘easy beauty’ (Bosanquet, 1914).

2. Materials and Methods

Eighteen volunteers (14 females and four males aged 20–53, Mage=28, SD = 8.21), recruited at the Department of Psychology at Moscow State University, participated in the study. The sample size was based on our pilot study results: with N=17 the effect size was 0.475 (Cramer’s V). Informed consent was obtained; the experiment has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki).

We used eight materials as haptic stimuli: natural silk, wood (spruce), rabbit fur, metal (aluminum), glass, natural leather, rubber, sandpaper. These materials varied on two of the main dimensions of touch experience, investigated in psychophysics research: smoothness (smooth vs. rough) and hardness (hard vs. soft; Hollins et al., 1993; Picard et al., 2003; Yoshida, 1968).

Three films with elements of aesthetics of beauty (Wong Kar-wai’s ‘2046’, Gennady Shpalikov’s ‘Long and Happy Life’, Lucino Visconti’s ‘Death in Venice’) and three films with elements of aesthetics of ugliness (Peter Greenaway’s ‘A Zed and Two Noughts’, Aleksei German’s ‘Hard to Be a God’, Marco Ferreri’s ‘The Grande Bouffe’) were picked by an expert in aesthetics. As already mentioned, the elements of ugliness are not supposed to diminish the aesthetic value of the film. All the films used in our experiment are films of highly recognized film directors and have received prestigious awards at film festivals.

Elements of beauty and ugliness are very often mixed in one film, just like minor and major modes can be mixed in one music piece. We did not show the whole film to our participants. Instead we presented a three-minute sample. Apart from the fact that each sample/scene contained elements of beauty (or ugliness), all samples were logically isolated (with its own beginning, culmination, and end). Finally, in choosing ‘beautiful’ and ‘ugly’ films and the scenes in them, we tried to achieve uniformity in these stimuli. For example, coloratura (one black-and-white film with elements of beauty, one black-and-white film with elements of ugliness), musical coloratura (one film with a musical theme with elements of beauty and one with elements of ugliness). We also asked our participants to rate each film for emotional intensity using a feeling thermometer (on a scale from 0 to 100) and for aesthetic property (on a scale from 0 for aesthetically not appealing to 100 for aesthetically very appealing) to check if there is a difference between the films with elements of beauty and ugliness. A Kruskal–Wallis test showed that there was no significant difference in aesthetic evaluation between ‘beautiful’ and ‘ugly’ films, χ2(1)=2.843, p=0.092 (mean rank score of 59.56 for ‘beautiful’ films and 49.44 for ‘ugly’ films), nor in their emotional intensity score, χ2(1)=1.565, p=0.211 (mean rank score of 50.74 for ‘beautiful’ films and 58.26 for ‘ugly’ films).

We used Osgood’s semantic differential technique to evaluate cinema and tactile experience. This technique measures affective responses to different stimuli by the participant’s rating of these stimuli on a seven-point bipolar adjective scale (from −3 to +3). The following bipolar scales were used: good/bad, dirty/clean, young/old, stupid/clever, loud/quiet, fast/slow, disgusting/nice, active/passive, sweet/bitter, brave/cowardly, happy/sad, strong/weak, hungry/full (Osgood et al., 1957; Snider and Osgood, 1969). Artemieva (2007) showed that these 13 scales have insignificant dispersion when participants evaluate visual stimuli (shapes). Moreover, there is a significant retest correlation in the evaluation of shapes using these scales (Artemieva, 2007).

2.1. Procedure

The experiment was performed in a laboratory room in daylight. The films were presented on a notebook screen in a window of 10 × 14 cm without headphones. The auditory complexation (Reatlek High Definition Audio) was 16 bits, 44 100 Hz (60% of 100). The resolution of the display (monitor PnP, Intel® HD Graphics) was set to the maximum possible, 1366 × 768 pixels. The materials (9 cm length, 9 cm height) were fixed on a plank (110 cm). The distance between the materials on the plank was 4.3 cm. The plank and the notebook were placed on a desk in front of the participant. The distance between the display and the eyes of the participant was about 60 cm.

We first asked our participants to scale each material they were touching on the 13 Osgood’s bipolar scales. The participants used a sleeping mask when touching the materials to eliminate visual stimulation during the experiment. We next showed our participants the first film sample and asked them to rank materials from best to worst associated with it. We asked the participants to rate the sample on the 13 scales as well. The same process was followed for the remaining film samples. During the experiment, the order of the materials on the plank was randomly changed. The experimenters did not name either the materials or the films and their directors to the participants, until the end of the experiment. After the experiment the participants were asked if they had seen these films before the experiment, as this could affect their experience during the experiment. Three participants reported to already have seen one film in six; only one participant reported she had already seen three films.

2.2. Statistical Analysis

2.2.1. Descriptive Statistics

We calculated the mean ratings of films and materials on the 13 semantic differential scales. The chi-square statistic was used to test if films and materials were scaled significantly different from chance. Because of the small sample size, we decided to rearrange the seven-point scale (−3, +3) to a three-point scale (−1, +1).

2.2.2. Cinema–Haptic Cross-Modal Association Framework

We used non-parametric chi-square statistics to test whether there was a cinema–haptic association. In deriving our expectations, we used the two following models, each with two variables: films and materials ‘best associated’ with them (materials ranked first) and films and materials ‘least associated’ with the films (materials ranked eighth). To define the models, we calculated the mean strength of association for each film–material pair defined as the simple average of the chosen material ratings for the related film across all 18 participants. As there were eight materials to associate with each film, the highest rating between the film and the material was set to be eight points, and the smallest one point. The results were tested using Friedman’s test to define the existing statistically significant differences for film–material associations. To examine what were the differences, we used post-hoc analysis with Wilcoxon’s signed-rank tests conducted using Benjamini and Hochberg’s procedure for multiple comparisons to control the false discovery rate (FDR).

2.2.3. Aesthetics of Beauty/Ugliness–Haptic Cross-Modal Association Framework

For the purpose of analysis, we combined data obtained for films with elements of beauty (Wong Kar-wai’s ‘2046’, Gennady Shpalikov’s ‘Long and Happy Life’, Lucino Visconti’s ‘Death in Venice’) into one dataset. The same was done for the films with elements of ugliness (Peter Greenaway’s ‘A Zed and Two Noughts’, Aleksei German’s ‘Hard to Be a God’, Marco Ferreri’s ‘The Grande Bouffe’). The same statistical analysis as described above was used to investigate whether materials were associated with elements of beauty and ugliness, except that instead of Friedman’s test and a post-hoc analysis we used only Wilcoxon’s signed-rank test with a Benjamini and Hochberg procedure for multiple comparisons.

2.2.4. Emotional/Semantic Basis of Cinema–Haptic Associations Testing Framework

A correlation analysis was performed to test the mediation effect of emotional/semantic representations on cinema–haptic cross-modal associations. We calculated correlations between the direct evaluation of six films through semantic differential scales and indirect evaluation of these films through the CHP (cinema–haptic perception) index. Conceptually, the CHP index [analogous to the music-association (MCA) score developed by Palmer and colleagues, 2013) for a given film on a given semantic differential scale (say ‘dirty/clean’) is a linearly weighted average of the chosen scale ratings of the three materials chosen as most consistent with that film minus an analogous weighted average of the chosen scale ratings of the three materials ratings chosen to be least consistent with that film. The significance of the correlation between the direct and indirect film evaluations was tested using Wilcoxon’s signed-rank test using the Benjamini and Hochberg procedure for multiple comparisons to control the FDR.

Finally, we wanted to discover whether the number of scales on which the material and the film were rated similarly mattered for the strength of the cross-modal associations, so we conducted a correlation analysis between the strength of the cross-modal associations and the way the film and the material were scaled on the 13 semantic differential scales (Spearman correlation coefficients).

The above analyses were performed using IBM SPSS Statistics v. 20.0 (Armonk, NY).

3. Results

The mean ratings of films and materials on the 13 semantic differential scales varied between −2.83 and 2.11 (see Table 1). The value −2.83 represents the association between the negative adjective ‘dirty’ and the German film (‘Hard to Be a God’). The value 2.11 was scored for the association between the positive adjective ‘good’ and the Visconti film (‘Death in Venice’).

Table 1.

Mean values of films and materials on 13 semantic differential scales. In bold: significant values according to a chi-square test

Table 1.

3.1. Cinema–Haptic Cross-Modal Association Testing Results

We performed chi-square tests to reveal possible cinema–haptic associations. The chi-square test used the data from the contingency tables based on (film)/(associated material) information. Such an association was found for film–boundary ranked material (first or eighth rank) pairs: χ2(35)=62.45, p=0.003, Cramer’s V=0.340 and χ2(30)=51.08, p=0.010, Cramer’s V=0.308, respectively.

We calculated the strength of associations between materials and films (see Table 2). As can be seen from Table 2, the strongest association was between the Visconti film (‘Death in Venice’) and silk, while the weakest was between the Kar-wai film (‘2046’) and fur and the Visconti film (‘Death in Venice’) and sandpaper.

Table 2.

Strength of associations (means and standard deviations) between materials and films

Table 2.

The Fridman test revealed that the cinema–haptic association for the materials described below differs depending on the movie tested and that this difference is statistically significantly different from chance. The Friedman test was significant for the following materials: sandpaper [χ2(5)=22.936, p=0.0001] and silk [χ2(5)=20.342, p=0.001]. The associations between six films and the other materials were not significantly different after applying Benjamini and Hochberg’s procedure for multiple comparisons: metal [χ2(5)=12.215, p=0.032], glass [χ2(5)=11.649, p=0.040], fur [χ2(5)=13.348, p=0.020], rubber [χ2(5)=10.178, p=0.070], wood [χ2(5)=5.561, p=0.351] and leather [χ2(5)=4.509, p=0.479].

Post-hoc analysis revealed that sandpaper was stronger associated with the German film (‘Hard to Be a God) than with all other films: Kar-wai (‘2046’; Z=2.417, p=0.016, r=0.403), Greenway (‘A Zed and Two Noughts’; Z=2.967, p=0.003, r=0.494), Visconti (‘Death in Venice’; Z=3.331, p=0.001, r=0.555), Shpalikov (‘Long and Happy Life’; Z=3.280, p=0.001, r=0.547) and Ferreri (‘The Grande Bouffe’; Z=2.623, p=0.009, r=0.437). Silk was stronger associated with the Visconti film (‘Death in Venice’) than with four other films: Greenway (‘A Zed and Two Noughts’; Z=2.854, p=0.004, r=0.476), German (‘Hard to Be a God’; Z=2.968, p=0.003, r=0.495), Shpalikov (‘Long and Happy Life’; Z=2.455, p=0.014, r=0.409) and Ferreri (‘The Grande Bouffe’; Z=2.849, p=0.004, r=0.474). Silk was also stronger associated with the Kar-wai film (‘2046’) than with the Greenway (‘A Zed and Two Noughts’; Z=2.556, p=0.011, r=0.426), and Ferreri films (‘The Grande Bouffe’; Z=2.857, p=0.004, r=0.476).

3.2. Aesthetics of Beauty/Ugliness–Haptic Cross-Modal Association Testing Results

We calculated the strength of the associations between materials and these films (see Table 3). The strongest associations were between the ‘beautiful’ films and silk and the ‘ugly’ films and rubber. The weakest associations were between sandpaper and ‘beautiful’ films and fur and ‘ugly’ films. There was a systematic association between materials ‘best associated’ with films with elements of beauty and ugliness [χ2(7)=16.01, p=0.025, Cramer’s V=0.385]. There was also a systematic association between materials ‘least associated’ with films with elements of beauty and ugliness [χ2(6)=20.78, p=0.002, Cramer’s V=0.439].

Table 3.

Strength of associations (means and standard deviations) between materials and films with elements of beauty and ugliness

Table 3.

Wilcoxon’s signed-rank test with a Benjamini and Hochberg procedure for multiple comparisons showed that mean scores of associations between ‘beautiful’ and ‘ugly’ films and the following materials were statistically significantly different: sandpaper (Z=3.025, p=0.002, r=0.288) and silk (Z=3.889, p=0.0001, r=0.301). Sandpaper was more associated with ‘ugly’ films compared to ‘beautiful’ ones. Silk was more associated with ‘beautiful’ films compared to ‘ugly’ ones. Associations between the other materials and films were not significant: metal (Z=2.216, p=0.027, r=0.211), rubber (Z=1.552, p=0.121, r=0.148), wood (Z=1.672, p=0.094, r=0.159), glass (Z=0.264, p=0.792, r=0.025), leather (Z=0.912, p=0.362, r=0.087) and fur (Z=0.669, p=0.503, r=0.064).

3.3. Emotional/Semantic Basis of Cinema–Haptic Associations Testing Results

A correlation analysis, testing the role of emotional/semantic representations in cross-modal associations, revealed that the following six scales had significant p-values and thus mediate cinema–haptic associations: good–bad (p=0.001), dirty–clean (p=0.01), young–old (p=0.007), nice–disgusting (p=0.004), sweet–bitter (p=0.006), happy–sad (p=0.02). The rest of the scales had no significant p-values: stupid–clever (p=0.523), loud–quiet (p=0.777), slow–quick (p=0.266), active–passive (p=0.528), brave–cowardly (p=0.078), strong–weak (p=0.267), hungry–full (p=0.187).

Finally, we tested the following hypothesis: the bigger the number of semantic scales on which the film and the material are scaled similarly, the stronger are the cross-modal associations between them. We conducted a correlation analysis between the strength of the cross-modal associations (Table 2) and the way the film and the material were scaled on the 13 semantic differential scales (Spearman correlation coefficients). The correlation coefficient was significant: r=0.470, p=0.001.

4. Discussion

To our knowledge, this is the first study investigating cross-modal associations between haptic perception and cinema. We found that the associations between films and materials were not random. That is, specific materials are associated with certain films significantly different from chance. For example, silk and fur were strongly associated with the Visconti film (‘Death in Venice’). All of them were rated as good, clean, quiet and nice. On the other hand, sandpaper, rated as bad, dirty, loud and bitter, was weakly associated with that film.

Our result contributes to research of cross-modal associations between simple and complex stimuli from different modalities; it is the first study that shows that cross-modal associations exist between complex intermodal stimuli (cinema) and simple stimuli from one modality (haptic perception).

We further investigated whether certain materials are associated with films with elements of beauty and ugliness. We could confirm that hypothesis. According to our results, silk is more associated with ‘beautiful’ films, while sandpaper is with ‘ugly’ films. Probably, this result corresponds to the finding of Etzi et al. (2016); they found that smoother materials are associated with the label ‘beautiful’, while rougher materials are associated with the label ‘ugly’. However, it is also possible that smoothness is not the only driver for that kind of cross-modal association.

Although metal did not have a statistically significant association with any film, there was a tendency of associating metal with films including elements of ugliness, and there was a tendency of associating wood with films including elements of beauty. In this case, it is possible that ecological organic materials (silk and wood) are more strongly associated with films including elements of beauty, while artificial non-organic materials (metal and sandpaper) are more associated with films including elements of ugliness. Indeed, researchers, investigating the perception of naturalness in textures suppose that more natural materials have a positive emotional connotation (Overvliet and Soto-Faraco, 2011; Overvliet et al., 2016; Rozin, 2005). Further research with a bigger sample size is necessary for testing the hypothesis that organic and non-organic materials are differently associated with the perception of elements of beauty and ugliness in cinema. If this association is confirmed, it is also interesting to test if it expands to the elements of beauty and ugliness in arts other than cinema.

In this study we were interested not only in finding cross-modal associations in cinema and haptic perception, but also in explaining their nature. We found that these associations are, at least partly, mediated by affective representations about good/bad, nice/disgusting, dirty/clean, young/old, bitter/sweet and happy/sad. That result contributes to studies into the role of emotional processing in cross-modal associations. Emotional representations of happiness, sadness, goodness, badness, etc. mediate relations between music and colors, odors and music, odors and colors, tastes and shapes (Guerdoux et al., 2014; Levitan et al., 2015; Palmer et al., 2013; Velasco et al., 2016).

For example, the German film (‘Hard to be a God’) was strongly associated with sandpaper. Both film and material were scaled as bad, dirty, old, disgusting, bitter and sad. One of the weakest associations was between the same material (sandpaper) and the Visconti film (‘Death in Venice’). Indeed, the Visconti film was scaled as good, clean, nice, sweet and happy, while sandpaper was scaled as the opposite (bad, dirty, disgusting, bitter and sad). Only on one scale (young–old) were both film and material scaled the same (i.e., ‘old’). Thus, probably, the procedure of this study can be used as a method to measure implicitly the aesthetic perception of a movie.

However, the question remains whether the quantity of semantic scales (e.g., number of scales on which the material and the film are scaled similarly) or only their quality (some scales are more important than others) matters for the strength of these cross-modal associations. For example, the association between the film by Kar-wai (‘2046’) and fur was weak; however, both of them were scaled as good, clean, young and nice. Only on one scale — happy/sad — were they scaled differently (film was scaled as sad, while fur as happy).

Answering that question, we found a significant correlation between the strength of cross-modal associations and the way they are scaled on semantic differential. When the film and the material show similar tendencies when scaled on all semantic scales, the cross-modal association is stronger. Thus, the number of shared specific emotional properties of films and materials is related to the strength of these cross-modal associations. Shared emotional properties, which underlie cross-modal associations between haptic perception and cinema, support the semantic coding hypothesis: the existence of a semantic network which underlies cross-modal associations (Martino and Marks, 1999).

Our study showed that only emotional semantic properties of films and materials were linked to each other (e.g., good/bad, nice/disgusting). That was not the case for properties related to factors of activity and power (e.g., active–passive, strong–weak) despite the fact that many studies of cross-modal associations showed that these properties mediate cross-modal associations as well as emotional properties (Albertazzi et al., 2015; Palmer et al., 2013; Velasco et al., 2016). There are two possible explanations for our results. Firstly, it is possible that a low mediation ability of features related to activity and power in our study was due to the small sample size. Secondly, it is also possible that for cinema–haptic cross-modal associations emotional/semantic properties of objects play a more important role compared to activity and power features.

There are some important limitations to consider in the present research. We chose to investigate cinema perception keeping in mind that cinema itself is a very complex stimulus. Although we tried to reach uniformity of stimuli in our experiment by choosing more or less equal characteristics of ‘beautiful’ and ‘ugly’ films (the aesthetic evaluation and emotional intensity, as rated by our participants, were not different for ‘beautiful’ and ‘ugly’ films), uniformity in terms of style and composition was not achieved. Thus, the results of the present study should be interpreted with caution, knowing that these differences might play a role in cross-modal associations.

Secondly, generalization of the results of the present study to other materials is limited. The materials we used varied on two dimensions of touch experience (smoothness and hardness), while there are other touch dimensions that we did not test in this study (e.g., coldness, thickness). Thirdly, in the present study we chose bipolar semantic differential scales instead of unipolar scales. We wanted to test 13 scales and thus chose bipolar scales to shorten the duration of the experiment. However, several researchers assume that bipolar scales may bias (overestimate) the obtained correlations (Albertazzi et al., 2015). Fourthly, the small sample size may also affect the reliability of our conclusions. Our sample was small compared to the samples in other studies of cross-modal associations. Fifthly, the subjects did not wear headphones to eliminate noise during the tactile examination, so they could hear noises. This fact could affect haptic perception.

Finally, despite the fact that beauty and ugliness are two of the most fundamental notions in aesthetics, there is still no clear definition of them. Perhaps, future studies of cross-modal associations involving cinema stimuli should focus on more concrete and better defined aesthetical categories (e.g., specific aesthetic movement in cinema, film maker’s style). This could also allow the generalization of the results we found to other types of movies.

Acknowledgements

The authors thank Ekaterina V. Pechenkova and two anonymous reviewers for their valuable advice. The authors thank volunteers who participated in this study.

Funding and Conflict of interest

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors declare no conflict of interest.

References

  • Afra K. (2015). ‘Vertical montage’ and synaesthesia, Music Sound Mov Image 9, 3361.

  • Albertazzi L., Canal L., Micciolo R. (2015). Cross-modal associations between materic painting and classical Spanish music, Front. Psychol. 6, 464. DOI:10.3389/fpsyg.2015.00424.

    • Search Google Scholar
    • Export Citation
  • Antunes L. R. (2012). The vestibular in film: orientation and balance in Gus Van Sant’s cinema of walking, Essays Philos. 13, 522549.

    • Search Google Scholar
    • Export Citation
  • Artemieva E. I. (2007). Psikhologiia Subieektivnoi Semantiki [The Psychology of Subjective Semantics]. LKI, Moscow, Russia.

  • Astrinaki E. (2012). Enhancing presence: sensory integration and proprioception in cinema, Am. Soc. Aesthet. Grad. E-J. 4(2), 19.

  • Beauchamp M. S., Ro T. (2008). Neural substrates of sound-touch synesthesia after a thalamic lesion, J. Neurosci. 28, 1369613702.

  • Bosanquet B. (1914). Three Lectures on Aesthetic. Bobbs Merrill, Indianapolis, IN, USA.

  • Day S. (2005). Some demographic and socio-cultural aspects of synaesthesia, in: Synaesthesia: Perspectives From Cognitive Neuroscience, Robertson L. C., Sagiv N. (Eds), pp.  1133. Oxford University Press, New York, NY, USA.

    • Search Google Scholar
    • Export Citation
  • Eisenstein S. (1940). Vertikalnyi montazh [Vertical montage], in: Psikhologicheskie Voprosy Iskusstva, pp.  175184. Smysl, Moscow, USSR.

    • Search Google Scholar
    • Export Citation
  • Etzi R., Spence C., Zampini M., Gallace A. (2016). When sandpaper is ‘Kiki’ and satin is ‘Bouba’: an exploration of the associations between words, emotional states, and the tactile attributes of everyday materials, Multisens. Res. 29, 133155.

    • Search Google Scholar
    • Export Citation
  • Flournoy T. (1893). Des Phenomenes de Synopsie (Audition Coloree): Photismes, Schemes, Visuels, Personnifications. F. Alcan, Paris, France.

    • Search Google Scholar
    • Export Citation
  • Guerdoux E., Trouillet R., Brouillet D. (2014). Olfactory–visual congruence effects stable across ages: yellow is warmer when it is pleasantly lemony, Atten. Percept. Psychophys. 76, 12801286.

    • Search Google Scholar
    • Export Citation
  • Hasson U., Landesman O., Knappmeyer B., Vallines I., Rubin N., Heeger D. J. (2008). Neurocinematics: the neuroscience of film, Projections 2, 126.

    • Search Google Scholar
    • Export Citation
  • Hollins M., Faldowski R., Rao S., Young F. (1993). Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis, Percept. Psychophys. 54, 697705.

    • Search Google Scholar
    • Export Citation
  • Kant I. (2008). Critique of Judgement [1790]. Oxford University Press, Oxford, UK and New York, NY, USA.

  • Levitan C. A., Ren J., Woods A. T., Boesveldt S., Chan J. S., McKenzie K. J., Dodson M., Levin J. A., Leong C. X., Van den Bosch J. J. (2014). Cross-cultural color–odor associations, PLoS One 9, e101651. DOI:10.1371/journal.pone.0101651.

    • Search Google Scholar
    • Export Citation
  • Levitan C. A., Charney S. A., Schloss K. B., Palmer S. E. (2015). The smell of jazz: Crossmodal correspondences between music, odor, and emotion, in: Proceedings of the 37th Annual Meeting of the Cognitive Science Society, pp. 1326–1331. Pasadena, CA, USA.

  • Ludwig V. U., Simner J. (2013). What colour does that feel? Tactile–visual mapping and the development of cross-modality, Cortex 49, 10891099.

    • Search Google Scholar
    • Export Citation
  • Marinetti F. T. (1989). The Futurist Cookbook [1932]. Penguin Books, London, UK.

  • Martino G., Marks L. E. (1999). Perceptual and linguistic interactions in speeded classification: tests of the semantic coding hypothesis, Perception 28, 903923.

    • Search Google Scholar
    • Export Citation
  • Osgood C. E., Suci G. J., Tannenbaum P. H. (1957). The Measurement of Meaning. University of Illinois Press, Urbana, IL, USA.

  • Overvliet K. E., Soto-Faraco S. (2011). I can’t believe this isn’t wood! An investigation in the perception of naturalness, Acta Psychol. 136, 95111.

    • Search Google Scholar
    • Export Citation
  • Overvliet K. E., Karana E., Soto-Faraco S. (2016). Perception of naturalness in textiles, Mater. Des. 90, 11921199.

  • Palmer S. E., Schloss K. B., Xu Z., Prado-León L. R. (2013). Music–color associations are mediated by emotion, Proc. Natl Acad. Sci. USA 110, 88368841.

    • Search Google Scholar
    • Export Citation
  • Picard D., Dacremont C., Valentin D., Giboreau A. (2003). Perceptual dimensions of tactile textures, Acta Psychol. 114, 165184.

  • Rozin P. (2005). The meaning of ‘natural’ process more important than content, Psychol. Sci. 16, 652658.

  • Schifferstein H. N. J., Tanudjaja I. (2004). Visualizing fragrances through colors: the mediating role of emotions, Perception 33, 12491266.

    • Search Google Scholar
    • Export Citation
  • Simner J., Ludwig V. U. (2012). The color of touch: a case of tactile-visual synaesthesia, Neurocase 18, 167180.

  • Slobodenyuk N., Jraissati Y., Kanso A., Ghanem L., Elhajj I. (2015). Cross-modal associations between color and haptics, Atten. Percept. Psychophys. 77, 13791395.

    • Search Google Scholar
    • Export Citation
  • Snider J. G., Osgood C. E. (Eds) (1969). Semantic Differential Technique: a Sourcebook. AldineTransaction, Chicago, IL, USA.

  • Spence C. (2011). Crossmodal correspondences: a tutorial review, Atten. Percept. Psychophys. 73, 971995.

  • Steven M. S., Hansen P. C., Blakemore C. (2006). Activation of color-selective areas of the visual cortex in a blind synesthete, Cortex 42, 304308.

    • Search Google Scholar
    • Export Citation
  • Velasco C., Woods A. T., Marks L. E., Cheok A. D., Spence C. (2016). The semantic basis of taste–shape associations, Peer J. 4, e1644. DOI:10.7717/peerj.1644.

    • Search Google Scholar
    • Export Citation
  • Wenzel C. (1999). Kant finds nothing ugly? Br. J. Aesthet. 39, 416422.

  • Yoshida M. (1968). Dimensions of tactual impressions (1), Jpn. Psychol. Res. 10, 123137.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Afra K. (2015). ‘Vertical montage’ and synaesthesia, Music Sound Mov Image 9, 3361.

  • Albertazzi L., Canal L., Micciolo R. (2015). Cross-modal associations between materic painting and classical Spanish music, Front. Psychol. 6, 464. DOI:10.3389/fpsyg.2015.00424.

    • Search Google Scholar
    • Export Citation
  • Antunes L. R. (2012). The vestibular in film: orientation and balance in Gus Van Sant’s cinema of walking, Essays Philos. 13, 522549.

    • Search Google Scholar
    • Export Citation
  • Artemieva E. I. (2007). Psikhologiia Subieektivnoi Semantiki [The Psychology of Subjective Semantics]. LKI, Moscow, Russia.

  • Astrinaki E. (2012). Enhancing presence: sensory integration and proprioception in cinema, Am. Soc. Aesthet. Grad. E-J. 4(2), 19.

  • Beauchamp M. S., Ro T. (2008). Neural substrates of sound-touch synesthesia after a thalamic lesion, J. Neurosci. 28, 1369613702.

  • Bosanquet B. (1914). Three Lectures on Aesthetic. Bobbs Merrill, Indianapolis, IN, USA.

  • Day S. (2005). Some demographic and socio-cultural aspects of synaesthesia, in: Synaesthesia: Perspectives From Cognitive Neuroscience, Robertson L. C., Sagiv N. (Eds), pp.  1133. Oxford University Press, New York, NY, USA.

    • Search Google Scholar
    • Export Citation
  • Eisenstein S. (1940). Vertikalnyi montazh [Vertical montage], in: Psikhologicheskie Voprosy Iskusstva, pp.  175184. Smysl, Moscow, USSR.

    • Search Google Scholar
    • Export Citation
  • Etzi R., Spence C., Zampini M., Gallace A. (2016). When sandpaper is ‘Kiki’ and satin is ‘Bouba’: an exploration of the associations between words, emotional states, and the tactile attributes of everyday materials, Multisens. Res. 29, 133155.

    • Search Google Scholar
    • Export Citation
  • Flournoy T. (1893). Des Phenomenes de Synopsie (Audition Coloree): Photismes, Schemes, Visuels, Personnifications. F. Alcan, Paris, France.

    • Search Google Scholar
    • Export Citation
  • Guerdoux E., Trouillet R., Brouillet D. (2014). Olfactory–visual congruence effects stable across ages: yellow is warmer when it is pleasantly lemony, Atten. Percept. Psychophys. 76, 12801286.

    • Search Google Scholar
    • Export Citation
  • Hasson U., Landesman O., Knappmeyer B., Vallines I., Rubin N., Heeger D. J. (2008). Neurocinematics: the neuroscience of film, Projections 2, 126.

    • Search Google Scholar
    • Export Citation
  • Hollins M., Faldowski R., Rao S., Young F. (1993). Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis, Percept. Psychophys. 54, 697705.

    • Search Google Scholar
    • Export Citation
  • Kant I. (2008). Critique of Judgement [1790]. Oxford University Press, Oxford, UK and New York, NY, USA.

  • Levitan C. A., Ren J., Woods A. T., Boesveldt S., Chan J. S., McKenzie K. J., Dodson M., Levin J. A., Leong C. X., Van den Bosch J. J. (2014). Cross-cultural color–odor associations, PLoS One 9, e101651. DOI:10.1371/journal.pone.0101651.

    • Search Google Scholar
    • Export Citation
  • Levitan C. A., Charney S. A., Schloss K. B., Palmer S. E. (2015). The smell of jazz: Crossmodal correspondences between music, odor, and emotion, in: Proceedings of the 37th Annual Meeting of the Cognitive Science Society, pp. 1326–1331. Pasadena, CA, USA.

  • Ludwig V. U., Simner J. (2013). What colour does that feel? Tactile–visual mapping and the development of cross-modality, Cortex 49, 10891099.

    • Search Google Scholar
    • Export Citation
  • Marinetti F. T. (1989). The Futurist Cookbook [1932]. Penguin Books, London, UK.

  • Martino G., Marks L. E. (1999). Perceptual and linguistic interactions in speeded classification: tests of the semantic coding hypothesis, Perception 28, 903923.

    • Search Google Scholar
    • Export Citation
  • Osgood C. E., Suci G. J., Tannenbaum P. H. (1957). The Measurement of Meaning. University of Illinois Press, Urbana, IL, USA.

  • Overvliet K. E., Soto-Faraco S. (2011). I can’t believe this isn’t wood! An investigation in the perception of naturalness, Acta Psychol. 136, 95111.

    • Search Google Scholar
    • Export Citation
  • Overvliet K. E., Karana E., Soto-Faraco S. (2016). Perception of naturalness in textiles, Mater. Des. 90, 11921199.

  • Palmer S. E., Schloss K. B., Xu Z., Prado-León L. R. (2013). Music–color associations are mediated by emotion, Proc. Natl Acad. Sci. USA 110, 88368841.

    • Search Google Scholar
    • Export Citation
  • Picard D., Dacremont C., Valentin D., Giboreau A. (2003). Perceptual dimensions of tactile textures, Acta Psychol. 114, 165184.

  • Rozin P. (2005). The meaning of ‘natural’ process more important than content, Psychol. Sci. 16, 652658.

  • Schifferstein H. N. J., Tanudjaja I. (2004). Visualizing fragrances through colors: the mediating role of emotions, Perception 33, 12491266.

    • Search Google Scholar
    • Export Citation
  • Simner J., Ludwig V. U. (2012). The color of touch: a case of tactile-visual synaesthesia, Neurocase 18, 167180.

  • Slobodenyuk N., Jraissati Y., Kanso A., Ghanem L., Elhajj I. (2015). Cross-modal associations between color and haptics, Atten. Percept. Psychophys. 77, 13791395.

    • Search Google Scholar
    • Export Citation
  • Snider J. G., Osgood C. E. (Eds) (1969). Semantic Differential Technique: a Sourcebook. AldineTransaction, Chicago, IL, USA.

  • Spence C. (2011). Crossmodal correspondences: a tutorial review, Atten. Percept. Psychophys. 73, 971995.

  • Steven M. S., Hansen P. C., Blakemore C. (2006). Activation of color-selective areas of the visual cortex in a blind synesthete, Cortex 42, 304308.

    • Search Google Scholar
    • Export Citation
  • Velasco C., Woods A. T., Marks L. E., Cheok A. D., Spence C. (2016). The semantic basis of taste–shape associations, Peer J. 4, e1644. DOI:10.7717/peerj.1644.

    • Search Google Scholar
    • Export Citation
  • Wenzel C. (1999). Kant finds nothing ugly? Br. J. Aesthet. 39, 416422.

  • Yoshida M. (1968). Dimensions of tactual impressions (1), Jpn. Psychol. Res. 10, 123137.

Content Metrics

All Time Past Year Past 30 Days
Abstract Views 177 39 0
Full Text Views 252 60 5
PDF Downloads 49 32 3