Crossmodal Texture Perception Is Illumination-Dependent

Visually perceived roughness of 3D textures varies with illumination direction. Surfaces appear rougher when the illumination angle is lowered resulting in a lack of roughness constancy. Here we aimed to investigate whether the visual system also relies on illumination-dependent features when judging roughness in a crossmodal matching task or whether it can access illumination-invariant surface features that can also be evaluated by the tactile system. Participants ( N = 32) explored an abrasive paper of medium physical roughness either tactually, or visually under two different illumination conditions (top vs oblique angle). Subsequently, they had to judge if a comparison stimulus (varying in physical roughness) matched the previously explored standard. Matching was either performed using the same modality as during exploration (intramodal) or using a different modality (crossmodal). In the intramodal conditions, participants performed equally well independent of the modality or illumination employed. In the crossmodal conditions, participants selected rougher tactile matches after exploring the standard visually under oblique illumination than under top illumination. Conversely, after tactile exploration, they selected smoother visual matches under oblique than under top illumination. These ﬁndings conﬁrm that visual roughness perception depends on illumination direction and show, for the ﬁrst time, that this failure of roughness constancy also transfers to judgements made crossmodally


Introduction
The ability to correctly identify objects and materials is a crucial part of our everyday life, affecting the decisions we make and the actions we perform.Most material properties can be perceived using several of our senses (e.g., vision, touch, audition) either in isolation, i.e., unimodal (also called unisensory) perception, in combination, i.e., multimodal (also called multisensory) perception, or in separation where different senses contribute independently but influence and/or inform each other, i.e., crossmodal (also called crosssensory) perception (Spence et al., 2009).Here, we are interested in the latter.Specifically, we wanted to measure the formation of perceptual predictions about tactile object properties based on visual information.Making such crossmodal inferences becomes particularly important when tactile information is not instantly available, such as during online shopping where we are trying to infer the softness/roughness of textiles based on photographs (Xiao et al., 2016).While, in general, such crossmodal inferences result in a satisfactory outcome, situations occur where there are strong discrepancies between the predicted tactile properties and the actual tactile experience (once the product arrives).One likely reason for these discrepancies might be that the visual perception of surface features is confounded by features of the illumination of the surface.It is particularly difficult to disentangle these two aspects in images where -as in the case of online shopping -we often have little or no information about illumination direction and other relevant contextual factors (Fleming et al., 2003;Kingdom, 2008).However, when we deal with (real) 3D textures, we also have access to illumination-invariant visual cues to surface texture, such as binocular disparity (Ho et al., 2006), which could make our visual roughness perception more robust and potentially improve crossmodal inferences.Previous research has shown that those cues do not seem to be used in purely visual tasks (Ho et al., 2006), but we do not know if this also applies to crossmodal judgements.Consequently, we aimed to investigate the effect of illumination direction on the perception of surface roughness of (real) 3D textures using a crossmodal matching task.
Our rationale is the following: As the tactile system has no access to the illumination-dependent features that are used in purely visual assessments of surface roughness, there are two possible predictions with regard to crossmodal judgements.Either, as postulated previously by Björkman (1967), crossmodal judgements are made based on features that are accessible to both systems (i.e., the visual system would use different, illumination-independent, features than in purely visual tasks) or, the illumination-dependent cues are mapped into tactile cues and vice versa and consequently crossmodal tactile judgements would exhibit illumination dependence.
We chose surface roughness as a material property to measure, as it is one of the most prominent perceptual dimensions of 3D texture perception (e.g., Baumgartner et al., 2013;Drewing et al., 2017;Hollins et al., 1993Hollins et al., , 2000;;Okamoto et al., 2012) and has been extensively studied in relation to both vision and touch (e.g., Bergmann Tiest and Kappers, 2007;Brown, 1960;Eck et al., 2013;Lederman, 1981;Lederman et al., 1986).Generally, studies investigating the accuracy and relative contributions of vision and touch to the (unimodal and bimodal) perception of surface roughness have found that judgements of surface roughness made by the two modalities are highly correlated (Baumgartner et al., 2013;Bergmann Tiest and Kappers, 2007) and also tend to be similarly accurate (e.g., Jones and O'Neil, 1985;Lederman & Abbott, 1981) at least for coarser textures, i.e., up to 1000 grit for sandpapers (Heller, 1989;Klatzky and Lederman, 2010).The physical roughness of a 3D surface is mainly defined by the shape, size, and distribution/density of particles on the surface which have also been shown to affect perceived roughness (Drewing, 2018;Drewing et al., 2004;Klatzky and Lederman, 2010;Lederman and Taylor, 1972;Natsume et al., 2019;Taylor and Lederman, 1975).Consequently, a variety of different features and sensory cues are available to estimate surface roughness.Both the visual system and the tactile system evaluate the spacing and height of the elements on the surface as well as the width of the grooves and ridges (Klatzky and Lederman, 2010).The tactile system uses skin deformation on contact and experienced vibration as cues to assess those surface features (e.g., Drewing, 2018;Natsume et al., 2017).The visual system may use a variety of cues to size and spacing of surface elements.As mentioned above, there are two broad groups of visual cues to surface roughness: image-based cues, i.e., 2D image statistics, and cues related to the 3D structure of the surfaces, e.g., cues to size like binocular disparity (Ho et al., 2006).In contrast to the image-based cues, cues related to the 3D surface structure will be less affected by illumination changes (Ho et al., 2006).Ho et al. (2006) investigated how visually perceived 3D roughness is influenced by the direction from which a surface is illuminated.In their study, participants were asked to indicate which of two computer-generated 3D images of textures appeared to be rougher using a two-interval forced-choice task.The images presented varied on eight levels of roughness and were rendered under three different illumination angles (elevations of 50, 60 and 70 degrees).Employing a staircase procedure, they found that perceived surface roughness consistently increased with decreasing illumination angle.Interestingly, this effect persisted even when objects providing additional information about the direction of illumination were added to the scene.They showed that participants' performance could be well modelled using a linear combination of four cues to roughness broadly related to the distribution of shadows and shading in the image, e.g., the proportion of image in the shadow, the mean and standard deviation in the luminance of non-shadowed pixels, and texture contrast (as defined by Pont &and Koenderink, 2005).They argued that the demonstrated failure of roughness constancy in surface texture perception is explained by the visual system using these cues to estimate surface roughness (see also, Ho et al., 2007).Ho and colleagues refer to those cues as 'pseudocues' to roughness because they are not illumination-invariant, therefore, they are only valid indicators of surface roughness under constant illumination.Here, we will adopt their terminology to differentiate between illumination-invariant and illumination-dependent cues to roughness.
But what are the implications of those results for crossmodal texture perception?In perceptual crossmodal matching tasks, participants are required to use information acquired by one sensory modality, such as vision or touch, to make a judgement in a different sensory modality, using touch or vision (Björkman, 1967).In other words, to make crossmodal matches, information about surface properties from one modality has to be mapped into the other modality.Björkman (1967) suggested that crossmodal matching tasks may force participants to base their judgements on distinctive features that are common to the two modalities.As mentioned above, features related to surface roughness that are available to both modalities include the distribution, size, and height of the particles on the surface.There are both visual (e.g., binocular disparity) and tactile cues (e.g., skin indentation) to particle size.Thus, particle size can be considered a common feature according to Björkman (1967).
The pseudocues identified by Ho et al. (2006) are related to the shadows and shading on a surface which are purely visual features and thus inaccessible to the tactile system.Hence, they are not common features in the sense of Björkman's notion.However, since those pseudocues are correlated with particle size and other roughness features, they might still be used to make tactile predictions in a crossmodal task.Consequently, tactile matches, although inherently illumination-invariant, may inherit illumination dependence from visual cues (i.e., a texture would be expected to feel rougher the shallower the illumination angle).Conversely, it is possible that information acquired by the tactile system is mapped into pseudocues when selecting visual matches (i.e., a texture experienced tactually will be associated with smoother visual textures under decreasing illumination angle because visual roughness appearance will increase).
To test whether crossmodal matches are affected by illumination direction, we designed a matching experiment in which real 3D surfaces varying in physical roughness were illuminated from two different positions (top vs oblique).Under each illumination angle, participants were asked to explore a standard texture either by vision or touch and subsequently had to select a matching texture either using the same modality (intramodal conditions) or using the other modality (crossmodal conditions).There are two competing hypotheses for crossmodal matching.If illumination-invariant cues are used, we expect no systematic differences between the matches made in the two illumination conditions.However, if illumination-dependent cues are used, we expect systematic changes in the matches depending on illumination: in the condition in which surfaces are explored visually, oblique illumination should result in rougher tactile matches compared to illumination from the top.In the condition in which the surfaces are explored tactually, oblique illumination during matching should result in smoother visual matches (as the surfaces appear visually rougher) compared to illumination from the top.
In line with Ho et al. (2007), our findings confirm our second hypothesis that visual textures indeed tended to appear visually rougher under oblique illumination.Importantly, results from crossmodal matching are in line with the prediction that illumination-dependent cues also affect crossmodal matching performance.

Participants
Thirty-four volunteers (10 males, M age = 23 years, SD age = 6.3, age range: 17-47 years) were recruited for this study.Prior to the experiment, we assessed participants' near visual acuity using the SLOAN letter chart (40 cm distance, 756400, Good-Lite, Elgin, IL, USA).One female participant did not pass the minimum required near visual acuity of 20/20 and was thus ineligible to participate.Data of one further male participant were excluded due to noncompliance with the task instructions (i.e., touching stimuli during visual inspection).Hence, data analysis is based on a final sample of 32 participants (31 right-handed and one left-handed by self-report).All participants provided written informed consent and were either granted course credits or reimbursed with £20 for the completion of the experiment that lasted 120 to 150 minutes.The study was approved by the local ethics committee of the School of Psychology at the University of Aberdeen (PEC/4800/2021/9).

Stimuli and Setup
Seven black silicon carbide sandpapers with varying grit values (i.e., 60, 80, 120, 180, 240, 600, 800) served as our stimuli.Grit values of sandpapers refer to the number of openings per square inch in the sieve used to apply the particles to the papers (Stevens and Harris, 1962).Thus, lower grit values indicate rougher and higher grit values finer abrasive papers.We chose grid values based on previous research (Kangur et al., 2022;Lederman and Abbott, 1981), and pilot experiments to ensure stimuli were perceptually distinguishable and increased monotonically in perceived roughness.The setup used for stimulus presentation consisted of a black rectangular box (H: 60 cm, W: 73 cm, D: 30 cm) placed on a table with two openings: the upper one (W: 15 cm, H: 10 cm) for the visual and the lower one (W: 15 cm, H: 3 cm) for the tactile exploration of the stimuli (comparable setup to, Kangur et al., 2022;Lederman and Abbott, 1981).Two sandpapers (7.5 cm × 12.5 cm in size) of each grit value were affixed on a rotating table with a diameter of 60 cm which was placed inside the box just below the tactile exploration opening (see Fig. 1A).The tactile stimuli were arranged in a circle around the perimeter of the rotating table, and the visual stimuli were placed in an adjacent inner circle.Sandpapers on the rotating table were ordered counterclockwise from low to high grit values.The 180 grit sandpaper was duplicated and positioned again between the roughest (60 grit) and smoothest (800 grit) sandpaper.This stimulus served as the standard stimulus while the remaining seven stimuli served as comparison stimuli (see Fig. 1B).
Furthermore, the box allowed us to vary the illumination of the visual stimuli by changing the illumination angle.The sandpapers were illuminated by six LED lights (colour temperature 6000 K, distance between LEDs 1.5 cm) arranged in two strips of three lights.The lights either illuminated the sandpapers from the top (i.e., directly from above) or from the back at an oblique angle (∼11.5 degrees with respect to the stimulus surface, see Fig. 1A).The lights in both conditions were equidistant (10 cm) from the stimuli and the eye-to-stimulus viewing distance was 25-30 cm.

Procedure
The experiment consisted of a matching task (completed first) and a subsequent visual rating task.The aim of the rating task was to determine if the perceived visual roughness of textures changed systematically with illumination angle.The matching task was split into intra-and crossmodal matching conditions (described below) with the intramodal conditions always performed prior to the crossmodal conditions.

Matching Task
Participants were seated in a height-adjustable chair in front of the setup in a darkened room.Their task was to visually or tactually explore the standard stimulus presented to them (i.e., 180-grit sandpaper) and then to decide whether a subsequently presented comparison stimulus (explored visually or tactually) provided a 'good match' for the previously explored standard.Participants were not provided with a definition of what constituted a good match.In line with the arguments of Björkman (1967), we considered it preferable to leave the perceptual properties unspecified (such as 'roughness' or 'smoothness') to avoid directing participants' attention to certain attributes or cues of the texture stimuli.Instead, we wanted participants to be free to identify the cues they found most helpful/relevant to perform the matching task.That is, when we use the term 'good match' here, we refer to the perceived similarity in appearance between standard and comparison stimulus (i.e., a good match means that standard and comparison are perceived as similar) and not the accuracy of the matches (i.e., their physical distance to the standard stimulus).This choice was made as our main interest was in if, and how, matches change under different illumination conditions.
Each trial started with the experimenter rotating the turntable in a way that positioned the standard within the visual and tactile windows.Once the experimenter had positioned the stimulus, they verbally stated: 'Standard' as a cue for participants to start exploring the standard stimulus.The participants then explored the standard by vision or touch (only one modality at the time) and verbally informed the experimenter once they had completed their exploration.Subsequently, the experimenter rotated the turntable to position the comparison stimulus in the visual and tactile windows and verbally stated: 'Comparison' as a cue for participants to start exploring the comparison stimulus.Participants provided their response on whether they perceived the comparison stimulus to be a good match for the standard stimulus by pressing a key (i.e., 'yes' or 'no') on a hand-held button box.Note that during visual exploration and matching trials, lights in the presentation box were turned on simultaneously with the verbal cue and were turned off as soon as participants had completed their exploration and before the turntable was rotated.In the tactile exploration and matching trials, participants rested their dominant hand on an arm-rest in front of the tactile window while the experimenter rotated the turntable.Once they were informed that the stimulus was placed, they moved their hand through the tactile window to start exploration.
In the intramodal conditions, which participants performed first, exploration and matching were completed within the same modality (i.e., both visually or both tactually).There were three different intramodal conditions: visual exploration and matching under top illumination (VV top), visual exploration and matching under oblique illumination (VV obl), and tactile exploration and matching (TT).The order of the visual and tactile conditions was counterbalanced across participants.In each intramodal condition, each of the seven standard-comparison combinations (i.e., 180-60, 180-80, 180-120, 180-180, 180-240, 180-600, 180-800) was presented four times in randomised order resulting in 28 trials per condition and 84 intramodal trials in total.
Intramodal matching was followed by crossmodal matching where participants were either asked to explore the standard visually and judge whether the comparison stimulus provided a good match tactually (VT condition) or vice versa, to explore the standard tactually and judge whether the comparison stimulus was a good match for the standard stimulus visually (TV condition).
In each of the two crossmodal conditions, we also varied the illumination during visual exploration of the standard (VT top vs VT obl) or visual exploration of the comparison stimulus during matching (TV top vs TV obl).Presentation and illumination conditions were blocked and balanced across participants in a Latin Square fashion.As in the intramodal conditions, each standardcomparison combination was presented four times in randomised order, resulting in 28 trials per block and 112 crossmodal trials in total.

Rating Task
Following the completion of the matching task, participants were asked to rate the visual roughness of the sandpaper stimuli used for matching (i.e., 60, 80, 120, 180, 240, 600, 800 grit) in the two illumination conditions (i.e., top vs oblique).The task was conducted last in the experiment as it involved a direct assessment of surface roughness and, as described above, we did not want to direct participants' attention to specific aspects of the surface during the matching task.
The setup used for visual stimulus presentation was identical.The experimenter started each rating trial by positioning the sandpaper in the visual opening and subsequently turning on the lights.Participants' task was to visually explore the stimulus and verbally rate its perceived roughness on a sevenpoint Likert scale (between 1: 'Not rough at all' and 7: 'Very rough').No time restrictions were imposed during the exploration of the stimuli.Illumination conditions were blocked and counterbalanced across participants.Within each illumination condition, the seven stimuli were presented in randomised order and each rated four times resulting in 28 rating trials per illumination condition and 56 trials in total.

Predictions
In the intramodal conditions (VV and TT), we expected that visual matches would be unaffected by illumination as both visual exploration and matching happened under the same illumination conditions.We also expected intramodal visual matches to be similar to the intramodal tactile matches as both modalities are assumed to be similarly accurate in determining surface roughness.In the visual rating condition (VR), we expected the surfaces to be rated as being rougher when illuminated from an oblique angle compared to illumination from the top.
For crossmodal matching, the predictions depend on whether or not illumination-invariant cues are used to perform the task.If illuminationinvariant cues are used to perform the task, there should be no systematic differences between the matches made in the two illumination conditions.On the other hand, if illumination-dependent cues are used, we expect that in the VT condition in which surfaces are explored visually, oblique illumination should result in the selection of rougher tactile matches compared to illumination from the top (as the surfaces look rougher under oblique illumination during exploration).In the TV condition in which the surfaces are explored tactually, oblique illumination should result in smoother visual matches compared to illumination from the top (as the matches look rougher under oblique illumination).

Data Analysis
Since grit values do not form a uniform perceptual scale, we assigned ordinal roughness ranks (1-7) to grit values for data analysis, with '1' referring to the smoothest sandpaper (800 grit) and '7' referring to the roughest sandpaper (60 grit).Note that after transformation, larger numbers now refer to rougher stimuli and small numbers to smoother stimuli.
We calculated the mean match to the standard for each participant, matching, and illumination condition based on averaging the roughness ranks associated with 'yes' responses.Responses in the intramodal and crossmodal matching conditions were analysed separately using repeated-measures analyses of variance (ANOVAs).Visual roughness ratings under top and oblique illumination were compared using a paired-samples t-test.For all analyses, a significance level of α = 0.05 was used.Where applicable, α-levels for posthoc comparisons were Bonferroni-Dunn-corrected. Means are presented with ±1 SEM (standard error of the mean; between subjects).

Results
Based on previous observations that vision and touch tend to be similarly accurate when judging surface texture (Jones & O'Neil, 1985;Lederman & Abbott, 1981), we expected similar matching performance across the three intramodal conditions.Figure 2A shows how often each stimulus was selected to represent a good match in each of the three intramodal conditions across all participants and trials.Generally, participants selected both the 180-grit (rank 4) and the 240-grit (rank 3) sandpapers similarly often as good matches for the standard while all other stimuli were considerably less frequently identified as good matches (see the Supplementary Material for a full statistical analysis of the data).Furthermore, distributions were very similar in all conditions.We then calculated the number of times participants identified a good match in each of the three intramodal conditions.Each of the seven comparison stimuli was presented four times, thus if participants only identified the 180-grit stimulus as a good match, they should provide a 'yes' response in four out of 28 trials.For each participant, we summed all yes-responses across all trials in one condition.Across participants, the number of identified matches did not differ between the three intramodal conditions, F 2,62 = 0.22, p = 0.81, η 2 p = 0.007, and on average, participants identified 10.0 ± 0.5 good matches (i.e., 'yes' responses).Note that our participants commonly reported (during debriefing) that they were convinced that there was more than one standard used during the experiment.Moreover, as can be seen from Fig. 2B, the mean of the 'yes' responses was very similar in all three intramodal conditions.A one-way repeated-measures ANOVA on the data confirmed no effect of condition (VV obl, VV top, TT), F 2,62 = 0.89, p = 0.42, η 2 p = 0.03.Thus, similar to previous findings, we observed comparable performance when matching surface textures by either vision or touch.As expected, when visual exploration and matching were performed under identical illumination, the direction of illumination did not affect performance.
The main aim of this experiment was to identify if a lack of roughness constancy in vision would transfer to perceptual judgements in crossmodal conditions.Figure 3A and 3B show how often each stimulus was selected to represent a good match in each of the crossmodal conditions across all participants and trials.Distributions were broadly similar in the TV and VT matching conditions but skewed in opposite directions for the two illumination conditions (dark pink bars: top; light lilac bars: oblique) consistent with the notion that surfaces appear visually rougher when illuminated from an oblique angle (i.e., light lilac bars skewed towards rougher values in VT condition and skewed towards less rough values in TV condition).In other words, the number of good matches identified for the different grit values seemed to change differently in the two matching conditions depending on illumination (see the Supplementary Material for a full statistical analysis of the data).
As for intramodal matching, we computed the number of times participants identified a good match in each of the four crossmodal conditions.Again, if participants identified only the 180-grit stimulus as a good match, they should provide four 'yes' responses in each condition.For each participant, we summed all yes-responses across all trials in each condition.Our analysis revealed that, again, the number of identified matches did not differ between the four crossmodal conditions, F 3,93 = 1.51, p = 0.23, η 2 p = 0.05, and on average, participants identified 11.1 ± 0.4 good matches.
Figure 3C shows the average values of those 'yes' responses in the two crossmodal matching conditions as a function of illumination condition (top: dark pink bars; oblique: light lilac bars).Descriptively, participants appeared to select rougher tactile matches after exploring the standard visually (VT) under oblique illumination as compared to top illumination.Furthermore, after tactile exploration (TV) they tended to select smoother visual matches under oblique illumination.A 2 (matching condition: TV vs VT) × 2 (illumination: top vs oblique) repeated-measures ANOVA confirmed a significant interaction effect between the factors, F 1,31 = 20.8,p < 0.001, η 2 p = 0.40.The analysis also revealed a main effect of matching condition, F 1,31 = 10.77,p = 0.003, η 2 p = 0.26, with participants selecting overall slightly rougher matches in the VT conditions than in the TV conditions.There was no main effect of illumination angle, F 1,31 = 0.91, p = 0.35, η 2 p = 0.03.The significant interaction effect was followed up with pairwise comparisons testing the effect of illumination in the two matching conditions separately.This analysis confirmed an effect of illumination on the matches selected in the VT condition, t 31 = 2.62, p = 0.014, d = 0.46.Following a visual inspection of the standard under oblique illumination, participants selected, on average, rougher tactile matches than following inspection of the standard under top illumination (M diff = 0.31 ± 0.12).Moreover, the same analysis conducted for the TV condition confirmed that after tactile exploration, participants selected significantly different visual matches (i.e., smoother) under oblique illumination than under top illumination (M diff = −0.45± 0.10), t 31 = −4.32,p < 0.001, d = 0.76.This pattern of results is consistent with the prediction that crossmodal judgements are susceptible to the effects of illumination.
Finally, to test if visual appearance of the stimuli did consistently change with illumination, we also asked participants to rate all stimuli according to their perceived roughness on a seven-point Likert scale under both top and oblique illumination.Figure 4 shows the average ratings for all seven stimuli used in our matching task as well as the average roughness rating across them.A 2 (illumination: top vs oblique) × 7 (grit value: 60, 80, 120, 180, 240, 600, 800) repeated-measures ANOVA on the rating data revealed a significant main effect of grit value, F 6,186 = 732.65,p < 0.001, η 2 p = 0.96, as well as a main effect of illumination condition, F 1,31 = 11.93,p = 0.002, η 2 p = 0.28.The main effect of grit value confirms that, as expected, perceived roughness increases with larger grit values.Post-hoc tests confirmed that all comparisons between stimuli were significant (p < 0.001) apart from the difference between the 60-grit and 80-grit sandpapers (p = 0.56).Furthermore, post-hoc tests confirmed that stimuli were perceived as looking rougher under oblique illumination than under top illumination (M diff = 0.34 ± 0.10).Finally, the analysis also revealed a significant interaction effect between the factors, F 6,186 = 11.11,p < 0.001, η 2 p = 0.26, indicating that the size of the illumination effect varied for the different stimuli/grit values (see Fig. 4).

Discussion
Here, we aimed to explore the effect of illumination angle on crossmodal visuo-tactile perception of surface texture.We asked participants to explore two stimuli, a standard followed by a comparison stimulus, and to report whether they perceived the two stimuli to match -once under top and once under oblique illumination.Participants were also asked to rate the stimuli based on their perceived visual roughness.Overall, our results suggest that the crossmodal matches and the visual roughness ratings were reliably affected by the angle from which the surfaces were illuminated.That is, matching behaviour as well as visual roughness ratings were found to be consistent with the notion that textures looked rougher when illuminated from an oblique angle as compared to illumination from the top (Ho et al., 2006(Ho et al., , 2007)).
First, we confirmed that, as expected, intramodal visual matches were unaffected by illumination direction.In other words, match distributions were very similar in both conditions and both the 180-grit standard and the adjacent smoother 240-grit sandpaper were most commonly identified as good matches.This is consistent with the rating data that also shows that the 180-grit and the 240-grit stimuli were perceived as being more similar than the 180-grit standard and the adjacent rougher 120-grit stimulus.Furthermore, consistent with previous literature reporting that visual and tactile judgements are similarly accurate and/or highly correlated (e.g., Bergmann Tiest and Kappers, 2007;Guest and Spence, 2003;Jones and O'Neil, 1985), we also found a high similarity between intramodal visual and tactile matches with respect to both the mean matches and the match distributions.
Importantly, the rating data confirmed that all three pre-requirements for our crossmodal conditions to work were fulfilled: firstly, confirming the findings of Ho et al. (2006), we found that participants generally rated stimuli as looking rougher under oblique illumination than under top illumination.Secondly, perceived roughness increased monotonically with decreasing grit value and thirdly, stimuli ratings differed reliably between the different grit values (with exception of the 60-and 80-grit comparison).
In the crossmodal conditions, participants explored the standard in one modality (vision or touch), and then judged the matching stimulus using the other modality (touch or vision).Importantly, we found that crossmodal matches were clearly affected by illumination, confirming that crossmodal judgements inherit the illumination dependence of visually perceived surface roughness.This suggests that the visual system relies on the same cues to surface roughness in intra-and crossmodal conditions.Moreover, the finding that matches and explicit roughness ratings were similarly affected by illumination suggests that participants evaluated similar surface features relying on the same (illumination-dependent) cues in both tasks.
Our findings are consistent with Ho et al.'s suggestion that the visual perception of surface roughness depends on image statistics describing the distribution of shadows and shading on the surface (i.e., pseudocues).The shadow and shading distributions are related to a purely visual feature (i.e., shadows and shading) which cannot be perceived by the tactile system.Hence, shadows and shading are not a commonality between the visual and tactile systems.Thus, our findings are in conflict with Björkman's suggestion that crossmodal judgements are based on commonalities between modalities and show, for the first time, that illumination-dependent cues to roughness are also used in crossmodal perception.
Studies on associative crossmodal learning have shown that participants can learn (even arbitrary) connections between surface properties (e.g., Adams et al., 2004;Ernst, 2007;Jacobs and Fine, 1999).Ho et al. (2006), speculated that the usage of the pseudocues for visual roughness perception might be the result of the visual system (wrongly) associating those cues with tactile cues to surface roughness.As long as illumination remains relatively constant, surface roughness can be validly estimated based on pseudocues as they are correlated with certain roughness related surface features such as particle size.This is further confirmed by our observation that intramodal visual matches could be equally well identified in either illumination condition.However, the use of pseudocues becomes problematic when either visual matches have to be made in different illumination contexts or when matches have to be made between an illumination-dependent and an illumination-invariant modality (as in our study).However, based on our study, it is hard to estimate the magnitude and relevance of this effect on real-life crossmodal texture judgements as, for example, required during online shopping where differences between textures are much more pronounced than in the sandpaper stimuli used in our study.
Our findings on crossmodal matching are also in line with a previous study from our group on bimodal visuo-tactile matching where we investigated the relative contributions of vision and touch to roughness perception (Kangur et al., 2022).In this study, we employed a discrepancy paradigm in which a smoother visual stimulus (150 grit) was always paired with a much rougher tactile stimulus (60 grit), and matches had to be selected uni-or bimodally.Similar as in the current study, we also varied the illumination angle.We found that both modality weights assigned during bimodal integration and the selected matches were less variable under oblique than under top illumination, in line with the assumption that the visual stimulus appeared rougher under oblique illumination, thus reducing the perceived discrepancy between visual and tactile stimuli.
In conclusion, our findings confirm that the predictions we make about the tactile properties of textures and surfaces of objects vary with illumination direction.This is likely due to the two modalities basing their predictions on different features and the learnt probabilistic relationship between them.

Figure 1 .
Figure 1.Illustration of setup and stimuli (not drawn to scale).(A) Schematic drawing of the setup.Note that apart from the visual stimulus on the turntable, the lower, grey-shaded part of the box was not visible to participants.(B) Illustration of the standard and comparison stimuli arrangement on the turntable.'V' refers to the stimuli that were used for visual presentation, 'T' for tactile presentation.Colours are for illustrative purposes only.In the experiment all abrasive papers were black.

Figure 2 .
Figure 2. (A) Histogram showing the number of good matches identified across all participants in the three intramodal conditions.(B) Mean matches in the intramodal conditions [i.e., VV (visual conditions) and TT (tactile conditions)].Higher ranks show rougher matches (i.e., lower grit values of abrasive papers, see section 2.5.Data Analysis for more detail).Error bars denote ±1 SEM (between subjects).SEM, standard error of the mean.

Figure 3 .
Figure 3. (A, B) Histograms showing the number of good matches identified across all participants in the TV condition where matches were explored by touch and matched by vision (A) and the VT condition where matches were explored by vision and matched by touch (B).(C) Mean matches in the crossmodal matching conditions (i.e., VT and TV) under top (dark pink bars) and oblique (light lilac bars) illumination.Higher ranks show rougher matches (i.e., lower grit values of abrasive papers; see section 2.5.Data Analysis for more detail).Error bars denote ±1 SEM (between subjects).SEM, standard error of the mean.

Figure 4 .
Figure 4. Visual ratings of stimuli roughness.Ratings shown as a function of grit value and illumination condition (left) as well as averaged across all stimuli in the two illumination conditions (right).Error bars denote ±1 SEM (between subjects).SEM, standard error of the mean.