Self as a prior: The malleability of Bayesian multisensory integration to social salience

Our everyday perceptual experiences are grounded in the integration of information within and across our senses. Due to this direct behavioural relevance, cross-modal integration retains a certain degree of contextual flexibility, even to social relevance. However, how social relevance modulates cross-modal integration remains unclear. To investigate possible mechanisms, Experiment 1 tested the principles of audio-visual integration for numerosity estimation by deriving a Bayesian optimal observer model with perceptual prior from empirical data to explain perceptual biases. Such perceptual priors may shift towards locations of high salience in the stimulus space. Our results showed that the tendency to over-or underestimate numerosity, expressed in the frequency and strength of fission and fusion illusions, depended on the actual event numerosity. Experiment 2 replicated the effects of social relevance on multisensory integration from Scheller & Sui, 2022 JEP:HPP , using a lower number of events, thereby favouring the opposite illusion through enhanced influences of the prior.


BACKGROU N D
It is well conceived that perception can be understood as a probabilistic process, whereby every piece of sensory information (i.e., cue) that is picked up by the sensory systems is associated with a certain degree of uncertainty (Angelaki et al., 2009;Bülthoff & Yuille, 1996;Drugowitsch & Pouget, 2012;Ernst & Banks, 2002;Knill & Pouget, 2004;Körding et al., 2007;Ma, 2012;Van Bergen et al., 2015;van Bergen & Jehee, 2019).This uncertainty depends on how well our sensory systems are tuned to process specific environmental features.For instance, as the layout of the retina allows the visual system to process more bits of information in parallel, vision is typically more dominant in spatial tasks, while the auditory system is better attuned to the temporal information structure (Bresciani et al., 2005(Bresciani et al., , 2008;;Scheller et al., 2018;Shams, Ma, et al., 2005;Wozny et al., 2008).The relative uncertainty of the sensory systems explains various perceptual phenomena and illusions, reflecting how perception disambiguates environmental information by weighing different sensory cues by their individual reliabilities (Alais & Burr, 2004;Bruns, 2019;Heron et al., 2004;Magnotti & Beauchamp, 2017;Shams et al., 2000;Shams & Beierholm, 2022;Shams, Ma, et al., 2005).
One of these cases is the sound-induced flash illusion (SIFI; Andersen et al., 2004;Shams et al., 2000Shams et al., , 2002;; see Hirst et al., 2020;Keil, 2020 for reviews), an illusory percept of temporal numerosity estimation.Here, a number of events are sequentially presented to two sensory modalities, typically vision and audition and observers need to judge how many events were presented.The more reliable modality, typically audition, influences the less reliable modality, vision, leading to an illusory percept of an additional event (fission; Shams et al., 2002) or one event less (fusion; Andersen et al., 2004).Illusions like the SIFI not only provide a tool to study perception in a more naturalistic, multisensory context but also offer insights into the ways that our perceptual system resolves ambiguity, reduces sensory uncertainty, and generates a more veridical representation of our environment.
The SIFI, as well as many other perceptual phenomena, has been shown to be best represented as a Bayesian optimal percept.That is, the difference in reliability (inverse of sensory uncertainty) between sensory modalities defines the weight that is placed on each numerosity estimate during integration (Shams, Ma, et al., 2005): Audition influences visual numerosity perception more strongly than vision influences auditory numerosity.Reliability weighting in integration is typically beneficial, as it allows the perceptual system to attain higher accuracy and maximal precision, and is therefore often referred to as an 'optimal' process (Alais & Burr, 2004;Ernst & Banks, 2002;Hillis et al., 2004;Rohde et al., 2016;Shams, Ma, et al., 2005;Trommershäuser et al., 2012).
A further key feature of Bayesian optimal integration is that prior beliefs are integrated with sensory cues into the final percept.These can, for instance, relate to the belief about statistical regularities in the feature's distributions, typically built up over repeated perceptual experience (Adams et al., 2004;Cicchini et al., 2018;Flanagan et al., 2008;Knill, 2007).For example, expecting to see a specific number of events, or assuming auditory and visual events to arise from the same source, can directly influence the final, multisensory, numerosity estimate (Odegaard et al., 2016;Wang et al., 2019;Wozny et al., 2008).By embedding prior experience in the perceptual estimation of stimulus features, the Bayesian integration framework allows to better understand how processes that are typically described as purely 'bottom-up' (sensory input-driven) and 'top-down' (belief and expectation-driven) can be conceived as a weighted combination of the evidence our perceptual system has at any given moment in time.

Multisensory integration is flexible to social context
The malleability of cue integration to sensory reliability and prior beliefs can have far-reaching implications, making perception susceptible to contextual influences.As such, it may explain why social contexts can directly affect perceptual integration (Bogdanova et al., 2021;Braithwaite et al., 2017;Dewe et al., 2018;Heed et al., 2010;Scheller & Sui, 2022a;Wahn et al., 2020).For instance, previous research showed that the degree of the SIFI can be modulated by the presence or absence of a conspecific (Wahn et al., 2020).Even more, the mere social relevance of stimuli (i.e., whether a stimulus relates to oneself or another person) can alter the degree of audio-visual integration in the SIFI (Scheller & Sui, 2022a).Specifically, when a certain visual numerosity is self-relevant, the illusory effect that is caused by a simultaneously presented but irrelevant auditory cue is decreased.This implies that self-relevance may bias perception towards the visual estimate.However, the mechanisms underlying these effects remain elusive.
Interestingly, the previously reported effect of self-relevance was dependent on the direction of the numerosity mismatch, that is, it was only significant in the fusion illusion (three flashes + two beeps), while the fission illusion (two flashes + three beeps) was influenced by social association to a lesser degree (Scheller & Sui, 2022a).This raises the question whether the social relevance effect in cue integration depends on different processing stages that lead to either fission or fusion illusions (Bolognini et al., 2011;Mishra et al., 2007Mishra et al., , 2008)), or whether it is better explained as a probabilistic prior that biases the integration of sensory information into a perceptual whole.In line with the latter notion, it has been shown that self-relevance can modulate prior expectancies of upcoming stimuli (Li et al., 2022;Sui et al., 2014) and that the self as a preceding spatial cue, compared to cues related to other people, can improve task performance by reducing the uncertainty of the location of upcoming targets (Liu et al., 2016).
The present study's aim to elucidate the mechanisms via which social relevance affects multisensory integration further taps into a critical question, which is how the self operates during perceptual processing (Golubickis & Macrae, 2022;Reuther & Chakravarthi, 2017;Scheller & Sui, 2022a, 2022b) leading to well-established self-reference/self-prioritization effects (Markus, 1977;Rogers et al., 1977).It has been argued that self-relevance modulates different stages of information processing through the activation of a self-representation (Sui & Humphreys, 2015a, 2015b), leading to the prioritization of self-relevant information.Studies dating back to the early days of cognitive psychology have demonstrated evidence on how the self affects attention, memory and decision-making (Moray, 1959;Rogers et al., 1977) and considerable evidence on these aspects has accumulated over the past few decades (e.g., Constable et al., 2019;Conway, 2005;Falbén et al., 2020;Humphreys & Sui, 2016;Scheller & Sui, 2022b).However, evidence on whether and how the self permeates to perceptual processing levels is inconclusive.
Social relevance effects on perceptual processing have direct implications for the way we interact with our environment.For example, imagine being at a party and trying to listen to a conversation with someone while music is playing in the background.Even though the music is not relevant to the conversation, your brain is automatically integrating the auditory information of the music with other sensory information, such as visual cues of the person's facial expressions and body language, creating a coherent experience.However, if a song comes on that you particularly like or dislike, your brain may prioritize this information over the conversation because it is more self-relevant to you.This is just one example of how self-relevance can automatically influence information processing in a multisensory context.By investigating the effects of self-relevance on multisensory integration, we gain a better understanding of how the self serves as an anchor/reference of accessing the external world, which has significant implications for various fields, including social cognition, neuroscience, and self-related mental and brain disorders (Liu et al., 2022;Noel et al., 2018;Northoff, 2016;Sui et al., 2021;Zhang et al., 2023).

Explaining self-relevance effects on fission and fusion: Different processing levels or uncertainty-dependent biases?
Comparing findings across previous studies indicates that fission and fusion illusions are not expressed equally strongly, but that their appearance may indeed be governed by different underlying mechanisms or stimulus characteristics (Andersen et al., 2004; for reviews see Hirst et al., 2020;Keil, 2020).For instance, larger stimulus eccentricity facilitates the occurrence of fission compared to fusion illusions (Chen et al., 2017).At the neural level, fusion has been shown to be largely influenced by mid-latency processing in parietal and superior temporal regions (Innes-Brown et al., 2013;Mishra et al., 2008), while fission additionally depends on earlier processing in visual and auditory sensory corteces (Bolognini et al., 2011;Mishra et al., 2010;Shams, Iwaki, et al., 2005).It is therefore possible that self-relevance, by funnelling attentional resources towards self-associated information, affects fusion by exclusively modulating later processing stages.This would assume that fusion, more than fission, could be penetrated by higher-order concepts such as social valuation.
However, while social association has been shown to mostly affect fusion, fission was still affected, albeit to a lesser extent (Scheller & Sui, 2022a).Specifically, the frequency of fission illusions was significantly modulated by social relevance in those individuals that generally showed a stronger processing facilitation towards self-associated information, suggesting a general malleability of the fission illusion.Furthermore, other studies showed that the fission illusion can be altered by selective attention (Kamke et al., 2012;Mishra et al., 2010) by enhancing the relative reliability of the visual stimulus (Odegaard et al., 2016).Finally, Wang et al. (2019) reported that, while the fission illusion is susceptible to numerosity expectations (prior belief to see a higher or lower number of events), the fusion illusion in their task was not affected by expectations.These mixed findings are surprising: would this suggest that socially induced numerosity relevance and numerosity expectation affect integration on fission and fusion trials via different mechanisms?After all, fusion has been comparably less studied and has been considered less stable than fission (Hirst et al., 2020;Wang et al., 2019).
While this is one possibility, the mixed findings may be better explained within a Bayesian optimal integration framework with a pre-existing numerosity prior.Indeed, previous studies reported that numerosity estimation is likely affected by numerosity priors, both for small (Philippi et al., 2011) and large number lines (Anobile, Cicchini, et al., 2012;Cicchini et al., 2014;Katzin et al., 2021;Pomè et al., 2021).During integration, such feature-specific (numerosity) priors may weigh into the integration of audio-visual information, biasing the final, integrated estimate towards their weighted, linear average (see Experiment 1 for more details).Below we show that, in the SIFI, this would predict a stronger expression of fission illusions for smaller numbers and a stronger expression of fusion illusions for larger numbers.Interestingly, this pattern can be found in the literature when comparing across studies: while the majority of SIFI studies that reported the fission illusion used 1 and 2 events (e.g., McGovern et al., 2014;Shams et al., 2002), the inclusion of more events increased the incidences of the fusion illusion (e.g., Andersen et al., 2004;Scheller & Sui, 2022a).What is more, by placing this framework into a social context, we can gain insights into the mechanisms that govern how the self affects perception.That is, self-relevance may affect integration in just the same way as a perceptual (numerosity) prior: by biasing the final percept towards the perceptual estimate that is most likely to occur and more relevant to us.

The present study
The present study tested whether self-relevance influences multisensory integration in similar ways to a perceptual prior in a central, Bayesian integration process.To that end, we conducted two experiments.
The first experiment outlined whether numerosity estimation can generally be well-described as Bayesian optimal percept with an added numerosity-specific prior.It demonstrates how biases and illusion frequencies can be explained by the uncertainty-dependent integration of sensory cues and numerosity-specific priors.To do so, we first quantified how sensory uncertainty, biases, and illusion frequency change with increasing numerosity in audio-visual numerosity estimation.The quantified relationship between visual uncertainty and numerosity was then implemented in a Bayesian optimal observer model to estimate prior parameters and predict the degree of bias in visual numerosity estimation (influence of the prior) as well as the relative frequency of fission and fusion illusions in audiovisual numerosity estimation.Note that, as we are interested in the integration of two sensory cues and perceptual priors under minimal conflict, the Bayesian optimal observer model was used instead of the extended causal inference model.While the latter is more commonly employed in recent years it is not primarily concerned with perceptual biases, but with the tendency to integrate or segregate information under conflict.Overall, the first experiment allowed us to derive a framework from which testable predictions about the expected effects of self-relevance could be drawn.It showed not only that a numerosity-specific prior can explain biases in unisensory perception, but how such bias can lead to asymmetries in the expression of fission and fusion illusion in multisensory contexts, depending on the number of events included in the task.
After showing that illusion frequencies can directly derive from the influence of perceptual (numerosity) priors, the second experiment tested whether self-relevance could affect multisensory processing via similar mechanisms.That is, it assessed whether the illusion that was more susceptible to the influence of priors was more strongly modulated by self-relevance.Such an illusion-specific modulation suggests that the self can enhance the expectancy of a specific numerosity, thereby shifting the perceptual prior towards areas of high self-relevance.If this was the case, self-association should reduce the fusion illusion if a larger number of events are included (as has been shown in Scheller & Sui, 2022a) and reduce the fission illusion when fewer events are included (Experiment 2, this study).Alternatively, if the self affects multisensory processing only at very specific levels in the processing hierarchy, self-association should not modulate the more frequent illusion, but only the fusion illusion, independently of how many events are included in the task.As such, the second experiment was able to arbitrate between two possible mechanisms via which social concepts can penetrate perception in multisensory integration.

E X PER I M EN T 1 -N UM EROSIT Y ESTI M ATION A S OP TI M A L PERCEP T W IT H CEN TR A L T EN DENC Y PR IOR
To outline how sensory uncertainty changes as a function of absolute numerosity and allows for a stronger bias, Experiment 1 consisted of a lab-based numerosity estimation task with 22 adult observers (15 female; 26.7 ± 5.3 years), who were asked to report the number of perceived events (flashes and beeps).Between one and five visual flashes were presented either alone (V) or combined with the same number of auditory beeps (V + A).Additionally, incongruent bimodal trials were included in which flash-beep combinations were presented with an additional beep or one less beep, allowing to measure the influence of audition leading to the fission or fusion illusion, respectively.Observers were instructed to report how many visual flashes they saw.To assure that they were able to determine the number of auditory events and to quantify uncertainty in both unimodal conditions, a final, blocked condition was conducted with only auditory (A) events in which the number of beeps had to be determined.
As this experiment was conducted to derive how uncertainty changes with increasing numerosity and to show how bias and illusion frequency can be explained by those changes within the Bayesian integration framework, no quantitative predictions were made that afford a sample size calculation.However, the patterns we observed in uncertainty, bias and illusion frequency were qualitatively similar when only 10 participants (as tested in Shams, Ma, et al., 2005) were included in the analysis, suggesting that the pattern of results remains stable within the group.

Stimuli and conditions
Visual flashes consisted of uniform light grey discs, approximately 2.47 cm in diameter, presented on a dark grey background.Each flash was presented for the duration of 21 ms, with a 63 ms inter-stimulus interval (ISI).Auditory beeps consisted of 3.5 kHz sound bursts, presented for the duration of 10 ms with a 58 ms ISI.Pink noise was played throughout the actual task to allow match presentation in the main experiment in which noise allowed for a more standardized signal-to-noise-ratio (SNR) across different setups (SNR 3-4kHZ = 5.4).Audio-visual event timings were aligned relative to the trail's midlatency.Each possible numerosity was presented 14 times per condition in an intermixed fashion.

Results
We measured perceptual biases and sensory uncertainty in each of the two unimodal (V, A) and the bimodal congruent (V + A) conditions for each observer.The perceptual bias refers to the mean deviation from the correct number of events.The uncertainty for each sensory cue was defined by the standard deviation of responses, measured for each presented numerosity.Error rate (absolute bias) and illusion frequency were assessed in the incongruent bimodal conditions, in which one more (fission) or less (fusion) beep was added to the audio-visual stimulus streams.
Results are summarized in Figure 1, showing that numerosity estimation was prone to biases (Figure 1a,b), with a small overestimation at lower numbers and a larger underestimation at higher numbers.A mixed effects model with the number of flashes and sensory condition as fixed effects and subject as random effect indicated that the bias significantly depended on the number of flashes (F(4,84) = 186.64;p < .001;see Figure 1b).This effect of number further differed significantly between sensory conditions (F(8,168) = 3.83; p < .001),with the strongest biases in the visual condition.Follow-up contrasts indicated that, in the auditory condition, there was no pronounced bias for one to three beeps ( p > .987),while four and five beeps were significantly underestimated ( p < .001).In the visual condition, one and two flashes were slightly, but not significantly, overestimated ( p < .174),while three flashes were marginally ( p = .056),and four and five flashes were significantly underestimated ( p < .001).In the congruent condition, one to three events were not significantly biased, with a tendency to be overestimated (1: p = .487;2: p = .192;3: p = .957).Four and five events were significantly underestimated ( p < .001) in this condition.
Uncertainty in numerosity estimation significantly depended on the number of events, increasing with higher numerosity (F(4,84) = 44.48;p < .001;see Figure 1b).This is in line with results from previous studies showing that uncertainty increased with larger numbers, both within the larger (Cicchini et al., 2014;Katzin et al., 2021;Pomè et al., 2021), and smaller number range (0-10; Philippi et al., 2008).This number-dependent increase in uncertainty further depended on sensory condition (F(8,168) = 2.1; p = .03).Follow-up contrasts showed that the visual and bimodal uncertainty was higher than auditory uncertainty for up to three events ( p < .005).For four events, uncertainty in the auditory condition increased such that the level of uncertainty in the auditory condition did not differ from the congruent ( p = .994)nor visual conditions ( p = .195).For five events, auditory uncertainty decreased slightly again, possibly because five events reached the upper range limit.The higher uncertainty in the bimodal compared to the auditory condition is not surprising, given that participants were asked to report the number of flashes and ignore the beeps.Auditory and visual uncertainty estimates from event numerosities one to four were used to derive the change in unimodal uncertainty with increasing numerosities for the framework presented in the following section.
Figure 1c indicates that the error rate (absolute bias) increased with numerosity, not only in the bimodal congruent but also in the incongruent trials.Notably, the error rate and illusion frequency, derived from slopes in Figure 1a, suggested that a lower number of events (1 vs. 2) increases the fission illusion, while a higher number of events (2 vs. 3, 3 vs. 4, 4 vs. 5) increases the fusion illusion.

Bayesian integration framework explains numerosity biases and illusion frequency
The present results can be well conceived within the Bayesian optimal integration framework, whereby biases (towards prior beliefs and expectations) exhibit a stronger influence on the final percept when the reliability of the sensory estimate is low. 1 These can either be general number biases such as central 1 Typically, an estimate's reliability is directly linked to the sensory system's precision in representing a given feature and is not assumed to change along with the feature.Rather, it has been assumed that the perception of magnitude (e.g., numerosity) follows a logarithmic relationship.However, recent research showed that the Bayesian framework with central tendency priors may better conceive spatial numerosity representation than logarithmic encoding (Anobile, Cicchini, et al., 2012;Anobile, Turi, et al., 2012;Cicchini et al., 2014Cicchini et al., , 2022;;Testolin & McClelland, 2021).
20448295, 0, Downloaded from https://bpspsychub.onlinelibrary.wiley.com/doi/10.1111/bjop.12683 by University Of Aberdeen The Uni, Wiley Online Library on [05/10/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License tendency priors, which have been evidenced for almost every quantity, including numerosity, size, space, and time (Anobile, Cicchini, et al., 2012;Anobile, Turi, et al., 2012;Aston et al., 2022;Cicchini et al., 2022), or even social or general attentional templates (Scheller & Sui, 2022a;Wang et al., 2019), which shift the expectation to perceive a certain number of events towards a specific numerosity.Priors describe how a person's perception tends to gravitate towards average (i.e., central tendency) or highly salient values within the perceptual space.
Within the Bayesian optimal integration framework, all available information that is not strongly conflicting 2 is combined in a reliability-weighted fashion (Ernst & Banks, 2002;Rohde et al., 2016;Shams, Ma, et al., 2005;Trommershäuser et al., 2012), meaning that the information with the lower uncertainty (higher precision) has a stronger influence on the final perceptual estimate than the information with the higher uncertainty (lower precision).The weights i that are applied to each piece of information i (audio, visual) are given by their respective uncertainties i via 2 If large conflicts in the two cues are present, information is less likely to be integrated by rather treated as independent cues.This is further described in the framework of the causal inference model (Körding et al., 2007;Shams & Beierholm, 2010) (1) The degree with which visual and auditory uncertainty increase is derived from data reported in Experiment 1 (see also Data S1).Note that we assumed the absolute uncertainty of the auditory stimulus to be higher than estimated in Experiment 1, in which auditory stimuli were attended while, in the present case, auditory stimuli are not attended (Odegaard et al., 2016).(e) Illusion strength of the fusion and fission illusions as a function of the number of flashes that are included in a given study (flash pairs, i.e., how many events are presented in contrast to each other).Illusion strength is derived from the model parameters (left panel) and empirically measured in Experiment 1 (right panel).Illusion strength is quantified as the deviation of the final estimate from the presented number of flashes.Negative illusion strength values in the fission illusion suggest a stronger pull of the prior than the auditory estimate on the final estimate of flashes.Hence, it describes how the strength of the prior-induced bias increases when visual uncertainty gets larger.To derive predictions of numerosity estimates under the model, we estimated the prior parameters (location P , and uncertainty P ) based on data of the visual-only condition, in which only visual evidence and prior information was available to the participant.The location and uncertainty of the prior was chosen such that it maximized the probability of the experimental data under the model, which assumed that all available information was fully integrated (according to Equations 1 and 2), and that visual uncertainty changed across the numerosity space.This was achieved with a negative log likelihood search over the possible parameter space ( P = [1:5], P = [0.1:2]), starting from 20 different locations to reduce the risk of getting stuck in local minima.The visual location parameter was given by the visual stimulus numerosity ( V = [1:5]) while the visual uncertainty parameter ( P ) was determined by the relationship of visual uncertainty and numerosity, derived across participants in Experiment 1.This relationship was modelled with a second-degree polynomial fit to the average uncertainty with which participants perceived one to four events (see Data S1 for more details).
The prior parameters were then used in a model that predicted the bias and illusion strength in those conditions in which auditory and visual stimuli were presented together (congruent, fission, and fusion trials).Here, the auditory location parameter was again given by the stimulus numerosity ( A = [1:5]).The auditory uncertainty for a specific numerosity ( A ) increased with the number of events n, derived from Experiment 1. Crucially, as participants were asked to report the number of visual flashes, and not auditory beeps (as in the auditory condition of Experiment 1), the relationship measured in Experiment 1 was adjusted.Specifically, the relative auditory uncertainty was increased to give more weight to the visual estimate, however, the rate of change in uncertainty across the number line was kept consistent (see Data S1 for more details).
The mean estimated location parameter ( P = 2.33 ± 0.63; mean ± SD) indicated the presence of a central tendency prior, which can be expected for this stimulus range (Anobile, Cicchini, et al., 2012;Aston et al., 2022;Pomè et al., 2021).This prior location further allows to explain the reduction in bias in the visual and combined conditions between 2 and 3 events in the previous study (see Figure 1b).
Figure 2a demonstrates how the probability of perceiving a certain number of flashes shifts towards different numerosities in the presence of this number-specific prior (σ P = 0.7 ± 0.33; mean ± SD). 3 Here, most participants would expect to see between 2 and 3 flashes on most trials (also indicated by the data in Experiment 1), but if the visual uncertainty is low (more reliable), the prior only has a weak influence on the final estimate.As visual uncertainty increases with increasing stimulus number n, the prior exhibits a stronger influence on the final estimate towards 2-3 flashes.The average reported number of visual flashes is plotted as red dashed line in Figure 2a and indicates a good fit of the predicted and measured biases across conditions (β = 1.07;R 2 = .970).Model comparisons further indicated that the model with numerosity-specific prior provided a better fit for the data than a model without numerosityspecific prior (ΔAICc = −705 ± 172; ΔBIC = −660 ± 171; mean ± CIs 95 ; see Data S1 for more information).
When sound is added, it biases the final numerosity judgement towards the presented number of beeps by its respective reliability (Figure 2b,c).As the relative reliability of the auditory estimate increases, so does its impact on the final percept -yet, as its uncertainty increases, the weights of the visual and auditory estimates become not only more similar but also approach that of the prior.Hence, by combining conflicting auditory and visual estimates with a central tendency prior, the frequency with which an extra illusory flash (fission) or a flash less (fusion) is perceived is not equal across the number line.Instead, as uncertainty increases with larger event numerosities, the strength of the bias increases, leading to a larger fusion illusion strength (Figure 2b,e).For fewer event numerosities, uncertainty is low while the bias is reversed (higher number bias), leading to a stronger fission illusion (Figure 2c,e).Empirically, this is reflected in the frequency (Figure 1c) and strength (Figure 2e) of illusions when different numbers of flashes are used in the study design.

E X PER I M EN T 2 -EF F EC TS OF SOCI A L R EL E VA NCE ON M U LT ISENSORY I N T EGR AT ION
Showing that the strength and frequency of the fission and fusion illusions, two markers of multisensory integration, are well represented by the Bayesian framework with numerosity prior, Experiment 2 more specifically tested how social relevance influences this integration: Following the idea that self-relevance biases numerosity estimation as an additional prior that enhances the saliency of the associated numerosity (e.g., via selective attention or expectancy; Odegaard et al., 2016;Wang et al., 2019) we would expect to find modulations of self-relevance in the fusion illusion when presenting higher numbers, and in the fission illusion when presenting lower numbers.Again, as the relative uncertainty of the visual estimate increases with larger numbers, the strength of the bias increases, leading to a larger illusion strength on fusion compared to fission trials.This bias could then be reduced if visual reliability is enhanced.However, whether the effect takes place in the fission or fusion illusion depends on the relative placement of the prior to the numerosity (1 & 2 flashes: fission > fusion; 2 & 3 flashes: fusion > fission).
Alternatively, if self-relevance affects integration at a specific, later processing stage, with fission being more prone to low-level sensory characteristics while only fusion can be modulated by higherlevel concepts (Hirst et al., 2020;Mishra et al., 2007), we would expect to find modulations of social salience in the fusion illusion, irrespective of the number of events being presented.
To test these two competing ideas, we replicated the first two experiments presented in Scheller and Sui (2022a) in a single experiment with fewer events.Based on the predictions made by the Bayesian framework with numerosity prior, we would expect more frequent fission illusions at baseline when 1 and 2 events were used, while more frequent fusion illusions at baseline were reported when 2 and 3 events were used (Scheller & Sui, 2022a).

Participants
Eighty adult participants (55 females, mean ± SD age 26.1 ± 4.8 years) took part in this study.All participants had normal or corrected-to-normal vision.Forty participants were randomly assigned to a group that associated the own social identity with the lower numerosity, while the other 40 participants were assigned to a group that associated the own social identity with the higher numerosity (and vice versa for the other social identity).To allow for comparability with effects reported for two and three events the sample size was matched to our previous study (see Scheller & Sui, 2022a, for sample size estimation).Participants were recruited online through Prolific, University advertisements, and word of mouth.The study was approved by the University of Aberdeen Psychology Research Ethics Committee (#4669/2021/1), and all participants gave informed consent.

Stimuli & conditions
All events consisted either of one or two successive auditory beeps or visual flashes (unimodal), or their combination (bimodal).Except for the absolute number of events (one vs. two), all stimulus characteristics were kept consistent with those reported in Scheller and Sui (2022a).Bimodal events were either presented congruently, i.e., same number of visual and auditory events, or incongruently, i.e., different number of auditory compared to visual events.Unimodal and bimodal congruent trials were used to determine self-prioritization effects in the present sample.Incongruent trials were used to determine the malleability of multisensory integration by social association and were further split into those trials in which an additional auditory beep was presented with a single visual flash (fission) and trials in which two visual flashes were paired with one less auditory beep (fusion).

Procedure & design
The first part of the experiment consisted of a baseline condition (number task) that asked participants to judge how many flashes were presented.To maintain control of the decisional dimension all judgements were given via label-matching, that is, a number label was shown, and participants judged whether the label matched the number of events that were shown.In the second part of the experiment, participants repeated the same task after associating two identities ('self' or 'other participant') with the two visual numerosities (e.g., self was associated with one flash, the other participant with two flashes).They then had to judge how many flashes were presented by reporting the associated identity via labelmatching: Similar to the baseline condition, a label was shown, however, this time it was an identity instead of a number.Participants had to indicate whether the label matched the identity that was associated with the event numerosity.
Labels were always shown 500 ms after the offset of the stimulus.Equal proportions of matching and mismatching labels were shown for each condition, and similar to previous studies (see Scheller & Sui, 2022a;Sui et al., 2012;Sui & Humphreys, 2017;Verplanken & Sui, 2019), matching trials were analysed.The individually assigned identity-numerosity mappings (self = 1, other = 2 or self = 2, other = 1) were counterbalanced across the sample (n S1O2 = 40, n S2O1 = 40) to dissociate effects of social association from the absolute number at group-level.

Data analysis
Data analysis was split into three sections.First, to confirm that the present sample expressed selfprioritization effects in numerosity estimation, logistic mixed effect models were applied to accuracy data across sensory conditions.In all models, task type (number vs. social) and social identity (self vs. other) were included as fixed factors.Participants were assigned as random intercepts, to reflect the repeated measurement for each sensory condition at the participant level.Self-prioritization effects would be expressed by an interaction of task type and social identity, with self-associated numerosities being responded to more accurately than other-associated numerosities in the social, but not the number task.Note that the 'social identity' in the number task refers to the respective number of events that was associated with this identity (i.e., 50% of participants associated the self with one event and 50% with two events) and does not equate to a specific number of events.
Second, we established whether the relative frequency of the fission and fusion illusions were reversed across the two experiments, as predicted by our data from Experiment 1 and the Bayesian framework with numerosity prior.To that end, we compared the frequency of fission and fusion illusions at baseline in the present experiment with the frequency of fission and fusion illusions at baseline in experiments 1 and 2 reported in Scheller and Sui (2022a), which used 2 and 3 events (each n = 80).While the relative frequency of fusion illusions was higher in the latter experiment, a numerosity prior between 2 and 3 events would assume that the relative frequency would be reversed in the present experiment, i.e., a higher frequency of fission illusions can be expected.
Third, to test the main hypothesis of whether self-relevance exclusively affects the fusion illusion at higher processing levels or biases a central integration process via a social salience prior, we measured the influence of self-association on the strength of fission and fusion illusions.To that end, we applied logistic, mixed effect models to accuracy data of the bimodal incongruent conditions, separately for those trials in which an additional sound was presented (inducing the fission illusion) and for the trials 20448295, 0, Downloaded from https://bpspsychub.onlinelibrary.wiley.com/doi/10.1111/bjop.12683 by University Of Aberdeen The Uni, Wiley Online Library on [05/10/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License in which one less sound was presented (inducing the fusion illusion).Again, an interaction effect of task type and social association would indicate that social salience (the different relevance applied to self-and other-associated information) would affect the integration of audio-visual information.If an interaction effect was present in the fusion illusion, this would provide evidence for the hypothesis that self-relevance affects specific stages of processing.If an interaction effect was present in the fission illusion, this would provide evidence for the hypothesis that social relevance biases a central integration process similar to an expectation prior.To allow for a more direct comparison, the results from Scheller and Sui (2022a) are presented alongside the present results.

F I G U R E 3
Top row shows numerosity discrimination accuracy (marginal effects of interaction terms) as a function of task type (number vs social task) and associated social identity (self vs. other) in the present experiment, in which 1 and 2 events were used.For direct comparability, the bottom row shows the same data reported in † Scheller and Sui (2022a) in which 2 and 3 events were used (data from experiments 1 and 2 in that study, which were identical, were combined.Hence data from the present experiment and Scheller & Sui, 2022a, amounts to n = 80 each).All parameters are identical between the two studies, with the difference in the number of events used.Significant interactions indicate that numerosity discrimination estimates were influenced by social salience.**p < .01;***p < .001.For comparison, using 2 and 3 events, the previous study found significant self-prioritization effects in the bimodal congruent (OR (7980) = 2.185, CI 95 [1.59 3.00], p < .001)and auditory (OR (3191) = 2.69, CI 95 [1.59 4.55], p < .001),but not the visual modality (OR (3192) = 1.229,CI 95 [0.84 1.80], p = .287).The latter was due to a pre-existing bias in the visual number task ( p = .048).The significant interactions in the bimodal congruent and auditory modalities were driven by self-associated numerosity being responded to more accurately than other-associated numerosity in the social task (bimodal congruent: p < .001;auditory: p < .001)but not in the number task (bimodal congruent: p = .354;auditory: p = .853).
To establish whether the frequency of the fission and fusion illusions changed with a different number of events we compared the illusion frequencies at baseline between the present experiment and that reported in (Scheller & Sui, 2022a), which only differed in the number of events.The frequency of the two illusions strongly depended on the number of events being presented (OR (640) = 1.243,CI 95 [1.16 1.33], p < .001;see Figure 4a), as indicated by a logistic mixed effects model with illusion type and event numerosity as fixed and participant nested within each experiment as random effect.When one and two events were presented, fission was more frequently experienced than fusion ( p < .001).When two and three events were presented, fusion was more frequently experienced than fission ( p = .002).Note that all other factors were kept consistent between the two experiments.As differences in the frequency with which each illusion was experienced were suggesting that perceived numerosities were biased towards a number-specific prior (see Bayesian integration framework explains numerosity biases and illusion frequency), we next tested whether, in the present experiment, the fusion illusion or the fission illusion was modulated by self-relevance.As would be predicted by a social salience prior in the Bayesian framework, the fission illusion was modulated by self-relevance in the present task, as shown by a significant interaction of task type and social identity association (OR (3983) = 0.629, CI 95 [0.46 0.85], p = .003;see Figure 4b).Here, the frequency of illusions was more strongly reduced, relative to the number task, when the correct number of flashes was associated with the self ( p < .001)while the benefit of social association with the other participant was smaller ( p = .012).In contrast, self-relevance did not affect the fusion illusion in the present study (OR (3986) = 0.962, CI 95 [0.68 1.36], p = .828;see Figure 4b).

DISCUS SION
A large part of our perceptual experience grounds in the integration of information within and across our different senses, providing a rich and coherent representation of the world around us.Due to its direct behavioural relevance, the perceptual integration of sensory information needs to retain a certain degree of contextual flexibility.Previous research showed that multisensory integration can be influenced not only by the integrated history of perceptual experiences (Beierholm et al., 2009;Cicchini et al., 2014;Ernst, 2007;Gau & Noppeney, 2016) but even by the social salience that is attached to this information (Scheller & Sui, 2022a).Specifically, the influence of a third, irrelevant auditory sound on two visual flashes, leading to the audio-visual fusion illusion (a marker of multisensory integration) was reduced as a result of mere self-association with the visual numerosity.The present study tested whether self-relevance affects multisensory integration in the same way that priors do in a central, Bayesian integration process.That is, it tested whether self-relevance reduces the fission or fusion illusions depending on the sensory uncertainty associated with the visual numerosity estimate (and hence the illusion frequency), rather than modulating only the fusion illusion at higher processing levels.
Contrasting reports of illusion frequency from previous studies using the SIFI (e.g., Andersen et al., 2004;McGovern et al., 2014;Shams et al., 2002) suggested differences in the relative frequency of fission and fusion illusions, depending on the number of events that were included in the study design.However, no study has explicitly tested this before.Experiment 1 assessed how a perceptual numerosity estimate can be derived from auditory and visual information, when the sensory information in both modalities provides redundant, matching information, and when it provides conflicting information.In line with previous research, suggesting that numerosity estimation is affected by numerosity-specific priors (Anobile, Cicchini, et al., 2012;Anobile, Turi, et al., 2012), we showed that a Bayesian optimal observer framework (Ernst & Banks, 2002;Rohde et al., 2016;Shams, Ma, et al., 2005) with central tendency prior can explain how perceptual biases change across the number line.This framework did not only allow to explain the perceptual biases in the visual-only condition but also the strength of the fission and fusion illusions when a mismatching auditory event was presented.
When flashes or beeps were presented alone, both the model and empirical data showed that sensory uncertainty increased with increasing numerosity, allowing for a stronger influence of the numerosityspecific prior.Increases in sensory uncertainty with higher event numbers are not surprising, given that the temporal estimation of numerosity requires the serial integration of temporal event contours (onset, offset) over varying periods of time, thereby not only accumulating evidence but also noise (Martin et al., 2017).The increase in sensory uncertainty led to large underestimation biases when more events were presented, while the underestimation bias for fewer events was small.This dependency of bias directionality and strength on numerosity was most pronounced in the visual cue, which was less reliable than the auditory cue, in line with previous research (Shams, Ma, et al., 2005).Indeed, several studies suggested that the increasing uncertainty, in conjunction with specific biases such as a central tendency prior, are better able to explain the apparent logarithmic numerosity mapping that is commonly represented by Weber's law (e.g., Anobile, Cicchini, et al., 2012;Anobile, Turi, et al., 2012;Cicchini et al., 2014Cicchini et al., , 2022;;Testolin & McClelland, 2021).
Based on the Bayesian optimal observer model with central tendency prior, we expected the fission illusions to be most prominent when one or two events were presented, and to decrease when more events were presented.The fusion illusion, in contrast, was expected to increase when more events were presented.Our empirical data from both experiments confirmed this.Together, the results from Experiment 1 showed that the relative frequency of either illusion can be well-described in the Bayesian optimal observer framework with numerosity-specific prior.
Based on this finding, we were able to test whether self-relevance affected multisensory integration either by modulating a single specific processing level, governing only the fusion but not the fission illusion, or whether it may be better represented by a shift in the numerosity-prior towards the number of high self-relevance, thereby affecting the illusion that was more prevalent depending on the number of events.In a previous study, using 2 and 3 events, social relevance largely modulated the fusion illusion (Scheller & Sui, 2022a).Results from Experiment 2 in the present study extended this finding, showing that, when 1 and 2 events were used, social relevance modulated the fission illusion, i.e., the illusion that was more strongly influenced by the numerosity prior.This suggests that, rather than targeting an isolated, specific process governing the fusion illusion, social relevance affects multisensory integration by altering the influence of perceptual priors, leading to a reduction of illusion frequency for self-associated information.In other words, the self acts like a prior in biasing the multisensory percept towards the stimulus space of high self-relevance (self-associated numerosity).As a result, the self-associated information is given a stronger weight in the integration process, leading to a final percept that is more strongly based on self-, rather than other-associated information.
Our data show a stronger influence of the self-associated sensory information on the final percept, specifically when sensory uncertainty is high, suggesting that the self acts like a prior in the integration process.Within the Bayesian integration framework, there are alternative mechanisms that may lead to the influences of social representations on cue integration, and should be considered.For instance, it is possible that social relevance directly alters the sensory uncertainty with which a specific numerosity is represented in the visual modality.Here, it may decrease the sensory uncertainty associated with the visual estimate through a two-stage integration process, similar to the hierarchical ordering of forced fusion and causal inference (Aller & Noppeney, 2019;Körding et al., 2007;Rohe et al., 2019;Rohe & Noppeney, 2015;Shams & Beierholm, 2010): by being integrated with the sensory estimates either before or after it is integrated with the numerosity-specific prior.Hence, it is unclear whether the self is represented as its own prior, or whether it shifts the location of the perceptual central tendency prior towards areas of high self-relevance.Alternatively, by selectively enhancing early attentional filtering towards the self-associated numerosity, processing resources may be used more efficiently, increasing the perceptual precision with which this number is represented (Odegaard et al., 2016).While the two possibilities that the self acts as a prior, or directly reduces uncertainty in the sensory estimate, are difficult to disentangle behaviourally, the present study applied social relevance to the stimulus dimension rather than to a sensory modality.It thereby mimics an additional prior (enhancing the salience of a specific numerosity) rather than changing in reliability of a specific sensory channel (enhancing the salience of information in one sensory modality).Hence, the present results are more in line with the interpretation that the self affects multisensory integration via shifts in priors.
One might wonder whether the observed data may be better explained by the causal inference model, that is, via changes to the prior that participants assume about the causal structure of the two cues.For instance, self-association may modulate the belief that the two cues arise from the same source, thereby enhancing the fusion or segregation of the auditory and visual cues.Indeed, the causal inference model has been repeatedly shown to describe multisensory perceptual processes better than forced fusion (Körding et al., 2007;Shams & Beierholm, 2010, 2022;Wozny et al., 2008).Furthermore, recent research in social cognition suggests that self-association enhances feature binding, which fits with the assumptions of causal inference (Lee et al., 2021;Schäfer et al., 2016;Sui & Humphreys, 2015a, 2015b).However, there are a number of reasons that render this explanation unlikely.Firstly, the biases that are observed in small conflicts, such as the ones that we used in the incongruent audio-visual condition (±1 event), are typically well-described with the full integration (forced fusion) model, while larger conflicts are necessary to establish the conditions under which cues are progressively segregated (Shams & Beierholm, 2011).This is further supported by our data showing that predictions that directly derive from full, reliability-weighted integration of auditory and visual cues (Equations 1 and 2) align well with the combined estimates in the incongruent cue pairings (biases) in Experiment 1 (see Figure 2).Effectively, this supports the notion that numerosity information from the two modalities was fully integrated by the relative sensory reliabilities.Secondly, while the causal inference prior is concerned with the perceiver's assumed causal structure of the environment, it does not make predictions about the changes in bias with increasing numerosity.Only perceptual priors or scaling differences in the feature dimension (numerosity) can account for the observed increases in uncertainty and biases across the number line (Anobile, Cicchini, et al., 2012;Cicchini et al., 2014Cicchini et al., , 2022)).Therefore, socially induced changes in the causal inference prior would not be able to explain that the directionality and strength of the bias depends on numerosity.Instead, the present findings are best explained by perceptual priors that can be modulated by social association.However, whether social association can modulate priors at different levels, such as beliefs about the causal structure of the environment, remains an interesting question for future investigations.
To ascertain that social association led to self-prioritization in numerosity perception, a prerequisite of its effects on multisensory integration, we tested the effects of social association on accuracy with which numerosity was perceived using unimodal and bimodal congruent stimuli.Results indicated robust self-prioritization effects: self-associated numerosities were responded to more accurately than other-associated information, only in the social but not the number (baseline) task, replicating findings from Scheller and Sui (2022a).However, in contrast to the aforementioned study, the present results also indicated self-prioritization in the visual-only condition that was even stronger than that in the auditory condition.This was due to an absence of an identity-based effect at baseline, which was present in Scheller and Sui (2022a).This suggests individual-specific differences in the reliance on prior information when higher numerosities are used.As higher numerosities (in the temporal domain) are perceived with higher uncertainty, the influence of perceptual priors is more pronounced, and so may be individual differences in the relative reliability of prior and sensory uncertainty.
Lastly, our findings lend support to the account of the 'integrative self' (Sui, 2016;Sui & Humphreys, 2015b), which assumes that the activation of our self-representation directly modulates perceptual-cognitive processing stages (Scheller & Sui, 2022b).Specifically, our findings suggest that the self-representation can alter perception by enhancing the relative reliability of perceptual priors during cue integration.
Taken together, our findings provide evidence that social salience affects multisensory integration by biasing the combined percept towards areas of high self-relevance, in line with the Bayesian optimal observer framework.Notably, conceptualizing the self as a prior (information that ought to be expected) may allow to explain not only multisensory integration but also a variety of effects that self-association has on different levels of information processing, including attentional filtering, perception, memory, decision-making, and action planning (Desebrock & Spence, 2021;Falbén et al., 2020;Humphreys & Sui, 2016;Scheller & Sui, 2022b;Sui & Humphreys, 2015b;Woźniak et al., 2018).It emphasizes that the self plays a key role in our experiences and affects how we process information to generate a coherent perception of us and our environment.It furthermore highlights the malleability of perception to higher-order concepts and social factors.

AU T HOR C ON T R I BU T IONS
Meike Scheller: Conceptualization; data curation; formal analysis; investigation; methodology; project administration; software; supervision; validation; visualization; writing -original draft; writing -review and editing.Huilin Fang: Data curation; investigation; validation; writing -review and editing.Jie Sui: Conceptualization; funding acquisition; resources; supervision; validation; writing -review and editing.

1
Results from numerosity estimation task in Experiment 1 (N = 22) using between 1 and 5 events.(a) Figure shows the perceived number of flashes as a function of condition (line type) and presented number of flashes (different colours).Dotted lines indicate perceived number of beeps when only beeps were presented, while dashed lines indicate the perceived number of flashes when only flashes were presented.Solid lines, means and error bars (95% confidence intervals; CIs 95) indicate reported number of events when both flashes and beeps were presented together, either congruently (at 0) or with one beep more (at 1) or less (at −1).Biases in number estimation become more evident with higher numbers, as can be seen by the larger deviation of the reported from the presented numerosity, especially for four and five events.(b) Mean perceptual bias and uncertainty (response SD), averaged for the 22 subjects, are shown as a function of the number of events presented, split by sensory condition.Shaded areas indicate CIs 95 bands.(c) Upper panel shows error rate (absolute bias) for the different illusion conditions and the congruent condition.When auditory and visual event numbers were matched (no illusion), the error increased with increasing numerosity.When one additional auditory event was presented, the fission illusion was perceived, especially for low numerosities.When one less auditory event was presented, the fusion illusion was perceived, especially for higher numerosites.Error bars indicate CIs 95 .Bottom panel shows the mean (+SEM) predicted fission and fusion strengths for the two numerosity combinations used in the present study and our previous study(Scheller & Sui, 2022a), based on Experiment 1 data.Illusion strength was estimated from individuals' regression slopes between the congruent and respective illusion conditions (i.e., slope of solid lines in panel (a), estimated for each individual participant; see alsoBresciani et al., 2006).The maximum a posteriori probability estimate ( com ) of the combined posterior distribution, indicated by the black dashed line in Figure2, indicates the most likely percept and is given by the weighted sum of all estimates i Probability density distributions of perceiving a specific number of flashes when (a) only visual flashes are presented or when visual flashes are paired with (b) one sound less or (c) one additional sound.Visual and auditory probability distributions are centred on the numbers of flashes that were presented.In all cases, a numerosity-prior (grey distribution; P = 2.33; P = 0.7) was added, biasing the final perceptual numerosity estimate towards the centre of the response space.Visual and auditory uncertainty, indicated by the width of the blue and red probability distributions, respectively, increase with increasing numerosity.Location estimates for the combined condition ( com ), derived from Equation2, are indicated by black vertical dashed lines and are numerically reported above the respective posterior probability distribution (green).Red vertical dashed lines indicate the mean numerosity estimates in Experiment 1, showing that the model describes the observed data well.Note that the parameters chosen for this demonstration are based on the data collected in Experiment 1 and depend on the specific task parameters.(d)

F
I G U R E 4 (a) Panel indicates the frequency with which each of the two illusions (fission, fusion) was perceived, depending on the number of events that were discriminated ( † data taken fromScheller & Sui, 2022a).(b) Panel shows the illusion frequency (marginal effects of interaction terms) for fission trials (left), in which an additional sound was presented, and fusion trials (right), in which one less sound was presented, as a function of task type (number vs social) and social identity (self vs other) in the present task.(c) Panel shows data from† Scheller and Sui (2022a), where 2 and 3 events were used, for comparison.Error bars show CI 95 .Significant interactions indicate that integration of audio-visual numerosity estimates was influenced by social salience.**p < .01.