Carrier‐frequency specific omission‐related neural activity in ordered sound sequences is independent of omission‐predictability

Regularities in our surroundings lead to predictions about upcoming events. Previous research has shown that omitted sounds during otherwise regular tone sequences elicit frequency‐specific neural activity related to the upcoming but omitted tone. We tested whether this neural response is depending on the unpredictability of the omission. Therefore, we recorded magnetencephalography (MEG) data while participants listened to ordered or random tone sequences with omissions occurring either ordered or randomly. Using multivariate pattern analysis shows that the frequency‐specific neural pattern during omission within ordered tone sequences occurs independent of the regularity of the omissions. These results suggest that the auditory predictions based on sensory experiences are not immediately updated by violations of those expectations.


| INTRODUCTION
Even if people are not consciously aware of regularities in the environment, the brain does pick up on those and forms predictions accordingly.Such predictive processing can be seen in different modalities, for example, touch (e.g.Dercksen et al., 2023;Yu et al., 2022), vision (Goujon et al., 2015) and hearing (Winkler et al., 2009).
At a fundamental level, predictive processing enables us to form valid models of our environments (or updating them when necessary) as well as the consequences of our actions (Friston, 2010).An example par excellence is the learning of speech in early life, with studies showing the astounding statistical learning capacities of infants (Saffran, 2020).Predictions can also be 'stubborn'; that is, old predictions are not updated based on unexpected and omitted outcomes (Yon et al., 2023).Abnormal predictive processing, especially an overweighting of predictions, may be a crucial mechanism that enables the development of phantom perception (Partyka et al., 2019;Sterzer et al., 2018).In many everyday settings, predictive processing especially in an anticipatory sense-that is, Abbreviations: ERF, event related field; F, frequency prediction; fMRI, functional magnetic resonance imaging; MEG, magnetoencephalography; MMN, Mismatch negativity; MVPA, multivariate pattern analysis; SD, standard deviation; SOA, stimulus onset asynchrony; T, temporal prediction.
relating to future events-enables more efficient processing, such as in iconic cocktail party settings (Schubert et al., 2023).
In auditory cognitive neuroscience, there has been a long and established tradition of experimental paradigms that probe predictive processes looking at event-evoked violations of regularity (so-called oddball-paradigms; Garrido et al., 2009;Schröger, 1998).Statistical regularity and by extension putative predictive auditory processing can be parametrically manipulated via so-called Markov sequences using, for example, pure tones (Barascud et al., 2016).Applying multivariate pattern analysis techniques, including time-and condition generalization (King & Dehaene, 2014), we could recently show how increasing regularity of the pure tone sequence leads to increased feature specific neural activity (Demarchi et al., 2019).This was observed in particular prior to the onset of the anticipated sound (presented at 3 Hz) and following so-called omissions.The illustration of sound feature specific neural activity during 'surprising' silent periods goes beyond the demonstration of a general reaction of the brain, known as evoked or induced omission responses (Raij et al., 1997;Todorovic et al., 2011).For the latter, using a repetition suppression design, Todorovic and de Lange (2012) and Todorovic et al. (2011) showed that the omission of a repetition leads to stronger responses when it was unexpected as compared with when it was more likely.Expectancy in this case was manipulated by overall frequency of the omission, either being frequent or rare depending on the condition.However, expectancy could also be manipulated via temporal predictability, keeping the overall frequency of omissions constant.Especially, how the temporal predictability of the occurrence of an omission influences the features specificity of neural responses as shown in our previous work (Demarchi et al., 2019) is unknown.
The goal of the present study is to address this open question.For this purpose, we presented regular and random tone sequences in which 10% omissions were embedded that were either strictly regular (i.e.every 10th tone) or appeared (pseudo-)randomly while recording MEG data.For ease of navigating through the manuscript, we will call the regularity of the tone sequence frequency prediction (F+) and the regularity of the omissions temporal prediction (T+), even though those terms are only rough approximations of the underlying processes.Our results show that regular sequences of tone frequencies are reliably linked to enhanced feature specific neural responses during omissions, irrespective of whether it was predictable or not.This suggests, predictions about upcoming tones are not reliably updated even though omissions violate those predictions (cf.stubborn predictions for visual cortex, Yon et al., 2023).

| Participants
Forty-nine (23 female, 26 male) native German speakers participated in this study.Their age ranges from 18 to 37 (Mean = 23.32,SD = 3.66).Participants reported normal vision and hearing.Participation was voluntary and in line with the declaration of Helsinki and the statutes of the University of Salzburg.All participants provided informed consent and were financially compensated for participation or via study credit.The study was approved by the ethical committee of the University of Salzburg.

| Stimuli and procedure
Stimuli and procedure are similar to the ones in the study by Demarchi et al. (2019).Auditory stimuli were presented binaurally using MEG-compatible pneumatic in-ear headphones (SOUNDPixx, VPixx technologies, Canada) in sequences.These sequences were composed of four different pure tones, ranging from 200 to 2000 Hz, logarithmically spaced (200 Hz, 431 Hz, 928 Hz, 2000 Hz) each lasting 100 ms (5 ms linear fade in/out).Tones were presented at a rate of 3 Hz.Overall eight blocks were presented to participants, each containing 1600 stimuli.Each block was balanced with respect to the number of presentations per tone frequency (400 per tone).Within the block, 10% of the stimuli were omitted, thus yielding 400 omission trials (100 per omitted sound frequency).While within each block, the overall number of trials per sound frequency was set to be equal, blocks differed in the order of the tones and omissions.In more detail, there were four conditions, based on the order of tone sequence (ordered and random) and omission sequence (ordered and random).Random tone condition (see Figure 1b) was characterized by equal transition probability from one sound to another, thereby preventing any possibility of accurately predicting an upcoming stimulus.In the ordered tone condition, presentation of one specific sound was followed with high (75%) probability by another specific sound thereby generating frequencyspecific predictions.The probability for self-repetitions was set to 25% in both tone sequence conditions.The random omission condition had the omissions occurring randomly, whereas in the ordered omission condition, every 10th tone was omitted (balanced for the four different tones) thereby generating temporal predictions.Importantly, the absolute number of omissions was the same for both omission sequences.This resulted in four conditions in total (see Figure 1a,c): frequency and temporal prediction (F+T+, when both tones and omissions occurred regularly), no frequency and no temporal prediction (FÀTÀ, when both tones and omissions occurred randomly), frequency but no temporal prediction (F+TÀ, when tones occurred regularly but omissions randomly) and no frequency but temporal predictions (FÀT+, when tones occurred randomly but omissions regularly).During the stimulation, a video of Cirque du Soleil ('Worlds Away') was silently presented.The experiment was programmed in MATLAB 9.1 (The MathWorks, Natick, Massachusetts, USA) using the open source Psychophysics Toolbox (version 3.0.14,Brainard, 1997).

| Data acquisition and pre-processing
MEG data were recorded using a 306-channel Triux MEG system (Elekta-Neuromag Ltd., Helsinki, Finland) from 102 locations with 102 magnetometers and 204 planar gradiometers in a magnetically shielded room (AK3B, Vakuumschmelze, Hanau, Germany).The MEG signal was recorded at a sampling rate of 1000 Hz and high-pass filtered at .1 Hz and low-pass filtered at 330 Hz.Prior to the experiment, the head shapes of each participant were digitized including fiducials (nasion, pre-auricular points) as well as around 300 points on the scalp using a Polhemus Fastrak Digitizer (Polhemus).We used a signal space separation algorithm as provided by the MEG manufacturer and implemented in the Maxfilter program (version 2.2.15) to remove external noise from the MEG signal (mainly 16.6 Hz from the nearby train and 50 Hz plus harmonics) and realign data to a common standard head position (across different blocks based on the measured head positions at the beginning of each block).
Offline, data were analysed using the Fieldtrip toolbox (Oostenveld et al., 2011).To reduce computational resources, only magnetometers were further analysed.A .1-Hz high-pass filter and a 30-Hz low-pass filter were applied, and the delay between trigger and stimulus onset due to the tubes delivering the acoustic signal was corrected.Data were cut into segments around the single tones and omissions (100 ms prestimulus and 300 ms poststimulus) and then resampled to 250 Hz.Data were then either averaged time-locked to the omission onset for the event related field (ERF) analysis or used for decoding.

| Decoding
All decoding was done using MVPA Light (Treder, 2020).Accuracy was calculated with a multiclass LDA classifier and the preprocessing settings of 'demean' and 'undersample'.Data were always trained on tones from the condition with random tone sequence and random occurrence of omissions (FÀTÀ, non-overlapping trials with testing data).
For a first analysis of omission cross-decoding, testing data was chosen as trials corresponding to all F+ and FÀ omissions independent of the regularity of the omissions.We then also calculated the cross-decoding of the conditions separately for regularity of omissions and tone sequence, resulting in four conditions (random tonesrandom omissions FÀTÀ, random tones-ordered omission FÀT+, ordered tones-random omissions F+TÀ, ordered tones-ordered omissions F+T+).Data were smoothed across three time points.

| Statistics
Decoding of all F+ omissions was contrasted with decoding of all FÀ omissions using cluster-based permutation (0-300 ms, 1000 repetitions, two-tailed, alpha .05).To compare the influence of the regularity of the omissions on this effect (i.e.temporal prediction), we extracted the individual differences of decoding accuracy between F+ and FÀ separately for T+ and TÀ for the whole statistical cluster of the contrast between F+ and FÀ.We analysed those differences with t tests and with Bayes Factor.This was also done for the individual differences underlying the maximal statistical value.
The same statistical analysis was done for the ERFs.

| Control analysis
In order to control for possible decoding effects carried by the previous tones, we recoded the omission periods according to the previous tone.We then trained the data again with the random tones from the FÀTÀ sequences and tested with omission periods coded for the previous tone separately for F+ and FÀ sequences (as was done for the main analysis).The same statistical contrast was then calculated as before: decoding of all F+ omissions (coded as the previous tone) was contrasted with decoding of all FÀ omissions (coded as the previous tone) using cluster-based permutation (0-300 ms, 1000 repetitions, two-tailed, alpha .05).

| Carrier frequency of omitted tones can be decoded in ordered tone sequences
In a first step, we wanted to replicate the key findings of Demarchi et al. (2019) showing enhanced decoding accuracy of carrier frequency during the omission period in ordered tone sequences compared with random tone sequences.Therefore, we contrasted the cross decoding of all F+ omission with FÀ omissions and yielded an positive and an negative cluster suggesting effects of frequency prediction (Figure 2b).The positive effect (p = .002)occurred for the whole testing period but with a focus around 40-to 80-ms and around 70-to 100-ms training time (positive peak at 48-ms testing time and 96-ms training time, maximum t value = 6.498).The negative effect (p = .02)occurred with little variation for the whole testing period between 30-and 50-ms training time (minimum t value = À4.585,40-ms training time, 68-ms testing time).F+ omissions yielded higher frequency-specific decoding than FÀ omissions (see also grand averages in Figure 2a).This is similar to Demarchi et al. (2019), although our effect is more restricted regarding training time (centres around 100 ms compared with 100-300 ms).

| Decoding of omitted tones is irrespective of expecting the omission
We then looked at the temporal predictions within those frequency predictions, that is, looking at contrasts between omission within ordered and random tone sequences (F+ vs. FÀ) separately for ordered (T+) versus random (TÀ) occurring omissions during the significant effect from above.No differences were found based on the regularity of the omissions using t test (positive effect: t(96) = À.759 p = .45,Bayes Factor = .275;negative effect: t(96) = 1.045 p = .299,Bayes Factor = .345,Figure 2b,d); that is, the temporal regularity of the omission occurrence does not influence sound frequency specific neural activity during the silent period.This result was stable also when using the statistical maximum to extract underlying individual values (positive effect: t(96) = À.2975 p = .767;negative effect: t(96) = 1.045 p = .299,Bayes Factor = .345).
As illustrated by the control analysis (Figure 2e), the carrier frequency of the previous tone was also decodable in the omission period ( p = .024).While partly overlapping with the omission effect related to the not presented carrier frequency, importantly, the overall effect as well as the maximum (t value = 4.006, training time 212 ms, testing time 52 ms) of the previous tone rather influenced later time periods regarding the training time (Figure 2c).

| Differences in omission ERFs between ordered and random tone sequences
Exploring the effects of tone regularity and omission prediction on the evoked response similar to studies exploring MMN (Garrido et al., 2009) and repetition suppression (Todorovic & de Lange, 2012), we contrasted the ERFs of all F+ omission with FÀ omissions.A difference occurred in right lateral sensors (p = .05)starting from 120 ms on with a peak at 228 ms (maximum t value = 4.858, Figure 3).Comparing the contrast between T+ and TÀ for the maximum effect between F+ and FÀ omission ERFs revealed no differences (t(96) = À.712, p = .478,Bayes Factor = .266).Also, when extracting the whole cluster, the difference did not reach significance (t(96) = À.736, p = .463,Bayes Factor = .271).

| DISCUSSION
In this study, we investigated how the temporal predictability of the occurrence of an omission influences the ability of the brain to form frequency-specific predictions of upcoming tones.For this purpose, we combined a paradigm of tone sequence regularity (Markov sequence, e.g.Chait et al., 2008) with varying omission regularity and multivariate pattern analysis (Treder, 2020).
Omitting single tones within otherwise ordered tone sequences compared with random tone sequences offers the possibility to investigate the expectations about upcoming tones that are reflected in brain activity specific to the frequency of those upcoming tones.Using four levels of tone orderedness in a previous study, Demarchi et al. (2019) showed that frequency-specific decoding of expected but omitted tones is dependent on the regularity of the tone sequence.The more ordered the sequence is, the stronger the frequency-specific decoding.In our study, we find a similar effect of increased decoding accuracy with a peak around 100 ms for omission periods of expected tones (F+) compared with not expected tones (FÀ).The statistical contrast shows that this increased decoding accuracy of F+ omissions lasts for the entire testing period of the omission (and around 100-ms training time) with a peak around 100 ms.Other studies have previously reported brain activity responding to a missing or deviant stimulus, for example, in silent oddball paradigms (Busse & Woldorff, 2003;Karamürsel & Bullock, 2000) or studies presenting only one stimulus of previously paired associations (Bendixen et al., 2009;Stekelenburg & Vroomen, 2015).In line with Demarchi et al. (2019), the present study shows that the brain response to the missing stimulus, here missing tone, shows clear predictions of frequency-specific information.Here, we find not only an increased frequency-specific decoding for the omissions in ordered tone sequences but also decreased decoding that lasts the entirety of the testing period.As can be seen in the illustration of the grand averages, this is due to below chance frequency decoding in the ordered tone sequences.Below chance decoding has previously been described by others (Carlson et al., 2011;King & Dehaene, 2014;Petruo et al., 2021) and suggested to reflect reversed brain patterns (King & Dehaene, 2014) with possible underlying processes of task set deactivation or changes of response strategies (Petruo et al., 2021).This effect was not reported in Demarchi et al. (2019) possibly due to higher statistical power (more participants) in the present study.
F I G U R E 3 Left: topographical display of the statistical maximal difference at 228 ms between all omission event related fields (ERFs) from ordered tone sequences (F+) and random tone sequences (FÀ).Significant sensors marked by asterixes.Right: averaged ERFs of the significant sensors.Black line indicates the significant time period.
Regarding the increased decoding accuracy found here as well as by Demarchi et al. (2019), the influence of the predictability of a missing tone itself however is not clear as all omission occurred irregularly in the study by Demarchi et al. (2019).
Therefore, in the present study, we manipulated the regularity of the occurrence of the omissions (temporal prediction) by having a random condition (TÀ) similar to Demarchi et al. (2019) and a regular condition (T+) where every tenth tone was omitted.Importantly, keeping the total number of omissions constant across conditions is different to, for example, Todorovic and de Lange (2012) and Todorovic et al. (2011) using a repetition suppression design where the expectancy of omissions was modulated by often or rarely occurring omissions.In their study, the omission of a repetition led to stronger responses when it was unexpected as compared with when it was more likely.Similarly, in Bekinschtein et al. (2009), using the local-global paradigm global and local violations of regularity was modulated by frequency of occurrence.
To explore the influence of the temporal predictions of omissions on the frequency prediction of upcoming tones, we looked at the individual data underlying the maximal statistical values of the frequency prediction effect (Figure 2b) and calculated again the contrast of omissions from F+ and FÀ tone sequences separately for T+and TÀ omissions.The temporal predictability of omissions has no influence on the frequency prediction as there is no change on the frequency-specific decoding due to the temporal prediction of the omission.This means, even if an omission is anticipated in an ordered tone sequence, the brain activity still captures the frequency-specific pattern of the omitted tone.This finding aligns with functional magnetic resonance imaging (fMRI) decoding results by Yon et al. (2023), showing that perceivers hold an 'undue reliance on old predictions' on visual outcomes without updating those based on unexpected and omitted visual outcomes.The idea of stubborn predictions explains the failing of the brain to update its predictions based on sensory evidence.As underlying mechanisms, computational limitations are discussed as well as estimation biases towards the predictive power of own actions.In the present study, another possible explanation for this undue reliance on established predictions might be that in an experimental paradigm, the toneto-tone predictions in the ordered condition occur on a shorter time scale and therefore more often than the omissions (every 10th tone in the ordered condition) thereby differing in the number of learning trials.
Regarding a possible influence of the previous tone, we conducted a control analysis where we coded the omission periods according to the previous tone and ran the decoding analysis again.Indeed, the previous tone was also decodable during the omission period.Importantly, the effect of the previous tone did not overlap with the maximum values of the omission effect.This suggests that the carrier frequency of the previous tone and the omitted carrier frequency are represented simultaneously during the omission period, however engaging somewhat different neural processes (as seen by the difference in the training time in Figure 2c).
While our analysis of ERFs cannot make statements about frequency specificity, it offers insights regarding evoked responses during an expected versus unexpected silent period within either regular or random tone sequences.Comparing ERFs of all ordered omissions and ERFs of all random omissions showed a difference between the two, with ERFs during random omissions evoking stronger negative deflections.Compared with the decoding differences between all ordered and all random omission, however, the effect started (onset around 100 ms) and peaked (228 ms) notably later.Omission ERFs yielded only a moderate component structure, which might be due presenting auditory only stimulation.Based on previous findings on omission responses, coupling the (omitted) stimuli to an action or another stimulus might elicit stronger omission responses (e.g.Dercksen et al., 2020;Dercksen et al., 2023;Stekelenburg & Vroomen, 2015).
The stubborn predictions during omissions, that is, increased responses within ordered tone sequences compared with random tone sequences independent of the predictability of the omissions, are visible with both decoding and ERFs.The earlier statistical significance and earlier statistical peaks in decoding compared with ERF suggest decoding frequency-specific neuronal responses might be a more sensitive approach.
Our study shows that frequency-specific information is predicted by the brain during silent episodes within otherwise regular sequences and that this specificity is not dependent on the regularity of those silent periods.It further suggests that investigating frequency specific prediction by decoding picks up on differences earlier than using an ERF approach.

F
I G U R E 1 Overview of the experimental design.(a) Illustration of the four conditions regarding tone and omission regularity; (b) transition matrices of tones sequences in the different conditions; (c) schematic examples of different sequences corresponding to the four conditions.The upper two sequences show regular tone sequences (F+) and the lower two random tone sequences (FÀ).The first and third tone sequence show sequences with regular omission (T+) and the second and last sequences random omission (TÀ).Omissions occurred in all conditions in 10% of the trials (absence of sound).

F
I G U R E 2 (a) Grand averages of decoding accuracy of all omission trials from ordered tone sequences (F+, upper) and random tone sequences (FÀ, lower panel).(b) Grand averages of decoding accuracy of all four conditions separately: omission trials from ordered tone sequences (F+, upper) and random tone sequences (FÀ, lower panel), omissions occurring ordered (T+, left) and randomly (TÀ, right).(c) Statistical contrast of decoding accuracy of all omission trials from ordered tone sequences (F+) and random tone sequences (FÀ).The black lines indicate the significant clusters.White area indicates the significant difference from the control analysis (see e).(d) Boxplots of the differences between omissions within F+ and FÀ trials separately for omission occurring ordered (T+) or randomly (TÀ).(e) Control analysis of recoding the omission periods according to the previous tone.Statistical contrast of decoding accuracy of all omission trials from ordered tone sequences (F+) and random tone sequences (FÀ).The black lines indicate the significant cluster.