Microdialysis perfusion of 5-HT into hypoglossal motor nucleus differentially modulates genioglossus activity across natural sleep-wake states in rats
Article first published online: 5 AUG 2004
The Journal of Physiology
Volume 532, Issue 2, pages 467–481, April 2001
How to Cite
Jelev, A., Sood, S., Liu, H., Nolan, P. and Horner, R. L. (2001), Microdialysis perfusion of 5-HT into hypoglossal motor nucleus differentially modulates genioglossus activity across natural sleep-wake states in rats. The Journal of Physiology, 532: 467–481. doi: 10.1111/j.1469-7793.2001.0467f.x
- Issue published online: 5 AUG 2004
- Article first published online: 5 AUG 2004
- (Received 24 August 2000; accepted after revision 18 December 2000)
- 1Serotonin (5-hydroxytryptamine, 5-HT) excites hypoglossal (XII) motoneurons in reduced preparations, and it has been suggested that withdrawal of 5-HT may underlie reduced genioglossus (GG) muscle activity in sleep. However, systemic administration of 5-HT agents in humans has limited effects on GG activity. Whether 5-HT applied directly to the XII motor nucleus increases GG activity in an intact preparation either awake or asleep has not been tested.
- 2The aim of this study was to develop a novel freely behaving animal model for in vivo microdialysis of the XII motor nucleus across sleep-wake states, and test the hypothesis that 5-HT application will increase GG activity.
- 3Eighteen rats were implanted with electroencephalogram and neck muscle electrodes to record sleep-wake states, and GG and diaphragm electrodes for respiratory muscle recording. Microdialysis probes were implanted into the XII motor nucleus and perfused with artificial cerebrospinal fluid (ACSF) or 10 mm 5-HT.
- 4Normal decreases in GG activity occurred from wakefulness to non-rapid eye movement (non-REM) and REM sleep with ACSF (P < 0.01). Compared to ACSF, 5-HT caused marked GG activation across all sleep-wake states (increases of 91-251 %, P < 0.015). Importantly, 5-HT increased sleeping GG activity to normal waking levels for as long as 5-HT was applied (3-5 h). Despite tonic stimulation by 5-HT, periods of phasic GG suppression and excitation occurred in REM sleep compared with non-REM.
- 5The results show that sleep-wake states differentially modulate GG responses to 5-HT at the XII motor nucleus. This animal model using in vivo microdialysis of the caudal medulla will enable the determination of neural mechanisms underlying pharyngeal motor control in natural sleep.
The genioglossus (GG) muscle of the tongue contributes to effective lung ventilation by maintaining an open pharyngeal airway. Decreased GG activity in sleep, especially REM sleep (Sauerland & Harper, 1976) can lead to airway narrowing, increased upper airway resistance and hypoventilation (Henke et al. 1992). In individuals with already anatomically narrow upper airways, such GG suppression can produce airway occlusion and obstructive sleep apnoea (Remmers et al. 1978), a serious sleep-related breathing disorder affecting approximately 4 % of adults (Young et al. 1993). However, despite increased knowledge of the major effects of sleep on GG activity, it is still not known which brainstem neural circuits and neurotransmitters modulate hypoglossal (XII) motor output to GG muscle in wakefulness and natural sleep.
In vitro studies using neonatal tissue slices have shown that 5-HT depolarizes and increases the excitability of XII motoneurons (Berger et al. 1992). 5-HT also facilitates XII motoneurons in decerebrate cats (Kubin et al. 1992; Douse & White, 1996). Medullary raphe neurons provide the 5-HT inputs to XII motor nucleus (Manaker & Tischler, 1993) and show decreasing discharge from wakefulness to non-REM and REM sleep (Jacobs & Azmitia, 1992). There is also decreased discharge of medullary raphe neurons projecting to XII motor nucleus in a pharmacological model of REM sleep evoked by carbachol microinjection into the pontine reticular formation of decerebrate cats (Woch et al. 1996). This pharmacological REM-like state in decerebrate cats is also associated with reduced 5-HT at the XII motor nucleus (Kubin et al. 1994).
Together, these observations provide appropriate circuitry for the notion that increased raphe 5-HT activity in wakefulness may excite XII motoneurons and increase GG activity, whereas withdrawal of 5-HT in sleep may decrease GG activity (Kubin et al. 1998). However, it has not yet been tested whether 5-HT at XII motor nucleus modulates GG activity in intact freely behaving animals, nor whether the potentially excitatory effects of 5-HT are dependent upon the prevailing sleep-wake states. This latter consideration may be especially important because in contrast to potential postsynaptic excitation of XII motoneurons (Berger et al. 1992; Kubin et al. 1992; Douse & White, 1996), 5-HT can also suppress XII motoneuron activity (Morin et al. 1992) by pre-synaptic inhibition of excitatory inputs (Singer et al. 1996). Such suppression of XII motoneuron activity by 5-HT may be involved in switching motor output appropriate for specific behaviours (Singer & Berger, 1996). These latter observations complicate the simple extrapolation of results from in vitro slice and in vivo decerebrate preparations that cannot exhibit spontaneous behaviours, and highlights the need for studies to be performed in an intact freely behaving preparation to determine the effects of 5-HT delivery to the XII motor nucleus across natural sleep-wake states.
Whether 5-HT applied directly to XII motor nucleus exerts a net excitatory (Berger et al. 1992; Kubin et al. 1992; Douse & White, 1996) or suppressant (Morin et al. 1992; Singer & Berger, 1996) effect on XII motor output in an intact preparation is also important because systemic administration of 5-HT agents in humans has limited success in increasing GG activity and preventing obstructive apnoeas (Schmidt, 1983; Hanzel et al. 1991; Slamowitz et al. 1998; Kraiczi et al. 1999; Sunderram et al. 2000). Although systemic administration of a combination of 5-HT agents decreased respiratory disturbance in a canine model of obstructive sleep apnoea, there was little effect on pharyngeal dilator muscle activity as measured in sternohyoid (Veasey et al. 1999). It is not known if the limited effects on GG or sternohyoid activities were due to problems associated with the peripheral administration of drugs failing to sufficiently increase 5-HT levels at XII motor nucleus, or whether there is lack of excitation by 5-HT in an intact behaving preparation.
Therefore, an aim of this study was to develop a freely behaving animal preparation for in vivo microdialysis of the caudal medulla to deliver neurotransmitters to the XII motor nucleus across natural sleep-wake states. Apart from acute single microinjections into the nucleus of the tractus solitarius in awake rats (Callera et al. 1997; Haibara et al. 1999; Czapla et al. 1999), to our knowledge this is the first description of chronic manipulation of neurotransmission in such a caudal region of medulla using microdialysis in an intact freely behaving preparation awake or asleep. Using this new preparation, this study tests the hypothesis that chronic delivery of 5-HT to the XII motor nucleus will selectively increase GG activity across wakefulness, non-REM and REM sleep.
Rats were housed individually, maintained on a 12-12 h light dark cycle (lights on 07.00 h) and had free access to food and water. The University of Toronto Animal Care Committee approved all procedures.
Studies were performed on 18 male rats (Charles River, mean body weight 344 g, range 241-439 g). Sterile surgery was performed under anaesthesia induced with intraperitoneal ketamine (85 mg kg−1) and xylazine (15 mg kg−1). Rats were also intraperitoneally injected with buprenorphine (0.03 mg kg−1), atropine sulphate (1 mg kg−1) and saline (3 ml, 0.9 %). The rats spontaneously breathed a 50:50 mixture of room air and oxygen through an anaesthesia mask (Freedman, 1992). Any additional anaesthesia was given by inhalation (halothane, typically 0.2-2 %). Effective anaesthesia was judged by abolition of the hind limb withdrawal and corneal blink reflexes. Body temperature was monitored with a rectal probe (BAS Inc., West Lafayette, IN, USA) and maintained between 36 and 38 oC.
With the rats supine the ventral surface of GG muscle was exposed via a submental incision and dissection of the geniohyoid and mylohyoid muscles. Insulated wire hook electrodes (38 gauge) were placed bilaterally, under direct vision, into GG using a 23 gauge needle and the technique of Basmajian & Stecko (1962). To record diaphragm (Dia) activity, two insulated, multi-stranded stainless steel wires (AS636: Cooner Wire, Chatsworth, CA, USA) were sutured onto the costal Dia via an abdominal approach. To ensure adequate electrode placements during surgery the GG and Dia signals were monitored on chart (Grass Model 79D polygraph, 7P511 amplifiers) and loudspeaker (AM8 Audio Amplifier, Grass). The GG and Dia wires were tunnelled subcutaneously to a neck incision and the submental and abdominal incisions closed with absorbable sutures.
The rats were placed in a stereotaxic apparatus (Kopf Model 962, Tujunga, CA, USA) with blunt ear bars. To ensure consistent positioning between animals, the flat skull position was achieved with an alignment tool (Kopf model 944). To record the electroencephalogram (EEG), two stainless steel screws (1.5 mm diameter) attached to insulated wire (30 gauge) were implanted in the skull (Horner et al. 1997). Two insulated, multi-stranded stainless steel wires were sutured onto the dorsal neck muscles to record the neck electromyogram (EMG).
The microdialysis guides (CMA/11, Acton, MA, USA) were placed through a small hole drilled at the junction of the interparietal and occipital bones. The guides were implanted 14.0 ± 0.5 mm (mean ±s.d.) posterior to bregma (range 12.9-14.9 mm), 0.3 ± 0.1 mm lateral to the midline (range 0.2-0.4 mm) and aimed 1-3 mm above XII using co-ordinates from a stereotaxic atlas (Paxinos & Watson, 1986). At the end of surgery the electrodes were connected to pins inserted into a miniature plug (STC-89PI-220ABS, Carleton University, Ottawa, Ontario, Canada). The plug and microdialysis guides were affixed to the skull with dental acrylic and anchor screws.
Upon completion of surgery, the rats were transferred to a clean cage and kept warm under a heating lamp until full recovery as judged by normal locomotor activity, grooming, drinking and eating. Rats were given soft food for the first day after surgery. All rats recovered fully after surgery and were housed individually. The rats recovered for 6.4 ± 2.0 days (mean ±s.d., range 3-13 days) before the experiments. To verify electrode placements, post-mortem X-rays were obtained in seven rats using a cabinet X-ray system (Faxitron Series, Model 43855A, Hewlett Packard, McMinnville, OR, USA). Images were obtained for 10 s at 47 kVp using standard mammography film. Figure 1 shows an example of the electrode and guide cannula placements.
For recordings, a lightweight shielded cable was connected to the plug on the rat's head. The cable was attached to a counterbalanced swivel that permitted free movement. All rats were studied in a noise-attenuated, electrically shielded cubicle (EPC-010, BRS/LVE Inc. Laurel, MD, USA). A one-way mirror allowed visual monitoring without disrupting the rat. For habituation the rats were connected to the cable and swivel apparatus the day before the experiments.
The electrical signals were amplified and filtered (Super-Z head-stage amplifiers and BMA-400 amplifiers/filters, CWE Inc., Ardmore, PA, USA). The EEG was filtered between 1 and 100 Hz, whereas the neck, GG and Dia EMGs were filtered between 100 and 1000 Hz. Calibration of the EEG and EMG signals was performed using the built-in microvolt calibrator (20 μV to 1 mV) on the head-stage amplifiers. The electrocardiogram was removed from the Dia EMG using an oscilloscope and an electronic blanker (Model SB-1, CWE Inc.). The moving-time averages (time constant = 200 ms) of the neck, GG and Dia EMGs were also obtained (Coulbourn S76-01, Lehigh Valley, PA, USA). Signals were recorded on chart (TA11, Gould, Valley View, OH, USA) and tape recorder (Model 4000A, A. R. Vetter Co. Inc., Rebersburg, PA, USA). The raw EEG and GG signals, along with the moving-time averages of the GG, Dia and neck EMGs were also digitized and recorded on computer (Spike 2 software, 1401 interface, CED Ltd, Cambridge, UK).
Protocol and microdialysis
Experiments were typically performed between 10.00 h and 19.00 h, i.e. during the time the rats normally sleep. Signals were monitored continuously in each of four conditions: (i) baseline before microdialysis, (ii) during microdialysis perfusion of artificial cerebrospinal fluid (ACSF) into XII, (iii) during microdialysis of ACSF containing drug, e.g. 5-HT, and (iv) during a second baseline condition after removal of the microdialysis probe and recovery from the effects of 5-HT.
In the baseline condition before insertion of the microdialysis probe, recordings were made across multiple cycles of wakefulness, non-REM and REM sleep. The dummy cannula was then removed from the guide and the microdialysis probe was inserted (CMA/11 14/01). The probes projected 1-3 mm from the tip of the guide, and were 240 μm in diameter with a 1 mm cuprophane membrane and a 6000 Da cut-off. The ease of insertion of the probe ensured that the rat was minimally disturbed by the procedure. The probes were connected to FEP Teflon tubing (inside diameter 0.12 mm) in turn connected to 1.0 ml syringes via a zero dead space switch (Uniswitch, B.A.S. West Lafayette, IN, USA). The probes were continually flushed with ACSF at a flow rate of 2.1 μl min−1 using a syringe pump and controller (MD-1001 and MD-1020, BAS Inc). With the length of tubing used for the chronic experiments the lag time for fluid to travel to the tip of the probe was 8 min 5 s at this flow rate. The composition (mm:) of the ACSF was NaCl, 125; KCl, 3; KH2PO4, 1; CaCl2, 2; MgSO4, 1; NaHCO3, 25; and glucose, 30. The ACSF was warmed to 37 oC and bubbled with CO2 to a pH of 7.39 ± 0.03 (mean ±s.d.). The CaCl2 was added after warming the ACSF to 37 oC (Nattie & Li, 2000).
After insertion of the microdialysis probe, ACSF was continuously perfused for at least 2 h across sleep-wake states. The perfusion medium was then switched to 10 mm 5-HT (creatinine sulfate complex, FW: 387.4, Sigma) dissolved in ACSF, and perfusion was also maintained for at least 2 h across sleep-wake states. Perfusion of 50 mm 5-HT was also performed in eight rats. At the end of the experiments the microdialysis probes were removed and the dummy cannula replaced. In 15 of 18 rats the electrical signals were also recorded the following morning to determine if recovery from 5-HT effects (see Results) was complete and responses had returned to baseline.
The moving-time averages of the GG, neck and Dia EMGs were analysed in wakefulness, non-REM and REM sleep determined by standard EEG and EMG criteria (Horner et al. 1997). Data were selected for analysis at least 60 min after insertion of the microdialysis probe or switching between drugs. Although a burst of GG activity was observed when the probe was initially inserted and penetrated XII motor nucleus (Fig. 2), this burst was transient and typically lasted < 5 min. In addition, any GG responses to the drugs had already reached the steady state well before this 60 min period.
The physiological record was split into contiguous 5 min bins, and the first periods of uninterrupted wakefulness (> 1 min), non-REM (> 1 min) and REM sleep (> 45 s) in each bin were selected and the final 30 s analysed. As such, multiple sleep-wake episodes in each experimental condition were analysed while avoiding selection bias. To further minimize bias in selecting periods, the EEG and neck EMG were scored for sleep-wake states without reference to the GG or Dia. Mean GG and neck EMGs, Dia amplitude and respiratory rates were calculated in 5 s epochs from the moving-time averages (above electrical zero) for each 30 s period and were quantified in arbitrary units. Electrical zero was the voltage recorded with the amplifier inputs earthed. Each rat served as its own control with all interventions being performed in one experiment therefore allowing for consistent effects of experimental condition (e.g. 5-HT) to be observed across sleep-wake states within and between rats.
At the end of the study the rats were given an overdose of sodium pentobarbital by intraperitoneal injection (100 mg (100 g)−1) and then perfused intracardially with 0.9 % saline and 10 % formalin. To assist in locating sites of microdialysis, the probes were perfused with Methylene Blue dye (2 % solution, FW: 319.9, Sigma) for 15 min with the rat in the sphinx position. Brains were then removed and fixed in 10 % formalin. The medullary regions were blocked, transferred to 30 % sucrose and cut in 50 μm coronal sections with a cryostat (Leica, CM1850). Each section from the regions spanning the blue dye and lesion site were mounted and then stained with Neutral Red. Although the Methylene Blue dye was useful in locating microdialysis sites upon initial gross examination of unstained sections, this dye was not clearly visible after Neutral Red staining. Accordingly, after staining, the lesion sites made by the probes were used to locate the sites of microdialysis, and these sites were marked on standard brain maps (Paxinos & Watson, 1986).
The analyses performed for each statistical test are included in the text where appropriate. For all comparisons, differences were considered significant if the null hypothesis was rejected at P < 0.05 using two-tailed t test. Where post hoc comparisons were performed after analysis of variance with repeated measures (ANOVA), the Bonferroni corrected P value was used to infer statistical significance. For two-way ANOVA, the factors were sleep-wake states (i.e. wakefulness, non-REM and REM sleep) and the drug used for microdialysis (e.g. ACSF vs. 5-HT). The choice of parametric or non-parametric analysis for paired or unpaired samples depended on whether the data were normally distributed. Analyses were performed using Sigmastat (Jandel Scientific, San Rafael, CA, USA). Data are presented as means ±s.e.m. unless otherwise stated.
Stereotypical motor acts were associated with phasic GG activation in these freely behaving rats (Fig. 3). In contrast, quiet wakefulness was associated with tonic GG activation, and phasic respiratory-related GG activity was rarely observed (Fig. 3 and below). Respiratory-related GG activity was, however, almost always recorded at the time of surgery in each rat (perhaps due to higher arterial PCO2) and post-mortem examinations confirmed that the electrodes were still in GG muscle after the experiments (Fig. 1).
Anatomical location of microdialysis probes
Figure 4 shows an example of the lesion sites made by the microdialysis probe in one rat. Figure 5 shows the distribution of microdialysis sites from all experiments. In the 18 rats studied, microdialysis probes were successfully implanted into XII in 10 rats as judged by histology. In each of these 10 rats a burst of GG activity was also observed when the probe was initially inserted and penetrated XII, and this proved useful as a preliminary confirmation of probe placement (Fig. 2). This burst of GG activity during probe insertion was transient (typically lasting < 5 min) and did not occur in the neck or Dia signals (Fig. 2). The brain of one rat was damaged upon processing, but since a similar burst of GG activity was also observed in that rat when the probe was initially inserted on the day of the experiment, the results from this rat were included with the other ten. As such, a total of 11 rats were judged to have probes implanted into XII. The probes in the other seven rats were placed at various other sites in the caudal medulla but outside XII (Fig. 5).
GG responses to 5-HT
Figure 6 shows an example of the GG responses to microdialysis perfusion of 5-HT into the XII motor nucleus awake and asleep. During control ACSF infusion, note the normal changes in EEG, neck and Dia activities from wakefulness to non-REM and REM sleep (Horner et al. 1997). Note also the higher GG activity in wakefulness and the phasic GG bursts in REM sleep. Following 5-HT delivery to XII in this freely behaving rat, there was clear GG activation compared with ACSF across all sleep-wake states. Note, however, that GG activity in REM sleep with 5-HT showed periods of both clear suppression, and phasic excitation, compared with non-REM sleep. It was also commonly observed that phasic GG twitches in REM sleep were greater with 5-HT than with ACSF alone (Fig. 6).
The onset of GG activation following the switch from ACSF to 5-HT is shown in Fig. 7. The time lag for eliciting a GG response following the switch to 5-HT in this example was 8 min 30 s. This lag is consistent with the expected time for 5-HT in the perfusate to arrive at the probe tip and diffuse into XII (see Methods). For the rats with probes placed in XII the group mean time lag for eliciting a GG response was 13 min 22 s ± 1 min 12 s.
It is also important to note that once activated by 5-HT, GG activity remained tonically elevated for as long as the 5-HT was applied (range 1 h 40 min to 5 h 36 min in the different experiments, mean = 3 h 19 min ± 26 min). An example of this tonic and persistent GG activation by 5-HT is shown in Fig. 7 for a rat in non-REM sleep at different times during the experiment.
A total of 792 sleep-wake periods were analysed in the eleven rats with microdialysis probes implanted into the XII motor nucleus. Of these, 38.1 % occurred in wakefulness, 33.5 % in non-REM sleep and 28.4 % in REM sleep. In these 11 rats, 226 sleep-wake periods were analysed for the baseline condition, 187 periods for ACSF, 227 periods for 5-HT and 152 periods for recovery baseline. For the seven rats with probes outside XII, a total of 447 sleep-wake periods were analysed across sleep-wake states and across the four experimental conditions. Figure 8 shows the grouped mean data in the 11 rats with probes implanted into XII.
Effects of 5-HT on GG activity.
Figure 8A shows that 5-HT at the XII motor nucleus led to significant increases in GG activity across all sleep-wake states compared with the three control conditions (i.e. baseline, ACSF and recovery baseline). There was a statistically significant effect of 5-HT on GG activity compared with the three controls (P < 0.015, two-way ANOVA). This stimulating effect of 5-HT occurred independently of sleep-wake states (P > 0.345, two-way ANOVA), i.e. it was consistent across wakefulness, non-REM and REM sleep. There was no difference in GG activity between the three control conditions (P > 0.620, two-way ANOVA).
GG activity across sleep-wake states.
Figure 8A also shows that GG activity varied as a function of sleep- wake state within each of the four experimental conditions (all P≤ 0.004, one-way ANOVA). For the three controls, GG activity was significantly higher in wakefulness compared with non-REM sleep (all P < 0.01). However, GG activity did not change further from non-REM to REM (P > 0.512) as it was already minimally active in non-REM (Figs 6 and 8A). With 5-HT, GG activity also significantly decreased from wakefulness to non-REM sleep (P < 0.01, Fig. 8A), and overall activity did not change further from non-REM to REM P = 0.738. However, as previously shown in Fig. 6, GG activity during 5-HT was highly variable in REM sleep compared with non-REM, and was associated with periods of both clear GG suppression and phasic excitation. The high variability of GG activity in REM sleep was confirmed by comparisons of the mean coefficients of variation of GG activity (mean = 11.7 ± 1.9 % in non-REM vs. 36.4 ± 6.2 % in REM, P= 0.003, paired t test). As such, although overall mean GG activity was similar between REM and non-REM sleep with 5-HT, such an analysis obscured the changes with time in GG activity that typified REM. Accordingly, GG activity in REM sleep was subjected to a more detailed analysis (see below).
Neck muscle activity.
There was no effect of 5-HT on neck activity compared with the three control conditions (P > 0.261, two-way ANOVA). For each experimental condition, neck EMG varied consistently across sleep- wake states in (P < 0.001, one-way ANOVA, Fig. 8B), with higher activity in wakefulness compared with non-REM and REM sleep (P < 0.01, paired t tests). Neck EMG activity also decreased slightly but consistently from non-REM to REM sleep across all four experimental conditions (Fig. 8B, all P < 0.028, paired t tests).
Figure 8C shows the effects of 5-HT on mean Dia amplitude. There was no difference in Dia amplitudes with 5-HT compared with the three control conditions (all P > 0.156, two-way ANOVA). Figure 8D shows the effects of 5-HT on respiratory rate. Respiratory rates typically decreased from wakefulness to non-REM sleep and then increased in REM. Although the direction of this change was consistent across all four experimental conditions, the increased respiratory rate in REM sleep was small with 5-HT at the XII motor nucleus. There was a stronger effect of sleep-wake state on respiratory rate in the baseline, ACSF and recovery baseline conditions (P= 0.011, 0.003, 0.075, respectively, one-way ANOVA) compared with 5-HT P = 0.389. Moreover, for respiratory rate there was a statistically significant interaction between experimental condition and sleep-wake state when the three control conditions were compared with 5-HT (P < 0.05, two-way ANOVA). This result confirmed that the effect of sleep-wake states on respiratory rate depended on the presence of 5-HT, with respiratory rates in REM being reduced with 5-HT (Fig. 8D).
Modulation of GG activity from non-REM to REM sleep
REM sleep was associated with periods of both clear phasic GG suppression and excitation compared with non-REM sleep. Figure 9 (top) shows an example of GG suppression in REM sleep coincident with the onset of neck muscle atonia and EEG desynchrony followed by periods of transient GG excitation also in REM sleep. Figure 9 also shows how these GG data were further analysed to provide a more detailed description of the GG variability in REM sleep than that provided by the coefficient of variation previously described (see GG activity across sleep-wake states). For this analysis the frequency histograms of GG EMG values (from the moving average signal) are shown in this rat for the time spent in both non-REM and REM sleep with 5-HT. This histogram was calculated from 60 5 s non-REM epochs and 12 REM epochs. This figure shows that in this rat GG values were normally distributed in non-REM sleep with 0 % of the values below 20 EMG units and only 2 % of values above 40 units. In contrast, the data were not normally distributed in REM sleep, with 29 % of values occurring below 20 units (i.e. periods of GG suppression compared with non-REM sleep with 5-HT) and 25 % of values above 40 units (i.e. periods of phasic GG excitation in REM sleep with 5-HT).
Such periods of GG suppression and excitation in REM sleep, compared with non-REM, occurred in 32 of 34 REM episodes analysed with 5-HT. To further quantify this effect of REM sleep on GG activity with 5-HT, the frequency histograms of GG values for the time spent in both non-REM and REM sleep were calculated from the moving average signal for each rat (i.e. in the same fashion as in Fig. 9). An average of 42 ± 5.9 5 s non-REM epochs and 21.8 ± 5.4 REM epochs were analysed in each rat for this histogram. To facilitate comparisons between rats, the GG values were normalised for each rat to the mean GG in non-REM sleep. The group frequency histograms of GG values (Fig. 10A and B) show that, with 5-HT at the XII motor nucleus, REM sleep was associated with more EMG values at low levels in REM sleep compared with non-REM sleep, i.e. periods of relative GG suppression in REM sleep. In addition, Fig. 10 shows that there were more EMG values at high levels compared with non-REM sleep, i.e. periods of phasic GG excitation in REM sleep. These effects of REM sleep on GG activity can also be observed in the cumulative frequency histogram shown in Fig. 10C. Analysis confirmed that with 5-HT at the XII motor nucleus the cumulative distribution of GG EMG values across the range of values encountered in each rat was significantly different between non-REM and REM sleep (P < 0.001, two-way ANOVA, Fig. 10C). Further analyses showed that significantly more GG activity occurred in REM sleep at values below 40 % and above 170 % of those encountered in non-REM sleep (P < 0.05, paired t tests), showing that major periods of REM sleep with 5-HT were associated with GG suppression and excitation relative to non-REM. Nevertheless as can be seen in the cumulative histogram (Fig. 10C) that GG activity in REM sleep 50 % of the time was similar to non-REM sleep. This result is consistent with the overall result shown in Fig. 8, i.e. that average GG values between these two sleep states were not statistically different.
GG responses to 5-HT in rats with probes inside and outside the XII motor nucleus
GG responses to 10 mm 5-HT were significantly larger in the 11 rats with probes implanted into the XII motor nucleus compared with the 7 rats with probes outside XII. For probes inside versus outside XII, GG responses to 5-HT were increased in wakefulness (61.6 ± 9.4 vs. 38.2 ± 4.6 arbitrary (Arb.) units; mean percentage increase, 61 %; P= 0.029, unpaired t test), non-REM sleep (33.0 ± 6.9 vs. 12.0 ± 3.5 Arb. units; increase = 175 %, P= 0.047) and REM sleep (28.1 ± 5.0 vs. 11.0 ± 2.6 Arb. units; increase, 155 %; P= 0.021). With control perfusion of ACSF there were no significant differences in GG activity across sleep- wake states for probes inside vs. outside XII (wakefulness: 32.3 ± 6.3 vs. 24.1 ± 3.1 Arb. units; P= 0.344; non-REM sleep: 9.4 ± 2.0 vs. 8.5 ± 2.5 Arb. units; P= 0.802; REM sleep: 12.2 ± 2.3 vs. 8.7 ± 2.4 Arb. units; P= 0.321).
Comparisons of responses to 10 mm and 50 mm 5-HT
Responses to 50 mm 5-HT were investigated when small or minimal GG responses to 10 mm 5-HT were observed on the day of the study. Responses to 50 mm 5-HT were tested in 3 of 11 rats with probes in XII and 5 of 7 rats with probes outside XII. GG responses to 50 mm 5-HT were increased in sleep (i.e. non-REM and REM sleep) compared with responses to 10 mm 5-HT (42.9 ± 9.5 vs. 19.7 ± 4.3 Arb. units; mean increase, 118 %, P= 0.022, paired t test). Responses to 50 and 10 mm 5-HT were not statistically different in awake rats (93.5 ± 60.8 vs. 35.6 ± 4.9 Arb. units; P= 0.844).
This study describes a novel freely behaving animal model for in vivo microdialysis of the caudal medulla to deliver neurotransmitters to the XII motor nucleus across natural sleep-wake states. Microdialysis perfusion of 5-HT into XII caused tonic GG activation, and this increased GG activity persisted for as long as the 5-HT was applied, which was over several hours in these experiments. These results support the suggestion that 5-HT delivery to the XII motor nucleus provides a ‘wakefulness stimulus’ for XII motoneurons. Indeed, 5-HT increased sleeping GG activity to normal waking levels, a result that may have important clinical relevance (see below).
The data also showed that the prevailing sleep-wake states exerted significant modulating effects on the GG responses to 5-HT at the XII motor nucleus. This observation highlights the importance of studies in intact freely behaving animals in extrapolating results from reduced preparations to natural sleep-wake states. In particular, periods of GG suppression and excitation still occurred in REM sleep compared with non-REM despite the continuing tonic background activation provided by the exogenous 5-HT.
Determining the neurotransmitters and receptors modulating XII motor nucleus across sleep-wake states will be important in identifying the mechanisms underlying sleep-wake state-dependent GG activity. This knowledge is significant to the understanding of the control of those pharyngeal muscles involved in the maintenance of upper airway patency. Such studies will also be important in guiding therapeutic strategies for obstructive sleep apnoea in animal models of this disorder (Veasey et al. 1999; Tuck et al. 1999) and in humans to determine if GG can be selectively activated in sleep.
Relevance of GG responses to 5-HT at the XII motor nucleus
The current results show that tonic GG activation occurred when 5-HT was applied directly to the XII motor nucleus. As such, the limited effect on pharyngeal dilator muscle activity observed in previous studies using systemic administration of 5-HT drugs in humans (Schmidt, 1983; Hanzel et al. 1991; Slamowitz et al. 1998; Kraiczi et al. 1999; Sunderram et al. 2000) is probably due to problems associated with the peripheral administration of drugs rather than a potential lack of excitation by 5-HT at the XII motor nucleus. Importantly, the present study also showed that 5-HT delivery to the XII motor nucleus caused tonic increases in GG activity for as long as the 5-HT was applied. Previous studies in reduced preparations have only examined the transient effects of 5-HT on XII motor output following acute single microinjections (Kubin et al. 1992; Douse & White, 1996). Overall, these data suggest that attempts to increase GG activity over time (e.g. overnight) with 5-HT agents in the clinical population of obstructive apnoea patients are worthwhile. In the present study selective GG activation was achieved by delivery of 5-HT to the XII motor nucleus directly via anatomical approaches and microdialysis. However, the ability to target the relevant neural systems selectively in the clinical situation to increase pharyngeal dilator muscle activity without affecting other major systems (e.g. sleep and mood) or respiratory pump muscle activity (Richmonds & Hudgel, 1996) will be a major challenge.
Effects of 5-HT on XII motor output across sleep-wake states
In non-REM sleep the tonic increase in GG activity with 5-HT may be best explained by depolarization of XII motoneurons (Berger et al. 1992). In REM sleep, however, periods of both phasic GG suppression and excitation occurred despite this tonic GG stimulation by 5-HT. As such, REM sleep differentially modulated the GG responses to 5-HT at the XII motor nucleus. The phasic GG excitation in REM sleep with 5-HT (Fig. 6) may be explained by 5-HT-mediated increases in XII motoneuron excitability (Berger et al. 1992) in response to the transient membrane depolarizations that accompany REM sleep, as observed in other motoneuron pools (Glenn et al. 1978). The mechanisms mediating the phasic suppressions in GG activity in REM sleep are not so clear, however, and inhibitory or disfacilitatory mechanisms may each play a role to a greater or lesser degree. In this respect postsynaptic inhibitory mechanisms play a major role in hypotonia of lumbar and trigeminal motoneurons both in natural REM sleep and in the REM-like state produced by pontine carbachol (Morales et al. 1987; Pedroarena et al. 1994). However, whether such inhibitory mechanisms make an important contribution to the suppression of XII motor output in REM sleep is questionable according to studies in decerebrate or anaesthetized animals after carbachol (Kubin et al. 1993, 1996; Yamuy et al. 1999). However, pontine carbachol does not reproduce the whole range of electrocortical and respiratory changes elicited in natural REM sleep, particularly phasic events (Horner & Kubin, 1999; Horner, 2000), which may be involved in transient inhibitions of XII motor output (Yamuy et al. 1999). It remains to be determined in separate experiments if the periods of GG suppression observed in REM sleep in this study are due to the recruitment of inhibitory mechanisms acting to counteract the excitation produced by the exogenous 5-HT.
It is also possible that the observed suppression of GG activity in REM sleep, compared with non-REM, is due to further withdrawal of endogenous excitatory neurotransmitters such as 5-HT, with co-released thyrotropin-releasing hormone and substance P (Jacobs & Azmitia, 1992), and noradrenaline (Aston-Jones & Bloom, 1981). Endogenous release of these neurotransmitters would probably be at a minimum just prior to, and during, REM sleep. This animal model will allow the determination of whether disfacilitation contributes to the GG suppression in REM compared to non-REM sleep.
Nevertheless, the possibility that neuronal mechanisms associated with REM sleep may modulate the GG activation produced by excitatory inputs to the XII motor nucleus, as observed in this study, is of clinical relevance. For example, periods of reduced GG activity in REM sleep despite tonic GG activation (in this case produced by exogenously applied 5-HT) may explain the observation that treatment with 5-HT agents was unable to prevent suppression of pharyngeal dilator muscle activity from non-REM to REM sleep in a canine model of obstructive apnoea (Veasey et al. 1999).
Although the previous discussion has focused on neural mechanisms associated with sleep processes as being responsible for the periods of phasic GG suppressions and excitations in REM sleep, non-sleep-related mechanisms may also play a role. For example, increased respiratory rates in REM sleep may lead to reductions in arterial PCO2 and contribute to decreased GG activity. Although the contribution of this mechanism cannot be discounted in this study, this observation highlights the fact that studies in freely behaving animals, while offering distinct advantages over reduced preparations in the study of natural sleep mechanisms, also have some disadvantages in terms of experimental control of related variables during behaviours. In contrast, while studies in reduced preparations do not allow the study of natural sleep mechanisms, reflex modulations of GG activity via changes in blood gases and breathing pattern can be controlled (Kubin et al. 1998).
GG activity was also increased in wakefulness after 5-HT (Fig. 8A). This increased GG activity is perhaps due to increased excitability of XII motoneurons by the exogenously applied 5-HT (Berger et al. 1992) along with increased amounts of other endogenous excitatory neurotransmitters in wakefulness such as 5-HT, thyrotropin-releasing hormone, substance P and noradrenaline (Aston-Jones & Bloom, 1981; Jacobs & Azmitia, 1992; Parkis et al. 1995).
The receptors responsible for the effects of 5-HT at the XII nucleus were not identified in this study. The mRNAs for several 5-HT receptor types are present at the XII nucleus (Okabe et al. 1997), and of these, types 2A and 2C most probably mediate the excitatory effects of 5-HT on XII motoneurons (Kubin et al. 1992). In contrast, preliminary data suggest that the type 3 receptor has little excitatory effect on XII motor output (Veasey et al. 2000). The animal model described in this study will enable experiments to determine the receptor types involved in modulating XII motor output in freely behaving animals for 5-HT and other neurotransmitters across sleep-wake states.
Specificity of responses
It was a concern that neural structures close to XII may be influenced by diffusion of 5-HT from the microdialysis probe and cause GG activation independently of an effect at XII. Although direct measurement of the spread of 5-HT from the microdialysis probe, or spread of a fluorescent marker of similar molecular weight (Nattie & Li, 2000), was not performed in this study, other precautions were taken to determine if 5-HT responses were likely to be localized to effects at XII. As such, measurements in the freely behaving rats included neck muscle activity (as a control postural muscle) and Dia activity (as a control respiratory muscle).
The results showed that a general increase in postural motor output would not explain the increased GG activity after 5-HT in the freely behaving rats as no changes in neck muscle activity were observed compared with the three control conditions (Fig. 8B). Activation of medullary raphe neurons would also not explain the increased GG activity in the present study because increased 5-HT at this site would probably have inhibited raphe activity via the 5-HT1A autoreceptor (Jacobs & Azmitia, 1992; Woch et al. 1996). That there was no change in Dia amplitude between the 5-HT and control conditions in these rats also argues against possible 5-HT effects at the medullary respiratory neurons (McCrimmon et al. 1995) in affecting XII activity. Preliminary studies in anaesthetized rats also showed that that there were no changes in breathing pattern or blood pressure with 5-HT application to the XII motor nucleus (Horner et al. 2000). The absence of a change in blood pressure in those studies under anaesthesia suggests that the 5-HT responses at the XII motor nucleus were probably independent of cardiovascular effects at the nucleus of the tractus solitarius (Callera et al. 1997).
The previous discussion provides strong evidence supporting XII as the effective site of action for 5-HT in this study. In this context, the reduced respiratory rate with 5-HT in REM sleep was surprising, especially because respiratory rate was not different between 5-HT and the three control conditions for both non-REM sleep and wakefulness (Fig. 8D). It is possible that a reduced requirement for alveolar ventilation in REM sleep after 5-HT delivery to the XII nucleus may lead to the reduced respiratory rate, presumably because of reduced upper airway resistance caused by GG activation leading to increased tidal volume for the same Dia activity. It remains to be tested if the GG activation caused by 5-HT at XII does produce beneficial effects for the maintenance of airway patency, e.g. dilatation and/or stiffening of the pharyngeal airway (Horner, 1996). It is also possible, however, that diffusion of 5-HT from the XII motor nucleus to surrounding structures such as the nucleus of the tractus solitarius could potentially affect respiratory rhythm via modulation of afferent inputs. Why such an effect could have affected respiratory rate in a state-dependent manner, and only in REM sleep, is unclear at present. It is possible that the absence of a change in respiratory rate after 5-HT was applied to the XII motor nucleus in preliminary experiments in anaesthetized rats (Horner et al. 2000) may have occurred because these rats were vagotomized.
Other methodological considerations
It has been reported that rats studied under conditions of lowered ambient temperatures exhibit periods of phasic respiratory GG activity when awake and asleep (Megirian et al. 1985). However, respiratory-related GG activity was not observed in this group of rats studied at normal ambient temperatures. Phasic GG activation was only observed during active behaviours such as eating and licking, i.e. behaviours in which GG would be expected to be active. However, respiratory-modulated GG activity was typically recorded in these same rats at the time of surgery, perhaps due to higher arterial PCO2 because of anaesthesia. These data, and the observation that post-mortem examinations confirmed that the electrodes were still in GG muscle after the studies, make us confident that the electrodes were recording GG activity during the experiments.
The tissue surrounding the lesion sites in these rats showed evidence of gliosis (Fig. 4). We suspect that the gliosis occurred after the first insertion of the microdialysis probe, and developed over the course of several days before the rats were killed. The rats were not killed immediately after the experiment because we wanted to determine if the GG responses to 5-HT had returned to baseline the day after the experiments. Nevertheless, the observation of gliosis is important because repeated insertions of microdialysis probes or microinjection cannulae is a common design in many experiments in the literature. As such the potential for more tissue damage and gliosis in those experiments is a concern as this would act as a barrier to drug diffusion thereby limiting the validity of comparisons between interventions if they were not performed on the same day. All the present experiments were performed on one day to avoid this potential problem.
That GG activity across sleep-wake states was similar for each of the three control conditions with and without the microdialysis probe (i.e. ACSF vs. both baseline conditions, Fig. 8A) is also an important observation. This result shows that insertion of the microdialysis probe caused no adverse influences on the XII nucleus since GG activity was similar with and without the probe in place.
The dose that was chosen in the present study to deliver 5-HT to the XII motor nucleus was larger than that employed in previous studies using tissue slices (10-100 μm, Berger et al. 1992). Nevertheless, the dose chosen in our experiments is comparable to previous microinjection experiments (5 mm, Kubin et al. 1992, 1996). Although it is difficult to determine the exact 5-HT concentration around XII in the present study, the delivered concentration is likely to be considerably less than the 10 mm applied in the perfusion medium. Measurement of microdialysis delivery of glutamate at 2.0 μl min−1 for longer than 30 min showed only 11-25 % delivery of the applied dose (Alessandri et al. 1996). However, the 5-HT concentrations used in the present study were probably reasonable since GG was activated to physiological levels, e.g. sleeping GG activity with 5-HT approximated the levels measured in normal waking without 5-HT (Figs 6 and 8A).
- 1996). Application of glutamate in the cortex of rats: a microdialysis study. Acta Neurochirurgica-Supplementum 67, 6–12. , , , , & (
- 1981). Activity of norepinephrine-containing locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep-waking cycle. Journal of Neuroscience 1, 876–886. & (
- 1962). A new bipolar electrode for electromyography. Journal of Applied Physiology 17, 849. & (
- 1992). Modulation of neonatal rat hypoglossal motoneuron excitability by serotonin. Neuroscience Letters 143, 164–168. , & (
- 1997). Cardiovascular effects of microinjection of low doses of serotonin into the NTS of unanesthetized rats. American Journal of Physiology 272, R1135–1142. , , , & (
- 1999). Tyrosine kinase inhibitors modulate the ventilatory response to hypoxia in the conscious rat. Journal of Applied Physiology 87, 363–369. , , & (
- 1996). Serotonergic effects on hypoglossal neural activity and reflex responses. Brain Research 726, 213–222. & (
- 1992). A sliding gas-delivery system and ventilating chamber for small animals fixed in standard stereotaxic instruments. Behavior Research Methods, Instruments and Computers 24, 423–425. (
- 1978). Membrane potential of spinal motoneurons during natural sleep in cats. Sleep 1, 199–204. , & (
- 1999). Sympathoexcitatory neurotransmission of the chemoreflex in the NTS of awake rats. American Journal of Physiology 276, R69–80. , & (
- 1991). Response of obstructive sleep apnea to fluoxetine and protriptyline. Chest 100, 416–421. , & (
- 1992). Load compensation and respiratory muscle function during sleep. Journal of Applied Physiology 72, 1221–1234. , , & (
- 1996). Motor control of the pharyngeal musculature and implications for the pathogenesis of obstructive sleep apnea. Sleep 19, 827–853. (
- 2000). Impact of brainstem sleep mechanisms on pharyngeal motor control. Respiration Physiology 119, 113–121. (
- 1999). Pontine carbachol elicits multiple rapid-eye-movement sleep-like neural events in urethane anesthetized rats. Neuroscience 93, 215–226. & (
- 1997). Serotonin at the laterodorsal tegmental nucleus suppresses rapid-eye-movement sleep in freely behaving rats. Journal of Neuroscience 17, 7541–7552. , , , & (
- 2000). Microdialysis perfusion of serotonin into the hypoglossal motor nucleus increases genioglossus muscle activity across sleep-wake states in freely behaving rats. Sleep 23 (S2), A2. , , , & (
- 1992). Structure and function of the brain serotonin system. Physiological Reviews 72, 165–229. & (
- 1999). Effect of serotonin uptake inhibition on breathing during sleep and daytime symptoms in obstructive sleep apnea. Sleep 22, 61–67. , , , & (
- 1998). Control of upper airway motoneurons during REM sleep. News in Physiological Sciences 13, 91–97. , & (
- 1993). Suppression of hypoglossal motoneurons during the carbachol-induced atonia of REM sleep is not caused by fast synaptic inhibition. Brain Research 611, 300–312. , , , & (
- 1994). Changes in serotonin level in the hypoglossal nucleus region during carbachol-induced atonia. Brain Research 645, 291–302. , , , , & (
- 1992). Serotonergic excitatory drive to hypoglossal motoneurons in the decerebrate cat. Neuroscience Letters 139, 243–248. , , & (
- 1996). Interaction of serotonergic excitatory drive to hypoglossal motoneurons with carbachol-induced REM sleep-like atonia. Sleep 19, 187–195. , , , & (
- 1995). Glutamate, GABA, and serotonin in ventilatory control. In Regulation of Breathing, ed. Dempsey, J. A. & Pack, A. I., pp. 151–218. Dekker, New York , USA , & (
- 1993). Origin of serotonergic afferents to the hypoglossal nucleus in the rat. Journal of Comparative Neurology 334, 466–476. & (
- 1985). Respiratory roles of genioglossus, sternothyroid, and sternohyoid muscles during sleep). Experimental Neurology 90. 118–128. , & (
- 1987). Motoneuron properties during motor inhibition produced by microinjection of carbachol into the pontine reticular formation of the decerebrate cat. Journal of Neurophysiology 57, 1118–1129. , , , & (
- 1992). Compared effects of serotonin on cervical and hypoglossal activities: an in-vitro study in the newborn rat. Journal of Physiology 451, 605–629. , & (
- 2000). Muscimol dialysis in the retrotrapezoid nucleus region inhibits breathing in the awake rat. Journal of Applied Physiology 89, 153–162. & (
- 1997). Serotonin receptor mRNA expression in the hypoglossal motor nucleus. Respiration Physiology 110, 151–160. , & (
- 1995). Actions of norepinephrine on rat hypoglossal motoneurons. Journal of Neurophysiology 74, 1911–1919. , & (
- 1986). The Rat Brain in Stereotaxic Co-ordinates. San Diego, Academic Press & (
- 1994). The control of jaw-opener motoneurons during active sleep. Brain Research 653, 31–38. , , & (
- de, & (1978). Pathogenesis of upper airway occlusion during sleep. Journal of Applied Physiology 44, 931–938. ,
- 1996). Hypoglossal and phrenic motoneuron responses to serotonergic active agents in rats. Respiration Physiology 106, 153–160. & (
- 1976). The human tongue during sleep: electromyographic activity of the genioglossus muscle. Experimental Neurology 51, 160–170. & (
- 1983). L-Tryptophan in the treatment of impaired respiration in sleep. Bulletin European de Physiopathologie Respiratoire 19, 625–629. (
- 1996). Presynaptic inhibition of glutamatergic synaptic transmission to rat motoneurons by serotonin. Journal of Neurophysiology 76, 799–807. , & (
- 1996). Presynaptic inhibition by serotonin: A possible mechanism for switching motor output of the hypoglossal nerve. Sleep 19, S146–149. & (
- 1998). Serotonergic and cholinergic influences on pharyngeal muscles. Sleep 21(S), 140. , , , & (
- 2000). Serotonergic stimulation of the genioglossus and the response to nasal continuous positive airway pressure. American Journal of Respiratory and Critical Care Medicine 162, 925–929. , & (
- 1999). Monitoring respiratory function and sleep in the obese Vietnamese pot-bellied pig. Journal of Applied Physiology 87, 444–451. , , & (
- 1999). The effects of trazodone with L-tryptophan on sleep-disordered breathing in the English bulldog. American Journal of Respiratory and Critical Care Medicine 160, 1659–1667. , , , & (
- 2000). Anatomical and physiological correlates of the 5-HT3 receptor within the hypoglossal nucleus. Sleep 23 (S2), A110. , , , , , & (
- 1996). Behaviour of raphe cells projecting to the dorsomedial medulla during carbachol-induced atonia in the cat. Journal of Physiology 490, 745–758. , , & (
- 1999). Hypoglossal motoneurons are postsynaptically inhibited during carbachol-induced rapid-eye-movement sleep. Neuroscience 94, 11–15. , , , & (
- 1993). The occurrence of sleep-disordered breathing among middle-aged adults. New England Journal of Medicine 328, 1230–1235. , , , & (
This work was supported by operating funds from the Medical Research Council (MRC) of Canada (MT-15563), Ontario Thoracic Society and the Elsie W. Crann Memorial Trust Award from the University of Toronto. The authors also gratefully acknowledge equipment grants from the Canada Foundation for Innovation and Ontario Research and Development Challenge Fund. R.L.H is a recipient of an MRC of Canada Scholarship. S.S is recipient of an NSERC Canada studentship. The authors thank Ms Maria Mendes, Mount Sinai Hospital for performing the X-rays. Dr Richard Stephenson, Department of Physiology, is thanked for critical reading of the manuscript.
A. Jelev and S. Sood contributed equally to this paper.