We investigate the propagation of electromagnetic waves in a cylindrical waveguide with an arbitrary cross section filled with a nonlinear material. The electromagnetic field is expanded in the usual eigenmodes of the waveguide, and the coupling between the modes is quantified. We derive the wave equations governing each mode with special emphasis on the situation with a dominant TE mode. The result is a strictly hyperbolic system of nonlinear partial differential equations for the dominating mode, whereas the minor modes satisfy hyperbolic systems of linear, nonstationary, and partial differential equations. A growth estimate is given for the minor modes.
 Electromagnetic waves can be guided in space by a hollow waveguide, where the walls are electric conductors. This can be used to guide waves traveling from one point to another, and provides a controlled environment in which measurements can be made. In order to be able to interpret these measurements, we need to investigate what influence the waveguide structure and material or filling have on the wave propagation.
 The waves can be decomposed in modes, which can be defined as the eigenfunctions of a transversal differential operator. These modes are orthogonal, and the wave equation for each mode decouples completely from the other modes for a linear, homogeneous filling. However, in practice there may still be a mode coupling due to irregularities or finite conductivity in the walls, particularly when degenerate modes are present. When the waveguide is filled with a nonlinear material an additional coupling is introduced, and the equations do not decouple even when assuming perfectly conducting walls. However, it is still expedient to use the standard waveguide modes due to the possibility of using the resulting equations with a mode-matching algorithm in direct and inverse scattering problems [Petit, 1980; Roberts, 1988].
 In this paper we study the propagation of transient waves in nonlinear waveguides, i.e., waves generated by an arbitrary signal. The theory of linear waveguides is well established since the major efforts during the second world war. The analysis is often made in the frequency domain, but since the nonlinear filling not only couples the modes but also induces a coupling between the different frequency components, we choose to treat the problem entirely in the time domain. The propagation of transient waves in waveguides has been treated for linear materials by, among others, Bernekorn et al. , Kristensson , and Cotte , and some general references are Collin , Jackson [1999, chapter 8], and Olyslager .
 There has been a number of papers on nonlinear waveguides. Some recent contributions consider the problem of self focusing, where the field energy inside the waveguide moves closer to the center as the wave propagates. A few early studies are found in the works of Kelley  and Chiao et al.  and a more recent one is [Polstyanko et al., 1997]. In the work of Rozzi and Zappelli  a modal field expansion is applied for a dielectric slab waveguide at a fixed frequency. The resulting equations are mainly used to determine where the energy will be localized.
 This paper is organized as follows. In section 2 we introduce Maxwell's equations and the instantaneous constitutive relations. The waveguide geometry is presented in section 3, together with the relevant expansion functions. We use these expansion functions to obtain wave equations for each mode in section 4, and the explicit results for a parallel-plate waveguide are calculated in section 5. Since even this simple example proves very challenging, we make the reasonable assumption that almost all the energy is contained in one mode in section 6, which enables us to derive a system of quasi-linear, homogeneous, hyperbolic differential equations for the dominant mode, and a system of inhomogeneous hyperbolic equations with source terms for the minor modes. Some energy relations are derived for both the dominant and the minor modes, allowing an estimate of the growth of the minor modes. The final conclusions and discussions are given in section 7.
 In this paper we use a slight modification of the Heaviside-Lorentz units [Jackson, 1999, p. 781], where the electromagnetic fields are scaled so that they all have the physical dimension J1/2 m−3/2
where ESI, HSI, DSI and BSI are the electric and magnetic field strength and flux density, all in SI units. The permittivity, permeability, and the speed of light in vacuum are denoted by ϵ0, μ0, and c0 = (ϵ0μ0)−1/2, respectively.
where Fe and Fm are dimensionless functions of E2(r, t) = ∣E(r, t)∣2 and H2(r, t) = ∣H(r, t)∣2, respectively. We use the squared absolute values as arguments instead of the absolute values themselves since this is beneficial in the final equations. In the absence of sources, Maxwell's equations now read
In the following section we describe the expansion of the electromagnetic fields in waveguide modes.
3. Waveguide Mode Expansion Functions
 The geometry of the waveguide is depicted in Figure 1. The canonical problem in determining the transverse behavior of the fields is given by the scalar eigenproblems
where the acronyms TE and TM stand for Transverse Electric field and Transverse Magnetic field, respectively. The real numbers λnTE and λnTM correspond to eigenvalues of the transverse Laplace operator ∇T2 = ∂x2 + ∂y2, given the respective boundary condition, and are arranged in ascending order 0 < λ1 ≤ λ2 ≤ ⋯. The canonical solutions ϕnTE and ϕnTM are normalized by requiring
which implies that the scalar eigenfunctions are dimensionless. We use these eigenfunctions to expand the electric and magnetic field as (where is the unit vector in the z direction)
The expansion coefficients unTE, vnTE etc are real-valued functions of z and t, whereas the six-vector expansion functions UnTE, VnTE etc are functions of x and y. This set of expansion functions is complete [Collin, 1991, p. 329], and orthonormal in the scalar product
for instance, (UmTE, UnTE) = 1 if m = n and 0 if m ≠ n.
4. Decomposition in Modes
 We continue the analysis by constructing the scalar product of the expansion functions with Maxwell's equations, in order to remove the transverse dependence. Hence, upon denoting an arbitrary expansion function by Ψm, we examine equations of the form
for each possible Ψm.
4.1. Linear Terms
 It is convenient to separate the curl operator according to
So far, all our work is well known from the corresponding linear analysis, and the orthogonality relations of the expansion functions makes it easy to evaluate the linear terms in integral (8). We now turn our attention to the nonlinear term.
4.2. Nonlinear Term
 In this subsection, we analyze the integral
For the time being, we ignore the time derivative and consider the exact form of the integral for all possible expansion functions Ψm. Considering the vectorial properties, many terms in (12) vanish, leaving only the integrands
Since the scalar products contain the functions Fe(E2) and Fm(H2), the remaining terms do not simplify, and the modes couple. We contend that the nonlinearity mainly affects the wave propagation properties. Accordingly, we propose to neglect the nonlinear effect over the cross section to some extent. However, we shall preserve the prominent mode-coupling mechanism.
 An obvious approach is to expand Fe(E2) and Fm(H2) in a Taylor series and explicitly calculate the corresponding integrals. Since the expressions for E2(x, y, z, t) and H2(x, y, z, t) are rather complex, we wish to delay this approach for a while. Instead we suggest to substitute E2(x, y, z, t) and H2(x, y, z, t) with some suitable functions independent of the transverse variables x and y, i.e., 2(z, t) and 2(z, t)
and treat the terms in square brackets as perturbations which are neglected in the leading order equations. Since the factors Fe(2) and Fm(2) are independent of x and y, they can be pulled out of the scalar products in (13) and (14), and we can then use orthogonality. Though, as we show in section 4.3, we must generally choose a different 2 and 2 for each expansion function Ψm, which we denote by an index, 2ψm and 2ψm. Since the electric field is associated only with UmTE, VmTM, and WmTE, and the magnetic field is associated only with VmTE, WmTE, and UmTM, we do not emphasize the TE or TM mode when indexing 2ψm and 2ψm in the following. The resulting equations are then deduced from (10), (11), (13), and (14) as
for the TE modes, and
for the TM modes. These equations are strictly hyperbolic, and the equation structure is exactly the same for both TE and TM modes; only the functions Fe and Fm must be interchanged. A system of partial differential equations ∂tu + A(u)∂zu = 0 is strictly hyperbolic if all eigenvalues of the matrix A(u) are real and distinct [Evans, 1998, p. 573]. In our case, it can be shown that there is one positive eigenvalue, one negative eigenvalue and one zero eigenvalue.
 In the next section we discuss the approximation leading to this result, but we must first consider an important detail. Each of the systems (16) and (17) must be supplemented by three initial conditions (three dependent variables, three equations, three conditions), but only two initial conditions can be chosen independent of each other. Since we know that ∇ · B = ∇ · D = 0 inside the waveguide, we must introduce the additional constraints
to maintain compatibility with the original equations. Note that λmTE and λmTM correspond to the x and y derivatives. These constraints can also be derived from the last two equations in (16) and (17), respectively.
4.3. Error Estimates
 To assess the validity of the approximation made above, let us concentrate on the electric field. Since the explicit representation of E2 = E · E is rather complicated if we use the expansion functions UnTE, VnTM and WnTM, we suppress this difficulty by using generic expansion functions Ψn and corresponding expansion coefficients ψn
The error we wish to estimate is formulated as the scalar product
and upon expanding Fe(E2) in a Taylor series about E2 = ψm2 we see that this term is at most O(Fe″(ψm2) ∫∫Ω(E2 − ψm2)2E dx dy) if
This can be accomplished by choosing
which is the exact result for a Kerr material, i.e., Fe(E2) = 1 + E2. For other materials this represents an approximation to the first order in a Taylor series expansion of the constitutive function Fe(E2), and we see that this choice of ψm2 is equivalent to the Taylor expansion suggested earlier. However, by delaying the introduction of the Taylor expansion until this stage, we have gained an opportunity to make other choices of ψm2, depending on the situation at hand.
 It is clear that in general ψm2 depends on which expansion function Ψm is used. We can calculate the scalar products (Ψm, Ψk · ΨlΨn) analytically for a few geometries, especially the rectangular and the parallel-plate waveguide. For general geometries, we must resort to numerical calculations of the scalar products. In the following section, we show that even when we can calculate all integrals analytically, the problem is still quite a challenge.
5. Parallel-Plate Waveguide With Nonmagnetic Material
 We analyze one of the simplest possible waveguides, i.e., the parallel-plate waveguide filled with a nonmagnetic material. The geometry is depicted in Figure 2, and the expansion functions are
If the two plates are held at different potentials, we also have the TEM mode,
but we assume the TEM mode to be absent in this section. One set of nonzero scalar products we need to compute is
It is clear that, even for the simplest cases, we have a formidable problem. For instance, we see that each mode generally couple to infinitely many others with a nondecreasing coupling factor. This means that in order to proceed, we should impose more restrictions on the problem.
6. Dominant TE Mode in a Nonmagnetic Material
 In the previous section it is shown that the relation (23) is quite complicated to handle explicitly. In this section, we look at a simplified case for a nonmagnetic material (Fm(H2) = 1), where almost all the energy is contained in the first TE mode, which we denote TE1. This mode is chosen since in the absence of TEM modes it generally corresponds to the lowest eigenvalue λ1 [Courant and Hilbert, 1953, p. 410], and should be the easiest to generate. The assumption of a dominant TE1 mode implies
The expansion coefficients v1TE and w1TE are associated with the magnetic field, and do not enter our calculations below. Therefore we expand all ψm2 with respect to ∣u1TE∣ → ∞, and assume these quantities to be sufficiently regular for differentiation of their expansions to be warranted.
6.1. Leading Terms for UmTE
 With the above assumption on the relative sizes of the expansion coefficients, a close look at the relation in (23) reveals that the leading term associated with the dominant mode is
where the index “u1” is used to indicate the relation to the expansion function U1TE. For the other TE modes, we have
and the constants αu1, αum and βum are defined as
Notice that the second term in um2 is one order less in u1TE than the first. This term is needed since the first will cancel in the term that governs the principal wave dynamics. As soon as βum ≠ 0, the first term can be very large for very small umTE. This is the origin of the mode coupling, and causes the excitation of new modes which were not present in the beginning. This is made more clear in the sections to follow. In Figure 3, the distribution of the coupling factors is shown for the rectangular and circular waveguides.
 In section 4.2 we deduced the equations (16) for the TE modes, where the time derivative ∂t(Fe(um2)umTE) is included. From now on, we suppress the higher-order terms in the expansion of um2. For the dominant mode the time derivative is then
and for the other TE modes it is
Using the leading order expansion Fe(um2) = Fe((u1TE)2αum) + Fe′((u1TE)2αum) βum(u1TE)3/umTE, we find Fe(um2) − Fe′(um2)βum(u1TE)3/umTE = Fe((u1TE)2αum) + higher-order terms. This implies
i.e., the wave speed for TE mode m depends only on the dominant mode. As is shown in the next section, this is valid for the TM modes as well.
6.2. Leading Terms for VmTM and WmTM
 It can be shown that the leading terms in vm2 and wm2 corresponding to VmTM and WmTM, respectively, are
Observe that there is no term (u1TE)3/wmTM in the expression for wm2 since the corresponding βwm is always zero. The constants αvm, βvm and αwm are
Calculations analogous to the previous subsection show that the pertinent time derivatives are
6.3. Resulting Equations for a Dominant TE Mode
 Gathering the results derived above, we find that to the leading order, the equations for the dominant TE mode are
for the rest of the TE modes. For the TM modes, we have
Some conclusions from these equations are as follows.
 1. The model is suitable when the dominant mode is not affected by the minor modes. It is modeled by a system of quasi-linear partial differential equations without source terms, which can be reduced to the scalar problem −∂z2u + ∂t2[F(u2)u] + λ2u = 0.
 2. The minor modes travel through an inhomogeneous medium with source terms. Both the inhomogeneity and the source terms are induced by the dominant mode.
6.4. Energy for the Dominant Mode
 By multiplying the first, second and third equation in equation (38) by u1TE, v1TE and w1TE, respectively, and adding the equations, we obtain
After some algebra, we find that this can be written as
where we have introduced the energy density function
This is exactly the energy density for propagation in an unbounded medium; the influence of the waveguide is reduced to a scaling constant αu1. The energy density function can be calculated explicitly for some common cases. For the Kerr medium, where Fe(E2) = 1 + E2 (after scaling to dimensionless fields), we have
It is easy to show that for a medium where Fe can be expanded in a polynomial series, the energy density can be expressed in a related series. Another medium which has been used is the saturated Kerr medium [Kristensson and Wall, 1998; Sjöberg, 1999; Tikhonenko et al., 1996], where Fe(E2) = 1 + E2/(1 + E2). The energy density for this medium is
since the integral of the z derivative correspond to the field values at infinity, and can be assumed to disappear for finite times T. This means that the energy of the dominant mode is conserved in this approximation.
6.5. An Estimate for the Mode Spreading
 We have shown that one of the most distinct features of nonlinearity in the propagation of guided waves is that the modes are no longer independent, but rather couple in an intricate manner. In the case of a dominant mode, this coupling appears as creating inhomogeneities and source terms for the minor modes. In this section we estimate how fast these minor modes grow when the dominant mode is known.
 We refrain from using the explicit representation of the source terms in equations (38), (39), and (40), and return to the generic TE case (cf. (16))
Again, we use um2 = βum(u1TE)3/umTE + αum(u1TE)2 + higher-order terms. The differential equation for the power per unit length now reads
For the dominant mode, we found a total energy, i.e., a total time derivative. This is no longer possible for the other modes, but a step in the desired direction is
Integrating this equation over z, implies
where we have assumed that shocks do not occur, i.e., the time derivative is bounded. We apply Grönwall's inequality [see, e.g., Evans, 1998, p. 624], to find
This seems to imply that if we put no energy in this mode from the start, it will stay silent. However, as we clearly see in the first line of (39), there is a source term which depends solely on u1TE and initiates the minor modes. The estimate (51) is simply not valid in the limit umTE → 0 since
Thus we can expect a relatively rapid growth when umTE is very small and βum ≠ 0. In the case of the parallel-plate waveguide, it is easily seen that the coupling factor is βum = Cδm,3.
7. Conclusions and Discussion
 The coupling between the modes produces equations for a general mode analysis which is hard to perform. In case one mode is dominant, we obtain a tractable problem. The dominant mode is described by a nonlinear system of homogeneous partial differential equations, and the minor modes are described by a linear system of inhomogeneous partial differential equations. The equations describing the propagation of the dominant mode are inert with respect to the minor modes, and should be object for further study.
 Mode spreading is always present. The mechanisms behind this deserves further examination. Some open questions are: (1) is there an equilibrium in the mode distribution, (2) when is the mode spreading strong enough to influence the dominant mode, and (3) are there sharper estimates of the growth of the minor modes? It should be stressed that the relative ease of implementing a finite difference algorithm for the three-dimensional analysis of a rectangular waveguide makes the numerical study of the “true” mode spreading possible.
 Since we have employed the usual waveguide modes for the analysis, the equations derived in this paper may be of interest in a mode matching algorithm, especially for the inverse scattering problem. A remaining problem is the propagation through the boundary between a nonlinear material and, for instance, vacuum.
 The author acknowledges fruitful discussions with Gerhard Kristensson and several colleagues at the Electromagnetic Theory Group in Lund. He also thanks the anonymous referees for constructive suggestions which helped improving this paper.
 The work reported in this paper is partially supported by a grant from the Swedish Research Council for Engineering Sciences and its support is gratefully acknowledged. A large portion of this paper was written during a stay at the Department of Mathematics and Statistics of the University of Canterbury, Christchurch, New Zealand, and their warm hospitality is most appreciated. The author also thanks the Royal Physiographical Society of Lund (Kungliga fysiografiska sällskapet i Lund), for a grant enabling this visit.