The combined impact of rotation and magnetic fields on oscillations of stellar fluids is still not well known theoretically. It mixes Alfvén and inertial waves. Neutron stars are a place where both effects may be at work. We aim to solve this problem in the context of the r-mode instability in neutron stars, as it appears when these modes are coupled to gravitational radiation.
We consider a rotating spherical shell filled with a viscous fluid of infinite electrical conductivity and analyse propagation of model perturbations when a dipolar magnetic field is bathing the fluid layer. We perform an extensive numerical analysis and find that the m= 2 r-mode oscillation is influenced by the magnetic field when the Lehnert number (the ratio of Alfvén speed to rotation speed) exceeds a value proportional to the one-fourth power of the Ekman number (a non-dimensional measure of viscosity). This scaling is interpreted as the coincidence of the width of internal shear layers of inertial modes and the wavelength of the Alfvén waves. Applied to the case of rotating magnetic neutron stars, we find that dipolar magnetic fields above 1014 G are necessary to perturb the r-mode instability.