A stochastic jump diffusion (SJD) process is used to describe movement of sediment particles in open channel flows. The stochastic jump diffusion particle-tracking model (SJD-PTM) is governed by the stochastic differential equation (SDE). The SDE consists of three main terms including a mean drift motion term, a Wiener process representing random turbulent motion, and a Poisson process describing the abrupt movement of particles caused by probabilistic occurrences of the extreme flow perturbations. The proposed SJD-PTM can characterize the probabilistic properties of sediment particle movement by simulating the most probable trajectory and ensemble variances of particle movement associated with both flow turbulence and random occurrences of extreme flow perturbations. For the particle movement in response to extreme flow occurrences, we have introduced the particle relaxation time (temporal lag) to quantify the delayed response of sediment particles to large flow perturbations because of the particle inertia effect. The particle relaxation time is derived from a consideration of the forces exerted on particles under the effect of flow accelerations. The particle relaxation time is demonstrated to be dependent on particle size and density as well as the particle Reynolds number. The impact of drag and added mass forces on the ensemble mean and variance of particle trajectory is also evaluated. The particle relaxation time is found to be a factor that diminishes the impact of the extreme flow perturbations. It is shown that heavier and larger particles normally have a larger particle relaxation time, a decreased particle jump magnitude and a smaller ensemble variance of particle trajectories in the occurrences of extreme flow perturbations. It is concluded that the proposed SJD-PTM with the particle relaxation time can better describe the movement of sediment particles of nonnegligible size in response to extreme flow perturbations. The ability to quantify the variances of particle movement more comprehensively is one of the major advantages associated with the SJD-PTM for longer-term sediment transport modeling and predictions. The proposed SJD-PTM is verified against two distinct experimental data.