Joint modeling of recurrent events and a terminal event has been studied extensively in the past decade. However, most of the previous works assumed constant regression coefficients. This paper proposes a joint model with time-varying coefficients in both event components. The proposed model not only accommodates the correlation between the two type of events, but also characterizes the potential time-varying covariate effects. It is especially useful for evaluating long-term risk factors' effect that could vary with time. A Gaussian frailty is used to model the correlation between event times. The nonparametric time-varying coefficients are modeled using cubic splines with penalty terms. A simulation study shows that the proposed estimators perform well. The model is used to analyze the readmission rate and mortality jointly for stroke patients admitted to Veterans Administration (VA) Hospitals.