This article investigates the impact of spatially correlated unobservable variables on the refinancing, selling and default decisions of mortgage borrowers. Virtually the entire mortgage literature acknowledges that borrower-specific characteristics, such as culture, education or access to information, play an important role in mortgage termination decisions. While we do not observe these variables directly, we note that borrowers of similar background tend to cluster together in neighborhoods. We estimate a competing risks hazard model with random effects using a three-stage maximum likelihood estimation approach. We utilize the space-varying coefficient method to modify the covariance structure according to the spatial distribution of the observations. Beyond a significant improvement of the model performance, this yields a number of insightful implications for mortgage termination behavior. For instance, borrowers of the affluent “West Side” of Los Angeles County both refinance and move at a higher rate than predicted by the standard maximum likelihood estimation method. At the same time, borrowers from some lower-valued neighborhoods tend to stay longer than expected with their mortgages and properties.