The enigmatic seismicity in the New Madrid Seismic Zone (NMSZ) has been attributed to some abnormal lithospheric structure, including the presence of dense mafic intrusions and a low-viscosity lower crust. However, the area's detailed lithospheric structure remains unclear. Here we invert 2,056 teleseismic P and 12,226 local P first arrival times from a recent nine-year dataset to infer the lithospheric velocity structure beneath the NMSZ. Our results show that the seismically active zone is associated with a local, NE–SW trending low-velocity anomaly in the lower crust and upper mantle, instead of high-velocity intrusive bodies proposed in previous studies. The low-velocity anomaly is on the edge of a high-velocity lithospheric block, consistent with the notion of stress concentration near rheological boundaries. This lithospheric weak zone may shift stress to the upper crust when loaded, thus leading to repeated shallow earthquakes.