Based on an analysis of the direction of maximum horizontal compressive stress as a function of depth as observed at different scientific wells along the San Andreas fault, it has recently been suggested that the scale-invariant fluctuations in the stress orientation over intervals from tens of centimeters to several kilometers are directly related to the local earthquake magnitude-frequency statistics. Here we mathematically analyze the possibility of such a relationship and show that the magnitude-frequency statistics alone is insufficient to explain the scaling of the stress orientation fluctuations. While stress perturbations caused by slip on adjacent faults of various sizes can still be responsible for these fluctuations, the average amplitude of the induced changes in the orientation would have to increase nonlinearly with the fault size. As the example of two research wells near the San Andreas fault also shows, the specific nonlinear form would have to depend sensitively on the specific geographic location. We conclude that the observed scale-invariant fluctuations in the stress orientation are more likely a consequence of a combination of local seismicity rates and the specific local fault structure.