The influence of surface roughness is central in understanding the behavior of various types of shear zones including faults, landslides, and deformation in glacial till. All of these systems contain a non-planar rough wall, which interacts with either a gouge zone or another wall. We use the 3-D discrete element method (DEM) to investigate both the effect of boundary roughness and friction. Granular non-cohesive gouge is sandwiched between rough walls with large grooves, or smooth walls composed of spherical particles that can be adjusted to control roughness. Roughness and gouge properties are scaled to laboratory friction experiments. We vary friction between the particles and the wall and monitor shear strength, height, coordination number, distribution, and orientation of particle forces, localization, and porosity distribution in the shear zone. We find that, on the first-order, strength is controlled by particle-particle friction and mechanical coupling of the fault zone wall to the gouge. Rough boundaries (RMS roughness > grain radius) force more shear within the gouge zone, dilating the layer and sliding more grains, which leads to large stress necessary to shear the layer. When large amplitude roughness is removed, and roughness is at the grain-scale, the coupling, and thus the strength, is controlled by both wall and particle friction as well as fine-scale boundary roughness. These differences are reflected in profiles of shear within the gouge zone and offset at the boundary in smooth models. From our simulations, we quantify how and why rough natural faults will have a higher overall strength.