The site-specific attachment of nanoparticles is of interest for biomaterials or biosensor applications. Polymer brushes can be used to regulate this adsorption, so the conditions for selective adsorption of phosphonate-functionalized nanoparticles onto micropatterned polymer brushes with different functional groups are optimized. By choosing the strong polyelectrolytes poly(3-sulfopropyl methacrylate), poly(sulfobetaine methacrylate), and poly[2-(methacryloyloxy)ethyl trimethylammonium chloride], it is possible to direct the adsorption of nanoparticles to specific regions of the patterned substrates. A pH-dependent adsorption can be achieved by using the polycarboxylate brush poly(methacrylic acid) (PMAA) as substrate coating. On PMAA brushes, the nanoparticles switch from attachment to the brush regions to attachment to the grooves of a patterned substrate on changing the pH from 3 to 7. In this manner, patterned substrates are realized that assemble nanoparticles in pattern grooves, in polymer brush areas, or substrates that resist the deposition of the nanoparticles. The nanoparticle deposition can be directed in a pH-dependent manner on a weak polyelectrolyte, or is solely charge-dependent on strong polyelectrolytes. These results are correlated with surface potential measurements and show that an optical trap is a versatile method to directly probe interactions between nanoparticles and polymer brushes. A model for these interactions is proposed based on the optical trap measurements.