The design, synthesis, and characterization of a hierarchically ordered composite whose structure and optical properties can be reversibly switched by adjustment of solvent conditions are described. Solvent-induced swelling and de-swelling is shown to provide control over the internal packing arrangement and hence, optical properties of in situ synthesized metal nanoparticles. Specifically, a gold-nanoparticle-containing ionic-liquid-derived polymer is synthesized in a single step by UV irradiation of a metal-ion-precursor-doped, self-assembled ionic liquid gel, 1-decyl-3-vinylimidazolium chloride. Small-angle X-ray scattering (SAXS) studies indicate that in the de-swollen state, the freestanding polymer adopts a perforated lamellar structure. Optical spectroscopy of the dried composite reveals plasmon resonances positioned in the near-IR. Strong particle–particle interactions arise from matrix-promoted formation of aggregated 1D clusters or chains of gold nanoparticles. Upon swelling in alcohol, the composite undergoes a structural conversion to a disordered structure, which is accompanied by a color change from purple to pale pink and a shift in the surface plasmon resonance to 527 nm, consistent with isolated, non-interacting particles. These results demonstrate the far-field tuning of the plasmonic spectrum of gold nanoparticles by solvent-mediated changes in its encapsulating matrix, offering a straightforward, low-cost strategy for the fabrication of nanophotonic materials.