Sexual dichromatism in birds is often attributed to selection for elaboration in males. However, evolutionary changes in either sex can result in plumage differences between them, and such changes can result in either gains or losses of dimorphism. We reconstructed the evolution of plumage colors in both males and females of species in Maluridae, a family comprising the fairy-wrens (Malurus, Clytomias, Sipodotus), emu-wrens (Stipiturus), and grasswrens (Amytornis). Our results show that, across species, males and females differ in their patterns of color evolution. Male plumage has diverged at relatively steady rates, whereas female coloration has changed dramatically in some lineages and little in others. Accordingly, in comparisons against evolutionary models, plumage changes in males best fit a Brownian motion (BM) model, whereas plumage changes in females fit an Ornstein Uhlenbeck (OU) multioptimum model, with different adaptive peaks corresponding to distributions in either Australia or New Guinea. Levels of dichromatism were significantly associated with latitude, with greater dichromatism in more southerly taxa. Our results suggest that current patterns of plumage diversity in fairy-wrens are a product of evolutionary changes in both sexes, driven in part by environmental differences across the distribution of the family.