Although bird song has been an important model for investigating questions of behavior development, cultural evolution and population differentiation, the quantitative methods of analysis have been problematic. Here we develop and apply quantitative randomization methods to test hypotheses about these processes in a natural population of birds. Songs of the African brood-parasitic straw-tailed whydahs (Vidua fischeri) and songs of their host species, the purple grenadier (Granatina ianthinogaster), were compared in audiospectrograms for similarity to test the following hypotheses: Whydahs mimic the songs of their host species, they have local song dialects, neighboring males match their song themes, local males match the songs of local hosts, remote populations have different songs according to their geographic distance, and songs undergo cultural evolution over time across generations. Randomization analyses were completed using (1) Mantel matrix statistics and (2) tree-based measures employing Sankoff optimization of Manhattan matrices and approximate randomizations. Our results provide evidence for song mimicry, local song dialects, matching song themes between neighboring males, song matching of local whydah mimics and grenadier song models, correspondence of song differences and geographic distance, and cultural continuity with change in song traditions within a local population. These randomization methods may be useful in other studies of animal communication, and they are sufficiently general for use both with distance matrices derived either from naturalistic impressions of song similarity as in our example or from acoustic measurements.