Humpback whales (Megaptera novaeangliae) sing long, complex songs which continually change over time. In certain time periods the songs contain repeating patterns which structurally resemble human rhyming, and which, like human rhyming, tend to occur in highly rhythmical contexts. We speculate that humpback whales may use these repetitions as mnemonic devices, much as humans are thought to use rhymes. With this speculation in mind we examine 548 humpback whale songs from 7 years in the eastern North Pacific and 12 years in the western North Atlantic oceans. We find that rhyme-like structures are most likely to occur in songs containing the most material to be remembered.