The evolutionary potential of any given population is of fundamental importance for its longer term prospects. Modern land-use practices often result in small and isolated populations, increasing the risk of extinction through reduced genetic diversity as a consequence of inbreeding or drift. Such genetic erosion may also interfere with a population's evolutionary potential. In this study, we investigate the consequences of inbreeding on evolutionary potential (the ability to increase cold resistance) in a laboratory population of the tropical butterfly Bicyclus anynana. To explore constraints on evolution, we applied artificial selection to chill-coma recovery time, starting from three levels of inbreeding (outbred control, one or two full-sibling matings). Ten generations of selection produced highly divergent phenotypes, with the lines selected for increased cold tolerance showing about 28% shorter recovery times after cold exposure relative to unselected controls. Correlated responses to selection in 10 different life-history and stress-resistance traits were essentially absent. Inbred lines showed a weaker response to selection, indicating reduced evolutionary potential and thereby constraints on evolution. Inbreeding depression was still measurable in some traits after the course of selection. Traits more closely related to fitness showed a clear fitness rebound, suggesting a trait-specific impact of purging. Our findings have important implications for the longer term survival of small populations in fragmented landscapes.