The sustainable seafood movement is over a decade old. It has done much to raise awareness regarding improper production and harvest of seafood and to derive a course to lessen the deleterious environmental impacts of this industry. Certification has been a key tool, yet few programmes have demonstrated comprehensive improvements. Here, the degree of aquaculture improvement through the implementation of certification was assessed using data from the Global Aquaculture Alliance (GAA) Best Aquaculture Practice (BAP) shrimp standard. An examination of 323 audits from 192 farms spanning 2005–2012 showed that overall, 35% of the farms were conditionally certified, indicating they had to improve prior to being certified. This version of the BAP shrimp farm audit had both compliance and scored components. Out of the 28 critical points, only six were in full compliance by all farms during all audits and hence provided no value to determine farm performance. Farms that passed the audit without compliance issues had a greater aggregate scored value than those that farms with noncompliances. However, performance-based metrics exhibited few differences between the compliant and noncompliant farms. Overall, issues pertaining to water quality were a leading cause of farms being scored as noncompliant, although they were distributed among the seven different water quality parameters. Certification systems have not been designed specifically to demonstrate adherence to continual improvement. Because of this, and the multitude of factors with which a fully compliant farm needs to acquiesce, the specific means by which certification improves aquaculture and the overall value of improvement will remain challenging to demonstrate.