Over-exploitation and habitat degradation are the two major drivers of global environmental change and are responsible for local extinctions and declining ecosystem services. Here we compare the top-down effect of exploitation by fishing with the bottom-up influence of habitat loss on fish communities in the most diverse of ecological systems, coral reefs. Using a combination of multivariate techniques and path analyses, we illustrate that the relative importance of coral cover and fishing in controlling fish abundance on remote Fijian reefs varies between species and functional groups. A decline in branching Acropora coral is strongly associated with a decline in abundance of coral-feeding species, and a decrease in coral-associated habitat complexity, which has indirectly contributed to reduced abundance of small-bodied damselfish. In contrast, reduced fishing pressure, brought about by declining human populations and a shift to alternate livelihoods, is associated with increased abundance of some piscivores and fisheries target species. However, availability of prey is controlled by coral-associated habitat complexity and appears to be a more important driver of total piscivore abundance compared with fishing pressure. Effects of both fishing and coral loss are stronger on individual species than functional groups, as variation in the relative importance of fishing or coral loss among species within the same functional group attenuated the impact of either of these potential drivers at the functional level. Overall, fishing continues to have an influence on Fijian fish communities; however, habitat loss is currently the overriding agent of change. The importance of coral loss mediated by climate change is expected to have an increasing contribution to fish community dynamics, particularly in remote locations or where the influence of fishing is waning.