Human-induced alterations in the birth and mortality rates of species and in the strength of interactions within and between species can lead to changes in the structure and resilience of ecological communities. Recent research points to the importance of considering the distribution of body sizes of species when exploring the response of communities to such perturbations. Here, we present a new size-based approach for assessing the sensitivity and elasticity of community structure (species equilibrium abundances) and resilience (rate of return to equilibrium) to changes in the intrinsic growth rate of species and in the strengths of species interactions. We apply this approach on two natural systems, the pelagic communities of the Baltic Sea and Lake Vättern, to illustrate how it can be used to identify potential keystone species and keystone links. We find that the keystone status of a species is closely linked to its body size. The analysis also suggests that communities are structurally and dynamically more sensitive to changes in the effects of prey on their consumers than in the effects of consumers on their prey. Moreover, we discuss how community sensitivity analysis can be used to study and compare the fragility of communities with different body size distributions by measuring the mean sensitivity or elasticity over all species or all interaction links in a community. We believe that the community sensitivity analysis developed here holds some promise for identifying species and links that are critical for the structural and dynamic robustness of ecological communities.