Predicting which crystalline modifications can exist in a chemical system requires the global exploration of its energy landscape. Due to the large computational effort involved, in the past this search for sufficiently stable minima has been performed employing a variety of empirical potentials and cost functions followed by a local optimization on the ab initio level. However, this might introduce some bias favoring certain types of chemical bonding and entails the risk of overlooking important modifications that are not modeled accurately using empirical potentials. In order to overcome this critical limitation, it is necessary to employ ab initio energy functions during the global optimization phase of the structure prediction. In this paper, we review the current state of the field of structure prediction on the ab initio level.