The photovoltaic parameters, i.e., the short-circuit current, open-circuit voltage and device fill factor, of bulk heterojunction solar cells that use perylene diimide (PDI) derivatives as electron acceptors are often far below the theoretically expected values for reasons still not entirely understood. This article demonstrates that the photovoltaic characteristics of blend films of regioregular poly(3-hexylthiophene) (rr-P3HT) and PDI molecules are improved upon using a core-alkylated PDI derivative instead of the often used N-alkylated PDI molecules. A doubling of the power conversion efficiency of P3HT:PDI solar cells by using the core-alkylated PDI derivative is observed leading to an unprecedented power conversion efficiency of 0.5% for a P3HT:PDI solar cell under AM1.5 solar illumination. Furthermore, the optical properties of the novel PDI derivative are compared to two standard exclusively N-alkylated PDI derivatives by steady-state and time-resolved photoluminescence spectroscopy in solution and solid state. The experiments reveal that aggregation in the solid state determines the photophysics of all PDI derivatives. However, the emission energy and excited state lifetime of the aggregates are clearly influenced by the alkyl-substitution pattern through its effect on the packing of the PDI molecules. X-ray diffraction experiments before and after thermal annealing of PDI:polystyrene and PDI:P3HT blends reveal subtle differences in the packing characteristics of the different PDI derivatives and, problematically, that P3HT ordering is suppressed by all of the PDI derivatives.