In the present work, we correlate the photophysical and photovoltaic properties with the respective film morphologies of three different blends made of the fluorene copolymers poly(9,9′-dioctylfluorene-co-benzothiadiazole) (F8BT), poly[9,9′-dioctylfluorene-co-N-(4-butylphenyl)diphenylamine] (TFB), and poly[9,9′-dioctyfluorene-co-bis-N,N′-(4-butylphenyl)-bis-N,N-phenyl-1,4-phenylenediamine] (PFB) when blended with a perylene tetracarboxylic diimide (PDI) derivative. Additional photophysical studies in reference PDI blends of the electronically inert poly(styrene) matrix address the enhanced PDI intermolecular solid-state interactions. We resolve the process of resonance energy transfer from excited polymer hosts to PDI and the process of photoinduced hole transfer from PDI to the polymer hosts. We deduce the efficiency of charge-transfer PDI photoluminescence (PL) quenching and we discuss the power-law PL kinetics seen in the as-spun systems. Next we determine the dependence of the device external quantum efficiency (EQE) of these blends, in a range of annealing temperatures and PDI loadings. Differential scanning calorimetry enables precise selection of annealing temperatures. Optical microscopy shows that annealing enhances the order characteristics in the PDI aggregates in the F8BT:PDI system. In the case of the TFB:PDI and PFB:PDI blends, AFM studies suggest the formation of PDI-rich domains on the film/air interface. The degree of order in the π–π stacking of the PDI monomers is inferred by the UV–Vis and PL spectra of the blends. The extent of order characteristics in PDI aggregates is correlated with the thermal properties of the hosts that control PDI molecular mobility upon annealing. The efficient dispersion of disrupted PDI crystallites is proposed to form appropriate percolation networks that favor balanced extraction of photogenerated carriers.