Two similar structural versions of a molecular donor, in which two terminal hexyl-substituted bithiophene units are connected to a central dithienosilole (DTS) through electron deficient thiadiazolopyridine (PT) units, and which differ only in the position of pyridyl N-atoms, were explored to study the interplay of crystallization and vertical phase segregation as a result of annealing. The donor materials exhibit greatly contrasting photovoltaic performance despite similarity in molecule structure. The difference in position of the pyridal N-atom which points away (distal configuration; compound 1) or towards (proximal configuration; compound 2) from the DTS core, modifies the aggregation/molecular packing in the solid state, resulting in differences in the phase segregation and formation of crystalline domains. A systematic study of the temperature dependence of photovoltaic performance reveals a parameter trade-off: as annealing temperature increases, higher donor crystallinity and ordering results, but increased donor segregation near the surface or decrease in electrode selectivity also occurs, resulting in increased interfacial recombination and hence reduction in open-circuit voltage (Voc). The higher crystalline nature of 2 is found to have a higher impact on cell performance despite a competing undesired effect at the donor/aluminum cathode interface, contributing to its superior performance to 1 when blended with [6,6]phenyl-C61-butyric acid methyl ester (PC61BM). Molecule 2 exhibits a performance increase of a factor of two after thermal annealing at 100 °C, achieving a power conversion efficiency of 5.7%.