In principle, parental relatedness, parental age, and the age of parental gametes can all influence offspring fitness through inbreeding depression and the parental effects of organismal and postmeiotic gametic senescence. However, little is known about the extent to which these factors interact and contribute to fitness variation. Here, we show that, in Drosophila melanogaster, offspring viability is strongly affected by a three-way interaction between parental relatedness, parental age, and gametic age at successive developmental stages. Overall egg-to-adult viability was lowest for offspring produced with old gametes of related, young parents. This overall effect was largely determined at the pupa–adult stage, although three-way interactions between parental relatedness, parental age and gametic age also explained variation in egg hatchability and larva-pupa survival. Controlling for the influence of parental and gametic age, we show that inbreeding depression is negligible for egg hatchability but significant at the larva–pupa and pupa–adult stages. At the pupa–adult stage, where offspring could be sexed, parental relatedness, parental age, and gametic age interacted differently in male and female offspring, with daughters suffering higher inbreeding depression than sons. Collectively, our results demonstrate that the architecture of offspring fitness is strongly influenced by a complex interaction between parental effects, inbreeding depression and offspring sex.