Saturated fatty acids (SFA) have been reported to alter organelle integrity and function in many cell types, including muscle and pancreatic β-cells, adipocytes, hepatocytes and cardiomyocytes. SFA accumulation results in increased amounts of ceramides/sphingolipids and saturated phospholipids (PL). In this study, using a yeast-based model that recapitulates most of the trademarks of SFA-induced lipotoxicity in mammalian cells, we demonstrate that these lipid species act at different levels of the secretory pathway. Ceramides mostly appear to modulate the induction of the unfolded protein response and the transcription of nutrient transporters destined to the cell surface. On the other hand, saturated PL, by altering membrane properties, directly impact vesicular budding at later steps in the secretory pathway, i.e. at the trans-Golgi Network level. They appear to do so by increasing lipid order within intracellular membranes which, in turn, alters the recruitment of loose lipid packing-sensing proteins, required for optimal budding, to nascent vesicles. We propose that this latter general mechanism could account for the well-documented deleterious impacts of fatty acids on the last steps of the secretory pathway in several cell types.