Multiple mutations in different subunits of the tethering complex Conserved Oligomeric Golgi (COG) have been identified as a cause for Congenital Disorders of Glycosylation (CDG) in humans. Yet, the mechanisms by which COG mutations induce the pleiotropic CDG defects have not been fully defined. By detailed analysis of Cog8 deficiency in either HeLa cells or CDG-derived fibroblasts, we show that Cog8 is required for the assembly of both the COG complex and the Golgi Stx5-GS28-Ykt6-GS15 and Stx6-Stx16-Vti1a-VAMP4 SNARE complexes. The assembly of these SNARE complexes is also impaired in cells derived from a Cog7-deficient CDG patient. Likewise, the integrity of the COG complex is also impaired in Cog1-, Cog4- and Cog6-depleted cells. Significantly, deficiency of Cog1, Cog4, Cog6 or Cog8 distinctly influences the production of COG subcomplexes and their Golgi targeting. These results shed light on the structural organization of the COG complex and its subcellular localization, and suggest that its integrity is required for both tethering of transport vesicles to the Golgi apparatus and the assembly of Golgi SNARE complexes. We propose that these two key functions are generally and mechanistically impaired in COG-associated CDG patients, thereby exerting severe pleiotropic defects.