Swedenborgite-type cobaltites and ferrites and derivatives, represent a very important class of oxides, LnBaM4O7 and CaBaM4O7 (M = Co, Fe). They are one of the rare families of mixed valent transition metal oxides, where the transition element exhibits exclusively the tetrahedral coordination. Their complex crystal chemistry, involving closely related structures with various symmetries (orthorhombic, hexagonal cubic) and possibility of oxygen “hyperstoichiometry”, leading to LnBaM4O7+δ oxides is described herein, as well as their ability to exhibit structural transitions vs. temperature. It is also shown that the triangular geometry of these new strongly electron correlated systems, plays an important role in their physics, particularly in the competition that appears between the 1D magnetic ordering and the 2D magnetic frustration. The generation of new magnetodielectric properties for CaBaCo4O7 is also emphasized.