Weakly collisional magnetized cosmic plasmas have a dynamical tendency to develop pressure anisotropies with respect to the local direction of the magnetic field. These anisotropies trigger plasma instabilities at scales just above the ion Larmor radius ρi and much below the mean free path λmfp. They have growth rates of a fraction of the ion cyclotron frequency, which is much faster than either the global dynamics or even local turbulence. Despite their microscopic nature, these instabilities dramatically modify the transport properties and, therefore, the macroscopic dynamics of the plasma. The non-linear evolution of these instabilities is expected to drive pressure anisotropies towards marginal stability values, controlled by the plasma beta βi. Here this non-linear evolution is worked out in an ab initio kinetic calculation for the simplest analytically tractable example – the parallel (k⊥= 0) firehose instability in a high-beta plasma. An asymptotic theory is constructed, based on a particular physical ordering and leading to a closed non-linear equation for the firehose turbulence. In the non-linear regime, both the analytical theory and the numerical solution predict secular (∝t) growth of magnetic fluctuations. The fluctuations develop a k−3∥ spectrum, extending from scales somewhat larger than ρi to the maximum scale that grows secularly with time (∝t1/2); the relative pressure anisotropy (p⊥−p∥)/p∥ tends to the marginal value −2/βi. The marginal state is achieved via changes in the magnetic field, not particle scattering. When a parallel ion heat flux is present, the parallel firehose mutates into the new gyrothermal instability (GTI), which continues to exist up to firehose-stable values of pressure anisotropy, which can be positive and are limited by the magnitude of the ion heat flux. The non-linear evolution of the GTI also features secular growth of magnetic fluctuations, but the fluctuation spectrum is eventually dominated by modes around a maximal scale ∼ρilT/λmfp, where lT is the scale of the parallel temperature variation. Implications for momentum and heat transport are speculated about. This study is motivated by our interest in the dynamics of galaxy cluster plasmas (which are used as the main astrophysical example), but its relevance to solar wind and accretion flow plasmas is also briefly discussed.