High mechanical performances of macroscopic-scale fibers hierarchically constructed with carbon nanotubes (CNTs) are attracting great interest in the materials community owing to their merits of light weight and multiple functionalities. However, from the viewpoint of structural design, many fundamental issues, for example, modulus, strength, and deformation mechanisms of such CNT fibers are not yet well understood. In this Full Paper, the axial compression of hierarchical CNT fibers embedded in epoxy is investigated with the assistance of in situ Raman spectroscopy. Experimental results reveal that the conspicuous stiffening and strengthening effects of embedded CNT fibers are dominated by the constituent CNTs within the fiber, and have not yet been observed for conventional carbon fibers. Moreover, hierarchically structured CNT fibers exhibit notable flexibility without permanent deformation and failure under large-strain compression.