Rhizobial soil bacteria can form a symbiosis with legumes in which the bacteria fix atmospheric nitrogen into ammonia that can be utilized by the host. The plant, in turn, supplies the rhizobia with a carbon source. After infecting the host cell, the bacteria differentiate into a distinct bacteroid form, which is able to fix nitrogen. The bacterial BacA protein is essential for bacteroid differentiation in legumes producing nodule-specific cysteine-rich peptides (NCRs), which induce the terminal differentiation of the bacteria into bacteroids. NCRs are antimicrobial peptides similar to mammalian defensins, which are important for the eukaryotic response to invading pathogens. The BacA protein is essential for rhizobia to survive the NCR peptide challenge. Similarities in the lifestyle of intracellular pathogenic bacteria suggest that host factors might also be important for inducing chronic infections associated with Brucella abortus and Mycobacterium tuberculosis. Moreover, rhizobial lipopolysaccharide is modified with an unusual fatty acid, which plays an important role in protecting the bacteria from environmental stresses. Mutants defective in the biosynthesis of this fatty acid display bacteroid development defects within the nodule. In this review, we will focus on these key components, which affect rhizobial bacteroid development and survival.