Afforestation has been accepted as a key measure for preventing soil erosion on the Chinese Loess Plateau for 40 years. In this study, we assessed the ecological success of afforestation by comparing afforested with pre-afforested (croplands) and natural recovery sites in a typical watershed on the Loess Plateau. We evaluated the ecosystem response in terms of vegetation structure, plant diversity, and several key ecological processes of soil moisture, soil nutrients, and soil anti-erodibility. Compared with the croplands, we found that the following indexes were significantly enhanced in afforested sites: vegetation structure and species diversity (species richness, Margalef index, Shannon–Wiener index, and Sorensen's similarity index), soil nutrients (organic carbon, total nitrogen, extractable ammonium nitrogen, available potassium, and available phosphorous), and soil anti-erodibility indexes (water-stable soil aggregates, mean weight diameter, and the ratio of soil structure dispersion). Afforestation offered few additional advantages when compared with natural recovery sites. More importantly, afforestation had significant negative effects on soil desiccation, with negative impacts on the long-term sustainability of these ecosystems. In order to develop self-sustaining and functional ecosystems, our results suggest that natural revegetation offers a more adaptive and appropriate method of ecological restoration on the Loess Plateau.