The lung constantly interacts with the environment through thousands of liters of air that are inhaled daily. This continually transports toxic chemicals and particles or pathogenic microorganisms deep into the respiratory system, posing a challenge to physicochemical barriers and the local immune system. Thus, complex structures and mechanisms have evolved to recognize and fend off environmental dangers while at the same time allowing efficient gas exchange. Here we review our current knowledge regarding cellular mechanisms of the immune system in context with the highly specialized anatomical features of the airways and especially the alveolar compartment. The focus is on fungal and viral infections, merging anatomical aspects well known to pulmonologists with fundamental immunological concepts. We discuss the specialized morphological constraints of immune cells compressed under a continuous layer of the surfactant lining within alveoli as well as the importance of functional polarization of respiratory tract epithelia. Furthermore, we summarize the different types of innate and adaptive immune cells and their relative contribution to lung homeostasis with respect to localization. Finally, we provide a list of currently unresolved questions with high relevance for the field that might serve as food for thought regarding future research directions.