Many pathogens use mucosal surfaces to enter and propagate within the host, making particularly desirable vaccines that target immune responses specifically to mucosal compartments. The majority of mucosal vaccine design strategies to date have been empirical in nature. However, an emerging body of basic immunological knowledge is providing new insights into the regulation of tissue-specific lymphocyte trafficking and differentiation. These insights afford the opportunity for the rational design of vaccines that focus immune responses at mucosal surfaces. Mucosal cellular immunity may prove critical for protection in the context of HIV infection, and thus there has been considerable interest in developing vaccines that target HIV-specific cellular immune responses to the gastrointestinal and vaginal mucosa. However, the optimal strategies for eliciting mucosal cellular immune responses through vaccination remain to be determined. Here, we review both recent vaccine studies and emerging paradigms from the basic immunological literature that are relevant to the elicitation of potent and protective mucosal cellular immune memory. Increasing the synergy between these avenues of research may afford new opportunities for mucosal vaccine design.