In two experiments, it was investigated how preverbal infants perceive the relationship between a person and an object she is looking at. More specifically, it was examined whether infants interpret an adult’s object-directed gaze as a marker of an intention to act or whether they relate the person and the object via a mechanism of associative learning. Fourteen-month-old infants observed an adult gazing repeatedly at one of two objects. When the adult reached out to grasp this object in the test trials, infants showed no systematic visual anticipations to it (i.e. first visual anticipatory gaze shifts) but only displayed longer looking times for this object than for another before her hand reached the object. However, they showed visual anticipatory gaze shifts to the correct action target when only the grasping action was presented. The second experiment shows that infants also look longer at the object a person has been gazing at when the person is still present, but is not performing any action during the test trials. Looking preferences for the objects were reversed, however, when the person was absent during the test trials. This study provides evidence for the claim that infants around 1 year of age do not employ other people’s object-directed gaze to anticipate future actions, but to establish person–object associations. The implications of this finding for theoretical conceptions of infants’ social-cognitive development are discussed.