This article focuses on the contribution to vocabulary learning of the use of bilingual dictionaries during reading by 293 Japanese university students studying English as a foreign language. The results of the study show that students who used a dictionary scored significantly better on a vocabulary test than students who did not use a dictionary. However, evidence appeared for differential item functioning: Some items were harder for the group that used dictionaries. A possible explanation for this tendency is that students who were unable to locate the appropriate gloss in the dictionary were misled as to the meaning of the word. Moreover, students who used a dictionary read nearly half as quickly as the group that did not use dictionaries.