This paper defends the claim that, in order to have a concept of time, subjects must have memories of particular events they once witnessed. Some patients with severe amnesia arguably still have a concept of time. Two possible explanations of their grasp of this concept are discussed. They take as their respective starting points abilities preserved in the patients in question: (1) the ability to retain factual information over time despite being unable to recall the past event or situation that information stems from, and (2) the ability to remember at least some past events or situations themselves (typically because retrograde amnesia is not complete). It is argued that a satisfactory explanation of what it is for subjects to have a concept of time must make reference to their having episodic memories such as those mentioned under (2). It is also shown how the question as to whether subjects have such memories, and thus whether they possess a concept of time, enters into our explanation of their actions.