The idea that thinking is a form of talking to oneself was discussed in classical Greece, analyzed by the Medievals and treated as a central issue by the American pragmatists. But whether inner speech is a language unto itself, distinct from outer language, has not been determined. To this end I ask how Saussure's defining ideas about language apply to inner speech. I show that Saussure's ideas, while partly usable, are mainly a poor fit. Inner speech is a variety of language, or perhaps of dialect, with its own unique structure. Given that it is a unique window into human consciousness, I briefly discuss some of the research areas on which it sheds light.