In the early part of the paper, I attempt to explain a dispute between two parties who endorse the compositionality of language but disagree about its implications: Paul Horwich, and Jerry Fodor and Ernest Lepore. In the remainder of the paper, I challenge the thesis on which they are agreed, that compositionality can be taken for granted. I suggest that it is not clear what compositionality involves nor whether it obtains. I consider some kinds of apparent counterexamples, and compositionalist responses to them in terms of covert indexicality and unspecific meanings. I argue that the last option is the best for most of the cases I consider. I conclude by stressing, as against Horwich and Fodor and Lepore, that the appropriate question concerns the extent to which compositionality obtains in a natural language, rather than whether it obtains or not, so that the answer is essentially messy, requiring detailed consideration of a wide range of examples.