The purpose of this study is to show how conceptual analysis may be used to investigate the validity of a research instrument. The instrument chosen here, the Scientific Attitude Inventory (SAI), is by far the most popular of its type, yet the studies in which it has been used give reason to question its validity. Using the conceptual perspectives developed in the study, it is possible to show that many of the items which might be thought to tap attitudes can be interpreted quite differently. In this way, the discrepant results obtained by using the SAI in quantitative research can be explained. The report concludes that we can be less than certain of what is measured by the SAI, and that it needs reworking before it can be used with confidence. The report ends with some general concerns about attitude measurement and its place in science education research.