Accurate identification of humpback whales from photographic identification data depends on the quality of the photographs and the distinctiveness of the flukes. Criteria for evaluating photographic quality and individual distinctiveness were developed involving judgments about overall quality or distinctiveness and about specific aspects of each. These criteria were tested for the level of agreement among judges. The distinctiveness scheme was tested for the independence of distinctiveness judgments and photographic quality. Our results show that judges could agree when evaluating specific and overall aspects of photographic quality and individual distinctiveness. The level of agreement varied for different pairs of judges, and less adept judges were identified. Ability to agree on evaluations of photographic quality was independent of the experience of the judges. Overall photographic quality and overall distinctiveness were successfully predicted from more specific variables, but the agreement between judges for these was not significantly greater than the agreement for the overall measures judged directly. There was no correlation between individual distinctiveness and photographic quality for four of the five judges, but the power of this rest may be low. Analyses of photographic identification data frequently require evaluations of photographic quality and individual distinctiveness. To obtain reliable results from such analyses, evaluation schemes and judges should be tested to ensure reliable and consistent evaluations.