A common method to assess behavioral types in personality research involves the use of a single emergence test (employed by researchers working on fish, avian, mammal, amphibian, and invertebrate taxa), whereby a shorter latency to emerge from a holding container into a novel environment is inferred to represent greater ‘boldness’. Although any behavior might be context specific, studies using this single assay type must assume it reflects boldness in other similar contexts, otherwise it cannot reflect personality (defined as consistency across time and/or contexts). We attempted to validate whether a single assay of this type is correlated with other similar assays of boldness under more familiar, and less stressful, situations. We compared single emergence test scores of two species of damselfish (Pomacentrus wardi; P. amboinensis) in a novel environment, with two different behavioral assays of the same fish in subsequent repeated trials in home tanks. Although behavior was highly repeatable in home tanks, we found no correlation between emergence test scores in the novel environment and measures of latency to emerge from shelter following disturbance, or activity levels, on the first, second, or third observations in home tanks; there was also no correlation when we used average home tank scores from mixed models that accounted for individual differences (i.e., plasticity) in the rate of habituation (latency) and acclimation (activity). Our results therefore lead us to question the validity of using this single emergence test assay as a predictor of general boldness and to question the use of any single assay of behavior in personality research.