General solutions to consistency problems in multiple hypothesis testing



Coherence and consonance are the two important concepts to ensure consistent conclusions drawn from multiple tests. Unlike coherence, consonance of a multiple testing procedure (MTP) has not yet received much attention. Although most of the MTPs used in practice are consonant, cases still exist that dissonant tests have to be used due to their unique advantages, for example, the likelihood ratio tests and sum tests. Consonance adjustments are necessary for such tests to avoid difficulty in interpretation. Moreover, in terms of detecting elementary hypotheses, a consonant test procedure is uniformly more powerful than the corresponding dissonant one. In this paper, several general methods using the partitioning principle to construct (strongly) consonant and strongly coherent MTPs are proposed. As a result, any dissonant (but coherent) multiple level α test can be improved by a consonant one without any cost in terms of inferences on elementary hypotheses.