### Abstract

- Top of page
- Abstract
- 1. Introduction
- 2. Model and inequality constrained hypotheses
- 3. Bayes factors and prior probabilities
- 4. Issues with prior specification
- 5 Alternative prior specification methods
- 6. Simulation study
- 7. Concluding remarks
- Acknowledgements
- References
- Appendix A: Derivation of the Bayes factor
- Appendix B: Bayes factor based on
- Appendix C: Generalized inverse
- Appendix D: Information paradox
- Appendix E: Intrinsic Bayes factors

Several issues are discussed when testing inequality constrained hypotheses using a Bayesian approach. First, the complexity (or size) of the inequality constrained parameter spaces can be ignored. This is the case when using the posterior probability that the inequality constraints of a hypothesis hold, Bayes factors based on non-informative improper priors, and partial Bayes factors based on posterior priors. Second, the Bayes factor may not be invariant for linear one-to-one transformations of the data. This can be observed when using balanced priors which are centred on the boundary of the constrained parameter space with a diagonal covariance structure. Third, the information paradox can be observed. When testing inequality constrained hypotheses, the information paradox occurs when the Bayes factor of an inequality constrained hypothesis against its complement converges to a constant as the evidence for the first hypothesis accumulates while keeping the sample size fixed. This paradox occurs when using Zellner's *g* prior as a result of too much prior shrinkage. Therefore, two new methods are proposed that avoid these issues. First, partial Bayes factors are proposed based on transformed minimal training samples. These training samples result in posterior priors that are centred on the boundary of the constrained parameter space with the same covariance structure as in the sample. Second, a *g* prior approach is proposed by letting *g* go to infinity. This is possible because the Jeffreys–Lindley paradox is not an issue when testing inequality constrained hypotheses. A simulation study indicated that the Bayes factor based on this *g* prior approach converges fastest to the true inequality constrained hypothesis.

### 1. Introduction

- Top of page
- Abstract
- 1. Introduction
- 2. Model and inequality constrained hypotheses
- 3. Bayes factors and prior probabilities
- 4. Issues with prior specification
- 5 Alternative prior specification methods
- 6. Simulation study
- 7. Concluding remarks
- Acknowledgements
- References
- Appendix A: Derivation of the Bayes factor
- Appendix B: Bayes factor based on
- Appendix C: Generalized inverse
- Appendix D: Information paradox
- Appendix E: Intrinsic Bayes factors

In past decades there has been considerable development of the evaluation of hypotheses containing inequality constraints. A practical advantage for this development is that applied researchers often formulate their expectations using inequality constraints. In a repeated measures model, for instance, one may expect that the first measurement mean is smaller than the second measurement mean, the second measurement mean is smaller than the third measurement mean, and the third measurement mean is smaller than the fourth measurement mean: . Subsequently, when evaluating an inequality constrained hypothesis, researchers obtain a direct answer to the question to what degree their expectations are supported by the data. Testing inequality constrained hypotheses also has a technical advantage because they result in more statistical power. The simplest example is that the classical one-sided *p*-value of the standard null hypothesis against a one-sided alternative is half the size of the two-sided *p*-value when the one-sided alternative is supported by the data. See Barlow, Bartholomew, Bremner, and Brunk (1972), Robertson, Wright, and Dykstra (1988), and Silvapulle and Sen (2004) for an overview of classical methods for testing inequality constrained hypotheses. For recent developments on this topic using Akaike-type information criteria and Bayesian selection criteria, see Hoijtink, Klugkist, and Boelen (2008) and Hoijtink (2011), and the references therein.

In this paper, a Bayesian approach will be considered. A straightforward method for evaluating inequality constrained hypotheses in a Bayesian framework is by computing the posterior probabilities that the inequality constraints hold. This approach, however, may not result in desirable testing behaviour. For example, let us consider the repeated measures model with four measurements where the interest is in testing (i.e., the means monotonically increase) versus (i.e., the means do not monotonically increase). Now let us consider a hypothetical data set with 10 observations with sample means of , and a maximum likelihood (ML) estimate of the covariance matrix of ; this corresponds to an effect size of , which implies a ‘small’ to ‘medium’ effect (Cohen, 1988). The posterior probabilities that the inequality constraints of and hold are given by and , respectively. These probabilities can simply be estimated as the proportion of posterior draws satisfying the inequality constraints. Hence, receives more posterior evidence than based on these posterior probabilities. Because is supported by the data with a small to medium effect, testing hypotheses based on the posterior probabilities may not be preferable.

In the Bayes factor, prior specification plays a crucial role when testing a null hypothesis with equality constraints on the parameters due to the Jeffreys–Lindley paradox (Lindley, 1957; Bartlett, 1957; Jeffreys, 1961). The paradox implies that the evidence for the null hypothesis can become arbitrarily large when specifying the prior variance large enough. When testing inequality constrained hypotheses however, the Jeffreys–Lindley paradox does not play a role because the prior probability that the inequality constraints hold is not affected by the prior variance when the unconstrained prior is centred on the boundary of the constrained space (Klugkist & Hoijtink, 2007). Therefore, Bayes factors are well suited for testing inequality constrained hypotheses. However, three issues must be taken into account when using the Bayes factor for this purpose.

First, the complexity of the inequality constrained parameter spaces can be ignored. As will be shown in this paper, Bayes factors based on non-informative improper priors and partial Bayes factors based on posterior priors (O'Hagan, 1995; Berger & Pericchi, 1996, 2004) ignore the complexity when testing inequality constrained hypotheses.

Second, Bayes factors may not be invariant for linear one-to-one transformations of the data. This is the case, for instance, when using the constrained posterior priors of Mulder, Hoijtink, and Klugkist (2010), which can be seen as a multivariate generalization of Jeffreys' (1961) notion of balanced priors. Mulder *et al*. proposed an automatic prior specification method that resulted in priors that were located on the boundary of the constrained parameter space with diagonal covariance structure. One of the important implications of the proposed methodology was that every ordering of the parameters was equally likely a priori.

Third, the information paradox can be observed. This paradox has been investigated when testing the standard null hypothesis (an equality constrained hypothesis); see Zellner (1986), Berger and Pericchi (2001), and Liang, Paulo, Molina, Clyde, and Berger (2008). The paradox states that the Bayes factor for an unconstrained hypothesis against the null hypothesis does not converge to infinity as the evidence against the null hypothesis accumulates, for fixed sample size. In this paper, we present an equivalent formulation of the paradox when testing inequality constrained hypotheses. Similarly to when testing equality constrained hypotheses, the information paradox can also be observed when testing inequality constrained hypotheses when using Zellner's (1986) *g* prior.

For this reason, two methods are developed that avoid the three issues mentioned above. In the first method, minimal training samples are transformed such that the resulting posterior priors are located on the boundary of the constrained space with identical covariance structure to that of the original posterior prior. The second approach is based on the *g* prior, where *g* goes to infinity. This is allowed because the Jeffreys–Lindley paradox is not an issue when testing inequality constrained hypotheses. The resulting Bayes factor is relatively simple to compute and avoids the information paradox.

The paper is organized as follows. First, the multivariate normal linear model and the inequality constrained hypotheses are formulated in Section 2. Section 3 describes the Bayes factor when testing inequality constrained hypotheses, including a short discussion of the choice of prior probabilities for the hypotheses. In Section 4 the three issues are discussed in detail in relation to posterior priors, balanced priors, and the *g* prior. Section 5 describes two new Bayesian methods for testing inequality constrained hypotheses. A simulation study is discussed in Section 6. Some concluding remarks are made in Section 7.

### 2. Model and inequality constrained hypotheses

- Top of page
- Abstract
- 1. Introduction
- 2. Model and inequality constrained hypotheses
- 3. Bayes factors and prior probabilities
- 4. Issues with prior specification
- 5 Alternative prior specification methods
- 6. Simulation study
- 7. Concluding remarks
- Acknowledgements
- References
- Appendix A: Derivation of the Bayes factor
- Appendix B: Bayes factor based on
- Appendix C: Generalized inverse
- Appendix D: Information paradox
- Appendix E: Intrinsic Bayes factors

Models will be considered that belong to the multivariate normal linear model, where a *p*-dimensional dependent variable depends on a *k* × *p* mean parameter matrix **Θ** via *k* explanatory variables and error covariance matrix **Σ** according to

- (1)

for *i* = 1,…,*n*, so that **Y** = **XΘ** + **E**, where , , and , with . Under this model, the non-informative improper Jeffrey's prior is given by , resulting in the posterior

- (2)

where *N*(*θ*;**m**,**K**) denotes the multivariate normal density with mean **m** and covariance matrix **K**, denotes the inverse Wishart density with scale matrix **C** and degrees of freedom *ν*,** ***θ* is the vectorization of **Θ**, that is, *θ* = vec(**Θ**), the ML estimate is given by , and the sum of squares equals . Furthermore, the marginal posterior of **Θ** has a matrix *t* distribution given by .

Throughout this paper, testing problems are considered between a finite number of *T* inequality constrained hypotheses on the mean parameters, where or with . Furthermore, it is assumed that a solution exists to the system of equations , where and . This implies that the inequality constrained parameter spaces share a common boundary. Furthermore, we shall refer to the hypothesis with no constraints on *θ* as the unconstrained hypotheses, denoted by . Note here that the likelihood function under , denoted by , is a truncation of the likelihood function under , that is, , where is the identity function. Therefore, the hypothesis index will be omitted in the likelihood *h* and parameters *θ* and **Σ** when possible in order to simplify the notation.

### 3. Bayes factors and prior probabilities

- Top of page
- Abstract
- 1. Introduction
- 2. Model and inequality constrained hypotheses
- 3. Bayes factors and prior probabilities
- 4. Issues with prior specification
- 5 Alternative prior specification methods
- 6. Simulation study
- 7. Concluding remarks
- Acknowledgements
- References
- Appendix A: Derivation of the Bayes factor
- Appendix B: Bayes factor based on
- Appendix C: Generalized inverse
- Appendix D: Information paradox
- Appendix E: Intrinsic Bayes factors

A natural choice for the prior under an inequality constrained hypothesis is to take the truncation of the unconstrained prior under (Berger & Mortera, 1999; Klugkist, Laudy, & Hoijtink, 2005), that is, where the prior probability is a normalization constant. Consequently, the Bayes factor of against can be expressed as

- (3)

(see Appendix A). Hence, the Bayes factor is equal to the ratio of the posterior probability that the inequality constraints hold under , which serves as a measure of ‘relative fit’ of , and the prior probability that the inequality constraints hold under , which serves as a measure of ‘relative complexity’ or size of (both relative to ).

It is important to note that the prior probabilities that the inequality constraints hold under , , are not equal (or proportional) to the prior hypothesis probabilities . The must be chosen separately. We recommend using equal prior probabilities for , for *t* = 1,…,*T*, that is, , as a default choice (unless there is specific prior knowledge). If we use the prior probabilities that the inequality constraints hold as prior probabilities for the hypotheses, that is, , the posterior probabilities of the hypotheses become equal to (or proportional to, in the case of more than two hypotheses) the posterior probabilities that the inequality constraints hold. For example, when testing versus using the unconstrained prior , so that and , the posterior hypothesis probabilities become

Hence, the relative complexity measures of the inequality constrained hypotheses, , are cancelled out so that the resulting posterior hypothesis probabilities become equal to the posterior probabilities that the inequality constraints hold, . This would result in a bias towards the ‘larger’ hypothesis , for *p* > 2. In the remainder of this paper, posterior hypothesis probabilities and prior hypothesis probabilities will not be further discussed. Therefore, when discussing the posterior (or prior) probability that the inequality constraints of hold under , we refer to (or ).

### 6. Simulation study

- Top of page
- Abstract
- 1. Introduction
- 2. Model and inequality constrained hypotheses
- 3. Bayes factors and prior probabilities
- 4. Issues with prior specification
- 5 Alternative prior specification methods
- 6. Simulation study
- 7. Concluding remarks
- Acknowledgements
- References
- Appendix A: Derivation of the Bayes factor
- Appendix B: Bayes factor based on
- Appendix C: Generalized inverse
- Appendix D: Information paradox
- Appendix E: Intrinsic Bayes factors

A simulation study was carried out to investigate the differences between the Bayes factors that were discussed in this paper. We consider the testing problem of versus for i.i.d. data that come from , for *i* = 1,…,*n*, for four different populations: and ; and ; and ; and and . A balanced Bayes factor was obtained from a relatively non-informative conjugate balanced prior, where , in which , for *v* = 1 and 2, and . The results are displayed in Figure 1. Each line represents the expected weight of evidence for against (i.e., the natural logarithm of the Bayes factor) of a different Bayes factor, based on 100 simulated data sets from each of the four populations when increasing *n* from 4 to 25. Based on these results we conclude the following. First, the Bayes factor , based on the *g*-prior with *g* ∞, converges fastest to the true hypothesis, followed by the median IBF based on the transformed minimal training samples, , and the balanced Bayes factor . Second, when the population is located on the boundary, such as , the Bayes factors , , and prefer over because the relative complexity of is smaller than , while the relative fit is equal. Third, the Bayes factor based on the non-informative improper prior, , that is, the ratio of posterior probabilities that the inequality constraints hold, shows a preference for the more complex .