Correspondence site: http://www.respond2articles.com/MEE/

# Category count models for resource management

Article first published online: 7 JUN 2012

DOI: 10.1111/j.2041-210X.2012.00191.x

© 2012 The Author. Methods in Ecology and Evolution © 2012 British Ecological Society

Additional Information

#### How to Cite

Fackler, P. L. (2012), Category count models for resource management. Methods in Ecology and Evolution, 3: 555–563. doi: 10.1111/j.2041-210X.2012.00191.x

#### Publication History

- Issue published online: 7 JUN 2012
- Article first published online: 7 JUN 2012
- Received 3 October 2011; accepted 16 January 2012 Handling Editor: Robert Freckleton

- Abstract
- Article
- References
- Cited By

### Keywords:

- dynamic programming;
- implicit spatial models;
- Markov models

### Summary

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

**1. **A category count model consists of a fixed set of objects (often sites) each of which is classified as one of a set of mutually exclusive categories. Additionally, the category membership of each object evolves over time as a Markov process. The evolution of the objects can be affected by choosing the number of objects in each category that receive alternative management actions.

**2. **Category count models have been used for a variety of resource management applications, including conservation of endangered species, land management and the management of pest infestations.

**3. **This paper provides for the first time a general framework for such models, briefly reviews existing applications that fit the general framework, discusses the non-trivial problem of how transition probabilities can be computed as well as some of the challenges facing analysts using this framework.

**4. ** The framework is applied to an existing application (the management of habitat for a threatened species), demonstrating the importance of modelling the stochasticity inherent in the problem.

### Introduction

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Spatially explicit models have been used to address a variety of resource use and management issues (Getz & Haight 1989; Hansi 1999). Although this kind of model has a considerable degree of flexibility, it also suffers from a severe ‘curse of dimensionality’ (Bellman 1957) that limits the number of sites that can be analysed. One response to this is to search for near-optimal heuristics. Another is to simplify the model by, for example, using spatial aggregation to reduce the number of sites. An alternative approach treats each site as distinguished only in terms of a relatively small set of categories and only the number of sites in each category is required. This approach loses information about the explicit relationships among the sites, and thus, these models are often referred to as spatially implicit. The approach, however, provides a way to simplify the data requirements for specifying a model and makes the determination of optimal strategies more feasible by reducing the problem size.^{1}

Numerous examples of such models exist in the literature on resource and ecological management. Categories might describe population characteristics such as present/not present or low/moderate/high population levels. The categories might also be used to describe habitat quality or successional stage. Categories might also be defined as mixtures of habitat and population characteristics such as the successional stage and occupancy status or habitat quality and nesting success.

Possingham (1996), Nicol & Possingham (2010) and Johnson *et al.* (2011) use the approach to address the management of endangered species through translocation and/or habitat enhancement. Shea & Possingham (2000) and Bogich & Shea (2008) apply the framework to the control of invasive species. Richards *et al.* (1999) use the framework to develop a strategy for managing a reserve to enhance biodiversity. Martin *et al.* (2011) uses the approach to develop a strategy for balancing recreational access to natural sites with potential disturbance to nesting birds.

In general, the category count approach assumes that each site evolves as a Markov chain process. This requires the specification of the probability that an individual site changes to a particular category in the next period given its current category and any treatment that is applied to it. This probability cannot depend on details of the characteristics of individual sites, such as the location of occupied sites, but can depend on the total number of sites in each category and treatment as well as on other information that applies to all of the sites.

The goal of this paper is threefold. First, it provides a general framework for a number of problems addressed in the literature. Second, it discusses how transition probabilities can be computed for the general case, thereby allowing more complicated models to be analysed than has been previously possible (Matlab code implementing the procedure is available as the catcountP function of the MDPSolve package, which can be accessed at https://sites.google.com/site/mdpsolve/). Third, it describes how these models are used to specify and solve optimization problems. The framework is then applied to the management of habitat for the Florida scrub jay (*Aphelocoma coerulescens*), a species listed as threatened under the Endangered Species Act. The example demonstrates that the explicit use of transition probabilities is important for obtaining the correct management strategy.

### Category count models

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Category count models apply to collections of a fixed number (*N*) of individuals or elements. In most of the applications familiar to the author the elements are spatial sites, but they could be members of an animal population or have other interpretations altogether. However, for expositional convenience and to make the discussion less abstract, the elements will be referred to as sites. Each site can be classified into one of *n* possible categories and a manager can apply alternative treatments to each site, there being a total of *m* possible category/treatment combinations. The category to which each site belongs evolves over time as a Markov chain. The transition probabilities for any specified category and treatment could be fixed constants depending only on a site's own current category and treatment or could, additionally, depend on the total number of sites in each category receiving each treatment. The transition probabilities could also depend on other environmental state variables that are common to all sites.

To establish a notation for the problem let lower case variables apply to individual sites and upper case to aggregates of sites. Thus, *s*_{h} and denote the current and next period category for site *h*, and *x*_{h} represents its category/treatment combination. *S* and *S*^{+}, on the other hand, are both *n* vectors representing the present and next period number of sites in each category, and *X* is an *m*-vector with *X*_{i} representing the number of sites in category/treatment *i*. Adding up restrictions imply that ∑_{j}*S*_{j} = ∑_{i}*X*_{i} = *N*. Furthermore, *X* must be consistent with *S*, so the total number of sites in the category/treatment combinations for a given category in *X* must sum to the number of sites in the category (i.e., in *S*). Throughout, *s* will be referred to as the category, and *S* will be referred to as the state.

The number of values that the state variable *S* (which recall is an *n* vector) can take on is the same as the number of ways to place *N* objects into *n* categories, which is well known to be

- (eqn 1)

(this is sometimes known as the multiset coefficient). While this could be a large number, it is far fewer than the *n*^{N} possible values of the state that arise in a fully explicit spatial model (where the state is an *N*-vector in which each element can take on values in the set {1,…,*n*}). Notice that if there are only two categories (*n* = 2), we get *N*+1 possible combinations. This only increases linearly with the number of sites, whereas a fully explicit spatial model would have 2^{N} values of the state (i.e., exponential increase in *N*).

For any individual site, let *p*_{ij}(*X*,*W*) be the probability that the site will be in category *j* next period, given that it is currently in category/treatment *i*. This probability is common across all sites but can depend on *X* as well as on *W*, which represents the values of additional state or random noise variables that are common across all sites. For example, *W* could represent the level of soil moisture, which could affect the probability of advancement to a higher successional stage. If this is observed prior to the decision, it should be treated as an additional state whereas if it is observed after the decision it should be treated as a noise variable.

The manager's problem is to choose *X* as a function of *S* and *W* given some reward function *R*(*X*,*W*) and a rate at which future rewards are discounted. Specifically, the manager seeks to determine the decision rule *X*(*S*,*W*) that solves

- (eqn 2)

subject to the dynamic behaviour of *S* and *W* and to the constraint that *X* is consistent with *S*. Here *δ* is a discount factor with 0 < *δ* ≤ 1. Expressed in this way, the problem is a standard stochastic dynamic programming (SDP) problem (Williams *et al.* 2001).

One of the main difficulties in implementing this approach is determining the transition probabilities for the whole collection of sites. An example will illustrate how this can be done in a relatively simple case with no actions and no *W*. Suppose that a site is categorized as (i) not occupied or (ii) occupied, with a constant site transition matrix

- (eqn 3)

Suppose that the state is defined as the number of occupied sites. When there are two sites (*N* = 2), the state transition matrix is given in Table 1.

Future state | Current state | ||
---|---|---|---|

0 | 1 | 2 | |

0 | (1−c)^{2} | (1−c)e | e ^{2} |

1 | 2(1−c)c | (1−c)(1−e)+ce | 2e(1−e) |

2 | c ^{2} | c(1−e) | (1−e)^{2} |

Notice, in particular, that there are two ways that the system can move from one occupied site to one occupied site. First, both sites can remain in the current occupancy categories, with probability (1−*c*)(1−*e*), or both sites can change occupancy status, with probability *ce*. The total probability is the sum of the probabilities of these two events.

When there are more than two sites, the probability of the state changing from *S* to *S*^{+} becomes more complicated. One way to express the correct probability is to compute the probability of having exactly *K* of the occupied sites remain occupied. This implies that *S*−*K* change from occupied to unoccupied, that *S*^{+}−*K* change from unoccupied to occupied and that (*N*−*S*)−(*S*^{+}−*K*) remain unoccupied. The probability of the transition from *S* to *S*^{+} is the sum of the probabilities over possible values of *K*, that is, values that satisfy *K* ∈ {0,…,min(*S*,*S*^{+})}:

- (eqn 4)

As the number of categories increases, the expressions for the transition probabilities become increasingly complicated unless there are zeros values in the individual site transition matrix. Such simplifications have often been used in existing studies to obtain closed form expressions like the one in eqn 4. A computational method for computing the probabilities without the need to obtain such expressions is given latter. First, however, a number of examples from the literature will illustrate the approach.

### Examples from the literature

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

This section briefly describes a number of examples of category count models that exist in the resource management literature. To highlight the unique contribution of the current paper, it should be noted that the papers discussed here all make simplifying assumptions about possible site transitions to obtain closed form expressions for transition probabilities. In particular, a number of these papers break the transitions into separate time stages to obtain closed form expressions with each stage having a relatively simple transition matrix (such as allowing each category to move to a restricted set of possible categories in the next period). In addition, the need for relatively simple closed form expressions for transition probabilities might limit the sets of possible actions that are allowed. The results presented in the next section obviate the need for such simplifications in model specifications.

The examples discussed here are only ones in which a category count-type model is embedded in a dynamic decision problem. There are, however, other studies that use the category count framework. Stochastic patch-occupancy models (SPOMs) are a widely studied example of the kind of model discussed here. In an SPOM, *N* sites are either occupied or not, so *n* = 2. In heterogeneous SPOMs, the sites are differentiated in aspects other than their occupancy; these are spatially explicit resulting in a full set of 2^{N} states. In a homogeneous SPOM, however, the sites are all distinguished only in terms of their occupancy status. In this case, only the number of occupied sites is relevant for decision-making, with *N*+1 possible states. For example, Hill & Caswell (2001) develop a two category site occupancy model.

There are numerous examples of problems that fit the framework of category count models but in which the transition is treated as deterministic (Johnson *et al.*, 2011; Martin *et al.*, 2011). Generally, such models specify a site transition matrix *p* but use it as a deterministic projection matrix in which . The approach taken here, in contrast, has . It is sometimes the case that the decision rule is not much affected using a deterministic model, but the Scrub Jay example discussed later demonstrates that this is not always the case.

Possingham (1996) developed a model of site occupancy in which each site is either unsuitable, empty of or occupied by a species of interest. Control actions consisted of doing nothing, reintroducing the species to a site from another site or altering the site to make it suitable for habitation by the species. Actions are restricted such that only one site treatment is possible in each period. The goal is to maximize the probability that at least one site is still occupied at some terminal date.

Nicol & Possingham (2010) have a more elaborate model in which each site can either be occupied or not, and the size of the sites and the number of suitable sites can be increased. Here the state variable is the triple (number of suitable sites, number of occupied sites and size of the sites). Controls are do nothing, increase the number of suitable sites and increase the area of the existing sites. Only one site or one unit of site size can be added per period. As in the previous study, the goal is to maximize survival probability at some terminal date.

Richards *et al.* (1999) develop a model in which each site can be in one of three successional stages, with the third stage being a climax stage. In addition, sites can return to stage 1 because of fire. Control actions consist of doing nothing, altering fire probabilities and engaging in various numbers of controlled burns. The objective is to maintain the landscape with a diverse set of successional stages.

Shea & Possingham (2000) discussed the optimal strategy for the establishment of biological controls over a number of sites. The sites are classified as to whether the site is empty of the control species or the species is present on a site as either an insecure or secure population. Actions consist of choosing the number of sites that will receive either a small or large release of the control species. The goal is to maximize a weighted sum of the number of occupied sites at a specified terminal date, with secure sites receiving more weight than insecure sites.

Bogich & Shea (2008) modelled a pest infestation problem in which each site is categorized according to whether there is no, moderate or severe infestation on the site. Actions consisted of doing nothing, decreasing the probability of colonization or directly reducing the number or severity of the infested sites. The goal is to minimize a weighted sum of the number of infested sites at some terminal date [errors in the original paper are discussed in Bogich & Shea (2011)].

### Transition probabilities

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Category count models are defined in terms of the category transition probabilities for individual sites. To solve the decision problem, however, the state transition probabilities must be determined. Given the independence of *W*, the state transition probabilities satisfy

- (eqn 5)

The second term on the right-hand side is assumed to be given, so the problem of determining the state transition probabilities involves determining *Prob*(*S*^{+}|*X*,*W*).

Although computing *Prob*(*S*^{+}|*X*,*W*) for simple models is straightforward, computing it for the general case is not a trivial problem and does not appear to have been done before. The difficulties arise because there are alternative ways in which a future state can occur given the current starting state, as was previously illustrated. One way to compute the desired probabilities is to enumerate the ways that a given transition can occur and sum the probabilities associated with them. Let the *m* × *n* matrix *E*_{ij} represent the numbers of sites changing to category *j* given that it is currently in the category/treatment pair *i*. Given *S*^{+} and *X*, *E* must satisfy the constraints that ∑_{j}*E*_{ij}=*X*_{i} and that . Any matrix *E* with the correct row and column sums is termed valid. The probability that any specific valid *E* is realized is

- (eqn 6)

The transition probability is the sum of the probabilities associated with all of the valid *E* matrices.

The enumeration of the valid *E* matrices is the same problem as that of finding all possible contingency tables with given row and column sums. This problem has been extensively studied and is reviewed in Verbeek & Kroonenberg (1985) (see also Greselin 2003). It is known that the number of tables increases exponentially in the number of rows and columns. In the current application, however, the presence of zero values in the *p* matrix allows us to eliminate from consideration some of the valid *E* matrices. Specifically, any matrix for which *E*_{ij} ≠ 0 when *p*_{ij} = 0 is infeasible and need not be considered. Although it is acceptable to consider all valid *E*, eliminating the infeasible ones leads to considerable streamlining in the algorithm used to compute the transition probabilities. As this can be more time-consuming than solving the dynamic programming algorithm itself, such streamlining can be quite welcome. Unfortunately even with the streamlining possible with 0 entries, the enumeration approach is unacceptably slow even for moderate applications.

Given that *p*_{ij} is the probability that a site transitions to category *j* from category/treatment pair *i* and that *X*_{i} is the number of sites in category/treatment pair *i*, the number of sites in each category starting in category/treatment *i* is distributed as a multinomial variates with *X*_{i} trials and probability vector *p*_{i·} (the *i*th row of *p*). As all of the site transitions are independent, this implies that the number of sites next period in each category is the sum of *m* independent multinomial variates.

The fact that the probability we seek can be written as the sum of multinomials provides an alternative algorithm for obtaining the exact probabilities.^{2} Let *Y*_{i}∼*MN*(*X*_{i},*p*_{i·}) be an *n*-vector representing the number of sites in each category that start in category/treatment *i*. Also define *Z*_{1} = *Y*_{1} and, for *i* > 1, *Z*_{i} = *Z*_{i−1}+*Y*_{i}. Noted that *Y*_{i} sums to *X*_{i} and so *Z*_{i} sums to .

The domain of *Z*_{i} is the set of *n*-vectors with non-negative integers that sum to *B*_{i}. The probability of the sum of a pair of independent random variables is the convolution of the probabilities; thus, for *i*>1,

- (eqn 7)

As *Y*_{i} is a multinomial variate, the probabilities associated with it are easily computed, and hence, we can proceed recursively over the *m* values of *i* until we obtain *Prob*(*Z*_{m}) that is the distribution we seek.

Noting that the random variable of interest is the sum of independent multinomials makes it very straightforward to compute the mean and covariance of *S*^{+}. Using standard results for the multinomial distribution, the expected number of sites in category *j* next period is

- (eqn 8)

its variance is

- (eqn 9)

and the covariance between and is

- (eqn 10)

Noted that this results in a covariance matrix of rank *n*−1 because *S*^{+} is constrained to sum to *N*.

It is well known that the multinomial distribution converges to the normal distribution as the number of trials increases. Therefore, the sum of independent multinomial variates will converge to the sum of independent normals. The analogue of the number of trials is the *X*_{i}. Unfortunately, this is not necessarily large just because number of sites, *N*, is large. Indeed, the values of *X*_{i} range from 0 to *N* (and must sum to *N*), so a normal approximation will always suffer from small sample problems for some feasible values of *X*. Whether this is an important consideration depends on how likely it is that small numbers of sites in some category are actually observed. If there are a large number of sites and there tend to be numerous sites in each category, the normal approximation will provide a useful alternative to the exact distribution. Given that the computational complexity of computing the exact distribution increases rapidly with the number of sites, using the normal approximation may be the best alternative.

### Issues in specifying and solving category count problems

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Category count models entail a specific structure that can cause difficulties for some standard dynamic programming algorithms (for example the CompEcon software described in Miranda & Fackler (2002) or the ASDP program described in Lubow (1995)). First, the model involves a set of state variables that are defined over a simplex rather than over a rectangular domain. Having software that is able to handle this easily can be a considerable help in specifying and interpreting models. With software that specifically uses rectangular grids to specify multidimensional state processes (such as ASDP) one approach that has been used in practice is to define infeasible states (states with more sites than exist) and to restrict either the actions that can occur in these states and/or the transition rule for these states. In the author's experience, such ‘fixes’ lead to unpredictable and potentially incorrect results.

More problematic is the case when the control also has this feature. An implication is that there are generally different numbers of possible actions for each value of the state variable. If software requires that there are equal numbers of values of the actions for each state, one would be forced to introduce infeasible actions.

To see why, consider a example in which there are two categories, two actions and two sites. To make the discussion easier to follow, the categories will be referred to as *A* or *B* and the treatments as *C* or *D*. The state can be defined as the number of sites in either state, which takes on values of 0, 1 or 2. Suppose that it is defined to be the number in category *A*. If we define the action as the number of sites receiving treatment *C*, it can also take on values of 0, 1 or 2. If we consider all the combinations (of which there are 9), we get

#C=A | #T=C |
---|---|

0 | 0 |

1 | 0 |

2 | 0 |

0 | 1 |

1 | 1 |

2 | 1 |

0 | 2 |

1 | 2 |

2 | 2 |

This listing of the possible choices loses valuable information however. Generally, we want to know not only how many sites receive a given treatment but also to which sites the treatment should be applied. Consider, for example, a pest infestation problem in which sites are either infested or not and we can do nothing or apply a treatment that helps to control the infestation. Clearly it makes a difference which sites we apply the treatment to; it is reasonable to suppose that the infested sites should be treated preferentially (although it might be useful to do preventative treatment if the treatment costs are low enough relative to damages of infestation).

Suppose that we therefore define the action to be the number of sites in each category/treatment combination (of which there are four in our example). We need to enumerate the values only for three of these as the fourth must make the sum of them all equal *N*. There are *N*+1 = 3 values each variable can take on, *m*−1 = 3 unique category/treatment combinations: #*C* = *A*,*T* = *C*, #*C* = *B*,*T* = *C* and #*C* = *A*,*T* = *D*) and there are (*N*+1)^{m−1} = 27 values for the state/action combinations. These are listed here:

#C = A,T = C | #C = B,T = C | #C = A,T = D | #C = B,T = D |
---|---|---|---|

0 | 0 | 0 | 2 |

0 | 0 | 1 | 1 |

0 | 0 | 2 | 0 |

0 | 1 | 0 | 1 |

0 | 1 | 1 | 0 |

0 | 1 | 2 | −1 |

0 | 2 | 0 | 0 |

0 | 2 | 1 | −1 |

0 | 2 | 2 | −2 |

1 | 0 | 0 | 1 |

1 | 0 | 1 | 0 |

1 | 0 | 2 | −1 |

1 | 1 | 0 | 0 |

1 | 1 | 1 | −1 |

1 | 1 | 2 | −2 |

1 | 2 | 0 | −1 |

1 | 2 | 1 | −2 |

1 | 2 | 2 | −3 |

2 | 0 | 0 | 0 |

2 | 0 | 1 | −1 |

2 | 0 | 2 | −2 |

2 | 1 | 0 | −1 |

2 | 1 | 1 | −2 |

2 | 1 | 2 | −3 |

2 | 2 | 0 | −2 |

2 | 2 | 1 | −3 |

2 | 2 | 2 | −4 |

The last column gives the values of #*C* = *B*,*T* = *D* that make the values of the first three columns sum to *N*. Notice, however, that many values in this column are negative, that is, they are infeasible (because the values in the first three columns sum to more than *N*). If we eliminate the infeasible combinations, we arrive at feasible combinations:

#C = A,T = C | #C = B,T = C | #C = A,T = D | #C = B,T = D |
---|---|---|---|

0 | 0 | 0 | 2 |

0 | 0 | 1 | 1 |

0 | 0 | 2 | 0 |

0 | 1 | 0 | 1 |

0 | 1 | 1 | 0 |

0 | 2 | 0 | 0 |

1 | 0 | 0 | 1 |

1 | 0 | 1 | 0 |

1 | 1 | 0 | 0 |

2 | 0 | 0 | 0 |

Clearly, it is computationally wasteful to consider all of the infeasible alternatives. In a small example like this, the difference would not be noticeable, but in a large problem, it can lead to excessive processing time; some models might, in fact, be unable to be solved because of memory limitations.

Although the size of category count problems is far smaller than for a comparable fully explicit spatial model, the number of states and state/action combinations can nonetheless grow quickly. Table 2 gives these numbers for alternate values of *N* when *n* = 3 and there are two possible actions for each category, resulting in *m* = 6.

N | No. of states | No. of states/actions | Megabytes |
---|---|---|---|

5 | 21 | 252 | 0·01 |

10 | 66 | 3003 | 0·19 |

15 | 136 | 15,504 | 2·01 |

20 | 231 | 53,130 | 11·70 |

25 | 351 | 142,506 | 47·70 |

30 | 496 | 324,632 | 153·56 |

35 | 666 | 658,008 | 417·93 |

40 | 861 | 1,221,759 | 1003·20 |

45 | 1081 | 2,118,760 | 2184·28 |

50 | 1326 | 3,478,761 | 4399·14 |

When a problem gets too big, it is possible that memory limitations will preclude storing the whole transition probability (*P*) matrix in memory. The last column in Table 2 lists the number of megabytes needed to store the *P* matrix (using 8 byte, double precision numbers). It is clear that even for this relatively small problem, memory could quickly become a bottleneck in working with this model.

#### Defining Treatments to Reduce the Number of Actions

There are several options if memory is a limitation (other than getting a bigger machine). First, it is possible to compute the necessary probabilities as needed, but this means recomputing them many times (at each iteration of the dynamic programming algorithm). A second alternative is to set very small probabilities to zero and store the probabilities using sparse matrix methods.

The third alternative is to seek ways to reduce the number of actions. To date, the literature has relied on this method to limit problem size. Specifically, two strategies have been used. The first approach is to require that some treatment be applied to all sites. The second approach that has appeared in the literature is to require that a treatment be defined as a deterministic change in a site's category. For example, if sites are categorized by their dominant vegetative cover, replanting or burning changes the site's category. This is an advantage because the action simply changes the number of sites in each category prior to the stochastic transition. Thus, the same probability distribution applies to any state/action combination with the same post-treatment state.

In many cases, however, limiting how treatments are defined to these approaches is too restrictive. Two other possible approaches to limiting the number of actions arise because of budget considerations or by imposing a hierarchy on the treatments. Budget considerations may limit the number of sites to which a given treatment can be applied. The set of potential actions is then limited to those for which the treatment costs for a given value of *X* do not exceed the allowable budget.

One might also define a treatment hierarchy to limit the possible number of actions. For example, suppose that a specified treatment must be applied first to category *A* sites. To illustrate using the example given earlier suppose that treatment *D* is applied preferentially to category *A*, so #*C* = *B*,*T* = *D* can be positive only if #*C* = *A*,*T* = *C* is zero. This would eliminate *X* = [1 0 0 1] because the single application of treatment *D* is applied to a category *B* site but could have been applied to a category *A* site. With larger values of *n* and *m*, this adjustment can make a large difference in the problem size as is illustrated in Table 3 for *n* = 3 and *m* = 6, with the hierarchical adjustment applied to the second treatment.

N | No. of states/actions | |
---|---|---|

No adjustments | Hierarchical adjustment | |

5 | 252 | 126 |

10 | 3003 | 726 |

15 | 15,504 | 2176 |

20 | 53,130 | 4851 |

25 | 142,506 | 9126 |

30 | 324,632 | 15,376 |

35 | 658,008 | 23,976 |

40 | 1,221,759 | 35,301 |

45 | 2,118,760 | 49,726 |

50 | 3,478,761 | 67,626 |

#### Staged Transitions

Existing models utilizing this framework generally define the state transition in stages. This adds flexibility to the modelling approach and also may simplify the computation of the transition probability matrix. Suppose, for example, that one category is a succession stage in which a site either stays in its current type or transitions to the next type. The individual *p* matrix would have the form

- (eqn 11)

In the second stage, a site might transition to the previous type or [as in the model of Richards *et al.* (1999)] the transition might be to the first type (resetting). With the former assumption, the *p* matrix would have the form

- (eqn 12)

and with the latter assumption the *p* matrix would have the form

- (eqn 13)

In all of these cases, there is a unique set of events (a unique *E* matrix) for each (*S*,*S*^{+}) pair, and the state transition matrices (denoted *C*, *D* and *F*) can be obtained directly from the associated *p* matrices. The full transition matrix is the product of the stage transition matrices, either *P* = *DC* or *P* = *FC*.

The use of such a staged approach requires that the parameter values are specified for a staged model and that they be estimated correctly to reflect this. In particular, such staged transitions allow for the possibility of so-called hidden turnovers that arise, for example, when a site that goes extinct is recolonized in the same period. Hidden turnovers and the associated estimation issues are discussed in Clark & Rosenzweig (1994).

As a parenthetical note, in site occupancy models, it is often useful to model the individual site probabilities to depend on the current state (*p*(*S*)). For example, the probability that an empty site becomes colonized can depend on the current number of occupied sites:

- (eqn 14)

where *S*_{1} is the number of occupied sites. Buckley & Pollett (2010) describe several alternative ways to specify colonization probabilities for this type of model.

### Habitat management for an endangered species

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Merritt Island National Wildlife Refuge (MINWR) is one of the few remaining areas supporting populations of Florida scrub jays (*A. coerulescens*), which are listed as threatened under the Endangered Species Act. Johnson *et al.* (2011) describe an approach to managing this reserve that attempts to maximize the growth rate of the scrub jay population.

Briefly, the model they use operates at a management unit level, which is divided into *N* evenly sized sites.^{3} Each site is classified into one of four categories that describe the height of the vegetation on the site. The four categories represent the following successional stages: short, optimal, mixed and tall. The growth rates of the scrub jay population depend on the proportion of sites in each category with rates of 0·735, 1·130, 0·875 and 0·850 for the four categories.

There are three actions available to the managers. First they can do nothing. Second they can burn the unit. Burning, however, is only partially effective at transforming the landscape. Third, they can restore the unit by clearing, which results in all of the sites being transformed to the short category. The actions are applied at the unit level, meaning that all sites receive the same treatment. Furthermore, owing to its high cost, restoration is only undertaken once on a unit. Once a unit is restored, it is only managed using the doing nothing and burning actions.

Site transition probabilities are estimated using the data in Table 1 of Johnson *et al*. that contain the 2004–2005 transitions for 1689 10 hectare sites. The probabilities are estimated using the sample frequencies for each of the eight possible initial category and burn/no burn combinations. Site transition probabilities associated with the do-nothing and burn actions are estimated to be^{4}

- (eqn 15)

and

- (eqn 16)

The site transition probabilities associated with the restoration action are

- (eqn 17)

In Johnson *et al*., these matrices are interpreted as projection matrices with a deterministic state transition given by *S*^{+} = *pS*. This is equivalent to replacing the actual transitions with the expected value of the transition. Given that *pS* will generally not exactly equal a grid point, linear interpolation is used to assign ‘probability’ weights to nearby grid points.

Figure 1 displays the optimal management strategy for an *N* = 10 site management unit when the transitions are assumed to be deterministic. This is analogous to Fig. 2 of the original paper; differences are attributed to different ways of handling the interpolation with the original using tensor product interpolation on a rectangular grid (F.A. Johnson, personal communication) and the results presented here using Freudenthal (triangular) interpolation on a simplex grid. Noted that only low values of category one are shown; if more than 4 of 10 sites are in the short category, the do-nothing strategy is always optimal.

Figure 2 results from using the category count transition matrices, in which each site is assumed to evolve as an independent Markov chain, with the aforementioned *p* matrices defining the transition probabilities. This description of the stochastic nature of the transitions is consistent with the estimation approach used to obtain the individual site transition matrices. There are some significant differences between the two strategies especially if the unit has not yet been restored. When the transitions are treated as stochastic, it is optimal to use the restoration action in far fewer states than when the transitions are treated as deterministic. This is consistent with financial option pricing theory; given that the restoration action can only be performed once, it can be viewed as an option that, once exercised, is gone forever. It is generally true that it is optimal to exercise an option in fewer states as system uncertainty increases (see, e.g., Hull 2000).

With the deterministic model, the long-run proportions of sites in the four categories are 0·1264, 0·1466, 0·6789 and 0·0481 and burning was optimal 91·5% of the time. In contrast when the stochastic framework is used the long-run proportions of sites in the four categories are 0·1052, 0·1376, 0·7093 and 0·0480 with burning occurring only 85·5% of the time. The population growth rates are similar, 0·8935 for the deterministic model and 0·8942 for the stochastic (these figures assume, of course, that the purported model is correct).

Three comments are relevant at this point. First, although in some ways the deterministic and stochastic approaches might be viewed as alternatives, the stochastic approach has a desirable internal consistency in that the parameters were estimated with precisely this model as the assumed data generating process. Second, although it may be the case that the deterministic and stochastic approaches lead to similar management strategies, it is difficult to characterize a priori the conditions under which this will be true (unless all rewards and transitions are linear in the state). Third, if an adaptive management approach to parametric uncertainty is taken, it is important that all sources of environmental uncertainty be considered to avoid underestimating the degree of sampling error, which in turn determines the updating process and the speed of learning. All of these comments suggest a preference towards using the stochastic approach.

As mentioned earlier, the simple category count approach applies to a situation in which all sites evolve independently. In many cases, however, either the number of sites in the various categories or some external environmental variable may influence the individual site transition probability matrix. In the context of the scrub jay management problem, there are a number of environmental variables that could have such an influence. One is the amount of rainfall, which has a direct influence on both the succession probabilities (succession slows in dry years) and on the fire probabilities (dry years make fire more effective at reducing vegetative cover). Unfortunately, the transition matrices used here are derived using maximum likelihood estimates of a multinomial model with a single year of data. To examine the effect of environmental covariates will require a multiyear sample. With more years of data, it would be possible to estimate the effects of covariates such as rainfall on the site transition matrices. This would lead to a set of category count transition matrices that either depend on the covariates, if these are known at the time the decision is made, or on a single category count transition matrix that averages out the effect of the covariates, with the averaging taken with respect to the probability distribution of the covariates.

### Concluding comments

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

This paper has developed a general framework for spatially implicit models for a set of otherwise homogeneous sites (or other elements) that differ only in their category and treatment. In these models, the category for each site evolves independently as a Markov chain (possibly with the transition probabilities dependent on common environmental factors). The environmental state consists of the number of sites in each of the categories, and the action is defined as the number of sites in each category/treatment combination.

Although such models have appeared before in the literature, this paper extends the framework by providing a general method for computing transition probabilities. This allows more categories and treatments to be used than have appeared in previous models, and it allows the treatments to be defined in a more flexible fashion. Software for performing these calculations should make these models more attractive to analysts developing site management strategies and is available on the internet.

The approach is applied to a conservation management problem involving habitat management for a threatened species. An optimal management plan is developed and contrasted with one that assumes a deterministic transition. Important differences exist in the two approaches.

One logical extension of this work is to integrate the approach with an adaptive management strategy which would explicitly recognize that the site transition probabilities are not known precisely. The use of deterministic transitions would severely bias the updating of belief probabilities in an adaptive management context. Future work will develop methods for addressing parameter uncertainty.

- 1
Whether the loss of information in a spatially implicit approach is outweighed by the gain in solvability depends on the nature of the problem being asked. Some insight into this issue is provided in Perry & Enright (2007) and Frank & Wissel (1998).

- 2
A search of the literature did not produce any results dealing with the probability distribution of the sum of multinomial random variables. It did, however, uncover an unpublished report (Butler & Stephens 1993) that used essentially the same method to solve the simpler problem of computing probabilities for sums of binomial random variables.

- 3
The original paper described the model in terms of the proportion of the unit in each category and discretized the proportion using

*N*+1 proportion levels between 0 and 1. Multiplying the proportion by*N*provides an equivalent way of representing the model in a category count framework. - 4
To be consistent with Johnson

*et al*., the site transition matrices are displayed in column-stochastic form, so rows represent future categories and columns represent current categories.

### Acknowledgements

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

Thanks are due to Fred Johnson for discussions of the application to Florida scrub jays and to him and Mitch Eaton for comments on the manuscript, and also to Jim Nichols and Julien Martin for useful discussions on structured decision-making and dynamic programming. This research has been supported in part by funds from USDA Hatch Project 0215736.

### References

- Top of page
- Summary
- Introduction
- Category count models
- Examples from the literature
- Transition probabilities
- Issues in specifying and solving category count problems
- Habitat management for an endangered species
- Concluding comments
- Acknowledgements
- References

- 1957) Dynamic Programming. Princeton University Press, Princeton, New Jersey. (
- 2008) A state-dependent model for the optimal management of an invasive metapopulation. Ecological Applications, 18, 748–761. & (
- 2011) A state-dependent model for the optimal management of an invasive metapopulation: erratum. Ecological Applications, 21, 1886. & (
- 2010) Analytical methods for a stochastic mainland-island metapopulation model. Ecological Modelling, 221(21), 2526–2530. & (
- 1993) The Distribution of the Sum of Binomial Random Variables. Technical Report 467. Department of Statistics, Stanford University, Stanford, California. & (
- 1994) Extinction and colonization processes: parameter estimates from sporadic surveys. American Naturalist, 143, 583–596. & (
- 1998) Spatial aspects of metapopulation survival – from model results to rules of thumb for landscape management. Landscape Ecology, 13, 363–379. & (
- 1989) Population Harvesting: Demographic Models of Fish, Forest, and Animal Resources. Princeton University Press, Princeton, New Jersey. & (
- 2003) Counting and enumerating frequency tables with given margins. Statistica & Applicazioni, I(2), 87–104. (
- 1999) Metapopulation Ecology. Oxford University Press, Oxford. (
- 2001) The effects of habitat destruction in finite landscapes: a chain-binomial metapopulation model. Oikos, 93, 321–331. Direct Link: & (
- 2000) Options, Futures and Other Derivative Securities, 4th edn. Prentice Hall, Englewood Cliffs, New Jersey. (
- 2011) A Markov decision process for managing habitat for Florida Scrub-Jays. Journal of Fish and Wildlife Management , 2, 234–246. , , , , & (
- 1995) SDP: generalized software for solving stochastic dynamic optimization problems. Wildlife Society Bulletin, 23, 738–742. (
- 2011) An adaptive-management framework for optimal control of hiking near golden eagle nests in Denali National Park. Conservation Biology, 25(2), 316–323. , , , , , , & (
- 2002) Applied Computational Economics and Finance. MIT Press, Cambridge, Massachusetts. & (
- 2010) Should metapopulation restoration strategies increase patch area or number of patches? Ecological Applications, 20(2), 566581. & (
- 2007) Contrasting outcomes of spatially implicit and spatially explicit models of vegetation dynamics in a forest-shrubland mosaic. Ecological Modelling, 207, 327–338. & (
- 1996). Decision theory and biodiversity management: how to manage a metapopulation. Frontiers of Population Ecology (eds R.B. Floyd, A.W. Sheppard & P.J. De Barro), pp. 391–398. CSIRO, Melbourne. (
- 1999) Optimal fire management for maintaining habitat diversity. Ecological Applications, 9, 880–892. , & (
- 2000) Optimal release strategies for biological control agents: an application of stochastic dynamic programming to population management. Journal of Applied Ecology, 37 (1), 77–86. & (
- 1985) A survey of algorithms for exact distributions of test statistics in r x c contingency tables with fixed margins. Computational Statistics & Data Analysis, 3, 159–185. & (
- 2001) Analysis and Management of Animal Populations. Academic Press, San Diego, California. , & (