I have been Editor-in-Chief for about 10 months now. Over that period of time, I have processed hundreds of manuscripts and considered hundreds of reviews. In doing so, I have noticed an emphasis on analysis at the expense of a better understanding of the ecological system under study. I mention this not to belittle statistical advances made within various disciplines of wildlife science. In truth, I admire and appreciate our role as leaders in developing new and rigorous approaches for data analysis. But in doing so, have we compromised our roles as biologists and managers? I do not pretend to know the answer, but the question continues to nag at me.
All parts of a study and the resulting manuscript(s) are important. Perhaps the foundation of any study includes asking appropriate questions, designing studies correctly, and measuring the right variables. Indeed, whether questions are posed as objectives, hypotheses, or alternative models, they must be relevant and addressable. They must be based on knowledge of the system under study to identify key information gaps. They should not be exploratory, but they should be well-informed and focused. The questions must further our understanding of the system and improve management options. After all, this is the Journal of Wildlife Management.
Volumes have been written about study design, so it would be folly for me to expound on all virtues of a well-designed study here. In gist, studies should be designed that capture variation in a system across time and space to enable inferences at the appropriate scale(s) to address the study question(s). Ideally, placement of plots and sampling units should embrace concepts of randomization and replication to enable broad inference. Sample sizes, of course, must be adequate for unbiased and precise parameter estimates. I understand and appreciate constraints imposed by budgets and logistics and we must often work within them. The bottom line, however, is that the study design must be adequate to address the question(s).
A myriad of variables can be measured in any study. Few of us have enough resources to measure everything, so we must pare down that list to those most biologically relevant. Measuring everything with the hope that something will be “significant” is akin to all-possible subsets analysis or model selection considering all possible combinations of variables without developing a set of a priori models.
Intertwined with facets of designing and conducting a study is data analysis. Herein lays my concern. We have many tools and options for conducting analysis. They are evolving constantly. As I look back over the course of my career, I have seen the transition from univariate statistics to multivariate statistics to model selection and parameter estimation. Every step along the way, the new approach is regarded as the best approach. Authors assume that if they do not embrace and employ the latest analysis, their manuscript will not be published. Statistical methods sections are growing in size, as are the corresponding results sections. Tables rarely provide simple descriptive statistics, but now list dozens of candidate models and corresponding diagnostic metrics. Discussions often dwell more on the merits and limitations of the analysis than on the relevance of their results to biology and management. Enough!
Please do not misconstrue my message here. As I noted above, wildlife science is a leader in developing new and more powerful ways to tackle a dataset. All things considered, any one dataset may be analyzed in multiple ways. My intent here is not to enter into the significance-testing versus model-selection debate. Both offer viable alternatives for approaching a dataset. I am reminded of a quip from my station statistician. He basically said that if you walk into a room with a dataset and give it to 10 statisticians, they will analyze it in 10 different ways. The point being that one has options for analyzing a dataset.
My take-home message here is that I would rather see a well-designed study with appropriate statistics and clear management implications than a poorly designed study with intricate analyses and equivocal results. I would rather see a manuscript that uses analysis as a tool than as the end. Let's not be paralyzed by analysis at the expense of furthering our understanding of ecological systems and providing viable wildlife management options. That is, after all, what we are all about!