The Caulkins et al. paper  gives a fair critique of the issues relating to attempts by us and others to develop a comprehensive single scale against which all drugs can be compared. I would not argue with their criticisms of our approach.
We ourselves pointed out that it has a number of flaws. For example, in the paper that prompted the Caulkins et al. response , we specifically mention that factors such as legal status were bound to influence use and harms. We also argued for a more sophisticated analysis in which drug benefits would also be taken into account; however, how one does this is still uncertain. One way would be to develop another list of benefits and plot the scores orthogonal to the harm ones. The alternative matrix approach Caulkins et al. suggest would, to some extent, overcome some of the limitations of our approach and provide a more refined analysis. However, it would be very hard to do, which I presume is why they have not done it themselves and presented the outputs. Perhaps a test analysis for one or two drugs would help to clarify the necessary process and the kind of results that might be obtained. If it appeared technically feasible and the results promising, then we and others might well take it up. Indeed, if the results from a more sophisticated matrix approach are ever published, then we would be pleased to use the information it contains in any future drug harms assessments.
The second criticism is that a single ranking score is of little value. I would contend that despite its scientific limitations, the composite score that we developed with the two dimensions of harms to self and others, by virtue of its clarity and simplicity, gives a very useful perspective on drug harms. This in itself is a significant improvement on our earlier one-dimensional scale, which nevertheless was beneficial for public health decisions, legal and political thinking and public education. It should be remembered that all decisions regarding drug classifications resolve harms into a single scale point for each drug so people, particularly politicians, are used to making and working with such estimations. Our two-dimensional matrix is especially helpful, as it clearly separates the harms to self and those to others. This allows the ‘self’ scores to drive interventions for individual benefit, including education and legal controls, whereas the ‘others’ scores can inform and hopefully direct policy aimed at reducing the total burden of harms from any drug.
Anyone interested in alcohol and other drugs, from law enforcement to education and from health improvement to international policy, needs some measure that allows them to understand and communicate relative harms and risks. I believe we have provided the best currently available analysis of an extremely complex multifaceted data set. It isn't perfect but is nevertheless good enough to be useful, and if it encourages the pursuit of the best then more to the good!