Commentary on Hyggen (2012): Subtle perils of large general population surveys


Jessor & Jessor [1] found that cannabis use was associated with less conformist attitudes. Hyggen [2] also found that people who admit to using cannabis in the past year value work a little less, and proposes that cannabis use in Norway remains relatively rare, hence still deviant, unlike countries such as the United Kingdom [3,4]. Norwegian prevalence is approximately average for Europe [5], and 60% of Hyggen's sample had been exposed to, or used, cannabis at some time, which suggests normalization in Norway as well. Therefore, reduced work commitment is unlikely to be due to an a-motivated subculture, although perhaps it could be due to people whose life values extend beyond work to leisure and family, because being a parent and alcohol use also reduced work commitment; or might it be due to the effects of cannabis, for cause is implied in the paper's title? In fact, Hyggen's results are too weak to support a relationship of any consequence.

  • 1Figure 1 exaggerates the difference in work commitment (no real zero on axis, no error bars), which is actually only 0.4 units on a scale with unknown variance, having a range from 1 (strongly disagree) to 5 (strongly agree). If this is correct, then involved cannabis users agree slightly, albeit significantly, less with work commitment statements, but the average remains on the ‘more committed’ half of the scale.
  • 2For the between-groups analyses there were only 63 (4%) ‘involved’ cannabis users who differ on work commitment from the other groups. Moreover, the data were weighed for analysis by means unspecified. Weighting tends to exaggerate the impact of smaller groups in multivariate analyses, and there is also the potential for systematic response bias in a group of only 63 subjects.
  • 3The regression analysis fails to consider the probable effect size in the real world. The R2 is about 0.07, meaning that the entire model predicts only about 3% of the variance in work commitment scores. That a tiny relationship is statistically significant is a consequence of the large sample size, not of the importance of the relationship.
  • 4Both ‘cannabis use’ and ‘work commitment’ are self-report variables which could covary because of reporting biases, such as that the same people may be more willing to admit more unconventional/less socially acceptable values in surveys.
  • 5Use in previous 12 months is a very weak index of ‘cannabis use’, ranging from daily users to people who had had one puff at a party.

Hyggen's approach is not unusual, and illustrates some of the pitfalls of using epidemiological methods to research psychological constructs and social factors: (i) to hurry to the complex—and ‘significant’—statistical analyses without lingering carefully enough on basic descriptive statistics; (ii) to make category mistakes in interpreting variables derived from questionnaire answers. This is the equivalent of eating the menu in a restaurant (confusing the sign with the signified in semiotic terminology), by treating how people answer questions as uncomplicated representations of their thoughts and behaviour. Similar difficulties pervade the more influential literature on cannabis and psychosis: (iii) fishing expeditions seem acceptable if the findings conform to contemporary prejudices. In the epidemiology of drugs this means that any association between illicit drug use and some harm will be accepted relatively uncritically, as here; (iv) ignoring problems of small subsamples and small effects. In epidemiological research on physical disease, the reasonably safe assumption is made that disease mechanisms are relatively invariant across people and unaffected by participants’ volition. It may be appropriate, then, to compare a small group of people with a disease to a much larger group without disease. When, as here, assignment to the small group is based instead on subjective answers to questionnaire items, then such comparisons need to be approached with caution. (v) In the interests of brevity, drug use questions in general surveys are often excessively attenuated to the point that no useful quantity–frequency data are available. (vi) Again for brevity, surveys often focus on worrisome behaviours without contextualizing them within life-style. Cannabis and alcohol ‘involvement’ might indicate stress, anxiety, depression and reduced work ethic, or might indicate a work–life balance involving leisure and socialization: there is no way to tell.

Declaration of interests