The writers of this thoughtful paper address issues of significance to those involved in making judgements about children’s health and well-being. They rightly identify some challenges to current approaches of risk identification, which is very welcome. The paper raises several questions, and I would like here to explore a few of the implications of the position they outline.
Essentially, their case is that the use of screening instruments to identify the degree of risk of child abuse or neglect in individual families is theoretically and methodologically flawed, and ethically questionable. I agree with their assessment of the ethical position, and here address only theoretical and methodological issues.
Taylor et al. (2008) claim that the role of the health visitor does not include ‘prediction’. This claim is strictly speaking true in so far as they do not (and cannot) state that a specific event will happen in the future – which is a dictionary definition of prediction. However, it is also true that the screening instruments they describe similarly do not (and cannot) state with certainty that specific events will happen – they offer a quantified degree of probability that they will happen. If we understand the word ‘predict’ in the context of risk assessment to mean ‘estimate the probability of’, then it appears that the role of the health visitor must include this cognitive activity, and this appears to be acknowledged by the authors. The activity of assessing risk does involve an attempt to look into a range of possible futures, and to identify which is most likely, given the information we have now. This is an inescapable feature of professional assessment (and indeed everyday life) whether or not it is articulated, acknowledged or even recognised, and it is on the basis of these probability estimations that we make our choice of actions.
Why is it important to recognise this? Because there is a robust body of research which demonstrates that when it comes to thinking about probabilistic events, people are prone to significant cognitive biases in reasoning (Tversky & Kahneman 2000). For example, a well known bias is known as ‘anchoring’: if we estimate the probability of risk to be very high, for example, even in the face of significant amounts of incoming evidence to the contrary, we do not lower that probability as much as we logically should. These biases prove resistant to education and training, and are therefore extraordinarily difficult to overcome.
Screening instruments attempt to overcome these natural human biases by directing the clinician’s attention to those factors which evidence has shown to have a direct relationship with the condition which is being looked for (in this case child neglect or abuse). They offer systematic approaches to accumulating and weighing the information in clinical situations available to the professional (Ruttiman 1994). Since they are developed by researchers, they are rigorously tested (or should be), and therefore we know how well they function as instruments of ‘prediction’– i.e. probability assessment. The more complex, uncertain, and diffuse the phenomenon under investigation (child abuse/neglect), the more likely they are to have low sensitivity and specificity: the more rare the phenomenon in the population the more likely they are to have low predictive values, as Taylor et al. correctly point out. (For a fuller explanation of these relationships, see Harbison 2006). This is true of judgements made using a screening instrument; however evidence suggests it is also true of unassisted human judgement (Camerer & Johnson 1997).
On the basis of perceived deficits in screening instruments, the writers recommend that health visitors should rely on their own ‘reasoned assessments and judgements…using evidence and practice wisdom’. The apparent underlying assumption here is that doing so would reduce the dangers of incorrectly identifying risk. However, there is no real evidence to support that assumption. Whereas, the ‘reasoning’ of the screening instrument is visible and open to scrutiny, and shows how evidence pertinent to the situation is taken into account, the same is not necessarily true of the ‘reasoning’ of the health visitor.
What is interesting in debates like these about the value of aids to judgement is that the predictive strengths of the tool are carefully calculated, while the predictive strengths of the unaided practitioner(s) are seldom considered. Taylor et al. are correct to say that little confidence can be attributed to screening instruments in this field at their current stage of development. However no-one to my knowledge has produced evidence as to the confidence we can attribute to health visitors’ judgements in assessing risk, compared to a screening instrument. We should be cautious about making assumptions about the accuracy of human judgements without subjecting these to the same rigorous testing which screening instruments undergo. I am not saying that health visitors make less accurate risk assessments of child abuse and neglect using their professional judgement than using screening tools: I am simply saying that I am aware of no evidence either way.
And finally, Taylor et al. rightly point out why we should think very carefully about these initiatives – the consequences of misjudgements in the area of child neglect/abuse can be devastating. They understandably advocate the need for resources for all parents and families to be supported in order to reduce risk. However as we all know, resources are finite, and there are opportunity costs to be considered, therefore, it seems likely that resources will always be limited. Therefore, I would suggest that whether they articulate it or not, health visitors have to make judgments about how time and resources are spread around the caseload, and furthermore those judgements are based on assessments of how badly individual families need those resources – including assessments of the degree of risk to the children. It does not I think do clinicians any favours to obscure the necessity for making these judgements, and therefore to obscure the processes by which they make them. But I do understand the reluctance to face this necessity, which contributes to the general resistance to screening tools and on a wider scale, other forms of decision support (Winkler 2001). People do not like the articulations of uncertainty which screening tools provide, and would prefer to believe that reliance on ‘professional judgement’ will eliminate uncertainty, or at least reduce this to an acceptable minimum. But sadly the evidence from judgement research suggests that this may be an unattainable goal.