In probation and, in Scotland, in criminal justice social work, the evaluation of effectiveness is fraught with difficulty. Engaging workers in the process of defining effective probation and in developing effectiveness measures is essential if workers are to have any investment in evaluation. The research study reported here attempted to explore workers' definitions of effective probation and found evidence of considerable diversity of opinion on this vital issue. In general though, workers preferred to measure effectiveness in terms of ‘outputs’ or ‘outcomes’– reducing reoffending, changing attitudes, increasing victim empathy and alleviating needs. The study concluded that there are some grounds for cautious optimism about the capacity, opportunity and motivation of criminal justice social workers to rise to the challenge that the effectiveness agenda represents. However, effectiveness evaluation needs to be enacted in a pluralistic manner, recognising the ethical and practical complexity of the issue, as well as the diversity of workers' perspectives.