The need for deference to well-grounded claims to expert authority often seems to conflict with democratic ideals and practices of equality and contestation. In this article I identify a parallel tension in Mill's work on authority. Against the idea that Mill's thought is contradictory, I argue that in both his early and later work he was very clear about the tension between the good of thinking for oneself and the necessity of epistemic dependence, and in particular deference to a consensus of experts. He does not resolve this tension, but he makes it productive in the figure of the ‘competent observer’, who exercises judgement in deferring to authorities. Mill's contribution is to focus on the process of questioning and scrutiny that underpins this sort of judgement. I conclude with some observations about the value and limitations of Mill's account of authority for understanding contemporary problems of expertise in democratic systems.