Soft law, or non-legislative modes of policy making, is becoming increasingly common today. The Nordic countries have a long tradition soft law, not least in central–local relations, where non-binding agreements are frequently used to coordinate policies. A key question springing from soft law theory is that of compliance. Why do independent actors comply if they are not formally obliged to do so, and what happens if they do not comply? This article addresses the question of how compliance can be achieved during policy conflict between actors at different governing levels by investigating a case of health care reform in Sweden. An important finding in the study is that compliance was reached ‘in the shadow of hierarchy’. The central government resorted to the threat of regular legislation to force the county councils to comply. This finding points to the fact that sanctions and the presence of a hierarchical order may play an important role even in soft law governance. The study also shows that an additional important reason that the voluntary agreement between the county councils and central government was honoured in the end by both parties can be attributed to the efforts of a mediating actor: the organization representing the county councils in their negotiations with the government. Finally, the study also illustrates how various forms of informal social pressures such as shaming, peer pressure and moral responsibility can help enforce local compliance in a case of open policy conflict. Arguably, all these compliance mechanisms also have relevance outside the Nordic setting.