Comment on ‘Evaluating indices of conservation success: a comparative analysis of outcome- and output-based indices’



Sally Cunningham, Darwin Initiative Secretariat, International Biodiversity Sub-programme, Zone 1/15, Temple Quay House, 2 The Square, Temple Quay, Bristol BS1 6EB, UK.


We welcome Howe and Milner-Gulland's (2012) paper published in Animal Conservation. The Department for Environment, Food and Rural Affairs is keen to retain the Darwin Initiative's reputation in achieving real, measurable difference to conservation of biodiversity in developing countries. Papers such as this support the conservation community in learning how better to evaluate the success of biodiversity conservation initiatives. We believe papers such as this contribute to evaluating the success of programmes such as the Darwin Initiative through evaluating their component parts – the projects.

However, the paper makes at least one critical assumption about how the Darwin Initiative evaluates its success, which is incorrect. The paper assumes that the ‘standard measures’ collected from every Darwin project are used as a measure of its effectiveness. In fact, as rightly pointed out by the authors, this is a list largely of inputs, activities and outputs. Few are outcome-orientated indicators, and therefore cannot be used as indicators to evaluate the success of the Darwin Initiative. These standard measures have been collected from the Darwin Initiative projects since its inception in 1992. Therefore, these measures provide a 20-year dataset of the types of activities, inputs and outputs generated by the Darwin Initiative projects. They do not provide a 20-year dataset of the effectiveness of the Darwin Initiative in biodiversity conservation.

As the paper points out, the standard measures do not allow us to measure outcomes or wider impacts of projects, which should be the measure of effectiveness of any conservation programme. For example, a project that attains seven MSc qualifications may have limited conservation impact compared to a project that achieves no formal qualifications for its staff but has achieved substantial changes in governance structures capable of supporting conservation of biodiversity.

The Darwin Initiative's approach to evaluating the success of its funding has relied on the Organization for Economic Cooperation and Development's Development Assistance Criteria for evaluating development assistance (OECD, 1991), which are: relevance, effectiveness, efficiency, impact and sustainability. To evaluate these criteria, the Darwin Initiative has largely focused on the logical framework and the narrative of project final reports, in particular the purpose statement (now called the outcome statement). At application, each Darwin Initiative project has its own target (in the form of the purpose statement), which is approved at application stage by the Darwin Expert Committee. Projects are funded on the basis that they seek to achieve this purpose statement.

The Darwin Initiative reviews the effectiveness of its projects on a project-by-project basis. Effectiveness is evaluated by reviewing how successful the project has been at achieving its purpose statement (outcome). A successful project is one that can provide sufficient evidence of achievement, which generally relates to the indicators used. These indicators are expected to be SMART (specific, measurable, achievable, realistic and time-bound) and outcome-orientated. Material evidence needs to be supplied for these indicators to be deemed evidence of achievement. Projects that do not achieve these intended targets are reviewed to better understand why they have not achieved and to draw out lessons for future projects.

With the focus on outcome rather than input, activity or output, the standard measures are not used as indices of effectiveness of the Darwin Initiative projects. The Darwin Initiative is a diverse programme with projects targeting a range of biodiversity issues, the standard measures fail to capture the full range of outputs and actions that make up a successful Darwin Initiative project. Therefore, the standard measures should be seen only as a collective dataset of inputs, activities and outputs that can be used as part of the evidence of achievement (outcome). In 2012, we have in fact removed the standard measures from application materials, as we felt it was placing too high an emphasis on quantitative targets of inputs and activities and was confusing to applicants. No decision has been made yet on whether we will continue to collect data on the standard measures in the future.

We felt it was important to make this clarification, because this journal is read by many Darwin Initiative grant holders and prospective applicants. Please be reassured, the Darwin Initiative will continue to evaluate the success of its funding based on the outcomes of its completed projects and not on the scale of activities or value of inputs.

Darwin Initiative Secretariat, International Biodiversity Policy Unit, Department for Environment, Food and Rural Affairs (Defra).