Digital library evaluation is a complex field, as complex as the phenomena it studies. The interest of the digital library society still remains vibrant after all these years of solidification, as these systems have entered real-life applications. However the community has still to reach a consensus on what evaluation is and how it can effectively be planned. In the present article, an ontology of the digital library evaluation domain, named DiLEO, is proposed, aiming to reveal explicitly the main concepts of this domain and their correlations, and it tries to combine creatively and integrate several scientific paradigms, approaches, methods, techniques, and tools. This article demonstrates the added value features of the ontology, which are the support of comparative studies between different evaluation initiatives and the assistance in effective digital library evaluation planning.