A web application for reasoning on probabilistic description logics knowledge bases

The aim of the Semantic Web is making information and resources from the Web automatically processable by machines. Usually, the uncertainty characterizing much of this information is addressed by means of a probabilistic semantics. Following the vision of a “Probabilistic Semantic Web”, a plethora of probabilistic semantics have been proposed: some of them change the syntax and/or the semantics itself of the knowledge representation language, others allow one to annotate axioms of a knowledge base with a probability value. Among the latter, the DISPONTE semantics exploits probabilistic annotations to extend query answering with the capability of returning the probability of a query being true in a domain. In order to promote the adoption of Probabilistic Semantic Web we first developed BUNDLE, a framework that can exploit different underlying (probabilistic and non‐probabilistic) reasoners to perform probabilistic inference under the DISPONTE semantics. In this paper we present a web application for BUNDLE, to show how DISPONTE is easily usable even in already established applications and systems. It allows users to query a DISPONTE knowledge base written or uploaded directly in the application interface by using just a web browser, without the need to install any software on their machine. It is accessible on the web at https://bundle.ml.unife.it/ and also provides some examples for familiarizing with the application. The results of a usability evaluation involving human participants are also reported, showing the relevance and the practical impact of the tool and possible ways for improvement.

data, called Web Ontology Language (OWL), which became a W3C recommendation in 2012 and is now at its second version 2 (OWL 2). These formalisms provide languages to model domains where it is necessary to represent specific information and to perform inference on it, such as software engineering, medical diagnosis, digital libraries, databases, and web-based information systems. These domain models are called ontologies, or knowledge bases (KBs). OWL 2 is designed to facilitate ontology development and sharing via the Web, with the ultimate goal of making web content more accessible to machines. Description Logics (DLs) are particularly useful for representing ontologies and have been adopted as the basis of the Semantic Web. They are fragments of First-Order Logic possessing nice computational properties such as decidability and, sometimes, low complexity. 3 In the last decades, it has become evident that the Semantic Web data is often uncertain, so it is extremely important to represent and reason with uncertain information. Application areas in which uncertain aspects of domain concepts need to be modelled can greatly benefit from uncertainty representation of any kind in DLs. For instance, almost every bio-medical ontology contains uncertain concepts of some sort. The well-known medical ontology SNOMED CT 4 comprises a variety of classes with names such as "Probable tubo-ovarian abscess", "Natural death with probable cause suspected", "Probable diagnosis". Other than the medical field, a wider use of such formalisms could positively affect many other domains. Applications of (certain) ontologies can be found in software engineering and software technology 5,6 for sharing knowledge of the problem domain and using a common terminology among all stakeholders and filtering the knowledge when defining models and metamodels. The W3 Consortium proposed a document for "potential uses of the Semantic Web in Systems and Software Engineering". * Ontologies have been used also in the legal domain. 7 We believe that all these fields could benefit from an uncertain representation, as well as, for example, fault diagnosis or recommender systems, where probability could be used, in the former case, to calculate a fault risk value, showing whether a production line could be affected by a fault and, in the latter case, to rank possible recommendations. These needs motivated many researchers to study and present probabilistic semantics capable of handling the uncertainty inherent in the Semantic Web. For example, Fuzzy DLs have been widely studied in the literature for representing vagueness, [8][9][10][11] based on the idea that concepts and roles can be interpreted as fuzzy sets and fuzzy binary relations. Giordano et al. 12 show that the input/output behavior of self-organizing maps, which have been proposed as possible ways to explain the psychological mechanisms underlying category generalization, after training can be described by a fuzzy description logic interpretation as well as by a preferential interpretation. Yu et al. 13 propose a Causal Probability Description Logic framework which introduces a "causality probability" concept into the KB level of the Probabilistic Description Logic. 14 The goal is to help learn comprehensive health causal knowledge and relations among the diseases and symptoms from the UK National Health Service website linking with DBpedia datasets. DISPONTE, for "distribution semantics for probabilistic ontologies", [15][16][17] instead allows the user to annotate axioms of a knowledge base with a probability representing the degree of belief in their truth. It is inspired by the distribution semantics 18 proposed in the field of Probabilistic Logic Programming (PLP). This semantics for PLP allows one to compute the probability of a query from a probability distribution over normal logic programs.
In the literature several non-probabilistic reasoners are available-Pellet, 19 Fact++, 20 JFact, † HermiT 21 -which extract implicit information from ontologies, as well as probabilistic reasoners-Pronto, 22 BORN, 23 onto2problog, 24 TRILL, 25,26 TRILL P27 and TORNADO 28 -which can compute the probability of a query posed on a probabilistic DL. BUNDLE ("BDDs for uncertain reasoning on description logic theories") 29,30 is a system for performing probabilistic inference from DISPONTE knowledge bases by means of Binary Decision Diagrams (BDDs): BUNDLE computes the probability of queries from a covering set of explanations returned by an underlying reasoner. Initially built for exploiting Pellet as an underlying reasoner to collect explanations, it was extended 31 in order to: (1) interface with Fact++, JFact, and HermiT in the same way as Pellet; (2) interface with TRILL, TRILL P and TORNADO which, instead, do not require the conversion of explanations into a BDD.
In this paper, we present a web application for BUNDLE for performing inference on user-defined KBs under DISPONTE (https://bundle.ml.unife.it/). The application allows the user to upload or write a KB in OWL -using several syntaxes such as Resource Description Framework (RDF)/XML, Turtle, or OWL/XML -and query it. The user can choose the maximum number of explanations to be returned and a reasoner among the ones with which BUNDLE interfaces. Then the query, the ontology and the user settings are sent to the server which sends back the answer. BUNDLE can answer different types of queries and can compute their probability, whether a query is entailed from the KB, the set of explanations that explain the entailment and the execution time split in the different inference phases.
The first goal of this application is to demonstrate the ease of adopting and reasoning under the DISPONTE semantics. In fact, DISPONTE exploits the built-in annotations in the OWL language to add uncertainty in the KB, allowing one to integrate many systems designed for OWL in BUNDLE without modifying them internally, but only exploiting them to gather the information needed to calculate the probability of the query. The second goal is popularizing the use of probabilistic DLs, making it easy to test the potential of probabilistic DL reasoners, and addressing the lack of web interfaces for probabilistic reasoning on ontologies.
The web application is inspired by our previous work, 32 where a web interface for the TRILL, TRILL P and TORNADO reasoners was presented. Besides them, to the best of our knowledge, there is no other application for probabilistic Semantic Web reasoning. In a broader sense, we can mention cplint-on-SWISH 33 and the ProbLog 34 online editor as other web applications for probabilistic reasoning, but they are tailored to PLP languages. For the OWL language, WebProtégé 35 is the web application of the well-known KB editor Protégé. ‡ However, despite the stand-alone version of Protégé can encapsulate (non-probabilistic) reasoners via plug-ins, its online counterpart does not allow to reason on the designed KB. Differently from WebProtégé, the BUNDLE web application can answer queries (also w.r.t. non-probabilistic KBs) without the need to install any software except a browser, thus it is available to everyone. On the other hand, without the web application, to try BUNDLE (or any other reasoners wrapped into BUNDLE except for TRILL) the user would have three different possibilities: (1) download the jar file or compile the code § after installing at least a Java Runtime Environment (similarly to the stand-alone version of Protégé), or (2) create an application importing BUNDLE as a dependency (possibly using Maven ¶ ), or (3) download the Docker image from the Docker Hub. # These methods require a more advanced computer science knowledge.
We validated the application through a questionnaire administered to an audience of researchers from all over the world, reached via various mailing lists in the field of Semantic Web and DLs. The questionnaire consists of the Post-Study System Usability Questionnaire (PSSUQ) Version 3, 36,37 to which some questions specific to the application were added. The choice of PSSUQ is due to the fact that its results are significant even with few answers, which can help in questionnaires such as this one, where it is not known in advance how many people are going to fill it out.
The paper is organized as follows: Section 2 and 3 introduce Description Logics and probabilistic Description Logics, respectively. Section 4 describes the BUNDLE framework and Section 5 the Web application with some running examples. Section 6 discusses the results of the usability questionnaire, used to evaluate and validate the potentialities of the application. Section 7 concludes the paper.

DESCRIPTION LOGICS
Descriptions Logics (DLs) are a family of knowledge representation formalisms that differ in how concepts (sets of individuals of the domain) and roles (sets of pairs of individuals) can be defined. In order to illustrate DLs, we describe in the following (D) , one of the most common fragments. It was introduced by Horrocks et al. 38 and it is of particular importance being semantically equivalent to OWL 2. Let C be a set of atomic concepts, R a set of atomic roles and I a set of individuals. A role could be an atomic role R ∈ R, the inverse R − of an atomic role R ∈ R or a complex role R•S. We use R − to denote the set of all inverses of roles in R.
Each A ∈ C, ⊥, and ⊤ are concepts and if a ∈ I, then {a} is a concept called nominal. If C, C 1 , and C 2 are concepts and R ∈ R ∪ R − , then (C 1 ⊓ C 2 ), (C 1 ⊔ C 2 ), and ¬C are concepts, as well as ∃R.C, ∀R.C, ≥ nR.C, and ≤ nR.C for an integer n ≥ 0.
A knowledge base (KB)  = ( ,  ,  ) consists of a TBox  , an RBox , and an ABox . An RBox  is a finite set of transitivity axioms Trans(R), role inclusion axioms R ⊑ S, and role chain axioms R•P ⊑ S, where R, P, S ∈ R ∪ R − . A finite set of role inclusion axioms  h is called role hierarchy. Given  h , we can define ⋅ ⊑ as the transitive-reflexive closure of where C and D are concepts. We use C ≡ D to abbreviate the conjunction of C ⊑ D and D ⊑ C. An ABox  is a finite set of concept membership axioms a ∶ C and role membership axioms (a, b) ∶ R, where C is a concept, R ∈ R and a, b ∈ I. ‡ https://protege.stanford.edu/. § Available at https://bitbucket.org/machinelearningunife/bundle/src/master/. ¶ https://maven.apache.org/. # Available at https://hub.docker.com/r/giuseta/bundle. A KB is usually assigned a semantics using interpretations of the form  = (Δ  , ⋅  ), where Δ  is a non-empty domain and ⋅  is the interpretation function that assigns an element in Δ  to each individual a, a subset of Δ  to each concept C, and a subset of Δ  × Δ  to each role R. The mapping ⋅  is extended to complex concepts as follows (where R  (x, C) = {y|⟨x, y⟩ ∈ R  , y ∈ C  } and #X denotes the cardinality of the set X): (D) also allows the definition of datatype roles, which connect an individual to an element of a datatype such as integers, floats and so forth.
The satisfaction of an axiom E in an interpretation  = (Δ  , ⋅  ), denoted by  ⊧ E, is defined as follows:  satisfies  , , and . In this case, we say that I is a model of . A KB is satisfiable if it has a model. Inference in a DL is decidable if the problem of checking the satisfiability of any possible KB representable with that DL is decidable. In particular, (D) is decidable iff there are no number restrictions on non-simple roles. Moreover, to maintain decidability every set of (complex) role inclusions must be acyclic.
A query Q over a KB  is usually an axiom for which we want to test the entailment from the KB, written  ⊧ Q. The entailment test may be reduced to checking the unsatisfiability of a concept in the knowledge base, that is, the emptiness of the concept. For example, the entailment of the axiom C ⊑ D may be tested by checking the unsatisfiability of the concept C ⊓ ¬D, while the entailment of the axiom a ∶ C may be tested by checking the unsatisfiability of a ∶ ¬C. Example 1. Consider the following KB "Crime and Punishment" containing 4 axioms i : This KB corresponds to Example 1 at https://bundle.ml.unife.it/examples. It states that if you killed someone then you are a nihilist and whoever is a nihilist is a "great man" (TBox). It also states that Raskolnikov killed Alyona and Lizaveta (ABox). The KB entails the query Q=raskolnikov:GreatMan. However, he does not seem to be such a great man.

PROBABILISTIC DESCRIPTION LOGICS
DISPONTE, 17,25,39 for "distribution semantics for probabilistic ontologies", applies the distribution semantics 18 of Probabilistic Logic Programming to Description Logics KBs. This semantics allows one to label any axiom (either in  or in ) with a real value p ∈ [0, 1] representing a probability. These axioms are called probabilistic axioms and take the form where E is a DL axiom and p ∶∶ E means that we have degree of belief p in axiom E. A DISPONTE KB can be defined as a pair of sets ( ,  ), where  is the set of non-probabilistic axioms and  is the set of probabilistic axioms. Following this definition, every non-probabilistic KB is a DISPONTE KB where  = ∅. DISPONTE is based on the idea that a set X = {X i |p i ∶∶ E i ∈  } of independent Boolean random variables X i is associated with the probabilistic axioms, so that P(X i = 1) = p i , that is, the probability of X i taking on value 1 is p i , the probability of axiom E i .
Given a query Q, DISPONTE computes its probability by building worlds, that is, non-probabilistic KBs to which a semantics can be usually assigned. A query is entailed by a world if it is true in every model of the world. A world can be obtained by specifying atomic choices: an atomic choice is a couple (E i , k) where E i is the ith probabilistic axiom and k ∈ {0, 1}, indicating whether E i is chosen to be included in a world (k = 1) or not (k = 0). A composite choice is a consistent set of atomic choices, that is, only one decision is taken for each axiom). The probability of a composite choice is where p i is the probability associated with axiom E i ; the product is justified by the fact that the random variables associated with the axioms are independent. A total composite choice, that is, a composite choice that contains an atomic choice (E i , k) for every probabilistic axiom, is called a selection. A selection identifies a non-probabilistic KB w , called a world, in this way: where  is the set of certain, that is, non-probabilistic, axioms. A composite choice identifies a set of worlds = {w | ∈  , ⊇ }, where  is the set of all selections. The probability of a world w , P(w ), corresponds to the probability of a selection and is given by is a probability distribution over worlds. Let us indicate with  the set of all worlds. Given a query Q and the set of all worlds  , the probability of Q is the sum of the probabilities of the worlds w in which the query is true: It is worth noting that in case that KB has  = ∅, every query Q entailed by the KB has probability P(Q) = 1.0. The following example illustrates inference under DISPONTE semantics.

Example 2.
Let us consider the knowledge base and the query Q=raskolnikov:GreatMan of Example 1 where three axioms have been made probabilistic: Here  = { 2 }. We then believe that whoever is a nihilist is a "great man" with probability 0.2 ( 1 ) and Raskolnikov killed Alyona and Lizaveta with probability 0.6 and 0.7 respectively ( 2 and 3 ). This KB has eight worlds and Q is true in three of them, corresponding to the selection : However, as the number of worlds is exponential in the number of probabilistic axioms, for large KBs it is infeasible to find all the worlds where a query is true. To reduce reasoning time, we can resort to finding justifications for the query and then compute the probability of the query from them. The problem of finding the set of justifications for a given query is a non-standard reasoning service useful for tracing derivations and debugging ontologies. This problem has been investigated by various authors. [40][41][42][43] The definition of justification can be derived from that of "explanation".
An explanation for a query Q is a subset of logical axioms  of a KB  such that  ⊧ Q. A justification is an explanation minimal w.r.t. set inclusion. Formally, an explanation  ⊆  is a justification if, for all  ′ ⊂  ,  ′ ⊭ Q, that is,  ′ is not an explanation for Q. According to DISPONTE, a composite choice is an explanation for Q if Q is entailed by every world of .
In particular, algorithms find a covering set of explanations for the query, where a set of composite choices K is covering with respect to Q if every world w ∈  in which Q is entailed is in . In other words, a covering set K identifies all the worlds in which Q succeeds.
In practice, the problem of computing the probability of a query is reduced to that of finding a covering set of justifications and then transforming it into a covering set of pairwise incompatible explanations. Two composite choices 1 and 2 are incompatible if their union is not consistent. A set K of composite choices is pairwise incompatible if for all 1 ∈ K, 2 ∈ K, 1 ≠ 2 implies that 1 and 2 are incompatible. The probability of a pairwise incompatible set of composite choices is P(K) = ∑ ∈K P( ). Given a query Q and a covering set of pairwise incompatible explanations K, the probability of Q is: where K is the set of worlds identified by the set of explanations K.
The set of all justifications for the query Q w.r.t. a KB  is the covering set of justifications for Q, denoted by All-Just(Q, ). This set can be retrieved through non-probabilistic justification finding algorithms. The problem of enumerating all justifications that entail a given query is called axiom pinpointing. 41 Example 3. Consider the KB and the query Q=raskolnikov:GreatMan of Example 2. We have the following covering set of pairwise incompatible explanations for the query: The probability of the query, according to formula (1), is

INFERENCE WITH THE BUNDLE FRAMEWORK
The reasoner BUNDLE 17,29,30 computes the probability of a query w.r.
The probability that f ALL−JUST(Q, ) (X) takes value 1 corresponds to the probability of Q and can be computed by translating f ALL−JUST(Q, ) (X) into the language of Binary Decision Diagrams (BDDs). A BDD is a rooted graph used to represent a function of Boolean variables, with one level for each variable. Each node of the graph has two children corresponding either to the 1 value or the 0 value of the variable associated with the node. Its leaves are either 0 or 1. BDDs are built combining simpler BDDs with Boolean operators. Once the diagram is built, function Probability described by Kimmig and colleagues, 44 a simple dynamic programming algorithm, can be applied to efficiently compute the probability of Q at the root of the graph. Example 4 (Example 2 cont.). Let us consider the KB and the query of Example 2. If we associate random variables X 1 with axiom 1 , X 2 with 2 , and X 3 with 3 , the BDD representing the set of explanations is shown in Figure 1. By applying function Probability 44 to this BDD we get and therefore P(Q) = PROBABILITY (n 1 ) = 0.176, which corresponds to the probability computed according to DISPONTE.
BUNDLE is characterized by a modular architecture, 31 shown in Figure 2, which allows inference: 1. from a non-probabilistic reasoner through implementations of the "Hitting Set algorithm", 45 which builds a Hitting Set Tree to incrementally obtain all the justifications. BUNDLE then builds a BDD for computing the probability of the query from the covering set of justifications. In this case, BUNDLE supports four OWL reasoners: Pellet BlackBox A black-box approach offered by the OWL API. **43 The OWL API is a Java API for the creation and manipulation of OWL 2 ontologies.

OWL Explanation
A library that is part of the OWL Explanation Workbench. 43 The latter also contains a Protégé plugin, underpinned by the library, which allows users to find justifications for entailments in their OWL 2 ontologies.
All the supported non-probabilistic reasoners can be paired with the BlackBox or the OWL Explanation methods, while only Pellet can exploit the GlassBox method.
The GlassBox and the BlackBox methods implement a custom version of the Hitting Set algorithm, while OWL explanation implements the standard version. 45 2. by directly querying the three probabilistic reasoners provided by the TRILL ("Tableau Reasoner for descrIption Logics in proLog") System: TRILL, 25 justifications. When using one of the TRILL Interfaces, the Probability Computation module, which converts a covering set of justifications into a BDD, is not used. Note that BUNDLE can return both the probability of the query and the set of justifications, each one with the associated probability, if combined with TRILL, while it will return only the probability of the query if interfaced with TRILL P and TORNADO (see also Table 1). Table 1 provides an overview of the reasoners and methods supported by the BUNDLE framework. The integration of both probabilistic and non-probabilistic reasoners in BUNDLE is made possible as DISPONTE exploits the "annotations", a type of OWL construct standardized in the language which allows for a smooth conversion of a certain KB into a probabilistic one, with minimal changes to the syntax. For instance, probabilistic axiom 1 of Example 2 can be defined as: <SubClassOf> <Annotation> <AnnotationProperty IRI="https://ml.unife.it/disponte#probability"/> <Literal datatypeIRI="http://www.w3.org/2000/01/rdf-schema#Literal">0.2</Literal> </Annotation> <Class IRI="#Nihilist"/> <Class IRI="#GreatMan"/> </SubClassOf> Note that the IRI to add the DISPONTE annotation is https://ml.unife.it/disponte#probability. In this way, every DISPONTE KB can also be queried by a non-probabilistic reasoner without any change. Clearly, probabilistic inference is this case will not be possible.
More details about the use of the DISPONTE annotation can be found at https://ml.unife.it/disponte. Thanks to its modularity, BUNDLE can be easily extended in three main ways: • By adding a non-probabilistic reasoner that implements the OWLReasoner interface of the OWL API library; • By adding a non-probabilistic reasoner that is able to perform the reasoning task of justification finding. This reasoner should implement the ExplanationReasoner interface defined in BUNDLE; • By adding a probabilistic reasoner that implements the ProbabilisticReasoner interface defined in BUNDLE, as done for the TRILL System.
Up to now BUNDLE was available as a standalone desktop application or as a library (see the work presenting the BUNDLE's modular architecture 31 for technical details). In the next section we are going to present a Web application, which allows the user to test the framework without installing any software on the local machine.

Design
The application design is based on the Spring Framework, which provides a comprehensive programming and configuration model for modern Java-based applications and allows to focus on application-level business logic, without unnecessary ties to specific deployment environments. In particular, Spring Boot and Spring MVC were employed in order to respectively embed the web server and configure its execution (the former), and for structuring the Web application according to the model-view-controller pattern (the latter).

User interface
The Web application is composed of three main views, all accessible from the menu bar placed at the top of the page: Home the main view, where a user lands when navigating to the URL https://bundle.ml.unife.it/ (Figure 3); Examples a static page containing the description of both toy and real-world ontologies included in the Web application and testable by clicking on the correspondent button ( Figure 4); Tutorial a static page, showing how to use the Web application.
The Home page is divided in two panels. On the left side users can write or upload an ontology. On the right side, users can select several queries, set the corresponding parameters, and visualize the results. In the following F I G U R E 3 Home Page.

F I G U R E 4 Examples page containing some predefined ontologies.
we illustrate how the Web application works, both the front-end and the back-end, by referring to the execution of Example 2.
A DISPONTE KB can be loaded ( Figure 5 left side "Ontology Setting") in three different ways: by (1) uploading an OWL file, (2) writing from scratch the KB in the text area on the left side, or (3) loading a predefined ontology from the Examples page by pressing the button Try now (Figure 4). Note that, when a KB is uploaded or chosen from the examples, its content is shown in the text area, where it can be eventually modified and saved as an .owl file. Then users can load the KB into BUNDLE, by pressing the button Load Ontology. At this point, the KB will be sent to the server, as shown in step 1 in Figure 6. Here, the back-end initializes a data structure containing the current session ID, reads the KB, and instantiates an OWLOntology object (which is saved into the session data structure), and extracts all the individuals, classes, and properties to populate the drop-down menus in the right panel.
In the right panel ( Figure 5, "Query Setting") the user is asked to choose the query type from a first drop-down menu (step 2, Figure 6). The available query types are: Depending on the query type, a set of consequent drop-down menus will pop up below, after pressing the Submit button, for setting the query parameters (step 3, Figure 6). For example, by selecting the "Instance" query as shown in Figure 5 on the right, two drop-down menus for parameter setting will be shown, one with all the individuals and one with all the concepts in the ontology.
Finally, it is possible to set the maximum number of justifications to find and select one of the reasoners integrated into BUNDLE. By now, HermiT, Pellet, JFact, Fact++, and TRILL are available. TRILL P and TORNADO are not available because they do not return a set of justifications.
By hitting the Execute Query button, the arguments of the query are collected and sent to the server, where the selected reasoner will be initialized and the query performed. The collected justifications are sent to the client and shown to the user, together with the execution time (in seconds), the query probability, the probability of each justification and the probability of each axiom used in the justifications, as shown in the bottom right panel "Query Results" in Figure 5. Together with the total inference time, the time taken in the different reasoning steps is shown by means of a color bar.

F I G U R E 6
The client-server interactions to solve a query on the BUNDLE Web application.
Each color represents a step (in order): initialization of the reasoner (green), search for justifications (yellow), construction of the BDD, and computation of the probability of the query (purple). To see how much time each step took, the user can hover the segment with the mouse pointer or touch it in case a touch screen is used. When running TRILL, the two segments for the justification search and probability computation are replaced by a single segment considering both operations. This is due to the fact that TRILL builds the BDD and computes the probability internally. Now the user can (step 4, Figure 6): (1) change the parameters of the query, (2) change the query type, (3) modify the KB in the text area and re-load it, or change the KB by uploading a new one. All the computation is done server-side, in this way it is possible to change the parameters of the queries without reloading the KB at each operation.
The entire workflow of the interaction between client and server is shown in Figure 6. The application applies a timeout on the execution of the query. This timeout was set to 180 s, at the end of which BUNDLE shows the justifications found so far and computes the probability of the query based on these justifications. When the timeout is reached, a message warns the user about the fact that not all the justifications may be shown and that the probability may be a lower bound of the effective probability (Figure 7). The motivation behind the use of a timeout is that BUNDLE is a testing tool useful for developing and experimenting, but it is not suitable for heavy computations, for which a local installation should be used. 31 However, the application is extremely useful to perform a comparison between the embedded reasoners. For example, if we consider the following probabilistic KB for 1 ≤ i ≤ n, where n ≥ 1: the query Q = B 0 ⊑ B n has 2 n justifications, even if the KB has a size that is linear in n. For n = 2 for example, there are four different justifications:  where the C i,j s stand for the atomic choices (C i,j , 1), while with n = 8 there are 256 justifications. Table 2 shows the average running times to solve the query Q with the different reasoners with n = 2, 4, 6, 8. Times are averaged over 5 runs for each reasoner for each size of the KB. As shown in Table 2, Fact++, HermiT, JFact, Pellet, and TRILL can find all the justifications for the KBs with n equals to 2, 4, and 6 (with 4, 16, and 64 justifications respectively and query probability of 0.25402, 0.06452, and 0.01639 respectively). On the other hand, with n = 8 HermiT could not find all the 256 justifications within the given time limit of 180 s. In this case, while the exact probability P(Q) = 0.00416, HermiT computes the lower bound P(Q) = 0.00398 from F I G U R E 8 Axioms from BRCA. They state that the lack of exercise may increase the risk of having breast cancer as well as having the first child at old age. 147 justifications (HermiT was able to find 147 justifications on average in 180 s). Each justification has a probability of 2.8e −4 , as shown in Figure 7.

Examples
In this section more examples of queries available in the application are presented: • Probabilistic reasoning with a medical ontology, where the subjective probabilities representing risk factors for a certain individual can be combined with objective probabilities representing the statistical knowledge. This ontology is the "Breast Cancer Risk Assessment" (BRCA) † † whose original version was presented by Klinov and Parsia: 46 it states the risk factors of breast cancer depending on several factors such as age, drugs taken, ethnicity and so forth. It is a very challenging ontology to reason with and the creators state 46 "that reasoners that can handle this ontology will work for an interesting range of applications". BRCA has  (D) expressiveness and contains 491 axioms, 154 different classes, and 15 different properties (12 object properties and 3 data properties). The development of the ontology was driven by the idea that risk assessment were reduced to probabilistic entailment by introducing probabilistic axioms, defining for example that women aged 20-30 are unlikely to develop the disease in the short term: 0.004 ∶∶ WomanAged2030 ⊑ WomanUnderShortTermBRCRisk. Figure 8 shows that known factors for the disease are lack of exercise or having the first child at a late age. They can be encoded using the following axioms: In the version available in the Web application, we added an individual, "Helen", aged 40-50. If we query the KB to know the risk of developing breast cancer in her lifetime (Q=Helen:WomanUnderLifetimeBRCRisk) we obtain a probability value of 0.123 with 4 different explanations, as shown in Figure 5, where the reasoner HermiT was used. Instead, if we query to know Helen's short term risk (Q=Helen:WomanUnderShortTermBRCRisk) we obtain the following explanation with probability 0.025, telling us that a woman aged 40-50 has an overall low risk to develop breast cancer ‡ ‡ :  Figure 9 shows the results obtained by the Web application.
A KB is inconsistent if it contains contradictory information, so no parameters are required for running this kind of query. Consider the KB of Example 2: if we add the following axioms ∃killed.⊤ ≡ Killer, Killer disjointWith GreatMan, we will obtain that raskolnikov is both a killer and a great man, leading to a contradiction. As shown in Figure 10 A class C is unsatisfiable when the presence of at least one individual in class C forces any model of the ontology to contain a logical contradiction. In the previous example, the class Killer is unsatisfiable. If we remove from the KB all the individuals (raskolnikov, lizaveta, alyona) or just the individual belonging to Killer the resulting KB is consistent, because it does not contain any contradiction. With BUNDLE, it is possible to get the justifications explaining why a class (or more classes) is unsatisfiable, as shown in Figure 11. In this case, the justification F I G U R E 9 BUNDLE used to find justifications for the query Q = Helen ∶ WomanUnderShortTermBRCRisk w.r.t. the BRCA KB. Note that in this KB the class Nihilist is satisfiable because there may be individuals belonging to it that did not kill anyone. This is because of the use of subsumption, which tells that if an individual killed someone, she/he belongs to the class Nihilist but the vice versa is not always true (there can be individuals belonging to Nihilist that did not kill anyone). On the other hand, a Killer belongs also to class ∃killed.⊤ and vice versa because of the equality of the two concepts. Differently, if an individual is a killer, it is certainly a nihilist, leading to a contradiction as seen before.

VALIDATION OF BUNDLE WEB APPLICATION
To validate the effectiveness and practical impact of the application we prepared a questionnaire to be administered to the users of the application. The questionnaire was divided in two sections, one about usability and one targeted to the specific features of the application. The link to the questionnaire and to the application was published on several international mailing lists, both related to Semantic Web or, more generally, to artificial intelligence. In this way, we ensured we could reach interested researchers who could give more appropriate and accurate answers. We collected 13 answers, despite the number of accesses to the application was 77 in the same period of time. Knowing in advance the difficulty to collect responses for such questionnaires, the first section of the questionnaire was prepared based on the Post-Study System Usability Questionnaire (PSSUQ) Version 3 template, 36,37 also called Computer System Usability Questionnaire (CSUQ). PSSUQ has a high reliability, 47 even with a small number of participants: for example, Tullis and Stetson 48 found that a sample size of 12 answers generated the same results as a larger sample size 90% of the time. PSSUQ Version 3 consists of 16 questions following a 7-point Likert Scale, that is, every question has 7 options to choose from on a scale between Strongly Agree (score 1) to Strongly Disagree (score 7): 1. Overall, I am satisfied with how easy it is to use this system. 2. It was simple to use this system. 3. I was able to complete the tasks and scenarios quickly using this system. 4. I felt comfortable using this system. 5. It was easy to learn to use this system. 6. I believe I could become productive quickly using this system. 7. The system gave error messages that clearly told me how to fix problems. 8. Whenever I made a mistake using the system, I could recover easily and quickly. 9. The information (such as online help, on-screen messages, and other documentation) provided with this system was clear. 10. It was easy to find the information I needed. 11. The information was effective in helping me complete the tasks and scenarios. 12. The organization of information on the system screens was clear. 13. The interface of this system was pleasant. 14. I liked using the interface of this system. 15. This system has all the functions and capabilities I expect it to have. 16. Overall, I am satisfied with this system.
We then added 4 extra questions for the second section, explicitly written for our application and following the same 7-point Likert Scale: 1. I was able to use the interface without the help of the tutorial.  A v g ± std. dev.
1. Overall, I am satisfied with how easy it is to use this system.

1,63
Note: The last column reports the average score for each question and its standard deviation. TA B L E 4 Score obtained in each sub-scale averaged on the collected answers. Every question was associated with a text field allowing the user to comment the score given to the answer if greater or equal than 4. Moreover, a final free text field was added to give the possibility to the users to write suggestions or comments about the BUNDLE Web Application.
The PSSUQ provides an "overall score" by averaging the scores of all 16 items. Moreover, it also has 3 sub-scales, namely System Usefulness considering items 1-6, Information Quality considering items 7-12, and Interface Quality considering items 13-15. Note that the 16th item is used only in the global score. By adding 4 more questions, we added two scores: an Overall Score considering all the 20 questions, and a System Features score considering items 17-20. All the scores range between 1 and 7, the lower the better. Table 3 reports the distribution of the scores for each question, while Table 4 reports the sub-scale and overall scores computed by averaging the collected answers. All the scores take on low values, indicating a general appreciation for the tool, with in most case a standard deviation near 1.0. This means that the application is easily usable and that most users are satisfied with the features and examples proposed, so it could indeed promote an effective spread of Probabilistic Semantic Web by means of an easy and ready-to-use reasoning system accessible from a web browser.
It is worth noting that, together with the scores, we decided to collect also "free" comments in order to improve the interface. Some of them suggested interesting features that we are going to implement as a future work: improvements to the graphic interface, to the indication of errors, filters in dropdown menus, more indication on how specify the queries, support for browsing the ontology and for composing a query. Both the available tutorial and examples are considered sufficiently useful and interesting, while regarding the necessity to add more real life examples there is more variability in the answers, thus two new ontologies have been added: BioPAX Level 2, § § a well-known biological ontology modelling metabolic pathways, and Good Relations, ¶ ¶ an ontology for the exchange of e-commerce information, that is, data on products, offers, points of sale, prices, terms and conditions, on the Web. The project is supported by, among others, Google, Yahoo, Bing, and Yandex as part of their schema.org initiative.

CONCLUSIONS AND FUTURE WORK
In this paper we presented a Web application based on the BUNDLE framework. BUNDLE computes the probability of a query w.r.t. DISPONTE KBs by first searching for all justifications for the query by means of several ontological sub-reasoners: it can exploit state-of-the art systems such as Fact++, JFact, HermiT, and Pellet to collect the justifications, and then compute the probability of the query by means of Binary Decision Diagrams. Alternatively, it can call the TRILL reasoner to get both the set of justifications and the probability of the query.
With this application we hope to further popularize the Probabilistic Semantic Web and to spread the adoption of its tools, by giving the opportunity to perform inference without the trouble of installing software on a machine. § § Containing 2331 axioms. More information on the project at http://www.biopax.org/. ¶ ¶ Containing 1141 axioms. More information on the project at https://www.heppnetz.de/projects/goodrelations/.
Since its first publication online, in June 2020, the application has seen many improvements. Moreover, up to the 1st of February 2023 a total of 681 different users visited the application from all over the world, with 1030 sessions. From July 2022 to February 2023, we observed 270 users with 417 sections and a mean session time of 6 min and 28 s, demonstrating the increasing interest in the tool and in its capabilities. Thanks to the usability questionnaire results, improvements to the application are being developed at this time.
The BUNDLE Web application is available at https://bundle.ml.unife.it/.

ACKNOWLEDGMENT
This work was partly supported by the "National Group of Computing Science (GNCS-INDAM)".

DATA AVAILABILITY STATEMENT
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.