Explainable AI methods in cyber risk management

Artificial intelligence (AI) methods are becoming widespread, especially when data are not sufficient to build classical statistical models, as is the case for cyber risk management. However, when applied to regulated industries, such as energy, finance, and health, AI methods lack explainability. Authorities aimed at validating machine learning models in regulated fields will not consider black‐box models, unless they are supplemented with further methods that explain why certain predictions have been obtained, and which are the variables that mostly concur to such predictions. Recently, Shapley values have been introduced for this purpose: They are model agnostic, and powerful, but are not normalized and, therefore, cannot become a standardized procedure. In this paper, we provide an explainable AI model that embeds Shapley values with a statistical normalization, based on Lorenz Zonoids, particularly suited for ordinal measurement variables that can be obtained to assess cyber risk.

the ordinal classification of risks prevents the calculation of the value at risk. Although ordinal data cannot be used to calculate the value at risk, they can be used to rank risks by their "criticality," so as to prioritize interventions and, therefore, trigger mitigating actions. To our knowledge, there are very few papers that suggest how to deal with ordinal cyber data. Exceptions, that are however limited to specific issues, are Afful-Dadzie and Allen, 10 who focus on the problem of the scarcity of available data, and Hubbard and Evans, 11 Sexton et al., 12 Hubbard and Seiersen, 13 and Facchinetti et al. (2019), 14 who introduce descriptive scoring methods.
We propose to fill this gap in the literature, providing an explainable machine learning model aimed at accurately predicting the ordinal severity levels of cyber risks. To achieve this goal, we develop a methodology that combines rank-based regression models with a rank-based Shapley value approach. We test our model on a real data set of cyber events, ordered by severity levels. The application shows that the proposed methodology is both accurate, from a predictive viewpoint, and interpretable, from an explainable viewpoint. In addition, the usage of Lorenz Zonoids to assess model performance allows to obtain results more robust with respect to data quality issues, which may lead to outlying observations. The paper is organized as follows: the next section contains our proposal; Section 3 contains the empirical findings obtained applying our model to real cyber data; finally Section 4 contains some concluding remarks.

METHODOLOGY
Our proposal derives from the combination of two research streams. The first one concerns the development of models to analyze ordinal data arising in the cyber risk setting. The second one concerns the development of explainable methods to understand the results of advanced learning models. The result of the combination is a novel method for cyber risk management, which is, at the same time, predicitively accurate, interpretable, and robust.

Rank regression models in cyber risk management
As the cyber events are typically rare and not repeatable, it is quite natural to measure them with a less demanding ordinal approach rather than using quantitative data, which are often not available. Ordinal data for cyber risk measurement can be summarized, by means of a pair of statistics for each event type: the frequency of the event, how many times it has occurred, in a given period; and the corresponding severity, the mean observed loss. In the context of ordinal data, the severity can be expressed on an ordinal scale, characterized by distinct levels, arranged according to the corresponding magnitude. To understand the main factors impacting on cyber risks, each observed severity can be associated to a vector of explanatory variables, such as the type of attack, the technique of the attack, the victim type, and the geographical area where the event has occurred. The statistical models typically used to explain an ordinal response variable with a set of explanatory variables are the ordered logit or probit models (see, for instance Refs. 15 and 16). These, however, may be difficult to summarize and interpret, especially in applied contexts. We therefore resort to a linear regression model for a response variable that takes ordinal values and, in order to avoid an arbitrary assignment of the measurement scale, we resort to ranks.
Let be a response variable, expressed through ordered categories. A rank 1 = 1 to the smallest ordered category of and a rank ( −1 + −1 ) to the following ordered categories, where −1 is the absolute frequency associated with the ( − 1)-th category and = 2, … , , are assigned. Based on this transformation, the phenomenon described by the variable can be reformulated in terms of its ranks , where: with 1 = 1, 2 = 1 + 1 and = −1 + −1 . Given explanatory variables ( 1 , … , ), a regression model for can be specified as follows: whose unknown parameters can be estimated by the classical ordinary least squares (OLS) method.

The Shapley-Lorenz decomposition in cyber risk management
When dealing with data coming from highly regulated fields, such as energy, finance, and health, we may resort to simple or complex machine learning models. Simple machine learning models, including linear or logistic regression models, are highly interpretable but provide a limited predictive accuracy. Complex machine learning models, such as neural network models and decision tree models, fulfill the requirement of high predictive accuracy at the expense of interpretability. In order to meet both the conditions of predictive accuracy and interpretability, the idea is basically to boost accurate machine learning models with novel methodologies able to explain the predictive output. Recently, Giudici and Raffinetti17 have proposed a global explainable artificial intelligence (AI) model, named Shapley-Lorenz decomposition, which combines the interpretability power of the local Shapley value game theoretic approach (see, e.g., Ref. 18) with a more robust global approach based on the Lorenz Zonoid model accuracy tool (see, e.g., Ref. 19). The Lorenz Zonoids can be seen as a generalization of the receiver operating characteristic (ROC) curve in a multidimensional setting and, therefore, the Shapley-Lorenz decomposition has the advantage of combining predictive accuracy and explainability performance into one single diagnostics. Furthermore, the Lorenz Zonoid is intended as a measure of the mutual variability, robust to the presence of outlying observations, and can be exploited to develop partial dependence measures that allow to detect the additional contribution of a new predictor into an existing model. Shapley values were introduced as a pay-off concept from cooperative game theory. When referring to machine learning models, the notion of pay-off corresponds to the model prediction. Thus, for any single statistical unit (1 = 1, … , ), the pay-offs are defined as wherê( ′ ) denotes the predicted values generated by the machine learning models depending only on ′ predictors; ( ′ ∪ ) denotes the predicted values generated by the machine learning models depending both on the | ′ | predictors and the additional included predictor. For a set of statistical units ( = 1, … , ), the pay-off notion translated in terms of Lorenz Zonoids ( (⋅)) is given by wherê′ ∪ and̂′ are the vectors specifying the predicted values generated by the machine learning models, which include the additional explanatory variable , and the predicted values generated by the machine learning models, which do not include the explanatory variable , whereas (̂′ ∪ ) and (̂′ ) describe the (mutual) variability of the response variable explained by the models including the ′ ∪ predictors and the ′ predictors, respectively.
The Shapley-Lorenz decomposition expression is the result of a combination between the Shapley value-based formula and the Lorenz Zonoid tools. Formally, the contribution of the additional variable , expressed in terms of the differential contribution to the global predictive accuracy, equals to where (̂′ ∪ ) and (̂′ ) measure the marginal contribution provided by the inclusion of variable ; is the number of available predictors; ( ) ⧵ is the set of all the possible model configurations that can be obtained with − 1 variables, excluding variable ; | ′ | denotes the number of variables included in each possible model. We remark that (̂′ ∪ ) and (̂′ ) in Equation (5) can be expressed as function of the covariance operators, that is, (̂′ ∪ , (̂′ ∪ )) and (̂′ ) = 2 (̂′ ) (̂′ , (̂′ )), where (̂′ ∪ ) and (̂′ ) are the expected values of̂′ ∪ and̂′ , respectively; (̂′ ∪ ) and (̂′ ) are the rank scores associated with thê′ ∪ values and thê′ values. Due to its building characteristics, the Shapley-Lorenz decomposition presents as an agnostic eXplainable AI method, which can be applied to the predictive output, regardless of which model generated it. This feature makes it suitable in all the contexts where response variables with different nature are involved. The focus on cyber risk data represents an example of application in the presence of an ordinal target variable. We remark that in such a case the response variable is transformed into ranks according to Equation (1).

APPLICATION
The purpose of this application is to evaluate the performance of our cyber risk measurement proposal, based on the combination between rank regression models and the Shapley-Lorenz decomposition approach. We employ the Clusit cyber loss database,4 which consists of 6865 worldwide observations on serious cyber attacks, in the years 2011-2017. An attack is classified as "serious" if it has led to a significant impact, in terms of economic losses and/or damages to reputation. In this paper, we focus on a sample data, consisting of 808 cyber attacks observed in 2017, the year in which most data were observed. Severity levels are reported according to the type of attacker, technique of attacks, victims, and the corresponding continent of origin. Moreover, given the data at hand, we evaluate the model on the full sample, without splitting it into training and test sets.
We remark that the data, similarly to all cyber database available, may contain outlying observations, which can derive either from intrinsic characteristics or from measurement errors. In both cases it is necessary that the developed machine learning models are robust to outliers and data variations. This aspect suggests the use of rank-based regression models, on one hand, and of model assessment performances, which are similarly robust to data anomalies.
In terms of descriptive statistics, Tables 1 and 2 report the frequency distribution of the cyber loss severity, and of the considered four explanatory variables.
Our purpose is to detect the factors, among attacker, attack technique, victim type, and location (continent), which most affect the severity levels. To achieve this aim we first have applied our proposed rank regression model. From a descriptive viewpoint, the 2 is equal to 0.6183, and the -value of the associated -test is smaller than 0.001. In Table 3 ), are presented. We break each of the four categorical variables into dummies, with the baseline cases being "Cybercrime" for type of attacker, "Africa" for continent, "Automotive" for victim, and "0day" for attack technique. Together with the linear regression coefficients, the related -values are also provided, showing that the geographical area, where the cyber attack occurs, has not a significant impact on its severity degree. Thus, the continent variable can be removed from the full model in favor of a more parsimonious model. In addition, the only significant effects at a significance level = 5% are: espionage/sabotage, hacktivism and information warfare for the type of attacker variable; entertainment/news, GDO/retail, online services/cloud, and research-education for the victim-type variable; phishing/social engineering and unknown for the attack technique variable.
In general, by looking at the estimated linear regression coefficients, the different cyber attack levels have the effect of decreasing the severity degree with respect to the baseline of "Cybercrime." On the contrary, the levels characterizing the attack technique and the victim type have the effect of increasing the severity degree with respect to the baselines of "0-day" and "Automative," respectively.
Although the rank regression model appears explainable by definition, it is the results of a model selection procedure whose obtained coefficients are conditional on the single chosen model. Differently, the Shapley value approach provides a measure of explainability for each single feature variable, which is based on the consideration of all possible model  On the other hand, the local Shapley values can be computed in accordance with Equation (5), by replacing the Lorenz Zonoids included in the square brackets with the pay-off in Equation (3). Note that the latter is based on a euclidean distance between predicted values under different models, differently from the Shapley-Lorenz values, based on the Gini distance, more suited to deal with ordinal variables, and more robust to outliers. We further remark that, if on the one hand,  Table 4.
From Table, 4 note that, according to the Shapley-Lorenz values, the variable describing the type of victim provides the highest marginal contribution in the prediction of cyber severity, across all the possible model configurations. A further impacting variable is associated with the type of attacker, while variables with the lowest contributions are those representing the attack technique and the continent, which is intended as the geographical area where the event has occurred.
More precisely, the continent variable gives the minimum contribution to the explanation of the severity degree associated with the cyber attacks, confirming the findings derived from the application of our proposed rank regression model, according to which the continent variable is not significant.
In the Shapley-Lorenz approach perspective, the type of victim and attacker variables explain the 11.5% and the 7.2% of the mutual variability associated with the cyber attack severity degree over all the possible model configurations, respectively. The type of technique variable explains 5.8% and the continent variable only 3.2%. From an interpretational point of view, this indicates that preventive actions and mitigation measures, such as insurance coverage, should vary according to the type of victim (in our case economic activity types) rather than on the attack technique and/or on the location of the victim.
From Table 4 also note that the Shapley values, being not normalized, are more difficult to be interpreted in terms of the variables' contributions. The variable impacting more on the severity degree of the cyber attack is the type of attack, while the variable with the least effect, coherently with which emerges from the Shapley-Lorenz-based approach, is the continent variable. In addition, the technique of attack seems to explain more than the victim type, contrary to what happens when the Shapley-Lorenz values are considered. These discrepancies may be motivated by the Shapley value construction, which, as previously discussed, involves the sum of the deviation of each variable's Shapley value from the overall mean, and consequently is less robust to the presence of outlying observations. As our experiments show, this issue can be appropriately overcome by the implementation of the Shapley-Lorenz approach.

CONCLUDING REMARKS
The paper proposes a new methodology to assess cyber risks, using loss data at an ordinal scale, easier to acquire with respect to continuous data. Consistently with the ordinal nature of the data, the proposed methodology is based on a combination between rank regression model fit and Lorenz-based assessment models.
The combination of the two approaches leads to the identification of the drivers of cyber risk, which are more important to control and mitigate with insurance.
The application of the proposed method to the available data confirms that the proposed method is quite satisfactory, and provides an accurately predictive, explainable, and robust machine learning method for cyber risk management.

N O T E
1 It is worth noting that the rank scores (̂′ ∪ ) and (̂′ ) are not connected with the ranks appearing in Equation (1). Indeed, through Equation (1) the ordinal target variable is transformed into a discrete quantitative variable through the employment of ranks, which, contrary to the rank scores (̂′ ∪ ) and (̂′ ), which denote the positions of thê′ ∪ and̂′ values, are computed according to the procedure suggested in Section 2.1.

A C K N O W L E D G M E N T S
The authors thank the European Network for Business and Industrial Statistics (ENBIS) for useful suggestions and comments during the webinars organized with the Special Interest Group on Risk Management, coordinated by the first author. The work of the authors is receiving support from the European Union's Horizon 2020 training and innovation programme "FIN-TECH," under the grant agreement number 825215 (Topic ICT-35-2018, Type of actions: CSA). The paper is the result of the joint collaboration between the two authors.