How to apply responsible leadership theory in practice: A competency tool to collaborate on the sustainable development goals

Correspondence Katrin Muff, LUISS Business School, Rome, Italy. Email: katrin.muff@gmail.com Abstract In the era of the sustainable development goals (SDGs) and calls for climate action, business is challenged to respond more effectively to societal and environmental challenges. Collaboration with stakeholders requires an ability for broader collaboration competencies. These form a part of an established literature on responsible leadership (RL) competencies. Human resources managers, consultants and educators in charge of developing such competencies demand clarity on the definition and a practical measurement tool for RL. This paper addresses both by proposing a RL definition and a RL competency model, that has been operationalized into a free online tool for individuals and groups. This free Competency Assessment for Responsible Leadership (CARL) tool is used to assess stakeholder engagement, individual and group leadership development, and internal sustainable development transformation work. The CARL online tool serves to systematically analyze and develop RL competencies, both in business and educational practices, and helps advance the RL theory based on insights from practice.


| INTRODUCTION
Ever since the financial crisis of 2008, the call for responsible leaders in and beyond business has become louder. The continuing string of corporate scandals have all but grown. The ethical-based scandals at Siemens, Volkswagen, UBER and many more have shown that business has not learned its lesson from scandals such as ENRON. This endless series of scandals has damaged the trust in companies and business in general. Some meanwhile suggest that it was the misconduct of leadership and irresponsible leaders that were the main causes for the financial crisis (Pless & Maak, 2011). In the past few years, significant societal changes including the global MeToo and climate movements have resulted in further demands for regulatory requirements for the finance industry in particular, and business in general.
The call for responsible leadership in association with good corporate citizenship has been sharpened by the introduction of the sustainable development goals (SDGs) in 2015. The understanding that business plays a key role in resolving societal issues has shaped the expectations in civil society that business must take into consideration public interests, not just private interests (Dyllick & Muff, 2016).
Such expectations pose some difficult questions regarding the focus of business leaders: Should they focus predominantly on internal concerns or should they focus more on external, societal concerns? Which stakeholders should they give preference and how should they go about in answering this question? Any attempt to answer these questions quickly demonstrates a need to move beyond a simple "right or wrong" thinking. These questions point to a need to move beyond a simplistic "either -or" thinking to embrace the systemic conditions of wicked problems that require an "and -and" thinking capacity. The requirement to deal with conflicting or contradicting stakeholder interests beyond and outside traditional organizational, boundaries asks for the ability to deal with moral dilemmas and the capacity to behave in fair and ethical ways (Maak & Pless, 2006a;Maak & Pless, 2006b). The concept of virtuousness in organizations with virtuousness being defined as representing the best of the human condition, adds further depth to this idea (Cameron & Winn, 2012). Early leading companies, such as Danfoss in Denmark, understood that the 2008 financial crisis cannot impede the implementation of CSR initiatives, but rather provides an opportunity to walk-the-talk (Lehmann, Toh, Christensen, & Ma, 2010).
Business practitioners are looking at scholars for guidance and support in clarifying the requirements for responsible leadership (RL).
We have been repeatedly approached to come up with a clear and simple definition of RL that can be used for personal and leadership development purposes (Muff & Mayenfisch-Tobin, 2014). These demands include in particular three questions: 1. Determining the current state of RL competencies-for an individual or for a group.
2. Assessing the short and long-term effectiveness of RL competencies development.
3. Evaluating existing training offerings in terms of their RL impact.
Back in 2006, the World Business Council for Sustainable Development (WBCSD) engaged a taskforce of business leaders to map out "tomorrow's leaders," putting responsibility and long-term success measures at the beginning and end of strategy and stakeholder engagement (WBCSD, 2011). It had become increasingly clear, that RL could help provide a better understanding of what might be required from leaders to improve how society thinks about them and their firms (Pless, Maak, & Waldman, 2012).
This paper provides an answer to these specific demands and presents an online-tool tool for RL competency assessment together with the process used and steps taken to get there. Boyatzis (2008) has argued with conviction that the competencies needed to be effective in the 21st century can indeed be developed, supporting our ambition to not only define but also measure these. Such a tool serves practical purposes, and will allow more and better generation of data that shall seek to advance the scholarly discourse. The perspective applied is conceptual and it uses comparative analysis. The paper is structured as follows: Section 2 gives a brief overview of the literature with the goal to come up with a clear definition and framework (grid) for RL. Section 3 operationalizes the RL grid into a comprehensive model of 45 competencies across 15 aspects that form the operationalizing RL competencies into a survey. Section 4 demonstrates how the operationalized RL model is translated into an advanced online tool.
Section 5 summarizes the prototype development process that accompanied the operationalization and online tool development.
Section 6 reviews the three questions by practice and discusses potential applications in business and education. Section 7 draws some conclusions, hints at limitations and further research needs.

| TOWARD A DEFINITION OF RESPONSIBLE LEADERSHIP
Under the heading of RL we find a connection of two different fields of research; research on corporate social responsibility and research on leadership, thereby combining the organizational and the individual level (Waldman, 2011). The increasing research in the past two decades can be categorized into four distinct categories: teaching responsible management, organizing for responsible education, responsible individual learning, and responsible organizational learning (Cullen, 2019). This article focuses on responsible individual learning.
This section reviews the scholarly discussion in the area of RL over time and identifies common themes in an attempt to consolidate the insights into a clear RL definition for individuals and groups. The review shows how five competency dimensions of RL emerged in the literature: creating, managing and securing good relations with multiple stakeholders, ethically correct and values-based behavior, a continuously developed self-awareness, good understanding of the interdependencies of a larger system, and the ability to lead change and innovation towards sustainable development. In order to assess these dimensions from a developmental perspective, three action domains will be added to complete the proposed two-dimensional framework. "Thus our definition of responsible leadership is that leaders are those who take responsibility towards the bottom-line and shareholders of the organization, while at the same time -through reconciliation -take responsibility towards integrating a diverse workforce, multicultural customers and suppliers, local and global communities, NGOs, environmental concerns and society at large. These leaders recognize and respect multiple demands, interests, needs and conflicts stemming from diverse responsibilities and reconcile them by mobilizing and successfully engaging the organization and varying stakeholders." Maak reinforced the importance of stakeholder relations and contributed a new element to the discussion, namely the active engagement of responsible leaders also outside their organization, in society: building, cultivating and sustaining trustful relationships to different stakeholders, both inside and outside the organization, and in coordinating responsible action to achieve a meaningful commonly shared business vision" (Maak, 2007). Pless further substantiated the idea of active civil engagement and suggested the need and capacity for leading change by embracing the idea of effectiveness as a further element to the catalog of requirements for RL: "a person who reconciles the idea of effectiveness with the idea of corporate responsibility by being an active citizen and promoting active citizenship inside and outside the organization" (Pless, 2007). Pless also embraced earlier definitions of ethically correct and values-based behavior and action and connected these with the idea of creating social change and sustainable value: "values-based and ethical principles driven relationship between leaders and stakeholders who are connected through a shared sense of meaning and purpose through which they raise one another to higher levels of motivation and commitment for achieving sustainable value creation and social change" (Pless, 2007). This notion of capacity for change and active civil engagement was further strengthened by Quinn and D'Amato who added the importance of creating systemic change towards sustainable development: "Globally responsible leadership asks business organizations to pay attention to the impact their operations have on the planetrequiring a systemic view and addressing diverse economic, social and environmental issues." (Quinn & D'Amato, 2008).
GRLI's renewed and updated call for action for globally responsible leadership in many ways reflected the discussions of these initial years.
It highlighted the interconnection between a good understanding of system interdependencies and values-based action that is aligned to social progress: "Responsible leadership implies the grounding of actions in a system of values, which recognize societal interdependence and long-term sustainable development. A truly integral perspective considering a responsibility for the organization and the broader system in which it operates is of essence (Küpers, 2011). If the firm wishes to lend meaning to its actions and wants to give a purpose to economic progress by aligning it to societal progress, ethics are essential to enlighten tough choices and to guide behavior. The main ethical question for our time is to choose what kind of world we want to build together with the immense resources we have at our disposal" (GRLI, 2008). Quinn and D'Amato's focus on company internal activities also rang true with GRLI, which stressed the internal leadership dimension of RL highlighting the demands of being an effective change agent within one's organization: "Leadership is the art of motivating, communicating, empowering and convincing people to engage with a new vision of sustainable development and the necessary change that this implies. Leadership is based on moral authority. Moral authority requires convictions, character and talent" (GRLI, 2008). With this duality of internal and external focus, there was another emerging duality expressed by Dassah who points toward the duality between short-term economic thinking and long-term sustainability: "Responsible leaders see beyond their organisations, anticipate and embrace socio-environmental concerns and go beyond short-term profit to long-term sustainability as the ultimate mark of success." (Dassah, 2010).
Subsequently, two additional perspectives were added. Mirvis and colleagues DeJongh, Googins, Quinn and Van Velsor who presented a multidimensional perspective of RL, suggesting three different levels of RL In addition to the traditional individual level they suggested an organizational and societal level: "Responsible leadership is a function of individual leader (the "Me"), of responsible organizations (the "We"), and of responsible business in the larger ecosystem of investors, consumers, competitors, regulators, and other interests (the "Us") that provide a context for and also have to act responsibly to legitimate and sustain responsible business leadership." (Mirvis, DeJongh, Googins, Quinn, & Van Velsor, 2010). This idea builds on the insight developed by Bolden and Gosling (2006) who have argued for a more discursive approach that goes beyond an individual notion of leadership to include also more collective leadership aspects.
It became increasingly apparent that defining RL required also a broader look at other disciplines such as psychology (Ketola, 2010).
Vögtlin and Muff put a focus on the inner dimension of RL by pointing out the need for self-awareness and reflective capacity. Vögtlin pointed out the need for awareness and consideration of the consequences for all stakeholders: "Responsible leadership can thus be understood as the awareness and consideration of the consequences of one's actions for all stakeholders, as well as the exertion of influence by enabling the involvement of the affected stakeholders and by engaging in an active stakeholder dialogue." (Vögtlin, 2011a). Muff suggested more pointedly the need for an understanding of oneself: "Responsible leadership requires a deeper empathy and values-based ethic: an innate understanding of oneself, as well as of colleagues, organizations, communities, the environment, and how all these factors relate to one other." (Muff, 2013).
Increasingly, the descriptions of RL became more complex and comprehensive and started to shift to definitions, which may be inter- taking into account the potential, long-term impact and indirect consequences of their actions." ). Vögtlin's definition focusses strongly on the importance of stakeholder relations: "Responsible Leadership is to be understood as leading-action, which is expressed by leaders being (1) conscious of the consequences of their actions for all stakeholders and thus include these stakeholders in their actions and decisions. Furthermore, it implies that leaders exert influence with the goal to (2) encourage an active stakeholder dialogues and public exchange of opinions, to which (3) any concerned party has the opportunity to participate, with the intention that (4) the interests of concerned stakeholders are considered and balanced in a discursive process" (Vögtlin, 2011b, translated from German).
In 2015, this debate and discussion around the dimensions of RL allowed us to weave these different dimensions into a formal definition: (based on Liechti, 2014).
A responsible leader demonstrates a deep understanding of the interdependencies of the system and the own person, is distinguished by an ethical and values-based attitude, and able to build long-term relations with different stakeholders embracing their needs, while initiating change towards sustainable development.
We have since worked with this definition, which comprises the five competency dimensions briefly outlined at the beginning of this section: creating, managing and securing good relations with multiple stakeholders, ethically correct and values-based behavior, highly developed self-awareness, good understanding of the interdependencies with a larger system, and the ability to lead change and innovation towards sustainable development. Table 1  This definition is-5 years later-still relevant. The increasingly obvious societal and environmental challenges and the louder demands by civil society to embrace the climate crisis has further validated the various aspects of the definition. The world has developed into even more complexity, highlighting the need for RL even more clearly than ever before (Knight, 2018).
While the definition of RL provides an answer to the question, "What does RL mean?" or "What are the key competencies of RL?" there is a second core question we want to address: "What are the relevant domains of action?" In order to define the domains of action, on the model uses the three action domains as used by Datar, Garvin, and Cullen (2010): knowing, doing, and being. Euler and Hahn (2007) refer to them as knowledge, skills, and attitudes. By putting together these two dimensions, we obtain a two-dimensional framework, the "Responsible Leadership Grid". It includes the five competency dimensions and the three domains of action what results in 15 aspects overall (see Table 2).
These action dimensions seem to be passing the test of time with Laasch and Moosmayer (2016) pointing out the need to consider competencies in an action dynamic beyond knowing, thinking and doing to include relating, seeing, and becoming.
In the next section, the paper investigates the operationalization of these 15 competency areas into a comprehensive RL model with 45 sub-competencies.

| OPERATIONALIZING RESPONSIBLE LEADERSHIP INTO SUB-COMPETENCIES
The idea of "competencies" is not undisputed (Bolden & Gosling, 2006;Carroll, Levy, & Richmond, 2008;Hollenbeck, McCall, & Silzer, 2006) and may still be best framed by the philosophical underpinnings that consider a more integrated conception of competence. Hager and Beckett (1995) have pointed out that the tasks and the person cannot be considered separately, and that competence cannot be reduced to lists and attributes. They propose an integrated view that considers the context of activity and actor in a holistic manner.
Competence is, hence, relational rather than one-dimensional.
The question arises how these 15 aspects of RL in its five competency dimensions and three domains of action can be operationalized and later be made measurable? Based on a literature review and tested in our prototyping (see Section 5), we identified three subcompetencies for each of the 15 aspects resulting in 45 subcompetencies across the whole RL Grid. Let us look at each of the five competency dimensions one after the other to specify its nine specific sub-competencies.
Let us look first at the sub-competencies and its sources in the dimension of stakeholder relations ( being open and trustworthy, and appreciating the positive in diversity. In looking at the sub-competencies and sources in the ethics and values dimension we identified the following sub-competencies ( into being honest and integer, seeking fairness, and being responsible toward society and sustainability (serving the common good).
The sub-competencies and sources in the self-awareness dimension include the following elements (Table 5) Table 6 provides an overview of the sub-competencies and its sources in the systems thinking dimension. The knowledge domain features understanding how the systems works, understanding the interdependencies and interconnections of the system, and understanding sustainability challenges and opportunities. The skills domain is grouped into dealing with complexity and ambiguity (including working across disciplines), estimating consequences of decisions on the system, seeing the big picture and the connections rather than the parts. The attitudes domain includes working across disciplines and boundaries, defending a long-term perspective, and providing a transgenerational perspective.   Muff et al., 2013, p. 33;Maak &Pless, 2006b, pp. 37 and42, Pless &Schneider, 2006, p. 218 Acting as a role model Marquardt & Berger, 2000, p. 27;Pless & Schneider, 2006, p. 218 Attitudes Being honest and integer Svanström et al., 2008, p. 348 Seeking fairness Sterling & Thomas, 2006, pp. 360 and 363 Being responsible towards society and sustainability (serving the common good) Muff et al., 2013, p. 37, Sterling & Thomas, 2006 T A B L E 5 Sub-competencies and their sources in the self-awareness dimension Understanding interdependencies and interconnections of systems Svanström et al., 2008, p. 348;Sterling & Thomas, 2006, p. 364 Understanding sustainability challenges and opportunities Svanström et al., 2008, p. 347 Skills Dealing with complexity and ambiguity (working across disciplines) Muff et al., 2013, p. 33, Pless & Schneider, 2006, p. 217, Svanström et al., 2008 Estimating consequences of decisions on the system (identifying connections) Marquardt & Berger, 2000, p. 24;Sterling & Thomas, 2006, p. 364, Svanström et al., 2008 Seeing the big picture and the connections rather than the parts (thinking in systems) Marquardt & Berger, 2000, p. 24;Muff et al., 2013, p. 33;Scalberg, 2005, p. 383;Sterling & Thomas, 2006, p. 364 Svanström et al., 2008Wiek et al., 2011, p. 207 Attitudes Working across disciplines and boundaries Sterling & Thomas, 2006, pp. 362 and 363 Defending a long-term perspective Based on Wilson, Kendall, & Brooks, 2006, p. 20 Providing a trans-generational perspective Muff et al., 2013, p. 33 T A B L E 7 Sub-competencies and their sources in the change and innovation dimension Acting to bring about change and translating ideas into action (involving and inspiring others) Sterling & Thomas, 2006, pp. 360 and 363, Svanström et al., 2008, p. 348, Wiek et al., 2011 Questioning the status-quo and identifying steps of change for a sustainable future (visionary thinking) Muff et al., 2013, p. 33;Svanström et al., 2008, pp. 347-348;Wiek et al., 2011, p. 207 Attitudes Being open, curious and courageous Svanström et al., 2008, p. 348 Being flexible and adaptable for change Muff, 2012, p. 655 Being visionary in finding solutions for society's problems (and having endurance) Muff et al., 2013, p. 33, Svanström et al., 2008 Their technology is based on the measurement of user reaction times, taking into account reading speed.  Leadership Scale. All three online questions that were amended originated from the Playfulness survey. Table 9 provides an overview of the different survey questions' sources: A comprehensive overview of the finalized questions is provided in the next five tables below. economics online tools, and five were adopted from traditional RL offline surveys.

| PROTOTYPING THE OPERATIONALIZATION FOR THE ONLINE QUESTIONNAIRE
In order to prototype the process of operationalizing the 45 competencies into an online questionnaire and tool we used four different assessments: 1. The survey questions were tested during a two-year period in a suitable Master course at a university in order to assess the relevance of the RL dimensions in a real-life situation.
2. The survey was amended to fit the limited time availability of business practitioners in a sizable in-company assessment exercise, with the purpose to understand the usefulness and applicability of the survey results and the RL Grid in real-life.
3. The outcomes of the first and the second assessments were shared with thought leaders from academia and practice in order to highlight potential shortcomings and challenges.
4. The online tool was exposed to a Beta-test involving 103 users in order to allow the required calibration of the response rates and the generation of the automated results.

| Survey question assessment in a Master course
A Master course in the area of applied business sustainability using a student-based pedagogy was selected to test the operationalization of the RL Grid. The course was considered due to its innovative and experiential approach that was considered an ideal platform for developing responsible leadership competencies in a Master's level university course (Dyllick & Muff, 2014

TS2
• Estimating consequences of decisions on the system HEXAXO survey (Fehr) Q20 recoded I make decisions based on the feeling of the moment rather than on careful thought.

TS3
• Seeing the big picture and the connections rather than the parts Own formulation (recoded) I sometimes get lost in details.

TA1
• Working across disciplines and boundaries Own formulation (SY03) When looking for solutions I integrate insights from diverse disciplines.

TA2
• Defending a long-term perspective Playfulness survey (Fehr) Q2 I prefer to plan ahead rather than living from day to day.

TA3
• Providing a trans-generational perspective Own formulation When making decisions one should also consider future generations.
During a 2-year period, the RL operationalization was used both before and after the course as a way to test the operationalization with multiple-choice questions. The survey investigated to what degree and extent this specific course develops competencies of responsible leaders.
Both the execution and completion objectivity criteria were confirmed by ensuring identical instructions to all students when completing the survey, and the multiple-choice survey. The reliability was evaluated and confirmed through the Cronbach-alpha, albeit only slightly in some domains (the Cronbach-alpha should be significantly above 0.5 but is only slightly above 0.5 in some areas). Given the small sample size (n = 33) for the quantitative survey, the Wilcoxon test for normal distribution was not granted, as the data were ordinal rather than interval scaled. The boxplot in Figure 1 visually compares these averages of 5.05 (before) versus 5.23 (after) on a scale of 1 (entirely wrong) to 7 (entirely correct) indicating a "very significant" statistical relevance (p-value of .0087).
As it is insightful to look at the opportunities presented by using a  F I G U R E 1 Boxpot of the average self-assessment scores before and after the course [Colour figure can be viewed at wileyonlinelibrary.com] students judging their competency as lower than at the beginning of the course. Such awareness can be explained through an increasing understanding in the course about a competency resulting in the selfassessment: "Oh, maybe I didn't know as much as I thought about this particular competency." Table 16 offers a summative view of the most significant competency gains when comparing the before and after self-evaluations of those aspects with a very strong statistical relevance (p-value < .01 only).
Six aspects emerge as particularly important (in order of size of difference), covering all domains of action (knowing, doing, and being) as well as most competency dimensions (with the exception of change and innovation): In summary, the prototype that operationalized the RL definition has proven very useful and insightful in assessing responsible leadership competencies in an educational practice setting. It is encouraging to see that the prototype approach has provoked such considerations and has allowed opening the research field more widely.

| Testing the usability of the RL grid for business
A practice application was used to clarify and improve the prototype. A leading Swiss telecom provider wanted to know how their managers selfassessed their responsible leadership competencies. The company had a desire in particular to (a) undertake a RL-relatedself-assessment of its management and (b) assess their existing leadership development programs and trainings in view of their RL impact. The organization had previously undergone an internal process to define what RL meant and had subsequently engaged with Business School Lausanne to assess their findings. The prototype was translated into a company internal survey tool to be used by a representative sample (n = 89) of their three management levels (Zoppi, 2016). The survey triggered the identification of three competency areas as "blind-spots" areas, which were underdeveloped.
The RL grid with its 15 aspects and 45 sub-competencies proved an insightful tool for the company and its concern with analyzing and developing specific RL competencies. In addition, the RL grid helped the company to display their existing training offers in a clear way that highlighted overlaps and gaps. Based on a number of follow-on interviews allowed the company to translate the identified blind spots into actions to be added to their training and development priorities (Zoppi, 2016).

| Expert review of the learning and insights
A group of experts of Responsible Leadership scholars from theory and practice as well as HR representatives of the involved company met to discuss the outcome of the responsible leadership assessment in the company, in the context of corporate responsibility and business sustainability. The experts reviewed previous prototyping steps in an attempt to triangulate learning around the RL prototype. These insights as well as the interconnection with business sustainability and its transformative common space are reflected in a conceptual article on the topic (Muff, 2016).

| Beta-testing of the online tool for calibration
The Beta-test sample of 102 participants served to calibrate the social desirability factor using the association strength of the responses with the question at +0.7. Responses that occurred with an association strength of +0.7 and higher were considered as "completed," lower results were rejected due to the risk of social desirability. The Betatest furthermore served to refine the response coding per T A B L E 1 5 Overview of competency gains in the responsible leadership grid of Liechti (2014)  competency sub-dimension. A completed answer resulted in one point in a given dimension. As each of the 15 competency areas consisted of three aspects, three correct answers result in a green code (or 3 points), two in a yellow (or 2 points) and one in an orange code (or 1 point). No correct answers resulted in a red code (or zero points).
The calibration procedure has enabled us to verify the scale to attribute to the various competency level in each of the 15 competency areas, by assigning the value 1 to zero of the three elements acquired, the value 2 for having one of the three elements acquired, value 3 for two and value 4 for all three competencies acquired and developed (see Figure 2). The maximum value for all 15 competency areas would thus 15 times 4 equals 60.
We adjusted the three action domains in order to weight the varying difficulties in obtaining the different domains of action. Using a scale of 100%, we attributed 25% to "knowledge," 33% to "skills" and 42% to "attitude." Alternatively, expressed differently, "knowledge" is weighted at 60% of "attitude" and "skills" is weighted at 80% of "attitude." Using this action domains scale, we translate the original competency areas value of 60 into a scale 100 points, thus weighting the five competency areas of "knowledge" with a factor of 1.25, the five competency areas of "skills" with a factor of 1.25 and the five competency areas with a factor of 2.1 (see Table 17).
These calibrations and adjustments allow generating an overall score for the competencies of responsible leadership on a scale of 0 to 100. This provides a sample with an overall sense of where he is at a given point. Furthermore, we are providing insight into how a sample is doing as compared with the maximum value for the five competency dimensions and the three action domains (see Table 18).
Since its inception, the tool has been used widely in business education around the world.

| POTENTIAL APPLICATIONS AND USE FOR BOTH BUSINESS AND EDUCATION
There is an increasing understanding that there is a need for a fundamental mindset shift from "inside-out" to "outside-in" perspective to enable companies to contribute positively to society (Dyllick & Muff, 2016). This organizational mindset shift is strongly connected to an individual mindset shift from a "self-oriented" to a "systems-oriented" rationale. A "self-oriented" rationale is defined by mostly egotistic values combined with short-term thinking. A "system-oriented" rationale encompasses mostly altruistic values and long-term thinking (Pureza and Lee, 2020).
This research journey started with three clear demands from practice: 1. Determining the current state of RL competencies -for an individual or for a group.
2. Assessing the short and long-term effectiveness of RL competencies development.
3. Evaluating existing training offerings in terms of their RL impact.
Our prototyping suggests that the CARL is an answer to the first two points (1)  A comparative analysis of the before and after assessments, completed by 10 and 8 participants respectively, allows the participants and lecturer to observe the following developments (see Table 19): • The course brought an overall increase in responsible leadership competencies of 16%.
• The most change in the competency dimensions occurred in terms of "stakeholder engagement" and "self-awareness" (33 and 34%, respectively). have contributed to significant positive changes. Participants as well as interveners are excellent sources for finding potential avenues of answers to this question. Given a change in the number of respondents, we suggest not drawing conclusions from variations below 20% change; these may have causes that are beyond the means of analysis available here. We have noted that negative change does not necessarily imply a loss of a competency, but reflects the increase of awareness of a participant because of having been exposed to certain concepts, experiences, or reflections which may result in participants rating their knowledge, skills or competencies lower than previously estimated in any given dimension. It is therefore not necessarily appropriate to question too much a reduction of a competency, unless before-after invention show consistency in such data over time.
As indicated, pedagogically trained and experienced faculty may be interested to look at a class of students take a "before-after" assessment, in order to understand blind spots and learning opportunities at the beginning of a course. Such insight may serve as an additional measure to understand the impact of any learning intervention over a given period at the end of the course. Given the current limitations of suggestions provided by the tool in terms of how to improve a current level of responsibility, it is considered premature to share course-based surveys with faculty that is not specifically trained in understanding the impact of a preassessment of a class prior to teaching. It has been suggested that such pre-assessments highly influence a teacher (positively and negatively) and it remains yet to be better understood how we can set up this survey to limit unintended negative consequences of such sharing of data at a course level.

| Evaluating a broad range of existing training offerings in terms of responsible leadership impact
The experience with the Swisscom company case has shown that by using the RL grid, the human resources team was able to assess the effectiveness of their large training and development offer in accordance to the 15 areas identified, allowing them to understand which dimensions where not addressed at all and where they may have overlapping offers that could be optimized in a next phase.
With Millennials entering the workforce, it becomes clear to what degree this new generation is attracted to sustainable companies (Alonso-Almeida Del Mar & Llach, 2018). This underlines not only the need but also the opportunity for companies to assess how it can upgrade its training to develop RL across its organization and leaders at all levels.
F I G U R E 4 Legend on next page.

T A B L E 1 9
Before-<span></span>after<span data-pi-mid="915"></span> comparison of a sample MBA course [Colour table can be viewed at wileyonlinelibrary.com] As a result of a number of business schools around the world have started using this assessment, it has been suggested, that this online tool might well be way to answer to the question posed by the U.N. Principles of Responsible Management Education (UN PRME), namely to what degree do you actually educate responsible leaders.
Business School Lausanne in Switzerland, for instance, has started to do a survey for all incoming students in all programs (bachelor to doctoral) since September 2016 and at the end of studies across all the same programs. They are using the CARL in combination with the SuliTest (www.sustainabilitytest.org) as a way to assess progress of students in two dimensions of their vision: responsibility and sustainability.
It is our hope that the work reflected in this paper and the resulting CARL tool which has been in use for nearly 4 years now, contributes to the development of more responsible leaders both as continuing development at the firm and during a more formal business education.
In a time where companies are challenged to not only reduce their negative footprint but also to focus on increasing their positive handprint, a model and tool that measures both the "avoid harm" and the "do good" behaviors of leaders and managers (Stahl & De Luque, 2014) must serve-so we trust-a positive impact of business.

| CONCLUSIONS, LIMITATIONS, AND FURTHER RESEARCH
This paper demonstrates that the three questions posed by Managers can indeed be answered: 1. The CARL provides a pragmatic and rapid way to determine the state of existing RL competencies of individuals and groups.
2. The CARL assesses the short and long-term effectiveness of dedicated leadership development for a team, division or entire company; and also for a single class, course, or entire program; 3. The RL Grid with its 15 areas provides a pragmatic way of evaluating existing training offerings in terms of RL impact.
The paper, which has existed for a long period as a white paper, has also served to contribute to one of the three conceptual shifts in the field of responsible management studies, namely the shift that transformed the CSR and business ethics discussion from an organizational to an individual level, where we look at the responsible manager (Laasch, 2018).
The practical application of RL shall also serve the need for an entrepreneurship education. Lans, Blok, and Wesselink (2014) point out that there is a need for more work in linking sustainability education and entrepreneurship education.
Last but not least, this article seeks to contribute to the development of measures and assessment, one of five areas that require further research in the field of responsible leadership theories, as pointed out by Waldman and Balven (2014).
Future data that will be compiled with the current and future use of the online tool CARL will allow the further refinement and development of how we best define RL. The tool will generate insight into which of the dimensions are most challenging to develop and which may actually decrease once a participant gains awareness of a given dimension (as is shown in Section 6 in the area of knowledge). Data collected will support further research that will allow a refinement and improvement of the tools and its applications. The current simplistic recommendations that the online tool automatically generates to every user and group can and must be further developed once sufficient data is available. The same applies to the initial attempt of quantifying the progress as outlined in Section 6.
Limitations of this paper include the explorative nature of the approach taken in this paper. The comparative research and the subsequent prototyping in education and practice are solid in their methods as far as such methods may generate solid approaches.
While a majority of the survey questions of the online tool have been tested in similar situations, the tool itself does need further practice tests that can only be provided with the usage of the tool. It is important to analyze the effectiveness of the tool once sufficient additional data has been generated.
The CARL serves as a database to enrich the current discourse of RL with real-life data that is sought to further advance the related discussion. Thanks to all current and future users and to all those colleagues active in further advancing responsible leadership on an individual, an organizational and a societal level.

Katrin Muff
https://orcid.org/0000-0001-5501-5451 • Estimating consequences of decisions on the system Note: there was no suitable online questions for the dimension "self-awareness." These were sourced from existing off-line surveys. In italics, the online survey questions selected for the competencies. a Note: there was no suitable online questions for the dimension "self-awareness." These were sourced from existing off-line surveys. b In italics, the online survey questions selected for the competencies.