Defining nurse regulation and regulatory body performance: a policy Delphi study


  • D.C. Benton RN, MPhil,

    Chief Executive Officer, Corresponding author
    • International Council of Nurses, Geneva, Switzerland
    Search for more papers by this author
  • M.A. González-Jurado RN, PhD,

    Professor of Nursing
    1. Universidad Complutense de Madrid, Escuela Universitaria de Enfermería, Fisioterapia y Podología, Madrid, Spain
    Search for more papers by this author
  • J.V. Beneit-Montesinos MD, PhD

    Professor of Research
    1. Universidad Complutense de Madrid, Escuela Universitaria de Enfermería, Fisioterapia y Podología, Madrid, Spain
    Search for more papers by this author

Correspondence address: Mr David C. Benton, International Council of Nurses, 3 Place Jean-Marteau, Geneva 1201, Switzerland; Tel: +41-229080100; Fax: +41-229080101; E-mail:



Professional self-regulation is a privilege and needs to deliver against the underpinning social contract between the professional and citizens who are receiving care and services.


The aims of this study were to generate, international consensus on a contemporary definition of professional nurse regulation; and to articulate the key features of a highly performing regulatory body, and postulate which regulatory model and administrative arrangements are best suited to attain the key features.


A highly diverse and globally recruited random stratified sample of 75 experts was approached to participate in a classic three-round policy Delphi study. Quantitative and qualitative data were generated and subjected to thematic and statistical analysis. Both non-parametric and descriptive statistical techniques were used in relation to quantitative data.


Consensus on a revision of the current International Council of Nurses definition of professional nurse regulation was developed and a set of 47 key features of high-performing regulatory bodies was agreed. Although a strong preference for the delegated self-regulatory model (43%) and single-board administrative approach (48%) was expressed the underlying rationale for such a preference was unclear.


The research makes an important contribution to an underdeveloped field of study. The case for conducting more quantitative investigations to ascertain the best regulatory model and associated administrative approach has been made.

Nurse professional self-regulation is a privilege and, in addition to protecting the public, it is said to enhance the political, social and economic power of the occupational group concerned (Freidson 1994).

Over the past decades, the nursing profession and the role of the nurse have been subject to considerable change, yet to date, investigation and analysis of the topic of nurse regulation and its impact has not been studied to any great extent using rigorous research-based approaches. Although a wide range of opinion papers can be found (Esterhuizen 1996; Rowell 2003), these lack academic rigour and as such do not provide an evidential base to support the research-based redesign of contemporary professional regulation.

Despite the availability of various regulatory models, self-regulation is the preferred approach advocated by the professions to be used to regulate practice (Balthazard 2010). This approach is said to utilize the expertise of the profession, act to protect the public and simultaneously grant professions the necessary autonomy to act in a manner that avoids political imperatives. Alternatives to this approach are often dependent on the government. This can entail either multidisciplinary arms-length agencies or as part of ministries of health and/or education. At this time, there is some evidence that governments are, through changes to legislation that give ministers increased powers to intervene or through the establishment of oversight structures that monitor performance and challenge the regulatory bodies decision-making processes and outcomes, eroding the powers of some well-established self-regulatory bodies (CHRE 2012).

Without the underpinning evidence, it is difficult to logically advocate for any particular position. Indeed, such advocacy efforts by the profession can be undermined because opponents to professional nurse regulation can argue that its motives are less about public protection and more about self-interest (Salter 2004).


The aims of this study were to generate international consensus on a contemporary definition of professional nurse regulation; and to articulate the key features of a highly performing regulatory body, and postulate which regulatory model and administrative arrangements are best suited to attain the key features.


The Delphi approach is a method that was developed as part of studies conducted during the Cold War to help predict the potential impact of technology in warfare (Dalkey & Helmer 1963). It is now used extensively and is an accepted method of gathering data from defined experts who, through an iterative cycle of questionnaires, generate, anonymously, coherence and clarity on various aspects of the topic under investigation (Hsu & Sanford 2007).

Normally, the Delphi approach uses two to four rounds of questionnaires (Polit & Beck 2008). Linstone & Turoff (2002) contend that for most research studies, a three-round design is sufficient to attain stable results with further rounds running the risk of being viewed as excessive by panellists resulting in lower response rates. Accordingly, a three-round design was used in this study that provided, at the end of the first and second rounds, collated and anonomized feedback to the experts.

This approach has a number of advantages over other methods of gathering expert opinion, such as the ability to provide anonymity to participants; a controlled and structured feedback process; and the use of a variety of relatively simple statistical measures to allow interpretation of the results in real time by the participating experts (Dalkey 1972; Hsu & Sanford 2007). The advantages of this technique are said to mitigate against any one individual dominating the group and also avoid conformity through peer pressure (Dalkey 1972). The authors would also add that, in the case of this study where experts are drawn from around the world and where English is not always the panellist's mother tongue, that the technique provides the panellists with additional time to reflect and consider the issues.

Turoff (2002) identified that there are many approaches to the ways that the Delphi technique can be designed and implemented. In particular, he highlighted that the ‘Policy Delphi’, unlike the early applications of the technique that used homogenous groups of experts, uses a heterogeneous group that seeks to capitalize on diversity to generate robust solutions.

Rayens & Hahn (2000) also noted that the Policy Delphi is particularly well suited to identifying divergence and consensus in highly complex issues. According to Slocum (2005) to increase the likelihood of achieving a robust exploration of the topic, researchers should plan from the outset to carefully formulate the issues ensuring that clarity of wording is given specific attention. Further to this, the same author also highlights the need to explore and identify reasons and assumptions underpinning any disagreements of views as well as being prepared to evaluate and re-evaluate the data and emergent issues after each round.

At each stage of the Delphi questionnaire design, a pilot check for clarity (both grammatical and content) was introduced. This check involved utilizing a small group of experts who are both familiar with nurse regulation and the methodology being used. Suggestions made were considered carefully. Where necessary, refinements to the instruments were made prior to the questionnaires being sent to the expert respondent group.

The panel of experts

A random stratified sample of subjects was recruited to form the Policy Delphi expert panel. They were drawn from different parts of the world, from jurisdictions that have differing legal traditions, a variety of regulatory models and use a range of approaches to the administration of the legislation thereby providing heterogeneous sources. In addition, differing perspectives were sought, namely: those who are service based – nurse directors; those in charge of the regulatory bodies – registrars; educators of nurses; leaders from national nursing associations; and lay person views – patient representatives and other health-related disciplines.

Optimizing response

Turoff & Hiltz (1996) contend that panellists are motivated to participate actively if they feel they will obtain value from the experience, hence great care was taken in developing the initial letter of contact and all subsequent communication. Furthermore, the same authors recommend that in selecting experts four criteria should be applied. Firstly, they should have the necessary knowledge and expertise relating to the topic under investigation. Secondly, they have the capacity and are willing to participate. Thirdly, they have sufficient time and availability to meet the planned schedule. Finally, they have effective communication skills.

Hsu & Sanford (2007) highlight attrition as a particular risk of the Delphi design. Accordingly, panellists were given timescales for return of completed instruments and were given reminders a few days before the due date if they had not already completed and returned the instrument.

Panellist anonymity

Panellists were given an assurance that their individual responses and views would only be shared or disclosed with other panellists in an unidentifiable format both during the various rounds of the Delphi and in the write-up of the research.

Designing the Delphi instruments

The instruments capture a mixture of qualitative and quantitative data. An open-ended, 4-point Likert scale was used to solicit the views of the various experts. A 4-point Likert scale is particularly suited to the ‘Policy Delphi’ approach as it forces respondents to take a definitive ‘for’ or ‘against’ position (Turoff 2002). The same author also noted that the lack of a neutral point promotes debate.

Criteria used to assess the questions posed in the instruments

A number of researchers have identified that soliciting the views of panellists on the desirability, feasibility, importance and confidence of the items offers valuable insights and a means of deciding which items are best suited to addressing the research (Adler & Ziglio 1996).

In relation to quantitative data, according to Onwuegbuzie & Leech (2006), Delphi studies tend to use measures of central tendency such as mean, mode and median along with measures of dispersion including standard deviations and interquartile ranges. These techniques avoid overly complicated statistical measures and are usually familiar to most panellists and therefore are ideally suited to providing feedback on the collective position of the panellists' views on the various questions. Hasson et al. (2000) concur that on the whole median scores are used when Likert-type scales are employed and that using interquartile ranges offers panellists the necessary information on variability and degree of consensus. Additionally, two dimensions of interest, the Continuum of Professional Nurse Regulation and Administrative Approaches, offer independent categorical data rather than a Likert scale and, in these cases, results were reported back in the third round as the percentage of respondents who opted for a particular category.

Qualitative feedback including a summary of anonomized responses was provided in response to the various open-ended questions used in both rounds one and two. The summary of comments facilitated panellists in exploring areas where there were differences of opinion. Panel members were encouraged to offer explanations of the reasons for their views particularly when their quantitative score differed significantly from the median of the group.

Finally, non-parametric statistics (chi-squared test of association) were used to determine whether there was any evidence of response group bias.

Defining consensus

It has been frequently noted that researchers do not adequately specify what they mean by consensus (Dempsey et al. 2011). This is viewed as a weakness of design. Furthermore, even when consensus has been specified in a quantitative way, the level set may vary considerably. The Oxford English On-line Dictionary defines consensus as ‘general agreement’. Keeney et al. (2006) in their review of the Delphi techniques noted that research studies had interpreted the degree of agreement as anything from 51 to 100% and, while they did state that a higher percentage may be important should the decision result in life or death outcomes, no convincing rationale for determining a specific percentage was offered.

Because this is a Policy Delphi, the critical issue is not the degree of agreement per se, but understanding what underlies that agreement or indeed disagreement. On this basis, it can be argued that a percentage needs to be set that stimulates exploration of underlying reasons. In the Western world, many organizations utilize a set of rules of debate known as ‘Robert's Rules’ (Robert's Rules Association 2011). In this schema, important policy or constitutional changes need to carry at least a two-thirds majority to pass. Accordingly, because this level sits within the range 51–100%, 67% agreement was used as one measure to indicate consensus in this study. However, Rayens & Hahn (2000) have argued that for a four point Likert scale an interquartile range of 1 or less should be used in conjunction with an agreed percentage. Therefore, a threshold of 67% and interquartile range of 1 or less was used to indicate consensus.

Ethical approval

Ethical approval was sought and obtained from the University of Complutense's Research Ethics Committee. Responses were kept confidential through allocating a unique identity code that was used throughout the study to label data; all identifying information relating to the panellists was removed. The key to the identity codes was kept separate from the data.


In a three-round Delphi study, there was the potential risk of non-response error – that is, a distortion in the results because of differences between those who participate in and respond to the various cycles of data collection (Dillman 2000). However, by examining the responses received against the total sample population across key variables and utilizing the chi-squared test of association, it was possible to identify the likely presence of response error.

It is important to note that although the questionnaires used in the Delphi study were available in both English and Spanish, not all respondents had one of these languages as their mother tongue. Accordingly, the results are the product of responses from individuals who could have misunderstood some questions. To minimize this eventuality, respondents were encouraged to seek clarification as necessary.

Finally, the Policy Delphi methodology does not require that all respondents provide answers to each cycle of the study or even responses to all questions contained within a single instrument. This resulted in some missing data although the level of said missing data was remarkably low.


A total of 75 individuals with differing professional or lay experience were invited to participate in the Delphi study. Not all panellists responded to all three questionnaires and as a result, the response rate varied across the three rounds from 51 to 63%. However, this meant that a total of 59 panellists responded to one or more of the rounds giving a total response rate of 79%.

Data were analysed using the chi-squared test of association to determine whether there were any significant differences in terms of the response rate for each of the rounds in relation to the dimensions of interest (legal tradition of the jurisdiction – civil, common or Islamic law; background of expert – regulator, educator, professional association, lay respondent; gender; gross national income category of the jurisdiction – high, upper middle, lower middle and low). The chi-squared test of association identified no statistically significant differences across the various rounds in terms of the dimensions of interest. This confirms that, as intended, a heterogeneous sample of respondents was recruited as per the requirements for a Policy Delphi study.

A definition of professional nurse regulation

Four questions in the round one Delphi questionnaire contributed data to the analysis of the definition of professional nurse regulation. Panellists were asked to consider the current International Council of Nurses (ICN) definition of professional self-regulation and to identify material they felt might be redundant as well as suggesting concepts that needed to be added. While many respondents, more that 50% (n = 24) in round one, felt that the existing definition was adequate, a number of consistent responses were received that suggested that changes to the current International Council of Nurses' definition of professional regulation should be proposed. Analysis of the comments resulted in the identification of five component elements that should be included in any updated definition of professional nurse regulation. The elements are italicized and set out in the list below.

  • The subjects of the regulation – to whom regulation applies (practitioners, the registrant, nurse and profession).
  • The purpose of the regulation – a general statement of intent (to bring order, identity, consistency, control and accountability).
  • The outcome of regulation – what it is designed to achieve (protect the public, facilitate efficient and effective movement of registrants from one jurisdiction to another, alignment of professional practice with contemporary needs).
  • The mechanisms of regulation – basis upon which action is taken (legislation, voluntary-based approaches, co-regulatory models).
  • The means – detailed processes and approaches used to regulate the nurse (codes of conduct, examination, standards, continuing competence).

In terms of developing a revised definition the following steps were taken. The existing ICN definition was used as the basis for revision with deletion of redundant text, structuring to include the five key elements and insertion of missing concepts (enforcement and continuing competence were added). In addition, a refocusing of emphasis towards registrants rather than members was included. The agreed final version including the elaboration of the five key elements can be seen in Fig. 1.

Figure 1.

Final revised and extended definition of professional nurse regulation.

This elaborated format was felt to add significant clarity to the definition. However, it did draw mixed views on the length of the text. Some panellists (4, 10%) felt it was too long. Others wished to add some of the elaborative material to the core definition (3, 8%). The majority of respondents felt that the length was about right. A number of respondents did offer comments indicating that within their local context some further amendments would be required to adjust the definition to the specifics of the jurisdictional model, for example, the absence of licensed practical nurses.

Identifying key features of a high-performing regulatory body

A total of 49 potential features were identified and tested during the three-cycle review. Of the 49 potential features, 47 reached the pre-set required level of positive percentage consensus on all four criteria – desirability, feasibility, importance and confidence and the interquartile range was ≤1 for each of the measures. At the end of round two, a number of the panellists suggested that there was a need to cluster the 47 features into a series of themes and subthemes (Fig. 2). The round three questionnaire was in part used to ascertain whether the modifications introduced as a result of the round two feedback resulted in consensus on the clustering into themes and subthemes. As can be seen from Table 1 both the percentage consensus and the interquartile range results indicate that the new approach was well received.

Figure 2.

Final thematic mapping of the 47 key statements relating to high-performing regulatory bodies.

Table 1. Quantitative results relating to round three thematic presentation of performance features
StatementResponseFrequencyPercentage + consensusInterquartile rangeMedian
The structure helps clarify the focus of the featuresStrongly agree2889.1901
Strongly disagree0
The major headings provide an appropriate classification of featuresStrongly agree2286.1111
Strongly disagree2
The sub themes provide an appropriate classification of the detailed featuresStrongly agree2188.8911
Strongly disagree0

Model of regulation and associated administrative arrangements revisited

Panellists were asked to consider each of the key features identified and specify which of five regulatory models they felt most capable of delivering high performance. A high-performing regulatory body was operationally defined as one that delivers its core and subsidiary functions in a consistent, effective and efficient manner. Each of the five models of regulation (no regulation, pure self-regulation, delegated self-regulation, supervised regulation and government-based regulation) were operationally defined. Also, six different, although not mutually exclusive, administrative approaches to the governance of the regulator model were defined (multidisciplinary single-board umbrella, multidisciplinary with individual boards and shared operational services, multidisciplinary with individual autonomous board, single-board, regulatory unit, division of power).

Respondents appeared to have had great difficulty in offering critical comment on the ability of the models or approaches to deliver high performance. Some general comments on the importance of various features were received, but on the whole, respondents failed to elaborate on whether a particular model or administrative approach might be better suited to deliver the described aspect. However, considering the scores awarded to regulatory models, delegated self-regulation was identified as best suited to delivering the required features (43% of the scores available). This approach received more than double the number of endorsements than the next two most favoured approaches – supervised self-regulation and pure self-regulation, each with 21% of the scores.

In terms of administrative approaches the results were even more dramatic with three times the number of endorsements going to the single-board approach, 48% followed by the multidisciplinary, and 16% individual boards.

One position did appear to be consistently held and was explicitly and independently promoted by seven different respondents from various parts of the world who had experience of using different approaches including operating a regulatory unit. In summary, it was felt that a well-defined single board with clear purpose and adequate resources would seem to be the most efficient and accountable to discharge the features.

Despite the apparent reasoning – efficiency and serving the public better – all respondents limited the veracity of their positions by introducing a degree of uncertainly as indicated by words such as ‘would seem’, ‘perhaps’ or ‘most likely’.


The response rate to this study reached the upper limits of what could have been expected for this type of design. Gordon (1994) suggested Delphi studies typically attain a response rate of 40–75%. Accordingly, this study can be considered as offering a good basis upon which to reach definitive conclusions. Indeed, when comparing the response rate to that achieved by McAtee (2009) who attained a 22.3% response focused upon a much narrower aspect of regulation, albeit with a wider group of regulatory bodies, then the response rate of this study is far superior. This high level of response is thought by the authors to be attributed to the careful construction of the introductory letter and through diligently following the recommendations made by Slocum (2005) and Turoff & Hiltz (1996) regarding both panel selection and ongoing engagement. The authors also postulate that sending reminder e-mails three days before the due date for responses may have contributed to the good response rate.

Although there were high levels of response to the first and second round Delphi instruments (61 and 63%, respectively) there was some reduction in the response rate in the third round (52%). Keeney et al. (2006) have documented this phenomenon and, according to these authors, this indicates a classic symptom that respondents are starting to disengage from the study. The same authors go on to suggest that, in terms of the time taken to complete the instrument, respondents are no longer attaining the same level of personal benefit from participation. The reduction in response rate was therefore anticipated because, at the end of round two, a number of respondents indicated, in their general comments to the round two instrument, that the time taken to complete the questionnaire was considerable. Nevertheless, the response rate was still sufficient to obtain meaningful results.

This research has provided an important basis for examining the performance of regulatory bodies. The fact that such a diverse range of experts could agree on key features is both remarkable and reassuring. The subsequent clustering of the 47 statements under four main themes with 14 subthemes not only provides potentially important avenues for further research, but also a pragmatic foundation for developing a means of assessing the comparative performance of regulatory bodies across jurisdictions. Although beyond the scope of the current research study, Table 2 provides an illustrative example of how one of the 47 individual statements taken from Supporting Information Fig. 2 could be further developed to provide a metric for regulatory body performance. Each would require an agreed and easily understandable explanation of the statement. There would need to be recognition of any related statements and how to differentiate the statement under consideration from related metrics. Performance would need to be scored against a series of level indicators – each with their own description so as to ensure reliability of scoring.

Table 2. Illustrative example of metric for a specific statement
Statement – Continuing competence procedures are in place that use data from multiple sources.
Context – This statement is part of the responsibilities and functions cluster and specifically relates to those measures relating to competence and conduct.
Exclusions – This statement should exclude any actions relating to the assessment of competence associated with initial registration and licensure.
Level of performanceDescription of how to score performance
Outstanding performance
  • The regulatory body conducts and publishes research on the continuing competence activities of registrants in an aggregated format.
  • The regulatory body conducts correlation studies with the continuing competence activities used by registrants and their roles and functions.
  • The regulatory body has examples of best practice that they promote and share with peers, governments and their registrants.
Superior performance
  • In addition to meeting the requirements for adequate performance the body also delivers the following.
  • The regulatory body uses the results of competence and conduct cases to issue guidance to registrants on topics that they may wish to consider as part of their continuing competence activities.
  • The regulatory body considers the changing population demographics and strategic health and social policy documents to help shape guidance to registrants on how to meet continuing competence requirements.
  • The continuing competence system is fully computerized with registrants being able to submit and upload their evidence via a web portal that can be reviewed by the registrant via a user name and password feature.
Adequate performance
  • Legislation is in place and the regulatory body has developed, implemented and publicized process with which registrants must comply.
  • The regulatory body is able to identify those nurses that are in compliance with the required legislation.
  • The regulatory body takes systematic action against those nurses that do not comply with the required process.
Inadequate performance
  • The legislation does not currently have provision for continuing competence assessment.
  • The legislation has provision for continuing competence assessment but this has not been implemented.

Assessment of a regulatory body either through self-assessment or through a peer-to-peer process would provide an opportunity to identify both strengths and weaknesses. Weaknesses would offer an opportunity for performance improvement. By identifying and promoting examples of outstanding performance, peers may be able to benefit from the experiences of other jurisdictions and may even introduce a degree of normative influence.

The development of a set of metrics could also offer a useful addition to the induction programmes of new staff and board members providing direction in terms of content and focus for their orientation.

In relation to identifying a model and associated administrative approach most likely to deliver high performance across the range of features, it is clear that while there is a clear preference and consensus by the panel of experts for the delegated self-regulator model delivered via a single-board structure, the underlying rationales for such a choice remain far from clear. This result, bearing in mind that nurse regulation is a subset of regulation per se, might have been anticipated based on the work of Maggetti (2010) who, when reviewing the wider domain of regulation, noted that few studies were available to guide model or administrative approach choice.

The country specific rather than a comprehensive knowledge of regulatory systems was indeed a point highlighted by several respondents in this study and does seem to point to a major gap in the knowledge base of those that regulate nursing or who have influential roles in its leadership. With such a gap in knowledge, it is unsurprising that informed and objective critical commentary on the various models is difficult to obtain. Importantly, this study has exposed that not only is there little research on this key topic, but also national experts tend to have a limited understanding of models other than the one operating in their own jurisdiction. Despite this, it may be possible to build on the suggestion made earlier, namely, to utilize data from self- or peer assessments of performance to undertake correlation studies. A fully developed instrument would generate quantitative data that, based on a logistic regression design, could identify the presence or absence of a best regulatory model and administrative approaches. Although this is beyond the scope of the current research, such an investigation would make a valuable addition to the development of an ongoing programme of research in this neglected area of study.


The updated definition of regulation makes an important contribution to our ability to understand and enter into dialogue across jurisdictions. While some of the expert respondents felt that a shorter definition was desirable, the longer definition and the supporting key elements have proven useful in accommodating significant variations in the global application of the concepts and provides an important policy foundation for the direction and control of the nursing profession.

High-profile failures of self-regulation and delays in processing the applications for trans-jurisdictional mobility can result in increased government scrutiny and changes to the autonomy of the regulator (CHRE 2012; Salter 2004). In recent times, administrative failures, lack of focus on core business, and dysfunctional relationships among appointed leaders have, in at least one jurisdiction, been the source of further debate on the need to move regulation controls closer to government (CHRE 2012). Without an internationally agreed set of indicators that can be used to objectively assess performance individual regulatory bodies will be particularly vulnerable to political and media criticism. Hence, the results of this study provide a set of measures that can be developed further and used to track progress over time (annual accountability reporting) and offer a means for national and international jurisdictional benchmarking. Further research is required, but nevertheless, these results provide a tangible and practical first step to closer alignment and functioning of professional regulation across jurisdictions.


This paper was developed as part of a doctoral research study registered with the University of Complutense, Madrid, entitled ‘An International Comparative Analysis of the Regulation of Nurse practice’. The Authors wish to thank all those experts who contributed to the Delphi study and the double blind peer reviewers who provided helpful critical comments that enables us to strengthen and clarify this important contribution to the literature.

Author contributions

David C Benton RN, MPhilPrincipal author
  • Identification of Need
  • Conceptualization of Problem
  • Undertaking the research
  • Writing of initial draft
Máximo Antonio González-Jurado RN, PhDCo-author
  • Assistance in design of methods
  • Validation of analysis
  • Review of draft paper
Juan Vicente Beneit-Montesinos MD, PhDCo-author
  • Assistance in design of methods
  • Validation of analysis
  • Review of draft paper


This study has not been supported by any funding or grant source.

Conflict of interest

The principal author is the Chief Executive Officer of the International Council of Nurses and has an interest in making sure that ICN policy positions and principles reflect the latest thinking and evidence.