Developing a national framework of quality indicators for public hospitals




The current study describes the development of a preliminary set of quality indicators for public Greek National Health System (GNHS) hospitals, which were used in the “Health Monitoring Indicators System: Health Map” (Ygeionomikos Chartis) project, with the purpose that these quality indicators would assess the quality of all the aspects relevant to public hospital healthcare workforce and services provided.


A literature review was conducted in the MEDLINE database to identify articles referring to international and national hospital quality assessment projects, together with an online search for relevant projects. Studies were included if they were published in English, from 1980 to 2010. A consensus panel took place afterwards with 40 experts in the field and tele-voting procedure.


Twenty relevant projects and their 1698 indicators were selected through the literature search, and after the consensus panel process, a list of 67 indicators were selected to be implemented for the assessment of the public hospitals categorized under six distinct dimensions: Quality, Responsiveness, Efficiency, Utilization, Timeliness, and Resources and Capacity.


Data gathered and analyzed in this manner provided a novel evaluation and monitoring system for Greece, which can assist decision-makers, healthcare professionals, and patients in Greece to retrieve relevant information, with the long-term goal to improve quality in care in the GNHS hospital sector. Copyright © 2014 John Wiley & Sons, Ltd.


The evaluation of quality in healthcare has been a growing field in the last couple of decades, as the need for evidence-based decision-making, quantifiable improvement, and information useful for benchmarking has been manifested in many aspects of caregiving (Campbell et al., 2000). Quality of performance in healthcare services is of uttermost importance for stakeholders related to the healthcare industry, whether regarding health professionals, policy makers, or service users. Given the fact that the majority of annual health expenditure is invested on hospital healthcare at national level, mostly because of the high costs associated with hospital care and the profile that such organizations hold to the public eye (Marshall et al., 2006), projects regarding quality in hospital performance have been developed and quality indicators have been employed in a variety of settings and facing different goals. According to Ham et al. (2012), future demand of health services can be predicted through the monitoring of data on the utilization of services, and at population level, epidemiological data not only can help monitor health profiles/lifestyles but also can contribute to the delivery of better health outcomes and care for “at risk” groups at community and household levels. Additionally, it is important to put in the hands of both clinicians and patients data and information on health and healthcare, because only then can the health and social care system fully meet the demands of its population (Ham et al., 2012).

In line with the already widespread trend of healthcare quality assessment, there has recently been intense debate on evaluating performance of the public Greek National Health System (GNHS), with major service providers and policy-sensitive receivers in Greece being the GNHS hospitals. Specialized inpatient care in Greece is provided either by public hospitals, mainly GNHS hospitals, or by private clinics. GNHS hospitals are characterized as general or as specialized, the latter signifying referral healthcare centers for a single or a specific number of specialties (Economou, 2010). General GNHS hospitals, which are the main providers of hospital care, with at least one such facility available at each district, offer a variety of diagnostic and therapeutic healthcare services on outpatient or inpatient basis, while during each hospital's on-call day, outpatient departments receive patients requiring emergency care. At its total, personnel and infrastructure costs within public, GNHS hospitals are financed by the state, under regulations of the Ministry of Health (MoH), whereas healthcare services provided to patients are either covered through arrangements with social insurance funds, by private insurance, or by out-of-pocket money by patients.

Although operation of public GNHS hospitals is governed by the MoH at a central basis, the volume of resources needed at each district has not been carefully determined or updated according to a national scheme; administrative and capacity parameters had not been standardized and reliably updated in the past for public hospitals, and this was a need that the MoH was called upon to fulfill. Moreover, while data have been collected from public hospitals, either through routine internal recording, national census, or hospital-based surveys, only few efforts of utilizing the data to the patients' advantage or for evaluation purposes had been made in the past. Most efforts related to attempts of intra-organizational quality assessment have been made with no in-depth analysis on possible improvement of procedures. The arising needs for rational and efficient distribution of healthcare resources, improvement of population's general health status, and progress in the field of specialized healthcare posed the challenge to provide reliable and updated information on available services and quality of public hospitals, and to collect benchmarking information at national level, which could lead, in the near future, to renovation, a needs-based strategically planned public healthcare system and reduction of disparities in healthcare usage.

Greece, lying in the jurisdictional territory of organizational initiatives such as WHO's and Organisation for Economic Co-operation and Development's (OECD's) quality projects, has been providing public hospital data for the purposes of the aforementioned quality frameworks; however, those initiatives are international and not specifically sensitized to current healthcare demands of the Greek population, plus they mainly cover matters of clinical quality aspects of healthcare—not of capacity or management of hospital care services. The national health resources monitoring project “Health Monitoring Indicators System: Health Map” (Ygeionomikos Chartis) is a pioneer project at national level in Greece, recently engaged at developing standards for healthcare services, while quality indicators useful for assessment of health system's performance have been carefully selected according to major topics at hand. Participation in data collection and provision was mandatory for all public GNHS hospitals, to record the full extent of information, which would be valuable for improvement decisions and for benchmarking procedures among hospitals.

The current study describes the development of a preliminary set of quality indicators for public GNHS hospitals, which were used in the “Health Map” project, with the purpose that these quality indicators would assess the quality of all the aspects relevant to public hospital healthcare workforce and services provided.


The development of the quality indicators that were used for the preliminary version of the Greek “Health Map” project regarding the public hospitals' sector derived from a process including, firstly, a literature review and, secondly, a consensus procedure.

Literature review

When handling quality measurements in the form of quality indicators, one expects them to have been developed and validated following scientific soundness and usefulness, and to cover all challenging dimensions of health caregiving. Indicator development and realization is not always governed by hierarchical listing of information necessity but can sometimes be regulated by availability of records, making the scope of healthcare assessment rather insufficient and the inferential power of the reports rather limited, and possibly further obscuring the consumers' decision-making processes (Freeman, 2002; Copnell et al., 2009; Evans et al., 2009). Therefore, preceding a comprehensive approach to healthcare monitoring, a systematic review of previous evidence is essential to collect information on usefulness of measures that could be utilized.

On the basis of the aforementioned items, before developing a national set for quality assessment in hospitals, a review of quality indicator projects on hospital performance already employed in other countries at national or international level was carried out. Electronic literature search was conducted in the MEDLINE database to identify articles referring to international and national hospital quality assessment projects. Studies were included if they were published in English, from 1980 to 2010. The search was performed using various combinations of the following search terms/keywords: quality indicators, hospital, measure, assess, and/or evaluate. The reference lists of the selected studies were reviewed as well to trace relevant information on the projects selected, and internet search (i.e., via Google) was also performed, with the same keywords, to retrieve information for more existing projects and/or relevant gray literature. At the end, the official internet website of the selected papers was traced and investigated for updated description or technical information on the indicators included in the corresponding projects.

No publication types were excluded from the current review, apart from reviews, editorials, and letters, because these types of publication were considered to provide inadequate information on the design and implementation of the projects in question. The studies that referred to projects implemented at hospitals were analyzed further. The literature review and the selection and eligibility of the initial list of indicators that met the inclusion criteria were performed independently by two reviewers (E. S. and P. P.), and in case of disagreement, a third reviewer was consulted to reach final decision (A. R.). There was an 80–90% agreement between the initial two reviewers, regarding the selected list of indicators.

Consensus procedure

Selection of quality standards required to reduce the amount of the indicators that would emerge through the literature search; therefore, the initial list of quality indicators had to be reduced to a practical and balanced amount of standards/indicators. For this reason, following the step of assembling information on existing indicators, 40 experts on hospital care—independent, specialized, and/or recognized experts who covered all the aspects of the topic under scrutiny (such as policy makers, health professionals, researchers, and healthcare managers)—were asked to deliver assistance in successfully selecting useful standards for Greek public hospitals. The consensus panel evaluation of the indicators took place in Athens in November 2010. Because a relative quality-evaluating framework or previous scientific research on quality was practically nonexistent in the Greek territory, it was decided that a consensus procedure for the experts invited should be conducted (Jones and Hunter, 1995; Campbell et al., 2002). A consensus panel was organized, with a preparatory round performed on postal level and the final round performed through a tele-voting procedure. The preliminary list of quality indicators identified through literature review was initially posted to the experts participating in the consensus procedure, in questionnaire form, to gain familiarity. The same group of experts was, within one month, called to participate in a tele-voting panel; all of the experts summoned accepted the invitation.

Experts were asked to rate, through tele-voting, the quality indicators in terms of importance (impact on health status, policy relevance, and susceptibility to being influenced by the public health system performance) and feasibility (data availability, reporting burden), on a 5-point Likert-type scale ranging from “1—Not important” to “5—Very important” and from “1—Not feasible” to “5—Very feasible”. Experts were also encouraged to propose additional standards/indicators and to propose rephrasing of indicators that were unclear at initial presentation. Through this procedure, only quality indicators that were both evaluated as highly important and highly feasible by the expert panel were included in the preliminary “Health Map” set of quality indicators for hospital care.

Data analysis

The indicators selected for use in public hospitals by the consensus panel were categorized by the authors under dimensions. The indicators varied in terms of features and domains they covered; thus, a descriptive approach was preferred to analyze the selected projects. The reviewers compared each project's volume and dimensional composition with the overall trend for quality assessment in hospital care services, while considering the objectives and data collection and analysis potentials of each project.

Finally, after the selection of standards that would be used, technical specifications and data collection tools were also finalized and prepared for distribution. In addition to the standardized collection tools proposed, some quality indicators would require the additional use of surveys and data from other sources (e.g., census data), which were also proposed for the future by the experts of the panel. The standards and tools suggested for evaluation of quality in hospital care were selected keeping in mind the need for thriftiness in requirements and personnel burden.

The Health Map project

In Greece, no standardized tool had ever systematically been used for recording healthcare data in specific; apart from the healthcare data being collected by the Hellenic Statistical Authority (EL.STAT.)—among other types of data. The Health Map project was designed on the basis of data collection tools that have been implemented in other countries for health and healthcare monitoring and evaluation purposes. The result of the selection process was the development of a web-based information platform that would be updated regularly with raw data on health and healthcare from public and private providers around Greece. The MoH urged all healthcare providers to participate in the data collection process. The data collected concerned the recording of a variety of information (quantitative and qualitative variables/items) on the healthcare workforce and the health services provided by the public GNHS (Ethniko Systima Ygeias).

The information database of the “Health Map” has been grounded on four parameters: (i) the insertion, updating, coding procedure, and archiving of data; (ii) the geographical–topical classification and organization of indicators; (iii) the publication of customizable reports; and (iv) the availability of data to the users through a friendly environment that supports the access via the worldwide web (WWW) and electronic mail (e-mail). All “Health Map” online forms included a unified set of structured or semi-structured questionnaires all requiring information on predefined categories. Data sources have been documented with their variables and procurement methods. The Health Map project's information database is currently available to the public via the official website (


Quality indicators that could be used for evaluation of hospital performance were identified through the literature review, and after removal of duplicate standards, an initial list was selected with 1698 quality indicators (Table 1) that could potentially be used for monitoring hospital care services. More specifically, through the literature search, we managed to retrieve several articles on assessment of quality in specialized inpatient healthcare services, including two reviews of such projects and indicator frameworks (Groene et al., 2008; Copnell et al., 2009), with the result of 20 main projects being identified through our literature review as relevant to the aim of the study. Description of the projects selected is presented in Table 1.

Table 1. Hospital care quality-assessing projects and their corresponding dimensions and number of indicators (total N = 1698) selected through the literature review
Country and year of launching the projectOrganization: Project titleSourceDimensions of indicatorsNumber of indicators*
  1. “Number of indicators” refers to hospital-specific indicators of a more general spectrum found in respective projects

International, 2003WHO: Performance Assessment Tool for Quality Improvement in Hospitals (PATH)

Veillard et al., 2005

Clinical effectiveness, safety, efficiency, patient-centeredness, staff orientation, responsive governance41 (17 core)
International, 2002OECD: Health Care Quality Indicators (HCQI) project—Patient Safety

Arah et al., 2006

Clinical effectiveness, safety, patient-centeredness, timeliness15
USA, 1984/International, 1991Maryland Hospital Association (MHA): Quality Indicator Project (QIP)/Press Ganey Associates: International QIP (IQIP)

Kazandjian et al., 2003

Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness47
Europe, 2004The European Commission: European Public Health Outcome Research and Indicators Collection (EUPHORIC)

Torre et al., 2007

Clinical effectiveness, efficiency45
Europe, 2007The European Commission: Safety Improvement for Patients in Europe (SImPatIE) (Safety Improvement for Patients in Europe (SImPatIE), Europe, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, patient-centeredness42
Australia, 1989The Australian Council on Healthcare Standards (ACHS): Clinical Indicators project

Collopsy et al., 2000

Clinical effectiveness, safety, efficiency, timeliness338
Australia, 2004Australian Institute of Health and Welfare (AIHW): Sentinel events in Hospitals project/National Health Performance Committee (NHPC) (Australian Institute of Health and Welfare (AIHW), 2011a; Australian Institute of Health and Welfare (AIHW), 2011b) [Accessed on 15 December 2011]; and [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, staff orientation, timeliness8 for sentinel events/10 hospital-related in NHPC
Canada, 2001Ontario Hospital Association (OHA)/Hospital Report Research Collaborative (HRRC) (Hospital Report Research Collaborative, Canada, 2011) /Health System Performance Research Network (HSPRN): Hospital Reports [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, responsive governance158
Canada, 2010Canadian Institute for Health Information (CIHI): Canadian Hospital Reporting Project (CHRP) (Canadian Institute for Health Information, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, staff orientation, timeliness32
Denmark, 2000Danish Ministry of Health/Danish National Board of Health (et al.): The National Indicator Project (NIP) (The Danish National Indicator Project, Denmark, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness87
France, 2003French Ministry of Health: COMPAQHGrenier-Sennelier et al., 2005 [in French]Clinical effectiveness, patient-centeredness, staff orientation43
Germany, 2000BQS—Bund The Gemeinsamer Bundesausschuss (GBA) agency: Bundesgeschaftsstelle Qualitatssicherung (BQS) (Bundesgeschaftsstelle Qualitatssicherung, 2007)Bundesgeschaftsstelle Qualitatssicherung. (Bundesgeschaftsstelle Qualitatssicherung, 2007) Qualitat sichtbar machen: BQS—Qualitatsreport 2006. Dusseldorf: BQS, 2007 [in German]Clinical effectiveness194 (26 core)
The Netherlands, 2003The Dutch National Institute for Public health and the Environment (RIVM): Dutch Healthcare Performance Reports (hospitals)

Berg et al., 2005

Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness39
United Kingdom, 2000NHS Quality Improvement (Scotland): Clinical Indicators Support Team (Clinical Indicator Support Team, 2011) [Accessed on 15 December 2011]Clinical effectiveness64
United States of America, 1997Joint Commission on Accreditation of Health Care Organizations (JCAHO): Hospital Accreditation Standards (Joint Commission on Accreditation of Health Care Organizations (JCAHO), 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness70
United States of America, 1999National Quality Forum (NQF): Endorsed Standards (ES) for Hospital care (National Quality Forum, USA, 2011)[Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness113
United States of America, 2000Agency for Healthcare Research and Quality (AHRQ): Quality Indicators (Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety27 for patient Safety, 34 for inpatient care, 18 pediatric
United States of America, 2000The Leapfrog Group: Hospital Quality and Safety Survey (Leapfrog Hospital Quality and Safety Survey, USA, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness54
United States of America, 2004Centres for Medicaid and Medicare Services (CMMS)/Hospital Quality Alliance (HQA): Hospital Compare (Centres for Medicaid and Medicare Services (CMMS)/Hospital Quality Alliance (HQA): Hospital Compare, USA, 2011) [Accessed on 15 December 2011]Clinical effectiveness, safety, efficiency, patient-centeredness, timeliness101 (38 core)
Switzerland, 2000The Canton of Zurich: Verein Outcome

Luthi et al., 2002

Clinical effectiveness, safety, efficiency, patient-centeredness, responsive governance118

Most existing frameworks were launched in the last decade, varied in number of used dimensions and indicators, and were employed at national level. However, five international frameworks were also identified: Performance Assessment Tool for Quality Improvement in Hospitals (PATH), which was originally implemented by WHO and is perhaps the most global project to present (Veillard et al., 2005); International Quality Indicator Project (IQIP), which is the international modification of the Quality Indicator Project (QIP) originally developed in the USA (Kazandjian et al., 2003); the European Public Health Outcome Research and Indicators Collection (EUPHORIC) (Torre et al., 2007) and the Safety Improvement for Patients in Europe (SImPatIE) (Safety Improvement for Patients in Europe (SImPatIE), Europe, 2011) projects, both developed and funded by the European Commission (EC) and mainly focused on health outcomes and patient safety, respectively, throughout the European region; plus, OECD's Health Care Quality Indicators (HCQI) project's indicators focusing on patient safety (Arah et al., 2006). Among frameworks implemented at national level, most existing quality indicator sets originated from the USA, serving a variety of objectives, while Canada, Australia, and many European countries also presented hospital-assessing quality projects. Mainly depending on the voluntary or mandatory nature of each framework, the results of the data analysis can be disclosed to the broader public, used for internal quality purposes only, or dispensed at a centralized, benchmarking organization.

Performance Assessment Tool for Quality Improvement in Hospitals—sometimes compared with other quality frameworks as the golden standard (Groene et al., 2008)—was originally launched in 2003 and is run by the WHO Regional Office in Europe. PATH addresses a variety of national hospital settings in a comprehensive manner, including six essential dimensions of quality in hospital performance in four main domains (clinical effectiveness, efficiency, staff orientation, and responsive governance) and two transversal perspectives (safety and patient-centeredness) (Arah et al., 2003). Its main purpose of development was to assist in a multidimensional assessment and improvement of quality in hospital care, at the local, national, or international level, making use of data previously remaining unused. Four of the dimensions used in PATH are also suggested by the Institute of Medicine (IOM), namely safety, effectiveness, efficiency, and patient-centeredness (Vanselow et al., 1995). IOM also suggests the use of indicators measuring timeliness and equity in evaluating healthcare quality, therefore empowering indicators related to accessibility and, in the case of equity, indicators with a broad field of implementation (Copnell et al., 2009). All the dimensions mentioned earlier, namely those mentioned in PATH and by the IOM, were evaluated while developing the “Health Map” set of quality indicators. Finally, from the selected projects, as mentioned earlier, a list of 1698 indicators was assessed for eligibility in the current study to be evaluated by the consensus panel.

The consensus panel process resulted in the selection of 67 indicators (Table 2), which comprised the preliminary set of indicators of the Greek “Health Map” project and were used in the pilot study of the project—being the basis of the ongoing web-based information system developed for the Greek “Health Map” study—which was firstly implemented in its current form in 2011. The indicators selected for use in public hospitals by the consensus panel were categorized under six distinct dimensions (N = number of indicators included): Quality (N = 12), Responsiveness (N = 11), Efficiency (N = 10), Utilization (N = 5), Timeliness (N = 4), and Resources and Capacity (N = 25).

Table 2. The NHIF quality indicators (N = 67) selected for the assessment of hospital care, by dimensions and sub-dimensions/features
  • *

    AMI, acute myocardial infarction; CABG, coronary artery bypass graft; ICU, intensive care unit; VTE, venous thromboembolism; VAP, ventilator-associated pneumonia; UTI, urinary tract infection; ER, emergency room; CT, computerized tomography; MRI, magnetic resonance imaging; DSA, digital subtraction angiography; PET, positive emission tomography; HCU, high care unit.

• Inpatient mortality from selected causes (AMI*, stroke, pneumonia, CABG*, hip fracture, pneumonia)
• Readmission rate for selected causes (AMI, stroke, pneumonia, CABG, hip fracture, pneumonia, asthma, diabetes mellitus)
• Unscheduled readmission to ICU*
• Perioperative mortality
• Perinatal mortality due to complications (mother, child)
• Cancer patients successfully surviving surgery/chemotherapy/transplant
• In-hospital avoidable VTE*
• Hospital-acquired infections (VAP*, urinary catheter associated UTI*, central line associated blood stream infection, surgical site infection, infections in neonates)
• Medical errors per sector (following surgery, improper or delayed treatment, iatrogenic complications)
• Obstetric trauma
• Staff injury
• Staff needle puncture incidents
Patient centeredness
• Patient feedback management
• Pain control
• Satisfaction from personnel
• Explanation of procedures, treatment and discharge information
• Satisfaction from hospital environment (cleanliness, quietness, privacy)
Staff orientation
• Staff burnout
• Staff absenteeism
• Staff working overtime
• Satisfaction from working environment
• Clearly defined responsibilities in staff
• Continuous education for health professionals
• Length of stay
• ICU length of stay
• Hospital bed coverage
• Admission/discharge rate
• Cost of inpatient services per patient day
• Exams ordered at the ER*, per patient
• Laparoscopic/open surgery rate
• Single-day stay for selected surgeries
• Caesarian section rate
• Surgery postponed or canceled
• Patients visiting the ER department
• Admissions for acute conditions
• Usage of equipment/facilities
• Usage of laboratory exams
• Surgical Theater use
• Time needed for initial clinical examination at the ER after arrival
• Time needed for admission after arrival at the ER
• Time needed for selective surgical treatment
• Patients leaving without being examined
Human resources
• Permanent personnel (per discipline)
• Detached personnel (per discipline)
• Temporary personnel (per discipline)
• Personnel educational level (per discipline)
• Intra-sector nurses to physicians ratio
Information technology
• Computers for the personnel
• Computers with Internet access
• Computers with modern applications
• Use of electronic medical records
• Hospitals having a webpage
• Telephone center
Infrastructure and facilities
• Surgical theaters
• Beds per sector
• Beds per room
• Short-term stay beds
• Space for patient baggage
• Toilet in patients' rooms
• Intra-communication facilities in patients' rooms
• Oxygen facilities in patients' rooms
• Air-conditioning facilities in patients' rooms
• Telephone facilities in wards
• Imaging facilities (radiography, ultrasound, CT*, MRI*, mammography, gamma-camera, DSA*, PET*)
• ICU and HCU* unit(s)
• Hemodialysis facilities
• Management of hospital waste

Because of the early stage of quality-assessing efforts in the sector of hospitals in Greece and taking into consideration the lack of nationwide information on hospital personnel, equipment, facilities, and infrastructure, an extensive variety of quality indicators was dedicated to administrative subjects, meaning the dimension of resources and capacity. All administrative indicators proposed were expected to be measured by raw data that directly originated from the “Health Map” questionnaires and that can usually be recorded by organizational staff in hospitals as routine information, but are usually not collected centrally, standardized, or coded; these measures are aligned with previous knowledge of the managerial status in Greek public hospitals, and indicators were carefully proposed according to this knowledge (or therefore lack of).

On the other hand, the remaining indicators of the “Health Map” set were balanced among all other dimensions, most indicators dealing with quality, including clinical effectiveness and safety standards, which are areas highly prioritized in current quality indicator sets for hospitals and reflect evidence-based medical applications. Indicators on clinical effectiveness and safety in the “Health Map” framework's quality set covered a wide range of in-hospital outcomes and were mostly selected from larger sets used abroad or globally, first and foremost being the set of PATH indicators, while other quality frameworks such as the Joint Commission on Accreditation of Health Care Organizations (JCAHO), (Joint Commission on Accreditation of Health Care Organizations (JCAHO), 2011) IQIP, the EC-funded projects (EUPHORIC, and SImPatIE) and OECD's HCQI were also carefully considered for additional quality indicators; indicators in these sets were found to be extremely important though feasibility in the GNHS was not always their strength, on the basis of data collection so far, while standardized data collection tools were expected to moderate this difficulty. Responsiveness, meaning patient-centeredness and staff orientation, and timeliness indicators, based on similar measurements in global frameworks such as PATH and the Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project, (Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project, 2011) were also rated high in terms of importance, while valuable sources of data relevant to these areas, especially for measures regarding patient feedback, were surveys among in-hospital patients. Lastly, indicators related to efficiency and utilization, most of which would make use of administrative data collected by the standardized questionnaires, were two dimensions that counterbalanced the set of quality indicators and were found to be widely used abroad as well as highly appreciated by the consensus panel.


Hospital care, being the health sector that receives the largest volume of financial support and massive policy-related interest, has generally been in the epicenter of healthcare evaluation, especially during the last decade that most of the internationally utilized healthcare quality projects were launched. In our study, to proceed to the development of a quality indicator framework for Greek public hospitals, we attempted to depict the current global status of hospital evaluation by reviewing existing sources of quality indicators for hospitals; we also tried to identify possible areas that would require additional attention in the GNHS and, specifically, in public hospitals, which in turn would denote subjects and dimensions that would need surplus caution in their evaluation approach.

In our review, we identified several quality projects regarding hospital performance, most of which were employed at national level, while five projects expanded to an international fit for hospital evaluation (i.e., PATH, IQIP, OECD's HCQI, EUPHORIC, and SImPatIE). Differences on volume and final selection of dimensions of quality measurements were noted, and were also expected to a point, as each project appealed to a specific accommodation and was implemented through a distinct strategy, with some projects being compulsory, while others were voluntary and served as a guide to improving quality (Groene et al., 2008; Copnell et al., 2009), and as projects were based on different methodological backgrounds, although the basic quality dimensions are usually covered by the IOM's framework; these differences make an attempt of crosscheck of indicators difficult between projects. Differences in definitions and methods of data collection and analysis of quality indicators do not allow comparisons of findings across different frameworks, so in our case, this would present problems regarding the parallel operation of alternative quality-assessing projects in the same geographical area (e.g., of Europe or Greece) where internationally implemented projects operate in parallel with national projects. This situation, if not accounted for, might result into multiple schemes exploiting the same funds, creating an unnecessary burden of data gathering, discouraging participation, and creating competing interests, while it could also lead to biased duplication of results from the same area (Groene et al., 2008); therefore, it is extremely useful to map quality indicators according to project and aims, and to acknowledge issues regarding similar frameworks in the same areas, especially before developing a novel evaluation agenda. In the “Health Map” project, caution was applied so that indicators selected converged to measurements previously proposed and charted internationally, with priority given to projects funded by WHO, the OECD, and the EC (i.e., OECD's HCQI, PATH, EUPHORIC, SImPatIE); therefore, the expected results will generally be in accordance with other foreign quality frameworks, while the cost of data selection will minimize.

In the set of quality indicators finally selected by the “Health Map”, it is important to denote that all data gathered and analyzed centrally were later on redistributed to healthcare providers, suggesting sectors that could progress, but, in addition, disseminated to consumers of healthcare services, that is, patients, who never before had the quality-assessing or benchmarking information for public healthcare facilities around Greece; public disclosure is a trait common among quality projects that are based on non-voluntary participation (Groene et al., 2008). This feature is very important, as participation in the evaluation “Health Map” framework is mandatory for all Greek public healthcare facilities; therefore, information would universally cover public healthcare in the Greek territory and, most probably, provide organizational motives in the direction of investing on quality improvement (McGuckin et al., 2006).

In general, most indicators used in the identified quality frameworks abroad were process indicators, probably easier and more feasible to measurement (Donabedian, 2005) as patients' stay in contemporary hospital facilities has been reduced to a minimum, due to modern medical care. Therefore, rendering outcome measures has been found to be inadequate (Mant, 2001; Lilford et al., 2004) in quality assessment during recovery stage (e.g., postoperatively, recovering from infectious disease) as patients tend to leave the hospital environment relatively early (Groene et al., 2008), and structural measurements have often been made redundant because of previously recorded information in most countries, mostly as a result of accrediting standardization (Copnell et al., 2009). However, taking into account the lack of a previous national standardization framework in hospital care in Greece and the need for updating information on capacity and healthcare volume support by public hospitals according to region, led into a rather extensive utilization of structural indicators in the “Health Map” quality indicator set, comprising the additional dimension of resources and capacity, including infrastructure, facilities, personnel, and technology indicators; the use of structural indicators was also prioritized in the “Health Map” because of relative easiness of data collection at this early stage of a national quality-assessing attempt.

In our literature review, it was noted that the most focused-upon dimension of quality in hospital care has been clinical effectiveness (Groene et al., 2008; Copnell et al., 2009), being evaluated at some extent in all current frameworks; this might reflect the modern trend of following and appreciating clinical guidelines and evidence-based medicine and might also reflect the need for cost-effectiveness practices and strategies (Mainz, 2003). The second most valued and emphasized dimension in hospital care was patient safety, with several projects clearly focusing on this particular sector (SImPatIE, HCQI, AHRQ, (Agency for Healthcare Research and Quality (AHRQ): Quality Indicators project, 2011) Leapfrog, (Leapfrog Hospital Quality and Safety Survey, USA, 2011) AIHW (Australian Institute of Health and Welfare (AIHW), 2011a; Australian Institute of Health and Welfare (AIHW), 2011b) patient safety is thought to be the distinguishing value in healthcare quality between providers and systems and might also reflect the current hierarchy in healthcare priorities (Kazandjian et al., 2005; Copnell et al., 2009). These dimensions also received adequate attention in the “Health Map” set of hospital indicators, as many indicators referred to results in following effective clinical care and at the same time ensuring safety, primarily for patients; as already stated, priorities in healthcare, in Greece as much as abroad, include following guidelined clinical decision-making and avoiding patient discomfort and possible legal proceedings. However, at this point, outcome indicators were used more than process standards, as administrative data recorded at the hospital level (and not patient or sector specific), including hospital registries, were considered the primary source of raw data for clinical effectiveness and safety issues.

Responsiveness, mainly patient-centeredness and also staff orientation and satisfaction, are also valued in many quality frameworks of our literature review. The anthropocentric ideals governing the inclusion of quality indicators assessing patient-centeredness were much appreciated in the “Health Map” quality framework (Wylie and Wagenfeld-Heintz, 2004; Clark et al., 2006); therefore, a number of indicators assigned to this dimension were included in the final “Health Map” set. As mentioned before, data regarding patient-centeredness indicators would not be collected through the web-based form of “Health Map” questionnaires, but would rather require the assistance of patient surveys; a number of Greek hospitals already use forms of patient-oriented questionnaires to handle patient satisfaction and complaints; however, the results are usually left unused; therefore, utilization of such information might prove to be valuable in the field of enhancing satisfaction and quality of life. Staff orientation also qualified as important in the “Health Map” set, in the direction of efficacy of the healthcare workforce, considering that satisfied health professionals are the most productive ones (Cleary et al., 2010; Schulte and Vainio, 2010); hospital administrative data, such as records of leaves or overtime duty, standardized for the “Health Map” questionnaires but, also, in-hospital health workers' surveys would help in acquiring data fit for use in such standards.

Efficiency indicators, similar to other quality projects, were also highlighted in the “Health Map” framework's public hospitals set, as it is of great importance and policy-making significance that, in the current financially challenging period, orthological use of existing resources is made and that accounting for expenditure in a cost-effectiveness guided manner takes place. Standards assessing patient stay, charges related to in-hospital procedures, and proper and adequate use of surgical techniques were highly valued in the direction of assessing and improving profitability, productivity, and thus, efficiency of services provided by public hospitals (Clarke, 1996; Martinussen and Midttun, 2004; Huerta et al., 2008). Utilization indicators, such as surgical theater use, admissions, and usage of equipment and exams, were also considered important and ended up comprising a discrete dimension in “Health Map” quality standards; most of these measurements can also reflect accessibility matters; however, because of lack of previous systematic information on volume and allocation of services provided per facility and per district, this dimension received considerable attention in the “Health Map” set of quality standards in the direction of acceptable delivery of hospital services and reasonable use of resources, next to efficiency standards (Barrett et al., 2005; Kraus et al., 2005). Although utilization standards were through of as more related to accountability and justification matters, another theoretical component of accessibility, timeliness, was included, also as a distinct dimension, receiving a number of indicators, all of which encompassed in other quality projects' sets. Timeliness standards emphasize on the possible difficulties in managing the workload in hospitals (Bernstein et al., 2009), as these facilities sometimes receive responsibility for health needs left unfulfilled (e.g., patients leaving without being examined, patients admitted with delay) because of considerable preference in public hospital care versus private healthcare, with financial as well as qualitative criteria on the part of patients, and because of perhaps a less proactive role of primary public healthcare services in Greece (Economou, 2010).

On the other hand, matters of responsiveness in hospital governance were not considered as most important or as easy to measure at the current starting phase of the “Health Map” framework; therefore, no indicators specifically dedicated to this dimension were included. Also, linking hospital care to the community and to public health is much valued; however, because of the need to be parsimonious in our “Health Map” quality measures, such matters were given priority in assessment of public primary healthcare, a different parallel project of the “Health Map”, rather than within the hospital environment. Lastly, equity is perhaps the most important factor of ensuring a solid and functional healthcare system, to improve the health status of the general population and reduce health disparities; indicators relating to this dimension might not notably be developed in the existing frameworks, but equity may be the driving force beneath any comprehensive quality-assessing system (Hebb et al., 2003; Coffey et al., 2005). In the “Health Map” quality framework, similar to the global trend, indicators specifically dedicated to equity were not included; however, many of the multi-level indicators used to estimate the overall quality and accreditation in performance of public hospitals (e.g., space in rooms, timeliness in meeting acute healthcare needs, understanding information) address the subject of equity (Copnell et al., 2009).

It is perhaps useful to note some limitations of the current study regarding the selected “Health Map” set of quality indicators for public hospitals. First, indicators selected generally pertained to a hospital-wide range rather than being sector or disease specific, as this was a starting effort to assess hospital quality in Greece. Therefore, some more clinically focused indicators, on specific subjects (e.g., cardiac and stroke care, treatment options, and mobility and rehabilitation), were not included. These subjects are traditionally addressed in quality indicator projects (e.g., EUPHORIC, JCAHO, IQIP, and PATH), and although they might be relevant to a distinct specialty (e.g., cardiology and infection control), they also reflect severe conditions that inflict massive workload, costs, and focus on part of the healthcare system (Copnell et al., 2009; Evans et al., 2009) and should therefore be included in a generalized hospital assessment quality set; this was not the case in “Health Map” as main hospital data sources could not cover collection of such detailed information. The addition of omitted, but important, measurements in future versions of the “Health Map” indicators set was highly recommended by the experts of the consensus panel and will be given serious attention in upcoming selections.

Furthermore, the measurements that were selected referred to acute illness care only, while aspects of outpatient healthcare provision, such as health promotion, chronic disease management and palliative care, were not addressed in the current set of hospital indicators; such subjects are more systematically considered and fulfilled in the primary care setting (Vanselow et al., 1995; Starfield, 1998); therefore, standards regarding these subjects were left to primary healthcare quality assessment, consisting a parallel project of “Health Map” as already mentioned. Also, in a traditional manner and to converge with global quality frameworks (Copnell et al., 2009), quality and effectiveness of care in the national hospital indicator set majorly involved physicians' performance, although all health-related disciplines were accounted for in recording the current GNHS resources and capacity; perhaps in the future, additional quality measures may need to be indorsed to cover the need of a universal appraisal in effectiveness and improvement of healthcare provision. Finally, it should also be noted that, at present, the GNHS is under parallel reformation of the coding system related to diagnosis-related groups and hospitalization-related costs, endeavoring to adopt international classification and to update information technology utilized (Polyzos et al., 2013); therefore, success of the concurrent projects of reorganization and of evaluation in hospital services cannot be guaranteed yet, because realization and usefulness of performance measures remains to be validated in the future.


In the current study, we investigated contemporary quality indicator frameworks already implemented abroad or internationally, as the first step to advance to a national quality framework for public GNHS hospitals. We also attempted to isolate specific deficiencies originating from the existing systems of data recording and updating regarding performance of hospitals in Greece, and specific needs rising from contemporary challenges in the Greek society. With the use of evidence from the international literature and electronic sources, along with information on shortages in sectors of quality in healthcare in the Greek territory, the “Health Map” quality indicator set was developed, following, but not restricted to, standards of global quality projects, while all dimensions of quality in hospital care were discussed at its developing point, each dimension receiving an essential number of dedicated quality indicators. Data gathered and analyzed in this manner have been expected to provide useful and novel information to decision-makers, healthcare professionals, and patients in Greece and to assist in a future overall estimation and improvement of quality in care in the GNHS hospital sector.


This work was part of the “Health Monitoring Indicators System: Health Map” project funded by the European Social Fund in the framework of the Axis 5.1, 5.2, 5.3 of the European Project “Development of Human Resources” (2007–2013). The contents of this paper are solely the responsibility of the authors and do not necessarily represent the official views of the sponsor.