Performance of Thailand’s universal health coverage scheme: Evaluating the effectiveness of annual public hearings

Abstract Background Legislative provisions in Thailand's National Health Security Act 2002 mandate annual public hearings for providers, beneficiaries and other stakeholders in order to improve the performance of the Universal Health Coverage Scheme (UCS). Objective This study aims to explore the annual public hearing process, evaluate its effectiveness and propose recommendations for improvement. Method In‐depth interviews were conducted with 29 key informants from various stakeholder groups involved in annual public hearings. Results The evaluation showed that the public hearings fully met the criteria of influence over policy decision and partially met the criteria of appropriate participation approach and social learning. However, there are rooms for improvement on public hearing's inclusiveness and representativeness of participants, adequacy of information and transparency. Conclusions Three recommendations were proposed a) informing stakeholders in advance of the agenda and hearing process to enable their active participation; b) identifying experienced facilitators to navigate the discussions across stakeholders with different or conflicting interests, in order to reach consensus and prioritize recommendations; and c) communicating policy and management responses as a result of public hearings to all stakeholders in a timely manner.


| THEORE TIC AL FR AME WORK
Meaningful participation refers to effective construction and implementation of participation as well as public acceptance of public participation outcomes. 16 Approaches to evaluating meaningful participation depends on the specific issues under consideration and related contexts 17 ; therefore, there is no single accepted evaluative framework applicable to all cases. 17,18 This study investigated evaluation frameworks for meaningful participation from various scholars 10,16,[19][20][21] and selected 6 evaluation criteria that fit the context of annual public hearings of health providers and UCS beneficiaries.
These are (a) inclusiveness and representativeness, (b) adequacy of information, (c) appropriate participation approach, (d) social learning, (e) transparency and (f) influence over policy decision making.
The details on the evaluation criteria, definition and indicators are presented in Table 1.

| ME THOD
Using the evaluation framework in Table 1, this study assessed public hearings at regional and national levels. One province per region (Northern, North-Eastern, Central and Southern) was randomly selected as the study site. As national level public hearings are conducted in Bangkok, the public hearings in Bangkok were additionally evaluated. In total, five study sites were selected for the evaluation.
Data were collected through in-depth interviews with key informants, chosen from a list of previous annual public hearing participants. In addition to executives from the NHSO, at least one key informant from each stakeholder group was selected: public hearing organizers, health-care providers, Civil Society Organization (CSO) members and local administrative organization members ( Table 2). Six criteria for meaningful participation (Table 1) were used as the framework for the evaluation. Data were mapped into the framework matrix to analyse whether the public hearing process met each criteria. Based on the findings from interviews and data triangulation, achievement was measured into three levels: fully meeting the criteria (the finding supports all indicators), partially meeting the criteria (the finding supports some indicators) or not meeting the criteria (the finding do not support any indicators).

| The process of annual public hearings
The guiding principles of the public hearings and appointment of the ad-hoc committee for proposing annual public hearing agendas are endorsed by the NHSB (NHSO3).
In 2019, the NHSB approved 8 issues to be addressed at the annual public hearing. Seven issues are 'cross cutting issues for all regions', which include the following: (a) type and scope of health services, (b) service standards, (c) NHSO management, (d) national health security fund management, (e) local health fund management, (f) public participation, and (g) perception and rights protection. The 8th issue area in the public hearing agenda is local or context specific (NHSO1,2,3; NHSR2,3,6; CSO4).
The public hearings begin at the regional level, with regional forums at each of the 13 NHSO regions, followed by a national forum TA B L E 1 Evaluation criteria for meaningful participation of annual public hearing Evaluation criteria Definition Indicator

1) Inclusiveness and representativeness of participants
Both direct stakeholders 16,24 and wider public 10 are involved in the public participation.
1) Potential affected stakeholders by the new proposals of UCS attend the annual public hearing. Interested lay public attend the annual public hearing.

2) Adequacy of information
Adequate information is provided to the participants to be able to seek for assistance, 27 to strengthen their understanding and to be able to participate actively. 26 1) The public hearing organizers disseminate necessary information related to the agenda well in advance through various channels to the potential participants.
2) The public hearing organizers assist the participants to prepare the information to discuss in public hearing.
Appropriate participation approach Observations of contexts, culture, norms, timing, and diversity of participants are considered 17,22,27 as key attributes of appropriate public participation approaches. 28,29 1. The approach and process of the public hearing is flexible and appropriate which fit to the context, culture and norms of the community. 2) All groups of stakeholders participating feel comfortable with public hearing approach.

Social learning
Participants listen to each other and find a solution which accommodates mutual interests. [30][31][32][33][34] The experienced facilitators support and empower participants to express their opinions in a constructive manner. 32 1) The public hearing is held in a neutral place.
2) All stakeholders are free to express opinions in a friendly environment and work together to reach the conclusion on common interests and reconcile the differences.
3) The facilitators have extensive experience in moderating public hearing sessions.

Transparency
The outcomes of the public participation reflects the discussions, conclusions and decisions of participants and can be justified.. 10,20 1) The participants clearly understand how the issues of public hearing are framed. 2) The mechanism to reach mutual agreements of all participants has been jointly developed by all participants.

Influence
The outcome of public participation process influenced over policy decision making. 16 1) The participants are informed that their opinions are put into policy decision-making process.
2) The policy decision which reflects the public hearing outcomes are published and publicly available. TA B L E 2 the number of key informants by group and site in Bangkok. The regional forums are organized by the NHSO regional health security offices that are responsible for selecting the number and arrangement of each forum to the suit local situation (NHSO2,3; NHSR2,6).

Key Informant
In the Central and North Eastern regions, the regional health security offices assign working groups, comprised of representatives from health-care providers, CSOs and local administrative organizations, to organize the regional forums (NHSR2,6). For provincial forums, in the Central region, the working group assigns the provincial CSO that coordinates the UCS to organize the provincial hearings; each province then nominates a representative to the regional forum. Whereas in the North Eastern region, the working group assigns local universities to organize the provincial forums, the discussions from the provincial forums then inform the regional public hearings (NHSR2; PRO4,6; CSO3; LAO3). In contrast, both provincial and regional level public hearings in the Northern and Southern regions are organized and managed by office staff of the regional health security offices (NHSR#6, CSO2; LAO1,2).
Recommendations and feedback based on the forum discussions at the regional public hearings are compiled and submitted to the Regional Health Security Board and the Regional Standard and Quality Control Board (NHSR3, CSO4 and PRO7). In addition, according to the public hearing organizers, the Regional Health Security Board and the Regional Standard and Quality Control Board have co-hosted regional public hearings since 2017, allowing both boards to provide immediate response to key issues discussed at the hearings (NHSO3). Confirmed by all key informant groups, challenges that can be addressed at the regional level are taken into consideration by the regional boards and not forwarded to the national level public hearings (NHSR2,3,4,5,6; PRO2; CSO2,4; LAO1,5).
Similarly, the provincial level challenges and recommendations are not referred to the regional hearings and are instead addressed provincially (NHSR2,3,5,6; CSO4). This streamlines the bureaucratic inertia of addressing major concerns.
The recommendations from the regional public hearings to be

| The evaluation of annual public hearings for health-care providers and UCS beneficiaries
Using the evaluation framework proposed in Table 1, this study sought to identify current gaps and areas where the public hearings can be improved. 22,23

| Inclusiveness and representativeness
Literature suggests that partners in public participation should include individuals, groups or organizations that may be affected by the policy decision. 16

| ADEQUACY OF INFORMATION
Information provision is crucial in ensuring meaningful participation.
It is important that participants receive adequate information prior to attending public hearings to be able to strengthen their understanding and actively participate in the process. 16,26 It is suggested that experts should help participants prepare evidence and presentations for proposals. 27 The public hearing organizers should not only disseminate necessary information related to the agenda but should also assist participants in preparing information to discuss during the forums.
This study found that public hearing organizers and other Due to lack of information provision and absence of stakeholder preparation, improvements are still necessary for the public hearings to meet the criteria of adequacy of information.

| APPROPRIATE PARTICIPATION APPROACH
The best public participation method should take into account the local contexts, culture, social norms in the community and timing and characteristics of stakeholders. 17,22,27 The organizers should select participation methods where all groups of stakeholders can equally express their opinions and avoid neglecting minorities 28,29 Thus, the approach and process of the public hearing should be flexible and appropriate to the local context. Additionally, all groups of stakeholders should feel comfortable with the selected public participation approach.
This study found that the NHSO allows each region to manage public hearings to suit their local needs (NHSO3). In the North Eastern, Central and Southern regions, the provincial level hearings are conducted prior to the regional hearings. In contrast, in the Northern region, only one public hearing at the regional level is conducted. However, the public hearing approaches are based on the ability of organizers instead of the preferences of stakeholders. Key informants from all stakeholder groups agreed that only one approach to the public hearings is insufficient and alternative methods should be explored such as website surveys, questionnaires or mobile applications (NHSO2,3; NHSR4,5,6; PRO5,8; CSO4; LAO1,2).
In addition, some informants reported that they were not comfortable in expressing their opinions at the regional or national forums due to the formality of the process (NHSR3; LAO1,2,4; CSO3). Additionally, key informants from the provider group reported that they did not feel free to express true opinions or their needs in front of beneficiaries due to fear of leading to conflict (PRO4,5,6,7). This study found that the annual public hearing forums are flexible across regions based on the ability of regional organizers. However, some participants have reported being uncomfortable in expressing their opinions in such formal settings or when confronted with potential opponents. Therefore, the annual public hearings only partially meet the criteria of appropriate participation approach.

| Social learning
Public participation supports social learning among members of society, where participants listen to each other and try to find solutions which accommodate mutual interests. 30

| Transparency
Transparency in public participation refers to the openness in framing issues and the mutual decision-making mechanisms among stakeholders. 10,20 The public participation process should be transparent, so the public can see how decisions are being made. 16 In order to measure transparency of annual public hearings, two indicators were considered. Firstly, whether the organizers explained how the issues of public hearings are framed. Secondly, whether the organizers developed and utilized mechanisms to attain mutual agreements on decision-making processes among participants.
For the public hearings, the NHSO released seven issues as a framework for the hearings. However, many key informants did not understand why these seven issue areas in the public hearing agenda were selected, and found that the issue areas were too general (PRO8) and not aligned with participants interests (PRO7,9; NHSR5).
In the participants' perspective, the issues were not clearly framed, and therefore, participants are unsure whether their issues fit with agenda items (PRO5,8).
'My concern was not related to the seven issues on the agenda. I would like to propose about primary health care, but I was not sure which agenda I should propose' This study found that the organizers only compile opinions from participants, with no mechanism to attain mutual agreements on decision making among participants. Key informant from the provider group felt their proposals were not fully captured as the organizers had already pre-prepared a conclusion (PRO2).
The public hearing organizer key informants indicated that the NHSO regional offices were responsive by immediately taking key actions as per the recommendations suggested by the participants (NHSR2,3,4,5,6). However, this was countered by the key informants from both provider and beneficiary groups that believed the organizers (NHSO regional offices) only complied issues without taking action, as there were many recommendations suggested that were not prioritized due to lack of mechanism for consensus and prioritization of issues of importance across stakeholders (PRO9; CSO2,5).
'We only go and voice our concerns. There is no process of consensus or agreement'.
As there were no explanations to the public on the reasonings behind the selection of seven issues in the public hearings and no mechanisms to achieve a consensus on the various proposals, the annual public hearings in Thai UCS do not meet the criteria of transparency.

| Influence over policy decision making
The public hearing organizers should inform the stakeholders and general public on how the public participation process influence policy decision making. 16,37 The perception of stakeholders that their contributions influence policy decision making bolsters stakeholders' enthusiasm for involvement in future participation. 38 In order to assess the influence of public hearings over the policy decision making, two indicators were used. Firstly, whether the participants are informed on the process by which their opinions are considered by policy makers. Secondly, whether the policy decisions reflecting the public hearing outcomes are published and publicly available.
This study found that the organizers always inform the participants that the output of the public hearings will be considered by the

| CON CLUS I ON AND RECOMMENDATIONS
This study evaluated the annual public hearings for the improvement of UCS performance by applying a six criteria evaluation framework.
The evaluation was based on the experiences and opinions of 29 key informants from various stakeholder groups and analysed through categorizing the information into the pre-determined evaluation criteria.
The evaluation found that the public hearings fully meet the criteria of influence over the policy decision making as several key policy reforms have been generated from public hearing findings. The public hearings partially meet the criteria of appropriate participation approach and social learning. However, the public hearings have a deficiency in meeting the three other criteria of inclusiveness and representativeness of participants due to low diversity of participants and lack of involvement from wider public; adequacy of information due to lack of information provision and absence of stakeholders' preparation; and transparency due to lack of capacity to reach consensus on the proposals across different stakeholders by the moderator of the public hearings.
There are some limitations to this study. Firstly, due to budget constraints, the authors could only randomly select 5 out of the 13 NHSO regions as study sites; therefore, the results may not be nationally representative. Secondly, the evaluation was dependent on either positive or negative experiences of key informants which may introduce biases towards certain issues or organizations.
The strength of the annual public hearing is that it is mandatory by law through the NHSA. The public hearings not only result in policy and management responses to improve the UCS performance, but also serve as a key platform for health-care providers and beneficiaries to meet and resolve common problems through interactive discussions from different perspectives, actors and interests in a constructive manner.
Three recommendations emerged from this study. Firstly, the study found that the NHSO allows its regional offices to organize the public hearing, which has been gradually adapted to suit the local context, culture and norms of each region. However, the public hearing organizers should inform the stakeholders in advance about the agenda and process of the public hearing, and how to prepare their proposals and recommendations. Participating in complex issue like UCS requires the support from organizers to prepare proposals.
We agree with Sinclair and Diduck 39 that empowering participants should be done such as providing funding support and access to experts who can assist in understanding the system and prepare for active participation is important.
Secondly, this study found that public hearing organizers only compile the issues from participants, with no mechanisms to reach consensus on which proposal is accepted and is prioritized for further actions. Therefore, the public hearing organizers should identify experienced facilitators to navigate the discussions across stakeholders with different interests and concerns. The facilitators should support deliberative discussions among stakeholders, especially health-care providers and beneficiaries that often have different opinions and insist on their own perspectives, to reach a consensus and agree on issues that are to be prioritized. This study found that approaches which push too much towards compromise does not support social learning. We agree with Schusler, Decker and Pfeffer 40 that the moderator should urge participants to bring up the conflicting points of view in order to identify shared values and common solutions which require further deliberation in the future.
Lastly, the study found that the NHSB fully used the findings and recommendations from public hearings to improve UCS performance; however, the lack of effective communication to stakeholders is a major gap. Influence of public hearings over decisions has positive effect on acceptability by all concerned parties. 41 Therefore, the NHSO should establish feedback mechanisms for timely reporting of policy and management responses to all concerned stakeholders and share them to the wider public beyond the individual, group or organizations that have attend the public hearings.

ACK N OWLED G EM ENT
The authors thank Parinda Sanerattanaprayoon, Nisarat Thammarat, Supannee Kumpermpool, Chanathip Marom and Sirilak Faktim for coordinating with several stakeholders during field work and data collection.

CO N FLI C T O F I NTE R E S T
Authors declare that they have no conflict of interest.

DATA AVA I L A B I L I T Y S TAT E M E N T
Data sharing is not applicable to this article as no new data were created or analysed in this study.