Policy and practice suggestions to improve performance on the UNAIDS 90‐90‐90 targets: Results from a nominal group technique with HIV experts in Southwest Ethiopia

Abstract Objective This paper aims to evaluate the potential solutions to address negative outcomes of HIV care and treatment, that were proposed by HIV care providers, researchers and HIV programme managers in Southwest Ethiopia. Methods A nominal group technique (NGT) was conducted with 25 experts in December 2017 in Jimma, Southwest Ethiopia. The NGT process included (a) an analysis of the previously qualitative study conducted with various Ethiopian HIV stakeholders who proposed possible solutions for HIV care and treatment; (b) recruitment of a panel of HIV experts in policy and practice to rate the proposed solutions in Ethiopia before a discussion (first round rating); (c) discussion with the panel of experts on the suggested solutions; and (d) conducting a second round of rating of proposed solutions. Content analysis and Wilcoxon signed rank test were applied to analyse the data. Results Eighteen of the 25 invited panel of experts participated in the NGT. The following proposed solutions were rated and discussed as relevant, feasible and acceptable. In order of decreasing importance, the solutions were as follows: filling gaps in legislation, HIV self‐testing, the teach‐test‐link‐trace strategy, house‐to‐house HIV testing, community antiretroviral therapy (ART) groups, providing ART in private clinics and providing ART at health posts. Conclusions The current study findings suggested that, to address HIV negative outcomes, priority solutions could include mandatory notification of partner's HIV status, HIV self‐testing and the involvement of peer educators on the entire HIV care programme.

improved knowledge and trust in ART care, supportive environment from family, provider-initiated HIV counselling and testing services (PICT), and voluntarily HIV testing and counselling services (VCT).
The qualitative paper 25 also identified the following barriers that led to poor HIV care and treatment outcomes: HIV stigma, poor knowledge and trust in ART care, poor access and availability to ART care services, direct and associated cost of ART, structural factors resulting from patriarchal society, alternative perspectives of traditional healers and fragmented health-care system. As described above, to address these barriers, the qualitative paper reported a set of proposed solutions to improve HIV policy and practice in Ethiopia. 25 This paper is an extension of our previous qualitative paper. 25 Following the qualitative study, 25 we undertook a nominal group technique (NGT) to rank the relevance, feasibility and acceptability of the proposed solutions receiving expert advice from key stakeholders.
These experts comprised: academics, service providers and HIV programme managers. The aim of the current paper is to report on a consensus development study, using the NGT, which attempted to rank these solutions. The NGT outcomes are expected to inform measures to enhance and strengthen HIV care practices, guide future HIV policy development and generate new research ideas for further studies to improve Ethiopia's performance to the UNAIDS targets.

| Nominal Group Technique (NGT) study design
The present study used the NGT, a design for consensus development, [26][27][28] to rank the proposed solutions provided by our qualitative paper 25 and to improve HIV care and treatment in Ethiopia. As a planning technique, NGT involves four main steps: (a) silent generation of idea(s), (b) a round-robin, (c) clarification and voting and (d) ranking or rating. 29 The NGT can be modified, for example, by generating ideas initially from a literature review or an exploratory study instead of 'silent generation'. 30,31 Box 1 presents the details of the modified NGT process employed for the current study.

| Size and selection of NGT expert participants
Guided by Chitu and colleagues, 32 appropriate experts were identified and invited via e-mail to attend the NGT. We utilized consensus (Adapted from Jones & Hunter, 1995) HIV: human immunodeficiency virus. development techniques provided elsewhere, 28,33 for the definition of an 'expert'. For experts to participate, we: (a) prepared a list of relevant disciplines and organizations before nominating names of the experts; (b) developed a list of potential experts from the identified disciplines and organizations described in step one; (c) contacted experts listed in step 2 and requested them to nominate other experts; (d) ranked experts listed in step 2 and 3 according to their expertise, disciplines and organization; and (e) invited experts in order of their ranking until the targeted sample size was achieved. Twenty-five (25) participants-including HIV programme managers, researchers and service providers, were invited to participate in the NGT, and only 18 accepted the invitation to participate.

| NGT idea generation process
We modified the 'silent generation' step by obtaining the ideas from our previous qualitative paper 25 and from further suggestions made by the participants of the NGT process. From the findings of our qualitative paper, the following seven potential solutions related to HIV testing and treatment were proposed. These included (a) HIV self-testing, (b) house-to-house HIV testing, (c) the peer educatorsled teach-test-link-trace strategy, (d) ART provision in private clinics, (e) community ART groups, (f) ART provision in health posts and (g) filling the gaps in the legislation. Box 2 describes the definitions of these interventions.

| NGT discussion and ranking of solutions process
The process of the current NGT study was facilitated by the three study authors (HAG, KW and LM). Participants were asked to rate or evaluate each suggestion based on criteria of relevance, feasibility and acceptability (Box 2) using a self-administered questionnaire. The questionnaire had a three-point Likert scale as follows: agree (1), neural (2) and disagree (3), and had a free space for explanatory remarks. Two rounds of rating were conducted by the experts. For the initial rating round (R-1), experts were asked to rate the seven suggested solutions obtained from the qualitative

Box 2 Operational definitions of seven interventions for HIV care and treatment and three criteria of prioritization
The operational terms used for the seven suggestions to improve HIV care and treatment are defined as follows.
• HIV self-testing: refers to a screening process whereby a person who wants to know one's HIV status collects a specimen, perform a test and interpret the result in private.
• House-to-house HIV testing: refers to conducting HIV testing in every house by HEWs or trained lay counselors or peer educators-this process includes collecting a specimen, performing a test, interpreting the result and referral for further follow-up test or linkage (if the result is positive).
• Teach-test-link-trace strategy: involves formally employing and assigning of peer educators (HIV + persons disclosed themselves publicly) with HEWs (health extension workers) to teach the community about HIV, conduct HIV testing, linking into ART care and trace lost patients coined as Teach-test-link-trace strategy (TTLT).
• ART in private clinics: comprises the provision of ART care in private health clinics by the health workers employed in the clinic. ART will be provided for free by the government.
• Community ART groups: refers to a process whereby stable HIV + persons (who disclose publicly) living in near places establish a group and take their medications turn by turn or in rotation. They choose a leader who arranges monthly meeting to count pills and check the overall ART adherence. The people on ART will be told to come to the clinic whenever they feel ill.
• ART in health posts: involves the provision of ART in health post by HEWs. Health post is a grassroot level of primary health-care structure in Ethiopia with two community health workers (named health extension workers) serving on average 5000 people.
• Filling gaps in legislation: refers the need of legislation to suing an HIV + man who does not disclose his status to his wife after repeated counselling (as this prevents the woman from timely engagement to HIV care and prevent HIV transmission to child (if pregnant)) or vice versa. Another scenario is the need of legislation to suing religious leaders or witch doctors who declare HIV cure while not-as this is a false witness and against legislation of the nation. In addition, if the religious leaders or witch doctors tell patients to throw the pills and if patients die or sick seriously as a result of this, he/she is responsible to the death or attempt, and this is against legislation of the nation.
The operational criteria used during NGT to rate or evaluate proposed suggestion in a self-administered questionnaire included: • Relevance: refers to a scientific or societal benefit of a specified intervention to the context of HIV diagnosis, treatment linkage and retention in care.
• Feasibility: refers to technical, logistical, cultural (including religious) and legal considerations to implement a specified intervention.
• Acceptability: refers how well is received by the target population considering attitude, burden, effectiveness, ethics and opportunity cost. study. 25 Guided by the following question, 'What new programs do you suggest to improve HIV diagnosis, ART linkage or ART retention apart from these suggested solutions?' the NGT participants provided additional suggestions based on their expertise. The old and new suggestions were merged, but the new not changed the number of potential solutions to be ranked. After evaluation of the R-1 rating, a rigorous discussion was held. Participants were asked to re-rate (R-2) the solutions and provide justification if they did differ significantly with group opinion. After completion of the rating, facilitators collected the rating as part of data. The entire NGT study data comprised the quantitative component derived from NGT rating and the qualitative component comprising the audiorecording of the NGT discussions, the notes made from the justifications remarks and researchers' notes made during the NGT processes. Data were analysed as described below.

| NGT data management and analyses
We applied descriptive and inferential statistics to analyse the quantitative component of the NGT. To examine the statistical difference between the ratings in R-1 and R-2, we used a Wilcoxon signed rank test. 34 Thus, 21 hypotheses (7*3, ie seven suggested solutions and three criteria for each) were tested. The hypotheses were as follows: H A (alternative hypothesis): There was a difference in rating of the suggested solutions between rounds 1 and 2.
The qualitative data were analysed using content analysis 35 in six steps as follows. We: (a) transcribed audio-recording, notes from justification of made remarks and field notes, (b) familiarized with data through repetitive reading of the transcripts, (c) identified, listed notes (codes) and tallied the codes, (d) indexed the nodes and classified them, (e) read through the classification to name each theme and (f) reviewed the steps from a-e to map, interpret and confirm that the categories were exhaustive, mutually exclusive and independent. 35  Health), and five were lecturers. 36 All panel members took part in Round-1 (R-1) rating and the discussion; 16 members participated in Round-2 (R-2) rating. Two experts, a clinician and a programme coordinator, dropped out from the discussion due to unforeseen emergency commitments. Tables 1 and 2 describe individual rating of proposed measures for improving HIV care and treatment in R-1 and R-2, respectively.

| RE SULTS
After calculating the average score of relevance, feasibility and acceptability, in R-1, the seven measures were recommended in the following order: (a) legislation, (b) ART in private clinics, (c) HIV self-testing, (d) teach-test-link-trace strategy, (e) community ART groups, (f) ART in health post and (g) house-to-house HIV testing.
In R-2, following discussion and recalculation, the proposals, in their average score, were re-ranked as follows: (a) legislation, (b) HIV self-testing, (c) teach-test-link-trace strategy, (d) house-to-house testing, (e) community ART groups, (f) ART in private clinics and (g) ART in health posts. In both rounds, legislation and self-testing were among the top three recommended solutions, while ART in health post was among the least popular suggestions. Following the discussion, the teach-test-link-trace strategy moved up from fourth to third place and house-to-house HIV testing moved up from seventh to fourth place. Meanwhile, ART in private clinics dropped from second to sixth place. The Wilcoxon signed rank test showed no statistical difference between R-1 and R-2 for all suggestions (Table 3). In the open discussion, all of the suggested measures were discussed, and expert panels have forwarded directions on how these programmes can be implemented. Below, we have summarized the findings with the suggested solutions.

| Filling the gaps in the legislation (Legislation)
The participants strongly agreed that new legislation is needed Note: HIVST is a process whereby a person who wants to know one's HIV status collects a specimen, perform a test and interpret the result in private-this is a screening test. H2H refers to conducting HIV testing in every house by HEWs or trained lay counsellors or peer educators-this process includes collecting a specimen, performing a test, interpreting the result and referral for further follow up test or linkage (if the result is positive). TTLT involves formally employing and assigning of peer educators (HIV + persons disclosed themselves publicly) with HEWs (health extension workers) to teach the community about HIV, conduct HIV testing, linking into ART care and trace lost patients coined as Teach-test-link-trace strategy (TTLT). ARTHP is the provision of ART in health post by HEWs. ARTPC is the provision of ART care in private health clinics by the health workers employed in the clinic. ART will be provided for free by the government. CAGs is a process whereby stable HIV + persons (who disclose publicly) living in near places establish a group and take their medications turn by turn or in rotation. They choose a leader who arranges monthly meeting to count pills and check the overall ART adherence. The people on ART will be told to come to the clinic whenever they feel ill. Legislation refers the need of legislation to suing an HIV + man who does not disclose his status to his wife after repeated counselling (as this prevents the woman from timely engagement to HIV care, and prevent HIV transmission to child (if pregnant)) or vice versa. Another scenario is the need of legislation to suing religious leaders or witch doctors who declare HIV cure while not-as this is a false witness and against legislation of the nation. In addition, if the religious leaders or witch doctors tell patients to throw the pills and if patients die or sick seriously as a result of this, he/she is responsible to the death or attempt, and this is against legislation of the nation.
resistance. To address the above-mentioned problems, the expert panel recommended a legislative framework that would make traditional healers and/or religious curers responsible for their actions.  1  2  3  3  2  3  1  1  1  3  3  3  3  2  3  1  1  1  1  1  1   P2  1  2  2  1  2  2  1  1  2  2  2  2  3  3  3  2  2  2  1  1  1   P3  1  1  1  1  1  1  2  2  2  2  2  2  3  3  3  2  2  2  1  1  1   P4  1  3  1  2  3  3  1  2  3  1  3  2  1  1  1  1  1  1 1  1  1  2  2  2  1  1  1  2  2  2  3  3  3  2  2  2  1  1  1   P16  1  3  3  1  3  1  1  3  1  2  3  1  1  2  3  2  3  1  1  Note: HIVST is a process whereby a person who wants to know one's HIV status collects a specimen, perform a test and interpret the result in private-this is a screening test. H2H refers to conducting HIV testing in every house by HEWs or trained lay counsellors or peer educators-this process includes collecting a specimen, performing a test, interpreting the result and referral for further follow-up test or linkage (if the result is positive). TTLT involves formally employing and assigning of peer educators (HIV + persons disclosed themselves publicly) with HEWs (health extension workers) to teach the community about HIV, conduct HIV testing, linking into ART care and trace lost patients coined as Teach-test-link-trace strategy (TTLT). ARTHP is the provision of ART in health post by HEWs. ARTPC is the provision of ART care in private health clinics by the health workers employed in the clinic. ART will be provided for free by the government. CAGs is a process whereby stable HIV + persons (who disclose publicly) living in near places establish a group and take their medications turn by turn or in rotation. They choose a leader who arranges monthly meeting to count pills and check the overall ART adherence. The people on ART will be told to come to the clinic whenever they feel ill. Legislation refers the need of legislation to suing an HIV + man who does not disclose his status to his wife after repeated counselling (as this prevents the woman from timely engagement to HIV care, and prevent HIV transmission to child (if pregnant)) or vice versa. Another scenario is the need of legislation to suing religious leaders or witch doctors who declare HIV cure while not-as this is a false witness and against legislation of the nation. In addition, if the religious leaders or witch doctors tell patients to throw the pills and if patients die or sick seriously as a result of this, he/she is responsible to the death or attempt, and this is against legislation of the nation.

| Teach-Test-Link-Trace strategy (TTLT)
In the past, HIV was thought to be a death sentence. However, be- So, get ready for the resources and involve patients.
We are seeing the benefit live. Participants suggested that the use of peer educators in HIV care is important for the following reasons: (a) they themselves are HIV positive and can easily convince people to test for HIV via sharing of their experiences; (b) they are known to have high capability of maintaining confidentiality and people will trust them better compared to health extension workers (HEWs); and (c) they can also secure a job from the program. Nevertheless, the experts described that the issue of logistics may limit the implementation of such a programme.

| House-to-House HIV testing
The majority of panel members agreed that house-to-house HIV testing is not easily feasible or acceptable. For example, the experts noted, if the house-to-house HIV testing is carried out using HEWs, Although less feasible and acceptable, the programme was rated relevant. In this regard, the experts acknowledged some benefits would include increasing access to community, that is, services to be at home, and in some ways, it might reduce HIV-related stigma. It was also suggested that this programme can also be integrated with HIV self-testing.

| Community ART groups
The experts rated the community ART groups programme relevant but less acceptable and feasible for the following reasons: (a) it was feared that people may not disclose their status; and (b) it was noted that ART could be abused, that is sold or exchanged by the group of leaders, leading to non-adherence and drug resistance.
However, some experts noted that the programme is novel and suggested that it may be implemented in rural areas, since many people in rural areas travel long distances (hundreds of kilometres) to get in to ART services. Furthermore, the programme may help HIV group members to establish their own social networks. For example, an Associate Professor of Reproductive Health said, … community ART group is so novel, especially to our society-our people travel hundreds of kilometers to get ART services. If we have people who disclose their HIV status, they can take the pills turn by turn. This is fantastic.

| ART in Private Clinics
The panel members had major reservations to providing ART in private clinics for the following reasons including:(a) a potential to misuse ART drugs, (b) treatment could be expensive due to high costs associated with HIV care and (c) the likelihood of failure of private clinics allocating enough time to monitor patient adherence to the drug regime or to trace lost patients. Some experts also suggested the quality of HIV care services in private clinics might be poorer than in public clinics due to profit-making needs.
However, some experts supported decentralization of ART into private clinics. It was envisaged that some HIV patients who could afford these services would prefer to use private clinics to reduce the chance of beings seen by others. On this basis, a plan to commence ART services in one private clinic in Jimma Town Health Office in collaboration with Regional Health Bureau gained some support from the panel. As one HIV programme manager observed: We have heard that the district health office will start ART in a recognized private clinic… this is good and become as an alternative option. I know there are rich people who prefer to go to private clinics for collecting drugs.

| ART in health post
The panel members suggested that HEWs are not suitable to providing optimal ART service in the health post for the following reasons:

| Limitations
The NGT of the present study has some limitations. Firstly, although experts suggested ideal recommendations known in theory, these could be less applicable in practice, the so-called 'self-fulfillment prophecy'. 50 Secondly, reaching consensus was an additional challenge in the NGT, a common limitation which has been noted in other consensus methods. 28 Nevertheless, ultimate consensus was not the only aim, and our suggested solutions explored the minority suggestions to cater for feasibility in practice. Thirdly, even though we identified the expert panellists carefully, some disciplines, geographic areas and professions were possibly not as well represented as others. Fourthly, there was no statistical difference for the suggested solutions between the two rounds, and this could have been due to having compacted values of the Likert scale instrument.

| CON CLUS IONS
The top three interventions rated by experts included mandatory notification of HIV status to partner, HIV self-testing and the in- and the current study gave a glimpse of the possibility to roll these programs out in Ethiopia. However, before considering to implement these interventions, a nationwide based study should be carried out using multi-method approach.

ACK N OWLED G EM ENTS
We are grateful to the study participants and their respective institutions.

CO N FLI C T O F I NTE R E S T
All authors declare that they have no competing interests.

E TH I C S A PPROVA L
The NGT received ethical approvals from the Social and Behavioural Research Ethics Committee of Flinders University, South Australia

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request.