Results from a difference-in-differences evaluation of health facility HIV and key population stigma-reduction interventions in Ghana.

Abstract Introduction Stigma undermines all aspects of a comprehensive HIV response, as reflected in recent global initiatives for stigma‐reduction. Yet a commensurate response to systematically tackle stigma within country responses has not yet occurred, which may be due to the lack of sufficient evidence documenting evaluated stigma‐reduction interventions. With stigma present in all life spheres, health facilities offer a logical starting point for developing and expanding stigma reduction interventions. This study evaluates the impact of a “total facility” stigma‐reduction intervention on the drivers and manifestations of stigma and discrimination among health facility staff in Ghana. Methods We evaluated the impact of a total facility stigma‐reduction intervention by comparing five intervention to five comparable non‐intervention health facilities in Ghana. Interventions began in September 2017. Data collection was in June 2017 and April 2018. The primary outcomes were composite indicators for three stigma drivers, self‐reported stigmatizing avoidance behaviour, and observed discrimination. The principal intervention variable was whether the respondent worked at an intervention or comparison facility. We estimated intervention effects as differences‐in‐differences in each outcome, further adjusted using inverse probability of treatment weighting (IPTW). Results We observed favourable intervention effects for all outcome domains except for stigmatizing attitudes. Preferring not to provide services to people living with HIV (PLHIV) or a key population member improved 11.1% more in intervention than comparison facility respondents (95% CI 3.2 to 19.0). Other significant improvements included knowledge of policies to protect against discrimination (difference‐in‐differences = 20.4%; 95% CI 12.7 to 28.0); belief that discrimination would be punished (11.2%; 95% CI 0.2 to 22.3); and knowledge of and belief in the adequacy of infection control policies (17.6%; 95% CI 8.3 to 26.9). Reported observation of stigma and discrimination incidents fell by 7.4 percentage points more among intervention than comparison facility respondents, though only marginally significant in the IPTW‐adjusted model (p = 0.06). Respondents at intervention facilities were 19.0% (95% CI 12.2 to 25.8) more likely to report that staff behaviour towards PLHIV had improved over the last year than those at comparison facilities. Conclusions These results provide a foundation for scaling up health facility stigma‐reduction within national HIV responses, though they should be accompanied by rigorous implementation science to ensure ongoing learning and adaptation for maximum effectiveness and long‐term impact.

through interpersonal acts of discrimination that can range from verbal and physical abuse to social isolation and gossip [17,18]. Perceived stigma is the perception of the prevalence of stigmatizing attitudes and behaviours, for example within the general community or in a health facility [19][20][21][22]. Anticipated stigma is the fear that stigma will happen, if for example a family member learns a person is living with HIV, whether or not it is actually experienced [23]. Perceived and anticipated stigma are sometimes described together as felt stigma [24,25]. Individuals facing stigma may also internalize stigma [22,25,26]. Intersecting stigma occurs when individuals who have identities linked to multiple marginalized groups, for example based on socio-economic status, race, sexual orientation, gender identity or health conditions face multiple intersecting stigmas [27,28]. Embedded within human rights imperatives, recognition of the necessity and urgency of responding to stigma and resulting discrimination is critical to achieving global targets for HIV testing, linkage to care and viral suppression. As stated in the Global Partnership for Action to Eliminate All Forms of HIV-Related Stigma and Discrimination [29]:"Without addressing HIV-related stigma and discrimination, the world will not achieve the goal of ending AIDS as a public health threat by 2030" (p. 5).
Stigma occurs and needs to be addressed in all life spheres and institutionsin the home, community, workplace, places of worship, schools, health care facilities, etc. The health facility offers a logical place to develop and scale-up a national response to stigma for multiple reasons. It is, in most places, the gateway into HIV treatment and often prevention. As a chronic condition, HIV requires lifelong engagement with the health system. In addition, health care workers are members of their communities and are often looked to for guidance on health and other issues [30][31][32]. Health workers are a potentially powerful force not only to reduce stigma in their health facilities, but to be change agents in their homes and communities. Stigma also undermines the health workforce, impacting the health and wellbeing of health workers [33]. The importance of tackling stigma in health facilities is underscored by the UNAIDS-led global Agenda for Zero Discrimination in Health-care Settings [34] and corresponding policy guidance [35]. Global initiatives like the Fast Track Cities have also made addressing stigma a key pillar of their response, including a focus on health facilities [36].
In Ghana in particular, stigma and discrimination remain a pervasive issue, as documented both through measurement of stigmatizing attitudes in the general population [57] and through the experiences of people living with HIV (PLHIV) [57][58][59]. In 2014, only 8.0% of women and 14.1% of men in the general population expressed accepting attitudes as measured by four standardized indicators (caring for a family member in one's home, buying fresh vegetables from PLHIV, allowing a teacher living with HIV to continue working, and keeping a family member's positive HIV status secret). PLHIV reported that experienced stigma and discrimination was significant, including social exclusion and gossip, while noting that fear of transmission through casual contact was a key driver of stigma [57]. A health facility stigma assessment in Ghana [58] identified the physical layout of healthcare services; lack of in-depth knowledge about HIV; fear of contracting the disease; and low levels of respect and dignity based on cultural and moral grounds as contributing factors for stigma and discrimination experienced by PLHIV in health facilities. HIV stigma in Ghana has been documented as a barrier to antiretroviral therapy (ART) services [60]; voluntary testing and counselling (VCT) uptake [61,62]; and to parents informing children living with HIV of their seropositive status [5,63].
In its contribution to literature outlining potential interventions within health facility stigma-reduction, this study evaluates the impact of a "total facility" stigma-reduction intervention on the drivers and manifestations of stigma and discrimination among health facility staff in Ghana.

| Setting and Intervention
We evaluated the impact of a total facility stigma-reduction intervention in five health facilities in Ghana compared to five non-intervention facilities selected to be broadly comparable with regard to facility size (staff complement and patient load); subnational region; and type of facility (district-level hospitals). The initial phase of the study collected stigma data from the four highest HIV-caseload civilian facilities in five regions -Ashanti, Brong Ahafo, Eastern, Greater Accra and Westernfor a total of 20 facilities. From these, a subset of 10 facilities were selected for the second phase, with one intervention facility and one comparison facility selected from each region based on staff size (medium-sized, district-level facilities) and facility interest to participate. There was no randomization of facilities.
After baseline data collection in June 2017 from health facility staff, we conducted participatory data dissemination and review workshops with staff from all 20 first-phase facilities in August/September 2017. In these workshops, staff identified stigma and discrimination challenges and then generated potential solutions. Non-intervention facilities received no further intervention via the study during this period, though some facilities independently conducted stigma-reduction activities. After the evaluation reported in this manuscript, the non-intervention facilities are in the process of receiving the intervention approach through scale-up led by a local Ghanaian organization with support from The Global Fund.
Targeting the whole facility beyond HIV services, the intervention approach included a two-day participatory stigma-reduction training for all staff levels (clinical and non-clinical) with delivery by staff and clients from the facilities who were trained as stigma-reduction facilitators. Trainings were delivered between September and November 2017 to all categories of staff, with a target of reaching 70% of the facility workforce. The trainings were based on pre-existing global training materials that are available online [64]. These are targeted towards actionable HIV stigma drivers and were adapted to the Ghanaian context through an in-country stakeholder workshop. Final Ghanaian training materials are available online [65]. Ultimately, 1228 staff members were trained -79% of staff at intervention facilities (with individual facilities ranging from 61% to 97% training coverage). Each facility also created an eight to ten member "champion team, " which was provided $5000 USD to develop facility-specific ancillary activities, including launch events, anti-stigma-and-discrimination banners and posters, additional staff trainings, printed codes of ethics, reporting mechanisms, and staff nametags to enable identification and reporting of stigma and discrimination. More detail on the intervention is available online in a final project report [66]. We conducted follow-up surveys in April 2018 and disseminated results to staff at the 10 study facilities as well as at a national dissemination meeting for policymakers, donors and community stakeholders.

| Sampling
We conducted repeated cross-sectional surveys, stratified by health facility and job category, of a random sample of staff at the five intervention and five comparison facilities two to three months before and five to six months after the training portion of the intervention. The total sample size was 2308 participants (1154 in each of the pre-and post-intervention periods). To preserve anonymity, we did not attempt to identify the same participants for the follow-up survey. The sample was designed to provide 80% power to detect a 10-percentage point change in the only indicator for which pre-baseline estimates were available: observation of colleagues gossiping about PLHIV. The sampling frame consisted of a complete listing of all staff from which an off-site statistician generated the target sample. If respondents were unavailable or declined to participate, substitutes were chosen in order from a predetermined list. The survey was designed to be self-administered if participants could read and to be administered by interviewers to participants who could not read or preferred the administered survey.

| Instruments and Variables
Survey instruments are provided in Data S1. The survey included modules to capture demographic information; three actionable stigma and discrimination drivers (perceptions of health facility policies, fears of contracting HIV in the workplace, and attitudes towards PLHIV and key populations); and stigma and discrimination manifestations (self-reported unnecessary infection control behaviours and observed discrimination by other healthcare workers). The follow-up survey also included items about exposure to the intervention and perceived changes since baseline.
Survey questions were developed by adapting a globally validated tool for health worker stigma and discrimination [67]. Items were adapted during a one-day participatory workshop with Ghanaian collaborators and other stakeholders and then tested during a rapid pilot at one health facility which had not been selected for further surveys. Surveys were administered in English, Dangme, Akuapem Twi and Ga. English surveys were translated by professional translators and then checked by mother-tongue speakers on the research term and any issues resolved with the translators.
The primary outcomes outlined below were composite indicators for each domain, dichotomized from a series of items.

| Facility policies
One variable for reporting knowledge that facility guidelines protect PLHIV, men who have sex with men (MSM), and sex workers from discrimination; one variable for believing that the respondent would be punished if discriminating against PLHIV and all of four key populations (MSM, sex workers, sexually active adolescents and persons who inject drugs); and one variable for agreeing or strongly agreeing that the facility has adequate infection control policies and personal protective equipment and knowing that the facility has both a posted post-exposure prophylaxis (PEP) policy and PEP access.

| Fear
Reporting being a little worried, worried, or very worried to conduct one or more of four care activities with clients living with HIV, for fear of HIV acquisition.

| Attitudes
Three variables for agreeing or strongly agreeing with one or more stigmatizing statement about PLHIV in general, women living with HIV, and MSM; and one variable for agreeing or strongly agreeing that the respondent would prefer not to provide services to one or more key population.

| Self-reported provision of care in a stigmatizing manner
Reporting usually avoiding physical contact, wearing double gloves, wearing gloves during all aspects of care, or using extra precautions with PLHIV but not other patients.

| Observed stigma and discrimination
Observing one or more instances of healthcare workers at the same facility being unwilling to care for, providing poorer care quality to, talking badly about, or disclosing status about PLHIV or one of four key populations in the last six months.
A secondary outcome was participant's general perception about whether behaviour towards clients had changed in the facility over the year prior to the endline survey, dichotomized as a little/much better versus no change or a little/much worse.
The principal intervention variable was whether the respondent worked at an intervention or comparison facility. Analyses were by intention to treat, so participants were classified as at intervention facilities even if they did not individually receive training, as intention-to-treat analysis was expected to capture the effect of average programme implementation. We also collected demographic variables for use in adjusted analyses: age (18 to 24 years, 10 year increments from 24 to 54, and 55+); region; sex; staff category (senior medical, other medical, and administrative or support staff); tenure at the facility (<2 years, two to five years, 5+ years); whether the respondent has ever worked in an HIV-specific department; and quintiles of the number of PLHIV the respondent reports personally providing care to in the past month.

| Statistical analyses
We present basic descriptive statistics of the sample in Table 1. In Table 2, we report the number and proportion of respondents in the follow-up sample who reported exposure to the main intervention components: trainings in infection control; patients' rights to informed consent, privacy, and confidentiality; HIV stigma and discrimination; and key population stigma and discrimination; or who participated in any other stigma and discrimination reduction activity since August 2017 when the intervention began.
Intervention effects are presented in Table 3 as differences-in-differences: the difference in before-to-after changes between intervention and comparison areas for each composite indicator. To estimate differences-in-differences, we fit saturated linear probability models using generalized linear models with an identity link function and binomial error distribution. Each outcome was regressed on indicator variables for intervention (intervention vs. comparison); time (pre vs. post-intervention); and their interaction, with the coefficient on the interaction term being the estimated difference-in-differences (DID). Primary analyses were conducted on the full sample, and secondary analyses were restricted only to clinical providers. All analyses were by intention-to-treat, and we treated facilities designated as comparison but which partially implemented as comparison facilities in the analysis.
In addition to unadjusted estimates, we estimate intervention effects using inverse probability of treatment weighting (IPTW) [68,69]. The IPTW models generated weights to balance the following covariates across the four intervention-bytime groups: subnational region and staff members' age, sex, employment category, employment tenure, work history in an HIV-specific department and number of PLHIV personally treated. Separate IPT weights were constructed for the full sample and the clinical staff subsample. We then fit DID models identical to the primary analyses except incorporating IPT weights. We provide full details of the IPTW approach and balance diagnostics in Data S2. All analyses adjust standard errors for clustering by facility.
In the main analysis, we did not adjust for multiple comparisons because all outcomes were theoretically informed and predetermined and no analyses were added post hoc. Because some readers might expect adjustment; however, we present an analysis in Data S4 that reports risk of false discovery because of multiple outcomes. We present false discovery rates as q-values produced via Simes' method. We used Stata v15.1 for all analyses and statistical code and output are provided in Data S3.
The institutional review boards at Health Media Labs and the Ghana Health Services Ethical Review Board approved the study, and all participants provided written informed consent.

| RESULTS
Characteristics of the analytical sample of 2308 participants (1154 in each of the pre-and post-intervention periods) are in Table 1. They are broadly comparable across the four intervention-by-time groups, and balance of characteristics before and after IPT weighting is provided in Data S2. IPT-weighted models met generally accepted criteria for balance of potential confounders [70].
As expected, exposure to trainings and other activities to reduce stigma and discrimination was higher among intervention than comparison staff during the post-intervention period, with each of the trainings reaching at least three-quarters of intervention respondents, and about two-thirds being exposed to at least one additional intervention activity ( Table 2). More than half of comparison respondents reported receiving training on infection control and universal precautions (65%) and informed consent, privacy, and confidentiality (57%) during the same time period. A lower but still substantial percentage of comparison respondents reported receiving HIV stigma and discrimination training (45%), key population stigma and discrimination training (33%) and other relevant activities (32%).
Using intention-to-treat analyses, we observed favourable intervention effects for all outcome domains except for stigmatizing attitudes in both the full sample and the medical staff-only subsample (Table 3). In the IPTW-adjusted analyses of the full sample, preferring not to provide services to PLHIV or a key population member improved by 11.1 percentage points (95% CI 3.2 to 19.0) more in intervention than comparison facility respondents. Knowledge of policies to protect against discrimination (DID = 20.4 percentage points [pp], 95% CI 12.7 to 28.0); belief that discrimination would be punished (DID = 11.2pp, 95% CI 0.2 to 22.3); and knowledge of and belief in the adequacy of infection control policies (DID = 17.6pp, 95% CI 8.3 to 26.9) all improved significantly. Observing stigma and discrimination incidents improved by 7.4 percentage points more among intervention than comparison facility respondents (95% CI À 0.3 to 15.2), though this result was only marginally significant (p = 0.06). Results were very similar in the unadjusted analyses.
We found comparable improvements in the subsample composed only of medical staff (Table 3). Here too, there were no significant improvements in stigmatizing attitudes, but all other outcome domains improved significantly with similar intervention effect estimates to those identified in the full sample. In addition to the domains measured in the full sample, two domains were limited only to medical staff. Fear of conducting at least one care activity improved by 24.7 more percentage  All estimates are constructed such that a positive DID is an improvement (even if the original measure was scaled so that higher numbers were an undesirable outcome). Fear and avoidance outcomes are restricted only to clinical staff because the sizable majority of non-clinical staff answered "not applicable" to the single item applicable to them. DID, difference-in-differences; IPTW, inverse probability of treatment weighting; MSM, men who have sex with men; PEP, post-exposure prophylaxis.
When asked whether there had been a change in behaviour towards PLHIV over the last year, intervention facility respondents were 19.0 percentage points (95% CI 12.2 to 25.8) more likely to say it was better in IPTW-adjusted analyses (Table 3). Results were similar in the medical staff subpopulation (18.0pp; 95% CI 11.2 to 24.9) and in unadjusted analyses. Substitution of false discovery rate q-values for p-values does not meaningfully change conclusions.

| DISCUSSION
Evidence continues to grow of the negative relationship of stigma to pre-exposure prophylaxis [1], HIV testing [2][3][4], linkage and retention in care [5][6][7], medication adherence [8][9][10][11][12][13] and viral load suppression [2,14,15]. More visible global recognition of stigma in fuelling the HIV epidemic and in undermining the HIV response has been forthcoming through recent global declarations [29,35,71]. This includes a recognition of the importance of addressing stigma in health facilities [72], yet concerted efforts to tackle stigma at any scale within countries are lacking, apart from Thailand [73]. One reason for such inaction may be the small, but growing, body of evidence of effective intervention models for HIV-stigma reduction, particularly in health facilities [46,[49][50][51][52][53][54][55][56]. This paper adds to that literature. This evaluation of the Ghana "total facility" intervention found strong and consistent evidence of improvements, across indicator domains, apart from stigmatizing attitudes. Fear of acquiring HIV while providing care for clients living with HIV improved significantly in intervention versus comparison facilities, as did associated stigmatizing avoidance behaviours such as double gloving. These findings mirror results of interventions in China [55] and Vietnam [56] that also included addressing fear of HIV transmission in the health facility as a key component of stigma-reduction interventions. A further study in China underscores the importance of addressing fear directly and through a focus on standard precautions as a key strategy in reducing health facility stigma and discrimination [74]. There was no statistically significant DID between the intervention and control facilities on the composite stigmatizing attitudes variable. This is not unexpected as changing deeply held attitudes may require a longer and more intense intervention than changing fear and behaviours based in incorrect knowledge or strengthening the facility environment to facilitate delivery of non-stigmatizing care. Additionally, more items in the composite attitude variables results in less change being expected because all of a respondent's stigmatizing attitudes would have to be reduced to alter a single "none-or-any" composite. That changing (and measuring change in) stigmatizing attitudes is potentially more complicated than for other immediately actionable drivers of stigma is also highlighted by a study in Vietnam, where the arm of the study that received a focus on social stigma (attitudes) had a similar drop in stigmatizing attitudes to the arm that received only a focus on fear-based stigma [56].
There are limitations to this study that should be noted. The short-time frame of the evaluation did not allow for assessment of longer-term intervention effects and therefore, could overestimate more transient effects of the intervention or underestimate those that require longer intervention periods. The nature of the setting, design and funding did not allow for randomization of the facilities to intervention or comparison. However, facility selection, the DID approach and IPTW should substantially account for differences between the intervention and comparison groups. Some stigma-reduction activities did occur in the comparison group of facilities, which likely diminishes some of the intervention effect. All facilities received their baseline results before the intervention began in the intervention sites, as required by ethical guidelines, which may have spurred some comparison facilities to begin addressing their baseline results. As presented in Table 2, respondents in the comparison arm did report participating in a range of training related to specific stigma drivers (e.g. Infection control and standard precautions) and other general, non-specified stigma-reduction activities. While the percent of staff reporting such exposure was significantly lower in comparison to intervention sites, the data does indicate some activities were happening in the comparison facilities. Social desirability bias is always present in stigmareduction studies. One way that the study worked to reduce this potential bias was by implementing data collection strategies to reassure health care staff respondents that their responses were anonymous, including self-filled questionnaires (where literacy was not a challenge); placing of questionnaires by respondents in envelopes which they sealed and that were put in a locked box; and not collecting any personal identifiers on the questionnaire. That the attitudes indicator, one that we would expect to be particularly sensitive to social desirability bias, was the one indicator that did not change significantly perhaps indicates a mitigation of social desirability in the study.
The rationale for addressing stigma for an effective HIV response and to achieve national and global targets is clearly underscored by the evidence of the negative link between stigma and testing [2,3], treatment [10][11][12][13] and viral load suppression [2,14,15]. The importance and urgency of responding to stigma in a consistent and scaled manner is visible in recent global calls for stigma-reduction initiatives [29,35,71,72]. Translating these global calls into action on the ground will require national HIV responses to include stigmareduction as a core feature, alongside prevention and treatment. Health facilities are a logical and feasible place to initiate this work, as demonstrated by this study and others. Ghana has begun building on this pilot intervention, expanding the intervention to other facilities with The Global Fund's support, while Thailand is in the midst of expanding approaches beyond a pilot study, with their stigma-reduction intervention currently implemented in over 100 health facilities. (Personal Communication, Dr. Taweesap Siraprapasiri, Thailand MOPH).

| CONCLUSION
Addressing HIV stigma is not only a public health imperative but a human rights one. Tackling HIV stigma in the health system is a logical place to develop and scaleup a national response to HIV stigma. This paper adds to the small but growing literature demonstrating that HIV stigma can effectively be addressed in health facilities and is a challenge welcomed by facility staff and management. This evidence provides a solid foundation for developing and testing the feasibility and efficacy of scaling up health facility stigma-reduction within countries. It will be critical that these efforts are accompanied by rigorous implementation science to ensure ongoing learning and adaptation to maximize effectiveness and long-term impact. Additional support for the analysis was provided by RTI, International. The content of this manuscript is the sole responsibility of the authors and does not necessarily reflect the views or policies of the Global Fund, the U.S. Agency for International Development or the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) and does not imply endorsement by the U.S. Government. We are grateful to all the study respondents, health facility staff and clients, for sharing their time and insights and to management of intervention and non-intervention facilities, who supported the participation of staff and clients. We are especially grateful to the stigma-reduction facilitators and champion team members for their dedication to stigma-reduction. In-country implementation of both research and intervention were carried out by a team of talented and dedicated staff from the Educational Assessment Research Center's (EARC) who worked in partnership with the Ghana AIDS Commission (GAC) and the National AIDS Control Program (NACP), with support from colleagues at HP+. eliminate all forms of HIV related stigma and discrimination. 22nd Annual International AIDS Conference. Amsterdam; 2018 74. Li L, Liang LJ, Lin C, Wu Z. Addressing HIV stigma in protected medical settings. AIDS Care. 2015;27 (12):1439-42.

SUPPORTING INFORMATION
Additional information may be found under the Supporting Information tab for this article.