CD4 count outperforms World Health Organization clinical algorithm for point-of-care HIV diagnosis among hospitalised HIV-exposed Malawian infants

Authors

  • Madalitso Maliwichi,

    Corresponding author
    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    • Corresponding Author Madalitso Maliwichi, University of North Carolina Project Lilongwe, Tidziwe Centre, 100 Mzimba Road, Kamuzu Central Hospital, Private Bag A104, Lilongwe, Malawi. E-mails: mmaliwichi@unclilongwe.org; maliwichim@yahoo.com

    Search for more papers by this author
  • Nora E. Rosenberg,

    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    2. Department of Medicine, Division of Infectious Diseases, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC, USA
    Search for more papers by this author
  • Rebekah Macfie,

    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    Search for more papers by this author
  • Dan Olson,

    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    Search for more papers by this author
  • Irving Hoffman,

    1. Department of Medicine, Division of Infectious Diseases, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC, USA
    Search for more papers by this author
  • Charles M. van der Horst,

    1. Department of Medicine, Division of Infectious Diseases, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC, USA
    Search for more papers by this author
  • Peter N. Kazembe,

    1. Baylor College of Medicine Abbott Fund Children's Clinical Centre of Excellence-Malawi, Lilongwe, Malawi
    Search for more papers by this author
  • Mina C. Hosseinipour,

    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    2. Department of Medicine, Division of Infectious Diseases, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, NC, USA
    Search for more papers by this author
  • Eric D. McCollum

    1. University of North Carolina Project Lilongwe, Lilongwe, Malawi
    2. Department of Pediatrics, Division of Pulmonology, Johns Hopkins School of Medicine, Baltimore, MD, USA
    Search for more papers by this author

Abstract

Objective

To determine, for the WHO algorithm for point-of-care diagnosis of HIV infection, the agreement levels between paediatricians and non-physician clinicians, and to compare sensitivity and specificity profiles of the WHO algorithm and different CD4 thresholds against HIV PCR testing in hospitalised Malawian infants.

Methods

In 2011, hospitalised HIV-exposed infants <12 months in Lilongwe, Malawi, were evaluated independently with the WHO algorithm by both a paediatrician and clinical officer. Blood was collected for CD4 and molecular HIV testing (DNA or RNA PCR). Using molecular testing as the reference, sensitivity, specificity and positive predictive value (PPV) were determined for the WHO algorithm and CD4 count thresholds of 1500 and 2000 cells/mm3 by paediatricians and clinical officers.

Results

We enrolled 166 infants (50% female, 34% <2 months, 37% HIV infected). Sensitivity was higher using CD4 thresholds (<1500, 80%; <2000, 95%) than with the algorithm (physicians, 57%; clinical officers, 71%). Specificity was comparable for CD4 thresholds (<1500, 68%, <2000, 50%) and the algorithm (paediatricians, 55%, clinical officers, 50%). The positive predictive values were slightly better using CD4 thresholds (<1500, 59%, <2000, 52%) than the algorithm (paediatricians, 43%, clinical officers 45%) at this prevalence.

Conclusion

Performance by the WHO algorithm and CD4 thresholds resulted in many misclassifications. Point-of-care CD4 thresholds of <1500 cells/mm3 or <2000 cells/mm3 could identify more HIV-infected infants with fewer false positives than the algorithm. However, a point-of-care option with better performance characteristics is needed for accurate, timely HIV diagnosis.

Abstract

Objectif

Déterminer pour l'algorithme de l’OMS pour le diagnostic de l'infection VIH sur le lieu des soins, les niveaux de concordance entre les pédiatres et les agents cliniques non-médecins, et comparer les profils de sensibilité et de spécificité de l'algorithme de l’OMS et différents seuils de CD4 par rapport aux tests PCR du VIH chez les nourrissons hospitalisés au Malawi.

Méthodes

En 2011, des nourrissons âgés de moins de 12 mois, exposés au VIH, hospitalisés à Lilongwe, au Malawi ont été évalués de façon indépendante avec l'algorithme de l’OMS à la fois par un pédiatre et un agent clinique. Du sang a été prélevé pour le dosage des CD4 et le dépistage moléculaire du VIH (PCR sur ADN ou ARN). En utilisant le test moléculaire comme référence, la sensibilité, la spécificité et la valeur prédictive positive (VPP) ont été déterminées pour l'algorithme de l’OMS et des seuils de CD4 de 1500 et 2000 cellules/mm3 par les pédiatres et les agents cliniques.

Résultats

Nous avons recruté 166 nourrissons (50% de sexe féminin, 34% <2 mois, 37% infectés par le VIH). La sensibilité était plus élevée avec les seuils de CD4 (<1500, 80%; <2000, 95%) qu'avec l'algorithme (médecins, 57%; agents cliniques, 71%). La spécificité était comparable pour les seuils de CD4 (<1500, 68%, <2000, 50%) et l'algorithme (pédiatres, 55%; agents cliniques, 50%). Les valeurs prédictives positives étaient légèrement meilleures pour les seuils de CD4 (<1500, 59%, <2000, 52%) que pour l'algorithme (pédiatres, 43%; agents cliniques 45%) à cette prévalence.

Conclusion

L'utilisation de l'algorithme de l’OMS et les seuils de CD4 a donné lieu à de nombreuses erreurs de classification. Des seuils de CD4 <1500 cellules/mm3 ou <2000 cellules/mm3 sur le lieu des soins a pu identifier plus de nourrissons infectés par le VIH avec moins de faux positifs que l'algorithme. Cependant, une option sur le lieu des soins avec de meilleures caractéristiques de performance est nécessaire pour un diagnostic précis et rapide du VIH.

Abstract

Objetivo

Determinar los niveles de concordancia entre pediatras y trabajadores sanitarios no médicos (TSnM) para el algoritmo de la OMS para el diagnóstico in situ de la infección por VIH, y comparar los perfiles de sensibilidad y especificidad del algoritmo de la OMS y diferentes umbrales de CD4 con la prueba de VIH por PCR en bebés hospitalizados en Malawi.

Métodos

En el 2011, bebes expuestos al VIH y menores de 12 meses hospitalizados en Lilongüe, Malawi fueron evaluados de forma independiente con el algoritmo de la OMS tanto por un pediatra como por un TSnM. Se tomaron muestras de sangre para realizar pruebas de CD4 e VIH (mediante PCR de ADN o ARN). Utilizando las pruebas moleculares como referencia, se determinaron la sensibilidad, especificidad y valor predictivo positivo (VP+) para el algoritmo de la OMS y los umbrales del conteo de CD4 de 1500 y 2000 células/mm3 para pediatras y TSnM.

Resultados

Incluimos 166 bebés (50% mujeres, 34% <2 meses, 37% infectados con VIH). La sensibilidad era mayor utilizando los umbrales de CD4 (<1500, 80%; <2000, 95%) que con el algoritmo (médicos, 57%; TSnM, 71%). La especificidad era comparable para los umbrales de CD4 (<1500, 68%, <2000, 50%) y el algoritmo (pediatras, 55%, TSnM, 50%). Los VP+ eran un poco mejores utilizando los umbrales de CD4 (<1500, 59%, <2000, 52%) que el algoritmo (pediatras, 43%, TSnM 45%) con esta prevalencia.

Conclusión

El desempeño del algoritmo de la OMS y los umbrales de CD4 resultó en muchas clasificaciones erróneas. El uso de los umbrales de CD4 de <1500 células/mm3 o <2000 células/mm3 podría identificar más bebes infectados con VIH con menos falsos positivos que el algoritmo. Sin embargo, se requiere de una opción in situ con características de desempeño mejoradas para el diagnóstico preciso y temprano del VIH.

Introduction

Despite great strides in prevention of mother-to-child transmission (PMTCT) of HIV infection in developing countries, approximately 330 000 children acquire HIV each year, more than 90% of these in sub-Saharan Africa (UNAIDS 2012). More than half of these HIV-infected children will die within 2 years without antiretroviral therapy (ART) (Newell et al. 2004; Marinda et al. 2007). Early ART initiation within the first few months of life is even more beneficial, reducing infant mortality by 76% and HIV progression by 75% (Violari et al. 2008).

Although early ART initiation hinges on timely infant diagnosis, diagnosing HIV in this age group poses major challenges. First, point-of-care antibody tests cannot definitively diagnose HIV because maternal antibodies circulate in some infants until 18 months of age (Moodley et al. 1995). Second, HIV PCR testing using DNA or RNA is definitive, but results are frequently delayed due to slow turnaround times at specialised central referral laboratories and loss to follow-up (Jones et al. 2005). Ultimately, up to 70% of HIV-infected children may not receive HIV PCR results (Braun et al. 2011). As a consequence, HIV-related mortality continues among undiagnosed infants in HIV-endemic African countries.

Two potential alternatives to infant HIV PCR testing are the WHO clinical algorithm for symptomatic HIV infection and point-of-care CD4 testing (World Health Organization 2006a). The WHO clinical algorithm enables clinicians to immediately initiate potentially life-saving ART for infants meeting its criteria (World Health Organization 2006b), circumventing the long delays plaguing current PCR testing systems. Malawi, a southern African country with epidemic HIV, recommended use of this algorithm during routine care, with subsequent HIV DNA PCR confirmation starting in 2008 (Ministry of Health Malawi 2008). To date, little is known about the sensitivity and specificity profiles of this algorithm when used by non-physician clinicians, as previous work focused on algorithm performance solely by paediatricians with sensitivities of 23–77% and specificities between 53% and 93% (Inwani et al. 2009; Peltier et al. 2009; Mutesu-Kapembwa et al. 2010; Grundmann et al. 2011). This knowledge gap is critical because non-physician clinicians, not paediatricians, deliver the majority of clinical care in Malawi and other high HIV prevalence African countries.

Point-of-care CD4 cell count testing also offers a potential diagnostic alternative to HIV PCR. CD4 count testing can be made available at the point of care with existing technology. Performance characteristics at different CD4 count thresholds have not been explored. Furthermore, direct comparisons between the performance of different CD4 count thresholds, CD4 percentages, the WHO algorithm and gold standard PCR testing have not been thoroughly studied. If point-of-care CD4 testing were to outperform the WHO clinical algorithm, it would provide an immediate laboratory-based point-of-care option for infant HIV diagnosis and ART eligibility.

Our primary objectives for this study were (i) to determine WHO algorithm agreement levels between paediatricians and non-physician clinicians and (ii) to compare sensitivity and specificity profiles of the WHO algorithm and different CD4 thresholds against HIV PCR testing in hospitalised Malawian infants. We hypothesised that paediatricians would outperform non-physician clinicians in using the WHO algorithm and that a CD4 threshold which outperforms clinical diagnosis could be identified.

Methods

Study setting

This study was performed among hospitalised children in the paediatric wards of Kamuzu Central Hospital (KCH) in Lilongwe, Malawi. With paediatrics, KCH serves as both a referral and district hospital, has 215 beds, admits more than 13 000 children annually and has an inpatient paediatric HIV prevalence of 8.5% (McCollum et al. 2010). Since 2008, hospitalised children and their caregivers have been routinely offered HIV antibody testing as part of an inpatient paediatric HIV testing programme in accordance with Malawi guidelines (McCollum et al. 2010). Children are subsequently eligible for DNA PCR if HIV antibody positive and younger than 12 months. A WHO algorithm evaluation by either a paediatrician or clinical officer (CO) also occurs, but less routinely, because assessment depends upon the practitioner's training and preference.

Study procedures

This substudy took place between February and November 2011 and was nested within a prospective randomised controlled trial (McCollum et al. 2012). The parent study's primary objective was to compare outcomes of standard of care that included a paediatrician examination with the WHO algorithm plus laboratory-based HIV DNA PCR (turnaround time of several weeks) with a rapidly processed HIV RNA PCR test (turnaround time of 48 h). Study subjects of the parent study were hospitalised at KCH, were consented by a guardian, were <12 months of age and were HIV-exposed without a definitive HIV status. At enrolment, blood was collected from all participants for PCR HIV testing (either DNA or RNA PCR per randomisation outcome), CD4 absolute cell count and percentages, complete blood count and differential, malaria smear and blood culture. Chest radiographs and tuberculin skin tests were also carried out, with induced sputum tests performed only for patients with clinical suspicion of Mycobacterium tuberculosis or Pneumocystis jirovecii infection.

Study infants testing HIV antibody positive using the standard Malawi HIV testing algorithm were eligible for this substudy. The algorithm consisted of serial testing with Determine HIV-1/2 (Alere) first, followed by Unigold Recombigen HIV-1/2 (Trinity Biotech) for those testing antibody positive. Both a study paediatrician and one non-physician CO evaluated each substudy infant, also at the time of enrolment into the parent study, filled out an algorithm checklist for each criterion and assigned either a positive or negative HIV status per WHO algorithm criteria. For an infant to be considered algorithm positive, they needed either two HIV-related conditions (oral thrush, severe or very severe pneumonia, or severe sepsis) or one AIDS-specific condition (P. jirovecii pneumonia, oesophageal candidiasis, treatment-unresponsive severe acute malnutrition, extra-pulmonary tuberculosis disease, Kaposi sarcoma, cerebral toxoplasmosis with onset after 1 month of age or cryptococcal meningitis). The paediatrician and COs were blinded to one another's clinical evaluations and to PCR results. However, they were not blinded to the PMTCT and breastfeeding history of the mother–infant pair.

All COs working in the KCH paediatric wards were invited to participate in the substudy, provided written informed consent, underwent a half-day training in the WHO algorithm and study procedures and completed a questionnaire and written competency test. COs were the practitioners of interest because they are the primary cadre of non-physician clinicians in Malawi and provide the majority of Malawian paediatric hospital care.

We retrospectively assessed CD4 performance at multiple percentage and absolute CD4 count thresholds. Infants with values below the CD4 threshold were classified ‘positive’, and those with values above the threshold were classified as ‘negative’. We assessed CD4 percentages because they are preferred for HIV management in infants, and absolute counts because point-of-care technology is available currently, although not used in this assessment.

Analytic methods

Normally distributed continuous covariates were described using means and standard deviations, and categorical characteristics were presented as proportions. Level of agreement in assignment of overall WHO algorithm status and individual algorithm conditions were compared between COs and the paediatrician using proportion of overall agreement and Cohen's kappa statistic.

The reference standard used for HIV infection was a positive HIV DNA PCR or RNA PCR with >10 000 copies/ml. The performance of the paediatrician, COs and CD4 thresholds (both absolute count and percentages) was compared to this standard. We also compared each individual WHO algorithm condition to this standard. Sensitivity and specificity were calculated, along with 95% confidence intervals (CI).

Given these sensitivities and specificities, the positive predictive value (PPV), negative predictive value (NPV) and corresponding 95% CIs were calculated at HIV prevalence levels from 0% to 100%, including the prevalence in this population. Additionally, we calculated the total number of errors expected (false positives plus false negatives) in a population of 1000 infants at each CD4 count threshold. We varied two sets of assumptions. First we varied the relative weight of a false-negative and false-positive result (i.e. that a false-positive and a false-negative result were equal or that a false-negative result would be three times worse than a false-positive result). We also varied the prevalence of HIV infection in the population from 5% (the projected prevalence under improved PMTCT policies) and 37% (the prevalence in this substudy).

All paediatric data were analysed using SAS 9.3 (SAS Institute, Cary, NC, USA). CO characteristics were analysed in Microsoft Excel. Completeness and accuracy in conduct and reporting of this study was assessed using the STARD initiative checklist (Bossuyt et al. 2003).

Ethical approval

We received ethical approval from the Malawi National Health Sciences Research Committee and the Institutional Review Boards at The University of North Carolina at Chapel Hill and Baylor College of Medicine.

Results

Of all hospitalised infants <12 months, 13.1% (323 of 2465) were HIV-exposed (Figure 1). Research staff enrolled 300 infants into the parent study, of whom 237 (79%) tested HIV antibody positive and were eligible for this substudy. Both the study paediatrician and one CO examined 70.0% (166 of 237) of these HIV antibody-positive infants. COs were not available to examine all 237 eligible infants due to logistical conflicts.

Figure 1.

The proportion of infants included and excluded from the parent trial and this subanalysis. The proportions of infants who were HIV infected and HIV uninfected were determined with HIV RNA or HIV DNA PCR.

Table 1 presents patient characteristics. Exactly half (83 of 166) of study subjects were female with a median age of 3.3 months (interquartile range = 1.5–7.4 months) irrespective of gender. The majority of infants (84%) were breastfeeding, of whom 64% (89 of 139) were breastfeeding exclusively. Most infants received at least partial PMTCT as 66% of mothers reported antenatal ART prophylaxis and 59% reported that their infant received ART prophylaxis after birth.

Table 1. Patient characteristics
 = 166(100%)
  1. PMTCT indicates prevention of mother-to-child transmission.

Gender
Male83(50%)
Female83(50%)
Age
<2 months57(34%)
2–6 months58(35%)
>6 months51(31%)
Maternal PMTCT
Yes107(66%)
No56(33%)
Infant PMTCT
Yes98(59%)
No68(41%)
Mother alive
Yes161(98%)
No4(2%)
Breastfeeding status
Exclusive89(54%)
Mixed50(30%)
No27(16%)

Participating COs were generally young (mean age 25.4 years, range 21–32 years) and without substantial clinical or paediatric HIV experience (mean 2.3 years clinical experience; mean 1.6 years paediatric HIV experience [Table 2]). Prior to this study, only two of seven COs were both nationally certified in HIV and also trained in paediatric HIV. COs and paediatricians arrived at the same diagnosis on 67% (112 of 166) of the infants. An overall ‘fair’ agreement levelViera & Garrett 2005 in the assignment of WHO clinical algorithm status was achieved between COs and the study paediatrician (mean kappa 0.35, 95% CI: 0.21, 0.49), ranging from 0.18 to 0.59. We also assessed overall agreement levels for the diagnosis of specific WHO clinical algorithm conditions between participating COs and the study paediatrician using the kappa statistic, where 0 is no agreement, 1 is complete agreement (data not shown), and negative values reflect less agreement than would be expected at random. Except for almost no agreement found for the diagnosis of severe malnutrition (kappa = 0.03), kappa agreement levels ranged from 0.28 for sepsis to 0.62 for oral Candida.

Table 2. The characteristics of clinical officers as well as their level of agreement with the paediatrician on each diagnosis. Observed agreement is the proportion of cases in which the clinical officer and paediatrician indicated the same response on the algorithm
 CO #1CO #2CO #3CO #4CO #5CO #6CO #7
  1. HIV indicates human immunodeficiency virus; IMCI, Integrated Management of Childhood Illnesses; WHO, World Health Organization.

  2. The kappa statistic reflects observed agreement relative to expected agreement.

Male genderYYYYYYY
Age26253221232328
Years of Paediatric HIV Experience1330112
Years of Clinical Experience1341313
National IMCI certificationNNYNYNY
National HIV certificationNYYNYNN
Paediatric HIV trainedNYYYYNN
Children evaluated with WHO algorithm294738135223
Observed agreement79%75%71%71%62%56%74%
Kappa statistic0.59 (0.31–0.87)0.50 (–0.24–1.00)0.30 (–0.47–1.00)0.42 (0.14–0.71)0.25 (–0.21–0.72)0.18 (–0.04–0.40)0.41 (0.04–0.77)

We next examined the sensitivity of the WHO algorithm when applied by clinicians and CD4 thresholds as compared to gold standard DNA or RNA molecular testing (Table 3). Overall, 61 of 166 infants (37%) were HIV infected. The sensitivity of the WHO algorithm was lower when used by paediatricians (57%) vs. COs (71%). However, 5 of 43 cases classified as algorithm positive by COs did not actually meet criteria according to their own evaluation. For example, some checked ‘HIV positive’ even if the infant only had one HIV-related condition. If COs had correctly interpreted the WHO algorithm for all patients, the sensitivity of the algorithm would have decreased to 62%. For both paediatricians and COs, the most sensitive clinical condition was severe or very severe pneumonia (71% for paediatricians and 57% for COs). Sensitivity increased with both CD4 percentages (28% sensitive at CD4 < 15% to 85% sensitive at CD4 < 30%) and CD4 count (25% sensitive at CD4 < 500 cells/mm3 to 95% sensitive at CD4 < 2000 cells/mm3).

Table 3. The number of individuals with true-positive, false-negative, true-negative and false-positive results, as well as sensitivity, specificity and corresponding 95% confidence intervals. WHO clinical criteria for symptomatic HIV infection in infants include HIV antibody positive, and ≥2 severe HIV-associated clinical conditions or ≥1 AIDS-specific condition. HIV-associated conditions included sepsis, severe or very severe pneumonia, and oral Candida. Cryptococcol meningitis, KS, TB and other stage 4 conditions were not identified in any children by a paediatrician during this study and are therefore excluded from the table, even though they are AIDS-defining conditions
 True +False −True −False +Sensitivity (95% CI)Specificity (95% CI)
  1. HIV, human immunodeficiency virus; IMCI, Integrated Management of Childhood Illnesses; WHO, World Health Organization.

PCR as the gold standard
Paediatrician
WHO algorithm positive3526584757.4 (44.1, 70.0)55.2 (45.2, 65.0)
Sepsis55694118.2 (2.7, 18.1)89.5 (82.0, 94.7)
Severe or very severe pneumonia4318307570.5 (57.4, 81.5)28.6 (20.2, 38.2)
Oral Candida2140951034.4 (22.7, 47.7)90.5 (83.2, 95.3)
Any two HIV conditions1546891624.6 (14.5, 37.3)84.8 (76.4, 91.0)
Pneumocystis jirovecci pneumonia2239703536.1 (24.2, 49.4)66.7 (56.8, 75.6)
Oesophageal Candida853103213.1 (5.8, 24.2)98.1 (93.3, 99.8)
Severe malnutrition25910323.3 (0.4, 11.4)98.1 (93.3, 99.8)
Clinical Officer
WHO algorithm positive4318525370.5 (57.4, 81.5)49.5 (39.6, 59.5)
WHO PD by algorithm conditions3823683762.3 (49.0, 74.4)64.8 (54.8, 73.8)
Sepsis2041782732.8 (21.3, 46.0)74.3 (64.8, 82.3)
Severe or very severe Pneumonia3526366957.4 (44.1, 70.0)34.3 (25.3, 44.2)
Oral Candida214099634.4 (22.7, 47.7)94.3 (88.0, 97.9)
Any two HIV conditions1942881731.2 (19.9, 44.3)83.8 (75.4, 90.3)
Pneumocystis jirovecci pneumonia1843812429.5 (18.5, 42.6)77.1 (67.9, 84.8)
Oesophageal Candida55610508.2 (2.7, 18.1)100 (96.6, 100.0)
Severe malnutrition55610328.2 (2.7, 18.1)98.1 (93.3, 99.8)
CD4 percentage
<15%1744104127.9 (17.2, 40.8)99.0 (94.8,100.0)
<20%402198765.6 (52.3, 77.3)93.3 (86.8, 97.3)
<25%4615881775.4 (62.7, 85.5)83.8 (75.4, 90.3)
<30%529772885.2 (73.8, 93.0)73.3 (63.8, 81.5)
CD4 absolute count
<5001546103224.6 (14.5, 37.3)98.1 (93.3, 99.8)
<10003724921360.7 (47.3, 72.9)87.6 (79.8, 93.2)
<15004912713480.3 (68.2, 89.4)67.6 (57.8, 76.4)
<2000583525395.1 (86.3, 99.0)49.5 (39.6, 59.5)

In addition to studying the sensitivity of the WHO algorithm and CD4 thresholds, we also studied specificity. In all, 105 of 166 infants (63%) were HIV-uninfected. Paediatricians achieved a higher specificity (55%) with use of the WHO algorithm as compared to COs (50%). When algorithm performance was corrected for 16 patients misclassified by COs as algorithm positive, despite their own evaluation indicating otherwise, the specificity of the algorithm increased from 50% to 65%. Compared with HIV PCR testing, the specificity decreased with higher CD4 percentage and absolute count thresholds. Specifically, specificity was 99% at a CD4 percentage <15% and 73% at CD4 < 30%. Specificity was 98% at an absolute CD4 count <500 cells/mm3 and 50% at a CD4 < 2000 cells/mm3.

At the 37% prevalence, the PPV for CD4 percentage thresholds ranged from 94% at CD4 < 15% to 65% at CD4 < 30% (Figure 2c). For absolute CD4 count testing, the PPV ranged from 88% at CD4 < 500 cells/mm3 to 52% at CD4 < 2000 cells/mm3 (Figure 2b). NPV ranged from 72% at CD4 < 15% to 90% at CD4 < 30% and 69% at CD4 count <500 cells/mm3 to 95% at CD4 < 2000 cells/mm3.

Figure 2.

Positive and negative predictive values of (a) the WHO algorithm as a function of HIV prevalence of (b) CD4 count thresholds as a function of HIV prevalence of (c) CD4 percentage thresholds as a function of HIV prevalence.

Several trends emerged when assessing PPV and NPV using the WHO algorithm, and all CD4 thresholds over a spectrum of prevalence levels (Figure 2). First, at any prevalence, both PPV and NPV of CD4 < 1000, CD4 < 1500 and CD4 < 2000 were better than PPV and NPV among either of the clinician cadres. At the study prevalence (37%), PPV was best (88%) at CD4 < 500, and at this value CD4 NPV was almost identical to that of both clinician cadres. CD4 < 2000 was the worst CD4 count threshold for PPV (52%), but still better than use of the WHO algorithm by either type of clinician.

Because CD4 count thresholds outperformed the WHO algorithm and could become available at the point of care, we explored which thresholds would minimise the number of errors (Table 3). At 37% prevalence, when false negatives and false positives were weighted equally, the total number of errors increased monotonically from 29 per 100 infants screened at a CD4 threshold of <500 cells/mm3 to 34 per 100 infants screened at a threshold of <2000 cells/mm3. When a false negative was weighted three times more heavily, the opposite trend emerged: the total number of errors decreased monotonically from 85 per 100 infants screened at <500 cells/mm3 to 37 per 100 infants screened at <2000 cells/mm3. At a 10% prevalence when false negatives and false positives were weighted equally, the total number of errors increased monotonically from 9 per 100 infants screened at <500 cells/mm3 to 46 per 100 infants screened <2000 cells/mm3. But when a false negative was weighted three times more heavily, the relationship between CD4 count threshold and number of errors per 100 infants screened was not monotonic: 24 errors at CD4 < 500 cells/mm3, 23 errors at CD4 < 1000 cells/mm3, 35 errors at CD4 < 1500 cells/mm3 and 47 errors at <2000 cells/mm3(Table 4). The choice of the optimal CD4 count threshold is quite sensitive to both the relative weight of false-negative and false-positive result and the prevalence.

Table 4. Total number of expected errors (FP plus FN) in a hypothetical population of 1000 infants at different prevalence levels of HIV and different CD4 thresholds. It further displays number of expected error with false negatives (FN) weighed three times more heavily than false positives (FP)
PrevalenceCD4 thresholdHIV positiveHIV negativeSensitivitySpecificityFPFNFN × 3 (weighted)Total errorsTotal errors (×3)
  1. HIV, human immunodeficiency virus; FP, false positive; FN, false negative.

0.37<5003706300.250.9812279837291849
0.37<10003706300.610.8878145436224514
0.37<15003706300.800.6820473219277423
0.37<20003706300.950.503181854336373
0.2<5002008000.250.9815151452166468
0.2<10002008000.610.889979236178335
0.2<15002008000.800.6825939118299377
0.2<20002008000.950.504041029414433
0.1<5001009000.250.98177522693243
0.1<10001009000.610.8811239118151230
0.1<15001009000.800.682922059311351
0.1<20001009000.950.50455515459469
0.05<500509500.250.98183811356131
0.05<1000509500.610.881182059137177
0.05<1500509500.800.683081030318337
0.05<2000509500.950.5048027482487

Discussion

Unacceptably, high rates of mother-to-child HIV transmission and paediatric mortality continue in part because a reliable point-of-care HIV diagnostic test for infants is not yet routinely available in resource-constrained African countries with epidemic HIV. This study examined two types of existing point-of-care diagnostic approaches that could be utilised for infant HIV diagnosis until an acceptable molecular test is available, the WHO clinical algorithm for symptomatic HIV infection in infants and the CD4 test. Overall, we found the accuracy of infant HIV diagnosis using the WHO clinical algorithm or CD4 to be inferior to gold standard molecular HIV testing (DNA or RNA PCR) in hospitalised Malawian infants. Both COs and paediatricians missed a large share of HIV-infected infants using the WHO algorithm. Higher CD4 count thresholds (>1500 cells/mm3 or >25%), although also suboptimal, performed markedly better at identifying HIV-infected infants in our patient population.

This study addressed an important knowledge gap regarding the use of the WHO algorithm by COs, a cadre of non-physician clinicians who provide the majority of paediatric HIV care in Malawi and are common throughout sub-Saharan Africa (McCollum et al. 2013). The performance of the WHO algorithm by the paediatrician in this study was comparable to that observed in WHO algorithm validation studies carried out in Kenya (Inwani et al. 2009) and Zambia (Grundmann et al. 2011) (low sensitivity) and Rwanda (Peltier et al. 2009) (low specificity). Our study found that there was only fair agreement with algorithm use between COs and paediatricians (Kappa = 0.35) on an individual patient basis. However, COs and paediatricians had similar performance overall with respect to the sensitivity and specificity of the algorithm when compared to molecular HIV testing. COs displayed a trend towards better sensitivity and paediatricians displayed a trend towards better specificity, although these differences were not statistically significant at an alpha level of 0.05. In summary, using the WHO algorithm, COs were able to assess infant HIV status as well as paediatricians in this setting.

Although the two cadres performed similarly to each other, the WHO algorithm is poorly sensitive and specific for detecting infant HIV in this setting. Using the WHO algorithm, at least 30% of HIV-infected infants were not identified by either clinician cadre and the majority of those identified as HIV infected by both cadres were actually HIV-uninfected. Similar performance of the WHO algorithm was also observed in other studies assessing this algorithm (Inwani et al. 2009; Peltier et al. 2009; Mutesu-Kapembwa et al. 2010; Grundmann et al. 2011).

As point-of-care molecular testing is not yet available, this study explored the performance of different CD4 thresholds in an effort to identify an existing alternative to the WHO algorithm. CD4 count testing could be made available at the point of care. In some settings, CD4 count is comparable to CD4 percentages, the preferred metric in infants (Centers for Disease Control & Prevention 1994; Gebo et al. 2004; Goicoechea & Haubrich 2005). We have shown that using CD4 count thresholds can improve sensitivity over clinical diagnosis without compromising PPV at any given prevalence. Previous studies have assessed the performance of the CD4 < 25% threshold in similar infant populations and found sensitivities between 55% (Peltier et al. 2009) and 72% (Inwani et al. 2009) and specificities from 39% (Inwani et al. 2009) to 88% (Mutesu-Kapembwa et al. 2010). Other studies have assessed clinical criteria performance in combination with CD4 < 25% and demonstrated improved algorithm performance (Inwani et al. 2009; Peltier et al. 2009; Mutesu-Kapembwa et al. 2010; Grundmann et al. 2011).

Choosing an optimal CD4 count threshold is challenging and requires careful consideration. Initiating HIV-uninfected infants on ART is the compromise of using a less specific HIV diagnostic test and should be carried out with caution. Falsely identifying infants as HIV infected may result in medication side effects, misdirected use of scarce healthcare resources and psychosocial strain on the family. To avoid these potential issues, a highly specific diagnostic is desirable. On the other hand, maximising sensitivity is likely to minimise mortality because more HIV-infected infants can be initiated on ART in a timely fashion. Thus, we consider weighing a false negative more heavily than a false positive to be reasonable. A CD4 count of 2000 cells/mm3 identified nearly all the HIV-infected infants (95%). At 37% prevalence, assuming a false negative is three times worse than a false positive, a CD4 count threshold of 2000 cells/mm3 optimises the potential diagnostic algorithm.

But the optimal CD4 threshold will vary depending on the prevalence of HIV among infants and should therefore be individually tailored to each setting's HIV prevalence prior to implementation and reviewed over time as HIV prevalence changes. For example, in Malawi, HIV prevalence is likely to decline over time due to Option B+, a PMTCT programme started in 2011 that provides free lifetime ART to all HIV-infected pregnant and breastfeeding women (Ministry of Health Malawi 2011). In our setting, if prevalence among hospitalised infants declines to 10%, the optimal CD4 count threshold would be <1000 cells/mm3. As our study relied on data preceding Option B+ implementation, additional modelling or a pilot study assessing different CD4 thresholds at the point of care for HIV-exposed Malawian infants in this new context could be informative.

There are several reasons why the WHO algorithm may have underperformed compared with molecular testing. First, the individual conditions used in the algorithm themselves may not be good predictors of HIV in this population. For example, sepsis and pneumonia are prevalent among both HIV-infected and HIV-uninfected infants in this population (Table 3) and thus poorly specific for predicting HIV. Second, individual algorithm conditions may have been inaccurately classified, thereby limiting accuracy of clinicians who adhere to the algorithm. Third, COs misclassified some infants by not adhering to the algorithm. Previous evidence has reported poor adherence to clinical criteria for both pneumonia diagnosis and oxygen eligibility by Malawian COs (McCollum et al. 2013). In this study, we similarly report inconsistent CO adherence to the WHO algorithm for infant HIV diagnosis.

The results observed in this hospital-based study may not be generalisable to outpatient Malawian clinics due to several factors. Hospitalised infants are generally sicker than HIV-exposed infants presenting to outpatient care, and HIV prevalence is also higher in the hospital compared with the community. Therefore, fewer infants presenting for routine outpatient care are likely to meet algorithm criteria for an HIV-positive diagnosis. Only one paediatrician and seven COs participated in this single facility study. A larger multicenter study in diverse settings with more practitioners is needed to confirm our findings.

In conclusion, accurate, timely HIV infant diagnosis is critical for access to life-saving ART, but is not occurring in the current Malawian infant HIV diagnosis system that relies on slowly processed DNA PCR tests. Ideally, an infant HIV point-of-care diagnostic that is highly sensitive, specific and affordable is desired. Until such a test is available, we recommend implementing point-of-care CD4 testing as an infant diagnostic, with thresholds based on local HIV prevalence levels.

Acknowledgements

We gratefully acknowledge the study participants and their caregivers for their participation. This work was supported by the National Institutes of Health through the Fogarty International Center and International Clinical Research Fellows Program at Vanderbilt University; the National Heart Lung and Blood Institute; the University of North Carolina Center for AIDS Research; and the UNC Hopkins Morehouse Tulane Fogarty Global Health Fellows Program. Partial results were presented at the 7th IAS Conference on HIV Pathogenesis, Treatment and Prevention (IAS 2013) as a poster.

Ancillary