SEARCH

SEARCH BY CITATION

Keywords:

  • developing country;
  • systematic review;
  • validity;
  • method comparison
  • pays en développement;
  • l’examen systématique;
  • la validité;
  • la méthode de comparaison
  • países en vías de desarrollo;
  • revisión sistemática;
  • validación;
  • método comparativo

Summary

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Objectives  To systematically review comparative research from developing countries on the effects of questionnaire delivery mode.

Methods  We searched Medline, EMbase and PsychINFO and ISSTDR conference proceedings. Randomized control trials and quasi-experimental studies were included if they compared two or more questionnaire delivery modes, were conducted in a developing country, reported on sexual behaviours and occurred after 1980.

Results  A total of 28 articles reporting on 26 studies met the inclusion criteria. Heterogeneity of reported trial outcomes between studies made it inappropriate to combine trial outcomes. Eighteen studies compared audio computer-assisted survey instruments (ACASI) or its derivatives [personal digital assistant (PDA) or computer-assisted personal interview (CAPI)] against another self-administered questionnaires, face-to-face interviews or random response technique. Despite wide variation in geography and populations sampled, there was strong evidence that computer-assisted interviews lowered item-response rates and raised rates of reporting sensitive behaviours. ACASI also improved data entry quality. A wide range of sexual behaviours were reported including vaginal, oral, anal and/or forced sex, age of sexual debut, condom use at first and/or last sex. Validation of self-reports using biomarkers was rare.

Conclusions  These data reaffirm that questionnaire delivery modes do affect self-reported sexual behaviours and that use of ACASI can significantly reduce reporting bias. Its acceptability and feasibility in developing country settings should encourage researchers to consider its use when conducting sexual health research. Triangulation of self-reported data using biomarkers is recommended. Standardizing sexual behaviour measures would allow for meta-analysis.

Comment améliorer la validité des reports sur les comportements sexuels: Revue systématique des modes de délivrance des questionnaires dans les pays en développement

Objectifs: Examiner de façon systématique la recherche comparative dans les pays en développement sur les effets du mode de délivrance des questionnaires.

Méthode: Nous avons effectué une recherche sur Medline, Embase et PsychINFO ainsi que dans les revues de la conférence ISSTDR. Les essais randomisés contrôlés et les études quasi expérimentales ont été inclus s’ils comparaient deux ou plusieurs modes de délivrance des questionnaires, avaient été menés dans un pays en développement, rapportaient sur les comportements sexuels et avaient été menés après 1980.

Résultats: 28 articles rapportant sur 26 études répondaient aux critères d’inclusion. L’hétérogénéité des résultats des essais rapportés entre les études a rendu inapproprié la combinaison des résultats des essais. 18 études ont comparé des instruments d’enquête audio assistés par ordinateur (ACASI) ou leurs dérivés (PDA ou CAPI) à un autre questionnaire auto-administré, à des interviews de face-à-face, ou à la technique de réponse aléatoire. Malgré de grandes différences dans la géographie et les populations échantillonnées, il y avait des preuves solides que les interviews assistées par ordinateur abaissaient les taux de réponses types et élevaient le report de comportements de nature délicate. ACASI améliorait également la qualité des données saisies. Un large éventail de comportements sexuels ont été rapportés, y compris par voie vaginale, orale, anale et/ou des rapports sexuels forcés, l’âge du premier rapport sexuel, l’utilisation du préservatif lors du premier et/ou dernier rapport sexuel. La validation des auto-reports sur base de biomarqueurs était rare.

Conclusions: Ces données réaffirment que les modes de délivrance des questionnaires affectent les comportements sexuels auto-rapportés et que l’utilisation de ACASI peut considérablement réduire le biais des reports. Son acceptabilité et sa faisabilité dans les pays en développement devrait encourager les chercheurs à envisager son utilisation lors de la conduite de recherche en santé sexuelle. La triangulation des données provenant d’auto-reports en utilisant des biomarqueurs est recommandée. La standardisation des mesures des comportements sexuels permettrait la méta-analyse.

Como mejorar la validez del reporte de comportamiento sexual: revisión sistemática de las formas de entregar los cuestionarios en paises en vías de desarrollo

Objetivos: Realizar una revisión comparativa y sistemática en países en vías de desarrollo sobre los efectos del modo de entrega de los cuestionarios.

Métodos: Hemos realizado una búsqueda en Medline, EMbase y PsychINFO y los resumenes de las conferencias de la ISSTDR. Se incluyeron ensayos aleatorizados controlados y estudios casi-experimentales si se comparaban dos o más formas de entregar un cuestionario, si se habían realizado en países en vías de desarrollo, se reportaban comportamientos sexuales, y se habían realizado después de 1980.

Resultados: Los criterios los cumplieron 28 artículos que reportaban 26 estudios. La heterogeneidad de los resultados reportados por los ensayos entre estudios hacía que fuese inapropiado combinar los resultados de los ensayos. 18 estudios compararon encuestas realizadas con instrumentos de audio computarizados (ACASI) o sus derivados (PDA o CAPI) frente a cuestionarios auto-administrados, entrevistas cara a cara, o la técnica de la respuesta al azar. A pesar de la amplia variedad geográfica y de población muestreada, la evidencia era clara en el sentido de que en las entrevistas asistidas con ordenador disminuían las tasas de respuesta de ítems y aumentaban las tasas de respuesta a preguntas sensibles. ACASI también mejoró la calidad de los datos adquiridos. Se reportó un amplio rango de comportamientos sexuales, incluidos el sexo vaginal, oral, anal / o forzado, edad de la primera relación, uso de preservativos en la primera o última relación sexual. La validación de los autoreportes utilizando biomarcadores era rara.

Conclusiones: Estos datos reafirman que la modalidad de entrega de los cuestionarios si afecta los comportamientos sexuales auto-reportados y que el uso de ACASI podría reducir significativamente la parcialidad debida al proceso de reportaje. Su aceptabilidad y viabilidad en lugares en países en vías de desarrollo debería animar a los investigadores a considerar su uso para llevar a cabo investigaciones sobre salud reproductiva. Se recomienda la triangulación de datos auto-reportados utilizando biomarcadores. El estandarizar las medidas de comportamiento sexual permitiría realizar meta-análisis.


Background

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

After almost three decades, the burden of the HIV epidemic remains squarely in developing countries where the disease is primarily spread sexually (UNAIDS 2008; Wilson & Halperin 2008). Sexual behaviour is complex and influenced by many factors such as socio-economic, cultural, biological and psychological conditions, many of which cannot be easily externally validated or objectively measured.

Another measurement challenge surrounds the private aspect of sexual behaviours. As sexual behaviour cannot ethically be observed directly, we must rely on individuals’ self-reports of their sexual experiences. Although cultural norms differ around the world, most sexual behaviours are socially censured, and strong pressure to conform to societal norms causes self-reports to be fraught with bias, particularly around recall and social desirability.

Over the past two decades, diverse studies (condom usage, discordant couples, condom negotiations) have explored the relationship between risky behaviours and their impact on HIV. Researchers have noted a gap between the validity and reliability of the self-reported measure and other outcomes. Ensuing under-reporting of sexual behaviours renders it difficult to interpret trends in HIV prevalence or incidence, to design appropriate behavioural interventions and to interpret their effects. This requires greater attention be given to improving measurement techniques. The scope for change ranges from questionnaire wording (Wellings et al. 2001; Elam & Fenton 2003; Mavhu et al. 2008) and ensuring privacy and confidentiality (Catania et al. 1990; Fenton et al. 2001; Tourangeau & Yan 2007), to improving questionnaire delivery modes (Mensch et al. 2003, 2008; Tourangeau & Yan 2007).

Traditionally, the field has relied on interviewer-administered questionnaires to collect self-reported sexual behaviour information. Growing concern for improved validity has prompted researchers to explore other questionnaire delivery modes. This search has been most radically transformed with the recent advent of computer programming in questionnaire design (Turner et al. 1998; Jones 2003).

In the early 1990s, Catania et al. (1990) reviewed the methodological challenges for assessing sexual behaviour, including the effects of questionnaire delivery mode on measurement error. Their conclusion called for more rigorous research into the assessments of sexual behaviours and emphasized that the foundation lay in improving the reliability and validity of its measurement. All 20 studies included in the review on response bias in sexual behaviour research were conducted in North America. At the end of the 1990s, Weinhardt et al. (1998) re-examined the literature since Catania’s review on reliability and validity of questionnaire delivery method and assessed the evidence relating to questionnaire delivery mode and reporting of sexual behaviours. Despite developments in the field, such as the use of computer technology, Weinhardt et al. believed that comparative research of questionnaire delivery modes remained limited. Moreover, of the 30 studies included in their review, only three were from developing countries, and only one of which compared questionnaire delivery modes (Konings et al. 1995).

Two recent developments provided the impetus for this systematic review. First, there has been a rise in the number of comparative studies on questionnaire delivery modes in developing country settings. While the bulk (14/26) of the studies reviewed here took place between 2000 and 2004, almost two-thirds (18/26) were published between 2005 and 2008. Secondly, there has been an increase – albeit still tentative -- in the use of computer-administered modes in developing country settings on sexual behaviour research. Within the comparative research conducted in developing countries, more than half include computer-administered modes. These two developments prompted this systematic review to examine the body of evidence and to summarize the findings around different questionnaire delivery modes regarding sexual behaviour measurement.

Methods

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Inclusion criteria

Studies were selected for review if they met the following criteria: published or cited in peer-reviewed journals that compared two or more questionnaire delivery modes; conducted in a developing country; included data that reported on sexual behaviour (vaginal, anal, or oral sex, condom use, risky sexual behaviours, contraceptive use), and published between 1 January 1980 and 31 December 2008. In addition, abstracts for the conference proceedings from the International Congress of Sexually Transmitted Infections (ISSTDR) were examined from 2001 onwards (2001, 2003, 2005 and 2007). In this review, the countries listed as having ‘emerging’ or ‘developing economies’ by the International Monetary Fund World Economic Outlook report were considered ‘developing’ countries (IMF 2009). Studies were included if they were evaluated in an experimental (RCT), quasi-experimental (i.e. had non-randomized comparison group) or test–retest design. Studies were excluded if they compared one questionnaire delivery mode against a biological marker, the impact of interviewer gender on questionnaire responses or data reported by married couples rather than between questionnaire methods. Unpublished studies emanating from references of published articles and studies published in non-English language journals were considered for inclusion.

Search strategy

Three databases were searched: Medline, EMbase and PsychINFO using key MeSH terms and text words relevant to each data base (Table 1). Duplicates were manually discarded.

Table 1.   Medline search strategy
 1. data collection/mt or health surveys/ or interviews as topic/ or narration/ or questionnaires/
 2. Sentinel Surveillance/
 3. “reproducibility of results”/
 4. validity.m_titl.
 5. reliability.m_titl.
 6. (“methodological study” or “methodological studies”).mp.
 7. evaluation studies as topic/ or “reproducibility of results”/ or validation studies as topic/
 8. (“face to face” or FTFI).mp.
 9. SAQ.mp.
10. self report$.mp.
11. (“randomized response” or “randomised response”).mp.
12. coital diar$.mp.
13. (ACASI or CAPI or CASI).mp.
14. (respondent or “non respondent”).mp.
15. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14
16. exp Reproductive Behavior/
17. exp Sexual Behavior/
18. HIV Infections/pc
19. Sexually Transmitted Diseases/pc
20. (“sexual behavior” or “sexual behaviour”).mp.
21. (“sex behavior” or “sex behaviour”).mp.
22. (“sexual behavioral” or “sexual behavioural”).mp.
23. (“oral sex” or “anal sex”).mp.
24. (sexual adj9 activit$).mp.
25. (“sexual orientation” or “sex orientation”).mp.
26. (“unprotected sex” or “protected sex”).mp.
27. (“unsafe sex” or “safe sex”).mp.
28. high risk sex.mp.
29. (“extra marital” or extramarital).mp.
30. (“sexual intercourse” or “first intercourse”).mp.
31. coital frequenc$.mp.
32. (“sexual partner$” or “multiple partner$”).mp.
33. (“condom use” or “condom usage”).mp.
34. “use of condoms”.mp.
35. (“reproductive behavior” or “reproductive behaviour”).mp.
36. exp Contraception Behavior/
37. contracepti$.mp.
38. “family planning”.mp.
39. 16 or 17 or 18 or 19 or 20 or 21 or 22 or 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32 or 33 or 34 or 35 or 36 or 37 or 38
40. Developing countries/or exp africa/or exp caribbean region/or exp central america/or exp latin america/or exp south america/or asia/or exp asia, central/or asia, southeastern/or borneo/or brunei/or cambodia/or east timor/or indonesia/or laos/or malaysia/or mekong valley/or myanmar/or philippines/or thailand/or vietnam/or asia, western/or bangladesh/or bhutan/or exp india/or afghanistan/or iran/or iraq/or lebanon/or turkey/or nepal/or pakistan/or sri lanka/or far east/or china/or mongolia/or exp europe, eastern/or exp luxembourg/or mediterranean region/or exp transcaucasia/or exp indian ocean islands/or pacific islands/or exp pacific islands/
41. 15 and 39 and 40
42. 41
43. limit 42 to (humans and yr = “1980–2008”)
Analysis.

Titles and abstracts were used to screen for relevance to the literature review. If the questionnaire delivery method was not mentioned in the title or abstract, it was assumed that there was no mode comparison. Where the title or abstract were not sufficient to make a determination, the article was downloaded and read. Reference lists for all included articles were examined. Three percentage of articles were re-examined blind by one co-investigator (FC) to check that inclusion criterion were being met. Articles where inclusion criteria were unclear (= 119) and all articles included in the review were jointly discussed by two of the investigators (FC & LL).

Results

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Of the 6824 references reviewed (Medline = 3261; Embase = 1761; PsycINFO = 1800, ISSTDR = 2), 28 articles reporting on 26 studies met the inclusion criteria (Figure 1). Articles reporting on results from more than one study were analysed separately. Where studies were reported in more than one article; these results were combined for analysis. Studies ranged geographically (from China, to Hanoi, to rural Malawi) and in their selection of respondents (from female sex workers to South African students). Self-administered questionnaires (SAQ) were used as the comparison in five studies (Table 2) whereas interviewer-administered questionnaires were the comparison in 16 studies (Table 3). Seven studies included SAQ, an interviewer-administered mode, and at least one other mode for comparison (Table 4).

image

Figure 1.  Diagram of Systematic Review.

Download figure to PowerPoint

Table 2.   Method comparison using self-administered questionnaire (SAQ)
Author, publication date Year study took place Geographic locationDescription of study populationStudy design & Key outcome measuresModes comparedInternal comparison & Biomarker presenceResults
  1. Modes: ACASI, audio computer-assisted survey instrument, where questions and responses are heard through headphones and respondent enters their response through the computer (desktop or laptop); PDA, personal digital assistant is a hand-held computer where questions and responses are read (and/or heard) and respondents enter responses directly into computer, phone-ACASI, survey conducted over the phone using computerized data entry system (phone keypad used to enter responses); SAQ, self-administered questionnaire using paper and pen.

  2. *P < 0.05; **P < 0.01; ***P < 0.001.

Rumakom et al. 2005; 1999 ThailandRepresentative sample of Thai college students males = 293; females = 372 Six FGDs on acceptability (31 ppts)Randomized control trial Mode acceptability response rate for sexual behavioursSAQ ACASI/no photos ACASI/photosNo NoIncreased reporting and appreciation of ACASI  Females 2× as likely to report masturbation in both ACASI modes than SAQ (ACASI-no photos = 18.2%vs. ACASI photos = 16.4%vs. SAQ = 6.8%*).  Males mean number of sexual partners in last 3 months. (ACASI photos = 1.40**vs. ACASI no photos = 1.31 vs. SAQ = 0.78);  Males mean number of partners in last month (ACASI no photos = 0.70 vs. ACASI photos = 0.88*vs. SAQ = 0.57). Minimal inconsistency in reporting with SAQ
Seebregts et al., (2009) 2003 Cape Town, South AfricaTwo hundred 8th grade students in one public school, mean = 14 yearsRandomized control trial Level of agreement compared using test–retest mode acceptabilitySAQ PDA 3 weeks later, retook Q using same modeYes NoTest–retest reliability on sexual risk behaviours was similar in both modes (differences were not statistically significant) Acceptability: most students thought PDA would be more confidential when answering sex Qs
Fielding et al. 2006; January–March 2001 China15 092 adult volunteer blood donorsQuasi-experimental Item non-response rate response rate for sexual risk behavioursSAQ Phone-ACASINo NoIncreased reporting with phone-ACASI  Homosexual (1.5% c.v 0.7%)  Bisexual (2.6%vs. 0.6%***)  Sex in past year (56.5%vs. 50.3%**  Thinks partner has sex with another (7.0%vs. 4.3%** For risk behaviour Qs, non-item response rate was higher in SAQ than phone (SAQ = 8.0–14.2%vs. phone-ACASI = 0.4–3.1%)
Bernabe-Ortiz et al. 2008; August 2005 Lima, Peru200 household surveys 18–29 years, mean = 22.9 yearsTest–retest Response agreement by educational levelSAQ PDAYes, immediately after NoHigh levels of agreement (<85%) in test–retest:  Spearman rho between 0.76–0.95  Agreement improved with increased education
Jaspan et al. 2007; August 2004–March 2005 Cape Peninsula, South Africa212 adolescents, 11–19 years, mean = 14.5 years, 68% femaleTest–retest Feasibility & acceptability item non-response rate response rate for sexual behavioursSAQ PDAYes, 2 weeks later NoNo differences based on questionnaire mode order. Approximately 7× more missing items in SAQ than PDA. High levels of agreement (85%) in test–retest:  Kappa = 0.5 or greater;  Correlation coefficient 0.65–0.91 No differences in reporting of sexual behaviours, except ‘ever had sex’ (SAQ = 36%vs. PDA = 26%,**.
Table 3.   Method comparison using face-to-face interviewing (FTFI)
Author, publication date Year study took place Geographic locationDescription of study populationStudy design & Key outcome measuresModes comparedInternal comparison & Biomarker presenceResults
  1. Modes: ACASI, audio computer-assisted survey instrument, where questions and responses are heard through headphones, and respondent enters their response through the computer (desktop or laptop); ASCQ, assisted self-completed questionnaire, where questions are read out loud by trained interviewer in a group setting (respondents spaced far apart); CD, coital diary where respondents self-complete a record of their sexual activity over time; CAPI, computer-assisted personal interview, where interviewer administers & enters responses into computer; FTFI, face-to-face interview where trained interviewer asks questions and records respondent’s answers; ICVI, interviewer-controlled voting instrument where sensitive questions are recorded by respondent onto sheet and placed in locked ballot box; In-depth interview, trained interviewer spends considerable time (1 day to a few weeks) with respondent collecting data, questions are usually open-ended; Interactive interview, FTFI with several audio-visual aids (5 segment audio drama; male and female dolls, confidential response sheet); phone-ACASI, survey conducted over the phone using computerized data entry system (phone keypad used to enter responses); Phone interview, trained interviewers asks questions over a telephone.

  2. *< 0.05; **< 0.01; ***< 0.001.

Jaya et al. 2008; August–November 2004 India1058 (males = 583; females = 475), 15–19 years (mean males = 17, mean females = 16). 1/3 in schoolRandomized control trial Item non-response rate Response rates for 10 itemsFTFI ACASI Interactive interviewYes, same day: FTFI vs. ACASI (both ways) FTFI vs. interactive Interview (both ways) NoOverall low item non-response rate (0.2–1.0%) Increased reporting in ACASI vs. FTFI:  Males ever sex (27%vs. 21%*  Males forcibly touched (26.2%vs. 21.4%* Increased reporting in FTFI vs. ACASI:  Been emotionally attached to someone (males 28.3%vs. 17.6%; females 17.2%vs. 6.0%***  Touched someone sexually (males 21.7%vs. 10.7%; females 6.0%vs. 1.3%; P < 0.001).  Females receive forced sex (38.4%vs. 14.2***). Increased reporting in interactive interviewing vs. FTFI:  Males ‘ever had sex’ (28%vs. 20%**  Males forcibly touched (27%vs. 22%*  Females ‘ever had sex’ (7%vs. 2%; P = 0.002)
Hewett et al. 2008; April–November 2004 Brazil818 females, 18–40 yearsRandomized control trial Response rates for STI risk behavioursFTFI ACASINo Yes: Chlamydia Gonorrhoea TrichomoniasisIncreased reporting in ACASI vs. FTFI:  Anal sex in last 6 months.: 33%vs. 24%**;  Never condom use with vaginal sex in last month: 59%vs. 51%*;  Mean no. sex acts w/out condom in last month: 7.0%vs. 5.8%* Increased reporting in FTFI vs. ACASI:  Vaginal sex in last 6 months.: 83%vs. 90%** ACASI revealed stronger associations between risk behaviour and presence of STIs. 16 of 21 measures were positively associated (7 were statistically significant). In FTFI, STI positive respondents were more likely to underreport risk behaviour than STI negative respondents.
Mensch et al. 2008; June-July 2004 Balaka District, Malawi501 unmarried female adolescents aged 15–21Randomized control trial Randomized to method and to interviewer Test–retest using nurse interview Response rate for sexual behaviours Results compared to similar population of unmarried females in Kenya, seeHewett et al. 2004FTFI FTFI/ACASI Nurse interview at time of biomarker collectionNo Yes HIV Gonorrhea, Chlamydia TrichomoniasisIncreased reporting in FTFI compared to ACASI in initial question on ‘ever sex’ Increased reporting in ACASI compared to FTFI for all other sexual behaviour questions:  When composite measure (‘ever had sex’ and ‘ever had sex with any partner’) used: (57.8%vs. 48.3%*)  Partners who are not boyfriends: (42%vs. 25%) (see Hewett et al. 2004 for similar results in Kenya) More inconsistencies with ACASI users, first indicating no sex, then indicating a type of sexual partner
Caceres et al. 2007; December 2000–May 2001 Fuzhou, China Chennai, India Lima, Peru St Petersburg, Russia Manhenga & Nkayi, ZimbabweBoth genders surveyed China: 199, 18–40 years; markets India: 63, 18–40 years; slums Peru: 69, 18–30 years; slums Russia: 60 18–30 years; college dorms Zimbabwe: 54, 16–30 years; rural villagesRandomized control trial Levels of agreement Questionnaire duration Response rates for 11 risky behaviour QsCAPI ACASIYes 2–3 days later NoReporting of sexual behaviours was similar in both modes, except in China where increased reporting using CAPI over ACASI (P < 0.01) ACASI took longer to complete than CAPI, especially in India & Zimbabwe
Allen et al. 2007; March–April 2004 Tanzania150 women who are participants in microbicide feasibility trial 23% <25 years 35% >35 yearsRandomized control trial Mode acceptability response rates for sexual behavioursFTFI CD 3 types of support: (a) min (n = 49) weekly collection, FTFI at end (1 month) (b) med (n = 46) weekly collection, FTFI in last 7 days (c) intense (n = 46) like (b) +unscheduled visit to help with CD comprehensionNo NoIncreased reporting with CD than FTFI As level of support increased, majority of sexual behaviours reported also increased.  Difference between medium and intensive levels of support is smaller and not sufficient to outweigh extra cost incurred for intensive support
Phillips et al. 2007; unknown date Bangalore, India595 men (MSM and transgendered) mean age = 29 yearsRandomized control trial (2/3 FTFI vs. 1/3 ICVI) Response rates for sexual behavioursFTFI ICVINo NoIncreased reporting with ICVI for paid sex with FSW in the last year (28%vs. 8%***) No statistically significant differences in reporting for other sexual behaviours
Minnis et al., 2007 June 2001–August 2003 Zimbabwe655 (60%) 18–35 years sexually active females from larger hormonal contraception-HIV study. HIV negative, not pregnant or trying to conceiveRandomized control trial Response rates for reporting pregnancy occurrence Validity of self-reported HC used against disbursement recordsFTFI ACASIYes, in later study visits interviewed using different mode NoMode order had no effect on responses. Increased reporting of pregnancy in ACASI (OR = 1.5) Increased inconsistent reporting in ACASI: 20% reported not using HC when it was distributed to them vs. 5% in FTFI (note inconsistency reflects increase in socially undesirable status, because woman was expected to take HC)
Simoes et al. 2006; September 2002–October 2003 Rio de Janeiro, Brazil610 adults seeking drug Tx, who have used drug in past months., have no psychiatric conditions or cognitive impairment. (18–93 years; mean = 36 years); 85% malerandomized control trial response rates for risky sexual behavioursFTFI ACASINo NoIncreased reporting in ACASI, except for reports of sexual activity and having multiple partners.  MSM (AOR = 2.52*)  Sex for drugs (AOR = 1.88*)  Drugs for sex (1.30)  Money for sex (AOR = 1.37*)  Sex with HIV+ partner (AOR = 1.32)  Inconsistent condom use (AOR = 1.34) ACASI users had decreased reporting of HIV testing, a socially desirable behaviour (AOR = 0.82).
Hanck et al. 2008;  April–June 2006 India812 CSWs 18+ yearsQuasi-experimental picked every 3rd for ICVI Response rates for 7 a priori sensitive QsFTFI ICVI (n = 269) with cards for less literate womenNo NoIncreased reporting of risky sexual behaviour with ICVI.  Client anal sex w/out condom (AOR = 1.5*)  Client oral sex w/out condom (AOR 1.8*);  Always used condoms w/ regular clients (AOR = 0.4**).
Bernabe-Ortiz et al. 2008; August 2006 Lima, PeruHousehold surveys 18–29 years, mean 22.7 years N = 198Quasi-experimental response comparison by mode; allocated by random assignment of interviewer agreementFTFI CAPI (using PDA)No NoNo differences in reporting between modes Lower missing values and lower inconsistent responses using CAPI
Gregson et al. 2002; July 1998–January 2000 Manicaland Province, ZimbabweSexually experienced, literate individuals 15–49 years, 1sr survey (road side trading centres, commercial and subsistence farming communities) (n = 7823)Quasi-experimental Item non-response rate Response rates for sexual behaviour Qs Completeness and internal consistency of sexual behaviour questions in questionnaireFTFI ICVINo NoLow item non-response rate overall; higher in ICVI. Increased reporting of multiple sexual partnerships with IVCI:  Current AOR males = 1.33* AOR females = 5.21***  Past month: AOR males = 1.71** AOR females OR = 2.92***  Past year AOR males = 1.35** AOR females = 1.97** Multiple sex partners in past month:  Young males: AOR = 1.78**  Single AOR = 1.95**  Married males cohabiting AOR = 2.08*  Males in subsistence farming AOR = 6.04***  Young females AOR = 2.88*  Older females AOR = 2.99*  Married women AOR = 9.20* Extramarital partners of married cohabiting women (currently, past month, & past year) were only reported in ICVI.
Gregson et al. 2004; July 1998–January 2000 July 2001–January 2003 Manicaland Province, Zimbabwe15–49 literate men and women (mean males = 28 years; mean females = 30 years) 2nd survey of a population-based cohort n = 6191Quasi-experimental compared 1stround (seeGregson et al. 2002above) with 2ndround response rates of sexual behavioursFTFI ICVI cohort used same method they had in 1st survey; new respondents (younger) used ICVI unless low literate who used FTFINo NoIncreased reporting with ICVI found in first survey (see Gregson et al. 2002)diminished, esp. for uncommon behaviours.  In males, magnitude of OR was smaller than in first survey  In females, increased reporting for ICVI only with ‘new sexual partner in last year (OR = 2.37; 95% CI 1.67–3.37***); Rates of missing responses & internal consistency remain similar to first survey.
Konings et al.1995; 1993 Uganda490 15–49 years (male & female), +60 intentionally selected CSWsQuasi-experimental (2 modes were alternated) Response consistency response rates sexual behavioursFFTI (long 5–30 min) FTFI (short 10–15 min) In-depth interview internal sub-sampleYes sub-sample of 75 respondents 3 weeks later NoIncreased reporting for in-depth interview compared with both FTFIs.  Non-regular partner in last 12 month:   Males: short = 33.9%vs. long = 35.3%vs. in-depth 45.5%  Females: short = 10.7%vs. long = 13.9%vs. in-depth 31.8% Consistency: 23.3% respondents did not report sex in Qs but did report during in-depth interview.
Plummer et al. 2004a,b; September-December 1998 Mwanza, Tanzania4958 youth who participated in both FTFI and ASCQ* FTFI/bio = 9283: school grades 4–6, mean males = 15.5 years & mean females = 14.8 years. ASCQ = 6079: school grades 5–6, mean males = 15.6 years & mean females = 14.6 yearsTest–retest Agreement of responses Response rates of sexual behavioursFTFI ASCQYes In 14 communities, ASCQ took place 5 weeks earlier, in six communities the reverse Yes HIV Gonorrhoea Chlamyida pregnancy (girls only)Increased reporting of sexual behaviours in ASCQ. Consistency:  77% agreement between 2 modes re ‘ever had vaginal intercourse’  62% of males & 41% of females report sex in both surveys. Increased reporting for males in ASCQ (56%vs. 52%***) Increased reporting in ASCQ:  Ever sex: 40%vs. 38%  Males forced sex: 5.8%vs. <0.1%  Females forced sex: 12.3%vs. 0.2%  ‘I don’t know’ was greater in ASCQ than FTFI. Among youth with positive biomarkers:  Males had increased reporting of self-reported sexual activity on ASCQ vs. FTFI (75%vs. 58%);  Females had decreased reporting of self-reported sexual activity on ASCQ vs. FTFI (31%vs. 45%)
Sedyaningsih-Mamahit & Gortmaker 2003; April 1995 Jakarka, Indonesia459 CSWs working in 228 brothelsTest–retest Reliability between FTFI two weeks apart. Response rates on condom useFTFI CDYes 50 randomly selected for test–retest. 50 randomly selected to complete CD & collect used condom wrappers NoLow response rate overall: over 2 wk period, of 50 CDs handed out, 40 (80%) were completed in at least one of the 2 weeks and 20 (40%) were completed in both weeks. Increased reporting in CD:  ‘Never’ condom use: 40% c.v 35%  ‘Occasional condom use’: 50%vs. 20% Agreement between 2 weeks of CD:  Condom ‘never use’: WK1 = 50%vs. WK2 = 40%  ‘Occasional condom use: WK1 = 40% c.v WK2 = 30%  ‘Always’ remained 20% for both weeks
Ramjee et al. 1999; August–October 1998 Durban, South Africa79 CSWs partici-pating in vaginal microbicide trial;  Mean age = 25 years(range 18–44)Test–retest  AgreementComparison of sexual behaviour reportsCD FTFI weekly recall (WR) FTFI daily recall (DR) (WR asks how many partners in last month; DR asks how many on Monday, Tuesday, etc)Yes; each ppt was offered chance to participate in all three methods NoIncreased reporting using CD: CD vs. WR:  Number of clients: 23.3 vs. 13.6***  Number of condoms: 20.77 vs. 10.32*** CD vs. DR:  Number of days worked: 6.7 vs. 5.2**  Number of clients: 21.7 vs. 17.4*  Anal sex with clients: 3.9 vs. 0.8** Complete agreement between vaginal sex with clients.
Table 4.   Multiple Comparisons (SAQ, FTFI, and others)
Author, publication date Year study took place Geographic locationDescription of study populationStudy design & Key outcome measuresModes comparedInternal comparison & Biomarker presenceResults
  1. Modes: ACASI, audio computer-assisted survey instrument, where questions and responses are heard through headphones and respondent enters their response through the computer (desktop or laptop); Audio-SAQ, self-administered questionnaire where questions, instructions, and responses, are heard through headphones; CD, coital diary where respondents self-complete a record of the sexual activity over time; FTFI, face-to-face interview interviewer administered questionnaire where trained interviewer asks questions and records respondent’s answers; FTFI/ACASI,Face to face interview for non-sensitive questions, followed by ACASI for sensitive questions; In-depth interview, trained interviewer spends considerable time (1 day to a few weeks) with respondent collecting data, questions are usually open-ended; PASI, palm-assisted self-interviewing which is a derivative of ACASI using a hind-held minicomputer or palmtop; RRT, random response technique: interviewer administered where respondent is randomly asked either sensitive question or non-sensitive question and the interviewer records the response but does not know which question is being answered. SAQ, self-administered questionnaire using paper and pen.

  2. Author notes study had limitations in that interviewers did not respect random allocation of modes and political tension in the area possibly causing mistrust around data collection in these rural communities.

  3. ‡ACASI, laptop was closed - respondent listened to Qs and used external keypad.*< 0.05;**< 0.01;**< 0.001.

Langhaug et al. 2007;  January–April 2006  Rural Zimbabwe1495 youth (males 827; females 668) mean age = 18.2 years,Randomized control trial item non-response rate Response rates for sexual behavioursICVI SAQ Audio-SAQ ACASIYes YesItem non-response rate greatest in SAQ and Audio-SAQ*** Sexual activity Audio-SAQ AOR = 2.05 [95% CI: 1.2–3.4]; ACASI AOR = 2.0 [95% CI: 1.2–3.2]) with no reporting difference for ICVI and SAQ (AOR = 1.01.0 [95% CI: 0.6–1.8). ACASI users reported a lower age at first sex (0.7–1.7 years lower)*
Potdar & Koenig 2005  July–November 2003  Pune, Indiaunmarried males aged 18–22 years 900 male college students (from 4 colleges); 600 males living in 2 slumsRandomized control trial Response rates for sexual behavioursCollege: FTFI SAQ, FTFI/ACASISlums: FTFI, FTFI/ACASINo NoIncreased reporting in ACASI:  College males:   Heterosexual sex: AOR = 1.8*   Oral sex with female: AOR = 2.08   Homosexual sex: AOR = 8.10**;   Experienced coercive sex: AOR = 11.35** Slum males:  Masturbation: (AOR = 22.53);  Oral sex with female: (AOR = 2.4);  Anal sex with female (AOR = 3.87);  Oral sex with male: (AOR = 3.20). Increased reporting in FTFI:  Vaginal sex: AOR = 0.23*  Anal sex with men: AOR = 0.59.
Le et al., 2006  unknown  Hanoi  Viet Nam2761 15–24 years, living in 2 towns; mean age = 20 yearsRandomized control trial Response rates sexual behaviours, self efficacy Qs, and attitudes & normsFTFI SAQ ACASINo NoIncreased reporting in ACASI:  ‘Ever sex’: (adol. 15–9 years OR = 2.79**; unmarried OR = 1.77***)  Sex w/ CSW: (adol. 15–19 years OR = 4.88; unmarried OR = 2.08**).  Sex before marriage: (both genders; females 2x greater***). Decreased reporting in ACASI (note that both of these are socially desirable behaviours)   Less confidence in accessing condoms***.   Refuse to complete question on correct   condom use: FTFI = 42%; SAQ = 44%;   ACASI = 28% (females were more likely to   refuse to answer this question)
Mensch et al. 2003;  April–October 2000  Nyeri, KenyaUnmarried adolescents aged 15–21 (n = 4358)Randomized control trial Asked f/up sex Qs only if respondent said they had had sex Inconsistent reporting; response rates for sexual behavioursFTFI SAQ ACASI‡ F/up exit interviewNo NoDecreased reporting of premarital sex in SAQ & ACASI than in FTFI.  Males: SAQ OR = 0.42***, ACASI OR = 0.34***  Females: SAQ OR = 0.38***, ACASI OR = 0.36*** Increased lack of trust with ACASI: ¼ of surveys took place with others present. However this seems not to have affected responses: 41% of males with others present reported having had sex vs. with 39% for those who took it independently.
Mensch et al. 2003;  Hewett et al. 2004;  April-July 2002  Kisumu, Kenyaunmarried adolescents aged 15–21 Kisumu (n = 2172) Hewett et al. 2004 reports on 709 unmarried female adolescents aged 15–21 from larger study by Mensch et al. 2003; Hewett et al. 2004 also only compares FTFI (n-349) with ACASI (n = 360)Randomized control trial 80% power to detect 10% difference between modes All sex Qs were asked irrespective of response to ‘ever had sex’ Q. Inconsistent reporting; item non-response rate Response rates for sexual behavioursFTFI SAQ ACASI‡ F/up exit interviewNo NoIncreased reporting with ACASI:  Sex with stranger: OR = 4.25***;  Tricked or coerced: OR = 3.35***;  Ever had STI (OR = 2.52*);  Females ‘had more than 1 partner’: OR = 2.35***. Decreased reporting with ACASI:  Males ‘ever sex’: OR = 0.58*** (against FTFI);  Females ‘ever sex: OR = 0.66** (against SAQ);  Females ‘ever sex’: 2x less (against FTFI). Consistency: ACASI had increased rates of inconsistent reporting 14.7% ACASI refused to answer at least one sensitive Q; FTFI answered all sensitive Qs.
van Griensven et al. 2006;  late 2002  Chiang Rai, Thailand1282 students aged 15–21 from 2 vocational schools; 49.9% male, 60.4% aged 16–19 yearsRandomized control trial Item non-response rate Inconsistent reporting Response rates for sexual behavioursFTFI SAQ ACASI PASINo Yes Urine tested for amphetamine type substance, nicotine, and their metabolitesMissing data and inconsistencies were found in 80% of SAQ questionnaires, 14% of FTFI questionnaires; no missing data or inconsistencies were for found in PASI or ACASI questionnaires. Increased reporting with FTFI vs. PASI for used contraception at last sex (a more socially desirable behaviour): Ever had sexual contact (e.g. but any sexual stimulation) FTFI = 64.7%; SAQ = 41.2%; ACASI = 53.8% PASI = 43.3% Increased reporting for ACASI/PASI (SAQ not reported here as was always in between FTFI and ACASI/PASI:
 FTFIACASIPASI
Ever oral sex13.242.937.3
Ever sold sex0.95.56.7
Ever bought sex2.57.18.2
Ever had genital sore or ulcer2.58.06.7
Partner/self ever been pregnant7.611.711.0
Difference between PASI and ACASI not statistically significant.
Lara et al. 2004;  November 2000  May 2001  MexicoFemales 15–55 years in 3 pop’ns: (a) 1480 in 3 public hospitals (11% in-patient) (b) 612 in rural community (c) 1000 in house-to-house survey in Mexico CityQuasi-experimental response rates for abortionFTFI SAQ ACASI RRTNo NoRRT yielded highest reporting of attempted abortion in all 3 study samples (only statistically significant in one):  Hospitals: 22%vs. ACASI = 13%vs. FTFI = 12%** If omit RRT & examine only literate women, using logistic regression, SAQ reports highest number of attempted abortions. Highest level of missing responses was with SAQ with 6.8% of questionnaires having missing data; SAQ was not feasible for illiterate women.

This analysis makes the general assumption that an increase in reporting of a socially censured behaviour indicates more accurate reporting (reducing social desirability bias and increasing validity) (Konings et al. 1995; Durant & Carey 2000; Brener et al. 2003; Kreuter et al. 2008).

Comparison with SAQ

All five studies that compared another method against SAQ used an audio computer-assisted survey instrument (ACASI) or a derivative thereof (e.g. personal digital assistant (PDA) or phone-ACASI) (see Table 2). Two were randomized control trials (Rumakom et al. 2005; Seebregts et al. 2008), one was quasi-experimental (Fielding et al. 2006) and two focused on test–retest (Jaspan et al. 2007; Bernabe-Ortiz et al. 2008). Three studies included school attenders (Rumakom et al. 2005; Jaspan et al. 2007; Seebregts et al. 2008).

Comparison of response rates.

Where non-response rates were reported, SAQ performed poorly against ACASI and its derivatives. In South Africa, SAQ respondents were seven times more likely to have missing items than respondents using PDA (Jaspan et al. 2007). In China, non-response rates to items were 8–14% with SAQ and 0.4–3.0% with phone-ACASI (Fielding et al. 2006).

Comparisons of reporting of sexual behaviours.

In general, SAQ respondents reported lower levels of risk exposure than respondents using computer self-administered modes (e.g. in China, bisexual sex (SAQ 0.6%vs. ACASI 2.6%, < 0.001; sex in the past year SAQ 50.3%vs. ACASI 56.5%, = 0.006; and belief that their partner had sex with others SAQ 4.3%vs. ACASI 7.0%, = 0.005) (Jaspan et al. 2007). The exception was among South African youth where there were no differences between SAQ and PDA except for ‘ever sex’, which was more often reported in SAQ (36%) than PDA (26%; = 0.003) (Jaspan et al. 2007).

Reliability between modes.

In three studies, respondents used SAQ followed by PDA (Jaspan et al. 2007; Bernabe-Ortiz et al. 2008; Seebregts et al. 2008). Response agreement between the two modes was high at 85% and a Kappa value of 0.5 or more.

Acceptability of modes.

Respondents were more likely to report that ACASI and its derivatives felt more confidential when answering sexual behavioural questions. For example, in the Thai study, 7.8% of college females who used SAQ reported feeling embarrassed answering the sexual behaviour questions, whereas fewer than 1.5% of female ACASI respondents reported this (Rumakom et al. 2005).

Comparisons with face-to-face interviewing (FTFI)

Seven of the 16 studies compared face-to-face interviewing against ACASI (Table 3). Five studies explored an adaptation of face-to-face interviewing which allowed respondents to self-report sensitive questions on a ballot card (Gregson et al. 2002, 2004; Phillips et al. 2007; Hanck et al. 2008; Jaya et al. 2008). One study from India compared an additional mode termed ‘interactive interviewing’ which included a tape-recorded drama and dolls to desensitize respondents around sensitive issues (Jaya et al. 2008). Four other studies compared face-to-face interviewing against in-depth interviewing (Konings et al. 1995), coital diaries (Ramjee et al. 1999; Allen et al. 2007) or used a derivative of SAQ where the questions were read aloud in a group setting (Plummer et al. 2004a,b).

As shown in Table 3, eight studies were randomized control trials (Simoes et al. 2006; Allen et al. 2007; Caceres et al. 2007; Minnis et al. 2007; Phillips et al. 2007; Hewett et al. 2008; Jaya et al. 2008; Mensch et al. 2008), five were quasi-experimental studies (Konings et al. 1995; Gregson et al. 2002, 2004; Bernabe-Ortiz et al. 2008; Hanck et al. 2008), and three focused solely on test–retest (Ramjee et al. 1999; Plummer et al. 2004a,b).

Comparison of response rates.

As interviewer presence renders it more difficult for respondents to ignore a question, few studies (3/15) reported non-item response rates. In Peru, where data entry was handled manually (FTFI) or directly into a PDA, missing responses were significantly more common using FTFI (< 0.001) (Bernabe-Ortiz et al. 2008).

Comparisons of reporting of sexual behaviours.

Overall, respondents using face-to-face interviewing reported fewer sensitive behaviours than respondents using other questionnaire delivery modes. Six of 7 studies that compared face-to-face interviewing with ACASI (RCT = 6) showed increased reporting of various sexual behaviours in ACASI. For example, urban Brazilian women were more likely to report STI risk behaviours in ACASI (e.g. anal sex in the last 6 months: 33%vs. 24%, < 0.01; no condom use with vaginal sex in the last month: 59%vs. 51%, < 0.01) (Hewett et al. 2008).

In a study in Zimbabwe on hormonal contraception where eligibility of respondents was conditional on specific behaviours, ACASI users were more likely to report an undesirable behaviour (Minnis et al. 2007). For example, not getting pregnant was a study condition and more pregnancies were reported in ACASI interviews than in face-to-face ones (OR = 1.5; 95% CI 1.1–1.9). ACASI users were also more likely to report multiple partners than FTFI users (OR = 5.7; 95% CI 2.1–15.2).

Data from India produced less consistent results where face-to-face interviewing was compared against both ACASI and interactive interviewing (Jaya et al. 2008). Among females, more sexual behaviours were reported using interactive interviewing than face-to-face interviewing, and they consistently reported fewer sexual behaviours using ACASI than FTFI. However, males using ACASI were more likely to report having had sex (26.9%vs. 21.4%; = 0.03) and having been forcibly touched (26.2%vs. 21.4%; = 0.09) than males using FTFI.

In two studies, there were no reported differences between ACASI and interviewer data collected using a computer (CAPI) (Jaspan et al. 2007; Bernabe-Ortiz et al. 2008). This was attributed to smaller sample sizes.

When face-to-face interviewing was compared with a non-ACASI mode (ICVI, coital diaries, in-depth interviewing and assisted self-completed questionnaires-see definitions at the end of Table 3), all eight studies found lower rates of reporting of sensitive behaviours using face-to-face interviewing. This was true for both socially desirable and socially undesirable behaviours. While data from coital diaries provided increased reporting of sexual behaviours (Allen et al. 2007), diary loss or incompletion posed a problem (Ramjee et al. 1999; Sedyaningsih-Mamahit & Gortmaker 2003). Four studies examined interviewer-controlled voting instrument (ICVI), where respondents marked their responses to sensitive questions privately and then posted them into a locked box (Gregson et al. 2002, 2004; Phillips et al. 2007; Hanck et al. 2008). Overall, there was an increase in reports of sexual behaviours among ICVI users (e.g. in Zimbabwe: multiple sex partners males OR = 1.33, =  0.028; females OR = 5.21, < 0.001). Only among transgendered males was there increased reporting of some behaviours but not others (Phillips et al. 2007).

Comparison of the same respondents’ answers to different questionnaire delivery modes.

In the three studies which examined respondent agreement, where differences occurred, reporting was less likely to have occurred during face-to-face interviews (Konings et al. 1995; Plummer et al. 2004a,b; Mensch et al. 2008). Tanzanian pupils were administered the same questionnaire in two modes (FTFI or ASCQ where respondents had the questions read aloud to them in a single-gender group setting) 5 weeks apart (Plummer et al. 2004a,b). Sixty-two percent of males and 41% of females reported having sex in both surveys (Plummer et al. 2004a). There was 64.4% agreement in reporting of age at first sex and 47.3% agreement around the number of sexual partners (Plummer et al. 2004b). Respondents were more likely to report condom use, forced sex, and pregnancy in ASCQ. For example, forced sex was reported by only one female in both surveys, but it was reported by 12.3% of females in ACSQ and by 0.2% in FTFI (Plummer et al. 2004b). In Uganda, 23.3% of respondents reported not engaging in sex when asked in the face-to-face interview but then reported sexual activity in an in-depth interview (Konings et al. 1995).

Comparison of SAQ and FTFI against other modes

There were seven studies that compared both SAQ and FTFI with at least one other mode. Six of these were conducted as randomized control trials (Mensch et al. 2003; Hewett et al. 2004; Potdar & Koenig 2005; Le et al. 2006; van Griensven et al. 2006; Langhaug et al. 2007; one used a quasi-experimental design (Lara et al. 2004; Table 4).

Comparison of response rates.

In both studies that reported on non-response rates to items, computer-administered questionnaires had the fewest missing data (van Griensven et al. 2006; Langhaug et al. 2007).

Comparisons of reporting of sexual behaviours.

Data on rates of reporting sexual behaviours are less clear. In India, among college males using SAQ, ACASI or FTFI, men responding to ACASI reported heterosexual sex (AOR = 1.8, < 0.05), oral sex with a female (AOR = 2.08, < 0.05), homosexual sex (AOR = 8.1, < 0.05) and having experienced coercive sex (AOR = 11.35, < 0.01) more often (Potdar & Koenig 2005). Among young men living in slums, ACASI users were more likely to report masturbation (AOR = 22.53, < 0.001) and oral sex with a woman (AOR = 2.4, < 0.010) but less likely to report vaginal sex (AOR = 0.23, < 0.001) than FTFI users.

In Viet Nam young people’s attitudes and norms around sexual behaviours and condom use were compared using SAQ, FTFI and ACASI (Le et al. 2006). Respondents using ACASI were more likely to report liberal attitudes around premarital sex (a socially censured norm) and to report less confidence in their ability to access condoms (a socially condoned behaviour). Female refusals to answer questions about condom use were most frequent among SAQ and FTFI users. ACASI users were also more likely to report sex (e.g. unmarried males’ sex with commercial sex workers OR = 2.8, < 0.05; males aged 15–19 having sex OR = 2.79, < 0.05).

Three studies compared four modes (Lara et al. 2004; van Griensven et al. 2006; Langhaug et al. 2007). In the Thai study, data generally showed no difference in reporting of sexual behaviours between SAQ, ACASI and palm-assisted self-interviewing (PASI). However, regarding self-reports of the most sensitive behaviours, there was a statistically significant difference (< 0.001) between PASI and FTFI (history of oral sex 37.3%vs. 13.2%; sex today/yesterday 19.3%vs. 6.1%; sold sex 8.2%vs. 0.9%; bought sex 8.2%vs. 2.5%). In the Zimbabwean study, after adjusting for covariates, Audio-SAQ and ACASI users were twice as likely to report sexual activity as SAQ users (Audio-SAQ AOR = 2.05 [95%CI: 1.2–3.4]; ACASI AOR = 2.0 [95%CI: 1.2–3.2]), with no reporting differences between ICVI and SAQ users (ICVI AOR = 1.0 [95%CI: 0.6–1.8) (Langhaug et al. 2007). ACASI users reported a younger age at first sex (0.7–1.7 years lower) (< 0.045). In a post-survey questionnaire, ACASI users reported improved ability to answer questions honestly (= 0.004) and believed their answers would be kept secret.

ACASI only performed poorly compared to the random response technique (RRT). RRT asks a respondent to answer one of two questions picked randomly where the interviewer is blinded to the question being answered. One question is sensitive and the other is not but for which there is a known probability (e.g. where you born in April?) for that population. Mathematical techniques allow indirect estimates of the proportion reporting the sensitive behaviour (Lara et al. 2004). In three sub-populations in Mexico, RRT users reported the highest rates of attempted abortion. Differences were only statistically significant in the hospital survey (RRT = 22%; SAQ = 19%; ACASI = 13%; FTFI = 12%, = 0.012) (Lara et al. 2004).

Comparison with biological markers

Whilst six of the 26 studies used biological markers as external comparators, only five examined sexual biomarkers (Plummer et al. 2004a,b; Langhaug et al. 2007; Hewett et al. 2008; Mensch et al. 2008), with one study evaluating drug use (van Griensven et al. 2006). Sexual biomarkers comprised Chlamydia, gonorrhoea, trichomoniasis, HIV and current pregnancy in females. While prevalence of biological markers varied among studies involving adolescents, absolute numbers were relatively small. In the study from Brazil among urban women, those who used ACASI had stronger associations between their reported risk behaviours and STIs (Hewett et al. 2008). STI positive respondents using FTFI were more likely to underreport sexual behaviours than their STI negative peers.

Discussion

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

The results outlined here reaffirm that questionnaire delivery modes do affect self-reported sexual behaviour (Tourangeau & Smith 1996; Turner et al. 1998; des Jarlais et al. 1999; Metzger et al. 2000; Tourangeau & Yan 2007). This systematic review is the first to examine data from developing countries that compares self-reports of sexual behaviours between various questionnaire delivery modes. Despite wide variation in geography and populations sampled, we found strong evidence that computer-assisted interviewing decreases item non-response rates and increases rates of reporting of sexual behaviours. This was true when ACASI and its derivatives (PASI, phone-ACASI, PDA) were compared with other self-administered and interviewer-administered questionnaire delivery modes. Comparative research using ACASI was predominantly conducted among young people (11 out of 16 studies were exclusively with young people and the other five included individuals aged 18 and older), whose reporting of sexual behaviours is very likely to be socially censured (Mensch et al. 2003, 2009; Hewett et al. 2004; Potdar & Koenig 2005; Rumakom et al. 2005; van Griensven et al. 2006; Le et al. 2006; Jaspan et al. 2007; Langhaug et al. 2007; Seebregts et al. 2009). Data entry errors were also reduced when controlled by a computer programme: in studies with no differences in reporting of sexual behaviours, ACASI still improved the quality of data entry (Jaspan et al. 2007; Bernabe-Ortiz et al. 2008; Seebregts et al. 2009).

Validation of self-reports against biomarkers for sexual activity was rarely available. Where sample sizes were sufficient, results suggest more accurate reporting using ACASI than face-to-face interviewing.

These studies also support the acceptability and feasibility of using computers in developing country settings. In those studies where it was examined, ACASI and its derivatives were found acceptable, easy to use, and respondents, particularly young women, reported feeling more comfortable using a computer to report sensitive behaviours than they did with other methods (Rumakom et al. 2005; Le et al. 2006; Langhaug et al. 2007). Similar findings have emerged from Zimbabwe and the US (Millstein & Irwin 1983; Kissinger et al. 1999; Metzger et al. 2000; van de Wijgert et al. 2000; Kurth et al. 2004). Acceptability of computer technologies may vary geographically and be related to level of exposure. Generally, an increased sense of trust and sense of privacy is expressed by those who live in countries where computers are less commonly used. Only one study compared ACASI against ICVI and found less reporting of sensitive behaviours using ICVI (Langhaug et al. 2007). Results of studies using ICVI or interactive interviewing which did not compare ACASI emphasize that any effort made to improve privacy levels when answering sensitive questions increases the reporting of sexual behaviours (Gregson et al. 2002; Hanck et al. 2008; Jaya et al. 2008). ACASI has not been compared against interactive interviewing. This research is needed to better establish their comparative strengths and limitations.

Equally encouraging is the work of Mensch et al. (2008) in Malawi and Hewett et al. (2004 in Kenya), which suggests that levels of literacy per se may not affect ACASI, as it was used successfully among rural youth. Potdar and Koenig (2005) used ACASI (and not SAQ) among young people living in Indian slums, where despite differences in reporting to college-educated peers, no mention was made of their inability to use the computers. However, evidence from three studies is not sufficient to suggest that ACASI can always be used in settings with low literacy rates; more feasibility research is required.

While coital diaries also demonstrated increased reporting of sensitive behaviours (Ramjee et al. 1999; Allen et al. 2007), they carry requirements which render them less suitable for large surveys: more logistical support, more time for data entry and specific training to ensure appropriate completion. Coital diaries have a low completion rate (20% in one study). However, including coital diary data from a sub-sample of a large survey population can complement the data collected by other means.

One of the strengths of this review is that a number of the studies reported both socially censured and socially condoned behaviours (Lau et al. 2003; Sedyaningsih-Mamahit & Gortmaker 2003; Potdar & Koenig 2005; Le et al. 2006; Simoes et al. 2006; Minnis et al. 2007; Hanck et al. 2008). Conclusions drawn from these studies are strengthened when users of a mode are not only more likely to report socially censured behaviours but also less likely to report socially acceptable behaviours (Catania et al. 1990). Reports for computer self-administered questionnaires followed this pattern. In India, more college men using ACASI reported behaving violently after drinking than those using FTFI (3.0 vs. 1.7%) (Potdar & Koenig 2005). Similarly, in a study in Zimbabwe where hormonal contraceptive use was a prerequisite for participation, women were more likely to report that they were not using them in ACASI than in FTFI (Minnis et al. 2007). Equally heartening is the growing comparative literature around questionnaire delivery modes in developing countries and the increased interest in ACASI and its derivatives.

There are some limitations to this review. A number of studies did not show statistically significant differences around reporting of sexual behaviours between questionnaire delivery modes. This is in part attributable to the small sample size of these studies, or when youth were sampled, to the small number who reported sexual behaviours overall. Studies did not report the same sexual behaviour outcomes, nor did they always disaggregate their data by gender or age. This made it difficult to make comparisons across studies. Only five studies included biological markers of sexual behaviour as part of their analysis. Biological markers offer complimentary evidence that can be used to explore directions of effect. For most sexual behaviour variables, it is assumed that higher levels of reporting indicate more valid reporting, but the ability to triangulate against objective data improves our understanding of the differences in self-reported sexual behaviours between questionnaire delivery modes. Researchers should incorporate biological markers (or other externally valid outcomes) into evaluations whenever possible to broaden the evidence within comparative studies. We did not examine the effect of interviewer age or gender. Research on this has been extensive but inconclusive (Blanc & Croft 1992; Catania et al. 1996; Elam & Fenton 2003; Wellings et al. 1990).

In 2003, a technical meeting on “Measurement of Trends in Sexual Behaviour” called for more rigorous comparative studies before anything more definitive could be concluded (Cleland et al. 2004). Since then, there has been a noteworthy increase in the number of published articles in peer-reviewed journals comparing questionnaire delivery modes. Most articles considered in this review, which focussed exclusively on research performed in development country settings, were published after 2003 (= 21/28). Their data strongly suggest the use of computer-assisted methods.

This is important in view of the fact that the principal data collection tool for sexual behaviours in developing countries remains the interviewer-administered questionnaire. While interviewer-administered questionnaires remain an important tool for collecting survey data, this review provides good evidence that self-administered options, especially those using computers, will enhance data quality, particularly of socially sensitive data.

Acknowledgements

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We express our profound gratitude to Angela Young, the wonderful librarian who assisted in structuring this systematic review.

References

  1. Top of page
  2. Summary
  3. Background
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • Allen CF, Lees SS, Desmond NA et al. (2007) Validity of coital diaries in a feasibility study for the Microbicides Development Programme trial among women at high risk of HIV/AIDS in Mwanza, Tanzania. Sexually Transmitted Infections 83, 490496.
  • Becker S & Sosa D (1992) An experiment using a month-by-month calendar in a family planning survey in Cosa Rica. Studies in Family Planning 23, 386391.
  • Bernabe-Ortiz A, Curioso WH, Gonzales MA et al. (2008) Handheld computers for self-administered sensitive data collection: a comparative study in Peru. BMC Medical Informatics & Decision Making 8, 11.
  • Blanc AK & Croft TN (1992) The Effect of the Sex of the Interviewer on Responses in Fertility Surveys: the Case of Ghana. Annual Meeting of the Population Association of America, Denver.
  • Brener ND, Billy J & Grady WR (2003) Assessment of factors affecting the validity of self-reported health-risk behaviour among adolescents: evidence from the scientific literature. Journal of Adolescent Health 33, 436457.
  • Caceres CF, Celentano DD, Coates TJ et al. (2007) The feasibility of audio computer-assisted self-interviewing in international settings. AIDS 21(Suppl. 2), S49S58.
  • Catania JA, Gibson DR, Chitwood DD & Coates TJ (1990) Methodological problems in AIDS behavioral research: influences on measurement error and participation bias in studies of sexual behavior. Psychological Bulletin 108, 339362.
  • Catania JA, Binson D, Canchola J, Pollack LM & Hauck W (1996) Effects of interviewer gender, interviewer choice, and item wording on responses to questions concerning sexual behavior. Public Opinion Quarterly 60, 345375.
  • Cleland J, Boerma JT, Carael M & Weir SS (2004) Monitoring sexual behaviour in general populations: a synthesis of lessons of the past decade. Sexually Transmitted Infections 80(Suppl. 2), ii1ii7.
  • Durant LE & Carey MP (2000) Self-administered questionnaires versus face-to-face interviews in assessing sexual behavior in young women. Archives of Sexual Behavior 29, 309322.
  • Elam G & Fenton KA (2003) Researching sensitive issues and ethnicity: lessons from sexual health. Ethnicity & Health 8, 1527.
  • Fenton KA, Johnson AM, McManus S & Erens B (2001) Measuring sexual behaviour: methodological challenges in survey research. Sexually Transmitted Infections 77, 8492.
  • Fielding R, Lam TH & Hedley A (2006) Risk-behavior reporting by blood donors with an automated telephone system. Transfusion (Paris) 1, 289297.
  • Gregson SA, Zhuwau T, Ndlovu J & Nyamukapa CA (2002) Methods to reduce social desirability bias in sex surveys in low-development settings: experience in Zimbabwe. Sexually Transmitted Diseases 29, 568575.
  • Gregson SA, Mushati P, White PJ, Mlilo M, Mundandi C & Nyamukapa C (2004) Informal confidential voting interview methods and temporal changes in reported sexual risk behaviour for HIV transmission in sub-Saharan Africa. Sexually Transmitted Infections 80(Suppl. 2), ii36ii42.
  • Van Griensven F, Naorat S, Kilmarx PH et al. (2006) Palmtop-assisted self-interviewing for the collection of sensitive behavioral data: randomized trial with drug use urine testing. American Journal of Epidemiology 163, 271278.
  • Hanck SE, Blankenship KM, Irwin KS, West BS & Kershaw T (2008) Assessment of self-reported sexual behavior and condom use among female sex workers in India using a polling box approach: a preliminary report. Sexually Transmitted Diseases 35, 489494.
  • Hewett PC, Mensch BS & Erulkar AS (2004) Consistency in the reporting of sexual behaviour by adolescent girls in Kenya: A comparison of interviewing methods. Sexually Transmitted Infections 80(Suppl. 2), ii43ii48.
  • Hewett PC, Mensch BS, Ribeiro MCSD et al. (2008) Using sexually transmitted infection biomarkers to validate reporting of sexual behavior within a randomized, experimental evaluation of interviewing methods. American Journal of Epidemiology 168, 202211.
  • International Monetary Fund (2009) IMF emerging and developing economies list: world economic outlook database. http://www.imf.org/external/pubs/ft/weo/2009/01/wepdata/groupshtm#oem .
  • Des Jarlais DC, Paone D, Milliken J et al. (1999) Audio-computer interviewing to measure risk behaviour for HIV among injecting drug users: A quasi-randomised trial. Lancet 353, 16571662.
  • Jaspan HB, Flisher AJ, Myer L et al. (2007) Brief report: methods for collecting sexual behaviour information from South African adolescents – a comparison of paper versus personal digital assistant questionnaires. Journal of Adolescence 30, 353359.
  • Jaya J, Hindin MJ & Ahmed S (2008) Differences in young people’s reports of sexual behaviors according to interview methodology: a randomized trial in India. American Journal of Public Health 98, 169174.
  • Jones R (2003) Survey data collection using Audio Computer Assisted Self-Interview. Western Journal of Nursing Research 25, 349358.
  • Kissinger P, Rice J, Farley T et al. (1999) Application of computer-assisted interviews to sexual behavior research. American Journal of Epidemiology 149, 950954.
  • Konings E, Bantebya G, Carael M, Bagenda D & Mertens T (1995) Validating population surveys for the measurement of HIV/STD prevention indicators. AIDS 9, 375382.
  • Kreuter F, Presser S & Tourangeau R (2008) Social desirability bias in CATI, IVR, and web surveys: the effects of mode and question sensitivity. Public Opinion Quarterly 72, 847865.
  • Kurth AE, Martin DP, Golden MR et al. (2004) A comparison between audio computer-assisted self-interviews and clinician interviews for obtaining the sexual history. Sexually Transmitted Diseases 31, 719726.
  • Langhaug LF, Cheung YB, Pascoe SJS, Mavhu W, Chirawu P & Cowan FM (2007) Comparing four questionnaire delivery methods for collection of self-reported sexual behaviour data in rural Zimbabwean youth. 17th ISSTDR, Seattle.
  • Lara D, Strickler J, Olavarrieta DiazC & Ellerston C (2004) Measuring induced abortion in Mexico: A comparison of four methodologies. Sociological Methods & Research 21, 529558.
  • Lau JTF, Tsui HY & Wang QS (2003) Effects of two telephone survey methods on the level of reported risk behaviours. Sexually Transmitted Infections 79, 325331.
  • Le LC, Blum RW, Magnani R, Hewett PC & Do HM (2006) A pilot of audio computer-assisted self-interview for youth reproductive health research in Vietnam. Journal of Adolescent Health 38, 740747.
  • Mavhu W, Langhaug LF, Manyonga B, Power R & Cowan FM (2008) What is ‘sex’ exactly? Using cognitive interviewing to improve validity of sexual behaviour reporting among young people in rural Zimbabwe. Culture, Health, and Sexuality 10, 563572.
  • Mensch BS, Hewett PC & Erulkar AS (2003) The reporting of sensitive behavior by adolescents: a methodological experiment in Kenya. Demography 40, 247268.
  • Mensch BS, Hewett PC, Gregory R & Helleringer S (2008) Sexual behavior and STI/HIV status among adolescents in rural Malawi: an evaluation of the effect of interview mode on reporting. Studies in Family Planning 39, 321334.
  • Metzger DS, Koblin B, Turner CF et al. (2000) Randomized controlled trial of audio computer-assisted self-interviewing: Utility and acceptability in longitudinal studies. American Journal of Epidemiology 152, 99106.
  • Millstein SG & Irwin CE (1983) Acceptability of computer-acquired sexual histories in adolescent girls. Journal of Pediatrics 103, 815819.
  • Minnis AM, Muchini A, Shiboski S et al. (2007) Audio computer-assisted self-interviewing in reproductive health research: reliability assessment among women in Harare, Zimbabwe. Contraception 75, 5965.
  • Phillips AE, Lowndes CM, Boily MC et al. (2007) Comparison of informal confidential voting and face-to-face interviewing in collecting sexual research data from men who have sex with men (MSM) and transgenders (HIJRA) in Bangalore. 17th ISSTDR, Seattle.
  • Plummer ML, Ross DA, Wight D et al. (2004a) “A bit more truthful”: the validity of adolescent sexual behaviour data collected in rural northern Tanzania using five methods. Sexually Transmitted Infections 80(Suppl. 2), ii49ii56.
  • Plummer ML, Wight D, Ross DA et al. (2004b) Asking semi-literate adolescents about sexual behaviour: the validity of assisted self-completion questionnaire (ASCQ) data in rural Tanzania. Tropical Medicine & International Health 9, 73754.
  • Potdar R & Koenig MA (2005) Does Audio-CASI improve reports of risky behavior? evidence from a randomized field trial among young urban men in India Studies in Family Planning 36, 107116.
  • Ramjee G, Weber AE & Morar NS (1999) Recording sexual behavior: comparison of recall questionnaires with a coital diary. Sexually Transmitted Diseases 26, 374380.
  • Rumakom P, Guest P, Chinvarasopak W, Utarmat W & Sontanakanit J (2005) Obtaining accurate responss to sensitive questions among Thai students: a comparison of two data collection techniques. In: Sex Without Consent (eds SJejeebhoy, IShah & SThapa) Zed Books, London, pp. 318332.
  • Sedyaningsih-Mamahit E & Gortmaker S (2003) Reproducibility and validity of self-reported condom use in Jakarta. Southeast Asian Journal of Tropical Medicine & Public Health 34, 136146.
  • Seebregts CJ, Zwarenstein M, Mathews C et al. (2009) Handheld computers for survey and trial data collection in resource-poor settings: development and evaluation of PDACT, a Palm Pilot interviewing system. International Journal of Medical Informatics 78, 721731.
  • Simoes AA, Bastos FI, Moreira RI, Lynch KG & Metzger DS (2006) A randomized trial of audio computer and in-person interview to assess HIV risk among drug and alcohol users in Rio De Janeiro, Brazil. Journal of Substance Abuse Treatment 30, 237243.
  • Tourangeau R & Smith TW (1996) The impact of data collection mode, question format, and question context. Public Opinion Quarterly 60, 275304.
  • Tourangeau R & Yan T (2007) Sensitive questions in surveys. Psychological Bulletin 133, 859883.
  • Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH & Sonenstein FL (1998) Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science 280, 867873.
  • UNAIDS (2008) AIDS epidemic update: December 2007. UNA-IDS/07.27E/JC1322E.
  • Weinhardt LS, Forsyth AD, Carey MP, Jaworski BA & Durant LE (1998) Reliability and validity of self report measures of HIV-related sexual behavior: progress since 1990 and recommendations for research and practice. Archives of Sexual Behavior 27, 155180.
  • Wellings K, Nanchahal K, Macdowall W et al. (2001) Sexual behaviour in Britain: early heterosexual experience. Lancet 358, 18431850.
  • Van De Wijgert J, Padian NS, Shiboski S & Turner CF (2000) Is audio computer assisted interviewing a feasible method of surveying in Zimbabwe? International Journal of Epidemiology 29, 885890.
  • Wellings K, Field J, Wadsworth J, Johnson AM, Anderson RM & Bradshaw SA (1990) Sexual lifestyles under scrutiny. Nature 348, 276278.
  • Wilson D & Halperin DT (2008) “Know your epidemic, know your response”: a useful approach, if we get it right. Lancet 372, 4236.