Response patterns to interactive SMS health education quizzes at two sites in Uganda: a cohort study


Corresponding Author Linda Oskam, Royal Tropical Institute, KIT Biomedical Research, Meibergdreef 39, 1105 AZ Amsterdam, the Netherlands. Tel.: +31(0)20 5665446; Fax: +31(0)20 6971841; E-mail:



The use of mobile phones can improve and strengthen (preventive) health care in low- and middle-income countries. We aimed to retrospectively assess the response patterns of participants in free SMS health education quizzes in Uganda.


Study participants were employees of two companies and their community networks. We investigated how quickly individuals responded to quiz question(s) and assessed possible influencing factors. Cox regression and anova analyses were used.


Fifty percentage of responders answered within 50 min. The response chance declined with every additional day after sending an incentive via SMS (Hazard Ratio 0.993, CI 95% 0.981–0.984). Quiz topics influenced both participation rates and response time. Response time was shortest for questions on HIV and sexual behaviour. Response rates were high for HIV (79%) and malaria (78.4%), but only 37.4% for demographic topics. Network providers had a substantial effect on response behaviour.


Interactive SMS programs are a fast method to reach the target population and incentives increase response rates. The most important factor influencing response time and participation rate is the network provider. Future research should focus on developing evidence-based guidelines for the design, implementation and evaluation of SMS-based interventions.



L'utilisation des téléphones portables peut aider dans les soins de santé dans les pays à revenus faibles et intermédiaires. Nous avons cherché à évaluer rétrospectivement le comportement de réponse des participants dans un test par SMS gratuit sur l’éducation à la santé en Ouganda.


Les participants dans l’étude étaient des employés de deux sociétés et leurs réseaux communautaires. Nous avons étudié la rapidité avec laquelle les individus ont répondu aux questions tests et avons évalué les possibles facteurs influençant. La régression de Cox et les analyses anova ont été utilisées.


50% des répondants ont répondu dans les 53 min. Les chances de réponse ont diminué chaque jour supplémentaire après l'envoi d'une incitation par SMS (Rapport de risque 0.993; IC 95%: 0.981–0.984). Les sujets du test influençaient les taux de participation et le temps de réponse. Le temps de réponse était le plus court pour les questions concernant le VIH et le comportement sexuel. Le sujet de la question influençait les taux et le temps de réponse. Les taux de réponse étaient élevés pour le VIH (79%) et le paludisme (78.4%), mais seulement de 37.4% pour les sujets démographiques. Les fournisseurs de réseaux avaient un effet considérable sur le comportement de réponse.


Les programmes interactifs par SMS sont une méthode rapide pour atteindre la population cible et les incitations augmentent les taux de réponse. Le facteur le plus important qui influence le temps de réponse et le taux de participation est le fournisseur de réseau. Les recherches futures devraient se concentrer sur l’élaboration de directives fondées sur des preuves pour la conception, la mise en œuvre et l’évaluation des interventions basées sur les SMS.



El uso de teléfonos móviles puede ayudar en los cuidados sanitarios en países con ingresos bajos y medios. Buscábamos evaluar de forma retrospectiva la respuesta en el comportamiento de participantes en una encuesta gratuita por SMS sobre educación sanitaria realizada en Uganda.


Los participantes del estudio eran empleados de dos compañias y de sus redes comunitarias. Investigamos qué tan rápido los individuos respondían las preguntas de la encuesta y evaluamos posibles factores que podían influenciar. Se utilizaron los análisis de regresión de Cox y de anova.


Un 50% de quienes respondieron lo hicieron en 53 min. La posibilidad de responder disminuyó con cada día adicional después de haber enviado un incentivo por SMS (Hazard Ratio 0.993, IC 95% 0.981–0.984). Los temas de la encuesta tuvieron influencia tanto en las tasas de participación como en los tiempos de respuesta. El tiempo de respuesta fue el más corto para las preguntas sobre VIH y comportamiento sexual. Los temas de las preguntas tenían influencia sobre la tasa de respuesta y el tiempo. La tasa de respuesta era alta para el VIH (79%) y la malaria (78.4%), pero solo de un 37.4% para temas demográficos. Los proveedores de las redes tienen un efecto sustancial sobre el comportamiento de respuesta.


Los programas de SMS Interactivos son un método rápido para llegar a la población diana, y con incentivos se aumenta la tasa de respuesta. El factor más importante que influye en el tiempo de respuesta y la tasa de participación es el proveedor de la red. Investigaciones futuras deberían enfocarse en el desarrollo de unas guías basadas en la evidencia para el diseño, la implementación y la evaluación de las intervenciones basadas en SMS.


Using mobile communication technologies is an exciting development to improve and strengthen health care in low- and middle-income countries (LMIC) (mHealth). mHealth targets prevention, diagnosis and treatment of disease by improving health services, health education and healthy behaviour (WHO 2008; Krishna et al. 2009; UNAIDS 2009; Free et al. 2010; Kew 2010; Maher et al. 2010). Text messaging (SMS) is becoming an important tool to reduce the burden on healthcare systems as it is inexpensive and uses existing infrastructure to facilitate communication (Cole-Lewis & Kershaw 2010).

The not-for-profit organisation Text to Change (TTC) uses an interactive SMS quiz system (TMQ) to address various health issues in LMIC. In a retrospective analysis, we investigated potential influencing factors on parameters for response patterns to one of TTCs SMS interventions in Uganda.



TMQ participants were employees and their community networks (colleagues, relatives, friends, business relations, neighbours etc.) of two large companies representing both rural (cohort 1; sugar cane factory, ‘KSWL’, 2833 persons) and urban (cohort 2; cobalt mine factory, ‘KCCL’, 1935 persons) cohorts. The inclusion criterion was participation in the TMQ of 2009 and/or 2010. Characteristics of the study population are given in Table 1.

Table 1. Network and Quiz Characteristics of the study population (N = 4768)
 Number of questions in quizIncentives (N)Participantsa (N)Participantsa (%)
  1. a

    Participants in that joined the TMQ 2009 and 2010 are counted in both years, thus % counts up to 114.4.

  2. b

    Totals may differ per characteristic from the total number of participants, due to the fact that not all participants answered all (demographic) questions.

Network provider
Provider 1  2044.3
Provider 2  407585.5
Provider 3  982.1
Provider 4  3757.9
Missing  160.3
Total  4768100
Quiz characteristics
Urban (KCCL) 200924b6795a16.7a
Urban (KCCL) 201030b61369a28.7a
Rural (KSWL) 200924b61544a32.4a
Rural (KSWL) 201033b51745a36.6a

Data collection

Data were gathered from these cohorts and built on telephone numbers registered in the TTC database. Telephone numbers were anonymised and TMQ questions, responses to these questions, the response time and the chosen network provider by individual phone users were extracted from the database.

Telephone numbers were excluded from further analysis if we could assume beyond reasonable doubt that a telephone number was shared by two or more persons, that is, when multiple contradictory replies to the questions on sex and age were received. Information on demographic data (sex, age, education level) was gathered via SMS. Answers were only included in the analyses after formal registration for the quiz.

Quiz set-up

Text to Change uses a voluntary, free-of-charge, opt-in interactive SMS education program. There were 111 questions in four TMQs (2 years, two cohorts) divided over the topics family planning (8), HIV/AIDS/Sexually transmitted diseases (STD) (14), malaria (5), tuberculosis (5), sexual behaviour (12), medical male circumcision (8) and population demographics (4). The time of day that questions were sent out was categorised.

Five or six incentives per quiz were used to stimulate participation. Incentives consisted of SMS reminders on the possibility to win prizes or to undergo free HIV testing. Prizes entailed T-shirts, mobile phones or prepaid airtime (0.57–2.84€). Everyone who subscribed to the TTC SMS quiz was eligible for receiving an incentive. At random, a telephone number was extracted from the TTC database, and this person was informed through SMS and a telephone call that he or she had won a prize. Hereafter, it was communicated to the other participants that they did not win a prize. Furthermore, SMS was used to send reminders to participants that they were eligible to win a prize if they participated in the quiz (incentives were not dependent on the number of right questions or participation rate).

Quiz questions contained a question-specific keyword and two-choice answers (Table 2), and respondents received feedback. If a reply did not follow the instructions (thus not interpretable for the receiving electronic equipment), participants received an SMS requesting them to try again. Some participants sent in several answers; in this case, the first interpretable answer was used.

Table 2. TMQ text message example
QuestionSexually transmitted infections (STIs) can lead to infertility in both men and women
1. True
2. False
To reply type: BOTH<space><Answer Nr> & send to 8181
Possible answersBOTH 1; BOTH 2
Feedback messages
Right answer: (BOTH 1) Well done! STIs can lead to infertility in both men and women. You could have an STI without having clear symptoms. Seek early treatment if you have any doubts
Wrong answer (BOTH 2)No, it's TRUE! STIs can lead to infertility in both men and women. You could have an STI without having clear symptoms. Seek early treatment if you have any doubts
Incorrectly formulated (e.g. BOTH its false)Sorry, the answer ‘its false’ was not recognised, please correct any misspellings and try again

Endpoint variables

Endpoint variables were response time (RT), the percentage of answered questions and participation rate. The RT is the time in minutes between sending out a question and receiving a response. The calculated RT was the median response time over all the questions that the participant answered after opting in (LogRTMed). If the RT was missing (by not answering or not opting in to the quiz), a time of 7200 min was set as maximum. The percentage of answered questions was defined as the number of answered questions divided by the number of received questions after opting in. The participation rate was defined as the percentage of participants that responded to one or more TMQ questions after enrolment.

Statistical analysis

Response behaviour was evaluated, focusing on whether an individual responded to quiz question(s) and the time to response in relation to incentives and the utilised network provider. A uni- and multivariate Cox proportional hazard model (Cox regression) was used, and a robust standard error was calculated to correct for clustering of questions within participants; the influence of incentives on the participation rates and RT was also investigated. Factors influencing the percentage of answered questions or an individual's median RT were evaluated through analysis of variance (anova). As this median RT was skewed to the right, the 10 log value of the median RT was used.


The study was carried out in compliance with the Helsinki Declaration (2008). It was approved and registered (IS 75) by the National HIV/AIDS Research Committee in Kampala, Uganda, an institute accredited by the Uganda National Council for Science and Technology.


Study population

In total, 4768 people were included: 1654 (34.7%) people participated only in 2009, 2429 (50.9%) only in 2010 and 685 (14.4%) in both years. Sixteen telephone numbers (1.09%) were excluded because of suspected phone-sharing. Of all participants, 32.4% stated their sex, 29.4% their age and 27.5% their education level (data not shown). Notably, 85.5% used network provider 2 (Table 1).

Participation rates

The number of received questions differed as participants could opt in at any time. In 2009, the median number of received questions was 17 [Inter quartile range (IQR) 12–23]; this was 24 in 2010 (IQR 15–30). Approximately 30% of the enrolled participants never answered any of the received quiz questions (40% in 2009, 17.1% in 2010). In 2009, a median 25% (IQR 0–85%) of the received questions were answered while this was 57% (IQR 12–85%) in 2010.

The quiz topic also proved to be an important influencing factor: 79% of the HIV and 78% of the malaria questions were answered, but this was only 37% for questions on population demographics (Table 4).

The most evident influencing factor in a multivariate analysis on the percentage of answered questions per participant was the network provider: Provider 2 (HR 1.239, 95% CI 1.076–1.426) was associated with a higher response rate and a faster response time (Tables 3 and 4).

Table 3. Multivariate anova on the percentage of answered questions after opting in, per individual
Multivariate (N = 1030)Percentage of answered questionsadf F ChangeCI 95% P
  1. df, degrees of freedom; CI (95%): 95% confidence interval.

  2. a

    Per participant, derived from the univariate analysis.

  3. b

    Reference variable.

Provider 127.01−0.0199−0.350 to −0.04
Provider 257.910.1370.084 to 0.188
Provider 337.71−0.183−0.330 to 0.02
Provider 4b41.710
Table 4. Cox proportional hazards method on response time (RT) per individual per question
VariablesPer cent of participants that responded to the questions (%)aMedian RT of respondent (min)Time that 50% of population responded (min)Univariate analysis
dfHRCI (95%) P
Quiz company
    Multivariate analyses
dfHRCI (95%) P
  1. df, degrees of freedom; HR, hazard ratio; CI (95%), 95% confidence interval, MMC, medical male circumcision; STD: sexually transmitted diseases.

  2. a

    Derived from frequency analyses.

  3. b

    Reference value.

  4. c

    Only one question.

Participance year
Overall65.3506423 <0.0001
Provider 130.88410.5240.343–0.802
Provider 268.85046811.2391.076–1.426
Provider 348.53610.6720.557–0.810
Provider 4b52.74211
Quiz topic
Sexual behaviour63.82926711.1931.087–1.310 
Population demographics37.46311.0190.964–1.077 
Family planning53.374560210.9690.913–1.028 
Time sent
8:00–11:00 hoursb63.75058711
11:05–14:00 hours66.74240710.9930.958–1.027
14:05–17:00 hours63.174562610.8340.805–0.864
17:05–20:00 hours91.13226611.0290.973–1.087
20.05–8:00 hoursc10098137410.7100.568–0.887

Response time

The median RT of the responders was 50 min (IQR 18–260 min) (Table 4). The median RT of the entire study population was much higher with an IQR of 67–5626 min, depending on the question.

Most questions (71/111) were sent out between 8:00 and 11:00 hours. Others were sent out between 11:05 and 14:00 hours (22/111), 14:05 and 17:00 hours (13/111), and 17:05 and 20:00 hours (4/111); one question was sent out at midnight. Questions sent between 14:05 and 17:00 hours were less frequently answered (HR 0.834; 95% CI 0.805 and 0.864) and after a longer delay than questions sent out at other times. The RT was also influenced by the quiz topics (Table 4), although this was also dependent on the participation rates.

The effect of incentives

Every additional day after an incentive, the chance of answering declined with 0.7% compared to the response chance when the question was sent on the same day as the incentive, dropping to 35.6% of the initial percentage after 92 days.


The purpose of this study was to analyse response patterns in voluntary opt-in, free-of-charge SMS quizzes in Uganda. The influence of demographic factors on response patterns was not further investigated due to a high percentage of missing data.

Because of the large study population, almost all variables show a significant influence on response patterns, but minimal changes were often irrelevant for daily practice. The most relevant factor is the network provider. Provider 2 is the most widely used provider (85%), but inter-provider differences remained relevant. The most plausible explanation is that network coverage and quality in Uganda differs between providers, resulting in a delay or failure to send or deliver SMS. However, the association is not very strong (HR 1.239, 95% CI 1.076–1.426), and it is possible that other demographic characteristics [sex, age and education level did not vary across providers (results not shown)] such as socio-economic status or place of living may have differed between the providers, possibly confounding this result.

Incentives improve participation rates. This finding is in accordance with Fjeldsoe et al. (2009) and Trevena et al. (2006) and illustrates that it is good for the response rate to give frequent incentives. In our study population, SMS sent out between 11 and 14 or 17 and 20 h had the highest response rate. To improve participation rates and RT, the timing of SMS questions should be carefully evaluated. The optimal time to send out an SMS may vary across populations, as specific work patterns may vary.

Quiz topics differed in response rate and time (Table 2). The most obvious explanation is that some topics are less appealing or too personal. Questions addressing population demographics elicited a significantly lower response rate suggesting that participants might have been suspicious about how personal information was used. Furthermore, SMS semantics or personal relevance of the question topic could have influenced the response rates to different quiz topics.


A major limitation of our study was its retrospective setting, leading to incorrigible high percentages of missing data. Furthermore, the TMQ was not primarily intended for research and thus not designed using evidence-based principles. Data on population demographics were poorly answered. Approximately one-third of the participants responded to these questions (32.4% stated their sex, 29.4% their age and 27.5% their education level), rendering these data at risk for selection bias. Therefore, we were unable to further analyse our demographic data.

The nature of SMS predisposes to a selection bias: people with no or low literacy skills, insufficient knowledge on SMS, without enough funds to buy, own or share a mobile phone, or people living in areas with bad network coverage are hard to reach. We expect these problems to decrease rapidly in LMIC with the generally young, technology-minded population and improving network coverage. A selection bias may have occurred in our study population. However, as the community networks were also involved, it is likely that our study population is diverse and representative for the general Ugandan population. Furthermore, the risk of a selection bias was reduced by investigating populations from rural and urban settings.

Conclusion and recommendations

Despite the limitations of this retrospective analysis, we have found strong evidence showing the network provider to be pivotal in the participation rates, response times and thus the success rates of SMS intervention programs in LMIC. Incentives were efficient in stimulating participation. Response rate was also influenced by quiz topic. To increase success rates of promising SMS programs, we recommend to carefully select network providers and analyse delivery reports for accurate evaluation of the response patterns. The influences of population characteristics should be evaluated in a prospective controlled trial. Furthermore, future research should evaluate the impact of health focused SMS interventions in both high- and low-income countries on health-seeking behaviour.