Prospective Multicenter Evaluation of the Expert System “KABISA TRAVEL” in Diagnosing Febrile Illnesses Occurring After a Stay in the Tropics
Jozef Van den Ende, MD, PhD, Department of Clinical Sciences, Institute of Tropical Medicine, Kronenburgstraat, 43/3, B-2000 Antwerp, Belgium. E-mail: email@example.com
Background. KABISA TRAVEL is a clinical decision support system developed by the Institute of Tropical Medicine of Antwerp, Belgium, for the diagnosis of febrile illnesses after a stay in the tropics. This study aimed to compare the diagnostic accuracy of KABISA TRAVEL with that of expert travel physicians.
Methods. From December 2007 to April 2009, travelers with fever after a stay in the tropics were included in a multicenter trial conducted in travel referral centers in the Netherlands, Italy, Spain, and Belgium. Physicians were asked (1) to rank their first assessment diagnoses, (2) to enter in KABISA TRAVEL clinical and laboratory data available within 36 hours, and (3) to interact with the tutor until its final diagnostic ranking. Both physicians and KABISA TRAVEL rankings were then compared with the final diagnosis confirmed by reference methods. The clinical utility was also surveyed.
Results. A total of 205 cases with confirmed diagnosis were evaluated (male/female ratio: 1.85; mean age: 35 y). Most patients were western travelers or expatriates (60%) and were returning from sub-Saharan Africa (58%). Travel physicians and KABISA TRAVEL ranked the correct diagnosis in the first place for 70 and 72% of the cases, respectively, and within the top five both for 88% of them. Travel physicians reported having been suggested useful further investigations in 16% of the cases, and having been helped for obtaining the diagnosis in 24%. This was reported more frequently when they had initially missed the diagnosis (suggestion: 48% in missed vs 12% in found diagnoses, p < 0.001; helpful: 48% in missed vs 21% in found diagnoses, p = 0.005).
Conclusions. KABISA TRAVEL performed as well as expert travel physicians in diagnosing febrile illnesses occurring after a tropical stay. Clinicians perceived the system as more helpful when they had not immediately considered the correct diagnosis.
Fever is a leading reason for consultation in travel clinics, together with diarrhea and skin disorders.1 It is also the most challenging travel-related symptom because the differential diagnosis is wide, most tropical febrile diseases present with nonspecific features, and severe complications may sometimes rapidly develop.2,3 Clinical decision support systems (CDSS) have been developed with the aim to improve patient care. Several types of computer-based support systems exist such as diagnostic systems (also called expert systems), disease management systems, and drug dosing or prescribing systems. Various CDSS have been evaluated in different medical fields and have often demonstrated useful guidance for practitioners.4
So far, two CDSS have been designed for specific e-assistance in diagnosing infectious diseases, and in particular travel-related conditions: the Global Infectious Diseases and Epidemiology Network (GIDEON) (http://www.gideononline.com)5–7 and Fever Travel (http://www.fevertravel.ch) developed by the University of Lausanne, Switzerland.8 Each support system has a different design and focus. GIDEON is an expert system based on a probabilistic (Bayesian) approach and relies on an impressive global epidemiological database as an aid to diagnose infectious diseases worldwide. It focuses rather on infectious diseases specialists, gives a probability ranking of possible diagnoses with extensive documentation of diseases, but needs payment. Fever Travel has an algorithmic design based on both evidence and expert opinion, with the purpose of providing guidance in the management of travel-related conditions in nonendemic settings, mainly for clinicians not familiar with tropical diseases. It suggests further work-up, reference to travel specialist or hospitalization, and even presumptive treatments. Fever Travel is freely downloadable.
KABISA is a computer-based tutorial for tropical medicine, which has been used since 1992 for teaching at the Institute of Tropical Medicine, Antwerp, Belgium, as well as in many teaching centers overseas.9 Kabisa is Swahili for “hand in the fire, I'm absolutely certain,” referring to a clinician experiencing a straightforward pattern recognition. In 2008 the logical engine of this software was used for the development of an interactive expert system, KABISA TRAVEL (version IV). This system relies on a database currently containing >300 diseases and >500 findings, which are classified in five main categories (epidemiological characteristics, symptoms, clinical signs, laboratory data, results of imaging). Prevalence of diseases and frequency of related findings were entered according to evidence-based data obtained from a large prospective study in our center which explored the etiology of fever after a tropical stay as well as to the global epidemiological results published by the GeoSentinel group.1,3,10
When the user enters a present (or absent) finding, the software calculates the disease probabilities and provides a ranking of hypotheses. It relies on an adapted Bayesian approach. Following Bayes' theorem, pretest odds are multiplied by successive likelihood ratios, but the latter are recalculated at every step as the false positive rate depends on the spectrum of diseases still active at that moment of consultation (“dynamic specificity”).11 Second, the false positive rate calculated with the active hypotheses' prevalences and the related sensitivities is corrected by a “noise factor,” that is, presence of a nonspecific finding such as a cough without an identifiable cause. Third, if two conditionally dependent findings are entered, only the one with the highest positive likelihood ratio is accounted for. Finally, at every step, the sum of all probabilities is reset at 100%.
At any time during the consultation, the user can also ask the help of the tutor module that lists the relevant findings to explore, or suggests step-by-step further testing, with reassessment of the case each time a new finding is entered (“wizard” button). The tutor will not end the case before the probability of a diagnosis is considered high enough by the system (over the treatment threshold), before all relevant excluders for this disease have been exhausted, and before competing dangerous and treatable diagnoses are sufficiently excluded.
Following this study and coinvestigator's suggestions, KABISA TRAVEL has been upgraded recently, but with no major modifications. The software is now freely accessible at www.kabisa.be: KABISA V; setting “Travel clinic”; module “Expert.”
A first single-center retrospective study has evaluated the KABISA TRAVEL in 54 febrile travelers presenting at a Belgian emergency ward and demonstrated that 93% of the cases were correctly diagnosed.12 The present study intended to assess prospectively the diagnostic accuracy of the KABISA TRAVEL in different European settings dealing with travel-related pathology, and to compare it to travel physicians' performances. Secondary objectives were to evaluate the clinical utility of the KABISA TRAVEL software and the specific contribution of the tutor.
Materials and Methods
Study Period and Study Locations
From December 2007 to April 2009, travelers with fever after a stay in the tropics were included prospectively in a multicenter trial conducted in 10 referral travel clinics located in the Netherlands, Italy, Spain, and Belgium (nine tertiary referral hospitals with travel clinics and one outpatient referral travel clinic). Anonymous data from all collaborating centers were centralized and analyzed at the Institute of Tropical Medicine, Antwerp, Belgium.
Study Population and Definitions
We prospectively enrolled patients of any age presenting at one of the study centers with ongoing fever occurring within 3 months after a stay in the tropics. Ongoing fever was defined by an axillary temperature of 38°C or higher, documented by the patient or a physician whenever in the past 3 days before the first consultation. Tropics and subtropics corresponded to all countries at least partly situated between the 35°-northern and 35°-southern latitude, except the United States, European countries, Japan, and Australia.
The study patients were clinically managed by each coinvestigator (all of them being physicians with expertise in travel medicine) according to the usual standard of care in each site/country.
Within 36 hours maximum after initial contact, when the results of the first-line laboratory and radiological investigations were available, the coinvestigator had to record first in a preformatted electronic datasheet a ranking list with the five most probable diagnoses he suspected in his patient (further referred to as “travel physician diagnoses”). This step was mandatory before being allowed to open KABISA TRAVEL software, and the list of suspected diagnoses was automatically saved. Then, the coinvestigator could select out of the provided list of clinical and laboratory findings the ones he considered relevant for the case under investigation, starting always by those classified under “general data” (age category, visited region, incubation period, traveler category, type of travel). Further on, he entered the symptoms, signs, laboratory and imaging findings he collected during the initial workup, also including absent findings and/or negative test results he found relevant to report. KABISA TRAVEL calculated the disease probabilities and provided a ranking list of diagnoses and did so each time a new finding was entered. After feeding the software with all relevant present and absent findings, the coinvestigator had to systematically ask the tutor to intervene. He had then to answer all suggestions from the tutor and had to provide the required additional information in order to allow the system to fully explore the case. When the probability of a diagnosis was considered high enough by the system, and when competing diagnoses were sufficiently excluded, the program concluded that the clinician could rely upon its final ranking list (“KABISA TRAVEL diagnoses”). By closing the software, an automatic report of the consultation was saved for the coinvestigator.
Later on, he had to complete each saved initial report with the final diagnosis and the result of the test(s) that confirmed unequivocally the diagnosis. This final diagnosis obtained by reference methods was considered as “correct (or reference) diagnosis.” Two questions regarding the clinical utility of the software were asked at the end of the clinical report to be sent by the coinvestigators: “Did the expert system influence your choice of complementary exams?” and “Did the expert system help you in finding the correct diagnosis?” The final anonymous reports were then sent by e-mail to the main investigator and contained the following outcome indicators: the top five “travel physician diagnoses,” the top five “KABISA TRAVEL diagnoses,” the final “correct diagnosis” with its test of confirmation, all automatically registered (present or absent) findings entered by the travel physicians, all automatically registered findings required by KABISA TRAVEL, and the answers to the two closing questions on clinical utility. Cases with no definite final diagnosis or incomplete data were excluded from the main analysis. A subanalysis of the congruence between initial travel physicians and KABISA TRAVEL diagnoses with the final (nonconfirmed) diagnoses was also performed in a secondary step. During the study period, any comment or suggestion from the collaborating centers was encouraged, but the software was not modified until the conclusion of the study.
Ethics and Responsibility
All study patients were managed according to the usual standard of care in each collaborating center. Only observational data were collected and anonymously sent to the main investigator. Only the treating physician knew the identity of his patients. This study had no interventional purpose and travel physicians were reminded, when closing the KABISA TRAVEL software, that they had the final responsibility for their patients and that the software was only an aid for diagnosis and not a decider itself.
The study was designed, conducted, and analyzed independently of any sponsoring. The protocol got the ethical clearance from the review boards of the ITMA and of the University Hospital of Antwerp.
Data were entered in an Access database (Microsoft Office 2003). Analysis was performed with Stata version 10 (StataCorp, USA). The chi-square test was used to compare categorical variables. Comparison of proportions was performed with the Pearson chi-square test and the MacNemar's test. Kruskal Wallis test was used to compare median. All tests were two-tailed, and p values <0.05 indicated statistical significance.
Study Population Characteristics
Of 246 registered cases, 205 patients with confirmed diagnosis were included in the study. Cases were excluded because final diagnosis was not confirmed (n = 36), inclusion criteria were not respected (two patients returned from nontropical countries), or clinicians' diagnoses were missing or doubtful (n = 3).
The study population was composed of 190 adults (123 men and 67 women) and 15 children (9 boys and 6 girls); 69% of them had been admitted (Table 1). The mean age was 35 years (range 0.5–73 y). Of the 205 included patients, 98 (48%) were western travelers, 44 (21%) were travelers native of tropical countries who had visited friends and relatives in their country of origin, 39 (19%) were migrants arriving from the tropics, and 24 (12%) were western expatriates. Sub-Saharan Africa was the most frequent place of stay (58%), followed by Southeast Asia (24%), Latin America (11%), and North Africa or the Middle East (6%). One patient stayed in more than one region.
Table 1. Frequency of reference diagnoses per study sites
|Tropical diseases||28 (64)||29 (67)||16 (47)||10 (59)||11 (65)||12 (86)||13 (93)||7 (78)||6 (75)||1 (20)||133 (65)|
|Falciparum malaria||21 (48)||15 (35)||2 (6)||—||8 (47)||—||8 (57)||3 (33)||1 (12)||1 (20)||59 (29)|
|Nonfalciparum malaria*||6 (14)||6 (14)||2 (6)||1 (6)||—||3 (21)||2 (14)||1 (11)||1 (12)||—||22 (11)|
|Dengue||—||6 (14)||3 (9)||5 (29)||—||4 (29)||1 (7)||3 (33)||3 (37)||—||25 (12)|
|Rickettsial infections†||—||—||4 (12)||1 (6)||1 (6)||2 (14)||—||—||—||—||8 (4)|
|Enteric fever‡||—||1 (2)||3 (9)||1 (6)||1 (6)||1 (7)||1 (7)||—||—||—||8 (4)|
|Chikungunya||—||—||1 (3)||—||—||2 (14)||1 (7)||—||—||—||4 (2)|
|Amebic liver abscess||1 (2)||1 (2)||1 (3)||1 (6)||—||—||—||—||—||—||4 (2)|
|Other tropical diseases§||—||—||—||1 (6)||1 (6)||—||—||—||1 (12)||—||3 (1)|
|Cosmopolitan diseases||16 (36)||13 (30)||16 (47)||7 (41)||5 (29)||2 (14)||1 (7)||2 (22)||2 (25)||4 (80)||68 (33)|
|Bacterial enteritis||3 (7)||4 (9)||3 (9)||1 (6)||1 (6)||—||—||1 (11)||—||1 (20)||14 (7)|
|Infectious mononucleosis-like syndrome¶||—||4 (9)||5 (15)||—||—||2 (14)||1 (7)||—||1 (12)||—||13 (6)|
|Respiratory tract infections#||2 (5)||—||4 (12)||1 (6)||2 (12)||—||—||—||1 (12)||1 (20)||11 (5)|
|Skin/soft tissue infections||2 (5)||1 (2)||—||1 (6)||—||—||—||—||—||1 (20)||5 (2)|
|Tuberculosis**||2 (5)||—||2 (6)||—||—||—||—||1 (11)||—||—||5 (2)|
|Hepatitis A||4 (9)||—||—||—||—||—||—||—||—||—||4 (2)|
|Leptospirosis||—||1 (2)||—||3 (18)||—||—||—||—||—||—||4 (2)|
|Genitourinary infections††||1 (2)||—||1 (3)||1 (6)||—||—||—||—||—||—||3 (1)|
|Other infections‡‡||2 (5)||3 (7)||1 (3)||—||2 (12)||—||—||—||—||1 (20)||9 (4)|
|Noninfectious causes§§||—||1 (2)||2 (6)||—||1 (6)||—||—||—||—||—||4 (2)|
|Hospitalized patients||44 (100)||36 (84)||11 (32)||9 (53)||12 (71)||3 (21)||9 (64)||9 (100)||4 (50)||5 (100)||142 (69)|
Etiologies and Outcome
The reference (or “correct”) diagnoses are detailed per collaborating center in Table 1. Most febrile episodes were because of tropical diseases (65%), mainly malaria (40%) and dengue (12%). Among the cosmopolitan infections (33%), bacterial enteritis (7%), infectious mononucleosis-like syndrome (6%), and respiratory tract infections (5%) were the most common etiologies. Four (2%) patients had a noninfectious cause of fever. Of note, 93% (55/59) of the patients with Plasmodium falciparum malaria were hospitalized. Three deaths occurred in total: one patient with Marburg hemorrhagic fever, one with severe malaria, and one with lymphoma.
Diagnostic Accuracy of KABISA TRAVEL Versus Clinicians
Table 2 shows the number of “correct diagnoses” mentioned as first diagnosis (top one ranking) or among the first five diagnoses (top five ranking) by travel physicians and by KABISA TRAVEL. The correct diagnosis was proposed at the first rank by the travel physicians in 70% of the cases, and by KABISA TRAVEL in 72% (p = 0.56), and was cited in the top five ranking of both “competitors” in 88% of the cases (p = 0.85). No significant difference between the performances of travel physicians and of KABISA TRAVEL was observed for any specific diagnosis.
Table 2. Diagnostic accuracy (top one and top five ranking) of travel physicians and of KABISA TRAVEL against reference diagnoses (n = 205)
|Tropical diseases (n = 133)||104 (78)||110 (83)||123 (92)||123 (92)|
|Falciparum malaria (n = 59)||57 (97)||54 (91)||58 (98)||59 (100)|
|Nonfalciparum malaria (n = 22)||12 (55)||17 (77)||19 (86)||20 (91)|
|Dengue (n = 25)||21 (84)||23 (92)||23 (92)||24 (96)|
|Rickettsial infections (n = 8)||5 (62)||5 (62)||6 (75)||6 (75)|
|Enteric fever (n = 8)||5 (62)||6 (75)||8 (100)||7 (87)|
|Chikungunya (n = 4)||—||—||4 (100)||2 (50)|
|Amebic liver abscess (n = 4)||3 (75)||3 (75)||3 (75)||3 (75)|
|Other tropical diseases (n = 3)||1 (33)||2 (67)||2 (67)||2 (67)|
|Cosmopolitan diseases (n = 68)||38 (56)||36 (53)||56 (82)||56 (82)|
|Bacterial enteritis (n = 14)||10 (71)||10 (71)||14 (100)||14 (100)|
|Infectious mononucleosis-like syndrome (n = 13)||3 (23)||1 (8)||9 (69)||8 (61)|
|Respiratory tract infections (n = 11)||4 (36)||5 (45)||8 (73)||11 (100)|
|Skin/soft tissue infections (n = 5)||4 (80)||5 (100)||5 (100)||5 (100)|
|Tuberculosis (n = 5)||5 (100)||3 (60)||5 (100)||3 (60)|
|Hepatitis A (n = 4)||4 (100)||4 (100)||4 (100)||4 (100)|
|Leptospirosis (n = 4)||3 (75)||2 (50)||3 (75)||3 (75)|
|Genitourinary infections (n = 3)||2 (67)||3 (100)||3 (100)||3 (100)|
|Other infections (n = 9)||3 (33)||3 (33)||5 (56)||5 (56)|
|Noninfectious causes (n = 4)||1 (25)||1 (25)||2 (50)||1 (25)|
|Total (n = 205)||143 (70)||147 (72)||181 (88)||180 (88)|
Of note, tropical conditions were more frequently correctly diagnosed both by travel physicians and by KABISA TRAVEL than the cosmopolitan diseases as first diagnosis (78% vs 56%, p = 0.001 and 83% vs 53%, p < 0.001, respectively) and within the top five ranking (92% vs 82%, p = 0.03, for both comparisons). These differences disappeared when the malaria cases were removed from analysis, except for KABISA TRAVEL for the first diagnosis (75% vs 53%, p = 0.013).
Eleven diagnoses were not included in the five first proposals of travel physicians neither in those of KABISA TRAVEL; 13 were included by KABISA TRAVEL but not by travel physicians, and 14 were included by travel physicians but not by KABISA TRAVEL. Reasons for missed diagnoses by KABISA (14) were absence of findings (2) or diseases (1) in the database, not updated incidence (3), wrong computation (2), and atypical clinical presentation (9). When both clinicians and KABISA did not include the correct diagnosis in the first five (11), for KABISA atypical presentation was the only and constant cause. For missed diagnoses by clinicians alone no data are available in this study. Details on missed diagnoses are provided in Table 3.
Table 3. Description of diagnoses missed by KABISA TRAVEL or travel physicians, or both
|Tropical diseases||6 (chikungunya [n=2]; infection with Salmonella paratyphi A; P malariae malaria; Marburg hemorrhagic fever; tick-borne rickettsiosis)||6 (tick-borne rickettsiosis; P falciparum malaria, P ovale malaria; P malariae malaria; dengue, acute schistosomiasis)||4 (infection with R typhi, amoebic liver abscess; P malariae malaria, dengue)|
|Cosmopolitan diseases||7 (acute cytomegalovirus infection [n=2], acute HIV infection [n = 2], miliary tuberculosis, tuberculous lymphadenitis, herpetic stomatitis)||7 (acute cytomegalovirus infection [n = 3]; pharyngitis/tonsillitis [n = 2]; Q fever; chronic bronchitis)||5 (leptospirosis; septicemia; acute toxoplasmosis; rubella; secondary syphilis)|
|Noninfectious diseases||1 (sarcoidosis)|| ||2 (systemic lupus erythematosus, lymphoma)|
Finally, when analyzing the subset of 36 patients with final nonconfirmed diagnoses, it is interesting to note that the initial diagnoses proposed by travel physicians and by KABISA TRAVEL were rather similar and often close to this final clinical suspicion (Table 4).
Table 4. Comparison of final nonconfirmed diagnoses with the initial “travel physician” and “KABISA TRAVEL” diagnoses in the 36 patients excluded
|Typhoid fever||4||4||3||2 (typhoid (2), Campylobacter enteritis (2))||1 (typhoid (1), P falciparum (3))|
|Gastroenteritis (viral? Traveler diarrhea? Bacterial? Parasitic?) all causes:||3||3||3||3 (C enteritis, giardiasis, shigellosis)||3 (Salmonella enteritis, bacterial enteritis, C enteritis)|
|Viral infection (influenza like)||4||3||3||1 (influenza, acute CMV, P ovale, atypical pneumonia)||1 (influenza, acute CMV, P vivax, atypical pneumonia)|
|Viral upper respiratory tract infection||3||3||2||2 (influenza (2), giardiasis)||1 (acute sinusitis, P falciparum, pulmonary embolism)|
|Pharyngitis: viral? Cocksackie?||1||1||1||0 (acute CMV)||0 (acute CMV)|
|Rickettsial infection||1||1||1||0 (dengue)||1|
|Leptospirosis||1||1||0||1 (leptospirosis)||0 (P falciparum)|
|Bacterial adenitis||1||0||1||0 (acute CMV)||1|
|Pneumonia, Steven Johnson syndrome||1||0||1 (pneumonia)||0 (rickettsiosis)||0 (rickettsiosis)|
|SSTI, pneumonia||1||0||1 (for both diagnoses)||0 (rickettsiosis)||0 (P falciparum)|
|Septicemia||1||0||0||0 (P falciparum)||0 (P falciparum)|
|Acute liver failure: unknown etiology||1||0 (viral hepatitis both)||0 (hepatitis A)||0 (hepatitis A)|
|Pneumonia resistant to usual CAP treatment; suspected melioidosis||1||Melioidosis not proposed|
|Severe pneumopathy: no response to AB, response to corticoids||1||Infectious etiology not confirmed; sarcoidosis?|
|Unknown etiology||12|| || || || |
|Total||36|| || || || |
Clinical Utility of KABISA TRAVEL
The answers to the survey about the clinical utility of KABISA TRAVEL are reported in Table 5. Data were available for 198 patients. Travel physicians reported that they had been influenced by KABISA TRAVEL for the choice of further investigations in 16% of the cases, but much more frequently when they did not initially find the correct diagnosis (48% vs 12%, p < 0.001). They reported to have been helped for finding the correct diagnosis in 24% of the cases, and also more often when the correct diagnosis had not been mentioned in the initial list (48% vs 21%; p = 0.005).
Table 5. Clinical utility of KABISA TRAVEL (“Did the expert system influence your choice of complementary exams?”; “Did the expert system help you in finding the correct diagnosis?”)
|Correct diagnosis initially found by travel physicians (n = 175)||21 (12%)||154 (88%)||37 (21%)||138 (79%)|
|Correct diagnosis not initially found by travel physicians (n = 23)||11 (48%)||12 (52%)||11 (48%)||12 (52%)|
|Total diagnoses with complete available answers (n = 198)||32 (16%)||166 (84%)||48 (24%)||150 (76%)|
Contribution of the KABISA TRAVEL Tutorship
A median of 10 findings was spontaneously entered by the coinvestigators for all cases under study. After the travel physicians had entered all data they found relevant, KABISA TRAVEL still requested additional information in 81.5% of the cases. A median of three additional findings was asked by the system before considering the cases as fully explored.
When the correct diagnosis was initially found, the median number of suggestions by KABISA TRAVEL was 2 (vs 5 when it was not immediately considered, p < 0.05).
This prospective multicenter study showed that KABISA TRAVEL performed as well as travel physicians in diagnosing febrile illnesses in returning travelers. Its diagnostic accuracy reached almost 90% of the challenged cases when considering the top five ranking list. In addition, KABISA TRAVEL was perceived as helpful in suggesting further investigations and final diagnoses in a sizeable proportion of the cases, in particular when the diagnosis was not immediately clear. Also, in the majority of the cases, the tutor asked for additional information before providing a final ranking with sufficient probability.
The high number of malaria cases in our study might be explained by the high proportion of hospitalized patients. Even if no single clinical or biological feature has good sensitivity and specificity to predict malaria in febrile patients,13,14 malaria was rarely missed both by clinicians and KABISA TRAVEL in our study.
Comparison of the most frequent diagnosis with other published studies is biased by the definition of diagnoses and by the selection criteria: O’Brien studied only hospitalized patients, Ansart only outpatients, Bottieaux and Wilson mixed populations (27 and 26% hospitalized, respectively).3,9,13,15 Malaria was the most common diagnosis (resp. 27, 19.9, 27, and 21%), followed by respiratory track infection (24, 7.4, 10.5, and 14%), and gastroenteritis (14, 18.4, 7.1, and 15%). The results of our study are not really different for the latter two, except for the high prevalence of malaria which is uncommon, especially the high proportion of nonfalciparum malaria. The high proportion of dengue might be because of the worldwide increase over the last decade.16,17
Most missed diagnoses were because of nonspecific case presentation. KABISA was less well performing than clinicians. Although the numbers are equal, the balance of suggested serious and treatable diseases seems rather favorable for clinicians. Also here, we should state that KABISA only gives hints, during the initial workup, and is not a final decider. Every session ends with this warning.
Several limitations must be mentioned. First, no public call was made for this study; only the institution where the system has been developed and other centers with formerly existing links took part in this investigation. Second, we cannot be sure that all cases were prospectively entered in the KABISA TRAVEL within 36 hours after the first clinical contact. As the electronic report files were not locked, also some modifications might have been made a posteriori by the physicians before sending them. Third, it is also possible that not all eligible cases who presented in each center during the study period have been actually included, and it is possible as well that some selection of cases had occurred, favoring, for example, unusual cases or typical tropical cases. To which extent it has impacted on the performances of both competitors is however difficult to quantify. Fourth, because of the sample size, the effectiveness of KABISA in diagnosing less common tropical or cosmopolitan conditions could not be fully assessed. Fifth, one of the expert clinicians was from the institution where the program was made; this might have introduced some bias. On the other hand, this clinician was never involved in the development of KABISA. And finally, the questions on clinical utility were rather subjective.
Comparison With Other CDSS
GIDEON provides a ranking list of most probable diagnoses, after clinicians have entered epidemiological and clinical data. Its major strength is its comprehensive, flexible, and constantly updated database of more than 300 infectious diseases, also nontropical. However, the system does not interact with the user, except through a “why not” function explaining why a given diagnosis has not been considered (absence of a relevant finding or presence of an irrelevant one). The diagnostic workup does not go beyond this stage, and important diagnoses may sometimes be missed because a nonrelated finding has been entered (even if nonspecific) or because a good predictor was absent.18–20
Fever Travel proposes a dichotomous or branched approach based on pertinent questions extracted from a comprehensive literature review.21 It helps clinicians in focusing on the most relevant findings to look for when evaluating a patient with fever after travel and suggests further testing, reference, or hospitalization and even presumptive treatment. A prospective multi centric evaluation of Fever Travel software is under way.
Like GIDEON, KABISA TRAVEL gives a ranking of hypotheses based on a modified Bayesian logic. Like Fever Travel, it is free of charge. It offers an additional function (“tutor”) asking actively the user to look for findings which have not been entered yet and which are strong confirmers or excluders of diagnoses still in competition. Through this “corrective tutorship,” the final result is less influenced by the relevance (or irrelevance) of the findings entered by the clinician (which is problematic in GIDEON). The quality of “data entry” is a frequent weakness of expert systems because it depends highly on the expertise and sophistication of the user. A further difference with other CDSS is the inclusion of the threshold concept: dangerous and treatable diseases (“not to miss diagnoses”) are explored first and until all relevant findings are exhausted. Finally the most robust strength of KABISA TRAVEL resides in the use of recent and evidence-based data extracted from large and multicentric prospective studies.1,3,9
Whether this system improves patient outcome remains to be explored, but such an exploration is very difficult to conduct for any CDSS.4
It is worth mentioning that complete discrepancies between travel physicians and KABISA TRAVEL occurred in only 15% of all cases. Most diagnoses were found by both travel physicians and the expert system and this is maybe due to the rather high proportion of malaria cases, since laboratory diagnosis must be straightforward in travel clinics, in the absence of good clinical predictors. Diagnostic congruence between both “competitors” was fair also when malaria cases were removed or for cosmopolitan infections, and it was even so for diagnoses with no final confirmation. Finally about 5% of the cases were not found by either “competitor,” and corresponded to atypical presentation, or complex or rare diseases, where the diagnosis could only be found with tests that are normally not available within the first 36 hours. There is however still room for improvement, by analyzing the reasons for having missed diagnoses. Absence of diagnoses or findings in the database, nonupdated incidences, and erroneous computation were errors identified and corrected after the study.
The good performance of KABISA TRAVEL compared to clinicians with expertise in travel medicine encourages promoting its use not only by travel physicians and infectious diseases specialists but also by first-line practitioners (family or emergency physicians). However, a prospective assessment in primary care settings should be first conducted, as first-line physicians are much less exposed to travel-related diseases, possibly causing errors of manipulation and an effect on pre-test probability. This might enhance the importance of the contribution of the “tutorship.” Anyhow, by its interactive and dynamic approach, we are rather convinced that KABISA TRAVEL may provide diagnostic guidance for primary care practitioners and may have an additional educative impact regarding tropical and travel medicine.
KABISA TRAVEL performed as accurately as experienced travel physicians in diagnosing febrile illnesses occurring after a stay in the tropics and was perceived as rather helpful when the etiology was not immediately obvious to them. Further study is needed to evaluate its beneficial impact on diagnostic performances of physicians not familiar with travel medicine.
Declaration of Interests
The authors state they have no conflicts of interest to declare.