Framework synthesis to inform the ideation and design of a paper‐based health information system (PHISICC)

Abstract Background Health information systems (HIS) are meant to support decision‐making at all levels of the system, including frontline health workers. In field studies in Côte d’Ivoire, Mozambique and Nigeria, we observed health workers' interactions with the HIS and identified twelve decision‐making components of HIS. The objective of this framework synthesis is to portray these components in HIS research, in order to inform the ideation of a paper‐based HIS intervention (PHISICC). Methods We searched studies in the Cochrane Central Register of Controlled Trials (CENTRAL), Embase, Epistemonikos, Medline, in‐Process on the Ovid platform, OpenGrey, PDQ Evidence (“pretty darnd quick” Evidence), the World Health Organization (WHO) Global Health Library and included studies focussing on HIS interventions, data quality, information support tools and data use for decision‐making in the context of the governmental health care sector. We assessed the methodological quality of studies using the Critical Appraisal Skills Programme tool. We synthesised the findings based on the decision‐making components of HIS and thematic areas. Results The search identified 6784 studies; 50 were included. Most of the 50 studies had quality concerns. All studies included at least one of the decision‐making components: the most prominent were the technical aspects of ‘recording’ and ‘reporting’. Data use for decision‐making was much less represented. Conclusion HIS research focuses on the more technical aspects of HIS. Further research on HIS, given the strong push towards HIS digitalisation, should consider putting at the centre the human experience of decision‐making and data use, in order to make HIS relevant for quality of care.


| INTRODUCTION
The health and wellbeing of individuals and populations is directly affected by decisions made by policy makers and planners, managers, health care providers and by the population itself. 1 Health Information Systems (HIS) are meant to inform decision-making across all levels of the healthcare system. 1 Information within health services is typically produced through Routine Health Information Systems (RHIS) or Health Management Information Systems (HMIS).
Most of the HIS definitions and approaches are circumscribed to RHIS or HMIS and have a technical focus on the mechanisms and tools by which data and information are produced, supported and transformed. 2,3 However, decisions are also made based on other information sources, such as research, evaluation reports or even less formal sources, such as colloquial, personal or experiential evidence. [4][5][6] In the frame of a multi-country, transdisciplinary research project (Paper-based Health Information System in Comprehensive Care, PHISICC 7,8 ) we recently carried out an effectiveness systematic review on the effects of ZUSKE Et al. 1954 Results: The search identified 6784 studies; 50 were included. Most of the 50 studies had quality concerns. All studies included at least one of the decision-making components: the most prominent were the technical aspects of 'recording' and 'reporting'. Data use for decision-making was much less represented.
Conclusion: HIS research focuses on the more technical aspects of HIS. Further research on HIS, given the strong push towards HIS digitalisation, should consider putting at the centre the human experience of decision-making and data use, in order to make HIS relevant for quality of care.

K E Y W O R D S
decision-making, framework, health information system, primary health care, systematic review • Informed by existing frameworks and by fieldwork in three African countries, we identified twelve decision-making components of HIS.
• In this framework synthesis, we found that the most common decision-making components in the HIS literature were those related to the technical aspects of data 'recording' and 'reporting'; data use for decision-making was much less represented.
• For HIS instruments to facilitate improved health care delivery, clinical and public health decision-making components of HIS should be put at the centre for HIS research, ideation and implementation.
interventions to improve HIS. This evidence was complemented with the characterisation of the HIS in the three countries where PHISICC took place (Côte d'Ivoire, Mozambique and Nigeria), which together with the effectiveness systematic review and the framework synthesis we present here informed the ideation and design of the intervention. The PHISICC intervention consisted of a suite of paper-based tools for antenatal care, delivery, postnatal care, vaccination, sick child, general consultation, Human Immunodeficiency Virus (HIV) and tuberculosis. The tools shared a standard visual language with decision-making hints, as a clinical decision support tool. The intervention was tested in three randomised controlled trials.
The rationale for a framework synthesis was twofold; one, an effectiveness review assessing the effects of interventions targeting HIS on the quality and use of data and on health-related outcomes in primary health care (PHC) in low and middle-income countries. 9,10 This systematic review was not primarily conceived to understand how HIS interventions work or how users interact with them, and had limited findings on those issues. Two, preliminary research carried out in the three PHISICC countries, using stakeholders' consultations as well as a mix of public health and Human Centred Design approaches, emphasised the need to refocus HIS research, from the more technical issues related to data management to the actual use of data for decision-making by frontline health workers. The field work allowed us to identify a series of components of frontline health workers' decision-making to guide our thinking (see Table 1). The list was produced following an inductive approach based on the nature of decision-making, decision-making domains and levels of decision-making identified in the field.
The objectives of this framework synthesis were (i) to map the frontline health workers' decision-making components in the research literature; (ii) to explore the links between information and its use; and (iii) to describe barriers and opportunities for information use for decision-making within HIS. This framework synthesis contributed to inform the ideation of an innovative paper-based HIS (PHISICC).

| MATERIALS AND METHODS
We carried out a systematic review of the literature to produce a framework synthesis, 11 and reported it following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting items (see Supplementary file 1). We implemented a search strategy to the following sources in April 2016: CENTRAL (The Cochrane Central Register of Controlled Trials), Embase, Epistemonikos, Medline, in-Process on the Ovid platform, OpenGrey, PDQ Evidence ("pretty darn quick" Evidence), the World Health Organization (WHO) Global Health Library and a reference list of the chair of our technical advisory group. The search included terms related to HIS and decision-making and was not adapted or limited to the specific decision-making components in Table 1, because we precisely aimed at identifying to which extent we could map these components 'from the field' in the HIS literature. The complete search strategy for the observational studies, including qualitative research and case studies, can be found in the Supplementary file 2.
The studies' inclusion criteria were: observational, qualitative and case studies, addressing decision-making capacity by PHC frontline health workers (i.e. nurses, clinicians, midwives as well as community health workers) or managers at district or at higher levels of the system. Studies related to patients' decision-making were included if an information tool, device or subsystem was used by the health workers as well. We included studies containing information on perceptions, experiences and lessons learnt relevant for decision-making and in relation to the supportive function of HIS in the PHC system. In cases where the main focus of a study was not on decision-making components or the supportive components of data, tools or applications, the study was still included, if: (a) the characteristics of the problems described in the study were related to decision-making or the use of data, tools or applications; (b) the findings, discussion or conclusions of the study were related to relevant components, problems or solutions for decision-making processes or for the use of data, tools or applications. Studies had to refer to HIS interventions, data quality or information tools and provide any evidence in the results or discussion on barriers or facilitators of decision-making. We considered studies in the context of the governmental, public PHC system or where the health care context was mixed or not clearly identifiable. Studies conducted in private health care facilities or hospitals were excluded. We included studies carried out in any country and in any language.
First, we screened titles and abstracts for relevance. Studies that could not be clearly ruled out as irrelevant to our aims were kept and the full texts were retrieved to apply the inclusion criteria. Articles were screened only once by any of the reviewers, being inclusive in case of doubt. At inclusion/exclusion stage, some articles were double scrutinised by a second author (either Xavier Bosch-Capblanch (XBC) or Christian Auer (CAU)) either because of doubts by the first reviewer Meike-Kathrin Zuske (MZU) or arbitrarily as data were shared among the reviewers team. At least half of the articles would have been inspected by at least two reviewers. Data from the included studies were entered into a data extraction form specifically designed for this review to record the characteristics of studies, the mapping of decisions-making components and statements related to the main findings. The form was also used for the quality assessment. The methodological quality of the studies was assessed by a single reviewer supervised by the corresponding author, using the Critical Appraisal Skills Programme (CASP) tool. 12 Since different study designs were included, we selected the CASP criteria in order to have a common set of items across all observational study designs.
Eleven questions were used and a quality score for each study was calculated by adding up positive answers. Studies were classified as having no concern (11 items passed), minor concerns (9 or 10 items), moderate-minor concerns (7 or 8 items), moderate concerns (5-6 items), moderate-serious concerns (2-4 items), or serious concerns (0 or 1 items passed).
For the first objective (mapping of decision-making components), we considered all included studies. When a component was found within a study, this was marked as a 'Yes' in the correspondent data sheet; if a component was clearly not mentioned in a study, then we marked it as a 'No'; if unclear, we recorded it as such. We used frequency analysis to populate the decision-making components, by summing the positive hits for each component.
For the objectives two and three, we selected a subset of included studies with higher quality scores, in order to avoid unnecessary bias which would affect the synthesis of findings. After an initial coding of extracted findings in the form of statements, these were charted thematically 11 and grouped into barriers, opportunities and strategies for the use of information.

| RESULTS
The search strategy produced 6784 records. After removing 271 duplicates, the remaining 6513 records were

| Characteristics of included studies
The characteristics of included studies are summarised in Table 2 and Supplementary file 4. The publication years of the included 50 studies ranged from 1994 to 2015 (median 2009). Study designs were: 31 cross-sectional; eight mixed-methods; eight qualitative; two case control and one cohort study. Of the included studies, 29 were conducted in PHC facilities, seven at several health care levels; and two at the district level. In 12 studies, the setting was not clearly identifiable.
The majority of the studies were located in middle-income countries ( Sixteen of the 50 included studies had mother and child health care as the main health care area. The setting was general practice without a specific focus in 10 studies. Three studies focussed on infectious diseases in general; two studies on cancer care, two on family health, two on hypertension and two on diabetes. Health financing, HIV/ Aids, Malaria, tuberculosis and reproductive health were the main topics in one study each. A combination of different health areas other than maternal and child health was found in two studies. In three studies, the health subject was not clearly identifiable. The HIS was exclusively addressed in another three studies.
Thirty-six studies focused on an information tool; seven studies tackled data quality issues; and another seven focussed on a variety of health information issues.
We report the quality of studies by criteria ( Table 3) and also the distribution of overall quality scores ( Table 4).
The larger group of studies had minor to moderate quality concerns, for example, not reporting how findings were integrated into the existing body of evidence or confounders not taken into account (17 studies). Seven studies had moderate to serious quality issues. The main reasons were lack of adequate description of the sampling or recruitment strategy (7), a lack of clearly identifiable outcomes (6), not taking confounders into account (7), a weak description of the analysis performed (5) and no integration of findings into the context of other research (7). The full results of the quality assessment for each of the included studies can be found in Supplementary file 5.

| Mapping decision-making components in the literature
Twenty-nine studies focussed on decision-making at the PHC facilities; two examined decision-making at the district level of the PHC system. Seven studies focussed at several health care levels. In 12 studies, the setting was not clearly identifiable.
The 35 studies addressing decision-making at the PHC facilities examined the use of electronic health record systems (7), decision algorithms for disease management (6), HBR (5), factors related to the use of electronic health technology in general (4), the partograph (3), the use of paper-based medical records (5), SMS reminders for patient follow-up (2), mobile cameras for screening and treatment (2) and the use of a communication guide for nurses (1). The two studies conducted at district level focussed on decision-making in the frame of an assessment of WHO district HIS (DHIS) and in an evaluation of the district health management information system.
Combinations of different decision levels were found in 13 studies. In one study, the use of a primary care information system for planning and evaluation of actions at health facility and district level was examined. Eight studies compared the registry entries from the health facilities with the DHIS in the frame of data quality assessments.

Type of study
Cohort 1

Country income level classification
High income country HIC 7

Upper-middle income country UMIC 24
Lower-middle income country LMIC 9 Low income country LIC 10

Participants
Health care providers 31 Manager 2

Health care area
Maternal and child health 16 General practice 10 Infectious diseases 3

Health information 3
Cancer care or prevention 2

T A B L E 2 Characteristics of included studies
Single studies focussed on the following issues: the applicability of a basic information system for the planning of work of nurses in PHC facilities and for planning of family health activities at the higher levels; influencing factors on the intention of PHC physicians to collect data exhaustively in medical registries; challenges on health facility and district levels of implementing an Integrated Disease Surveillance and Response strategy using the current HMIS; reasons for under-reporting of notifiable communicable diseases; and factors related to the PHC facilities, higher levels as well as policy and planning.
Different types of decision-making were identified. Clinical decisions were most often addressed (19 studies).
They were related to diagnosis and treatment (4), treatment only (4) and management of deliveries (5). In three stud- Four studies focussed on managerial decisions exclusively: two addressed evaluation and planning of family health activities; one considered the implementation of HIS policy and objectives, technical feasibility, financial and political viability, administrative operability, internal quality of information, external resources, managerial support and systems impact; and one mentioned managerial decisions without any clear specification.
One study reported on public health decisions only, defined as the identification of strategies and interventions in the context of integrated disease surveillance.
In 13 studies, a combination of different types of decisions was mentioned and in another 13 studies, the decision type was not clearly identifiable. By focussing on the content of the different types of decisions, we found that more tasks were related to managerial decisions than clinical and public health decisions.

T A B L E 4 Frequency distribution of quality assessment scores
Regarding the tools supporting decision-making, 21 studies examined paper-based decision support tools. Six studies examined digital and five studies explicitly named computers as decision supporting tools. A combination of tools was found in 13 studies. In five studies, the support tool was not clearly identifiable.
When mapping the decision-making components in the literature, we found that the single component most frequently found was 'considering the outcomes', which refers to the recording practices documenting the results of clinical or public health events. Transforming information and reporting were found on 21 and 20 occasions, respectively and are activities that tend to go together (i.e. the fact of reporting often entails a previous data transformation process). Twenty studies considered information to assess a situation or problem. The component 'best course of action' was addressed in 16 studies and 'knowledge' in 14. Information used for follow-ups was retrieved in 11 studies. Much less frequently found components included those that may entail mechanisms to improve the system (e.g. auditing technical guidance) and communication. The least frequently found related to the aids to health workers or decision-makers to better understand and cope with the HIS itself ( Table 5).
Looking at the overall categories, (A) information for immediate decision-making components were found 71 times, (B) information to document health care events 41, (C) to maintain quality of care 27, and (D) related to use of the information system 7 times.

| Information use for decision-making
The 14 studies with minor quality concerns were further examined to understand how information was used in decision-making, as well as barriers and opportunities for the use of information. We extracted in total 429 statements.

| Use of information: Barriers and opportunities
Barriers and challenges for the use of information were linked to the health system, the information tools used, to health managers and to health workers. Box 1 summarises the key messages.
Major barriers from the health system for the use of information in PHC were identified in terms of weak polit-    It has to be noted that both studies addressing fears of health workers were related to paper-based information tools.
Another health worker-related challenge mentioned was the individual variation in the ease of use of informa- Box 2 lists key opportunities for the use of information for decision-making as found in the included studies. One opportunity related to electronic information systems was their potential to share information over differ-

Box 2: "Opportunities for the use of information for decision-making"
Health system Well-implemented health programmes (e.g. vaccination) create positive environment for the use of information

| DISCUSSION
We conducted a framework synthesis of the research literature in order to inform the development of an innovative paper-based HIS (PHISICC), having as a reference a list of twelve decision-making components resulting from the characterisation of HIS in three African countries. Across all included studies, we could not retrieve any additional decision-making component beyond those that we had already identified ( Table 1), which suggests that our framework fits the current research literature, in terms of HIS use for frontline health workers' decision-making.
Indeed, the bulk of included studies addresses or mentions in one way or another most of the decision-making components and most of them are limited to specific health care areas (as opposed to the whole system). However, when we looked in-depth in a selection of studies, these tended to focus on the technicalities of information systems (e.g. how digital systems run) rather than on their use for decision making (e.g. how digital systems might lead to better or more timely diagnoses).
The technical focus of most findings is consistent with requests to health workers to periodically synthesise and 'report' their activities and resource consumption to higher levels of the health system, where information is aggregated and forwarded to national 13,14 and international levels. 15 Data quality is a recurrent issue in international health and development: twenty years ago it was already defined in terms of relevance, completeness, timeliness and accuracy 16 ; 10 years later, the criteria were similar 2 including, for example, consistency and validity 17 ; even recent evidence suggests that the focus on data quality has hardly changed. 18 It is striking that the actual use of data and how data and information relates to decision-making is much less studied.
These seems to be consistent with other research evidence. For example, a recent scoping review 19 focussed on intervention studies that aimed to improve data quality and use within RHIS in low-and middle-income countries. The review found that a combination of interventions, addressing both behavioural and technical factors, improved data quality and use. Another systematic review 20 looked at challenges associated with the use of data from RHIS in lowand middle-income countries. The review found that the challenges most frequently addressed (or reported) were of technical nature.
In times when recommendations on digital health are increasingly widespread, 21 donors, partners and countries should take note of the numerous design and implementation challenges that digitalisation may bring, as illustrated in some of our examples in this review and elsewhere. 22 Paper-based systems, on the other hand, also have caveats, such as the error-proneness of writing and transcribing information. Some challenges are strikingly basic (e.g. the need for planning or training), while others reach beyond the domain of health (e.g. equipment and infrastructures).
Decision-making is indeed a complex process, where data and evidence is just one of the components. Barriers and facilitators to the use of evidence have been described in the literature for years across several settings. 23,24 Most studies in this review point at technical or managerial challenges but some studies also reported on the need to address "human factors" in the interaction with HIS; for example, building trustful and respectful relations with patients, taking into account psychosocial factors or the needs of health care provision. These 'human factors' reach further and are consistent with findings in other related reviews. 25 Even though data use for decision-making is widely mentioned across the literature, the in-depth analysis could not reveal any substantial insight or example on how HIS should be tailored to improve decision-making, either in clinical care (e.g. quality of care), public health (e.g. coverage of preventive measures) or health care management (e.g. resource forecasts). The interplay between data and decision-making is hardly studied; and where it is, it offers disappointing findings. 26

| Limitations of this framework synthesis
Despite having followed standard systematic review methods, our framework synthesis has several limitations, some of them deriving from the diverse level of detail in the included studies. We may have left out some relevant studies, although we have experienced a certain degree of saturation and repetition in the terms and concepts reported and completeness in relation to the components that emerged from the preceding field work. Furthermore, included studies are not necessarily rooted into a particular HIS framework which may lead to inconsistencies in the terminology, definitions or in the understanding of HIS in different settings. We acknowledge that our search strategy may become out of date; however, this framework synthesis was done in order to inform the ideation of a paper-based HIS intervention (PHISICC), which it did; and on the other hand, we believe that the studies published later on would hardly change our findings as suggested by the evidence found in more recent reviews. 19,20 Lastly, we did not register the protocol of this review. Open access funding provided by Universitat Basel.

| CONCLUSIONS
Our framework synthesis of available research revealed a strong focus on the technicalities of recording and reporting of information to higher levels of the health system, rather than on decision-making.
The twelve decision-making components developed during field work in Côte d'Ivoire, Mozambique and Nigeria proved to be comprehensive and serve as a guide to design innovative paper-based HIS (PHISICC), which are more user-friendly, facilitating decision-making.