Self‐reported medication intake vs information from other data sources such as pharmacy records or medical records: Identification and description of existing publications, and comparison of agreement results for publications focusing on patients with cancer ‐ a systematic review

To identify and describe publications addressing the agreement between self‐reported medication and other data sources among adults and, in a subgroup of studies dealing with cancer patients, seek to identify parameters which are associated with agreement.

which evaluate factors that might affect agreement between self-report and other data sources are lacking.

K E Y W O R D S
drug prescriptions, medical records, pharmaceutical preparations, pharmacies, pharmacoepidemiology, self-report, validation studies 1 | BACKGROUND Data on medication use is required in several research areas to enable researchers to analyze use patterns and medication-related costs. Relevant information can be obtained from healthcare provider records or health insurance claims, for instance. 1 Each source has its specific limitations, with claims only including reimbursed medications 2 and provider records only covering documentation for one healthcare provider, thus requiring several institutions to be covered to gain complete documentation of one patient's treatment. 3 Furthermore, in some cases patients' consent is required to gain access to data. Finally, claims and medical records reflect the receipt or filling of prescriptions, 4,5 whereas patients' self-reports are likely the most accurate reflection of actual intake of medication. 6 Researchers therefore sometimes have to rely on selfreported medication use. Obtaining this information via interview or questionnaire is resource-intensive and at risk of interviewer or recall bias. 7 As no data source provides complete information, there is no general reference standard. 1 Depending on the research question, one source might be more appropriate than another. Scientific knowledge about agreement of self-reported medication intake and information from other sources is needed to help interpret results from a single source. To date, numerous studies have examined agreement between two or more sources, but only two systematic reviews published over 20 years ago were found. Harlow and Linet 8 compared questionnaire data with medical records, while Evans and Crawford 9 compared any self-report (eg, questionnaires, interviews) with any other data source. Both reviews only analyzed medication use as part of healthcare utilization. Given these limitations and the obsolete systematic data, we decided to conduct a systematic review.
The objectives were: (a) to identify publications comparing self-reported medication use and medication information from other data sources for adult individuals and to describe their key characteristics; (b) seek to identify parameters which might be associated with agreement in a subgroup of studies dealing with cancer patients.
PsycINFO via OVID, and the Cochrane Methodology Register / Cochrane Central Register of Controlled Trials via Wiley. Databasespecific searches combined the terms "medication," "self-report," "other data sources," and "agreement." The complete search strategy for each database is provided in File S1. Reference lists were checked and relevant articles known by the authors included to identify further potentially eligible records.

| Selection of publications meeting the eligibility criteria
Two researchers independently checked title, abstract and full text for relevance according to the eligibility criteria and subdivided the records into "included" or "excluded". Every disagreement was discussed for consensus. The inter-rater reliability was calculated using Cohen's kappa.

| Critical appraisal and risk of bias within publications
We assessed the quality of all included publications using the Scottish Intercollegiate Guidelines Network (SIGN) methodology checklist for diagnostic studies. 11 We were therefore able to use one tool irrespective of different study designs. To apply the checklist, we had to define which data source should be used as the index test and which as reference standard. The reference standard defined by the authors was used to do so. Where authors did not indicate a reference standard, the non-self-report data source was taken as reference.
The checklist appraises the methodological quality to minimize bias (high, acceptable, low, unacceptable) and the direct applicability/external validity (yes, no) using 13 items in four domains: (a) patient selection, (b) index test, (c) reference standard, and (d) flow and timing. As described in detail in File S2, three items were excluded as not applicable to our review. Accordingly, the quality of the publications was rated high if 9 to 10 criteria were met, acceptable if 6 to 8 were met, low if 1 to 5 were met and unacceptable if no criteria were met. Direct applicability was given if all three relevant criteria were fulfilled. Two researchers completed the checklist for half of the articles each, with an overlap of every fifth article. Differences were discussed and resolved. Results are presented in Table 1.

| Data extraction
Key characteristics are provided for all included publications. As described below, we also focused on a subgroup of studies dealing with cancer patients and three medication groups, namely estrogen, hormone therapy (HT) and chemotherapy. We analyzed the degree of agreement in these subgroup studies and tried to identify factors which are associated with agreement such as study design aspects and patient characteristics.
We extracted key characteristics, namely sample size and description, analyzed medications, comparison data source, and outcome, for all included publications. Furthermore, we extracted the authors' main findings concerning agreement. Detailed study characteristics such as data collection period, cancer type, study design, self-report modality, comparison data source, analyzed patient characteristics, analyzed medication, statistic methods, and type of outcome/s were extracted for publications focused on patients with cancer. We focused on the underlying original study design, for example case-control study or cohort study, when extracting data regarding study design. The study design of the validation study, which in most cases is cross-sectional, was not considered here.
All information was extracted by one researcher using piloted forms and was subsequently checked by another researcher to reduce errors or bias. We did not contact any of the corresponding authors for further information. No meta-analysis was performed due to the high methodological heterogeneity of the included studies.
Given the large number of inclusions and their high heterogeneity, it was necessary to set a narrower focus and go into greater detail. To do so, we looked for the largest possible set of homogeneous and comparable studies. We were unable to identify homogeneous studies when starting at the level of the analyzed drugs.
We therefore decided to look for an appropriate population as a starting point and identified patients with cancer. Furthermore, we considered the three most frequent medication groups within these studies, namely the hormone estrogen as specific active agent, and hormone therapy (HT) and chemotherapy as two superordinate medication groups, where HT comprises estrogen. Additionally, we focused on the proportion correct and kappa values as an agreement measurement, as most researchers calculated those estimates.
In order to identify parameters which might affect or be associated with agreement, we searched among these studies which analyzed agreement with estrogen, HT and chemotherapy in cancer patients for any which had performed regression analyses or stratified analyses. We expected that those analyses would focus particularly on patient characteristics as age, sex, and sociodemographic position.
Since patient characteristics are generally not modifiable, we considered it reasonable to look for further parameters upon which researchers have an influence in study planning, such as (a) study design, (b) comparison data source and (c) self-report modality.  the inclusion criteria, of which 50 were not found by database searching. The inter-rater reliability for full-text-screening was 0.84.
The selection process is presented in Figure 1.
Critical appraisal found 56 publications (47%) to be of high quality and another 58 (48%) to be of acceptable quality. Results are shown in Table 1.

| Data extraction
Among the 120 included publications, we found 18 articles dealing with cancer patients. Results regarding the three selected medication groups were found in 11 of them. All but one of the 11 articles 81 provide information on the association of study design aspects or patient characteristics with agreement.

| Characteristics of publications
We included 120 full-text articles and described their characteristics in Table 1 Table 4 shows studies which analyzed factors associated with the agreement. The factors include study design aspects and patient characteristics. Among the 10 publications, the authors used either stratification methods or regression models to analyze possible associations.

| Factors associated with agreement for patients with cancer
Age, education, and marital status were the most frequently analyzed patient characteristics. Nevertheless, no consistent association between these three factors and agreement can be derived.
Younger age is associated with better agreement in 2 of 11 analyses, 89,96 higher education in 2 of 10, 67,99 and being married in 2 of 10. 16,96 It must be considered that confidence intervals concerning education overlap in one of the two studies. 67 Furthermore, two publications found cancer recurrence to be associated with lower agreement, 16

| Agreement of self-report with other data sources in patients with cancer
Overall, agreement for estrogen, HT, and chemotherapy in cancer patients was relatively good. This may be due to cancer likely being a very serious diagnosis for patients.
In the studies included in our review, agreement was best for chemotherapy. This may be due to patient characteristics or study design. However, chemotherapy can probably be deemed a drastic experience and might therefore be better remembered.
Tentative suggestions were found that agreement might be slightly better for the broader HT medication group than for the specific agent estrogen. This observation is based solely on the

| Factors associated with agreement
Among the studies dealing with cancer patients, 10 studies analyzed associated factors by using stratification or regression analysis. They most commonly tried to identify possible associations of different patient characteristics with the agreement. Although a number of studies assessed age, education and marital status, we could not find a consistent association between these factors and agreement.
Only two studies analyzed design-related factors, comparing selfreports with two other data sources, therefore only allowing for a cautious comparison of data sources. It must be considered, however, that the Paganini-Hill et al 95 study, which found better agreement estimates for MR/physician than for pharmacy records, was conducted over three decades ago. In the meantime, data quality from each of the sources may have improved. The second study, Spengler et al, 111 dates from the same era, and shows marginally better proportion correct estimates for MR/clinic (85%) than for MR/physician (83%). With regard to MR/clinic, the authors stated that records may have been incomplete.

| Strengths and limitations
Our work has a number of strengths. To the best of our knowledge, this is the first systematic review to identify studies which compare self-reports and other data sources specifically for medications. Unlike the two existing reviews on this topic, 8,9 we did not analyze healthcare utilization overall but focused on medications. That focus enabled us to provide an overview of this research area and identify a considerable number of relevant publications and describe their characteristics. Furthermore, the subsequent focus on studies including patients with cancer enabled us to provide a descriptive overview of different parameters and the agreement estimates.
However, our review also has some weaknesses. Firstly, we were unable to detect a number of relevant articles by routine database searching (50 out of 120), despite searching five different databases and including specific medications in our search strategy. This may be due to some publications having conducted agreement measurements as a piggy-back analysis and not reported it in the title/abstract/keywords. A number of potential improvements to the search strategy can be gleened by taking a closer look at the nondetected publications. Firstly, "drug taking" would complement the terms "use/utilization/consumption." Secondly, in addition to "drug/medication," specific treatment methods should be added, such as "breast cancer treatment/hormone therapy/vaccination/chemotherapy." Thirdly, the definition of a specific comparison data source might be helpful, for example "prescriptions/medical records." The majority of nonroutinely identified publications came from reference lists (37 of 50).
Researchers looking to reproduce the search will thus be able to find most matches systematically.
Secondly, the 120 studies included are hardly comparable due to great methodological differences. We therefore did not perform a meta-analysis, instead opting to describe study characteristics to provide an overview of methods and analyzed aspects.

| CONCLUSION
This review includes 120 publications that measure the agreement between self-reports and other data sources and focuses on 18 of them which refer to cancer patients. We identified relatively good agreement for hormone therapy, estrogen, and especially for chemotherapy. No consistent pattern could be found regarding factors associated with agreement for either patient characteristics or design-related parameters. The latter was rarely analyzed, indicating approaches for further research. Studies with experimental design can be helpful to counteract the impact of different comparison data sources or self-report modalities under otherwise identical conditions.

ACKNOWLEDGEMENTS
Thanks to Jeremy Groves for providing language help in the preparation of the manuscript. This research did not receive any specific grant from funding agencies in the public, commercial, or not-forprofit sectors. Open access funding enabled and organized by Projekt DEAL.