The question of data generalizabilty and transferability of economic and clinical data for economic evaluation is inherent to every health economic decision problem—at national, regional, or institutional level—where either data are external to the decision problem or the relevant subset of data is not providing a sample size to inform the decision with sufficient certainty. This affects data transferability between jurisdictions, regions, populations, and institutions like hospitals and is a result of many reasons like differences in treatment patterns, severity of disease, and population characteristics.
With a growing number of jurisdictions requesting economic data in support of their health technologies allocation decision-making procedures and with the requests for data being supported by national guidelines on the conduct of economic evaluation, it could be seen highly relevant to study current guidance on the transferability of economic and clinical data for economic evaluations and its variations. The article “What Do International Pharmacoeconomic Guidelines Say about Economic Data Transferability” in this issue presents the results of a study reviewing the positions of various national guidelines in relation to the transferability and generalizability of data and reviews the methods suggested for addressing issues of transferability .
In the absence of a hypothesis and derived critical implications no gain in knowledge can be generated from the descriptive assessment of the guidance on data transferability—in an epistemological sense. However, a descriptive approach might be deemed relevant as baseline to recommend good research practices in dealing with aspects of transferability including analytic strategies and guidance for considering the appropriateness of evidence from other countries .
The more population or jurisdiction specific the data, the less generalizable/transferable are the data for other populations or jurisdictions. Not unexpectedly, the review of pharmacoeoconomic guidelines finds the generalizabilizy/transferability of economic data to be limited. Consequently to the nature of the data, baseline risk has a low transferability, clinical data respective data on relative risk has high transferability, and resource use and costs data have low transferability. Only little guidance has been found on analytical strategies and the transferability of health-state valuations or utility estimates. This might define an area where additional (methodological) guidance will add value for both the generation and critical review of data in health resources allocation decisions.
With the inclusion/exclusion criteria for national pharmacoeconomic guidelines being driven by easiness of data accessibility (obtainable from the ISPOR Web site and language versions available) the findings do not only lack generalizability; with the exclusion of Asian countries the national guidelines of countries have not been studied where data generalizability/transferability is of particular relevance.
If the aspiration is to explain differences in national guidelines and to deduct normative recommendations, we are under different methodological constraints. There are two main aspects that might be worthwhile to critically reflect on and that this editorial would like to propose to the discussion:
- 1Optimal level of data generalizability/transferability. The consideration of generalized/transferred data is a process of diligent and careful balancing in the context of the decision problem: the incremental costs of accepting uncertainty in the estimates versus the incremental benefit of (earlier) reallocation of health-care resources, the incremental informational value of additional evidence respective the incremental gain in the accuracy of the estimates, and thus the decision quality versus the incremental costs to generate and assess additional data. Any normative recommendation will have to imply superiority with respect to cost of the decision-making process and costs and benefits of the implications of the decision ensuring the incremental benefits match the incremental cost of additional jurisdiction-specific data provided.
- 2Optimal regulation of data generalizability/transferability. Further on, there is the question of effective and efficient regulation if a decision has been made about the above. What should be regulated in “coded” guidance (e.g., in national pharmacoeconomic guidelines) and at which level of detail? To what extent is the “case law” of reviewers' practice a more effective and/or efficient instrument? What are the costs and benefits of harmonization of requirements respective regulations versus the benefits of diversity across decision bodies? What provides static, what provides dynamic regulative effectiveness and efficiency? Dynamic efficiency is an alternative paradigm to neoclassical efficiency. In fact there is likely for various reasons to be trade-offs between static and dynamic efficiency. When we are considering dynamic efficiency good policy cannot be mechanically judged in terms of whether it achieves the optimal level of data transferability at the moment. Policy is a far more nuanced process that has to be carefully evaluated in terms of its effect on the retroactive effects and adaptive flexibility. Which type of regulation is superior: guidance on the methods to handle data, the way data should be reviewed, or the type of data to be generated? And maybe even more important: What have been the regulative aims of the respective national pharmacoeconomic guidelines reviewed? The optimal way of regulating data transferability and generalizability depends on multiple factors and will likely vary between jurisdictions.
In the absence of a theoretical foundation given by the authors and in light of the considerations above the author's hypothesis that the guidance on transferability is dependent on the issue date of the guidelines as well as on the “degree of development of health economic methods” is not comprehensible, the lack of critical implications is absconding the hypotheses from falsifiability. Further on the operationalization (e.g., iHea members per inhabitant) could be deemed insufficient even for the hypothesized causality. Difference in the aim of the guidelines, size of country, health-care funding, and decision makers are only listing a few other variables important to consider.
Finally and in conclusion to the train of thoughts above, national guidance has to be seen in its entity as a mix of coded and uncoded recommendations and expectations. The actual implementation and interpretation of the national guidelines in real-life practice and the level of economic data generalizability/transferability accepted by assessors is a better proxy of data generalizability/transferability. With the example of the decision practice of the National Institute of Clinical Excellence in the UK the authors provide additional information for one of the national guidelines incorporated in the review. However, the generalizability of the observed application in one jurisdiction to control for the actual implementation is limited.
The authors conclude that the review suggests increasing the level of standardization, the frequency of national guideline updates, and the level of detail to be incorporated. Solely based on the expected burden or costs associated with increasing data requirements the normative recommendations appear unbalanced and are not supported by the design of the study conducted.
In summary, there are several methodological and practical issues surrounding the transferability of economic data that are important to address. A review of what national guidelines for economic evaluations say about transferability is important in understanding the context in which transferability is currently practiced and discussed. Recommendations on good research practices for dealing with aspects of generalizability/transferability are filling an important gap. However, in order for the applied science of Pharmacoeconomics and Outcomes Research to make up for its epistemological aim and the aim of providing normative judgments, the methodological foundation of normative judgments has to be given the same importance as the methodological foundation the scientific community is seeking to establish as good research practices.