Recent advancement in integrating artificial intelligence and information technology with real‐world data for clinical decision‐making in China: A scoping review

Striking innovations and advancements have been achieved with the use of artificial intelligence and healthcare information technology being integrated into clinical real‐world data. The current scoping review aimed to provide an overview of the current status of artificial intelligence‐/information technology‐based clinical decision support tools in China.


INTRODUCTION
Clinical decision-making (CDM) is defined as the process of diagnosis or treatment plan development based on the available clinical information, often with the incorporation of known patient preferences. 1It typically consists of three integrated parts, namely diagnosis, assessment of severity, including risk assessment and severity classification, and disease management. 2 Traditionally, clinical decisions are heavily driven by knowledge and experience, including the patient's subjective and objective data, physician's experiences, geographic location, the qualification of the hospital, etc., resulting in subjectivity, uncertainty, and variation. 3Evidence-based CDM, on the other hand, has been increasingly adopted by clinicians in recent decades.Established on three key dimensions, including the existing best available clinical evidence, with randomized controlled trials (RCTs) as the gold standard, the physician's clinical experiences, and the patient's own preferences, evidence-based CDM integrates scientifically sound evidence with the subjectivity of physician's experiences and patient's value to make optimal clinical decisions. 4th the exponential increase in the volume of healthcare data and the rapid development of health information technology, real-world data (RWD) have become an indispensable component fueling the development and innovation of CDM.RWD are data associated with patient health status routinely collected from the daily basis of clinical practice, including electronic medical records (EMRs), medical claims, disease registry data, patient-generated data, etc. 5,6 The clinical evidence generated through the analyses of RWD by real-world study (RWS) is called real-world evidence (RWE).Clinicians and researchers are now equipped with a vast amount of data from nontraditional research settings, further uncovering new insights and streamlining the process of CDM.RWE has distinct advantages compared to evidence generated by traditional RCTs.Unlike RCTs with restricted criteria for participant selection, well-designed RWE may provide clinicians with supplemental safety and effectiveness information on therapy in a large and heterogeneous patient population in a real-world setting. 7][10] More importantly, the generation of RWE is typically impacted by different RWD elements, including concomitant drugs, geographic location, and the patients' preferences, which brings more clinical insights into the evidence-based CDM. 8,11 the era of medical big data, RWD possesses the potential to go beyond supplementing the current clinical knowledge.The fast development of health information technology and artificial intelligence (AI) has forged a new direction to advance CDM, taking an essential step forward in individualized care.AI could be integrated into RWD for mining RWE and developing of operational tools, such as clinical decision support systems (CDSSs).CDSSs refer to various computerized and noncomputerized tools and interventions that help clinicians to make evidence-based medical decisions. 12The purposes of CDSSs include patient protections, reduced misdiagnosis, improved patient prognosis, increased medical efficacy, and healthcare quality, etc. 13,14

Protocol and registration
A prespecified review protocol was developed without registration.
The current review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension for Scoping Reviews (PRISMA-ScR). 17The protocol is available on request from the corresponding author.

Eligibility criteria
The predefined eligibility criteria were developed based on the PICOTS framework (

Selection of sources of evidence and data charting
Initial screening of titles and abstracts of relevant literature, which complied with the prespecified inclusion/exclusion criteria, was performed by the reviewer to determine eligibility (XL).Full-text review was subsequently performed to evaluate relevance (XL).Results of both initial screening and full-text review underwent group discussion and verification, with any uncertainties addressed and resolved through collective deliberation (XL, CY, and JZ).A standardized data

Synthesis of results
The included studies were qualitatively synthesized.A descriptive summary was performed based on the data charting form.

Results and characteristics of sources of evidence
The initial search of the electronic databases yielded a total of 2,075 studies, including 1,690 English literature and 385 Chinese literature.
An additional search of bibliography and gray literature resulted in 19 peer-reviewed articles and 16 commercially available CDSSs.After the initial scan, 93 studies underwent full-text review.Following the exclusion of studies that did not meet the eligibility criteria, a total of 50 studies and tools, including 37 peer-reviewed studies and 13 commercially available CDSSs, were included in the final review.Reasons for full-text exclusion were the prediction of health outcomes at the population level, which included the secular trend in disease mortality and incidence for a specific region (n = 4), comparative assessment of the real-world performance of risk prediction models (n = 9), insufficient information of RWD sources (n = 19), and no full text available (n = 11).
A PRISMA flow chart of study selection was shown in Figure 1. 18Summary of data extracted from 37 peer-reviewed articles was shown in Table S1.
Of 50 selected studies and tools, 32.0% were developed for disease diagnosis, 54.0% for risk prediction and classification, and 14.0% for disease management.An overview of the characteristics of included studies and tools was summarized in Table 3. Frequencies of AI algorithms, both overall and by clinical use, in 37 peer-reviewed studies were shown in Supplemental Figures.

Therapeutic areas
As the core technique of AI, ML has the advantage of mining associations and clinical evidence from a vast amount of structured and unstructured clinical RWD to meet more generalized clinical needs. 19vered disease areas included different types of cancers, [20][21][22][23][24] cardiovascular and cerebrovascular diseases, 25-29 pneumonia, 28,29 endocrine diseases based on musculoskeletal prediction, 29,30 retinal abnormalities, 31 hepatobiliary diseases, 32 gestational diabetes, 33 and acute leukemia. 34,35

Sample size and data collection
The sizes of datasets used to derive AI model varied substantially.In general, the sample sizes of unstructured imaging data were larger than that of structured EMR data.The number of images or participants used to develop imaging recognition tools ranged from 1,252 32 to 207k, 31 comparing with 227 35 to 1,000 33 cases for structured EMRs.
Retrospective data collection was prevalently used, 23,24,[33][34][35] and only one study collected data prospectively. 32Further, Lin et al. 31 used a mixed data collection method that retrospectively extracted fundus images for model development and prospectively collected data for external validation.

AI algorithms
Deep learning (DL) was primarily leveraged for unstructured imaging data.Convolutional neural networks (CNNs) were the most frequently used techniques, 23,24,31 with varying architectures like Xception, 23 Inception-ResNetV2, 31 and ResNet. 24Deep neural network was adopted by Xiao et al. 32 For structured EMR data, Catboost optimized by genetic algorithm, a supervised machine learning, was used by Liu et al. 33 for gestational diabetes diagnosis.A combination of supervised learning, including support vector machine (SVM), decision tree, and random forest, and unsupervised learning, including k-means, was adopted to diagnose acute leukemia. 34,35

5.1
Risk prediction tools

Therapeutic areas
Chronic diseases were popular areas of research and development.

Data sources and clinical benefits
Existing databases and clinical EMRs were two major data sources for the development of AI-enabled risk prediction tools.Three models were derived from existing population-based cohorts, which included the DATADRYAD dataset, 44 the China Kadoorie Biobank, 36 the dataset built by a prospective cohort study. 4549,53 One additional study leveraged disease registry, the China Acute Myocardial Infarction registry. 43rposes of developed tools varied from predicting populations at risks, including women who are at high risk of gestational diabetes in early pregnancy 45 and patients who are at risk of progression to severe COVID-19, 48,49 to predicting both short-term and long-term disease risk and prognosis.42][43]47,[50][51][52]

AI algorithms
Supervised machine learning algorithms were primarily used to support risk prediction.45,47 Other algorithms included gradient boosted tree, 36 artificial neural network, 42 random forest, 37,53 k-nearest neighbor, 48 logistic regression, 51 and Naïve Bayes. 52Evidential reasoning rule and focused-CNN with Boruta algorithm were used respectively to select features of ICU admission and in-hospital mortality of trauma patients 50 and patients who were at risk of progression to severe COVID-19. 49Furthermore, two algorithms were developed specifically to address time-to-event data, including one deep learning technique, BLeNet, 41 and one supervised learning technique, XGBoost-Surv. 46

Model interpretation
Some studies have highlighted the importance of model interpretability for the purpose of real-world clinical application.Captum interpretation library 41 and SHapley Additive exPlanations (SHAP) tool 37,42,46 were two commonly adopted tools for clinical interpretability.It should be stressed that a trade-off between predictive ability and clinical explainability was suggested by Li et al. 43 in choosing the optimal in-hospital mortality prediction model for patients with ST-elevation myocardial infarction.The study found that the random forest model had the best predictive performance but needed to improve clinical interpretability due to the large number of different decision trees.
In comparison, the results of chi-squared automatic interaction detector tree models were easily comprehended by clinicians but with a compromised predictive ability.Depending on different real-world scenarios and clinical needs, the choice of ML technique might vary.

Data sources
Clinical EMRs were the major RWD sources used to support disease classification.Single-center EMRs were leveraged by four studies, which included two studies with structured EMRs 54,55 and another two studies with unstructured imaging data. 56,57Two studies utilized unstructured text 58 and imaging data 59 from multi-center EMRs.Additionally, two population-based cohorts were identified, including the Shanghai Integrated Diabetes Prevention and Care System 60 and the China National Stroke Screening and intervention program. 61On the other hand, it should be noted that the above-mentioned studies only involved single source of RWD.In contrast, Yan et al. 62 proposed a multimodal classification approach combining low-dimensional structured EMR data with high-dimensional pathological images from single-center.

Clinical benefits and therapeutic areas
There were two main purposes of the developed tools, including differentiation and classification.Differentiation tools differentiated malignant tumors from benign nodules for thyroid nodule, 59 pulmonary nodule, 57 and breast cancer. 62Classification tools classified clinical stages for lung cancer, 54 different stroke risks, 61 traditional Chinese medicine syndromes of liver cancer, 58 various causes for fever of unknown origin, 55 multidisease and multilesion diagnosis, 56 and the severity of diabetic retinopathy. 60

Sample size and data collection
Similar to predictive and prognostic tools, RWD used to support disease classification were collected retrospectively.RWD sources that were multi-center and population-based had larger sample sizes as compared to that of single-center sources.The sample sizes for multicenter and population-based studies were at least 10k, 58 while at most 2,502 54 for single-center studies.

AI algorithms
Machine learning was utilized by four studies, including extreme learning machine network with particle swarm optimization, 58 Naïve Bayesian model, 54 LightGBM, 55 and random forest. 61ep learning was used by five studies, including convolutional neural networks, 56,57 ResNet, 60 richer fusion network, 62 and ThyNet. 59

DISEASE MANAGEMENT
In contrast to the remarkable progresses being made in diagnostic, predictive, and classification tools, the AI-/technology-based tools informing diseases management have been substantially constrained.Of five tools identified, four of them were commercially available cases collected from gray literature, with limited information regarding RWD sources, sample sizes, and validation methods.
Therefore, instead of thematic analysis, narrative descriptions were performed.

iPharma
To support drug administration, the AI-based iPharma system exploited a variety of ML techniques to deliver individualized medication regimens. 63By mining the association of treatment regimens with disease outcomes based on clinical EMR data, iPharma assisted physicians and pharmacists in administering appropriate dose at appropriate time for drugs such as vancomycin and Warfarin.

CDSS for liver cancer treatment recommendation
Incorporated into single-center EMRs, a CDSS recommending treatment regimen for hepatocellular carcinoma was developed.A merged model of multiple subclassifiers was applied to calculate the recommendation coefficient for different therapy choices and a DeepSurv algorithm was leveraged to predict survival outcomes and risk of recurrence. 64Internal validation showed a concordance rate for the recommended treatment between the system and a multidisciplinary team of 0.951.

iNCDSS
The iNCDSS system for diabetes management combined ML algorithms, the experiences of physicians, and clinical guidelines to develop personalized insulin regimens and to support insulin dose adjustment for better glycemic managements. 65

Technology-based disease management tools
Relying on information technologies that were integrated into patientcentered disease management interventions, the personalized disease management tool for hemophilia, myPKFiT, was approved by the NMPA for guiding dosing regimens to inform both healthcare providers and patients with hemophilia in China. 66Based on two to three blood success of surpassing or being comparable to human performance. 70,71 contrast, limited tools have been developed to inform disease management decisions, which might be related to the heterogeneity and complexity of treatment regimens.The existing literature on drug combinations illustrated that instead of prescriptive analytics that aimed to optimize drug combinations and treatments, great emphasis was placed on outcome prediction by most ML-related developments. 72cordingly, there existed a disproportionate interest and investment in tools facilitating disease management, as compared to predictive and diagnostic tools, which was possibly due to the complexity of disease management decisions with the optimization of a wide range of medical and lifestyle-related factors. 73th the integration of diversified AI algorithms and health-  74,77 In particular, unique patterns of treatment seeking behaviors in China resulted in the lack of longitudinal data that greatly constrained the ability to support long-term clinical decisions. 74,78Also, without the use of AI, manual transcription from the EMR system to research databases with regard to data contained in the unstructured field of EMR, such as imaging and pathological results and clinician notes, further raised challenges hindering the exploitation of full potential of RWD informing CDM. 79 address these issues, it should be necessary to establish an independent clinical research-specific platform that directly converts EMR data to the platform with the use of AI techniques such as natural language processing. 79Longitudinal data reported by the patients could be integrated into the platform, in combination with researchspecific and routinely collected data.More importantly, a collaborative approach through multistakeholder engagement was the prerequisite for the successful establishment of such a platform, including hospitals, health technology and pharmaceutical companies, academia, and regulatory agencies. 74,78,80The collective efforts would secure and promote data sharing, building a solid foundation for high-quality RWD, effectively informing CDM in China.
Statistical models like logistic and Cox regressions are typical models used for conventional variable selection and model building. 81wever, conventional predictive models usually have limited capability of processing large-sized datasets and addressing nonlinearity, greatly preventing long-term disease prognosis.ML-based prediction models, on the other hand, tended to have better predictive performance.Supervised machine learning and deep learning techniques were commonly employed, with XGBoost, random forest, Bayes network, and CNN as popular algorithms.Even though the trend of decline over time could not be prevented, the loss of predictive ability was potentially balanced by the predictive benefits gained with the integration of ML.In predicting the risk of myopia among school-aged children, a nonlinearity of risk of myopia development after three years was found, suggesting an incremental added value of ML for long-term disease prognosis. 53Although clear benefits on predictive ability have been demonstrated by the AI-enabled models in comparison with conventional models, 19,82  Despite the fast-growing research on AI in combination with RWD to support CDM in China, the complexity of AI has considerably impeded its clinical acceptability and applicability.3][94][95] In the era of medical big data, the training of ML/DL based on aggregated clinical source data dramatically increases the precision, but not internal validity. 19,96The gained value of ML/DL algorithms would be maximized under the appropriate conditions of high-quality RWD and methodologically rigorous RWE design. 19However, the overall quality of RWD in China varied dramatically due to the heterogeneity of RWD sources and structures, 74,[78][79][80] which reduced the acceptability of RWD and resulted in low credibility of AI-based clinical decision support tools in China.8][99] Main factors contributing to the deficiency of RWD quality included missing data, 80 insufficient source data verification, 79 inconsistence data standards, 77 and lack of common data model. 77,78 unleash the untapped potential of RWD in informing CDM in China, improving RWD data quality through multidisciplinary collaboration is of high priority. 74,78,80,99reover, the increased value of AI-integrated tools was primarily quantified by the improvement in parameters of discriminative power, such as AUC and C-statistics.However, none of these parameters sufficiently reflected reliability and clinical utility. 82,100,101The selected model based on the highest AUC or C-index value might not fit the complex clinical environment in the real-world the best, potentially leading to the reluctance to employ AI-enabled tools in hospitals.Furthermore, trade-offs were suggested between clinical interpretability and model complexity and performance. 37,1024][105] The lack of transparency and comprehension of the reasoning logic behind AI created the black box problem that clearly compromised the trust of healthcare providers. 105 make AI-based CDSSs clinically useful, model evaluation considering clinical utility should be embedded into the model building process.Decision curve analysis that quantifies the net benefit to inform clinical value has been well-established but has yet to be exploited in building predictive models, 82,106,107 especially in China.
Net benefit incorporated the clinical outcomes of the decisions made based on the built model, providing more clinical insights into whether the benefits would outweigh the risks. 106Moreover, visualization would greatly improve model interpretability, 102,103 contributing to the overall comprehension of the results provided by AI.Promising technical development such as the explainable AI was also proposed to promote transparency and to "open the black box" of AI, 105,108 offering potential solutions that enabled deeper understanding, appropriate trust, and effective management of AI.

LIMITATIONS
A potential limitation of current review concerned with the continuous evolvement of the field of RWD, as the indexing of publications on RWD might need to be more consistent, and the current search

CONCLUSION
The fast-evolving field of RWD holds great promise in data-driven innovation for healthcare communities.The integration of information technology/AI into RWD in building operational tools further brings hope for the possibility of personalized medicine.However, existing barriers concerning both the accessibility and quality of RWD and confidence in AI considerably hindered the widespread acceptability and applicability of the current AI-/technology-enabled clinical decision support tools in clinical practice.To effectively promote the systematic use of RWD and drive a paradigm shift in CDM in China, there is an urgent need for multidisciplinary collaboration of various key stakeholders.The continuous development of RWD will further advance the realization of evidence-based and high-quality goals in CDM in China.
Thematic synthesis was conducted regarding the AI-/technologybased clinical decision support tools in China.Specifically, four main themes were determined deductively based on different clinical use of RWD, which were disease diagnosis, risk prediction, disease classification, and disease management.Subthemes, where applicable, emerged inductively and were determined through group discussion until consensus was reached (XL, JZ, and CY).In addition, existing challenges preventing extensive RWD use in CDM in China were addressed, along with suggestions from the existing literature.

8 DISCUSSION
samples, myPKFiT was able to construct personalized pharmacokinetic curves based on the Bayesian estimation of PK parameters of factor VIII, including characteristics of drug metabolism such as peak and trough levels.Likewise, a technology-enabled national diabetes care and support program, the Lilly Connected Care Program (LCCP), was launched to improve the patient's blood glucose control.67In collaboration with WeChat, the largest social application in China, the LCCP enabled personalized diabetes management by allowing the clinicians to monitor the dynamic patient-generated blood glucose data in real-time.Additionally, individualized system-based reminders and self-management support were available to diabetic patients.The retrospective analysis of patients enrolled in the program showed a significant improvement in glycemic control as compared to the baseline, demonstrating the program's potential of guiding diabetic decision-making.68,69This scoping review was the first study comprehensively investigating the scope and applicability of AI-/technology-based clinical decision support tools in China.Our study indicated that risk prediction and classification were the most productive areas for AI applications being integrated into RWD to facilitate CDM in China, which accounted for 54% of the identified studies and tools.Imaging recognition tools for disease diagnosis were the most extensively marketed tools in the current clinical practice in China, possibly due to the well-established techniques of medical image analysis and the well-demonstrated clinical care information technology, RWD guiding CDM in China has been advanced to meet more generalized clinical needs and to address the existing pitfalls of models developed based on conventional approaches.Main therapeutic areas of interest focused on chronic diseases, including cardiovascular and cerebrovascular diseases, various types of cancers, and diabetes.Other popular research subjects ranged from differentiating malignant tumors from benign nodules to predicting progression to severe COVID-19.Despite the striking progress being made with the application of AI, RWD have limited real-world impact on CDM,7 particularly in the context of China.The real-world application of the developed tools had been rarely found.Consequently, a gap between heavy research investment and limited clinical implementation existed.To meet the wide range of clinical needs, regularly utilized RWD sources included single-center EMRs, either structured or unstructured, which was consistent with the findings of Xie et al.74The widespread use of single-center EMRs would further lead to the disproportionate adoption of internal validation for model evaluation.The lack of access to multi-center EMRs greatly restricted the generalizability of the operational tools that were internally validated, resulting in limited real-world clinical application.In China, the implementation of EMR systems has been largely government-driven.In general, the evaluation metrics of EMR in China can be classified into three stages, namely, low stage (levels 0-2 emphasizing source data collection), medium stage (levels 3-4 emphasizing data sharing across departments within the same hospital and basic clinical decision support), and high stage (levels 5-8 emphasizing information sharing across different hospitals and intelligent clinical decision support).75Released by the National Health Commission of the People's Republic of China, the "National Monitoring and Analysis of the Performance Appraisal of the National Tertiary Public Hospitals in 2020" showed that, as of 2020, the average EMR level of the participated tertiary public hospitals was 3.65, 76 indicating a medium stage of EMR development that enabled data exchange among different departments.In addition, 91.26% of the participated tertiary hospitals reached level 3 or above and 65.26% reached level 4 or above, essentially achieving information sharing within the hospital and providing preliminary support for CDM.Given this context, the widespread adoption of single-center EMRs and the limited clinical uptake of intelligent tools in China might possibly be attributed to the current state of EMR system construction in China.Besides, since clinical EMRs were not designed for research use, research-specific data collection was often required.Even within the same RWD source, such as single-center EMRs, the availability of individual data element might vary across hospitals.
strategy might not produce a comprehensive result of relevant studies.Moreover, electronic database in technology, like IEEE Xplore, was not searched since the focus of current review was the application of RWD in developing AI-/technology-based CDSSs in China.Publications in IEEE Xplore tended to highlight the theoretical mechanism of ML algorithms, with minimum description and attention paid to RWD data sources and data quality.In addition, relying on a single reviewer for both initial screening and full-text eligibility assessment could introduce bias.To minimize possible biases and ensure reliability, however, results generated by the single reviewer were discussed, reviewed, and verified through collective deliberation.
Knowledge-based and model-based CDSSs are two main types of CDSSs that are commonly used worldwide.Knowledge-based CDSS, also called expert system, facilitates CDM by human-computer communication through the reasoning conducted by the existing knowledge base. 15Data model-based CDSS automatically learns rules with 16re data to support CDM systems to achieve personalized medicine.16Withthe national effort promoting the integration of information technology/AI into RWD to facilitate CDM in China, technology/AI-enabled tools have been gaining growing popularity and have become the main focus of research and development in the healthcare industry, encouraging major advancement and potentially driving a paradigm shift in CDM in China.However, to date, studies have yet to be conducted comprehensively assessing the scope and applicability of information technology/AI being integrated into RWD in CDM in China.To bridge the gap, this scoping review aimed to provide an overview of the current status of AI-/technology-based clinical decision support tools in China.Possible barriers impeding the consistent use of RWD integrated with AI in support of CDM were also identified with suggestions provided by the existing literature.

TA B L E 2 PubMed search query. PubMed search query
determined with the initial screening of relevant literature and were revised as required after the full-text review.The main information extracted from each article included basic information (authors, year of publication, and databases), types of CDM, therapeutic areas, RWD data source, AI algorithms, validation methods, limitations, challenges, etc.