In‐depth analysis of patterns in selection of different physiologically‐based pharmacokinetic modeling tools: Part II — Assessment of model reusability and comparison between open and non‐open source‐code software

Whilst the reproducibility of models in the area of systems biology and quantitative systems pharmacology has been the focus of attention lately, the concept of ‘reusability’ is not addressed. With the advent of the ‘Model Master File’ dominating some regulatory discussions on pharmaceutical applications of physiologically‐based pharmacokinetic (PBPK) models, reusability becomes a vital aspect of confidence in their use. Herein, we define ‘reusability’ specifically in the context of PBPK models and investigate the influence of open versus non‐open source‐code (NOSC) nature of the software on the extent of ‘reusability’. Original articles (n = 145) that were associated with the development of novel PBPK models were identified as source models and citations to these reports, which involved further PBPK model development, were explored (n > 1800) for reuse cases of the source PBPK model whether in full or partial form. The nature of source‐code was a major determinant of external reusability for PBPK models (>50% of the NOSC models as opposed <25% of open source‐code [OSC]). Full reusability of the models was not common and mostly involved internal reuse of the OSC model (by the group who had previously developed the original model). The results were stratified by the software utilised (various), organisations involved (academia, industry, regulatory), and type of reusability (full vs. partial). The clear link between external reuse of models and NOSC PBPK software might stem from many elements related to quality and trust that require further investigation, and challenges the unfounded notion that OSC models are associated with higher uptake for reuse.


| INTRODUCTION
In the companion report (Rajput et al., 2022) associated with this current analysis, the explicit or implicit rationale in using open source-code (OSC) physiologically-based pharmacokinetic (PBPK) models was explored. With the current ongoing discussions around the concept of the 'Model Master File' (Zhao, 2021), the long-term use of developed models ('reusability') is becoming more important.
Whilst the aspects related to model credibility and quality (Baker, 2016;Frechen & Rostami-Hodjegan, 2022) as well as reproducibility (Kirouac et al., 2019;Manninen et al., 2017;Milkowski et al., 2018;Tiwari et al., 2021) have been the subject of several assessments, to our knowledge, the concept of 'reusability' in the context of modeling has only been discussed once (Rodrigues Matos et al., 2013) with the following definition describing reusability as: "A mathematical model will be fully reusable if it can be used as a simulation component within other mathematical models, limited only by its physics scope. That is, a modeling and simulation (M&S) methodology will support full reusability if it guarantees an isomorphism between models and their associated real systems, limited only by the physic scope".
However, this definition seems to be overly complex and fails to adequately cover the scope of model 'reusability'. Hence, there is no surprise that this report has never been cited or used in the context of reusability. In contrast to the area of modeling, 'reusability' is a well-known concept in computer science and software engineering (Meyer, 1987). It is described within that context in the Wikipedia entry (Reusability -Wikipedia, 2022) and Hooper and Chester (1991): "Reusability is the use of existing assets in some form within the software product development process; these assets are products and by-products of the software development life cycle and include code, software components, test suites, designs and documentation".
Since reusability in the context of modeling may have distinct features, we adapted the above definitions for reusability for the purpose of modeling (Table 1)  and had specific search terms as indicated in the Supplementary Material. These were merged to create a master set of articles, as shown in Figure S1. Using predefined inclusion and exclusion criteria (Table 2), each publication was scanned to ascertain whether it could be assigned to the set of 'Original PBPK Source Models'.
In the second step we aimed to assess any reuse cases of the 'Original Source PBPK Models' ( Figure S2) by investigating the citations made to these articles (using the 'Cited Reference' search options within the Web of Science). Citations that met the reusability criteria, as defined in Table 3, were exported to Excel and formed the database that we defined as 'Cases of Reuse'. It should be noted that application of the definitions to a small proportion of the reports was not straightforward, and they fell into a grey zone, some personal judgement was needed to help with their assignment.

| Categorising reports
Taking the affiliations of the authors into account, the type of organisation that contributed to each report was determined. A single report could be assigned to multiple organisational types (e.g. both academia and regulatory agencies). As a result, the types of organisational associations are not regarded as exclusive. Hence, three groups of organisations (Academia, Regulatory Agency, and Pharmaceutical Industry) were established, and items were assigned to one or more of these. Hospitals, universities, research centres, and other educational institutions were all included in the definition of ALDIBANI ET AL. 'Academia'. Public or governmental organisations that are responsible for adhering to legal requirements governing the drug development process in a particular country, for example, the U.S. Food and Drug Administration (FDA) and Medicines and Healthcare products Regulatory Agency were classified as 'Regulatory Agency'.
The 'Pharmaceutical Industry' encompassed commercial businesses focusing on research, development, marketing, and/or distribution of drugs. These included contract research organisations (CRO's).
The type of modeling platform used to build the PBPK model in each study was determined, paying specific attention to whether it used an OSC or NOSC software. A single report was assigned to multiple platforms if more than one software package had been employed.

| Determining type of reuse
Each of the potential reuse cases were reviewed to determine if they were indeed a PBPK model-building exercise. All the 'Original PBPK Source Models' were scored for the cases of reusability that occurred (whether once or multiple times). For the 'use' cases we determined T A B L E 1 Definitions of each term used throughout the study and applied during data sorting and analysis, complemented by Table 1 in Part I (Rajput et al., 2022) and adapted from Rostami-Hodjegan and Bois (2021).

Reusability
The reutilisation of I) the model in its entirety (the workspace), II) the system (drug independent) components file(s), III) the drug-dependent component file(s) IV) the modeling strategy or V) the aforementioned components with modifications.

Partial reusability
The reutilisation of a single component of a previously published model. For example, only the system component or the drug-dependent component.

Full reusability
The reutilisation of a previously published model in its entirety (the workspace) or the reutilisation of both system components and drug-dependent components.
External reusability Reusability by author(s) who do not have the same affiliations as the author(s) of a previously developed PBPK model.

Internal reusability
Reusability of a previously published PBPK model by the same author(s) who initially published it or by an author(s) who has the same affiliations.
T A B L E 2 Inclusion and exclusion criteria used in the triage of the Master Search results.

Inclusion criteria Exclusion criteria
Type of publication

| Database of articles forming the analysis
The initial search identified 376 publications, and 145 of these arti-  F I G U R E 1 An onion diagram illustrating the proportion of articles filtered from the master Search triage process ( Figure S2), which led to the remaining 145 articles. The fill colours used in the diagram reflect the colour coding that was applied in data processing.

| Reusability: Comparison between open source-code and non-open source-code physiologically-based pharmacokinetic models
Comparing the reusability of models developed by OSC and NOSC platforms showed some similarities as well as discrepancies. As evident from the radar chart in Figure 3, out of 145 articles that developed 'Original PBPK Source Models', nearly 60% of the models developed within NOSC platforms had reuse cases whether once or multiple times (i.e. subsequent applications met the criteria for reusability set a priori in our study), whereas only 43% of models developed by OSC platforms were reused (whether once or multiple times).
The radial bar chart shown in Figure 4 indicates the Relative Percentage of Reusability for each software package. Relative Percentage of Reusability represents the proportion of 'Reused' and 'Fully Reused' cases relevant to the total of cases for all platforms.
The reusability of the 'Original PBPK Source Models' stratified based on the platforms used is shown in Figures 5 and 6. The cases of reuse for each platform, categorised on a partial versus full and external versus internal basis, are shown as branches of the proportion of 'Original Source PBPK Models' that had a reuse incidence. As shown in these figures, the 'Original PBPK Source Models' that were developed in the NOSC platforms showed a higher likelihood of being reused, with Simcyp and GastroPlus being reused in 60% and 50% of the cases, respectively. The reusability cases for Simcyp and GastroPlus models were mainly external (64% and 71%, respectively).

| DISCUSSION
In the first instance, reusability and reproducibility may be interpreted as synonyms. However, this is not an accurate assumption, as demonstrated by recent debates on this subject (Manninen

F I G U R E 2
The distribution of platforms leveraged to develop 'Original PBPK Source Models' (n = 145) that were investigated for reusability. Conjunctions represent articles that utilised a combination of platforms.

F I G U R E 3
Radar chart comparing the reusability parameters for 'Original PBPK Source Models' developed (n = 145) via both OSC and NOSC platforms with a table illustrating the actual percentage of reusability.

296
-F I G U R E 4 Two radial bar charts comparing the proportion of reused and fully reused cases of PBPK models developed via NOSC and OSC software.

F I G U R E 5
Pie-charts detailing the percentage of reusability for the NOSC platforms that were used to develop the 'Original PBPK Source Models', with branches showing the percentage of reusability cases that were external/internal, partial/full. et al., 2017;Milkowski et al., 2018). Reproducibility in the context of modeling is adequately defined by Tiwari et al. (2021) as follows: "The ability to build the code de novo and/or ensure the mathematical expressions are correctly represented and reproduce the simulation results in a software different from the one originally used".
Their definition is not the subject of the current report, although the concept of reusability is linked with reproducibility and ultimately the quality and trust in the use of models (Frechen & Rostami-Hodjegan, 2022;Schuck et al., 2015). Concerns with the credibility of scientific publication outcomes are not exclusive to PBPK M&S (modeling and simulation); rather, they worry scientists in all domains, as underlined in Baker's report (Baker, 2016). According to the findings ALDIBANI ET AL. Considering that external reusability is more reflective of true 'reuse', our analysis demonstrated that confidence in models built within NOSC platforms is significantly higher than that in models built within OSC platforms. This can be attributed to a series of factors associated with NOSC platforms that go beyond the userfriendly graphical interface and the availability of a predefined model library, since the latter two are also available in OSC platforms and not specific to NOSC modality. Educational support material, and more extensive model verification by a larger set of researchers, would enhance the level of trust that a modeler puts on a previously developed model (El-Khateeb et al., 2021). It must be noted that these opinions are inferred from interpretation of the published reports and, as indicated in Part I of this study (Rajput et al., 2022), modelers seldomly include the rationale for the selection of a certain model or platform in their report (Rajput et al., 2022).

| CONCLUSION
There is no consensus on defining reusability in the context of modeling. Hopefully, this report will help to initiate some efforts by FDA (Zhao, 2021), can be considered as tailwind to raise the awareness about reusability of the models and their determinants.
The findings of this study, regarding PBPK models developed in F I G U R E 6 Pie-charts detailing the percentage of reusability for the two most popular OSC platforms that were used to develop the 'Original PBPK Source Models', with branches showing the percentage of reusability cases that were external/internal, partial/full.