Utilizing mixture design response surface methodology to determine effective combinations of plant derived compounds as prostate cancer treatments

Abstract Background Prostate cancer (PC) is estimated to cause 13.1% of all new cancer cases in the United States in 2021. Natural bioactive compounds have drawn the interest of researchers worldwide in their efforts to find novel treatments for PC. Many of these bioactive compounds have been identified from traditional Chinese medicine (TCM) remedies often containing multiple bioactive compounds. However, in vitro studies frequently focus on the compounds in isolation. Aim We used mixture design response surface methodology (MDRSM) to assess changes in PC cell viability after 48 h of treatment to identify the optimal mixture of all 35 three‐compound combinations of seven bioactive compounds from TCM. Methods and Results We used berberine, wogonin, shikonin, curcumin, triptolide, emodin, and silybin to treat PC3 and LNCaP human PC cells at their IC50 concentrations that we calculated. These compounds modulate many chemotherapeutic pathways including intrinsic and extrinsic apoptosis, increasing reactive oxygen species, decreasing metastatic pathways, inhibiting cell cycle progression. We hypothesize that because these compounds bind to unique molecular targets to activate different chemotherapeutic pathways, they will act synergistically to decrease tumor cell viability. Results from MDRSM showed that two‐way combinations were more effective than three‐way or single compounds. Most notably wogonin, silybin, emodin and berberine responded well in two‐compound combinations with each other in PC3 and LNCaP cells. We then conducted cell viability tests combining two bioactive compound ratios with docetaxel (Doc) and found significant results within the LNCaP cell line. In particular, mixtures of berberine and wogonin, berberine and silybin, emodin and berberine, and emodin and silybin reduced LNCaP cell viability up to an average of 90.02%. The two‐compound combinations were significantly better than docetaxel treatment of LNCaP cells. Conclusion Within the PC3 cells, we show that a combination of berberine, wogonin and docetaxel is just as effective as docetaxel alone. Thus, we provide new combination treatments that are highly effective in vitro for treating androgen‐dependent and androgen‐independent PC.


| INTRODUCTION
Prostate cancer (PC) is the most common cancer in men in the United States. Estimates range from 1 in 10 to as high as 1 in 5 men will be diagnosed with PC. Thankfully, the overall 5-year survival rate for PC patients is 97.8%. This statistic, however, masks the lethality of metastatic and castration-resistant PC (mCRPC) which has a 5-year survival rate of just 30.6%, and causes 34 000 deaths annually in the United States alone. 1 On average, patients diagnosed with mCRPC pass away 9-13 months after diagnosis because current treatments have limited effectiveness against mCRPC. [1][2][3] There is a substantial need to improve treatments against lethal metastatic, androgendependent and androgen-independent PC while keeping side effects in noncancerous cells to a minimum.
Treatments for PC include surgery (i.e., prostatectomy and removal of regional lymph nodes), radiotherapy, androgen deprivation therapy (ADT), and chemotherapy. Prostatectomy and radiotherapy are more frequently used for localized PC alone or in conjunction with ADT or chemotherapy. Chemotherapy, ADT, and androgen receptor (AR) inhibitors are the most common treatment options for distant PC. 4 PC can also develop resistance to these therapies but combining therapies can improve treatment response and limit development of drug resistance. 5 Specifically, research is being done to improve PC treatment through a combination of chemotherapy with bioactive compounds. For example, treatment of PC3 cells with docetaxel and vitamin E led to a significant decrease in the cell viability compared to docetaxel treatment alone. 6 Researchers are also studying bioactive compounds found in traditional Chinese medicines (TCM) to determine their chemotherapeutic effects. 7 TCM bioactive compounds are used in complex combinations, reducing the concentration of a single bioactive compound needed. The current usage of TCM bioactive compounds to treat human sickness indicates that these compounds may have low toxicity to the body. Researchers have also shown that certain TCM bioactive compounds have multiple anticancer effects. TCM bioactive compounds are a viable option to study as a treatment option in combination with chemotherapy in PC patients. Our study specifically focuses on combining TCM bioactive compounds with docetaxel as a PC treatment option. We explore combinations of seven TCM bioactive compounds, with known anticancer effects, to treat PC. Specifically, we tested three-way combinations of TCM compounds and then tested the most effective combinations in tandem with docetaxel treatment. The compounds we chose included berberine (BB), 8,9 curcumin (Cur), [10][11][12] emodin (Em), [13][14][15] wogonin (Wo), 16 shikonin (Shk), 17,18 triptolide (Ttd), 19,20 and silybin (Sy) (silibinin). [21][22][23][24] Each of these compounds target unique anticancer pathways (Table 1), and therefore may have synergistic effects. [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24] For example, previous studies have demonstrated that berberine increases reactive oxygen 8 species and triptolide inhibits the androgen receptor 20 in prostate cancer. Treating prostate cancer cells with both compounds would simultaneously target two anti-cancer pathways which could lead to synergistic effects and lower the concentration of the drugs and reduce off target effects.
One of the most common statistical methods used to measure drug combinations is the Chou-Talalay method because of its ability to distinguish between synergistic, antagonistic, or additive interactions. 32 However, this method is limited because it can only test combinations of two compounds and requires them to be in constant ratios of each other. In order to test our three-compound combinations based on the half maximal inhibitory concentration (IC50 values), we used a response surface methodology (RSM). RSMs are used to identify combination ratios that maximize a desired effect. Examples include Box-Behnken, fractional factorial, Plackett-Burman, and mixture design response surface methodology (MDRSM). 33 Response surface methodologies are often used in food science, engineering, and manufacturing, but used less frequently in biomedical research. We decided to use MDRSM to test our TCM bioactive compound combinations.
We used MDRSM to identify the most effective mixtures of all 35 three-compound combinations possible from the seven TCM bioactive compounds we chose. MDRSM measures the effects of three or more compounds, requires fewer experimental runs than other response surface statistical methods, and has been used to measure three-way chemotherapy combinations against PC in vitro. 33 We  Figure 1 and

| Mixture design response surface methodology (MDRSM)
We tested all 35 three-compound combinations in triplicate in LNCaP (Table 3), PC3 (Table 4), and DU-145 (Supplemental Table 1) cells using MDRSM to predict the optimal combination by fitting the data to a response surface (  LNCaP cells responded most frequently to berberine, wogonin, and emodin. Berberine, wogonin, and emodin either alone or in combination contributed to 75.56% of all optimal treatments against LNCaP cells ( Table 6). Combinations of berberine, wogonin and emodin also predicted high reduction of LNCaP cell viability. This is especially true in the most effective combinations even when excluding the top two optimal treatments, both of which contained berberine and wogonin, but had significant lack of fit ( Table 3). The lack of fit statistic establishes that the model and data fit well. If the p-value for the lack of fit is less than or equal to 0.05 we rejected the model due to a poor fit between the model and data. Excluding the top two optimal treatments, the top 4 optimal treatments contained combinations of berberine, wogonin, and emodin with predicted LNCaP cell viability below 8% (Table 3 (rows C-F)).
PC3 cells had four optimal treatments which predicted less than 50% cell viability (Table 4 (rows A-D)). Three of the optimal treatments were mixtures of berberine and wogonin (Table 4 (rows B-D)).
While the most effective treatment appeared to be shikonin alone, this trend was not repeated. Berberine and wogonin consistently appeared to be part of optimal treatments contributing to 47.06% of optimal treatments against PC3 cells, whereas shikonin only contributed to 7.84% of the optimal treatments ( Table 6). The predicted most effective combination of berberine and wogonin consisted of 52% BB and 48% Wo which resulted in only 43.22% predicted cell viability (Table 4 (row B)). Since berberine and wogonin were found in combination so often in the optimal treatments and reduced cells up to greater than 50%, we further tested a MDRSM combination on a docetaxel-resistant PC3 cell line. Previously in our lab, we grew the DR-PC3 cell line to demonstrate chemotherapy resistant cells. 6 We treated the DR-PC3 cell line with a three-way combination of shikonin, berberine, and wogonin. The optimal treatment consisted of 100% wogonin with a predicted 52.17% cell viability rate (Table 5 and Supplemental Figure 4). Although a combination of berberine and wogonin was not most effective, more research should be conducted to further determine wogonin's ability to treat chemotherapy resistant PC. From our PC3 MDRSM results, a total of 16 optimal treatments were combinations and 19 optimal treatments were single compounds; the single compounds frequently predicted less PC3 reduction than combinations (Table 4). All 35 three-compound combinations were run in a DU-145 cell line as well, but the MDRSM results showed that overall single compounds were more effective than combinations (Supplemental Table 1 and Supplemental Figure 3). The top six most optimal treatments against DU-145 cells were a single compound and the seventh most optimal treatment was barely a combination with just a small fraction of silybin combined with an overwhelming majority of curcumin (Supplemental Table 1 (row G)). The next three optimal treatments were also a single compound; overall, the top 10 optimal DU-145 treatments essentially contained just a single compound. Curcumin was the most effective individual treatment for DU-145 cells (Supplemental Table 1). Further research needs to be done to determine if curcumin is a viable treatment option for patients with prostate cancers similar to the isolated metastatic DU-145 cell line. Due to this lack of mixture potency, tests on the DU-145 line were not continued in this research. In contrast to the DU-145 cell line, 56 out of the top 10 optimal treatments for LNCaP and PC3 cells were combinations (the top two combinations for LNCaP cells were excluded due to lack of fit).
To further show how PC3 and LNCaP cells responded to the seven compounds, we summarized the number of times each compound appeared as part of the optimal treatment from Tables 3 and 4 into Table 6. The total number of compounds in all the optimal treatments is 96 and comes from counting every time a compound is a part of the predicted optimal treatment. There were 29 two-compound optimal treatments and 38 single compound optimal treatments which results in 96 compounds participating as part of the optimal treatments. Table 5 shows how frequently any given compound contributed to the optimal treatment indicating the potential of each compound at reducing hormone-responsive and non-responsive metastatic PC. While in combination, berberine contributed the most to the optimal treatments; berberine contributed to 21.88% of all optimal treatments. The next highest contributor, silybin, was found in the optimal combinatorial treatments 11.46% of the 96 combinations.
Wogonin contributed 10.42% to all optimal treatments when in combination. Although emodin was found alone most often in the optimal treatment, it also contributed to 9.38% of all optimal treatments when in combination. Due to these four compounds contributing frequently to the optimal treatment when in combination, we continued using these compounds and combined them with docetaxel to further explore treatment options.

| Determining optimal ratios of two compounds
Based on the results above showing that some compounds were quite effective in combination, we tested combinations of berberine, silybin, wogonin, and emodin to see if they would work in tandem with docetaxel treatment. Before testing combinations of these four compounds with docetaxel, we determined optimal two-compound combination ratios (between berberine, silybin, wogonin and emodin) through AlamarBlue cell viability tests. The two-compound combination ratios could not be taken directly from the MDRSM ratios above because each ratio would have a third confounding variable. To determine the optimal two-compound combination ratios, we tested (1) compound 1 at its IC50 value, (2) compound 2 at its IC50 value, (3) compound 1 at 50% of its IC50 value in tandem with compound 2 at 50% of its IC50 value, (4) compound 1 at 75% of its IC50 value in tandem with compound 2 at 25% of its IC50 value, and lastly (5) compound 1 at 25% of its IC50 value in tandem with compound 2 at 75% of its IC50 value (Figures 3 and 4).
The results for the PC3 and LNCaP cell lines showed some statistical differences between treatment ratios, but a single statistically effective ratio could not be determined as several of the mixtures decreased cell viability. Because there was not a statistically best ratio, we decided to continue tests on highly reductive ratios for each of the two-compound combinations run. The most reductive treatment to PC3 cells consisted of a 75:25 combination of emodin: silybin which resulted in only 52.09% cell viability ( Figure 3A). We see that 50% berberine and 50% silybin caused reduction of PC3 cell viability to 57.46% ( Figure 3B). A 25:75 berberine:wogonin ratio resulted in PC3 cell viability of 58.32% ( Figure 3C). Lastly, treatment with 50% emodin and 50% berberine caused only 58.63% of cells to remain viable ( Figure 3D). Interestingly, for each of the four combinations listed above, the analyses shows that at least one of the compounds alone was not statistically different than both in combination. Since we were hoping to reduce the amount of each compound administered in an effort to reduce toxic side effects, we focused on two-compound combinations even if not always statistically significant. Studies on the following combinations were run on PC3 cells with docetaxel treatment: 75% Em and 25% Sy, 50% BB and 50% Sy, 50% BB and 50% Wo, and 50% Em and 50% BB. These four combinations collectively will be referred to as the PC3-4 from now on.
Similar tests were run to determine the optimal ratio of these combinations in LNCaP cells. Our tests confirmed that the % LNCaP cell viability was much more reduced by these bioactive compounds than PC3 cells. Our most reductive combination consisted of 25:75 berberine: silybin which allowed only 9.95% cell viability ( Figure 4A).
Surprisingly, a 25:75 ratio of emodin: berberine reduced LNCaP cells allowing only 10.40% to survive and function ( Figure 4B). This is interesting when compared to the MDRSM data which shows that a 66.9: 33.1 emodin: berberine ratio was most effective within a 3 compound mixture of berberine, emodin, and triptolide ( silybin combination resulted in a 11.66% LNCaP cell viability ( Figure 4C). These results show some potential for treating different stages of prostate cancer with the same combination ratios of emodin and silybin. Lastly, we found that a 50:50 berberine: wogonin combination was also highly reductive allowing only 12.89% cell viability although this was not statistically different than other berberine and wogonin combinations ( Figure 4D). Since there was not one statistically optimal ratio for each of the four compound combinations, we continued tests on the four compound combination ratios that resulted in the lowest average % cell viability. For the LNCaP cell line, this included: 25% BB and 75% Sy, 25% Em and 75% BB, 75% Em and 25% Sy, and 50% BB and 50% Wo. These four combinations collectively will be referred to as the LNCaP-4 from now on.   ( Figure 6D). These results are more important to consider based on previous studies showing that wogonin significantly reduced LNCaP cells but not prostate epithelial cells. 36 This group, which will be called the LNCaP-Doc4, includes: 75% Em and 25% Sy, 25% Em and 75%
To further determine the mechanistic actions of these com- We tested the assumption that the combination of three compounds would be more effective than a single compound alone. Contrary to what we hypothesized, none of the combinations' response surface analyses predicted an optimal treatment that included all three compounds. This may have been caused by using the IC50 concentrations as the full dose treatment because MDRSM calls for fractions of the chosen full dose (i.e., 50%, 33.33%, or 16.67% of the IC50) ( Figure 2), and these lower concentrations of the compound may not be biologically effective. This problem is more likely to occur with the compounds that have a stepwise shape to their IC50 curve (Figure 1).
Perhaps choosing a higher drug concentration for the 100% treatments, that is points 1-3 in Figure 2A, would ensure the concentrations used at the subsequent mixture points, points 4-10 in To further assess the effectiveness of berberine, wogonin, emodin, and silybin, we tested highly effective two-compound combinations in tandem with docetaxel. From our results, we see combinations of emodin and silybin, berberine and silybin, emodin and berberine, and berberine and wogonin are more effective than docetaxel alone. When used with docetaxel these combinations reduce Note: The compound abbreviation column represents the number of times the compound was the optimal treatment alone. The (+) column is for the number of times the compound contributed to an optimal treatment. *Of the 70 unique treatments 29 optimal treatments were combinations, each only using two compounds, the other 38 optimal treatments are a single compound leaving (29*2) + 38 = 96 total.
docetaxel by as much as 75% without decreasing the chemotherapeutic efficacy. The reduction of docetaxel could reduce the nausea, diarrhea, mouth sores, and hair loss that are frequently reported as side effects of docetaxel treatment. There appears to be a characteristic in the metastatic androgen-dependent cells that enhance their susceptibility to the two-compound combinations. Also, this attribute seems to have has also been noted for additional anti-cancer effects in many in vivo studies against various cancer types and for inducing apoptosis through increasing reactive oxygen species (ROS). 8,41 Wogonin has been shown to increase p53, PUMA, Bax, and cytochrome C release from the mitochondria leading to apoptosis. Wogonin also has been reported to modulate several signal transduction pathways including inhibiting the Akt pathway to suppress tumor growth. 16  Our study also indicates that compound combinations are as effective/more effective than docetaxel. In the androgen independent PC3 cell line, a combination of 37.5% BB, 37.5% Wo and 25% Doc reduced cell viability at an average of 53%. This treatment combination was just as effective as docetaxel treatment alone. After a patient's PC becomes androgen independent and progresses on or after a 2nd generation AR inhibitor, the patient is then usually treated with chemotherapy such as docetaxel. We propose further research on this combination of docetaxel, berberine and wogonin as a treatment option for androgen-independent patients before they receive full dosages of chemotherapies.
In the androgen-dependent LNCaP cell line, all 4 two-compound combinations significantly decreased cell viability in comparison to docetaxel. Specifically, 75% Em and 25% Sy reduced LNCaP cells almost 30% more than docetaxel treatment ( Figure 6A). These 42-compound combinations should be tested further as potential treatment for PC patients while they are using androgen deprivation therapies and before they become androgen-independent. Lastly, our studies show

| IC50 value calculation and statistical analysis
We used GraphPad Prism 8 (Graphpad Software, San Diego, CA, USA) to calculate IC50 values for each of the 7 compounds used using gradation treatment and following the cell viability method stated in section 4.3. Using concentrations above and below the IC50 concentration, a variable slope non-linear regression model was fit to the experimental results (r 2 > 0.95). The concentration which resulted in a 50% reduction of cell viability was set as the IC50 for each compound based on at least 3 biological runs. We also used GraphPad Prism 8 to run statistical analyses on Figures 3-6. The data was first tested for outliers using the ROUT method with a Q = 10%. Then the data was analyzed to determine statistical significance using the Ordinary oneway ANOVA, Tukey's multiple comparisons test (Figures 3-6). MDRSM uses the experimental data from these 10 points to build a statistical model that predicts the optimal combination of the chosen compounds.
We analyzed the compound combinations using JMP Pro15 software (SAS Institute, Cary, NC, USA). ABCD mixture design was used for factor analysis and generation of the ternary plots. The methods followed Oblad et al methods. 34 We used the same simplex lattice augmented with four additional points resulting in the 10 experimental points; by using the least-squares method, coefficients were estimated for use in the quadratic mixture model. Each 96-well plate had the 10 mixture treatments and was normalized to a DMSO vehicle control which was set at 100% viability.

| Flow cytometry
Cells were plated on 6 well plates and allowed to adhere for a 24-hour period. Cells were then treated. Each 6-well plate had a DMSO vehicle control that was set to 100% viability with which the other treatments were normalized to; each plate also had a cell-free