Validation of a secondary dose check tool against Monte Carlo and analytical clinical dose calculation algorithms in VMAT

Abstract Purpose Patient‐specific quality assurance (QA) is very important in radiotherapy, especially for patients with highly conformed treatment plans like VMAT plans. Traditional QA protocols for these plans are time‐consuming reducing considerably the time available for patient treatments. In this work, a new MC‐based secondary dose check software (SciMoCa) is evaluated and benchmarked against well‐established TPS (Monaco and Pinnacle3) by means of treatment plans and dose measurements. Methods Fifty VMAT plans have been computed using same calculation parameters with SciMoCa and the two primary TPSs. Plans were validated with measurements performed with a 3D diode detector (ArcCHECK) by translating patient plans to phantom geometry. Calculation accuracy was assessed by measuring point dose differences and gamma passing rates (GPR) from a 3D gamma analysis with 3%–2 mm criteria. Comparison between SciMoCa and primary TPS calculations was made using the same estimators and using both patient and phantom geometry plans. Results TPS and SciMoCa calculations were found to be in very good agreement with validation measurements with average point dose differences of 0.7 ± 1.7% and −0.2 ± 1.6% for SciMoCa and two TPSs, respectively. Comparison between SciMoCa calculations and the two primary TPS plans did not show any statistically significant difference with average point dose differences compatible with zero within error for both patient and phantom geometry plans and GPR (98.0 ± 3.0% and 99.0 ± 3.0% respectively) well in excess of the typical 95%clinical tolerance threshold. Conclusion This work presents results obtained with a significantly larger sample than other similar analyses and, to the authors' knowledge, compares SciMoCa with a MC‐based TPS for the first time. Results show that a MC‐based secondary patient‐specific QA is a clinically viable, reliable, and promising technique, that potentially allows significant time saving that can be used for patient treatment and a per‐plan basis QA that effectively complements traditional commissioning and calibration protocols.

mended that PSQA is performed routinely 11,12 for VMAT treatment plans, in order to detect any potential error due for example to inaccurate calculation of the dose distribution by the treatment planning system (TPS) or failure of record-and-verify system, as well as to inaccurate MLC movements. 13,14 Typically, QA protocols compare the dose distribution planned by the TPS with the dose delivered to a homogeneous water-equivalent phantom that contains detectors. 10,15 More specifically, in the pretreatment patient-specific VMAT QA, dose measurements are usually carried out either at the reference point with a small volume air-filled ionization chamber 16 or with 2D devices like film dosimeters 17,18 or 2D detectors like electronic portal imaging devices 19,20 and arrays of ion chambers. 21,22 However, all these methods are not optimal. A single point measurement is insufficient for the verification of the complex dose distributions of VMAT plans. Film dosimetry has a good resolution but requires a time-consuming readout system. Electronic 2D detectors have a rapid response but are usually limited by their low resolution. 23 Gel and plastic dosimeters 24,25 have been developed to enable a full 3D dose verification matching more accurately the patient geometry. Unfortunately, such methods require time-consuming procedures and significant human resources, which are not practical for busy treatment centers. 26,27 Moreover, instabilities caused by storage procedure and manufacturing processes, and limitations in repeated usage have been reported. 28,29 More recently, systems based on 2D measurements that allow pseudo-3D dose reconstruction have been proposed to overcome these limitations. 30 Conventional PSQA procedures are generally not optimal for busy radiation therapy centers, as, typically, data collection and verification of the dose distributions are time-consuming for the clinical staff. 31,32 Moreover, machine time needed by the phantom-based measurements is subtracted to the available patient treatment time.
In its Report 83, 33

2.A | The SciMoCa software package
The main SciMoCa algorithm has been described in detail by Hoffmann et al. 47 It exploits the source modeling concept 48,49 to develop a clinical beam model specific for each treatment machine. SciMoCa is able to reconstruct 3D dose distributions from the CT dataset associated with the plan using the DICOM image datasets, the RT structure set, and the RT plan information returned by the TPS. The user is free to select the grid resolution (minimum 0.5 mm, maximum 10 mm per dimension), the statistical uncertainty (0.5%, 1%, 1.5%, 2%) and dose-to-water or dose-to-medium calibration. In general, in MC-based algorithms the statistical uncertainty controls the level of statistical noise remaining within the final calculation. A decrease in PIFFER ET AL. | 53 the statistical uncertainty value leads to an increase in the number of simulated histories, resulting in a lower level of statistical noise present in the computation. It is therefore assumed to understand that this factor is related to the dose calculation accuracy and calculation time.

2.B | Linac calibration in SciMoCa
An Elekta Synergy Linac equipped with Elekta Beam Modulator MLC, with 80 leaves 4 mm wide at isocenter, was selected for this study. To model the accelerator head, SciMoCa has been commissioned using the same set of measurements used to commission the reference TPS (Monaco, version 5.11.02, by Elekta, Stockholm, Sweden, and Pinnacle 3 , version 9.10, by Philips Radiation Oncology Systems, Fitchburg, USA). A similar procedure was followed to load the Hounsfield Units to mass density calibration curve, obtained using 14 materials ranging from air to aluminum including six organic compound types (lung, adipose, breast, brain, liver, and bone). The 6MV beam delivered by Synergy has been commissioned on the basis of 11 depth dose curves and cross-profiles measured at five depths

2.C.2 | Patient and phantom plans
A CT scan with 3 mm slice thickness was used for all VMAT plans. The contours were drawn manually by expert physicians and the planning goals were considered achieved when the prescribed dose covered to at least 95% of the target volume. Collimator angles of 0°or 3°, leaf motion constraints of 0.2 mm per degree of gantry rotation, one control point every 2°and a minimum segment size of 2 cm 2 have been set for Pinnacle 3 TPS. The parameters used for Monaco TPS were 21 mm maximum leaf travel per second, 5.5°maximum gantry travel per second, 256 control points and a minimum segment width of 1 cm. The same calculation parameters have been set for SciMoCa and reference TPS; in particular, a grid size of 2 mm (in all directions) has been used. For the two systems based on MC algorithms, the dose has been reported as dose-to-medium, and a statistical uncertainty of 0.5% has been selected.
To compare the software also with direct measurements, each of the 50 treatment plans was translated into a phantom verification plan. The verification plans were created by transferring with

2.D | Comparison of SciMoCa and primary TPS plans
In order to validate the accuracy of SciMoCa second-check dosimetry system, the obtained results were checked both against the TPS plans and direct measurements. At present, only partial sets of clinical action levels and/or tolerance guidelines are available for Sci-MoCa calculations. Therefore, the two most commonly used metrics were applied in the comparison tests: the isocenter point dose difference and the gamma analysis, following well established PSQA action levels in published Ref. [52] The relative isocenter point dose difference %D diff was calculated using the following equation: where D ref is the reference dose and D test is the evaluated dose. The action level chosen for %D diff was 3%, following well-established procedures. 7 The dose difference was averaged over each patient class (see Section 2.C.1) and the statistical significance of the difference between the means was assessed with the Mann-Whitney U test (p<0:05). The dose distributions were compared performing a gamma comparison and checking the gamma passing rate (GPR), assuming global normalization in absolute dose, dose difference ΔD ¼ 3% and distance to agreement DTA = 2 mm, with a low-dose region exclusion threshold of 10% (100% is the maximum dose) as recommended for rotational IMRT QA in Ref. [10] Following the protocol adopted in our department, a plan was considered acceptable if GPR was above a tolerance level of 95%. 53 The action level was set at 9% based on clinical experience. The average, the standard deviation and the maximum and minimum obtained values over each patient class were calculated for each metric. Several authors suggest that the degree of modulation is one of the parameters that best describes complexity of VMAT treatments and has an impact on the precision and accuracy of beam delivery. This could be quantified by the gamma comparison [54][55][56] . As for %D diff , the reference and the test doses were the measured and calculated doses, respectively. The agreement between the measured and software-calculated dose distributions was instead evaluated with SNC Patient software (v6.4.1, Sun Nuclear Corporation, Melbourne, FL, USA). 59 The measured dose distributions were selected as reference set, whereas software-calculated dose distributions as evaluation set.

3.B | Comparison against measurements
The results of the validation of SciMoCa and the two reference TPSs against direct measurements are summarized in Table 1

3.C | Comparison between SciMoCa and primary TPSs
The results of the comparison of SciMoCa with the two primary TPSs for patient and phantom plans are summarized in Table 2. The average, the standard deviation, and the maximum and minimum measured values over each patient class are reported for each T A B L E 1 Calculations vs. measurements. Gamma analysis results of validation of SciMoCa and TPS plans against dose measurements. The average, the standard deviation and the maximum and minimum measured values over each patient class are reported for each metric. Average values over Monaco and Pinnacle 3 plans and for the full dataset are also given.  The full set of measurements is reported in Fig. 4 where the 2D scatter plot of %D diff between SciMoCa and TPS patient and phantom plans is shown. A good agreement is observed between the two sets of calculations for both patient and phantom geometries (

3.D | Analysis of correlations with MCSv
The distributions of the VMAT modulation complexity scores MCSv for the plans of the two TPSs considered in this work are shown in Fig. 5, whereas Fig. 6

SUPPORTING INFORMATION
Additional supporting information may be found online in the Supporting Information section at the end of the article. and for the full dataset are also given.