Inter-observer variability of clinical target volume delineation for bladder cancer using CT and cone beam CT


  • F Foroudi,

    Corresponding author
    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 1 A Haworth,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 2 A Pangehel,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 3 J Wong,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 3 P Roxby,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 2 G Duchesne,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • 1,2 S Williams,

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author
  • and 1 KH Tai 1

    1. 1 Division of Radiation Oncology, 2Physical Sciences and 3Radiation Therapy Services, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
    Search for more papers by this author

  • F Foroudi MB BS, MPA, FRANZCR; A Haworth BSc, PhD; A Pangehel RTT; J Wong RTT, MSc; P Roxby MA, MSc; G Duchesne BSc, MD, FRANZCR; S Williams BSc, MB BS, FRANZCR; KH Tai MB BS FRANZCR.

  • Conflicts of interest: None.

Dr Farshad Foroudi, Division of Radiation Oncology, Peter MacCallum Cancer Centre, St Andrews Place, East Melbourne, Vic. 3002, Australia.


To compare the image quality of cone beam CT (CBCT) with that of planning CT (pCT) scan, and quantify inter-observer differences in therapeutic indices based on these scans prior to the introduction of an adaptive radiation therapy protocol for bladder cancer. Four consecutive patients were selected with muscle invasive bladder cancer receiving radical dose radiation therapy. Four radiation oncologists specializing in genitourinary malignancies contoured the clinical target volume (CTV) and rectum on both a pCT and a randomly chosen CBCT of the same patient. A conformity index (CI) for CTV and the rectum was determined for both pCT and CBCT. The maximal lateral, anterior, posterior, cranial and caudal extensions of the CTV for both CT and CBCT were determined for each observer. Variation in volumes of both the CTV and rectum for both pCT and were also compared using Varian Eclipse planning software (Varian Medical Systems, Palo Alto, CA, USA). Using pCT the mean CI for the CTV was 0.79; using CBCT the mean CI for the CTV was 0.75. For the rectum, the mean CI for using CT was 0.80 and for CBCT was 0.74. Greatest variation on CBCT CTV contours was seen in the supero-inferior direction with variation up to 2.1 cm between different radiation oncologists. With the variation in CI for pCT and CBCT of the CTV and rectum (0.04 and 0.06 respectively), CBCT is not significantly inferior to the pCT in terms of inter-observer contouring variability.


The urinary bladder and malignancies arising from it are soft tissue structures that are poorly defined on traditional megavoltage (MV) portal imaging. Due to limited contrast between soft tissue structures, bony anatomy has previously been the standard for verification of patient position at the time of bladder cancer treatment. However, bony anatomy is well known to be a poor surrogate for the position of soft tissue structures that may vary in shape and size during the course of radiotherapy treatment. Even with large treatment margins, it has been suggested that poor bladder cancer control rates in the past may be a result of a geographic miss of tumour volumes.1,2

As cone beam computed tomography (CBCT) is becoming available on many modern linear accelerators, it is possible to localize the bladder and determine its shape, size and position with the patient in the treatment position immediately prior to treatment. Such information at the treatment unit potentially allows daily ‘online’ positioning of the fields based on bladder position as well as adaptation of treatment fields based on bladder size. Such ‘adaptive’ radiotherapy has typically been used to refer to the use of information from previous treatments (i.e. ‘offline’) to modify subsequent treatments.3 While historically the term ‘adaptive’ has been applied to positional adjustments, as systems become more robust, field size and shape adjustments are also made possible.

Introducing such techniques into standard clinical practice requires rigorous quality assurance procedures to ensure safety and efficacy. One goal of an adaptive treatment regimen may be to reduce treatment margins, but this carries with it a risk of under-dosing the tumour. Therefore, a good understanding of the uncertainties in treatment delivery is required to ensure adequate treatment margins are applied. Furthermore, image quality from CBCT needs to be of adequate standard to enable rapid decision-making in relation to soft tissue position and appropriate selection of treatment plan dependant on organ size. Image quality is important when using an adaptive treatment planning protocol that requires acquisition of daily CBCT images over a small number of fractions to create a series of adaptive plans for subsequent treatment fractions.4 Thus, prior to commencing an online adaptive radiotherapy protocol for muscle invasive bladder cancer, a review was carried out of the image quality of the CBCT for patients undergoing radiotherapy for muscle invasive bladder cancer. The primary hypothesis for this study was to confirm that that there is no significant difference in contouring between the planning CT and CBCT.


Four consecutive muscle invasive bladder cancer patients who were being treated with radical intent were selected for review as part of a Peter MacCallum Cancer Centre ethics committee approved pilot study. The protocol involved five daily CBCT scans prior to radiation therapy in the first week of treatment then weekly CBCT. Patients varied in age between 58 to 75 years, with three males and one female. Histology was grade 3 TCC in three patients (T2-T3) and one limited stage small cell cancer of the bladder. All patients were node negative, did not show evidence of distant metastases and treated to 64 Gy in 32 daily fractions. Exclusion criteria for the study included prosthetic hip replacements, previous radiotherapy and pregnancy.

All patients were imaged and treated on a Varian Trilogy Linear accelerator (Varian Medical Systems, Palo Alto, CA, USA) using the kV imaging system to acquire CBCT images.

System description

CBCT is a 3-D imaging modality that has recently become available on linear accelerators for radiotherapy treatment.5–7 The Varian CBCT system used for this study consists of an x-ray tube and an amorphous silicon flat panel detector with the imaging beam orthogonal to the treatment beam. The CBCT dataset was acquired during a rotation of 360o of the gantry and a copper filter (thickness 0.15 mm) was mounted on the x-ray tube side of the bow tie filter. Images were acquired using standard factory settings of 125 kVp, 80 mA and 20 ms per projection. Images were reconstructed at an axial slice thickness of 0.25 cm. Treatment planning CT (pCT) scans were obtained using the diagnostic quality Phillips Brilliance wide bore CT scanner (Phillips Medical Systems, Best, the Netherlands). The setting for the treatment planning CT was 3 mm slice thickness, 140 kVp, and variable mAs depending on patient size.

Inter-observer variation

All images were reviewed by four radiation oncologists specializing in pelvic malignancies. The pCT and CBCT images were loaded into the Eclipse planning system (Varian Medical Systems, Palo Alto, CA, USA) for contouring. For each patient, the treatment pCT and one arbitrarily selected CBCT was selected for contouring by all observers. For each scan, the clinical target volume (CTV) and rectum were contoured, with all radiation oncologists blinded to the others’ contouring. Patient information was provided using a standardized form with cystoscopic and pathological details. Once the CTV was drawn by the radiation oncologist, a planning target volume (PTV) was created using a 2 cm uniform expansion of the CTV in Eclipse.

Contouring guidelines

The standard contouring guidelines for the department were provided, which state as follows:

clinical target volume (CTV) should include the whole bladder, the tumour bed region, the proximal urethra and, in the male patient, if there is involvement of the bladder neck and/or prostatic fossa, the entire prostatic urethra is to be contoured. Any extra-vesical extension (e.g. into peri-vesical fat) should be included into the CTV. No attempt should be made to cover nodal groups including first echelon lymph nodes (obturator) in the immediate vicinity of the bladder. Unusual anatomical variations (e.g. cystoceles and diverticula) should be covered. In cases involving the ureteric orifice, the distal ureter should be covered within the CTV to a distance of 1 cm.

The rectum was contoured from the inferior field edge to an upper limit of either the field edge or at the anatomical level where the rectum curves into the sigmoid colon; these levels were provided to each observer to ensure consistency.

Scoring of image quality

The four radiation oncologists evaluated image quality for the purpose of contouring the bladder and rectum for 20 CBCT. The quality of the CBCT scans was evaluated with respect to the ease of contouring the bladder and rectum, using a scale of 0 to 10. The radiation oncologists were instructed to use ‘0’ as extremely poor and ‘10’ as a perfect image.

Patient factors impacting on image quality

The anterior-posterior (AP) separations of the four patients were measured using the Varian Eclipse treatment planning system to investigate the relationship between patient separation and image quality.8

Inter-observer variability in shape and volume

Using the Varian Eclipse treatment planning computer, the dimensions of all delineated CTV and rectums from the pCT and CBCT were determined using the measure function and axial and sagittal planes that intersected the isocentre. Similarly, the planning computer tools were used to determine the volumes of all structures.

The inter-observer variability between pCT and CBCT was assessed by calculating the conformity index (CI) and maximum variability ratio (MVR) using the individual radiation oncologists’ contours. CI describes the variability between contours in terms of overlap, size and position.9 The CI is defined as the ratio of overlapping volumes (common volumes) and the total encompassing delineated volume of all contours (total volume). The CI has a value of 0 to 1 with a result of 1 indicating a 100% agreement of delineated volumes (the overlapped volume is identical to the total encompassed volume). A result of 0.5 indicates observers agree on only 50% of the total delineated volume (common volume is equal to half of the total volumes). The MVR of the CTV is the ratio of the maximum and minimum CTV.8 If the results show no significant difference in CI and MVR, then both pCT and CBCT have similar inter-observer contouring variability. The total encompassing delineated volume and the overlapping volume between the oncologists’ contours was calculated using the Eclipse planning system Boolean function and in an Excel (Microsoft, Redmond, WA, USA) spreadsheet.


CBCT scoring

The four observers scored image quality for all four patients scans. For the pCT scan, the average scores for each of the patients were 9.0, 8.75, 9.0 and 9.25. The corresponding CBCT images had mean scores of 5.1, 5.5, 4.7 and 5.5. (see Table 1). With reference to the scoring scale, the CBCT image can be classified as adequate (score of 4) to good (score of 6) while the pCT images are very good to excellent. This indicated that the image quality of CBCT is inferior to the pCT in terms of subjective review by radiation oncologists.

Table 1.  Planning CT and cone beam CT image quality score by four radiation oncologists and patient separations in cm. A score of 0 is extremely poor and a score of 10 is a perfect image
PatientMean pCT scoreMean CBCT scoreAP separationLAT separation
  1. AP, anterior-posterior; CB, cone beam CT; LAT, lateral.

Patient size and CBCT image quality

We found that CBCT image quality may have a relationship with the size of the patient as measured by the AP separation (see Table 1). The patient with the largest AP separation (patient 3) had the lowest image quality for the CBCT (4.7). This indicated that an increase in AP separation may result in a decrease in image quality as graded by the observers. The other three patients had relatively close AP separations (18.3–20.9) and their corresponding scores were between 5.1 and 5.5. Figure 1a shows the overlapping contours on the axial CBCT and 1b the corresponding pCT views for patient 2. Similarly, Figure 2 shows the delineated volumes on the CBCT for the same patient.

Figure 1.

a) Cone Beam CT image. b) Corresponding planning CT image with superimposed CTV.

Figure 2.

a) Interobserver CTV variation based on planning CT. b) Interobserver rectal variation based on planning CT. c) Interobserver CTV and rectal variation based on CBCT in the same patient.

CTV variability for bladder cancer

Conformity index (CI)

The CI was calculated for four structures: the pCT-based CTV and rectum, and the CBCT CTV and rectum (Table 2).

Table 2.  Conformity index scoring showing conformity index between the four radiation oncologists for both planning CT and cone beam CT images
 Radiation oncologistsCI pCT CTVCI CBCT CTVCI pCT rectumCI CBCT rectum
  1. CI, conformity index; CBCT, cone beam CT; CTV, clinical target volume; pCT, planning CT.

Patient 1A; B0.770.780.820.78
A; C0.780.810.810.66
A; D0.840.80.780.53
B; C0.760.70.80.68
B; D0.810.770.810.8
C; D0.790.760.820.65
Individual mean 0.790.770.810.68
Patient 2A; B0.620.760.760.72
A; C0.710.70.670.69
A; D0.860.80.770.73
B; C0.740.760.740.72
B; D0.810.810.830.74
C; D0.720.750.710.73
Individual mean 0.740.760.750.72
Patient 3A; B0.810.720.800.81
A; C0.810.770.830.84
A; D0.820.780.790.78
B; C0.780.740.750.81
B; D0.790.750.770.71
C; D0.760.740.760.78
Individual mean 0.800.750.780.79
Patient 4A; B0.820.740.870.78
A; C0.80.760.840.85
A; D0.870.660.840.75
B; C0.80.760.860.77
B; D0.840.690.880.81
C; D0.780.630.840.74
Individual average 0.820.710.860.78
Overall median 0.800.760.810.75
Overall average 0.790.750.80.74

Table 3 shows the variation in dimensions of the CTV along each of the axes that pass through the treatment isocentre for the pCT and CBCT scans for each patient. The maximal variation being in the superior-inferior (z) axis and minimum being in the left-right (x) axis. The variation in the ranges of volume for the CTV using the planning CTV and CBCT is shown in Table 4. The variation in CTV volumes was not greater using the CBCT compared to the pCT. Using the pCT the mean CI was 0.79, while using CBCT the mean CI was 0.75 (Table 3). There is only a difference of 0.04 between pCT and CBCT scans.

Table 3.  Mean variation in clinical target volume dimensions measured for each of the four patients
  1. AP, anterior-posterior; CBCT, cone beam CT; CTV, clinical target volume; LR, mediolateral; SI, superior-inferior. Maximum–minimum range (SD). All dimensions in cm.

16.9–7.7 (0.8)6.5–7.3 (0.8)4.2–5.4 (1.2)8.3–8.8 (0.5)6.8–7.2 (0.4)4.5–5.6 (1.1)
25.4–6.5 (0.9)5.9–7 (1.1)2.5–3.3 (0.8)5.3–6.2 (0.9)5.9–6.5 (0.6)3.3–3.7 (0.4)
310.5–11 (0.5)7.1–7.6 (0.5)6.2–7.4 (1.2)9.3–10.9 (1.6)6.2–6.8 (0.6)6.2–7.3 (1.1)
48.3–9 (0.7)8.5–9.4 (0.9)9.2–11.4 (2.2)6.7–7.8 (1.1)6.4–7.2 (0.8)5.7–7.8 (2.1)
Table 4.  Maximum–minimum variation in clinical target volume for each of the four observers (SD)
  1. CBCT, cone beam CT; CTV, clinical target volume. All volumes in cm3.

1120.5–144.3 (10.8)167.2–206.5 (18.4)
246.1–63.4 (8.0)55–72 (7.7)
3192.4–203.9 (6.2)166.5–177.1 (4.7)
4351.1–441.4 (41.6)111.3–169.9 (24.0)

Figure 2a shows inter-observer variation based on pCT while Figure 2c shows inter-observer variation based on CBCT.

Maximum variation ratio (MVR)

The MVR indicates the factor difference between the maximum and minimum.10 Within our study, for pCT CTV, the range of MVR is from 1.06 to 1.38 (mean 1.23) compared to CBCT MVR from 1.06 to 1.53 (mean 1.29). The absolute difference of the mean is 0.06 between pCT and CBCT (Table 5).

Table 5.  Planning CT (pCT) and cone beam CT (CBCT) maximum variation ratio (MVR) on clinical target volume (CTV)
Patient CTV maximumCTV minimumCTV averageMVR
  1. All volumes in cm3.



Table 6 shows considerable inter-observer variation in delineated rectal volumes. This is mostly a result of differences in length of the rectum contoured. The analysis was therefore repeated but with the rectum contours modified to have the same length between different oncologists. Using the modified rectal volume delineations, for the pCT the mean CI was 0.80, while using CBCT the mean CI was 0.74. Using the original rectal volume delineation (with different lengths), for the pCT the mean CI was 0.64, while using CBCT the mean CI was 0.58. Figure 2b shows inter-observer variation in contouring of the rectum based on CT while Figure 2c shows variation based on CBCT.

Table 6.  Variation in rectal volumes on CT and cone beam CT
 pCT rectum vol. 11CBCT rectum vol. 11
  1. CBCT, cone beam CT; pCT, planning CT. All volumes in cm3.

Pt 144.2–52.836.8–66.9
Pt 232–101.848.4–109.2
Pt 346.9–72.764–86.1
Pt 4107.8–160.191–180.4


An accepted way of examining the quality of imaging is through an inter-observer study. The first inter-observer study in radiotherapy was reported in 197712 and since then many such studies have been published.10,13–15 Logue et al. reported a pCT-based inter-observer study of bladder cancer contouring finding that variation was less when the whole bladder and a margin was contoured compared to a tumour and a margin.10 As whole bladder coverage is our current practice, this inter-observer study was based on contouring the whole organ.

Cone beam CT has the potential to provide information for accurate patient positioning on the treatment couch based on soft tissue organ position. It also allows for tailoring treatment field to organ size and shape on a daily basis. Such online adaptive treatment protocols, however, will only be effective if image quality is of an adequate standard for quick and reproducible identification of target structures. In the diagnostic radiology practice it is well established that a full bladder and the use of contrast improves delineation of the organ.16 Unfortunately in radiation therapy an empty bladder is required when irradiating the whole bladder to reduce the volume irradiated. Use of contrast is not feasible on a daily basis throughout treatment.

Challenges noted for bladder CTV delineation included partial volume effects caused by proximity of organs (e.g. superior border of the bladder and small bowel). Image quality at the interface between the bowel and bladder, particularly over the superior aspect of the bladder, may be deleteriously affected as a result of motion artifact due to bowel peristalsis given the long time to acquire images compared to diagnostic CT. This may also have contributed to the widest variation in the superior-inferior (z axis) direction of the CTV contoured by the radiation oncologists. This is of particular concern as the bladder expands anteriorly and superiorly; which may make the choice of a bladder plan difficult for radiation therapy staff on a daily online basis.17 An online adaptive bladder protocol will require radiation therapists to make decisions based on CBCT, thus, future studies confirming the therapist’s ability to make reliable decisions based on CBCT would be useful.

The image quality score for pCT was 9 and for CBCT was 5. We found that while the CBCT image quality was not of diagnostic (CT simulator) quality, the images were of sufficient quality to be used for daily image guidance as part of an online adaptive protocol for bladder carcinoma. The absolute difference on CTV in CI was 0.04 and on MVR was 0.06.

For the rectum, the CI values were 0.80 versus 0.74 for pCT and CBCT respectively. The absolute difference in CI is 0.06, which is slightly inferior to the CTV value. Again, the 0.06 difference showed that the variability in CBCT between observers is not considered significantly lower than pCT.

While the inter-observer variation between CT and CBCT did not vary significantly, it is interesting to note that there was such variation despite all the radiation oncologists working in the same department and having the same contouring guidelines, particularly when using the pCT. One limitation of the study was the small number of patients with bladder cancer available to carry out the inter-observer study. A limited sample size such as with the current study means that it is exploratory in nature and requires larger studies to conclusively prove a null hypothesis between inter-observer variation on pCT and CBCT. Even with small numbers, inter-observer studies have shown such a variation in delineation of CTV on pCT, including whole bladder CTV delineation.10 Inter-observer studies are the commonly accepted measure of examining new imaging technology, such as CBCT. Clearly, the ability to set a gold standard of pathological correlation is difficult using radiation therapy treatment room imaging equipment. Hence, there remains a theoretical risk that while the observers are contouring similarly with the CBCT they are systematically contouring an ‘incorrect volume’.

The purpose of this study was to asses CBCT image quality for the purpose of using these images in an adaptive bladder cancer treatment protocol. Such a study is one of the first steps in evaluating new imaging technology for implementation into clinical protocols. Similar measures of agreement in shape, volume and position of the bladder on the CT and CBCT images between the experienced radiation oncologists suggests the images are of adequate quality.


Cone beam CT can provide useful soft tissue information in radiotherapy. Although the image quality is inferior when compared to conventional pCT, it still offers sufficient soft tissue information for online adaptive radiotherapy for bladder cancer. This study confirms that the conformity index for CBCT is not significantly inferior to that of conventional pCT in the contouring of the CTV and rectum contouring by radiation oncologists in radical radiotherapy of muscle invasive bladder cancer.


The Victorian Cancer Agency for providing pilot study funding.