University of New South Wales Cancer Research Centre, University of New South Wales, Sydney, Australia
Prince of Wales Clinical School, Sydney, Australia
University of New South Wales Cancer Research Centre, University of New South Wales and Prince of Wales Clinical School, Level 1, South Wing Edmund Blacket Building, Prince of Wales Hospital, Randwick, NSW 2031, Australia
Medical oncology is embracing information technology to standardize care and improve patient outcomes, with a range of Web-based systems used internationally. The authors' aim was to determine the factors affecting the uptake and use of a Web-based protocol system for medical oncology in the Australian setting.
The authors conducted 50 interviews and observed medical oncology physicians, nurses, and pharmacists in their treatment setting at 6 hospitals in different geographic locations.
The Web-based system plays a major role in guiding oncology treatment across participating sites. However, its use varies according to hospital location, clinician roles, and experience. A range of issues impact on clinicians' attitudes toward and use of the Web-based system. Important factors are clinician‒specific (eg, their need for autonomy and perceptions of lack of time) or environmental (eg, hospital policy on protocol use, endorsement of the system, and the availability of appropriate infrastructure, such as sufficient computers). The level of education received regarding the system was also found to be integral to its ongoing use.
Healthcare systems worldwide are embracing information technology (IT) to disseminate best practice evidence, guide treatment processes, improve patient outcomes, and reduce potentially avoidable medical errors.1-4 National and regional organizations are developing tools and promoting IT innovations across a wide range of medical conditions, and medical oncology is no exception.5-9
Cancer organizations such as the British Columbia Cancer Agency10; Cancer Care Ontario11; the National Comprehensive Cancer Network12; Macmillan Cancer Support, United Kingdom13; and the Cancer Institute New South Wales, Australia14 have developed online cancer information systems. Each of these Web-based systems incorporates a range of key features, such as evidence-based or consensus guidelines regarding the choice of specific treatment options, detailed instructions regarding the preparation and administration of chemotherapy (including dose calculations), guidance for monitoring treatment effects, and assistance for dose modifications. Patient-specific information concerning treatments and their side effects are also the primary focus or a significant component of most of these systems.
Some jurisdictions such as the Canadian provinces mandate the use of this online information, to ensure patients managed under their reimbursement schemes are treated according to best practice guidelines. However, the use of most of these online systems is voluntary. Therefore, their dissemination is a necessary but not sufficient condition to guarantee use.15 Consequently, education and promotion become integral to attracting and retaining end-users.
The Australian-based Cancer Institute Standard Cancer Treatments Program (CI-SCAT), called eviQ from September 2009, is a state-funded, public domain program of evidence-based treatment protocols and patient information. CI-SCAT is a nonmandated system focusing primarily on supporting the way in which care is implemented after treatment decisions are made (as opposed to a guideline advocating for the choice of a particular therapy). Key features of CI-SCAT are the incorporation of peer clinical opinion (from specialist physicians, nurses, pharmacists, and allied health practitioners) to ensure treatment protocols reflect achievable rather than idealized practice situations and timely updating of the protocols when new evidence emerges.
The website contains nearly 1000 protocols, encompassing medical oncology (n = 219), hematology (n = 163), nursing (n = 184), radiation oncology (n = 79), cancer genetics (n = 165), and supportive care (n = 90). Individual protocols provide detailed treatment information, including the indication for the protocol, dosing instructions, patient-specific dose calculations (this feature is password protected), drug costs (including reimbursement status), detailed administration instructions (including pre- and postmedications) and nurse procedures, links to key sources of evidence, side effects management, and interaction information.
CI-SCAT was launched in August 2005 and was accompanied by a large-scale education program between September 2005 and September 2006. Since its inception, use has increased to over 600,000 Website visits per month, with approximately 30,000 individual user sessions. Although these statistics attest to the growing popularity of the site, they are not necessarily indicative of utilization at the point of care. Unlike other areas of medicine such as primary care in which IT systems have been implemented to manage a range of acute and chronic conditions,16-19 medical oncology is lagging behind in terms of evaluating the use and effectiveness of electronic decision support systems in real world clinical practice. To our knowledge, there are no published evaluations of online oncology resources.
In the current study, we undertook a qualitative evaluation to establish how CI-SCAT is being used at the point of care, attitudes of clinicians regarding its use and benefits, and the barriers to its uptake.
MATERIALS AND METHODS
We conducted 1-on-1 semistructured interviews with oncology physicians, nurses, and pharmacists representing the range of clinicians who were potential users of CI-SCAT at the point of care.
We also performed an observational study, whereby oncology clinicians were observed in their work environment (eg, during physician-patient consultations and in patient treatment areas). This component was incorporated into the research design to allow us to fully understand the full-spectrum of care associated with the implementation of oncology treatment and where CI-SCAT dovetailed into the workflow, and to validate the self-reported practices of clinicians.
Ethics approval was obtained from each study site and the appropriate area health services, and ratified by the University of New South Wales Human Research Ethics Committee (HREC 07,228). Each participant gave written consent to participate in interviews and to being observed.
Study Setting and Participants
In Australia, oncology protocols are implemented primarily in the ambulatory care (outpatient) setting, and most treatment is funded under the auspices of Medicare, Australia's universal healthcare system. This arrangement occurs regardless of a patient's health insurance status. We therefore identified 6 public hospital medical oncology outpatient departments in the state of New South Wales, which were stratified according to their geographic location (2 each of metropolitan, regional, rural). The metropolitan hospitals are large tertiary referral centers affiliated with universities. The metropolitan hospitals cover a relatively small geographic area in comparison with the regional centers (75 vs up to 1200 square kilometers), but their oncology patient throughput is approximately equal at 50 to 100 patients/week. The rural hospitals cover even larger geographic areas (approximately 3400 square kilometers) and see up to 35 patients per week.
We invited all oncology physicians (attending physicians and residents in advanced training programs), nurses (nurse unit managers, clinical nurse consultants, clinical nurse specialists, registered nurses), and pharmacists from each site to participate in our study. Rural hospitals do not use oncology pharmacists or have medical staff on site; rather, medical staff visit from metropolitan hospitals once or twice monthly.
We conducted a pilot study in May 2007 with 64 senior oncology nurses representing hospitals from around Australia.20 These study findings formed the basis of the semistructured interview schedule used in the current study. The schedule was pilot tested with 5 clinicians, representing medicine, nursing, and pharmacy, from sites not included in the main study and was revised according to participant suggestions (the schedule is available from the authors on request).
One researcher (I.M.H.) conducted all interviews and observations between November 2007 and March 2008. We conducted 49 face-to-face interviews and 1 telephone interview of approximately 30 minutes in duration. The interview schedule was adapted throughout the process according to issues arising from earlier interviews. Data saturation, according to the main themes, was achieved by the end of the study. All interviews were audio recorded and transcribed verbatim, and the transcripts were proofread for accuracy against the recording. The transcripts were deidentified for analysis. Participants were given the opportunity to review their transcripts and delete any additional information they believed could identify themselves, other colleagues, or their hospital.
Between 2 and 5 days (12-40 hours per site) was spent in each setting observing in clinics and patient treatment rooms. Observation items such as the number of chairs/beds in treatment areas; the frequency with which CI-SCAT was used and by whom; CI-SCAT's level of integration into the treatment process; discussion between clinicians regarding CI-SCAT; relative ease of CI-SCAT access and use; and alternative information sources used were documented in writing at the time or as soon as convenient after the site visit.
We analyzed the interview content by coding the dialogue in each transcript and determining the themes arising from each code. Three members of the research team (I.M.H., J.M.F., and S.-A.P.) generated independent initial coding schemes, and differences were resolved by discussion. I.M.H. elaborated the coding scheme and applied the final codes, with confirmation of consistency through blinded dual coding of all transcripts (by J.M.F.). This process ensures the reliability and validity of qualitative data.21 The transcripts were imported into NVivo 7 (QSR International, Cambridge, Mass), which we used to organize the data and stratify it according to clinician type. Two team members (either I.M.H. and S.-A.P. or I.M.H. and J.M.F.) then independently identified emerging themes for each code, and compared differences between clinician groups and hospital location.
All observational notes were reviewed for each site and compared with the self-reported interview data provided by the clinicians groups. Our results are reported based on the combined interview and observational data.
We interviewed 50 clinicians (16 physicians, 30 nurses, and 4 pharmacists), obtaining representation of different clinician groups within each hospital (Table 1). All nurses were observed in the patient treatment areas, and 10 physicians were observed in their clinics or chemotherapy write-up meetings. The remaining clinicians (n = 10) did not consent to being observed.
Table 1. Clinicians Interviewed and Hospital Size
Total Clinicians at Each Hospital
NUM indicates Nurse Unit Manager; CNC, Clinical Nurse Consultant; CNS, Clinical Nurse Specialist; RN, Registered Nurse; L, large (56+ cases seen per week); NI, not interviewed due to unavailability at hospital or declined interview; M, medium (26-55 cases seen per week); S, small (1-25 cases seen per week).
Total No. of clinicians
On the basis of the interview and observational data, we found several key issues that appear to impact on clinicians' attitudes toward and use of CI-SCAT.
CI-SCAT Is Well Known and Highly Regarded
All the medical oncology departments and staff therein have a common knowledge and awareness of CI-SCAT. Many clinicians, particularly nurses, highly regard CI-SCAT, referring to it as “fantastic,” “convenient,” their “first port of call,” a “good resource,” an “excellent program,” and “integral in the administering of chemotherapy.” They described the site as “user friendly,” “well structured,” and “intuitive.” However, a minority of clinicians have a contrary view of the resource, resulting in their infrequent use of it.
CI-SCAT Supports the Full Spectrum of Chemotherapy Care
As was its intent, CI-SCAT supports all aspects of the chemotherapy treatment process (eg, treatment, preparing for administration, administering, and monitoring treatment effects (Fig. 1)) but clinician roles and the tasks they perform in relation to these processes vary between hospitals. In most hospitals, CI-SCAT is “entrenched in what (clinicians) do” and a “part of the routine.” CI-SCAT protocols are regarded as a “vital” source of in-depth information for administering chemotherapy regimes. Consistent with the intent of the system's design, valued and highly utilized features include dosing modification and safety information (drug toxicity, drug interactions, and the management of side effects), guidance regarding the use of pre- and postmedications accompanying chemotherapy, direct links to the evidence supporting the protocols, drug costs, chemotherapy administration details, nursing procedures, patient scheduling, and education resources. However, clinicians reported their utilization of CI-SCAT's dose calculator to be far lower than the aforementioned areas of the website. This is partly because not all clinician groups, in particular nurses, have a “need” to use it and the requirement to “log-in.” In addition, although a few senior physicians use the calculator regularly, other clinicians use alternative calculator tools that are built into their hospital software, out of familiarity and ease.
Although it was not the intent of the system, CI-SCAT is also used by some clinicians to guide treatment choices when initiating treatment or to validate treatment choices and information thereafter.
CI-SCAT Use Is Determined by Multiple Factors
The extent of experience in practicing in medical oncology and in using CI-SCAT influences attitudes toward its use. Senior physicians are familiar with the protocols they use commonly and believe they have no need to consult CI-SCAT when using these. However, in circumstances in which the protocols are new or rarely used, they will consult CI-SCAT. Indeed, clinic observations demonstrated limited use of CI-SCAT in cases in which clinicians use protocols on a regular basis. However, residents and junior nurses perceived their use of CI-SCAT to be greater than their senior colleagues, because they are still learning and are not so familiar with the protocols (Table 2).
Table 2. Clinician-Specific Factors Affecting CI-SCaT Use
CI-SCaT indicates Cancer Institute Standard Cancer Treatments Program.
“·for the standard protocols I use all the time·they're imbedded in my brain matter. So for those protocols, I don't usually need to access CI-SCaT, but certainly for other protocols that are outside my comfort zone, they're the ones that I will sort of tend to use with the website, with CI-SCaT.” (Senior physician)
“It's up to the individual as to whether they want to use that·protocol or not.” (Senior physician)
“It is very valuable for registrars to have a very good reference without them always having to ring and hassle consultants. That allows you more autonomy in making decisions.” (Junior physician)
Perception of time
“·there is so much to do·there's not much time in that consultation before you move on to the next one to get onto the CI-SCaT site and work your way through.” (Senior physician)
“The nursing staff get very busy days with people churning through so they might, rather than refer to the computer for an answer, they might ask the next senior person.” (Senior nurse)
“I would actually make the time because it actually only takes 2 minutes to get on it and have a look at something. I feel that that saves me probably a couple of hours worth of worrying·” (Junior nurse)
“I am sure that if I was more computer literate, ambidextrous with it, I would have used it more. Not many people are as bad as I am using computers, which I am hoping to die before I have to learn.” (Junior physician)
Most clinicians highlighted the benefits of CI-SCAT use in their clinical practice (Table 3), and this certainly increased their likelihood of using the program. However, senior physicians emphasized that their clinical autonomy should be retained, and one cannot be made to use CI-SCAT. In contrast, junior physicians believe CI-SCAT allows them a degree of autonomy in their decision making because they are able to research the specifics of a protocol intensively before they are required to call on the support of their senior colleagues. Furthermore, nurses within rural hospitals in particular depend on CI-SCAT, because they are more “isolated” and do not have physicians or pharmacists on site, requiring them to be more autonomous. Indeed, we observed the use of CI-SCAT as the first port of call when there was a complication during chemotherapy administration.
Despite its aforementioned benefits, the perceived lack of time of many clinicians means they do not have enough time to refer to CI-SCAT. Certainly, we observed in some hospitals that CI-SCAT was not always used during busy periods. For some, it is thought to be more time‒efficient to seek information from colleagues than to refer to computer-based information. However, nurses in particular use CI-SCAT, because it saves them time when they need information and it is “better to be safe than sorry.”
Several clinicians believed their lack of IT skills is a barrier to CI-SCAT use. Furthermore, some older clinicians who may not have had IT training often prefer to “read things in their hand” than access a computer. In addition, clinicians who found CI-SCAT easy to use and navigate were those very comfortable using IT.
Environmental: policy and culture
Some hospitals sanction an official policy approving CI-SCAT as the primary protocol source, overriding local protocols (Table 4). Other hospitals have an informal understanding that CI-SCAT is to be used and that “most people will sort of default to CI-SCAT,” although this also depends on the individual clinician and their preferences. Whereas nursing and pharmacy staff generally endorse CI-SCAT use, endorsement by physicians varies. Many senior physicians “widely endorse” it, encourage use of it, and direct staff to CI-SCAT. Others are more cautious, saying it has to be used “judiciously,” and 1 advanced resident commented that “it's been endorsed, definitely·I've been told also it's not the be-all-and-end-all·it's not binding·”
Table 3. Benefits to CI-SCaT Use
CI-SCaT indicates Cancer Institute Standard Cancer Treatments Program.
“·what we're doing is backed up by research and it's the most up-to-date way of doing things.” (Junior nurse)
“·just to know that the information on CI-SCaT is evidence-based and it is current best practice and I feel confident using that protocol that is on there because I know that it's the best source that you can get at the moment·you can't ask for more than that.” (Junior nurse)
Standardization and uniformity
“·it just makes you actually have a form of standardization as high as possible.” (Senior physician)
“·it standardizes everything, so·it just means that everybody is giving things the same way.“ (Junior nurse)
“·we know that the treatment that we are giving here·it's the same as you would get anywhere else·it's not any more of a hick rural regional place that you don't get good care and you do get good care. You get the same as what you'd get somewhere else·I think that's really important.” (Senior nurse)
“It tends to ensure that people get what they're supposed to get I think.” (Senior physician)
“I think it just gives you accurate information you can clearly apply to every patient·and end up with a treatment program that's well-documented in a way that should have minimum mistakes made.” (Senior physician)
“It tells me my job and how to do it properly.” (Junior nurse)
“So for me personally I found it to be a real lifesaver·it's like a safety net.” (Junior nurse)
“·reducing people's anxiety and increasing their confidence·It takes away that sort of fear that I don't know what I'm doing·” (Senior physician)
“·it's a centralized repository so I don't have to go and look at a whole bunch of other places.” (Senior physician)
“You can focus on patient quality improvement activities rather than medicine quality improvement activities.” (Senior nurse)
“·we don't have to go chasing that information, what's best practice and creating our own policies, that has taken a huge burden off all of us because we were all involved in creating policies and reviewing policies locally here, so that has been beneficial.” (Senior nurse)
“It sort of consolidates a lot of science that I would have normally looked to for guidance. I used to previously use [other sources] for assistance, but this has sort of consolidated that. It's a much more useful site.” (Senior physician)
“It's there, it's easy. You can go bang, bang, bang and you're in and you have got all the information up there.” (Senior nurse)
Table 4. Environmental Factors Affecting CI-SCaT Use
CI-SCaT indicates Cancer Institute Standard Cancer Treatments Program; CNC, Clinical Nurse Consultant; IT, information technology.
“The policy is that we'll use CI-SCaT·what the Committee have agreed to is that the Cancer Institute protocols override local protocols.” (Senior nurse)
“It's our big fat resource now and we have been encouraged to use that as our means of best practice to check the CI-SCaT protocol before you go and administer anything.” (Junior nurse)
“I think definitely because [physician] has been so passionate about it, it has made a big difference.” (Junior physician)
“Our CNC in oncology had an amazing influence on her team and if she thought anything was a good idea well we went with it. That's the way we always worked.” (Senior nurse)
“The computer could be down or something. We had the helicopter the other day ran into a tree and went over the power lines. So I mean stuff like that, that could happen.” (Junior nurse)
“We've got 3 staff here and 1 laptop and that's it, it's very hard to manage with just the 1 laptop.” (Senior nurse)
“Because we do all of our notes on the computers, we definitely don't have enough computers.” (Senior nurse)
Many clinicians highlighted the need to have “respected,” ‘passionate’ people who are “zealots” within the hospital to drive practice. Within 2 hospitals in particular, CI-SCAT use was very much driven by clinicians such as this, which helped to increase its use.
The availability of IT facilities varies greatly and is a major factor relating to CI-SCAT use. Although there is sufficient computer access for the majority of physicians, with most having a computer in their clinic, nurses in metropolitan and rural hospitals identified insufficient computers for the number of staff. However, although CI-SCAT is intended to be used online to ensure use of the most up-to-date information, to overcome the lack of computer access, CI-SCAT protocols are printed and inserted into the patient files at some hospitals. In addition, available computers also have to be used for tasks other than referring to CI-SCAT, so it is not used as often as it could be because of the access barrier. Computers and hospital computers can also be slow or go offline, which is a “terrible frustration”; however, at the majority of hospitals, Internet access is reliable.
Education Is Key
Despite the influence of many positive attitudes and factors on its use in everyday practice, a key issue is clinician education on CI-SCAT. This is often incorporated into the orientation process for new staff, and most hospitals received a formal education session from the Cancer Institute New South Wales in the period 2005 to 2006, after its official launch. However, because of high staff turnover, it is recognized that single training sessions are insufficient to promote and maintain high levels of CI-SCAT use. Indeed, it was highlighted that further education is important and would be valuable. Clinicians believe more exposure to CI-SCAT is required, not only to raise awareness, but also to allow people to use it to its full capabilities.
CI-SCAT is 1 of an extensive array of Web-based resources developed worldwide to support clinicians treating cancer patients.10-14 Our findings demonstrate clearly the integral role this form of decision support system can play in the treatment process and the factors influencing its uptake. Importantly, this study increases our understanding of the challenges facing the implementation and uptake of voluntary Web-based systems in oncology practice.
Our findings are based primarily on the self-reported practices of clinicians from a sample of medical oncology departments in 1 Australian state. Thus, the outcomes reported may not necessarily reflect the opinions and practices of other clinicians working within the study hospitals or those at other hospital sites. Our respondents are likely to be supporters and users of CI-SCAT and therefore the results, based on interview and direct observation, may be more positive than one would anticipate across the state or country as a whole. However, the study strengths relate to the multisite and multimethod approach. Sampling a range of sites allowed us to encompass hospitals in diverse geographic regions and compare experiences and infrastructure in this realm. Although the foundations of the study were based on clinician self-report, we also validated these findings against direct observation. In addition, we interviewed multiple clinicians to ensure the perspectives of all professions acting in the diverse roles responsible for decision making and implementation of chemotherapy. Furthermore, our results corroborate other behavioral and clinical research findings.22-24
Clearly, this Web-based resource plays a major role within the hospitals participating in our study; it is used widely, although to various degrees. At 3 years postlaunch, clinicians' attitudes were generally positive, which translated into relatively high levels of use at the point of care and in turn greater awareness of its benefits in patient outcomes. Indeed, many clinicians recognized the advantages, from the clinician and patient perspective, of standardizing treatment processes and reducing variability in practice. Previous evidence also suggests that positive attitudes toward the utility of healthcare IT, particularly in relation to improved patient safety, are important determinants of their adoption among clinicians.24
In addition, the active involvement of clinicians in developing the protocols and/or taking the role of a “change champion” at individual hospitals have contributed promotion at these sites, which in turn appears to have impacted on its use. These 2 issues have been reported as key components for the uptake of evidence in clinical practice, electronic or otherwise. Clinicians involved in developing clinical information systems from the outset have a sense of ownership toward them, and active participation reviewing evidence for guidelines is an integral part of their acceptance.22, 24, 25 Furthermore, the presence of respected and trusted colleagues implementing innovations in medical settings to improve care is commonly recognized as a factor influencing their uptake.26, 27
Finally, although clinicians receive CI-SCAT education, its quality and quantity is highly variable. However, its provision appears to be a significant enabler of use. This is not surprising, considering there is much evidence to support the use of educational outreach programs in improving clinician's prescribing behavior and adherence to guidelines, for example.28-30
Although positive attitudes toward CI-SCAT appear to equate with more use, it is also evident that many clinicians believe their experience negates the need to refer to CI-SCAT on a regular basis. This approach could have a range of fiscal, effectiveness, and safety implications.31, 32 Clinicians will not be accessing the most up-to-date treatment information; at the most basic level this will mean subtle changes in dosing, but more importantly they are likely to miss critical safety updates if the protocols are not checked regularly.
In hospitals in which CI-SCAT is embedded in practice through incorporation into treatment workflows and hospital policies, and in which the provision of IT is sufficient to ensure unimpeded and timely access, CI-SCAT use appears to be greater than in hospitals in which these factors are not in place. Certainly, adequate and appropriate infrastructure is essential for the uptake of any electronic decision support system, and the published literature shows clearly that the more IT systems are integrated into clinician workflow, the more likely clinicians are to adopt them.23 In addition, and not surprisingly, rural hospitals appear more heavily dependent on CI-SCAT as compared with metropolitan hospitals, given their limited human resources.
The findings of the current study demonstrate the multifaceted nature of the forces acting on oncology clinicians in their day-to-day practice, reinforcing existing knowledge of the influences on uptake of IT systems in hospital and community practice.
Clearly, these findings will most closely mimic the implementation needs and requirements for voluntary as opposed to mandated systems. Certainly, literature suggests a clear difference between user attitudes and acceptance of technologies when they are mandated and highlights the importance of fostering positive attitudes in these situations.33 However, even in jurisdictions in which there are mandates for use of electronic systems, this does not necessarily guarantee full adoption by clinicians, and so the factors highlighted in this study remain integral to ensure the use of these systems to their full potential.34, 35
In light of these findings, and given the considerable efforts, attention, and resources being used to develop electronic decision support tools for medical practitioners worldwide, it is paramount that software and content development represent only 1 part of a multifaceted approach to implementation. This study and others emphasize the importance of understanding the target audience, the environment in which they operate, and the basic infrastructure required to implement such a system. Although we can learn from the extensive literature emerging in this area, one clearly needs to account for and understand local factors, constraints, and workflows to guarantee any level of success. Furthermore, early interventions need to be accompanied by continuing education and endorsement both for long-term sustainability and to ensure that systems reach their full capabilities. Only then can we work fully toward the ultimate aim of systems such as these, providing patients with the best quality care and maximizing positive treatment outcomes.
We thank the Cancer Institute Standard Cancer Treatments Program secretariat for its assistance in the development and implementation of the project and the clinicians who gave their valuable time to participate in the research.
Conflict of Interest Disclosures
Funded by a Cancer Institute New South Wales Health Services Research Fellowship.