The Clinical and Translational Science Institute (CTSI) offers a comprehensive physician–scientist training experience for learners enrolled in a certificate or degree-granting program. Learners from a variety of educational backgrounds (i.e., medicine, dentistry, rehabilitation sciences, public health) representing a continuum of learners, from medical students to established professionals, are enrolled in the programs. In 2009, the education core was awarded a 2-year supplemental grant to develop and implement a CBE framework for the CTSIs certificate and degree-granting programs. To implement this type of educational program, three steps were necessary. First, the competencies that outline the knowledge, skills, and attitudes of a CTS researcher were defined and operationalized (curriculum development). Next, because the CTSI had existing educational programs, an alignment between the existing course objectives and the new curricular objectives was done (curriculum alignment). Finally, instructors were made aware of the instructional and assessment methods that would be supportive of a CBE framework (curricular support).
Prior to beginning curriculum development, several comprehensive literature reviews focused on CBE-related domains were conducted. Understanding the history of CBE and how CBE was implemented in disciplines already using the paradigm (i.e., medicine, dentistry, psychology, engineering) helped inform a similar structure within CTS. In addition, it was necessary to know course requirements for other CTSIs and research-based graduate programs in order to develop program-specific curricular models.
Beginning with the end goal was fundamental to outlining the competencies that would underlie the curriculum. For instance, when one envisions a clinical and translational scientist, what type of person comes to mind? Ideally, this is an individual with the requisite knowledge and skills needed to function optimally within the profession. The complexity of this end product, however, necessitated the development of a hierarchical competency structure in which the lowest level operationally defined the competencies in measureable ways translating the competencies into a functional curriculum (Figure 1).
Starting with the ideal end goal defined above, two broad competency clusters were delineated (Figure 2). The requisite knowledge and skills that define the field (Foundational Skills) form one broad competency cluster of the CTS researcher. The other competency cluster outlines the skills a researcher would need to function as a professional within the field (Functional Skills). The next lower level in the hierarchical structure is competency domain. Competency domains begin to define the range of knowledge, skills, and attitudes of a CTS researcher by specifying fundamental aspects within the discipline. For Foundational Skills, the CTS researcher relies on applied knowledge and skills in data collection (Research Design) and data use (Data Analysis). For Functional Skills, not only do the competency domains outline communication and norms within the profession (Professional Skills), but include additional skills deemed necessary to work in a multi- or interdisciplinary context (Leadership and Teamwork).
It is important to note that at this level, the competency structure could equally apply to an undergraduate or graduate program, or a master's or doctoral program at the graduate level. Therefore, the structure is too broad to serve as a curriculum. In fact, all educational programs at the CTSI share the same competency clusters of Foundational Skills and Functional Skills and the same competency domains of Research Design, Data Analysis, Professional Skills, and Leadership and Teamwork. It is at the specific competency level; the next lower level, that the expectations of the certificate and degree-granting programs begin to be differentiated.
Most specific competencies, however, are still almost identical regardless of program. For instance: Research Design encompasses Problem Formulation, Methodology, Sampling, and Measurement; Data Analysis encompasses Data Management and Bioinformatics and Applied Analytic Techniques; and Professional Skills encompasses Oral Communication, Written Communication, and Ethics and Professional Norms. The specific competencies defining Leadership and Teamwork, however, differ depending on the program. The certificate program only requires that learners develop teamwork skills. The master's level program also includes the development of teamwork skills, but adds the specific competency of project management based on the idea that at this level, the learner should be working as part of a research team and developing management skills during that time. On the other hand, doctoral program expectations for this specific competency are leadership and teaching. In the capacity of a doctoral learner, it is expected that the skills necessary to lead a research project are being developed and that the learner develop skills as an instructor to further the discipline.
All specific competencies are operationalized through associated learning objectives that specify how the competencies are to be measured. Learning objectives that were too broad to be measureable were further defined by behavioral objectives which then served as the measureable outcomes. Additionally, it is the learning objectives that define levels of mastery; with the certificate, master's, and doctorate degrees requiring progressively greater levels of mastery in the objectives associated with each specific competency. Defining learning objectives for achieving mastery by specifying the amount of foundational knowledge (breadth) and specific knowledge (depth) outlines the process of moving from novice to expert as competence is achieved. For instance, while a master's level learner might be expected to identify the elements and rationale involved in constructing data security plans, a doctoral learner would be expected to actually construct a feasible data security plan. When developing learning objectives, several philosophical questions were considered:
- Since everything cannot be accomplished in the finite period of the degree program, what is a reasonable amount of content and experience?
- Is there a minimum amount of knowledge that learners should know about all methodologies?
- At what point is it appropriate for learners to focus only on one methodology of interest?
Additionally, learning objectives were written in such a way to accommodate the diversity of specialization within the field of CTS. Writing single objectives to accommodate each specialization would be overwhelming. For instance, consider the two following objectives; “Learners will evaluate the inclusion and exclusion criteria for a clinical trial and implications for recruitment and generalizability” versus “Learners will identify appropriate study populations, control groups and comparison groups for research problems.” Writing the objective in the second way allows the first objective to fit within it. Thus, each specialization could fit within the same curricular structure. This was particularly important to the CTSI since there are four specialization tracks offered. Occasionally an objective was too specific to a particular area of specialization and deemed not appropriate for all learners to know, such as writing behavior modification plans or learning to use laboratory-based technologies to conduct basic science research. Therefore, track-specific curricula were developed for each track despite the substantial overlap among them.
An additional consideration when writing objectives is the desired scope of the program. For instance, programs could decide that their graduates need to have an understanding of a variety of research methodologies regardless of their specialization. Alternately, programs could have the expectation that their graduates understand research methodologies within their area of specialization only. Of course, there are many gradations between these dichotomies.
One final set of competencies were delineated; Ethics and Cultural Sensitivity. These competencies were considered to be of such high importance and of such a different nature that they were labeled Super-Ordinate Competencies. These super-ordinate competencies permeate the entire educational structure and are interwoven throughout the learning objectives at all levels of mastery just as they are integrated into all aspects of CTS research. For instance, within the specific competency of measurement learners are asked to identify the need to adapt measuring instruments based on the cross-cultural diversity of research participants.
Because the CTSI has an existing program, the next step involved mapping the new CBE objectives to existing courses. Additionally, the National Council for Research Resources (NCRR) offers a comprehensive outline of the field as currently defined and the current curriculum should overlap this document as much as possible. While the alignment with the NCRR document was a straight-forward matching exercise, the curriculum alignment was more complicated. In order to complete it, four aspects of each course were catalogued; level of content coverage, instructional method used, level of assessment coverage, and assessment method used. This was done through a comprehensive review of course materials including videotaped lectures, lecture slides, and assessment materials. The project team created a matrix to document content and assessment coverage, as well as methods used, in order to visually reveal gaps in coverage (Figure 3). From an instructional perspective, questions to consider regarding comprehensiveness of curriculum are:
- Where do learners have an opportunity to demonstrate knowledge and practice skills?
- Is there sufficient overlap in content between courses to allow reinforcement of content and practice of skills in different contexts?
- Do learners have enough opportunities to practice the integration of knowledge and skills?
- What teaching strategies are used to deliver content? Are these appropriate methods to foster the development of competence?
- What assessment strategies are used? Are these appropriate methods to capture the complexity of competence?
The alignment matrix for each course was discussed with the course instructor and the percent agreement between the project team and the instructor was calculated. The percent agreement exercise was important because it revealed how instructors saw the elements of their course and whether or not the project team perceived those elements in the same way. Reports were generated that outlined recommendations for additions to content and assessment strategies for closing the identified gaps. Recommendations also included teaching and assessment techniques that would be appropriate or more appropriate for the new curricular framework. In addition, the alignment matrix was designed to be a discussion point for program directors and faculty to evaluate their program with respect to the first three questions outlined earlier.
Curricular support materials
The final step was the compilation of teaching and assessment resources for faculty. Instructional methods were chosen for their potential to be interactive, engaging, and encouraging of self-directed learning. The project team wrote information sheets on several instructional methods including: guided practice, questioning, motivation, adult learning theory, simulation, case study, and small groups. Each information sheet contained a description of the method, strengths and weaknesses of the method, and best practices for the method.
Assessment is a critical aspect of CBE providing evidence of competence and establishing that learning has been achieved at the desired standard of performance. Because of the nature of CBE, assessment methods that allow for the integration of knowledge, skills, and attitudes and have the potential to mimic problems encountered in everyday situations were given priority for inclusion. The assessment toolbox features educational modules on general aspects of assessment and specific assessment methods such as tests, performance assessments, peer assessment, and self-assessment. In addition, scoring rubrics, designed for types of assignments typically used, were also included in the assessment toolbox to allow for immediate use by instructors.