SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

This paper describes the design and development of a learning object repository for a new statewide higher education initiative. The Texas Higher Education Coordinating Board is funding the redesign of large undergraduate courses; the redesigned courses are intended to improve student learning, retention and outcomes. The learning object repository stores and provides access to content from these courses. Content has been decomposed into discrete learning object varying in levels of granularity. The paper provides details on the proof-of-concept implementation developed in Phase I of a two-phase project. Special attention is given to key aspects such as the levels of granularity, metadata, technology, and user testing. Issues that emerged in Phase I are informing all facets of the next iteration of the repository.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

The Texas Higher Education Coordinating Board (THECB) was created by the Texas Legislature in 1965 to “provide leadership and coordination” for the Texas higher education system. The THECB strives to insure excellence, participation, success, and research in higher education in the State of Texas. Under a current mandate from the Texas Legislature, the THECB initiated the Texas Course Redesign Project, which provides funding to redesign entry-level academic courses to improve student learning outcomes. The goal of the project is to develop and encourage widespread adoption of technology enhanced courses and instructional resources that promote student success. The redesigned courses will be freely available to all Texas public institutions of higher education.

To assist the THECB in encouraging widespread adoption of learning resources developed in the Texas Course Redesign Project, the Texas Center for Digital Knowledge at the University of North Texas proposed the development of a learning object repository (LOR) to store, manage, and make accessible, content from the redesigned courses. Specifically, the Texas Center for Digital Knowledge suggested that THECB could leverage the Texas Course Redesign Project investment in course redesign by making entire courses and components of courses available for reuse and repurposing. As a result, THECB is funding a two-phase LOR design and development project intended to support course development. For Phase I, completed in December 2007, a Texas Center for Digital Knowledge project team developed a proof-of-concept repository application. Phase II is a two-year effort developing the next instance of the LOR, and the administration, workflow, and policies necessary for a production system. Phase II will require the Texas Center for Digital Knowledge to implement lessons learned in Phase I.

This project report summarizes the design and development process which resulted in the project deliverables for Phase I. Phase I had a short turn around time and many of the steps taken by the project team happened simultaneously. However, the design and development process was informed by current LOR literature and our experience with similar projects. This report will be organized into four general sections. The background section is next, which includes the rationale for the subsequent sections.

Background

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

Reusing and sharing resources for educational purposes is not a new concept. There is a long history of faculty sharing and reusing learning resources in the form of textbooks, published papers, syllabi, assessments, and so forth. Like traditional paper-based materials, digital learning materials can also be shared and reused (Campbell, 2003; Pras, 2001). Campbell (2003) describes a useful three step process for integrating learning objects into educational settings. The first step addresses interoperability. The content is decontextualized into discrete learning objects. The focus of this step are issues such as granularity which is a term used to describe the size of a unit of learning. Campbell (2003) also includes reusability and levels of aggregation in this factor. Issues regarding aggregation are related to granularity. Some of the course content cannot be fully decomposed and decontextualized into discrete learning objects; it may have dependencies on related content, where it only has meaning in conjunction with the related content. Therefore, in some cases, discrete learning objects must be combined to create larger more meaningful units of content. Generally, a learning object repository must provide flexibility in storing course content at whatever level of granularity is considered most appropriate. Also, a repository must provide a storage environment for both learning objects and descriptions (i.e., metadata) of the objects to help users find, identify, select, and obtain relevant and useful learning objects. A unique characteristic of the THECB LOR is that it addresses entire courses, from which smaller learning objects can be derived. In addition, the quality of the learning objects is emphasized since all content in the LOR is from the newly redesigned courses.

Campbell (2003) addresses these issues in step two in which considers the educational properties of the learning objects. Insuring the learning objects are pedagogically meaningful is a major focus. In addition, subject specificity and learning and teaching styles must be considered. Campbell's (2003) third step focuses on technological issues. This step considers user access and tools, adherence to international standards and specifications, and the configuration of repositories. The final step in Campbell's (2003) process is the consideration of cultural factors. Issues related to current teaching practice, learner profiles, protection of intellectual property rights, and encouraging communities of practice around the leaning objects' content are grouped under this factor.

The Phase I proof-of-concept proposal required us to complete the following four objectives which are sequenced using Campbell's (2003) factors and process:

  • Decompose a THECB redesigned course in at least three levels of granularity and store the decomposed course content in the learning object repository;
  • Identify and implement an appropriate metadata scheme to describe and manage the learning objects in the repository;
  • Enable search and browse capabilities in the repository to support end user tasks of finding, identifying, selecting, and accessing the learning objects; and
  • Provide course content in packages that conform to the IMS content packing standard for import into learning management systems that support the IMS standard.

The rest of the report will follow this sequence. First, we will discuss how the content was decomposed. Next we will describe the work we completed to determine a metadata profile. The following sections describe the process we used to determine the technological and user requirements for the system. Finally, we describe our evaluation process and end by discussing how Phase I development is informing Phase II work.

Decomposition of course content

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

Learning object repositories have a variety of materials that can be reused for instruction. In the case of the THECB LOR, the focus is on providing the complete course as one learning object as well as discrete learning objects derived from the complete course. A U.S. History I course was chosen to demonstrate the functionality and viability of a LOR for reuse and repurposing learning objects. The redesigned course content was decomposed into discrete learning objects. This process yielded nearly 300 learning objects containing text, images, audio, video, and flash animations. A major component in Phase I was making decisions on the levels of granularity for the derived learning objects. While the literature indicates that more granular learning objects have an increased potential for reuse (Hummel, Manderveld, Tattersall, & Koper, 2004; Wiley, Waters, Dawson, & Lambert, 2004), there is little empirical data about what is an appropriate level of granularity for learning objects.

For purposes of the proof-of-concept, the goal was to demonstrate the various levels of granularity that can be considered. The U.S. History I course has a relatively traditional structure. The project team used that structure to provide the levels of granularity. These levels of granularity became the organizing structure of the repository (see below). The following are the five levels of granularity used, from the least to the most granular level:

  • Complete course as a learning object
  • Unit: Each unit consists of two or more lessons
  • Lesson: Each lesson belongs to a unit, and each lesson has two or more topics
  • Topic: Each topic belongs to a lesson
  • Free-standing Learning Object

Each of these learning objects consists of one or more files. The project team worked with the course creator to bundle the appropriate files together for each of these levels of granularity to submit into the repository and describe with a metadata record. The level of Free-standing Learning Object provided a mechanism to describe and make available specific learning objects (e.g., self-tests, glossary review, interactive maps, etc.) that while created for this specific course may be reusable or repurposed in other contexts besides U.S. History. Once the course content (i.e., individual files) was organized into the levels of granularity, the project team began submitting these files and creating metadata records for each learning object. The repository was then organized (using the DSpace construct of community, collection, subcollections, etc.) into the following for presentation to the end user (see also Figure 1):

  • Complete Course
  • Course Units
  • Course Lessons and Related Assessments
  • Course Topics
  • Free-Standing Learning Objects
thumbnail image

Figure 1. Organization of the Learning Objects in Repository

Download figure to PowerPoint

In addition, there were other course materials that were important to provide but that did not fit into the levels of granularity described above, and these include:

  • Course Structure: contains a syllabus and course map that outlines the components of this particular course
  • Course-wide Assessments: assessments in the form of mid-term and final examinations, developed specifically for this U.S. History I course
  • Case Studies and Case Study Teaching Guides: contains multiple items that comprise interactive learning materials that may be associated with a particular unit, lesson, or topic, but comprise a separate, course-contextualized learning object

The final collection presented to the user is IMS Content Packages. Another objective for the proof-of-concept was to demonstrate how learning objects can be easily ingested into a learning management system (LMS). The IMS Content Package is a specification for wrapping up all files associated with a learning object, along with metadata to produce a zip file which can be directly ingested into a LMS such as Angel, Blackboard, and Sakai. A user can discover an appropriate learning object in the repository. The metadata for that object contains a URI to an associated IMS Content Package, which can then be downloaded.

Metadata application profile

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

Metadata is critical for any digital repository to support a variety of functions including management and end user interaction. DSpace provides Dublin Core (DC) metadata elements by default. For the THECB LOR, neither simple nor qualified DC elements are rich or precise enough for learning objects. The project team developed a metadata application profile (Heery and Patel, 2000) that addressed a variety of needs of the application, the objects, and the users.

The project team examined various potential metadata schema and/or profiles to see which would address the majority of needs. We investigated questions such as which metadata schema might serve as the basis for the THCEB LOR application profile, which elements should be designated mandatory, and other issues identified in the guidance document (e.g., IFLA, 2003). The project team studied 24 learning object repositories listed by the Florida Distance Learning Consortium (2007) for its OnCoRe Blueprint project (funded by the Fund for the Improvement of Postsecondary Education, U.S. Department of Education). The repositories revealed the metadata supporting those applications are based primarily on two schemas: DC and IEEE 1484 Learning Object Metadata (IEEE LOM). Figure 1 lists some of the repositories and the metadata schemas used.

Afterwards, the project team compared and analyzed metadata elements in 13 learning object repositories to look at the use of elements and specifically the commonly used elements to guide the choice of elements for the THECB LOR. For example, all of the examined repositories use the elements: Title, Subject, Description, Learning Object Type, Authors or Creator, Rights. More than half of them use these elements: Identifier (URL), Technical Requirement, Media Format, Typical Learning Time, and Interactivity.

Table 1. Comparison of Learning Object Repository Metadata Applications
Repository nameBasis of metadata for application
Gateway to Educational Materials (GEM)GEM (based on DC)
Education Network Australia (Edna)AGLS (Australian Government Locator Service) Metadata Element Set
European Knowledge Pool System (ARIADNE)LOM
Campus Alberta Repository of Educational Objects (CAREO)CanCore (an application of LOM)
The Digital Library for Earth System Education (DLESE)AND (an application of NSDL-DC)
Jorum (a JISC-funded collaborative venture in UK Higher and Further EducationUK LOM (an application of LOM)
National Science Digital Library (NSDL)NSDL-DC (based on DC and LOM)
Florida Distance Learning Consortium (OrangeGrove)SCORE (based on LOM and GEM)
The Learning Federation (TLF)DC and LOM
Food and Agriculture Organization Learning Resources (FAOLR)DC, LOM, and AGS (Agricultural Metadata Element Set)

The project team concluded that a combination of DC and LOM elements (with one element from the Gateway to Educational Materials metadata) would address requirements for the prototype, and the resulting application profile documents the team's decision. Figure 2 lists the display form and element names defined in the THECB LOR application profile (note that some of these elements are DSpace-specific).

Table 2. Metadata Elements in THECB LOR Application Profile
Intended Educational Audience (dc.audience.educationLevel)Course Structure (dc.relation.isPartOf)
Author's Affiliation (dc.contributor.affiliation)IMS Content Package (dc.relation.uri)
Authors (dc.contributor.author)Rights (dc.rights)
Other Contributors (dc.contributor.other)Access and Use Rights (dc.rights.accessRights)
Date Published (dc.date.dateAccepted)Access and Use License (dc.rights.license)
Date Accessioned (dc.date.dateAccessioned)Rights Holder (dc.rights.rightsHolder)
Date Avalable (dc.date.dateAvailable)Subjects (dc.subject)
Date Issued (dc.date.dateIssued)Title (dc.title)
Summary (dc.description.abstract)Other Title (dc.title.alternative)
Learning Object's Content (dc.description.tableOfConents)Learning Object Type (dc.type)
Learning Object Description (dc.description.uri)Individual Cataloger (gem.cataloging.individualCataloger)
Media Format (dc.format)Domain Cataloger (gem.cataloging.individualCatalogerDomain)
Size (dc.format.extent)Discipline and Subdiscipline (lom.classification.taxonPath.taxon)
Format (dc.format.mimetype)Difficulty Level (lom.educational.difficulty)
URI (dc.identifier.uri)Interactivity Level (lom.educational.interactivityLevel)
Instructional Method (dc.instructionalMethod)Interactivity Type (lom.educational.interactivityType)
Language (dc.language)Typical Learning Time (lom.educational.typcialLearningTime)
Creation and Ownership Information (dc.provenance)Technical Requirement (lom.technical.requirement)
Publisher (dc.publisher) 

Functional requirements for the LOR

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

Functional requirements address the behaviors of the repository application and types of interactions the application should support. Given the relatively short timeframe for Phase I (originally May – August 2007), the project team did not carry out a formal needs assessment to identify user requirements. For the proof-of-concept, the project team's focus was on demonstrating likely functionality based on our understanding of the objectives of Texas Course Redesign Project. The project team documented that understanding by identifying a set of functional requirements based on a general understanding of what the LOR should support in the areas of administration, submission of learning objects, and end user access to the repository and its contents. The project team chose the DSpace digital repository platform for the proof-of-concept. Although the functional and technical requirements are generalizable, they were specified in the context of a DSpace-based LOR.

The resulting functional requirements served several purposes. First, they guided the development and configuration of the repository application. Second, they provided the baseline for use in the system-level assessment of the repository. Its function serves three kinds of users: repository administrators, course managers or submitters, and ender users or course users. According to different kinds of users, the functional requirements were categorized to address:

  • Administration
  • Submission of learning objects
  • End user access to learning objects.

Requirements related to administration

This set of requirements addresses various administrative tasks and responsibilities for managing the LOR. Many of these tasks are carried out using the DSpace administrative interface, but some tasks require access to the server to edit configuration files, set indexing policies, and other responsibilities. The functional requirements include: managing user accounts, setting authorizations and permissions, implementing organization structure for learning objects, customizing submission workflow, implementing metadata scheme, customizing metadata input form, and managing items in repository

Requirements related to submission of learning objects

This set of requirements addresses the entire process of submitting learning objects into the repository. The submission process involves logging into the LOR with an authorized username and password, creating metadata records for items submitted, uploading the files associated with a learning object into the repository, and workflow related considerations. The functional requirements include registering for user account, editing profile, and submitting items to one or more collection and creating metadata.

Requirements related to end user access to learning objects

There may be two or more sets of users who may be restricted in terms of viewing and downloading metadata, items, and content packages. All users should be able to search and browse the LOR and at the least see the metadata records associated with the learning objects, even if not all users will have access to the files associated with the learning objects. The functional requirements guide the configuration of the repository and address end user interaction with the repository which includes searching full-text and metadata elements, browsing by various access points or categories, viewing simple and complete metadata records, and accessing learning objects as allowed by user account authorization.

Technology for the LOR application

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

For the proof-of-concept repository, the project team used DSpace (Version 1.4.2), an open source software repository platform developed by MIT Libraries and Hewlett-Packard Labs. DSpace was installed on a server under the administrative control of project staff, and configuration was completed according to the functional requirements, for example, customizing the look and feel of the application and incorporating appropriate metadata elements required for the learning objects.

The project team chose Linux (Fedora) operating system and installed several instances of DSpace repositories for development and testing by several team members. A final production instance incorporated the results and decisions based on development and testing. The Handle server was installed on the production instance to provide efficient, extensible, and secure identifier and resolution services for the items in the repository. DSpace provides an administrative interface for many configuration options, including: a metadata registry for identifying elements available to the application, access controls, workflow controls, managing user groups, and other administrative tasks. Additional configuration required editing DSpace text and other files on the server itself.

Submission of learning objects and metadata creation

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

The proof-of-concept implementation used the basic submission workflow provided in DSpace. The project team configured the DSpace metadata registry to accommodate the elements needed, and customized the submission pages to assist in metadata creation. Graduate students from history and library and information sciences carried out the metadata creation and submission. The project team used a workflow option in DSpace so that the history students (subject matter experts) completed metadata elements related to subject representation, summary of learning objects, type of learning object, difficulty and time expected to complete the learning object. Additionally, the project team set certain default values (e.g., the creator name, license, etc.) to speed manual metadata creation.

A critical concern was controlling the subject terms assigned to the learning objects. While DSpace provides a function for using controlled vocabularies to limit the set of terms metadata creators can use, the proof-of-concept needed a way to dynamically develop (i.e., add to and enhance) a controlled vocabulary as well as using it during the process of metadata creation. The project team developed a controlled vocabulary application that resides outside of DSpace but links to the DSpace metadata entry interface (Figure 2).

thumbnail image

Figure 2. Metadata Entry Page and Controlled Vocabulary Interaction

Download figure to PowerPoint

The vocabulary tool was developed using MySQL, an open source database management system, Javascript, and PHP. The vocabulary was tested while submitting hundreds of items to the repository by the group of six people working concurrently from different locations. The controlled vocabulary tool provides the following functions:

  • Provide an online version of a controlled vocabulary that metadata creators can link to and have open when they are determining what terms to put into the Subject field;
  • Let metadata creator click on a term in the online controlled vocabulary and have it automatically pasted in to the Subject field;
  • Allow the metadata creator to add new terms to the controlled vocabulary through a simple to use submission form; and
  • Make newly submitted terms immediately available for all users working on item submissions.

End user interaction with repository

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

This section briefly describes the look and feel of the repository along with features implemented for the proof-of-concept. Since Phase I focused on demonstrating functionality, customization to the user interface (UI) was not a priority. Some changes were made to the out-of-the-box DSpace interface to demonstrate that customization is possible. Phase II focuses more on UI issues. In Phase II, the project team is using Manakin, an XML-based tool available in the current DSpace distribution, to improve presentation of information, user interaction, etc. Users can initiate search, browse, sign-in, and other functions.

The project team configured the proof-of-concept to address the functional requirements discussed above, which resulted in the following (see Figure 1):

Browse capabilities

The THECB LOR provides four main browsing options:

  • Browse by Disciplines and Courses: a hierarchical list of disciplines, sub-disciplines, and courses;
  • Browse by Titles: a list of titles of learning objects contained in the LOR;
  • Browse by Subjects: a list of subject terms assigned to the learning objects; and
  • Browse by Date Published: a list of learning objects in reverse chronological order.

Search capabilities

The THECB LOR supports a basic search and advanced Boolean searching by Title, Subject, Learning Object Type, Media Format, Intended Educational Audience, Instructional Method, and Author Affiliation within a selected discipline, courses, or the whole LOR. It also allows for full-text searches of textual learning objects. Each search returns a list of objects ordered alphabetically by title.

Viewing and downloading

The THECB LOR enables the user to open and view learning objects by clicking on the “View/Open” link in the grey box at the bottom of each record (Figure 3). The user can download the IMS Content Package associated with a learning object for import into a learning management system.

thumbnail image

Figure 3. Viewing and Downloading Learning Objects from Metadata Record Display

Download figure to PowerPoint

Testing and evaluating the repository

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

The project team conducted two assessments of the proof-of-concept: system-level functionality assessment and user scenario testing. System-level functionality assessment used a checklist based on the functional requirements identified at the outset of the project, and focused on three categories: administration of THECB LOR, submission of learning objects, and end user access to learning objects. These three sections encompassed 13 functionalities with 70 specific, testable sub-functions. The assessment determined that 63 of 70 or 90% of the sub-functions are working as specified in the THECB LOR proof of concept proposal. Phase II development is being informed by the results of this assessment.

For the end user testing, the project team created use scenarios, each of which presented a real life situation to exercise functionality in the repository. Four subject matter experts, with no previous experience with the LOR and who served as representatives of the intended user group for the LOR, completed actions that focused on browse tasks, search tasks, and a combination of tasks, including downloading objects. The specific purpose of the testing was to determine if new users who were familiar with repository content could locate particular information and download this information so they could use it for scenario-defined teaching purposes. Users could ask questions and make comments during the process. At the end of the experience, project team members interviewed each user in order to clarify any questions or issues about what the observer recorded and to get any additional comments from the user. Overall, these users were able to use the LOR successfully, but the testing identified issues that are being addressed in Phase II. For example, terminology used in the LOR confused some subject matter experts such as the distinction between the term “topic” and the term “lesson” that represent levels of granularity. Phase II includes much more in-depth user testing, usability, and other assessments to ensure the LOR responds to user needs. This Phase I assessment served more as a reality check about the LOR and the direction the development is taking rather than a full-blown usability and user assessment.

Key issues from proof-of-concept implementation

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

As stated above, the goal for Phase I was to demonstrate the proof-of-concept LOR's capability to leverage redesigned course content for reuse and repurposing. We completed four objectives to reach this goal. We decomposed a redesigned course into 5 levels of granularity, developed a metadata scheme, search and browse capabilities, and provided content packages for exporting. In carrying out Phase I work, the project team identified issues that are informing development underway in Phase II. Several key issues emerged which will be considered in Phase II. In some cases we link issues uncovered in Phase I development to the current research literature to demonstrate the utility of project reports in identifying areas for further research:

  • Levels of granularity:Although there is discussion in the literature about the concept of granularity (Duncan, 2003; Sicilia, & García, 2003; Wiley, 2000, 2002), there is little or no empirical evidence of the efficacy of particular levels or criteria on which to base a choice of level. This may be the most important decision to consider when designing a LOR. The proof-of-concept implementation provides several levels and it is possible that a LOR should provide multiple levels. However, as indicated from our user testing, it is clear there is a paramount need to determine appropriate terminology and definitions to apply to the levels for end user understanding.
  • Reuse and repurposing:The user testing also indicated that the concepts of reuse and repurposing in the context of levels of granularity are not well understood. Reuse and repurpose have been defined in the learning object literature (Boyle, 2003; Duval & Hodgins, 2003; Hawryszkiewycz, 2002; Sicilia & García, 2003; Verbert, Duval, Meire, Jovanovic, & Gasevic, 2006; Wiley, 2000). These well accepted definitions were used for the current proof of concept study. These definitions of reuse and repurpose do not appear to be mutually exclusive. For example, any time a learning object is reused it must fit within an instructional context. The learning object will fit within a sequence of instructional materials. Therefore the reuse of a learning object will require additional effort by users to integrate the learning object into an instructional sequence. The definition of repurposing includes using learning objects in ways that were not originally intended. Depending on the level of granularity an object could be reused for a different purpose than intended without having to make modifications. As the level of granularity increases the likelihood of the learning object being used as intended by the originator increases. However, smaller learning objects are less likely to be used as intended and they could be reused in this capacity with little effort (Harvey, 2005; Wiley, 2000). In conclusion, learning objects are integrated into instruction by way of reuse and repurposing, but these definitions do not hold together for all uses.
  • Adequacy of metadata for end users:The THECB LOR application profile defines a set of metadata elements that serve several functions; a primary function is to communicate to users what the learning object is regarding subject, type, difficulty, etc. The current metadata scheme, however is deficient in supporting an understanding how and if the object by itself can be reused or repurposed. Instructional designers analyze learning needs and then formulate learning objectives to address those needs. Consequently, learning objectives may be a good starting point to initially determine if a learning object may fit the user's purpose. The current metadata does not identify learning objectives the learning object addresses. Reuse and repurposing may be dependent upon learning objectives; the challenge will be having the course designer (not the metadata record creator) to supply learning objectives of individual learning objects (especially at the finer levels of granularity).
  • Representing dependencies of learning objects:If individual learning objects contain dependencies to other learning objects their reusability is compromised. The user may not be able to use an object at the level of granularity they are seeking because of these dependencies. The metadata may need to better represent dependencies that currently provided in the THECB LOR application profile.
  • Preparing course content for efficient repository submission:Phase I used a course being redesigned at UNT, and the creator of the course was a co-principal investigator on the project. This allowed for easy communication and cooperative work to get the course content files organized. However, the physical structure of files associated for the delivery of the course did not reflect the bundling of files for submission into the repository. Filename conflicts, relative and absolute URLs in HTML files, and other issues emerged when the rebundling for submission. Balancing the repository requirements for bundling files associated with a learning object with the course designer's needs and structure presents challenges that Phase II will explore. Stated simply, course developers are likely not designing their courses in terms of learning objects; disaggregation and decomposition of course content is needed for reuse as discrete components of instruction.

These issues warrant attention by developers of LORs as well as other issues that surfaced in the proof-of-concept implementation. The project team anticipates more issues emerging in Phase II which will be more focused on user needs, usability, and user testing.

Conclusion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

This two-phase research and development project has particular characteristics and requirements not necessarily shared by other learning object repositories. Instead of the repository simply collecting a variety of learning objects for others to use, the THECB LOR is focused on managing, storing, and providing access to entire courses and learning objects derived from those courses. Phase I surfaced technical, organization, workflow and other questions, some of which have been described in this paper. Issues such as reusability, repurposing, levels of granularity, and others, however, will likely be faced by other LOR developers and users. These issues will not be resolved by technologists (or even information scientists); rather, instructional designers, subject matter experts, librarians, technologists, usability experts, metadata specialists, and others will need to collaborate in successful designing and deploying efficient and effective LORs.

Our project team will continue defining and refining some of these key questions to demonstrate how they can be answered in the context of this LOR. Phase II provides the opportunity to do this through, for example, enhancing the technical infrastructure (e.g., using DSpace Manakin for user interface development) and revising the metadata elements to better communicate aspects of dependencies, learning objects, etc. In addition, the project team is also exploring and conducting research on the following issues:

  • Identification of user requirements and subsequent usability testing against those requirements
  • Policy issues related to intellectual property when learning object are modified by those reusing/repurposing
  • Administration of an production system
  • Distributed submission of items into the repository
  • Machine-processes to assist metadata generation

Phase II will add 4-6 redesigned courses from disciplines of mathematics, foreign language, and English. Issues defined above will be further clarified in collaboration with other course developers. Upon completion of Phase II, THECB will have a near-production level system populated with substantial high-value course content.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References

The Texas Center for Digital Knowledge and the project team gratefully acknowledge the funding from the Texas Higher Education Coordinating Board through the Texas Course Redesign Project, Grant #CR72105 – A Proof-of-Concept Repository for Learning Objects.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Background
  5. Decomposition of course content
  6. Metadata application profile
  7. Functional requirements for the LOR
  8. Technology for the LOR application
  9. Submission of learning objects and metadata creation
  10. End user interaction with repository
  11. Testing and evaluating the repository
  12. Key issues from proof-of-concept implementation
  13. Conclusion
  14. Acknowledgements
  15. References
  • Boyle, T. (2003). Design principles for authoring dynamic, reusable learning objects. Australian Journal of Educational Technology, 19 (1), 4658.
  • Campbell, L. (2003). Engaging with the learning object economy. in A.Littlejohn (Ed.), Reusing Online Resources. London: Kogan Pages.
  • Duncan, C. (2003). Granularization. In A.Littlejohn (Ed.). Reusing Online Resources: A Sustainable Approach to E-Learning. London: Kogan Page.
  • Duval, E., & Hodgins, W. A. (2003). A LOM research agenda. The Twelfth International World Wide Web Conference, Alternate Paper Tracks. Retrieved July 2, 2007, from http://www.www2003.org/cdrom/papers/alternate/P659/p659-duval.html.html.
  • Harvey, B. (2005). Learning objects and instructional design. International Review of Research in Open and Distance Learning, 6 (2).
  • Hawryszkiewycz, I. (2002). Designing learning activities from learning objects. Paper presented at the ASCILITE 2002 Auckland.
  • Heery, Rachel & Patel, Manjula. (2000). Application profiles: Mixing and matching metadata schemas.
  • Ariadne, 25. Retrieved January 24, 2008, from http://www.ariadne.ac.uk/issue25/app-profiles/.
  • Hummel, H., Manderveld, J., Tattersall, C., & Koper, R. (2004). Educational modeling language and learning design: New opportunities for instructional reusability and personalized learning. International Journal of Learning Technology, 1 (1), 111126.
  • IFLA. (2003). Guidance on the structure, content, and application of metadata records for digital resources and collection (draft). Retrieved January 24, 2008, from http://www.ifla.org/VII/s13/guide/metaguide03.pdf.
  • Pras, A. (2001). Sharing telematics courses – The CANDLE project. Retrieved January 24, 2008, from http://www.simpleweb.org/nm/research/results/publications/pras/2001-09-04-eunice.pdf.
  • Sicilia, M.-A., & García, E. (2003). On the concepts of usability and reusability of learning objects. The International Review of Research in Open and Distance Learning, 4 (2), Retrieved June3, 2006 from: http://www.irrodl.org/index.php/irrodl/article/view/2155/2236.
  • Verbert, K., Duval, E., Meire, M., Jovanovic, J., & Gasevic, D. (2006) Ontology-based learning content repurposing: the ALOCoM framework. International Journal on E-Learning, 5 (1), 6774. Retrieved November 20, 2007 from:
  • http://www.editlib.org/index.cfm?fuseaction=Reader.ViewAbstract&paperid=21750
  • Wiley, D. (2000). Connecting learning objects to instructional design theory: A definition, a metaphor, and a taxonomy. In D. A. Wiley (Ed.). The Instructional Use of Learning Objects: Online Version.
  • Wiley, D. A. (2002). The instructional use of learning objects. Agency for Instructional Technology and the Association for Educational Communications and Technology. Retrieved July 7, 2007, from http://www.reusability.org/read/.
  • Wiley, D. A., Waters, S., Dawson, L., & Lambert, B. (2004). Overcoming the limitations of learning objects. Journal of Educational Multimedia and Hypermedia, 13 (4), 507512.