Integrating industry developed courseware in IT scripting courses

The alignment of higher education with the skill needs of employers and industry at large is a topic of active debate, particularly for engineering education. Formal and informal content developed by companies, technology‐specific associations, consulting firms, and individuals forms a body of knowledge that might fast‐track careers toward very specific opportunities. By contrast, academic courseware is known for its pedagogical rigor while being slow to adapt to the fast‐paced changes in technology, particularly information technologies, and reliance on expensive course materials that burden students. This paper explores the drivers for integrating industry‐developed courseware in an existing, higher‐ed course on Scripting for Information Technology in the Department of Technology Systems at East Carolina University and the implications of such integration. We hypothesize that industry courseware is a streamlined, cost‐effective way to provide students with the knowledge they need in the workforce. Students were exposed to the two types of content and their scholarly performance was analyzed. Student perception of the two types of content was assessed through a postclass survey. The student results and the feedback they provided suggest that a hybrid approach is more suitable to address the learning preferences and predispositions of students. This case study concludes that textbook and industry‐generated content can be integrated into existing courses and improving them in terms student outcomes and in terms of alignment with industry's hiring expectations.


| INTRODUCTION
Higher education has come under increasing pressure over the past decade to transform from the structure and format developed during the industrial revolution to a new format that would fit the current information revolution.It faces a chorus of critics that include business leaders who face shortages in new graduates who fit the industry needs, parents who are justifiably looking at the return on their investment in their children's education, and finally, students themselves who were born digital citizens and have a different experience with information acquisition than previous generations.

| Trends
The push to change is most apparent in engineeringrelated majors and courses with three main drivers for change commonly brought up.First, is the perceived misalignment between the content of the standard curriculum and the current plan of record for the industry.This lack of alignment means new graduates will not have the basic knowledge that is currently in demand.Moreover, the slow process of updating the curriculum which is in stark contrast with the rapid developments in technology means the misalignment will grow rapidly over time [24].The second driver is the perception that traditional educational methods are not as effective in preparing today's workforce as they were in the past.Diverse delivery methods, new interactive experiences, and student adaptive learning are just some factors affecting knowledge delivery methods in higher education.Another key trend, driven primarily by industry, is microcredentialing and industry certification [6,14].Finally, the third driver for change is the availability of diverse, curated and not curated, structured, and not structured educational content which caters to some aspects of the previously mentioned two drivers and is generated by professional, industry-specific organizations, by industry-leading companies under the umbrella of Academies and even individual with significant practical experience.This content covers the spectrum from informal to formally organized and structured for effective training.This content is mostly available in digital format thus providing easy and significantly less expensive access to it, providing a path toward specific goals with immediate benefits such as certifications.
The positions taken in the debate on the transformation of higher education can be extreme, with some of those pushing for change claiming that for a successful career, higher education is no longer necessary.Highschool graduates should simply focus on highly rated, short courses that deliver industry-recognized certifications and access to in-demand jobs while skipping the expensive higher education programs.Platforms such as Coursera [10] and Udemy [39] provide an inexpensive path towards the acquisition of a skill or technical knowledge.Similarly, academies built by individual corporations such as Cisco's Network Academy [8] or Palo Alto's Academy [29] provide well-structured, curated content covering topics relevant to the core competencies of the hosting corporations.
The industry-centered academies have been actively and successfully integrated in many higher education programs with aligned concentrations.The traditional Cisco Academy courses have been integrated in engineering programs with concentrations in networking, Red Hat Academy [35] academy courses have been integrated in programs with systems concentrations.Educators are provided with the support needed to conduct the course, the training to prepare for its delivery, and the tools to manage it.Students benefit from industry-recognized training and certifications while instructors benefit from up-to-date materials and revenue opportunities to help develop and maintain the content.Successful integrations have been documented in higher education institutions around the World [18,19,25,37].In the Department of Technology Systems at East Carolina University [12] we have extensive experience in successfully creating multiple courses based on the content provided by industry academies.In some cases, these courses were created at the initiative of individual faculty and in many cases were driven by the guidance provided by the department's industry advisory board.
Courses provided by industry academies are credited with an increase in the number of internships and job opportunities for our students, but they also pose some challenges.First, the structure and focus of industrydeveloped content is typically on specific goals, and knowledge that must be acquired by the student and less on the pedagogical best practices of teaching it.The industry curriculum is focusing on hands on experiences with close to real-life practical exercises.This is in contrast with the traditional academic courseware focused on layering knowledge and exposing the student to curated, at times artificial exercises meant to underline specific concepts.These differences appeal to some students and not to others.Second, the long-term costs of programs leveraging industry content should be considered.Industry academies are corporate units that evolved over time to where they are no longer seen as a cost center with the primary goal of training the next generation of users and buyers, they are becoming revenue generators.This, in turn, means that courses, course materials, labs, or certification costs could increase over time, thus requiring updated justifications.
The benefits of the industry academies are very clear [38] and while in some cases, for certain subjects, the content they provide can represent the full extent of the class, it is important to remember that they do not represent a solution for all educational needs of today's higher-ed students.As highlighted in the debate over the future of higher-ed, industry-generated curriculum integration must be considered in the context of the target topic, the type of knowledge that needs to be delivered, and the target audience.Experience shows that augmenting industry curriculum with content, tools, or support infrastructure can enhance their value [22,26].

| Case study
The industry-developed curriculums have evolved significantly over time.Originally, the content was a sanitized version of internal, knowledge base content or documentation.These early versions of the content were meant to address user knowledge gaps and promote aspects of specific products.Later, the content was structured around specific competencies and in support of certifications, both of which were meant to increase the proficiency of existing users and as a result, reduce support costs and increase customer loyalty.Finally, with guidance and support from experienced educators, industry academies developed complete curriculum and delivery tools with the additional goals of preparing future customers and generating revenue.Since the content is correlating with the industry plan of record, the industry-developed curriculum is typically updated much faster than regular academic courseware.Moreover, considering the goal of corporations to prepare both customers and potential future employees for their specific needs, the curriculum is expanding into new areas of expertise.
Cisco's Networking Academy is a good example of the evolution of industry academies.It started around Cisco's broad suite of certifications with content developed internally by engineers and over time it became a welldefined organization within Cisco, leveraging internal expertise as well as external academic and nonacademic advisers.Today, the Network Academy curriculum is used by countless educational institutions and for-profit training organizations.The content and the certifications are well-regarded by professionals, both in terms of quality and in terms of opportunities generators [9].As the networking industry is evolving toward a focus on operational automation [27], DevOps support [17], and overall network softwarization [4], current and future networking engineers need to acquire software development and scripting knowledge.The most used language in networking and systems automation is Python [7].Naturally, academic programs focusing on networking are adding related curriculum and Cisco's Networking Academy developed its own related courses and certifications.
Within the ECU's department of technology systems which provides a networking concentration and is a Cisco Networking Academy affiliate, we have been delivering NetAcad courses in networking for years, both in person and for remote education.We have the infrastructure to support the related labs and we encourage our students to pursue Cisco certifications.Aware of the changes in the industry and with guidance from our industry advisory board, we decided to add a course focused on teaching students the fundamentals of Python and prepare them for the new requirements of network engineering, security engineering, and systems management jobs: ICTN 2732-Scripting for Information Technology [21].This is a three-credit hours course delivered in both face-to-face and distance-ed formats.Originally, the course was based exclusively on "Starting Out with Python" by Tony Gaddis [15] and the Pearson MyLab-related content.
The chosen textbook was used for several course deliveries with positive student feedback and results.Nevertheless, the course needs adjustments to align it more clearly with the original goals.The course was not meant to compete with a Computer Science offering, it was meant to provide students with the right amount of scripting knowledge and the opportunity to apply it to problems related to network and security analysis, management, and automation.To address this optimization, there were several approaches proposed: (1) Replace the textbook with industry generated curriculum developed specifically for the same goals set for the course, (2) Complement the textbook with industry-generated content to connect the foundational knowledge provided by the textbook with practical applications related to management and automation, and (3) Develop the necessary complementary content such as application aligned labs.
This paper explores the first two paths mentioned above while the third path will be described in a separate, upcoming paper.This paper analyzes the results of adding Cisco' NetAcad related courses "PCAP: Programming Essentials in Python" [31] to the existing textbook.The NetAcad PCAP course is free, it covers most of the topics in the Gaddis textbook and it is certification aligned with the Python Institute (Certified Entry-Level Python Programmer, Certified Associate in Python Programming) [33].Nevertheless, there are structural differences between the two types of content.Gaddis content is developed to build around pedagogical best practices, making it easier to assimilate by a diligent student while the PCAP content is more like a set of tutorials.Testing is also different where the textbook focuses on somewhat artificial scenarios meant to underline and test specific knowledge pieces, while PCAP testing is more like certifications.For these reasons, this case study analyzes the student evaluation of the two types of content and their performance with both content types.The goal of the analysis is to identify an efficient way to use industry-generated content in ICTN 2732.
As part of the process of exploring course updates, an additional NetAcad course was also tested: Model Driven Programmability [28].This additional content is primarily focused on applying the scripting knowledge acquired to real-life scenarios.This content has since evolved into a workshop focused on infrastructure automation.The impact of this content will be discussed in a separate paper.

| METHODS
The results described in this study are the output of a thematic analysis of data collected during the Spring 2022 semester delivery of one section of the ICTN 2732 course.This course is delivered every spring semester and it is very popular among students in IT-related concentrations.

| Case study
The course had 27 students in the senior class, evenly distributed across the Computer Networking and Cybersecurity concentrations.The course was delivered synchronously in person and consisted of lectures.Lab assistance was provided primarily during office hours or when explicitly requested.In this delivery, the following resources were used: Participating students were familiar with Cisco NetAcad before this class since they all had to take required courses based on the Cisco Academy content.The new labs included in the course were another dimension of the instructor's goal to align the course and tests with industry.The student assessment of that content is the subject of a different study.

| Approach
The explicit goal of the delivery was to assess the value of various educational resources to better align the course with the industry needs and, if feasible, to reduce costs for individual students.
To meet this goal, this instance of the course was designed as follows: 1. Teach the first 10 chapters of the textbook, conduct and grade the labs and tests for each chapter.2. Independent study of the PCAP course content which covers mostly the same topics in the first 10 chapters of the textbook, take the tests for each module, take the first half course evaluation tests, the second half evaluation test, and the final PCAP test.This exercise enables the students to revisit information already presented to them and test it but from a different perspective.Along with the benefits of improved retention, the goal is to have students evaluate the two types of content.3. Independent study of the MDP course which immerses them into more complex, real-life problems they must solve using Python.4. Do the internally developed labs.which expose the students to more complex problems and tools. 5. Final test covering exclusively the textbook knowledge.6. Conduct mandatory survey on the comparison between textbook and PCAP.7. Conduct mandatory survey regarding the newly developed labs.
This design facilitates alignment of content between the textbook and the PCAP course, alignment of tests, and hands-on exercises.It also provides the instructors with the opportunity to measure student learning at several points throughout the course as students get exposed to both types of content.
Data was collected in the form of exam performance and mandatory end-of-class survey.The final exam was implemented on Canvas, and it consisted of 50 questions of various formats such as: multiple-choice answers, freeform answers, and code review answers.The questions covered most topics taught in the course.Students had 100 min to complete the online test and the instructor had to grade free form and code review answers.The survey is detailed in Appendix A and consisted of eight quantitative questions and two qualitative, free-form questions.The quantitative questions required students to evaluate quality of content, ease of understanding, quality of the tests, their own performance with each curriculum, and their overall assessments and recommendations.In the free-form questions, students were asked to justify their preference for one course or the other.The survey was mandatory, it was conducted online, and it was anonymous.

| REVIEW AND DISCUSSION OF RESULTS
The main goal of the exercise was to understand student perception of the two types of content, textbook, and industry-developed curriculum, and the means of testing the acquired knowledge.All students in this section of the class were surveyed at the end of the course.The results of the survey are discussed in this section along with the performance of the students in the class.

| Course quality
The first area of evaluation is the perceived quality of the content with the first set of questions asking the students to grade each content type independently.The results are presented in Table 1 and shown graphically in Figure 1.
The results indicate a balanced perspective on the quality of the two types of content, with most students feeling comfortable with both formats.Extreme data points likely reflect students with strong preferences in terms of information acquisition, students strongly preferring the pedagogically focused approach vs students preferring the pragmatic, exploratory approach.
The next question focused on a specific aspect of the overall perceived quality of the course, the question asked the students to evaluate which course was easier to follow.The question was also structured in a way that forces the student to take a clear position on the subject, without degrees of preferences.The results of the survey are graphically shown in Figure 2.
The feedback to this question indicates most of the students prefer to acquire knowledge through content that is focused on a pedagogical approach to delivery.It is likely that some of those who expressed preference for the Cisco content might have chosen that answer because they were already familiar with the key concepts after going through the Pearson content first.This hypothesis is difficult to verify without running the class in two distinct groups of students, each group being exposed to only one type of content.
Following these questions, we can conclude that while the two types of content were similarly appreciated by the students, most students found the Pearson textbook to be easier to follow.These responses indicate that pedagogically focused content would be best suited to deliver fundamental concepts and to bring most students to a level of competence that would enable them to expand into other, more practical forms of knowledge acquisition.At the same time, it is important to recognize that there are students who will prefer a more asynchronous acquisition of knowledge with emphasis on hands-on learning.

| Exam quality
The next step in the evaluation process is to request student feedback on the assessment processes implemented by the Pearson and Cisco courses respectively.The pedagogical focus of the Pearson course is reflected in its testing process as well.Test questions and particularly coding-based tests implemented in the "MyLab" platform consist of purposefully contrived scenarios meant to emphasize a concept.This approach, common to traditional textbooks, comes in glaring contrast with the testing approach of the Cisco content which is focused on the application of knowledge and real-life scenarios.Students would prefer one testing mechanism over the other based on their preferred way to acquire knowledge and based on their predisposition to pursue more challenging assignments that might take them outside the boundaries identified by textbooks.The results of the exam quality assessment are captured in Table 2.
Figure 3 displays the student feedback on the perceived quality of the two types of exams.Unlike the perceived course quality, distinctly different distributions describe the two types of texts.The majority of students qualify the Pearson exams as Good while most students qualify the Cisco exams as Excellent and Very Good.This clear difference in preferences would indicate that while the pedagogically focused approach to knowledge delivery is preferred by most students, when it comes to testing, the focus on practical examples is viewed more positively.These results align with other evaluations of Cisco Networking Academy courses embedded in ICT curriculum [34].
Along with the survey data on the perceived quality of the exams, we wanted to review the performance of the students for the two types of tests.It is important to mention again the timeline of student access to the two types of tests.Students were required to take individual tests for each Pearson module during the semester.They then went through the Cisco content and took tests covering the same knowledge base followed by a final, comprehensive exam on the Cisco content.After completing the Cisco course, students were tested again with a final, comprehensive exam consisting of questions from the Pearson module tests.While the two courses cover mostly the same concepts, the sequence and structure of delivery is not aligned around the same modules, so the test results were grouped into two halves, each including the modules covering the same topics.Figure 4 shows the proportion of students who performed better in the exam delivered by one course vs the other.For example, in the test covering the first half of the content, 60% of the students performed better in the Cisco test versus 40% of the students who performed better in the Pearson test.
The data shows that most students performed better in the Cisco tests for the first and second half content exams.These results indicate that the student's command of the course topics was significantly improved following the review done through the Cisco course which followed the Pearson course.This improvement leads to a significant improvement of the performance in the final exam for the Pearson content where most students performed better than in the Cisco final test.This pattern is similar to the patterns observed when combining complementary content in the process of teaching Python for practical applications [36].The pattern confirms the positive value of repetition and the positive value of reviewing the same content from a different perspective, in a different format.
Students were also asked to assess their own performance in the two courses.This assessment is to a certain extent biased by the fact that the last test taken in the course, their freshest memory of the testing experience, was related to the Pearson content.Nevertheless, while over half of the students correctly estimated that they did better in the Pearson course (56%) versus Cisco course, these results do not reflect the results for the final exams in Figure 4.

| Overall evaluation
At the end of the survey, we decided to ask the students a set of questions regarding their overall perception of the two courses.The two questions and the results of the survey are shown in Table 3.The first and second questions in the table were discussed earlier in the paper and are listed here for reference purposes.
In Sections 3.1 and 3.2 we discussed the fact that most students view the Pearson content as easier to follow but they prefer the Cisco course approach to testing knowledge.When students are asked to do an overall evaluation of the two courses and to decide which Literature review studies [30] have shown there is a clear need for innovative approaches to curriculum design and content delivery focused on competency [1] and self-study, aligned with the specific needs of ICT industry segments.Most studies suggest the involvement of industry resources in the development and delivery of effective ICT training.The case study presented in this paper is an empirical application of multiple models applied to ICT education such as Challenge-Based Learning [16], Experiential Learning [2], Job-Oriented Learning [11], Work Integrated Learning [5].While the approach of this study was focused on content choices, the envisioned blended curriculum supports these educational models.

| FUTURE WORK
In this study, we analyzed student perception and student performance impact of blended content in teaching Python scripting ITC students with networking and cybersecurity concentrations.The results support the conclusion that effective curriculum benefits from a combination of the two types of content to achieve better student performance and to align student learning with industry and job market needs.The authors identified two major dimensions for subsequent work: (1) refinement of student performance analysis and (2) quantitative analysis of various course structures that can be implemented based on the lessons learned.
A systematic methodology for assessing student performance in blended content courses would be highly beneficial to future curriculum development.This is one area of investigation that will be pursued by the authors based on the lessons learned from this initial study.More rigorous control of second-order variables would facilitate a clearer picture of key factors in the timing of blended content delivery.
While this study provided a high-level evaluation of the advantages of combining content types rather than focusing on a single type, it did not investigate the optimal ways in which the content should be organized to maximize results.Smart content integration [3] is important in achieving the full benefits of blended learning [34].Moreover, while the focus is on Python, a scripting language fit for the needs of these majors and a programming language ease to learn [13], studies show that combining Python with a secondary language helps students grasp concepts better and more efficiently [23].The authors intend to perform quantitative evaluations of various approaches to developing an effective curriculum using blended learning.Additionally, the authors intend to work on analyzing the impact of delivery methods in the context of mixed teaching methods [40] and studentcentric design [20].

| CONCLUSIONS
Our case study of inserting industry-created courseware in an existing, higher education course on Python scripting revealed that by itself, such content does not meet the needs of all students.Instead, the material proved more valuable in complementing more traditional, pedagogically focused content.Industry content can expose students to practical perspectives on the topic covered and prepare them for industry-accepted certifications.It can also help reinforce concepts delivered through traditionally developed content.The results of this paper are very important in guiding the process of optimizing existing curriculum by combining it with content developed by industry-led academies and professional groups.These optimizations will help prepare students for the current demands of the job market and the expectations of employers.
5th edition of Gaddis' textbook and the related tests and labs provided via Pearson's MyLab.Pearson prices the textbook at $74.99 and access to the lab at $49.99[32] • Cisco's NetAcad PCAP course materials and tests, free for the students • Cisco's NetAcad MDP course materials and tests, free for the students • A set of new labs, purpose-built for the class by the course coordinator to help students apply their Python knowledge to problems similar to real-life operations.
Quality of content survey results.
T A B L E 1 F I G U R E 2 Responses to the question: "Which content is easier to follow?".T A B L E 2 Quality of tests survey results.
Student test results.Quality of test survey results.contentshould be used in future iterations of the class, the Cisco content ended up with a slight edge over the Pearson content.Despite the slightly favorable overall view of the Cisco course, it is worth noting that this edge could be since students started working on this material after they already learned the key concepts while completing most of the Pearson course.